diff --git a/.gitattributes b/.gitattributes index a6344aac8c09253b3b630fb776ae94478aa0275b..f0bdeab8b4e2750df0636d9e1a1905047cc060b5 100644 --- a/.gitattributes +++ b/.gitattributes @@ -1,35 +1,2 @@ -*.7z filter=lfs diff=lfs merge=lfs -text -*.arrow filter=lfs diff=lfs merge=lfs -text -*.bin filter=lfs diff=lfs merge=lfs -text -*.bz2 filter=lfs diff=lfs merge=lfs -text -*.ckpt filter=lfs diff=lfs merge=lfs -text -*.ftz filter=lfs diff=lfs merge=lfs -text -*.gz filter=lfs diff=lfs merge=lfs -text -*.h5 filter=lfs diff=lfs merge=lfs -text -*.joblib filter=lfs diff=lfs merge=lfs -text -*.lfs.* filter=lfs diff=lfs merge=lfs -text -*.mlmodel filter=lfs diff=lfs merge=lfs -text -*.model filter=lfs diff=lfs merge=lfs -text -*.msgpack filter=lfs diff=lfs merge=lfs -text -*.npy filter=lfs diff=lfs merge=lfs -text -*.npz filter=lfs diff=lfs merge=lfs -text *.onnx filter=lfs diff=lfs merge=lfs -text -*.ot filter=lfs diff=lfs merge=lfs -text -*.parquet filter=lfs diff=lfs merge=lfs -text -*.pb filter=lfs diff=lfs merge=lfs -text -*.pickle filter=lfs diff=lfs merge=lfs -text -*.pkl filter=lfs diff=lfs merge=lfs -text -*.pt filter=lfs diff=lfs merge=lfs -text -*.pth filter=lfs diff=lfs merge=lfs -text -*.rar filter=lfs diff=lfs merge=lfs -text -*.safetensors filter=lfs diff=lfs merge=lfs -text -saved_model/**/* filter=lfs diff=lfs merge=lfs -text -*.tar.* filter=lfs diff=lfs merge=lfs -text -*.tar filter=lfs diff=lfs merge=lfs -text -*.tflite filter=lfs diff=lfs merge=lfs -text -*.tgz filter=lfs diff=lfs merge=lfs -text -*.wasm filter=lfs diff=lfs merge=lfs -text -*.xz filter=lfs diff=lfs merge=lfs -text -*.zip filter=lfs diff=lfs merge=lfs -text -*.zst filter=lfs diff=lfs merge=lfs -text -*tfevents* filter=lfs diff=lfs merge=lfs -text +*.jpg filter=lfs diff=lfs merge=lfs -text diff --git a/README.md b/README.md index baf730569160ddedfb8533d80fd5de6a1027e7f2..4684c425c96e61e7e4f47e680b7e58fc8434b741 100644 --- a/README.md +++ b/README.md @@ -1,13 +1,150 @@ --- -title: Animap Gpu -emoji: ๐Ÿ“Š +title: Animap Inference +emoji: ๐Ÿ„ colorFrom: green -colorTo: yellow +colorTo: gray sdk: gradio -sdk_version: 6.25.0 -python_version: '3.12' -app_file: app.py +sdk_version: 5.49.1 +app_file: space_app.py +python_version: "3.12.12" pinned: false +license: apache-2.0 +short_description: Livestock models that refuse to invent a result. --- -Check out the configuration reference at https://huggingface.co/docs/hub/spaces-config-reference +# Animap inference + +This runs validated livestock models against farm photographs and returns what +the model actually produced. Its most important property is what it **refuses**: +a capability with no checksummed artefact behind it answers `unavailable`, not a +placeholder and not a plausible-looking number. + +Twenty-eight capabilities are registered. **Two run.** `cattle_detection` and +`poultry_count` execute a YOLOX-m ONNX artefact whose sha256 is verified against +its model card at start-up. The other twenty-six say so plainly, which is the +honest answer rather than a gap. + +The source is `services/inference` in the Animap repository. `app.py` mounts +`app.main:app` โ€” the same FastAPI service the Azure Container App runs โ€” under a +Gradio page, so `/health`, `/capabilities` and `/jobs` behave exactly as they do +in production and the page is a demonstration of them. + +**It was a Docker Space until 2026-08-24 and is a Gradio one now**, for one +reason: ZeroGPU is Gradio-SDK only. The Docker image ran the identical Azure +bytes, which was the better provenance story, and it could not be given a GPU at +any price โ€” which left CountGD, the one capability a GPU actually unblocks, +unmeasurable. See **What this Space is not** for what the change cost. + +## What is open and what is not + +| Endpoint | Auth | Why | +|---|---|---| +| `GET /health` | none | Carries no farm data, and a platform probe has to reach it | +| `GET /capabilities` | none | The published contract: what may be claimed, and what may not | +| `POST /jobs` | **bearer token** | Runs a model against a farm's photographs | +| `GET /jobs/{id}` | **bearer token** | Returns a farm's result | + +`space/publish.py --set-secret` mints `ANIMAP_INFERENCE_TOKEN` and sets it as a +Space secret. **Check it rather than assuming it**: `GET /health` reports +`"authenticated": false` when no token is configured, so a deployment that +reached the internet without one says so to anyone who asks. + + curl -s https://bluman1-animap-inference.hf.space/health + +## Running a model + +Two public-domain captures are baked in, so a real detection can be obtained +without an Azure account. `space/FIXTURES.md` in the repository lists their ids, +their sources and the human count on record for each. + +```bash +curl -s -X POST https://bluman1-animap-inference.hf.space/jobs \ + -H "authorization: Bearer $ANIMAP_INFERENCE_TOKEN" \ + -H 'content-type: application/json' \ + -d '{"capability_key":"cattle_detection", + "subject_type":"herd", + "subject_id":"00000000-0000-0000-0000-000000000001", + "farm_id":"00000000-0000-0000-0000-000000000002", + "media_ids":["aa5e8481-8be6-509d-b1fa-f1a178c7cda0"], + "captured_at":"2026-08-22T10:00:00Z"}' +``` + +A frame that settles at the first grid answers in well under a second. A dense +one runs all three grids and takes a few seconds; there is no queue, because no +capability yet takes tens of seconds. + +Read `warnings` before you read the number. A count is of the animals **visible +in one frame** โ€” never the herd size, never a flock population, and never a +house reconciliation. When the count keeps rising as the frame is read more +finely, or exceeds twenty, the service publishes `count_withheld` and no number +at all. That refusal is a feature and it is measured: see the `known_limits` and +`validation_notes` on each model card. + +## What the SDK change cost, and what it did not + +**Lost: the image is no longer byte-identical to Azure's.** A Gradio Space has +no Dockerfile, so the claim *"this Space builds from the same Dockerfile"* is +gone and cannot be got back while ZeroGPU is Gradio-only. What runs is the same +`app/` tree with the same `requirements.txt`, which is close and is not the same +thing, and this file says so rather than letting the old sentence stand. + +**Lost: a build-time licence gate.** The Docker build failed if an AGPL runtime +arrived. There is no build to fail now. + +**Kept: every gate that actually protects a result.** `app.py` runs +`scripts/install_models.py --check` before it imports the service, so an +artefact that disagrees with its model card stops the Space at start-up rather +than being found by a farm's job. `providers.discover()` still refuses to serve +a capability whose artefact fingerprints as a copyleft runtime, and `/health` +still publishes `artefact_licenses` so a deployment in breach is visible from +outside. + +**Gained: the ability to be given a GPU.** Nothing here reaches for CUDA yet โ€” +YOLOX-m and DINOv3 are both ONNX on CPU โ€” so this buys no speed-up today. It is +the prerequisite for CountGD, which gets MAE 14.84 on broiler houses against the +deployed detector's 156.80 and has never been runnable anywhere in this project. + +## What this Space is not + +**It is not the production media path, and it must not be read as evidence for +one.** This was true of the Docker Space and the SDK change did nothing to it. Production reads captures out of an Azure Blob container using the +Container App's managed identity โ€” no key, no SAS, nothing stored. A Space is +not inside Azure and has no managed identity, so that credential is unavailable +to it. The alternatives a Space *could* use are a storage account key or a SAS +token in a secret, and neither is the production posture: one hands a public +Space full access to every farm's evidence, and the other expires. + +So this Space serves `ANIMAP_MEDIA_PROVIDER=local` against the two baked-in +frames. Everything downstream of the pixels โ€” the quality gate, the detection +pyramid, the counting guard, the observation vocabulary โ€” is the production code +path exactly. Everything upstream of them is not. + +**No farm data reaches this Space.** It cannot read `animapmedia`, and the only +captures it holds are two public-domain photographs from Wikimedia Commons. + +## Weights + +YOLOX-m, Apache-2.0, from the Megvii `0.1.1rc0` release, **vendored into this +repository under Git LFS rather than fetched at build time**. The service +verifies its sha256 against `models/cattle_detection/model_card.json` at +start-up and refuses to load an artefact that does not match. Vendoring is what +lets the build step stay `install_models.py --check` โ€” verify, never fetch โ€” +which is the posture ADR 0005 asks for and the same command the Azure build +runs. + +`models/cattle_identity` is deliberately absent. Its artefact is DINOv3 under a +bespoke Meta licence whose two published texts disagree about an attribution +obligation, and publishing a copy into a public Space is redistribution. That +capability answers `unavailable` here, and correctly. + +No AGPL-3.0 software is installed and none may be. `requirements.txt` omits +`ultralytics`, `.dockerignore` excludes `*.pt`, the build fails if one arrives +anyway, and `providers.discover()` refuses to serve a capability whose artefact +fingerprints as a copyleft runtime. `GET /health` publishes +`artefact_licenses`, so a deployment in breach is visible from outside. + +## Attribution + +Third-party notices travel with the image in `THIRD_PARTY_NOTICES.md`. The two +demonstration captures are CC0 and public domain; their sources are in +`space/FIXTURES.md`. diff --git a/THIRD_PARTY_NOTICES.md b/THIRD_PARTY_NOTICES.md new file mode 100644 index 0000000000000000000000000000000000000000..be67a7447af5d94f88877d540768a97d5af07d8f --- /dev/null +++ b/THIRD_PARTY_NOTICES.md @@ -0,0 +1,68 @@ +# Third-party assets + +## Models + +### DINOv3 + +Built with DINOv3 + +Animap's cattle identity embedding is a local ONNX export of +`timm/vit_small_patch16_dinov3.lvd1689m`. The weights are covered by Meta's +bespoke DINOv3 licence, not by Apache-2.0. + +Two published texts of that licence differ. The `LICENSE.md` shipped with the +weights, dated 19 August 2025, ends clause 1.b.i at providing a copy of the +agreement. The text at +`ai.meta.com/resources/models-and-libraries/dinov3-license`, dated 14 August +2025 and linked from the Hugging Face model card, additionally requires that you +prominently display "Built with DINOv3". Section 8 lets Meta amend the licence +unilaterally with immediate effect. + +Displaying the attribution satisfies both readings, which is why it appears +here. **This file is not yet enough.** "Prominently display" points at a surface +a person using Animap can see, and the app has no about screen. The outstanding +surfaces are listed in `attribution_outstanding` in +`services/inference/app/adapters/licences.py`, and `tests/test_attribution.py` +asserts that this file carries the string and that those surfaces still do not. + +- Source: https://huggingface.co/timm/vit_small_patch16_dinov3.lvd1689m +- Licence: https://github.com/facebookresearch/dinov3/blob/main/LICENSE.md + +### DINOv2 + +The alternate embedding backbone, `facebook/dinov2-small`, is licensed +Apache-2.0 for both code and weights. It requires no attribution beyond the +licence text travelling with any redistributed copy, and Animap redistributes +none. + +- Source: https://huggingface.co/facebook/dinov2-small +- Licence: https://github.com/facebookresearch/dinov2/blob/main/LICENSE + +### YOLOX + +The shipped detector is YOLOX-m, from Megvii's YOLOX repository, under +Apache-2.0. Megvii publishes no separate licence for the released ONNX weights; +the repository's licence is read as covering the artefacts it distributes, and +that inference is recorded in `docs/adr/0017-ultralytics-licence.md`. + +- Source: https://github.com/Megvii-BaseDetection/YOLOX +- Licence: https://github.com/Megvii-BaseDetection/YOLOX/blob/main/LICENSE + +## Material Design Icons + +Animap includes the following icons from Material Design Icons by +Pictogrammers: + +- `cow` +- `turkey` +- `arrow-left` +- `home-outline` +- `bell-outline` +- `dots-horizontal` +- `camera-outline` + +Material Design Icons is licensed under the Apache License 2.0. + +- Project: https://pictogrammers.com/library/mdi/ +- License: https://pictogrammers.com/docs/general/license/ +- Source: https://github.com/Templarian/MaterialDesign diff --git a/app/__init__.py b/app/__init__.py new file mode 100644 index 0000000000000000000000000000000000000000..e69de29bb2d1d6434b8b29ae775ad8c2e48c5391 diff --git a/app/adapters/__init__.py b/app/adapters/__init__.py new file mode 100644 index 0000000000000000000000000000000000000000..2dae5429cf19963270d739a00ae8107dfd2d9afa --- /dev/null +++ b/app/adapters/__init__.py @@ -0,0 +1,43 @@ +"""Adapters for the zero-training model stack (directive ยง3, ยง4, ยง40.1). + +One interface over things with nothing else in common: a segmenter, a frozen +backbone, an open-vocabulary detector, a hosted reasoner, and a pair of +deterministic methods that need no weights at all. What they share is +governance, and that is what `base.py` unifies โ€” what a thing costs, what +licence it really carries, where it should run, and whether it can run at all. + +Read `base.py` first. The two properties it exists to hold are that an adapter +cannot produce a result with no model behind it, and that a cost is either +measured or reported as unmeasured. + +`licences.py` is the control ADR 0017 asked for, generalised: what each runtime +actually loads under, rather than what a card says about itself. +""" + +from app.adapters.base import ( + Adapter, + AdapterError, + AdapterSpec, + AdapterUnavailable, + Availability, + MeasuredCost, + Measurement, + Modality, + Placement, + Region, + Task, +) + +__all__ = [ + "Adapter", + "AdapterError", + "AdapterSpec", + "AdapterUnavailable", + "Availability", + "MeasuredCost", + "Measurement", + "Modality", + "Placement", + "Region", + "Task", +] diff --git a/app/adapters/audio/__init__.py b/app/adapters/audio/__init__.py new file mode 100644 index 0000000000000000000000000000000000000000..2ed3e58ed66f0a34247ed46abaf3089c424debf1 --- /dev/null +++ b/app/adapters/audio/__init__.py @@ -0,0 +1,197 @@ +"""Audio, as a first-class adapter with no weights in it. + +Directive ยง26 names four things to test for poultry respiratory audio: + +1. SAM Audio; +2. Perception Encoder audio / AV embeddings; +3. hosted multimodal audio reasoning; +4. classical audio features. + +**This package is the fourth, and it is first because ยง4 says so** โ€” *"Do not +use a neural model when deterministic signal processing is better"* โ€” and +because this repository already has one clean result from taking that rule +seriously: `adapters/signal/periodicity.py` recovers a metronome's stated 96 +beats per minute as 96.48 from an FFT and nothing else. + +Whether it is *better* here than the three neural options is not something this +package can settle on its own. `experiments/poultry_respiratory/README.md` +records what each of the other three would take and which of them was reachable +from this machine, which is what ยง36 asks for before anything is called +unavailable. + +## Layout + +`decode.py` โ€” a phone recording into a NumPy array, via an `ffmpeg` binary, +because the service has no audio library at all. + +`features.py` โ€” STFT, spectral flux, machinery and speech screens. NumPy only. + +`events.py` โ€” the ยง26 screen: cough- or sneeze-like events, or a refusal +carrying one of the registry's own rejection codes. +""" + +from __future__ import annotations + +from pathlib import Path + +from app.adapters.audio.decode import ( + MAX_SECONDS, + SAMPLE_RATE_HZ, + AudioUnreadable, + DecoderMissing, + Recording, + available, + ffmpeg_path, + from_samples, + read_audio, +) +from app.adapters.audio.events import ( + MIN_CAPTURE_SECONDS, + Event, + RespiratoryScreen, + screen, +) +from app.adapters.base import ( + Adapter, + AdapterSpec, + AdapterUnavailable, + Availability, + MeasuredCost, + Modality, + Placement, + Task, +) + +RESPIRATORY_AUDIO_SPEC = AdapterSpec( + adapter_id="respiratory-audio-flux", + runtime="ffmpeg-numpy", + tasks=(Task.MEASURE,), + modalities=(Modality.AUDIO,), + directive_role=( + "ยง26 poultry respiratory audio โ€” 'stand quietly in the house and record " + "30 seconds', reported as cough/sneeze-like events and a spot screen. " + "ยง4's classical route: spectral flux, band energies and an " + "autocorrelation pitch test, with no weights. ยง27 keeps continuous " + "monitoring a separate capability that needs a fixed microphone." + ), + #: There is no artefact โ€” but unlike the other deterministic adapters this + #: one is not unconditionally available, because it needs an external + #: binary. That distinction is the reason `availability()` below is not + #: simply `Availability(True)`. + requires_artefact=False, + placement=Placement.ON_DEVICE, + placement_reason=( + "An STFT over 60 seconds at 16 kHz is 7,500 frames of 512-point FFT โ€” " + "milliseconds of arithmetic, and a phone has both the CPU and the " + "recording already. Running it on the device keeps ADR 0002's " + "offline-first promise for a capability a farmer uses standing in a " + "shed, and means the audio never leaves the phone. The obstacle is not " + "compute, it is the decoder: Android supplies its own, so a port would " + "replace `decode.py` and nothing else." + ), + measured=MeasuredCost( + hardware=( + "Apple M-series laptop (NOT the target container), OMP_NUM_THREADS=1" + ), + threads=1, + sample=( + "60.0 s of 16 kHz mono holding 12 injected transients โ€” decode " + "excluded, since that is FFmpeg's cost and not this module's. " + "Spectrogram, onset envelope, adaptive threshold, machinery and " + "speech screens, and the per-event gates" + ), + runs=9, + median_seconds=0.094, + peak_rss_mb=263.4, + measured_on="2026-08-22", + ), + notes=( + "**This detector does not work, and that is measured rather than " + "suspected.** Over 6,346 real poultry-house clips from two CC BY 4.0 " + "datasets it separates Sick from Healthy at AUC 0.4141 โ€” below chance โ€” " + "because a healthy house is a noisy one and spectral flux counts " + "activity. Frozen CLAP embeddings over the identical clips do better, " + "so the signal is there and the failure is the method's. " + "`experiments/poultry_respiratory/` has the tables. No figure from it " + "may be shown to a farm as an accuracy.\n\n" + "**No poultry-house recording with event-level cough annotation exists " + "under a free licence**, so the thing this adapter claims to do โ€” count " + "events โ€” has never been scored by anybody. An earlier version of this " + "note said the benchmark measured detector behaviour on audio mixed " + "with events at known times and signal-to-noise ratios. No such mixture " + "was ever built; every threshold in events.py is chosen rather than " + "derived, and each one now says so.\n\n" + "**It needs an `ffmpeg` binary and the one on a laptop is not the one " + "to ship.** See `adapters/licences.py:ffmpeg-numpy`: the build this was " + "developed against is `--enable-gpl --enable-nonfree`, which is the one " + "configuration FFmpeg may not be redistributed under at all." + ), +) + + +class RespiratoryAudioAdapter(Adapter): + """ยง26's spot screen. No weights, one external binary. + + Deliberately not a subclass of `deterministic.DeterministicAdapter`. That + class answers `Availability(True)` unconditionally, and the sentence + justifying it โ€” *"there is no artefact to be absent"* โ€” is true of optical + flow and false here. Inheriting it to save six lines would be inheriting a + claim that does not hold. + """ + + spec = RESPIRATORY_AUDIO_SPEC + + def availability(self) -> Availability: + # Must not decode anything: `/health` calls this often enough that + # spawning a subprocess to answer it would be its own outage. + if not available(): + return Availability( + False, + "No `ffmpeg` binary on PATH, and this service has no audio " + "decoding library โ€” no soundfile, no librosa, no av. A phone " + "recording cannot be read at all without it.", + "Install FFmpeg, and read adapters/licences.py:ffmpeg-numpy " + "before choosing a build โ€” the common Homebrew and static " + "builds are GPL or non-free.", + ) + return Availability(True) + + def load(self) -> "RespiratoryAudioAdapter": + state = self.availability() + if not state.ready: + raise AdapterUnavailable(state) + return self + + def measure(self, audio_path: Path | str, **kwargs) -> RespiratoryScreen: + """Decode and screen one recording. + + Goes through `load()` rather than assuming it: a caller who skipped it + would get a `FileNotFoundError` from deep inside `subprocess` instead of + the availability answer this adapter exists to give. + """ + self.load() + return screen(read_audio(audio_path), **kwargs) + + +def audio_adapters() -> list[Adapter]: + return [RespiratoryAudioAdapter()] + + +__all__ = [ + "MAX_SECONDS", + "MIN_CAPTURE_SECONDS", + "RESPIRATORY_AUDIO_SPEC", + "SAMPLE_RATE_HZ", + "AudioUnreadable", + "DecoderMissing", + "Event", + "Recording", + "RespiratoryAudioAdapter", + "RespiratoryScreen", + "audio_adapters", + "available", + "ffmpeg_path", + "from_samples", + "read_audio", + "screen", +] diff --git a/app/adapters/audio/decode.py b/app/adapters/audio/decode.py new file mode 100644 index 0000000000000000000000000000000000000000..e825de781099609ac20182cb0bdff9ac731f419a --- /dev/null +++ b/app/adapters/audio/decode.py @@ -0,0 +1,250 @@ +"""Getting a phone recording into a NumPy array, and refusing when it cannot. + +Directive ยง26 asks the farmer to "stand quietly in the house and record 30 +seconds". Whatever the phone hands over โ€” `.m4a`, `.opus`, `.ogg`, `.3gp` โ€” has +to become mono float samples at one known rate before any of the arithmetic in +`features` or `events` means anything. + +**The service has no audio library.** There is no `soundfile`, no `librosa`, no +`av`; `cv2` bundles FFmpeg but exposes no audio path, and `wave` in the standard +library reads WAV and nothing else. So this module shells out to an `ffmpeg` +binary, which is a real external dependency and is treated as one: `available()` +looks for it, and the adapter that wraps this reports unavailable rather than +raising from the middle of a request. + +**Two things about that binary are recorded rather than assumed.** + +*It writes a file, not a pipe.* The obvious form is `-f s16le -` into +`subprocess`. That fails on any FFmpeg configured with a muxer whitelist, and +the build on this machine is one โ€” `--disable-muxers --enable-muxer='webm,opus, +mp4,wav,...'` has no `s16le` in it, so the pipe form exits 234 with *"Requested +output format 's16le' is not known"*. Writing a temporary `.wav` and reading it +back with `wave` costs one file and works against every build, including the +minimal ones. + +*Its licence is not this repository's to assume.* See +`adapters/licences.py:ffmpeg-cli`. The build here is `--enable-gpl +--enable-nonfree`, which is the one configuration FFmpeg may not be +redistributed under at all. That does not reach Animap's own code โ€” calling a +separate program over a pipe is not linking, and `app/adapters/licences.py` +records the reasoning โ€” but it does mean **the binary on a developer's laptop is +not the binary a deployment may ship**, and that is a deployment finding rather +than a footnote. +""" + +from __future__ import annotations + +import shutil +import subprocess +import tempfile +import wave +from dataclasses import dataclass +from pathlib import Path + +import numpy as np + +#: What everything downstream assumes, in hertz. +#: +#: 16 kHz resolves to 8 kHz, and a chicken snick's energy is in the low +#: kilohertz โ€” Mahdavian et al. and the broader poultry-audio literature place +#: sneeze and rale energy under 6 kHz. 22.05 or 44.1 kHz would carry more of the +#: transient's top edge and would quadruple the STFT cost for a band nothing +#: here measures. +#: +#: **It is fixed rather than passed through** because every threshold in +#: `events.py` is a property of the analysis band, and the periodicity module +#: next door is a standing lesson in what happens when a constant derived at one +#: band is applied at another. +SAMPLE_RATE_HZ = 16_000 + +#: Longest clip this will decode, in seconds. ยง26 asks for 30 and prefers 60; +#: ยง27 is explicit that continuous monitoring is a different capability with a +#: fixed microphone. Ten minutes is far past a spot check and is here to stop a +#: mis-sent file eating the container's memory, not to express a product limit. +MAX_SECONDS = 600.0 + +#: How long to let FFmpeg run. A 60-second clip transcodes in well under a +#: second; anything near this is a malformed file FFmpeg is chewing on. +DECODE_TIMEOUT_SECONDS = 120 + + +class AudioUnreadable(RuntimeError): + """The recording could not be decoded, so there is nothing to measure.""" + + +class DecoderMissing(RuntimeError): + """No `ffmpeg` on PATH. A missing tool, not a broken recording.""" + + +@dataclass(frozen=True) +class Recording: + """Mono float samples in [-1, 1], plus what it took to get them.""" + + samples: np.ndarray + sample_rate_hz: int + #: The file it came from, for a result that has to be traced back. + source: str + #: Before resampling, so a clip recorded at 8 kHz is diagnosable later โ€” it + #: has no content above 4 kHz however it is resampled, and half the band + #: this module analyses is empty for it. + source_sample_rate_hz: int + source_channels: int + + @property + def duration_seconds(self) -> float: + return len(self.samples) / self.sample_rate_hz + + @property + def is_silent(self) -> bool: + """No signal at all, as distinct from no events. + + A muted microphone and a quiet house are different findings and only + one of them is about the birds. + """ + return float(np.max(np.abs(self.samples), initial=0.0)) < 1e-6 + + +def ffmpeg_path() -> str | None: + return shutil.which("ffmpeg") + + +def ffprobe_path() -> str | None: + return shutil.which("ffprobe") + + +def available() -> bool: + return ffmpeg_path() is not None + + +def _probe(path: Path) -> tuple[int, int]: + """The source rate and channel count, or `(0, 0)` when ffprobe is absent. + + Recorded rather than required. The decode does not need it โ€” FFmpeg + resamples whatever it finds โ€” but a rate of 8,000 explains an empty upper + band better than any later measurement can, and losing that to a missing + optional tool would be worse than reporting it as unknown. + """ + probe = ffprobe_path() + if probe is None: + return 0, 0 + try: + result = subprocess.run( + [probe, "-v", "error", "-select_streams", "a:0", "-show_entries", + "stream=sample_rate,channels", "-of", "csv=p=0", str(path)], + capture_output=True, text=True, timeout=30, check=True, + ) + except (subprocess.SubprocessError, OSError): + return 0, 0 + parts = result.stdout.strip().split(",") + try: + return int(parts[0]), int(parts[1]) + except (IndexError, ValueError): + return 0, 0 + + +def read_audio( + path: Path | str, + *, + sample_rate_hz: int = SAMPLE_RATE_HZ, + max_seconds: float = MAX_SECONDS, +) -> Recording: + """Decode to mono float32 at `sample_rate_hz`. + + Raises `DecoderMissing` when there is no FFmpeg and `AudioUnreadable` when + there is one and the file defeats it. The two are separate exceptions + because they need different answers: install a tool, or ask for a different + recording. + """ + path = Path(path) + binary = ffmpeg_path() + if binary is None: + raise DecoderMissing( + "No `ffmpeg` on PATH. This service has no audio decoding library โ€” " + "no soundfile, no librosa, no av โ€” so a phone recording cannot be " + "read at all without it. Install FFmpeg, and read " + "adapters/licences.py:ffmpeg-cli before choosing a build." + ) + if not path.is_file(): + raise AudioUnreadable(f"{path} is not a file.") + + source_rate, channels = _probe(path) + + with tempfile.TemporaryDirectory(prefix="animap-audio-") as workspace: + decoded = Path(workspace) / "mono.wav" + command = [ + binary, "-v", "error", "-nostdin", "-y", + "-i", str(path), + # `-t` before the output rather than `-ss`: the cap is on how much + # is decoded, and a spot check has no reason to start late. + "-t", f"{max_seconds:.3f}", + "-map", "a:0?", + "-ac", "1", + "-ar", str(sample_rate_hz), + "-acodec", "pcm_s16le", + "-f", "wav", + str(decoded), + ] + try: + result = subprocess.run( + command, capture_output=True, text=True, + timeout=DECODE_TIMEOUT_SECONDS, + ) + except subprocess.TimeoutExpired as expired: + raise AudioUnreadable( + f"FFmpeg did not finish decoding {path.name} within " + f"{DECODE_TIMEOUT_SECONDS} s." + ) from expired + except OSError as failure: + raise AudioUnreadable(f"Could not run ffmpeg: {failure}") from failure + + if result.returncode != 0 or not decoded.is_file(): + raise AudioUnreadable( + f"FFmpeg could not decode {path.name} " + f"(exit {result.returncode}): {result.stderr.strip()[:300]}" + ) + + with wave.open(str(decoded)) as handle: + frames = handle.getnframes() + width = handle.getsampwidth() + raw = handle.readframes(frames) + + if width != 2: + # Only reachable if a future edit changes `-acodec`; asserted rather + # than assumed because reading 16-bit as 32-bit is silent and produces + # a plausible-looking waveform of noise. + raise AudioUnreadable( + f"Expected 16-bit samples from the decode step, got {width * 8}-bit." + ) + + samples = np.frombuffer(raw, dtype=" Recording: + """A `Recording` over samples that are already in hand. + + The mixer in the respiratory experiment builds its composites in memory, and + routing them through a temporary file to get a `Recording` would mean the + measured pipeline and the tested pipeline differed by an encode. + """ + samples = np.asarray(samples, dtype=np.float32).ravel() + return Recording( + samples=samples, + sample_rate_hz=int(sample_rate_hz), + source=source, + source_sample_rate_hz=int(sample_rate_hz), + source_channels=1, + ) diff --git a/app/adapters/audio/events.py b/app/adapters/audio/events.py new file mode 100644 index 0000000000000000000000000000000000000000..b496d9d9a452ae9b641c016070719c3d858641c2 --- /dev/null +++ b/app/adapters/audio/events.py @@ -0,0 +1,491 @@ +"""Cough- and sneeze-like events in a 30-second recording, and the refusals. + +Directive ยง26 in full: + + Stand quietly in the house and record 30 seconds. + โ†’ "Cough/sneeze-like events detected" + โ†’ "Spot respiratory screen only" + Do not present as continuous surveillance. + +**The output is deliberately weaker than the thing farmers want.** ยง27 makes +continuous cough monitoring a separate capability that needs a fixed +microphone, and ยง30 lists *"24/7 respiratory surveillance from one 30-second +recording"* among the claims that may never be made. Nothing here counts +coughs per bird, per hour, or per house. It counts *events that look like a +cough or a sneeze* in one recording, and the word "like" is load-bearing. + +## The shape this module borrows + +`adapters/signal/periodicity.py` is the pattern, and its opening line is the +one that matters here too: *the hard part is not finding a peak โ€” every +spectrum has a peak.* Every recording has transients. Slamming doors, feeders, +a boot on litter, a bird landing, a microphone rubbing a coat. So this module +is mostly gates, and a refusal is a first-class result: `count` is `None` +whenever `usable` is False, and there is no way to read a number out of a +screen that refused. + +## The three refusals, and where they come from + +The capability registry names them. `app/capabilities.py` gives +`poultry_respiratory` a `reject_if` of exactly +`("recording_too_short", "machinery_dominates", "speech_dominates")`, and each +one is implemented here under that name so a rejection the product declares is +a rejection the code can actually produce. + +## What is measured, and what is not + +`experiments/poultry_respiratory/` holds the benchmark, and the honest summary +is short: **this detector does not work.** Over 6,346 real poultry-house clips +from two CC BY 4.0 datasets it separates Sick from Healthy at AUC 0.4141 โ€” +below chance โ€” because a healthy house is a noisy one and spectral flux counts +activity. Frozen CLAP embeddings over the identical clips do better, so the +signal is there and this is a method failure. + +**No recording of a poultry house with event-level cough annotation exists under +a free licence**, so the thing this module actually claims to do โ€” count events +โ€” has never been scored by anybody, here or elsewhere. An earlier version of +this paragraph said the benchmark measured "detector behaviour on real farm +noise mixed with real transient events at known times and known +signal-to-noise ratios". No such mixture was ever built. Every threshold below +is chosen rather than derived, and each one now says so. +""" + +from __future__ import annotations + +from dataclasses import dataclass, field + +import numpy as np + +from app.adapters.audio.decode import Recording +from app.adapters.audio.features import ( + FLUX_BAND_HZ, + Spectrogram, + adaptive_threshold, + harmonicity, + machinery_dominance, + onset_strength, + spectrogram, + speech_dominance, +) + +#: ยง26's capture, and `app/capabilities.py`'s `minimum_capture_seconds` for +#: `poultry_respiratory`. Kept equal to the registry's number on purpose: a +#: capture minimum the app advertises and a capture minimum the code enforces +#: that differ by a second is a bug report nobody can reproduce. +MIN_CAPTURE_SECONDS = 30.0 + +#: Amplitude below which the microphone is considered dead rather than the house +#: quiet. Full scale is 1.0, so this is roughly โˆ’80 dBFS โ€” beneath the +#: self-noise of any phone microphone. +SILENCE_PEAK = 1e-4 + +#: Window the onset threshold adapts over, in seconds. Long enough to hold +#: several seconds of background between events; short enough to follow a fan +#: cycling. Two seconds at an 8 ms hop is 250 frames, and the median of 250 +#: frames is unmoved by the handful an event occupies. +ADAPTIVE_WINDOW_SECONDS = 2.0 + +#: How many local median-absolute-deviations above the local median a frame's +#: spectral flux must reach to be a candidate. +#: +#: **CHOSEN, NOT DERIVED, and an earlier version of this comment said the +#: opposite.** It claimed the experiment swept this value against composites of +#: real farm noise and real transient events at known times. No such composites +#: exist, no such sweep was run, and +#: `experiments/poultry_respiratory/config.yaml` says so in its own +#: `thresholds` block. A hostile audit found the two files contradicting each +#: other and it was the shipped code that was lying. +#: +#: 6.0 is a robust-statistics default โ€” roughly four standard deviations for +#: Gaussian noise, via the 1.4826 MAD-to-sigma factor. What IS measured is what +#: it does: over 6,346 real poultry-house clips it fires on 22.1% of Healthy +#: clips and 5.2% of Sick ones, which is the wrong way round. That is a fact +#: about the method rather than about the constant, and a sweep would move the +#: rate without moving the ordering โ€” but nobody has run one, so that sentence +#: is an argument and not a measurement. +ONSET_K = 6.0 + +#: Two onsets closer together than this are one event. A double sneeze exists +#: and this will merge it, which biases the count *down* โ€” the safe direction +#: for a screen whose failure mode is alarming a farm about birds that are fine. +REFRACTORY_SECONDS = 0.12 + +#: An event's duration must fall inside this, in seconds. Below the floor is a +#: click or a sample dropout; above the ceiling is a door, a vehicle or a bird +#: landing on the microphone. Poultry snicks and sneezes are reported in the +#: literature at roughly 50โ€“250 ms, and the band is widened either side because +#: nothing here has measured a Nigerian house. +EVENT_SECONDS = (0.02, 0.60) + +#: Share of an event's energy that must sit above 1 kHz. +#: +#: The gate that removes thumps. A boot on litter, a feeder chain and a slammed +#: door are loud and low; a snick is not. +#: +#: **Chosen, not derived**, like `ONSET_K` above and for the same reason: the +#: sweep an earlier comment credited it to was never run. It belongs to the +#: analysis band in `features.FLUX_BAND_HZ` and would not transfer to another. +MIN_HIGH_BAND_FRACTION = 0.30 + +#: Where "high band" starts, in hertz. +HIGH_BAND_HZ = (1_000.0, 8_000.0) + +#: Normalised autocorrelation above which an event is pitched, and therefore a +#: vocalisation rather than a respiratory transient. A cluck, a crow and a +#: spoken vowel repeat; a sneeze does not. +MAX_HARMONICITY = 0.55 + +#: `machinery_dominance` above which no count is published. A recording this +#: bottom-heavy and this stationary is a fan, and the birds under it are not +#: being heard. +#: +#: **Chosen, and measured to be inert.** Across all 6,346 clips of +#: `experiments/poultry_respiratory/` the statistic peaks at 0.4999 and never +#: reaches this, so `machinery_dominates` is a rejection the product declares +#: and that set never triggers. Either those recordings are not fan-dominated, +#: or the threshold is too high to be useful; nothing separates the two, and +#: the synthetic drone in `tests/test_audio.py` scores well above it, which is +#: how a value this high came to look reasonable. +MAX_MACHINERY_DOMINANCE = 0.55 + +#: `speech_dominance` above which no count is published. +#: +#: **Chosen, and an earlier comment claimed it came from composites of real +#: speech over real farm noise. Those do not exist.** What is known about it is +#: one measurement: over 6,346 real poultry-house clips it fires 26 times, which +#: is the only one of the registry's three rejection codes that set exercises. +#: Nobody has listened to those 26 clips to check whether anybody is talking in +#: them. +MAX_SPEECH_DOMINANCE = 0.20 + + +@dataclass(frozen=True) +class Event: + """One candidate, with the evidence that made it one. + + Every field a gate looked at is kept, whether the event passed or not, so a + threshold can be re-derived from stored results rather than by going back to + recordings nobody kept. `periodicity.Periodicity` keeps its diagnostics for + the same reason and it is the thing that made re-deriving its constants + possible a month later. + """ + + start_seconds: float + peak_seconds: float + end_seconds: float + onset_strength: float + #: Multiples of the local MAD above the local median, at the peak frame. + #: This is the closest thing the method has to a per-event score, and ยง37 + #: forbids it reaching a farm: nothing has calibrated it. + prominence: float + high_band_fraction: float + harmonicity: float + peak_level_dbfs: float + #: Empty when the event was kept. Otherwise the gate that removed it. + rejected_because: str = "" + + @property + def duration_seconds(self) -> float: + return self.end_seconds - self.start_seconds + + @property + def kept(self) -> bool: + return not self.rejected_because + + +@dataclass(frozen=True) +class RespiratoryScreen: + """A count of cough- or sneeze-like events, or an account of why not. + + `count` is `None` whenever `usable` is False. There is deliberately no way + to read a number out of a refused screen โ€” the property `app/counting.py` + and `periodicity.Periodicity` both have, where a withheld number is not a + number of zero. + + **A usable screen reporting zero is a real result and a different one.** It + says the recording was analysable and held nothing cough-like, which is what + a healthy house sounds like. + """ + + usable: bool + count: int | None + events: tuple[Event, ...] + #: Everything the gates rejected, kept rather than dropped: a screen that + #: found forty transients and passed none of them is a different situation + #: from one that found none, and only this field distinguishes them. + rejected: tuple[Event, ...] + reason: str = "" + #: The registry's own rejection name, when one applies. Matches + #: `app/capabilities.py`'s `reject_if` for `poultry_respiratory` so a caller + #: can route on it rather than parsing prose. + rejection_code: str = "" + diagnostics: dict[str, float] = field(default_factory=dict) + + @property + def statement(self) -> str: + """The ยง26 wording, and nothing stronger. + + Authored here rather than in a caller because ยง26 gives the sentence + and ยง27 gives the qualifier, and separating them is how the qualifier + gets lost between a service and a screen. + """ + if not self.usable: + return self.reason + if self.count == 0: + return ( + "No cough- or sneeze-like events detected in this recording. " + "Spot respiratory screen only โ€” a 30-second sample is not " + "continuous monitoring." + ) + plural = "" if self.count == 1 else "s" + return ( + f"{self.count} cough/sneeze-like event{plural} detected. " + f"Spot respiratory screen only โ€” a 30-second sample is not " + f"continuous monitoring." + ) + + +def _segment(strength: np.ndarray, threshold: np.ndarray, peak: int) -> tuple[int, int]: + """Where the event around a peak frame starts and stops. + + Walks outwards to the first frame at or below the local threshold. Bounding + an event by its own threshold rather than by a fixed width is what lets the + duration gate mean something: a fixed window would give every event the same + duration and the gate would never fire. + """ + start = peak + while start > 0 and strength[start - 1] > threshold[start - 1]: + start -= 1 + end = peak + last = len(strength) - 1 + while end < last and strength[end + 1] > threshold[end + 1]: + end += 1 + return start, end + + +def _candidates( + spec: Spectrogram, strength: np.ndarray, threshold: np.ndarray, + refractory_frames: int, +) -> list[tuple[int, int, int]]: + """`(start, peak, end)` frame indices, strongest peak first, non-overlapping. + + Strongest-first rather than left-to-right: when two onsets fall inside one + refractory period the louder one should be the event, not whichever happened + to come first. + """ + above = np.flatnonzero(strength > threshold) + if above.size == 0: + return [] + + taken = np.zeros(spec.frames, dtype=bool) + found: list[tuple[int, int, int]] = [] + for peak in above[np.argsort(-strength[above])]: + peak = int(peak) + if taken[peak]: + continue + start, end = _segment(strength, threshold, peak) + low = max(0, peak - refractory_frames) + high = min(spec.frames, peak + refractory_frames + 1) + if taken[low:high].any(): + continue + taken[min(low, start):max(high, end + 1)] = True + found.append((start, peak, end)) + return sorted(found, key=lambda triple: triple[1]) + + +def _describe( + spec: Spectrogram, recording: Recording, strength: np.ndarray, + threshold: np.ndarray, span: tuple[int, int, int], +) -> Event: + """Measure one candidate against every gate, without applying any of them.""" + start, peak, end = span + magnitude = spec.magnitude[start:end + 1] + total = float(np.sum(magnitude ** 2)) + high_bins = spec.band(*HIGH_BAND_HZ) + high = float(np.sum(magnitude[:, high_bins] ** 2)) if high_bins.size else 0.0 + + start_sample = int(spec.times[start] * recording.sample_rate_hz + - spec.window_seconds * recording.sample_rate_hz / 2) + end_sample = int(spec.times[end] * recording.sample_rate_hz + + spec.window_seconds * recording.sample_rate_hz / 2) + window = recording.samples[max(0, start_sample):max(0, end_sample)] + peak_amplitude = float(np.max(np.abs(window), initial=0.0)) + + local_spread = float(threshold[peak] - np.median(strength)) + return Event( + start_seconds=round(float(spec.times[start]) - spec.window_seconds / 2, 4), + peak_seconds=round(float(spec.times[peak]), 4), + end_seconds=round(float(spec.times[end]) + spec.window_seconds / 2, 4), + onset_strength=round(float(strength[peak]), 4), + prominence=round( + float((strength[peak] - threshold[peak]) / max(abs(local_spread), 1e-9)), 4 + ), + high_band_fraction=round(high / max(total, 1e-30), 4), + harmonicity=round(harmonicity(window, recording.sample_rate_hz), 4), + peak_level_dbfs=round( + float(20.0 * np.log10(max(peak_amplitude, 1e-10))), 2 + ), + ) + + +def _gate(event: Event) -> str: + """The first gate this candidate fails, or an empty string. + + A candidate usually fails more than one, so the **order decides what the + rejection is called**, and the reason is read by a person diagnosing a + capture rather than by a machine. So the order is by how informative the + answer is, not by how cheap the check is. + + **Pitch is tested before frequency band, and a test is why.** A synthetic + cluck โ€” a 400 Hz fundamental with harmonics to 4 kHz โ€” measures harmonicity + 0.83 and puts 0.20 of its energy above 1 kHz. Both gates reject it and both + are telling the truth, but band-first labels it *"low-frequency"*, which + describes a slamming door. Pitch-first labels it *"pitched โ€” a vocalisation, + not a respiratory transient"*, which describes what it is. Vocalisation is + the commonest confounder in a poultry house by an enormous margin, and + mislabelling the common case to save an autocorrelation on a bounded number + of candidates is the wrong trade. + + Duration stays first because a 5 ms click and a 2-second vehicle are not + usefully described by either of the other two. + """ + low, high = EVENT_SECONDS + if event.duration_seconds < low: + return f"too short ({event.duration_seconds * 1000:.0f} ms)" + if event.duration_seconds > high: + return f"too long ({event.duration_seconds * 1000:.0f} ms)" + if event.harmonicity > MAX_HARMONICITY: + return ( + f"pitched ({event.harmonicity:.2f} autocorrelation, against " + f"{MAX_HARMONICITY:.2f}) โ€” a vocalisation, not a respiratory transient" + ) + if event.high_band_fraction < MIN_HIGH_BAND_FRACTION: + return ( + f"low-frequency ({event.high_band_fraction:.2f} of its energy above " + f"1 kHz, against {MIN_HIGH_BAND_FRACTION:.2f})" + ) + return "" + + +def screen( + recording: Recording, + *, + min_capture_seconds: float = MIN_CAPTURE_SECONDS, + onset_k: float = ONSET_K, + flux_band_hz: tuple[float, float] = FLUX_BAND_HZ, +) -> RespiratoryScreen: + """ยง26's spot screen, end to end. + + The parameters exist so the experiment can sweep them. Every default is the + value `experiments/poultry_respiratory/` derived, and a caller passing a + different one is running a different method with different accuracy. + """ + duration = recording.duration_seconds + base = {"duration_seconds": round(duration, 2)} + + if duration < min_capture_seconds: + return RespiratoryScreen( + False, None, (), (), + reason=( + f"The recording is {duration:.0f} seconds. Stand quietly and " + f"record for at least {min_capture_seconds:.0f} โ€” a house that " + f"is coughing does not do it on cue, and a shorter sample is a " + f"count of whatever happened to be in it." + ), + rejection_code="recording_too_short", + diagnostics=base, + ) + + if recording.is_silent or float( + np.max(np.abs(recording.samples), initial=0.0) + ) < SILENCE_PEAK: + return RespiratoryScreen( + False, None, (), (), + reason=( + "The recording holds no audible signal. Check that the " + "microphone was not covered." + ), + # Not one of the registry's three codes, and deliberately not + # forced into one: a dead microphone is not a house full of + # machinery. The registry should grow a fourth, and until it does + # the honest thing is to leave this blank rather than mislabel it. + rejection_code="", + diagnostics=base, + ) + + spec = spectrogram(recording.samples, recording.sample_rate_hz) + machinery = machinery_dominance(spec) + speech = speech_dominance(spec) + diagnostics = dict( + base, + machinery_dominance=round(machinery, 4), + speech_dominance=round(speech, 4), + frames=spec.frames, + peak_dbfs=round(float(20.0 * np.log10( + max(float(np.max(np.abs(recording.samples), initial=0.0)), 1e-10) + )), 2), + rms_dbfs=round(float(20.0 * np.log10( + max(float(np.sqrt(np.mean(recording.samples.astype(np.float64) ** 2))), 1e-10) + )), 2), + onset_k=onset_k, + flux_band_low_hz=flux_band_hz[0], + flux_band_high_hz=flux_band_hz[1], + ) + + if machinery > MAX_MACHINERY_DOMINANCE: + return RespiratoryScreen( + False, None, (), (), + reason=( + f"Machinery drowns this recording: {machinery:.0%} of it is " + f"steady low-frequency noise, against a " + f"{MAX_MACHINERY_DOMINANCE:.0%} limit. Record again away from " + f"the fans, or when they cycle off." + ), + rejection_code="machinery_dominates", + diagnostics=diagnostics, + ) + + if speech > MAX_SPEECH_DOMINANCE: + return RespiratoryScreen( + False, None, (), (), + reason=( + f"Somebody is talking through this recording. Record again " + f"without speaking โ€” ยง26 asks you to stand quietly, and a voice " + f"close to the microphone hides every bird in the house." + ), + rejection_code="speech_dominates", + diagnostics=diagnostics, + ) + + strength = onset_strength(spec, band_hz=flux_band_hz) + threshold = adaptive_threshold( + strength, + window_frames=int(round(ADAPTIVE_WINDOW_SECONDS / spec.hop_seconds)), + k=onset_k, + ) + refractory = int(round(REFRACTORY_SECONDS / spec.hop_seconds)) + + kept: list[Event] = [] + rejected: list[Event] = [] + for span in _candidates(spec, strength, threshold, refractory): + event = _describe(spec, recording, strength, threshold, span) + failure = _gate(event) + if failure: + rejected.append( + Event(**{**event.__dict__, "rejected_because": failure}) + ) + else: + kept.append(event) + + diagnostics["candidates"] = len(kept) + len(rejected) + diagnostics["events_per_minute"] = round( + len(kept) / (duration / 60.0), 3 + ) if duration > 0 else 0.0 + + return RespiratoryScreen( + usable=True, + count=len(kept), + events=tuple(kept), + rejected=tuple(rejected), + diagnostics=diagnostics, + ) diff --git a/app/adapters/audio/features.py b/app/adapters/audio/features.py new file mode 100644 index 0000000000000000000000000000000000000000..a70ec39b2044897713374ec58e4a5ef65344f8c9 --- /dev/null +++ b/app/adapters/audio/features.py @@ -0,0 +1,350 @@ +"""Classical audio features, in NumPy, with nothing learned. + +Directive ยง26 lists four things to test for poultry respiratory audio and +"classical audio features" is the last of them. Directive ยง4 is why it is the +one built first: *"Do not use a neural model when deterministic signal +processing is better."* The sibling module `adapters/signal/periodicity.py` is +this project's evidence that the rule pays โ€” a metronome's stated 96 beats per +minute comes back as 96.48 from an FFT and no weights at all. + +**A snick is a transient, and transients are what spectral flux is for.** A +chicken's sneeze or snick is a short broadband burst with a sharp attack. A +ventilation fan is the opposite: loud, broadband, and unchanging. Neither an +absolute level nor a spectrum tells them apart, and the *rate of change* of the +spectrum separates them at a glance. Everything here exists to compute that +difference and the three or four quantities needed to know whether it means +anything. + +**No mel filterbank, and that is deliberate.** Mel spacing exists to model human +pitch perception; nothing here is about a human ear, and a linear STFT keeps +every threshold in this file expressible in hertz, which is the unit the +poultry-audio literature states its bands in. + +Nothing in this module decides anything. It returns numbers; `events.py` applies +the thresholds and, more often, refuses. +""" + +from __future__ import annotations + +from dataclasses import dataclass + +import numpy as np + +#: STFT window, in seconds. 32 ms at 16 kHz is 512 samples, giving 31.25 Hz +#: bins. Short enough that a 60 ms snick spans several frames rather than being +#: smeared into one; long enough to resolve the low-frequency fan energy that +#: `machinery_dominance` has to measure. +WINDOW_SECONDS = 0.032 + +#: Hop, in seconds. 8 ms at 16 kHz is 128 samples โ€” a quarter of the window, so +#: an onset is localised to about a hundredth of a second. ยง26 counts events; it +#: does not need better timing than that, and a finer hop is linear cost for +#: nothing. +HOP_SECONDS = 0.008 + +#: The band spectral flux is summed over, in hertz. +#: +#: **The low edge is the load-bearing one.** Ventilation fans, extractor +#: machinery and wind on a microphone put most of their power below a few +#: hundred hertz, and a broadband flux measure that includes them tracks the +#: machinery instead of the birds. 400 Hz is above the fundamental of a fan and +#: below where a snick's energy starts. +#: +#: The high edge is the Nyquist of the analysis rate, so the band is "everything +#: above the machinery" rather than a claim about where a snick stops. +FLUX_BAND_HZ = (400.0, 8_000.0) + +#: The band `machinery_dominance` measures, in hertz. Below `FLUX_BAND_HZ`'s low +#: edge on purpose: the two are meant to be disjoint, so a recording can be +#: loud in one and quiet in the other and the pair of numbers says which. +MACHINERY_BAND_HZ = (20.0, 400.0) + +#: Voiced speech puts its fundamental here. An adult male's fundamental sits +#: around 85โ€“180 Hz and a female's around 165โ€“255; the band is widened at both +#: ends because a farmer's voice reaches a microphone through a shed. +VOICE_BAND_HZ = (80.0, 300.0) + +#: Syllable rate, in hertz. The modulation spectrum of running speech peaks +#: between 3 and 5 Hz across languages and speakers โ€” one of the most stable +#: facts in speech acoustics. The band is widened to 2โ€“8 to cover fast and slow +#: talkers without reaching the 10โ€“20 Hz region where a flock's own chatter +#: sits. +SPEECH_MODULATION_BAND_HZ = (2.0, 8.0) + +#: Percentile taken over time to estimate the stationary noise floor of each +#: frequency bin. The 10th rather than the minimum, because a single quiet frame +#: โ€” a dropout, a gap between fan blades โ€” would otherwise set the floor for the +#: whole recording. +FLOOR_PERCENTILE = 10.0 + + +@dataclass(frozen=True) +class Spectrogram: + """Magnitude STFT, with the axes needed to say what a number means.""" + + #: `(frames, bins)`, magnitude โ€” not power. Flux is computed on log + #: magnitude, and squaring first only doubles it. + magnitude: np.ndarray + #: Bin centre frequencies, in hertz. + frequencies: np.ndarray + #: Frame centre times, in seconds. + times: np.ndarray + hop_seconds: float + window_seconds: float + + @property + def frames(self) -> int: + return int(self.magnitude.shape[0]) + + def band(self, low_hz: float, high_hz: float) -> np.ndarray: + """Indices of the bins inside a band. Empty when the band is above + Nyquist, which a caller must handle rather than divide by.""" + return np.flatnonzero( + (self.frequencies >= low_hz) & (self.frequencies <= high_hz) + ) + + def band_energy(self, low_hz: float, high_hz: float) -> np.ndarray: + """Per-frame energy in a band. Zeros when the band is empty.""" + bins = self.band(low_hz, high_hz) + if bins.size == 0: + return np.zeros(self.frames, dtype=np.float64) + return np.sum(self.magnitude[:, bins] ** 2, axis=1) + + +def spectrogram( + samples: np.ndarray, + sample_rate_hz: float, + *, + window_seconds: float = WINDOW_SECONDS, + hop_seconds: float = HOP_SECONDS, +) -> Spectrogram: + """A Hann-windowed magnitude STFT, framed with `np.lib.stride_tricks`. + + Written out rather than imported because the service has neither `scipy` nor + `librosa`, and adding either for one function would be a dependency a + 2 vCPU container carries forever. + """ + samples = np.asarray(samples, dtype=np.float64).ravel() + window_length = max(8, int(round(window_seconds * sample_rate_hz))) + hop = max(1, int(round(hop_seconds * sample_rate_hz))) + + if samples.size < window_length: + return Spectrogram( + magnitude=np.zeros((0, window_length // 2 + 1)), + frequencies=np.fft.rfftfreq(window_length, d=1.0 / sample_rate_hz), + times=np.zeros(0), + hop_seconds=hop / sample_rate_hz, + window_seconds=window_length / sample_rate_hz, + ) + + frame_count = 1 + (samples.size - window_length) // hop + frames = np.lib.stride_tricks.as_strided( + samples, + shape=(frame_count, window_length), + strides=(samples.strides[0] * hop, samples.strides[0]), + writeable=False, + ) + magnitude = np.abs(np.fft.rfft(frames * np.hanning(window_length), axis=1)) + + return Spectrogram( + magnitude=magnitude, + frequencies=np.fft.rfftfreq(window_length, d=1.0 / sample_rate_hz), + # Frame *centres*, so an onset time is the middle of the window that + # holds it rather than its leading edge. An 8 ms hop makes the + # difference small and a systematic 16 ms offset in every published + # event time would still be wrong. + times=(np.arange(frame_count) * hop + window_length / 2.0) / sample_rate_hz, + hop_seconds=hop / sample_rate_hz, + window_seconds=window_length / sample_rate_hz, + ) + + +def onset_strength( + spec: Spectrogram, *, band_hz: tuple[float, float] = FLUX_BAND_HZ +) -> np.ndarray: + """Half-wave rectified spectral flux over a band, one value per frame. + + Log magnitude rather than linear, so the measure is a *relative* change and + a quiet snick between fan cycles counts as much as a loud one next to the + microphone. Rectified, because a spectrum falling away is the tail of an + event and only its arrival is an onset. + + The first frame is zero by construction: there is no frame before it to + differ from, and a large opening value is the artefact that makes a + recording's first moment look like an event. + """ + bins = spec.band(*band_hz) + if spec.frames < 2 or bins.size == 0: + return np.zeros(spec.frames, dtype=np.float64) + + # +1e-10 rather than a smaller floor: below about 1e-12 the log of a silent + # bin dominates the difference and every silence boundary reads as an onset. + logs = np.log(spec.magnitude[:, bins] + 1e-10) + flux = np.maximum(np.diff(logs, axis=0), 0.0).sum(axis=1) + return np.concatenate([[0.0], flux]) + + +def adaptive_threshold( + strength: np.ndarray, *, window_frames: int, k: float +) -> np.ndarray: + """Local median plus `k` local median-absolute-deviations. + + **Median and MAD, never mean and standard deviation.** The events being + looked for are exactly the outliers, so a mean threshold is pulled up by the + events it is meant to find, and a recording with many of them raises its own + bar until it reports few. The median is unmoved by anything under half the + window. + + Computed by sorting a strided view โ€” O(nยทw log w) and a few milliseconds for + a 60-second clip, against a rolling-median implementation this repository + would have to own. + """ + strength = np.asarray(strength, dtype=np.float64) + n = strength.size + if n == 0: + return np.zeros(0) + window = max(3, min(int(window_frames) | 1, n if n % 2 else n - 1)) + if window < 3: + centre = float(np.median(strength)) + spread = float(np.median(np.abs(strength - centre))) + return np.full(n, centre + k * spread) + + half = window // 2 + padded = np.pad(strength, half, mode="reflect") + view = np.lib.stride_tricks.sliding_window_view(padded, window) + centre = np.median(view, axis=1) + spread = np.median(np.abs(view - centre[:, None]), axis=1) + return centre + k * spread + + +def stationary_floor(spec: Spectrogram) -> np.ndarray: + """Per-bin noise floor: the `FLOOR_PERCENTILE`-th percentile over time. + + What a fan leaves behind. A bin carrying only machinery has a floor close to + its mean; a bin carrying only events has a floor near zero. + """ + if spec.frames == 0: + return np.zeros(spec.magnitude.shape[1]) + return np.percentile(spec.magnitude, FLOOR_PERCENTILE, axis=0) + + +def machinery_dominance(spec: Spectrogram) -> float: + """How much of the recording is unchanging low-frequency noise, in [0, 1]. + + Two factors multiplied, because either alone is wrong: + + **How much energy is below 400 Hz.** A shed with the fans on is bottom-heavy + and a shed with them off is not. + + **How much of that low-band energy is stationary** โ€” the per-bin floor over + the per-bin mean. A fan is nearly all floor. A door slamming is loud, low, + and not floor at all, and a measure that only looked at the band would call + it machinery. + + The product is what `events.py` compares against a threshold, and a + recording can be very loud below 400 Hz without tripping it as long as the + low band is *changing*. + """ + if spec.frames == 0: + return 0.0 + total = float(np.sum(spec.magnitude ** 2)) + if total <= 0.0: + return 0.0 + + low = spec.band(*MACHINERY_BAND_HZ) + if low.size == 0: + return 0.0 + + low_energy = float(np.sum(spec.magnitude[:, low] ** 2)) + share_of_total = low_energy / total + + floor = np.percentile(spec.magnitude[:, low], FLOOR_PERCENTILE, axis=0) + mean = np.mean(spec.magnitude[:, low], axis=0) + stationarity = float(np.mean(floor / np.maximum(mean, 1e-12))) + + return float(share_of_total * stationarity) + + +def speech_dominance(spec: Spectrogram) -> float: + """How much the recording looks like somebody talking, in [0, 1]. + + Also two factors, and again both are needed: + + **A 2โ€“8 Hz modulation peak.** Running speech opens and closes the vocal + tract at the syllable rate, which puts a peak in the modulation spectrum of + its energy envelope between 3 and 5 Hz. Fans have no modulation; a flock's + chatter modulates faster and less regularly. + + **Energy in the voicing band.** 80โ€“300 Hz is where a human fundamental + lives. On its own it is useless โ€” a fan is louder there โ€” which is why it is + a factor rather than a test. + + **This is a screen, not a speech detector, and it has never been measured + against real speech in a poultry house.** It is here because the capability + registry names `speech_dominates` as a rejection reason, and a rejection + reason with no implementation behind it is the shape of thing this project + keeps finding in its own past. + """ + if spec.frames < 8: + return 0.0 + + envelope = np.sqrt(spec.band_energy(20.0, 8_000.0)) + if float(np.max(envelope, initial=0.0)) <= 0.0: + return 0.0 + + envelope = envelope - envelope.mean() + frame_rate = 1.0 / spec.hop_seconds + power = np.abs(np.fft.rfft(envelope * np.hanning(envelope.size))) ** 2 + modulation = np.fft.rfftfreq(envelope.size, d=1.0 / frame_rate) + + # Bin 0 is the mean, already removed; including it would make every + # recording's modulation peak its own DC. + band = np.flatnonzero( + (modulation >= SPEECH_MODULATION_BAND_HZ[0]) + & (modulation <= SPEECH_MODULATION_BAND_HZ[1]) + ) + rest = np.flatnonzero((modulation > 0.0) & (modulation < 40.0)) + if band.size == 0 or rest.size == 0: + return 0.0 + modulation_share = float(np.sum(power[band]) / max(float(np.sum(power[rest])), 1e-30)) + + total = float(np.sum(spec.magnitude ** 2)) + voice = float(np.sum(spec.magnitude[:, spec.band(*VOICE_BAND_HZ)] ** 2)) + voice_share = voice / max(total, 1e-30) + + return float(min(1.0, modulation_share) * voice_share) + + +def harmonicity(frame_samples: np.ndarray, sample_rate_hz: float, + *, min_hz: float = 80.0, max_hz: float = 2_000.0) -> float: + """Strength of the strongest autocorrelation peak in a pitch range, in [0, 1]. + + A cluck, a crow and a spoken vowel are pitched: the waveform repeats, so its + normalised autocorrelation has a tall peak at the period. A snick, a sneeze + and a fan are not, and theirs does not. + + Returns 0 when the segment is too short to hold a full period at `min_hz`, + which is a refusal rather than a low score โ€” a 40 ms segment cannot be asked + whether it repeats at 80 Hz. + """ + x = np.asarray(frame_samples, dtype=np.float64).ravel() + if x.size < 8: + return 0.0 + x = x - x.mean() + energy = float(np.dot(x, x)) + if energy <= 0.0: + return 0.0 + + min_lag = max(1, int(sample_rate_hz / max_hz)) + max_lag = int(sample_rate_hz / min_hz) + if max_lag >= x.size: + return 0.0 + + # Full autocorrelation via FFT; the direct form is O(nยฒ) and this is called + # once per candidate event. + size = 1 << int(np.ceil(np.log2(2 * x.size))) + spectrum = np.fft.rfft(x, size) + correlation = np.fft.irfft(spectrum * np.conj(spectrum), size)[: x.size] + if min_lag >= max_lag: + return 0.0 + return float(np.clip(np.max(correlation[min_lag:max_lag]) / energy, 0.0, 1.0)) diff --git a/app/adapters/base.py b/app/adapters/base.py new file mode 100644 index 0000000000000000000000000000000000000000..3ee35e6c3b6eabfe62c2ad0deb99bd9e09193de6 --- /dev/null +++ b/app/adapters/base.py @@ -0,0 +1,418 @@ +"""What every adapter agrees to, and the two things none of them may do. + +The zero-training directive (ยง3, ยง4, ยง40.1) names a stack of pretrained models โ€” +SAM, DINOv3, MegaDescriptor, Grounding DINO, CountGD, a hosted reasoner โ€” plus a +deterministic OpenCV/NumPy path that ยง4 says to prefer whenever it is better. +Those have almost nothing in common at the point of use: one returns masks, one +returns a 384-dimensional vector, one returns a breath rate. So this file does +**not** try to give them a single `run`. + +What they do have in common is governance, and that is what is unified here: + +**An adapter says what it costs, and `None` means nobody measured it.** +`MeasuredCost` has no defaults and no published-figure fallback. A latency copied +from a paper is a claim about somebody else's GPU, and the farms this serves run +a 2 vCPU / 4 GiB container. + +**An adapter cannot produce a result it has no model for.** There is deliberately +no `run` on this class โ€” the same reason `providers.InferenceProvider` has none. +A base implementation would be a way to return something plausible with nothing +behind it, and that is the single failure this service exists to prevent. Work +happens on the object `load()` returns, and `load()` raises when the artefact is +absent. + +The task protocols below are the narrow interfaces callers actually use. They are +kept as small as `detectors.Detector` is, for the same reason: everything added +here is something the replacement adapter has to reimplement on the day a licence +forces a swap, and ADR 0017 is the record of that day arriving. +""" + +from __future__ import annotations + +from dataclasses import dataclass, field +from enum import Enum +from typing import Protocol, runtime_checkable + +import numpy as np +from PIL import Image + + +class Task(str, Enum): + """What an adapter produces. A model may do several.""" + + DETECT = "detect" + SEGMENT = "segment" + EMBED = "embed" + COUNT = "count" + TRACK = "track" + POSE = "pose" + #: Structured reasoning from a hosted multimodal model. Named apart from the + #: rest because ยง4 is explicit that it is "an experimental visual reasoner, + #: not an authority", and a caller should have to type the difference. + REASON = "reason" + #: Deterministic signal processing โ€” optical flow, FFT, contour geometry. + #: No weights, no licence question, and ยง4 says to prefer it where it wins. + MEASURE = "measure" + + +class Modality(str, Enum): + IMAGE = "image" + VIDEO = "video" + AUDIO = "audio" + + +@dataclass(frozen=True) +class MeasuredCost: + """Latency and memory from a run that actually happened. + + Every field is required. There is no `estimated` variant and no default, + because the only thing worse than not knowing what an adapter costs on a + 2 vCPU box is believing a number nobody produced there. + + `hardware` is free text on purpose: it has to be able to say "MacBook, 8 + performance cores, not the target" as easily as it says the container SKU, + and a reader needs to see which one they are looking at. + """ + + hardware: str + threads: int + #: What it ran on, specifically enough to re-run. A slug from + #: `evaluation/dataset.json`, or a count of frames from a named set. + sample: str + runs: int + median_seconds: float + peak_rss_mb: float + measured_on: str + + @property + def fits_cpu_service(self) -> bool: + """Whether this would survive the 2 vCPU / 4 GiB CPU worker. + + **This is a placement hint, not a verdict on the model.** A model that + returns False here belongs on a GPU host, and that is a deployment + decision rather than a reason to drop a capability. The distinction is + recorded because the earlier version of this file got it wrong and would + have excluded most of directive ยง3 on the strength of a container size. + """ + return ( + self.peak_rss_mb <= CPU_SERVICE_MEMORY_CEILING_MB + and self.median_seconds <= INLINE_LATENCY_CEILING_SECONDS + ) + + +#: Peak RSS above which an adapter will not sit comfortably beside the API on the +#: existing CPU worker. ADR 0018 measured YOLOX-m at 591 MB and YOLOX-x at +#: 1,003 MB. 2,000 MB leaves the Python process, onnxruntime's arenas and +#: Pillow's decode buffers room inside 4 GiB. +CPU_SERVICE_MEMORY_CEILING_MB = 2000.0 + +#: Wall-clock above which a capability cannot run inline on a request, wherever +#: it is hosted. ADR 0018's phrasing: "A 16-second inline request is not a +#: request; it is a timeout with a result attached." Past this a capability needs +#: a queue, not a bigger box. +INLINE_LATENCY_CEILING_SECONDS = 8.0 + + +class Placement(str, Enum): + """Where a leg of the stack should run. + + Three tiers, and the choice between them is a product decision as much as an + engineering one. Animap is offline-first (ADR 0002): a capability that needs + a round trip is one a farm cannot use in a shed with no signal, so pushing + work off the phone has a cost that a latency table does not show. + """ + + #: On the phone. The only tier that works with no signal at all. + ON_DEVICE = "on_device" + #: The existing 2 vCPU / 4 GiB CPU container, beside the API. + CPU_SERVICE = "cpu_service" + #: A GPU host. Available, and the right answer for most of directive ยง3 โ€” + #: the models it names are GPU-class work and it was written knowing that. + GPU_SERVICE = "gpu_service" + + +@dataclass(frozen=True) +class AdapterSpec: + """An adapter's identity, licence position and measured cost. + + This is committed code rather than a JSON card, and that is the point. + `providers.ModelArtefact` reads a card, and a card is written by whoever + writes the card โ€” ADR 0017 records a watchdog defeating the licence gate by + declaring `Apache-2.0` over a path to AGPL weights. The `runtime` here names + which loader runs, which is a fact about the code and not a claim about + terms, and `adapters.licences` holds what that runtime's weights are really + licensed under. + """ + + adapter_id: str + #: Which loader runs. The key into `licences.RUNTIME_LICENCES`, and the only + #: field the licence gate trusts. + runtime: str + tasks: tuple[Task, ...] + modalities: tuple[Modality, ...] + #: What the zero-training directive asks this model for, quoted closely + #: enough that a reader can find the section. + directive_role: str + #: False for the deterministic methods โ€” optical flow, FFT, contour + #: geometry. They need no weights, so they have no artefact to be absent and + #: no licence to refuse, which is most of why ยง4 prefers them. + requires_artefact: bool = True + #: `None` until somebody runs it and writes the number down. Reported as + #: "not measured", never filled in from a paper. + measured: MeasuredCost | None = None + #: Where this leg should run. A recommendation with a reason, not a + #: constraint โ€” see `Placement`. + placement: Placement = Placement.CPU_SERVICE + #: Whether a GPU is needed for this to be usable at all, as opposed to + #: merely faster. Recorded separately from `placement` because "runs on CPU + #: but slowly" and "does not run on CPU" are different facts and only the + #: second one closes a door. + requires_gpu: bool = False + placement_reason: str = "" + notes: str = "" + + +@dataclass(frozen=True) +class Availability: + """Whether an adapter can run, and if not, what would change that. + + `remedy` exists because "unavailable" without it is the answer that gets + read as "broken". The service already distinguishes *"no validated model + exists"* from *"this is not planned"* in `main._unavailable_reason`, and an + adapter that cannot say which of those it is has lost the distinction. + """ + + ready: bool + #: Empty when ready. Otherwise says what is missing, not what went wrong. + reason: str = "" + remedy: str = "" + + def __post_init__(self) -> None: + if not self.ready and not self.reason: + raise ValueError( + "An unavailable adapter must say why. A bare False is what a " + "caller renders as a silent failure." + ) + + +class AdapterError(RuntimeError): + """The adapter is present but could not do the work.""" + + +class AdapterUnavailable(AdapterError): + """No model behind this adapter, so there is nothing to run. + + Raised by `load()`, never returned as a result. A caller that catches this + reports `unavailable` โ€” the state `JobState.UNAVAILABLE` already exists for, + and which is the honest answer for most of the stack today. + """ + + def __init__(self, availability: Availability) -> None: + self.availability = availability + message = availability.reason + if availability.remedy: + message = f"{message} {availability.remedy}" + super().__init__(message) + + +class Adapter: + """A pretrained model, or a deterministic method, behind one interface. + + **There is no `run` here, and adding one would be the bug.** Subclasses + expose whichever task protocol they satisfy โ€” `Embedder`, `Segmenter`, + `Reasoner` โ€” and only after `load()` has succeeded against a real artefact. + A default implementation on this class would be a way to answer a farmer + with no model in the loop. + """ + + spec: AdapterSpec + + def availability(self) -> Availability: + """Whether this adapter could run right now. + + Must not load anything. Called on `/health` and `/capabilities`, which + a platform probe hits often enough that reading a hundred megabytes of + weights to answer it would be its own outage. + """ + raise NotImplementedError + + def load(self) -> "Adapter": + """Prepare the runtime, or raise `AdapterUnavailable`. + + Returns self so a caller can write `adapter.load().embed(image)` and + have no path to `embed` that skipped the check. + """ + raise NotImplementedError + + def describe(self) -> dict[str, object]: + """Everything a governance reader needs, including what is unmeasured.""" + from app.adapters import licences + + availability = self.availability() + licence = licences.RUNTIME_LICENCES.get(self.spec.runtime) + cost = self.spec.measured + return { + "adapter_id": self.spec.adapter_id, + "runtime": self.spec.runtime, + "tasks": [t.value for t in self.spec.tasks], + "modalities": [m.value for m in self.spec.modalities], + "directive_role": self.spec.directive_role, + "ready": availability.ready, + "reason": availability.reason, + "remedy": availability.remedy, + "licence": licence.licence if licence else "unknown runtime", + "licence_source": licence.source_url if licence else "", + "servable": bool(licence and licence.servable), + "placement": self.spec.placement.value, + "requires_gpu": self.spec.requires_gpu, + "placement_reason": self.spec.placement_reason, + # The absence is the finding, so it is spelled rather than nulled. + "measured": ( + { + "hardware": cost.hardware, + "threads": cost.threads, + "sample": cost.sample, + "runs": cost.runs, + "median_seconds": cost.median_seconds, + "peak_rss_mb": cost.peak_rss_mb, + "measured_on": cost.measured_on, + "fits_cpu_service": cost.fits_cpu_service, + } + if cost is not None + else "not measured" + ), + "notes": self.spec.notes, + } + + +# --- Task protocols. Narrow on purpose. -------------------------------------- + + +@dataclass(frozen=True) +class Region: + """A box, a mask, or both. The common currency of detection and segmentation. + + `box` is in source-image pixels, matching `detectors.Detection`, so a caller + that already knows how to read a YOLOX box does not learn a second convention. + `mask` is a boolean array at source-image resolution, or `None` when the + adapter only localises. + """ + + label: str + score: float + box: tuple[float, float, float, float] + mask: np.ndarray | None = None + area_fraction: float = 0.0 + + @property + def has_mask(self) -> bool: + return self.mask is not None + + +@runtime_checkable +class Embedder(Protocol): + """Frozen features. ยง3's instruction for DINOv3 is explicit that this comes + before any fine-tuning: "Start with frozen embeddings + nearest-neighbor + retrieval." + """ + + #: Length of the vector `embed` returns. Recorded because a retrieval index + #: built at one dimension and queried at another fails silently. + dimensions: int + + def embed(self, image: Image.Image) -> np.ndarray: + """One L2-normalised float32 vector. + + Normalised by the adapter rather than the caller, so cosine similarity + is a dot product everywhere and no index has to remember which + convention it was built under. + """ + ... + + +@runtime_checkable +class Segmenter(Protocol): + def segment( + self, image: Image.Image, *, concepts: tuple[str, ...] = () + ) -> list[Region]: + ... + + +@runtime_checkable +class OpenVocabularyDetector(Protocol): + """Text-prompted detection. ยง4 names Grounding DINO as the fallback for when + SAM's concept prompting is weak.""" + + def detect_text( + self, image: Image.Image, prompts: tuple[str, ...] + ) -> list[Region]: + ... + + +@runtime_checkable +class ExemplarCounter(Protocol): + """Zero-shot counting, optionally guided by example boxes (ยง4, CountGD).""" + + def count( + self, + image: Image.Image, + *, + text: str = "", + exemplars: tuple[tuple[float, float, float, float], ...] = (), + ) -> "CountEstimate": + ... + + +@dataclass(frozen=True) +class CountEstimate: + """A count, or an honest refusal to publish one. + + `value` is `None` when the method ran and the result should not be shown โ€” + the same shape `app/counting.py` already uses, where withholding is a first + class outcome rather than an exception. ยง6.3 requires the distinction + between *visible count*, *unique birds observed* and *reconciled population* + to survive to the UI, so `kind` carries it. + """ + + value: float | None + kind: str + withheld_reason: str = "" + confidence: float | None = None + + +@runtime_checkable +class Reasoner(Protocol): + """A hosted multimodal model. + + ยง4: "All calls must return structured JSON" and "The multimodal model is an + **experimental visual reasoner**, not an authority." Both are enforced in + the adapter rather than left to the prompt โ€” see `adapters/multimodal.py`. + """ + + def reason( + self, + images: list[Image.Image], + *, + schema: dict, + rubric: str, + ) -> dict: + ... + + +@dataclass(frozen=True) +class Measurement: + """A number a deterministic method produced, with its own quality verdict. + + Signal processing fails differently from a model: it does not become + uncertain, it becomes wrong in a way that still returns a float. So a + measurement carries the evidence that the signal was there at all โ€” + `support` is whatever the method uses to know it measured something rather + than measuring noise, and `usable` is its own judgement about that. + """ + + kind: str + value: float | None + unit: str + usable: bool + support: dict[str, float] = field(default_factory=dict) + detail: str = "" diff --git a/app/adapters/claims.py b/app/adapters/claims.py new file mode 100644 index 0000000000000000000000000000000000000000..3fbb82814c0b2c2f2047d00a0ae2a0dfaf454ab8 --- /dev/null +++ b/app/adapters/claims.py @@ -0,0 +1,2781 @@ +"""What a hosted reasoner is allowed to say, checked rather than requested. + +Directive ยง4 asks for a hosted multimodal model and says two things about it that +are not decoration: *"All calls must return structured JSON"* and *"The +multimodal model is an **experimental visual reasoner**, not an authority."* +This module is the second sentence written as code. + +**The control is the shape of the answer, not a scan of its prose.** An earlier +version of this defence was a list of banned sentences, and a watchdog published +24 of 24 evasions through it: ยง10's forbidden sentence with its words reordered, +a Cyrillic `ั` in place of a Latin one, a zero-width space, a doubled space, and +`best_estimate: 2.6347` โ€” the exact false precision ยง7 forbids โ€” because nothing +looked at the number at all. Every one of those beat a substring match, and a +substring match will keep losing, because the attacker picks the string. + +So the order of the checks is inverted from what it was: + +1. **Parse strictly.** `NaN`, `Infinity` and duplicate keys are refused before + anything reads the object. `json.loads` accepts all three by default. +2. **Validate the shape against a closed schema.** Unknown keys are refused, + which is the check that matters most: a model cannot make a claim in a field + nobody declared. Types, enums, bounds, array lengths and numeric granularity + are all checked, so `2.6347` fails on `multipleOf` rather than on a regex. +3. **Validate the vocabulary against the capability.** Claims are identifiers + drawn from `Capability.acquisition.allowed_claims`, never sentences. A claim + that is not in the allowed set has nowhere to be emitted from, and one in + `forbidden_claims` or in the registry-wide `FORBIDDEN_CLAIMS` is refused by + name. +4. **Validate the observation/interpretation split** by constructing + `app.schemas.Observation` and `app.schemas.Interpretation`. `multimodal.py` + used to claim it did this and did not import the module. +5. **Refuse a quantity in free text that no structured field carries, in the + unit the prose wrote it in.** `check_quantities`, and the newest of these. It + reads the digits rather than the sentence, so a rephrasing does not move it โ€” + and it reads what they are *measured in*, so a percentage cannot ride on a + bird's weight in kilograms. +6. **Only then, scan the prose** โ€” as a backstop, and labelled as one. + +## `evidence` was free text, and that is what every attack used + +*"There is no field for a diagnosis to go in"* is the argument for steps 2 and +3, and for a long time it was not true: `schema_for` declared `evidence` and +`limits` as arrays of free-text strings, and `evidence` is what a farmer reads +under the claim. So prose could reach a person, and every successful attack on +this module landed there. + +**The paragraph that used to be here said the directive ruled out enumerating +it.** It quoted ยง7's worked output โ€” `"evidence": ["prominent hooks", "visible +pins", "limited fat around tail head"]` โ€” and called those *sentences a person +reads* that a closed vocabulary would flatten. Read again, they are the +opposite: they are observable features from a rubric, and the same is true of +ยง10's *"Multiple raised nodular lesions visible"*, ยง19's four lines, ยง11's six +hoof findings, ยง12's six faecal appearances and ยง38's bands. Every one is an +item a capability can enumerate, and ยง4's *"All calls must return structured +JSON"* means this rather than *"parses, and has these top-level keys"*. + +So `evidence` and `limits` are enums now, per capability, built from +`AcquisitionProtocol.allowed_evidence` and `allowed_limits`. ADR 0024 carries +the decision and what it costs. Three agents hardened the prose scan over that +field; each closed real phrasings and each was beaten by a rephrasing, and the +verdict converged on *"the identifier control is real, the publication control +is theatre."* **Anything that scans prose loses to an author who rephrases** โ€” +this repository learned it once already in Android's `NoInventedDataTest` โ€” so +the field stopped being prose. + +Three consequences run through this module: + +- **Step 5 skips a string the registry itself authored, and nothing else.** + Every phrase in the two enums was written by a person in + `app/capabilities.py` and reviewed in a diff, so a digit inside one is a digit + a reviewer approved rather than one a model chose โ€” ยง18's *"Approximate grade + on the 0-4 rubric"*, ยง6.3's benchmark footnote. The exemption is per string + (`_declared_phrases`) and deliberately not per field: `reason()` accepts a + caller-supplied schema with no capability and an open `evidence`, and this + step is the only thing standing on that path. A string nobody in the registry + wrote is prose whatever field it arrived in. +- **So step 5 still carries `observations[].unit`, invented keys, the + caller-schema path, and any payload built by hand.** It is not deleted and it + is not weakened; it has stopped being the only thing between a model and a + farmer. +- **`observations[].value` had to be closed at the same time.** It is + `float | str | None`, and the string half was the other place a farmer reads + words. Enumerating `evidence` and leaving that open would have moved the + problem one field sideways rather than solving it, so `check_observation_values` + is step 3b and `ClaimQuantity.values` is where the words are declared. Three + claims out of the registry's 94 keep free text โ€” an animal's name and two + breeds โ€” and each says in the registry why, all three sit under human + confirmation, and a free value still has to be shaped like a name rather than + like a sentence (`_NAME_SHAPE`). + +## Step 6 is deliberately last and deliberately weakest + +It exists for the assertions that carry no number and sit under a claim the +vocabulary did allow. Text is NFKC-normalised, stripped of zero-width and +formatting characters, folded through a confusables table, lowercased, +digit-folded, and reduced to whitespace-separated alphanumeric tokens. Concepts +are matched as *conjunctions within a window* rather than as substrings, so +reordering the words of a forbidden sentence does not evade it. + +**It is still a vocabulary, and a vocabulary loses to a phrasing nobody +listed.** Step 5 is the answer wherever a claim carries a digit. Where it does +not, one concept reduces the cost as far as it goes: `disease_named` refuses a +disease name that is *not* qualified, rather than catching one that is asserted. +That inverts what a gap costs โ€” a disease nobody listed fails to be caught, +instead of a phrasing nobody listed getting published โ€” and names are a smaller, +far more stable space than phrasings. It is why *"This is coccidiosis"*, which +says no word the older paired rule looked for, is refused. + +The concepts that pair a subject with an assertion do not have that shape and +cannot be given it, because their subjects are not enumerable. Whatever they +miss is caught by a human, because ยง32 makes confirmation a feature. +""" + +from __future__ import annotations + +import json +import math +import re +import unicodedata +from dataclasses import dataclass +from typing import Any + +from app.capabilities import FORBIDDEN_CLAIMS, Capability, OutputSpec +from app.schemas import Interpretation, Observation + + +class ContractViolation(RuntimeError): + """The model returned something the product may not publish. + + Raised rather than repaired. A repaired answer is one nobody can trace back + to what the model actually said, and ยง33 requires the verbatim response to + survive for later training. + """ + + +# --- Step 1: parsing that does not accept what JSON's defaults accept --------- + +#: `json.loads` maps these to Python floats without complaint, and every one of +#: them reaches a farmer as a number. `NaN` in particular compares false against +#: every bound a validator could check, so it has to be refused at the door. +_NON_FINITE = ("NaN", "Infinity", "-Infinity") + + +def _refuse_constant(name: str) -> Any: + raise ContractViolation( + f"The reasoner emitted the JSON constant {name}, which is not a number a " + f"result may carry. JSON's own grammar excludes it and Python's parser " + f"accepts it anyway." + ) + + +def _refuse_duplicate_keys(pairs: list[tuple[str, Any]]) -> dict[str, Any]: + """Reject an object that names the same key twice. + + Last-wins is the default, and it is a way to smuggle a value past a reader: + the object a person reviews and the object a program parses are different + objects. + """ + seen: dict[str, Any] = {} + for key, value in pairs: + if key in seen: + raise ContractViolation( + f"The reasoner returned the key {key!r} twice in one object. " + f"Which value counts depends on the parser, so neither does." + ) + seen[key] = value + return seen + + +def parse_strict(raw: str) -> dict[str, Any]: + """One JSON object, or a refusal. Never a repair. + + ยง4 says the calls must return structured JSON. A response that did not is a + failed call, and scraping an object out of prose is how a malformed answer + becomes a stored result. + """ + try: + parsed = json.loads( + raw, + parse_constant=_refuse_constant, + object_pairs_hook=_refuse_duplicate_keys, + ) + except ContractViolation: + raise + except json.JSONDecodeError as exc: + raise ContractViolation( + f"The reasoner did not return JSON ({exc}). The response is " + f"discarded rather than repaired." + ) from exc + + if not isinstance(parsed, dict): + raise ContractViolation( + f"The reasoner returned a {type(parsed).__name__}, not an object." + ) + + # `1e999` overflows to `inf` inside the number grammar, so `parse_constant` + # never sees it. The only way to catch that one is to look at the values. + _refuse_non_finite(parsed, "$") + return parsed + + +def _refuse_non_finite(value: Any, path: str) -> None: + if isinstance(value, float) and not math.isfinite(value): + raise ContractViolation( + f"{path} is {value!r}, which is not a finite number. A literal like " + f"`1e999` overflows inside JSON's number grammar, so it arrives as " + f"infinity without ever being a named constant." + ) + if isinstance(value, dict): + for key, item in value.items(): + _refuse_non_finite(item, f"{path}.{key}") + elif isinstance(value, list): + for index, item in enumerate(value): + _refuse_non_finite(item, f"{path}[{index}]") + + +# --- Step 2: a closed schema, validated --------------------------------------- + +_TYPES: dict[str, tuple[type, ...]] = { + "object": (dict,), + "array": (list,), + "string": (str,), + "number": (int, float), + "integer": (int,), + "boolean": (bool,), + "null": (type(None),), +} + + +def harden(schema: dict[str, Any]) -> dict[str, Any]: + """Close every object in a schema that does not explicitly open itself. + + JSON Schema's default is `additionalProperties: true`, which means the + default posture of a hand-written schema is *"say anything you like as long + as you also say these"*. That is the opposite of what ยง4 wants, and it is + what let a `diagnosis` key ride along beside a conforming `range`. A caller + that genuinely needs an open object writes `additionalProperties: true` and + a reviewer sees it in the diff. + """ + if not isinstance(schema, dict): + return schema + out = dict(schema) + if "properties" in out: + out["properties"] = { + key: harden(value) for key, value in out["properties"].items() + } + declared = out.get("type") + names = [declared] if isinstance(declared, str) else list(declared or ()) + if "properties" in out or "object" in names: + # Closed even when the schema declares no properties at all. A caller + # writing `{"type": "object", "required": []}` has declared that nothing + # is expected, and the reading that lets a model answer it with anything + # is the reading that let `{"notes": {"deep": [...]}}` through. + out.setdefault("additionalProperties", False) + if "items" in out: + out["items"] = harden(out["items"]) + return out + + +def validate(value: Any, schema: dict[str, Any], path: str = "$") -> list[str]: + """Every way this value fails this schema, rather than the first. + + All of them, because a caller fixing a prompt wants the whole list and a + reviewer reading a refusal wants to see how far off the response was. + """ + errors: list[str] = [] + if not isinstance(schema, dict): + return errors + + declared = schema.get("type") + if declared is not None: + names = [declared] if isinstance(declared, str) else list(declared) + permitted = tuple(t for name in names for t in _TYPES.get(name, ())) + # `bool` is a subclass of `int`, so a schema asking for a number would + # otherwise accept `true`. + matched = isinstance(value, permitted) and not ( + isinstance(value, bool) and "boolean" not in names + ) + if not matched: + errors.append( + f"{path} is {type(value).__name__}, and the schema asks for " + f"{' or '.join(names)}" + ) + return errors + + if "enum" in schema and value not in schema["enum"]: + errors.append( + f"{path} is {value!r}, which is not one of the " + f"{len(schema['enum'])} values this field may take" + ) + + if "const" in schema and value != schema["const"]: + errors.append(f"{path} must be {schema['const']!r}, and is {value!r}") + + if isinstance(value, dict): + errors.extend(_validate_object(value, schema, path)) + elif isinstance(value, list): + errors.extend(_validate_array(value, schema, path)) + elif isinstance(value, str): + errors.extend(_validate_string(value, schema, path)) + elif isinstance(value, (int, float)) and not isinstance(value, bool): + errors.extend(_validate_number(value, schema, path)) + + return errors + + +def _validate_object(value: dict, schema: dict, path: str) -> list[str]: + errors: list[str] = [] + properties = schema.get("properties", {}) + + for key in schema.get("required", []): + if key not in value: + errors.append(f"{path}.{key} is required and was omitted") + + if schema.get("additionalProperties", True) is False: + for key in value: + if key not in properties: + errors.append( + f"{path}.{key} is not a field this capability declares. A " + f"claim made in an undeclared field is a claim nobody " + f"reviewed" + ) + + for key, item in value.items(): + if key in properties: + errors.extend(validate(item, properties[key], f"{path}.{key}")) + return errors + + +def _validate_array(value: list, schema: dict, path: str) -> list[str]: + errors: list[str] = [] + if "minItems" in schema and len(value) < schema["minItems"]: + errors.append(f"{path} has {len(value)} items, and needs {schema['minItems']}") + if "maxItems" in schema and len(value) > schema["maxItems"]: + errors.append( + f"{path} has {len(value)} items, and may have at most " + f"{schema['maxItems']}" + ) + if "items" in schema: + for index, item in enumerate(value): + errors.extend(validate(item, schema["items"], f"{path}[{index}]")) + if schema.get("ascending") and len(value) == 2: + try: + if value[0] > value[1]: + errors.append(f"{path} is {value!r}, and a range runs low to high") + except TypeError: + pass + return errors + + +def _validate_string(value: str, schema: dict, path: str) -> list[str]: + errors: list[str] = [] + if "maxLength" in schema and len(value) > schema["maxLength"]: + errors.append( + f"{path} is {len(value)} characters, against a " + f"{schema['maxLength']} limit. A field that holds an essay is a " + f"field that holds an argument" + ) + if "minLength" in schema and len(value) < schema["minLength"]: + errors.append(f"{path} is shorter than {schema['minLength']} characters") + if "pattern" in schema and not re.search(schema["pattern"], value): + errors.append(f"{path} does not match {schema['pattern']!r}") + return errors + + +def _validate_number(value: float, schema: dict, path: str) -> list[str]: + errors: list[str] = [] + if "minimum" in schema and value < schema["minimum"]: + errors.append(f"{path} is {value}, below the minimum {schema['minimum']}") + if "maximum" in schema and value > schema["maximum"]: + errors.append(f"{path} is {value}, above the maximum {schema['maximum']}") + step = schema.get("multipleOf") + if step: + # ยง7: "Do not output BCS 2.63. False precision is worse than a broad + # useful estimate." The rubric's own granularity is half a point, so a + # value off that grid is not a more precise answer โ€” it is a claim the + # rubric cannot make. + # + # **This is why `multipleOf` can be relied on for halves here, and the + # question is a fair one to ask**: `2.63 % 0.5` is `0.1299999999999999` + # rather than `0.13`, and JSON Schema implementations disagree about + # what tolerance to allow. Two facts settle it for this service. First, + # nothing here uses a third-party validator โ€” `jsonschema` is not a + # dependency and this function is the only thing that reads the keyword + # โ€” so the behaviour is whatever is written below and not whatever a + # library chose. Second, the comparison is on the *quotient* with an + # absolute tolerance, not on a remainder, and 0.5 and 1 are exactly + # representable in binary, so every legal value divides exactly: + # measured, 2.63 and 2.6347 are refused and 1.0 through 5.0 in halves + # are all accepted. + # + # The tolerance does admit a value 1e-9 off the grid โ€” 2.5000000001 + # passes. That is float noise from an exporter or a vendor's JSON + # serialiser, it renders to a farmer as 2.5, and it is not the false + # precision ยง7 is about. + remainder = abs(value / step - round(value / step)) + if remainder > 1e-9: + errors.append( + f"{path} is {value}, which is not a multiple of {step}. ยง7 " + f"forbids narrowing a rubric to look precise" + ) + return errors + + +def _numeric_schema( + minimum: float | None, maximum: float | None, step: float | None +) -> dict[str, Any]: + """The three keywords, omitting the ones this capability does not declare.""" + schema: dict[str, Any] = {} + if minimum is not None: + schema["minimum"] = minimum + if maximum is not None: + schema["maximum"] = maximum + if step is not None: + schema["multipleOf"] = step + return schema + + +def check_numeric_bounds( + payload: dict[str, Any], capability: Capability +) -> list[str]: + """Every number in this response, against what the registry says is possible. + + **The check the bounds were declared for and did not have.** `OutputSpec` + carries `plausible_min`, `plausible_max` and `step` on all sixteen numeric + capabilities, `GET /capabilities` publishes them, and until this function + existed nothing read them: `schema_for` emitted bare numbers, so a watchdog + published 100,000 breaths a minute, a 0.004 kg cow, a footpad grade of 99 on + a 0-4 rubric, and `best_estimate: 2.63` on `cattle_bcs` โ€” ยง7's own named + forbidden example. + + `schema_for` now carries the same three keywords on `range` and + `best_estimate`, which is where those two landed and where the constraint is + unambiguous. This runs as well as that, and not instead of it, for two + reasons: + + - **`observations[].value` cannot be bounded by the schema alone.** One + capability's vocabulary mixes quantities โ€” ยง18 pairs a 0-4 footpad grade + with a sampled prevalence in percent โ€” so a single `minimum`/`maximum` on + that field would refuse the directive's own worked example. The bound has + to be chosen per claim, which is what `OutputSpec.measured_claims` and + `bounds_for` are for and what a flat JSON Schema property cannot express. + - **A caller-supplied schema must not be a way past it.** `reason()` lets a + caller override the contract; the override is hardened and validated, but + it carries whatever bounds the caller wrote. This check reads the registry + instead, so the numbers a capability may publish do not depend on which + schema was handed in. + """ + output = capability.acquisition.output + errors: list[str] = [] + + def check(value: Any, bounds: dict[str, Any], path: str) -> None: + if bounds and isinstance(value, (int, float)) and not isinstance(value, bool): + errors.extend(_validate_number(value, bounds, path)) + + # **A zero-width `range` is an exact figure wearing a range's clothes.** + # `[412, 412]` satisfies `minItems`, `maxItems`, `ascending` and every + # bound, so a watchdog published an exact weight through the field that + # exists to prevent one โ€” ยง22 prints "350-430 kg" and calls the initial + # range wide precisely because the capability cannot support one number. + # + # The convention that blocked this turned out not to be service-wide: it + # was one helper in `tests/test_dispositions.py` building `[edge, edge]` + # to assert a boundary reading still publishes, which made the suite + # depend on the hole. That helper now builds a real band. + errors.extend(output.range_violations(payload.get("range"))) + + if output.is_numeric: + headline = _numeric_schema( + output.plausible_min, output.plausible_max, output.step + ) + # `range` and `best_estimate` carry the capability's own quantity by + # definition โ€” there is nothing else they could be reporting. + for index, item in enumerate(payload.get("range") or []): + check(item, headline, f"$.range[{index}]") + check(payload.get("best_estimate"), headline, "$.best_estimate") + + for index, entry in enumerate(payload.get("observations") or []): + if not isinstance(entry, dict): + continue + claim = entry.get("type") + if not isinstance(claim, str): + continue + check( + entry.get("value"), + _numeric_schema(*output.bounds_for(claim)), + f"$.observations[{index}].value", + ) + return errors + + +# --- Step 3 and 4: vocabulary, and the observation/interpretation split ------- + + +def check_vocabulary( + payload: dict[str, Any], capability: Capability +) -> list[str]: + """Every claim identifier in this response, checked against the registry. + + A claim is an identifier here, never a sentence. `allowed_claims` is a + closed set per capability, so *"lumpy skin disease confirmed"* has nowhere + to be emitted from โ€” there is no field it fits and no value it may take. + That is the control ยง10 actually needs, and it does not depend on guessing + how a model will phrase itself. + """ + allowed = set(capability.acquisition.allowed_claims) + forbidden = set(capability.acquisition.forbidden_claims) | set(FORBIDDEN_CLAIMS) + errors: list[str] = [] + + for field, values in _claim_fields(payload): + for value in values: + if value in forbidden: + errors.append( + f"{field} claims {value!r}, which this capability forbids " + f"by name in the registry" + ) + elif value not in allowed: + errors.append( + f"{field} claims {value!r}, which is not in " + f"{capability.key}'s allowed vocabulary. A claim nobody " + f"declared is a claim nobody reviewed" + ) + return errors + + +def _claim_fields(payload: dict[str, Any]) -> list[tuple[str, list[str]]]: + """The three places a claim identifier may appear, and only those.""" + found: list[tuple[str, list[str]]] = [] + claims = payload.get("claims") + if isinstance(claims, list): + found.append(("claims", [c for c in claims if isinstance(c, str)])) + for key, member in (("observations", "type"), ("interpretations", "label")): + entries = payload.get(key) + if isinstance(entries, list): + found.append(( + key, + [ + entry[member] + for entry in entries + if isinstance(entry, dict) and isinstance(entry.get(member), str) + ], + )) + return found + + +#: What a free-text observation value may look like: a **name**, not a sentence. +#: +#: Three claims carry a string this registry cannot enumerate โ€” an animal's own +#: name and two breeds โ€” and leaving them merely "free" reopened the field the +#: whole change exists to close. A watchdog published seventeen strings through +#: them, sixteen forbidden, one of them 3,166 characters long and rendered to a +#: farmer as an animal's name. +#: +#: **What a name is, and it is not a judgement call.** *"Kofi"*, *"White +#: Fulani"*, *"Sokoto Gudali"*, *"N'Dama"* and *"White-Fulani-like"* are one to +#: three words of letters, hyphens and apostrophes. A digit, a colon, a comma, a +#: full stop, a fourth word or a fortieth character is a sentence wearing a +#: name's field, and none of the five above needs any of them. +#: +#: Measured against the watchdog's seventeen: **fifteen die here** โ€” every one +#: carrying a digit, a colon or more than three words. `_LENGTH` alone kills the +#: paragraph; the token cap kills *"The animal has a fever"*, *"Do not harvest +#: this flock yet"* and *"The whole flock is underweight"*; the charset kills +#: *"Uniformity band: poor"*, *"Sample CV: 14.38."* and the rest. +#: +#: **Two survive, and probing wider than the watchdog did, five of seven +#: two-word assertions survive.** *"Febrile animal"*, *"Lameness score +#: severe"*, *"Uniformity poor"*, *"Flock underweight"* and *"Pregnant cow"* +#: are bare words in a name's shape, and **nothing that reads the string can +#: tell them from "White Fulani"** โ€” both are two capitalised words naming a +#: property of an animal. The two that do die, *"Dead bird"* and *"Lumpy skin +#: disease"*, die on `scan_prose`, which is the tier this module calls +#: beatable. So the last three free-text claims in the contract are defended by +#: a vocabulary again, and that is said here rather than left to be found. +#: +#: Three things bound it and none is the shape rule: the value reaches a farmer +#: **as a name**, under ยง6.4's *"This looks like โ€ฆ"* beside a Confirm button; +#: two or three words cannot carry a figure, a scope or an instruction; and both +#: capabilities are `HUMAN_CONFIRMATION`. **The real close is a product change +#: outside this service** โ€” the app knows the farm's register and could supply +#: the animal by id, and ยง6.5 could carry a breed list per region. Neither is +#: this module's to make, and neither is pretended here. +#: `tests/test_claims.py::TestAFreeTextValueMustBeShapedLikeAName` holds the +#: measurement. +_NAME_SHAPE = re.compile(r"^[^\W\d_][\w'\- ]*$", re.UNICODE) +_NAME_LENGTH = 40 +_NAME_WORDS = 3 + + +def _name_violations( + value: str, claim: str, index: int, reason: str +) -> list[str]: + """Whether this free-text value is shaped like a name. See `_NAME_SHAPE`.""" + path = f"$.observations[{index}].value" + if len(value) > _NAME_LENGTH: + return [ + f"{path} is {len(value)} characters. {claim!r} carries a name โ€” " + f"{reason} โ€” and a name is not {_NAME_LENGTH} characters long" + ] + if len(value.split()) > _NAME_WORDS: + return [ + f"{path} is {len(value.split())} words. {claim!r} carries a name, " + f"and more than {_NAME_WORDS} words is a sentence in a field that " + f"reaches a farmer as a name" + ] + if not _NAME_SHAPE.match(value): + return [ + f"{path} is {value!r}, which is not shaped like a name. {claim!r} " + f"carries one โ€” {reason} โ€” and a name has no digit and no " + f"punctuation beyond a hyphen or an apostrophe" + ] + return [] + + +def check_observation_values( + payload: dict[str, Any], capability: Capability +) -> list[str]: + """A word in an observation's `value`, against the words the claim declares. + + **The field enumerating `evidence` would otherwise have pushed the problem + into.** `Observation.value` is `float | str | None`; `check_numeric_bounds` + looks only at numbers, and ADR 0022 recorded the string half as a residual + it could not close: *"refusing strings would take the word away from the 50 + claims whose only honest value is a word."* + + That was true of a blanket refusal and false of an enumeration, which is the + same mistake this module made about `evidence`. So the words are declared + per claim in `ClaimQuantity.values` โ€” ยง38's four tick bands, ยง11's normal or + abnormal, ยง19's sites, ยง6.6's two sexes โ€” and everything else is refused. + Most claims declare none, which is not a gap: ยง12's six faecal appearances + are six *claims*, and a word beside `visible_mucus` adds nothing a farmer + reads while being somewhere to write a sentence. + + **Three claims out of the registry's 94 keep free text**, and the registry + says why on each: `identity_candidate` carries the animal's own name (ยง6.4's + *"This looks like Kofi"*), and `likely_breed` and `breed_like_phenotype` + carry a breed (ยง6.5's *"Likely White Fulani"* and *"White-Fulani-like + phenotype"*), because an enum of a farm's animals or of a region's breeds + would refuse the crossbred animal ยง6.5 exists to protect. All three sit + under `HUMAN_CONFIRMATION`. + + **Free does not mean unshaped, and the first version of this let it.** A + watchdog published sixteen forbidden strings through those three claims, + including four the suite asserted had nowhere to land โ€” ยง30's fever from + RGB written as *"The animal has a fever"*, ADR 0023's *"This flock is not + uniform enough to harvest"*, ยง24's *"Lameness score: severe"* โ€” plus a + 3,166-character paragraph rendered to a farmer as an animal's name. The + field was the one the whole change exists to delete, kept open and defended + by the prose rules this module's own docstring calls beatable. + + So a free value has to look like **a name**: `_NAME_SHAPE`. Sixteen of those + seventeen strings die on it. See that constant for what survives and why the + residual is what it is. + + Runs only with a capability, like the other registry-backed checks. Without + one there is no vocabulary to check against. + """ + output = capability.acquisition.output + errors: list[str] = [] + for index, entry in enumerate(payload.get("observations") or []): + if not isinstance(entry, dict): + continue + value = entry.get("value") + claim = entry.get("type") + if not isinstance(value, str) or not isinstance(claim, str): + continue + permitted, free_reason = output.words_for(claim) + if free_reason is not None: + errors.extend(_name_violations(value, claim, index, free_reason)) + continue + if value in permitted: + continue + if not permitted: + errors.append( + f"$.observations[{index}].value is the text {value!r}, and " + f"{claim!r} carries no word at all โ€” the claim is the finding, " + f"and a sentence in a value field is a sentence nobody reviewed" + ) + else: + errors.append( + f"$.observations[{index}].value is {value!r}, which is not one " + f"of the {len(permitted)} words {claim!r} may take" + ) + return errors + + +def check_observation_split(payload: dict[str, Any]) -> list[str]: + """The split `app/schemas.py` defines, enforced by constructing it. + + `Observation` is a measurable fact about the picture and `Interpretation` is + what it might mean, always hedged. The distinction is the product's + credibility โ€” a photograph can support a pattern and only a vet can confirm + a disease โ€” so the models are built here rather than described, and pydantic + raises when the response does not fit them. + """ + errors: list[str] = [] + for index, entry in enumerate(payload.get("observations") or []): + try: + Observation(**entry) + except Exception as exc: # pydantic ValidationError, or a non-dict + errors.append(f"$.observations[{index}] is not an Observation: {exc}") + + for index, entry in enumerate(payload.get("interpretations") or []): + try: + interpretation = Interpretation(**entry) + except Exception as exc: + errors.append( + f"$.interpretations[{index}] is not an Interpretation: {exc}" + ) + continue + if not interpretation.requires_review: + # The one field a model could set to turn a hedge into a verdict. + errors.append( + f"$.interpretations[{index}] sets requires_review to false. An " + f"interpretation the product presents as settled is a diagnosis, " + f"and ยง30 reserves those for a vet" + ) + return errors + + +# --- Step 5: quantities in free text ------------------------------------------ + +#: A number as a model writes one: `412`, `3,200`, `2.63`, `1.84`, `28`. +#: +#: `[\d,]*` swallows a thousands separator so `3,200` is one number rather than +#: two. Every comma is then deleted before parsing, which reads `2,5` as +#: twenty-five rather than as two and a half. That is wrong for a European +#: decimal comma and it is wrong in the safe direction: the mis-read value +#: matches nothing a structured field declared, so the response is refused +#: rather than published. +#: `(? list[str]: + """Every string in the payload that carries prose rather than a token. + + Keys are included as well as values: a caller-supplied schema can leave an + object open, and a model that cannot put a number in a declared field can + try to put one in a key it invents. + """ + if key in _NOT_FREE_TEXT: + return [] + if isinstance(value, str): + return [value] + if isinstance(value, dict): + out: list[str] = [] + for inner, item in value.items(): + out.append(inner) + out.extend(_free_text(item, inner)) + return out + if isinstance(value, list): + return [s for item in value for s in _free_text(item, key)] + return [] + + +def _declared_phrases(capability: Capability | None) -> frozenset[str]: + """Strings this capability's own registry entry authored, which are not prose. + + **The exemption is per string, not per field, and the difference matters.** + `evidence` and `limits` are enums now (ADR 0024), so every string a + conforming response puts in them was written by a person in + `app/capabilities.py` and read in a diff. A digit inside one is a digit a + reviewer approved โ€” ยง18's *"Approximate grade on the 0-4 rubric"*, ยง6.3's + benchmark footnote, ยง26's thirty seconds โ€” and asking a corroboration rule + to account for it would refuse the directive's own wording for nothing, + because a model cannot choose the digits without choosing the sentence. + + Exempting the *fields* instead was written first and was wrong. `reason()` + accepts a caller-supplied schema with no capability and an open `evidence`, + and `check_quantities` is the only thing standing on that path; a field-wide + exemption would have switched it off for every payload that named a + capability, including ones validated against a schema the caller wrote. A + string that is not in this set is prose whatever field it arrived in. + + Empty without a capability, and empty for `poultry_uniformity`, which + declares no vocabulary at all (ADR 0023). + """ + if capability is None: + return frozenset() + acquisition = capability.acquisition + return frozenset(acquisition.evidence_phrases) | frozenset( + acquisition.limit_phrases + ) + + +def _declared_quantities( + payload: dict[str, Any], capability: Capability | None = None +) -> set[tuple[float, str | None, bool]]: + """Every number `check_numeric_bounds` looked at, and what unit it is in. + + Exactly those three places, and deliberately not `confidence`: a confidence + is not a quantity about the animal, and letting `0.82` licence an `82` in + prose would be a hole rather than a convenience. + + **The unit comes from the registry, never from the payload.** + `observations[].unit` is a sixteen-character free-text field the model fills + in, so trusting it would let a response declare its own corroboration: + `{"type": "sample_mean", "value": 9.4, "unit": "percent"}` would licence a + coefficient of variation on a capability that measures kilograms. What the + claim is measured in is `ClaimQuantity.unit`, which a person reviewed in + `app/capabilities.py`. That is the whole reason this function now needs the + capability. + + A unit of `None` means *not known here*: either no capability was supplied, + or the claim declares no quantity at all. It corroborates only a prose digit + that carries no unit either โ€” see `check_quantities`. + + The third slot says whether the field is the capability's **headline** + quantity โ€” `range` and `best_estimate` โ€” or one observation among several. + `_corroborated` needs the distinction, and the reason is there. + """ + output = capability.acquisition.output if capability is not None else None + headline = _REGISTRY_UNIT_FAMILIES.get(output.unit) if output else None + found: set[tuple[float, str | None, bool]] = set() + + def take(value: Any, unit: str | None, is_headline: bool) -> None: + if isinstance(value, (int, float)) and not isinstance(value, bool): + found.add((float(value), unit, is_headline)) + + # `range` and `best_estimate` carry the capability's own quantity by + # definition โ€” there is nothing else they could be reporting โ€” so the + # headline unit is theirs. + for item in payload.get("range") or []: + take(item, headline, True) + take(payload.get("best_estimate"), headline, True) + for entry in payload.get("observations") or []: + if not isinstance(entry, dict): + continue + take(entry.get("value"), _observation_unit(output, entry.get("type")), False) + return found + + +def _observation_unit(output: OutputSpec | None, claim: Any) -> str | None: + """The family this claim's `value` is measured in, per the registry. + + Mirrors `OutputSpec.bounds_for`: the per-claim `quantities` table answers + first, and the headline unit answers for a claim named in `measured_claims` + that the table has not been told about. A claim declaring no quantity gets + `None`, which corroborates nothing carrying a unit โ€” and it also fails + `check_numeric_bounds` on the empty interval, so the response is refused + either way. + """ + if output is None or not isinstance(claim, str): + return None + quantity = output.quantity_for(claim) + if quantity is not None: + return _REGISTRY_UNIT_FAMILIES.get(quantity.unit) + if claim in output.measured_claims: + return _REGISTRY_UNIT_FAMILIES.get(output.unit) + return None + + +def _unit_beside(tokens: list[str], index: int) -> str | None: + """The unit token this digit is written in, or `None` if it carries none. + + **Nearest wins, and a tie goes to the token after the number**, because that + is where a unit binds in English: `"Coefficient of variation across the + weighed birds: 9.4%."` has `birds` one token before and `percent` one token + after, and the number is a percentage. Reading the first unit in the window + instead would have made that sentence a count of birds and corroborated it + against a bird's weight in kilograms, which is the evasion this whole rule + exists to close. + + The reach is `_MEASURE_WINDOW`, the same as the capture veto's, because a + range puts the unit past the far end of itself: *"350-430 kg"* leaves `kg` + two tokens from the 350 and `"Weight from these photos: 412 in kg."* leaves + it two tokens from the 412. + """ + for distance in range(1, _MEASURE_WINDOW + 1): + after = index + distance + if after < len(tokens) and tokens[after] in _UNIT_FAMILIES: + return tokens[after] + before = index - distance + if before >= 0 and tokens[before] in _UNIT_FAMILIES: + return tokens[before] + return None + + +#: Units that are punctuation, or are spelled across two words, rewritten into +#: one token `_PROSE_TOKEN` can see. +#: +#: **A unit nothing tokenises is a unit nothing checks.** `%` is punctuation, so +#: it never became a token and never vetoed anything until it was spelled out โ€” +#: *"Litter moisture from these photos: 28%."* was ยง30's rejected claim riding +#: on a character nothing looked at. A watchdog then found the same shape twice +#: more: `ูช` (U+066A, the Arabic percent sign) is **not** folded to `%` by NFKC, +#: so it renders beside a number exactly like a percent sign and arrived as an +#: unmarked digit; and `cm2` reaches the tokeniser as the bare token `cm`, +#: because `_PROSE_TOKEN` deliberately refuses a digit welded to a label. +#: +#: The second of those was worse than a missing unit โ€” it was a *wrong* one. +#: With `cm` mapped to the area family, *"The laceration measures 14 cm along +#: the flank."* was corroborated by a 14 cmยฒ wound area. So the area spellings +#: are welded to `squarecm` here and `cm` on its own stays a length. +_UNIT_SPELLINGS: tuple[tuple[str, str], ...] = ( + ("square centimetres", " squarecm "), ("square centimeters", " squarecm "), + ("square centimetre", " squarecm "), ("square centimeter", " squarecm "), + ("square cm", " squarecm "), ("sq cm", " squarecm "), + ("cmยฒ", " squarecm "), ("cm2", " squarecm "), + ("per cent", " percent "), + ("%", " percent "), ("ูช", " percent "), ("๏นช", " percent "), + ("โ€ฐ", " permille "), +) + + +def _spell_out_units(folded: str) -> str: + """Rewrite the units above into tokens. + + Order matters and the table is written longest spelling first, so `"square + centimetres"` is consumed before `"square cm"` could take part of it. + """ + for written, spelled in _UNIT_SPELLINGS: + if written in folded: + folded = folded.replace(written, spelled) + return folded + + +def _prose_numbers(text: str) -> list[tuple[float, bool, str | None]]: + """The numbers in this string, each with its capture excuse and its unit. + + Folded through NFKC, the invisible sweep and the confusables table first, so + a full-width `๏ผ”๏ผ‘๏ผ’` and a `4๏ผ‘2` split by a zero-width space both arrive as + 412. + """ + folded = unicodedata.normalize("NFKC", text) + folded = _INVISIBLE.sub("", folded) + folded = folded.translate(_CONFUSABLES).lower() + folded = _spell_out_units(folded) + tokens = _PROSE_TOKEN.findall(folded) + + out: list[tuple[float, bool, str | None]] = [] + for index, token in enumerate(tokens): + if not token[0].isdigit(): + continue + try: + value = float(token.replace(",", "")) + except ValueError: + continue + window = tokens[max(0, index - _CAPTURE_WINDOW):index + _CAPTURE_WINDOW + 1] + # A unit beside the number vetoes the capture excuse. Checked on the + # immediate neighbours only, because that is where a unit binds. + # **Wider than the capture window, and deliberately so.** The veto read + # one token either side while the excuse read two, so moving the unit + # one step out brought the excuse back: "Weight from these photos: 412 + # in kg.", "Age from this photo: 42, in months." and "Frames analysed: + # 3200 in the house." all published while "Across both photos, 412 kg." + # was refused. + # + # Equal widths were the first fix and they were still too narrow. The + # asymmetry is the right way round now: widening the veto refuses more, + # which a caller sees and can fix, and widening the excuse publishes + # more, which nobody sees. + veto = tokens[max(0, index - _MEASURE_WINDOW):index + _MEASURE_WINDOW + 1] + measured = any(t in _MEASURE_UNITS for t in veto) + excused = any(t in _CAPTURE_WORDS for t in window) and not measured + out.append((value, excused, _unit_beside(tokens, index))) + return out + + +def _spelled_quantities(text: str) -> list[str]: + """Quantities written as words, where the wording admits no other reading. + + Only two shapes count, and both are unambiguous: a number word beside a + scale word (`four hundred`, `three thousand`), and a number word followed by + `point` (`two point six three`). A bare number word is left alone, because + `"one view only"` is prose and not arithmetic. + """ + folded = unicodedata.normalize("NFKC", text) + folded = _INVISIBLE.sub("", folded) + folded = folded.translate(_CONFUSABLES).lower() + folded = _spell_out_units(folded) + tokens = _PROSE_TOKEN.findall(folded) + + found: list[str] = [] + for index, token in enumerate(tokens): + if token not in _NUMBER_WORDS: + continue + following = tokens[index + 1:index + 3] + preceding = tokens[max(0, index - 2):index] + if any(t in _NUMBER_SCALE for t in following + preceding): + found.append(f"{token} {' '.join(following[:1])}".strip()) + elif following[:1] == ["point"]: + found.append(" ".join([token] + following)) + elif token in _TENS and following[:1] and following[0] in _UNITS: + # **A compound is where ยง30's exact-X claims actually live.** The + # first version of this caught `four hundred` and `two point six` + # and named `"four twelve kg"` as the survivor โ€” while a watchdog + # published `forty-two months`, `twenty-eight percent`, `sixty + # head` and `three out of five`, which between them are exact age, + # exact litter moisture, herd size, lameness score and footpad + # grade. A tens word followed by a units word is a number and + # cannot be anything else. + found.append(f"{token} {following[0]}") + elif token in _TENS and following[:1] and following[0] in _MEASURE_UNITS: + # `sixty head`, `forty birds`. Tens only: "one bird was inactive" + # and "two frames usable" are prose, and a rule that read every + # number word beside a unit would refuse them. + found.append(f"{token} {following[0]}") + elif token in _UNITS and following[:2] in (["out", "of"], ["in", "ten"]): + # `three out of five`, `eight in ten` โ€” a ratio, which is a + # prevalence with the percent sign taken off. + found.append(f"{token} {' '.join(following[:2])}") + return found + + +def check_mixed_scripts(payload: dict[str, Any]) -> list[str]: + """Refuse a word built from two alphabets, whatever the two are. + + **This retires a game of whack-a-mole that was being lost.** `_CONFUSABLES` + is hand-written and `_fold_latin_lookalikes` asks Unicode for a base letter, + but the second only works on characters Unicode *names* as Latin โ€” so + `Lีฝmpy` (Armenian seh) and `แžumpy` (Cherokee) fell through to the + punctuation sweep, which deletes the character and leaves `lumpy` matching + nothing. Every script with a Latin lookalike is another entry nobody has + made yet, and there are dozens. + + Mixing alphabets inside one word is the thing all of those attacks have in + common, and it is not something honest evidence does. English prose about a + Nigerian farm does not put an Armenian letter in the middle of `lumpy`. So + the shape is refused rather than the characters being enumerated, and a word + written entirely in one script โ€” Cyrillic, Greek, Arabic, whatever a future + localisation needs โ€” is untouched. + + **What it does not do:** a homoglyph attack written *entirely* in another + script still reads as that script and passes. Nothing here can catch that, + and nothing should โ€” a whole word in one alphabet is a word, not an evasion. + """ + errors: list[str] = [] + for text in _free_text(payload): + for word in re.split(r"[^\w]+", unicodedata.normalize("NFKC", text)): + scripts = set() + for char in word: + if not char.isalpha(): + continue + name = unicodedata.name(char, "") + if name: + scripts.add(name.split()[0]) + if len(scripts) > 1: + errors.append( + f"the word {word!r} mixes the {' and '.join(sorted(scripts))} " + f"alphabets. A word spelled in two scripts is one a reader " + f"and a matcher see differently, which is the whole " + f"mechanism of a homoglyph evasion" + ) + return errors + + +def _corroborated( + value: float, + family: str | None, + declared: set[tuple[float, str | None, bool]], + capability: Capability | None, +) -> bool: + """Whether a structured field carries this quantity, in this unit. + + Three states, and the middle one is the one to argue with: + + - **The prose names a unit.** The corroborating field has to be in the same + family. This is the rule, and it is what stops a percentage riding on a + bird's weight in kilograms. + - **The prose names no unit.** Any declared value of the same magnitude + corroborates it, exactly as before this rule existed. + - **No capability was supplied.** Nothing here knows what any field is + measured in, so the check falls back to the magnitude alone, which is what + it has always done. `reason()` permits a caller-supplied schema with no + capability, and on that path `check_vocabulary` and `check_numeric_bounds` + do not run at all. + + ## The unmarked digit is a deliberate decision, and it is the weak one + + **Requiring a unit of every prose digit refuses the directive's own + wording.** ยง7's worked evidence is *"Body condition 2.5 to 3.0"* against + `range: [2.5, 3.0]`, and a body condition score is not written with a unit. + + **Restricting an unmarked digit to `range` and `best_estimate` was written, + measured and reverted.** It closed *"Uniformity 72."* on every carrier โ€” and + it refused ยง10's own worked shape, `["nodules", "flank", "raised", "5 or + more"]` beside `{"type": "nodular_lesions_visible", "value": 5}`, because + `cattle_skin` publishes no `range` and a lesion count is an observation. A + control that refuses the directive's example is not a control. + + There is no third option, and the reason is the one ADR 0022 already + recorded: **nothing about a number distinguishes two numbers.** ยง10's `5` and + an attacker's `72` are both a declared observation's value restated without + a unit; the discriminators tried โ€” one unit family in the vocabulary, the + claim also appearing in `claims` โ€” separate neither. So the honest statement + is that **the unit rule is only as strong as the model's willingness to + write a unit**, and the measurement of what that leaves is in + `check_quantities`' own *"What it cannot do"*. + """ + for magnitude, unit, _ in declared: + if not math.isclose(value, magnitude, rel_tol=1e-9, abs_tol=1e-12): + continue + if capability is None or family is None or unit == family: + return True + return False + + +def check_quantities( + payload: dict[str, Any], capability: Capability | None = None +) -> list[str]: + """A number a farmer reads must be a number the registry bounded. + + **This is the structural answer to the whole "exact X" family**, and it does + not read the sentence around the number. ยง30 rejects an exact weight, an + exact house population, an exact age in months, an exact litter moisture and + an exact per-bird weight; ยง7 rejects a narrowed BCS; ยง23 rejects a + whole-flock weight. Every one of those reaches a farm as *a digit in prose*, + and every one of them was published by an earlier version of this module + because the prose rules were looking for words. + + A number in `evidence` or `limits` is refused unless one of two things is + true: + + - **A structured field carries it, in the unit the prose wrote it in.** + `range`, `best_estimate` and `observations[].value` are the three places + `check_numeric_bounds` examines against `OutputSpec`, so a corroborated + number is a number the capability declared it can measure, on the grid it + declared, inside the bounds it declared. ยง37's own preferred output โ€” + *"Experimental weight estimate 350-430 kg"* โ€” publishes unchanged, because + `range` holds 350 and 430 and the capability's unit is `kg`. + - **A capture word sits within two tokens of it**, which makes it a fact + about the photograph rather than about the animal. + + Rephrasing does not help, which is the property the concept table below does + not have: *"212 ticks across the body"*, *"Total tick burden: 212"*, *"Tick + count 212"* and *"there were 212"* are the same refusal, because the control + reads the 212. + + ## The unit half, and why it was not here from the start + + **Corroborating a number is not corroborating a claim.** For one commit this + asked only whether the prose digit appeared in *some* structured field, and + never whether that field was about the same quantity. A watchdog published + ADR 0023's forbidden flock uniformity โ€” *"Flock uniformity: 72%."* โ€” from + **15 of the 28 capabilities**, riding on a count of animals, of eggs, of + ticks, of wounds, of lesions, of cracks, of birds, and on a drinker-line + index. Worst, `poultry_weight` corroborated **its own forbidden coefficient + of variation**: its three claims are bounded 0.02-10 kg, a broiler CV runs + 8-14, and *"Coefficient of variation across the weighed birds: 9.4%."* + published beside `{"type": "sample_mean", "value": 9.4, "unit": "kg"}` โ€” + a bird's weight in kilograms and a percentage, sharing nothing but their + digits. The watchdog's summary was *"the identifier control is real, the + publication control is theatre"*, and it was right. + + So the prose digit's unit has to match the corroborating field's. + `_UNIT_FAMILIES` maps a prose token to a family and `_REGISTRY_UNIT_FAMILIES` + maps a registry unit to the same families. **Measured, that closes 12 of the + 15 and leaves 3** โ€” all three percent-unit claims, and all three the wall + ADR 0022 already hit: nothing about a number can separate two percentages. + ADR 0023 and this module both said 14 and 2 for a commit; both were counting + through a payload whose `unit` label was too long for the schema, which hid + `poultry_litter` on either side of the sum. + + **The unit comes from the registry, never from the payload.** + `observations[].unit` is sixteen characters of free text the model writes, + so reading it would let a response declare its own corroboration. + + **Without a capability there is no unit half.** `reason()` accepts a + caller-supplied schema and no capability; on that path this falls back to + matching the magnitude alone, as do `check_vocabulary` and + `check_numeric_bounds` by not running at all. + + **What it cannot do**, and none of this is hedging. Every limit below is a + statement about *this function*, not about what the service publishes: since + ADR 0024 the two fields these sentences used to arrive in are enums, so none + of them reaches a farmer through `schema_for`. They still describe what + stands on the caller-supplied-schema path, in `observations[].unit`, and in + any payload assembled by hand. + + - **A capture word two tokens away still excuses any digit, before this + check runs at all.** *"Flock uniformity, from this photo: 72."* and + *"Uniformity across the weighed sample: 72."* were published from **28 of + the 28 capabilities with an otherwise empty payload**, because + `_CAPTURE_WORDS` is an allowlist that only ever permits and `near_capture` + short-circuits corroboration entirely. That was the largest hole in the + whole step; it is still open here and it is now unreachable through a + generated schema, measured at 0 of 28 by + `tests/test_directive_payloads.py`. Widening the veto to the whole string + was measured and reverted: it kills one phrasing, leaves the other, and + refuses honest limits like *"30 second clip, 1 bird in frame"*. + - **A digit with no unit token beside it is corroborated by any field of the + same magnitude.** *"Uniformity 72."* โ€” the watchdog's sentence with the + `%` and the word `flock` deleted โ€” publishes from all 15 carriers again. + This is a decision, not an oversight: requiring a unit everywhere refuses + ยง7's *"Body condition 2.5 to 3.0"*, and restricting an unmarked digit to + `range` and `best_estimate` was written and reverted because it refuses + ยง10's own worked shape. The argument, and the discriminators that were + tried and failed, are in `_corroborated`. `tests/test_claims.py:: + test_an_unmarked_digit_is_the_residual_no_unit_rule_can_reach` holds the + measurement. + - **The unit tables are vocabularies, and a vocabulary loses to a word + nobody listed.** A prose spelling missing from `_UNIT_FAMILIES` makes a + digit unmarked, which is the hole above; a *wrong* family refuses an + honest answer. Both directions have already happened, in one commit: a + watchdog published a percentage as `pct`, `percentage`, `per cent` and + `72ูช` through the first gap, and `cm` mapped to the area family let a 14 cm + laceration corroborate a 14 cmยฒ wound through the second. Neither + direction is free and the table is not finished. + - **It says nothing about a claim carrying no number at all**, which is most + of ยง30's disease list and the whole of `uniformity_band`. `scan_prose` + still has no uniformity concept, so *"Uniformity band: poor."*, *"This + flock is uniform."* and *"Flock uniformity is below the harvest threshold; + delay the harvest."* pass both prose tiers untouched. **What refuses them + now is that there is no string in any capability's `evidence` vocabulary + that says them** (ADR 0024), which is a different kind of control and the + only kind that was ever going to work here: ADR 0023 is right that + teaching the vocabulary tier the word *uniform* would refuse a farmer's + own note, and that argument is unchanged and no longer load-bearing. + - **A number spelled as words is caught only where the wording admits no + other reading**: beside a scale word (*"four hundred and twelve + kilograms"*, *"three thousand two hundred birds"*) or before `point` + (*"two point six three"*, ยง7's forbidden BCS read aloud). A bare number + word is deliberately left alone, so *"the high twenties percent"* and + *"four twelve kg"* still pass. Folding every number word was rejected and + stays rejected: `"one view only"` and `"three of the four teats"` would + then need corroborating, and a control that refuses honest prose is a + capability that never answers. + - **A number the model also declares is corroborated even when the prose + mischaracterises it, as long as the unit agrees.** Declaring + `probable_tick_count_sampled: 212` and writing *"212 ticks across the + body"* passes this check, because the digit is accounted for and both are + ticks. Scope is what the concept table is for, and that half is vocabulary + and beatable. + - **It refuses four figures the directive itself prints**, all of them from + before the unit rule and none repaired by it: + + - ยง18's *"Approximate grade: 2 / 4"*, on the `4`, which is the rubric's + own maximum rather than a reading. Excusing a scale endpoint would + licence an undeclared *"grade 4"*, which is the worst footpad lesion. + - ยง18's *"20 birds sampled"* and ยง23's *"Sample 10-20 birds"*, because + neither capability declares a claim counted in birds. Excusing a count + of birds as capture metadata would licence ยง30's *"3,200 birds in the + house"*, which has the same shape. + - ยง6.3's own uncertainty footnote, *"dense scenes may vary by + approximately 10-20%"*, because `poultry_count` declares no claim in + percent. + - ยง7's UI shows the previous scan's band beside this one โ€” *"Previous: + 3.0-3.5"* โ€” and a previous reading has no field in this response. + + The last three are registry gaps: a sample-size claim, an uncertainty + claim and a previous-reading claim would each publish immediately. This + module does not own the registry, and inventing an excuse here instead + would trade a visible refusal for an invisible hole. + """ + declared = _declared_quantities(payload, capability) + errors: list[str] = [] + seen: set[tuple[float, str | None]] = set() + + authored = _declared_phrases(capability) + + for text in _free_text(payload): + # A string the registry itself authored is not prose. See + # `_declared_phrases`; the exemption is per string so that a payload + # validated against a caller's own open schema is still read. + if text in authored: + continue + for value, near_capture, unit in _prose_numbers(text): + family = _UNIT_FAMILIES.get(unit) if unit else None + if near_capture or (value, family) in seen: + continue + if _corroborated(value, family, declared, capability): + continue + seen.add((value, family)) + if family is None: + errors.append( + f"free text states the quantity {value:g}, and no " + f"structured field carries it. ยง30 and ยง37 reject an exact " + f"figure a farm reads but nothing bounded โ€” put it in " + f"range, best_estimate or an observation, where the " + f"registry's own limits apply to it" + ) + else: + errors.append( + f"free text states the quantity {value:g} in {unit}, and " + f"no structured field carries that quantity in that unit. A " + f"digit corroborated by a field measuring something else is " + f"not corroborated โ€” put it in range, best_estimate or an " + f"observation on a claim the registry declares in {unit}" + ) + for phrase in _spelled_quantities(text): + errors.append( + f"free text states the quantity {phrase!r} in words. A spelled " + f"number cannot be checked against what the capability declared " + f"it can measure, so it is refused rather than corroborated โ€” " + f"write the figure in digits and put it in a structured field" + ) + return errors + + +# --- Step 6: the prose backstop ----------------------------------------------- + +#: Characters that carry no glyph and split a word for a substring matcher while +#: leaving it whole for a reader. Zero-width space, ZWNJ, ZWJ, word joiner, the +#: bidi controls, the byte-order mark, and the soft hyphen โ€” which is `Cf` in +#: name but is not caught by the category sweep on every Python build. +_INVISIBLE = re.compile(r"[ยญโ€‹-โ€โ€ช-โ€ฎโ -โค๏ปฟ]") + +#: Letters from other scripts that a reader cannot distinguish from Latin ones. +#: Hand-written rather than pulled from a confusables package, because this list +#: has to be reviewable in a diff and the full Unicode table is not. +_CONFUSABLES = str.maketrans({ + # Cyrillic + "ะฐ": "a", "ะฒ": "b", "ะต": "e", "ะบ": "k", "ะผ": "m", "ะฝ": "h", "ะพ": "o", + "ั€": "p", "ั": "c", "ั‚": "t", "ัƒ": "y", "ั…": "x", "ั–": "i", "ั˜": "j", + "ั•": "s", "ิ": "d", "าป": "h", "ิ›": "q", "ิ": "w", "ั‘": "e", "ั—": "i", + "ะ": "a", "ะ’": "b", "ะ•": "e", "ะš": "k", "ะœ": "m", "ะ": "h", "ะž": "o", + "ะ ": "p", "ะก": "c", "ะข": "t", "ะฃ": "y", "ะฅ": "x", "ะ†": "i", "ะˆ": "j", + "ะ…": "s", "ิ€": "d", "ิš": "q", "ิœ": "w", + # Greek + "ฮฑ": "a", "ฮฒ": "b", "ฮต": "e", "ฮน": "i", "ฮบ": "k", "ฮผ": "u", "ฮฝ": "v", + "ฮฟ": "o", "ฯ": "p", "ฯ„": "t", "ฯ…": "u", "ฯ‡": "x", "ฮณ": "y", "ฯƒ": "o", + "ฮ‘": "a", "ฮ’": "b", "ฮ•": "e", "ฮ–": "z", "ฮ—": "h", "ฮ™": "i", "ฮš": "k", + "ฮœ": "m", "ฮ": "n", "ฮŸ": "o", "ฮก": "p", "ฮค": "t", "ฮฅ": "y", "ฮง": "x", + # Latin lookalikes that NFKC leaves alone + "วƒ": "!", "ว€": "l", "ษก": "g", "ษ‘": "a", "ษฉ": "i", "แด": "o", "แด„": "c", +}) + +#: Digit-for-letter substitutions. Folded because `c0nfirmed` reads as +#: `confirmed` to a farmer and as a different string to a matcher. +#: +#: **Applied only to tokens that already contain a letter.** Folding `40` into +#: `ao` would destroy the one thing a temperature or a weight claim is made of, +#: and a bare numeral is never an obfuscated word. That distinction is what let +#: `"Fever, 40 celsius"` through an earlier version of this table. +_LEET = str.maketrans({"0": "o", "1": "i", "3": "e", "4": "a", "5": "s", "7": "t"}) + +_HAS_LETTER = re.compile(r"[a-z]") +_NON_ALNUM = re.compile(r"[^a-z0-9]+") + +#: Stands for "any bare numeral" inside a concept's synonym group. Spelled as a +#: character no normalised token can contain, so it cannot collide with a word. +ANY_NUMBER = "#" + + +#: Pulls the base letter out of a Unicode name like `LATIN SMALL LETTER DOTLESS +#: I` or `LATIN LETTER SMALL CAPITAL L`. +#: The modifier can sit either side of `LETTER` โ€” `LATIN LETTER SMALL CAPITAL L` +#: but `LATIN SMALL LETTER DOTLESS I` โ€” so the base letter is found as the first +#: single-letter word after `LETTER`, wherever the qualifiers landed. +_LATIN_NAME = re.compile(r"LETTER (?:[A-Z]+ )*?([A-Z])(?:\s|$)") + + +def _fold_latin_lookalikes(text: str) -> str: + """Map a Latin-script letter nobody hand-listed onto its base letter. + + **The hand-written table is not the control it looked like.** A homoglyph in + `_CONFUSABLES` is replaced; one that is not in it falls through to the + punctuation sweep, which *deletes* it โ€” so `Lumpy skฤฑn disease confฤฑrmed` + (U+0131 dotless i) became `lumpy skn disease confrmed` and matched nothing, + neither as tokens nor squashed. Deleting a letter defeats the squashed + matcher the table exists to feed, so a missing entry was worse than a + missing entry looks. + + Unicode names the base letter of every Latin variant, so ask it rather than + listing them. `สŸ`, `ฤฑ`, `ษข`, `แด…` and the rest of the small-capital and + phonetic blocks all fold here without anybody enumerating them. The + hand-written table stays for Cyrillic and Greek, whose names name a + different letter. + """ + if text.isascii(): + return text + out = [] + for char in text: + if char.isascii() or not char.isalpha(): + out.append(char) + continue + name = unicodedata.name(char, "") + match = _LATIN_NAME.search(name) if name.startswith("LATIN ") else None + out.append(match.group(1) if match else char) + return "".join(out) + + +def normalise(text: str) -> str: + """Fold a string down to the thing a reader would see. + + NFKC first, so full-width and ligature forms collapse; then the invisibles, + which are the cheapest evasion there is; then the confusables table; then + case, diacritics and digits; then every run of punctuation and whitespace + becomes one space. A doubled space, a zero-width joiner, a Cyrillic `ั` and + a hyphen all disappear here rather than being handled downstream. + """ + folded = unicodedata.normalize("NFKC", text) + folded = _INVISIBLE.sub("", folded) + folded = folded.translate(_CONFUSABLES) + folded = _fold_latin_lookalikes(folded) + folded = folded.lower() + # Decompose, drop the combining marks, recompose: `cลnfirmed` โ†’ `confirmed`. + folded = "".join( + c for c in unicodedata.normalize("NFD", folded) + if unicodedata.category(c) != "Mn" + ) + tokens = _NON_ALNUM.sub(" ", folded).split() + tokens = _join_letter_runs(tokens) + return " ".join( + token.translate(_LEET) if _HAS_LETTER.search(token) else token + for token in tokens + ) + + +def _join_letter_runs(tokens: list[str]) -> list[str]: + """Weld a run of single letters back into one token. + + **`_SQUASH_FLOOR` leaves every short acronym open, and a full stop is the + whole attack.** `F.M.D.` normalises to the tokens `f m d`; the squashed + matcher would find `fmd` in the joined text, but `fmd` is three characters + and the floor refuses needles shorter than five โ€” for the good reason that + ignoring word boundaries makes a short needle match inside innocent words. + So `F.M.D.`, `L.S.D.`, `P.P.R.` and `O.R.F.` all published while `FMD` + was refused. + + Welding the run restores the token, so the exact matcher handles it and the + floor keeps doing its job. It costs nothing legitimate: consecutive + single-letter words are punctuation-stripped initials, not prose. It also + catches the opposite trick, `l u m p y s k i n`, without relying on the + squashed pass. + """ + out: list[str] = [] + run: list[str] = [] + for token in tokens + [""]: + if len(token) == 1 and token.isalpha(): + run.append(token) + continue + if len(run) > 1: + out.append("".join(run)) + else: + out.extend(run) + run = [] + if token: + out.append(token) + return out + + +@dataclass(frozen=True) +class ForbiddenConcept: + """A claim the directive forbids, described as tokens rather than a sentence. + + `all_of` holds one synonym group per idea that has to be present. A response + trips the concept when it uses at least one phrase from every group inside + `within` tokens of each other โ€” so *"lumpy skin disease confirmed"*, + *"confirmed lumpy skin disease"* and *"lumpy skin disease: confirmed on the + flank"* are all the same finding, which is exactly what a substring matcher + could not say. + """ + + key: str + reason: str + all_of: tuple[tuple[str, ...], ...] + #: How close the groups must be, **in characters of the whitespace-stripped + #: text**. Characters rather than words, because an evasion that writes + #: `l u m p y s k i n` turns three words into eleven and any window counted + #: in words inflates with it. Roughly six characters to the word. + within: int = 48 + #: Which group carries the assertion, and is therefore the one a negation + #: can cancel. The first group by default; `disease_confirmed` puts its + #: certainty words second, so it says so. + asserting_group: int = 0 + #: Whether a qualifier anywhere in the sentence cancels the assertion, + #: rather than only one immediately before it. + #: + #: **This is a polarity switch, and it is the difference between a rule that + #: catches a phrasing and a rule that catches a subject.** `disease_confirmed` + #: leaves it false: it hunts an assertion, so a hedge three words earlier + #: must not excuse a `confirmed` later in the same sentence. `disease_named` + #: sets it true: it refuses the disease name itself, so anything that + #: genuinely qualifies the name has to be able to rescue it, wherever in the + #: sentence the model put it. + #: + #: The two are not redundant. Under the sentence rule *"Lumpy skin disease + #: confirmed, though a vet should review"* would be excused by the word + #: `review`; under the adjacent rule it is refused, and it stays refused. + qualified_by_sentence: bool = False + + +#: Words that turn an observation into a verdict. Kept as one group because +#: every concept below pairs certainty with a subject. +_CERTAIN = ( + "confirmed", "confirms", "confirm", "confirming", "confirmation", + "definitive", "definitively", "definite", "certain", "certainly", + "conclusive", "conclusively", "proven", "proves", "unmistakable", + "diagnosis", "diagnosed", "diagnostic", "positive for", "tested positive", + "is suffering from", "clearly has", "undoubtedly", "without doubt", + "beyond doubt", "verified", "no doubt", +) + +#: Diseases the directive names, plus the ones a reasoner reaches for on cattle +#: and poultry photographs. ยง10 is explicit that the legal weight of these +#: changes the wording rather than the feasibility. +#: +#: **A name here is refused unqualified, not only when it is confirmed.** The +#: earlier reading paired this list with `_CERTAIN`, so *"Lumpy skin disease +#: confirmed"* was caught and *"This is coccidiosis"*, *"This is FMD"*, *"The +#: bird has bumblefoot"* and *"This flock has Newcastle"* were all published โ€” +#: a diagnosis is a diagnosis whether or not the model also says `confirmed`. +#: `disease_named` fixes the polarity, and the fix changes what a gap in this +#: list costs: a disease nobody listed now fails to be *caught*, where before a +#: phrasing nobody listed failed to be caught. Names are a smaller and much more +#: stable space than phrasings, which is the whole reason the flip is worth +#: having. +_DISEASE = ( + "lumpy skin disease", "lumpy skin", "lsd", + "foot and mouth", "foot mouth", "fmd", "foot and mouth disease", + "mastitis", "brucellosis", "anthrax", "trypanosomiasis", "nagana", + "blackleg", "rinderpest", "cbpp", "contagious bovine pleuropneumonia", + "newcastle disease", "newcastle", "avian influenza", "bird flu", + "coccidiosis", "coccidiosis", "gumboro", "infectious bursal disease", + "marek", "fowl pox", "bumblefoot", "pododermatitis", "footrot", + "foot rot", "laminitis", "digital dermatitis", "ringworm", + "dermatophilosis", "mange", + # Added with the polarity flip. Each is a name a hosted model reaches for + # on a Nigerian cattle or poultry photograph, and none of them needed the + # word `confirmed` to reach a farm. + "infectious bronchitis", "infectious laryngotracheitis", "salmonellosis", + "pullorum", "fowl typhoid", "fowl cholera", "colibacillosis", + "mycoplasma", "chronic respiratory disease", "aspergillosis", + "necrotic enteritis", "enteritis", "gastroenteritis", "ascites", + "egg drop syndrome", "avian encephalomyelitis", "tuberculosis", + "johnes disease", "paratuberculosis", "leptospirosis", "bluetongue", + "peste des petits ruminants", "ppr", "east coast fever", "theileriosis", + "babesiosis", "heartwater", "cowdriosis", "besnoitiosis", "orf", + "screwworm", "myiasis", "helminthiasis", "parasitic infection", + "bacterial infection", "viral infection", "worm burden", + "parasite burden", "coccidial infection", "white line disease", + "sole ulcer", "interdigital dermatitis", "scaly leg mite", + # **Added after a watchdog published eleven synonyms first try**, which is + # the direct refutation of "names are a smaller and much more stable space + # than phrasings". Veterinary English carries a lay name, an agent name, a + # regional name and an eponym for most of these, and a farm reads them all + # the same way. The flip still helps โ€” a synonym fails to be caught rather + # than a phrasing being published โ€” but the list is not small and is not + # finished, and that has to be said out loud rather than assumed away. + "hoof and mouth", "hoof and mouth disease", "aphthous fever", + "ranikhet", "avian paramyxovirus", "eimeria", "cocci", + "foul in the foot", "foul of the foot", "founder", "hairy heel warts", + "heel warts", "neethling", "neethling disease", "knemidocoptes", + "scaly leg", "liver fluke", "fasciola", "ostertagia", "haemonchus", + "strongyle", "strongyles", "staph", "staphylococcus", "streptococcus", + "e coli", "salmonella", "pasteurella", "clostridial", "clostridium", + "black quarter", "quarter evil", "wooden tongue", "lumpy jaw", + "actinomycosis", "actinobacillosis", "milk fever", "grass tetany", + "three day sickness", "ephemeral fever", "sweating sickness", + "red water", "redwater", "gall sickness", "heartwater disease", + "new castle", "gumboro disease", "coryza", "infectious coryza", + "sour crop", "impacted crop", "vent gleet", "omphalitis", + # **Round two, after fifty names went in and five more landed anyway.** An + # Afrikaans name, a lesion name standing in for the disease, a genus, a + # virus family and a textbook periphrasis. The list is not finished and + # will not be; that is the standing cost of the vocabulary tier. + "knopvelsiekte", "toe necrosis", "nematodirus", "reovirus", + "viral arthritis", "vesicular disease", "doyles form", "doyle form", + "beaudette", "avian rhinotracheitis", "swollen head syndrome", + "gapeworm", "syngamus", "histomoniasis", "blackhead", "leucosis", + "lymphoid leucosis", "cachexia", "ketosis", "acidosis", "bloat", + "photosensitisation", "photosensitization", "besnoitia", +) + +#: Words that cancel an assertion rather than making one. +#: +#: **These exist because the backstop refused correct hedging.** ยง10's own +#: preferred output is *"Veterinary review recommended"*, and a response saying +#: *"a vet must decide whether this is lumpy skin disease, and no photograph can +#: confirm it"* is the product working exactly as the directive asks. A +#: proximity matcher cannot tell that from an assertion, because both put a +#: disease name near the word `confirm`. A refused legitimate answer is a +#: capability that never answers, so a certainty word immediately preceded by +#: one of these is read as negated. +#: +#: Only *preceding* text suppresses, and only within `_NEGATION_LOOKBEHIND` +#: characters. "Confirmed, though a vet should review" is still a confirmation; +#: "cannot confirm" is not. +_HEDGES = ( + "not", "cannot", "can not", "never", "unable to", "no photograph", + "no image", "nothing", "must decide", "whether", "rule out", "suspected", + "possible", "possibly", "may be", "might be", "could be", "pending", + "awaiting", "requires", "no way to", "does not", "do not", "cannot be", + # `neither` and `nor` earn their place on a real false positive: + # `"Neither is confirmable here"` tripped `confirm`, which the squashed + # matcher finds inside `confirmable`. Matching across word boundaries is + # what defeats an obfuscated `conโ€‹firmed`, and the price is that a longer + # innocent word containing a concept word also matches. Negation is the + # right place to pay it. + "neither", "nor", "none of", "unlikely", "inconclusive", "uncertain", +) + +#: What counts as qualifying a disease name, for `qualified_by_sentence`. +#: +#: Wider than `_HEDGES` on purpose, and used only by the concepts that refuse a +#: *subject* rather than an assertion. Those need every honest way of naming a +#: possibility to rescue the sentence, because the cost of getting it wrong is a +#: refused legitimate answer โ€” and ยง13 asks in so many words for the qualified +#: form, *"coccidiosis-associated visual pattern only when appropriate and +#: clearly qualified"*, which is where `associated` and `pattern` come from. +#: +#: **`vet`, `veterinary` and `review` are deliberately absent.** They are the +#: obvious additions and they would gut the rule: ยง10 wants *"Veterinary review +#: recommended"* on every skin result, so a model that always appends it would +#: always be excused, and `disease_named` would read as a control while refusing +#: nothing. What is here is epistemic โ€” possibility, resemblance, association, +#: alternatives โ€” and a sentence carrying none of it is asserting. +#: Qualifiers whose reach really is the whole clause. +#: +#: Short, and every entry earns it. A word here excuses a name anywhere in its +#: sentence, so it has to be one that cannot be true of one clause while the +#: next asserts the opposite. `not`, `cannot`, `whether`, `rule out`, +#: `inconclusive`, `differential` are statements about the *whole finding*. +#: **Written out rather than inherited from `_HEDGES`, because the two lists +#: overlap.** `_HEDGES` carries `may be`, `might be`, `could be`, `suspected`, +#: `possible` and `nothing` for the adjacent rule, and `_QUALIFIERS = _HEDGES + +#: (...)` put every one of them back at sentence scope โ€” so *"It may be +#: notifiable, but this is foot and mouth"* and *"It may look like nothing much +#: in this light, but it is lumpy skin disease"* went on publishing after the +#: split that was supposed to stop them. A word named weak is weak wherever else +#: it appears, and the only way to be sure of that is to not build this list out +#: of the other one. +_QUALIFIERS = ( + "not", "cannot", "can not", "never", "unable to", "no photograph", + "no image", "must decide", "whether", "rule out", "pending", "awaiting", + "requires", "no way to", "does not", "do not", "cannot be", + "neither", "nor", "none of", "unlikely", "inconclusive", "uncertain", + "differential", "differentials", "either", "versus", "query", + "cannot exclude", "not excluded", "unknown", "undetermined", + "not determinable", "none", +) + +#: Qualifiers that only count **beside** the thing they qualify. +#: +#: **This split is the most important correction in the file, and two rounds of +#: watchdog are why.** Everything here was once sentence-scoped, and every one +#: became a licence: +#: +#: "The nodular pattern here is lumpy skin disease." on `pattern` +#: "No test needed: this is lumpy skin disease." on `test` +#: "I have screened this animal and it has LSD." on `screen` +#: "Watch your step: the bird has bumblefoot." on `watch` +#: "It may look like nothing much, but it is LSD." on `may` +#: "Whatever it appears to be, this is LSD." on `appears` +#: "This flock is clear of everything except coccidiosis" on `clear of` +#: +#: **The first split kept `may`, `might`, `could` and `appears` at sentence +#: scope on the argument that they are genuine epistemic hedges. They are โ€” and +#: it made no difference.** A hedge is only honest about the clause it governs, +#: and `"It may look like nothing much in this light, but it is lumpy skin +#: disease"` hedges the appearance in order to assert the diagnosis. Scope, not +#: sincerity, is what distinguishes them, so the test is proximity for all of +#: them. +#: +#: `pattern` and `test` are absent entirely, weak or strong: ยง13's wording is +#: rescued by `associated` and *"a pattern consistent with X"* by `consistent +#: with`, so neither word has to appear anywhere. +#: +#: **A bare `no` and `not` belong here rather than in the strong list**, and +#: that is what finally fixes a family that has been mishandled twice. At +#: sentence scope `no` published *"There is no question, the bird has +#: bumblefoot"*. Hand-listing `no dead`, `no abscess`, `no lesion` and eleven +#: more fixed those sentences and left ~140 disease names with no way to say +#: "no mastitis" โ€” and, worse, let *"Sole abscess here, and no laminitis +#: anywhere near it"* publish an abscess on the strength of denying something +#: else. Adjacency says what the hand-list was groping for: a denial cancels the +#: thing it is *next to*, and nothing else. +_WEAK_QUALIFIERS = ( + # Bare `no` reaches four characters, which is not far enough to cross "sign + # of" โ€” so the multi-word denials are listed as the single units they are + # rather than by widening the reach, which would also let "no laminitis" + # cancel an abscess seven characters away. + "no", "not", "no sign of", "no signs of", "no evidence of", + "free of", "clear of", "without", + "possible", "possibly", "may", "may be", "might", "might be", "could", + "could be", "likely", "probable", "probably", "apparent", "appears", + "appear", "seems", "seem", "looks like", "look like", "resembles", + "resembling", "similar to", "consistent with", "suggestive", "suggests", + "suggest", "associated", "screen", "screening", "screened", "candidate", + "candidates", "watch", "risk", "laboratory", "lab result", "sample for", + "further testing", "if confirmed", "would need", "suspected", "nothing", +) + +_WEAK_QUALIFIER_REACH = 4 + +#: How far back a hedge reaches, in squashed characters. Deliberately short: +#: roughly three words, so it catches the clause the certainty word sits in. +#: +#: **A hedge never reaches across a sentence boundary**, however close it is. +#: Without that rule, *"Lumpy skin disease. Cannot be ruled out. Confirmed."* +#: and *"Rule out other causes. Lumpy skin disease confirmed."* both went +#: unnoticed, because a hedge fifteen characters back happened to belong to a +#: different sentence. Sentence membership is tracked from the raw text, before +#: normalisation flattens the punctuation that marks it. +_NEGATION_LOOKBEHIND = 20 + +#: What ends a sentence, for the purpose above. Semicolons and newlines count: +#: they separate independent clauses, which is what the rule cares about. +_SENTENCE_BREAK = re.compile(r"[.!?;\n\r]+") + +#: A full stop between two single letters, which is an acronym's punctuation and +#: not the end of a sentence. +#: +#: **Stripped before the sentence split, or `F.M.D.` is three sentences.** +#: `_join_letter_runs` welds `f m d` back into one token so `_SQUASH_FLOOR` stops +#: mattering for short acronyms โ€” but it only ever saw one letter at a time, +#: because the split ran first and put each in its own sentence. `F.M.D.`, +#: `L.S.D.`, `P.P.R.` and `O.R.F.` all published while `FMD` was refused. +_ACRONYM_DOT = re.compile(r"(?<=\b[A-Za-z])[.ยท](?=[A-Za-z]\b)") + +FORBIDDEN_CONCEPTS: tuple[ForbiddenConcept, ...] = ( + ForbiddenConcept( + key="disease_confirmed", + reason=( + "ยง10 forbids 'Lumpy skin disease confirmed' and ยง30 reserves a " + "diagnosis for a vet. A photograph can support a pattern" + ), + all_of=(_DISEASE, _CERTAIN), + asserting_group=1, + ), + ForbiddenConcept( + key="disease_named", + reason=( + "ยง30 rejects a definitive FMD or LSD diagnosis and ยง10 reserves one " + "for a vet. A name stated without a qualifier is that diagnosis, " + "whether or not the word 'confirmed' appears; ยง13 permits the " + "qualified form and only the qualified form" + ), + # One group, and no certainty word required. That is the whole point: + # "This is coccidiosis" and "The bird has bumblefoot" carry no word the + # paired rule above is looking for, and both were published. + all_of=(_DISEASE,), + qualified_by_sentence=True, + ), + ForbiddenConcept( + key="pregnancy_asserted", + reason=( + "Pregnancy is not visible in a photograph, and ยง31 rejects the " + "claim outright rather than hedging it" + ), + all_of=( + ("pregnant", "pregnancy", "in calf", "in kid", "gravid", "gestating"), + _CERTAIN + ("is", "definitely", "clearly"), + ), + within=24, + ), + ForbiddenConcept( + key="body_temperature_stated", + reason=( + "RGB photography does not measure temperature, and the system rules " + "say never to state one. ยง12 and ยง17 both reject a fever claim from " + "a visible-light image" + ), + # A subject and a quantity. A bare mention of fever is deliberately not + # caught here: `cattle_skin` has no allowed claim for it, so prose that + # merely names it cannot become a result, and a rule that fired on the + # word alone would refuse "no visible signs of fever". + all_of=( + ("temperature", "fever", "febrile", "pyrexia"), + (ANY_NUMBER, "celsius", "centigrade", "fahrenheit", "degrees", "deg"), + ), + within=24, + ), + ForbiddenConcept( + key="hoof_disease_named", + reason=( + "ยง11 allows a visible crack, lesion or swelling and forbids naming " + "the condition that caused it" + ), + all_of=( + ("hoof", "claw", "sole", "footpad", "foot pad"), + ("footrot", "foot rot", "laminitis", "digital dermatitis", + "white line disease", "abscess"), + ), + within=36, + asserting_group=1, + # **ยง11 asks for the negative finding and this refused it.** "No abscess + # seen at the sole" tripped `sole` + `abscess` with no way to say + # otherwise, because the concept took no qualifier at all. + # + # Sentence scope was the first fix and it was too generous: "Sole + # abscess here, and no laminitis anywhere near it" then published an + # abscess on the strength of denying something else. `no` is a weak + # qualifier, so it cancels the name it stands beside and no other. + qualified_by_sentence=True, + ), + ForbiddenConcept( + key="whole_population_stated", + reason=( + "ยง6.3 keeps visible count, unique birds observed and reconciled " + "population apart. A whole-house number from one frame is the " + "claim that collapses them" + ), + all_of=( + ("total", "whole house", "entire house", "entire flock", + "whole flock", "all birds", "house population", "flock size", + "herd size", + # Scope phrases a watchdog used to move the same claim outside the + # list above: "Whole-animal tick burden", "a third of the flock", + # "212 ticks across the whole animal". + "of the flock", "of the herd", "of the house", "of the birds", + "flock prevalence", "flock weight", "flock level", "whole animal", + "entire animal", "across the body", "whole body", "entire body", + "across the flock", "across the herd", "throughout the house", + "throughout the flock", "every bird", "each bird", + "every animal", "each animal", "all animals", "all the birds"), + # Fractions carry a prevalence with no digit in it, so the quantity + # gate never sees them: *"A quarter of the flock affected"* is + # `flock_prevalence_from_one_bird` written without arithmetic. + (ANY_NUMBER, "exactly", "precisely", "quarter", "third", "half", + "two thirds", "three quarters", "most", "majority", "nearly all", + "almost all", "every one", "each one"), + ), + within=24, + ), + + # ---- the six ยง30 claims that carry no number and name no disease -------- + # + # **Everything below is vocabulary, and vocabulary is beatable.** None of + # these subjects can be enumerated the way a disease name can, so none of + # them gets the polarity flip that makes `disease_named` cheap to maintain. + # They are here because a bare claim is worse than a weak control, and + # `tests/test_claims.py::TestEveryForbiddenClaimIsGraded` records each one as + # `vocabulary` rather than `structural` so nobody reads this table as + # stronger than it is. + ForbiddenConcept( + key="wound_etiology_named", + reason=( + "ยง9 asks for a description of what is visible โ€” size, swelling, " + "discharge, change since the last scan โ€” and a photograph cannot " + "say what made the wound" + ), + all_of=( + ("wound", "injury", "lesion", "laceration", "cut", "abrasion", + "gash", "wounds", "injuries"), + ("caused by", "cause was", "cause is", "cause of", "due to", + "result of", + "resulted from", "inflicted by", "etiology", "aetiology", + "bitten by", "struck by", "made by", "from a bite", + "from a machete", "from barbed wire"), + ), + within=36, + asserting_group=1, + qualified_by_sentence=True, + ), + ForbiddenConcept( + key="mortality_asserted", + reason=( + "ยง30 rejects dead-versus-sleeping certainty from one poultry image, " + "and ยง20's corrected form is an inactive candidate a person reviews" + ), + all_of=( + ("dead", "died", "dies", "deceased", "mortality", "carcass", + "carcase", "fatality", "fatalities", "expired", "dead bird", + "dead birds", "found dead"), + ), + qualified_by_sentence=True, + ), + ForbiddenConcept( + key="mortality_versus_sleeping", + reason=( + "ยง30 names dead-versus-sleeping certainty as the rejected claim, " + "and ruling sleep out is how a model states it" + ), + # **`mortality_asserted` cannot catch this one and this is why.** *"This + # bird is dead, not sleeping."* carries the word `not`, which the + # sentence-scoped qualifier reads as a denial โ€” and it is a denial, of + # sleep rather than of death. So the assertion is matched directly: + # ruling out the alternative is the claim, not a hedge against it. + all_of=( + ("dead", "died", "deceased", "carcass", "carcase", "mortality"), + ("not sleeping", "not asleep", "rather than sleeping", + "versus sleeping", "instead of sleeping", "definitely dead", + "certainly dead", "confirmed dead", "not resting", + "rather than resting", "not just sleeping"), + ), + within=36, + asserting_group=1, + ), + ForbiddenConcept( + key="hairline_crack_asserted", + reason=( + "ยง30 rejects subtle hairline crack detection from arbitrary ambient " + "egg photos. ยง26's corrected form is obvious visible damage, and a " + "crack too fine to be obvious needs candling" + ), + all_of=( + ("hairline", "hair line", "microcrack", "micro crack", + "fine crack", "hairline crack", "hairline fracture", + "star crack", "invisible crack"), + ), + qualified_by_sentence=True, + ), + ForbiddenConcept( + key="breed_forced", + reason=( + "ยง6.5 makes 'not forcing a breed' a required output. A crossbred " + "animal must be allowed to stay crossbred, so purity is the one " + "thing a phenotype may not assert" + ), + all_of=( + ("purebred", "pure bred", "pure blood", "full blood", "fullblood", + "thoroughbred", "pedigree", "unmixed", "not crossbred", + "no crossbreeding"), + ), + qualified_by_sentence=True, + ), + ForbiddenConcept( + key="breed_asserted", + reason=( + "ยง6.5 wants 'Likely White Fulani' or a White-Fulani-like phenotype. " + "A breed stated with certainty is the forced call it forbids" + ), + all_of=(("breed", "phenotype", "breeds"), _CERTAIN), + within=24, + asserting_group=1, + ), + ForbiddenConcept( + key="sex_asserted", + reason=( + "ยง6.6 makes sex a suggestion the user confirms. `likely_sex` and " + "`not_determinable_from_view` are the whole vocabulary" + ), + # Phrases rather than the bare nouns. `bull`, `cow`, `male` and `female` + # are ordinary words in livestock prose โ€” "the bull is standing square" + # is a capture note, not a sex claim โ€” so matching them alone would + # refuse most of what a reasoner writes about cattle. + all_of=( + ("sex is", "the sex", "sexed as", "sex determined", "is a bull", + "is a heifer", "is a steer", "is a cockerel", "is a cock", + "is male", "is female", "is a male", "is a female", + "definitely male", "definitely female", "clearly male", + "clearly female"), + ), + qualified_by_sentence=True, + ), + ForbiddenConcept( + key="sex_stated_with_certainty", + reason=( + "ยง6.6 makes sex a suggestion the user confirms, so certainty about " + "it is the failure rather than the phrasing" + ), + # The phrase list above cannot hold every way of writing a sex, and + # `"Sex: male, definitively."` proved it โ€” normalisation drops the colon + # and none of those phrases match. Pairing the subject word with a + # certainty word catches the shape instead of the wording. + all_of=(("sex", "sexed", "gender"), _CERTAIN), + within=24, + asserting_group=1, + ), + ForbiddenConcept( + key="identity_asserted", + reason=( + "ยง6.4 shows 'This looks like Kofi' and asks. An unconfirmed match " + "is a candidate, never a record" + ), + # **The weakest entry in this table, and the one to distrust.** An + # animal's name is not enumerable, so `"This is Kofi."` โ€” the exact + # sentence ยง6.4 is written against โ€” is not caught and cannot be. What + # is caught is a model that says out loud what it is doing. + all_of=( + ("identified as", "identification is", "identity is", "identity:", + "matched to", "recognised as", "recognized as", "registered as", + "the animal is named", "positively identified", "same animal as"), + ), + qualified_by_sentence=True, + ), + ForbiddenConcept( + key="treatment_directed", + reason=( + "Naming a drug is naming the disease it treats. ยง10 lists what a " + "skin result may propose โ€” create a case, isolate where " + "appropriate, request vet review, add a laboratory result โ€” and a " + "prescription is none of them, and is a vet's under ยง30" + ), + # **The one part of the implication problem that is enumerable.** + # "Treat with amprolium today" is a coccidiosis diagnosis with the + # diagnosis deleted, and no prose rule can see it โ€” the sentence names + # no disease, asserts nothing and carries no number. Drug names can be + # listed; the general case cannot, and `check_quantities`' docstring + # says so. + # + # ยง10's permitted actions are deliberately absent: `isolate`, `create a + # case`, `request vet review` and `add a laboratory result` are what the + # directive asks the product to offer. + all_of=( + ("treat", "treated", "treatment", "medicate", "medication", + "dose", "dosage", "prescribe", "prescription", "administer", + "amprolium", "oxytetracycline", "tetracycline", "ivermectin", + "albendazole", "levamisole", "tylosin", "enrofloxacin", + "penicillin", "sulphonamide", "sulfonamide", "antibiotic", + "antibiotics", "anthelmintic", "dewormer", "deworm", "acaricide", + "vaccinate", "vaccination", "withdrawal period", "cull", + "slaughter", "destroy the animal", "put down"), + ), + qualified_by_sentence=True, + ), + ForbiddenConcept( + key="continuous_surveillance_asserted", + reason=( + "ยง30 rejects 24/7 respiratory surveillance from one 30-second " + "recording, and ยง27 makes continuous monitoring a fixed-hardware " + "feature. A spot recording is a spot screen" + ), + all_of=( + ("continuous", "continuously", "24 7", "round the clock", + "around the clock", "constant", "constantly", "permanent", + "permanently", "ongoing", "always on", "nonstop", "non stop", + "24 hours a day", "day and night", "all day"), + # **`observation`, `watch` and `detection` were here and had to + # go.** `_strings` feeds the scan the payload's *key names* as well + # as its values, so the literal key `observations` sat next to the + # word `permanent` in `permanent_incisor_count` and refused a + # legitimate count of four teeth. `gi_health_watch` is an allowed + # claim for the same reason. A concept word that is also a field + # name or a claim name is a false positive waiting for a payload + # shaped the right way. + ("monitoring", "monitor", "monitored", "surveillance", "screening", + "coverage", "tracking", "round the clock"), + ), + within=36, + ), +) + + +#: Shortest squashed phrase that may be matched across token boundaries. `lsd` +#: is three characters and would fire inside the middle of unrelated words once +#: spacing is ignored; `confirmed` at nine will not. +_SQUASH_FLOOR = 5 + + +def _positions(tokens: list[str], phrases: tuple[str, ...]) -> list[int]: + """Where any of these phrases starts, as offsets into the squashed text.""" + return [start for start, _ in _spans(tokens, phrases)] + + +def _spans(tokens: list[str], phrases: tuple[str, ...]) -> list[tuple[int, int]]: + """Where any of these phrases occurs, as `(start, end)` in the squashed text. + + **Ends matter as much as starts, and measuring from the start alone was + wrong.** A weak qualifier is close to what it qualifies, but `consistent + with` is fourteen characters long, so `"a pattern consistent with + coccidiosis"` puts fourteen characters between the qualifier's start and the + disease and none between its end and the disease. Tuning a start-to-start + window to fit it also fitted `"Sole abscess here, and no laminitis anywhere + near it"`, where a denial of one condition cancelled the assertion of + another. Edge-to-edge distance separates them and needs no tuning. + + Two matchers, unioned, because they fail on opposite attacks. + + **Token-sequence matching** compares consecutive tokens, so `lsd` does not + fire inside `lsdx` and `c` does not fire inside `calf`. It is precise and it + loses to any evasion that changes where the word boundaries are. + + **Squashed matching** drops the boundaries entirely and searches the + concatenated text. It exists because of an evasion that beat the token + matcher: a zero-width space inside *"Lumpyโ€‹skin disease"*. Stripping the + invisible character โ€” which is the right thing to do โ€” welds the two words + into `lumpyskin`, and the phrase `lumpy skin disease` no longer matches + three consecutive tokens. Replacing the character with a space instead only + moves the problem, because `conโ€‹firmed` then splits in two. Ignoring + boundaries on both sides catches both, and it also catches the opposite + trick of writing `l u m p y s k i n`. + + Squashed matching is bounded by `_SQUASH_FLOOR` because ignoring boundaries + is what makes a short token match inside an innocent word. + + **It is also bounded by requiring one edge to land on a token boundary**, + and that is what stops the floor being the only guard. `"Moderate or worse + in 4 of every 10 birds, so 40%."` โ€” ยง18's own aggregate, in a farmer's + wording โ€” was refused as a body-temperature claim, because `of every` + squashes to `ofevery`, which contains `fever` at five characters exactly. + Neither edge of that match is a word edge, in either direction. + + **One edge, not both**, because an evasion moves the boundaries *inside* a + phrase and does not move its outer edges: `Lumpyโ€‹skin disease` welds two + tokens and still starts where `lumpy` starts, `conโ€‹firmed` splits one and + still ends where `confirmed` ends, and `"Neither is confirmable here"` keeps + the `confirm` hit its start shares with the token. What is dropped is a + needle floating with both edges inside other words, which is a coincidence + rather than a claim. + """ + hits: list[tuple[int, int]] = [] + squashed = "".join(tokens) + # Where each token begins in `squashed`, so a token-sequence hit and a + # squashed hit are reported on the same scale and can be compared. + starts: list[int] = [] + offset = 0 + for token in tokens: + starts.append(offset) + offset += len(token) + # Every token edge, for the alignment rule in the docstring above. + boundaries = set(starts) | {offset} + + for phrase in phrases: + if phrase == ANY_NUMBER: + hits.extend( + (starts[index], starts[index] + len(token)) + for index, token in enumerate(tokens) if token.isdigit() + ) + continue + wanted = phrase.split() + span = len(wanted) + if span == 0: + continue + for index in range(len(tokens) - span + 1): + if tokens[index:index + span] == wanted: + hits.append(( + starts[index], + starts[index] + sum(len(t) for t in wanted), + )) + + needle = "".join(wanted) + if len(needle) < _SQUASH_FLOOR: + continue + start = squashed.find(needle) + while start != -1: + end = start + len(needle) + if start in boundaries or end in boundaries: + hits.append((start, end)) + start = squashed.find(needle, start + 1) + + return sorted(set(hits)) + + +def _touching(qualifier: tuple[int, int], hit: tuple[int, int]) -> bool: + """Whether a qualifier sits against the thing it qualifies, either side. + + Edge to edge, not start to start. `"coccidiosis-associated"` has the + qualifier ending where the name begins and `"consistent with coccidiosis"` + has it beginning where the name ends, and both are zero apart by this + measure however long the words are. `"Sole abscess ... no laminitis"` has + seven characters of unrelated text between them and is not. + + **A qualifier inside the phrase it would cancel does not cancel it**, and + that is not a corner case: `breed_forced` lists *"no crossbreeding"* and + *"not crossbred"* as the assertions ยง6.5 forbids, and both contain a weak + qualifier at offset zero. Each was cancelling itself, so the one concept + written to catch a claim of purity refused nothing when that claim was + phrased as a denial of mixing. A denial that is part of the claim is the + claim. + """ + if _inside(qualifier, hit): + return False + gap = max(hit[0] - qualifier[1], qualifier[0] - hit[1]) + return gap <= _WEAK_QUALIFIER_REACH + + +def _inside(qualifier: tuple[int, int], hit: tuple[int, int]) -> bool: + """Whether the qualifier is part of the phrase rather than beside it.""" + return qualifier[0] >= hit[0] and qualifier[1] <= hit[1] + + +def scan_prose(text: str) -> list[str]: + """Concepts this text asserts, as a backstop behind the vocabulary check. + + Returns the reasons rather than raising, so a caller can report every + problem in one refusal. Documented as a backstop on purpose: a model that + wants to write a diagnosis can always find a phrasing nobody listed, and the + control that actually holds is that there is no field for it to go in. + """ + tokens, sentence_at = _tokenise(text) + if not tokens: + return [] + + hedges = _positions(tokens, _HEDGES) + # Spans rather than sentence numbers, because a qualifier *inside* the + # phrase it would cancel must not cancel it โ€” `breed_forced` lists "not + # crossbred" and "no crossbreeding", and each was suppressing itself, at + # sentence scope through `not` and at adjacent scope through `no`. ยง6.5's + # one concept for a claim of purity refused nothing when the claim was + # written as a denial of mixing, which is how a model writes it. + qualifiers = _spans(tokens, _QUALIFIERS) + nearby = _spans(tokens, _WEAK_QUALIFIERS) + + found: list[str] = [] + for concept in FORBIDDEN_CONCEPTS: + spans = [_spans(tokens, phrases) for phrases in concept.all_of] + groups = [[start for start, _ in group] for group in spans] + ends = {start: end for group in spans for start, end in group} + index = concept.asserting_group + if 0 <= index < len(groups): + if concept.qualified_by_sentence: + groups[index] = [ + hit for hit in groups[index] + if not any( + sentence_at(qualifier[0]) == sentence_at(hit) + and not _inside(qualifier, (hit, ends.get(hit, hit))) + for qualifier in qualifiers + ) + and not any( + _touching(near, (hit, ends.get(hit, hit))) + and sentence_at(start) == sentence_at(hit) + for near in nearby + for start in (near[0],) + ) + ] + else: + groups[index] = [ + hit for hit in groups[index] + if not any( + 0 < hit - hedge <= _NEGATION_LOOKBEHIND + and sentence_at(hedge) == sentence_at(hit) + for hedge in hedges + ) + ] + if any(not group for group in groups): + continue + if _within_window(groups, concept.within, sentence_at): + found.append( + f"{concept.key}: the response asserts it in prose. " + f"{concept.reason}." + ) + return found + + +def _tokenise(text: str): + """Normalised tokens, plus a way to ask which sentence an offset came from. + + Sentences are split from the *raw* text, because normalisation deletes the + punctuation that marks them โ€” it has to, since punctuation is one of the + things an evasion hides behind. So the split happens first and the + normalisation happens per sentence. + """ + tokens: list[str] = [] + #: One entry per squashed character, holding its sentence number. + owner: list[int] = [] + text = _ACRONYM_DOT.sub("", text) + # **An invisible character splits a token here rather than vanishing.** + # + # `normalise` deletes them, which is right for a reader โ€” `confirmed` + # is `confirmed` on screen. But deleting them here welds the words either + # side into one token, and the squashed rule requires a match to touch a + # token edge. Replacing every space in a sentence with a zero-width space + # therefore collapsed the whole sentence to a single token with no interior + # edges, and **155 of 155 disease names published behind one + # find-and-replace**, against 0 before the edge rule existed. + # + # Splitting keeps both readings: the tokens carry the boundaries a reader + # sees, and squashed matching joins them again, so `confirmed` is + # still caught as `confirmed` while `lumpyskindisease` now + # starts where `lumpy` starts. + text = _INVISIBLE.sub(" ", text) + for number, sentence in enumerate(_SENTENCE_BREAK.split(text)): + for token in normalise(sentence).split(): + tokens.append(token) + owner.extend([number] * len(token)) + + def sentence_at(offset: int) -> int: + return owner[offset] if 0 <= offset < len(owner) else -1 + + return tokens, sentence_at + + +def _within_window(groups: list[list[int]], window: int, sentence_at) -> bool: + """Whether one hit from each group sits inside `window` characters of the + others, **in the same sentence**. + + Brute force over the product would be exponential in the number of groups. + Every concept has two or three groups and a handful of hits each, so the + cheap answer is to walk one group's hits and ask whether every other group + has something close to it. + + **The sentence condition is new and it fixes a false positive on the + product's own idiom.** *"A pattern consistent with coccidiosis. A vet + confirms."* was refused, because `coccidiosis` and `confirms` fell within 48 + characters of each other across a full stop โ€” and ยง32 makes *"a vet confirms + it"* the sentence this whole product is built around, while `SYSTEM_RULES` + tells the model to write *"inconclusive for X"*. The hedge suppression was + already sentence-scoped; the window was not, so a concept could assemble + itself from two different statements. + + Nothing is lost by it. *"Lumpy skin disease. Confirmed."* stops being a + `disease_confirmed` hit and becomes a `disease_named` hit, because the first + sentence names a disease and qualifies nothing โ€” which is the shape + `disease_named` exists for. Array elements are joined with a space rather + than a full stop, so a claim split across two `evidence` items is still one + sentence and still caught. + """ + for anchor in groups[0]: + if all( + any( + abs(hit - anchor) <= window + and sentence_at(hit) == sentence_at(anchor) + for hit in group + ) + for group in groups[1:] + ): + return True + return False + + +def _strings(value: Any) -> list[str]: + """Every string anywhere in the object, keys included. + + Keys as well as values, because a model that cannot put a sentence in a + declared field can still try to put one in an undeclared key โ€” and while + `additionalProperties: false` already refuses that, a control that depends + on one check being reached first is a control with an ordering bug waiting + in it. + """ + if isinstance(value, str): + return [value] + if isinstance(value, dict): + out: list[str] = [] + for key, item in value.items(): + out.append(key) + out.extend(_strings(item)) + return out + if isinstance(value, list): + return [s for item in value for s in _strings(item)] + return [] + + +# --- The whole contract, in one call ------------------------------------------ + + +def enforce( + payload: dict[str, Any], + schema: dict[str, Any], + *, + capability: Capability | None = None, +) -> None: + """Raise `ContractViolation` unless this response may be published. + + Every check runs before anything raises, so the message names all of the + problems rather than the first one. A response that fails is discarded whole + โ€” ยง33 keeps the raw text, and a partially accepted object is a result nobody + can reconstruct. + """ + problems: list[str] = [] + problems.extend(validate(payload, harden(schema))) + if capability is not None: + problems.extend(check_vocabulary(payload, capability)) + problems.extend(check_numeric_bounds(payload, capability)) + problems.extend(check_observation_values(payload, capability)) + problems.extend(check_observation_split(payload)) + + # Runs whether or not a capability was supplied, because it compares the + # payload against itself as well as against the registry. With a capability + # it also requires the corroborating field to be in the *same unit* as the + # prose digit; without one it cannot know any field's unit and says so. + problems.extend(check_quantities(payload, capability)) + problems.extend(check_mixed_scripts(payload)) + + # **Scanned joined, not element by element.** A per-string scan never sees a + # sentence that spans two array entries, and `evidence` is an array of short + # strings โ€” so `["Lumpy skin disease", "Confirmed."]` slipped through + # something that catches `"Lumpy skin disease confirmed"` every time. That + # needs no attacker: a model listing evidence as bullet points produces it + # by accident, which makes it a correctness bug rather than an evasion. + # + # This repository has been bitten by the same shape before. Android's + # `NoInventedDataTest` checked each text node and was defeated by a value + # split across two of them; the fix was to read the concatenated subtree. + # **Anything that scans a container element-wise loses to an author who + # splits.** + # + # Both passes run and their findings are unioned, because joining can also + # hide a match โ€” normalisation and the proximity window behave differently + # on one long string than on several short ones. + strings = _strings(payload) + for text in strings: + problems.extend(scan_prose(text)) + problems.extend(scan_prose(" ".join(strings))) + + if problems: + # Deduplicated because one banned phrase repeated in three evidence + # strings is one finding, and a refusal that repeats itself reads as + # three separate faults. + unique = sorted(set(problems)) + raise ContractViolation( + "The reasoner returned something the product may not publish:\n - " + + "\n - ".join(unique) + + "\nThe response is discarded rather than edited โ€” a repaired " + "answer cannot be traced back to what the model actually said." + ) + + +def schema_for(capability: Capability) -> dict[str, Any]: + """The closed schema this capability's reasoner may answer in. + + Generated from the registry rather than written per capability, so a + capability's allowed vocabulary and the schema a model is handed cannot + drift apart โ€” the failure `BCS_SCHEMA` used to be exposed to, where the + prompt and its schema lived in different files. + """ + allowed = list(capability.acquisition.allowed_claims) + evidence = capability.acquisition.evidence_phrases + limits = capability.acquisition.limit_phrases + output = capability.acquisition.output + units = sorted( + {q.unit for q in output.quantities if q.unit} + | ({output.unit} if output.unit else set()) + ) + schema: dict[str, Any] = { + "type": "object", + "required": ["claims", "evidence"], + "additionalProperties": False, + "properties": { + # The only place a claim may be made, and it is a closed list of + # identifiers. Prose cannot reach this field. + "claims": { + "type": "array", + "items": {"type": "string", "enum": allowed}, + "maxItems": max(1, len(allowed)), + }, + "observations": { + "type": "array", + "items": { + "type": "object", + "required": ["type"], + "additionalProperties": False, + "properties": { + "type": {"type": "string", "enum": allowed}, + "value": {"type": ["number", "string", "null"]}, + # **Sixteen characters of free text, and the registry + # could not fit its own unit names in them.** + # `permanent_incisors` is eighteen and + # `percent_of_scanned_region` is twenty-five, so a + # response labelling its observation the way + # `app/capabilities.py` labels it was refused on a + # string length โ€” which is how ADR 0023's own + # measurement lost `poultry_litter` from both sides of + # a sum and reported 14 and 2 where the answer was 15 + # and 3. + # + # The field is decorative for enforcement: + # `_declared_quantities` reads `ClaimQuantity.unit` from + # the registry and never this, precisely because a + # model writing its own unit could declare its own + # corroboration. So it becomes an enum of the units this + # capability actually declares, which fixes the length + # defect and closes a sixteen-character field a farmer + # reads beside a number โ€” a watchdog published + # `unit: "412 kg exactly"` and `unit: "2.63 exactly"` + # through it once already. + "unit": { + "type": ["string", "null"], + "enum": [*units, None], + }, + "confidence": { + "type": ["number", "null"], "minimum": 0.0, + "maximum": 1.0, + }, + }, + }, + }, + "interpretations": { + "type": "array", + "items": { + "type": "object", + "required": ["label", "requires_review"], + "additionalProperties": False, + "properties": { + "label": {"type": "string", "enum": allowed}, + "confidence": { + "type": ["number", "null"], "minimum": 0.0, + "maximum": 1.0, + }, + # ยง30, as a schema constant rather than as a hope. + "requires_review": {"type": "boolean", "const": True}, + }, + }, + }, + "confidence": { + "type": "string", "enum": ["low", "medium", "high"], + }, + # **This was free text and is a closed vocabulary now** (ADR 0024). + # It is what a farmer reads under the claim, and it is where every + # successful attack on this contract landed: three agents hardened + # the prose scan over it and each was beaten by a rephrasing. + # + # The phrases are the capability's own, drawn from its own rubric โ€” + # ยง7's three body-condition features, ยง11's six hoof findings, ยง12's + # six faecal appearances โ€” and they are reader-facing strings rather + # than identifiers, so the words a farmer sees are reviewed in + # `app/capabilities.py` rather than assembled in a client. + # + # No phrase carries a per-capture number: ยง25's 17 ticks and ยง18's + # grade of 2 stay in `observations[].value`, which `OutputSpec` + # bounds on the unit the registry declared. + "evidence": { + "type": "array", + "items": {"type": "string", "enum": list(evidence)}, + "maxItems": max(1, len(evidence)), + }, + # Closed for the same reason, and closing it was not optional: + # `limits` is an array of strings on the same object, rendered under + # the same result, read by the same farmer. A control that shuts one + # narrative field and leaves the other beside it has moved the + # problem rather than solved it. + "limits": { + "type": "array", + "items": {"type": "string", "enum": list(limits)}, + "maxItems": max(1, len(limits)), + }, + }, + } + + if output.show_range: + # ยง37 and ยง7. A capability that declares a range must return one, and + # `best_estimate` stays nullable so a null remains a correct answer. + # + # **The bounds come from the registry, and used not to.** `items` was + # `{"type": "number"}` and `best_estimate` was `{"type": ["number", + # "null"]}`, so `respiratory_rate_range: [0, 100000]` and + # `best_estimate: 2.63` on `cattle_bcs` both validated โ€” the second + # being the value ยง7 names in the sentence "Do not output BCS 2.63". + # `OutputSpec` had said 1.0 to 5.0 in half points the whole time. + bound = _numeric_schema( + output.plausible_min, output.plausible_max, output.step + ) + schema["required"].append("range") + schema["properties"]["range"] = { + "type": "array", + "items": {"type": "number", **bound}, + "minItems": 2, "maxItems": 2, "ascending": True, + } + schema["properties"]["best_estimate"] = { + "type": ["number", "null"], **bound + } + + return schema diff --git a/app/adapters/deterministic.py b/app/adapters/deterministic.py new file mode 100644 index 0000000000000000000000000000000000000000..670353e685bf0cd7321aaa4c5958ca2a270f97ec --- /dev/null +++ b/app/adapters/deterministic.py @@ -0,0 +1,210 @@ +"""The deterministic methods, as first-class adapters. + +Directive ยง4, and it is the sentence most easily skipped in the whole document: + +> Do not use a neural model when deterministic signal processing is better. + +So optical flow, FFT periodicity, contour measurement and reference-marker +calibration are registered here alongside SAM and DINOv3 rather than living in a +utilities module. They sit in the same registry, answer the same +`availability()`, and appear in the same listing, because a reader comparing the +stack should see that two of the capabilities with the clearest path forward +need no weights at all. + +**These adapters are always available**, which no other adapter in this package +can say. There is no artefact to be absent, no card to be checksummed, no +licence to be refused and no gate to fail โ€” which is most of the argument for +preferring them. `availability()` still exists and still answers, because a +caller should not have to know which kind of adapter it is holding. + +The honesty property is not weaker here, it is only located differently. A +neural adapter refuses by having no model; these refuse by measuring whether the +signal was present, in `periodicity`'s two gates and in `geometry`'s refusal to +invent a scale. +""" + +from __future__ import annotations + +from pathlib import Path + +from app.adapters.base import ( + Adapter, + AdapterSpec, + Availability, + MeasuredCost, + Measurement, + Modality, + Placement, + Task, +) +from app.adapters.signal.geometry import ( + NoReference, + Scale, + measure_region, + scale_from_marker, +) +from app.adapters.signal.respiration import RespirationResult, respiratory_rate + +RESPIRATION_SPEC = AdapterSpec( + adapter_id="respiration-flow-fft", + runtime="opencv-numpy", + tasks=(Task.MEASURE,), + modalities=(Modality.VIDEO,), + directive_role=( + "ยง14 cattle respiratory rate โ€” video, flank region, optical flow, " + "periodicity, FFT, breaths per minute. ยง4 names optical flow, FFT and " + "periodic motion analysis as OpenCV work rather than model work." + ), + requires_artefact=False, + placement=Placement.CPU_SERVICE, + placement_reason=( + "No weights, and the arithmetic is cheap โ€” but the clip is long. Dense " + "flow costs 4.3 ms per frame pair at 320 px, so ยง14's 30โ€“60 second " + "capture is 4โ€“8 seconds of flow plus decode, measured at 9.7 s median " + "for a 31-second clip. **That is past the inline ceiling**, so this " + "capability needs a queue rather than a bigger box. It is also the " + "strongest on-device candidate in the stack: OpenCV is on the phone " + "already, the video never has to leave it, and ADR 0002's offline-first " + "promise is kept for free." + ), + measured=MeasuredCost( + hardware=( + "Apple M-series laptop (NOT the target container). OpenCV 5.0.0 " + "reports 11 threads and ignores setNumThreads(), so a " + "single-threaded figure could not be taken on this build" + ), + threads=11, + sample=( + "Cow_crosses_cattle_grid.webm, 925 frames, 30.86 s at 29.97 fps, " + "whole frame, decode plus flow plus spectrum" + ), + runs=7, + median_seconds=9.68, + peak_rss_mb=239.0, + measured_on="2026-08-21", + ), + notes=( + "**The latency figure is load-sensitive and should be read as a band, " + "not a point.** Seven runs give a 9.68 s median over a 9.29โ€“12.58 s " + "spread, and separate sessions on the same machine and the same clip " + "produced medians of 12.15 s and 14.17 s. The previously recorded " + "9.07 s / 331 MB does not reproduce in any configuration tried: memory " + "is consistently around 239 MB, and no threading setting moves the " + "latency, because this OpenCV build does not honour setNumThreads. " + "What survives all of it is the conclusion โ€” every measurement is past " + "the 8 s inline ceiling, so this capability needs a queue.\n\n" + "**What the metronome validates is the extractor, not this adapter.** " + "On footage whose Commons description states 96 beats per minute, " + "`signal.dominant_rate` returns 96.48, and 48.38 on a crop of the " + "pendulum alone, the swing being half the tick rate โ€” a 0.5% error " + "against a stated rate on real video. But `Metronome.webm` is 11.71 " + "seconds, and `respiration.MIN_CAPTURE_SECONDS` is 20, so " + "`measure()` refuses all three of those regions before any signal " + "processing runs. The only clip with a ground truth cannot reach the " + "code path this adapter exposes, and an earlier version of this note " + "read as though it had. `tests/test_adapters.py` asserts the gap so it " + "cannot be quietly re-closed in prose.\n\n" + "**No cattle rate is validated** โ€” all three real cattle clips are " + "refused, two for being shorter than the capture protocol and one for " + "having no clear rhythm. What is missing is not model work: it is a " + "thirty-second clip of a cow's flank with somebody's counted breath " + "rate beside it." + ), +) + +GEOMETRY_SPEC = AdapterSpec( + adapter_id="marker-geometry", + runtime="opencv-numpy", + tasks=(Task.MEASURE,), + modalities=(Modality.IMAGE,), + directive_role=( + "ยง4 geometry, contour measurement and reference-marker calibration; " + "ยง9's 'approximate visible area: 12โ€“16 cmยฒ' for a wound, and ยง22's " + "fallback scale when metric depth is unreliable." + ), + requires_artefact=False, + placement=Placement.ON_DEVICE, + placement_reason=( + "Marker detection and a contour area are microseconds of arithmetic on " + "a phone. Running it on the device means the farmer learns the card was " + "not in shot while still standing next to the animal, which is the " + "difference between a re-capture and a lost record." + ), + notes=( + "**Unmeasured, and unexercised on a real photograph.** No image " + "available to this project contains an Animap reference marker, so the " + "marker-detection half has never run on anything real. The arithmetic " + "either side of it is exercised by unit tests. Do not quote an area " + "from this until somebody has photographed a printed card beside a " + "ruler." + ), +) + + +class DeterministicAdapter(Adapter): + """Signal processing and geometry. Always available, never guessing.""" + + def __init__(self, spec: AdapterSpec) -> None: + self.spec = spec + + def availability(self) -> Availability: + # OpenCV and NumPy are production dependencies, so there is genuinely + # nothing to check. Importing cv2 here to prove it would make a health + # probe pay for a 60 MB import. + return Availability(True) + + def load(self) -> "DeterministicAdapter": + return self + + +class RespirationAdapter(DeterministicAdapter): + """ยง14, end to end.""" + + def __init__(self) -> None: + super().__init__(RESPIRATION_SPEC) + + def measure( + self, + video_path: Path | str, + *, + region: tuple[float, float, float, float] | None = None, + ) -> RespirationResult: + return respiratory_rate(video_path, region=region) + + +class GeometryAdapter(DeterministicAdapter): + """ยง9 and ยง4, once something in the frame has a known size.""" + + def __init__(self) -> None: + super().__init__(GEOMETRY_SPEC) + + def scale(self, image, marker_side_mm: float) -> Scale: + """Pixels per millimetre from a printed marker. + + Propagates `NoReference` rather than returning a default. A frame with + no marker has no scale, and the honest answer is a re-capture prompt. + """ + return scale_from_marker(image, marker_side_mm) + + def region_size(self, mask, scale: Scale) -> dict: + return measure_region(mask, scale) + + def try_scale(self, image, marker_side_mm: float) -> Measurement: + """The same thing, as a `Measurement` a runner can put in a result.""" + try: + found = self.scale(image, marker_side_mm) + except NoReference as absent: + return Measurement( + kind="scale", value=None, unit="px/mm", usable=False, + detail=str(absent), + ) + return Measurement( + kind="scale", value=round(found.pixels_per_mm, 4), unit="px/mm", + usable=True, + support={"relative_error": round(found.relative_error, 4)}, + detail=found.source, + ) + + +def deterministic_adapters() -> list[Adapter]: + return [RespirationAdapter(), GeometryAdapter()] diff --git a/app/adapters/embedding/__init__.py b/app/adapters/embedding/__init__.py new file mode 100644 index 0000000000000000000000000000000000000000..f98cbf3cedb400b345470e07e81aeb466691fac3 --- /dev/null +++ b/app/adapters/embedding/__init__.py @@ -0,0 +1,58 @@ +"""Frozen visual embeddings, and the identity index built on top of them. + +Two modules, split along the line ยง6.4 draws: + +- `backbones` โ€” the ONNX adapters that turn an image into one unit vector, and + the four specs ยง40.2 asks to be compared. Nothing in it knows what an animal + is. +- `identity` โ€” enrolment, matching, and the open-set decision. Nothing in it + knows what ONNX is. + +The split is not tidiness. The benchmark that chooses a backbone and the index +that serves a farm fail in different ways and are audited by different people, +and keeping the retrieval logic testable without a 837 MB artefact on disk is +what lets the open-set rules have unit tests at all. + +Everything `adapters.embedding` exported before the split is re-exported here, so +`from app.adapters.embedding import DINOV3_SPEC` still resolves. +""" + +from __future__ import annotations + +from app.adapters.embedding.backbones import ( + DINOV2_SPEC, + DINOV3_SPEC, + MEGADESCRIPTOR_SPEC, + MIEWID_SPEC, + OnnxEmbeddingAdapter, + preprocess, +) +from app.adapters.embedding.identity import ( + ENROLMENT_VIEWS, + Candidate, + Embedding, + EnrolledView, + IdentityIndex, + IdentityResult, + IndexMismatch, + OpenSetPolicy, + UnmeasuredThreshold, +) + +__all__ = [ + "DINOV2_SPEC", + "DINOV3_SPEC", + "ENROLMENT_VIEWS", + "MEGADESCRIPTOR_SPEC", + "MIEWID_SPEC", + "Candidate", + "Embedding", + "EnrolledView", + "IdentityIndex", + "IdentityResult", + "IndexMismatch", + "OnnxEmbeddingAdapter", + "OpenSetPolicy", + "UnmeasuredThreshold", + "preprocess", +] diff --git a/app/adapters/embedding/backbones.py b/app/adapters/embedding/backbones.py new file mode 100644 index 0000000000000000000000000000000000000000..b849c28633d1f6534c39265f73d698a64c5f552b --- /dev/null +++ b/app/adapters/embedding/backbones.py @@ -0,0 +1,397 @@ +"""Frozen visual embeddings, through onnxruntime. + +Directive ยง3 on DINOv3: "Do not assume fine-tuning is required. Start with frozen +embeddings + nearest-neighbor retrieval." That is what this is โ€” a backbone with +no head, one vector per image, and every capability that wants it (identity, +breed, BCS reference, fecal reference, footpad reference) built as retrieval on +top rather than as a trained classifier. + +**Why ONNX and not `transformers`.** ADR 0017 took roughly a gigabyte of torch +out of the serving image and cut start-up from about 28 seconds to under one. +Reaching for `transformers` at serve time hands all of that back for a model +whose forward pass is a fixed graph with no control flow. So torch is a +*build-time* tool: `scripts/export_embedding.py` runs it once on a developer's +machine, and the service ships an `.onnx` that `onnxruntime` โ€” already a +production dependency โ€” loads in about a fifth of a second. + +The export puts the pooling and the L2 normalisation inside the graph, so there +is no post-processing convention that can drift between whoever exported the +artefact and whoever serves it. A vector out of this adapter is always unit +length and cosine similarity is always a dot product. + +**The artefact is still governed by a card.** This is not a second way to load a +model โ€” `providers.load_card` checksums it and `adapters.licences.gate` checks +what the runtime really loads under, which is the check ADR 0017 added after a +watchdog defeated the card's own licence field. +""" + +from __future__ import annotations + +import threading +from functools import lru_cache + +import numpy as np +from PIL import Image + +from app.adapters.base import ( + Adapter, + AdapterError, + AdapterSpec, + AdapterUnavailable, + Availability, + MeasuredCost, + Modality, + Placement, + Task, +) +from app.adapters.licences import LicenceRefused, gate +from app.providers import ModelArtefact + +_session_lock = threading.Lock() + + +@lru_cache(maxsize=4) +def _session(artefact_path: str): + """One session per artefact. Building one costs about as much as an + inference, so a request that rebuilds it doubles its own latency.""" + import onnxruntime as ort + + return ort.InferenceSession( + artefact_path, providers=["CPUExecutionProvider"] + ) + + +def preprocess( + image: Image.Image, + size: int, + mean: tuple[float, float, float], + std: tuple[float, float, float], +) -> np.ndarray: + """Resize the short side, centre crop, normalise. RGB, NCHW, float32. + + This is torchvision's standard eval transform written out, for the same + reason `detectors/yolox_onnx.py` writes out its own NMS: pulling in a + training framework for one resize is what the ONNX path exists to avoid. + Getting it wrong does not raise โ€” it quietly returns worse vectors โ€” so the + numbers come from the card rather than from a constant here. + """ + rgb = image.convert("RGB") + width, height = rgb.size + scale = size / min(width, height) + resized = rgb.resize( + (max(size, round(width * scale)), max(size, round(height * scale))), + Image.BICUBIC, + ) + + new_width, new_height = resized.size + left = (new_width - size) // 2 + top = (new_height - size) // 2 + cropped = resized.crop((left, top, left + size, top + size)) + + array = np.asarray(cropped, dtype=np.float32) / 255.0 + array = (array - np.asarray(mean, dtype=np.float32)) / np.asarray( + std, dtype=np.float32 + ) + return np.ascontiguousarray(array.transpose(2, 0, 1)[None]) + + +class OnnxEmbeddingAdapter(Adapter): + """A frozen backbone that turns an image into one unit vector. + + Constructed from a `ModelArtefact` that `providers.load_card` has already + checksummed. It reads nothing else and downloads nothing. + """ + + def __init__( + self, + artefact: ModelArtefact | None, + spec: AdapterSpec, + *, + input_size: int = 224, + mean: tuple[float, float, float] = (0.485, 0.456, 0.406), + std: tuple[float, float, float] = (0.229, 0.224, 0.225), + dimensions: int = 768, + ) -> None: + self.artefact = artefact + self.spec = spec + self.input_size = input_size + self.mean = mean + self.std = std + self.dimensions = dimensions + self._session = None + self._input_name = "" + + def availability(self) -> Availability: + if self.artefact is None: + return Availability( + False, + f"No artefact is installed for {self.spec.adapter_id}.", + "Run scripts/install_models.py, which fetches what the " + "committed card names and refuses anything whose checksum " + "does not match.", + ) + try: + gate(self.spec.runtime, self.artefact.license) + except LicenceRefused as refusal: + return Availability( + False, + str(refusal), + "Move the capability to a permissively licensed backbone.", + ) + if not self.artefact.is_validated: + return Availability( + False, + f"{self.artefact.model_id} has empty validation notes, so " + f"nothing attests that it works.", + "Fill in what was tested, on what data, with what result.", + ) + return Availability(True) + + def load(self) -> "OnnxEmbeddingAdapter": + availability = self.availability() + if not availability.ready: + raise AdapterUnavailable(availability) + + assert self.artefact is not None # availability() proved it + with _session_lock: + session = _session(str(self.artefact.path)) + + inputs = session.get_inputs() + if len(inputs) != 1: + raise AdapterError( + f"{self.artefact.path.name} takes {len(inputs)} inputs; this " + f"adapter was written for a single image tensor." + ) + # Shape-checked rather than trusted, because an export at a different + # resolution produces vectors that are the right length and the wrong + # thing, and nothing downstream would notice. + expected = [3, self.input_size, self.input_size] + actual = list(inputs[0].shape[1:]) + if actual != expected: + raise AdapterError( + f"{self.artefact.path.name} takes {actual}, but the card " + f"describes a {expected} input. The artefact and its card " + f"disagree about what was exported." + ) + + self._session = session + self._input_name = inputs[0].name + return self + + def embed(self, image: Image.Image) -> np.ndarray: + if self._session is None: + raise AdapterError( + "embed() called before load(). There is no path to a vector " + "that skipped the artefact check, and this is it refusing." + ) + blob = preprocess(image, self.input_size, self.mean, self.std) + vector = self._session.run(None, {self._input_name: blob})[0][0] + + if vector.shape[0] != self.dimensions: + raise AdapterError( + f"The graph returned {vector.shape[0]} dimensions; the card " + f"says {self.dimensions}. An index built at one and queried at " + f"the other fails silently, so this fails loudly." + ) + return vector.astype(np.float32) + + def embed_many(self, images: list[Image.Image]) -> np.ndarray: + """One row per image. Kept separate because building a retrieval index + is the batch case and a request is the single case, and batching a + request would only add latency.""" + return np.stack([self.embed(image) for image in images]) + + +#: ยง3's first choice. Bespoke Meta licence with a live ambiguity about +#: attribution โ€” see `licences.RUNTIME_LICENCES["dinov3-onnx"]`. +DINOV3_SPEC = AdapterSpec( + adapter_id="dinov3-vits16", + runtime="dinov3-onnx", + tasks=(Task.EMBED,), + modalities=(Modality.IMAGE,), + directive_role=( + "ยง3 DINOv3 โ€” general visual embeddings, image similarity, retrieval, " + "cattle identity experiments, breed similarity, BCS and fecal and " + "footpad reference retrieval. Frozen, with nearest-neighbour on top." + ), + placement=Placement.CPU_SERVICE, + placement_reason=( + "86.6 MB of ONNX, 59 ms a frame and a 470 MB peak single-threaded. It " + "belongs beside the API, and it is small enough that an on-device " + "build is worth investigating โ€” ADR 0002 makes Animap offline-first, " + "and identity is exactly the capability a worker wants in a pen with " + "no signal." + ), + measured=MeasuredCost( + hardware="Apple M-series laptop (NOT the target container)", + threads=1, + sample="61 Commons frames, evaluation/dataset.json", + runs=61, + median_seconds=0.059, + peak_rss_mb=470.0, + measured_on="2026-08-21", + ), + notes=( + "**Measured at re-identification, which is what it is registered for.** " + "On 169 enrolled cattle from the CC BY 4.0 Zenodo 6324361 muzzle " + "database, five enrolment images each: closed-set top-1 0.977, top-3 " + "0.994, MRR 0.985, against a 0.0059 chance rate. Best of the two " + "servable backbones; the two unservable ones were only run on a " + "30-animal set, where all four saturate. " + "**And the open-set result is the one that shapes the product**: with no " + "threshold it names an unenrolled animal 100% of the time, because every " + "query has a nearest neighbour. The similarity cutoff that admits no " + "impostor accepts only 24.1% of the correct matches โ€” the two " + "distributions overlap badly, enrolled probes median 0.971 against " + "unenrolled median 0.904 with an unenrolled maximum of 0.978. A margin " + "rule does not rescue it. That is why the confirm step in ยง6.4 is " + "load-bearing rather than decorative. " + "No Nigerian and no zebu animal has been through this; the database is " + "US beef breeds. " + "Exported at 224 px so the comparison against DINOv2-small is " + "like-for-like โ€” timm resolves this checkpoint's native config to 256 px, " + "so these figures understate it slightly. On the older Commons proxy it " + "measured 1.000 species 1-NN and 1.000 Nigerian-cattle 1-NN against a " + "0.357 base rate, better than every other backbone and faster than all " + "but DINOv2." + ), +) + +#: The same interface over Apache-2.0 weights of the same size and embedding +#: width. Not a downgrade chosen for convenience โ€” ยง4 asks for the benchmark, +#: and `experiments/cattle_identity/` is where the two are compared. +DINOV2_SPEC = AdapterSpec( + adapter_id="dinov2-small", + runtime="dinov2-onnx", + tasks=(Task.EMBED,), + modalities=(Modality.IMAGE,), + directive_role=( + "ยง3 DINOv3's role, served from the Apache-2.0 generation. 22.06M " + "parameters against DINOv3 ViT-S/16's 21.60M, and the same 384-wide " + "patch embedding." + ), + placement=Placement.CPU_SERVICE, + placement_reason=( + "88.4 MB of ONNX, 81 ms a frame, 390 MB peak. The registered artefact, " + "because it is the one with no licence question attached." + ), + measured=MeasuredCost( + hardware="Apple M-series laptop (NOT the target container)", + threads=1, + sample="61 Commons frames, evaluation/dataset.json", + runs=61, + median_seconds=0.081, + peak_rss_mb=390.0, + measured_on="2026-08-21", + ), + notes=( + "Re-identification on the same 169 enrolled cattle as DINOv3: closed-set " + "top-1 0.957 against DINOv3's 0.977, top-3 0.986 against 0.994, MRR " + "0.971 against 0.985. Its " + "open-set behaviour is worse in the same shape โ€” 21.3% true accepts at " + "the 1% false-accept point against DINOv3's 24.1%. " + "On the Commons proxy, 0.984 species 1-NN and 0.727 Nigerian-cattle 1-NN " + "against a 0.357 base rate. **Measurably worse than DINOv3 on every " + "figure taken on both arms**, which is what makes DINOv3's licence " + "ambiguity worth somebody's time rather than an academic point: the " + "permissive fallback costs about two points of top-1 on a task where " + "the errors are somebody's cow." + ), +) + +#: ยง40.2's head-to-head, and the reason it can exist at all. +#: +#: **This runs and it is not servable, and both halves are deliberate.** The +#: weights are CC-BY-NC-4.0, which a commercial product cannot satisfy at any +#: size; what the founder lifted was the rule that a licence like that stops the +#: model being *measured*. So the artefact is installed, the adapter is built, +#: and `licences.gate` refuses it under the default `enforce` policy and records +#: it under `record`. `describe()` reports `servable: False` either way. +MEGADESCRIPTOR_SPEC = AdapterSpec( + adapter_id="megadescriptor", + runtime="megadescriptor-timm", + tasks=(Task.EMBED,), + modalities=(Modality.IMAGE,), + directive_role=( + "ยง4 and ยง40.2 MegaDescriptor โ€” wildlife re-ID embeddings for cattle " + "identity, benchmarked head-to-head against DINOv3." + ), + placement=Placement.CPU_SERVICE, + placement_reason=( + "837 MB of ONNX and a measured 1,296 MB peak, which is inside the " + "2,000 MB the CPU worker is judged against but 3.7x DINOv3's peak on " + "the same run. The 0.723 s median is also inside the inline ceiling. " + "It fits; it is simply not worth the room, because it lost the " + "benchmark it was installed to win." + ), + measured=MeasuredCost( + hardware="Apple M-series laptop (NOT the target container)", + threads=1, + sample="61 Commons frames, evaluation/dataset.json", + runs=61, + median_seconds=0.723, + peak_rss_mb=1295.8, + measured_on="2026-08-21", + ), + notes=( + "**ยง40.2 answered: DINOv3 wins, and not narrowly.** MegaDescriptor-L-384 " + "measured 0.934 species 1-NN and 0.636 Nigerian-cattle 1-NN against a " + "0.357 base rate; DINOv3 measured 1.000 and 1.000 on the same 61 frames " + "in the same run. It is beaten by Apache-2.0 DINOv2-small on both " + "accuracy figures as well, at roughly 10x the artefact size and 7x the " + "latency. Its nearest-neighbour cosines are much flatter โ€” 0.257 median " + "against DINOv3's 0.672. " + "**None of that measures re-identification**, which is what " + "MegaDescriptor is for: no available image set has the same animal " + "twice, so this says the space is worse *organised* for cattle and " + "geography, not that it cannot tell two White Fulani apart. " + "L-384 was chosen over the smaller variants because ยง40.2 names it and " + "because 837 MB of ONNX exports cleanly under the 2 GB protobuf limit; " + "T-224, S-224, B-224 and L-224 publish checkpoints of 204, 290, 473 and " + "1,922 MB and none was exported." + ), +) + +#: The other non-commercial contender, installed for the same reason and to no +#: better end. Its licence problem is quieter than MegaDescriptor's: nothing was +#: granted at all, and silence defaults to all rights reserved. +MIEWID_SPEC = AdapterSpec( + adapter_id="miewid-msv3", + runtime="miewid", + tasks=(Task.EMBED,), + modalities=(Modality.IMAGE,), + directive_role=( + "ยง4's 'Wildlife ReID embeddings' โ€” the alternative to MegaDescriptor, " + "benchmarked alongside it under ยง40.2." + ), + placement=Placement.CPU_SERVICE, + placement_reason=( + "206 MB of ONNX, 0.213 s a frame, 502 MB peak. Comfortably the cheapest " + "of the two wildlife re-ID models and still 2.9x DINOv3's latency for " + "the worst Nigerian retrieval of the four." + ), + measured=MeasuredCost( + hardware="Apple M-series laptop (NOT the target container)", + threads=1, + sample="61 Commons frames, evaluation/dataset.json", + runs=61, + median_seconds=0.213, + peak_rss_mb=502.2, + measured_on="2026-08-21", + ), + notes=( + "Measured 0.951 species 1-NN and 0.455 Nigerian-cattle 1-NN against a " + "0.357 base rate โ€” a lift of 1.27x on 11 frames, which is a failure to " + "show anything rather than a measured floor. Last of the four on the " + "figure that matters most for Nigerian farms. " + "**Building this artefact meant running a third party's Python.** The " + "upstream repo ships `modeling_miewid.py` instead of a `transformers` " + "architecture, so `scripts/export_embedding.py` loads it with " + "`trust_remote_code=True`. That is a build-step supply-chain exposure, " + "not a serving one โ€” the service loads a fixed ONNX graph with no " + "Python in it โ€” and the three modules were read before they were run. " + "Preprocessing deviates from the published transform: the model card " + "specifies `Resize((440, 440))` and this pipeline centre-crops, so its " + "figures here may understate it." + ), +) diff --git a/app/adapters/embedding/identity.py b/app/adapters/embedding/identity.py new file mode 100644 index 0000000000000000000000000000000000000000..e16c8a7144494d0e40790d174f28dd9cba4e8ee2 --- /dev/null +++ b/app/adapters/embedding/identity.py @@ -0,0 +1,657 @@ +"""Enrolment, matching, and the open-set decision for `cattle_identity`. + +Directive ยง6.4. An animal is enrolled from five views โ€” front face, left face, +right face, muzzle, side body โ€” each stored as one unit vector. A new photograph +is embedded, compared against every enrolled animal, and the farm is shown a +ranked list with *"This looks like Kofi"* on top and a confirm button under it. + +**Why there is no model here.** The exemplar is the animal. `poultry_house_count` +proved the shape on a different capability: CountGD reached MAE 14.84 on frames a +trained detector found 8.5% of the birds in, because it was shown three example +boxes instead of being retrained. An enrolled animal is the same trick โ€” five +photographs of Kofi are what teaches the system Kofi, and nothing is fitted. +That is also why this module imports no model code and takes vectors rather than +images: the backbone is chosen by `experiments/cattle_identity/`, and this file +must keep working when that choice changes. + +## The two rules that are load-bearing + +**An index belongs to one backbone.** DINOv3's 768 dimensions and MegaDescriptor's +1536 are not the only difference between them; two exports of the *same* backbone +with different pooling produce vectors of identical length and incompatible +meaning, which the model cards in `models/alternates/` each warn about +separately. A silent mismatch does not error โ€” it returns confident nonsense, +which is the failure this project can least afford on the capability every other +record hangs off. + +So the index pins the backbone id and the artefact digest it was built under, and +checks both against any `Embedding` handed to it. **A bare `ndarray` can only be +checked for width**, which is not provenance, and the count of those is kept on +`unverified_queries` rather than being waved through silently. This paragraph +previously claimed the index "refuses a query that arrives from anything else" +while the code compared only dimensions; a watchdog built an index tagged +`dinov3-vits16`, scored a foreign vector against it at similarity 1.0, and was +right to call the sentence false. Prefer `Embedding` at every call site that can +produce one. + +**A threshold is measured or it is absent.** Closed-set accuracy โ€” *given that +this animal is enrolled, is the top candidate right* โ€” is the easy half. The half +that decides whether the product is safe is open-set: an animal nobody enrolled +must not come back as Kofi. That boundary is a number, and a number nobody +measured is a guess with a decimal point on it (ยง37). `OpenSetPolicy.measured` +is the only constructor that produces thresholds and it demands the run id that +produced them; `OpenSetPolicy.unmeasured` produces none, still returns the ranked +candidates ยง6.4 asks for, and marks the result `open_set_verified=False` with a +warning the caller has to carry. It does not invent a cutoff, and it does not +suppress the capability either. + +## What a result is allowed to say + +`app/capabilities.py` already fixes the vocabulary: allowed claims are +`identity_candidate` and `no_confident_match`, the forbidden one is +`identity_without_confirmation`, and the four confirmation options are +`confirm`, `not_this_animal`, `choose_another_animal`, `register_new_animal`. +This module emits exactly those and nothing else. + +**A candidate carries a similarity and never a confidence.** Nothing calibrated +cosine similarity into a probability that the animal is Kofi, and two White +Fulani photographed in the same light score high because the light is the same. +`Candidate.confidence` is therefore `None` until something measures a mapping, +and `Interpretation.confidence` downstream stays `None` with it. + +## Why the ranked list is the shape, rather than one answer + +`evidence_correction.selected_interpretation` โ€” a foreign key at +`services/api/apps/evidence/models/correction.py` โ€” exists so that a farmer +picking the second name in the list records *"rank 2 was right"* rather than +*"the model was wrong"*. Its own docstring calls it the highest-value training +signal in the table. Returning one name would throw that gradient away at the +moment it is collected, which is the same mistake ยง32 records having been made +with confirmations. So `match` returns a list with ranks on it even when the top +candidate is obvious. +""" + +from __future__ import annotations + +from dataclasses import dataclass, field +from typing import Iterable, Mapping, Sequence + +import numpy as np + +#: ยง6.4's enrolment protocol, in the order the directive lists it. The registry +#: entry for `cattle_identity` declares the same five as `required_views`, and +#: `tests/test_identity.py` asserts the two agree โ€” a capture flow and an index +#: that disagree about what a view is called produce an enrolment that silently +#: stores nothing under the name the matcher looks for. +ENROLMENT_VIEWS: tuple[str, ...] = ( + "front_face", + "left_face", + "right_face", + "muzzle", + "side_body", +) + +#: The view ยง6.4 says to lead with, and the one the identity literature is about. +#: Coat pattern is not an option for Nigerian herds โ€” a White Fulani is white all +#: over, so the Holstein re-identification results that dominate the published +#: work transfer to the method and not to the signal. +PRIMARY_VIEW = "muzzle" + + +class IndexMismatch(ValueError): + """A query vector that did not come from the backbone this index was built on.""" + + +class UnmeasuredThreshold(ValueError): + """A threshold was asserted without the measurement that produced it.""" + + +def _unit(vector: np.ndarray) -> np.ndarray: + """Length-check and normalise. + + The ONNX exports normalise inside the graph, so this is usually a no-op โ€” and + it runs anyway, because an index is also built in tests and in notebooks from + vectors that did not come through the graph, and cosine similarity computed + as a dot product over a non-unit vector is wrong without being an error. + """ + array = np.asarray(vector, dtype=np.float32).reshape(-1) + norm = float(np.linalg.norm(array)) + if norm == 0.0: + raise ValueError( + "A zero vector has no direction, so it is nearest to everything and " + "to nothing. Something upstream returned an empty embedding." + ) + return array / norm + + +@dataclass(frozen=True) +class Embedding: + """A vector that says where it came from. + + **This exists because the docstring above used to be false.** It claimed the + index "refuses a query that arrives from anything else", and the code checked + only the *width* โ€” so a vector from a different pooling of the same backbone, + which is the exact case the model cards warn about and which produces the + right number of dimensions and the wrong meaning, sailed through. A watchdog + built an index tagged `dinov3-vits16` and scored a foreign vector against it + at similarity 1.0. + + Passing one of these makes the check real. A bare `ndarray` is still + accepted, because tests and notebooks legitimately have vectors with no + provenance, and `IdentityIndex.unverified_queries` counts how many arrived + that way so the gap is visible rather than assumed away. + """ + + vector: np.ndarray + #: The adapter that produced it, e.g. `dinov3-vits16`. + backbone_id: str + #: The sha256 of the ONNX artefact. Empty means the producer did not say, + #: which is checked as far as it can be and no further. + artefact_sha256: str = "" + + +#: What `enrol` and `candidates` accept: a vector, or a vector that can prove +#: where it came from. +Vector = "np.ndarray | Embedding" + + +@dataclass(frozen=True) +class EnrolledView: + """One photograph of one animal, as a unit vector. + + The image itself is not held. ยง33 requires the raw media to be preserved so a + later correction can be trained on, and that is the API's job โ€” `MediaAsset` + with a retention hold written by the correction. An index that also kept the + bytes would be a second copy nobody was pinning. + """ + + animal_id: str + view: str + vector: np.ndarray + #: The media id the vector came from, so a candidate can be traced back to a + #: photograph a person can look at. ยง6.4's *"Not Kofi"* is far more useful + #: when the farm can see which picture of Kofi the system thought matched. + media_id: str = "" + + +@dataclass(frozen=True) +class OpenSetPolicy: + """When a top candidate is good enough to show as a name. + + Two thresholds rather than one, because they fail differently. + `accept_similarity` catches the animal nobody enrolled: every candidate is + poor, and the best of them is still poor. `accept_margin` catches the case + that matters more on a real farm โ€” a herd of animals that genuinely look + alike, where the top two candidates are both strong and the ordering between + them is noise. A margin rule refuses to pick a name out of a tie, which is + exactly the situation *"This looks like Kofi"* is most likely to be wrong in + and most likely to be believed in. + """ + + #: `None` when nothing has been measured. Not a permissive default and not a + #: strict one; the absence is carried into the result instead. + accept_similarity: float | None + accept_margin: float | None + #: Where the numbers came from. Empty only for `unmeasured`. + measured_by_run_id: str = "" + measured_on: str = "" + #: Why there is no threshold. Empty only for `measured`. + unmeasured_reason: str = "" + + def __post_init__(self) -> None: + has_thresholds = ( + self.accept_similarity is not None or self.accept_margin is not None + ) + if has_thresholds and not self.measured_by_run_id: + raise UnmeasuredThreshold( + "A threshold decides whether a farm is shown an animal's name, " + "so it names the run that measured it. Use " + "OpenSetPolicy.unmeasured() to say honestly that nothing has." + ) + if not has_thresholds and not self.unmeasured_reason: + raise UnmeasuredThreshold( + "A policy with no thresholds has to say why it has none." + ) + + @classmethod + def measured(cls, *, accept_similarity: float, accept_margin: float, + run_id: str, measured_on: str) -> "OpenSetPolicy": + """Thresholds from a benchmark, citing it. + + `run_id` is a `results/run-*.json` in `experiments/cattle_identity/`, and + `measured_on` names the image set. Both travel onto every result the + policy decides, so a threshold that came from the wrong species or the + wrong farm is visible in the record rather than only in somebody's memory. + """ + if not run_id: + raise UnmeasuredThreshold("A measured policy cites the run that measured it.") + return cls( + accept_similarity=float(accept_similarity), + accept_margin=float(accept_margin), + measured_by_run_id=run_id, + measured_on=measured_on, + ) + + @classmethod + def unmeasured(cls, reason: str) -> "OpenSetPolicy": + """No thresholds, and the reason recorded. + + This is what ships until an open-set benchmark exists. It does **not** + disable matching โ€” ยง6.4 says to build immediately and the ranked list is + useful with a confirm button under it โ€” it records that the boundary + between *"this looks like Kofi"* and *"I do not know this animal"* has + never been measured, so the product must not lean on it. + """ + return cls( + accept_similarity=None, accept_margin=None, unmeasured_reason=reason + ) + + @property + def is_measured(self) -> bool: + return self.accept_similarity is not None or self.accept_margin is not None + + def to_json(self) -> dict: + return { + "accept_similarity": self.accept_similarity, + "accept_margin": self.accept_margin, + "measured_by_run_id": self.measured_by_run_id, + "measured_on": self.measured_on, + "unmeasured_reason": self.unmeasured_reason, + "is_measured": self.is_measured, + } + + +#: The default until an open-set benchmark exists. Named rather than constructed +#: at each call site so there is one place to change when one does, and so a grep +#: for it finds every caller that is still running without a measured boundary. +NO_MEASURED_THRESHOLD = OpenSetPolicy.unmeasured( + "No open-set benchmark has been run for cattle identity. Nothing has " + "measured how similar an unenrolled animal looks to the nearest enrolled " + "one, so there is no cutoff to apply and every candidate needs a person." +) + + +@dataclass(frozen=True) +class Candidate: + """One enrolled animal, ranked, with the view that matched it. + + `similarity` is a cosine in [-1, 1] and `confidence` is `None`. The two are + separate fields rather than one so that nothing downstream can quietly + promote the first into the second โ€” ยง37's whole complaint is that an + uncalibrated model score reaches a person as a promise. + """ + + rank: int + animal_id: str + #: The name a screen shows. Supplied by the caller from the farm's own + #: records, never derived from `animal_id`, for the same reason + #: `saveConfirmation` takes `displayText` separately from + #: `confirmationOption`: rendering a key as copy is how an id reaches a + #: person as a word. + display_name: str + similarity: float + #: Which of the five enrolled views scored highest, and the photograph it + #: came from. + matched_view: str + matched_media_id: str = "" + #: Never set by this module. Present because `Interpretation.confidence` + #: exists and something may one day calibrate it against confirmations. + confidence: float | None = None + + def to_json(self) -> dict: + return { + "rank": self.rank, + "animal_id": self.animal_id, + "display_name": self.display_name, + "similarity": round(self.similarity, 4), + "matched_view": self.matched_view, + "matched_media_id": self.matched_media_id, + "confidence": self.confidence, + } + + +@dataclass(frozen=True) +class IdentityResult: + """What ยง6.4 shows a farm, and what ยง32 stores when they answer. + + `claim` is one of the two `app/capabilities.py` allows for this capability. + `confirmation_options` is the registry's list verbatim, because the option a + person taps is written into `evidence_correction.confirmation_option` and + compared against the registry there; a list assembled independently here + would drift. + """ + + claim: str + candidates: tuple[Candidate, ...] + policy: OpenSetPolicy + enrolled_animals: int + #: True only when a measured policy accepted the top candidate. False both + #: when a measured policy rejected it and when no policy has been measured โ€” + #: which are different states, and `warnings` says which. + #: + #: **Both rejection branches returned `True` until this was corrected**, so + #: a refusal reached the API as `open_set_verified 1.0` beside + #: `no_confident_match` and a similarity under the cutoff โ€” a stored row + #: saying, of one run, both that nothing was verified and that something + #: was. Whoever re-derives the threshold from stored results reads this + #: column, and it is the one that has to mean a verdict. + #: + #: *Was the check performed* is a different question and has its own answer: + #: `policy.is_measured`, which travels in the same result. + open_set_verified: bool + warnings: tuple[str, ...] = () + #: ยง6.4's four buttons. Order matters: it is the order the directive lists. + confirmation_options: tuple[str, ...] = ( + "confirm", "not_this_animal", "choose_another_animal", "register_new_animal", + ) + + @property + def top(self) -> Candidate | None: + return self.candidates[0] if self.candidates else None + + def to_json(self) -> dict: + return { + "claim": self.claim, + "candidates": [c.to_json() for c in self.candidates], + "open_set_verified": self.open_set_verified, + "enrolled_animals": self.enrolled_animals, + "policy": self.policy.to_json(), + "warnings": list(self.warnings), + "confirmation_options": list(self.confirmation_options), + # Stated on the result rather than left to a UI, because the one + # claim ยง6.4 forbids is an identity asserted without a person, and + # the surface that would forget is the one furthest from this file. + "requires_confirmation": True, + } + + +@dataclass +class IdentityIndex: + """Every enrolled animal on one farm, as vectors. + + **`farm_id` is a label, not a mechanism, and saying otherwise was an + overstatement a watchdog caught.** Nothing in `enrol` or `candidates` reads + it. What actually keeps one farm's animals away from another's is that the + caller builds one index per farm and never enrols across them โ€” a discipline + this class records but does not enforce. The field is here so that a + mismatch is *detectable*: an index can say which farm it believes it holds, + and a caller that cached the wrong one can be caught by comparing. + + It matters because every other table in this product is `FarmScopedModel`, + and an index holding two farms' animals would answer *"which animal is + this"* with a neighbour's cow, which is both wrong and a disclosure. If this + ever moves behind an API that takes a farm id from a request, the check + belongs there and not here. + + Small by design: a farm has tens to low hundreds of animals and five views + each, so a brute-force matrix multiply over a few hundred rows is + microseconds and an approximate-nearest-neighbour structure would add a + dependency, an index-build step and a recall question in exchange for + nothing. The moment that stops being true is when one farm passes a few + thousand animals, and `match` is where it would change. + """ + + farm_id: str + #: The backbone that produced every vector in here, and the artefact digest + #: it was produced by. Both are checked against any `Embedding` that arrives; + #: a bare `ndarray` can only be checked for width, and each one that arrives + #: increments `unverified_queries`. + backbone_id: str + dimensions: int + artefact_sha256: str = "" + views: list[EnrolledView] = field(default_factory=list) + #: How many vectors were accepted without being able to prove their origin. + #: Not an error and not zero in practice โ€” it is the size of the gap between + #: what this class checks and what it would like to. + unverified_queries: int = 0 + + def _accept(self, vector: "np.ndarray | Embedding") -> np.ndarray: + """Check where a vector came from, then normalise it. + + Width alone is not provenance. Two exports of the same backbone with + different pooling produce vectors of identical length and incompatible + meaning โ€” every model card in `models/alternates/` warns about it + separately โ€” and the comparison would succeed rather than fail, which on + this capability means a confident wrong name. + """ + if isinstance(vector, Embedding): + if vector.backbone_id != self.backbone_id: + raise IndexMismatch( + f"This index was built on {self.backbone_id!r} and the " + f"vector came from {vector.backbone_id!r}. Two backbones' " + f"vectors are not comparable, and comparing them succeeds " + f"rather than fails." + ) + if ( + vector.artefact_sha256 and self.artefact_sha256 + and vector.artefact_sha256 != self.artefact_sha256 + ): + raise IndexMismatch( + f"Same backbone name, different artefact: the index was " + f"built under {self.artefact_sha256[:12]}โ€ฆ and the query " + f"came from {vector.artefact_sha256[:12]}โ€ฆ. A re-export " + f"with different pooling is the case this catches." + ) + array = vector.vector + else: + # Counted rather than refused. A test or a notebook has a bare array + # and no way to prove anything about it, and refusing would make the + # provenance check the reason nobody uses the class. + self.unverified_queries += 1 + array = vector + + unit = _unit(array) + if unit.shape[0] != self.dimensions: + raise IndexMismatch( + f"The vector is {unit.shape[0]}-dimensional and this index is " + f"{self.dimensions}. It was built on {self.backbone_id!r}." + ) + return unit + + # ---- enrolment ---------------------------------------------------------- + + def enrol(self, animal_id: str, vectors: Mapping[str, np.ndarray], + media_ids: Mapping[str, str] | None = None) -> tuple[str, ...]: + """Store one animal's views. Returns the ยง6.4 views still missing. + + Unknown view names are refused rather than stored: a typo'd `"muzzel"` + would enrol cleanly, never be queried by a muzzle capture, and leave a + farm wondering why one animal never matches. + + A partial enrolment is allowed and reported. Requiring all five would + mean a worker who cannot get a cow to hold still for a front-face shot + enrols nothing at all, and one good muzzle photograph is worth more than + a refused enrolment โ€” but the caller is told what is missing so a capture + flow can ask for the rest later. + """ + unknown = sorted(set(vectors) - set(ENROLMENT_VIEWS)) + if unknown: + raise ValueError( + f"{unknown} are not enrolment views. ยง6.4 names exactly " + f"{list(ENROLMENT_VIEWS)}, and a view stored under any other " + f"name is a view nothing will ever query." + ) + media_ids = media_ids or {} + for view, vector in vectors.items(): + try: + unit = self._accept(vector) + except IndexMismatch as mismatch: + raise IndexMismatch( + f"{view} of {animal_id}: {mismatch} Mixing backbones in one " + f"index produces matches that are arithmetic rather than " + f"evidence." + ) from mismatch + self.views.append( + EnrolledView( + animal_id=animal_id, view=view, vector=unit, + media_id=media_ids.get(view, ""), + ) + ) + return tuple(v for v in ENROLMENT_VIEWS if v not in vectors) + + @property + def animal_ids(self) -> tuple[str, ...]: + """Enrolled animals, in enrolment order and without repeats.""" + seen: dict[str, None] = {} + for view in self.views: + seen.setdefault(view.animal_id, None) + return tuple(seen) + + def views_for(self, animal_id: str) -> tuple[str, ...]: + return tuple(v.view for v in self.views if v.animal_id == animal_id) + + def missing_views(self, animal_id: str) -> tuple[str, ...]: + held = set(self.views_for(animal_id)) + return tuple(v for v in ENROLMENT_VIEWS if v not in held) + + # ---- matching ----------------------------------------------------------- + + def candidates(self, query: np.ndarray, *, top_k: int = 3, + restrict_to_views: Sequence[str] | None = None, + names: Mapping[str, str] | None = None, + exclude: Iterable[str] = ()) -> tuple[Candidate, ...]: + """The ranked animals, best first. + + An animal scores its **best** view, not its average. The query is one + photograph of one part of an animal, and averaging a muzzle close-up + against a side-body shot dilutes the view that actually matched with four + that could not have. Maximum over views is the standard multi-shot + retrieval rule and it is the right one here for a reason specific to + ยง6.4: the five enrolment views are deliberately *different pictures*, not + five samples of one distribution. + + `restrict_to_views` is how a muzzle capture asks to be compared against + muzzles only. Worth using when the capture flow knows what it took โ€” + comparing a muzzle print against a side-body vector contributes nothing + but a chance of a spurious high score. + + `exclude` drops named animals, which is what makes leave-one-out + evaluation possible without building a second index per query. + """ + names = names or {} + excluded = set(exclude) + wanted = set(restrict_to_views) if restrict_to_views else None + + unit = self._accept(query) + + pool = [ + v for v in self.views + if v.animal_id not in excluded + and (wanted is None or v.view in wanted) + ] + if not pool: + return () + + scores = np.stack([v.vector for v in pool]) @ unit + + best: dict[str, tuple[float, EnrolledView]] = {} + for view, score in zip(pool, scores): + current = best.get(view.animal_id) + if current is None or score > current[0]: + best[view.animal_id] = (float(score), view) + + ordered = sorted( + best.items(), + # Ties broken by animal id rather than left to dict order, so the + # same index and the same query always produce the same list. A + # ranked list that reorders between runs makes `selected_ + # interpretation` mean two different things on two devices. + key=lambda item: (-item[1][0], item[0]), + )[:top_k] + + return tuple( + Candidate( + rank=position, + animal_id=animal_id, + display_name=names.get(animal_id, animal_id), + similarity=score, + matched_view=view.view, + matched_media_id=view.media_id, + ) + for position, (animal_id, (score, view)) in enumerate(ordered, start=1) + ) + + def match(self, query: np.ndarray, *, policy: OpenSetPolicy = NO_MEASURED_THRESHOLD, + top_k: int = 3, restrict_to_views: Sequence[str] | None = None, + names: Mapping[str, str] | None = None, + exclude: Iterable[str] = ()) -> IdentityResult: + """The ยง6.4 answer: a ranked list, a claim, and four buttons. + + The claim is `no_confident_match` in three situations, and they are worth + distinguishing because only one of them is a model result: nothing is + enrolled yet, a measured policy rejected the top candidate on similarity, + or a measured policy rejected it on margin. All three are honest answers + and the third is the one that protects a farm from a confident wrong name. + """ + ranked = self.candidates( + query, top_k=top_k, restrict_to_views=restrict_to_views, + names=names, exclude=exclude, + ) + enrolled = len({v.animal_id for v in self.views} - set(exclude)) + warnings: list[str] = [] + + if not ranked: + return IdentityResult( + claim="no_confident_match", + candidates=(), policy=policy, enrolled_animals=enrolled, + open_set_verified=False, + warnings=( + "Nothing to compare against. No animal on this farm has an " + "enrolled view matching this capture.", + ), + ) + + if not policy.is_measured: + warnings.append( + "The open-set boundary is unmeasured, so this candidate has not " + "been checked against the possibility that the animal is not " + "enrolled at all. " + policy.unmeasured_reason + ) + return IdentityResult( + claim="identity_candidate", candidates=ranked, policy=policy, + enrolled_animals=enrolled, open_set_verified=False, + warnings=tuple(warnings), + ) + + top = ranked[0] + runner_up = ranked[1].similarity if len(ranked) > 1 else None + margin = None if runner_up is None else top.similarity - runner_up + + if policy.accept_similarity is not None and top.similarity < policy.accept_similarity: + warnings.append( + f"Best match scored {top.similarity:.3f}, under the " + f"{policy.accept_similarity:.3f} measured on {policy.measured_on} " + f"in run {policy.measured_by_run_id}. On that evidence this is " + f"more likely an animal nobody has enrolled." + ) + return IdentityResult( + claim="no_confident_match", candidates=ranked, policy=policy, + # False, because nothing was verified. The check ran and it + # said no; `policy.is_measured` is what records that it ran. + open_set_verified=False, + enrolled_animals=enrolled, + warnings=tuple(warnings), + ) + + if (policy.accept_margin is not None and margin is not None + and margin < policy.accept_margin): + warnings.append( + f"{ranked[0].display_name} and {ranked[1].display_name} are " + f"{margin:.3f} apart, under the {policy.accept_margin:.3f} " + f"measured in run {policy.measured_by_run_id}. Two animals look " + f"this alike; picking between them is not something this " + f"photograph supports." + ) + return IdentityResult( + claim="no_confident_match", candidates=ranked, policy=policy, + # False for the same reason as the branch above: two animals + # too alike to separate is a refusal, not a verified match. + open_set_verified=False, + enrolled_animals=enrolled, + warnings=tuple(warnings), + ) + + return IdentityResult( + claim="identity_candidate", candidates=ranked, policy=policy, + enrolled_animals=enrolled, open_set_verified=True, + ) diff --git a/app/adapters/fingerprints.py b/app/adapters/fingerprints.py new file mode 100644 index 0000000000000000000000000000000000000000..82bf462d941d06655572c1654574d9317fd776b3 --- /dev/null +++ b/app/adapters/fingerprints.py @@ -0,0 +1,891 @@ +"""What an artefact *is*, read from its bytes rather than from its card. + +**Nothing else in this service binds a model's identity to its contents.** +`providers.load_card` checksums the file, so the card and the bytes cannot drift +apart โ€” but a sha256 only says *"these are the bytes somebody wrote this card +about"*. It says nothing about what the bytes are. Change the `runtime` field +and the same checksum now describes AGPL Ultralytics weights served as +`cattle_identity` and reported as Apache-2.0, because every licence decision in +this package keys off a string the card supplies. + +ADR 0017's fix was a hard-coded `DISALLOWED_RUNTIMES = {"ultralytics"}`, and +`app/adapters/licences.py` generalised it into a table of what each runtime's +weights are really licensed under. Both improve on trusting the `license` field +and neither closes the hole, because both still start from the card's claim +about which loader runs. This module supplies the missing fact. + +## What it reads + +An ONNX file is a protobuf. Its framing carries the graph's structure โ€” input +and output names and shapes, every node's op type, every initializer's name and +dimensions โ€” and that structure is a fingerprint no card can edit without +editing the model. A DINOv3 export has a 16-pixel patch convolution, a +`reg_token` of shape `[1, 4, 384]`, twenty-four `gamma_1`/`gamma_2` LayerScale +vectors and a pair of RoPE `Sin`/`Cos` nodes. A DINOv2 export has a 14-pixel +patch convolution, Hugging Face's `encoder.layer.N.โ€ฆ` module-path naming, and no +`Sin` or `Cos` anywhere. YOLOX has 83 or more convolutions, purely numeric +initializer names, and a `[1, 8400, 85]` output. + +**The reader is written against the protobuf wire format directly, using only +the standard library.** That is a deliberate constraint rather than an +affectation: `onnx` lives in `requirements-export.txt`, which says *"Build-time +only. Never install this into the serving image."* A control that only runs +where its dependency is installed is a control that is absent in production, and +`onnxruntime` โ€” which is a production dependency โ€” cannot substitute. Its +`get_modelmeta()` exposes `producer_name`, `graph_name`, `domain`, `version` and +a metadata map, and no nodes, no initializers and no attributes at all. Through +onnxruntime alone, DINOv2 and DINOv3 both report `producer_name='pytorch'`, +`graph_name='main_graph'` and an output of `['batch', 768]`: indistinguishable. + +The scan never materialises `TensorProto.raw_data`. It walks the framing and +skips those byte ranges, so reading an 87 MB artefact costs about 22 ms and +68 MB of peak RSS โ€” roughly three times faster than the sha256 the loader +already pays for, and twenty-six times faster on the 396 MB YOLOX-x. + +A torch `.pt` is a zip. Its pickle is read with `pickletools.genops`, which +decodes opcodes without executing any of them, so identifying an Ultralytics +checkpoint never runs a line of its author's code. + +## What it does not do + +**It does not decide what may be served.** The founder's standing instruction is +that no model is dropped for its licence right now, so a mismatch between the +bytes and the card records itself and warns by default. What changes is that the +record is *true*: an exception logged under `ANIMAP_LICENCE_POLICY=record` names +the licence the bytes actually arrive under instead of the one the card claimed, +which is the whole reason for keeping a ledger. `ANIMAP_ARTEFACT_IDENTITY=refuse` +turns the same finding into a refusal, one variable on one deployment. + +**An unidentified artefact is not a mismatch.** A file this module has no +fingerprint for is recorded as unverified and loads. Treating "nobody has +written a fingerprint for MegaDescriptor yet" as "this file is lying" would make +every new model an incident, and the honest state of an unfingerprinted artefact +is that its identity rests on the card โ€” exactly where it rested before. + +**A refuted claim is a mismatch, and used not to be one.** `Identification. +refutes` draws the line the paragraph above was missing: a card naming a runtime +that *has* a fingerprint, over bytes that fail it, is a claim this module +disproved, not a file it has nothing to say about. Renaming an Ultralytics +export's initializers is enough to stop `ULTRALYTICS_ONNX` matching and was +enough to serve AGPL weights as `cattle_detection`; it is not enough to make +those bytes YOLOX, and `YOLOX_ONNX` is right there to say so. `providers.py` +reads it, so nothing loads on a card claim its own artefact contradicts. + +**A fingerprint identifies a family, not an export.** It matches on structure +that survives a re-export, so re-running `scripts/export_embedding.py` under a +different torch does not turn DINOv3 into an impostor. Pinning the exact bytes +is what sha256 is for, and doing it twice would only produce a check that fails +for the wrong reason. +""" + +from __future__ import annotations + +import logging +import mmap +import pickletools +import re +import zipfile +from dataclasses import dataclass, field +from pathlib import Path +from typing import Any, Iterator + +logger = logging.getLogger(__name__) + + +class NotReadable(RuntimeError): + """The file is not in a format this module can read the structure of.""" + + +# --- The ONNX protobuf reader ------------------------------------------------- +# +# Field numbers below come from onnx/onnx.proto. They are quoted in comments at +# each use because a bare integer is unreviewable, and one of them is a trap: +# `AttributeProto.ints` is field 8, while field 7 is `floats`. Reading 7 returns +# an empty kernel shape and no error. + +_WIRE_VARINT, _WIRE_64BIT, _WIRE_LENGTH, _WIRE_32BIT = 0, 1, 2, 5 + + +def _varint(buffer, index: int, end: int) -> tuple[int, int]: + result = shift = 0 + while True: + if index >= end: + raise NotReadable("a protobuf varint runs past the end of the file") + byte = buffer[index] + index += 1 + result |= (byte & 0x7F) << shift + if not byte & 0x80: + return result, index + shift += 7 + if shift > 63: + raise NotReadable("a protobuf varint is longer than 64 bits") + + +def _fields(buffer, start: int, end: int) -> Iterator[tuple[int, int, int, int, int]]: + """Walk one protobuf message, yielding `(number, wire_type, from, to, value)`. + + Length-delimited payloads are yielded as a byte range and never copied, + which is what keeps a 396 MB artefact off the heap. + """ + index = start + while index < end: + key, index = _varint(buffer, index, end) + number, wire = key >> 3, key & 7 + if number == 0: + raise NotReadable("protobuf field number 0 is not legal") + if wire == _WIRE_VARINT: + value, after = _varint(buffer, index, end) + yield number, wire, index, after, value + index = after + elif wire == _WIRE_64BIT: + yield number, wire, index, index + 8, 0 + index += 8 + elif wire == _WIRE_LENGTH: + length, after = _varint(buffer, index, end) + if after + length > end: + raise NotReadable("a protobuf field runs past its own message") + yield number, wire, after, after + length, 0 + index = after + length + elif wire == _WIRE_32BIT: + yield number, wire, index, index + 4, 0 + index += 4 + else: + raise NotReadable(f"protobuf wire type {wire} is not legal") + + +def _text(buffer, start: int, end: int) -> str: + return bytes(buffer[start:end]).decode("utf-8", "replace") + + +def _signed(value: int) -> int: + """Protobuf stores int64 as an unsigned varint, so negatives arrive huge.""" + return value - (1 << 64) if value >= (1 << 63) else value + + +def _packed(buffer, start: int, end: int) -> list[int]: + values: list[int] = [] + index = start + while index < end: + value, index = _varint(buffer, index, end) + values.append(_signed(value)) + return values + + +@dataclass +class OnnxStructure: + """Everything the framing of an ONNX file says about the model in it.""" + + ir_version: int = 0 + producer_name: str = "" + producer_version: str = "" + opset: int = 0 + graph_name: str = "" + #: `ModelProto.metadata_props`. Exporters write provenance here โ€” Ultralytics + #: writes `author`, `license`, `docs`, `task` and `stride` โ€” and it is the + #: cheapest way to recognise a toolchain that has otherwise been renamed. + #: **Read as evidence, never as authority**; see `_self_declared_refusal`. + metadata: dict[str, str] = field(default_factory=dict) + inputs: list[tuple[str, list[Any]]] = field(default_factory=list) + outputs: list[tuple[str, list[Any]]] = field(default_factory=list) + op_counts: dict[str, int] = field(default_factory=dict) + node_count: int = 0 + #: `(name, dims)` for every initializer. Names are the discriminator that + #: matters most โ€” a module path is a statement about which implementation + #: exported the graph. + initializers: list[tuple[str, list[int]]] = field(default_factory=list) + total_params: int = 0 + #: Kernel shape and strides of the first convolution. For a vision + #: transformer this is the patch size, which differs between every backbone + #: worth telling apart. + first_conv_kernel: list[int] = field(default_factory=list) + first_conv_strides: list[int] = field(default_factory=list) + first_conv_weight_dims: list[int] = field(default_factory=list) + #: Name of the first convolution's weight tensor. Kept because `GraphProto` + #: lists every node (field 1) before every initializer (field 5), so the + #: tensor a node refers to has not been read yet when the node is. + _first_conv_weight_name: str = "" + + def initializer(self, name: str) -> list[int] | None: + """Dimensions of one initializer, matched on the tail of its path. + + The tail rather than the whole name, because `scripts/export_embedding.py` + wraps the backbone twice and every path arrives prefixed `inner.inner.`. + A fingerprint written against the wrapper would break the day the + wrapper is renamed, which would be a change to this repository's code + and not to the model's identity. + """ + for path, dims in self.initializers: + if path == name or path.endswith("." + name): + return dims + return None + + def initializers_matching(self, pattern: str) -> int: + expression = re.compile(pattern) + return sum(1 for name, _ in self.initializers if expression.search(name)) + + +def scan_onnx(path: Path | str) -> OnnxStructure: + """Read an ONNX file's structure without loading its weights.""" + path = Path(path) + if path.stat().st_size == 0: + # `mmap` raises ValueError on a zero-length file, and a governance check + # that crashes on an empty artefact is a governance check that takes the + # service down instead of reporting one bad model. + raise NotReadable(f"{path.name} is empty.") + with path.open("rb") as handle: + if handle.read(2) == b"PK": + # Named specifically rather than left to a protobuf stack trace. + # "Wire format was corrupt" is what every parser says here, and it + # sends a reader looking for a damaged download instead of at the + # `.pt` sitting where an `.onnx` was expected. + raise NotReadable( + f"{path.name} begins with the ZIP magic 'PK', so it is a torch " + f"checkpoint or another archive, not an ONNX protobuf." + ) + handle.seek(0) + with mmap.mmap(handle.fileno(), 0, access=mmap.ACCESS_READ) as buffer: + return _scan_model(buffer) + + +def _scan_model(buffer) -> OnnxStructure: + structure = OnnxStructure() + saw_ir_version = False + for number, wire, start, end, value in _fields(buffer, 0, len(buffer)): + if number == 1 and wire == _WIRE_VARINT: # ir_version + structure.ir_version = _signed(value) + saw_ir_version = True + elif number == 2 and wire == _WIRE_LENGTH: # producer_name + structure.producer_name = _text(buffer, start, end) + elif number == 3 and wire == _WIRE_LENGTH: # producer_version + structure.producer_version = _text(buffer, start, end) + elif number == 8 and wire == _WIRE_LENGTH: # opset_import + for f2, w2, s2, e2, v2 in _fields(buffer, start, end): + if f2 == 2 and w2 == _WIRE_VARINT: # OperatorSetId.version + structure.opset = max(structure.opset, _signed(v2)) + elif number == 14 and wire == _WIRE_LENGTH: # metadata_props + key = value_text = "" + for f2, w2, s2, e2, _v2 in _fields(buffer, start, end): + if f2 == 1 and w2 == _WIRE_LENGTH: # StringStringEntry.key + key = _text(buffer, s2, e2) + elif f2 == 2 and w2 == _WIRE_LENGTH: # .value + value_text = _text(buffer, s2, e2) + if key: + # Truncated because `names` on a COCO export is a 3 KB dict and + # this is going into a log line and a ledger entry. + structure.metadata[key] = value_text[:200] + elif number == 7 and wire == _WIRE_LENGTH: # graph + _scan_graph(buffer, start, end, structure) + if structure._first_conv_weight_name: + structure.first_conv_weight_dims = ( + structure.initializer(structure._first_conv_weight_name) or [] + ) + if not saw_ir_version: + raise NotReadable( + "No ModelProto.ir_version field was found, so this is not an ONNX " + "model however well it parsed as protobuf." + ) + return structure + + +def _scan_graph(buffer, start: int, end: int, structure: OnnxStructure) -> None: + for number, wire, from_, to, _ in _fields(buffer, start, end): + if wire != _WIRE_LENGTH: + continue + if number == 1: # GraphProto.node + structure.node_count += 1 + _scan_node(buffer, from_, to, structure) + elif number == 2: # GraphProto.name + structure.graph_name = _text(buffer, from_, to) + elif number == 5: # GraphProto.initializer + _scan_initializer(buffer, from_, to, structure) + elif number == 11: # GraphProto.input + structure.inputs.append(_scan_value_info(buffer, from_, to)) + elif number == 12: # GraphProto.output + structure.outputs.append(_scan_value_info(buffer, from_, to)) + + +def _scan_node(buffer, start: int, end: int, structure: OnnxStructure) -> None: + op_type = "" + inputs: list[str] = [] + kernel: list[int] = [] + strides: list[int] = [] + for number, wire, from_, to, _ in _fields(buffer, start, end): + if wire != _WIRE_LENGTH: + continue + if number == 1: # NodeProto.input + inputs.append(_text(buffer, from_, to)) + elif number == 4: # NodeProto.op_type + op_type = _text(buffer, from_, to) + elif number == 5: # NodeProto.attribute + name, values = _scan_attribute(buffer, from_, to) + if name == "kernel_shape": + kernel = values + elif name == "strides": + strides = values + if not op_type: + return + structure.op_counts[op_type] = structure.op_counts.get(op_type, 0) + 1 + if op_type == "Conv" and not structure.first_conv_kernel: + structure.first_conv_kernel = kernel + structure.first_conv_strides = strides + if len(inputs) > 1: + structure._first_conv_weight_name = inputs[1] + + +def _scan_attribute(buffer, start: int, end: int) -> tuple[str, list[int]]: + name = "" + values: list[int] = [] + for number, wire, from_, to, value in _fields(buffer, start, end): + if number == 1 and wire == _WIRE_LENGTH: # AttributeProto.name + name = _text(buffer, from_, to) + # Field 8 is `ints`. Field 7 is `floats`, and reading it here returns an + # empty kernel shape with no error at all. + elif number == 8 and wire == _WIRE_LENGTH: + values.extend(_packed(buffer, from_, to)) + elif number == 8 and wire == _WIRE_VARINT: + values.append(_signed(value)) + return name, values + + +def _scan_initializer(buffer, start: int, end: int, structure: OnnxStructure) -> None: + dims: list[int] = [] + name = "" + for number, wire, from_, to, value in _fields(buffer, start, end): + if number == 1 and wire == _WIRE_VARINT: # TensorProto.dims + dims.append(_signed(value)) + elif number == 1 and wire == _WIRE_LENGTH: + dims.extend(_packed(buffer, from_, to)) + elif number == 8 and wire == _WIRE_LENGTH: # TensorProto.name + name = _text(buffer, from_, to) + # Field 9 is `raw_data`. `_fields` has already skipped past it without + # reading a byte, which is the whole reason this is fast. + structure.initializers.append((name, dims)) + count = 1 + for dimension in dims: + count *= dimension + structure.total_params += count + + +def _scan_value_info(buffer, start: int, end: int) -> tuple[str, list[Any]]: + name = "" + dims: list[Any] = [] + for number, wire, from_, to, _ in _fields(buffer, start, end): + if number == 1 and wire == _WIRE_LENGTH: # ValueInfoProto.name + name = _text(buffer, from_, to) + elif number == 2 and wire == _WIRE_LENGTH: # ValueInfoProto.type + for f2, w2, s2, e2, _v in _fields(buffer, from_, to): + if f2 != 1 or w2 != _WIRE_LENGTH: # TypeProto.tensor_type + continue + for f3, w3, s3, e3, _v3 in _fields(buffer, s2, e2): + if f3 != 2 or w3 != _WIRE_LENGTH: # Tensor.shape + continue + for f4, w4, s4, e4, _v4 in _fields(buffer, s3, e3): + if f4 != 1 or w4 != _WIRE_LENGTH: # Shape.dim + continue + dims.append(_scan_dimension(buffer, s4, e4)) + return name, dims + + +def _scan_dimension(buffer, start: int, end: int) -> Any: + for number, wire, from_, to, value in _fields(buffer, start, end): + if number == 1 and wire == _WIRE_VARINT: # dim_value + return _signed(value) + if number == 2 and wire == _WIRE_LENGTH: # dim_param + return _text(buffer, from_, to) + return "?" + + +# --- The torch checkpoint reader ---------------------------------------------- + +#: How many pickle opcodes to decode before giving up on finding the header. +#: Ultralytics writes its metadata dict first, so the interesting keys arrive +#: inside the first couple of dozen opcodes; the cap exists so a hostile file +#: cannot turn a governance check into a long walk. +_PICKLE_OPCODE_LIMIT = 4000 + + +@dataclass +class TorchStructure: + """What a `.pt` says about itself, read without executing its pickle.""" + + #: Module paths named by `GLOBAL` and `STACK_GLOBAL` opcodes. This is the + #: identity: a checkpoint that reconstructs `ultralytics.nn.tasks. + #: DetectionModel` needs Ultralytics installed to load, whatever it is called. + globals: list[str] = field(default_factory=list) + #: Top-level string keys paired with the string that follows them. Enough + #: for `license`, `version`, `date` and `docs`, and no more. + header: dict[str, str] = field(default_factory=dict) + + +def scan_torch(path: Path | str) -> TorchStructure: + """Read a torch checkpoint's pickle header without unpickling it. + + `pickletools.genops` decodes the opcode stream and yields it. It builds no + objects and imports no modules, so reading an artefact to find out whether + it is Ultralytics does not run Ultralytics โ€” which matters, because the + reason for asking is that this file might not be what its card says. + """ + path = Path(path) + try: + archive = zipfile.ZipFile(path) + except zipfile.BadZipFile as exc: + raise NotReadable(f"{path.name} is not a zip archive: {exc}") from exc + + with archive: + members = [n for n in archive.namelist() if n.endswith("data.pkl")] + if not members: + raise NotReadable( + f"{path.name} is a zip archive with no `data.pkl` member, so it " + f"is not a torch checkpoint." + ) + payload = archive.read(sorted(members, key=len)[0]) + + structure = TorchStructure() + strings: list[str] = [] + try: + for index, (opcode, argument, _position) in enumerate( + pickletools.genops(payload) + ): + if index > _PICKLE_OPCODE_LIMIT: + break + if opcode.name == "GLOBAL" and isinstance(argument, str): + structure.globals.append(argument.replace(" ", ".")) + elif isinstance(argument, str): + strings.append(argument) + except Exception as exc: # pickletools raises bare ValueError on junk + raise NotReadable(f"{path.name} holds an unreadable pickle: {exc}") from exc + + # `STACK_GLOBAL` takes its module and name off the stack rather than as an + # argument, so protocol-4 checkpoints leave the pair among the strings. + for index in range(len(strings) - 1): + if strings[index] in _TORCH_HEADER_KEYS: + structure.header.setdefault(strings[index], strings[index + 1]) + return structure + + +#: The keys worth reading out of a checkpoint header. Not a general reader โ€” +#: everything else in a `.pt` is weights. +_TORCH_HEADER_KEYS = frozenset({"license", "version", "date", "docs", "author"}) + + +# --- The fingerprints themselves ---------------------------------------------- + + +@dataclass(frozen=True) +class Fingerprint: + """The structure one runtime's artefacts always have. + + `must` holds predicates over the parsed structure. Every one has to hold, + and each carries the sentence that goes in a report when it does not โ€” so a + mismatch says *which* property failed rather than "fingerprint failed", + which is the difference between a diagnosis and an alarm. + """ + + runtime: str + kind: str # "onnx" or "torch" + must: tuple[tuple[str, Any], ...] + + def match(self, structure: Any) -> tuple[bool, list[str]]: + failures = [ + description + for description, predicate in self.must + if not _safe(predicate, structure) + ] + return (not failures), failures + + +def _safe(predicate, structure) -> bool: + """A predicate that raises is a predicate that did not hold. + + A fingerprint runs against files that are not what they claim to be, so + indexing off the end of a two-element list is an expected outcome rather + than a bug. + """ + try: + return bool(predicate(structure)) + except Exception: + return False + + +def _naming(structure: OnnxStructure, pattern: str, least: int) -> bool: + return structure.initializers_matching(pattern) >= least + + +#: DINOv3 ViT-S/16, as exported by `scripts/export_embedding.py`. +#: +#: `reg_token` is the strongest single property here โ€” register tokens are the +#: architectural change DINOv3 introduced over DINOv2, and no DINOv2 export has +#: one. RoPE is the second: DINOv3 replaces the learned position embedding with +#: rotary embeddings, so `Sin` and `Cos` appear in the graph and `pos_embed` +#: does not exist as an initializer at all. +DINOV3_ONNX = Fingerprint( + runtime="dinov3-onnx", + kind="onnx", + must=( + ("the patch convolution is not 16 pixels", + lambda s: s.first_conv_kernel == [16, 16] and s.first_conv_strides == [16, 16]), + ("there is no `reg_token` of shape [1, 4, 384]", + lambda s: s.initializer("reg_token") == [1, 4, 384]), + ("there is no `cls_token` of shape [1, 1, 384]", + lambda s: s.initializer("cls_token") == [1, 1, 384]), + ("there are not 24 LayerScale `gamma_1`/`gamma_2` vectors", + lambda s: _naming(s, r"\.gamma_[12]$", 24)), + ("the graph has no rotary position embedding (`Sin` and `Cos`)", + lambda s: s.op_counts.get("Sin", 0) >= 1 and s.op_counts.get("Cos", 0) >= 1), + ("the initializers are not named in timm's `blocks.N.` convention", + lambda s: _naming(s, r"blocks\.\d+\.", 24)), + ("it does not take one `pixel_values` tensor", + lambda s: [n for n, _ in s.inputs] == ["pixel_values"]), + ), +) + +#: DINOv2-small, as exported through `transformers`. +#: +#: The previous audit reported this one as identifiable by "Hugging Face +#: `Dinov2Model` naming". The literal string `Dinov2Model` does not occur +#: anywhere in the file โ€” a byte search finds no `Dinov2`, no `transformers` and +#: no `facebook`. What is genuinely there is the *module-path convention* of +#: `transformers.models.dinov2`: `embeddings.patch_embeddings.projection.weight` +#: and `encoder.layer.N.attention.attention.query.bias`. That is a reliable +#: discriminator and it is a different fact, so it is written down as one. +DINOV2_ONNX = Fingerprint( + runtime="dinov2-onnx", + kind="onnx", + must=( + ("the patch convolution is not 14 pixels", + lambda s: s.first_conv_kernel == [14, 14] and s.first_conv_strides == [14, 14]), + ("there is no `cls_token` of shape [1, 1, 384]", + lambda s: s.initializer("cls_token") == [1, 1, 384]), + ("it carries a `reg_token`, which DINOv2 does not have", + lambda s: s.initializer("reg_token") is None), + ("the initializers are not named in the `encoder.layer.N.` convention " + "that transformers' Dinov2Model produces", + lambda s: _naming(s, r"encoder\.layer\.\d+\.", 24)), + ("it has rotary position embedding nodes, which DINOv2 does not use", + lambda s: s.op_counts.get("Sin", 0) == 0 and s.op_counts.get("Cos", 0) == 0), + ("it does not take one `pixel_values` tensor", + lambda s: [n for n, _ in s.inputs] == ["pixel_values"]), + ), +) + +#: YOLOX, at any of the four published sizes. +#: +#: Deliberately size-agnostic. All four share one licence, which is the question +#: this module exists to answer, and `sha256` on the card already pins which one +#: is installed. A fingerprint that also distinguished `-s` from `-m` would fail +#: for a reason the checksum had already caught, with a worse message. +YOLOX_ONNX = Fingerprint( + runtime="yolox-onnx", + kind="onnx", + must=( + ("it does not take one 640-pixel `images` tensor", + lambda s: [n for n, _ in s.inputs] == ["images"] + and list(s.inputs[0][1])[1:] == [3, 640, 640]), + ("it does not emit YOLOX's [1, 8400, 85] decode grid", + lambda s: list(s.outputs[0][1]) == [1, 8400, 85]), + ("the stem convolution does not read a 12-channel focus slice", + lambda s: len(s.first_conv_weight_dims) == 4 + and s.first_conv_weight_dims[1] == 12), + ("it has fewer than 80 convolutions, so it is not a YOLOX backbone", + lambda s: s.op_counts.get("Conv", 0) >= 80), + ("the initializers are not YOLOX's bare numeric names", + lambda s: _naming(s, r"^\d+$", 100)), + ), +) + +#: Ultralytics YOLO as a torch checkpoint. +#: +#: This is ADR 0017's exploit read from the bytes. A card claiming +#: `runtime: dinov3-onnx` over `yolo11m.pt` passes every other check in this +#: service; it fails here at the first predicate, because an Ultralytics +#: checkpoint reconstructs `ultralytics.nn.tasks.DetectionModel` and cannot be +#: loaded by anything else. +ULTRALYTICS_TORCH = Fingerprint( + runtime="ultralytics", + kind="torch", + must=( + ("its pickle names no `ultralytics.` module, so nothing in it needs the " + "Ultralytics loader", + lambda s: any(g.startswith("ultralytics.") for g in s.globals)), + ), +) + +#: **Ultralytics after `yolo export format=onnx`, which is how the first version +#: of this module was defeated.** +#: +#: A watchdog exported `yolo11m.pt` to ONNX, wrote a card calling it +#: `yolox-onnx`, and served AGPL weights as `cattle_detection` under +#: `ANIMAP_LICENCE_POLICY=enforce` *and* `ANIMAP_ARTEFACT_IDENTITY=refuse`, with +#: an empty ledger. Nothing here recognised it: the torch fingerprint only reads +#: pickles, and an unidentified artefact loads on its card's word by design. A +#: ledger that reports success while the thing it audits walks past is worse +#: than no ledger. +#: +#: The predicates are structural rather than metadata-based on purpose. An +#: Ultralytics export does declare itself in `metadata_props` โ€” `author: +#: Ultralytics`, `license: AGPL-3.0 License (โ€ฆ)` โ€” and `_self_declared_refusal` +#: reads that too, but a stripped metadata block must not be a way through. +#: These survive stripping: +#: +#: | | YOLOX-m | YOLO11m exported | +#: |---|---|---| +#: | output | `output` `[1, 8400, 85]` | `output0` `[1, 84, 8400]` | +#: | initializer names | 224 bare numerals | 225 `model.N.โ€ฆ` | +#: | `Split` / `Softmax` | 0 / 0 | 10 / 2 (DFL head) | +#: | stem convolution | `[48, 12, 3, 3]`, focus slice | `[64, 3, 3, 3]` | +#: +#: Both take one `images` `[1, 3, 640, 640]` tensor, which is why input shape +#: alone was never going to be enough. +ULTRALYTICS_ONNX = Fingerprint( + runtime="ultralytics", + kind="onnx", + must=( + ("it does not take one 640-pixel `images` tensor", + lambda s: [n for n, _ in s.inputs] == ["images"] + and list(s.inputs[0][1])[1:] == [3, 640, 640]), + ("its output is not Ultralytics' channels-first [1, 84, N] decode head", + lambda s: len(list(s.outputs[0][1])) == 3 + and list(s.outputs[0][1])[0] == 1 + and 5 <= list(s.outputs[0][1])[1] <= 200 + and list(s.outputs[0][1])[2] > list(s.outputs[0][1])[1]), + ("the initializers are not named in Ultralytics' `model.N.` convention", + lambda s: _naming(s, r"^model\.\d+\.", 50)), + ("it has no distribution-focal-loss head, which every YOLOv8-and-later " + "export carries", + lambda s: s.op_counts.get("Split", 0) >= 1 + and s.op_counts.get("Softmax", 0) >= 1), + ), +) + +FINGERPRINTS: tuple[Fingerprint, ...] = ( + DINOV3_ONNX, DINOV2_ONNX, ULTRALYTICS_ONNX, YOLOX_ONNX, ULTRALYTICS_TORCH, +) + + +def fingerprint_for(runtime: str, kind: str) -> Fingerprint | None: + """The fingerprint that decides whether a file *is* this runtime. + + Keyed on both, because `ultralytics` has two โ€” a torch one and an ONNX one โ€” + and asking whether a `.pt` matches the ONNX predicates would answer no for a + reason that says nothing about the file. + """ + for fingerprint in FINGERPRINTS: + if fingerprint.runtime == runtime and fingerprint.kind == kind: + return fingerprint + return None + +#: Strings in an artefact's own metadata that name a runtime this service will +#: not serve from. Matched case-insensitively against every metadata value. +#: +#: **This may only make a verdict stricter, never laxer**, and that asymmetry is +#: the whole design. `licences.py` records the trap that proves it: +#: `BVRA/MegaDescriptor-L-384/config.json` declares `"license": "mit"` as +#: inherited timm boilerplate over CC-BY-NC-4.0 weights, so a loader that +#: believed an artefact's own permissive claim would ship a non-commercial +#: model. Believing a *restrictive* self-declaration has no such failure mode: +#: the worst case is refusing something that was fine, which is a conversation +#: rather than a breach. +_SELF_DECLARED_RUNTIMES: tuple[tuple[str, str], ...] = ( + ("ultralytics", "ultralytics"), + ("agpl", "ultralytics"), +) + + +# --- The finding -------------------------------------------------------------- + + +@dataclass(frozen=True) +class Identification: + """What the bytes turned out to be, and how confidently. + + `runtime` is `None` for both *"nobody has written a fingerprint for this"* + and *"this file could not be read at all"*, and `detail` is what tells them + apart. They are the same outcome for a caller โ€” identity unverified, load + on the card's word โ€” and different facts for a person reading the log. + """ + + #: The runtime whose fingerprint matched, or `None` if none did. + runtime: str | None + #: True when the file was read and no fingerprint matched. False when the + #: file could not be read as either ONNX or a torch archive. + readable: bool + detail: str + #: What the file was read as โ€” `"onnx"`, `"torch"`, or `""` when neither + #: reader could open it. A caller needs this to ask the inverse question: + #: *the card claims runtime X; was there a fingerprint for X this file could + #: have been tried against?* Only fingerprints of the same kind are ever + #: tried, so without it a caller cannot tell *"the card's claim was + #: disproved"* from *"the card's claim was never testable"*. + kind: str = "" + #: Facts worth keeping in the ledger whether or not anything matched. + evidence: dict[str, Any] = field(default_factory=dict) + #: Why each fingerprint that was tried did not match, keyed by runtime. + #: Present so a mismatch report can say what the file looked like instead. + #: + #: **These sentences are the predicates an attacker would have to break**, + #: and returning them is only defensible because nothing serialises them: + #: `app/main.py` puts no part of an `Identification` on `/health` or + #: `/capabilities`, and `tests/test_fingerprints.py` asserts that so a + #: future response model cannot quietly start carrying one. Anyone who can + #: read this field can already read this file โ€” writing a model card means + #: writing into the models tree โ€” so the leak that matters is the one across + #: an HTTP boundary, and that is the one under test. + near_misses: dict[str, list[str]] = field(default_factory=dict) + + @property + def identified(self) -> bool: + return self.runtime is not None + + def refutes(self, declared: str | None) -> list[str]: + """Why these bytes are not `declared`, or an empty list. + + **The inverse question, and nothing was asking it.** `identify` walks + the fingerprints looking for one that matches and answers `None` when + none does. That is the right answer for a toolchain nobody has + fingerprinted, and the wrong one for a card naming a runtime that *has* + a fingerprint. + + A watchdog renamed 225 initializers in an Ultralytics ONNX export from + `model.N.` to `m.N.`, stripped `metadata_props`, and served it as + `cattle_detection` on a card reading `runtime: yolox-onnx`, under + `ANIMAP_LICENCE_POLICY=enforce` and `ANIMAP_ARTEFACT_IDENTITY=refuse`, + with an empty ledger. The rename left `onnxruntime`'s output + bit-identical, so nothing about the model changed โ€” it only stopped + matching `ULTRALYTICS_ONNX`. Everything downstream then read the card, + because an unidentified artefact loads on its card's word by design. + + But `YOLOX_ONNX` exists, the file was read as ONNX, and it fails three + of that fingerprint's five properties. That is not an unidentified + artefact; it is a claim this module can disprove and did. The two must + not produce the same outcome. + + Empty when the claim was never testable โ€” no `declared`, an unreadable + file, or no fingerprint for `declared` of the kind this file was read + as โ€” because *"not checked"* has to stay distinguishable from *"checked + and false"*. That distinction is the whole reason an unfingerprinted + artefact still loads. + """ + if not declared or not self.readable: + return [] + if fingerprint_for(declared, self.kind) is None: + return [] + return list(self.near_misses.get(declared, [])) + + +def identify(path: Path | str) -> Identification: + """Read a model artefact and say which runtime's weights it holds.""" + path = Path(path) + structure: Any + kind: str + try: + structure = scan_onnx(path) + kind = "onnx" + except NotReadable as onnx_failure: + try: + structure = scan_torch(path) + kind = "torch" + except NotReadable as torch_failure: + return Identification( + runtime=None, readable=False, + detail=( + f"{path.name} is neither an ONNX graph ({onnx_failure}) nor " + f"a torch checkpoint ({torch_failure}), so nothing here can " + f"say what it is." + ), + ) + + evidence = _evidence(structure, kind) + near_misses: dict[str, list[str]] = {} + for fingerprint in FINGERPRINTS: + if fingerprint.kind != kind: + continue + matched, failures = fingerprint.match(structure) + if matched: + return Identification( + runtime=fingerprint.runtime, readable=True, kind=kind, + detail=( + f"{path.name} matches every structural property of " + f"{fingerprint.runtime}." + ), + evidence=evidence, + ) + near_misses[fingerprint.runtime] = failures + + declared = _self_declared_refusal(structure, kind) + if declared is not None: + runtime, where = declared + return Identification( + runtime=runtime, readable=True, kind=kind, + detail=( + f"{path.name} matches no structural fingerprint, but it names " + f"itself: {where}. A self-declaration is not authority about " + f"what an artefact may be used for โ€” a permissive one is " + f"exactly the MegaDescriptor trap โ€” but a restrictive one can " + f"only ever make this stricter, so it is believed." + ), + evidence=evidence, near_misses=near_misses, + ) + + return Identification( + runtime=None, readable=True, kind=kind, + detail=( + f"{path.name} was read as {kind} and matches no fingerprint in " + f"app/adapters/fingerprints.py. Its identity rests on its card, " + f"which is where it rested before this check existed." + ), + evidence=evidence, near_misses=near_misses, + ) + + +def _self_declared_refusal( + structure: Any, kind: str +) -> tuple[str, str] | None: + """A runtime the artefact names in its own metadata, if it is a refused one. + + The backstop behind the structural fingerprints, for the case they were + written to handle badly: a toolchain nobody has fingerprinted yet whose + exporter is honest about where the weights came from. Most exporters are โ€” + Ultralytics writes `author` and `license` into `metadata_props` โ€” and a + check that reads them costs nothing. + + Returns `None` for a permissive self-declaration, always. That direction is + where the trap lives. + """ + declared = ( + structure.metadata if kind == "onnx" else structure.header + ) + for key, value in declared.items(): + haystack = f"{key} {value}".lower() + for needle, runtime in _SELF_DECLARED_RUNTIMES: + if needle in haystack: + return runtime, f"{key}={value!r}" + return None + + +def _evidence(structure: Any, kind: str) -> dict[str, Any]: + """The handful of facts worth keeping about any artefact, matched or not.""" + if kind == "torch": + return { + "format": "torch", + "modules": sorted({g.rsplit(".", 1)[0] for g in structure.globals})[:8], + # Recorded, and deliberately not acted on. `licences.py` documents + # the MegaDescriptor trap: `config.json` there declares `"license": + # "mit"` as inherited timm boilerplate over CC-BY-NC-4.0 weights. An + # artefact's own claim about its terms is evidence about what + # upstream wrote, never authority about what Animap may serve. + "self_declared": dict(structure.header), + } + return { + "format": "onnx", + "producer": f"{structure.producer_name} {structure.producer_version}".strip(), + # Recorded, and read only in the strict direction. See + # `_SELF_DECLARED_RUNTIMES`. + "self_declared": dict(structure.metadata), + "opset": structure.opset, + "inputs": [f"{n}{d}" for n, d in structure.inputs], + "outputs": [f"{n}{d}" for n, d in structure.outputs], + "nodes": structure.node_count, + "initializers": len(structure.initializers), + "parameters": structure.total_params, + "first_conv_kernel": structure.first_conv_kernel, + } diff --git a/app/adapters/geometry/__init__.py b/app/adapters/geometry/__init__.py new file mode 100644 index 0000000000000000000000000000000000000000..2b671f8ed8ea663abcb056efa1574a1e03f85e36 --- /dev/null +++ b/app/adapters/geometry/__init__.py @@ -0,0 +1,128 @@ +"""Geometry to weight: ยง22's pipeline from the mask onward, with no weights. + +The models ยง22 names โ€” SAM for the mask, ARCore and VGGT for metric scale โ€” are +not here. What is here is everything downstream of them: a mask becomes body +dimensions (`body`), a scale turns those into centimetres, and a published +equation turns centimetres into a band (`equations`, `weight`). That split is +deliberate. The neural half needs a GPU host and a licence review; this half runs +on a phone, is exercised by unit tests, and is where most of the error lives. + +**Nothing in this package invents a scale.** `BodyMeasurements` requires a +`scale_relative_error` with no default, and `app.adapters.signal.geometry` +already refuses to return a scale from a frame with no reference in it. ยง22 is +explicit that "a photograph has no scale" is true of an arbitrary photograph; the +answer is to make the photograph non-arbitrary, not to guess. +""" + +from __future__ import annotations + +from app.adapters.base import ( + Adapter, + AdapterSpec, + Availability, + Modality, + Placement, + Task, +) +from app.adapters.geometry.body import ( + MaskUnusable, + PixelProfile, + profile, + rear_width_px, +) +from app.adapters.geometry.equations import ( + BY_ID, + EQUATIONS, + LESOSKY_SHZ, + ODADI_BORAN, + SCHAEFFER, + OutOfRange, + WeightEquation, + ellipse_perimeter_cm, +) +from app.adapters.geometry.weight import ( + BodyMeasurements, + EquationResult, + NotEnoughGeometry, + WeightBand, + estimate, +) + +BODY_GEOMETRY_SPEC = AdapterSpec( + adapter_id="body-geometry-weight", + runtime="opencv-numpy", + tasks=(Task.MEASURE,), + modalities=(Modality.IMAGE, Modality.VIDEO), + directive_role=( + "ยง22 cattle weight โ€” the second half of the pipeline: body length, " + "height, girth proxy, classical livestock weight equation, broad " + "estimate. ยง4 prefers deterministic geometry to a network wherever it " + "wins, and this step is arithmetic." + ), + requires_artefact=False, + placement=Placement.ON_DEVICE, + placement_reason=( + "Column sums over a mask and three closed-form equations. The expensive " + "parts of ยง22 are the mask and the metric depth, both of which are " + "somewhere else; this runs in microseconds wherever the mask is." + ), + notes=( + "**The equations are published and unfitted, which is the point and " + "also the limit.** None of the three was derived on Nigerian cattle, " + "and the closest โ€” Lesosky et al. on east African shorthorn zebu โ€” " + "publishes a ยฑ20% interval on its own animals before any vision error " + "is added. `experiments/cattle_weight` measures all three on 623 real " + "cattle: 10.6โ€“11.9% MAPE with a TAPE measurement, and ยฑ25% needed to " + "cover nine animals in ten.\n\n" + "**`body` has never run on a real animal mask.** Everything that turns " + "a mask into dimensions โ€” the column profile, the trunk span, the chest " + "window, the rear width โ€” is exercised only by unit tests over " + "rectangles in `tests/test_geometry_weight.py`. No segmented cow has " + "passed through it, because the one dataset carrying cattle " + "photographed from the side and the rear with the measurements beside " + "them serves zero files. Do not quote a dimension from this until " + "somebody has run it on a photograph." + ), +) + + +class BodyGeometryAdapter(Adapter): + """ยง22 from the mask onward. Always available; never guesses a scale.""" + + spec = BODY_GEOMETRY_SPEC + + def availability(self) -> Availability: + # NumPy is a production dependency and the equations are arithmetic, so + # there is nothing that can be absent. The honesty property lives in + # `weight.estimate`, which refuses rather than widening when no equation + # applies, and in `BodyMeasurements`, which has no default scale error. + return Availability(True) + + def load(self) -> "BodyGeometryAdapter": + return self + + def measure(self, measurements: BodyMeasurements, **kwargs) -> WeightBand: + return estimate(measurements, **kwargs) + + +__all__ = [ + "BODY_GEOMETRY_SPEC", + "BY_ID", + "EQUATIONS", + "LESOSKY_SHZ", + "ODADI_BORAN", + "SCHAEFFER", + "BodyGeometryAdapter", + "BodyMeasurements", + "EquationResult", + "MaskUnusable", + "NotEnoughGeometry", + "OutOfRange", + "PixelProfile", + "WeightBand", + "WeightEquation", + "ellipse_perimeter_cm", + "estimate", + "profile", + "rear_width_px", +] diff --git a/app/adapters/geometry/body.py b/app/adapters/geometry/body.py new file mode 100644 index 0000000000000000000000000000000000000000..21ff6f98f67b76bcbc2e90d7b9c3610353f528d1 --- /dev/null +++ b/app/adapters/geometry/body.py @@ -0,0 +1,196 @@ +"""From an animal mask to body dimensions, and the places that step is weak. + +Directive ยง22's pipeline goes `cattle masks โ†’ ARCore + VGGT geometry โ†’ body +length โ†’ height โ†’ girth/volume proxy`. This module is the arithmetic between a +mask and those three numbers. It is deterministic โ€” contours and column sums, no +weights โ€” which ยง4 says to prefer wherever it wins. + +## Three known weaknesses, stated here rather than discovered later + +**The silhouette's length is not the equation's length.** Schaeffer's rule wants +the distance from the point of the shoulder to the pin bone. A side-view mask +gives nose to tail, which is longer by however far the head sticks out, and the +head is the most mobile part of the animal. `trunk_span_px` exists to cut the +head and tail off by profile shape, and it is a heuristic; `experiments/ +cattle_weight` measures what it costs against tape-measured oblique body length. + +**The girth station is not visible.** Heart girth is measured immediately behind +the foreleg. A silhouette shows no foreleg boundary, so this module takes the +deepest part of the trunk inside a stated window and calls it the chest. On +cattle the deepest point is near the girth station, which is why the proxy is +defensible; it is not the same thing, which is why it is called a proxy. + +**A circumference is not in the picture at all.** Depth comes from a side view +and width from a rear view, and everything between them is an assumption about +cross-sectional shape. `equations.ellipse_perimeter_cm` is that assumption. + +None of these are reasons the capability is impossible. They are the reasons ยง22 +asks for a guided sweep and a rear view rather than a snapshot, and they are the +error terms an experiment has to size. +""" + +from __future__ import annotations + +from dataclasses import dataclass + +import numpy as np + +#: Fraction of the body length, measured from the front, inside which the chest +#: depth is sought. Cattle are deepest through the barrel just behind the +#: foreleg; the window keeps the search off the hindquarters, which on a +#: well-conditioned animal can be as deep and are not the girth station. +CHEST_WINDOW = (0.20, 0.55) + +#: A column holding fewer than this fraction of the mask's peak height is +#: treated as head, neck, tail or leg rather than trunk. Chosen so that a neck โ€” +#: roughly a third of the trunk's depth on cattle โ€” falls outside and the trunk +#: does not. It is a threshold on a silhouette, so it is a heuristic, and +#: `trunk_span_px` reports what it cut. +TRUNK_THRESHOLD = 0.55 + + +class MaskUnusable(ValueError): + """The mask does not describe a whole animal seen side-on. + + Raised rather than measured around. ยง22's `reject_if` already lists + `animal_heavily_occluded` and `wrong_pose`; a body length taken off a mask + with the hindquarters cut out of frame is a number that looks fine and is + wrong by 30%. + """ + + +@dataclass(frozen=True) +class PixelProfile: + """The silhouette reduced to what the measurements need. + + `column_height` is the mask's vertical extent in every column, which is the + only thing any of the derived numbers reads. Keeping it on the object means + an experiment can plot the profile that produced a bad measurement instead of + guessing at it. + """ + + column_height: np.ndarray + #: Inclusive column indices of the whole animal, head and tail included. + full_span: tuple[int, int] + #: Inclusive column indices of the trunk, after the profile threshold. + trunk_span: tuple[int, int] + #: Column index of the deepest trunk column inside `CHEST_WINDOW`. + chest_column: int + #: Fraction of the mask's pixels that fell outside `trunk_span`. Large means + #: a long neck, an outstretched head โ€” or a segmentation that included a + #: second animal, which is the failure worth catching. + trimmed_fraction: float + + @property + def full_length_px(self) -> float: + return float(self.full_span[1] - self.full_span[0] + 1) + + @property + def trunk_length_px(self) -> float: + return float(self.trunk_span[1] - self.trunk_span[0] + 1) + + @property + def chest_depth_px(self) -> float: + return float(self.column_height[self.chest_column]) + + @property + def withers_height_px(self) -> float: + """Peak trunk depth. **Not withers height above ground.** + + A standing animal's withers height is measured from the floor and needs + the ground plane, which a mask alone does not give. This is the top of + the trunk to the bottom of the trunk in the same column โ€” a body depth. + Named on the object as `withers_height_px` would be a lie, so it is not. + """ + low, high = self.trunk_span + return float(self.column_height[low:high + 1].max()) + + +def profile(mask: np.ndarray) -> PixelProfile: + """Reduce a side-on animal mask to a column profile and its landmarks. + + The mask is expected in image orientation โ€” rows are image rows โ€” and the + animal is expected to be roughly horizontal in frame. A cow photographed at + 45ยฐ produces a profile whose peak is not the chest and whose span is not the + body length, and nothing here detects that. ยง22's capture prompt exists + partly for this reason. + """ + binary = np.asarray(mask) > 0 + if binary.ndim != 2: + raise MaskUnusable(f"Expected a 2-D mask, got {binary.ndim} dimensions.") + if not binary.any(): + raise MaskUnusable("The mask is empty; nothing was segmented.") + + occupied_columns = np.flatnonzero(binary.any(axis=0)) + first, last = int(occupied_columns[0]), int(occupied_columns[-1]) + + # Vertical extent rather than pixel count per column. A count would be + # thinned by a hole in the mask โ€” a fence rail across the animal, a patch + # the segmenter lost โ€” and the extent is what a depth means. + rows = np.arange(binary.shape[0])[:, None] + masked_rows = np.where(binary, rows, -1) + top = np.where(binary.any(axis=0), np.where(binary, rows, binary.shape[0]).min(axis=0), 0) + bottom = masked_rows.max(axis=0) + column_height = np.where(binary.any(axis=0), bottom - top + 1, 0).astype(float) + + peak = float(column_height.max()) + if peak <= 0: + raise MaskUnusable("The mask has no vertical extent.") + + trunk_columns = np.flatnonzero(column_height >= TRUNK_THRESHOLD * peak) + if trunk_columns.size == 0: + raise MaskUnusable("No column reaches the trunk threshold.") + + # The largest *contiguous* run, not simply the first and last column over + # threshold. A raised head can clear the threshold on its own and would + # otherwise stretch the trunk span across the neck's gap. + breaks = np.flatnonzero(np.diff(trunk_columns) > 1) + starts = np.concatenate(([0], breaks + 1)) + ends = np.concatenate((breaks, [trunk_columns.size - 1])) + widest = int(np.argmax(ends - starts)) + trunk_low = int(trunk_columns[starts[widest]]) + trunk_high = int(trunk_columns[ends[widest]]) + + span = trunk_high - trunk_low + 1 + window_low = trunk_low + int(CHEST_WINDOW[0] * span) + window_high = trunk_low + max(int(CHEST_WINDOW[1] * span), 1) + window = column_height[window_low:window_high + 1] + if window.size == 0: + raise MaskUnusable("The trunk is too short to contain a chest window.") + chest_column = window_low + int(np.argmax(window)) + + inside = binary[:, trunk_low:trunk_high + 1].sum() + total = binary.sum() + + return PixelProfile( + column_height=column_height, + full_span=(first, last), + trunk_span=(trunk_low, trunk_high), + chest_column=chest_column, + trimmed_fraction=float(1.0 - inside / total) if total else 0.0, + ) + + +def rear_width_px(mask: np.ndarray) -> float: + """Widest horizontal extent of a rear-view mask. + + ยง22's fallback is "side + rear images", and this is the rear half of it: the + width that, with the side view's depth, gives an ellipse to take a girth + from. It returns the widest row rather than the mean because the chest is + the widest part of a rear silhouette below the hips โ€” but a rear view also + contains the hips, which on many animals are wider, and nothing here + separates them. That confusion is a known bias of this proxy and it runs in + the direction of over-estimating girth. + """ + binary = np.asarray(mask) > 0 + if binary.ndim != 2: + raise MaskUnusable(f"Expected a 2-D mask, got {binary.ndim} dimensions.") + if not binary.any(): + raise MaskUnusable("The mask is empty; nothing was segmented.") + columns = np.flatnonzero(binary.any(axis=0)) + left = np.where(binary, np.arange(binary.shape[1])[None, :], binary.shape[1]).min(axis=1) + right = np.where(binary, np.arange(binary.shape[1])[None, :], -1).max(axis=1) + widths = np.where(binary.any(axis=1), right - left + 1, 0) + if not columns.size: + raise MaskUnusable("The mask has no horizontal extent.") + return float(widths.max()) diff --git a/app/adapters/geometry/equations.py b/app/adapters/geometry/equations.py new file mode 100644 index 0000000000000000000000000000000000000000..48421d6a869f26187651dbaafd04e205aafc4eed --- /dev/null +++ b/app/adapters/geometry/equations.py @@ -0,0 +1,268 @@ +"""Published livestock weight equations, used exactly as published. + +Directive ยง22 ends its pipeline at "classical livestock weight equation". This +module is that step, and the constraint on it is the whole directive's premise: +**nothing here is fitted.** Every coefficient below was published by somebody +else, against their own cattle, and is reproduced unchanged. Fitting a curve to +the animals this repository can measure would turn a zero-training capability +into a one-dataset model and would make every accuracy figure a training score. + +That constraint costs something, and the cost is the point. An equation derived +from Kenyan shorthorn zebu applied to Chinese yellow cattle is out of domain, and +the error it makes is real information about how far a published relationship +travels. A fitted curve would hide exactly that. + +## What each equation needs + +Two shapes. `SCHAEFFER` is volumetric โ€” it needs a girth **and** a length, and it +is the one ยง22's pipeline is written for. The other two are girth-only linear or +power laws, which need one measurement and are therefore the ones that survive a +capture where the animal's length could not be resolved. + +## Why the error bands are here and not computed + +`published_relative_error` is what the **authors** reported on **their** animals. +It is a `PRIOR` in the sense `experiments.harness.metrics.UncertaintyBasis` means +it, and it must never be presented as an Animap measurement. Directive ยง37 keeps +those apart, and `estimate()` labels which one it is holding. +""" + +from __future__ import annotations + +import math +from dataclasses import dataclass +from typing import Callable + +#: Conversion constants, spelled out so the derived Schaeffer coefficient below +#: cannot drift from them and cannot be mistaken for a fitted number. +KG_PER_POUND = 0.45359237 +METRES_PER_INCH = 0.0254 + +#: Schaeffer's rule in pounds and inches: W = girthยฒ ร— length / 300. +SCHAEFFER_DIVISOR_LB_IN = 300.0 + +#: The same rule in kilograms and metres, derived rather than quoted: +#: +#: W_kg = KG_PER_POUND ร— (HG_m / m_per_in)ยฒ ร— (BL_m / m_per_in) / 300 +#: = KG_PER_POUND / (m_per_inยณ ร— 300) ร— HG_mยฒ ร— BL_m +#: +#: which comes to 92.27. Worked check, matching the figure the rule is usually +#: quoted with: a 70 in girth and 78 in length give 1,274 lb; 177.8 cm and +#: 198.12 cm give 577.9 kg, and 1,274 lb is 577.9 kg. +SCHAEFFER_COEFFICIENT_KG_M = KG_PER_POUND / (METRES_PER_INCH ** 3 * SCHAEFFER_DIVISOR_LB_IN) + + +class OutOfRange(ValueError): + """A measurement outside the range the equation was published over. + + Raised rather than extrapolated. A power law fitted between 60 cm and 200 cm + of girth returns a number for 10 cm as readily as for 150 cm, and the number + it returns for 10 cm is not a weight. + """ + + +@dataclass(frozen=True) +class WeightEquation: + """One published relationship, with the evidence for it attached. + + Every field except `apply` is bibliography. It is here rather than in a + README because the README is not what runs, and because a coefficient whose + source is three documents away is a coefficient somebody will eventually + "improve". + """ + + id: str + #: What it needs. `("heart_girth_cm",)` or `("heart_girth_cm", "body_length_cm")`. + inputs: tuple[str, ...] + citation: str + #: Verbatim, in the authors' own notation, so a reader can check the code + #: against the paper without opening the paper. + published_form: str + #: The animals it was fitted on. The single most important field here: it is + #: what makes an out-of-domain application visible as one. + population: str + sample_size: int + licence: str + source_url: str + #: Valid girth range in cm, from the paper where the paper states one and + #: from the paper's reported animals where it does not. + girth_range_cm: tuple[float, float] + #: The authors' own accuracy, as a fraction. A PRIOR, never an Animap + #: measurement. `None` where the paper reports no usable figure. + published_relative_error: float | None + published_error_basis: str + apply: Callable[..., float] + + def __call__(self, **measurements: float) -> float: + """Evaluate, or refuse. + + Refuses on a missing input rather than defaulting one, because a + volumetric equation silently handed a default length returns a weight + that varies only with girth and looks nothing like a failure. + """ + missing = [name for name in self.inputs if measurements.get(name) is None] + if missing: + raise OutOfRange( + f"{self.id} needs {', '.join(self.inputs)}; " + f"{', '.join(missing)} was not measured." + ) + girth = measurements["heart_girth_cm"] + low, high = self.girth_range_cm + if not low <= girth <= high: + raise OutOfRange( + f"{self.id} was published over girths of {low:g}โ€“{high:g} cm and " + f"this animal measures {girth:g} cm. Extrapolating a fitted curve " + f"past its own data is how a calf becomes a bull." + ) + return float(self.apply(**{k: measurements[k] for k in self.inputs})) + + +def _schaeffer(heart_girth_cm: float, body_length_cm: float) -> float: + return SCHAEFFER_COEFFICIENT_KG_M * (heart_girth_cm / 100.0) ** 2 * (body_length_cm / 100.0) + + +def _lesosky(heart_girth_cm: float) -> float: + # weight**0.262 = 0.95 + 0.022 ร— girth, inverted. The exponent is small, so + # the inverse power is large (1/0.262 โ‰ˆ 3.82) and the equation is extremely + # sensitive to girth: a 1% girth error becomes roughly 3% of weight at a + # typical adult girth. That sensitivity is a property of the published + # relationship, not of this implementation, and `experiments/cattle_weight` + # measures what it does to a vision-derived girth. + return (0.95 + 0.022 * heart_girth_cm) ** (1.0 / 0.262) + + +def _odadi(heart_girth_cm: float) -> float: + return -265.0 + 3.37 * heart_girth_cm + + +#: Schaeffer's rule. The volumetric one, and the only one here that uses a body +#: length โ€” which is why ยง22's pipeline derives a length at all. +SCHAEFFER = WeightEquation( + id="schaeffer", + inputs=("heart_girth_cm", "body_length_cm"), + citation=( + "Schaeffer's formula, in general agricultural use since the early 20th " + "century and with no single primary citation. Evaluated for Bos indicus " + "in Sarwar et al. (2018), 'Accuracy of estimates for live body weight " + "using Schaeffer's formula in non-descript cattle (Bos indicus), Nili " + "Ravi buffaloes and their calves using linear body measurements'." + ), + published_form="W (lb) = girth (in)ยฒ ร— length (in) รท 300", + population=( + "No stated origin population. Applied across cattle generally, which is " + "both why it is the default here and why it should be expected to be " + "biased on any particular breed." + ), + sample_size=0, + licence="Formula, not a copyrightable work. No licence attaches.", + source_url="https://www.tandfonline.com/doi/full/10.1080/09712119.2017.1302876", + # Deliberately wide, because this rule has no published population and so no + # published range. The bounds are the physical span of cattle from a large + # calf to a mature bull, and their purpose is to catch a segmentation failure + # that produced a girth of 8 cm, not to model a breed. + girth_range_cm=(80.0, 260.0), + published_relative_error=None, + published_error_basis=( + "No single published figure. Sarwar et al. report estimates 'not " + "significantly different from the weighbridge'; popular sources quote " + "around 5% near 500 kg. Neither is an interval this code can use, so " + "nothing is claimed." + ), + apply=_schaeffer, +) + +#: The strongest evidence of the three, and the closest to Animap's animals. +LESOSKY_SHZ = WeightEquation( + id="lesosky-shz-2012", + inputs=("heart_girth_cm",), + citation=( + "Lesosky M, Dumas S, Conradie I, et al. (2012). 'A live weightโ€“heart " + "girth relationship for accurate dosing of east African shorthorn zebu " + "cattle.' Trop Anim Health Prod 45(1):311โ€“316. doi:10.1007/s11250-012-0220-3" + ), + published_form="weight^0.262 = 0.95 + 0.022 ร— girth (weight in kg, girth in cm)", + population=( + "East African shorthorn zebu, western Kenya, one week old to fully " + "mature. Indigenous tropical Bos indicus, which is the closest published " + "population to the cattle Animap serves." + ), + sample_size=703, + licence="CC BY (the article states the Creative Commons Attribution License)", + source_url="https://pmc.ncbi.nlm.nih.gov/articles/PMC3552367/", + # The paper covers one-week-old calves upward; 40 cm is a newborn's girth + # and 220 cm covers a mature zebu bull with margin. + girth_range_cm=(40.0, 220.0), + published_relative_error=0.20, + published_error_basis=( + "The authors' own claim, quoted: '95% prediction intervals fall within " + "the ยฑ20% body weight error band regarded as acceptable when dosing " + "livestock'. Rยฒ(adj) 0.98 over 703 animals, 300 modelling and 403 " + "validation. This is THEIR interval on THEIR cattle โ€” a prior, not an " + "Animap measurement." + ), + apply=_lesosky, +) + +#: A linear girth-only rule, kept because it disagrees with the one above. Two +#: published equations that diverge on the same animal are the cheapest available +#: evidence about how much of the error is the equation rather than the geometry. +ODADI_BORAN = WeightEquation( + id="odadi-boran-2018", + inputs=("heart_girth_cm",), + citation=( + "Odadi WO (2018). 'Using heart girth to estimate live weight of heifers " + "(Bos indicus) in pastoral rangelands of northern Kenya.' Livestock " + "Research for Rural Development 30(1), article 16." + ), + published_form="LW (kg) = โˆ’265 + 3.37 ร— HG (cm)", + population=( + "160 Boran zebu and Boran ร— Small East African Zebu heifers, aged 1โ€“3 " + "years, on eight group ranches in Laikipia and Isiolo, northern Kenya. " + "**Heifers only** โ€” a linear rule fitted to a narrow age band, so it is " + "expected to fail on calves and on mature bulls, and the intercept of " + "โˆ’265 kg makes that failure loud rather than subtle." + ), + sample_size=160, + licence=( + "Livestock Research for Rural Development is open access; the article " + "states no explicit licence, so the equation is used as a fact and the " + "text is not reproduced beyond citation." + ), + source_url="https://www.lrrd.org/lrrd30/1/wood30016.html", + # Below about 79 cm this line returns a negative weight, which is the + # clearest possible demonstration of why a fitted line needs a range on it. + girth_range_cm=(120.0, 200.0), + published_relative_error=0.073, + published_error_basis=( + "Residual standard error 12.8 kg, which the author gives as 7.3% of mean " + "live weight. Rยฒ 0.90 over 160 heifers. A residual standard error is one " + "sigma on the fitting set, NOT a 95% interval and NOT an out-of-sample " + "figure; it is the most flattering of the three numbers here and should " + "be read as such." + ), + apply=_odadi, +) + +EQUATIONS: tuple[WeightEquation, ...] = (SCHAEFFER, LESOSKY_SHZ, ODADI_BORAN) +BY_ID = {equation.id: equation for equation in EQUATIONS} + + +def ellipse_perimeter_cm(depth_cm: float, width_cm: float) -> float: + """Ramanujan's second approximation to an ellipse's perimeter. + + This is the **girth proxy** ยง22's pipeline needs, and the place the pipeline + is most likely to be wrong. A side view gives chest depth, a rear view gives + chest width, and a circumference has to come from somewhere; treating the + cross-section as an ellipse is the standard move and the chest is not an + ellipse. It is flatter across the back and rounder underneath, so this is + expected to be biased, and `experiments/cattle_weight` measures the bias + against tape-measured heart girth rather than assuming it away. + + Ramanujan's approximation itself is accurate to better than one part in 10โต + for any eccentricity a chest could have, so none of the error this proxy + makes is the approximation's. + """ + a, b = depth_cm / 2.0, width_cm / 2.0 + if a <= 0 or b <= 0: + raise OutOfRange("An ellipse needs two positive semi-axes.") + return float(math.pi * (3 * (a + b) - math.sqrt((3 * a + b) * (a + 3 * b)))) diff --git a/app/adapters/geometry/weight.py b/app/adapters/geometry/weight.py new file mode 100644 index 0000000000000000000000000000000000000000..be726f1282f71a9a4f9941875f76483814920a7e --- /dev/null +++ b/app/adapters/geometry/weight.py @@ -0,0 +1,334 @@ +"""A weight band, never a weight. + +Directive ยง22's initial UI is *"Experimental weight estimate / 350โ€“430 kg"* with +*"Add scale weight"* beside it, and ยง37 gives the reason: a single number is a +claim about an animal, and nothing in this pipeline can make that claim. So there +is no function here that returns a float. `estimate` returns a `WeightBand`, and +a `WeightBand` has no `.value`. + +## Where the width comes from + +Two sources, kept apart because they behave differently and a farm should not be +shown their sum as though it were one thing. + +**Scale error** is the metric uncertainty on the measurements โ€” how well ARCore, +VGGT or a reference marker pinned what a pixel is worth. It propagates through +the equation, and for a volumetric rule it amplifies: weight goes as girthยฒ, so a +2% scale error is roughly 6% of weight once the length term is included. This +module propagates it numerically by re-evaluating the equation at the perturbed +inputs, which is exact for any equation shape and cannot get an analytic +derivative wrong. + +**Equation error** is how wrong the published relationship is on an animal it was +not fitted to. It is a `PRIOR` until an Animap benchmark measures it, and +`WeightBand` records which of the two it is holding so ยง37's separation survives +into the object the API returns. + +## Sensitivity is reported, not buried + +`sensitivity` is d(ln W)/d(ln girth) computed numerically at the animal's own +measurements. It is the number that decides whether this capability is worth +building: if a 1% girth error costs 4% of weight, then the whole question is what +metric accuracy the capture can deliver, and no amount of model work substitutes +for it. +""" + +from __future__ import annotations + +import math +from dataclasses import dataclass, field + +from app.adapters.geometry.equations import ( + EQUATIONS, + OutOfRange, + WeightEquation, +) + +#: Fractional perturbation used for the numerical derivative. Small enough that +#: the second-order term is negligible for these equation shapes, large enough +#: to stay well clear of float64 cancellation. +DERIVATIVE_STEP = 1e-4 + +#: The narrowest band this module will publish, as a fraction of the estimate. +#: ยง22 says "the initial range can be wide" and says nothing about a floor; this +#: one exists because a band narrower than the best published equation's own +#: interval would be Animap claiming to have improved on the paper it is quoting. +MINIMUM_RELATIVE_HALF_WIDTH = 0.10 + + +@dataclass(frozen=True) +class BodyMeasurements: + """What the geometry stage produced, and how well scaled it is. + + `scale_relative_error` is not optional and has no default. A measurement in + centimetres whose scale nobody characterised is a measurement in arbitrary + units with `cm` written after it, and this whole capability turns on the + difference. + """ + + heart_girth_cm: float | None + body_length_cm: float | None + #: Fractional 1-sigma error on the metric scale. From + #: `signal.geometry.Scale.relative_error` for a marker, from the depth + #: source's own characterisation for ARCore or VGGT. + scale_relative_error: float + #: Free text naming what established the scale โ€” a marker, an AR session, a + #: known-size object, or the fact that nothing did. + scale_source: str + withers_height_cm: float | None = None + chest_depth_cm: float | None = None + chest_width_cm: float | None = None + #: True when the girth came from an ellipse through a depth and a width + #: rather than from a tape. Carried because the proxy has a bias of its own + #: that is separate from the scale error. + girth_is_proxy: bool = False + + def as_inputs(self) -> dict[str, float | None]: + return { + "heart_girth_cm": self.heart_girth_cm, + "body_length_cm": self.body_length_cm, + } + + +@dataclass(frozen=True) +class EquationResult: + """One equation's answer on one animal, with what moves it.""" + + equation_id: str + kg: float + #: Half-width in kg from scale error alone. + scale_half_width_kg: float + #: d(ln W)/d(ln girth). Dimensionless amplification of girth error. + sensitivity: float + #: The authors' own error, as a fraction. `None` where they published none. + published_relative_error: float | None + population: str + + +@dataclass(frozen=True) +class WeightBand: + """What the app may show. Deliberately has no single value. + + `statement` is words rather than a number because ยง37's worked example is + words. `basis` says whether the width came from an Animap measurement or from + the papers, and it is the field that must reach the screen alongside the + kilograms. + """ + + low_kg: float + high_kg: float + equations: tuple[EquationResult, ...] + basis: str + caveats: tuple[str, ...] = field(default_factory=tuple) + + @property + def statement(self) -> str: + return ( + f"Experimental weight estimate, " + f"{round(self.low_kg / 5) * 5:g}โ€“{round(self.high_kg / 5) * 5:g} kg" + ) + + @property + def relative_half_width(self) -> float: + midpoint = (self.low_kg + self.high_kg) / 2.0 + return (self.high_kg - self.low_kg) / 2.0 / midpoint if midpoint else float("inf") + + +class NotEnoughGeometry(RuntimeError): + """No equation could run on what the capture produced. + + Raised rather than returning a very wide band. ยง22's `reject_if` list already + names `insufficient_geometry` as a re-capture prompt, and a band from no + usable measurement is not wide, it is meaningless. + """ + + +def _sensitivity(equation: WeightEquation, inputs: dict[str, float | None]) -> float: + """d(ln W)/d(ln girth), numerically, at this animal's own measurements. + + Numerical rather than analytic on purpose. The three equations have three + different shapes โ€” a power law, a cube-ish volume rule and a straight line โ€” + and a hand-differentiated version of each is three more places to be wrong. + """ + girth = inputs["heart_girth_cm"] + if girth is None: + return float("nan") + up = dict(inputs, heart_girth_cm=girth * (1 + DERIVATIVE_STEP)) + down = dict(inputs, heart_girth_cm=girth * (1 - DERIVATIVE_STEP)) + try: + high, low = equation(**up), equation(**down) + except OutOfRange: + # An animal sitting exactly on the boundary of the equation's published + # range: the perturbation steps outside it and the range guard fires, + # correctly. A derivative that cannot be taken is `nan`, not a crash โ€” + # `estimate` has already accepted this animal, and a band is still owed. + return float("nan") + if low <= 0 or high <= 0: + # The linear rule goes non-positive below its intercept. A log derivative + # does not exist there, and reporting one would be inventing a number. + return float("nan") + return (math.log(high) - math.log(low)) / (2 * DERIVATIVE_STEP) + + +def _scale_half_width(equation: WeightEquation, inputs: dict[str, float | None], + relative_error: float) -> float: + """Half-width in kg from perturbing every linear measurement together. + + Together, not independently. A scale error is a single multiplicative factor + on every length the capture produced โ€” it is one mistake about what a pixel + is worth, not several independent ones โ€” so girth and length move in step and + the errors add rather than partially cancelling. Treating them as independent + would understate the band, which is the direction that matters. + """ + if relative_error <= 0: + return 0.0 + centre = equation(**inputs) + sides: list[float] = [] + for sign in (+1, -1): + perturbed = {k: (v * (1 + sign * relative_error) if v is not None else None) + for k, v in inputs.items()} + try: + sides.append(equation(**perturbed)) + except OutOfRange: + # The perturbed animal falls outside the range this equation was + # published over. That is not nothing to report: it means a + # plausible capture error puts this animal off the end of the curve. + # Fall back to whichever side is still computable rather than + # silently returning a half-width of zero, and report `nan` when + # neither is โ€” `estimate` turns that into a caveat. + continue + if len(sides) == 2: + return abs(sides[0] - sides[1]) / 2.0 + if len(sides) == 1: + return abs(sides[0] - centre) + return float("nan") + + +def estimate(measurements: BodyMeasurements, *, + equations: tuple[WeightEquation, ...] = EQUATIONS, + measured_relative_error: float | None = None, + measured_basis: str = "") -> WeightBand: + """Every equation that can run, and the band that covers them. + + `measured_relative_error` is an Animap benchmark's figure. When it is absent + the band falls back to the published intervals and says so, because ยง37 will + not let a prior be presented as a measurement. When it is present it + **replaces** the published error rather than being combined with it: the + benchmark measured the equation on Animap's own geometry, so the published + interval is already inside it and adding both would double-count. + """ + inputs = measurements.as_inputs() + results: list[EquationResult] = [] + refusals: list[str] = [] + #: Equations the scale error itself pushed off their published range. + scale_dropped: list[str] = [] + + for equation in equations: + try: + kg = equation(**inputs) + except OutOfRange as refused: + refusals.append(str(refused)) + continue + results.append(EquationResult( + equation_id=equation.id, + kg=kg, + scale_half_width_kg=_scale_half_width( + equation, inputs, measurements.scale_relative_error), + sensitivity=_sensitivity(equation, inputs), + published_relative_error=equation.published_relative_error, + population=equation.population, + )) + + if not results: + raise NotEnoughGeometry( + "No published equation could be applied to this capture. " + + " ".join(refusals) + ) + + caveats: list[str] = [] + if measurements.girth_is_proxy: + caveats.append( + "The girth is an ellipse through a measured chest depth and width, " + "not a tape around the animal. A chest is not an ellipse." + ) + if measurements.scale_relative_error <= 0: + caveats.append( + "No scale error was supplied, so the band below carries none. That " + "is almost certainly wrong: every metric scale has an error." + ) + caveats.extend(refusals) + + # The band spans every equation's answer, widened by each one's own + # uncertainty. Spanning rather than averaging is deliberate: two published + # equations disagreeing by 15% on the same animal is evidence about the + # equations, and averaging it away would produce a narrow band built on a + # disagreement nobody was told about. + lows, highs = [], [] + for result in results: + if measured_relative_error is not None: + half = result.kg * measured_relative_error + elif result.published_relative_error is not None: + half = result.kg * result.published_relative_error + else: + # No published interval and no measurement. The scale term is all + # that is left, and it is not the dominant term, so the band would be + # falsely narrow. Widen to the floor and say why. + half = result.kg * MINIMUM_RELATIVE_HALF_WIDTH + caveats.append( + f"{result.equation_id} publishes no error interval, so its " + f"contribution to the band is a floor, not a measurement." + ) + if math.isnan(result.scale_half_width_kg): + # **The equation is dropped, not banded on its published interval + # alone.** When a scale error this large moves the animal off the + # end of the published range, what this equation does under that + # error is *unknown*, and unknown is not zero. Falling back to the + # published interval made the band NARROWER as the scale error grew + # โ€” at a 40% error the band spanned 0โ€“1015 kg and at 80% it + # collapsed back to 317โ€“521 kg, because every scale term had + # silently dropped out. A band that tightens as the capture gets + # worse is the most dangerous shape this function could have. + scale_dropped.append(result.equation_id) + continue + half = max(half, result.scale_half_width_kg) + lows.append(result.kg - half) + highs.append(result.kg + half) + + # **Losing any equation to the scale error is a refusal, not a narrower + # band.** Dropping one still shrinks the span whenever the dropped equation + # was the one setting an extreme โ€” at a 60% scale error two of the three + # fall off their published ranges and the band tightened from 1,015 kg wide + # to 799. A band that tightens as the capture gets worse is the most + # dangerous shape this function could have, so the capture is refused + # instead. ยง22's `reject_if` already lists `insufficient_geometry`, and a + # scale error this large is exactly that. + if scale_dropped or not lows: + raise NotEnoughGeometry( + f"A scale error of {measurements.scale_relative_error:.0%} moves " + f"this animal outside the published range of " + f"{', '.join(scale_dropped) or 'every equation'}, so what those " + f"equations would do under it is unknown rather than small. No band " + f"is given. Re-capture with a better scale reference." + ) + + low, high = min(lows), max(highs) + midpoint = (low + high) / 2.0 + floor = midpoint * MINIMUM_RELATIVE_HALF_WIDTH + if (high - low) / 2.0 < floor: + low, high = midpoint - floor, midpoint + floor + + if measured_relative_error is not None: + basis = measured_basis or "measured by an Animap benchmark" + else: + basis = ( + "PRIOR โ€” the equations' own published intervals, on the authors' own " + "cattle. No Animap benchmark stands behind this width." + ) + + return WeightBand( + low_kg=max(0.0, low), + high_kg=high, + equations=tuple(results), + basis=basis, + caveats=tuple(dict.fromkeys(caveats)), + ) diff --git a/app/adapters/licence_policy.py b/app/adapters/licence_policy.py new file mode 100644 index 0000000000000000000000000000000000000000..4b15c35999bc2f116114a9953f9fa261b112df5f --- /dev/null +++ b/app/adapters/licence_policy.py @@ -0,0 +1,188 @@ +"""Whether a governance problem stops a load, or only records itself. + +**Two settings, and they live here together so a deployment has one file to +read.** Both are environment variables, both are read at call time, and nothing +else in the codebase decides either. + + ANIMAP_LICENCE_POLICY=enforce # default. A refused licence does not load. + ANIMAP_LICENCE_POLICY=record # it loads, loudly, and the fact is kept. + + ANIMAP_ARTEFACT_IDENTITY=record # default. Bytes that contradict their card + # load, loudly, and the fact is kept. + ANIMAP_ARTEFACT_IDENTITY=refuse # they do not load. + +They are separate variables because they answer different questions. +`ANIMAP_LICENCE_POLICY` is a permission question โ€” *may Animap serve from these +terms?* โ€” and the founder's standing instruction is that the answer is currently +yes for everything, so the work is not blocked while the ledger is kept. +`ANIMAP_ARTEFACT_IDENTITY` is an integrity question โ€” *is this file what its +card says it is?* โ€” and its default is `record` for the same reason and no +other: a refusal here would stop model work today, and the value of the check +right now is that the ledger says what actually shipped. + +**`record` on either is only defensible while the record is true**, which is why +`app/adapters/fingerprints.py` exists at all: before it, an exception recorded +under `record` named the licence the *card* claimed, so a mislabelled artefact +produced a ledger entry that was wrong in exactly the case the ledger was for. + +**Detection always runs, under both settings.** That is deliberate and it is the +part worth protecting: ADR 0017 records a watchdog defeating the label-based +version of this check by writing `"license": "Apache-2.0"` over a path to AGPL +weights, and the fix was to key the check to the runtime instead. `record` mode +does not weaken the detection, it changes what happens after it fires. A model +that is a licence problem stays a licence problem that anybody can find, which +is the whole point of keeping the machinery rather than deleting it. + +**The default is `enforce`, and that is a decision rather than an oversight.** +Animap is a closed-source commercial product serving predictions from a private +API. AGPL-3.0's network clause and CC BY-NC's NonCommercial grant are live +exposures for exactly that shape of business, and ADR 0018 measured the cost of +avoiding the AGPL one at 8.8 points of coverage. Relaxing it is a decision with +a signature on it, so it is made by setting a variable on a deployment โ€” an act +that appears in configuration and in `/health` โ€” and not by a default that +nobody remembers choosing. + +If the posture is relaxed, `record` mode is built so the debt is payable later: +every load that would have been refused is logged at ERROR, kept in +`RECORDED_EXCEPTIONS`, and surfaced by `scripts/licence_ledger.py`. The question +"what did we ship that we should not have?" has a one-command answer instead of +being archaeology. +""" + +from __future__ import annotations + +import logging +import os +from dataclasses import dataclass, field +from datetime import datetime, timezone +from enum import Enum + +logger = logging.getLogger(__name__) + +#: The one setting. Named here so a grep for it finds the definition, the +#: reader, and the documentation in the same file. +POLICY_ENV = "ANIMAP_LICENCE_POLICY" + + +class LicencePolicy(str, Enum): + #: A licence Animap may not serve under stops the load. The capability goes + #: unavailable, which is a state the service already models honestly. + ENFORCE = "enforce" + #: It loads. The fact is logged at ERROR, recorded, and reported by + #: `/health` and the ledger. Nothing is silent and nothing is lost. + RECORD = "record" + + +DEFAULT_POLICY = LicencePolicy.ENFORCE + + +def current() -> LicencePolicy: + """Read at call time, not import time. + + Same reasoning as `main._configured_token`: a change takes effect on restart + rather than needing a rebuild, and a test can set it without reloading the + module. + """ + raw = os.environ.get(POLICY_ENV, "").strip().lower() + if not raw: + return DEFAULT_POLICY + try: + return LicencePolicy(raw) + except ValueError: + # An unrecognised value falls back to the strict setting rather than to + # the permissive one. A typo in a deployment variable must not be the + # thing that puts AGPL weights in front of a farmer. + logger.error( + "%s=%r is not a recognised policy; falling back to %s. Valid: %s.", + POLICY_ENV, raw, DEFAULT_POLICY.value, + ", ".join(p.value for p in LicencePolicy), + ) + return DEFAULT_POLICY + + +#: The other setting. Same shape, same reading-at-call-time, same fallback rule. +IDENTITY_ENV = "ANIMAP_ARTEFACT_IDENTITY" + + +class IdentityPolicy(str, Enum): + #: An artefact whose bytes contradict its card loads, and the contradiction + #: is logged at ERROR and recorded. **The licence decision is still made on + #: what the bytes are**, never on what the card claimed โ€” that is the part + #: that does not soften, because it is what makes the record true. + RECORD = "record" + #: It does not load. The capability goes unavailable, which the service + #: already models honestly. + REFUSE = "refuse" + + +#: `record`, and this is the founder's standing instruction rather than an +#: oversight: no model is dropped on a governance question right now, and the +#: value of this check today is an accurate ledger rather than a closed door. +DEFAULT_IDENTITY_POLICY = IdentityPolicy.RECORD + + +def identity_policy() -> IdentityPolicy: + raw = os.environ.get(IDENTITY_ENV, "").strip().lower() + if not raw: + return DEFAULT_IDENTITY_POLICY + try: + return IdentityPolicy(raw) + except ValueError: + # Falls back to the *default* rather than to the strict setting, which + # is the opposite of `current()` above and is deliberate: here the + # default is the permissive one, and a typo must not silently make a + # deployment stricter than it was configured to be either. + logger.error( + "%s=%r is not a recognised policy; falling back to %s. Valid: %s.", + IDENTITY_ENV, raw, DEFAULT_IDENTITY_POLICY.value, + ", ".join(p.value for p in IdentityPolicy), + ) + return DEFAULT_IDENTITY_POLICY + + +@dataclass(frozen=True) +class RecordedException: + """One load that the strict policy would have refused. + + Kept in memory for `/health`, and re-derivable from the logs, which is what + makes this survivable if the process restarts. The ledger reads the model + cards directly and does not depend on this list. + """ + + what: str + runtime: str + licence: str + reason: str + source_url: str = "" + at: str = field( + default_factory=lambda: datetime.now(timezone.utc).isoformat() + ) + + +#: Everything loaded under `record` that `enforce` would have stopped. Empty +#: under the default policy, and its non-emptiness is itself the finding. +RECORDED_EXCEPTIONS: list[RecordedException] = [] + + +def record_exception( + *, what: str, runtime: str, licence: str, reason: str, source_url: str = "" +) -> RecordedException: + """Note a licence problem that was allowed through, loudly.""" + entry = RecordedException( + what=what, runtime=runtime, licence=licence, + reason=reason, source_url=source_url, + ) + RECORDED_EXCEPTIONS.append(entry) + logger.error( + "LICENCE EXCEPTION: %s loaded on runtime %r under %s, which Animap may " + "not normally serve. Allowed because %s=%s. %s Reason: %s", + what, runtime, licence, POLICY_ENV, LicencePolicy.RECORD.value, + source_url, reason, + ) + return entry + + +def reset_recorded() -> None: + """For tests. Production never clears this โ€” a debt that can be cleared by + calling a function is not a debt.""" + RECORDED_EXCEPTIONS.clear() diff --git a/app/adapters/licences.py b/app/adapters/licences.py new file mode 100644 index 0000000000000000000000000000000000000000..85780108f1a08a1d63ca0695690501847197d6ea --- /dev/null +++ b/app/adapters/licences.py @@ -0,0 +1,639 @@ +"""What each runtime's weights are actually licensed under. + +**This table is the control, and the model card is not.** ADR 0017 records why: +a card declares its own licence, so the threat the licence gate was written for โ€” +somebody editing a card โ€” was precisely the case it could not catch. A watchdog +wrote `"license": "Apache-2.0"` over a path to `yolo11m.pt` and `discover()` +waved it through. The fix named one runtime in a frozenset. This generalises it. + +A runtime name is a fact about which loader executes, not a claim about terms. +`ultralytics` loads Ultralytics weights and nothing else; `dinov3-onnx` loads a +DINOv3 export and nothing else. So the licence is attached to the runtime, in +committed code, next to the URL it was read from and the date it was read. + +Three things are refused, and the third is new: + +1. A runtime this file does not know. No default, for the reason + `detectors.build` gives: a default would let a typo change which licensed + model produced a farmer's result. +2. A runtime whose real licence Animap may not serve under. +3. **A card whose declared licence disagrees with the runtime's real one.** This + is the check that catches the ADR 0017 exploit generically rather than by + name. A card claiming Apache-2.0 over a copyleft runtime is now a refusal + that says which of the two is lying. + +Nothing here is legal advice, and `app/dispositions.py` carries the same caveat. +Every entry records the primary source so the next reader checks the licence +rather than this paragraph. +""" + +from __future__ import annotations + +from dataclasses import dataclass + +from app.adapters import licence_policy + +#: Licence identifiers Animap may not serve from. Kept identical in spirit to +#: `providers.DISALLOWED_LICENSES`, which remains the gate on the card's own +#: declaration โ€” the two checks are deliberately separate, because one reads a +#: claim and one reads a fact, and the whole point of ADR 0017's fifth mechanism +#: is that the claim can be false. +DISALLOWED = frozenset({ + "AGPL-3.0", "AGPL-3.0-only", "AGPL-3.0-or-later", + "GPL-3.0", "GPL-3.0-only", "GPL-3.0-or-later", "GPL-2.0", + "SSPL-1.0", + "CC-BY-NC-4.0", "CC-BY-NC-SA-4.0", "CC BY-NC 4.0", "CC-BY-NC-SA-3.0", + "Non-Commercial Government Licence", + "CC-BY-NC-ND-4.0", +}) + + +class LicenceRefused(RuntimeError): + """This runtime may not be served, whatever its card says.""" + + +@dataclass(frozen=True) +class RuntimeLicence: + """The terms one loader's weights actually arrive under. + + `permits_commercial_use` and `permits_network_service` are the two questions + that decide whether Animap may use a model at all โ€” it is a closed-source + commercial product serving predictions from a private API โ€” and they are + recorded separately from the SPDX-ish identifier because bespoke research + licences answer them differently from anything SPDX has a name for. + """ + + runtime: str + #: The identifier as the publisher states it. Bespoke licences keep their + #: own name rather than being mapped onto the nearest SPDX id, because the + #: mapping is where the meaning gets lost. + licence: str + source_url: str + verified_on: str + permits_commercial_use: bool + permits_network_service: bool + #: Whether the weights sit behind an accepted-terms gate. A gated repo is not + #: automatically disqualifying, but it means `scripts/install_models.py` + #: cannot fetch it unattended, which changes the deployment story. + gated: bool = False + #: Text the product must display, verbatim, if the licence demands it. + #: + #: Machine-readable rather than a sentence in a note, because an attribution + #: obligation is the one licence term that is discharged by a line in a UI + #: rather than by a lawyer โ€” so it needs to be something a test can assert + #: and a build can carry into an about page. Empty means none is required. + attribution_required: str = "" + #: Repository-relative files that carry `attribution_required` today. + #: `tests/test_attribution.py` opens each one and fails if the string is not + #: in it, so this list is a claim that can be wrong for exactly one commit. + attribution_displayed_in: tuple[str, ...] = () + #: Surfaces that ought to carry it and do not yet, each with the reason. + #: + #: **This field exists because the alternative was a lie.** The DINOv3 model + #: card read "MITIGATION TAKEN: the product displays 'Built with DINOv3'" + #: while no Android screen, API response or web surface mentioned DINO at + #: all โ€” a licence obligation discharged by writing that it had been + #: discharged. The same test asserts the *negative*: every path here must + #: exist and must NOT contain the string, so the day somebody adds it the + #: suite says to move the entry up rather than letting the record rot. + attribution_outstanding: tuple[tuple[str, str], ...] = () + note: str = "" + + @property + def attribution_satisfied(self) -> bool: + """Whether every surface this obligation needs actually carries it.""" + return bool(self.attribution_required) and not self.attribution_outstanding + + @property + def servable(self) -> bool: + """Whether Animap may answer a farmer's request with this.""" + return ( + self.licence not in DISALLOWED + and self.permits_commercial_use + and self.permits_network_service + ) + + +def register(licence: RuntimeLicence) -> RuntimeLicence: + RUNTIME_LICENCES[licence.runtime] = licence + return licence + + +#: Runtime โ†’ the terms its weights arrive under. Populated below. +RUNTIME_LICENCES: dict[str, RuntimeLicence] = {} + + +def gate(runtime: str | None, declared_licence: str | None = None, + *, what: str = "") -> RuntimeLicence: + """Raise unless this runtime may be served and its card agrees with it. + + `declared_licence` is optional because the deterministic methods have no + card. When it is supplied it is checked *against* the table rather than + trusted, which is the whole difference between this and the card gate. + + **Two of the three refusals below soften under + `ANIMAP_LICENCE_POLICY=record`, and one does not.** The two that soften are + licence *permission* questions โ€” is this runtime unrecorded, and are its + terms ones Animap may not serve under. Those are the founder's call to + take, and under `record` they load with the fact kept. + + The one that does not soften is the card-versus-runtime *disagreement*. + That is not a permission question, it is an integrity question: a card whose + declared licence is not what its loader actually loads is a card that + poisons the ledger. And a ledger is exactly what `record` mode depends on โ€” + the whole argument for allowing a licence exception is that it stays + findable afterwards. Letting a lying card through would make the record of + the exception wrong, which defeats the mode rather than serving it. So it + raises under both policies, and the fix is to correct the card. + """ + policy = licence_policy.current() + + if not runtime: + raise LicenceRefused( + "No runtime named, so there is no way to know what licence the " + "weights arrive under. A card without a runtime can be inspected " + "and not run." + ) + + licence = RUNTIME_LICENCES.get(runtime) + if licence is None: + message = ( + f"Runtime {runtime!r} is not in the licence table, so nothing has " + f"checked what its weights are licensed under. Add an entry with " + f"the primary source you read, or the runtime cannot be served. " + f"Known: {', '.join(sorted(RUNTIME_LICENCES))}." + ) + if policy is licence_policy.LicencePolicy.ENFORCE: + raise LicenceRefused(message) + licence_policy.record_exception( + what=what or runtime, runtime=runtime, licence="UNRECORDED", + reason="No entry in the runtime licence table; terms are unknown.", + ) + # Returned as an explicit unknown rather than as something permissive, + # so a caller that reports `servable` reports False. + return RuntimeLicence( + runtime=runtime, licence="UNRECORDED", source_url="", + verified_on="", permits_commercial_use=False, + permits_network_service=False, + note="Loaded under a relaxed policy. Nobody has read its licence.", + ) + + if not licence.servable: + message = ( + f"Runtime {runtime!r} loads weights under {licence.licence}, which " + f"Animap may not serve from" + + ("" if licence.permits_commercial_use + else " โ€” it does not permit commercial use") + + ("" if licence.permits_network_service + else " โ€” it does not permit serving over a network") + + f". Read {licence.source_url}. {licence.note}" + ) + if policy is licence_policy.LicencePolicy.ENFORCE: + raise LicenceRefused(message) + licence_policy.record_exception( + what=what or runtime, runtime=runtime, licence=licence.licence, + reason=message, source_url=licence.source_url, + ) + + if declared_licence and declared_licence != licence.licence: + # The generic form of the ADR 0017 exploit, and it raises under every + # policy โ€” see the docstring. One of these two is wrong, and the card is + # the one that can be edited without review noticing. + raise LicenceRefused( + f"The card declares {declared_licence!r} but runtime {runtime!r} " + f"loads weights licensed {licence.licence!r} ({licence.source_url}). " + f"A card's licence field is a claim; the runtime is a fact about " + f"which loader runs. Fix whichever is wrong โ€” do not widen this " + f"check. This is an integrity check, not a permission check, so " + f"{licence_policy.POLICY_ENV} does not relax it: a relaxed policy " + f"is only defensible while the record of what was relaxed is true." + ) + + return licence + + +# --- The table. Every entry names where it was read and when. ----------------- + +# The two runtimes that predate this file, from ADR 0017, which quotes both +# licences at length. +register(RuntimeLicence( + runtime="yolox-onnx", + licence="Apache-2.0", + source_url="https://github.com/Megvii-BaseDetection/YOLOX/blob/main/LICENSE", + verified_on="2026-08-20", + permits_commercial_use=True, + permits_network_service=True, + note=( + "Megvii publishes no separate licence for the released ONNX weights. " + "The repository's Apache-2.0 is read as covering the artefacts the " + "repository distributes, which is an inference and is recorded as one " + "in ADR 0017." + ), +)) + +register(RuntimeLicence( + runtime="ultralytics", + licence="AGPL-3.0", + source_url="https://www.ultralytics.com/license", + verified_on="2026-08-20", + permits_commercial_use=False, + permits_network_service=False, + note=( + "The checkpoint carries `license: AGPL-3.0 License` in its own pickle " + "metadata, and the vendor's published position is that a closed-source " + "SaaS needs an Enterprise licence. Kept in the tree for evaluation/ " + "only (ADR 0017)." + ), +)) + +#: Deterministic signal processing. No weights, so no artefact and no terms to +#: check โ€” which is a substantial part of why ยง4 says to prefer it: "Do not use +#: a neural model when deterministic signal processing is better." The entry +#: exists so the gate has something to return rather than being special-cased, +#: and the licence named is the library's, since that is the only thing shipped. +register(RuntimeLicence( + runtime="opencv-numpy", + licence="Apache-2.0", + source_url="https://pypi.org/project/opencv-python/", + verified_on="2026-08-21", + permits_commercial_use=True, + permits_network_service=True, + note=( + "Read from the installed wheel's own metadata: opencv-python 5.0.0.93 " + "declares `License: Apache 2.0`, numpy 2.5.2 declares `BSD-3-Clause AND " + "0BSD AND MIT AND Zlib AND CC0-1.0`. **The wheel is not purely " + "permissive.** Its LICENSE-3RD-PARTY.txt says `FFmpeg is redistributed " + "within all opencv-python packages` and reproduces LGPL-2.1 and LGPL-3 " + "in full; on macOS wheels libbluray, libgnutls, libmp3lame, librtmp and " + "others are LGPL too. LGPL obligations attach on conveying, and Animap " + "conveys nothing to farmers, so this is materially weaker than the AGPL " + "question ADR 0017 declined to answer. It is recorded rather than " + "resolved: only the video-decode path touches FFmpeg, and the flow, FFT " + "and contour code does not." + ), +)) + +#: Deterministic audio (ยง26, ยง4). NumPy does the arithmetic; an `ffmpeg` binary +#: does the decode, because this service has no audio library at all โ€” no +#: `soundfile`, no `librosa`, no `av`, and `cv2` exposes no audio path. +#: +#: **The two halves of this entry answer different questions and both matter.** +#: +#: *Does FFmpeg's licence reach Animap's code?* No. `adapters/audio/decode.py` +#: runs `ffmpeg` as a separate program over a temporary file. Neither the LGPL +#: nor the GPL treats invoking a separate executable as creating a derivative +#: work, so nothing here is a combined work with FFmpeg, and this is materially +#: weaker than the linking question `opencv-numpy` above declines to answer. +#: +#: *Which build may be shipped?* Not the one this was developed against. Read +#: from `ffmpeg -version` on the development machine on 2026-08-22: the +#: configuration includes `--enable-gpl --enable-nonfree --enable-libfdk-aac`. +#: FFmpeg's own `LICENSE.md` says of `--enable-nonfree`: *"This will cause the +#: resulting binary to be unredistributable."* So the binary on this laptop +#: could not be put in a container image at all, whatever Animap's own licence +#: is. `permits_commercial_use` and `permits_network_service` below describe the +#: **default LGPL v2.1+ build** โ€” *"Most files in FFmpeg are under the GNU +#: Lesser General Public License version 2.1 or later"* โ€” which is what a +#: deployment must build or source. +register(RuntimeLicence( + runtime="ffmpeg-numpy", + licence="LGPL-2.1-or-later (default build); GPL-2.0-or-later with --enable-gpl; unredistributable with --enable-nonfree", + source_url="https://raw.githubusercontent.com/FFmpeg/FFmpeg/master/LICENSE.md", + verified_on="2026-08-22", + permits_commercial_use=True, + permits_network_service=True, + note=( + "Applies to a default LGPL v2.1+ FFmpeg invoked as a separate binary. " + "**The build present during development is not that build**: " + "`--enable-gpl --enable-nonfree --enable-libfdk-aac`, which FFmpeg's " + "LICENSE.md says makes the resulting binary unredistributable. Before " + "anything ships, build or source a default-configuration FFmpeg and " + "record its `-version` line beside this entry. LGPL obligations attach " + "on conveying; Animap conveys nothing to farmers from the server, but a " + "phone build that bundles a decoder does convey, and that is a different " + "question nobody here has answered. NumPy 2.5.2 declares `BSD-3-Clause " + "AND 0BSD AND MIT AND Zlib AND CC0-1.0`." + ), +)) + + +# --- Frozen embeddings (ยง3 DINOv3, ยง4 MegaDescriptor). ----------------------- + +register(RuntimeLicence( + runtime="dinov2-onnx", + licence="Apache-2.0", + source_url="https://github.com/facebookresearch/dinov2/blob/main/LICENSE", + verified_on="2026-08-21", + permits_commercial_use=True, + permits_network_service=True, + note=( + "Code and weights both Apache-2.0; `facebook/dinov2-small` carries " + "`license: apache-2.0` and is ungated. Its embedding is the same 384 " + "dimensions as DINOv3 ViT-S/16 at 22.06M against 21.60M parameters, so " + "it is the drop-in that costs nothing to be sure about." + ), +)) + +register(RuntimeLicence( + runtime="dinov3-onnx", + licence="DINOv3 License", + source_url="https://github.com/facebookresearch/dinov3/blob/main/LICENSE.md", + verified_on="2026-08-21", + permits_commercial_use=True, + permits_network_service=True, + gated=True, + # Cheapest possible discharge of the ambiguity below: display it and both + # readings of the licence are satisfied. One line in an about page against + # an hour of counsel's time. + # + # **It is not discharged yet, and the second list is the honest half.** A + # NOTICES file in a source repository is where a lawyer looks, not where a + # farmer does, and the Meta-hosted text says "prominently display". Until + # the string is on a screen somebody using Animap can see, the obligation is + # met on paper and not in the product. + attribution_required="Built with DINOv3", + attribution_displayed_in=("THIRD_PARTY_NOTICES.md",), + attribution_outstanding=( + ( + "apps/android/app/src/main/java/com/ccc2c/animap/ui/screens/more/" + "MoreScreen.kt", + "The app has no about screen and no third-party notices screen โ€” " + "`ui/screens/Notices.kt` is a form-message component despite the " + "name, and nothing under `ui/` mentions a licence. `MoreScreen` is " + "where an about entry would hang, and this is the surface that " + "decides whether 'prominently display' is satisfied, because it is " + "the only one a farmer can reach.", + ), + ( + "services/inference/app/main.py", + "`/health` and `/capabilities` name every adapter and its licence " + "and do not carry the attribution string, so an integrator " + "embedding Animap has no way to learn it owes one.", + ), + ), + note=( + "Bespoke Meta licence, not Apache-2.0 โ€” `facebookresearch/dinov3` " + "reports SPDX NOASSERTION and covers code and weights alike. It carries " + "no non-commercial clause, no acceptable-use policy and no monthly-user " + "trigger. **Two published versions of this licence disagree.** The " + "LICENSE.md shipped with the weights is dated 19 August 2025 and ends " + "clause 1.b.i at providing a copy of the agreement; the text at " + "ai.meta.com/resources/models-and-libraries/dinov3-license, which is " + "what the model card links to, is dated 14 August 2025 and adds a duty " + "to `prominently display \"Built with DINOv3\"`. Section 8 lets Meta " + "amend unilaterally with immediate effect. The `facebook/*` repos are " + "gated with manual approval; `timm/vit_small_patch16_dinov3.lvd1689m` " + "is ungated and ships the same LICENSE.md, which is where the shipped " + "export came from. Counsel's question, not an engineer's." + ), +)) + +#: Refused, and registered *in order to be* refused. An unknown runtime produces +#: "nobody has checked this"; a known-and-disallowed one produces the reason, +#: which is what a reader needs when they ask why the wildlife re-ID model the +#: directive names is not here. +#: +#: **Installed since 2026-08-21, and no more servable for it.** The founder +#: lifted the rule that a refused licence stops a model being exported and +#: measured, so `models/alternates/megadescriptor/` now holds a real artefact +#: and ยง40.2's benchmark has a number in it. Nothing below changed: the terms +#: did not, `servable` is still False, and `gate` still raises under `enforce`. +#: The only difference is that the refusal now costs a measurement rather than +#: preventing one. +register(RuntimeLicence( + runtime="megadescriptor-timm", + licence="CC-BY-NC-4.0", + source_url="https://huggingface.co/BVRA/MegaDescriptor-L-384/raw/main/README.md", + verified_on="2026-08-21", + permits_commercial_use=False, + permits_network_service=True, + note=( + "All eight BVRA MegaDescriptor repos carry `license: cc-by-nc-4.0` in " + "their card front-matter. CC BY-NC 4.0 ยง2(a)(1) grants rights for " + "NonCommercial purposes only and the family has no commercial " + "exception, so Animap cannot use these weights at any size. " + "**There is a trap in the artefact itself**: " + "`BVRA/MegaDescriptor-L-384/config.json` contains " + "`\"pretrained_cfg\": {... \"license\": \"mit\" ...}`, inherited " + "boilerplate from timm's original Swin config that describes " + "Microsoft's weights and not these. A loader that read the licence out " + "of the checkpoint would conclude MIT and be wrong โ€” which is the same " + "shape of failure as ADR 0017's edited card, arriving from upstream " + "instead of from a colleague." + ), +)) + +#: Also refused, for a different and more easily missed reason: nothing was +#: granted at all. Installed and benchmarked on the same footing as +#: MegaDescriptor since 2026-08-21, and unservable on the same footing too. +#: +#: One extra fact worth carrying, because it is a build-time risk this table's +#: usual questions do not ask about: this repository ships its architecture as +#: Python rather than as a `transformers` class, so exporting it means running +#: a third party's code. See `scripts/export_embedding.py._load_remote`. +register(RuntimeLicence( + runtime="miewid", + licence="none stated", + source_url="https://huggingface.co/conservationxlabs/miewid-msv3", + verified_on="2026-08-21", + permits_commercial_use=False, + permits_network_service=False, + note=( + "The wildlife re-ID alternative to MegaDescriptor, and it carries no " + "licence tag, no LICENSE file, and `cardData` of " + "`{\"library_name\": \"transformers\", \"tags\": []}`. The upstream " + "repo returns `\"license\": null`. Absent an express grant the default " + "is all rights reserved, so silence is a refusal and not a permission. " + "Third-party re-uploads tagged MIT cannot grant what they were never " + "given." + ), +)) + + +# --- Open-vocabulary detection and counting (ยง4). ----------------------------- + +register(RuntimeLicence( + runtime="grounding-dino-hf", + licence="Apache-2.0", + source_url="https://github.com/IDEA-Research/GroundingDINO/blob/main/LICENSE", + verified_on="2026-08-21", + permits_commercial_use=True, + permits_network_service=True, + note=( + "Stock Apache-2.0 with no appended restrictions; the HF re-releases " + "`IDEA-Research/grounding-dino-tiny` and `-base` carry " + "`license: apache-2.0` explicitly, which the GitHub weights do not โ€” " + "there the repo licence covering the release artefacts is an inference " + "from silence, the same shape as YOLOX above. The BERT text encoder it " + "loads is Apache-2.0. Objects365 and the other training sets carry " + "their own terms and whether those reach through to weights is " + "unsettled; not investigated." + ), +)) + +register(RuntimeLicence( + runtime="countgd", + licence="MIT", + source_url="https://raw.githubusercontent.com/niki-amini-naieni/CountGD/main/LICENSE", + verified_on="2026-08-21", + permits_commercial_use=True, + permits_network_service=True, + note=( + "MIT for the code (`Copyright (c) 2024 Niki Amini-Naieni`) and " + "`license: mit` on the `nikigoli/CountGD` weights. It vendors a " + "GroundingDINO fork whose deformable-attention module falls back to a " + "pure-PyTorch `grid_sample` path when the CUDA extension is absent, so " + "CPU-only execution is possible in principle. No CPU throughput figure " + "is published and none was measured here." + ), +)) + + +#: ยง26's audio-embedding leg, and **not the model the directive names.** +#: +#: ยง26 asks for *"Perception Encoder audio / AV embeddings"*. Meta publishes the +#: vision ports of Perception Encoder (`facebook/PE-Core-*`, Apache-2.0) and +#: `facebook/perception_encoder` returns 401 to an unauthenticated request, so +#: no audio tower was reachable. `facebook/sam-audio` returns 401 as well. +#: CLAP is the substitute that was reachable, and +#: `experiments/poultry_respiratory/` records it as a substitute in its run +#: notes rather than letting it stand in for what was asked for. +register(RuntimeLicence( + runtime="clap-hf", + licence="Apache-2.0", + source_url="https://huggingface.co/laion/clap-htsat-unfused", + verified_on="2026-08-22", + permits_commercial_use=True, + permits_network_service=True, + note=( + "`laion/clap-htsat-unfused` carries `license: apache-2.0` in its card " + "front-matter and is ungated. The LAION-Audio-630K training data it was " + "fitted on carries its own terms, and whether those reach through to " + "weights is the same unsettled question the Grounding DINO entry above " + "records for Objects365; not investigated here either." + ), +)) + + +# --- Segmentation (ยง3 SAM 3.1). ---------------------------------------------- + +register(RuntimeLicence( + runtime="sam2-onnx", + licence="Apache-2.0", + source_url="https://github.com/facebookresearch/sam2/blob/main/LICENSE", + verified_on="2026-08-21", + permits_commercial_use=True, + permits_network_service=True, + note=( + "SAM 2.1 is the last Apache-2.0 generation, and " + "`facebook/sam2.1-hiera-tiny` is 39.0M parameters and ungated. It does " + "promptable segmentation only โ€” no concept prompting โ€” which is the " + "capability ยง3 wants SAM 3 for. Named here as the variant that fits the " + "container, not as an equivalent." + ), +)) + +register(RuntimeLicence( + runtime="sam3", + licence="SAM License", + source_url="https://huggingface.co/facebook/sam3/resolve/main/LICENSE", + verified_on="2026-08-21", + permits_commercial_use=True, + permits_network_service=True, + gated=True, + note=( + "Bespoke Meta licence dated 19 November 2025, covering code and weights " + "alike โ€” `facebookresearch/sam3` is SPDX NOASSERTION, not Apache-2.0 as " + "SAM 2 was. No non-commercial clause, no acceptable-use policy, no " + "monthly-user trigger. Clause 1.b.i binds distribution or making the " + "materials `available to a third party`, which serving inference " + "outputs is normally not, but the wording is broader than " + "`distribute` and unadjudicated. Section 8 permits unilateral " + "amendment with immediate effect, so archive the text you accepted. " + "Gated with manual approval, and a build needs an HF token. **The " + "licence is not what stops this being deployed โ€” the size is.** See " + "`adapters/segmentation.py`." + ), +)) + + +# --- Hosted multimodal reasoning (ยง4). --------------------------------------- + +register(RuntimeLicence( + runtime="hosted-multimodal", + licence="vendor terms of service", + source_url="", + verified_on="2026-08-21", + permits_commercial_use=True, + permits_network_service=True, + note=( + "No weights arrive, so there is no artefact licence to check and this " + "table cannot do its usual job. What governs is the vendor's API terms " + "and its data-retention posture, and both are procurement questions " + "rather than loader questions. Recorded so the runtime is known rather " + "than refused as unrecognised, and deliberately *not* recorded as " + "cleared: nobody has read a specific vendor's terms here." + ), +)) + + +# --- Quadruped pose (ยง24). ---------------------------------------------------- +# +# Neither has run. Both are here because the licence question is decided long +# before the model is, and because ยง24 names the first one first โ€” so whoever +# picks up cattle gait meets the commercial obstacle in the ledger rather than +# discovering it after building on it. + +register(RuntimeLicence( + runtime="superanimal-quadruped-dlc", + licence="Modified MIT (academic and non-commercial use only)", + source_url=( + "https://huggingface.co/mwmathis/DeepLabCutModelZoo-SuperAnimal-Quadruped" + ), + verified_on="2026-08-22", + # **The two fields that matter, and the reason this entry exists.** Animap + # is a closed-source commercial product; this model card restricts use to + # academic and non-commercial purposes and offers commercial licensing on + # application to Prof. Mackenzie W. Mathis or EPFL's technology transfer + # office. That is not a reason to skip evaluating it โ€” the founder's + # standing instruction is that no model is excluded on licence today โ€” it is + # a reason for the position to be written down where a sale would find it. + permits_commercial_use=False, + permits_network_service=False, + gated=False, + note=( + "ยง24 names this model first for cattle gait: zero-shot quadruped pose " + "over 39 bodyparts, trained on 40k+ images. Its 39-bodypart vocabulary " + "is mapped in `app/adapters/pose/vocabulary.py` and " + "`app/adapters/pose/gait.py` consumes its output unchanged, so the only " + "thing between here and running it is video โ€” not code and not a " + "licence decision, which has not been taken. The card also forbids using " + "the model to harm any animal deliberately. **`servable` is False, so " + "`gate` refuses it under the default policy**; that is the correct " + "state for a model whose terms exclude the product's own business " + "model, and it should be revisited by asking EPFL rather than by " + "editing this line. Cite Ye et al., arXiv:2203.07436." + ), +)) + +register(RuntimeLicence( + runtime="vitpose-hf", + licence="Apache-2.0", + source_url="https://huggingface.co/usyd-community/vitpose-base-simple", + verified_on="2026-08-22", + permits_commercial_use=True, + permits_network_service=True, + note=( + "ยง36's required alternative to SuperAnimal, and **the shippable one**: " + "permissive, already installed in this service's virtualenv via " + "`transformers`, and runnable with no CUDA extension. Its AP-10K " + "vocabulary covers 54 species including cattle and is mapped in " + "`app/adapters/pose/vocabulary.py`. It supplies 11 of the 12 landmarks " + "gait needs โ€” AP-10K has no mid-back keypoint, so ยง24's back-line " + "movement is reported *unavailable* on it rather than zero. Not " + "measured: `experiments/cattle_gait` never ran a pose model, because no " + "openly licensed side-on cattle walking video with a locomotion score " + "could be obtained. AP-10K itself is CC BY 4.0, which is a fact about " + "the training data rather than about these weights." + ), +)) diff --git a/app/adapters/multimodal.py b/app/adapters/multimodal.py new file mode 100644 index 0000000000000000000000000000000000000000..40da0db780459ed4ce7cf653d56ace619efad526 --- /dev/null +++ b/app/adapters/multimodal.py @@ -0,0 +1,322 @@ +"""The hosted multimodal reasoner, and the rails it runs inside. + +Directive ยง4 asks for a Gemini-class model for BCS rubrics, dentition, wound +description, skin and hoof and footpad triage, breed suggestion, litter +condition, heat-stress signs and egg quality. It says two things about it that +are not decoration, and both are enforced here in code rather than in a prompt: + +> All calls must return structured JSON. + +> The multimodal model is an **experimental visual reasoner**, not an authority. + +**The second one is why this file is longer than an HTTP call.** A hosted model +will cheerfully write "lumpy skin disease confirmed" into a field called +`diagnosis`, and ยง10 forbids exactly that sentence. A prompt asking it not to is +a request; `app/adapters/claims.py` is a control, and everything about how it +works and why it is shaped that way lives in that module's own docstring. + +**This file used to claim a control it did not have.** The paragraph here said +every response was checked against `app/capabilities.FORBIDDEN_CLAIMS` and the +observation/interpretation split from `app/schemas.py`. The first was true and +useless โ€” `FORBIDDEN_CLAIMS` is 29 snake_case registry keys, and no hosted model +writes `lsd_diagnosis` in prose. The second was not true at all: this module +never imported `app/schemas.py`. What actually stood between a model and a +published diagnosis was a tuple of seven hard-coded sentences, and a watchdog +published 24 evasions out of 24 attempts through it โ€” a doubled space, a +Cyrillic `ั`, a zero-width character, ยง10's own sentence with the words +reordered, `best_estimate: 2.6347`, and `NaN`. + +`claims.enforce` replaces it and `claims.schema_for` builds the schema from the +capability, so the closed vocabulary a model is handed and the closed vocabulary +its answer is judged against are the same object. + +**No key is committed and none may be.** The adapter reads +`ANIMAP_MULTIMODAL_API_KEY` from the environment at call time, and its absence is +an honest `unavailable`, not a fallback to something plausible. On Azure +Container Apps that is a secret reference, the same posture `services/api` +already uses. + +**This adapter chooses no vendor, and that is deliberate rather than +unfinished.** `_transport` is injected; `app/adapters/transports/` holds the +implementations and `registry.py` passes whichever one `ANIMAP_MULTIMODAL_PROVIDER` +names. With none named the adapter reports itself unavailable and says so. + +Choosing a vendor commits Animap to a data-retention posture for photographs of +somebody's animals leaving the country, which is a procurement decision and not +one an adapter should make by importing an SDK. Keeping the transport injectable +is also what makes this file testable without a network and without a key: every +test below passes a callable. +""" + +from __future__ import annotations + +import os +from dataclasses import dataclass +from typing import Any, Callable + +from PIL import Image + +from app.adapters import claims +from app.adapters.base import ( + Adapter, + AdapterError, + AdapterSpec, + AdapterUnavailable, + Availability, + Modality, + Placement, + Task, +) +from app.capabilities import Capability + +# Re-exported rather than merely imported. `ContractViolation` was defined here +# before the contract moved to `claims.py`, and callers โ€” including +# `tests/test_adapters.py` โ€” import it from this module. Moving the definition +# without keeping the name would be a rename disguised as a refactor. +from app.adapters.claims import ContractViolation # noqa: F401 + +#: The environment variable this expects. Named here so a deployment can be +#: configured without reading the code, and so nobody invents a second name. +API_KEY_ENV = "ANIMAP_MULTIMODAL_API_KEY" + +#: Which hosted model to call. Kept in the environment rather than in code +#: because a result must name the version that produced it โ€” `schemas` +#: requires `model_version` โ€” and a hosted model's version changes without +#: anyone here editing a file. +MODEL_ENV = "ANIMAP_MULTIMODAL_MODEL" + +#: Prefixed to every rubric. It is not a safety incantation โ€” the checks below +#: are what actually hold โ€” but a model told the shape of the answer produces +#: fewer responses that have to be thrown away. +SYSTEM_RULES = """\ +You are a visual observer for a livestock record system. You are not a +veterinarian and your output is not a diagnosis. + +Return one JSON object matching the supplied schema and nothing else. No prose, +no markdown fence, no commentary. + +Report only what is VISIBLE in the images. Separate observation from +interpretation: an observation is a fact about the picture ("multiple raised +nodular lesions on the flank"), an interpretation is what it might mean +("abnormal skin pattern, veterinary review recommended"). + +Never name a disease unless you qualify it in the same sentence โ€” "a pattern +consistent with X", "could be X", "inconclusive for X". An unqualified name is a +diagnosis whatever the surrounding words, and it is discarded with the whole +response. Never state a body temperature, a pregnancy status, an exact weight, +an exact whole-house population, a cause for a wound, that a bird is dead rather +than resting, or a sex, breed or identity as settled. If the image does not +support a field, set it to null and say why in `limits`. A null is a correct +answer and a guess is not. + +Make claims only by choosing identifiers from the schema's own `claims` list. +Do not invent a field. Any key the schema does not declare is discarded with the +whole response. + +`evidence` and `limits` are closed lists, not free text. Choose the strings the +schema offers and change none of them. Do not paraphrase, do not combine two +into one, and do not add a word. If nothing in the list describes what you can +see, choose the limit that says so and let a person look; a response containing +a string the schema does not offer is discarded whole. + +Put every number in a structured field โ€” `range`, `best_estimate`, or an +observation's `value`. Never write a number into `evidence` or `limits`: the +strings there are fixed, and any figure a farmer reads has to come from a field +the capability declared it can measure. + +Where the schema asks for a range, give a range. Do not narrow it to look +precise.\ +""" + + +@dataclass(frozen=True) +class ReasonerResponse: + """What came back, and what it cost. + + `raw` is kept verbatim beside the parsed object because ยง33 requires the + model result to be preserved for later training, and a reparsed + reconstruction is not what the model said. + """ + + parsed: dict[str, Any] + raw: str + model: str + warnings: tuple[str, ...] = () + + +class HostedMultimodalAdapter(Adapter): + """A hosted reasoner behind a structured-output contract. + + Construct it with a `transport` โ€” a callable taking `(prompt, images, + schema, model, api_key)` and returning the response text. There is no + default, so this adapter cannot silently start talking to a vendor nobody + chose. + """ + + spec = AdapterSpec( + adapter_id="hosted-multimodal", + runtime="hosted-multimodal", + tasks=(Task.REASON,), + modalities=(Modality.IMAGE, Modality.VIDEO, Modality.AUDIO), + directive_role=( + "ยง4 hosted multimodal โ€” BCS rubric scoring, dentition " + "interpretation, wound description, skin/hoof/footpad triage, breed " + "suggestion, litter condition, heat-stress signs, egg external " + "quality, structured evidence extraction. ยง4: 'an experimental " + "visual reasoner, not an authority'." + ), + requires_artefact=False, + placement=Placement.CPU_SERVICE, + placement_reason=( + "No weights run here, so it costs the container a socket. It is " + "also the one leg that can never move to the phone: ADR 0002 makes " + "Animap offline-first, and every capability built on this one is a " + "capability a farm cannot use in a shed with no signal. That is a " + "connectivity tier, not a reason to drop it." + ), + notes=( + "Unmeasured, because no vendor is wired and no key is configured. " + "Latency and cost per call are the vendor's and must be measured " + "against a real account before any capability depends on it." + ), + ) + + def __init__( + self, + transport: Callable[..., str] | None = None, + *, + key_env: str = API_KEY_ENV, + model_env: str = MODEL_ENV, + ) -> None: + self._transport = transport + self._key_env = key_env + self._model_env = model_env + + # Read at call time rather than at import, so a secret rotation takes effect + # on restart rather than needing a rebuild โ€” the same reasoning as + # `main._configured_token`. + def _api_key(self) -> str: + return os.environ.get(self._key_env, "") + + def _model(self) -> str: + return os.environ.get(self._model_env, "") + + def availability(self) -> Availability: + if self._transport is None: + from app.adapters.transports import PROVIDER_ENV, PROVIDERS + + return Availability( + False, + "No hosted multimodal vendor is chosen for this deployment.", + f"Choosing one commits Animap to a retention posture for farm " + f"photographs, so it is a procurement decision rather than a " + f"deployment step. Set {PROVIDER_ENV} once it is made; this " + f"build can serve: {', '.join(sorted(PROVIDERS))}.", + ) + if not self._api_key(): + return Availability( + False, + f"{self._key_env} is not set, so the reasoner cannot be called.", + f"Set {self._key_env} as a Container App secret. Never commit " + f"it, and never add it to a provisioning script in this repo.", + ) + if not self._model(): + return Availability( + False, + f"{self._model_env} is not set, so a result could not name the " + f"model that produced it.", + f"Set {self._model_env} to the exact hosted model id. " + f"InferenceResult requires model_version, and the API refuses " + f"to store a run that cannot name its artefact.", + ) + return Availability(True) + + def load(self) -> "HostedMultimodalAdapter": + availability = self.availability() + if not availability.ready: + raise AdapterUnavailable(availability) + return self + + def reason( + self, + images: list[Image.Image], + *, + rubric: str, + capability: Capability | None = None, + schema: dict | None = None, + ) -> ReasonerResponse: + """One call, one JSON object, checked before it is returned. + + Pass `capability` and the schema is built from the registry, so the + vocabulary the model is handed is the vocabulary its answer is judged + against and the two cannot drift. Pass `schema` to override it โ€” the + override is still hardened, still validated, and still scanned, because + a caller supplying a permissive schema must not be a way around the + checks. + + **Passing neither a capability nor a schema is a refusal.** There is no + default open schema: an answer nobody declared a shape for is an answer + nobody can review. + """ + availability = self.availability() + if not availability.ready: + raise AdapterUnavailable(availability) + assert self._transport is not None + + if not images: + raise AdapterError("A visual reasoner needs at least one image.") + if capability is None and schema is None: + raise AdapterError( + "reason() needs a capability or a schema. A hosted model asked " + "for an answer with no declared shape can put a claim in any " + "field it invents, which is the failure this adapter exists to " + "prevent." + ) + + contract = schema if schema is not None else claims.schema_for(capability) + + prompt = f"{SYSTEM_RULES}\n\n{rubric}" + raw = self._transport( + prompt=prompt, + images=images, + schema=contract, + model=self._model(), + api_key=self._api_key(), + ) + + # Not repaired, not retried with a nudge, and not partially accepted. + # ยง4 says the calls must return structured JSON; one that did not is a + # failed call, and scraping an object out of prose is how a malformed + # answer becomes a stored result. + parsed = claims.parse_strict(raw) + claims.enforce(parsed, contract, capability=capability) + + warnings = [ + "This reading came from a general-purpose hosted model that has " + "never been trained on livestock, and it is experimental. A vet " + "confirms it." + ] + return ReasonerResponse( + parsed=parsed, raw=raw, model=self._model(), warnings=tuple(warnings) + ) + + +# **`BCS_SCHEMA` used to be here, and it is gone rather than repaired.** +# +# It was ยง7's body-condition contract written by hand: `minimum: 1.0`, +# `maximum: 5.0`, `multipleOf: 0.5` on `best_estimate` and on `range.items` โ€” +# the only bounded numbers anywhere in this service, and the reason the audit +# that found `best_estimate: 2.6347` could point at something that did work. +# +# It had no production caller. `reason(capability=cattle_bcs)` builds its +# contract from `claims.schema_for` on the line above, and the only imports of +# `BCS_SCHEMA` were in `tests/test_adapters.py` and `tests/test_claims.py`. **A +# control referenced only by its own tests is worse than none**, because a green +# suite reads as coverage of the path a farmer's result actually takes, and this +# one covered a path nothing takes. +# +# Deleting it costs nothing now that `schema_for` reads `OutputSpec`: for +# `cattle_bcs` it emits the same three keywords on the same two fields, from the +# registry rather than from a copy, and adds the closed `claims` vocabulary that +# the hand-written version never had. The tests that imported it now build the +# generated schema, so they exercise what production exercises. diff --git a/app/adapters/pose/__init__.py b/app/adapters/pose/__init__.py new file mode 100644 index 0000000000000000000000000000000000000000..3670d3ddfecb942d67a4b159cb6b11d402d10ad7 --- /dev/null +++ b/app/adapters/pose/__init__.py @@ -0,0 +1,142 @@ +"""Quadruped pose to gait features: ยง24 from the keypoints onward. + +The pose model is not here. What is here is everything a pose model feeds: +a keypoint vocabulary that more than one model can be mapped into +(`vocabulary`), a body frame that removes the animal's translation and its +changing apparent size (`tracks`), and the ยง24 features plus a screen that +cannot express a diagnosis (`gait`). + +That boundary is drawn where it is because of the licences, which are the part of +ยง24 most likely to be discovered late. + +## Licence positions, recorded because they decide what can be sold + +**DeepLabCut SuperAnimal-Quadruped โ€” Modified MIT, academic and non-commercial +use only.** The model card on Hugging Face states it plainly: use is restricted +to academic and non-commercial purposes, the model "may not be used to harm any +animal deliberately", and commercial licensing is available on application to +Prof. Mackenzie W. Mathis or EPFL's technology transfer office. ยง24 names this +model first, and nothing about that is a reason not to evaluate it โ€” but Animap +is a commercial product, so shipping it would need that licence. Read on +2026-08-22 from +`https://huggingface.co/mwmathis/DeepLabCutModelZoo-SuperAnimal-Quadruped`. + +**ViTPose โ€” Apache-2.0 for the code and the Hugging Face ports.** The AP-10K +vocabulary it can be run against covers 54 species including cattle. It is the +alternative ยง36 requires to be tried, and it is the one with no commercial +obstacle. + +`experiments/cattle_gait/README.md` records which of these actually ran and what +it produced. This docstring records only what their terms say. +""" + +from __future__ import annotations + +from app.adapters.base import ( + Adapter, + AdapterSpec, + Availability, + Modality, + Placement, + Task, +) +from app.adapters.pose.gait import ( + GaitFeatures, + GaitScreen, + LimbCycle, + PairAsymmetry, + STATEMENTS, + Verdict, + features, + screen, +) +from app.adapters.pose.tracks import ( + BodyFrame, + PoseSequence, + TrackUnusable, + body_frame, + in_body_frame, +) +from app.adapters.pose.vocabulary import ( + AP10K, + AP10K_ORDER, + CONTRALATERAL_PAIRS, + SUPERANIMAL_QUADRUPED, + VOCABULARIES, + Landmark, + missing, +) + +GAIT_FEATURES_SPEC = AdapterSpec( + adapter_id="gait-features", + runtime="opencv-numpy", + tasks=(Task.MEASURE,), + modalities=(Modality.VIDEO,), + directive_role=( + "ยง24 cattle gait โ€” stride timing, left/right symmetry, hoof " + "trajectories, back-line movement, head movement and stance duration, " + "derived from quadruped keypoint tracks. The pose model itself is a " + "separate adapter; this is the deterministic half ยง4 prefers." + ), + requires_artefact=False, + placement=Placement.CPU_SERVICE, + placement_reason=( + "Gradients and one FFT over a few hundred frames of keypoints โ€” " + "microseconds. It sits wherever the pose model does, and the pose model " + "is the thing that needs a GPU." + ), + notes=( + "**No clinical threshold exists and none is offered.** `screen()` " + "requires the caller to supply the asymmetry index it judges against, " + "because deriving one needs cattle a vet locomotion-scored and this " + "code has never seen any. What `experiments/cattle_gait` measures " + "instead is the method's own resolution โ€” the smallest asymmetry it can " + "separate from its noise โ€” which is a different claim and is labelled " + "as one." + ), +) + + +class GaitFeatureAdapter(Adapter): + """ยง24 from the keypoints onward. Always available; never diagnoses.""" + + spec = GAIT_FEATURES_SPEC + + def availability(self) -> Availability: + return Availability(True) + + def load(self) -> "GaitFeatureAdapter": + return self + + def measure(self, sequence: PoseSequence) -> GaitFeatures: + return features(sequence) + + def screen(self, sequence: PoseSequence, *, threshold: float, + threshold_basis: str = "unstated") -> GaitScreen: + return screen(sequence, threshold=threshold, threshold_basis=threshold_basis) + + +__all__ = [ + "AP10K", + "AP10K_ORDER", + "CONTRALATERAL_PAIRS", + "GAIT_FEATURES_SPEC", + "STATEMENTS", + "SUPERANIMAL_QUADRUPED", + "VOCABULARIES", + "BodyFrame", + "GaitFeatureAdapter", + "GaitFeatures", + "GaitScreen", + "Landmark", + "LimbCycle", + "PairAsymmetry", + "PoseSequence", + "TrackUnusable", + "Verdict", + "body_frame", + "features", + "in_body_frame", + "missing", + "screen", +] diff --git a/app/adapters/pose/gait.py b/app/adapters/pose/gait.py new file mode 100644 index 0000000000000000000000000000000000000000..35f0586882408e3a69a179e691bd22694c4a59d4 --- /dev/null +++ b/app/adapters/pose/gait.py @@ -0,0 +1,566 @@ +"""Gait features from keypoint tracks, and a screen that refuses to diagnose. + +Directive ยง24 asks for stride timing, left/right symmetry, hoof trajectories, +back-line movement, head movement and stance duration, and it gives the exact +sentence the output may not be: + +> Lameness score 3 caused by left rear hoof disease. + +That sentence is forbidden three times over โ€” it scores, it localises and it +attributes a cause โ€” and none of the three is a thing this pipeline could know. +So `GaitScreen` has no score field, no limb field and no cause field. There is +nothing to fill in, which is a stronger guarantee than a rule about what to fill +in with. The permitted wording is a fixed map from a three-valued verdict; a +caller cannot compose a sentence out of the features because the features are +numbers with no words attached. + +## What is measured, and in what units + +Everything is in the body frame `tracks` establishes โ€” trunk lengths and +seconds โ€” so nothing here needs a metric scale. That is the structural reason +gait is a more tractable capability than ยง22's weight: an asymmetry is a ratio, +and a ratio survives not knowing how big the animal is. + +**Asymmetry indices are unsigned.** `|left โˆ’ right| / mean`, never `left โˆ’ +right`. A signed index invites the reader to name a limb, and naming the limb is +half of the forbidden sentence. + +## The threshold problem, stated rather than solved + +Whether an asymmetry index of 0.12 means anything about a cow is a clinical +question, and answering it needs cattle whose locomotion a vet scored. This +module has never seen one. What it can establish without them is its own +**resolution**: the smallest asymmetry it can tell apart from its own noise, on +sequences where the true asymmetry is known because it was constructed. That is +what `RESOLUTION_FLOOR` holds, it is measured in `experiments/cattle_gait`, and +it is emphatically not a clinical threshold. + +`screen()` therefore returns `ASYMMETRY_OBSERVED` only for an index above a +threshold the caller supplies. There is no default, and there is no module-level +constant a caller can reach for and mistake for a validated one. +""" + +from __future__ import annotations + +from dataclasses import dataclass, field +from enum import Enum + +import numpy as np + +from app.adapters.pose.tracks import ( + PoseSequence, + TrackUnusable, + body_frame, + in_body_frame, +) +from app.adapters.pose.vocabulary import CONTRALATERAL_PAIRS, Landmark +from app.adapters.signal.periodicity import dominant_rate + +#: Search band for stride rate, in strides per minute for one limb. A cow at a +#: walk completes roughly 0.6โ€“1.3 strides per second. The band is deliberately +#: wider than that on both sides so the method is not a prior pulling the +#: estimate towards what a textbook says a cow does. +STRIDE_BAND_PER_MINUTE = (25.0, 120.0) + +#: **The respiratory module's thresholds do not transfer to this band and are +#: not reused.** `periodicity.MIN_PEAK_PROMINENCE` was derived at (8, 180) +#: cycles per minute against a metronome and camera noise; prominence is peak +#: power over the *in-band* median, so it is a property of the band. These are +#: separate constants, set from what a hoof trace looks like rather than +#: inherited, and they are weaker gates because a swinging limb is a far +#: stronger oscillation than a breathing flank. +STRIDE_MIN_PROMINENCE = 8.0 +STRIDE_MAX_HALF_DRIFT = 0.25 +STRIDE_MIN_CYCLES = 3.0 + +#: Fractional change in apparent trunk length above which the capture is refused. +#: A walk perpendicular to the camera holds the trunk's apparent length nearly +#: constant; one angled towards the lens does not, and the foreshortening lands +#: unevenly on the two sides of the animal. That produces an asymmetry in a sound +#: cow, which is the worst failure this capability has. +#: +#: **Not derived from cattle footage.** 0.25 is a geometric argument โ€” a 25% +#: change in apparent length is roughly a 25% change in distance, at which point +#: near-side and far-side limbs are being measured at materially different +#: scales. It should be re-derived the first time there is real footage. +MAX_TRUNK_DRIFT = 0.25 + +#: Width of the local-polynomial window used to differentiate a hoof's +#: body-frame position, as a fraction of that limb's own measured stride period. +#: +#: **Scaled to the stride rather than fixed in frames**, so the same constant +#: works at 15 fps and at 30, and on a slow walk and a brisk one. A fixed frame +#: count silently means something different in each of those cases. +#: +#: ## Why differentiation needs a window at all +#: +#: Stance is separated from swing by the sign of the hoof's velocity relative to +#: the trunk, and that velocity is small: at 30 fps, a 0.9 Hz stride and a +#: stride length of 0.55 trunk lengths, a planted hoof moves 0.0165 trunk +#: lengths per frame. Five pixels of keypoint noise on a 400 px trunk is 0.0125 +#: trunk lengths, and a three-point difference of that noise is **larger than +#: the signal being measured**. The first version of this module used +#: `np.gradient` with a three-frame moving average and reported a 24% left-right +#: asymmetry on a perfectly symmetric walk at that noise level. +#: +#: ## How 0.35 was chosen +#: +#: **Regenerate this table rather than trusting it:** +#: +#: python -m experiments.cattle_gait.run --arm window-derivation +#: +#: The rule, fixed before the sweep ran: the **smallest** fraction that +#: separates a 12% injected asymmetry from a sound walk at **every** keypoint +#: noise level tested, where "separates" means the lame 5th percentile clears +#: the sound 95th. Protocol: seeds **100โ€“111**, which are disjoint from the +#: `SEEDS = (0, 1, 2, 3, 4)` the benchmark runs on; hind pair; stance index; +#: 30 fps; `stance_reduction = stride_reduction = 0.12`. Margins as +#: `lame p5 โˆ’ sound p95`, positive separates: +#: +#: | fraction | 0 px | 2 px | 5 px | 10 px | separates everywhere | +#: |---|---|---|---|---|---| +#: | 0.10 | +0.1593 | โˆ’0.1731 | โˆ’0.4312 | โˆ’0.1697 | no | +#: | 0.15 | +0.1651 | +0.1416 | โˆ’0.1490 | โˆ’0.4063 | no | +#: | 0.20 | +0.1853 | +0.1398 | +0.0993 | โˆ’0.4075 | no | +#: | 0.25 | +0.1510 | +0.1148 | +0.1008 | โˆ’0.1693 | no | +#: | 0.30 | +0.1510 | +0.1003 | +0.0748 | โˆ’0.0214 | no | +#: | **0.35** | **+0.1154** | **+0.0737** | **+0.0674** | **+0.0326** | **yes** | +#: | 0.40 | +0.1154 | +0.0737 | +0.0674 | +0.0326 | yes | +#: | 0.50 | +0.0582 | +0.0395 | +0.0179 | +0.0149 | yes | +#: +#: 0.40 gives an identical row because the rounded window is the same number of +#: frames; 0.35 is the smaller of the two and is taken. +#: +#: **The `--arm window-derivation` command exists because a hostile audit could +#: not reproduce this table from the paragraph above it.** Its best-matching +#: reconstruction put the load-bearing 0.35/10 px cell at +0.012 against the +#: published +0.033, and sixteen other plausible readings of the protocol made +#: it negative. The numbers were right; what was missing was enough of the +#: protocol to re-derive them, which is the same failure as not having them. +#: +#: The derivation set is disjoint from the benchmark set, so the published +#: figures are not a training score. It is still **synthetic on both sides**, +#: and a window chosen against constructed kinematics has no claim on real +#: footage. +#: +#: ## What it costs +#: +#: 0.35 of a stride period is nearly as long as the swing phase itself, so the +#: stance-to-swing transition is blurred and a stance duration is resolved to +#: roughly a tenth of a cycle rather than to a frame. That is the trade being +#: made: the sign of the velocity survives noise, and the exact instant of +#: touchdown does not. Nothing here should be read at single-frame resolution. +STANCE_WINDOW_FRACTION = 0.35 + +#: Order of the local polynomial fitted inside that window. Quadratic rather +#: than linear because a hoof's body-frame trace curves through the swing, and a +#: straight-line fit over a third of a cycle would bias the derivative towards +#: the window's mean slope โ€” which is zero over a whole cycle. +STANCE_POLYNOMIAL_ORDER = 2 + + +class Verdict(str, Enum): + """The only three things this capability may conclude. + + Note what is not here: a grade, a limb, a cause, a severity. ยง24 names all + four in the sentence it forbids. + """ + + ASYMMETRY_OBSERVED = "asymmetry_observed" + NO_ASYMMETRY_OBSERVED = "no_asymmetry_observed" + NOT_ASSESSABLE = "not_assessable" + + +#: The permitted wording, fixed. A caller renders `screen().statement`; there is +#: no path by which a feature value becomes part of a sentence. +STATEMENTS: dict[Verdict, str] = { + Verdict.ASYMMETRY_OBSERVED: "Possible gait asymmetry", + Verdict.NO_ASYMMETRY_OBSERVED: ( + "No gait asymmetry observed in this recording" + ), + Verdict.NOT_ASSESSABLE: "Gait could not be assessed from this recording", +} + + +@dataclass(frozen=True) +class LimbCycle: + """One limb's stride, as measured. All times in seconds. + + `usable` false means the periodicity gates refused this limb, and every + figure below it should be read as diagnostic rather than as a measurement. + """ + + landmark: str + usable: bool + reason: str = "" + strides_per_minute: float | None = None + #: Fraction of the cycle the hoof spends moving backwards relative to the + #: trunk, which is the definition of stance used here. Cattle at a walk sit + #: around 0.6; a figure near 0.5 or above 0.8 suggests the separation failed + #: rather than that the animal is unusual. + stance_fraction: float | None = None + #: Mean duration of one stance phase. + stance_seconds: float | None = None + #: Peak-to-peak excursion along the body axis, in trunk lengths. + stride_length_trunks: float | None = None + #: Peak-to-peak excursion across the body axis, in trunk lengths. Hoof lift. + hoof_lift_trunks: float | None = None + peak_prominence: float = 0.0 + cycles_observed: float = 0.0 + + +@dataclass(frozen=True) +class PairAsymmetry: + """Left against right, for one pair of limbs. Unsigned, always. + + Every index is `|left โˆ’ right| / mean`, so 0 is symmetric and 0.2 means the + two sides differ by a fifth of their average. `None` where the underlying + limb measurement was refused. + """ + + pair: str + stance_index: float | None + stride_length_index: float | None + #: Contralateral phase offset as a fraction of a stride, folded so that 0 + #: means perfectly anti-phase (the sound pattern at a walk) and 1 means the + #: two limbs move together. + phase_index: float | None + left: LimbCycle + right: LimbCycle + + @property + def worst(self) -> float | None: + values = [v for v in (self.stance_index, self.stride_length_index, + self.phase_index) if v is not None] + return max(values) if values else None + + +@dataclass(frozen=True) +class GaitFeatures: + """Everything ยง24 asks to be derived, plus what could not be. + + `unavailable` names the features the model's vocabulary or the capture made + impossible. It is a list of reasons rather than a set of nulls, because a + null reads as zero to whoever writes the next summary. + """ + + pairs: tuple[PairAsymmetry, ...] + #: Peak-to-peak vertical excursion of the nose in trunk lengths. Head nodding + #: is a long-established lameness sign; the amplitude is reported and nothing + #: is concluded from it, because no threshold for it has been measured here. + head_movement_trunks: float | None + #: Deviation of the mid-back from the neck-to-tail line, in trunk lengths: + #: mean, then how much it varies over the clip. An arched back is another + #: established sign, and the same absence of a threshold applies. + back_arch_mean_trunks: float | None + back_arch_variation_trunks: float | None + trunk_drift: float + duration_seconds: float + fps: float + unavailable: tuple[str, ...] = field(default_factory=tuple) + + @property + def worst_asymmetry(self) -> float | None: + values = [p.worst for p in self.pairs if p.worst is not None] + return max(values) if values else None + + +@dataclass(frozen=True) +class GaitScreen: + """What the product may show, and the evidence under it. + + There is no `score`, no `limb` and no `cause`. `statement` reads from + `STATEMENTS` by verdict and interpolates nothing. + """ + + verdict: Verdict + features: GaitFeatures | None + reason: str + #: The asymmetry index the caller judged against, carried so a stored result + #: says what it was compared to. `None` on `NOT_ASSESSABLE`. + threshold: float | None = None + #: Whether that threshold came from measured cattle or from somewhere else. + #: There is no default: a caller that does not say gets "unstated", which is + #: what a reader needs to see. + threshold_basis: str = "unstated" + + @property + def statement(self) -> str: + return STATEMENTS[self.verdict] + + +def _derivative(signal: np.ndarray, window: int, + order: int = STANCE_POLYNOMIAL_ORDER) -> np.ndarray: + """First derivative by a sliding local polynomial fit (Savitzkyโ€“Golay). + + Fitting a polynomial across a window and reading its slope is far more + robust to independent per-frame noise than differencing neighbours, because + every sample in the window constrains the fit. The coefficients are the + second row of the pseudo-inverse of the Vandermonde matrix โ€” the row that + recovers the linear term โ€” so this is a plain least-squares fit written as a + convolution, with no SciPy dependency. + + Edges are handled by repeating the end samples. That biases the derivative + towards zero in the first and last half-window, which is why `_limb` drops + the first and last stance run before averaging. + """ + # Forced odd. Defensive rather than load-bearing: `half` floors, so a + # window of 8 builds the same 9-tap kernel as a window of 9. What it + # does buy is that the two comparisons below test the width actually + # used. `tests/test_gait.py` records the equivalence. + window = int(window) | 1 + if window < order + 2 or signal.size < window: + return np.gradient(signal) + half = window // 2 + offsets = np.arange(-half, half + 1, dtype=np.float64) + design = np.vander(offsets, order + 1, increasing=True) + coefficients = np.linalg.pinv(design)[1] + padded = np.pad(signal, half, mode="edge") + return np.convolve(padded, coefficients[::-1], mode="valid") + + +def _limb(sequence: PoseSequence, landmark: Landmark) -> LimbCycle: + """Stride rate, stance fraction and excursions for one hoof.""" + path = in_body_frame(sequence, landmark) + if path is None: + return LimbCycle( + landmark=landmark.value, usable=False, + reason=( + f"{landmark.value} was not tracked in enough frames, or the " + f"model has no keypoint for it." + ), + ) + + along, across = path[:, 0], path[:, 1] + rate = dominant_rate( + along, sequence.fps, STRIDE_BAND_PER_MINUTE, + min_prominence=STRIDE_MIN_PROMINENCE, + max_drift=STRIDE_MAX_HALF_DRIFT, + min_cycles=STRIDE_MIN_CYCLES, + # The sub-band shoulder gate is disabled here and that is a deliberate + # difference from respiration. A hoof's body-frame trace is a sawtooth, + # not a sinusoid, and a sawtooth's own harmonic structure plus the + # residual of an imperfectly removed trend put real power below the + # band. Leaving the gate on refused sound synthetic walks. What replaces + # it is the trunk-drift check, which catches the same underlying + # problem โ€” a capture that is drifting rather than cycling โ€” at its + # source rather than in the spectrum. + max_shoulder=float("inf"), + ) + + # Stance and swing, separated by the sign of the body-frame velocity. In + # stance the hoof is planted, so relative to a forward-moving trunk it + # travels backwards; in swing it overtakes the trunk. The zero crossing is + # the natural boundary and needs no threshold to be chosen. + # + # The differentiating window is scaled to this limb's own measured stride. + # When the rate was refused there is no period to scale by, so the window + # falls back to a fifth of the clip's own length โ€” the result is refused + # either way, and the figures below are diagnostics rather than a + # measurement. + period_frames = (sequence.fps * 60.0 / rate.cycles_per_minute + if rate.cycles_per_minute else along.size / 5.0) + window = max(3, int(round(STANCE_WINDOW_FRACTION * period_frames))) + velocity = _derivative(along, window) + in_stance = velocity < 0 + stance_fraction = float(in_stance.mean()) + + # Mean length of a stance run, in seconds. Computed from the runs rather + # than as `stance_fraction ร— period`, so a limb that takes one very long + # stance and three short ones is distinguishable from one taking four even + # ones. The first and last runs are dropped because the clip truncates them + # and a truncated stance reads as a short one. + edges = np.flatnonzero(np.diff(in_stance.astype(np.int8))) + runs: list[int] = [] + if edges.size >= 2: + boundaries = np.concatenate(([0], edges + 1, [in_stance.size])) + for start, end in zip(boundaries[:-1], boundaries[1:]): + if in_stance[start]: + runs.append(int(end - start)) + runs = runs[1:-1] if len(runs) > 2 else runs + stance_seconds = float(np.mean(runs) / sequence.fps) if runs else None + + return LimbCycle( + landmark=landmark.value, + usable=rate.usable, + reason=rate.reason, + strides_per_minute=round(rate.cycles_per_minute, 2) if rate.cycles_per_minute else None, + stance_fraction=round(stance_fraction, 4), + stance_seconds=round(stance_seconds, 4) if stance_seconds else None, + stride_length_trunks=round(float(along.max() - along.min()), 4), + hoof_lift_trunks=round(float(across.max() - across.min()), 4), + peak_prominence=round(rate.peak_prominence, 2), + cycles_observed=round(rate.cycles_observed, 2), + ) + + +def _index(left: float | None, right: float | None) -> float | None: + """`|l โˆ’ r| / mean`, or `None` when either side is missing. + + Unsigned on purpose โ€” see this module's docstring. Returns `None` rather + than 0 when the mean is zero, because two limbs that both measured zero have + not been shown to be symmetric, they have not been measured. + """ + if left is None or right is None: + return None + mean = (left + right) / 2.0 + if mean == 0: + return None + return round(abs(left - right) / mean, 4) + + +def _phase_index(sequence: PoseSequence, left: Landmark, right: Landmark, + strides_per_minute: float | None) -> float | None: + """How far the two limbs are from anti-phase, as a fraction of a stride. + + Cross-correlation of the two body-frame along-axis traces gives the lag at + which they best agree. At a walk the contralateral pair is half a cycle + apart, so a lag of half a period is the sound pattern and is mapped to 0. + + Returns `None` without a stride rate: a lag in frames means nothing until + there is a period to express it as a fraction of. + """ + if not strides_per_minute: + return None + left_path = in_body_frame(sequence, left) + right_path = in_body_frame(sequence, right) + if left_path is None or right_path is None: + return None + + a = left_path[:, 0] - left_path[:, 0].mean() + b = right_path[:, 0] - right_path[:, 0].mean() + if not np.any(a) or not np.any(b): + return None + + period_frames = sequence.fps * 60.0 / strides_per_minute + correlation = np.correlate(a, b, mode="full") + lags = np.arange(-len(a) + 1, len(b)) + # Only lags inside one period are meaningful; beyond that the correlation + # peak repeats and picking the global maximum would report a lag of three + # cycles as easily as one. + inside = np.abs(lags) <= period_frames + if not inside.any(): + return None + lag = float(lags[inside][int(np.argmax(correlation[inside]))]) + + offset = abs(lag) / period_frames # 0 = in phase, 0.5 = anti-phase + folded = offset % 1.0 + # Distance from 0.5, doubled so the index runs 0 (sound) to 1 (limbs moving + # together), matching the direction of every other index here. + return round(abs(folded - 0.5) * 2.0, 4) + + +def features(sequence: PoseSequence) -> GaitFeatures: + """Every ยง24 quantity this sequence supports, and a reason for each it does not.""" + frame = body_frame(sequence) + unavailable: list[str] = [] + + pairs: list[PairAsymmetry] = [] + for name, left_landmark, right_landmark in CONTRALATERAL_PAIRS: + left = _limb(sequence, left_landmark) + right = _limb(sequence, right_landmark) + + # **Both limbs must be usable before any index is computed.** A refused + # limb still carries a stance fraction and an excursion โ€” they are + # diagnostics, emitted so a threshold can be re-derived later โ€” and an + # earlier version of this function fed them straight into `_index`. On a + # two-second clip, where every limb is refused for holding too few + # strides, that published `Possible gait asymmetry` at an index of + # 0.145. A refusal that reaches the farm as a finding is the exact + # failure this whole service is built against; `tests/test_gait.py` + # asserts the short-clip case so it cannot come back. + both = left.usable and right.usable + rate = left.strides_per_minute if both else None + pairs.append(PairAsymmetry( + pair=name, + stance_index=_index(left.stance_seconds, right.stance_seconds) if both else None, + stride_length_index=_index(left.stride_length_trunks, + right.stride_length_trunks) if both else None, + phase_index=_phase_index(sequence, left_landmark, right_landmark, rate), + left=left, + right=right, + )) + if not both: + unavailable.append( + f"{name} pair: {left.reason or right.reason or 'limb not measurable'}" + ) + + nose = in_body_frame(sequence, Landmark.NOSE) + if nose is None: + head_movement = None + unavailable.append("head movement: the nose was not tracked well enough.") + else: + head_movement = round(float(nose[:, 1].max() - nose[:, 1].min()), 4) + + back = in_body_frame(sequence, Landmark.BACK_MIDDLE) + if back is None: + arch_mean = arch_variation = None + unavailable.append( + f"back-line movement: {sequence.vocabulary or 'this model'} supplies " + f"no mid-back keypoint, or it was not tracked well enough. ยง24 lists " + f"back-line movement among the quantities to derive and this " + f"vocabulary cannot supply it." + ) + else: + # Already the perpendicular distance from the neck-to-tail line, in + # trunk lengths: `in_body_frame`'s second column is the across-axis + # component and the axis runs between exactly those two anchors. + arch_mean = round(float(np.mean(back[:, 1])), 4) + arch_variation = round(float(np.std(back[:, 1])), 4) + + return GaitFeatures( + pairs=tuple(pairs), + head_movement_trunks=head_movement, + back_arch_mean_trunks=arch_mean, + back_arch_variation_trunks=arch_variation, + trunk_drift=round(frame.drift, 4), + duration_seconds=round(sequence.duration_seconds, 3), + fps=sequence.fps, + unavailable=tuple(unavailable), + ) + + +def screen(sequence: PoseSequence, *, threshold: float, + threshold_basis: str = "unstated") -> GaitScreen: + """ยง24's output: a screen, with the evidence, and never a diagnosis. + + `threshold` is required and has no default. The module has no validated + value to offer โ€” deriving one needs cattle a vet locomotion-scored, and this + code has never seen any โ€” so making the caller supply it forces the question + "where did this number come from?" to be answered at every call site rather + than inherited from a constant. + """ + try: + measured = features(sequence) + except TrackUnusable as refused: + return GaitScreen(Verdict.NOT_ASSESSABLE, None, str(refused)) + + if measured.trunk_drift > MAX_TRUNK_DRIFT: + return GaitScreen( + Verdict.NOT_ASSESSABLE, measured, + f"The animal's apparent size changed by {measured.trunk_drift:.0%} " + f"during the recording, against a {MAX_TRUNK_DRIFT:.0%} limit. It " + f"was walking towards or away from the camera rather than across " + f"it, and near-side and far-side limbs would be measured at " + f"different scales. Record again from the side, standing still.", + ) + + worst = measured.worst_asymmetry + if worst is None: + return GaitScreen( + Verdict.NOT_ASSESSABLE, measured, + "No limb pair produced a usable stride. " + + " ".join(measured.unavailable), + ) + + verdict = (Verdict.ASYMMETRY_OBSERVED if worst >= threshold + else Verdict.NO_ASYMMETRY_OBSERVED) + return GaitScreen( + verdict, measured, + reason=( + f"Largest left-right index {worst:.3f} against a threshold of " + f"{threshold:.3f}." + ), + threshold=threshold, + threshold_basis=threshold_basis, + ) diff --git a/app/adapters/pose/tracks.py b/app/adapters/pose/tracks.py new file mode 100644 index 0000000000000000000000000000000000000000..da0fc1dc317738ae913f767309480365efc28686 --- /dev/null +++ b/app/adapters/pose/tracks.py @@ -0,0 +1,212 @@ +"""Keypoint tracks over time, and the body frame that makes them comparable. + +A pose model returns pixel coordinates. Pixel coordinates are useless for gait on +their own, for two reasons that both have to be removed before any feature means +anything: + +**The animal moves across the frame.** ยง24's capture is a cow walking 5โ€“10 m +side-on, so a hoof's image x-coordinate is dominated by the animal's own +translation. Every hoof's trace looks the same: a ramp. + +**The animal's apparent size changes.** It walks closer to or further from the +camera, and a stride that measures 300 px at one end of the run measures 200 px +at the other. Comparing a left stride recorded at the near end to a right stride +recorded at the far end would find an asymmetry in a perfectly sound animal, and +that is the single most dangerous false positive this capability can produce. + +So everything downstream reads the **body frame**: positions expressed relative +to the trunk and divided by the trunk's own apparent length. The result is +dimensionless, immune to both problems, and needs no metric scale at all โ€” which +is why gait is tractable on a phone in a way that ยง22's weight is not. + +**What the body frame does not fix** is the camera not being perpendicular to the +walk. A cow walking towards the camera at an angle has a foreshortened stride +that shortens further as it approaches, and the trunk-length normalisation +partly absorbs that and partly does not. `PoseSequence.trunk_drift` reports how +much the trunk length changed over the clip so a caller can refuse a capture +where it changed a lot, rather than analysing it anyway. +""" + +from __future__ import annotations + +from dataclasses import dataclass + +import numpy as np + +from app.adapters.pose.vocabulary import Landmark + +#: Below this the keypoint is treated as absent for that frame. Pose models +#: report a heatmap peak value, not a probability, so this is a threshold on an +#: uncalibrated score and ยง37 forbids presenting it as a confidence anywhere. +#: 0.3 is the value ViTPose's and DeepLabCut's own example code use; it has not +#: been tuned on cattle and should not be quoted as if it had. +MIN_KEYPOINT_SCORE = 0.3 + +#: Fraction of frames a landmark must be present in before it is used at all. +#: A trace that exists in a third of the frames is mostly interpolation, and +#: interpolation between two distant stance phases invents a swing that never +#: happened. +MIN_LANDMARK_COVERAGE = 0.6 + + +class TrackUnusable(ValueError): + """The pose sequence cannot support a body frame. + + Raised rather than worked around. Without both trunk anchors there is no + body frame, and without a body frame every feature below measures the + camera's motion as though it were the animal's. + """ + + +@dataclass(frozen=True) +class PoseSequence: + """Keypoints over frames, plus the model vocabulary they are named in. + + `xy` is `(frames, landmarks, 2)` in image pixels and `score` is + `(frames, landmarks)`. `landmarks` is the tuple naming the second axis, so + an experiment holding a stored sequence never has to remember an ordering. + """ + + xy: np.ndarray + score: np.ndarray + landmarks: tuple[Landmark, ...] + fps: float + #: Which vocabulary the underlying model used. Carried for the record: a + #: feature that is unavailable because the model has no mid-back keypoint + #: should say which model. + vocabulary: str = "" + + def __post_init__(self) -> None: + frames, count, _ = np.shape(self.xy) + if (frames, count) != np.shape(self.score): + raise TrackUnusable("xy and score describe different numbers of keypoints.") + if count != len(self.landmarks): + raise TrackUnusable( + f"{count} keypoint columns against {len(self.landmarks)} names." + ) + if self.fps <= 0: + raise TrackUnusable("The clip reports no frame rate.") + + @property + def frames(self) -> int: + return int(np.shape(self.xy)[0]) + + @property + def duration_seconds(self) -> float: + return self.frames / self.fps + + def index(self, landmark: Landmark) -> int | None: + try: + return self.landmarks.index(landmark) + except ValueError: + return None + + def coverage(self, landmark: Landmark) -> float: + """Fraction of frames where this landmark scored above threshold.""" + column = self.index(landmark) + if column is None: + return 0.0 + return float((self.score[:, column] >= MIN_KEYPOINT_SCORE).mean()) + + def trace(self, landmark: Landmark) -> np.ndarray | None: + """This landmark's `(frames, 2)` path, low-score frames filled in. + + Returns `None` when the landmark is absent from the vocabulary or falls + under `MIN_LANDMARK_COVERAGE`. Gaps inside a well-covered trace are + linearly interpolated and the ends are held, which is the standard + treatment and is also a lie about the frames it fills; nothing here + should be read at single-frame resolution. + """ + column = self.index(landmark) + if column is None or self.coverage(landmark) < MIN_LANDMARK_COVERAGE: + return None + present = self.score[:, column] >= MIN_KEYPOINT_SCORE + frames = np.arange(self.frames, dtype=np.float64) + filled = np.empty((self.frames, 2), dtype=np.float64) + for axis in (0, 1): + filled[:, axis] = np.interp( + frames, frames[present], self.xy[present, column, axis] + ) + return filled + + +@dataclass(frozen=True) +class BodyFrame: + """The trunk, per frame: where it is, how long it looks, which way it faces. + + `length` is the neck-to-tail distance in pixels. It is the normaliser for + everything, so its stability over the clip is reported rather than assumed โ€” + see `drift`. + """ + + origin: np.ndarray + length: np.ndarray + #: Unit vector from tail to neck. The animal's forward direction in image + #: coordinates, so a cow walking right-to-left is handled without the caller + #: having to know which way it went. + forward: np.ndarray + + @property + def drift(self) -> float: + """Peak-to-peak change in apparent trunk length, as a fraction of the median. + + The single best available warning that the walk was not perpendicular to + the camera. A cow crossing the frame at a right angle holds its apparent + length within a few per cent; one walking towards the lens does not. + """ + median = float(np.median(self.length)) + if median <= 0: + return float("inf") + return float(self.length.max() - self.length.min()) / median + + +def body_frame(sequence: PoseSequence) -> BodyFrame: + """Trunk position, apparent length and heading, per frame.""" + neck = sequence.trace(Landmark.NECK) + tail = sequence.trace(Landmark.TAIL_BASE) + if neck is None or tail is None: + missing_names = [ + landmark.value for landmark, trace in + ((Landmark.NECK, neck), (Landmark.TAIL_BASE, tail)) if trace is None + ] + raise TrackUnusable( + f"No body frame: {', '.join(missing_names)} was not tracked in at " + f"least {MIN_LANDMARK_COVERAGE:.0%} of frames. Without both trunk " + f"anchors every gait feature would measure the camera's motion." + ) + + span = neck - tail + length = np.linalg.norm(span, axis=1) + if not np.all(length > 0): + raise TrackUnusable("The neck and tail keypoints coincide in some frames.") + return BodyFrame( + origin=tail, + length=length, + forward=span / length[:, None], + ) + + +def in_body_frame(sequence: PoseSequence, landmark: Landmark) -> np.ndarray | None: + """A landmark's path in trunk lengths, along and across the body axis. + + Returns `(frames, 2)` where column 0 is the along-body coordinate โ€” positive + towards the head โ€” and column 1 is across it, positive downwards in image + terms once the axis is fixed. Both are in units of trunk length, so a stride + amplitude of 0.4 means "four tenths of this animal's own body". + + Projecting onto the trunk axis rather than onto the image axes is what makes + the result independent of which way the animal walked and of a camera held + slightly off level. + """ + trace = sequence.trace(landmark) + if trace is None: + return None + frame = body_frame(sequence) + relative = trace - frame.origin + forward = frame.forward + # The perpendicular of a 2-D unit vector, taken consistently so "across" has + # a fixed sign for a given walk direction. + across = np.stack([-forward[:, 1], forward[:, 0]], axis=1) + along_component = (relative * forward).sum(axis=1) / frame.length + across_component = (relative * across).sum(axis=1) / frame.length + return np.stack([along_component, across_component], axis=1) diff --git a/app/adapters/pose/vocabulary.py b/app/adapters/pose/vocabulary.py new file mode 100644 index 0000000000000000000000000000000000000000..9a185060a7245c10900d6d4e8bbb52100c168680 --- /dev/null +++ b/app/adapters/pose/vocabulary.py @@ -0,0 +1,136 @@ +"""One keypoint vocabulary, and the maps from the models that use another. + +Directive ยง24 names DeepLabCut SuperAnimal-Quadruped, and ยง36 requires the +alternatives to be attempted before anything is called impossible. Those two +sentences together mean more than one pose model will be tried, and every pose +model has its own names for the same anatomy. Wiring gait analysis to any one of +them would make swapping models a rewrite of the analysis. + +So the analysis reads `Landmark`, and each model gets a map. A map that has no +entry for a landmark leaves it **absent**, and absent propagates: a gait feature +that needs the mid-back is reported as unavailable on a model with no mid-back +keypoint, never as zero. That distinction is the whole reason this file exists +rather than an integer index. + +## The maps are copied from the models' own definitions, not remembered + +`SUPERANIMAL_QUADRUPED` is the `bodyparts` list in DeepLabCut's own +`superanimal_quadruped.yaml`, read on 2026-08-22. `AP10K` is the `keypoint_info` +ordering in mmpose's `configs/_base_/datasets/ap10k.py`, read the same day. Both +are recorded in source order with their indices, because a pose model returns an +array and an off-by-one in this table silently swaps a cow's left hind hoof for +its right โ€” which would produce a confident, entirely fictional asymmetry. +""" + +from __future__ import annotations + +from enum import Enum + + +class Landmark(str, Enum): + """The anatomy gait analysis needs, named once. + + Deliberately small. Every entry here is read by something in `gait`; a + landmark nothing uses is a landmark whose mapping nobody checks. + """ + + NOSE = "nose" + #: Where the neck meets the trunk. The front anchor of the body frame. + NECK = "neck" + #: Base of the tail. The rear anchor of the body frame. + TAIL_BASE = "tail_base" + #: The middle of the topline, between neck and tail. Only some models have + #: it, and back-arch analysis is unavailable without it. + BACK_MIDDLE = "back_middle" + LEFT_FRONT_HOOF = "left_front_hoof" + RIGHT_FRONT_HOOF = "right_front_hoof" + LEFT_HIND_HOOF = "left_hind_hoof" + RIGHT_HIND_HOOF = "right_hind_hoof" + LEFT_FRONT_KNEE = "left_front_knee" + RIGHT_FRONT_KNEE = "right_front_knee" + LEFT_HIND_KNEE = "left_hind_knee" + RIGHT_HIND_KNEE = "right_hind_knee" + + +#: Contralateral pairs, front and hind. Symmetry is only ever computed within a +#: pair โ€” comparing a fore hoof to a hind hoof would find a phase difference on +#: every sound animal that walks. +CONTRALATERAL_PAIRS: tuple[tuple[str, Landmark, Landmark], ...] = ( + ("front", Landmark.LEFT_FRONT_HOOF, Landmark.RIGHT_FRONT_HOOF), + ("hind", Landmark.LEFT_HIND_HOOF, Landmark.RIGHT_HIND_HOOF), +) + +#: DeepLabCut SuperAnimal-Quadruped, 39 bodyparts. Source: the `bodyparts` list +#: in `deeplabcut/modelzoo/project_configs/superanimal_quadruped.yaml` on the +#: DeepLabCut `main` branch, read 2026-08-22. +#: +#: **Only 8 of the 39 are mapped**, and that is correct rather than lazy: the +#: other 31 are ears, antlers, jaw and eye points that no gait feature reads. +#: Mapping them would create thirty-one more entries nobody verifies. +#: +#: Two notes on the choices, because both are judgement calls: +#: `back_base` is taken as the neck-side trunk anchor rather than `neck_end`, +#: because `back_base` sits on the topline and `neck_end` does not, and a body +#: axis is more stable between two topline points. The model spells the upper +#: limb segment `thai` (its own spelling of "thigh"); the `knee` points are the +#: lower joint and the `paw` points are the ground contact, which is what stance +#: detection needs. +SUPERANIMAL_QUADRUPED: dict[Landmark, str] = { + Landmark.NOSE: "nose", + Landmark.NECK: "back_base", + Landmark.TAIL_BASE: "tail_base", + Landmark.BACK_MIDDLE: "back_middle", + Landmark.LEFT_FRONT_HOOF: "front_left_paw", + Landmark.RIGHT_FRONT_HOOF: "front_right_paw", + Landmark.LEFT_HIND_HOOF: "back_left_paw", + Landmark.RIGHT_HIND_HOOF: "back_right_paw", + Landmark.LEFT_FRONT_KNEE: "front_left_knee", + Landmark.RIGHT_FRONT_KNEE: "front_right_knee", + Landmark.LEFT_HIND_KNEE: "back_left_knee", + Landmark.RIGHT_HIND_KNEE: "back_right_knee", +} + +#: AP-10K's 17 keypoints, in the order mmpose defines them. Source: +#: `configs/_base_/datasets/ap10k.py`, `keypoint_info` ids 0โ€“16, read 2026-08-22. +#: AP-10K includes cattle among its 54 species, which is why it is here at all. +#: +#: **It has no mid-back point.** `Neck` and `Root of tail` are the only topline +#: keypoints, so `BACK_MIDDLE` is absent and back-arch analysis cannot run on a +#: model trained to this vocabulary. That is a real limitation of the model +#: choice, not a gap in this table, and `gait` reports it as unavailable. +AP10K_ORDER: tuple[str, ...] = ( + "L_Eye", "R_Eye", "Nose", "Neck", "Root of tail", + "L_Shoulder", "L_Elbow", "L_F_Paw", + "R_Shoulder", "R_Elbow", "R_F_Paw", + "L_Hip", "L_Knee", "L_B_Paw", + "R_Hip", "R_Knee", "R_B_Paw", +) + +AP10K: dict[Landmark, str] = { + Landmark.NOSE: "Nose", + Landmark.NECK: "Neck", + Landmark.TAIL_BASE: "Root of tail", + Landmark.LEFT_FRONT_HOOF: "L_F_Paw", + Landmark.RIGHT_FRONT_HOOF: "R_F_Paw", + Landmark.LEFT_HIND_HOOF: "L_B_Paw", + Landmark.RIGHT_HIND_HOOF: "R_B_Paw", + Landmark.LEFT_FRONT_KNEE: "L_Elbow", + Landmark.RIGHT_FRONT_KNEE: "R_Elbow", + Landmark.LEFT_HIND_KNEE: "L_Knee", + Landmark.RIGHT_HIND_KNEE: "R_Knee", +} + +VOCABULARIES: dict[str, dict[Landmark, str]] = { + "superanimal_quadruped": SUPERANIMAL_QUADRUPED, + "ap10k": AP10K, +} + + +def missing(vocabulary: str) -> tuple[Landmark, ...]: + """Landmarks this model cannot supply. + + Called before analysis so a caller learns what will be unavailable up front + rather than reading a result with silent holes in it. + """ + mapping = VOCABULARIES[vocabulary] + return tuple(landmark for landmark in Landmark if landmark not in mapping) diff --git a/app/adapters/registry.py b/app/adapters/registry.py new file mode 100644 index 0000000000000000000000000000000000000000..0d29f6823e5552a896a614910a9d24b7d84f49e7 --- /dev/null +++ b/app/adapters/registry.py @@ -0,0 +1,133 @@ +"""Every adapter the zero-training stack names, and the state each one is in. + +One listing, built the same way whether an adapter runs, is refused on licence, +is merely not installed, or needs a credential nobody has set. That uniformity is +the point: `main._unavailable_reason` already distinguishes *"no validated model +exists"* from *"this is not planned"*, and the same distinction has to survive +one level down or a reader is back to guessing which of six models is the one +blocking a capability. + +**Nothing here can produce a result.** The registry hands back adapters, and an +adapter that cannot run raises from `load()`. There is no path through this +module that returns an answer, which is the property `providers.discover()` has +and the reason this is a sibling of it rather than a replacement for it. +""" + +from __future__ import annotations + +import json + +from app.adapters.base import Adapter, Placement +from app.adapters.deterministic import deterministic_adapters +from app.adapters.embedding import ( + DINOV2_SPEC, + DINOV3_SPEC, + MEGADESCRIPTOR_SPEC, + MIEWID_SPEC, + OnnxEmbeddingAdapter, +) +from app.adapters.multimodal import HostedMultimodalAdapter +from app.adapters.transports import transport_from_env +from app.adapters.unavailable import unavailable_adapters +from app.providers import MODELS_DIR, ArtefactError, load_card + +#: Cards for adapters this package knows how to build, by adapter id. An +#: adapter whose card is absent still appears in the listing โ€” as unavailable, +#: with the path it was looking for โ€” because a silently missing entry is how a +#: capability disappears without anybody deciding it should. +#: +#: The last two are installed and permanently non-servable, which is a state +#: this listing has not had before. They are here rather than in +#: `unavailable.py` because they are no longer absences to explain: the +#: artefacts exist, `load()` works under `ANIMAP_LICENCE_POLICY=record`, and +#: what stops them reaching a farm is `licences.gate` refusing every request +#: under the default policy. `refused()` below still names exactly these two. +_EMBEDDING_CARDS = { + "dinov3-vits16": ("cattle_identity", "model_card.json"), + "dinov2-small": ("alternates/dinov2_embedding", "model_card.json"), + "megadescriptor": ("alternates/megadescriptor", "model_card.json"), + "miewid-msv3": ("alternates/miewid", "model_card.json"), +} + + +def _embedding(spec, folder: str, filename: str) -> Adapter: + card_path = MODELS_DIR / folder / filename + artefact = None + card: dict = {} + if card_path.is_file(): + try: + artefact = load_card(card_path) + card = json.loads(card_path.read_text()) + except (ArtefactError, json.JSONDecodeError): + # An unreadable card leaves the adapter unavailable rather than + # taking the listing down, matching `discover()`'s behaviour. The + # detail is logged there, not swallowed twice here. + artefact = None + return OnnxEmbeddingAdapter( + artefact, spec, + input_size=card.get("input_size", 224), + mean=tuple(card.get("image_mean", (0.485, 0.456, 0.406))), + std=tuple(card.get("image_std", (0.229, 0.224, 0.225))), + dimensions=card.get("embedding_dimensions", 768), + ) + + +def all_adapters() -> list[Adapter]: + """Every adapter, runnable or not, in the order the directive names them.""" + adapters: list[Adapter] = [] + for spec in (DINOV3_SPEC, DINOV2_SPEC, MEGADESCRIPTOR_SPEC, MIEWID_SPEC): + folder, filename = _EMBEDDING_CARDS[spec.adapter_id] + adapters.append(_embedding(spec, folder, filename)) + adapters.extend(unavailable_adapters()) + # **The transport is chosen by the deployment, not by this import.** + # `transport_from_env` returns None where no provider is configured, which + # leaves the adapter reporting itself unavailable with the reason it always + # gave โ€” and that is the whole posture: a vendor commits Animap to a + # retention stance for photographs of somebody's animals, so nothing here + # picks one by default. + # + # An unknown provider name raises rather than falling back. A typo that + # silently disabled fifteen capabilities would produce a listing identical + # to a deployment where nobody had chosen yet, which is the one failure + # nobody would go looking for. + adapters.append(HostedMultimodalAdapter(transport=transport_from_env())) + adapters.extend(deterministic_adapters()) + return adapters + + +def describe_all() -> list[dict]: + """The listing, for `/health` and for a person reading a deployment.""" + return [adapter.describe() for adapter in all_adapters()] + + +def ready() -> list[Adapter]: + return [a for a in all_adapters() if a.availability().ready] + + +def refused() -> list[Adapter]: + """Adapters a licence forbids, as distinct from ones nobody installed. + + Worth its own function because the two look identical in a listing and mean + completely different things to whoever is planning the next fortnight: one + is a download and one is never. + """ + from app.adapters.licences import RUNTIME_LICENCES + + out = [] + for adapter in all_adapters(): + licence = RUNTIME_LICENCES.get(adapter.spec.runtime) + if licence is not None and not licence.servable: + out.append(adapter) + return out + + +def by_placement() -> dict[str, list[str]]: + """Which leg belongs where. + + The container size is a hosting decision and not a verdict on a model, so + this is a recommendation with a measurement behind it rather than a filter. + """ + grouped: dict[str, list[str]] = {p.value: [] for p in Placement} + for adapter in all_adapters(): + grouped[adapter.spec.placement.value].append(adapter.spec.adapter_id) + return grouped diff --git a/app/adapters/signal/__init__.py b/app/adapters/signal/__init__.py new file mode 100644 index 0000000000000000000000000000000000000000..59345565120ac957b2ecd9bbe118fe44ccb1717c --- /dev/null +++ b/app/adapters/signal/__init__.py @@ -0,0 +1,3 @@ +"""Deterministic signal processing. Directive ยง4: "Do not use a neural model +when deterministic signal processing is better." +""" diff --git a/app/adapters/signal/flow.py b/app/adapters/signal/flow.py new file mode 100644 index 0000000000000000000000000000000000000000..2524b5d22fe23ed29a18cad1508dff443aa46dae --- /dev/null +++ b/app/adapters/signal/flow.py @@ -0,0 +1,138 @@ +"""Turning frames into a one-dimensional motion signal. + +Directive ยง4 lists optical flow first among the things to use OpenCV for, and +ยง14's respiration pipeline is built on it. This module does the video half โ€” +decode, downscale, dense flow, project to one number per frame โ€” and hands the +result to `periodicity`, which does the arithmetic half. + +**The projection is the only interesting decision here.** Dense flow gives a +vector field; a rate needs a scalar. Averaging the magnitude would work and is +wrong, because magnitude rectifies: a flank moving out and a flank moving back +both read positive, so the signal comes out at twice the breathing rate and the +error looks like a plausible answer. The mean *signed* flow keeps the direction, +and projecting onto the field's own principal axis means the caller does not +have to know whether the phone was held upright. +""" + +from __future__ import annotations + +from dataclasses import dataclass +from pathlib import Path + +import numpy as np + +#: Width the frames are resized to before flow is computed. Farnebรคck cost is +#: linear in pixels and the signal being measured is a whole region's mean, so +#: resolution buys nothing above a few hundred pixels. Measured on a 1920ร—1080 +#: clip: 4.3 ms per frame pair at 320 px. At full resolution the same clip is +#: roughly thirty-six times the pixels for the same number. +FLOW_WIDTH = 320 + +#: Farnebรคck's own parameters, at OpenCV's documented defaults for these +#: arguments. They are named rather than passed positionally because a reader +#: cannot otherwise tell `poly_n` from `levels`, and getting one wrong degrades +#: the field quietly. +_FARNEBACK = dict( + pyr_scale=0.5, levels=3, winsize=15, + iterations=3, poly_n=5, poly_sigma=1.2, flags=0, +) + + +class VideoUnreadable(RuntimeError): + """The clip could not be decoded, so there is nothing to measure.""" + + +@dataclass(frozen=True) +class MotionSignal: + """One number per frame pair, plus what it took to get there.""" + + #: Signed displacement along the dominant motion axis, in resized pixels. + values: np.ndarray + sample_rate_hz: float + frames: int + #: Ratio of the first principal component to the second. Near 1 means the + #: motion has no preferred direction, which is what a field of noise looks + #: like โ€” reported rather than acted on, because a real oscillation seen + #: head-on is also near 1. + anisotropy: float + frame_size: tuple[int, int] + + +def read_frames( + path: Path | str, + *, + max_frames: int = 1800, + width: int = FLOW_WIDTH, + roi: tuple[float, float, float, float] | None = None, +) -> tuple[list[np.ndarray], float]: + """Decode to greyscale, cropped and downscaled. + + `roi` is fractional โ€” `(x0, y0, x1, y1)` in 0โ€“1 โ€” so a caller that got a + flank box from a segmenter does not have to know what resolution the clip + is. ยง14's capture protocol is "hold the cow's flank in frame", and cropping + to it is what keeps a swishing tail out of the average. + """ + import cv2 + + capture = cv2.VideoCapture(str(path)) + if not capture.isOpened(): + raise VideoUnreadable(f"{Path(path).name} could not be opened.") + + sample_rate = float(capture.get(cv2.CAP_PROP_FPS)) + frames: list[np.ndarray] = [] + try: + while len(frames) < max_frames: + ok, frame = capture.read() + if not ok: + break + if roi is not None: + x0, y0, x1, y1 = roi + height, frame_width = frame.shape[:2] + frame = frame[ + int(y0 * height):int(y1 * height), + int(x0 * frame_width):int(x1 * frame_width), + ] + if frame.size == 0: + raise VideoUnreadable("The requested region is outside the frame.") + height, frame_width = frame.shape[:2] + scale = width / frame_width + frame = cv2.resize( + frame, (width, max(1, int(height * scale))), + interpolation=cv2.INTER_AREA, + ) + frames.append(cv2.cvtColor(frame, cv2.COLOR_BGR2GRAY)) + finally: + capture.release() + + if len(frames) < 2: + raise VideoUnreadable( + f"{Path(path).name} yielded {len(frames)} frames. There is no motion " + f"in a single frame." + ) + return frames, sample_rate + + +def motion_signal(frames: list[np.ndarray], sample_rate_hz: float) -> MotionSignal: + """Mean signed dense flow per frame pair, projected onto its principal axis.""" + import cv2 + + means = np.empty((len(frames) - 1, 2), dtype=np.float64) + previous = frames[0] + for index, current in enumerate(frames[1:]): + field = cv2.calcOpticalFlowFarneback(previous, current, None, **_FARNEBACK) + means[index] = (float(field[..., 0].mean()), float(field[..., 1].mean())) + previous = current + + centred = means - means.mean(axis=0) + # SVD rather than an eigendecomposition of the covariance: same axis, and it + # cannot return a negative eigenvalue when the field is nearly degenerate. + _, singular, components = np.linalg.svd(centred, full_matrices=False) + projected = centred @ components[0] + + return MotionSignal( + values=projected, + sample_rate_hz=sample_rate_hz, + frames=len(frames), + anisotropy=float(singular[0] / max(singular[1], 1e-12)), + frame_size=(frames[0].shape[1], frames[0].shape[0]), + ) diff --git a/app/adapters/signal/geometry.py b/app/adapters/signal/geometry.py new file mode 100644 index 0000000000000000000000000000000000000000..2b8bb30879a57b084f0e8a787edc49686dd13a1e --- /dev/null +++ b/app/adapters/signal/geometry.py @@ -0,0 +1,173 @@ +"""Measuring physical size from a photograph, when something in it has a size. + +Directive ยง4 lists geometry, contour measurement and reference-marker +calibration among the things to do with OpenCV rather than a network, and ยง9 +names the payoff: a wound reported as *"approximate visible area: 12โ€“16 cmยฒ"* +instead of "a wound", so a follow-up scan can say whether it is getting smaller. + +**The whole capability rests on one honest premise.** ยง22 puts it plainly โ€” "a +photograph has no scale" is true of an arbitrary photograph and not of one with +a known-size object in it. So there are exactly two entry points here: one that +takes a marker of stated physical size, and one that takes a scale somebody +else established. There is no third that guesses, because a wound area computed +from a guessed scale is a number with a unit attached to nothing. + +The `ยฑ` on every result is not decoration either. A marker localised to within a +pixel or two at the edges puts a few per cent of error into the linear scale and +twice that into an area, and ยง37 requires that to reach the farmer as a range +rather than being rounded away. +""" + +from __future__ import annotations + +import math +from dataclasses import dataclass + +import numpy as np + +#: Aruco dictionary used for the Animap reference marker. 4ร—4 with 50 ids has +#: the largest cell size for a given printed square, which is what survives +#: being photographed at arm's length on a phone in a barn. +ARUCO_DICTIONARY = "DICT_4X4_50" + +#: Assumed localisation error on each marker corner, in pixels. Propagated into +#: the reported range rather than ignored. +#: +#: **This is an assumption and not a measurement.** OpenCV's corner refinement +#: is typically sub-pixel on a well-lit marker, and 1.5 px is a deliberately +#: pessimistic stand-in for the barn case โ€” motion blur, a marker at an angle, +#: a printed square that has been in a pocket. It should be replaced by a +#: measurement the first time anybody photographs a marker at a known distance. +CORNER_UNCERTAINTY_PX = 1.5 + + +class NoReference(RuntimeError): + """Nothing in the frame establishes a physical scale.""" + + +@dataclass(frozen=True) +class Scale: + """Pixels per millimetre, with the error that comes from measuring it.""" + + pixels_per_mm: float + #: Fractional 1-sigma uncertainty on the above. + relative_error: float + source: str + + def length_mm(self, pixels: float) -> tuple[float, float]: + """A length and its half-width, both in millimetres.""" + value = pixels / self.pixels_per_mm + return value, value * self.relative_error + + def area_mm2(self, pixels: float) -> tuple[float, float]: + """An area and its half-width, in square millimetres. + + The relative error doubles going from a length to an area, which is the + reason ยง9's example is a range โ€” "12โ€“16 cmยฒ" โ€” and not a figure. + """ + value = pixels / (self.pixels_per_mm ** 2) + return value, value * 2.0 * self.relative_error + + +def scale_from_marker( + image: np.ndarray, marker_side_mm: float, *, dictionary: str = ARUCO_DICTIONARY +) -> Scale: + """Find a printed square marker of known size and derive pixels per mm. + + Raises rather than returning a default. A frame with no marker has no + scale, and the caller's correct response is to ask for a re-capture with + the card in shot โ€” not to receive a number that happens to be plausible. + """ + import cv2 + + grey = image if image.ndim == 2 else cv2.cvtColor(image, cv2.COLOR_BGR2GRAY) + aruco = cv2.aruco + detector = aruco.ArucoDetector( + aruco.getPredefinedDictionary(getattr(aruco, dictionary)), + aruco.DetectorParameters(), + ) + corners, ids, _ = detector.detectMarkers(grey) + if ids is None or len(corners) == 0: + raise NoReference( + "No reference marker in this frame, so nothing establishes what a " + "pixel is worth. Photograph again with the Animap card beside the " + "subject and in the same plane." + ) + + # The mean of the four sides, per marker, then across markers. Averaging the + # sides absorbs a little perspective; it does not correct for it, and a + # marker photographed at a steep angle still reads short. That is why the + # capture instruction says "in the same plane" rather than "in frame". + sides: list[float] = [] + for quad in corners: + points = quad.reshape(4, 2) + sides.extend( + float(np.linalg.norm(points[i] - points[(i + 1) % 4])) for i in range(4) + ) + + mean_side = float(np.mean(sides)) + if mean_side <= 0: + raise NoReference("The marker was found but has no measurable size.") + + pixels_per_mm = mean_side / marker_side_mm + # Two corners contribute to each side, independently, hence the root two. + relative_error = (CORNER_UNCERTAINTY_PX * math.sqrt(2.0)) / mean_side + # Spread between markers, when there is more than one, is real evidence + # about perspective and is folded in rather than averaged away. + if len(sides) > 4: + relative_error = math.hypot(relative_error, float(np.std(sides)) / mean_side) + + return Scale( + pixels_per_mm=pixels_per_mm, + relative_error=relative_error, + source=f"{len(sides) // 4} ร— {marker_side_mm:g} mm {dictionary} marker", + ) + + +def contour_area_px(mask: np.ndarray) -> float: + """Area of the largest connected region in a boolean mask, in pixels. + + The largest region rather than the sum, because a segmenter that returns a + wound plus three specks of noise should report the wound. A caller that + genuinely wants the total already has `mask.sum()`. + + **This is the polygon's area, not a pixel count, and it is smaller.** OpenCV + traces the contour through pixel centres, so a filled 40x40 square measures + 39x39 = 1,521 rather than 1,600 โ€” **4.9% low**, not 2.5%. The 2.5% is the + error on each *side*, and an area loses it twice; this comment used to quote + the linear figure for an area, and so did the 300x300 case, where the true + shortfall is **0.67%** rather than 0.3%. Both were measured on 2026-08-21. + + The gap is a perimeter effect and shrinks as the region grows, so at any + size worth reporting it sits inside the uncertainty `Scale` already carries + โ€” `CORNER_UNCERTAINTY_PX` alone puts a few per cent into the linear scale + and twice that into an area. Recorded because a reader checking the + arithmetic against `mask.sum()` will otherwise find a discrepancy and wonder + which is wrong. + """ + import cv2 + + binary = (np.asarray(mask) > 0).astype(np.uint8) + contours, _ = cv2.findContours(binary, cv2.RETR_EXTERNAL, cv2.CHAIN_APPROX_SIMPLE) + if not contours: + return 0.0 + return float(max(cv2.contourArea(c) for c in contours)) + + +def measure_region(mask: np.ndarray, scale: Scale) -> dict[str, float | str]: + """A masked region's physical size, as a range. + + Returns square centimetres because that is the unit ยง9 reports in, and + because a wound quoted in square millimetres invites the false precision + ยง37 warns about. + """ + pixels = contour_area_px(mask) + area_mm2, half_width_mm2 = scale.area_mm2(pixels) + return { + "area_cm2": round(area_mm2 / 100.0, 2), + "area_cm2_low": round(max(0.0, area_mm2 - half_width_mm2) / 100.0, 2), + "area_cm2_high": round((area_mm2 + half_width_mm2) / 100.0, 2), + "area_px": round(pixels, 1), + "scale_source": scale.source, + "scale_relative_error": round(scale.relative_error, 4), + } diff --git a/app/adapters/signal/periodicity.py b/app/adapters/signal/periodicity.py new file mode 100644 index 0000000000000000000000000000000000000000..18206799403ade253d4152b4032e2a5fdda62ab8 --- /dev/null +++ b/app/adapters/signal/periodicity.py @@ -0,0 +1,434 @@ +"""Finding a rate in a noisy 1-D signal, and refusing when there isn't one. + +Directive ยง4: "Do not use a neural model when deterministic signal processing is +better." Respiration is the clearest case in the whole capability matrix โ€” ยง14 +spells the pipeline out as video โ†’ flank region โ†’ optical flow โ†’ periodicity โ†’ +FFT โ†’ breaths per minute, and none of those steps wants a network. + +**The hard part is not finding a peak. Every spectrum has a peak.** The hard part +is knowing whether the peak means anything, and this module is mostly that. It +mirrors the shape `app/counting.py` arrived at: measure whether you are still in +the regime the method works in, and when you are not, publish nothing. + +Four gates, and all must pass: + +**Enough cycles** โ€” the clip has to hold `MIN_CYCLES` full periods, or the +estimate moves with where the recording happened to start. + +**Prominence** โ€” peak power over the median in-band power. Noise is flat, so a +real oscillation stands far above its neighbours and sensor noise does not. + +**Stability** โ€” split the clip, measure each half independently, and require both +to agree with the whole-clip estimate. A signal that is genuinely periodic gives +the same answer on any window of it. One that is a slow drift, a pan, or an +animal shifting its weight does not. + +**Sub-band shoulder** โ€” the power sitting *below* the search band, against the +in-band peak. This is the newest gate and it exists because the other three +published a confident rate for signals that do not oscillate at all. + +**Every threshold here belongs to one search band, and the band is now recorded +with them.** Prominence is peak power over median *in-band* power, so it is a +property of the band as much as of the signal, and it does not transfer. The +constants were derived at `(8, 180)` cycles per minute and the caller shipped +`(8, 90)`; at the narrower band the ordering inverts, a static patch of sensor +noise outranks a positive control, and no threshold separates the two classes at +all. `DERIVATION_BAND_CPM` pins the band the numbers came from, and +`tests/test_periodicity.py` asserts that the band the service actually searches +is that one. + +**The stability rule is written the way it is because of a bug real footage +caught.** The first version compared the two halves *to each other*. On a clip of +resting cattle both halves said ~26 cycles/min while the whole clip said 45.7, +and the two halves agreeing with each other sailed through a check that never +looked at the number being published. Each half is now compared to the estimate +that would actually be reported. + +**The shoulder rule is written because of a failure a synthetic sweep caught.** +`sqrt(t)`, `log(1+t)`, `sigmoid(t)` and `exp(t)` contain no oscillation of any +kind, and over 2,804 runs across durations from 20 to 90 seconds, 1,504 of them +โ€” 53.6% โ€” published a confident 8.34 to 10.39 cycles per minute through the +prominence and stability gates. They did not scrape past: at sixty seconds their +prominence runs to seven figures and their half-drift to 0.059, twice as good as +the tolerance. On a respiratory screen a fabricated rate is the dangerous +direction, and this was the exact case the gates were described as catching. +""" + +from __future__ import annotations + +from dataclasses import dataclass + +import numpy as np + +#: The search band every threshold below was derived at, in cycles per minute. +#: +#: **Recorded as a constant because a threshold without its band is not a +#: threshold.** Prominence is peak power over the *in-band* median, so narrowing +#: the band changes the denominator and every number in the tables below moves. +#: The service shipped `(8, 90)` against constants derived here, and at that +#: band: +#: +#: - the positive control is out of range entirely โ€” the metronome's stated +#: 96 cycles per minute sits above a 90 ceiling, so its whole-frame peak is +#: forced onto the pendulum's 48 subharmonic and its prominence collapses from +#: 1944.3 to 16.9; +#: - a walking cow scores 114.2 and a static patch of sensor noise scores 30.9, +#: so **two negatives outrank two of the three positives**; +#: - no prominence threshold separates the classes. The sorted list interleaves +#: three times and the best achievable is two misclassifications at any value. +#: +#: So the band was widened back to the one the evidence belongs to rather than +#: the constants being re-derived at a band where they cannot exist. +#: `tests/test_periodicity.py` asserts that `respiration.SEARCH_BAND_BPM` is +#: this band, which is what stops the two drifting apart again. +#: +#: **What widening costs is untested rather than nil.** No negative in the +#: derivation set has any content between 90 and 180 cycles per minute, so the +#: claim that the band is "narrow enough to exclude a swishing tail at the top" +#: is now unexamined at the top. Narrowing it again is defensible and it is a +#: re-derivation, not an edit. +DERIVATION_BAND_CPM = (8.0, 180.0) + +#: Peak power divided by median in-band power, below which no rate is reported. +#: +#: **Derived, on four real Wikimedia Commons clips, not chosen.** The clips are +#: pinned by sha256 in `experiments/cattle_respiratory/examples/sources.json`, +#: which is how this derivation is reproduced. The positives +#: are three regions of `Metronome.webm` (CC BY 4.0), whose Commons description +#: states the mechanism ticks at 96 beats per minute; the negatives are regions +#: of two cattle clips, a walking cow, and two static background patches that +#: contain nothing but sensor noise. +#: +#: All figures below are at `DERIVATION_BAND_CPM` and were reproduced from the +#: pinned clips on 2026-08-21, matching the recorded values to the last digit. +#: +#: | region | prominence | shoulder | reported | +#: |---|---|---|---| +#: | metronome, whole frame | 1944.3 | 0.0031 | 96.48 cyc/min | +#: | metronome, pendulum crop | 1480.7 | 0.0009 | 48.38 cyc/min | +#: | metronome, upper crop | 225.5 | 0.0033 | 96.49 cyc/min | +#: | cow walking across a grid, first 600 frames | 92.8 | 4.327 | โ€” | +#: | static background, metronome corner | 37.4 | 0.323 | โ€” | +#: | cattle at rest, whole frame | 12.1 | 0.064 | โ€” | +#: | cattle defecating, flank crop | 5.9 | 0.148 | โ€” | +#: | static background, road corner, first 600 frames | 5.4 | 4.312 | โ€” | +#: +#: The gap runs from 92.8 to 225.5 and 150 sits in it. The walking-cow row is +#: the worst case and it is a truncation: the first 600 frames read 92.8, and +#: the whole 925-frame clip reads 16.2. The tighter of the two is quoted, +#: because a threshold should be set against the hardest window a capture can +#: present and not against the average of one. **Three positives, all +#: from one video of one metronome, is a very thin basis** and this threshold +#: should be re-derived the moment there is real cattle footage that anybody has +#: counted breaths on. It is recorded here as a starting point with its evidence +#: attached, not as a validated constant. +MIN_PEAK_PROMINENCE = 150.0 + +#: How far either half's estimate may sit from the whole-clip estimate, as a +#: fraction of it. +#: +#: Same eight regions, same band. The three metronome regions drift 0.003, 0.015 +#: and 0.003; everything else drifts 0.372 or more, up to 2.885. 0.15 sits in +#: that gap with room on both sides. +#: +#: **This rule and the one above do not overlap on this set**, which is a +#: stronger separation than `app/counting.py` gets from its two rules โ€” and with +#: eleven regions measured, it is also much weaker evidence. Both are required +#: rather than either, because the cost of publishing a wrong breathing rate is +#: a clinical decision and the cost of refusing is a re-capture. +MAX_HALF_DRIFT = 0.15 + +#: Full cycles that must fit in the clip before a rate is worth reporting. +#: Below this the FFT has too few periods to resolve one, and the estimate moves +#: with where the clip happened to start. The walking-cow clip fails here too +#: (2.5 cycles), so this rule is not load-bearing on the current set โ€” it is a +#: guard for the short-capture case the set does not contain. +MIN_CYCLES = 4.0 + +#: The strongest spectral line *below* the search band, over the in-band peak. +#: Above this, no rate is published. +#: +#: **This is the gate that catches drift wearing an oscillation's clothes.** +#: `_detrend` removes a straight line exactly, and nothing removes a curve. A +#: signal that only ever increases has a `1/f`-shaped residual spectrum: power +#: keeps climbing as frequency falls, so whatever the in-band peak is, there is +#: far more power under the band than in it. A real oscillation puts its power +#: at its own frequency and leaves the sub-band bins empty. +#: +#: Measured, all at `DERIVATION_BAND_CPM`: +#: +#: | class | worst case | value | +#: |---|---|---| +#: | metronome positives | upper crop | **0.0033** | +#: | synthetic breathers, 8.5โ€“45 cyc/min with drift and noise | 8.5 at noise 0.3 | **0.040** | +#: | real footage, nearest miss | static road corner, full 925 frames | **0.962** | +#: | monotonic curves, worst of a 2,804-run sweep | โ€” | **253.8** | +#: +#: The sweep is `sqrt(t)`, `log(1+t)`, `sigmoid((t โˆ’ mid)/6)` and `exp(t/12)` at +#: 30 fps, over every duration from 20.0 to 90.0 seconds in 0.1 s steps, giving +#: 2,804 runs. **With the shoulder gate off, 1,504 of them publish a confident +#: 8.34 to 10.39 cycles per minute โ€” 53.6%. With it on, 0 do**, and all three +#: metronome positives and every synthetic breather still publish. +#: +#: The exact constants matter to the first number and not to the second. 1,504 +#: is what these four shapes give and it moves if the shapes or their constants +#: do โ€” an independent reviewer using a different sigmoid midpoint reported +#: 1,500. **The figure that carries the argument is the zero**, which is stable +#: across every variation tried, and `tests/test_periodicity.py` re-derives a +#: 284-run subset of the sweep so the claim is checked rather than remembered. +#: +#: **Two things it is not.** It is not a universal monotonic detector: after +#: `_detrend` removes a straight line *exactly*, a pure linear ramp leaves only +#: float rounding noise, whose spectrum is numerically unstable โ€” measured +#: shoulders from 0.14 to 1,312 depending on duration and scale. The ramp is +#: always refused, but which gate refuses it is not predictable and no claim +#: should rest on it. And the headroom above is against synthetic curves; the +#: **nearest real region is 0.962**, four per cent under the threshold, so the +#: margin against real-world footage is far thinner than the sweep suggests. +#: That row is the one to watch if this is ever tightened. +MAX_SUBBAND_SHOULDER = 1.0 + + +@dataclass(frozen=True) +class Periodicity: + """A rate, or an account of why there isn't one. + + `cycles_per_minute` is `None` whenever `usable` is False. There is + deliberately no way to read a number out of a failed measurement โ€” the same + property `app/counting.py` has, where a withheld count is not a count of + zero. + """ + + usable: bool + cycles_per_minute: float | None + reason: str = "" + #: The diagnostics, emitted whether or not a rate is published, so a + #: threshold can be re-derived from stored results rather than by going back + #: to footage nobody kept. + peak_prominence: float = 0.0 + half_drift: float = 0.0 + cycles_observed: float = 0.0 + resolution_cycles_per_minute: float = 0.0 + duration_seconds: float = 0.0 + #: Strongest sub-band line over the in-band peak. `inf` when the clip is too + #: short to have a bin below the band, which is a refusal rather than a pass. + subband_shoulder: float = 0.0 + #: The band this was measured in. **Recorded on every result**, because a + #: prominence without its band cannot be compared to anything and a stored + #: diagnostic that nobody can re-derive a threshold from is not a diagnostic. + band_cycles_per_minute: tuple[float, float] = DERIVATION_BAND_CPM + #: What each half said on its own. Kept because when a measurement is + #: refused, this pair is usually the reason a reader can see it. + half_estimates: tuple[float, float] | None = None + + +def _detrend(signal: np.ndarray) -> np.ndarray: + """Remove the linear component. + + A handheld capture drifts. Left in, the drift dumps power into the lowest + bins and drags the peak down towards zero, which is how a still frame comes + out as a very slow, very confident oscillation. + """ + n = len(signal) + t = np.arange(n, dtype=np.float64) + design = np.vstack([t, np.ones(n)]).T + coefficients, *_ = np.linalg.lstsq(design, signal, rcond=None) + return signal - design @ coefficients + + +@dataclass(frozen=True) +class _Peak: + """One spectrum's verdict, before any threshold is applied.""" + + frequency_hz: float + prominence: float + bin_width_hz: float + shoulder: float + + +def _peak( + signal: np.ndarray, sample_rate_hz: float, band_hz: tuple[float, float] +) -> _Peak | None: + """Dominant in-band frequency, its prominence, the bin width, the shoulder.""" + n = len(signal) + if n < 8: + return None + + windowed = _detrend(signal) * np.hanning(n) + power = np.abs(np.fft.rfft(windowed)) ** 2 + frequencies = np.fft.rfftfreq(n, d=1.0 / sample_rate_hz) + + low, high = band_hz + in_band = np.flatnonzero((frequencies >= low) & (frequencies <= high)) + if in_band.size < 3: + return None + + peak = in_band[int(np.argmax(power[in_band]))] + + # Bin 0 is excluded because `_detrend` has already taken DC out, so + # whatever is left there is arithmetic rather than signal. + below_band = power[1:int(in_band[0])] + if below_band.size == 0: + # A clip too short to have a single bin under the band cannot be asked + # this question, and the answer to a question that cannot be asked is + # not "pass". At the 8 cycles/min floor this needs 7.5 seconds, and + # `respiration.MIN_CAPTURE_SECONDS` is 20, so it is unreachable through + # the respiratory path and reachable by calling this function directly. + shoulder = float("inf") + else: + shoulder = float( + np.max(below_band) / max(float(power[peak]), 1e-30) + ) + + # Parabolic interpolation in log power. The bin width at 30 fps over ten + # seconds is about 5 cycles/min, which is coarse enough to matter for a + # breathing rate; three points around the peak give sub-bin resolution for + # four lines of arithmetic and no extra dependency. + offset = 0.0 + if 0 < peak < len(power) - 1: + left, centre, right = ( + np.log(power[peak - 1] + 1e-30), + np.log(power[peak] + 1e-30), + np.log(power[peak + 1] + 1e-30), + ) + curvature = left - 2.0 * centre + right + if abs(curvature) > 1e-30: + offset = float(np.clip(0.5 * (left - right) / curvature, -0.5, 0.5)) + + bin_width = float(frequencies[1] - frequencies[0]) + frequency = float(frequencies[peak]) + offset * bin_width + prominence = float(power[peak] / max(float(np.median(power[in_band])), 1e-30)) + return _Peak(frequency, prominence, bin_width, shoulder) + + +def dominant_rate( + signal: np.ndarray, + sample_rate_hz: float, + band_cycles_per_minute: tuple[float, float], + *, + min_prominence: float = MIN_PEAK_PROMINENCE, + max_drift: float = MAX_HALF_DRIFT, + min_cycles: float = MIN_CYCLES, + max_shoulder: float = MAX_SUBBAND_SHOULDER, +) -> Periodicity: + """The rate this signal repeats at, or a refusal. + + `band_cycles_per_minute` is a *search range*, not a claim about what is + normal for an animal. Narrowing it is how a caller stops the method locking + on to a gait or a fan; it is not a prior that pulls the estimate. + + **It is also the band every default threshold here was derived at**, and + passing a different one silently invalidates all four. It is a parameter + rather than a constant because the derivation itself has to sweep it, and + the result records the band it used so a stored diagnostic can be compared + with the constants it was judged against. + """ + signal = np.asarray(signal, dtype=np.float64).ravel() + n = len(signal) + duration = n / sample_rate_hz if sample_rate_hz > 0 else 0.0 + + band = (float(band_cycles_per_minute[0]), float(band_cycles_per_minute[1])) + + if sample_rate_hz <= 0: + return Periodicity( + False, None, "The clip reports no frame rate.", + band_cycles_per_minute=band, + ) + + low_hz, high_hz = (v / 60.0 for v in band_cycles_per_minute) + + # Nyquist. Asked before anything is computed, because above it the method + # does not degrade โ€” it aliases, and returns a confident wrong number. + if sample_rate_hz < 2.0 * high_hz: + return Periodicity( + False, None, + f"The clip samples at {sample_rate_hz:.1f} Hz, which cannot resolve " + f"{band_cycles_per_minute[1]:.0f} cycles per minute. Capture at a " + f"higher frame rate.", + duration_seconds=duration, + band_cycles_per_minute=band, + ) + + whole = _peak(signal, sample_rate_hz, (low_hz, high_hz)) + if whole is None: + return Periodicity( + False, None, + "The clip is too short to hold a spectrum.", + duration_seconds=duration, + band_cycles_per_minute=band, + ) + + rate = whole.frequency_hz * 60.0 + cycles = whole.frequency_hz * duration + resolution = whole.bin_width_hz * 60.0 + + half = n // 2 + first = _peak(signal[:half], sample_rate_hz, (low_hz, high_hz)) + second = _peak(signal[half:], sample_rate_hz, (low_hz, high_hz)) + + if first is None or second is None: + drift = float("inf") + halves = None + else: + halves = (first.frequency_hz * 60.0, second.frequency_hz * 60.0) + # Each half against the number that would be *published*, never against + # each other. Two halves can agree on something the whole clip does not + # say, and that is the case this rule exists to catch. + drift = max(abs(h - rate) for h in halves) / max(rate, 1e-9) + + diagnostics = dict( + peak_prominence=whole.prominence, + half_drift=drift, + cycles_observed=cycles, + resolution_cycles_per_minute=resolution, + duration_seconds=duration, + subband_shoulder=whole.shoulder, + band_cycles_per_minute=band, + half_estimates=halves, + ) + + if cycles < min_cycles: + return Periodicity( + False, None, + f"Only {cycles:.1f} cycles fit in {duration:.1f} s. At least " + f"{min_cycles:.0f} are needed before a rate means anything โ€” record " + f"for longer.", + **diagnostics, + ) + + if whole.prominence < min_prominence: + return Periodicity( + False, None, + f"No clear rhythm. The strongest rate in the search band stands only " + f"{whole.prominence:.0f}ร— above the background, and " + f"{min_prominence:.0f}ร— is the least that has separated a real " + f"oscillation from camera noise.", + **diagnostics, + ) + + if whole.shoulder > max_shoulder: + return Periodicity( + False, None, + f"This is drift, not a rhythm. Below the search band the spectrum " + f"holds {whole.shoulder:.0f}ร— more power than the peak inside it, " + f"against a {max_shoulder:.0f}ร— limit โ€” the signature of something " + f"that only moves one way. A rate measured from a slow drift is a " + f"number with nothing behind it.", + **diagnostics, + ) + + if drift > max_drift: + detail = ( + f" The two halves of the clip read {halves[0]:.0f} and " + f"{halves[1]:.0f}." if halves else "" + ) + return Periodicity( + False, None, + f"The rhythm is not steady: measured over each half of the clip it " + f"moves by {drift * 100:.0f}%, against a {max_drift * 100:.0f}% " + f"tolerance.{detail} Something in the frame is moving that is not " + f"the thing being measured.", + **diagnostics, + ) + + return Periodicity(True, rate, "", **diagnostics) diff --git a/app/adapters/signal/respiration.py b/app/adapters/signal/respiration.py new file mode 100644 index 0000000000000000000000000000000000000000..0fd5661ae7b6cec7d60a770b05fb586c6e929ba1 --- /dev/null +++ b/app/adapters/signal/respiration.py @@ -0,0 +1,168 @@ +"""Respiratory rate from a video, by optical flow and an FFT. + +Directive ยง14, and it is the clearest case in the whole capability matrix for +ยง4's rule that a neural model should not be used where deterministic signal +processing is better. The pipeline the directive specifies is exactly this: + + video โ†’ flank region segmentation โ†’ optical flow / pixel-motion signal + โ†’ periodicity โ†’ FFT / peak detection โ†’ breaths per minute + +**What this module adds to that list is the refusal.** `flow` produces a signal +from any video and `periodicity` finds a peak in any signal, so the only thing +standing between a still frame of a fence post and a confident respiratory rate +is the pair of gates in `periodicity`. Those gates were derived on real footage +and they are the reason this file is worth having. + +**Measured on every real cattle clip available, this publishes nothing**, and +that is the current honest state of the capability rather than a bug. None of +the three is the capture ยง14 asks for. Two are under twenty seconds and are +refused before any signal processing runs; the third is thirty-one seconds of a +cow walking across a cattle grid, gets as far as the spectrum, and is refused +there for having no clear rhythm โ€” peak prominence 16 against a threshold of 150. + +What the module demonstrates positively is the metronome: on footage whose +Commons description states 96 beats per minute, `dominant_rate` returns 96.48, +and on a crop of the pendulum alone 48.38, the pendulum's own cycle being half +the tick rate. That is a 0.5% error against a stated rate, on real video. + +**That demonstration does not run through `respiratory_rate`, and saying it does +would be false.** `Metronome.webm` is 11.71 seconds and `MIN_CAPTURE_SECONDS` is +20, so this function refuses it before `motion_signal` is called. The only clip +anybody has a ground-truth rate for cannot reach the gates it was used to +derive. That is a gap in the evidence rather than a bug in the code โ€” the +capture minimum is right, and what is missing is a long enough clip of something +with a known rate. + +**No cattle respiratory rate has been validated and none can be from what is +here.** What is missing is not model work โ€” it is a thirty-second clip of a +cow's flank with somebody's counted breath rate beside it. +""" + +from __future__ import annotations + +from dataclasses import dataclass +from pathlib import Path + +from app.adapters.base import Measurement +from app.adapters.signal.flow import motion_signal, read_frames +from app.adapters.signal.periodicity import DERIVATION_BAND_CPM, dominant_rate + +#: The range the peak is searched in, in breaths per minute. +#: +#: **A search range, not a claim about what is normal for cattle.** Nothing in +#: this repository establishes a normal respiratory range for a White Fulani in +#: Kaduna, and a band that encoded one would be a physiological claim smuggled +#: in as a constant. It is set wide enough to contain any plausible rate and +#: narrow enough to exclude a slow pan at the bottom. +#: +#: **It is `periodicity.DERIVATION_BAND_CPM` because it has to be.** This used +#: to read `(8, 90)` while every threshold in `periodicity` was derived at +#: `(8, 180)`, and prominence is peak power over the *in-band* median, so it +#: does not survive a change of band. At `(8, 90)` the positive control is out +#: of range โ€” the metronome's stated 96 breaths per minute is above the ceiling +#: โ€” two negatives outrank two of the three positives, and no threshold +#: separates the classes at all. The band was widened back to where the evidence +#: is rather than the thresholds being re-derived where they cannot exist, and +#: `tests/test_periodicity.py` asserts the two stay equal. +#: +#: **What that costs is untested, not nil.** No clip in the derivation set has +#: content between 90 and 180 breaths per minute, so a ceiling of 180 leaves the +#: top of the band unexamined โ€” a swishing tail is the case to worry about. +#: Narrowing it again is a re-derivation, not an edit. +SEARCH_BAND_BPM = DERIVATION_BAND_CPM + +#: ยง14 asks the farmer to "hold the cow's flank in frame for 30โ€“60 seconds". +#: Below this there are not enough breaths to resolve one โ€” at the bottom of +#: the search band, four cycles take thirty seconds โ€” and the periodicity gate +#: would refuse anyway. Checking it here lets the app say so before spending +#: the compute. +MIN_CAPTURE_SECONDS = 20.0 + + +@dataclass(frozen=True) +class RespirationResult: + """A rate, or a refusal, plus everything needed to re-derive a threshold.""" + + measurement: Measurement + frames: int + sample_rate_hz: float + duration_seconds: float + #: Where the signal came from, so a result recorded against the wrong part + #: of the animal is diagnosable later. + region: tuple[float, float, float, float] | None + + +def respiratory_rate( + video_path: Path | str, + *, + region: tuple[float, float, float, float] | None = None, + max_frames: int = 1800, +) -> RespirationResult: + """Breaths per minute from a clip, or an account of why not. + + `region` is the fractional flank box โ€” `(x0, y0, x1, y1)` in 0โ€“1 โ€” from a + segmenter or from the capture UI's own guide box. Passing one is strongly + preferable to not: the whole-frame signal averages the flank together with + everything else that moved, and on a clip of two animals it is the other + animal that wins. + """ + frames, sample_rate = read_frames( + video_path, max_frames=max_frames, roi=region + ) + duration = len(frames) / sample_rate if sample_rate > 0 else 0.0 + + if duration < MIN_CAPTURE_SECONDS: + return RespirationResult( + measurement=Measurement( + kind="respiratory_rate", value=None, unit="breaths/min", + usable=False, + support={"duration_seconds": round(duration, 2)}, + detail=( + f"The clip is {duration:.0f} seconds. Hold the flank in " + f"frame for at least {MIN_CAPTURE_SECONDS:.0f} โ€” a slow " + f"breather needs half a minute before four breaths have " + f"happened." + ), + ), + frames=len(frames), sample_rate_hz=sample_rate, + duration_seconds=duration, region=region, + ) + + signal = motion_signal(frames, sample_rate) + rate = dominant_rate(signal.values, sample_rate, SEARCH_BAND_BPM) + + support = { + "peak_prominence": round(rate.peak_prominence, 2), + "half_drift": round(rate.half_drift, 4), + "cycles_observed": round(rate.cycles_observed, 2), + "resolution_bpm": round(rate.resolution_cycles_per_minute, 2), + "subband_shoulder": round(rate.subband_shoulder, 4), + # Stored with every result, because a prominence is a property of the + # band it was measured in and a stored diagnostic that does not name its + # band cannot be used to re-derive anything. + "band_low_bpm": rate.band_cycles_per_minute[0], + "band_high_bpm": rate.band_cycles_per_minute[1], + "anisotropy": round(signal.anisotropy, 3), + "duration_seconds": round(duration, 2), + } + if rate.half_estimates: + support["half_1_bpm"] = round(rate.half_estimates[0], 2) + support["half_2_bpm"] = round(rate.half_estimates[1], 2) + + return RespirationResult( + measurement=Measurement( + kind="respiratory_rate", + # A refused measurement carries no number. The diagnostics are in + # `support` where a threshold can be re-derived from them, and + # nowhere that a caller could mistake for a result. + value=round(rate.cycles_per_minute, 1) if rate.usable else None, + unit="breaths/min", + usable=rate.usable, + support=support, + detail=rate.reason, + ), + frames=len(frames), + sample_rate_hz=sample_rate, + duration_seconds=duration, + region=region, + ) diff --git a/app/adapters/tiled/__init__.py b/app/adapters/tiled/__init__.py new file mode 100644 index 0000000000000000000000000000000000000000..b43e1cb269c66022da88b4c2498087dfb85fbee0 --- /dev/null +++ b/app/adapters/tiled/__init__.py @@ -0,0 +1,197 @@ +"""High-resolution tiled inference (ยง25), and the two ways to fill the tiles. + +Directive ยง25 lists four things to build for cattle ticks: + +1. SAM 3.1 visual exemplar prompting; +2. Grounding DINO; +3. **high-resolution tiled inference**; +4. a multimodal verifier. + +This package is the third, plus what the first two turn into when SAM 3 is out +of reach. `experiments/cattle_ticks/` is where the four are attempted and where +the record of what could not be attempted lives, which is what ยง36 asks for +before anything is called unavailable. + +## The pieces + +`tiles.py` โ€” cut a photograph into overlapping fixed-pixel tiles and merge what +comes back. Model-free arithmetic, and the part ยง25 actually names. + +`blobs.py` โ€” a difference-of-Gaussians proposer. ยง4's classical route: no +weights, milliseconds, generous by design. + +`openvocab.py` โ€” Grounding DINO prompted with a word. ยง4's open-vocabulary leg. + +`exemplar.py` โ€” a frozen DINOv3 embedding and a cosine, which is the reachable +form of ยง25's visual-exemplar prompting. + +## Why the pipeline is two-stage + +Because the counting result says so. `experiments/poultry_house_count/` measured +a COCO detector out by 157 birds a frame and the same frames counted to within +15 once the model was **shown three examples**. The exemplar was what closed the +gap, and it did not need to come from the frame being scored. Propose cheaply, +then verify against an exemplar, and report both stages separately so a failure +is attributable to one of them. + +## What a tiled pass costs + +Linear in tiles, and the tile count is quadratic in the photograph's side. A +4,000 ร— 3,000 photo at an 800 px side with 20% overlap is 6 ร— 5 = 30 tiles. With +the blob proposer that is milliseconds. With Grounding DINO at a measured 5.5 s +a frame on a laptop CPU it is nearly three minutes, which is why +`openvocab.GROUNDING_DINO_TILED_SPEC` is a queued GPU placement and says so. +""" + +from __future__ import annotations + +import time +from dataclasses import dataclass, field +from typing import Callable + +from PIL import Image + +from app.adapters.base import OpenVocabularyDetector, Region +from app.adapters.tiled.tiles import ( + MERGE_CONTAINMENT, + MERGE_IOU, + TILE_OVERLAP, + TILE_PIXELS, + Tile, + merge, + tiles_for, + to_frame, +) + +#: A function that turns one tile image into regions in **tile** coordinates. +#: `blobs.propose` is one; a bound `GroundingDinoAdapter.detect_text` is another. +OnTile = Callable[[Image.Image], list[Region]] + + +@dataclass(frozen=True) +class TiledResult: + """What a tiled pass found, and enough to say what tiling did. + + **`raw_count` and `count` are both here on purpose.** The difference between + them is how much of the answer was duplicate detections across seams, and a + result that reported only the merged number would make an over-aggressive + merge and a genuinely sparse image look identical. `experiments/cattle_ticks/` + reads both. + """ + + regions: list[Region] + tiles: tuple[Tile, ...] + #: Regions before the cross-tile merge, summed over tiles. + raw_count: int + #: Per-tile counts before the merge, in tile order. Kept because a tiled + #: pass whose detections all come from one tile is a different finding from + #: one spread evenly, and only this shows it. + per_tile: tuple[int, ...] + seconds: float + tile_pixels: int + overlap: float + #: Empty when every tile ran. Otherwise `(tile index, reason)` โ€” a tile that + #: raised is recorded, never silently skipped, because a pass that dropped a + #: third of its tiles must not report a count as though it had not. + failures: tuple[tuple[int, str], ...] = () + notes: dict[str, object] = field(default_factory=dict) + + @property + def count(self) -> int: + return len(self.regions) + + @property + def complete(self) -> bool: + return not self.failures + + +def run_tiled( + image: Image.Image, + on_tile: OnTile, + *, + tile_pixels: int = TILE_PIXELS, + overlap: float = TILE_OVERLAP, + iou: float = MERGE_IOU, + containment: float = MERGE_CONTAINMENT, +) -> TiledResult: + """Run `on_tile` over every tile and merge the results into frame coordinates. + + A tile that raises is recorded in `failures` and the pass continues. That is + the same choice `experiments/poultry_house_count/run.py` makes per frame, and + for the same reason: one unreadable tile should cost one tile, and the count + that comes back has to carry the fact that it was short. + """ + tiles = tuple(tiles_for(image.size, tile_pixels=tile_pixels, overlap=overlap)) + began = time.perf_counter() + + gathered: list[Region] = [] + per_tile: list[int] = [] + failures: list[tuple[int, str]] = [] + for tile in tiles: + try: + found = on_tile(tile.crop(image)) + except Exception as failure: # noqa: BLE001 โ€” recorded, not swallowed + failures.append((tile.index, f"{type(failure).__name__}: {failure}")) + per_tile.append(0) + continue + per_tile.append(len(found)) + gathered.extend(to_frame(region, tile, image.size) for region in found) + + merged = merge(gathered, image.size, iou=iou, containment=containment) + return TiledResult( + regions=merged, + tiles=tiles, + raw_count=len(gathered), + per_tile=tuple(per_tile), + seconds=round(time.perf_counter() - began, 3), + tile_pixels=tile_pixels, + overlap=overlap, + failures=tuple(failures), + ) + + +class TiledDetector: + """An `OpenVocabularyDetector` run tile by tile instead of whole-frame. + + Not an `Adapter`. It owns no weights, has no licence of its own and cannot + be unavailable โ€” it is a strategy applied to an adapter, and giving it a + spec would put a second entry in the listing for one model, which is exactly + the drift `adapters/registry.py` exists to prevent. + """ + + def __init__( + self, + detector: OpenVocabularyDetector, + *, + tile_pixels: int = TILE_PIXELS, + overlap: float = TILE_OVERLAP, + ) -> None: + self.detector = detector + self.tile_pixels = tile_pixels + self.overlap = overlap + + def detect_text( + self, image: Image.Image, prompts: tuple[str, ...] + ) -> TiledResult: + return run_tiled( + image, + lambda tile: self.detector.detect_text(tile, prompts), + tile_pixels=self.tile_pixels, + overlap=self.overlap, + ) + + +__all__ = [ + "MERGE_CONTAINMENT", + "MERGE_IOU", + "TILE_OVERLAP", + "TILE_PIXELS", + "OnTile", + "Tile", + "TiledDetector", + "TiledResult", + "merge", + "run_tiled", + "tiles_for", + "to_frame", +] diff --git a/app/adapters/tiled/blobs.py b/app/adapters/tiled/blobs.py new file mode 100644 index 0000000000000000000000000000000000000000..5d3f08b2fe367c32499cbda673b070dbc5a93729 --- /dev/null +++ b/app/adapters/tiled/blobs.py @@ -0,0 +1,245 @@ +"""Small dark blobs on a textured surface, found with a filter bank and no model. + +Directive ยง4: *"Do not use a neural model when deterministic signal processing +is better."* A tick on a cow is a small dark ellipse on hide, and finding small +dark ellipses is what a difference-of-Gaussians scale space has done since 1980. +It costs milliseconds where a tiled Grounding DINO pass costs minutes, it needs +no weights and no licence, and โ€” the part that matters for ยง25 โ€” it does not +have to be *right*. It has to be a **proposer**: cheap, generous, and biased +towards recall, so that an exemplar check or a multimodal verifier decides what +is actually a tick. + +That two-stage shape is the one the counting result argued for. CountGD did not +need a better frame; it needed to be shown once what the target looks like. A +proposer that finds every dark speck and an exemplar that says which specks +match is the same idea with the parts separated, and the separation is what lets +`experiments/cattle_ticks/` measure which half is failing. + +**What this cannot do, stated up front.** It has no idea what a tick is. On a +Friesian's flank it will propose every black patch edge; on a dusty hide it will +propose dirt; on an ear it will propose the shadow inside the ear. Every figure +in `experiments/cattle_ticks/` for this arm is a proposer's figure and the +false-positive rate is the interesting half of it, not an embarrassment. +""" + +from __future__ import annotations + +import numpy as np +from PIL import Image + +from app.adapters.base import Region + +#: Blob radii searched, in pixels, on the image as handed in. +#: +#: **These are pixel sizes, so they only mean something at a known scale**, and +#: that is the whole reason `tiles.py` tiles at a fixed pixel side rather than a +#: fraction. An engorged Rhipicephalus microplus female is around 8โ€“12 mm long +#: and an unfed one around 2โ€“3 mm; at a phone macro distance where a cow's ear +#: fills an 800 px tile โ€” roughly 150 mm across โ€” that is 10โ€“65 px. The range +#: below is wider at both ends because nothing here has measured a real capture +#: distance, and a proposer should over-propose. +RADII_PIXELS = (3.0, 4.5, 6.5, 9.0, 13.0, 18.0) + +#: Minimum response, as a multiple of the robust spread of the scale-space +#: response over the whole tile. Median plus kร—MAD, for the reason +#: `audio/features.adaptive_threshold` gives: the things being looked for are +#: the outliers, so a mean-and-standard-deviation threshold is raised by its own +#: targets. +RESPONSE_K = 4.0 + +#: Floor on the raw difference-of-Gaussians response, in units of image +#: intensity where full scale is 1.0. A blob must be an outlier for its tile +#: **and** actually visible. +#: +#: **Measured, and it does exactly one job.** On a synthetic field of Gaussian +#: noise at ฯƒ = 0.05 holding four discs of 0.45 contrast, this floor takes the +#: proposal count from 120 โ€” the cap, saturated by noise peaks โ€” to **4, the +#: four discs and nothing else**. On three real cattle photographs from +#: `evaluation/images/` it changes the count by **nothing at all**: 120 with it +#: and 120 without, because a real photograph's response distribution reaches +#: p99 0.102 and a maximum of 0.230, which is where the synthetic discs +#: themselves sit at 0.143โ€“0.150. +#: +#: So it removes noise-only proposals from a flat field and is inert on a real +#: image. It is not what separates ticks from hide, and nothing at this stage is. +MIN_ABSOLUTE_RESPONSE = 0.02 + +#: Most proposals one tile may return, strongest first. +#: +#: **On every real photograph tried this cap binds, and that is the honest +#: description of this stage rather than a defect.** Measured on +#: `cattle_ng_red_bororo.jpg` (1600 ร— 1067), `cattle_ng_kaduna_market_01.jpg` +#: and `cattle_ke_maasai.jpg`: 120 proposals whole-frame on each, and 720 raw +#: over six 800 px tiles merging to 445โ€“525. The proposer's job is recall, and +#: the discrimination happens in `exemplar.verify`. +#: +#: The number is a cost decision. Exemplar verification is a DINOv3 forward pass +#: per candidate at a **measured 122.5 ms** on a small crop under load โ€” not the +#: 59 ms `adapters/embedding.py` records for a whole frame โ€” so 120 per tile +#: over a 35-tile photograph is about **eight and a half minutes** of embedding +#: on a laptop CPU. An earlier version of this comment said 59 ms, 30 tiles and +#: 3.5 minutes, and was out by roughly a factor of two on each. +#: +#: Rank-and-truncate rather than a stricter threshold because a cap is a cost +#: bound that says what it is, and a threshold tuned until the count looks right +#: is a threshold tuned on the answer. +MAX_PER_TILE = 120 + +#: Centres closer than this multiple of the larger blob's radius are the same +#: blob found at two scales. +MERGE_RADIUS_FACTOR = 1.0 + +#: Label every proposal carries. Not `"tick"` โ€” that would be the proposer +#: claiming to have identified something, and it has not. `tiles.merge` compares +#: labels, so this also keeps proposals from merging with a detector's boxes. +LABEL = "blob" + +#: Which way round a blob has to be against its surroundings. +#: +#: **`both` is the default because the field images say it has to be.** The +#: obvious assumption is that a tick is a dark speck on pale hide, and on +#: `tick_cattle_calf_groin` โ€” dark Hyalomma on a calf's pale groin โ€” it is. On +#: `tick_cattle_hereford_neck` it is exactly backwards: engorged +#: Rhipicephalus microplus on a dark Hereford neck photograph as **bright** +#: specks against the coat. Same species of problem, opposite sign, and a +#: dark-only filter finds nothing at all on the second image. +#: +#: This was found by looking at the photographs rather than by reasoning about +#: ticks, and it is the single most useful thing the field set contributed to +#: the method. +POLARITY = ("dark", "light", "both") + + +def _scale_space(grey: np.ndarray, radii: tuple[float, ...]) -> np.ndarray: + """Scale-normalised Laplacian response at each radius, `(scales, h, w)`. + + Approximated by a difference of Gaussians, which is the standard and much + cheaper stand-in for the Laplacian of Gaussian. **Sign convention: positive + response means darker than surroundings**, which is what a tick is against + hide, and is why the difference is taken the way round it is. + + **The response is not multiplied by ฯƒยฒ, and an earlier version of this + function was.** Scale normalisation is genuinely required โ€” without it the + smallest scale wins everywhere โ€” but a difference of Gaussians already + carries it: DoG(ฯƒ, kฯƒ) โ‰ˆ (kโˆ’1)ยทฯƒยฒโˆ‡ยฒG, so the ฯƒยฒ is in the approximation. + Applying it a second time biases every response towards the largest radius, + and a smoke test on four synthetic discs of radius 5, 6, 9 and 13 px + reported **every one of them at the 18 px maximum**. Found because the test + checked the radii and not only the count. + """ + import cv2 + + responses = np.empty((len(radii), *grey.shape), dtype=np.float32) + for index, radius in enumerate(radii): + # ฯƒ = r/โˆš2 is the scale at which a DoG's response peaks for a circular + # blob of radius r. + sigma = float(radius) / np.sqrt(2.0) + inner = cv2.GaussianBlur(grey, (0, 0), sigmaX=sigma) + outer = cv2.GaussianBlur(grey, (0, 0), sigmaX=sigma * 1.6) + responses[index] = outer - inner + return responses + + +def propose( + image: Image.Image, + *, + radii: tuple[float, ...] = RADII_PIXELS, + response_k: float = RESPONSE_K, + max_proposals: int = MAX_PER_TILE, + polarity: str = "both", +) -> list[Region]: + """Candidate small dark blobs, strongest first, in image pixels. + + `score` is the response over the tile's own robust spread. **It is not a + probability and nothing calibrated it**, which is the same ยง37 position every + model score in this repository is in; the run records carry + `confidence_is_calibrated: false` and the harness enforces it. + + **The reported box is a size estimate and it runs about 30% small.** The + scale that peaks for a hard-edged disc is not the scale that peaks for the + Gaussian blob the filter is matched to. Measured on synthetic discs of + radius 5, 6, 9 and 13 px, the selected radii were 4.5, 4.5, 6.5 and 9.0 โ€” + the right ordering, consistently under. That is harmless for a proposer and + would not be harmless if a box from here were ever used to *measure* a tick, + which nothing does. + + Boxes are clipped to the frame, so a blob on an edge reports a box narrower + than its radius. Also harmless here, also worth knowing before reading a + width out of one. + """ + import cv2 + + grey = np.asarray(image.convert("L"), dtype=np.float32) / 255.0 + height, width = grey.shape + if height < 8 or width < 8: + return [] + + usable = tuple(r for r in radii if 2.0 * r < min(height, width)) + if not usable: + return [] + + if polarity not in POLARITY: + raise ValueError(f"polarity must be one of {POLARITY}, not {polarity!r}") + + responses = _scale_space(grey, usable) + if polarity == "light": + responses = -responses + elif polarity == "both": + # Magnitude, so a bright tick on a dark Hereford scores the same as a + # dark one on a pale calf. It costs the *sign*, which nothing here uses + # and which would be worth keeping the day something wants to know + # whether a hide is dark or pale. + responses = np.abs(responses) + + best_scale = np.argmax(responses, axis=0) + best = np.take_along_axis(responses, best_scale[None], axis=0)[0] + + centre = float(np.median(best)) + spread = float(np.median(np.abs(best - centre))) + if spread <= 0.0: + # A flat tile โ€” a blown-out highlight, a uniform background, a synthetic + # patch. No spread means no threshold can be set, and proposing + # everything would be worse than proposing nothing. + return [] + threshold = max(centre + response_k * spread, MIN_ABSOLUTE_RESPONSE) + + # Local maxima only. A 3x3 dilation equals the original exactly where a + # pixel is the largest in its neighbourhood, which is a peak test in one + # OpenCV call rather than a Python loop over a megapixel. + peak = (best >= cv2.dilate(best, np.ones((3, 3), np.uint8))) & (best > threshold) + ys, xs = np.nonzero(peak) + if ys.size == 0: + return [] + + order = np.argsort(-best[ys, xs]) + frame_area = float(width * height) + + kept: list[Region] = [] + centres: list[tuple[float, float, float]] = [] + for index in order: + y, x = float(ys[index]), float(xs[index]) + radius = float(usable[int(best_scale[int(ys[index]), int(xs[index])])]) + + # Same blob at two scales, or two peaks on one blob's shoulder. + duplicate = False + for cx, cy, cr in centres: + if (x - cx) ** 2 + (y - cy) ** 2 < ( + MERGE_RADIUS_FACTOR * max(radius, cr) + ) ** 2: + duplicate = True + break + if duplicate: + continue + + x0, y0 = max(0.0, x - radius), max(0.0, y - radius) + x1, y1 = min(float(width), x + radius), min(float(height), y + radius) + kept.append(Region( + label=LABEL, + score=round(float((best[int(y), int(x)] - centre) / spread), 4), + box=(x0, y0, x1, y1), + area_fraction=((x1 - x0) * (y1 - y0)) / frame_area, + )) + centres.append((x, y, radius)) + if len(kept) >= max_proposals: + break + return kept diff --git a/app/adapters/tiled/exemplar.py b/app/adapters/tiled/exemplar.py new file mode 100644 index 0000000000000000000000000000000000000000..b8e49f4cb9f95a0f3e11e5763e29b16cdddc6188 --- /dev/null +++ b/app/adapters/tiled/exemplar.py @@ -0,0 +1,230 @@ +"""Showing the system one example, and asking which candidates look like it. + +**This is the lesson from counting, applied to a different problem.** +`experiments/poultry_house_count/` is the strongest result in this tree: a COCO +detector was out by 157 birds a frame on commercial broiler-house imagery, and +CountGD given **three exemplar boxes** was out by 15. The exemplars were not +from the frame being counted โ€” they came from one frame in a different split, +fixed once and reused for all 452 โ€” and that arm was the *best* of the three. The +conclusion in that README is the one this module is built on: *"Nothing CountGD +needs is specific to the frame in front of it. It needs to be shown, once, what +a bird looks like from this camera."* + +Directive ยง25 asks for the same mechanism by name โ€” **SAM 3.1 visual exemplar +prompting** โ€” for ticks. SAM 3 is gated behind manual Meta approval, needs an +HF token a build cannot obtain unattended, and is a 3.44 GB checkpoint that +wants a CUDA 12.6 GPU. It was not reachable from this machine and +`experiments/cattle_ticks/README.md` records that as an attempt that could not be +made rather than as one that failed. + +So the exemplar mechanism here is built from what ยง3 *does* make reachable: a +frozen DINOv3 embedding and a cosine similarity. Embed one crop of a tick, embed +each candidate, keep the candidates whose vector points the same way. It is +weaker than SAM 3's โ€” there is no segmentation and no joint text-and-exemplar +conditioning โ€” and it is the same idea. + +## The three ways this is honest about what it is + +**It never proposes.** It only ranks and filters what a proposer found, so its +recall ceiling is the proposer's recall. A tick the blob filter missed cannot be +recovered here, and the experiment reports both numbers separately for exactly +that reason. + +**A similarity is not a probability.** Cosine between two frozen embeddings is +an uncalibrated number in [-1, 1] whose useful range depends entirely on what +the exemplars were. ยง37 forbids it reaching a farm, and the run records say +`confidence_is_calibrated: false`. + +**The exemplars are part of the method.** Change them and every figure changes. +So `ExemplarBank` records what each vector came from and hashes the images, and +the experiment pins that hash in its run record โ€” the same discipline +`poultry_house_count` applies to its three boxes from `train[0]`. +""" + +from __future__ import annotations + +import hashlib +from dataclasses import dataclass, replace + +import numpy as np +from PIL import Image + +from app.adapters.base import Embedder, Region + +#: How much larger than the candidate box the embedded crop is. +#: +#: **Not 1.0, and the reason is what the backbone is.** DINOv3 was trained on +#: photographs of things in context, and its embedding of a 12 px brown speck +#: filling the frame is dominated by colour and blur. Giving it the speck plus +#: the hide around it produces a vector about *a tick on a cow*, which is the +#: thing being matched. 1.8 is a starting point, swept in +#: `experiments/cattle_ticks/`, and the sweep is in that directory's config. +CONTEXT_FACTOR = 1.8 + +#: Smallest crop, in pixels, handed to the backbone. Below roughly this size the +#: 224 px resize is pure upsampling and the vector is describing interpolation. +MIN_CROP_PIXELS = 24 + + +@dataclass(frozen=True) +class ExemplarBank: + """Unit vectors for a handful of example crops, and where they came from. + + Frozen and content-addressed for the same reason `provenance.InputSet` is: + an accuracy figure measured against one set of exemplars is not an accuracy + figure for another, and a bank identified only by a variable name drifts + without anybody noticing. + """ + + #: `(n, d)`, L2-normalised. The embedding adapters normalise inside the ONNX + #: graph, so this is asserted rather than re-done. + vectors: np.ndarray + #: One per row: a human-readable id for the crop it came from. + ids: tuple[str, ...] + #: sha256 over the raw crop bytes, in row order. What pins the bank. + digests: tuple[str, ...] + #: Which backbone produced the vectors. A bank built with DINOv3 and queried + #: with DINOv2 is a silent failure, and this is what makes it a loud one. + embedder_id: str + + def __post_init__(self) -> None: + if self.vectors.ndim != 2 or self.vectors.shape[0] == 0: + raise ValueError( + "An exemplar bank with no exemplars in it is a bank that will " + "match nothing and report a threshold failure. Build it with at " + "least one crop." + ) + norms = np.linalg.norm(self.vectors, axis=1) + if not np.allclose(norms, 1.0, atol=1e-3): + raise ValueError( + f"Exemplar vectors are not unit length (norms " + f"{norms.min():.4f}โ€“{norms.max():.4f}). Cosine similarity here " + f"is a dot product and assumes they are." + ) + + @property + def digest(self) -> str: + """One hash over the whole bank, in a stable order.""" + joined = "\n".join( + f"{i}:{d}" for i, d in sorted(zip(self.ids, self.digests)) + ) + return hashlib.sha256(joined.encode("utf-8")).hexdigest() + + def similarity(self, vector: np.ndarray) -> tuple[float, str]: + """Best cosine against the bank, and which exemplar gave it. + + Maximum rather than mean. Ticks differ enormously by species, sex and + engorgement โ€” a flat unfed male and a grey engorged female do not look + alike โ€” so a bank is a set of appearances rather than a cluster with a + centre, and averaging them produces a vector that matches none of them. + """ + scores = self.vectors @ np.asarray(vector, dtype=np.float32) + best = int(np.argmax(scores)) + return float(scores[best]), self.ids[best] + + +@dataclass(frozen=True) +class Verified: + """One candidate, and what the exemplar check said about it. + + Both scores are kept. The proposer's response and the exemplar's similarity + fail in different directions โ€” a strong blob that looks nothing like a tick, + a faint blob that looks exactly like one โ€” and collapsing them into a single + number loses the ability to say which stage is wrong. + """ + + region: Region + similarity: float + nearest_exemplar: str + proposer_score: float + passed: bool + + +def _crop(image: Image.Image, box: tuple[float, float, float, float], + context: float) -> Image.Image: + """The candidate plus context, square, clipped to the frame. + + Square because the backbone centre-crops to a square anyway, and letting it + do that on a wide crop silently discards one axis of the context this + function exists to add. + """ + x0, y0, x1, y1 = box + cx, cy = (x0 + x1) / 2.0, (y0 + y1) / 2.0 + half = max(x1 - x0, y1 - y0) * context / 2.0 + half = max(half, MIN_CROP_PIXELS / 2.0) + + width, height = image.size + left = int(max(0, round(cx - half))) + top = int(max(0, round(cy - half))) + right = int(min(width, round(cx + half))) + bottom = int(min(height, round(cy + half))) + if right - left < 2 or bottom - top < 2: + return image.crop((0, 0, min(2, width), min(2, height))) + return image.crop((left, top, right, bottom)) + + +def build_bank( + embedder: Embedder, + crops: list[Image.Image], + ids: list[str], + *, + embedder_id: str, +) -> ExemplarBank: + """Embed example images once. The bank is then reused for every candidate. + + `crops` are whole images of the target โ€” an isolated tick, a close-up of one + on an ear โ€” not boxes on a scene. Cropping is the caller's decision, because + what counts as "the exemplar" is exactly the thing an experiment has to pin + and vary. + """ + if len(crops) != len(ids): + raise ValueError(f"{len(crops)} crops against {len(ids)} ids.") + + vectors = np.stack([embedder.embed(crop) for crop in crops]).astype(np.float32) + digests = tuple( + hashlib.sha256(crop.convert("RGB").tobytes()).hexdigest() for crop in crops + ) + return ExemplarBank( + vectors=vectors, ids=tuple(ids), digests=digests, embedder_id=embedder_id + ) + + +def verify( + embedder: Embedder, + image: Image.Image, + candidates: list[Region], + bank: ExemplarBank, + *, + threshold: float, + context: float = CONTEXT_FACTOR, +) -> list[Verified]: + """Score every candidate against the bank. Nothing is dropped. + + **Rejected candidates come back too**, carrying `passed=False`. A run that + proposed 400 blobs and passed 3 is a completely different situation from one + that proposed 3, and a function that returned only the survivors would make + the two indistinguishable in the record. The experiment stores both counts; + `app/counting.py` and `audio/events.py` keep their rejects for the same + reason. + + The label is rewritten to `probable_tick` on the ones that pass, so a merged + result cannot mix a verified candidate with a raw proposal โ€” `tiles.merge` + compares labels, and two objects with different labels are two objects. + """ + out: list[Verified] = [] + for candidate in candidates: + vector = embedder.embed(_crop(image, candidate.box, context)) + similarity, nearest = bank.similarity(vector) + passed = similarity >= threshold + out.append(Verified( + region=replace( + candidate, + label="probable_tick" if passed else candidate.label, + score=round(similarity, 4), + ), + similarity=round(similarity, 6), + nearest_exemplar=nearest, + proposer_score=candidate.score, + passed=passed, + )) + return out diff --git a/app/adapters/tiled/openvocab.py b/app/adapters/tiled/openvocab.py new file mode 100644 index 0000000000000000000000000000000000000000..f68d52240d6fabc2ff60486d8088fcf20466f65c --- /dev/null +++ b/app/adapters/tiled/openvocab.py @@ -0,0 +1,253 @@ +"""Grounding DINO, prompted with a word, as an `OpenVocabularyDetector`. + +Directive ยง4 names it for *"open-vocabulary object detection, fallback detection +when SAM concept prompting is weak, text-prompted localization"*, and ยง25 names +it for ticks specifically. It is also the only model in the whole ยง3/ยง4 stack +that is Apache-2.0 for code, weights and text encoder alike, ungated, and small +enough to run on a laptop โ€” `adapters/licences.py:grounding-dino-hf` has the +detail. + +**This is a real adapter and `adapters/unavailable.py` still lists a placeholder +for the same model.** That is deliberate and it is not a contradiction: +`GROUNDING_DINO_SPEC` there describes an adapter registered in +`adapters/registry.py` and driven from a `models/` card, which is the shape +every served model in this service has, and which this one does not have yet. +Nothing here is wired into `registry.all_adapters()`, so `/health` and +`/capabilities` report exactly what they reported before. What this file adds is +a way for `experiments/cattle_ticks/` to drive the model that ยง25 asks for, from +code that is in the diff. Promoting it to a registered adapter means exporting +it and writing a card, and that belongs to whoever owns `registry.py`. + +## Two things worth knowing before reading a number out of it + +**It is not a counter.** `experiments/poultry_house_count/` measured this exact +model, at this exact port, on 452 annotated frames: MAE 158.37 against a COCO +detector's 156.80, finding nothing at all in 222 of them. Its own paper's +Table 1 puts text-only Grounding DINO at FSC-147 val MAE 54.45 where CountGD +scores 7.10. Prompting an open-vocabulary detector with a word is not counting, +and this repository has the receipt. + +**Its box score is not a probability.** It is a sigmoid over a text-conditioned +logit that nothing has calibrated โ€” not on ticks, not on cattle, not on +anything. ยง37 forbids it reaching a farm as a confidence, and the run records in +`experiments/cattle_ticks/results/` carry `confidence_is_calibrated: false` so +the harness enforces that rather than trusting a docstring. +""" + +from __future__ import annotations + +from dataclasses import dataclass + +from PIL import Image + +from app.adapters.base import ( + Adapter, + AdapterError, + AdapterSpec, + AdapterUnavailable, + Availability, + Modality, + Placement, + Region, + Task, +) + +#: The port used throughout. `-tiny` rather than `-base` because it is what is +#: in the Hugging Face cache on the development machine and because +#: `experiments/poultry_house_count/` already benchmarked this exact checkpoint, +#: so a figure here is comparable with one there. `-base` is the stronger model +#: and nobody has run it; both directories say so. +MODEL_ID = "IDEA-Research/grounding-dino-tiny" + +#: Box and text thresholds. 0.3 is the value in the model card's own usage +#: example, used unchanged so it cannot have been tuned to any set here. It is +#: also what `experiments/poultry_house_count/` used, which is the other reason +#: not to move it. +THRESHOLD = 0.3 + +GROUNDING_DINO_TILED_SPEC = AdapterSpec( + adapter_id="grounding-dino-tiny-openvocab", + runtime="grounding-dino-hf", + tasks=(Task.DETECT,), + modalities=(Modality.IMAGE,), + directive_role=( + "ยง4 Grounding DINO โ€” open-vocabulary detection and text-prompted " + "localisation. ยง25 names it for tick detection over tiled close-ups, " + "which is what `adapters/tiled/` drives it for." + ), + placement=Placement.GPU_SERVICE, + requires_gpu=False, + placement_reason=( + "It runs on CPU โ€” the transformers port's deformable attention is plain " + "`grid_sample` with no CUDA extension โ€” and should not. ADR-adjacent " + "measurement in `adapters/unavailable.py` records 5.5 s a frame and a " + "2,121 MB peak on one whole frame, against a 4 GiB container also " + "holding the media. **Tiling multiplies that by the tile count**, which " + "is the cost that decides the placement: a 4,000 ร— 3,000 photograph is " + "30 tiles at an 800 px side and a 4,598 ร— 2,997 one is 35, so a tiled " + "pass is minutes on CPU and seconds on a GPU. This is a queued GPU " + "capability." + ), + notes=( + "172,250,626 parameters, 657 MiB of safetensors, Apache-2.0 for code, " + "weights and the BERT text encoder alike โ€” the cleanest licence in the " + "ยง3/ยง4 stack. Not registered in `adapters/registry.py`: there is no " + "`models/` card and no ONNX export, so the CPU worker's no-torch " + "property (ADR 0017) would be lost by wiring it in as it stands." + ), +) + + +@dataclass(frozen=True) +class _Loaded: + processor: object + model: object + device: str + + +class GroundingDinoAdapter(Adapter): + """Text-prompted detection on one image. Tiling is `tiled.TiledDetector`'s job. + + Kept single-image on purpose. Mixing the prompt-and-threshold concerns with + the cut-up-and-merge concerns is how a tiling bug becomes indistinguishable + from a prompting bug, and `experiments/cattle_ticks/` has to be able to run + this model with tiling on and off to say what tiling did. + """ + + spec = GROUNDING_DINO_TILED_SPEC + + def __init__( + self, + *, + model_id: str = MODEL_ID, + device: str = "cpu", + threshold: float = THRESHOLD, + ) -> None: + self.model_id = model_id + self.device = device + self.threshold = threshold + self._loaded: _Loaded | None = None + + def availability(self) -> Availability: + """Whether this could run without a download, checked without loading. + + Deliberately does not import torch. `/health` calling this must not pay + for a two-second import, and โ€” more to the point โ€” an adapter that + reports availability by loading the thing has no way to report that + loading it failed. + """ + try: + from app.adapters.licences import LicenceRefused, gate + except ImportError as absent: # pragma: no cover โ€” package always present + return Availability(False, f"licence table unreadable: {absent}", "") + try: + gate(self.spec.runtime) + except LicenceRefused as refusal: + return Availability( + False, str(refusal), + "This one does not become available by installing it.", + ) + + try: + import transformers # noqa: F401 + except ImportError: + return Availability( + False, + "transformers is not installed, so the Grounding DINO port " + "cannot be built.", + "pip install transformers โ€” nothing here is gated and nothing " + "needs a token.", + ) + + try: + from huggingface_hub import try_to_load_from_cache + except ImportError: # pragma: no cover โ€” a transformers dependency + return Availability(True) + + cached = try_to_load_from_cache(self.model_id, "config.json") + if not isinstance(cached, str): + return Availability( + False, + f"{self.model_id} is not in the Hugging Face cache, so running " + f"it needs a 657 MiB download.", + f"huggingface-cli download {self.model_id}, or run once with " + f"network access.", + ) + return Availability(True) + + def load(self) -> "GroundingDinoAdapter": + state = self.availability() + if not state.ready: + raise AdapterUnavailable(state) + if self._loaded is not None: + return self + + from transformers import AutoModelForZeroShotObjectDetection, AutoProcessor + + processor = AutoProcessor.from_pretrained(self.model_id) + model = AutoModelForZeroShotObjectDetection.from_pretrained(self.model_id) + model = model.to(self.device) + model.eval() + # Held rather than re-read per call: session construction is 16.3 s and + # 948 MB before a single frame, and a tiled pass makes dozens of calls. + self._loaded = _Loaded(processor=processor, model=model, device=self.device) + return self + + def detect_text( + self, image: Image.Image, prompts: tuple[str, ...] + ) -> list[Region]: + """Boxes for each prompt, in image pixels. + + Grounding DINO wants a caption, not a class list: the documented form is + lower-case phrases separated by " . " and terminated by one. Building it + here rather than asking the caller to means a caller cannot silently + prompt the model with something it was not trained to parse. + """ + if self._loaded is None: + raise AdapterError( + "Call load() before detect_text(). There is no path from an " + "unloaded adapter to a detection, which is the point." + ) + if not prompts: + return [] + + import torch + + loaded = self._loaded + # Lower-cased and dot-terminated per the model card. A capitalised + # prompt tokenises differently and the model's own examples are lower + # case throughout. + caption = " . ".join(p.strip().strip(".").lower() for p in prompts) + " ." + rgb = image.convert("RGB") + inputs = loaded.processor( + images=rgb, text=caption, return_tensors="pt" + ).to(loaded.device) + + with torch.no_grad(): + outputs = loaded.model(**inputs) + + results = loaded.processor.post_process_grounded_object_detection( + outputs, + inputs.input_ids, + threshold=self.threshold, + text_threshold=self.threshold, + target_sizes=[rgb.size[::-1]], + )[0] + + frame_area = float(rgb.size[0] * rgb.size[1]) or 1.0 + regions: list[Region] = [] + for box, score, label in zip( + results["boxes"].cpu().tolist(), + results["scores"].cpu().tolist(), + # `text_labels` in transformers v5; `labels` was the v4 name and + # holds token ids there, which would silently become integer labels. + results.get("text_labels", results.get("labels", [])), + ): + x0, y0, x1, y1 = (float(v) for v in box) + regions.append(Region( + label=str(label), + score=float(score), + box=(x0, y0, x1, y1), + area_fraction=((x1 - x0) * (y1 - y0)) / frame_area, + )) + return regions diff --git a/app/adapters/tiled/tiles.py b/app/adapters/tiled/tiles.py new file mode 100644 index 0000000000000000000000000000000000000000..bf41a1effd85ac18ac7fb7a2e3e1c584432f7027 --- /dev/null +++ b/app/adapters/tiled/tiles.py @@ -0,0 +1,291 @@ +"""Cutting a high-resolution photograph up so a small object survives the resize. + +Directive ยง25 lists **high-resolution tiled inference** as one of four things to +build for cattle ticks, and it is there for a concrete reason. A detector reads +a fixed input โ€” Grounding DINO's shortest side is 800 px, DINOv3's is 224 โ€” +so a 4,000 px close-up of a cow's ear is downsampled by five before the network +sees it, and a 40 px tick arrives as 8 px. Cut the same photograph into 800 px +tiles and the tick arrives at 40 px. Nothing about the model changed; the +object got five times larger. + +## Why this is not `app/tiling.py` + +That module exists and does the same arithmetic, and it is the right module for +what it does. It is bound to `detectors.Detection` and to a **fractional grid** โ€” +1ร—1, 2ร—2, 3ร—3 โ€” which is the correct control when the question is *"has the +count converged?"* over an unknown frame. It is the wrong control here. + +Ticks are a fixed-size object in an unknown-resolution frame, so what matters is +the ratio between the tick's pixels and the model's input, and that is set by a +**tile side in pixels**, not by a fraction of the picture. A 3ร—3 grid over a +1,200 px phone photo gives 400 px tiles that get *upsampled*; the same grid over +an 8,000 px macro gives 2,600 px tiles that are still downsampled by three. The +grid is the same and the thing that matters is not. + +So this module tiles at a fixed pixel side, and works on +`adapters.base.Region`, which carries a mask and comes from the open-vocabulary +and exemplar routes ยง25 actually names. + +## The merge, and the trap `app/tiling.py` already fell into + +Overlapping tiles find the same object twice, so duplicates have to go. IoU +alone does not do it: an object cut into quarters by tile boundaries gives four +quarter-boxes that barely overlap *each other* and survive an IoU test, and the +whole-frame box that contains all four is what removes them. That is why the +merge is **largest box first** and why containment is a separate rule from IoU. +Both were learned in `app/tiling.py` โ€” its comment records one cow becoming +three โ€” and re-deriving them here rather than importing them would have been +the same bug twice. + +The thresholds are deliberately the same values, for one reason worth stating: +nobody has re-derived them for objects of this size, and inventing new numbers +would have made it look as though somebody had. +""" + +from __future__ import annotations + +from dataclasses import dataclass, replace + +import numpy as np +from PIL import Image + +from app.adapters.base import Region + +#: Default tile side, in source pixels. Grounding DINO resizes its input to a +#: shortest side of 800, so an 800 px tile passes through at roughly 1:1 and a +#: tick keeps every pixel the camera gave it. Larger tiles throw resolution +#: away; smaller ones multiply the inference count without adding detail, and +#: cost is linear in tile count. +TILE_PIXELS = 800 + +#: How much neighbouring tiles overlap, as a share of the tile side. An object +#: sitting exactly on a cut would otherwise be two partial objects, neither +#: complete enough to detect. 0.20 matches `app/tiling.py`; at an 800 px tile it +#: is 160 px, which is four times the longest tick measured on the set in +#: `experiments/cattle_ticks/`, so no tick can straddle a seam without being +#: whole in at least one tile. +TILE_OVERLAP = 0.20 + +#: IoU above which two boxes are the same object. From `app/tiling.py`, which +#: derived it against cattle; **not re-derived for objects this small**, and the +#: cattle_ticks README lists that as a gap rather than hiding it here. +MERGE_IOU = 0.55 + +#: Intersection over the *smaller* box's area, above which the smaller box is +#: part of the larger rather than a second object. This is the rule that stops +#: an object cut across tile seams becoming several objects. See the module +#: docstring. +MERGE_CONTAINMENT = 0.85 + +#: Centre separation, as a fraction of the **larger** box's mean side, below +#: which two same-label regions are the same object. +#: +#: **The rule `app/tiling.py` does not have, added because small objects break +#: the two that it does.** IoU is a poor duplicate test at small sizes: two +#: 36 px boxes on the same tick, found in overlapping tiles and disagreeing by +#: 14 px, score IoU 0.44 and containment 0.61 โ€” under both thresholds, so the +#: tick is counted twice. That is what a smoke test on synthetic discs produced: +#: 208 raw regions merging to 150, where the four real discs appeared two or +#: three times each. The same 14 px offset on a cow-sized box is nothing, which +#: is why `app/tiling.py` never needed this and why adding it there is not +#: implied. +#: +#: **Normalised by the larger box, not the smaller, and that is what makes 0.6 +#: work.** Two detections of one 9 px-radius disc, found in overlapping tiles at +#: radii 4.5 and 6.5 with centres 7 px apart, score 0.54 against the larger box +#: and 0.78 against the smaller โ€” so only the larger normalisation separates +#: them from the case that must survive: two ticks touching. Touching circles of +#: equal radius r have centres 2r apart over a box side of 2r, which is exactly +#: 1.0, well clear of 0.6. Objects that overlap by half score 0.5 and merge, +#: which for small round objects found twice across a seam is the right answer. +MERGE_CENTRE_DISTANCE = 0.6 + + +@dataclass(frozen=True) +class Tile: + """One crop, and where it sits in the source frame.""" + + index: int + #: `(x0, y0, x1, y1)` in source-image pixels, left-top inclusive. + box: tuple[int, int, int, int] + #: Grid position, for a caller that wants to lay results out. + row: int + column: int + + @property + def origin(self) -> tuple[int, int]: + return self.box[0], self.box[1] + + @property + def size(self) -> tuple[int, int]: + return self.box[2] - self.box[0], self.box[3] - self.box[1] + + def crop(self, image: Image.Image) -> Image.Image: + return image.crop(self.box) + + +def tiles_for( + size: tuple[int, int], + *, + tile_pixels: int = TILE_PIXELS, + overlap: float = TILE_OVERLAP, +) -> list[Tile]: + """Cover a frame with overlapping fixed-size tiles. + + A frame smaller than one tile returns a single tile covering all of it, + which is whole-frame inference โ€” the honest degenerate case, and the one a + phone photo that has already been downscaled by a messaging app will hit. + + The last tile in each direction is pulled back to the frame edge rather than + being a narrow remainder. A 120 px strip is a strip in which nothing is + detectable and which still costs a full inference. + """ + width, height = int(size[0]), int(size[1]) + if width <= 0 or height <= 0: + return [] + tile_pixels = max(1, int(tile_pixels)) + if width <= tile_pixels and height <= tile_pixels: + return [Tile(index=0, box=(0, 0, width, height), row=0, column=0)] + + step = max(1, int(round(tile_pixels * (1.0 - overlap)))) + + def starts(extent: int) -> list[int]: + if extent <= tile_pixels: + return [0] + positions = list(range(0, extent - tile_pixels, step)) + # The final position is always flush with the far edge, so the last + # tile is a full tile rather than a remainder. + positions.append(extent - tile_pixels) + return positions + + found: list[Tile] = [] + for row, y0 in enumerate(starts(height)): + for column, x0 in enumerate(starts(width)): + found.append(Tile( + index=len(found), + box=(x0, y0, min(width, x0 + tile_pixels), + min(height, y0 + tile_pixels)), + row=row, column=column, + )) + return found + + +def to_frame(region: Region, tile: Tile, frame_size: tuple[int, int]) -> Region: + """Move a tile-relative region into source-frame coordinates. + + `area_fraction` is recomputed against the **whole frame**, not the tile. A + tick covering a fiftieth of its tile covers a two-thousandth of an 8,000 px + photograph, and everything downstream โ€” the quality gates, the burden band โ€” + reasons about the photograph. + + A mask, when one is present, is pasted into a frame-sized array rather than + being returned tile-relative. A boolean array whose coordinate system + depends on which tile produced it is the kind of thing that works until two + tiles disagree. + """ + x0, y0 = tile.origin + bx0, by0, bx1, by1 = region.box + box = (bx0 + x0, by0 + y0, bx1 + x0, by1 + y0) + frame_area = float(frame_size[0] * frame_size[1]) or 1.0 + + mask = region.mask + if mask is not None: + placed = np.zeros((frame_size[1], frame_size[0]), dtype=bool) + tile_width, tile_height = tile.size + placed[y0:y0 + tile_height, x0:x0 + tile_width] = mask[:tile_height, :tile_width] + mask = placed + + return replace( + region, + box=box, + mask=mask, + area_fraction=((box[2] - box[0]) * (box[3] - box[1])) / frame_area, + ) + + +def _overlaps( + a: tuple[float, float, float, float], b: tuple[float, float, float, float] +) -> tuple[float, float, float]: + """`(IoU, intersection over the smaller area, centre distance in objects)`. + + The third is the centre separation divided by the **larger** box's mean + side, so it is dimensionless and comparable between a tick and a cow. See + `MERGE_CENTRE_DISTANCE` for why the larger and not the smaller. It is + computed even when the boxes do not intersect, because two boxes can be + nearly concentric and still score zero on the first two when one is a + sliver. + """ + ax0, ay0, ax1, ay1 = a + bx0, by0, bx1, by1 = b + area_a = (ax1 - ax0) * (ay1 - ay0) + area_b = (bx1 - bx0) * (by1 - by0) + + centre_gap = float(np.hypot( + (ax0 + ax1) / 2.0 - (bx0 + bx1) / 2.0, + (ay0 + ay1) / 2.0 - (by0 + by1) / 2.0, + )) + larger_side = max( + ((ax1 - ax0) + (ay1 - ay0)) / 2.0, + ((bx1 - bx0) + (by1 - by0)) / 2.0, + ) + separation = centre_gap / larger_side if larger_side > 0 else float("inf") + + x0, y0 = max(ax0, bx0), max(ay0, by0) + x1, y1 = min(ax1, bx1), min(ay1, by1) + overlap = max(0.0, x1 - x0) * max(0.0, y1 - y0) + if overlap <= 0.0: + return 0.0, 0.0, separation + union = area_a + area_b - overlap + smaller = min(area_a, area_b) + return ( + overlap / union if union > 0 else 0.0, + overlap / smaller if smaller > 0 else 0.0, + separation, + ) + + +def merge( + regions: list[Region], + frame_size: tuple[int, int], + *, + iou: float = MERGE_IOU, + containment: float = MERGE_CONTAINMENT, + centre_distance: float = MERGE_CENTRE_DISTANCE, +) -> list[Region]: + """One object, one region, whichever tile found it. + + **Largest box first**, which is the ordering the containment rule needs: the + whole object has to be in the kept set before its fragments are tested + against it. Score order โ€” the usual choice for non-maximum suppression โ€” + would let a confident fragment claim the object and leave its siblings + unmatched, which is how one cow became three in `app/tiling.py`'s first + attempt. + + Labels are compared, so a tick overlapping a tag is two objects. Comparison + is case-insensitive because open-vocabulary detectors return the caption's + own casing and a caller prompting `"tick"` and `"Tick"` did not mean two + classes. + """ + def area(region: Region) -> float: + x0, y0, x1, y1 = region.box + return (x1 - x0) * (y1 - y0) + + frame_area = float(frame_size[0] * frame_size[1]) or 1.0 + kept: list[Region] = [] + for region in sorted(regions, key=area, reverse=True): + duplicate = False + for other in kept: + if other.label.casefold() != region.label.casefold(): + continue + overlap_iou, overlap_containment, separation = _overlaps( + other.box, region.box + ) + if (overlap_iou > iou or overlap_containment > containment + or separation < centre_distance): + duplicate = True + break + if duplicate: + continue + kept.append(replace(region, area_fraction=area(region) / frame_area)) + kept.sort(key=lambda r: r.score, reverse=True) + return kept diff --git a/app/adapters/transports/__init__.py b/app/adapters/transports/__init__.py new file mode 100644 index 0000000000000000000000000000000000000000..cbd20cadccdbfd912250ba5b30db5a80168a0044 --- /dev/null +++ b/app/adapters/transports/__init__.py @@ -0,0 +1,115 @@ +"""How a hosted reasoner is actually called, and which one. + +`app/adapters/multimodal.py` has the whole contract โ€” the rubric, the closed +schema, the structured-output validation, the forbidden-claims gate โ€” and takes +an injectable `transport: Callable[..., str]`. Until this package existed, +`registry.py` constructed it with **none**. There were zero HTTP calls in the +service and nothing had ever talked to a hosted model; the adapter reported +itself `unavailable` and fifteen capabilities waited on it. + +This package is the missing half, and nothing else. The transport's whole job: + + (prompt, images, schema, model, api_key) -> the model's response text + +Everything on either side of that already exists. A transport does not decide +what to ask, does not read the registry, does not interpret an answer and does +not get to relax a check โ€” `claims.parse_strict` and `claims.enforce` run on +whatever it returns, from the caller, where a transport cannot reach them. + +## Why the provider is a choice and not an import + +Choosing a vendor commits Animap to that vendor's retention posture for +photographs of somebody's animals, leaving the country. That is a procurement +decision rather than an engineering one, and `multimodal.py` refuses to make it +by importing an SDK โ€” which is why the transport is injected at all. + +So the shape here is a **registry keyed on a name**, and the name comes from the +environment: + + ANIMAP_MULTIMODAL_PROVIDER=anthropic + +An unset provider is the honest `unavailable` that was there before, not a +default that quietly picks somebody. A provider this build does not implement is +an error naming the ones it does, because a typo that silently disabled fifteen +capabilities would look exactly like the state this package exists to leave. + +## What a provider has to do to belong here + +Three things, and the third is the one that decides whether a capability is +worth shipping at all: + + * **Take images and a prompt.** All fifteen waiting capabilities are visual. + * **Return one JSON object matching a supplied schema**, natively rather than + by being asked nicely in prose. Directive ยง4 is explicit โ€” *"All calls must + return structured JSON"* โ€” and a provider without server-side schema + enforcement makes `claims.parse_strict` the only thing standing between a + farm and a model's prose, which is a retry loop rather than a contract. + * **Actually see what is in the photograph.** A model that cannot tell a + raised nodular lesion from mud on a flank produces confident, well-formed, + wrong JSON, and every gate downstream will pass it: the schema is closed, + the claim is in the vocabulary, the sentence is qualified. Nothing in this + service can catch a plausible wrong answer, and that is why the choice of + model is a capability decision rather than a cost one. +""" + +from __future__ import annotations + +import os +from typing import Callable + +#: Which hosted provider to call. Read at call time rather than at import, for +#: the same reason `multimodal.API_KEY_ENV` is: a change takes effect on a +#: restart rather than needing a rebuild. +PROVIDER_ENV = "ANIMAP_MULTIMODAL_PROVIDER" + + +class UnknownProvider(ValueError): + """A provider name this build has no transport for.""" + + +def _anthropic(): + from app.adapters.transports.anthropic_transport import anthropic_transport + + return anthropic_transport + + +#: Name -> a factory returning the transport callable. +#: +#: Factories rather than the callables themselves, so importing this module does +#: not import a vendor SDK. The inference container runs on CPU with no torch +#: and starts in under a second; that is a property worth keeping, and a +#: deployment that has chosen no provider should not be paying for an import it +#: will never call. +PROVIDERS: dict[str, Callable[[], Callable[..., str]]] = { + "anthropic": _anthropic, +} + + +def configured_provider() -> str: + """The provider name in the environment, lower-cased, or empty.""" + return os.environ.get(PROVIDER_ENV, "").strip().lower() + + +def transport_from_env() -> Callable[..., str] | None: + """The transport this deployment has chosen, or None if it has chosen none. + + None rather than a default, and it is the whole posture of this package: + an unset provider leaves `HostedMultimodalAdapter` reporting itself + unavailable with the reason it always gave, and no farm gets an answer from + a vendor nobody picked. + + Raises `UnknownProvider` for a name this build cannot serve. Loud, because + the failure it replaces is silent: a typo would leave fifteen capabilities + off with a listing that says *"no vendor is wired"* โ€” indistinguishable from + a deployment where nobody had chosen one yet. + """ + name = configured_provider() + if not name: + return None + factory = PROVIDERS.get(name) + if factory is None: + raise UnknownProvider( + f"{PROVIDER_ENV} is {name!r}, which this build has no transport " + f"for. It knows: {', '.join(sorted(PROVIDERS))}." + ) + return factory() diff --git a/app/adapters/transports/anthropic_transport.py b/app/adapters/transports/anthropic_transport.py new file mode 100644 index 0000000000000000000000000000000000000000..4784baee44192a2b56e44871c5a901251b6e897a --- /dev/null +++ b/app/adapters/transports/anthropic_transport.py @@ -0,0 +1,342 @@ +"""The Claude transport: images and a rubric in, one JSON object out. + +The first concrete provider behind `HostedMultimodalAdapter`, and the reason it +is first is the third requirement in this package's header โ€” a model that cannot +actually see what is in a photograph produces confident, well-formed, wrong JSON +that every gate downstream will pass. + +## What this file does and does not decide + +It does not build the prompt, does not build the schema, does not read the +capability registry, and does not judge the answer. `multimodal.reason` composes +`SYSTEM_RULES` with the capability's rubric, `claims.schema_for` builds the +closed schema from the registry, and `claims.parse_strict` plus `claims.enforce` +run on whatever comes back โ€” from the caller, where this module cannot reach +them. Everything here is the HTTP call and the shape of its arguments. + +## The schema the API accepts is narrower than the one the contract enforces + +`claims.schema_for` emits a full JSON Schema. Structured outputs accepts most of +it and refuses four things, each verified against the live API rather than read +off a page: + + array maxItems not supported + array minItems, where the value is > 1 not supported + number minimum / maximum / multipleOf not supported + enum containing null beside a union type โ€” "Enum value 'a' does not match + declared type '['string', 'null']'" + +`_for_structured_output` removes exactly those and changes nothing else. + +The `minItems` rule was the one a keyword probe missed: `minItems: 1` is +accepted, so only a real capability's schema โ€” `range`, which is exactly two +entries โ€” produced *"values other than 0 or 1 are not supported"*. + +**This narrows what the model is guided by and not what the answer is judged +against.** `multimodal.reason` calls `claims.enforce(parsed, contract, โ€ฆ)` with +the **original** schema, and `claims.validate` implements `maxItems`, `minimum`, +`maximum` and `multipleOf` itself โ€” the `2.6347` a watchdog once published still +fails on `multipleOf`, from the same line it always did. What is lost is the +model being *told* the bound up front, which costs a retry rather than a +control. + +The `unit` enum is the one worth naming. It is dropped rather than reshaped, and +that is safe for a stated reason: the field is decorative for enforcement โ€” +`_declared_quantities` reads `ClaimQuantity.unit` from the registry and never +from the response, precisely so a model cannot declare its own corroboration. + +## Structured output is server-side, not a request in prose + +`output_config.format` with a `json_schema` constrains the response at the API +rather than asking for JSON and hoping. That matters more than it looks: +`claims.parse_strict` does not repair, retry with a nudge, or partially accept โ€” +directive ยง4 says the calls must return structured JSON, so one that did not is +a failed call. Without server-side enforcement every malformed answer is a +capability that intermittently returns nothing, and the schema this service +already builds would be doing nothing but rejecting. + +The schema `claims.schema_for` emits is already closed โ€” `additionalProperties: +false`, explicit `required`, enums on every claim, evidence and limit string โ€” +and every one of those crosses intact. What is dropped is the numeric and length +*bounds* named in the section above, and `claims.validate` implements all of +them itself. + +## Why the images go in as base64 rather than by URL + +The capture lives in `animapmedia`, which the inference service reads through +its own managed identity (ADR 0012). A URL the vendor fetches would need that +blob to be publicly reachable, which is a photograph of somebody's animals on +the open internet in exchange for saving a base64 encode. + +## No retry, and no fallback model + +A refusal, a timeout or a malformed answer is reported rather than worked +around. The SDK's own transport-level retries stay on โ€” they cover connection +errors and 429s, which are not answers โ€” but nothing here re-asks a question +that was answered badly, and nothing silently substitutes a different model. +`InferenceResult` requires `model_version` and the release row records it, so a +result that cannot name the model that produced it is not evidence. +""" + +from __future__ import annotations + +import base64 +import io +import json +import logging +from typing import Any + +from PIL import Image + +from app.adapters.base import AdapterError + +logger = logging.getLogger(__name__) + +#: What a capability may spend on one answer. +#: +#: These are structured extractions โ€” a claim list, a handful of observations, +#: an evidence phrase โ€” and not prose. The ceiling is generous against that +#: shape rather than tuned, because the cost of a truncated answer is a whole +#: capture refused by `parse_strict` and re-queued. +MAX_TOKENS = 8_000 + +#: How long one call may take before it is a failure rather than a wait. +#: +#: Under `INFERENCE_TIMEOUT_SECONDS`' fifteen-second default on purpose: a run +#: that outlives the request path is kept queued and retried by +#: `drain_inference_queue`, and a transport that sat past that would turn a +#: recoverable wait into a request the API has already given up on. +TIMEOUT_SECONDS = 120.0 + +#: Sent as `image/jpeg` unless Pillow says otherwise. Captures are JPEG. +_MEDIA_TYPES = { + "JPEG": "image/jpeg", + "PNG": "image/png", + "GIF": "image/gif", + "WEBP": "image/webp", +} + + +def _encode(image: Image.Image) -> tuple[str, str]: + """One image as `(media_type, base64)`. + + Re-encoded to JPEG unless it is already one of the four types the API + accepts. A capture that reached this service as something else โ€” a frame + grabbed from a clip, a PNG from a screenshot harness โ€” would otherwise be + refused for its container rather than for anything about the animal. + + RGBA is flattened onto white before a JPEG encode, because JPEG has no alpha + and Pillow raises rather than guessing. White rather than black: a + transparent margin around a photograph is padding, and padding a farmer sees + is paper. + """ + fmt = (image.format or "").upper() + if fmt in _MEDIA_TYPES: + buffer = io.BytesIO() + image.save(buffer, format=fmt) + return _MEDIA_TYPES[fmt], base64.standard_b64encode(buffer.getvalue()).decode() + + prepared = image + if image.mode in ("RGBA", "LA", "P"): + prepared = Image.new("RGB", image.size, (255, 255, 255)) + converted = image.convert("RGBA") + prepared.paste(converted, mask=converted.split()[-1]) + elif image.mode != "RGB": + prepared = image.convert("RGB") + + buffer = io.BytesIO() + # Quality 90 rather than Pillow's default 75. The capabilities waiting on + # this read skin texture, tooth wear and footpad lesions, and a compression + # artefact at that scale is indistinguishable from the thing being looked + # for. The bytes are not the constraint here; the reading is. + prepared.save(buffer, format="JPEG", quality=90) + return "image/jpeg", base64.standard_b64encode(buffer.getvalue()).decode() + + +#: Keywords structured outputs refuses, by the type they sit on. +#: +#: Discovered by probing the live API one keyword at a time, not by reading a +#: list: `minItems`, `maxLength`, a bare union type and a plain `enum` on a +#: string are all accepted, so a broader strip would remove controls the API was +#: willing to enforce. +_UNSUPPORTED = { + "array": ("maxItems",), + "number": ("minimum", "maximum", "multipleOf"), + "integer": ("minimum", "maximum", "multipleOf"), +} + +#: `minItems` survives only as 0 or 1. +#: +#: *"'minItems' values other than 0 or 1 are not supported (got: [2, 5])"* โ€” the +#: `range` array is exactly two entries and this is what the API says about it. +#: A probe with `minItems: 1` passed, which is why the first pass missed it and +#: why this is a value rule rather than another entry above. +_MIN_ITEMS_CEILING = 1 + + +def _for_structured_output(node): + """`node` with the keywords structured outputs refuses taken out. + + Recursive and non-destructive โ€” the caller keeps the original, because the + original is what judges the answer. + """ + if isinstance(node, list): + return [_for_structured_output(item) for item in node] + if not isinstance(node, dict): + return node + + out = {key: _for_structured_output(value) for key, value in node.items()} + + # **Only when `type` is a type.** `claims.schema_for` has a property + # literally named `type` โ€” an observation's own kind โ€” so inside a + # `properties` node `out["type"]` is a schema rather than a declaration. + # Reading it as one raised `TypeError: unhashable type: 'dict'` the first + # time this ran against a real capability. + declared = out.get("type") + if isinstance(declared, str): + types = [declared] + elif isinstance(declared, list) and all(isinstance(t, str) for t in declared): + types = declared + else: + types = [] + + for kind in types: + for keyword in _UNSUPPORTED.get(kind, ()): + out.pop(keyword, None) + if kind == "array": + floor = out.get("minItems") + if isinstance(floor, int) and floor > _MIN_ITEMS_CEILING: + out.pop("minItems", None) + + # An enum whose values cannot all be the declared type. The API reads a + # union type plus an enum as a contradiction rather than as a widening, so + # the enum goes and the type โ€” which is what makes the field nullable โ€” + # stays. See the header for why this one costs nothing. + enum = out.get("enum") + if isinstance(enum, list) and len(types) > 1 and None in enum: + out.pop("enum", None) + + return out + + +def anthropic_transport( + *, + prompt: str, + images: list[Image.Image], + schema: dict[str, Any], + model: str, + api_key: str, +) -> str: + """One call to Claude, returning the response text verbatim. + + The signature is `HostedMultimodalAdapter`'s, keyword-only, and it returns + the raw string rather than a parsed object โ€” `ReasonerResponse` keeps `raw` + beside `parsed` because ยง33 requires what the model said to be preserved for + later training, and a reparsed reconstruction is not that. + + :param model: the exact hosted model id, from `ANIMAP_MULTIMODAL_MODEL`. + Never defaulted here. A result has to name the model that produced it, + and a transport quietly substituting one it preferred would make every + stored `model_version` a guess. + """ + # Imported inside the call, not at module scope. `transports/__init__` keeps + # its provider table as factories for the same reason: a deployment that has + # chosen no provider should not pay for a vendor SDK import, and the + # inference container's sub-second start is a property worth keeping. + import anthropic + + if not images: + raise ValueError("A visual reasoner needs at least one image.") + + client = anthropic.Anthropic( + api_key=api_key, + timeout=TIMEOUT_SECONDS, + # The SDK's default of 2. Connection errors and 429s are not answers, + # so retrying them is not re-asking a question that was answered badly โ€” + # which is the thing this transport does not do. + max_retries=2, + ) + + content: list[dict[str, Any]] = [ + { + "type": "image", + "source": {"type": "base64", "media_type": media_type, "data": data}, + } + for media_type, data in (_encode(image) for image in images) + ] + # The rubric last, after the images. It refers to them. + content.append({"type": "text", "text": prompt}) + + response = client.messages.create( + model=model, + max_tokens=MAX_TOKENS, + messages=[{"role": "user", "content": content}], + # **Server-side, from the registry's own schema**, minus the four + # keywords structured outputs refuses. `claims.enforce` still judges the + # answer against the original, and implements all four itself โ€” see the + # module header. + output_config={ + "format": { + "type": "json_schema", + "schema": _for_structured_output(schema), + } + }, + # Adaptive rather than a fixed budget: these are visual judgements whose + # difficulty varies by photograph โ€” a clear muzzle and an occluded hock + # are not the same question โ€” and a fixed budget is either wasted on the + # first or short on the second. + thinking={"type": "adaptive"}, + ) + + # **A refusal is reported, not retried and not swallowed.** It arrives as a + # 200 with `stop_reason: "refusal"` and no usable content, so reading + # `content` first would raise something that looks like a parse failure and + # hide what actually happened. + if response.stop_reason == "refusal": + details = getattr(response, "stop_details", None) + category = getattr(details, "category", None) + raise AdapterError( + f"The hosted model declined this request" + f"{f' ({category})' if category else ''}. Nothing is stored for it." + ) + + if response.stop_reason == "max_tokens": + # Truncated JSON is not partial evidence, it is malformed. Said here + # rather than left to `parse_strict`, because the fix is a ceiling and + # not a recapture, and a farm should not be asked to re-photograph an + # animal over a token limit. + raise AdapterError( + f"The hosted model's answer was cut off at {MAX_TOKENS} tokens, so " + f"it is not a complete JSON object. Nothing is stored for it." + ) + + text = next((b.text for b in response.content if b.type == "text"), None) + if text is None: + # Reachable when a response carries only thinking blocks. Structured + # output makes it very unlikely and never impossible, and an empty + # string here would reach `parse_strict` as a malformed answer with no + # explanation attached. + raise AdapterError( + "The hosted model returned no text block, so there is no JSON " + "object to read. Nothing is stored for it." + ) + + logger.info( + "Hosted reasoner %s answered in %s input and %s output tokens.", + model, response.usage.input_tokens, response.usage.output_tokens, + ) + return text + + +def _describe() -> str: + """What this transport is, for a listing. Imports nothing.""" + return json.dumps( + { + "provider": "anthropic", + "structured_output": "server-side json_schema", + "images": "base64, inline", + "max_tokens": MAX_TOKENS, + "timeout_seconds": TIMEOUT_SECONDS, + }, + sort_keys=True, + ) diff --git a/app/adapters/unavailable.py b/app/adapters/unavailable.py new file mode 100644 index 0000000000000000000000000000000000000000..44a21d423183d4935a4fd8303487ed97571f1f3d --- /dev/null +++ b/app/adapters/unavailable.py @@ -0,0 +1,298 @@ +"""Adapters for models that are not installed, and what each one is waiting on. + +**Every class here is a real adapter with a real reason, not a stub.** The +difference matters and it is the whole point of the file: a stub returns +something plausible, and these cannot return anything at all. `load()` raises, +there is no `detect` or `embed` or `count` to call, and the only thing they can +produce is an account of what is missing. + +Directive ยง36 says a capability may only be called unavailable after the +alternatives have been attempted and the attempt documented. This is where that +documentation lives in code rather than in a document nobody re-reads โ€” each +adapter carries the licence position, the measured or unmeasured cost, and the +specific thing that would make it runnable. + +Two kinds of absence are represented, and collapsing them would lose the +information a reader needs: + +- **Not installed.** It is usable and nobody has stood it up yet. SAM 3, + Grounding DINO, CountGD. +- **Not configured.** It needs a credential the deployment does not have. The + hosted reasoner. + +There used to be a third โ€” **refused**, for MegaDescriptor and MiewID โ€” and it +is gone because a refused licence is no longer a reason to leave a model +uninstalled. Both now run, both are still unservable, and both live in +`adapters/embedding.py`. `NotInstalled.availability()` still checks the licence +first, because the distinction it draws is the one that matters here: a model +Animap may not serve does not become available by installing it, and saying so +keeps somebody from spending a week finding out. +""" + +from __future__ import annotations + +from app.adapters.base import ( + Adapter, + AdapterSpec, + AdapterUnavailable, + Availability, + MeasuredCost, + Modality, + Placement, + Task, +) +from app.adapters.licences import LicenceRefused, gate + + +class NotInstalled(Adapter): + """An adapter whose weights are not on this machine. + + Holds a spec and a reason and nothing else. It exists so `/capabilities` + can say *which* model is missing and what it would take, rather than + returning the same "coming soon" for a model nobody has tried and a model + that is one download away. + """ + + def __init__(self, spec: AdapterSpec, *, reason: str, remedy: str) -> None: + self.spec = spec + self._reason = reason + self._remedy = remedy + + def availability(self) -> Availability: + # The licence is checked first and separately. A model Animap may not + # serve is not "not installed yet" โ€” installing it would not help, and + # saying so keeps somebody from spending a week on it. + try: + gate(self.spec.runtime) + except LicenceRefused as refusal: + return Availability( + False, str(refusal), + "This one does not become available by installing it.", + ) + return Availability(False, self._reason, self._remedy) + + def load(self) -> "NotInstalled": + raise AdapterUnavailable(self.availability()) + + +# --- ยง3 SAM 3.1 --------------------------------------------------------------- + +SAM3_SPEC = AdapterSpec( + adapter_id="sam3", + runtime="sam3", + tasks=(Task.SEGMENT, Task.DETECT, Task.TRACK), + modalities=(Modality.IMAGE, Modality.VIDEO), + directive_role=( + "ยง3 SAM 3.1 โ€” detection, segmentation, tracking, concept prompting, " + "visual exemplar prompting, auto-labelling, body-region extraction, " + "wound and lesion masks. The teacher model for most of ยง6โ€“ยง20." + ), + placement=Placement.GPU_SERVICE, + requires_gpu=True, + placement_reason=( + "848M parameters and a 3.44 GB fp32 checkpoint, and the official " + "install requirements name a CUDA 12.6 GPU. It will not sit beside the " + "API on the 2 vCPU / 4 GiB worker, and it does not need to โ€” this is " + "the leg to host on a GPU. A quantised CPU build is conceivable and " + "nobody has measured one." + ), + notes=( + "SAM 3.1 (27 March 2026) is a video-tracking speed-up over SAM 3, not a " + "new image model, and ships no transformers integration โ€” its own model " + "card says so. For still images, SAM 3 is the one to stand up; " + "transformers has supported it since v5.0.0. Both are gated with manual " + "approval, so a build needs an HF token and Meta can revoke access." + ), +) + +SAM2_SPEC = AdapterSpec( + adapter_id="sam2.1-hiera-tiny", + runtime="sam2-onnx", + tasks=(Task.SEGMENT,), + modalities=(Modality.IMAGE, Modality.VIDEO), + directive_role=( + "ยง3 SAM's segmentation role at a size the CPU worker can hold. 39.0M " + "parameters, Apache-2.0, ungated." + ), + placement=Placement.CPU_SERVICE, + notes=( + "**Not a substitute for SAM 3 and must not be recorded as one.** It " + "does promptable segmentation from a point or a box; it has no concept " + "prompting and no text, which is most of what ยง3 wants SAM 3 for. It is " + "here as the mask source for the pipelines that already know where to " + "look โ€” ยง14's flank region, ยง9's wound outline โ€” where a box from the " + "existing YOLOX detector is prompt enough." + ), +) + + +# --- ยง4 Grounding DINO -------------------------------------------------------- + +GROUNDING_DINO_SPEC = AdapterSpec( + adapter_id="grounding-dino-tiny", + runtime="grounding-dino-hf", + tasks=(Task.DETECT,), + modalities=(Modality.IMAGE,), + directive_role=( + "ยง4 Grounding DINO โ€” open-vocabulary detection, text-prompted " + "localisation, and the fallback when SAM's concept prompting is weak. " + "ยง25 also names it for tick detection over tiled close-ups." + ), + placement=Placement.GPU_SERVICE, + placement_reason=( + "It does run on CPU โ€” the HF implementation's deformable attention is " + "plain PyTorch `grid_sample` with no CUDA extension โ€” and the measured " + "cost below is why it should not run there anyway: 5.5 s a frame and a " + "2,121 MB peak, against a 4 GiB container that is also holding the " + "media. The second obstacle is torch, which ADR 0017 deliberately " + "removed; an ONNX export would fix that and nobody has attempted one." + ), + measured=MeasuredCost( + hardware="Apple M-series laptop (NOT the target container)", + threads=1, + sample=( + "3 Commons frames โ€” cattle_ng_kaduna_market_01, cattle_ng_red_bororo, " + "poultry_free_range_flock โ€” at 1600โ€“1920 px, prompts 'a cow.' and " + "'a chicken.', box and text thresholds 0.3" + ), + runs=3, + median_seconds=5.53, + peak_rss_mb=2121.0, + measured_on="2026-08-21", + ), + notes=( + "172,250,626 parameters, 657 MiB of safetensors, Apache-2.0 for code, " + "weights and the BERT text encoder alike โ€” the cleanest licence in the " + "whole ยง3/ยง4 stack. Session load was 16.3 s and 948 MB before a single " + "frame. " + "**The detection quality above is a spike and not a benchmark.** At " + "threshold 0.3 it returned 1 box on a Kaduna market frame, 2 on " + "cattle_ng_red_bororo โ€” which ADR 0018 records as holding 3 cattle, all " + "3 found by the shipped YOLOX โ€” and 5 on a free-range flock frame whose " + "human count is 12. One prompt, one threshold, no tuning, three frames. " + "It says the pipeline runs, and nothing about whether it is better than " + "what ships." + ), +) + + +# --- ยง4 CountGD --------------------------------------------------------------- + +COUNTGD_SPEC = AdapterSpec( + adapter_id="countgd", + runtime="countgd", + tasks=(Task.COUNT,), + modalities=(Modality.IMAGE,), + directive_role=( + "ยง4 and ยง6.3 CountGD โ€” zero-shot open-world counting, exemplar-guided, " + "for the poultry visible-count that a COCO detector cannot do. ยง40.3 " + "names the specific job: benchmark it against the 452 annotated " + "commercial broiler frames." + ), + placement=Placement.GPU_SERVICE, + requires_gpu=False, + placement_reason=( + "MIT, and its vendored GroundingDINO falls back to a pure-PyTorch path " + "when the CUDA extension is absent, so CPU execution is possible in " + "principle. 894 MiB of weights on top of torch, and no published CPU " + "throughput figure exists. GPU is the sane host; CPU is a measurement " + "nobody has taken." + ), + notes=( + "**This is the highest-value unbuilt thing in the stack.** ADR 0018 " + "measured the shipped detector finding nothing at all in 254 of 452 PIO " + "frames and 8.5% of the birds that were there โ€” a 92% undercount โ€” and " + "concluded a density method is needed. CountGD is that method, the " + "benchmark set is already on disk under evaluation/pio, and the licence " + "is MIT. Skip the optional --sam_tt_norm flag and the 2.4 GB SAM ViT-H " + "checkpoint is not needed." + ), +) + + +# --- ยง4 MegaDescriptor and MiewID have left this file ------------------------ +# +# They were here as placeholders while their licences were treated as a reason +# not to install them. The founder lifted that: a licence is no longer grounds +# for dropping a model, only for refusing to serve it. Both are now exported, +# benchmarked and registered as real adapters in `adapters/embedding.py`, with +# their cards under `models/alternates/megadescriptor/` and `.../miewid/`. +# +# Nothing was weakened by the move. `licences.gate` still refuses both under the +# default `enforce` policy, `registry.refused()` still names exactly these two, +# and `describe()` still reports `servable: False` whatever the policy โ€” the +# terms have not changed, and neither has what Animap may put in front of a +# farmer. What changed is that "refused" is now a fact with a measurement behind +# it rather than a reason the measurement never happened. + + +# --- ยง4 hosted multimodal ----------------------------------------------------- + +HOSTED_MULTIMODAL_SPEC = AdapterSpec( + adapter_id="hosted-multimodal", + runtime="hosted-multimodal", + tasks=(Task.REASON,), + modalities=(Modality.IMAGE, Modality.VIDEO, Modality.AUDIO), + directive_role=( + "ยง4 hosted multimodal โ€” BCS rubric scoring, dentition, wound " + "description, skin and hoof and footpad triage, breed suggestion, " + "litter condition, heat-stress signs, egg quality, structured evidence " + "extraction. ยง4 is explicit that it is 'an experimental visual " + "reasoner, not an authority'." + ), + requires_artefact=False, + placement=Placement.CPU_SERVICE, + placement_reason=( + "No weights run here, so it costs the container nothing but a socket. " + "It is the one leg that cannot ever move to the phone, which matters: " + "ADR 0002 makes Animap offline-first, and every capability built on " + "this one is a capability a farm cannot use in a shed with no signal." + ), + notes=( + "Unavailable because no API key is configured. See adapters/multimodal.py " + "for the env var and the structured-output contract." + ), +) + + +#: Every adapter the directive names that is not runnable here, with its reason. +#: Built as a function rather than a module-level dict so a caller gets fresh +#: objects and cannot mutate a shared registry. +def unavailable_adapters() -> list[Adapter]: + return [ + NotInstalled( + SAM3_SPEC, + reason="SAM 3 is not installed.", + remedy=( + "Request access on huggingface.co/facebook/sam3, set HF_TOKEN, " + "and host it on a GPU worker โ€” it will not fit the CPU one." + ), + ), + NotInstalled( + SAM2_SPEC, + reason="SAM 2.1 is not installed.", + remedy=( + "Export facebook/sam2.1-hiera-tiny to ONNX the way " + "scripts/export_embedding.py does, so the CPU worker keeps its " + "no-torch property." + ), + ), + NotInstalled( + GROUNDING_DINO_SPEC, + reason="Grounding DINO is not installed.", + remedy=( + "pip install transformers and fetch " + "IDEA-Research/grounding-dino-tiny, or export it to ONNX for the " + "CPU worker. Nothing is gated and nothing needs a token." + ), + ), + NotInstalled( + COUNTGD_SPEC, + reason="CountGD is not installed.", + remedy=( + "Clone niki-amini-naieni/CountGD, fetch the MIT weights from " + "nikigoli/CountGD, and benchmark against evaluation/pio โ€” the " + "452-frame set ADR 0018 says the shipped detector fails on." + ), + ), + ] diff --git a/app/capabilities.py b/app/capabilities.py new file mode 100644 index 0000000000000000000000000000000000000000..7f93d91139c7b1cfdbfa40a88340f0ee75dd0588 --- /dev/null +++ b/app/capabilities.py @@ -0,0 +1,3463 @@ +"""The capability registry, and the acquisition protocol each capability declares. + +ADR 0006 and ADR 0021. Every model-backed feature is data, not code โ€” which is +what collapses the prototype's 26 capability screens into two layouts. + +**This file was rewritten against the founder's zero-training directive** +(`Animap_AI_Inference_Zero_Training_Implementation_Directive.md`). The registry +used to answer *"can one model do this from one unconstrained RGB photograph?"* +and answered *no* nineteen times. The directive rejects that question. The +question it asks instead is *"what is the minimum practical capture protocol, +model combination, signal-processing method or retrieval method that makes this +observable enough to be useful?"* โ€” and a capability whose claim is corrected +survives instead of disappearing. + +Nothing measured was thrown away. Every figure the earlier pass recorded is +still in `app/dispositions.py`, verbatim, now attached to the classification it +argues about rather than to a verdict it was made to justify. + +## `state` is a claim ceiling, not an availability flag + +The most load-bearing distinction in this file. `state` says how strong a claim +this capability is *entitled* to make once its stack is wired. `is_runnable` +says whether an artefact and an adapter exist **today**. They are independent, +and a client must render both: 27 of the 28 capabilities are `experimental` and +26 of them are `runnable: false`, which reads as *"this is a real feature, it is +not switched on yet"* โ€” not as *"this works."* + +The twenty-eighth is `poultry_uniformity`, and it is `unsupported_claim` +(ADR 0023). It is the only capability whose claim was refused rather than +corrected, and the distinction above is what made the refusal legible: its +ceiling was never in doubt while the question was *when*, and collapsed the +moment somebody measured what its input would be worth. + +Two capabilities run: `cattle_detection` and `poultry_count`, on YOLOX-m under +Apache-2.0 (ADR 0017, ADR 0018). Both are `experimental` rather than +`production`, because their only validation is 31 frames labelled by one +non-expert and a `global-coco` provenance. + +## Why state plus flags, and not one enum or one set + +Directive ยง2 names seven states: `production`, `experimental`, +`human_confirmation`, `guided_capture`, `hardware_required`, `coming_soon`, +`unsupported_claim`. They do not form a ladder, and ยง18 proves it โ€” poultry +footpad is *"Experimental / Guided Capture / Human Confirmation"* at once. + +They answer two different questions: + +- **How far can this be trusted?** `production`, `experimental`, `coming_soon`, + `unsupported_claim`. Exactly one is true at a time. That is `CapabilityState`. +- **What has to happen for an output to exist at all?** Guided capture, a human + confirming, an accessory, a fixed installation. Any combination is possible, + including none. That is a set of `Requirement`. + +A single enum would need one member per combination. A single flat set would +let a capability be `production` and `coming_soon` together, which is +meaningless. State plus flags is the only one of the three that can express ยง18 +and cannot express nonsense. + +`Requirement` also carries `FIXED_INSTALLATION`, which ยง2 does not name but ยง1 +and ยง27 do โ€” *"continuous-monitoring version requires fixed camera/mic"*. That +is the eighth reframing, and it is a requirement rather than a state for the +same reason as the other three: ยง27 says the continuous form is a separate +capability, so the flag describes an acquisition mode, not a trust level. + +`PRODUCTION`, `COMING_SOON`, `HARDWARE_REQUIRED` and `FIXED_INSTALLATION` have +no member in the registry today โ€” `UNSUPPORTED_CLAIM` was on that list until +ADR 0023 and now holds `poultry_uniformity`. They stay because +`state` is a wire vocabulary that two other services and a phone read, and +adding a value later means a coordinated release across all of them. **Not +because anything renders them yet** โ€” the app's capability screens still carry +the old vocabulary in comments that this change makes stale, on 43 lines across +27 Kotlin files. `REJECTED_CLAIMS` below names which hardware and which fixed +installation would unlock which stronger claim. + +An earlier version of this paragraph added *"no Kotlin file reads `state` at +all"*, which has stopped being true: +`apps/android/โ€ฆ/capability/CapabilityEnums.kt` mirrors this vocabulary as an +executable Kotlin enum. It was untracked when this was written, so it is another +agent's in-flight work โ€” but a value added here now costs a release on the phone +in fact, not only in principle. + +`COMING_SOON` emptied when `poultry_uniformity` moved to `experimental`, and +that same capability then filled `UNSUPPORTED_CLAIM`. So `state` runs two live +values and two reserved ones. For a long time it ran *one* live value, which +looked like a vocabulary doing no work โ€” and was the honest reading of a +registry where every capability's *claim* was supportable and almost none of +them was *built*. The second fact is `is_runnable`'s to report, and it reports +it for 26 of the 28. + +**The vocabulary earned its keep the first time a measurement came back against +a capability rather than against a model.** An enum with one live value cannot +say *"this one is different"*, and by the time it needs to, adding the value is +a coordinated release across three codebases. + +## Where this file is stricter than the directive's own status lines + +Fifteen capabilities carry `GUIDED_CAPTURE` although the status line in their +directive section does not name it: `cattle_bcs` (ยง7), `cattle_gait` (ยง24), +`cattle_identity` (ยง6.4), `cattle_ticks` (ยง25), `cattle_weight` (ยง22), +`poultry_count` (ยง6.3), `poultry_eye_head`, `poultry_feather` and +`poultry_hock` (ยง19), `poultry_fecal` (ยง13), `poultry_heat_stress` (ยง17), +`poultry_inactive_birds` (ยง15), `poultry_respiratory` (ยง26), +`poultry_uniformity` and `poultry_weight` (ยง23). Only four sections name the +flag for a capability that holds it โ€” ยง8, ยง11, ยง14 and ยง18. + +For twelve of the fifteen the flag follows the *capture* the section specifies +rather than its status line, because ยง2 defines the flag by whether the user +must follow a protocol: a side and a rear-quarter view, a five-metre walk, four +to six samples, a held bird, a density-appropriate mode. Every one of those is +an instruction the farmer has to obey for the capture to carry the signal. + +**Three of the fifteen do not have that justification, and should not pretend +to.** ยง19 specifies no capture at all โ€” no views, no samples, no duration โ€” so +`poultry_hock`, `poultry_feather` and `poultry_eye_head` carry the flag because +*this file* invented their protocols (`held_bird_hock`, `whole_bird_dorsal`, +`held_bird_head`), not because the directive asked for them. Presenting a bird's +hock or its head to a camera really does need the bird caught and held, which is +why the flag is right; but the source is a judgement made here, and +`poultry_wound` shows how thin the line is โ€” it is a lesion close-up anywhere on +the body, exactly like `cattle_wound`, so it carries no flag. A reader who +thinks `poultry_feather` is no more "held" than `poultry_wound` has a fair +argument, and this note exists so they can find it. + +**An earlier draft of this paragraph said four and named four**, which was an +undercount taken from the wrong reading and not a narrower definition โ€” there is +no reading of the directive under which the number is four. The list above was +derived by matching every `GUIDED_CAPTURE` holder against the `## Status` block +of its own section. + +It runs one way only. ยง20's *second* status block โ€” *"Guided Capture / accessory +recommended"* โ€” belongs to the candling sub-capability for fine cracks, not to +`egg_quality`, whose own status line is plain `Experimental`. So `egg_quality` +carries no flag, and the candling route lives where the directive puts it: on +`hairline_crack_from_ambient_photo`'s `available_with`. + +## Compute is never a reason for a lower state + +Founder's instruction, and it constrains every classification in this file: a +capability is not downgraded because a model is expensive to run. SAM 3.1, +DINOv3, VGGT and a hosted multimodal reasoner do not fit on a phone and several +want a GPU. That is a **hosting** decision โ€” the inference legs can run +somewhere with GPUs โ€” and it produces a `Connectivity` tier, not a lower state +and not a `hardware_required` flag. + +So where the model runs is answered by two fields, and neither of them is +`state`: + +- **`connectivity`** โ€” what a farmer gets with no signal. After this rewrite, + 26 capabilities are `required`: the phone contributes nothing to the + answer, so the honest badge is *"Saves now, analysis runs when you + reconnect."* Two are `deferred`, because the Android app really does ship a + MediaPipe COCO detector over `efficientdet_lite0.tflite` that answers + *"cattle detected"* and counts visible birds offline + (`apps/android/.../capture/SubjectDetector.kt`). **That file's "about 30 ms" + is its own comment's claim with no benchmark behind it anywhere in this + repo** โ€” the detector is real, the latency is unverified. Per ADR 0002 every + tier still saves the capture; connectivity gates analysis, never capture. +- **`on_device_candidate`** โ€” whether this could plausibly move to the phone + later, which is the difference between a feature that works in a shed and one + that works in the yard. Set only where the directive's own method is + deterministic signal processing or an on-device detector that already runs, + not wherever it would be nice. + +## A licence is never a reason for a lower state either + +Second founder instruction, and it works the same way as the first: *"don't drop +anything because of licence requirement for now. just get them done."* + +Nothing here is `coming_soon` or `unsupported_claim` because the best model for +it is AGPL, non-commercial, research-only or gated. `cattle_identity` is the +case that proves it โ€” directive ยง4 names MegaDescriptor first, its weights are +CC-BY-NC-4.0, and the capability is `experimental` anyway, because what decides +that is whether a muzzle is photographable and whether the claim is confirmed, +not whose terms the embedding arrives under. + +So the licence is recorded as an **attribute of the models a capability leans +on**, in `model_stack`, and `licence_exposure()` answers *"which capabilities +currently depend on a copyleft or non-commercial model"* without anyone going +digging. The terms themselves live in `app/adapters/licences.py`, which is the +adapters agent's ledger, records a primary source and a verification date for +every runtime, and is the only thing here entitled to an opinion about terms. +This file names runtimes; it does not restate their licences. + +That ledger still refuses to *serve* a disallowed runtime, which is correct and +is a different decision from this one: refusing to run a model is a deployment +answer, and classifying a capability is a product answer. A capability whose +whole stack is unservable today is still classified on its signal, and +`licence_exposure()` is how that shows up before anything is sold. +""" + +from __future__ import annotations + +from dataclasses import dataclass, field, replace +from enum import Enum + + +class CapabilityState(str, Enum): + """How far a capability's output may be trusted. Exactly one is true. + + Ordering here is deliberate โ€” strongest first โ€” but it is not a ladder a + capability climbs one rung at a time. `coming_soon` is not weaker than + `experimental`; it is a different statement, about implementation rather + than about evidence. + """ + + #: Works reliably enough for normal use โ€” meaning an Animap benchmark + #: demonstrated it, not that a demo looked good (directive ยง37). Nothing + #: holds this today. + PRODUCTION = "production" + #: The capability works and the output carries broad uncertainty the user + #: can see. This is where the directive expects almost everything to sit + #: for v0. + EXPERIMENTAL = "experimental" + #: An engineering path exists and nobody has walked it yet. Distinct from + #: `experimental` with `runnable: false`: that one is wired-up work, this + #: one is not designed work. + COMING_SOON = "coming_soon" + #: The claim itself is not scientifically supportable in any capture + #: protocol this capability can offer. Directive ยง36 sets the bar: only after + #: the **observable signal** has been shown to be absent or impractical, and + #: a failed first model is not sufficient evidence. + #: + #: **One capability meets that bar: `poultry_uniformity`** (ADR 0023). It is + #: the first member this value has had, and what met the bar was not a failed + #: model โ€” there is no model in it at all. It was error propagation: a + #: coefficient of variation over measured weights carries the measurement + #: error as well as the flock's spread, the inflation does not shrink with + #: sample size, and the best published per-bird error for the method is a + #: floor rather than an estimate. That is the observable signal shown + #: impractical, which is the only thing ยง36 accepts. + #: + #: Individual claims meet it too, and more often โ€” see `REJECTED_CLAIMS`. + UNSUPPORTED_CLAIM = "unsupported_claim" + + +class Requirement(str, Enum): + """What has to happen before an output exists. Any combination, including none.""" + + #: Animap proposes and the user confirms or corrects. Directive ยง32: this is + #: a feature, not a failure, and every correction is stored as ground truth. + HUMAN_CONFIRMATION = "human_confirmation" + #: Available only when the user follows the declared capture protocol. + GUIDED_CAPTURE = "guided_capture" + #: Needs an optional sensor or accessory **in the farmer's hand** โ€” a + #: moisture probe, a thermal camera, a candling backlight (directive ยง29). + #: + #: **This never means server-side compute.** A capability that wants a GPU + #: for its inference leg is not `hardware_required`; that is a hosting + #: decision, and the inference legs can run wherever GPUs are. Where the + #: model runs is `Connectivity`'s question, not this flag's. + HARDWARE_REQUIRED = "hardware_required" + #: Needs a permanently placed camera or microphone (directive ยง27, ยง28). + FIXED_INSTALLATION = "fixed_installation" + + +class Modality(str, Enum): + """What the capture consists of. Named as directive ยง34 names them.""" + + RGB_PHOTO = "rgb_photo" + RGB_VIDEO = "rgb_video" + AUDIO = "audio" + CAMERA_POSE = "camera_pose" + DEPTH_OPTIONAL = "depth_optional" + + +class Connectivity(str, Enum): + """Maps 1:1 to the badge and footnote the app shows. + + **Every tier still saves the capture.** Even `required` reads *"Saves now, + analysis runs when you reconnect"* โ€” connectivity defers analysis, it never + blocks capture (ADR 0002). + + The tier is set by what the phone contributes to the **answer**, not by + whether it can help frame the shot. Capture guidance is present everywhere + and does not earn `deferred`; a partial result does. + """ + + #: The full result with no connection. Nothing holds this. + ON_DEVICE = "on_device" + #: *"Captures now, full assessment after sync."* Part of the answer is + #: computed on the phone. + DEFERRED = "deferred" + #: *"Saves now, analysis runs when you reconnect."* The phone computes none + #: of the answer. This is the honest tier for anything whose first stage is + #: a hosted model, and after the zero-training rewrite that is most of them. + REQUIRED = "required" + + +class Basis(str, Enum): + """Where a bound came from. The three are not equally strong. + + Recorded per bound rather than described once, because the weakest kind is + the one a reader has to be able to find. A guardrail is an engineering + judgement with no measurement behind it, and saying so is the difference + between a number somebody can challenge and a number that looks settled. + """ + + #: The unit defines it. A BCS scale runs 1 to 5; a footpad grade runs 0 to 4; + #: a share of a scanned region runs 0 to 100. Not a judgement. + SCALE = "scale" + #: The directive states or exemplifies it. + DIRECTIVE = "directive" + #: An engineering judgement made in this file, **with no measurement behind + #: it**, chosen wide enough that a true reading cannot hit it. These are the + #: ones to challenge. + GUARDRAIL = "guardrail" + + +@dataclass(frozen=True) +class EvidenceItem: + """One thing a capability may say in words, and where the wording came from. + + ## Why the words are a vocabulary and not a free-text field + + `evidence` and `limits` were declared as arrays of free strings, and that + was the one field in the contract a model could write a sentence into. Three + agents hardened the prose scan over it; each closed real phrasings and each + was then beaten by a rephrasing, and the watchdogs' verdict converged on + *"the identifier control is real, the publication control is theatre."* The + last attempt closed twelve of fifteen numeric carriers and its own watchdog + still published a forbidden uniformity figure from 28 of 28 capabilities, + and a digit-free assertion โ€” *"This flock is not uniform enough to + harvest"* โ€” had no defence at all and was never going to get one. + + **Anything that scans prose loses to an author who rephrases.** This + repository learned that once already, in Android's `NoInventedDataTest`, and + wrote it down there. So the field stops being prose. + + ADR 0024 carries the decision. The short form: the directive's own worked + output is a rubric's observable features, not sentences โ€” + + "evidence": ["prominent hooks", "visible pins", + "limited fat around tail head"] + + โ€” and ยง10's *"Multiple raised nodular lesions visible"*, ยง19's *"Reduced + feather coverage on back and tail"*, ยง11's six hoof findings and ยง12's six + faecal appearances are all items a capability can enumerate. ยง4's *"All + calls must return structured JSON"* is read as meaning this, rather than as + *"parses, and has these top-level keys"*. + + ## The phrase is the reader-facing string, not an identifier + + `claims` are snake_case identifiers a client renders. These are not: the + phrase is what a farmer reads, verbatim, so the review that matters happens + in this file's diff rather than in a client's string table. It also keeps + the directive's own wording intact where the directive prints a line โ€” ยง7's + three features are here character for character. + + ## A phrase never carries a per-capture number + + ยง25 prints *"17 probable ticks visible across sampled regions"*, and 17 is + not a property of the vocabulary. The invariant words are the phrase and the + figure stays in `observations[].value`, where `OutputSpec` already bounds it + on the unit the registry declared. **That is the whole of the old quantity + problem, gone rather than mitigated**: a model cannot write a digit into + words a farmer reads, because it cannot write the words. + + A phrase may carry a number that is a *constant* โ€” a rubric's endpoints in + *"Approximate grade on the 0-4 rubric"*, the seconds in a capture limit, + ยง6.3's own uncertainty footnote. Those are reviewed here and asserted as an + exact set by `tests/test_directive_payloads.py`, so adding one takes a diff. + """ + + #: What a farmer reads. Reviewed here and nowhere else. + phrase: str + #: Where the wording comes from: a directive section, or `product` for a + #: phrase this file wrote. **`product` is the one to challenge** โ€” it means + #: nobody outside this repository chose those words. + source: str + #: Why this phrase exists where the source does not make it obvious. Read by + #: a person, never by code. + note: str = "" + + +#: A capability may say nothing about the capture that every other capability +#: cannot also say, so the shared half of `limits` is declared once. +#: +#: **`limits` is enumerated for the same reason `evidence` is.** Leaving it free +#: would have moved the problem rather than solved it: it is an array of strings +#: on the same object, rendered under the same result, read by the same farmer. +#: A control that closes one narrative field and leaves the other beside it is +#: not a control. +_SHARED_LIMITS: tuple[EvidenceItem, ...] = ( + EvidenceItem("One view only", "ยง37", + note="ยง37 asks for product uncertainty to be shown rather " + "than a fabricated confidence."), + EvidenceItem("Single view; veterinary review recommended", "ยง10"), + EvidenceItem("Experimental result โ€” a person confirms it", "ยง32", + note="ยง32: human confirmation is a feature, not a failure."), + EvidenceItem("Capture quality reduces confidence in this reading", "ยง37"), + EvidenceItem("Assessed from what is visible in this capture only", "ยง30"), + # **The escape hatch, and it is deliberately contentless.** An enumerated + # vocabulary cannot describe the unanticipated, which is the real cost of + # this change (ADR 0024). What it can do is say that something unanticipated + # is there and route it to a person โ€” which is ยง32's own answer โ€” without + # letting the model say what it thinks it is. + EvidenceItem("Something outside this capability's vocabulary is visible; " + "have a person look at the capture", "product", + note="The only thing a model may say about an observation this " + "vocabulary has no phrase for. It carries no content on " + "purpose: a free-text escape hatch would be the old field " + "back under a new name."), +) + + +@dataclass(frozen=True) +class ClaimQuantity: + """What number, if any, one claim's `observations[].value` may carry. + + ## Why this exists at all + + `OutputSpec` bounded one quantity per capability and named the claims that + carry it. Every other claim's value was unbounded โ€” 73 of the registry's 93, + across 12 capabilities that named none โ€” so a reasoner could launder any + figure through an observation nobody had bounded, and the prose gate in + `app/adapters/claims.py` would then treat that figure as corroborated. The + named case was `cattle_gait`: ยง24 forbids *"Lameness score 3"*, and + `{"type": "gait_asymmetry", "value": 3}` published it. + + The fix cannot be a blanket bound over a capability's vocabulary, because a + vocabulary is not one unit โ€” ยง18 pairs a 0-to-4 footpad grade with a + prevalence in percent. So the unit is declared per claim, which is what this + is. + + ## `carries_number = False` is a claim about the claim, not a missing entry + + A body condition score, a count of lesions and a direction of travel are + three different kinds of thing, and only two of them are quantities. + *"Possible gait asymmetry"* has no magnitude; a number standing beside it is + not a more precise reading, it is a score the directive refuses. Declaring + that is the point of the field. + + ## How "no number" is enforced through a contract that cannot say it + + `bounds_for` is the hook `check_numeric_bounds` calls, and its answer is + three slots โ€” minimum, maximum, step. None of them says *"no number at + all"*, and returning nothing in all three means *permitted*, which is the + state this field exists to end. + + So a claim carrying no quantity declares an **empty interval**: a minimum + above its maximum, which no number satisfies. It is a deliberate encoding + and not a typo, and it is the only one those three slots have. + + **The first version of this returned 0 to 1** โ€” absent or present, on the + reasoning that a finding rides in a numeric field as a flag when it rides in + one at all. A watchdog defeated it immediately, on the exact capability the + field was built for: `{"type": "gait_asymmetry", "value": 1}` beside + *"Lameness score 1."* published, and 91 of the 93 pairs took the same shape. + A score of 1 is a score. The two digits that looked like they asserted + nothing asserted the thing ยง24 forbids. + + What a categorical claim keeps is the string. `Observation.value` is + `float | str | None`, `check_numeric_bounds` looks only at numbers, and + *"moderate"*, *"left flank"* and *"drinker line 3"* all publish โ€” so + refusing every number costs a claim nothing it should have been saying. + """ + + claim: str + #: `None` when this claim carries no quantity โ€” see the class docstring. + unit: str | None = None + minimum: float | None = None + maximum: float | None = None + #: The granularity the rubric supports, as JSON Schema's `multipleOf`. + step: float | None = None + basis: Basis | None = None + #: Why these numbers and not others. Read by a person, never by code. + why: str = "" + #: The words this claim's `observations[].value` may take, if any. + #: + #: **`Observation.value` is `float | str | None`, and the string half was + #: the last free-text channel a farm reads.** ADR 0022 recorded it as a + #: residual and could not close it: `check_numeric_bounds` looks only at + #: numbers, so `{"type": "gait_asymmetry", "value": "lameness score three"}` + #: published. Enumerating `evidence` without enumerating this would have + #: moved the problem one field sideways rather than solving it, which is the + #: failure ADR 0024 exists to avoid. + #: + #: **Empty means no word at all**, which is the state most claims are in and + #: is not a gap. ยง12's six faecal appearances are six *claims*; a `value` + #: beside `visible_mucus` adds nothing a farmer reads and is a place to + #: write a sentence. The claims that keep a word are the ones whose finding + #: really is a word โ€” a band, a severity, a site, a sex โ€” and the words are + #: listed here, in snake_case, because a client renders them. + values: tuple[str, ...] = () + #: Why this claim's value is free text, for the two that are. + #: + #: **Three of the registry's 94 claims carry a string nobody here can + #: enumerate**, and each names something from the farm's own world rather + #: than from a rubric: the animal's name (ยง6.4's *"This looks like Kofi"*) + #: and two breeds (ยง6.5's *"Likely White Fulani"* and its + #: *"White-Fulani-like phenotype"*). An enum of Nigerian cattle breeds would + #: refuse the crossbred animal ยง6.5 exists to protect, and an enum of animal + #: names is a farm's register, not a registry's vocabulary. + #: + #: **An earlier version of this comment said two, and named two.** The third + #: โ€” `breed_like_phenotype` โ€” was the one a watchdog published through, and + #: an undercount of an open channel is the worst kind: it names the holes and + #: leaves one out. + #: + #: All three sit under `HUMAN_CONFIRMATION`, so a farmer sees the word and is + #: asked; all three are scanned by every prose rule in + #: `app/adapters/claims.py`; and a free value has to be shaped like a name + #: rather than like a sentence, which is `claims._NAME_SHAPE` and is what + #: makes this a residual rather than a reopening of the field ADR 0024 + #: closed. + free_value_reason: str | None = None + + @property + def carries_number(self) -> bool: + return self.unit is not None + + @property + def carries_word(self) -> bool: + """Whether a string is a legal value for this claim at all.""" + return bool(self.values) or self.free_value_reason is not None + + @property + def enforced_bounds(self) -> tuple[float | None, float | None, float | None]: + """`(minimum, maximum, step)` as `check_numeric_bounds` will read them. + + For a claim that carries no quantity this is an interval containing no + number, because the caller's three slots have no way to say *"none"* and + returning nothing in all three means *permitted*. See the class + docstring. + + The refusal a reader sees says the value is below a minimum of 1 and + above a maximum of 0, which is confusing prose for a correct answer. It + is the price of expressing a fourth state through three slots, and the + wording is in `app/adapters/claims.py`, not here. + """ + if not self.carries_number: + return (_NO_NUMBER_FLOOR, _NO_NUMBER_CEILING, None) + return (self.minimum, self.maximum, self.step) + + +#: The width below which a `range` is not a range. Exact, because a range whose +#: ends are equal publishes a single figure however wide the units are. +_ZERO_WIDTH = 0.0 + +#: An interval no number falls inside, for a claim that carries no quantity. +#: Inverted on purpose โ€” see `ClaimQuantity.enforced_bounds`. +_NO_NUMBER_FLOOR = 1 +_NO_NUMBER_CEILING = 0 + + +@dataclass(frozen=True) +class OutputSpec: + """What the capability returns, and whether it is allowed to look precise. + + `show_range` is the flag that stops *"BCS 2.63"*. Directive ยง7 is blunt + about it: false precision is worse than a broad useful estimate. + + ## The bounds are refusals, not accuracy claims + + A watchdog validated `respiratory_rate_range` at **0 to 100,000 breaths a + minute** against the generated schema, because only the hand-written + `BCS_SCHEMA` carried `minimum` and `maximum`. Every other numeric output was + unbounded. So the bounds live here, next to the vocabulary, for the same + reason `forbidden_claims` does: the constraint belongs where the capability + is declared, not in whichever adapter happens to run it. + + **`plausible_min` and `plausible_max` mark the impossible, not the + unusual.** Outside them a value is not a poor estimate, it is a broken one โ€” + a cow that weighs four grams, a bird counted 1.4 times. They are deliberately + wide, because a bound that refuses a true outlier is worse than no bound: it + silently deletes the one capture that mattered. Nothing here should be read + as a statement about where a real answer usually falls. That is + `stated_uncertainty`'s job, in `app/dispositions.py`, where the measured + figures are. + + Each bound is marked with a `Basis` โ€” scale, directive or guardrail โ€” and + they are not equally strong. A guardrail is an engineering judgement with no + measurement behind it, and it says so rather than being left to look like a + finding. + + ## The headline bounds cover two fields, and `quantities` covers the rest + + `plausible_min`, `plausible_max` and `step` bound the capability's own + quantity, which is what `range` and `best_estimate` report and nothing else. + They were the whole of the numeric contract for one commit, and that left + every observation on a claim outside `measured_claims` unbounded โ€” 73 of the + registry's 93 allowed claims, in 12 capabilities that declared none at all. + A figure laundered through one of those was then treated as corroborated by + the prose gate, so ยง24's forbidden *"Lameness score 3"* published as + `{"type": "gait_asymmetry", "value": 3}`. + + `quantities` closes that by declaring, per claim, what number the claim may + carry โ€” or that it carries none. See `ClaimQuantity`, which also records + where its enforcement is weaker than its declaration. + + `step` is the same idea applied to granularity, and it is what catches + ยง7's own forbidden example. `BCS_SCHEMA` had to be repaired once because + `best_estimate: 2.6347` validated against a bare `{"type": "number"}`; a + half-point rubric cannot make a two-decimal claim, so `step` says 0.5 and + the schema refuses the rest. On a count it says 1, because 327.4 birds is + not a more careful answer than 327. + + ## These were declared here and read by nothing, for one commit + + Worth writing down, because the failure is easy to repeat and it was not + visible from this file. Every field above was correct, published by + `GET /capabilities`, and **wired to no check at all**: + `app/adapters/claims.py::schema_for` built `range`, `best_estimate` and + `observations[].value` as bare unbounded numbers, so a watchdog published + 100,000 breaths a minute, a cow weighing 0.004 kg and โ€” ยง7's own named + forbidden example โ€” `best_estimate: 2.63` on `cattle_bcs`. Impossible-high + in 16 of 16, negative in 16 of 16, off-step in 10 of 10. + + The tests could not see it, and said so in their own docstring: they + asserted that `step` was `0.5` and that `2.63 % 0.5 != 0`, which is a fact + about Python arithmetic rather than about this system. **A declaration + nothing reads is not a control**, and a test that reads the declaration + cannot tell the difference. `tests/test_dispositions.py` now publishes + payloads and expects refusals instead. + """ + + type: str + unit: str | None = None + show_range: bool = False + #: Inclusive. `None` for a categorical output, which has no numeric value to + #: bound โ€” twelve of the 28 capabilities return no number at all. + plausible_min: float | None = None + plausible_max: float | None = None + #: The granularity the rubric can actually support, as JSON Schema's + #: `multipleOf`. `None` means continuous. + step: float | None = None + #: Which of `allowed_claims` carry **this** quantity, so an observation's + #: `value` can be bounded by the right numbers. + #: + #: **A capability's vocabulary is not all one unit, and that is why the + #: bound cannot simply be applied to every observation.** ยง18's footpad + #: capability allows `approximate_grade` on a 0-4 rubric *and* + #: `sampled_prevalence`, whose own worked example is "20 birds sampled, 4 + #: moderate or worse, **20%**". A blanket `maximum: 4` over that vocabulary + #: would refuse the directive's own example. ยง14 pairs a respiratory rate of + #: 5-200 with `capture_quality`; both wound capabilities pair an area in cmยฒ + #: with `change_since_last_scan`, which is legitimately negative when a + #: wound is healing. + #: + #: So the registry names the claims the unit belongs to, and every other + #: claim's value stays unbounded โ€” which is the honest state, because + #: nothing here knows what unit those carry. Refusing a real reading is the + #: worse failure of the two, and this field is what stops the fix causing it. + #: + #: **It is no longer the whole answer.** Naming the claims that carry *this* + #: unit left every other claim unbounded, which is the hole `quantities` + #: closes. This field survives because it is what a client reads to know + #: which observations the headline bounds apply to, and because `range` and + #: `best_estimate` still carry the headline quantity and nothing else. + measured_claims: tuple[str, ...] = () + #: Every claim in `allowed_claims`, and what number each may carry. Spliced + #: in from `_QUANTITIES` by `_cap`, so the table can be read against the + #: vocabulary in one pass and a claim added without one is caught by a test + #: rather than by a watchdog. + quantities: tuple[ClaimQuantity, ...] = () + + @property + def is_numeric(self) -> bool: + """Whether this output carries a number a schema should bound. + + Keyed to `unit` rather than to `plausible_min`, so that an output which + gains a unit and no bounds fails a test rather than passing silently. + """ + return self.unit is not None + + def quantity_for(self, claim: str) -> ClaimQuantity | None: + """What this capability says `claim` may carry, or `None` if it is silent.""" + for quantity in self.quantities: + if quantity.claim == claim: + return quantity + return None + + def bounds_for(self, claim: str) -> tuple[float | None, float | None, float | None]: + """`(minimum, maximum, step)` for one claim's value, any of them `None`. + + The hook `app/adapters/claims.py::check_numeric_bounds` calls for every + observation, so what this returns is what the product will publish. + + `quantities` answers first and answers for every claim in the + vocabulary. The fall-back below is what the registry used to do and what + left 73 claims open: it applies the headline bounds to the claims that + carry the headline unit and returns nothing for the rest. It stays only + for a claim `_QUANTITIES` has not been told about, and + `test_dispositions.py` refuses that state โ€” so reaching it means a test + is missing, not that a claim is unbounded on purpose. + """ + quantity = self.quantity_for(claim) + if quantity is not None: + return quantity.enforced_bounds + if not self.is_numeric or claim not in self.measured_claims: + return (None, None, None) + return (self.plausible_min, self.plausible_max, self.step) + + def words_for(self, claim: str) -> tuple[tuple[str, ...], str | None]: + """What word this claim's `value` may be, as `(permitted, free reason)`. + + `([], None)` โ€” the default and the answer for most of the vocabulary โ€” + means *no string at all*. See `ClaimQuantity.values` for why that is a + decision rather than a gap. + + A claim `quantities` has not been told about also gets `([], None)`, + which is the strict answer. `test_dispositions.py` refuses that state + for any claim in the registry, so reaching it means a table entry is + missing rather than that a claim is open. + """ + quantity = self.quantity_for(claim) + if quantity is None: + return ((), None) + return (quantity.values, quantity.free_value_reason) + + def range_violations(self, values: object) -> list[str]: + """Why this `range` may not be published, or an empty list. + + **A zero-width range publishes an exact figure.** `[412, 412]` satisfies + every check a range has ever had here โ€” two items, ascending, both ends + inside the declared bounds โ€” while saying 412 kg exactly, which is the + claim ยง30 rejects and `show_range` exists to prevent. ยง22 prints + *"Experimental weight estimate, 350-430 kg"* and calls the initial range + wide; ยง37 contrasts that band with *"327 birds exactly"*; ยง38 asks for + broad outputs before precise ones. A capability that declares + `show_range` has said it cannot support a point value, so it may not + emit one through the field that exists to stop it. + + **This is not on the publication path yet**, and pretending otherwise + would repeat the failure this registry has already had once: the bounds + were declared, published and read by nothing for a commit. The only + caller that sees a `range` is `claims.enforce`, in `app/adapters/`. Until + it calls this, the rule is enforced by `tests/test_dispositions.py` and + by nothing else. ADR 0022 carries the one-call change and why it was not + made here. + """ + if not self.show_range or not isinstance(values, list) or len(values) != 2: + return [] + low, high = values + if not all( + isinstance(v, (int, float)) and not isinstance(v, bool) for v in (low, high) + ): + return [] + if high - low > _ZERO_WIDTH: + return [] + return [ + f"$.range is [{low}, {high}], which states an exact " + f"{self.unit or 'value'} rather than a band. This capability " + f"declares show_range because it cannot support a point value, and " + f"ยง37 rejects an exact figure the signal does not carry" + ] + + +@dataclass(frozen=True) +class AcquisitionProtocol: + """Directive ยง34: each capability declares how its signal is acquired. + + This is the contract the Android capture flow reads and the inference + provider validates against. The two claim lists at the bottom are the + mechanism that stops the product ever saying *"Lumpy skin disease + confirmed"* โ€” a claim not in `allowed_claims` has nowhere to be emitted + from, and one in `forbidden_claims` is asserted against. + """ + + #: A short name for the capture flow, shared with the Android app. + protocol: str + modality: tuple[Modality, ...] + output: OutputSpec + minimum_capture_seconds: int | None = None + preferred_capture_seconds: int | None = None + #: Directive ยง34's `cattle_gait` example. Walking distance, not camera range. + minimum_distance_m: float | None = None + #: Samples, for the capabilities whose unit of work is a bird or a dropping + #: rather than a second of video (ยง13, ยง18, ยง23). + minimum_samples: int | None = None + preferred_samples: int | None = None + required_views: tuple[str, ...] = () + optional_inputs: tuple[str, ...] = () + #: The buttons the user is offered. Empty unless `HUMAN_CONFIRMATION` is set. + confirmation_options: tuple[str, ...] = () + #: Where a result goes when it is not a farmer's decision to make (ยง10). + escalation: str | None = None + #: Workflows this capability must never hold up, whatever it returns or + #: fails to return. ยง6.6 is explicit that a sex suggestion must not block + #: registration, and that is a property of the capability rather than of a + #: screen โ€” so it is declared here, where a client reads it, instead of + #: being remembered by whoever builds the form. + never_blocks: tuple[str, ...] = () + #: Conditions under which no result may be produced. `resolution`, + #: `illumination` and `motion_blur` are the checks `app/quality.py` actually + #: implements; the rest are named by the directive and not yet implemented. + reject_if: tuple[str, ...] = () + #: What a **model** may emit for this capability. This is the enum that + #: `app/adapters/claims.py::schema_for` hands a reasoner, so anything listed + #: here is a field a model can fill in. A number the app works out from + #: records the model cannot see does not belong here โ€” see `derived_claims`. + allowed_claims: tuple[str, ...] = () + forbidden_claims: tuple[str, ...] = () + #: What a model may say **in words a farmer reads**, drawn from this + #: capability's own rubric. Spliced in from `_EVIDENCE` by `_cap`. + #: + #: **A skin finding is not a footpad finding**, so this is per capability + #: exactly as `allowed_claims` is. Sharing happens only where the directive + #: prints the same line for two captures โ€” ยง9 and ยง19 give `cattle_wound` + #: and `poultry_wound` the same wording, and they get the same phrase + #: because the directive gave it to them, not because the table was + #: convenient to write. + #: + #: Empty for `poultry_uniformity`, whose `allowed_claims` is also empty: + #: a capability that may claim nothing may say nothing (ADR 0023, ADR 0024). + allowed_evidence: tuple[EvidenceItem, ...] = () + #: What a model may say about the **capture** rather than about the animal. + #: `_SHARED_LIMITS` plus one phrase per condition in `reject_if`, spliced by + #: `_cap` so a rejection condition and the sentence a farmer reads for it + #: cannot drift apart. + allowed_limits: tuple[EvidenceItem, ...] = () + + @property + def evidence_phrases(self) -> tuple[str, ...]: + """The `evidence` enum, in declaration order.""" + return tuple(item.phrase for item in self.allowed_evidence) + + @property + def limit_phrases(self) -> tuple[str, ...]: + """The `limits` enum, in declaration order.""" + return tuple(item.phrase for item in self.allowed_limits) + #: Quantities this capability's result screen may show that **no model + #: produces**. The app computes them, from a capability output plus what the + #: farm already knows. + #: + #: This exists because ยง6.3 requires three poultry quantities to stay + #: distinct and only two of them are observations. Putting the third in + #: `allowed_claims` handed a hosted model a field called + #: `reconciled_flock_population` and let it write a whole-house number into + #: it โ€” which a watchdog duly did, with `value: 3200` from a partial pan. + #: That is ยง30's rejected claim arriving through the vocabulary that was + #: supposed to prevent it. + #: + #: **Deliberately not `forbidden_claims`.** A forbidden claim joins the + #: registry-wide `FORBIDDEN_CLAIMS`, which `app/counting.py` refuses + #: outright โ€” and a reconciled population is a legitimate number for the + #: product to show. It is illegitimate only as a *model's* claim. This field + #: keeps the concept visible, as ยง6.3 demands, without giving a reasoner + #: somewhere to put it. + derived_claims: tuple[str, ...] = () + + +@dataclass(frozen=True) +class Capability: + key: str + species: str + state: CapabilityState + requirements: frozenset[Requirement] + connectivity: Connectivity + save_target: str + acquisition: AcquisitionProtocol + #: What the model reports. A measurement carries no interpretation and needs + #: no confidence tile; a judgement carries both. Five of the prototype's + #: thirteen result screens omit confidence for exactly this reason. + output_kind: str = "judgement" + #: Capabilities that must ship first. + #: + #: **No capability declares one today, and the empty state is a result + #: rather than an oversight.** The only member was `poultry_uniformity`, + #: which waited on `poultry_weight` because ยง23 derives a flock spread from + #: per-bird weights. ADR 0023 removed it: the dependency was not merely + #: unmet, it was *unsuitable*, and a pointer that resolves into a refusal + #: tells a reader the blocker is sequencing when the blocker is suitability. + #: + #: The field stays because the relation is real and will recur โ€” a + #: capability computed from another capability's output is the shape ยง23 + #: describes and ยง6.3 nearly needs. What the one member proved is that + #: `depends_on` says *"this needs that input"* and never *"that input will + #: be good enough"*, and only the second question decides whether a claim + #: survives. + depends_on: tuple[str, ...] = () + #: Whether this could plausibly run on the phone later โ€” the difference + #: between a feature that works in a shed with no signal and one that works + #: in the yard. **This is an engineering property, never a reason for a + #: lower state**: a capability that needs a GPU is hosted where GPUs are. + #: + #: Set only on the strict test: the directive's own method is deterministic + #: signal processing (ยง4 โ€” optical flow, FFT, geometry, arithmetic), or an + #: on-device model that already ships. Everything gated on a SAM mask, a + #: DINOv3 embedding or a hosted multimodal call is false until the adapters + #: agent reports which legs are small enough, which is their measurement to + #: make and not this file's to guess. + on_device_candidate: bool = False + #: The runtimes this capability's declared stack leans on, named as + #: `app/adapters/licences.py` names them. **An attribute, never a gate**: a + #: non-commercial or gated runtime here does not lower `state`, it shows up + #: in `licence_exposure()`. A name absent from that ledger means nobody has + #: checked its terms yet, which is itself the useful answer. + model_stack: tuple[str, ...] = () + model_provider: str | None = None + model_version: str | None = None + geography_validated: tuple[str, ...] = field(default_factory=tuple) + notes: str = "" + + @property + def modality(self) -> str: + """The primary modality, in the single-word spelling the API's + `capabilities` table stores. + + Kept so that widening `modality` to a list does not force a column + change at the same time as a vocabulary change. + + **The first entry in `acquisition.modality` wins, not the richest + one.** Preferring video would report `poultry_count` as a video + capability because a controlled pan is one valid capture, when a single + photograph is the other and is what the shipped adapter reads. The + declared order is the capability's own answer to "what is this mostly". + + Four values change anyway, because the capture changed: `cattle_weight` + photo โ†’ video and `cattle_respiratory` audio โ†’ video (ยง14 is flank + video, not cough audio); `poultry_weight` and `poultry_uniformity` + video โ†’ photo (ยง23 is a held bird, not a flock scan). The API's + transcription still holds the old values (ADR 0021). + """ + first = self.acquisition.modality[0] + if first == Modality.RGB_VIDEO: + return "video" + if first == Modality.AUDIO: + return "audio" + return "photo" + + @property + def duration_seconds(self) -> int | None: + return self.acquisition.preferred_capture_seconds + + @property + def frames_required(self) -> int: + return max(1, len(self.acquisition.required_views)) + + @property + def is_runnable(self) -> bool: + """Whether this capability can produce a result **today**. + + Unchanged in substance from the four-state registry, and deliberately + so: it is what `app/providers.py` calls, and the property that stops a + status edit from unlocking a capability. A state alone still unlocks + nothing โ€” a checksummed artefact and a version have to be there too. + """ + return ( + self.state in (CapabilityState.PRODUCTION, CapabilityState.EXPERIMENTAL) + and self.model_provider is not None + and self.model_version is not None + ) + + @property + def may_be_promoted_to_production(self) -> bool: + """Whether this capability could legitimately reach `production`. + + Was `may_be_enabled` before ADR 0021; `enabled` is no longer a state. + The rule is the same one: `production` means the product acts on the + result without a human reading it first, and a model whose only recorded + validation is the distribution it was pretrained on has not earned that. + The sentinel geography blocks the promotion mechanically instead of + relying on someone remembering. + """ + return ( + self.is_runnable + and bool(self.geography_validated) + and UNVALIDATED_GEOGRAPHY not in self.geography_validated + ) + + @property + def needs_human_confirmation(self) -> bool: + return Requirement.HUMAN_CONFIRMATION in self.requirements + + @property + def needs_guided_capture(self) -> bool: + return Requirement.GUIDED_CAPTURE in self.requirements + + +#: Recorded in `geography_validated` when a model's only provenance is its +#: pretraining distribution. COCO is web-scraped photographs from everywhere and +#: nowhere; a Holstein in a Dutch field and a White Fulani in Kaduna are not the +#: same test. +UNVALIDATED_GEOGRAPHY = "global-coco" + +#: The quality checks `app/quality.py` implements, on every visual capture. +_GATE = ("resolution", "illumination", "motion_blur") + + +#: Every `reject_if` condition in the registry, and the sentence a farmer reads +#: when it fires. +#: +#: **Derived rather than written twice.** `_cap` turns a capability's own +#: `reject_if` tuple into its `limits` vocabulary, so a rejection condition and +#: the wording for it cannot drift apart, and a condition added without a phrase +#: fails a test rather than silently leaving a capability unable to explain +#: itself. That is the same relation `FORBIDDEN_CLAIMS` has to +#: `forbidden_claims`. +#: +#: The wording is imperative where the farmer can fix it in the yard and +#: descriptive where they cannot, because a limit that reads as an instruction +#: the farmer cannot follow is worse than one that reads as a fact. +_LIMIT_PHRASES: dict[str, str] = { + # `app/quality.py` implements these three. + "resolution": "Too low a resolution to assess โ€” move closer, or use a " + "better camera", + "illumination": "Too dark or too bright to assess", + "motion_blur": "Too much motion blur to assess", + # Framing and pose. + "move_closer": "Move closer", + "wrong_angle": "Wrong angle โ€” follow the capture guide", + "wrong_pose": "The animal is not standing in the pose this capture needs", + "animal_heavily_occluded": "The animal is heavily occluded", + "animal_not_walking": "The animal is not walking", + "animal_moving": "The animal moved during the recording", + "camera_moving": "The camera moved during the recording", + "subject_partly_out_of_frame": "Part of the subject is outside the frame", + # Body parts a protocol requires to be presented. + "muzzle_not_visible": "The muzzle is not visible", + "hooks_and_pins_not_visible": "The hooks and pins are not visible", + "teeth_obscured": "The lower front teeth are obscured", + "hoof_not_lifted": "The hoof was not lifted", + "sole_not_clean": "The sole was not cleaned before the photograph", + "foot_not_presented": "The underside of the foot was not presented", + "hock_not_presented": "The hock was not presented", + "head_not_presented": "The head was not presented", + "flank_not_in_frame": "The flank is not in frame", + "region_not_visible": "One of the guided regions was not visible", + # Scene and background. + "bird_not_isolated": "The bird is not isolated against a clear background", + "cluttered_background": "The background is too cluttered to separate the " + "bird from it", + "sample_not_isolated": "The sample is not isolated from the ground around it", + "floor_not_visible": "The floor is not visible", + "birds_not_visible": "The birds are not visible", + "eggs_overlapping": "The eggs overlap, so they cannot be counted separately", + # Capture protocol not followed. + "pan_too_fast": "The pan was too fast to track birds between frames", + "density_beyond_measured_regime": "This scene is denser than anything this " + "capability has been measured on", + "insufficient_geometry": "The sweep did not carry enough geometry to " + "estimate from", + "insufficient_sample": "Fewer birds were sampled than the protocol asks for", + "recording_too_short": "The recording is shorter than the 30 seconds ยง26 " + "asks for", + # Audio. + "machinery_dominates": "Fans or machinery dominate the recording", + "speech_dominates": "Speech dominates the recording", +} + + +#: Capability key โ†’ the runtimes its declared stack leans on. +#: +#: Held as one table rather than a field on each entry so that it can be read +#: against `app/adapters/licences.py` in a single pass. Every name is either in +#: that ledger or deliberately not: `vggt`, `deeplabcut-superanimal` and +#: `sam-audio` are named by directive ยง3, ยง4 and ยง26 and have no ledger entry +#: yet, so they report as unchecked rather than as cleared. +#: +#: The models are the directive's own, section by section. Where a capability +#: runs today the shipped runtime is listed alongside the target stack, because +#: both are true and the difference matters. +_STACK: dict[str, tuple[str, ...]] = { + # cattle + "cattle_detection": ("sam3", "grounding-dino-hf", "yolox-onnx"), # ยง6.1 + "cattle_identity": ("megadescriptor-timm", "dinov3-onnx"), # ยง6.4 + "cattle_breed": ("hosted-multimodal", "dinov3-onnx"), # ยง6.5 + "cattle_sex": ("hosted-multimodal",), # ยง6.6 + "cattle_weight": ("sam3", "vggt"), # ยง22 + "cattle_bcs": ("hosted-multimodal", "dinov3-onnx"), # ยง7 + "cattle_age_dentition": ("hosted-multimodal",), # ยง8 + "cattle_gait": ("deeplabcut-superanimal", "sam3"), # ยง24 + "cattle_ticks": ("sam3", "grounding-dino-hf", "hosted-multimodal"), # ยง25 + "cattle_wound": ("sam3", "hosted-multimodal", "opencv-numpy"), # ยง9 + "cattle_skin": ("sam3", "hosted-multimodal", "dinov3-onnx"), # ยง10 + "cattle_hoof": ("sam3", "hosted-multimodal", "dinov3-onnx"), # ยง11 + "cattle_feces": ("sam3", "hosted-multimodal", "dinov3-onnx"), # ยง12 + "cattle_respiratory": ("sam3", "opencv-numpy"), # ยง14 + # poultry + "poultry_count": ("countgd", "grounding-dino-hf", "sam3", "yolox-onnx"), # ยง6.3 + "poultry_fecal": ("dinov3-onnx", "hosted-multimodal"), # ยง13 + "poultry_inactive_birds": ("sam3", "opencv-numpy"), # ยง15 + "poultry_weight": ("sam3", "vggt"), # ยง23 + # ยง23 again: no model at all. The empty tuple is the point โ€” a coefficient + # of variation over twenty numbers has no licence exposure of any kind. + # + # **It is also why this capability's refusal could not have been found by + # reading this table.** An empty stack looks like the safest row here: no + # weights, no terms, no gated download, nothing to check. The thing that + # ended the capability was the accuracy of the numbers going *in*, which no + # licence ledger and no model inventory can see (ADR 0023). + "poultry_uniformity": (), + "poultry_footpad": ("hosted-multimodal", "dinov3-onnx", "sam3"), # ยง18 + "poultry_heat_stress": ("hosted-multimodal", "opencv-numpy"), # ยง17 + "poultry_litter": ("sam3", "hosted-multimodal", "dinov3-onnx"), # ยง16 + "poultry_respiratory": ("sam-audio", "hosted-multimodal"), # ยง26 + "poultry_hock": ("sam3", "hosted-multimodal", "dinov3-onnx"), # ยง19 + "poultry_feather": ("sam3", "hosted-multimodal", "dinov3-onnx"), # ยง19 + "poultry_wound": ("sam3", "hosted-multimodal", "dinov3-onnx"), # ยง19 + "poultry_eye_head": ("sam3", "hosted-multimodal", "dinov3-onnx"), # ยง19 + "egg_quality": ("sam3", "opencv-numpy", "hosted-multimodal"), # ยง20 +} + + +def licence_exposure() -> dict[str, list[str]]: + """Which capabilities lean on a model Animap may not currently serve. + + The question the founder asked this registry to be able to answer without + anyone going digging, and the reason `model_stack` exists. It reports three + kinds of exposure, which need different responses: + + - **refused** โ€” the ledger has read the terms and Animap may not serve from + them. `megadescriptor-timm` is CC-BY-NC-4.0; `miewid` grants nothing at + all. A capability leaning on one of these needs a substitute before it is + sold, not before it is built. + - **gated** โ€” servable, but behind an accepted-terms wall, so + `install_models.py` cannot fetch it unattended. A deployment problem. + - **unchecked** โ€” the runtime is not in the ledger, so nobody has read its + terms. This is not the same as a problem, and it is not the same as fine. + + **Nothing here changes a capability's state**, and a test asserts that. + Imported lazily because `app/adapters/` imports this module, and a + module-level import would close the cycle. + """ + from app.adapters.licences import RUNTIME_LICENCES + + exposure: dict[str, list[str]] = {} + for key, capability in REGISTRY.items(): + flags = [] + for runtime in capability.model_stack: + licence = RUNTIME_LICENCES.get(runtime) + if licence is None: + flags.append(f"{runtime}: unchecked โ€” not in the licence ledger") + elif not licence.servable: + flags.append(f"{runtime}: refused โ€” {licence.licence}") + elif licence.gated: + flags.append(f"{runtime}: gated โ€” {licence.licence}") + if flags: + exposure[key] = flags + return exposure + + +def _number(claim, unit, low, high, basis, why, step=None, values=()) -> ClaimQuantity: + """A claim carrying a quantity, and sometimes a band word as well. + + **A number and a word are not alternatives.** ยง38 asks for broad outputs + before precise ones and gives several capabilities both โ€” a body condition + is *"thin; appropriate; heavy"* **or** a half-point band, and ยง16 prints + *"Caking: High"* beside *"18% of scanned region"*. A claim that could only + answer in figures refused the coarser half of ยง38, which is the half the + directive says to build first. + """ + return ClaimQuantity(claim=claim, unit=unit, minimum=low, maximum=high, + step=step, basis=basis, why=why, values=tuple(values)) + + +def _no_number(claim, why) -> ClaimQuantity: + """A claim that carries no quantity and no word. `why` says what it is.""" + return ClaimQuantity(claim=claim, why=why) + + +def _words(claim, values, why) -> ClaimQuantity: + """A claim whose `observations[].value` is a word from a closed list. + + The list is the whole of what this claim can say, exactly as + `allowed_claims` is the whole of what a capability can claim. See + `ClaimQuantity.values`. + """ + return ClaimQuantity(claim=claim, values=tuple(values), why=why) + + +#: What a severity word may be, where the directive prints one and gives no +#: rubric for it. ยง9's *"Moderate surrounding swelling"* and ยง19's *"Moderate +#: visible hock lesion"* are the two printed cases. +#: +#: **Four words and no number**, which is the distinction ยง18 and ยง19 turn on: +#: ยง18 gives the footpad an explicit 0-to-4 rubric and gets `approximate_grade`, +#: and ยง19 gives the hock none, so a grade here would be a scale this file +#: invented. A word is a description; a number would be a score. +_SEVERITY = ("none", "mild", "moderate", "marked") + +#: Presence, for a finding whose only honest reading is whether it is there. +_PRESENCE = ("none", "present") + +#: Normal or abnormal, which is ยง11's and ยง19's own summary finding. +_NORMALITY = ("normal", "abnormal") + +#: Where on a bird a wound is, for ยง19's *"Open wound visible on left flank"*. +#: +#: **A site is enumerable and a breed is not**, which is why one is a list here +#: and the other is `free_value_reason`. A bird has a fixed anatomy; a farm's +#: animals and a region's breeds do not. +_BIRD_SITES = ( + "head", "comb", "wattle", "cheek", "eye", "beak", "neck", "breast", "keel", + "back", "abdomen", "left_wing", "right_wing", "left_flank", "right_flank", + "vent", "left_thigh", "right_thigh", "left_shank", "right_shank", + "left_hock", "right_hock", "left_foot", "right_foot", "toe", "tail", +) + +#: ยง16's band words, for a litter condition reported coarsely. +#: +#: ยง16's own printed line is *"Caking: High"*, and the four condition claims +#: were declared as percentages only โ€” so the directive's own example was +#: refused as *"'caked' carries no word at all"* while the percentage beside it +#: published. ยง38's *"build broad outputs before precise outputs"* is the +#: general form of that mistake. +_LITTER_BANDS = ("none", "low", "moderate", "high") + +#: Where on a cow a wound is, for ยง38's *"location"* bullet. +#: +#: Longer than a bird's list because a cow is bigger and because ยง25's guided +#: regions and ยง11's hoof are already named parts of this product's vocabulary. +_CATTLE_SITES = ( + "head", "left_eye", "right_eye", "muzzle", "left_ear", "right_ear", "neck", + "dewlap", "brisket", "shoulder", "left_flank", "right_flank", "back", + "rump", "tail_head", "udder", "groin", "left_fore_leg", "right_fore_leg", + "left_hind_leg", "right_hind_leg", "left_fore_hoof", "right_fore_hoof", + "left_hind_hoof", "right_hind_hoof", +) + +#: Where feather coverage is reduced, for ยง19's *"on back and tail"*. The +#: combination the directive prints is a member in its own right, because a +#: single-valued field cannot say two places at once and ยง19 says two. +_FEATHER_SITES = ( + "head", "neck", "back", "tail", "back_and_tail", "breast", "vent", "wings", +) + +#: Where discharge is, for ยง19's *"around left eye"*. +_HEAD_SITES = ( + "left_eye", "right_eye", "both_eyes", "nostrils", "sinus", "beak", "comb", + "wattle", +) + + +_SCALE = Basis.SCALE +_DIRECTIVE = Basis.DIRECTIVE +_GUARD = Basis.GUARDRAIL + +#: What a photograph can resolve, for a count of discrete marks in one close-up. +#: The ceiling is the frame's rather than the body part's, on the same reasoning +#: `cattle_detection` uses for animals: a bound that clipped a badly affected +#: animal would delete the capture that mattered most. +_LESIONS_IN_FRAME = "A count of separate marks in one close-up. The ceiling is " \ + "what a frame can resolve, not what a hoof or a flank " \ + "usually carries โ€” the worst animal on the farm is the one " \ + "this must not refuse." + +#: Capability key โ†’ what each of its allowed claims may carry. +#: +#: Held as one table rather than inline on 28 output specs for the same reason +#: `_STACK` is: it can be read against `allowed_claims` in a single pass, and +#: `test_dispositions.py` asserts that every claim in the registry appears here +#: exactly once. A claim added to a vocabulary without an entry fails a test +#: instead of quietly becoming unbounded, which is how the last 73 happened. +#: +#: **The bias is deliberately towards declaring a number.** A claim wrongly +#: marked as carrying none refuses a legitimate reading, and a refused reading is +#: invisible to everyone except the farmer who lost it. A claim wrongly given a +#: wide bound refuses less than it should, which a watchdog can find. So a number +#: is declared wherever one is meaningful, and `carries_number = False` is +#: reserved for claims where a figure would be an assertion the directive +#: refuses โ€” a lameness score, a disease probability, an exact physiology. +_QUANTITIES: dict[str, tuple[ClaimQuantity, ...]] = { + # ---- cattle --------------------------------------------------------- + "cattle_detection": ( + _no_number("cattle_detected", + "Whether an animal is in frame. The count is `visible_count`."), + _number("visible_count", "animals", 0, 500, _GUARD, step=1, + why="Animals in one frame, so the ceiling is what a frame can " + "physically hold rather than what a herd can."), + _no_number("subject_framed", + "Whether one animal is framed well enough for the per-animal " + "capabilities. A yes-or-no finding, not a count."), + ), + "cattle_identity": ( + replace( + _no_number("identity_candidate", + "An animal, named. A match score is not a property of " + "the animal and belongs in `confidence`, which the " + "schema already bounds 0 to 1 โ€” ยง37 is explicit that a " + "bare percentage beside a result is the wrong thing to " + "show."), + free_value_reason="The animal's own name, from this farm's " + "register. ยง6.4 prints 'This looks like Kofi', " + "and a registry cannot enumerate a farm's " + "animals. Confirmed by the user before it " + "becomes a record.", + ), + _no_number("no_confident_match", + "The absence of a match. There is nothing to count."), + replace( + _no_number("closest_candidate", + "An animal on this farm's register, named but not " + "asserted. It carries no number for the same reason " + "`identity_candidate` carries none, and for one more: a " + "similarity beside a name Animap has just declined to " + "settle on would be read as how nearly it settled. The " + "run's own score travels once, under " + "`top_candidate_similarity`, uncalibrated and named so."), + free_value_reason="The animal's own name, from this farm's " + "register โ€” the same value `identity_candidate` " + "carries, under the type that does not assert. " + "ยง6.4's 'Also considered', and picking one is a " + "correction rather than a confirmation.", + ), + ), + "cattle_breed": ( + replace( + _no_number("likely_breed", + "A breed name (ยง6.5). A photograph carries the " + "phenotype, and a number beside a breed would be the " + "pedigree share no camera can see."), + free_value_reason="A breed name. An enum of Nigerian cattle breeds " + "would refuse the animal ยง6.5 exists to protect " + "โ€” 'Do not force breed when crossbred/uncertain' " + "โ€” by refusing the breed nobody listed instead. " + "Confirmed by the user.", + ), + replace( + _no_number("breed_like_phenotype", + "A phenotype description. A percentage of breed " + "composition is a pedigree claim no photograph " + "carries."), + free_value_reason="The breed a phenotype resembles, for the same " + "reason as `likely_breed`. ยง6.5's " + "'White-Fulani-like phenotype'.", + ), + _no_number("crossbred_or_uncertain", + "ยง6.5's first-class answer, and a word. A crossbreeding " + "fraction would be the forced breed ยง6.5 forbids, written as " + "arithmetic."), + ), + "cattle_sex": ( + _words("likely_sex", ("male", "female"), + "Male or female (ยง6.6). Two words, which is the whole of what " + "ยง6.6 asks for."), + _no_number("not_determinable_from_view", + "The honest failure ยง6.6 asks for."), + ), + "cattle_weight": ( + _number("weight_range_estimate", "kg", 20, 1_200, _GUARD, + why="A newborn calf to an oversized bull. Wide on purpose: ยง22's " + "whole point is that the estimate is broad, and a bound that " + "clipped a real animal would delete the capture rather than " + "flag it."), + ), + "cattle_bcs": ( + _number("body_condition_band", "bcs_1_5", 1.0, 5.0, _SCALE, step=0.5, + # **ยง38's coarse band, which this claim could not say.** "Start: + # thin; appropriate; heavy โ€” or broad score band. Later: + # half-point BCS." Declaring only the half-point band answered + # the *later* half of that sentence and refused the *start*, + # which is ยง38 run backwards and is what ยง38 is written against. + values=("thin", "appropriate", "heavy"), + why="The 1-to-5 scale, at the half-point granularity ยง7's own " + "example uses. `step` is what refuses \"BCS 2.63\". ยง38's " + "three words are the coarse form the same claim may take " + "instead, and the section asks for them first."), + _number("condition_trend", "bcs_1_5_change", -4.0, 4.0, _SCALE, step=0.5, + why="A direction against the previous band, and ยง7 prints it as " + "words. Where a model puts the arithmetic instead, the " + "difference of two points on a 1-to-5 scale cannot exceed " + "four and cannot be finer than the scale it came from."), + # **A registry gap, closed.** ยง7's UI prints "Previous: 3.0-3.5" beside + # this scan's band, and until now no claim carried it โ€” so the + # directive's own screen line had no structured field and the quantity + # gate refused it. The previous score is already `optional_inputs`, so + # this restates an input the capability was handed rather than reading + # a second animal off the photograph. + _number("previous_band", "bcs_1_5", 1.0, 5.0, _SCALE, step=0.5, + why="The band the last scan produced, restated for comparison. " + "Same scale and same granularity as the reading itself, " + "because it is a reading itself. ยง7 prints it."), + ), + "cattle_age_dentition": ( + _number("age_band", "years", 0, 30, _GUARD, step=0.5, + why="Cattle do not reach 30 years. Half years rather than " + "continuous because the teeth resolve five states across " + "five years and nothing finer โ€” a two-decimal age from a " + "tooth photograph is the 0.06-years-RMSE claim its " + "disposition calls label leakage."), + _number("permanent_incisor_count", "permanent_incisors", 0, 8, _SCALE, + step=1, + why="Cattle carry eight permanent incisors, all on the lower " + "jaw. The unit defines the ceiling; a ninth is not a " + "surprising reading, it is a broken one."), + ), + "cattle_gait": ( + _no_number("gait_asymmetry", + "ยง24's whole point. 'Possible gait asymmetry' has no " + "magnitude, and the number a model reaches for is the " + "'Lameness score 3' the section names as the thing not to " + "output. This is the claim a watchdog laundered a 3 through."), + _no_number("possible_movement_issue", + "A screen, for the same reason. ยง24 refuses a score, and a " + "score is what a number here would be."), + ), + "cattle_ticks": ( + # **The count stays, and it is a queue rather than a total.** ยง25 prints + # it, `HUMAN_CONFIRMATION` is set, and every detection is shown for + # confirmation or removal โ€” so the number a farm ends up with is the + # number a person kept, not the number a detector produced. + # + # `experiments/cattle_ticks/` is what makes that framing load-bearing + # rather than decorative. On 24 composite frames holding 322 real ticks, + # with an exemplar bank sharing no photograph with the pasted ticks: + # **precision 0.798, recall 0.258**, F1 0.390, count MAE 9.2 and a bias + # of -9.08. Four detections in five are ticks; roughly three ticks in + # four are missed. That supports a confirmation queue and supports + # nothing a farm subtracts from. + _number("probable_tick_count_sampled", "probable_ticks", 0, 1_000, _GUARD, + step=1, + why="A sampled count across four guided regions, so the ceiling " + "is what those regions can hold rather than a whole animal. " + "ยง25's own example is 17. The measured support is a " + "confirmation queue and not a tally: precision 0.798 at " + "recall 0.258 on a holdout exemplar bank, so the number " + "under-reports and every entry is shown for a person to " + "keep or drop."), + _words("tick_burden_band", ("none_seen", "low", "moderate", "high"), + "ยง38's four words โ€” none seen, low, moderate, high. An " + "ordinal index would invite a farm to read a level as a " + "count, which is the conflation ยง25 forbids. **The band is " + "the weaker of the two claims here, not the safer one**: at " + "recall 0.258 the published arm returned zero detections on " + "two of five real cattle photographs holding 25-45 and " + "70-160 ticks, and 'none seen' is the one wording a farm " + "could act on without confirming anything. ยง38's word is " + "*seen*, and it has to keep carrying that weight."), + ), + "cattle_wound": ( + _number("visible_wound", "wounds", 0, 500, _GUARD, step=1, + why=_LESIONS_IN_FRAME), + _words("surrounding_swelling", _SEVERITY, + "ยง9 prints 'Moderate surrounding swelling'. The directive " + "gives no rubric for severity here, so a graded number would " + "be a scale this file invented โ€” ยง38's broad-before-precise " + "run backwards. Four words is what ยง9's own wording supports."), + _words("visible_discharge", _PRESENCE, + "ยง9 prints 'No obvious visible discharge'. Presence, not " + "volume; RGB carries no volume."), + _number("approximate_visible_area", "cm2", 0, 2_000, _GUARD, + why="An area on one animal. The ceiling is a large flank lesion, " + "not a plausible one. ยง9's own example is 12-16 cmยฒ."), + _number("change_since_last_scan", "cm2", -2_000, 2_000, _GUARD, + why="The difference between two areas, so it is legitimately " + "negative when a wound heals and cannot exceed the larger " + "area in either direction."), + # **ยง38 lists four things a wound result should start with โ€” visible, + # severity, location, change โ€” and this capability had three of them.** + # `poultry_wound` carried a location and `cattle_wound` did not, which + # was an omission rather than a decision: ยง19 prints *"Open wound + # visible on left flank"* for a bird and ยง9 gives the cow the same + # shape, reference marker included. + _words("wound_location", _CATTLE_SITES, + "ยง38's 'location'. Where the wound is, from a list, because a " + "cow has a finite number of places and a free-text site is a " + "sentence in a field a farmer reads."), + ), + "cattle_skin": ( + _number("nodular_lesions_visible", "lesions", 0, 500, _GUARD, step=1, + why=_LESIONS_IN_FRAME + " ยง10 itself prints 'Multiple raised " + "nodular lesions visible' and carries no digit, so the " + "count comes from the product rather than the section: the " + "prototype's skin screen shows '12 visible lesions', and " + "the API stores it as `lesion_count`."), + _no_number("abnormal_skin_pattern", + "ยง10's second preferred output, and a pattern. An extent " + "belongs on a lesion count or an area, neither of which this " + "claim is."), + _no_number("veterinary_review_recommended", + "ยง10's escalation. A recommendation has no magnitude."), + ), + "cattle_hoof": ( + _number("visible_crack", "cracks", 0, 500, _GUARD, step=1, + why=_LESIONS_IN_FRAME), + _number("visible_lesion", "lesions", 0, 500, _GUARD, step=1, + why=_LESIONS_IN_FRAME), + _words("swelling", _SEVERITY, + "A condition of the foot, in ยง11's own list. No rubric is given " + "for it and none is invented, so it carries a severity word and " + "no number."), + _words("erosion", _PRESENCE, + "A condition of the sole, not a countable object."), + _words("overgrowth", _PRESENCE, + "A condition of the horn. A length in millimetres would need " + "the reference marker ยง11 does not ask for."), + _words("normal_or_abnormal", _NORMALITY, + "ยง11's summary finding. Two states, and they are the two."), + ), + "cattle_feces": ( + _no_number("normal", "ยง12's six appearance states, and only appearances. " + "None of them is a quantity."), + _no_number("loose", + "One of ยง12's six appearance states. Consistency is what is " + "visible; a volume or a frequency is not."), + _no_number("watery", + "An appearance, as ยง12 lists it. Nothing in a single dropping " + "photograph carries a rate or a quantity."), + _no_number("visible_blood", + "Presence, worth surfacing on its own. A proportion would be " + "the strong disease claim ยง12 refuses from feces alone."), + _no_number("visible_mucus", "Presence, for the same reason as blood."), + _no_number("unusual_colour", + "A colour, as ยง12 lists it. A number here would be a colour " + "index nobody declared and no farmer could read."), + ), + "cattle_respiratory": ( + _number("respiratory_rate_range", "breaths_per_minute", 5, 200, _GUARD, + step=1, + why="ยง14's own worked example is 44-50 breaths a minute. The " + "bounds sit far outside it in both directions so that severe " + "distress still reports, while the 100,000 a watchdog " + "validated against the unbounded schema cannot."), + _number("capture_quality", "fraction", 0, 1, _SCALE, + why="ยง14 asks for the capture quality beside the rate. A share " + "of a capture that was usable cannot exceed the whole."), + ), + + # ---- poultry -------------------------------------------------------- + "poultry_count": ( + _number("visible_count", "birds", 0, 50_000, _GUARD, step=1, + why="A visible count from one frame or one pan. The densest " + "measured frame in this file is DFCCNet's ~166 birds; a " + "whole-house pan legitimately sees thousands, so the ceiling " + "sits where a visible count stops being physically possible " + "rather than where it stops being usual. 327.4 birds is not " + "a more careful answer than 327."), + _number("unique_birds_observed_during_scan", "birds", 0, 50_000, _GUARD, + step=1, + why="ยง6.3's second quantity, over the same pan, so the same " + "ceiling bounds it. It is not the house, which is why " + "`reconciled_flock_population` is a derived claim."), + ), + "poultry_fecal": ( + _no_number("normal", "ยง13's flock-level wording, over 4-6 samples."), + _no_number("abnormal", "ยง13's flock-level wording."), + _no_number("gi_health_watch", + "ยง13's middle band, over 4-6 samples. A figure beside a band " + "is the disease probability the section defers until local " + "validation exists."), + _no_number("elevated_gi_health_risk", + "ยง13's band. A number here would be the disease probability " + "the section says to avoid until local validation exists."), + _no_number("coccidiosis_associated_visual_pattern", + "ยง13 permits the pattern named and clearly qualified. A " + "probability beside a disease name is the claim the " + "qualification exists to prevent, and Newcastle's 62.7% " + "recall is why that matters here more than anywhere."), + ), + "poultry_inactive_birds": ( + _number("inactive_candidate", "birds", 0, 10_000, _GUARD, step=1, + why="A review queue from one section scan, not a mortality count " + "for a house. ยง15's example is 5 birds."), + _number("needs_review", "birds", 0, 10_000, _GUARD, step=1, + why="The same queue, counted the same way."), + ), + "poultry_weight": ( + _number("weight_range_estimate", "kg", 0.02, 10, _GUARD, + why="A day-old chick to an oversized cock or turkey."), + _number("sample_mean", "kg", 0.02, 10, _GUARD, + why="A mean over 10-20 held birds sits inside the same range as " + "any one of them."), + _number("sample_range", "kg", 0.02, 10, _GUARD, + why="An end of the sample's own spread, in the same unit."), + # The same registry gap as `poultry_footpad`'s, from the same shape of + # sentence: ยง23 says "Sample 10-20 birds" and then asks for statistics + # over them, so the count of birds behind the mean is a figure a farm + # reads and nothing carried it. + _number("birds_sampled", "birds", 0, 10_000, _GUARD, step=1, + why="ยง23's 'Sample 10-20 birds'. How many held birds the mean " + "and the range are computed over, which is what makes them " + "readable as a sample statistic rather than as a flock " + "figure. **It qualifies a mean and a range, never a " + "spread** โ€” ADR 0023 refuses the spread outright."), + ), + # `poultry_uniformity` has no entry, and the absence is the decision. + # + # It used to declare a coefficient of variation, a sample size and a + # uniformity band. All three are gone because the capability's vocabulary is + # now empty: `experiments/poultry_weight/` measured what a vision weight does + # to a spread, and the answer is that the spread is unrecoverable. ADR 0023 + # carries the argument, and the entry itself carries the numbers. + # + # A bound is a statement that a number is publishable inside it. There is no + # such number here, so declaring one would be the wrong shape of answer. + "poultry_footpad": ( + _number("approximate_grade", "grade_0_4", 0, 4, _SCALE, step=1, + why="ยง18's 'Approximate grade: 2 / 4'. A whole-number grade on a " + "0-to-4 rubric; 2.4 is not a finer reading of it."), + _number("sampled_prevalence", "percent", 0, 100, _SCALE, + why="ยง18's own aggregate โ€” '20 birds sampled, 4 moderate or " + "worse, 20%'. A share cannot exceed the whole, and a " + "blanket 0-to-4 ceiling over this vocabulary would refuse " + "the directive's own example."), + # **Two registry gaps, closed.** ยง18 prints its aggregate as three + # figures โ€” "20 birds sampled", "4 moderate or worse", "20%" โ€” and only + # the third had a claim. The other two were digits with no structured + # field, so the directive's own sentence was refused by the quantity + # gate and the refusal was recorded rather than fixed twice over. + # + # Named `birds_sampled` rather than `sample_size` deliberately: + # `sample_size` is in `FORBIDDEN_CLAIMS` (ADR 0023), where it was put + # because a sample size is only ever shown to qualify a spread and + # `poultry_uniformity` no longer has one. That reasoning is about a + # spread. ยง18's sample size qualifies a prevalence, which survives. + _number("birds_sampled", "birds", 0, 10_000, _GUARD, step=1, + why="ยง18's 'Sample 10-20 birds' and its printed '20 birds " + "sampled'. The ceiling is a guardrail: a farm sampling ten " + "thousand birds by hand is not a reading this should " + "refuse, and nothing measures where the real limit is."), + _number("moderate_or_worse", "birds", 0, 10_000, _GUARD, step=1, + why="ยง18's '4 moderate or worse' โ€” the numerator its 20% is " + "computed from. Counted in birds, over the same sample, so " + "it shares `birds_sampled`'s ceiling."), + ), + "poultry_hock": ( + _number("visible_hock_lesion", "lesions", 0, 500, _GUARD, step=1, + why=_LESIONS_IN_FRAME), + _words("lesion_severity", _SEVERITY, + "ยง19 prints 'Moderate visible hock lesion' and gives no " + "rubric. ยง18 does give one for the footpad, and the " + "difference is the point: a grade here would be a scale this " + "file invented, which is what its disposition means by " + "'a severity word is a description and not a score'."), + _words("normal_or_abnormal", _NORMALITY, + "ยง19's summary finding, and two states. A number " + "between them would be a severity scale ยง19 declines " + "to give."), + ), + "poultry_feather": ( + _number("reduced_feather_coverage", "percent", 0, 100, _SCALE, + why="Coverage is an area on a surface, which is what a " + "segmentation mask measures directly, and a share of a bird " + "cannot exceed the bird. ยง19 prints the location rather than " + "a figure; this bounds the figure where a model gives one."), + _words("coverage_location", _FEATHER_SITES, + "ยง19's 'on back and tail'. A place, not an extent โ€” and the " + "places are a bird's, so they are listed rather than left as a " + "string a model composes."), + _words("normal_or_abnormal", _NORMALITY, + "ยง19's summary finding, and two states. A number " + "between them would be a severity scale ยง19 declines " + "to give."), + ), + "poultry_wound": ( + _number("visible_wound", "wounds", 0, 500, _GUARD, step=1, + why=_LESIONS_IN_FRAME), + _words("wound_location", _BIRD_SITES, + "ยง19's 'on left flank'. A place, not an extent, and a bird has " + "a finite number of them."), + # The other half of ยง38's four bullets, which this capability was + # missing exactly as `cattle_wound` was missing the location. ยง9 gives + # the shape and the registry's own note already says ยง19's bird wound + # is "the same shape as `cattle_wound`". + _words("surrounding_swelling", _SEVERITY, + "ยง38's 'severity', in ยง9's own words. No rubric is given for a " + "bird any more than for a cow, so a word and not a grade."), + _words("visible_discharge", _PRESENCE, + "ยง9's 'No obvious visible discharge', on a bird. Presence; RGB " + "carries no volume."), + _number("approximate_visible_area", "cm2", 0, 500, _GUARD, + why="The same claim as `cattle_wound` on a much smaller animal, " + "so the ceiling is much smaller too."), + _number("change_since_last_scan", "cm2", -500, 500, _GUARD, + why="The difference between two areas, legitimately negative " + "when a wound heals."), + ), + "poultry_eye_head": ( + _words("visible_discharge", _PRESENCE, + "ยง19's 'Visible discharge around left eye'. Presence. RGB " + "carries no volume, and a graded number would edge towards " + "the Newcastle-adjacent naming this capability exists to " + "avoid."), + _words("discharge_location", _HEAD_SITES, + "ยง19's 'around left eye'. A place, and a head has few."), + _words("swelling", _SEVERITY, + "Presence and how much of it, with no rubric given for it โ€” so " + "a word rather than a grade."), + _words("normal_or_abnormal", _NORMALITY, + "ยง19's summary finding, and two states. A number " + "between them would be a severity scale ยง19 declines " + "to give."), + ), + "poultry_heat_stress": ( + _no_number("no_obvious_visual_signs", + "ยง17's first band. Three bands are what the section asks " + "for, and it closes by refusing exact physiology."), + _no_number("some_heat_associated_behaviours", + "ยง17's second band, and ยง38 gives the same three. A figure " + "here would be a flock rate read off a partial view."), + _no_number("heat_stress_associated_behaviour_elevated", + "ยง17's third band, and the wording it prints. The house is " + "what a probe measures; this measures the birds."), + # ยง17 and ยง38 both give this capability three bands and no figure, and + # ยง17 closes with "Do not claim exact physiology". A share of the birds + # panting is the number a reasoner reaches for, and it is a flock rate + # read off a partial view โ€” the shape ยง30 rejects for house population. + # This is the widest judgement in the table and the one to challenge. + _no_number("panting", + "A behaviour ยง17 lists and prints as a band. A share of birds " + "panting is a flock rate from a partial view, which is the " + "shape ยง30 rejects for house population."), + _no_number("wing_spreading", "A behaviour, banded (ยง17)."), + _no_number("reduced_activity", "A behaviour, banded (ยง17)."), + _no_number("clustering_at_drinkers", "A behaviour, banded (ยง17)."), + _no_number("abnormal_distribution", "A behaviour, banded (ยง17)."), + ), + "poultry_litter": ( + _number("loose", "percent_of_scanned_region", 0, 100, _SCALE, + values=_LITTER_BANDS, + why="ยง16 lists the conditions and the abnormal-area percentage " + "together, and its example reads '18% of scanned region'. A " + "share of what was scanned cannot exceed the whole, whichever " + "condition the share is of."), + _number("caked", "percent_of_scanned_region", 0, 100, _SCALE, + values=_LITTER_BANDS, + why="A share of the scanned region (ยง16)."), + _number("heavily_soiled", "percent_of_scanned_region", 0, 100, _SCALE, + values=_LITTER_BANDS, + why="A share of the scanned region (ยง16)."), + _number("wet_looking", "percent_of_scanned_region", 0, 100, _SCALE, + values=_LITTER_BANDS, + why="ยง16's own worked figure โ€” 'Wet-looking areas: 18% of " + "scanned region'."), + _number("abnormal_area_percentage", "percent_of_scanned_region", 0, 100, + _SCALE, + why="ยง16 names this output. A share cannot exceed the whole, and " + "it is a share of what was scanned rather than of the house."), + # **An identifier, not a measurement, and it is declared anyway.** This + # said `carries_number = False` on the argument that a name belongs in + # the string `Observation.value` also accepts. A watchdog showed what + # that cost: ยง16's own printed output is "Worst near drinker line 3", + # the string form is refused by the quantity gate in + # `app/adapters/claims.py` because nothing structured carries the 3, and + # refusing the numeric form as well left the directive's own line + # unpublishable in both. It published before this registry change, so + # that was a regression rather than a tightening. + # + # The cost of declaring it is small and worth naming: `poultry_litter` + # already carries five continuous 0-to-100 claims, so a bounded line + # number widens the window a figure can be laundered through by a few + # integers rather than opening a new one. + _number("worst_area_location", "drinker_line", 0, 200, _GUARD, step=1, + why="ยง16's 'Worst near drinker line 3'. The number names a " + "line, so this bounds an identifier rather than a " + "measurement โ€” the ceiling is more lines than a house has, " + "because refusing a real one would refuse the directive's " + "own output."), + ), + "poultry_respiratory": ( + # **This carried a count of 0 to 1,000 and now carries no number**, and + # the reason is a measurement rather than a caution. + # + # `experiments/poultry_respiratory/` ran the shipped detector over 6,346 + # clips from two commercial-farm datasets and scored **AUC 0.4141** + # against a chance line of 0.5. It is not merely weak, it runs backwards: + # healthy clips average **6.90 events a minute** and sick clips **1.16**, + # because a spectral-flux onset detector measures activity and a sick + # flock is quiet. No threshold repairs that โ€” `ONSET_K` moves the rate + # and the ordering between the classes is what is wrong. + # + # So the count is not a poor estimate of the events; it is a number whose + # direction is inverted. ยง26's own printed line โ€” "Cough/sneeze-like + # events detected. Spot respiratory screen only." โ€” carries no digit, so + # refusing the digit costs the directive's wording nothing. + # + # **The claim itself survives**, because the signal does: frozen CLAP + # embeddings on the identical clips, on a node-disjoint split, reach + # 0.5651 against a 0.5182 majority baseline. That is a thin 4.7-point + # margin and it is enough to fail ยง36's bar for calling the capability + # impossible. The method is wrong; the capability is not. + _no_number("cough_or_sneeze_like_events_detected", + "Whether cough- or sneeze-like events were heard, which is " + "what ยง26 prints and all it prints. The count is refused " + "because the only implementation scores AUC 0.4141 against " + "chance and fires more often on healthy flocks (6.90 " + "events/min) than on sick ones (1.16) โ€” an inverted number is " + "worse than none, and a farm cannot tell them apart."), + _no_number("spot_respiratory_screen", + "ยง26's own disclaimer โ€” 'Spot respiratory screen only'. It " + "qualifies the result; it is not one."), + ), + "egg_quality": ( + _number("count", "eggs", 0, 1_000, _GUARD, step=1, + why="Eggs on a photographed tray or trays. A standard tray holds " + "30; the ceiling allows a stack of them."), + _number("shape", "eggs", 0, 1_000, _GUARD, step=1, + why="ยง20's immediate capabilities are per-tray tallies, so each " + "reports how many of the photographed eggs it applies to and " + "shares `count`'s ceiling."), + _number("obvious_dirt", "eggs", 0, 1_000, _GUARD, step=1, + why="A tally over the same tray (ยง20)."), + _number("external_discolouration", "eggs", 0, 1_000, _GUARD, step=1, + why="A tally over the same tray (ยง20)."), + _number("obvious_visible_damage", "eggs", 0, 1_000, _GUARD, step=1, + why="A tally over the same tray (ยง20). Obvious damage only โ€” a " + "hairline crack needs candling and is forbidden here."), + ), +} + + +def _ev(phrase: str, source: str, note: str = "") -> EvidenceItem: + return EvidenceItem(phrase=phrase, source=source, note=note) + + +#: Capability key โ†’ every phrase that capability may put in `evidence`. +#: +#: **This is the change ADR 0024 makes, and it is a table rather than a rule.** +#: `evidence` used to be `{"type": "string", "maxLength": 200}`, which is the +#: field every successful attack on this service landed in. It is now an enum, +#: per capability, exactly as `claims` already was โ€” and the argument that it +#: could not be is answered by reading what the directive actually prints. ยง7's +#: worked output is three observable features from a rubric. ยง10 prints one +#: sentence, ยง19 prints four, ยง11 lists six hoof findings, ยง12 lists six faecal +#: appearances and ยง38 lists the bands. Not one of those is prose a model has to +#: compose. +#: +#: ## Three rules the table follows +#: +#: **The directive's wording is kept character for character where it prints +#: one.** ยง7's *"prominent hooks"* is here as `prominent hooks`, not as +#: `Prominent hooks are visible`. A vocabulary that paraphrases the source it +#: claims to implement is a vocabulary nobody can check against it. +#: +#: **Every finding has its negative.** *"No obvious visible discharge"* is ยง9's +#: own printed output and it is the shape the whole table needs: a control that +#: can only say the abnormal thing is a control that manufactures abnormal +#: results. Where the directive prints only the positive, the negative is +#: marked `product` and is this file's judgement. +#: +#: **No phrase carries a per-capture number.** ยง25's 17 ticks, ยง18's grade of 2 +#: and ยง6.3's 327 birds all live in `observations[].value`, `range` or +#: `best_estimate`, where `OutputSpec` bounds them on the unit the registry +#: declared. A constant may appear โ€” a rubric's endpoints, ยง6.3's uncertainty +#: footnote, the seconds ยง26 asks for โ€” and +#: `tests/test_directive_payloads.py` asserts the exact set of phrases that +#: carry a digit, so adding one costs a diff. +_EVIDENCE: dict[str, tuple[EvidenceItem, ...]] = { + # ---- cattle --------------------------------------------------------- + "cattle_detection": ( + _ev("Cattle detected", "ยง6.1"), + _ev("No cattle detected in this frame", "product", + note="ยง6.1 prints only the positive. A detector that cannot report " + "an empty frame reports a cow in every one."), + _ev("Number of cattle visible in this frame", "ยง6.1"), + _ev("One animal is framed well enough for the per-animal capabilities", + "ยง6.1"), + _ev("No single animal is framed well enough to assess on its own", + "product"), + ), + "cattle_identity": ( + _ev("This looks like a registered animal", "ยง6.4", + note="ยง6.4 prints 'This looks like Kofi'. The name is the farm's, " + "not this registry's, so it rides in the observation's value " + "and the phrase carries the hedge."), + _ev("No confident match against the animals already registered", "ยง6.4"), + _ev("Matched on the muzzle", "ยง6.4"), + _ev("Matched on the face", "ยง6.4"), + _ev("Matched on the side of the body", "ยง6.4"), + _ev("A candidate for you to confirm, never a record", "ยง6.4"), + ), + "cattle_breed": ( + _ev("Likely breed, from the visible phenotype", "ยง6.5"), + _ev("The phenotype resembles a breed without matching it cleanly", + "ยง6.5", note="ยง6.5's 'White-Fulani-like phenotype'."), + _ev("Crossbred or uncertain", "ยง6.5", + note="ยง6.5 makes this a first-class answer, not a failure."), + _ev("A suggestion for you to confirm or correct", "ยง6.5"), + ), + "cattle_sex": ( + _ev("Likely male", "ยง6.6"), + _ev("Likely female", "ยง6.6"), + _ev("Not determinable from this view", "ยง6.6"), + _ev("A suggestion for you to confirm or correct", "ยง6.6"), + ), + "cattle_weight": ( + _ev("Experimental weight estimate", "ยง22"), + _ev("Estimated from the guided side sweep and the camera's own pose", + "ยง22"), + _ev("Estimated against the reference marker", "ยง22", + note="ยง22's fallback where metric depth is unreliable."), + _ev("The initial range can be wide", "ยง22"), + _ev("Add a scale weight to verify this", "ยง22"), + _ev("A broad estimate, never an exact weight", "ยง30"), + ), + "cattle_bcs": ( + # ยง7's worked output, verbatim. These three are the reason this change + # is a reading of the directive rather than a departure from it. + _ev("prominent hooks", "ยง7"), + _ev("visible pins", "ยง7"), + _ev("limited fat around tail head", "ยง7"), + _ev("hooks not prominent", "product", + note="The negative of ยง7's first feature. Without it the rubric can " + "only describe a thin animal."), + _ev("pins not visible", "product"), + _ev("fat cover around the tail head", "product"), + _ev("Condition appears to be declining", "ยง7"), + _ev("Condition appears to be improving", "product", + note="ยง7 prints the declining case; a trend that only ever falls is " + "not a trend."), + _ev("Condition appears unchanged", "product"), + # ยง38's coarse band, which the section asks the product to build + # *before* the half-point one. The words are ยง38's own. + _ev("Thin", "ยง38"), + _ev("Appropriate", "ยง38"), + _ev("Heavy", "ยง38"), + _ev("Previous band, for comparison", "ยง7", + note="ยง7's UI line 'Previous: 3.0-3.5'. The band itself is the " + "`previous_band` observation; this is the words around it."), + ), + "cattle_age_dentition": ( + _ev("Permanent incisors visible", "ยง8", + note="ยง8 prints 'Four permanent incisors visible'. The four is " + "`permanent_incisor_count`."), + _ev("Deciduous teeth still present", "ยง8"), + _ev("Wear on the incisors", "ยง8"), + _ev("Estimated age band from the dentition rule table", "ยง8"), + _ev("A known birth date was supplied and is used instead", "ยง8"), + _ev("An age band, never an age to the month", "ยง8"), + ), + "cattle_gait": ( + _ev("Possible gait asymmetry", "ยง24"), + _ev("No obvious gait asymmetry", "product", + note="ยง24 prints only the positive."), + _ev("Stride timing is uneven between left and right", "ยง24"), + _ev("Hoof trajectories differ between left and right", "ยง24"), + _ev("Back-line movement while walking", "ยง24"), + _ev("Head movement while walking", "ยง24"), + _ev("Stance duration differs between limbs", "ยง24"), + _ev("A screen, not a lameness score", "ยง24"), + ), + "cattle_ticks": ( + _ev("Probable ticks visible across the sampled regions", "ยง25"), + _ev("None seen in the sampled regions", "ยง38", + note="ยง38's word is *seen*. `experiments/cattle_ticks/` measured " + "recall at 0.258, so this phrase carries the weight of a miss " + "and has to keep saying 'seen' rather than 'none'."), + _ev("Low tick burden", "ยง38"), + _ev("Moderate tick burden", "ยง38"), + _ev("High tick burden", "ยง38"), + _ev("Sampled burden across the guided regions, not a whole-animal count", + "ยง25"), + _ev("Each detection is shown for you to confirm or remove", "ยง25"), + ), + "cattle_wound": ( + _ev("Open wound visible", "ยง9"), + _ev("No open wound visible", "product"), + _ev("Moderate surrounding swelling", "ยง9"), + _ev("No obvious surrounding swelling", "product", + note="ยง9 prints the moderate case and the negative discharge; this " + "is the missing corner of the pair."), + _ev("No obvious visible discharge", "ยง9"), + _ev("Visible discharge", "ยง9"), + _ev("Approximate visible area, measured against the reference marker", + "ยง9"), + _ev("Smaller than at the last scan", "ยง9"), + _ev("Larger than at the last scan", "ยง9"), + _ev("Unchanged since the last scan", "ยง9"), + ), + "cattle_skin": ( + _ev("Multiple raised nodular lesions visible", "ยง10"), + _ev("Abnormal skin pattern", "ยง10"), + _ev("Veterinary review recommended", "ยง10"), + _ev("No abnormal skin pattern visible", "product"), + _ev("Lesions are raised and nodular", "ยง10"), + _ev("Lesions are spread over more than one region", "product", + note="ยง10's 'scan additional lesions' action needs a phrase that " + "says why."), + ), + "cattle_hoof": ( + # ยง11's own output list, one phrase each. + _ev("Visible crack", "ยง11"), + _ev("Visible lesion", "ยง11"), + _ev("Swelling", "ยง11"), + _ev("Erosion", "ยง11"), + _ev("Overgrowth", "ยง11"), + _ev("Appears normal", "ยง11"), + _ev("Appears abnormal", "ยง11"), + _ev("What is visible on the sole, not a named condition", "ยง11"), + ), + "cattle_feces": ( + # ยง12's six initial outputs, verbatim. + _ev("Normal", "ยง12"), + _ev("Loose", "ยง12"), + _ev("Watery", "ยง12"), + _ev("Visible blood", "ยง12"), + _ev("Visible mucus", "ยง12"), + _ev("Unusual colour", "ยง12"), + _ev("An appearance, not a diagnosis", "ยง12"), + ), + "cattle_respiratory": ( + _ev("Estimated respiratory rate from flank movement", "ยง14"), + _ev("The flank was held in frame for the whole recording", "ยง14"), + _ev("Capture quality for this recording", "ยง14"), + _ev("A spot measurement, not continuous cough surveillance", "ยง14"), + ), + + # ---- poultry -------------------------------------------------------- + "poultry_count": ( + _ev("Birds visible in this frame", "ยง6.3"), + _ev("Unique birds observed during the scan", "ยง6.3"), + _ev("A visible count, never the whole flock", "ยง6.3"), + _ev("Experimental ยท dense scenes can undercount", "ยง37"), + # **A digit in a phrase, and one of only three in the registry.** ยง6.3 + # prints this footnote with its numbers in it, and the quantity gate + # refused it for as long as `evidence` was free text, because + # `poultry_count` declares no claim in percent and no capture produces + # a 10 or a 20. They are not a reading โ€” they are a product-uncertainty + # constant, which is exactly the kind of number an enumerated phrase can + # hold safely: a model cannot choose the digits without choosing the + # sentence, and a person wrote the sentence. + # + # ยง6.3's own next line is *"Use measured benchmark when available"*, so + # this phrase has an expiry. When `experiments/poultry_count/` produces + # a farm-specific figure, this is the string that has to change. + _ev("Experimental ยท dense scenes may vary by approximately 10-20% " + "until farm-specific benchmarks are complete", "ยง6.3", + note="Replace with the measured benchmark when one exists โ€” ยง6.3 " + "says so in the line under it."), + ), + "poultry_fecal": ( + _ev("Normal", "ยง13"), + _ev("Abnormal", "ยง13"), + _ev("GI-health Watch", "ยง13"), + _ev("Elevated GI-health risk", "ยง13"), + # ยง13's own words, and the qualification is part of the phrase rather + # than left to the model: "coccidiosis-associated visual pattern only + # when appropriate and clearly qualified". + _ev("Coccidiosis-associated visual pattern", "ยง13"), + _ev("A flock-level reading over several samples, not one dropping", + "ยง13"), + ), + "poultry_inactive_birds": ( + _ev("Birds needing review", "ยง15"), + _ev("No inactive birds seen in this scan", "product"), + _ev("Inactivity measured across the scan, not from one frame", "ยง15"), + _ev("Each candidate is for a person to answer", "ยง15", + note="ยง15's four buttons are `confirmation_options`; naming them " + "here would put the word *dead* into a phrase a model can " + "choose, which is the claim ยง30 rejects."), + ), + "poultry_weight": ( + _ev("Sample-bird weight estimate from geometry", "ยง23"), + _ev("Average over the sampled birds", "ยง23"), + _ev("Range across the sampled birds", "ยง23"), + _ev("Each bird was held and photographed on its own", "ยง23"), + _ev("A sample statistic, not every bird in the flock", "ยง23"), + _ev("Add a scale weight to verify this", "ยง22", + note="ยง22's confirmation, which ยง23's workflow shares."), + ), + "poultry_footpad": ( + _ev("Approximate grade on the 0-4 rubric", "ยง18", + note="ยง18 prints 'Approximate grade: 2 / 4'. The 2 is the " + "observation; the 0 and the 4 are the rubric's own ends and " + "are a constant a person wrote here."), + _ev("Birds sampled", "ยง18"), + _ev("Moderate or worse", "ยง18"), + _ev("Share of the sampled birds that are moderate or worse", "ยง18"), + _ev("A farm sample, not a whole-flock grade", "ยง18"), + _ev("Correct the grade if it looks wrong", "ยง18"), + ), + "poultry_hock": ( + _ev("Moderate visible hock lesion", "ยง19"), + _ev("Visible hock lesion", "ยง19"), + _ev("No visible hock lesion", "product"), + _ev("Appears normal", "ยง19"), + _ev("Appears abnormal", "ยง19"), + _ev("One bird, described, not a flock rate", "ยง13", + note="ยง19 gives no sampling rule; ยง13's principle that one sample " + "does not diagnose a flock is what this says out loud."), + ), + "poultry_feather": ( + _ev("Reduced feather coverage on back and tail", "ยง19"), + _ev("Reduced feather coverage", "ยง19"), + _ev("Feather coverage appears normal", "product"), + _ev("Share of the visible surface with reduced coverage", "ยง19"), + _ev("Appears abnormal", "ยง19"), + _ev("One bird, described, not a flock rate", "ยง13"), + ), + "poultry_wound": ( + _ev("Open wound visible", "ยง19"), + _ev("No open wound visible", "product"), + # ยง38 asks a wound result for visible, severity, location and change. + # ยง19 prints the first and the third for a bird; ยง9 gives the wording + # for the second, and the registry already says this capability is the + # same shape as `cattle_wound`. + _ev("Moderate surrounding swelling", "ยง9"), + _ev("No obvious surrounding swelling", "product"), + _ev("No obvious visible discharge", "ยง9"), + _ev("Visible discharge", "ยง9"), + _ev("Approximate visible area, measured against the reference marker", + "ยง9", note="ยง19 gives the wound the same shape ยง9 gives it on " + "cattle, reference marker included."), + _ev("Smaller than at the last scan", "ยง9"), + _ev("Larger than at the last scan", "ยง9"), + _ev("Unchanged since the last scan", "ยง9"), + _ev("One bird, described, not a flock rate", "ยง13"), + ), + "poultry_eye_head": ( + _ev("Visible discharge around the eye", "ยง19"), + _ev("No visible discharge around the eye", "product"), + _ev("Swelling around the head", "ยง19"), + _ev("Appears normal", "ยง19"), + _ev("Appears abnormal", "ยง19"), + _ev("One bird, described, not a flock rate", "ยง13"), + ), + "poultry_heat_stress": ( + # ยง38's three bands, and ยง17's five behaviours. + _ev("No obvious visual signs", "ยง38"), + _ev("Some heat-associated behaviours", "ยง38"), + _ev("Heat-stress-associated behaviour elevated", "ยง17"), + _ev("Panting", "ยง17"), + _ev("Wing spreading", "ยง17"), + _ev("Reduced activity", "ยง17"), + _ev("Clustering at drinkers", "ยง17"), + _ev("Abnormal distribution across the floor", "ยง17"), + _ev("A behaviour screen โ€” this does not measure the birds themselves", + "ยง17", note="ยง17 closes with 'Do not claim exact physiology'."), + ), + "poultry_litter": ( + # ยง16's own output list. + _ev("Loose", "ยง16"), + _ev("Caked", "ยง16"), + _ev("Heavily soiled", "ยง16"), + _ev("Wet-looking", "ยง16"), + _ev("Share of the scanned region that is abnormal", "ยง16"), + _ev("Worst near a drinker line", "ยง16", + note="ยง16 prints 'Worst near drinker line 3'. The 3 names a line " + "and rides in `worst_area_location`."), + _ev("Litter condition, never a moisture percentage", "ยง16"), + ), + "poultry_respiratory": ( + _ev("Cough/sneeze-like events detected", "ยง26"), + _ev("No cough- or sneeze-like events detected", "product"), + _ev("Spot respiratory screen only", "ยง26"), + ), + "egg_quality": ( + # ยง20's immediate capabilities. + _ev("Eggs counted on the tray", "ยง20"), + _ev("Abnormal shape", "ยง20"), + _ev("Obvious dirt", "ยง20"), + _ev("External discolouration", "ยง20"), + _ev("Obvious visible damage", "ยง20"), + _ev("Nothing obviously wrong with the eggs in this tray", "product"), + _ev("Fine cracks need candling and are not assessed here", "ยง20"), + ), + # `poultry_uniformity` has no entry, and the absence is the decision โ€” the + # same decision `_QUANTITIES` records for it. Its `allowed_claims` is empty + # (ADR 0023), so `schema_for` generates an enum of zero claims; an evidence + # vocabulary would be the one place left to write the refused figure in + # words, which is precisely what ADR 0023's watchdog did through free text. +} + + +def _cap(key, species, save_target, state, requirements, acquisition, **kw) -> Capability: + """Register a capability. + + Connectivity defaults to `required`, which inverts the old default. Every + capability in the directive's stack opens with a hosted model โ€” SAM 3.1, + DINOv3, CountGD, VGGT, a multimodal reasoner โ€” so the phone contributes + nothing to the answer and *"Saves now, analysis runs when you reconnect"* is + what a farmer should be told. The two capabilities that really do compute + part of their answer on the phone say `deferred` explicitly, which is + exactly the ratio a default should have. + + The per-claim quantities are spliced in here rather than written on each + `OutputSpec`, so `_QUANTITIES` stays readable as one table against + `allowed_claims`. A capability missing from it keeps the empty tuple, which + `bounds_for` falls back from and a test refuses. + """ + quantities = _QUANTITIES.get(key, ()) + if quantities: + acquisition = replace( + acquisition, output=replace(acquisition.output, quantities=quantities) + ) + evidence = _EVIDENCE.get(key, ()) + # **A capability that may claim nothing may say nothing.** `poultry_uniformity` + # declares no claims (ADR 0023), and giving it capture limits would leave one + # array of reader-facing strings on a capability whose whole point is that it + # publishes none. The shared limits are earned by having a vocabulary. + limits = ( + _SHARED_LIMITS + + tuple( + EvidenceItem(_LIMIT_PHRASES[condition], "reject_if") + for condition in acquisition.reject_if + if condition in _LIMIT_PHRASES + ) + if evidence else () + ) + acquisition = replace( + acquisition, allowed_evidence=evidence, allowed_limits=limits + ) + return Capability( + key=key, + species=species, + state=state, + requirements=frozenset(requirements), + connectivity=kw.pop("connectivity", Connectivity.REQUIRED), + save_target=save_target, + acquisition=acquisition, + model_stack=_STACK.get(key, ()), + **kw, + ) + + +_EXPERIMENTAL = CapabilityState.EXPERIMENTAL +_CONFIRM = Requirement.HUMAN_CONFIRMATION +_GUIDED = Requirement.GUIDED_CAPTURE + + +#: The registry. 28 capabilities, classified against directive ยง6 to ยง30. +#: +#: It was 21 until ยง31's Group A was checked item by item and four of its twenty +#: entries turned out to have no key at all โ€” breed (ยง6.5), sex (ยง6.6), cattle +#: feces (ยง12), and ยง19's four visual observations, which are four keys rather +#: than one. Their absence cost more than four features: three claims the +#: directive rejects had no capability to hang on, so `FORBIDDEN_CLAIMS` never +#: saw them. +#: +#: The section reference on each entry is the directive's own verdict for it. +#: Where the directive and the earlier measurement disagreed, the directive +#: decides the classification and the measurement becomes the stated uncertainty +#: in `app/dispositions.py` โ€” which is what the founder asked for and is also +#: the only reading that survives ยง36. +REGISTRY: dict[str, Capability] = { + c.key: c + for c in [ + # ---- cattle ----------------------------------------------------- + # ยง6.1 "Build now". Runs today, and stays experimental rather than + # production because ยง37 separates measured accuracy from a demo: the + # only measurement is 31 frames labelled by one non-expert (ADR 0018). + _cap("cattle_detection", "cattle", "animal", _EXPERIMENTAL, (), + AcquisitionProtocol( + protocol="single_frame", + modality=(Modality.RGB_PHOTO,), + # guardrail. Animals in one frame, so the ceiling is what a + # frame can physically hold, not what a herd can. + output=OutputSpec(type="detection", unit="animals", + plausible_min=0, plausible_max=500, step=1, + # `cattle_detected` and `subject_framed` are + # yes-or-no findings, not counts of animals. + measured_claims=("visible_count",)), + required_views=("whole_animal",), + reject_if=_GATE, + allowed_claims=("cattle_detected", "visible_count", "subject_framed"), + forbidden_claims=("herd_size",), + ), + output_kind="measurement", + # `deferred`, not `required`: the Android app ships a MediaPipe + # COCO detector over `efficientdet_lite0.tflite` that answers + # "cattle detected" offline + # (`apps/android/.../capture/SubjectDetector.kt`). The service's + # YOLOX-m produces the count. Part of the answer really is on the + # phone, which is what this tier means. + connectivity=Connectivity.DEFERRED, + on_device_candidate=True, + model_provider="yolox-m-coco", model_version="0.1.1rc0", + geography_validated=(UNVALIDATED_GEOGRAPHY,), + notes="Counts the cattle visible in one frame, and reports whether " + "a single animal is framed well enough for the per-animal " + "capabilities. Never the herd size."), + + # ยง6.4 "Build immediately โ€” high priority". MegaDescriptor or DINOv3 + # frozen embeddings, nearest neighbour, and the user confirms. No custom + # muzzle model is waited for. + _cap("cattle_identity", "cattle", "animal", _EXPERIMENTAL, (_GUIDED, _CONFIRM), + AcquisitionProtocol( + protocol="enrolment_and_match", + modality=(Modality.RGB_PHOTO,), + output=OutputSpec(type="identity_candidates"), + required_views=("front_face", "left_face", "right_face", + "muzzle", "side_body"), + optional_inputs=("ear_tag_photo",), + confirmation_options=("confirm", "not_this_animal", + "choose_another_animal", "register_new_animal"), + reject_if=_GATE + ("muzzle_not_visible",), + # **`closest_candidate` is here because the refusal path is the + # common path.** At the measured operating point three quarters + # of correct matches are refused, so *"not a confident match, + # and these are the nearest on your register"* is what a farm + # sees most days โ€” and `Claims.kt::claimOf` denies by default, + # so a type absent from this tuple is refused as `NOT_ALLOWED` + # and the names render as an empty list. Omitting it discarded + # both the answer and `evidence_correction`'s highest-value + # training signal, which is a person picking rank 2. + # + # It is a ranked list and never a settled name. The two are + # separate types precisely so the device does not have to + # decide which sentence to show from a field that does not say; + # `deploy/semantics/cattle_identity.json` writes one sentence + # for each, and `identity_without_confirmation` below still + # forbids turning either into a record without a person. + allowed_claims=("identity_candidate", "no_confident_match", + "closest_candidate"), + forbidden_claims=("identity_without_confirmation",), + ), + # Named from `models/cattle_identity/model_card.json`, whose + # `model_id` and `version` these two must equal โ€” the release row + # records the same pair, and a result that cannot be matched back to + # an artefact is not evidence (`app/schemas.py`). + # + # **Measured before it was named.** Run `8db9e0bd1b30`, + # `experiments/cattle_identity/metrics.json`: 169 enrolled animals + # from the muzzle268 database, closed-set top-1 0.9772, top-3 0.9937, + # MRR 0.9853, against a 0.005917 chance rate. + model_provider="dinov3-vits16-onnx", model_version="1", + # **The open-set result is why this is `experimental` and why the + # sentinel is here rather than a country code.** With no threshold + # the matcher names an unenrolled animal 100% of the time, because + # every query has a nearest neighbour; the similarity cutoff that + # admits no impostor accepts 24.08% of the correct matches. The + # measurement is on US beef breeds โ€” no Nigerian and no zebu animal + # has been through it โ€” so `may_be_promoted_to_production` stays + # False mechanically rather than by anybody remembering. + geography_validated=(UNVALIDATED_GEOGRAPHY,), + notes="Directive ยง6.4. Shows 'This looks like Kofi' and asks. An " + "unconfirmed match is a candidate, never a record."), + + # ยง6.5 "Build now as Human Confirmation". A hosted multimodal model plus + # DINOv3 reference retrieval, and the directive is explicit that a + # crossbred animal must be allowed to stay crossbred. + _cap("cattle_breed", "cattle", "animal", _EXPERIMENTAL, (_CONFIRM,), + AcquisitionProtocol( + protocol="whole_animal_side", + modality=(Modality.RGB_PHOTO,), + output=OutputSpec(type="breed_suggestion"), + required_views=("side",), + optional_inputs=("head", "dam_breed", "sire_breed"), + confirmation_options=("confirm", "correct", "crossbred", + "not_sure"), + reject_if=_GATE + ("animal_heavily_occluded", "move_closer"), + allowed_claims=("likely_breed", "breed_like_phenotype", + "crossbred_or_uncertain"), + forbidden_claims=("forced_breed_when_crossbred",), + ), + # Answered by a hosted visual reasoner, not a file on disk. + # `providers.HostedModelProvider` says why the pair is stable + # here while the model that actually ran travels on the result. + model_provider="hosted-multimodal", model_version="1", + notes="Directive ยง6.5. 'Likely White Fulani', or " + "'White-Fulani-like phenotype' where the animal does not sit " + "cleanly in one breed. **Not forcing a breed is a required " + "output, not a failure** โ€” `crossbred_or_uncertain` is a " + "first-class answer and the user confirms either way."), + + # ยง6.6 "Build now as Human Confirmation". The shortest entry in the + # directive and the one with the sharpest workflow constraint. + _cap("cattle_sex", "cattle", "animal", _EXPERIMENTAL, (_CONFIRM,), + AcquisitionProtocol( + protocol="whole_animal_side", + modality=(Modality.RGB_PHOTO,), + output=OutputSpec(type="sex_suggestion"), + required_views=("side",), + optional_inputs=("rear_view",), + confirmation_options=("confirm", "correct", "not_sure"), + # ยง6.6: "Do not make this a blocker for registration." Declared + # rather than left to whoever builds the form, because a + # capability that quietly gates registration is discovered by a + # farmer standing in a paddock with an unregistered animal. + never_blocks=("animal_registration",), + reject_if=_GATE + ("animal_heavily_occluded", "wrong_pose"), + allowed_claims=("likely_sex", "not_determinable_from_view"), + forbidden_claims=("sex_without_confirmation",), + ), + # Answered by a hosted visual reasoner, not a file on disk. + # `providers.HostedModelProvider` says why the pair is stable + # here while the model that actually ran travels on the result. + model_provider="hosted-multimodal", model_version="1", + notes="Directive ยง6.6. 'Likely male', confirmed by the user, and " + "never in the way of registering an animal."), + + # ยง22 "Engineering spike โ†’ Experimental". The ยง34 schema example is this + # capability, and the fields below follow it. + _cap("cattle_weight", "cattle", "animal", _EXPERIMENTAL, (_GUIDED, _CONFIRM), + AcquisitionProtocol( + protocol="guided_side_sweep", + modality=(Modality.RGB_VIDEO, Modality.CAMERA_POSE, + Modality.DEPTH_OPTIONAL), + # guardrail. A newborn calf to an oversized bull. Wide on + # purpose: ยง22's whole point is that the estimate is broad, and + # a bound that clipped a real animal would delete the capture + # rather than flag it. + output=OutputSpec(type="weight_estimate", unit="kg", show_range=True, + plausible_min=20, plausible_max=1_200, + measured_claims=("weight_range_estimate",)), + minimum_capture_seconds=2, + preferred_capture_seconds=4, + required_views=("side",), + optional_inputs=("arcore_depth", "reference_marker", "rear_view"), + confirmation_options=("add_scale_weight", "correct", "retake"), + reject_if=_GATE + ("animal_heavily_occluded", "insufficient_geometry", + "wrong_pose"), + allowed_claims=("weight_range_estimate",), + forbidden_claims=("exact_weight_from_single_photo",), + ), + output_kind="measurement", + notes="Directive ยง22. A guided sweep with camera pose is not an " + "arbitrary photograph, and the range is allowed to be wide. " + "A scale reading outranks it and is what verifies it."), + + # ยง7. The directive rejects the previous 'not viable' outright: human + # disagreement means a broader band and a trend, not abandonment. + _cap("cattle_bcs", "cattle", "animal", _EXPERIMENTAL, (_GUIDED, _CONFIRM), + AcquisitionProtocol( + protocol="side_and_rear_quarter", + modality=(Modality.RGB_PHOTO,), + # scale. The 1-to-5 BCS scale, at the half-point granularity + # ยง7's own example uses (`2.5`, range `[2.5, 3.0]`). This is the + # bound `BCS_SCHEMA` was hand-written to carry and every other + # capability lacked; `step` is what refuses "BCS 2.63". + output=OutputSpec(type="body_condition_band", unit="bcs_1_5", + show_range=True, + plausible_min=1.0, plausible_max=5.0, step=0.5, + # `condition_trend` is a direction against a + # previous band, not a point on the scale. + measured_claims=("body_condition_band",)), + required_views=("side", "rear_quarter"), + optional_inputs=("previous_score",), + confirmation_options=("looks_right", "correct_score", "retake"), + reject_if=_GATE + ("hooks_and_pins_not_visible", "animal_heavily_occluded"), + allowed_claims=("body_condition_band", "condition_trend", + "previous_band"), + forbidden_claims=("bcs_point_score",), + ), + # Answered by a hosted visual reasoner, not a file on disk. + # `providers.HostedModelProvider` says why the pair is stable + # here while the model that actually ran travels on the result. + model_provider="hosted-multimodal", model_version="1", + notes="Directive ยง7. '2.5-3.0' with 'Looks right / Correct score', " + "and 'Condition appears to be declining' against the previous " + "band. Never 'BCS 2.63'."), + + # ยง8. An age band, from a deterministic rule table over what the visual + # model can see. Not a chronological age. + _cap("cattle_age_dentition", "cattle", "animal", _EXPERIMENTAL, (_GUIDED, _CONFIRM), + AcquisitionProtocol( + protocol="lower_incisor_close_up", + modality=(Modality.RGB_PHOTO,), + # guardrail. Cattle do not reach 30 years. `step` is 0.5 rather + # than continuous because the disposition is explicit that the + # teeth resolve five states across five years and nothing + # finer โ€” a two-decimal age from a tooth photograph is the + # 0.06-years-RMSE claim that entry calls label leakage. + output=OutputSpec(type="age_band", unit="years", show_range=True, + plausible_min=0, plausible_max=30, step=0.5, + # `permanent_incisor_count` counts teeth, of + # which cattle have eight, not years. + measured_claims=("age_band",)), + required_views=("lower_front_teeth",), + optional_inputs=("known_birth_date", "breed", "sex"), + confirmation_options=("known_dob", "correct", "retake"), + reject_if=_GATE + ("move_closer", "teeth_obscured", "wrong_angle"), + allowed_claims=("age_band", "permanent_incisor_count"), + forbidden_claims=("exact_age_from_teeth",), + ), + # Answered by a hosted visual reasoner, not a file on disk. + # `providers.HostedModelProvider` says why the pair is stable + # here while the model that actually ran travels on the result. + model_provider="hosted-multimodal", model_version="1", + notes="Directive ยง8. 'Estimated age: 3-4 years. Four permanent " + "incisors visible.' A known birth date always wins."), + + # ยง24 "Engineering spike / Experimental". SuperAnimal-Quadruped pose plus + # motion heuristics; the ยง34 example gives this capability's claim lists. + _cap("cattle_gait", "cattle", "animal", _EXPERIMENTAL, (_GUIDED,), + AcquisitionProtocol( + protocol="side_walk", + modality=(Modality.RGB_VIDEO,), + output=OutputSpec(type="gait_screen"), + minimum_capture_seconds=5, + preferred_capture_seconds=10, + minimum_distance_m=5.0, + required_views=("side",), + optional_inputs=("rear_view",), + reject_if=_GATE + ("animal_not_walking", "animal_heavily_occluded", + "wrong_pose"), + allowed_claims=("gait_asymmetry", "possible_movement_issue"), + forbidden_claims=("definitive_hoof_disease", "lameness_score"), + ), + notes="Directive ยง24. 'Possible gait asymmetry', not 'lameness " + "score 3 caused by left rear hoof disease'."), + + # ยง25 "Engineering spike / Human Confirmation". Sampled burden across + # four guided regions, with the detections shown for removal. + _cap("cattle_ticks", "cattle", "animal", _EXPERIMENTAL, (_GUIDED, _CONFIRM), + AcquisitionProtocol( + protocol="guided_region_sequence", + modality=(Modality.RGB_PHOTO,), + # guardrail. A sampled count across four guided regions, so the + # ceiling is what those regions can hold, not a whole animal. + output=OutputSpec(type="tick_burden", unit="probable_ticks", + plausible_min=0, plausible_max=1_000, step=1, + # `tick_burden_band` is none/low/moderate/high. + measured_claims=("probable_tick_count_sampled",)), + required_views=("ears", "neck_dewlap", "tail", "groin_udder"), + optional_inputs=("macro_lens", "zoom"), + confirmation_options=("confirm_detection", "remove_detection"), + reject_if=_GATE + ("move_closer", "region_not_visible"), + allowed_claims=("probable_tick_count_sampled", "tick_burden_band"), + forbidden_claims=("total_body_tick_count",), + ), + notes="Directive ยง25 and ยง38. '17 probable ticks visible across " + "sampled regions', plus a none/low/moderate/high band. " + "Sampled burden, never a whole-animal count."), + + # ยง9 "Experimental". Observations and, with a marker in frame, an area. + _cap("cattle_wound", "cattle", "animal", _EXPERIMENTAL, (), + AcquisitionProtocol( + protocol="lesion_close_up", + modality=(Modality.RGB_PHOTO,), + # guardrail. An area on one animal, in cmยฒ. The upper bound is + # a large flank lesion, not a plausible one. + output=OutputSpec(type="wound_observation", unit="cm2", + show_range=True, + plausible_min=0, plausible_max=2_000, + # Not `change_since_last_scan`: a healing wound + # shrinks, so that number is legitimately + # negative and this minimum would delete it. + measured_claims=("approximate_visible_area",)), + required_views=("wound",), + optional_inputs=("reference_marker", "previous_capture"), + escalation="veterinary_review", + reject_if=_GATE + ("move_closer",), + allowed_claims=("visible_wound", "surrounding_swelling", + "visible_discharge", "approximate_visible_area", + "change_since_last_scan", "wound_location"), + forbidden_claims=("wound_etiology",), + ), + # Answered by a hosted visual reasoner, not a file on disk. + # `providers.HostedModelProvider` says why the pair is stable + # here while the model that actually ran travels on the result. + model_provider="hosted-multimodal", model_version="1", + notes="Directive ยง9. Describes what is visible and, where a " + "reference marker is in frame, roughly how large. Follow-up " + "scans compare. Never names a cause."), + + # ยง10 "Experimental / escalation required". The legal exposure changes + # the wording and the workflow, not the computer vision. + _cap("cattle_skin", "cattle", "animal", _EXPERIMENTAL, (), + AcquisitionProtocol( + protocol="lesion_close_up", + modality=(Modality.RGB_PHOTO,), + output=OutputSpec(type="skin_observation"), + required_views=("affected_area",), + optional_inputs=("whole_animal", "additional_lesions"), + escalation="veterinary_review", + reject_if=_GATE + ("move_closer",), + allowed_claims=("nodular_lesions_visible", "abnormal_skin_pattern", + "veterinary_review_recommended"), + forbidden_claims=("lsd_diagnosis", "fmd_diagnosis", "fever_from_rgb"), + ), + # Answered by a hosted visual reasoner, not a file on disk. + # `providers.HostedModelProvider` says why the pair is stable + # here while the model that actually ran travels on the result. + model_provider="hosted-multimodal", model_version="1", + notes="Directive ยง10. 'Multiple raised nodular lesions visible. " + "Veterinary review recommended.' Creates a case and can " + "isolate the animal. Never 'Lumpy skin disease confirmed' โ€” " + "LSD and FMD are notifiable and a false positive creates a " + "legal duty (see `app/dispositions.py`)."), + + # ยง11 "Experimental / Guided Capture". The sole not being visible while + # the animal stands is a capture requirement, not a reason to drop it. + _cap("cattle_hoof", "cattle", "animal", _EXPERIMENTAL, (_GUIDED,), + AcquisitionProtocol( + protocol="lifted_hoof", + modality=(Modality.RGB_PHOTO,), + output=OutputSpec(type="hoof_observation"), + required_views=("sole",), + optional_inputs=("coronary_band", "interdigital_space"), + escalation="veterinary_review", + reject_if=_GATE + ("hoof_not_lifted", "sole_not_clean", "move_closer"), + allowed_claims=("visible_crack", "visible_lesion", "swelling", + "erosion", "overgrowth", "normal_or_abnormal"), + forbidden_claims=("definitive_hoof_disease",), + ), + # Answered by a hosted visual reasoner, not a file on disk. + # `providers.HostedModelProvider` says why the pair is stable + # here while the model that actually ran travels on the result. + model_provider="hosted-multimodal", model_version="1", + notes="Directive ยง11. 'Lift and clean the hoof before " + "photographing.' Observations, not a diagnosis."), + + # ยง12 "Experimental". Appearance only. The directive is explicit that + # cattle feces alone does not carry a strong disease claim. + _cap("cattle_feces", "cattle", "animal", _EXPERIMENTAL, (), + AcquisitionProtocol( + protocol="dropping_close_up", + modality=(Modality.RGB_PHOTO,), + output=OutputSpec(type="faecal_observation"), + required_views=("dropping",), + optional_inputs=("reference_marker", "previous_capture"), + escalation="veterinary_review", + reject_if=_GATE + ("move_closer", "sample_not_isolated"), + allowed_claims=("normal", "loose", "watery", "visible_blood", + "visible_mucus", "unusual_colour"), + forbidden_claims=("disease_diagnosis_from_feces",), + ), + # Answered by a hosted visual reasoner, not a file on disk. + # `providers.HostedModelProvider` says why the pair is stable + # here while the model that actually ran travels on the result. + model_provider="hosted-multimodal", model_version="1", + notes="Directive ยง12. Six visible states and nothing beyond them. " + "Blood and mucus are worth surfacing on their own; a " + "diagnosis from a dropping is not."), + + # ยง14 "Experimental / Guided Capture". Re-scoped from cough audio to + # respiratory *rate* from video: this is optical flow and an FFT, and it + # is a different capability from the continuous cough surveillance in + # ยง27 that a fixed microphone would provide. + _cap("cattle_respiratory", "cattle", "animal", _EXPERIMENTAL, (_GUIDED,), + AcquisitionProtocol( + protocol="flank_hold", + modality=(Modality.RGB_VIDEO,), + # guardrail, anchored on the directive. ยง14's own worked example + # is "44-50 breaths/min". The bounds sit far outside it in both + # directions so that severe distress still reports, while the + # 100,000 breaths a minute a watchdog validated against the + # unbounded schema cannot. + output=OutputSpec(type="respiratory_rate", unit="breaths_per_minute", + show_range=True, + plausible_min=5, plausible_max=200, step=1, + # Not `capture_quality`, which is a fraction. + measured_claims=("respiratory_rate_range",)), + minimum_capture_seconds=30, + preferred_capture_seconds=60, + required_views=("flank",), + reject_if=_GATE + ("flank_not_in_frame", "camera_moving", + "animal_moving"), + allowed_claims=("respiratory_rate_range", "capture_quality"), + forbidden_claims=("continuous_surveillance_from_spot_recording",), + ), + output_kind="measurement", + # ยง14's pipeline is segmentation, optical flow, FFT and peak + # detection. Everything after the mask is arithmetic ยง4 says to + # prefer over a neural model, so this is the strongest on-device + # candidate in the registry once a small segmenter exists. + on_device_candidate=True, + notes="Directive ยง14. 'Hold the cow's flank in frame for 30-60 " + "seconds' โ†’ segmentation โ†’ optical flow โ†’ FFT โ†’ 'Estimated " + "respiratory rate: 44-50 breaths/min'. A spot measurement, " + "explicitly not cough surveillance."), + + # ---- poultry ---------------------------------------------------- + # ยง6.2 and ยง6.3 "Build now as Experimental", on CountGD rather than a + # COCO detector. ยง28 adds the controlled pan for a whole house. + _cap("poultry_count", "poultry", "flock_cycle", _EXPERIMENTAL, (_GUIDED,), + AcquisitionProtocol( + protocol="controlled_pan", + modality=(Modality.RGB_PHOTO, Modality.RGB_VIDEO), + output=OutputSpec(type="visible_count", unit="birds", show_range=True, + # guardrail. A visible count from one frame or + # one pan. The densest measured frame in this + # file is DFCCNet's ~166 birds; a whole-house + # pan legitimately sees thousands, so the + # ceiling is set where a *visible* count stops + # being physically possible rather than where + # it stops being usual. step 1: 327.4 birds is + # not a more careful answer than 327. + plausible_min=0, plausible_max=50_000, step=1, + measured_claims=( + "visible_count", + "unique_birds_observed_during_scan")), + # ยง6.3's capture is "photo; controlled pan; density-dependent + # mode". The protocol is the density rule, not the duration: + # a single frame for a sparse yard, a controlled pan for a + # house. That is what `GUIDED_CAPTURE` is asserting here โ€” the + # user must capture in the mode the density calls for โ€” so + # there is no minimum duration, and the pan is the preferred + # rather than the only capture. + preferred_capture_seconds=15, + required_views=("floor_section",), + optional_inputs=("exemplar_box", "house_layout"), + reject_if=_GATE + ("pan_too_fast", "density_beyond_measured_regime"), + # Two of ยง6.3's three quantities, and only two, because only two + # are observations. A model looking at pixels can say how many + # birds are in frame and how many distinct birds it saw during a + # scan. It cannot know what the farm placed, what died, or what + # was counted last week, so it cannot reconcile anything. + allowed_claims=("visible_count", "unique_birds_observed_during_scan"), + # The third quantity, kept where ยง6.3 wants it โ€” distinct, named + # and impossible to conflate โ€” but out of the model's schema. + derived_claims=("reconciled_flock_population",), + forbidden_claims=("exact_house_population",), + ), + output_kind="measurement", + # `deferred` for the same reason as `cattle_detection`: the phone's + # COCO detector counts visible birds offline. A dense house needs + # the service and, on the directive's stack, CountGD. + connectivity=Connectivity.DEFERRED, + on_device_candidate=True, + model_provider="yolox-m-coco", model_version="0.1.1rc0", + geography_validated=(UNVALIDATED_GEOGRAPHY,), + notes="Directive ยง6.3. Three numbers that must never be conflated: " + "visible count, unique birds observed during a scan, and " + "reconciled flock population โ€” and **only the first two are " + "the model's to say**. Reconciliation needs the placement " + "count, the mortality log and the last scan, none of which is " + "in the photograph, so it is `derived_claims` and the app " + "computes it. 'Approximately 327 birds " + "visible', never 'the flock is 327'. **What runs today is " + "YOLOX-m on one frame**, which is honest for a yard of fifty " + "layers and withholds a number once the frame is denser than " + "a detector can read. **CountGD is the replacement ยง6.3 asks " + "for and it has been benchmarked** โ€” this sentence read 'has " + "not' while the run sat in the repository. " + "`experiments/poultry_house_count/`, run 70dbab213a4c, 452 " + "frames of a commercial broiler house: MAE 14.84 birds, MAPE " + "11.2%, per-frame recall 0.9555, and it found nothing on 0 " + "frames. That is the dense case YOLOX-m refuses outright. It " + "is not registered because there is no CountGD model card, no " + "adapter inside `app/`, and its checkpoint is not in the " + "repository โ€” so the ยฑ11% belongs to a run and not to " + "anything a farm can call today."), + + # ยง13 "Experimental / Human Confirmation". DINOv3 retrieval against + # labelled reference sets, then a structured multimodal review, over + # 4-6 samples so the output is flock-level. + _cap("poultry_fecal", "poultry", "flock_cycle", _EXPERIMENTAL, (_GUIDED, _CONFIRM), + AcquisitionProtocol( + protocol="multi_sample_dropping", + modality=(Modality.RGB_PHOTO,), + output=OutputSpec(type="gi_health_screen"), + minimum_samples=4, + preferred_samples=6, + required_views=("dropping",), + optional_inputs=("house_location", "bird_age_days"), + confirmation_options=("looks_right", "correct", "retake"), + escalation="veterinary_review", + reject_if=_GATE + ("move_closer", "sample_not_isolated"), + # ยง13's own preferred list, verbatim, plus the flock-level + # wording it asks for. The coccidiosis entry is the directive's: + # "coccidiosis-associated visual pattern only when appropriate + # and clearly qualified". + allowed_claims=("normal", "abnormal", "gi_health_watch", + "elevated_gi_health_risk", + "coccidiosis_associated_visual_pattern"), + forbidden_claims=("newcastle_diagnosis", "coccidiosis_diagnosis", + "disease_probability_without_local_validation"), + ), + # Answered by a hosted visual reasoner, not a file on disk. + # `providers.HostedModelProvider` says why the pair is stable + # here while the model that actually ran travels on the result. + model_provider="hosted-multimodal", model_version="1", + notes="Directive ยง13. One dropping does not diagnose a flock, so " + "the output is flock-level over 4-6 samples.\n\n" + "**Why one disease is named and the other is not.** An " + "auditor found this file refusing `newcastle_diagnosis` while " + "allowing `coccidiosis_associated_visual_pattern`, with no " + "reason recorded. The reason is real and it is now written " + "down; the asymmetry is also narrower than it looked.\n\n" + "It is not coccidiosis against Newcastle. It is a **visual " + "pattern** against a **diagnosis**: `coccidiosis_diagnosis` " + "is forbidden too, and was added when this was checked. " + "Neither disease may be diagnosed. One of them may be named " + "as an appearance, and three things decide which:\n" + "1. ยง13 names it โ€” 'coccidiosis-associated visual pattern " + "only when appropriate and clearly qualified' โ€” and names " + "nothing comparable for Newcastle. The directive draws the " + "line; this file follows it.\n" + "2. The evidence is not comparable. Newcastle is 376 of " + "6,812 images in the largest public set and its reported " + "recall is 62.7%, so more than a third of cases are missed; " + "coccidiosis is one of the three common classes that carry " + "that set's headline accuracy. See `app/dispositions.py`.\n" + "3. **Newcastle is notifiable in Nigeria and coccidiosis is " + "not.** A false positive on a notifiable disease creates a " + "legal duty a farmer never asked a phone to create. That is " + "the same constraint as `cattle_skin` and it is what makes " + "the cost of the two errors different in kind, not degree."), + + # ยง15 "Experimental / Human Confirmation". Time is the signal; a dead + # bird and a sleeping bird are the same photograph. + _cap("poultry_inactive_birds", "poultry", "flock_cycle", _EXPERIMENTAL, + (_GUIDED, _CONFIRM), + AcquisitionProtocol( + protocol="section_scan", + modality=(Modality.RGB_VIDEO,), + # guardrail. A review queue from one section scan, not a + # mortality count for a house. + output=OutputSpec(type="inactive_candidates", unit="birds", + plausible_min=0, plausible_max=10_000, step=1, + measured_claims=("inactive_candidate", + "needs_review")), + minimum_capture_seconds=15, + preferred_capture_seconds=30, + required_views=("floor_section",), + confirmation_options=("dead", "sick", "resting", "fine"), + reject_if=_GATE + ("pan_too_fast", "camera_moving"), + allowed_claims=("inactive_candidate", "needs_review"), + forbidden_claims=("dead_versus_sleeping_certainty",), + ), + notes="Directive ยง15. '5 birds need review', each answered " + "Dead / Sick / Resting / Fine. Useful immediately, and every " + "answer is a training label."), + + # ยง23 "Engineering spike / Experimental". A sample-bird workflow, not + # every bird in a crowded flock from arbitrary video. + _cap("poultry_weight", "poultry", "flock_cycle", _EXPERIMENTAL, (_GUIDED, _CONFIRM), + AcquisitionProtocol( + protocol="held_sample_bird", + modality=(Modality.RGB_PHOTO, Modality.CAMERA_POSE, + Modality.DEPTH_OPTIONAL), + # guardrail. A day-old chick to an oversized cock or turkey. + output=OutputSpec(type="weight_estimate", unit="kg", show_range=True, + plausible_min=0.02, plausible_max=10, + measured_claims=("weight_range_estimate", + "sample_mean", + "sample_range")), + minimum_samples=10, + preferred_samples=20, + required_views=("side", "top"), + optional_inputs=("arcore_depth", "reference_marker"), + confirmation_options=("add_scale_weight", "correct", "retake"), + reject_if=_GATE + ("bird_not_isolated", "cluttered_background", + "insufficient_geometry"), + # **The mean and the range survive; the spread does not.** + # `experiments/poultry_weight/` measured the mean out by + # 0.33% +/- 3.5 with camera weights against 0.18% +/- 3.2 with + # exact ones at fifteen birds โ€” statistically + # indistinguishable, because measurement error averages out of + # a mean. It squares into a variance, which is why the two + # claims below it are refused. + allowed_claims=("weight_range_estimate", "sample_mean", + "sample_range", "birds_sampled"), + # The two uniformity rejections are attached here as well as to + # `poultry_uniformity`, and that is deliberate: this is the + # capability that holds the per-bird weights, so this is where a + # reasoner is standing when it reaches for a spread. + forbidden_claims=("exact_poultry_weight", "whole_flock_weight_from_video", + "flock_uniformity_from_vision_weights", + "coefficient_of_variation_from_vision_weights"), + ), + output_kind="measurement", + notes="Directive ยง23. 10-20 held birds, each isolated against a " + "clear background. The flock number is a sample statistic " + "and says so.\n\n" + "**What ยง23's four statistics come to.** The section asks for " + "an average, a range, a coefficient of variation and a " + "uniformity. The first two survive a vision weight and the " + "last two do not โ€” measurement error averages out of a mean " + "and squares into a variance โ€” so `sample_mean` and " + "`sample_range` are allowed here and the spread is refused, " + "on this capability and on `poultry_uniformity` both. " + "`sample_range` is a band in kilograms shown as a band, which " + "is what ยง38 asks for; it is not a uniformity percentage a " + "farm culls on. **Its own inflation was not measured** โ€” the " + "experiment covered the mean and the CV โ€” so that it survives " + "is a decision about the shape of the claim rather than a " + "measurement of the claim."), + + # ยง23 again, and **the one capability in this registry whose claim is + # refused rather than corrected.** ADR 0023 is the decision; this is what + # it does to the entry. + # + # The history is worth keeping because the same argument was made twice + # and was wrong both times. It was `coming_soon`, on the reasoning that + # nobody had built it. An auditor called that the weakest classification + # in the file โ€” `state` is a claim ceiling, and availability is + # `is_runnable`'s to report โ€” so it moved to `experimental` with an unmet + # `depends_on`. That was the right correction to the wrong question. + # Nobody had asked what the input would be worth when it arrived. + # + # **`experiments/poultry_weight/` asked, and the answer ends the + # capability.** A coefficient of variation over measured weights carries + # the measurement error as well as the flock's own spread, because error + # adds variance. At the best published per-bird error for this method โ€” + # 7.8%, Mortensen et al. 2016, from a *fixed depth camera* over a + # commercial house, so a floor on a phone's error rather than an estimate + # of it โ€” against a commercial flock's own 11-18% spread: + # + # * 15 birds, true CV 12%: estimated CV 14.38%, **bias +2.40 points**, + # within two points of the truth 40.0% of the time. + # * The derived uniformity reads **6.96 points low**. A flock at 60% + # uniformity reports as 53%. + # * **More birds does not fix it.** +1.91 at n=10, +2.40 at n=15, + # +2.24 at n=20, +2.16 at n=30. The exact-weight control converges + # towards zero over the same sweep (-0.50 to -0.05); this does not, + # because the bias is systematic and not sampling noise. + # * Across true CVs of 8-18% at fifteen birds the CV bias runs +1.46 + # to +2.90 and the uniformity error -1.46 to -13.94 points. **It is + # worst where the flock is most uniform**, which is the opposite of + # useful. + # + # Two things follow, and the second is why `experimental` is not + # available as a softer landing. + # + # **The error is a bias, not a width.** `experimental` means the output + # carries broad uncertainty the user can see, and a visible band is that + # state's whole remedy. A band does not move a centre. A flock at 60% + # still reads 53% with error bars drawn round it, and it reads low in the + # direction that makes a bad flock look acceptable โ€” the direction a farm + # does not investigate. + # + # **ยง36's bar is met, and it is met by the signal rather than by a failed + # model.** ยง36 refuses to call a capability unavailable on one model's + # failure. This is not one model's failure: it is error propagation, it + # binds any vision method at that error level, and the error level is a + # published floor. `experiments/poultry_weight/` also prices the escape โ€” + # a per-bird error of 4% still leaves about -3 uniformity points from + # measurement bias alone, and the small figure at n=15 is two biases + # cancelling rather than an error small enough to ignore. + # + # **The simulation has a closed form and the two agree**, which is what + # takes this from one experiment's result to a property of the + # estimator. Measuring a weight with independent relative error `e` + # gives `CV_obs = sqrt(CV^2 + e^2 + CV^2 e^2)`; at a true CV of 12% and + # e = 7.8% that is +2.34 points against the simulation's +2.40, and at + # e = 6.6% it is +1.72 against +1.74. **There is no `n` in the closed + # form**, which is the sharpest statement of why more birds does not + # help. ADR 0023 carries the full comparison. + # + # `unsupported_claim` is then the only one of the four states left. + # `production` and `experimental` both assert a publishable output; + # `coming_soon` means an engineering path nobody has walked, and this one + # was walked. The state is a claim ceiling and this ceiling is zero. + # + # **What it does not say.** Uniformity from a *scale* is exact and always + # was โ€” ยง23's "requires no ML once weights exist" is true of scale + # weights and was never true of camera weights. That route survives, in + # `derived_claims`, and it is not an inference capability: a farmer + # typing ten numbers is not a capture, has no modality and runs no model. + # It is the `reconciled_flock_population` shape exactly โ€” a legitimate + # number for the product to show, illegitimate as a *model's* claim. + # + # The declared protocol below is kept although nothing may be published + # from it. ยง36 asks for the attempt to be documented, and an entry that + # deleted its own capture would invite the next reader to propose it + # again. + _cap("poultry_uniformity", "poultry", "flock_cycle", + CapabilityState.UNSUPPORTED_CLAIM, (_GUIDED,), + AcquisitionProtocol( + protocol="held_sample_bird", + modality=(Modality.RGB_PHOTO,), + # No unit and no bounds. The capability publishes no number, and + # a declared bound would say a number is publishable inside it. + output=OutputSpec(type="uniformity"), + minimum_samples=10, + preferred_samples=20, + required_views=("side", "top"), + reject_if=_GATE + ("insufficient_sample",), + # **Empty, and that is the control.** `schema_for` builds the + # reasoner's enum from this list, and `check_vocabulary` refuses + # anything absent from it, so an empty vocabulary is the + # strictest state available rather than an unguarded one: there + # is no field a figure can be written into. + # + # **It closes the schema and not the output**, and saying so + # here is the point. A watchdog took a uniformity percentage + # into free text on 14 of the 28 capabilities, because + # `check_quantities` corroborates a prose digit against any + # structured field rather than one in the same unit โ€” 72 beside + # `sampled_prevalence: 72` publishes "Flock uniformity: 72%". + # `poultry_weight` carries its own forbidden CV the same way: a + # bird's weight in kilograms is bounded 0.02-10 and a broiler CV + # is 8-14. So this removes the claim from the model's vocabulary + # and does not remove it from the product's prose. The + # difference is real and is not the whole job; ADR 0023 and + # `tests/test_claims.py::TestTheQuantityGateCorroboratesANumberAndNotAClaim` + # carry the measurement and the one-call fix. + allowed_claims=(), + # Six entries doing three jobs. + # + # The **first two** are the descriptive rejections, which is + # what `REJECTED_CLAIMS` renders to a client with a reason and + # a corrected form. The **middle three** are the identifiers + # this capability used to allow and a reasoner would actually + # reach for โ€” forbidding a descriptive name nobody emits while + # leaving the emitted one allowed would be theatre. + # `sample_size` is in that group because a sample size is only + # ever shown to qualify a spread, and there is no longer a + # spread for it to qualify. The **last** is `exact_poultry_weight`, + # which was already here and is ยง23's own rejection. + forbidden_claims=("flock_uniformity_from_vision_weights", + "coefficient_of_variation_from_vision_weights", + "coefficient_of_variation", "uniformity_band", + "sample_size", "exact_poultry_weight"), + # The surviving route, kept visible and kept out of every + # model's schema. ยง23's arithmetic over numbers a farmer read + # off a scale is exact, and `sampling-exact` measures what it + # delivers: unbiased, and within two points of the true CV 63.5% + # of the time at fifteen birds. + derived_claims=("uniformity_from_scale_weights", + "coefficient_of_variation_from_scale_weights"), + ), + output_kind="measurement", + # **`depends_on` is gone, and its absence is the finding.** It said + # `("poultry_weight",)`, which reads as *ship that and this + # unlocks*. Shipping `poultry_weight` does not unlock this; it + # supplies the exact input that makes the answer wrong. A dependency + # that resolves into a refusal is a worse pointer than none, because + # it tells a reader the blocker is sequencing when the blocker is + # suitability. + depends_on=(), + # Still true, and now it describes only the surviving arithmetic: a + # coefficient of variation over twenty typed numbers runs anywhere. + on_device_candidate=True, + notes="Directive ยง23. **Refused as a vision claim.** A coefficient " + "of variation or a uniformity percentage computed from " + "camera weights reads 6.96 points low at the best published " + "per-bird error (7.8%), on a fifteen-bird sample against a " + "commercial flock's own 11-18% spread โ€” a flock at 60% " + "uniformity reports as 53%, and thirty birds does not fix it " + "because the bias is systematic. Uniformity is a decision " + "variable: a farm culls, re-feeds or delays a harvest on it, " + "and the error runs in the direction that makes a bad flock " + "look acceptable. **The sample mean and the per-bird range " + "are untouched and survive on `poultry_weight`** โ€” " + "measurement error averages out of a mean and squares into a " + "variance, which is the whole of it. Uniformity from scale " + "weights is exact, is what ยง23's 'no ML once weights exist' " + "was always true of, and is a derived claim the app computes " + "rather than a capability that runs."), + + # ยง18 "Experimental / Guided Capture / Human Confirmation". The three + # requirements at once โ€” the case that decides the state-plus-flags shape. + _cap("poultry_footpad", "poultry", "flock_cycle", _EXPERIMENTAL, (_GUIDED, _CONFIRM), + AcquisitionProtocol( + protocol="held_bird_footpad", + modality=(Modality.RGB_PHOTO,), + # scale. ยง18's "Approximate grade: 2 / 4". A whole-number grade + # on a 0-4 rubric; 2.4 is not a finer reading of it. + output=OutputSpec(type="footpad_grade", unit="grade_0_4", + plausible_min=0, plausible_max=4, step=1, + # Not `sampled_prevalence`. ยง18's own example + # is "20 birds sampled, 4 moderate or worse, + # 20%", and a 0-4 ceiling would refuse it. + measured_claims=("approximate_grade",)), + minimum_samples=10, + preferred_samples=20, + required_views=("underside_of_foot",), + confirmation_options=("looks_right", "correct_grade", "retake"), + reject_if=_GATE + ("foot_not_presented", "move_closer"), + allowed_claims=("approximate_grade", "sampled_prevalence", + "birds_sampled", "moderate_or_worse"), + forbidden_claims=("whole_flock_grade_from_sample",), + ), + # Answered by a hosted visual reasoner, not a file on disk. + # `providers.HostedModelProvider` says why the pair is stable + # here while the model that actually ran travels on the result. + model_provider="hosted-multimodal", model_version="1", + notes="Directive ยง18. 'Approximate grade: 2 / 4', correctable, then " + "aggregated: '20 birds sampled, 4 moderate or worse, 20%'. " + "A slaughter line does this better; a farm cannot use one."), + + # ยง19 "Hock / feather / poultry wound / eye-head" โ€” **four keys, not + # one.** + # + # ยง19 writes them as one section with one model stack and four output + # examples, which is an argument for one key only if the stack is what + # distinguishes a capability. It is not. SAM 3.1 plus a multimodal model + # plus DINOv3 retrieval is also the stack for `cattle_wound`, + # `cattle_skin`, `cattle_hoof`, `cattle_feces` and `poultry_litter`, and + # nobody would merge those. + # + # What distinguishes a capability in this registry is the acquisition + # protocol, and these are four different captures on a farm: a held + # bird's leg joint, a whole bird's back and tail, an affected area + # anywhere on the body, and a head close-up. One key would need a + # `required_views` that fits none of them, which is exactly the failure + # the schema exists to prevent. + # + # None of the four claims a flock number from one bird. ยง19 gives no + # sampling rule, so none is invented here โ€” but ยง13's principle, that + # one dropping does not diagnose a flock, is the reason + # `flock_prevalence_from_one_bird` is forbidden rather than left open. + _cap("poultry_hock", "poultry", "flock_cycle", _EXPERIMENTAL, (_GUIDED,), + AcquisitionProtocol( + protocol="held_bird_hock", + modality=(Modality.RGB_PHOTO,), + output=OutputSpec(type="hock_observation"), + required_views=("hock",), + confirmation_options=(), + reject_if=_GATE + ("hock_not_presented", "move_closer"), + allowed_claims=("visible_hock_lesion", "lesion_severity", + "normal_or_abnormal"), + forbidden_claims=("named_disease_diagnosis", + "flock_prevalence_from_one_bird"), + ), + # Answered by a hosted visual reasoner, not a file on disk. + # `providers.HostedModelProvider` says why the pair is stable + # here while the model that actually ran travels on the result. + model_provider="hosted-multimodal", model_version="1", + notes="Directive ยง19. 'Moderate visible hock lesion.' One bird, " + "described, not a flock rate."), + + _cap("poultry_feather", "poultry", "flock_cycle", _EXPERIMENTAL, (_GUIDED,), + AcquisitionProtocol( + protocol="whole_bird_dorsal", + modality=(Modality.RGB_PHOTO,), + output=OutputSpec(type="feather_observation"), + required_views=("back", "tail"), + optional_inputs=("vent", "bird_age_days"), + reject_if=_GATE + ("bird_not_isolated", "move_closer"), + allowed_claims=("reduced_feather_coverage", "coverage_location", + "normal_or_abnormal"), + forbidden_claims=("named_disease_diagnosis", + "flock_prevalence_from_one_bird"), + ), + # Answered by a hosted visual reasoner, not a file on disk. + # `providers.HostedModelProvider` says why the pair is stable + # here while the model that actually ran travels on the result. + model_provider="hosted-multimodal", model_version="1", + notes="Directive ยง19. 'Reduced feather coverage on back and tail.' " + "Where the coverage is missing is the observation; why it is " + "missing is not claimed."), + + _cap("poultry_wound", "poultry", "flock_cycle", _EXPERIMENTAL, (), + AcquisitionProtocol( + protocol="lesion_close_up", + modality=(Modality.RGB_PHOTO,), + # guardrail. The same claim as `cattle_wound` on a much smaller + # animal, so the ceiling is much smaller too. + output=OutputSpec(type="wound_observation", unit="cm2", + show_range=True, + plausible_min=0, plausible_max=500, + measured_claims=("approximate_visible_area",)), + required_views=("wound",), + optional_inputs=("reference_marker", "previous_capture"), + escalation="veterinary_review", + reject_if=_GATE + ("move_closer",), + allowed_claims=("visible_wound", "wound_location", + "approximate_visible_area", + "change_since_last_scan", + "surrounding_swelling", "visible_discharge"), + forbidden_claims=("wound_etiology", "named_disease_diagnosis", + "flock_prevalence_from_one_bird"), + ), + # Answered by a hosted visual reasoner, not a file on disk. + # `providers.HostedModelProvider` says why the pair is stable + # here while the model that actually ran travels on the result. + model_provider="hosted-multimodal", model_version="1", + notes="Directive ยง19. 'Open wound visible on left flank.' The same " + "shape as `cattle_wound`, including the reference marker and " + "the refusal to name a cause."), + + _cap("poultry_eye_head", "poultry", "flock_cycle", _EXPERIMENTAL, (_GUIDED,), + AcquisitionProtocol( + protocol="held_bird_head", + modality=(Modality.RGB_PHOTO,), + output=OutputSpec(type="eye_head_observation"), + required_views=("head",), + optional_inputs=("second_eye",), + escalation="veterinary_review", + reject_if=_GATE + ("head_not_presented", "move_closer"), + allowed_claims=("visible_discharge", "discharge_location", + "swelling", "normal_or_abnormal"), + forbidden_claims=("named_disease_diagnosis", + "flock_prevalence_from_one_bird"), + ), + # Answered by a hosted visual reasoner, not a file on disk. + # `providers.HostedModelProvider` says why the pair is stable + # here while the model that actually ran travels on the result. + model_provider="hosted-multimodal", model_version="1", + notes="Directive ยง19. 'Visible discharge around left eye.' " + "Respiratory and Newcastle-adjacent signs show up here, " + "which is exactly why nothing is named โ€” see `poultry_fecal` " + "for the notifiable-disease constraint."), + + # ยง17 "Experimental". Vision measures the birds' response; a probe + # measures the house. They are complementary, not competing. + _cap("poultry_heat_stress", "poultry", "house", _EXPERIMENTAL, (_GUIDED,), + AcquisitionProtocol( + protocol="flock_behaviour_scan", + modality=(Modality.RGB_VIDEO,), + output=OutputSpec(type="heat_stress_screen"), + minimum_capture_seconds=15, + preferred_capture_seconds=30, + required_views=("floor_section",), + optional_inputs=("house_temperature", "house_humidity"), + reject_if=_GATE + ("camera_moving", "birds_not_visible"), + allowed_claims=("no_obvious_visual_signs", + "some_heat_associated_behaviours", + "heat_stress_associated_behaviour_elevated", + "panting", "wing_spreading", "reduced_activity", + "clustering_at_drinkers", "abnormal_distribution"), + forbidden_claims=("exact_physiology", "fever_from_rgb"), + ), + notes="Directive ยง17 and ยง38. Three bands, from the behaviours " + "actually visible. A temperature and humidity probe measures " + "the environment; this measures the birds."), + + # ยง16 "Experimental", with the claim renamed. Litter *condition* is + # visible; litter *moisture* is not (ยง29). + _cap("poultry_litter", "poultry", "house", _EXPERIMENTAL, (), + AcquisitionProtocol( + protocol="floor_area_scan", + modality=(Modality.RGB_PHOTO, Modality.RGB_VIDEO), + # scale. A share of the region that was scanned. ยง16's example + # is "wet-looking areas: 18% of scanned region", and a share + # cannot exceed the whole. + output=OutputSpec(type="litter_condition", + unit="percent_of_scanned_region", + plausible_min=0, plausible_max=100, + measured_claims=("abnormal_area_percentage",)), + required_views=("floor_area",), + optional_inputs=("drinker_line_id", "moisture_probe_reading"), + reject_if=_GATE + ("floor_not_visible",), + allowed_claims=("loose", "caked", "heavily_soiled", "wet_looking", + "abnormal_area_percentage", "worst_area_location"), + forbidden_claims=("exact_litter_moisture",), + ), + # Answered by a hosted visual reasoner, not a file on disk. + # `providers.HostedModelProvider` says why the pair is stable + # here while the model that actually ran travels on the result. + model_provider="hosted-multimodal", model_version="1", + notes="Directive ยง16. 'Caking: High. Wet-looking areas: 18% of " + "scanned region. Worst near drinker line 3.' Never a moisture " + "percentage without a probe."), + + # ยง26 "Engineering spike / Experimental". A spot screen, and ยง27 is + # explicit that needing continuous monitoring must not block it. + _cap("poultry_respiratory", "poultry", "flock_cycle", _EXPERIMENTAL, (_GUIDED,), + AcquisitionProtocol( + protocol="quiet_spot_recording", + modality=(Modality.AUDIO,), + # **This declared a count of 0 to 1,000 events and now declares + # no number at all.** The shipped detector scores AUC 0.4141 on + # 6,346 real poultry-house clips โ€” below the 0.5 chance line, + # and inverted rather than merely weak: 6.90 events a minute on + # healthy clips against 1.16 on sick ones. A capability whose + # only implementation runs backwards may not publish that + # implementation's count, and ยง26's own printed line carries no + # digit, so nothing the directive asks for is lost. + output=OutputSpec(type="respiratory_events"), + minimum_capture_seconds=30, + preferred_capture_seconds=60, + optional_inputs=("house_id", "bird_age_days"), + reject_if=("recording_too_short", "machinery_dominates", + "speech_dominates"), + allowed_claims=("cough_or_sneeze_like_events_detected", + "spot_respiratory_screen"), + forbidden_claims=("continuous_surveillance_from_spot_recording",), + ), + notes="Directive ยง26. 'Stand quietly in the house and record 30 " + "seconds.' Reports cough- or sneeze-like events and says " + "plainly that it is a spot screen."), + + # ยง20 "Experimental" for external quality. Fine cracks are a separate, + # accessory-gated claim โ€” see `REJECTED_CLAIMS`. + _cap("egg_quality", "poultry", "flock_cycle", _EXPERIMENTAL, (), + AcquisitionProtocol( + protocol="tray_photo", + modality=(Modality.RGB_PHOTO,), + # guardrail. Eggs on a photographed tray or trays. A standard + # tray holds 30; the ceiling allows a stack of them. + output=OutputSpec(type="egg_external_quality", unit="eggs", + plausible_min=0, plausible_max=1_000, step=1, + measured_claims=("count",)), + required_views=("tray",), + optional_inputs=("reference_marker", "backlight_accessory"), + reject_if=_GATE + ("eggs_overlapping", "move_closer"), + allowed_claims=("count", "shape", "obvious_dirt", + "external_discolouration", "obvious_visible_damage"), + forbidden_claims=("hairline_crack_from_ambient_photo",), + ), + output_kind="measurement", + # Answered by a hosted visual reasoner, not a file on disk. + # `providers.HostedModelProvider` says why the pair is stable + # here while the model that actually ran travels on the result. + model_provider="hosted-multimodal", model_version="1", + notes="Directive ยง20. Count, shape, dirt, discolouration and " + "obvious damage from an ordinary photograph. Hairline cracks " + "need candling and are not claimed without it."), + ] +} + + +@dataclass(frozen=True) +class RejectedClaim: + """A claim from directive ยง30 that Animap must never make. + + ยง30's own instruction is the important half: *"This does not mean the + underlying Animap capability should be removed. Change the acquisition + protocol or the claim."* So each entry names the capability that survives + and what it says instead โ€” and, where one exists, the hardware or + installation that would make the stronger claim supportable. + + `survives_as` is `None` for exactly one entry, and that is not an oversight. + """ + + claim: str + #: The directive's words for why the claim fails. + why: str + #: The registry key that carries the corrected form, if any. + survives_as: str | None + corrected_form: str + #: Directive ยง29: the accessory or installation that would change the answer. + available_with: str | None = None + + +#: Directive ยง30, in full. Eleven claims, plus `exact_poultry_weight` from ยง23. +REJECTED_CLAIMS: dict[str, RejectedClaim] = { + r.claim: r + for r in [ + RejectedClaim( + "exact_weight_from_single_photo", + "Exact cattle weight from an arbitrary freehand single RGB " + "photograph. An arbitrary photograph carries no scale.", + "cattle_weight", + "A guided 2-4 second side sweep with camera pose, and a range: " + "'Experimental weight estimate, 350-430 kg'.", + available_with="Animap reference marker, or a known-size calibration " + "object, when metric depth is unreliable (ยง22).", + ), + RejectedClaim( + "exact_house_population", + "Exact entire-house bird population from a casual partial pan.", + "poultry_count", + "Visible count, unique birds observed during a scan, and reconciled " + "flock population โ€” three distinct numbers, never conflated.", + available_with="A fixed multi-camera installation for stronger " + "whole-house counting (ยง28).", + ), + RejectedClaim( + "exact_age_from_teeth", + "Exact chronological age to the month from teeth.", + "cattle_age_dentition", + "An age band from a deterministic rule table: 'Estimated age: 3-4 " + "years. Four permanent incisors visible.'", + ), + RejectedClaim( + "fmd_diagnosis", + "Definitive foot-and-mouth diagnosis from one image.", + "cattle_skin", + "'Abnormal skin pattern. Veterinary review recommended.' A case, an " + "isolation prompt where appropriate, and room for a lab result.", + ), + RejectedClaim( + "lsd_diagnosis", + "Definitive lumpy skin disease diagnosis from one image.", + "cattle_skin", + "'Multiple raised nodular lesions visible. Veterinary review " + "recommended.'", + ), + RejectedClaim( + "fever_from_rgb", + "Fever measurement from a normal RGB image.", + "poultry_heat_stress", + "Heat-stress-associated behaviour, described from what is visible: " + "panting, wing spreading, clustering at drinkers.", + available_with="A thermal accessory or camera, for surface " + "temperature screening (ยง29).", + ), + RejectedClaim( + "pregnancy_from_rgb", + "Pregnancy diagnosis from an ordinary cattle photograph.", + None, + "Nothing. No capability in this registry claims it and none is " + "proposed: pregnancy is diagnosed by palpation or ultrasound, and " + "the corrected form is a recorded veterinary result rather than an " + "inference capability. This is the one ยง30 claim with no surviving " + "vision feature behind it.", + ), + RejectedClaim( + "exact_litter_moisture", + "Exact litter moisture percentage from RGB. Moisture has no " + "photometric signature.", + "poultry_litter", + "Litter condition: loose, caked, heavily soiled, wet-looking, the " + "share of the scanned region affected, and where it is worst.", + available_with="A moisture probe, radar, or other validated sensing " + "(ยง29).", + ), + RejectedClaim( + "dead_versus_sleeping_certainty", + "Dead-versus-sleeping certainty from one poultry image.", + "poultry_inactive_birds", + "Inactivity over a 15-30 second scan, surfaced as '5 birds need " + "review' with Dead / Sick / Resting / Fine.", + ), + RejectedClaim( + "hairline_crack_from_ambient_photo", + "Subtle hairline crack detection from arbitrary ambient egg photos.", + "egg_quality", + "Count, shape, obvious dirt, external discolouration and obvious " + "visible damage.", + available_with="Candling: controlled backlighting, or an inexpensive " + "backlight cradle (ยง20, ยง29).", + ), + RejectedClaim( + "continuous_surveillance_from_spot_recording", + "24/7 respiratory surveillance from one 30-second recording.", + "poultry_respiratory", + "'Cough/sneeze-like events detected. Spot respiratory screen only.' " + "The cattle equivalent is `cattle_respiratory`, a spot rate.", + available_with="A permanently placed microphone or camera, for " + "hourly cough rate against a house's own baseline " + "(ยง27).", + ), + RejectedClaim( + "exact_poultry_weight", + "The weight of every bird in a crowded flock from arbitrary video " + "(ยง23).", + "poultry_weight", + # **This sentence used to end "a mean, a range and a coefficient of " + # variation", and the last third of it is now rejected in its own + # right two entries below.** Left corrected rather than left + # standing: a rejection whose corrected form contains another + # rejection is how a refused claim gets back in. + "A 10-20 bird sample of held, isolated birds, reported as a mean and " + "a range. Not a coefficient of variation โ€” see " + "`coefficient_of_variation_from_vision_weights`.", + ), + # The two below are **not from ยง30**. Every other entry in this table is + # a claim the directive rejected in advance; these two are claims ยง23 + # asked for and a measurement refused. They are here because this is + # where the product's refusals live and a client renders them, and + # because ยง36 is explicit that a capability may only be called + # unavailable once the observable signal has been shown impractical โ€” + # which is a thing that can happen after the directive was written. + # ADR 0023 carries the evidence. + RejectedClaim( + "coefficient_of_variation_from_vision_weights", + "A flock's coefficient of variation computed from camera-estimated " + "bird weights. Measurement error adds variance the flock does not " + "have, so the CV is inflated by +2.40 points at the best published " + "per-bird error (7.8%) on a fifteen-bird sample against a true CV of " + "12% โ€” and the bias is systematic, so it survives the sample size: " + "+1.91 at ten birds, +2.16 at thirty. The exact-weight control " + "converges towards zero over the same sweep and this does not.", + "poultry_weight", + "A sample mean and a per-bird range over 10-20 held birds, which a " + "vision weight does support: the mean is out by 0.33% +/- 3.5 " + "against 0.18% +/- 3.2 with exact weights, because error averages " + "out of a mean and squares into a variance. The coefficient of " + "variation itself is computed from scale weights or not at all.", + available_with="A scale. ยง23's own `add_scale_weight` confirmation " + "already exists on `poultry_weight`, and over exact " + "weights the arithmetic is unbiased and lands within " + "two points of the true CV 63.5% of the time at " + "fifteen birds.", + ), + RejectedClaim( + "flock_uniformity_from_vision_weights", + "A flock uniformity percentage derived from camera-estimated bird " + "weights. It inherits the CV's inflation through " + "`2*Phi(10/CV) - 1` and reads **6.96 points low** at a true CV of " + "12% โ€” a flock at 60% uniformity reports as 53%. Across true CVs of " + "8-18% the error runs -1.46 to -13.94 points, worst where the flock " + "is most uniform. Uniformity is a decision variable a farm culls, " + "re-feeds or delays a harvest on, and the error runs in the " + "direction that makes a bad flock look acceptable.", + "poultry_weight", + "A sample mean and a per-bird range. The uniformity band itself is " + "withheld rather than widened: the failure is a bias and not a " + "width, so a visible uncertainty band would leave the centre exactly " + "where it is.", + available_with="A scale, as above. The conversion arithmetic is not " + "in doubt โ€” it matches Aviagen's twelve published " + "rows to within 0.4985 points โ€” and it is the weights " + "underneath it that fail.", + ), + ] +} + + +#: Claims the product must never make, whatever a model reports. +#: +#: Derived rather than hand-maintained, so a capability cannot declare a +#: forbidden claim that the global guard does not know about. `app/counting.py` +#: asserts every emitted observation type against this set. +FORBIDDEN_CLAIMS: frozenset[str] = frozenset(REJECTED_CLAIMS) | frozenset( + claim + for capability in REGISTRY.values() + for claim in capability.acquisition.forbidden_claims +) + + +def get(key: str) -> Capability | None: + return REGISTRY.get(key) diff --git a/app/counting.py b/app/counting.py new file mode 100644 index 0000000000000000000000000000000000000000..d55f641e6fa39588e73419c240be70ffc8ea451d --- /dev/null +++ b/app/counting.py @@ -0,0 +1,526 @@ +"""Counting, and knowing when not to. + +This is the layer that turns boxes into a claim. It exists separately from +`detection.py` because the hard part of counting livestock is not detection โ€” it +is being honest about the frames where detection stops working. + +Two capabilities share every line of it: + +**`cattle_detection`** counts cattle in a paddock. Cattle are large, separated, +and there are tens of them. A COCO detector already has a `cow` class and this +is genuinely the right tool. + +**`poultry_count`** counts birds in a frame. For a backyard flock or a yard of +fifty layers, the same detector works. For 12,000 broilers in a shed it does +not, and no threshold tuning will make it: the birds overlap, each one covers a +few hundred pixels, and non-maximum suppression merges the ones that remain. The +answer is a density head, not a better detector (ADR 0014). + +So this module measures whether it is still in the regime it was validated for, +and when it is not, **it reports no count at all**. A sample presented as a count +is the failure mode that costs the product its credibility: a farmer shown "18" +for a shed of several hundred does not conclude that the number means something +narrower than they thought. + +**How it knows.** It counts the frame three ways โ€” whole, 2x2, 3x3 โ€” and watches +what the count does (`app/tiling.py`). In a frame the detector can read, the +count stops moving, because there was nothing left to find. In a shed it never +stops, because there are always more birds behind the ones in front. That is a +measurement of what the detector is *missing*, which is the thing a saturation +guard has to know and the thing box sizes cannot tell it. + +The first version of this file guarded on box size instead, and the evaluation +set caught it: on a broiler house of a thousand birds it found a handful of large +foreground birds, concluded the frame was sparse, and published the handful as a +count. It withheld a number on three of twenty uncountable frames; the tiled test +withholds on twenty of twenty (ADR 0018). + +**The thresholds belong to the detector, not to the problem.** They were first +derived on YOLO11m and then inherited unchanged when the shipped model became +YOLOX-m, which cost 22 points of coverage for no gain in safety โ€” the grid a +frame settles at depends on how much the detector's whole-frame pass resolves, +and that is precisely what differs between detectors. Re-deriving them removed a +rule entirely and restored the coverage. If the artefact changes again, re-run +`evaluation/run.py` before trusting a number in this file. +""" + +from __future__ import annotations + +from dataclasses import dataclass +from statistics import median +from uuid import UUID, uuid4 + +from app.capabilities import Capability, FORBIDDEN_CLAIMS +from app.detectors import Detection, build +from app.media import MediaRef, MediaStore +from app.providers import ModelArtefact +from app.quality import QualityVerdict, assess +from app.schemas import ( + ConfidenceLabel, + InferenceLocation, + InferenceRequest, + InferenceResult, + Observation, + QualityCheck, +) +from app.tiling import Level, converged, pyramid, subject_count + +#: How much the count may grow when the frame is cut finer before the frame is +#: called unreadable. +#: +#: **Every number below was re-measured on the shipped YOLOX-m artefact** over +#: the 61-image set (ADR 0018). They previously came from YOLO11m, and carrying +#: them across backends was wrong: the grid at which a frame settles depends on +#: how much the detector's whole-frame pass can resolve, which is exactly what +#: differs between detectors. Re-deriving them moved coverage from 0.71 to 0.935 +#: without letting a single dense frame through. +#: +#: **The two classes overlap here, and the honest reading is that this rule does +#: not separate them on its own.** Last-refinement growth runs 0%โ€“75% on frames a +#: human could count and 7%โ€“162% on frames a human could not. At 20% it catches +#: seventeen of the twenty uncountable frames and wrongly withholds two countable +#: ones; the two dense frames that slip past it are caught by +#: `MAX_VALIDATED_COUNT`. Neither rule is sufficient alone, which is why they are +#: OR-ed rather than tuned against each other. +COUNT_GROWTH_TOLERANCE = 0.20 + +#: Median share of the frame a subject covers, below which no count is reported +#: *even after the count has settled*. A frame can settle simply because every +#: animal in it is a smudge the detector resolves the same way at every grid. +#: +#: **This rule never fires on the evaluation set, so it carries no evidence.** +#: Measured at the grid that ran: the smallest median on a countable frame is +#: 0.00077 (a hillside herd in the Turkish EฤŸribel pass) and the smallest on an +#: uncountable one is 0.00042 โ€” but every frame it would have caught was already +#: withheld by convergence. 0.0006 sits in a gap two frames wide. +#: +#: It is kept as a backstop for a frame type this set does not contain, and it is +#: labelled unexercised rather than described as if it were doing work. Do not +#: cite it as a reason the guard is safe. +SATURATION_MEDIAN_AREA_FRACTION = 0.0006 + +#: Median detection score below which the count is reported but capped at `low` +#: confidence and a recapture is asked for. Measured on the fixtures: cattle +#: 0.72, sparse hens 0.49, dense hens 0.47. Sparse and dense are only 0.02 +#: apart, so this cannot decide whether to publish a number โ€” it can only decide +#: how much to trust one. +LOW_CONFIDENCE_MEDIAN_SCORE = 0.50 + +#: Confusable detections per subject, above which the frame is *reported* as +#: class-confused. **This is an observation, not a suppression**, and it used to +#: be the latter. +#: +#: As a suppression rule it was measured net-harmful: on the 61-image set it +#: withheld exactly one count โ€” `cattle_ng_red_bororo`, three Red Bororo cattle, +#: where the detector found all three and also called some of them `horse` โ€” and +#: it caught none of the twenty uncountable frames, because convergence and the +#: count ceiling had already taken all twenty. It cost a correct answer on a +#: Nigerian frame and bought nothing. +#: +#: The signal is still worth recording: it was real when the small backend called +#: seven of thirteen hens `sheep`, and a future artefact may bring it back. So it +#: rides along as `confusable_detections` and a warning, where it informs a +#: reader without silently deleting a number. +CONFUSION_RATIO = 1.0 + +#: Minimum subjects before that ratio means anything. Below this a single +#: mislabelled animal would trip it. +CONFUSION_MIN_SUBJECTS = 3 + +#: The largest count that has been checked against ground truth. Above it the +#: service reports no number โ€” not because a larger count is necessarily wrong, +#: but because nobody has ever verified one. +#: +#: **Load-bearing, and tight.** It is what catches the two dense frames whose +#: counts settle anyway: a Swedish free-range yard that stabilises at 32 birds +#: and a Karamoja kraal that stabilises at 30, both holding many times that. The +#: largest correctly published count on the set is 20 animals against a human +#: count of 19, so the threshold sits directly on the edge of the evidence +#: rather than at a comfortable distance from it. +#: +#: Raise it by measuring more frames, not by deciding the detector is probably +#: fine up there. +MAX_VALIDATED_COUNT = 20 + +HIGH_CONFIDENCE_SCORE = 0.70 +MEDIUM_CONFIDENCE_SCORE = 0.50 + +#: Share of the frame one subject must cover before the per-animal capabilities +#: โ€” weight, body condition, skin โ€” have something they could work with. At 15% +#: of a 1080-line capture the animal is roughly 400 px across, which is the point +#: below which a girth measured off it is noise rather than a measurement. +ISOLATION_MIN_AREA_FRACTION = 0.15 + + +@dataclass(frozen=True) +class CountingProfile: + """Everything that differs between counting cattle and counting birds.""" + + capability_key: str + subject_noun: str + count_observation: str + #: COCO classes that are the subject. Only these are counted. + subject_classes: tuple[str, ...] + #: COCO classes the detector reaches for when it can no longer tell what it + #: is looking at. **Never counted** โ€” they are a signal, not a subject. + #: The shipped YOLOX-m does not confuse them on any of the 61 evaluation + #: images, so the signal is unexercised: see `CONFUSION_MIN_SUBJECTS`. + confusable_classes: tuple[str, ...] + unit: str + #: Said on every run, whatever the result. These are the claims the brief + #: forbids, stated before anyone can misread the number. + standing_warning: str + saturation_warning: str + #: Whether a single well-framed subject is what this capability is for. + reports_isolation: bool = False + + +CATTLE_DETECTION = CountingProfile( + capability_key="cattle_detection", + subject_noun="cattle", + count_observation="cattle_visible", + subject_classes=("cow",), + confusable_classes=("horse", "sheep"), + unit="animals", + standing_warning=( + "A count of the animals visible in this frame. It is not the herd size โ€” " + "animals behind others, behind cover, or out of frame are not in it." + ), + saturation_warning=( + "There are more cattle in this frame than can be counted from it. No " + "number is reported. Move closer, or frame a smaller part of the herd." + ), + reports_isolation=True, +) + +POULTRY_COUNT = CountingProfile( + capability_key="poultry_count", + subject_noun="birds", + count_observation="birds_visible", + subject_classes=("bird",), + confusable_classes=("sheep", "cat", "dog"), + unit="birds", + standing_warning=( + "A count of the birds visible in this frame. It is never the flock " + "population, and it must not be used to reconcile a house." + ), + saturation_warning=( + "This flock is denser than a detector can count. No number is reported, " + "because a detector undercounts a crowded shed by an amount nobody can " + "estimate. Frame a smaller section, closer in." + ), +) + + +class DetectionCountRunner: + """Runs one counting capability against one frame.""" + + def __init__(self, profile: CountingProfile) -> None: + self.profile = profile + + def run( + self, + *, + request: InferenceRequest, + capability: Capability, + artefact: ModelArtefact, + store: MediaStore, + request_id: UUID | None = None, + ) -> InferenceResult: + profile = self.profile + request_id = request_id or uuid4() + warnings: list[str] = [profile.standing_warning] + + # These capabilities read one frame. Saying so beats silently ignoring + # the rest, and beats pretending a count was aggregated across them. + if len(request.media_ids) > capability.frames_required: + warnings.append( + f"{len(request.media_ids)} frames were supplied; this capability " + f"reads {capability.frames_required}." + ) + + # `farm_id` and `captured_at` are what let a blob store find the object + # in one request instead of scanning a prefix (`app/media.py`). A local + # store ignores them. + image = store.open_image(MediaRef( + media_id=request.media_ids[0], + farm_id=request.farm_id, + captured_at=request.captured_at, + object_path=request.path_for(request.media_ids[0]), + )) + verdict = assess(image) + + if verdict.blocked: + return self._blocked(request, capability, artefact, verdict, warnings, request_id) + + detector = build(artefact) + levels = pyramid( + detector, image, profile.subject_classes, COUNT_GROWTH_TOLERANCE, + ) + final = levels[-1] + settled = len(levels) < 2 or converged( + levels[-2], levels[-1], profile.subject_classes, COUNT_GROWTH_TOLERANCE, + ) + + subjects = [d for d in final.detections if d.label in profile.subject_classes] + confusable = [d for d in final.detections if d.label in profile.confusable_classes] + + checks = list(verdict.checks) + checks.append(self._framing_check(subjects)) + checks.append(self._convergence_check(levels, profile, settled)) + if profile.reports_isolation: + checks.append(self._isolation_check(subjects)) + + saturation = self._withhold_reason(subjects, confusable, settled) + observations: list[Observation] = [ + # Emitted on every path, including the ones that publish no count, + # because these three are what let a threshold be re-derived later + # from stored results instead of re-run from photographs nobody kept. + Observation(type="counting_grid", value=float(final.grid), confidence=None), + Observation( + type="subjects_detected", value=float(len(subjects)), + unit=profile.unit, confidence=None, + ), + ] + if subjects: + observations.append(Observation( + type="median_subject_frame_fraction", + value=round(median(d.area_fraction for d in subjects), 5), + unit="fraction", confidence=None, + )) + observations.append(Observation( + type="largest_subject_frame_fraction", + value=round(max(d.area_fraction for d in subjects), 5), + unit="fraction", confidence=None, + )) + observations.append(Observation( + type="confusable_detections", + value=float(len(confusable)), + confidence=None, + )) + + if self._is_class_confused(subjects, confusable): + # Reported, never suppressing. Measured net-harmful as a guard; see + # `CONFUSION_RATIO`. + warnings.append( + f"The detector also labelled {len(confusable)} things in this " + f"frame as another animal, which is as many as it called " + f"{profile.subject_noun}. It may be struggling to tell what it " + f"is looking at, so treat the number as a rough indication." + ) + + if not subjects: + # **Not a count of zero.** "We could not find any birds" and "there + # are no birds" are different claims, and on a packed broiler house + # the detector produces the first while the second would be absurd. + # Emitting no count observation is what keeps the app from rendering + # a zero it would have to defend. + warnings.append( + f"No {profile.subject_noun} were found in this frame. That is not " + f"a count of zero โ€” it means nothing recognisable was detected. " + f"Capture again, closer in and better lit." + ) + confidence = ConfidenceLabel.LOW + recapture = True + elif saturation is not None: + warnings.append(profile.saturation_warning) + observations.append(Observation( + type="count_withheld", + value=saturation, + confidence=None, + )) + # `subjects_detected` above is already the floor. It is deliberately + # not named as a count anywhere on this path. + confidence = ConfidenceLabel.LOW + recapture = True + else: + mean_score = sum(d.score for d in subjects) / len(subjects) + observations.append(Observation( + type=profile.count_observation, + value=float(len(subjects)), + unit=profile.unit, + confidence=round(mean_score, 3), + )) + confidence = _label(mean_score) + recapture = False + + if median(d.score for d in subjects) < LOW_CONFIDENCE_MEDIAN_SCORE: + warnings.append( + f"The detector was unsure about most of these {profile.subject_noun}. " + f"Treat the number as a rough indication and capture again closer in." + ) + confidence = ConfidenceLabel.LOW + recapture = True + + if verdict.degraded: + # A frame the gate flagged cannot produce a high-confidence claim, + # whatever the detector's own scores say about it. + confidence = ConfidenceLabel.LOW + recapture = True + + return self._result( + request=request, + capability=capability, + artefact=artefact, + request_id=request_id, + observations=observations, + confidence=confidence, + checks=checks, + warnings=warnings, + recapture=recapture, + ) + + def _framing_check(self, subjects: list[Detection]) -> QualityCheck: + if subjects: + return QualityCheck(check="framing", passed=True) + return QualityCheck( + check="framing", passed=False, + detail=f"No {self.profile.subject_noun} found in this frame.", + ) + + def _isolation_check(self, subjects: list[Detection]) -> QualityCheck: + """Whether the per-animal capabilities could use this frame. + + `cattle_detection` runs before weight, body condition and skin, and each + of those needs one animal filling the frame. Reporting that here saves a + second capture attempt later. + """ + large = [d for d in subjects if d.area_fraction >= ISOLATION_MIN_AREA_FRACTION] + if len(large) == 1: + return QualityCheck(check="subject_isolation", passed=True) + return QualityCheck( + check="subject_isolation", passed=False, + detail=( + f"{len(subjects)} animals in frame and {len(large)} close enough to " + f"assess individually. Per-animal capabilities need one animal, " + f"filling the frame." + ), + ) + + def _convergence_check( + self, levels: list[Level], profile: CountingProfile, settled: bool + ) -> QualityCheck: + counts = [subject_count(level, profile.subject_classes) for level in levels] + trail = " โ†’ ".join( + f"{level.grid}x{level.grid}: {count}" for level, count in zip(levels, counts) + ) + if settled: + return QualityCheck(check="count_convergence", passed=True, detail=trail) + return QualityCheck( + check="count_convergence", passed=False, + detail=( + f"The count kept rising as the frame was read more finely " + f"({trail}), so animals are still hidden behind other animals." + ), + ) + + def _is_class_confused( + self, subjects: list[Detection], confusable: list[Detection] + ) -> bool: + """Whether the detector is reaching for neighbouring classes as often as + the right one. A reported signal, not a reason to withhold a count.""" + return ( + len(subjects) >= CONFUSION_MIN_SUBJECTS + and len(confusable) >= len(subjects) * CONFUSION_RATIO + ) + + def _withhold_reason( + self, + subjects: list[Detection], + confusable: list[Detection], + settled: bool, + ) -> str | None: + """Whether this frame has left the regime the detector was validated in. + + Any one signal is enough. They are OR-ed rather than AND-ed on purpose: a + guard that needs every signal to agree is a guard that goes quiet as soon + as one of them drifts, and going quiet here means publishing a number + that is wrong by an unknown factor. + + On the 61-image set the two working signals withhold every one of the + twenty uncountable frames and wrongly withhold two of the thirty-one + countable ones. The last two signals never fire; their comments say so. + """ + if not subjects: + return None + + # Seventeen of the twenty uncountable frames stop here. + if not settled: + return "count_did_not_converge" + + # The other two. A frame can settle and still be a shed: the detector + # runs out of things it can resolve, so the count stops moving for the + # wrong reason. A count larger than anything ever checked is the signal. + if len(subjects) > MAX_VALIDATED_COUNT: + return "beyond_validated_range" + + if median(d.area_fraction for d in subjects) < SATURATION_MEDIAN_AREA_FRACTION: + return "subjects_too_small" + return None + + def _blocked( + self, request, capability, artefact, verdict: QualityVerdict, warnings, request_id + ) -> InferenceResult: + failure = verdict.first_failure + warnings.append( + failure.detail if failure and failure.detail + else "The capture was not usable." + ) + return self._result( + request=request, + capability=capability, + artefact=artefact, + request_id=request_id, + observations=[], + confidence=None, + checks=list(verdict.checks), + warnings=warnings, + recapture=True, + ) + + def _result( + self, *, request, capability, artefact, request_id, observations, + confidence, checks, warnings, recapture, + ) -> InferenceResult: + forbidden = [o.type for o in observations if o.type in FORBIDDEN_CLAIMS] + if forbidden: + # Belt and braces. The registry holds these as data precisely so a + # runner can be stopped by them rather than reviewed against them. + raise ValueError(f"{capability.key} tried to emit a forbidden claim: {forbidden}") + + return InferenceResult( + request_id=request_id, + capability_key=capability.key, + model_id=artefact.model_id, + model_version=artefact.version, + inference_location=InferenceLocation.REMOTE, + subject_type=request.subject_type, + subject_id=request.subject_id, + observations=observations, + # A measurement carries no interpretation. A count of animals is a + # fact about the frame; what a farmer should do about it is not + # something this model knows (ADR 0006). + interpretations=[], + observation_confidence=confidence, + interpretation_confidence=None, + quality_checks=checks, + warnings=warnings, + recommended_recapture=recapture, + ) + + +def _label(score: float) -> ConfidenceLabel: + if score >= HIGH_CONFIDENCE_SCORE: + return ConfidenceLabel.HIGH + if score >= MEDIUM_CONFIDENCE_SCORE: + return ConfidenceLabel.MEDIUM + return ConfidenceLabel.LOW + + +#: Capabilities with an implemented adapter. A capability that has a validated +#: artefact but is absent from here returns 501 rather than a plausible result โ€” +#: see `models/README.md`. +RUNNERS: dict[str, DetectionCountRunner] = { + CATTLE_DETECTION.capability_key: DetectionCountRunner(CATTLE_DETECTION), + POULTRY_COUNT.capability_key: DetectionCountRunner(POULTRY_COUNT), +} diff --git a/app/detectors/__init__.py b/app/detectors/__init__.py new file mode 100644 index 0000000000000000000000000000000000000000..67e56a397f728152bf453f9333daba2725aca505 --- /dev/null +++ b/app/detectors/__init__.py @@ -0,0 +1,68 @@ +"""Detector backends, chosen by the model card. + +One COCO-pretrained detector serves every sparse-counting capability, because a +paddock of cattle and a yard of chickens are the same computer vision problem +with a different class index. What sits above these modules decides what a box +*means*; they only say where the boxes are. + +**Which backend runs is a field on the model card, not a code path.** That is +what made ADR 0017 cheap: moving both counting capabilities off AGPL-3.0 and on +to the Apache-2.0 backend was an edit to two reviewed JSON files, not a rewrite. + +Both backends are still registered here. Only one can answer a request โ€” +`providers.discover()` refuses a card naming the `ultralytics` runtime, whatever +that card declares about its licence โ€” and the Ultralytics adapter remains so +`evaluation/` can reproduce the comparison the decision rests on. +""" + +from __future__ import annotations + +from app.detectors.base import ( + COCO_CLASSES, + Detection, + Detector, + DetectorError, +) +from app.detectors.ultralytics_yolo import UltralyticsDetector +from app.detectors.yolox_onnx import YoloxDetector + +#: Card value โ†’ backend. A card naming anything else is refused rather than +#: defaulted, because defaulting would mean a typo silently changes which +#: licensed model produced a farmer's result. +RUNTIMES = { + "ultralytics": UltralyticsDetector, + "yolox-onnx": YoloxDetector, +} + +__all__ = [ + "COCO_CLASSES", + "Detection", + "Detector", + "DetectorError", + "RUNTIMES", + "UltralyticsDetector", + "YoloxDetector", + "build", +] + + +def build(artefact) -> Detector: + """Construct the backend a validated artefact asks for. + + `artefact` is a `providers.ModelArtefact` โ€” already checksummed against its + card. This function only decides which adapter reads it. + """ + runtime = getattr(artefact, "runtime", None) + if not runtime: + raise DetectorError( + f"{artefact.model_id} does not name a runtime on its card, so there " + f"is no way to know how to run it. Add `\"runtime\": \"โ€ฆ\"`, one of " + f"{', '.join(sorted(RUNTIMES))}." + ) + backend = RUNTIMES.get(runtime) + if backend is None: + raise DetectorError( + f"{artefact.model_id} asks for runtime {runtime!r}, which does not " + f"exist. Known runtimes: {', '.join(sorted(RUNTIMES))}." + ) + return backend(artefact.path) diff --git a/app/detectors/base.py b/app/detectors/base.py new file mode 100644 index 0000000000000000000000000000000000000000..61acea20901b83bf84b9f051a044a3be896b8d64 --- /dev/null +++ b/app/detectors/base.py @@ -0,0 +1,95 @@ +"""What every detector backend agrees to return. + +This file is why ADR 0017 was affordable, and it is not an abstract nicety. +Animap shipped an AGPL-3.0 model, decided not to, and swapped to an Apache-2.0 +one โ€” and because nothing above this file knows which backend produced a box, +that was an edit to two model cards rather than a rewrite. + +The lesson is worth keeping rather than congratulating: the interface earned its +keep on the day it was used, and it only worked because the second backend had +been kept tested while it was still hypothetical. `tests/test_alternate_runtime.py` +still runs a second artefact through the whole counting layer for exactly that +reason. + +**The thresholds above this file are not interface-neutral.** Swapping the +backend leaves `app/counting.py`'s numbers measuring the wrong detector; that +mistake was made once and cost 20 points of coverage (ADR 0018). +""" + +from __future__ import annotations + +from dataclasses import dataclass +from typing import Protocol + +from PIL import Image + +#: COCO's 80 classes, in the order every COCO-trained head emits them. Order is +#: load-bearing: `cow` is index 19 and `bird` is index 14, and a silently +#: shifted list would turn cattle into horses without failing anything. Backends +#: that carry their own name table should prefer it over this one. +COCO_CLASSES: tuple[str, ...] = ( + "person", "bicycle", "car", "motorcycle", "airplane", "bus", "train", + "truck", "boat", "traffic light", "fire hydrant", "stop sign", + "parking meter", "bench", "bird", "cat", "dog", "horse", "sheep", "cow", + "elephant", "bear", "zebra", "giraffe", "backpack", "umbrella", "handbag", + "tie", "suitcase", "frisbee", "skis", "snowboard", "sports ball", "kite", + "baseball bat", "baseball glove", "skateboard", "surfboard", + "tennis racket", "bottle", "wine glass", "cup", "fork", "knife", "spoon", + "bowl", "banana", "apple", "sandwich", "orange", "broccoli", "carrot", + "hot dog", "pizza", "donut", "cake", "chair", "couch", "potted plant", + "bed", "dining table", "toilet", "tv", "laptop", "mouse", "remote", + "keyboard", "cell phone", "microwave", "oven", "toaster", "sink", + "refrigerator", "book", "clock", "vase", "scissors", "teddy bear", + "hair drier", "toothbrush", +) + +DEFAULT_SCORE_THRESHOLD = 0.30 +DEFAULT_IOU_THRESHOLD = 0.45 + + +@dataclass(frozen=True) +class Detection: + label: str + score: float + #: Pixel coordinates in the *source* image, whatever the backend resized it to. + box: tuple[float, float, float, float] + #: Share of the frame the box covers. The counting layer uses this to decide + #: whether the subjects are close enough for a detector to be counting them + #: rather than guessing at them. + area_fraction: float + + +class DetectorError(RuntimeError): + """The artefact could not be loaded or does not have the expected shape.""" + + +class Detector(Protocol): + """One image in, boxes out. Deliberately the whole interface. + + Anything richer โ€” batching, tracking, class filtering โ€” belongs above this, + because every addition here is another thing a replacement backend has to + reimplement on the day the licence forces a swap. + """ + + def detect(self, image: Image.Image) -> list[Detection]: + ... + + +def to_detections( + rows: list[tuple[str, float, tuple[float, float, float, float]]], + frame_size: tuple[int, int], +) -> list[Detection]: + """Shared box bookkeeping, so each backend only produces label/score/box.""" + width, height = frame_size + frame_area = float(width * height) + detections = [ + Detection( + label=label, + score=score, + box=(x1, y1, x2, y2), + area_fraction=((x2 - x1) * (y2 - y1)) / frame_area, + ) + for label, score, (x1, y1, x2, y2) in rows + ] + detections.sort(key=lambda d: d.score, reverse=True) + return detections diff --git a/app/detectors/ultralytics_yolo.py b/app/detectors/ultralytics_yolo.py new file mode 100644 index 0000000000000000000000000000000000000000..2ac8bda32a017ae767eda5e1fb934f61ec8fa79c --- /dev/null +++ b/app/detectors/ultralytics_yolo.py @@ -0,0 +1,127 @@ +"""The Ultralytics YOLO backend. **AGPL-3.0.** + +**This backend cannot serve a request, and that is deliberate.** +`providers.discover()` refuses any card naming the `ultralytics` runtime, so +nothing reachable from `POST /jobs` can reach this file. It exists for +`evaluation/run.py`, which measures YOLO11m against the shipped YOLOX-m โ€” the +comparison ADR 0017's decision rests on. Deleting it would delete the evidence. + +Read `docs/adr/0017-ultralytics-licence.md` before reinstating it. The short +version: the strict text of AGPL-3.0 is narrower than ADR 0014 assumed โ€” ยง13's +source-offer duty is conditioned on modifying the program, and ยง0 puts network +interaction outside "convey" โ€” but Ultralytics publishes the position that a +closed-source SaaS using its models needs an Enterprise licence, the question is +genuinely unsettled, and YOLOX costs nothing measurable. Animap chose not to have +the argument. + +Running this requires `requirements-agpl.txt`, which the deployment never reads. +""" + +from __future__ import annotations + +import os +import threading +from functools import lru_cache +from pathlib import Path + +from PIL import Image + +from app.detectors.base import ( + DEFAULT_IOU_THRESHOLD, + DEFAULT_SCORE_THRESHOLD, + Detection, + DetectorError, + to_detections, +) + +INPUT_SIZE = 640 + +_lock = threading.Lock() + + +@lru_cache(maxsize=4) +def _model(artefact_path: str): + """Load a checkpoint once. + + `ultralytics` is imported here, not at module scope, because it pulls in + torch โ€” about 28 seconds and a gigabyte of resident memory on this machine. + Nineteen of twenty-one capabilities never touch a detector, and the refusal + path must not pay for one. + """ + try: + from ultralytics import YOLO + except ImportError as exc: # pragma: no cover - environment problem, not logic + raise DetectorError( + "The ultralytics package is not installed, so this artefact cannot " + "run. Install requirements.txt, or move the capability to the " + "yolox-onnx runtime." + ) from exc + + try: + return YOLO(artefact_path, task="detect") + except Exception as exc: + raise DetectorError(f"{Path(artefact_path).name} did not load: {exc}") from exc + + +class UltralyticsDetector: + """A checksummed Ultralytics checkpoint, ready to run.""" + + def __init__( + self, + artefact_path: Path, + score_threshold: float = DEFAULT_SCORE_THRESHOLD, + iou_threshold: float = DEFAULT_IOU_THRESHOLD, + ) -> None: + self.artefact_path = Path(artefact_path) + self.score_threshold = score_threshold + self.iou_threshold = iou_threshold + # CPU by default. A GPU that is present on the dev box and absent in + # production is a difference that shows up as a crash on deploy day. + self.device = os.environ.get("ANIMAP_INFERENCE_DEVICE", "cpu") + + with _lock: + self._model = _model(str(self.artefact_path)) + + names = getattr(self._model, "names", None) + if not names: + raise DetectorError( + f"{self.artefact_path.name} carries no class names, so its " + f"outputs cannot be mapped to a species." + ) + # The checkpoint's own table, not our COCO tuple. A model fine-tuned on + # cattle would have three classes, and reading index 19 out of it would + # be nonsense that never raised. + self.names: dict[int, str] = dict(names) + + def detect(self, image: Image.Image) -> list[Detection]: + rgb = image.convert("RGB") + try: + result = self._model.predict( + source=rgb, + conf=self.score_threshold, + iou=self.iou_threshold, + imgsz=INPUT_SIZE, + device=self.device, + verbose=False, + )[0] + except Exception as exc: + raise DetectorError(f"Prediction failed: {exc}") from exc + + boxes = result.boxes + if boxes is None or len(boxes) == 0: + return [] + + rows = [] + for class_id, score, box in zip( + boxes.cls.tolist(), boxes.conf.tolist(), boxes.xyxy.tolist() + ): + label = self.names.get(int(class_id)) + if label is None: + raise DetectorError( + f"{self.artefact_path.name} emitted class {int(class_id)}, " + f"which is not in its own name table." + ) + x1, y1, x2, y2 = (float(v) for v in box) + rows.append((label, float(score), (x1, y1, x2, y2))) + + return to_detections(rows, rgb.size) diff --git a/app/detectors/yolox_onnx.py b/app/detectors/yolox_onnx.py new file mode 100644 index 0000000000000000000000000000000000000000..5ee016d1b2efce04f42cac0a1f213bc0f6e03eee --- /dev/null +++ b/app/detectors/yolox_onnx.py @@ -0,0 +1,194 @@ +"""The YOLOX backend. Apache-2.0, and **the one that ships**. + +This was the escape hatch under ADR 0014, kept working so the AGPL-3.0 decision +stayed reversible. ADR 0017 took the exit, so it is now the only backend that can +answer a request: `cattle_detection` and `poultry_count` both run YOLOX-m through +onnxruntime, with no torch dependency. + +Keeping it tested while it was still the fallback is the reason the switch took +an afternoon rather than a quarter. An untested escape hatch is not one. + +**Nothing is downloaded here.** The artefact arrives through +`scripts/install_models.py`, is checksummed against a committed model card, and +this file is handed a path that already passed those checks. +""" + +from __future__ import annotations + +import threading +from functools import lru_cache +from pathlib import Path + +import numpy as np +import onnxruntime as ort +from PIL import Image + +from app.detectors.base import ( + COCO_CLASSES, + DEFAULT_IOU_THRESHOLD, + DEFAULT_SCORE_THRESHOLD, + Detection, + DetectorError, + to_detections, +) + +INPUT_SIZE = 640 + +#: YOLOX's own strides. The exported graph emits one flat tensor of 8,400 +#: anchor-free predictions and leaves the grid arithmetic to the caller, so +#: these have to match the export or every box lands in the wrong place. +_STRIDES = (8, 16, 32) + +#: Padding value from YOLOX's reference preprocessing. Mid-grey, so the letterbox +#: bars do not read as an object edge. +_PAD_VALUE = 114 + + +_session_lock = threading.Lock() + + +@lru_cache(maxsize=4) +def _session(artefact_path: str) -> ort.InferenceSession: + """One session per artefact, built once. + + Building a session costs about 200 ms and allocates the weights. A request + that pays that every time turns a 60 ms inference into a 260 ms one. + """ + return ort.InferenceSession(artefact_path, providers=["CPUExecutionProvider"]) + + +@lru_cache(maxsize=4) +def _grid(size: int) -> tuple[np.ndarray, np.ndarray]: + grids, strides = [], [] + for stride in _STRIDES: + cells = size // stride + xv, yv = np.meshgrid(np.arange(cells), np.arange(cells)) + grid = np.stack((xv, yv), 2).reshape(1, -1, 2) + grids.append(grid) + strides.append(np.full((1, grid.shape[1], 1), stride)) + return np.concatenate(grids, 1), np.concatenate(strides, 1) + + +def _preprocess(image: Image.Image) -> tuple[np.ndarray, float]: + """Letterbox to 640ร—640, BGR, 0โ€“255, unnormalised. + + Every part of that sentence is a YOLOX-specific choice and getting any of it + wrong degrades results quietly rather than raising. YOLOX folds the + mean/std normalisation into its first convolution, so feeding it 0โ€“1 floats + halves the input range and the model simply detects less. Channel order is + BGR because the reference implementation reads frames with OpenCV and never + converts. + """ + rgb = image.convert("RGB") + width, height = rgb.size + ratio = min(INPUT_SIZE / height, INPUT_SIZE / width) + new_w, new_h = int(width * ratio), int(height * ratio) + + resized = np.asarray(rgb.resize((new_w, new_h), Image.BILINEAR), dtype=np.uint8) + padded = np.full((INPUT_SIZE, INPUT_SIZE, 3), _PAD_VALUE, dtype=np.uint8) + padded[:new_h, :new_w] = resized + + bgr = padded[:, :, ::-1] + blob = np.ascontiguousarray(bgr.transpose(2, 0, 1)[None].astype(np.float32)) + return blob, ratio + + +def _nms(boxes: np.ndarray, scores: np.ndarray, iou_threshold: float) -> list[int]: + """Greedy per-class NMS. + + Written out rather than pulled from torchvision because the whole point of + the ONNX path is that this service does not carry a training framework into + production for one function. + """ + x1, y1, x2, y2 = boxes.T + areas = (x2 - x1) * (y2 - y1) + order = scores.argsort()[::-1] + keep: list[int] = [] + while order.size: + best = order[0] + keep.append(int(best)) + rest = order[1:] + if rest.size == 0: + break + xx1 = np.maximum(x1[best], x1[rest]) + yy1 = np.maximum(y1[best], y1[rest]) + xx2 = np.minimum(x2[best], x2[rest]) + yy2 = np.minimum(y2[best], y2[rest]) + overlap = np.maximum(0.0, xx2 - xx1) * np.maximum(0.0, yy2 - yy1) + iou = overlap / np.maximum(areas[best] + areas[rest] - overlap, 1e-9) + order = rest[iou <= iou_threshold] + return keep + + +class YoloxDetector: + """A validated YOLOX artefact, ready to run. + + Construct it from a path that `providers.load_card` has already checksummed. + It re-reads nothing and downloads nothing. + """ + + def __init__( + self, + artefact_path: Path, + score_threshold: float = DEFAULT_SCORE_THRESHOLD, + iou_threshold: float = DEFAULT_IOU_THRESHOLD, + ) -> None: + self.artefact_path = Path(artefact_path) + self.score_threshold = score_threshold + self.iou_threshold = iou_threshold + + with _session_lock: + session = _session(str(self.artefact_path)) + inputs = session.get_inputs() + if len(inputs) != 1 or list(inputs[0].shape[1:]) != [3, INPUT_SIZE, INPUT_SIZE]: + raise DetectorError( + f"{self.artefact_path.name} does not take a single " + f"3ร—{INPUT_SIZE}ร—{INPUT_SIZE} input, so it is not the artefact " + f"this adapter was written for." + ) + self._session = session + self._input_name = inputs[0].name + + def detect(self, image: Image.Image) -> list[Detection]: + width, height = image.size + blob, ratio = _preprocess(image) + + raw = self._session.run(None, {self._input_name: blob})[0] + if raw.ndim != 3 or raw.shape[2] != len(COCO_CLASSES) + 5: + raise DetectorError( + f"Expected an anchor-free head emitting {len(COCO_CLASSES) + 5} " + f"values per prediction, found {raw.shape}." + ) + + grid, strides = _grid(INPUT_SIZE) + centres = (raw[..., :2] + grid) * strides + sizes = np.exp(raw[..., 2:4]) * strides + # Scores are objectness ร— class probability, which is what makes a + # confident box of a wrong class score low rather than high. + scores = raw[..., 4:5] * raw[..., 5:] + + centres, sizes, scores = centres[0], sizes[0], scores[0] + class_ids = scores.argmax(1) + best = scores.max(1) + + above = best > self.score_threshold + if not above.any(): + return [] + centres, sizes = centres[above], sizes[above] + class_ids, best = class_ids[above], best[above] + + half = sizes / 2.0 + boxes = np.concatenate([centres - half, centres + half], axis=1) / ratio + boxes[:, 0::2] = boxes[:, 0::2].clip(0, width) + boxes[:, 1::2] = boxes[:, 1::2].clip(0, height) + + rows = [] + for class_id in np.unique(class_ids): + members = np.flatnonzero(class_ids == class_id) + for local in _nms(boxes[members], best[members], self.iou_threshold): + index = members[local] + x1, y1, x2, y2 = (float(v) for v in boxes[index]) + rows.append( + (COCO_CLASSES[int(class_id)], float(best[index]), (x1, y1, x2, y2)) + ) + return to_detections(rows, (width, height)) diff --git a/app/dispositions.py b/app/dispositions.py new file mode 100644 index 0000000000000000000000000000000000000000..5e7067e2df33c1d2d959f9cd13f29c8356581aa0 --- /dev/null +++ b/app/dispositions.py @@ -0,0 +1,1178 @@ +"""The evidence behind each capability, and what the product may say because of it. + +ADR 0021. Every capability in the registry has one of these. + +**This file used to hand down verdicts. It now hands down evidence.** The +earlier pass asked whether one model could do each job from one unconstrained +RGB photograph, recorded honestly what it found, and then wrote four verdicts โ€” +six of them `not_viable`. The founder's directive rejects that last step and +keeps the rest. So does this file: the measurements are unchanged, and the +conclusions drawn from them are not. + +**Nothing measured was deleted or edited.** `blocker`, `data_needed` and +`evidence` are carried across word for word, including the figures that argued +for the old verdict โ€” 58.1% observer agreement on body condition, 8.5% of birds +found in a commercial broiler house, kappa 0.57 on iOS and 0.38 on Android for +the same hoof model. Where a verdict changed, `superseded` records what it was +and why it moved, so the reframing is auditable rather than invisible. Deleting +those numbers would repeat the original mistake pointing the other way. + +The two figures the earlier pass corrected in the *previous* registry are also +still here, in the entries for `cattle_weight` and `cattle_bcs`: sample size was +never the binding constraint for either, and that finding survives the +reframing. It is now an argument about the shape of the claim rather than about +whether to build. + +## What each field is for + +Directive ยง37 asks for three things to be kept apart, and this file keeps them +in three fields: + +- **Measured accuracy** โ€” what a benchmark demonstrated. It lives in `blocker` + and `evidence`, with a URL for every number. +- **Product uncertainty** โ€” what Animap shows a person. `stated_uncertainty`. +- **Model confidence** โ€” what a model reports about itself. Not here; it comes + back on the result, per capability run. + +`group` is directive ยง31's activation order, which is the sequencing the founder +asked for and not a re-derivation of it. +""" + +from __future__ import annotations + +from dataclasses import dataclass, field +from enum import Enum + + +class ActivationGroup(str, Enum): + """Directive ยง31. The order the work should be attempted in. + + Not a difficulty ranking. Group A is the set that needs little or no custom + ML work; Group B needs an engineering spike with pretrained parts; Group C + needs hardware or a fixed installation. No registry capability is in Group C + โ€” the hardware-gated forms are individual claims, and they are listed in + `capabilities.REJECTED_CLAIMS` with the accessory that would unlock them. + """ + + A_IMPLEMENT_NOW = "a_implement_now" + B_ENGINEERING_SPIKE = "b_engineering_spike" + C_HARDWARE_OR_FIXED = "c_hardware_or_fixed" + + +@dataclass(frozen=True) +class Source: + claim: str + url: str + + +@dataclass(frozen=True) +class Superseded: + """The verdict this capability used to carry, and why it moved. + + Kept so that nobody has to take the reframing on trust. If the directive is + ever wrong about one of these, this is the field that makes the argument + recoverable. + """ + + verdict: str + summary: str + reframed_because: str + + +@dataclass(frozen=True) +class Disposition: + capability_key: str + group: ActivationGroup + #: One sentence, in the words the founder would use. This is the line that + #: decides whether the capability appears on a roadmap. + summary: str + #: What limits the claim. Not a list of difficulties โ€” the one constraint + #: that shapes what may be said. Carries the measured figures inline. + blocker: str + #: What the product shows a person alongside a result (ยง37). This is the + #: sentence that turns a measurement into an honest claim, and it is where + #: an unflattering number does its work rather than where it is buried. + stated_uncertainty: str + #: How much labelled data, and of what exactly. `None` when data is not what + #: is missing โ€” which, after the reframing, is most of them. + data_needed: str | None = None + evidence: tuple[Source, ...] = field(default_factory=tuple) + superseded: Superseded | None = None + + +def _d(key, group, summary, blocker, stated_uncertainty, + data_needed=None, evidence=(), superseded=None) -> Disposition: + return Disposition( + capability_key=key, group=group, summary=summary, blocker=blocker, + stated_uncertainty=stated_uncertainty, data_needed=data_needed, + evidence=tuple(Source(c, u) for c, u in evidence), + superseded=superseded, + ) + + +_A = ActivationGroup.A_IMPLEMENT_NOW +_B = ActivationGroup.B_ENGINEERING_SPIKE + + +DISPOSITIONS: dict[str, Disposition] = { + d.capability_key: d + for d in [ + # ---- the two that run ------------------------------------------- + _d("cattle_detection", _A, + "Runs today, and the directive's stack should replace the detector " + "rather than the capability. SAM 3.1 with a Grounding DINO fallback " + "(ยง6.1); YOLOX-m is what is wired up, not what this should end at.", + "Nothing blocks it. What limits it is validation: the only cattle " + "counting sets with usable licences do not exist, so its accuracy " + "rests on 29 photographs one non-expert annotator counted. The " + "Bristol cattle datasets are Non-Commercial and WAID carries no " + "licence at all, so neither may be used to check it (ADR 0018).", + "Experimental. MAE 0.62 animals and MAPE 11.1% over 16 published " + "frames of 18, labelled by one non-expert. Animals visible in one " + "frame โ€” never the herd.", + evidence=( + ("Bristol Cows2021, Non-Commercial Government Licence", + "https://data.bris.ac.uk/data/dataset/4vnrca7qw1642qlwxjadp87h7"), + ("MAE 0.62 animals, MAPE 11.1%, over 16 published frames of 18", + "evaluation/reports/yolox-m.json"), + )), + # ยง31 Group A item 4 names "poultry visible count with CountGD" as + # implement-immediately. Whole-house controlled counting is Group B and + # is the same registry key in a stronger form; the key follows the form + # that ships, and `exact_house_population` carries the rejection. + _d("poultry_count", _A, + "Build now as Experimental, on CountGD (ยง6.3). **The 8.5% figure " + "below measures YOLOX-m, and the directive's reading of it is the " + "correct one: it proves that model is wrong for the domain, not that " + "counting is impossible.** CountGD has not been tried.", + "COCO's `bird` class is wild birds. On PIO's 452 annotated frames " + "from two commercial broiler houses the detector found nothing at " + "all in 254 of them, and across the whole split it detected 8.5% of " + "the birds that were there โ€” a 92% undercount, with a median of " + "zero. The guard withholds a number on essentially every frame, " + "which is correct and is also the whole capability failing to " + "deliver in a commercial house. A poultry-trained detector fixes " + "this and PIO is the data for it: 1,035 training images, 253,429 " + "boxes, CC BY 4.0.", + "Experimental. 'Approximately 327 birds visible', and **the number " + "to show is Animap's own, not the directive's placeholder**: on the " + "61-frame evaluation set the shipped detector reaches MAPE 20.7% " + "with MAE 1.80 and a bias of -1.80 over 15 published poultry frames " + "of 16, which is a systematic undercount roughly double the 10-20% " + "band ยง6.3 suggests as an interim. ยง6.3 says to use the measured " + "benchmark once it exists; it exists. In a commercial house the " + "guard withholds a number instead, which is the honest output and " + "not a count. PIO's 452 frames are the set CountGD must be measured " + "on before any figure here is restated. Of ยง6.3's three quantities " + "the model may claim two โ€” visible count and unique birds observed " + "during a scan. **Reconciled flock population is not a vision " + "claim at all**: reconciling needs the placement count, the " + "mortality log and the previous scan, none of which is in the " + "photograph. It is `derived_claims`, the app computes it, and a " + "watchdog showed why that matters by getting a model to publish " + "'reconciled_flock_population: 3200' from a partial pan while the " + "identifier was still in `allowed_claims`.", + data_needed="None to buy. PIO is published under CC BY 4.0 and is " + "enough to fine-tune a house detector; the work is a " + "training run and an Apache-2.0 training stack, not a " + "labelling budget.", + evidence=( + ("PIO, CC BY 4.0. The archive holds 1,035 train + 452 val " + "images and 327,289 boxes; the record describes 1,435 images " + "from a commercial farm and a prototype house. Unreconciled.", + "https://doi.org/10.5281/zenodo.16686320"), + ("DFCCNet density map: MAE 12.07 at ~166 birds/frame", + "https://pmc.ncbi.nlm.nih.gov/articles/PMC10705762/"), + ("Poultry, through the whole service: coverage 0.938, MAE 1.80, " + "MAPE 20.7%, bias -1.80, over 15 published frames of 16", + "evaluation/reports/yolox-m.json"), + ), + superseded=Superseded( + "buildable_now", + "Runs today for yard flocks. **In a commercial house it does not " + "work, and the guard is what stands between that and a wrong " + "number.**", + "The verdict was right about YOLOX-m and was being read as a " + "verdict about counting. ยง6.2: a detector failing on commercial " + "broiler-house imagery proves the model is wrong for the domain. " + "ยง40.3 asks for CountGD benchmarked against these same 452 " + "frames before any capability decision, so the capability moves " + "to an engineering spike rather than resting on the old result.", + )), + + # ---- cattle, per animal ------------------------------------------- + _d("cattle_weight", _B, + "Build the spike (ยง22). A guided 2-4 second side sweep with camera " + "pose and optional ARCore depth is not 'an arbitrary single " + "photograph', which is the only thing the scale argument rules out.", + "A photograph carries no scale. Every low-error result in the " + "literature comes from a camera fixed above a race, where the " + "camera-to-animal distance is constant and the network learns the " + "scale implicitly. The one study using freehand photos at varying " + "angles reached 16.8% MAPE โ€” and only by sticking a circular sticker " + "of known size on every animal and normalising to it. `gates.py` " + "asks for 7%, which freehand capture does not reach.", + "Experimental, and the range is allowed to be wide: '350-430 kg'. " + "The nearest measured comparator is 16.8% MAPE from freehand photos " + "with metric depth and a size sticker, which on a 400 kg animal is " + "about +/-67 kg. A scale reading outranks the estimate and is what " + "verifies it. `gates.py` still asks for 7% before this reaches " + "`production`, and nothing here claims to have reached it.", + data_needed="1,200 weighbridge-paired samples is the right order and " + "the wrong lever. Published learning curve, fixed rig: " + "58 animals / 211 images gives 8-9% MAPE, 215 / 2,116 " + "gives 6.2%, 1,201 / 13,357 gives 3.0%. Pretraining on " + "another farm's herd took a 58-animal farm from 9.3% to " + "5.6%. So ~200 animals is the useful first target โ€” but " + "only once a marker board or a phone mount is part of " + "the capture.", + evidence=( + ("Freehand photos + metric depth + size sticker: MAPE 16.8%", + "https://cris.unibo.it/bitstream/11585/1027956/1/1-s2.0-S2772375525003326-main.pdf"), + ("Fixed RealSense, RGB-only: MAPE 3.79%, MAE 15.9 kg, 1,289 pairs", + "https://pmc.ncbi.nlm.nih.gov/articles/PMC10971323/"), + ("Learning curve, 58 to 1,201 cows: MAPE 9.3% to 3.0%", + "https://arxiv.org/pdf/2601.01044"), + ("Zebu heart-girth tape, 703 animals, R^2 0.98 โ€” the incumbent", + "https://pmc.ncbi.nlm.nih.gov/articles/PMC3552367/"), + ), + superseded=Superseded( + "needs_capture_change", + "Reachable to about 5-8% error with a fixed capture. **Not " + "reachable at all from a freehand photograph**, at any sample " + "size.", + "The capture does change โ€” that part was right, and it is now " + "the declared protocol rather than a reason to wait. ยง22 says " + "plainly not to disable cattle weight because no custom weight " + "model exists, and the 16.8% study is evidence that a marker " + "plus depth already works badly-but-usefully, which is what an " + "experimental band is for.", + )), + _d("cattle_bcs", _A, + "Build now as Experimental plus Human Confirmation (ยง7). **The " + "directive rejects the previous 'not viable' explicitly, and it is " + "right:** 58.1% observer agreement is an argument for a broader band " + "and a trend line, not for removing the feature.", + "The label is the ceiling. Four trained observers scoring 225 cows " + "agreed exactly 58.1% of the time, and practising vets span kappa " + "0.22 to 0.78. The gate asks for QWK 0.70 against a consensus, which " + "is roughly what two humans manage with each other โ€” so the model " + "would be measured against a ruler with the same error it is being " + "asked to beat. Worse for Nigeria: the published 5- and 9-point " + "grids are temperate-breed instruments, the scale stops resolving " + "below 2.5, and the one public dataset contains no animal thinner " + "than 3.25.", + "Experimental. A half-point band โ€” 'Body condition: 2.5-3.0' โ€” with " + "'Looks right / Correct score / Retake', and the previous band shown " + "beside it so the trend carries the meaning. The band is half a point " + "wide because four trained observers agree exactly 58.1% of the time; " + "a point score would claim resolution the ground truth does not have. " + "**Never 'BCS 2.63'.** Below 2.5 the published grids stop resolving " + "and the public dataset holds no animal thinner than 3.25, so a thin " + "animal is reported as thin rather than scored.", + data_needed="2,000 samples is the right order for the coarse-band " + "version โ€” published work reaches 81% on five bands from " + "1,270 phone images. It does nothing for the half-point " + "version, because more of a noisy label is still a noisy " + "label.", + evidence=( + ("Ferguson 1994: 58.1% exact agreement, 4 observers, n=225", + "https://www.journalofdairyscience.org/article/S0022-0302(94)77212-X/fulltext"), + ("DeLaval camera over-estimated 44% of cows below BCS 3.0, n=343", + "https://pmc.ncbi.nlm.nih.gov/articles/PMC6616514/"), + ("Depth camera, 53 cows, 4 coarse groups: 70%, and 0% on one class", + "https://academic.oup.com/tas/article/7/1/txad085/7230216"), + ("Tropical breeds need a different grid entirely", + "https://doi.org/10.1007/s11250-025-04328-4"), + ), + superseded=Superseded( + "not_viable", + "Not viable as a 1-5 score. A coarse thin / fit / fat band is " + "defensible; half-point resolution is not.", + "The measurement was right and the conclusion inverted the " + "product decision. ยง7: 'The previous conclusion that BCS is not " + "viable is rejected. Human disagreement means the system should " + "use broader uncertainty and emphasize trend.' The old sentence " + "already contained the answer โ€” a band is defensible โ€” and then " + "removed the capability anyway. The half-point *point score* " + "stays rejected, as `bcs_point_score`.", + )), + _d("cattle_identity", _A, + "Build immediately, high priority (ยง6.4). Frozen MegaDescriptor and " + "DINOv3 embeddings with nearest-neighbour retrieval, benchmarked " + "against each other. Do not wait for a custom muzzle model.", + "Nothing blocks the muzzle route: two CC BY 4.0 datasets are public " + "(268 and 459 animals) and four or five images per animal reaches " + ">90% closed-set accuracy. Two things need saying anyway. Accuracy " + "falls from 96% to 72% when a fifth of the animals presented are not " + "enrolled, which is the only regime a pastoralist herd is ever in. " + "And coat-pattern re-identification โ€” the method with the best public " + "data โ€” works on Holstein markings and has nothing to key on with " + "White Fulani, Sokoto Gudali or N'Dama.", + "Experimental. 'This looks like Kofi', with Confirm / Not Kofi / " + "Choose another animal / Register new animal. The open-set number is " + "why confirmation is mandatory rather than a courtesy: closed-set " + "accuracy of 96.3% falls to 72.5% once a fifth of the animals shown " + "are not enrolled, and a working herd is always in that regime. An " + "unconfirmed match is never written as an identity.", + data_needed="4-5 muzzle photographs per animal at enrolment. No " + "external labelling budget; the enrolment is the label.", + evidence=( + ("Beef cattle muzzle, 4,923 images, 268 animals, CC BY 4.0", + "https://zenodo.org/records/6324361"), + ("Over four images per animal gives >90% accuracy", + "https://pmc.ncbi.nlm.nih.gov/articles/PMC9179917/"), + ("Open-set drops 96.3% to 72.5% with 20% unseen animals", + "https://doi.org/10.3168/jds.2024-26069"), + )), + # ---- added at the coordinator's instruction, ยง31 Group A ------------ + # These four sections named capabilities the registry never held. Their + # absence was a gap rather than a decision, and it had a second cost: + # three claims the directive rejects had no capability to attach to. + _d("cattle_breed", _A, + "Build now as Human Confirmation (ยง6.5). A hosted multimodal model " + "plus DINOv3 reference retrieval, and a crossbred animal is allowed " + "to stay crossbred.", + "No measurement was made for this capability, by the earlier pass or " + "by this one, and none should be implied. What is known is " + "structural: Nigerian herds are substantially crossbred, the " + "reference photographs that retrieval would key on are the same " + "temperate-breed-heavy sets that made `cattle_bcs` unreliable below " + "2.5, and the phenotypes that matter here โ€” White Fulani, Sokoto " + "Gudali, N'Dama โ€” are the ones with the least public imagery. The " + "constraint is therefore the claim rather than the model: a breed " + "named confidently on a crossbred animal is wrong in a way the user " + "cannot easily correct, because it looks like knowledge.", + "Experimental. 'Likely White Fulani', or 'White-Fulani-like " + "phenotype' where the animal does not sit cleanly in one breed, and " + "the user confirms either way. **There is no accuracy figure for " + "this and the product must not imply one.** `crossbred_or_uncertain` " + "is a first-class answer rather than a failure, per ยง6.5's " + "instruction not to force a breed.", + evidence=()), + _d("cattle_sex", _A, + "Build now as Human Confirmation (ยง6.6), and never let it hold up " + "registering an animal.", + "No measurement was made and none is claimed. The visible cue is " + "external genitalia and, in some breeds, conformation and horn " + "shape, none of which is reliably in frame in a side photograph " + "taken at working distance โ€” which is why ยง6.6 asks for confirmation " + "rather than a decision, and why the honest failure mode is " + "'not determinable from this view' rather than a guess. The binding " + "constraint is a workflow one: an animal that cannot be registered " + "because a model would not commit is a worse outcome than an " + "unrecorded sex.", + "Experimental. 'Likely male', confirmed by the user. **It never " + "blocks registration** โ€” declared as `never_blocks` on the " + "acquisition protocol rather than left to a screen โ€” and " + "'not determinable from this view' is an allowed answer.", + evidence=()), + _d("cattle_feces", _A, + "Build now as Experimental (ยง12). SAM 3.1 segmentation, multimodal " + "visual reasoning and DINOv3 reference retrieval, reporting " + "appearance only.", + "The poultry evidence does not transfer, and that is the whole " + "point. The two CC BY 4.0 dropping datasets under `poultry_fecal` " + "are chicken droppings photographed in Tanzania; no comparable " + "public set of cattle feces was found by the earlier pass or this " + "one. So there is no retrieval index to build against yet and no " + "accuracy figure to quote. What is defensible without one is the " + "list ยง12 gives โ€” normal, loose, watery, visible blood, visible " + "mucus, unusual colour โ€” because each is an appearance a photograph " + "carries, and blood and mucus are worth surfacing on their own " + "whatever caused them.", + "Experimental, and appearance only: normal / loose / watery / " + "visible blood / visible mucus / unusual colour. **No accuracy " + "figure exists for cattle feces and none is implied.** ยง12 is " + "explicit that a strong disease claim must not be made from cattle " + "feces alone, so blood or mucus escalates to a vet rather than " + "naming a parasite or an infection.", + evidence=()), + _d("cattle_age_dentition", _A, + "Build now as an age *band* (ยง8). The teeth carry a band and the old " + "entry proved it โ€” five states across five years is exactly what an " + "age band is made of.", + "Incisor eruption resolves five states across five years and nothing " + "finer, with 50% transition points around 23, 30, 37 and 42 months โ€” " + "and males erupt about three weeks earlier, dairy types two to three " + "months earlier, and Bos indicus crosses keep their incisors longer. " + "Past five years you are reading wear, which is diet-confounded. A " + "published claim of 0.06 years RMSE from a tooth photograph is not " + "physically achievable and should be read as label leakage. It also " + "needs the animal restrained and its mouth opened.", + "Experimental. 'Estimated age: 3-4 years. Four permanent incisors " + "visible. Medium confidence.' The band is a year wide below five " + "years because the 50% eruption points sit around 23, 30, 37 and 42 " + "months and shift with sex and breed type; past five years the signal " + "is wear, which is diet-confounded, so the band widens to 'over five " + "years' rather than narrowing. A known birth date always wins. " + "**Never a month.**", + evidence=( + ("Whiting et al., ~60,000 cattle: eruption timing and its spread", + "https://doi.org/10.1017/S1751731112001656"), + ("Sheep analogue, the honest benchmark: 540 photos, 3 classes, 96.9%", + "https://doi.org/10.1080/09540091.2025.2506456"), + ), + superseded=Superseded( + "not_viable", + "Not viable as an age. The teeth do not carry one.", + "True of a chronological age and false of the capability. ยง8: " + "'Do not attempt exact chronological age. Age from dentition " + "should be an age-band feature.' The five resolvable states and " + "their transition months are the rule table ยง8 asks for, so the " + "old blocker is now the specification. `exact_age_from_teeth` " + "stays rejected.", + )), + _d("cattle_gait", _B, + "Build the spike (ยง24). DeepLabCut SuperAnimal-Quadruped is zero-shot " + "on quadruped pose, and a 5-10 metre side-on walk is a capture " + "protocol a person can follow without a lane.", + "Every published method films one animal at a time walking single " + "file past a camera on a fixed mount โ€” 2 m up and 4.5 m back, or 6 m " + "back down a 44 m walkway. The single-farm accuracies of 95-99% do " + "not survive contact with a second farm: two at-scale validations of " + "a commercial system across 3 and 7 farms reached kappa 0.23-0.41, " + "with 40% sensitivity against painful-lesion ground truth. Human " + "observers reach kappa 0.28-0.84 with each other.", + "Experimental, and screening only: 'Possible gait asymmetry.' The " + "cross-farm numbers are why there is no score โ€” a commercial system " + "validated across 3 and 7 farms reached kappa 0.23-0.41 at 40% " + "sensitivity, and human observers only reach kappa 0.28-0.84 with " + "each other. A missed asymmetry is expected at that sensitivity and " + "the wording must not imply a clear result. **Never 'lameness score " + "3 caused by left rear hoof disease'.**", + evidence=( + ("Fixed ZED camera, 2 m up, 4.5 m from the passageway", + "https://arxiv.org/pdf/2401.05202"), + ("244 articles, 25 scoring systems: inter-rater kappa 0.28-0.84", + "https://doi.org/10.1016/j.prevetmed.2014.06.006"), + ), + superseded=Superseded( + "needs_capture_change", + "Needs a walking lane and a fixed side-on camera. Not a phone " + "feature.", + "ยง24: 'Do not require a permanently mounted camera in v0.' The " + "fixed rigs in the literature exist to support a *score*, and " + "the v0 output is a screen โ€” 'possible gait asymmetry' โ€” which " + "the cross-farm kappa figures argue for rather than against. The " + "fixed-camera version remains the stronger form.", + )), + _d("cattle_ticks", _B, + "Build the spike (ยง25). Guided close-ups of four regions with phone " + "zoom, tiled high-resolution inference, SAM 3.1 exemplar prompting " + "and a multimodal verifier โ€” none of which the thermal study used.", + "Every tick computer-vision paper is laboratory work on detached " + "material โ€” eggs in a dish, larval mortality in a container. The best " + "attempt on live animals used thermal imaging and correlated with " + "manual counts at 0.62 on the hind end and 0.29 on the neck, because " + "ticks could not be told apart from hair. The standard phenotype " + "counts only adult females of 4.5 mm or more on one whole side of the " + "animal, and the sites that matter โ€” dewlap, escutcheon, udder, inner " + "thigh โ€” are folded, shaded, and not in any photograph a farmer takes.", + "Experimental. '17 probable ticks visible across sampled regions', " + "with every detection shown for confirmation or removal, plus a " + "none / low / moderate / high band. This is **sampled burden across " + "four regions, not a total-body count** โ€” the standard phenotype " + "counts adult females of 4.5 mm or more over one whole side, and the " + "folded sites are not photographable.\n\n" + "**There is an accuracy figure now, and it supports a confirmation " + "queue and nothing else.** An earlier version of this paragraph said " + "no figure existed and the product must not imply one; the first half " + "has stopped being true and the second half is why the first half " + "mattered. On 24 composite frames holding 322 real ticks pasted onto " + "real Nigerian and East African cattle, scored with an exemplar bank " + "sharing no photograph with the pasted ticks: **precision 0.798, " + "recall 0.258**, F1 0.390, count MAE 9.2, bias -9.08. Four detections " + "in five are ticks and roughly three ticks in four are missed, so the " + "number under-reports โ€” the safer direction for a screen a farmer " + "confirms, and still not a number to subtract from. **The holdout is " + "the load-bearing word**: with a bank that had seen the paste sources " + "recall reads 0.578, and more than half of that was near-duplicate " + "retrieval of pixels the method had been shown in advance. On twelve " + "real photographs it lands inside the annotator's interval on **0 of " + "12, and 0 of 5 cattle**, returning zero detections on a neck holding " + "70-160 ticks. Composite figures are an upper bound: the ticks are " + "copies of two specimens, none of them overlap, and all of them were " + "pasted on smooth pixels inside a detected cow.", + evidence=( + ("Thermal on live cattle: r 0.619 hind end, 0.285 neck", + "https://www.embrapa.br/busca-de-publicacoes/-/publicacao/1069406/"), + ("The standard count excludes anything under 4.5 mm", + "https://doi.org/10.3389/fimmu.2021.620847"), + ("Holdout exemplar bank, 24 composite frames, 322 real ticks: " + "precision 0.798, recall 0.258, F1 0.390, count MAE 9.2. Inside " + "the annotator's interval on 0 of 12 field photographs and 0 of " + "5 cattle", + "experiments/cattle_ticks/metrics.json"), + ), + superseded=Superseded( + "not_viable", + "Do not build. There is no prior art on live animals and the " + "geometry is against it.", + "ยง36 sets the bar for 'do not build': the observable signal must " + "be shown absent or impractical, and a failed first model is not " + "sufficient. One thermal study at r 0.62 and 0.29 is one model, " + "not the signal. ยง25 is explicit โ€” 'Do not remove' โ€” and the " + "geometry objection is answered by guiding the capture to the " + "four regions instead of hoping they appear. The total-body " + "count stays rejected as `total_body_tick_count`.", + )), + _d("cattle_wound", _A, + "Build now as Experimental (ยง9). SAM 3.1 for the mask, a multimodal " + "model for the observations, and OpenCV with a reference marker for " + "an area when one is in frame.", + "There is no public wound dataset. What exists is the lumpy-skin " + "image set, and it is not a wound set. Anything shipped here must " + "refuse to name a condition, for the reason recorded under " + "`cattle_skin`.", + "Experimental, and descriptive: 'Open wound visible. Moderate " + "surrounding swelling. No obvious visible discharge.' With a " + "reference marker in frame, 'Approximate visible area: 12-16 cmยฒ'. " + "There is no public wound dataset, so there is no accuracy figure and " + "the product must not imply one โ€” the value is the follow-up " + "comparison, not the first reading. **Never a cause.**", + data_needed="No public dataset found. A triage classifier needs on " + "the order of 1,000 photographs of intact and injured " + "skin from working Nigerian herds, labelled by a " + "veterinarian as 'needs attention' or not โ€” a binary " + "referral label, not a diagnosis.", + evidence=(), + superseded=Superseded( + "needs_labelled_data", + "Buildable as triage โ€” 'something here needs a vet' โ€” and only " + "that.", + "The verdict was nearly right and priced a classifier nobody " + "needs yet. ยง9 asks for visible observations from a multimodal " + "model plus a SAM mask, which needs no training set at all; the " + "1,000-photograph budget is what a *trained* triage classifier " + "would cost later.", + )), + _d("cattle_skin", _A, + "Build now as visual screening with mandatory escalation (ยง10). The " + "legal exposure changes the wording and the workflow. It does not " + "change whether a nodule is visible in a photograph.", + "Lumpy skin disease and foot-and-mouth are notifiable. Nigeria's " + "Animal Diseases (Control) Act 1988 s.8(1) obliges the person in " + "charge of an animal *suspected* to be infected to give notice and " + "isolate it, and s.8(4) lets a veterinary officer order slaughter; " + "the First Schedule lists FMD, lumpy skin disease **and " + "streptothricosis**, which is the commonest look-alike in West " + "Africa. So a false positive creates a legal duty that can end with a " + "healthy animal destroyed. The evidence underneath the published " + "models does not support that risk: every high accuracy traces to one " + "Mendeley set of 324 lumpy and 700 normal images with no stated " + "collection method and no veterinary or PCR confirmation, and the " + "same model drops from 96% to 85% as soon as the negatives include " + "other skin diseases. **The statute should be confirmed by counsel " + "before anyone relies on this paragraph.**", + "Experimental, described not named: 'Multiple raised nodular lesions " + "visible. Abnormal skin pattern. Veterinary review recommended.' The " + "96%-to-85% drop once other skin diseases enter the negatives is why " + "no disease is named โ€” streptothricosis is the commonest West African " + "look-alike and is on the same notifiable schedule. **Never 'Lumpy " + "skin disease confirmed'**, because under s.8(1) a suspicion creates " + "a legal duty and under s.8(4) that can end with a healthy animal " + "destroyed.", + evidence=( + ("The whole literature's dataset: 324 lumpy, 700 normal, no provenance", + "https://data.mendeley.com/datasets/w36hpf86j2/1"), + ("96% falls to 85.45% once other skin diseases are in the negatives", + "https://pmc.ncbi.nlm.nih.gov/articles/PMC11512320/"), + ), + superseded=Superseded( + "not_viable", + "The model is easy and the claim is the liability. Not viable as " + "anything that names a disease.", + "The two halves of that sentence point opposite ways and the " + "second one won. ยง10: 'Do not remove this feature because certain " + "skin diseases are legally important. The legal/regulatory issue " + "changes the workflow and wording, not the computer-vision " + "feasibility.' The naming stays rejected โ€” `lsd_diagnosis` and " + "`fmd_diagnosis` โ€” and the escalation workflow is now part of the " + "acquisition protocol.", + )), + _d("cattle_hoof", _A, + "Build now with guided presentation (ยง11). 'Lift and clean the hoof " + "before photographing' is a capture instruction, and the sole not " + "being visible while the animal stands is what it exists to fix.", + "Sole ulcer and white line disease are on the sole, invisible while " + "the animal is standing, and no public dataset covers them at all. " + "Digital dermatitis is photographable, and the honest number for it " + "is a 2024 field trial: mAP 0.95 offline, then kappa 0.57 on iOS and " + "0.38 on Android with the same model โ€” the phone changed the result " + "more than the lesion did. Capture protocol is a restrained animal, " + "feet sprayed with water, camera 35 cm perpendicular.", + "Experimental. Visible crack, lesion, swelling, erosion, overgrowth, " + "or normal โ€” never a named condition. The field numbers are the " + "reason: the same model that scored mAP 0.95 offline reached kappa " + "0.57 on iOS and 0.38 on Android, so the device changes the answer " + "more than the lesion does and the result must read as an observation " + "a vet acts on. Sole ulcer and white line disease have no public " + "dataset at all and are not claimed.", + evidence=( + ("mAP 0.95 offline, kappa 0.57 / 0.38 in the field on two phones", + "https://pmc.ncbi.nlm.nih.gov/articles/PMC11829201/"), + ), + superseded=Superseded( + "needs_capture_change", + "Needs the foot lifted, washed and held. Not reachable on a herd " + "with no crush.", + "ยง11: 'The fact that the sole is not visible while the animal is " + "standing is a capture requirement, not a reason to remove the " + "capability.' The lifting and washing are now `reject_if` " + "conditions and a capture prompt. A herd with no crush cannot " + "use this, which is a coverage limit rather than a feasibility " + "one.", + )), + _d("cattle_respiratory", _A, + "Build now as a respiratory *rate*, from video (ยง14). This is " + "segmentation, optical flow and an FFT โ€” signal processing, with no " + "training and no audio. **The capability was re-scoped:** the cough " + "evidence below belongs to continuous surveillance, which ยง27 makes a " + "separate, fixed-microphone capability.", + "The disease signal is a multi-day rise in cough rate against that " + "house's own baseline, read off a trend line. A single recording " + "cannot produce it. The best published cattle result โ€” 62 calves, " + "205 minutes of labelled audio, 385 coughs โ€” is 50.3% sensitive at " + "99.2% specificity, and the authors say plainly that algorithms do " + "not transfer between set-ups. Room acoustics alone swing precision " + "from over 80% to 54% between compartments of one building.", + "Experimental. 'Estimated respiratory rate: 44-50 breaths/min', with " + "the capture quality shown beside it. A range rather than a number " + "because the measurement is a peak-detection over a 30-60 second " + "window and both the animal and the camera move. **This is a spot " + "measurement and says so.** The 50.3%-sensitive cough figure applies " + "to the continuous form and is not this capability's number.", + data_needed="No public event-labelled cattle cough dataset exists. " + "Building one means continuous audio from a fixed " + "microphone with every cough time-stamped by a person: " + "the reference study needed 205 minutes to collect 385 " + "events.", + evidence=( + ("62 calves, 385 labelled coughs: SE 50.3%, SP 99.2%", + "https://t-stor.teagasc.ie/handle/11019/1751"), + ), + superseded=Superseded( + "needs_capture_change", + "Needs a microphone left in the house, not thirty seconds from a " + "phone.", + "True, and about a different capability. ยง14 asks for " + "respiratory rate from 30-60 seconds of flank video; ยง27 keeps " + "continuous cough monitoring as a separate fixed-microphone mode " + "and says the requirement for it must not block phone-based spot " + "screening. The old entry measured the second and cancelled the " + "first. `continuous_surveillance_from_spot_recording` stays " + "rejected.", + )), + + # ---- poultry ------------------------------------------------------- + _d("poultry_respiratory", _B, + "Build the spike as a spot screen (ยง26). SAM Audio to pull bird " + "sound out of fan and machinery noise, then audio embeddings or " + "hosted multimodal audio reasoning over 30 seconds.", + "Same structural problem โ€” the signal is a rate against a baseline โ€” " + "but sneezes are more frequent than cattle coughs and the reference " + "study reached 66.7% sensitivity at 88.4% precision on 51 chickens. " + "The published labels are bird-level rather than event-level, which " + "is what a detector needs.\n\n" + "**Animap's own detector has now been measured and it is below " + "chance.** Over 6,346 clips from two CC BY 4.0 commercial-farm " + "datasets, the shipped spectral-flux path in `app/adapters/audio/` " + "scores **AUC 0.4141** on sick against healthy, where chance is 0.5. " + "The direction says why: healthy clips average **6.90 events a " + "minute** and sick clips **1.16**. A healthy poultry house is a noisy " + "one โ€” birds move, peck and scratch, and every one of those is a " + "broadband transient an onset detector is built to find โ€” while a " + "sick flock is lethargic and quiet. The detector measures activity " + "and activity runs the wrong way. No threshold repairs it: `ONSET_K` " + "moves the event rate and the ordering between the classes is what is " + "wrong. Two caveats travel with the number and neither rescues it โ€” " + "74% of the sick-by-healthy pairs are ties, so it is a mostly inert " + "detector rather than a strongly anti-correlated one; and on the 141 " + "clips long enough to meet ยง26's own 30-second protocol it is 0.3253, " + "which agrees with the larger result.", + "**No number.** 'Cough/sneeze-like events detected. Spot respiratory " + "screen only.' โ€” ยง26's own wording, which carries no digit, and the " + "registry now agrees: the event count is declared as carrying no " + "quantity, so a figure has nowhere to be published from. The reason " + "is Animap's own measurement rather than caution: the only " + "implementation scores AUC 0.4141 against a chance line of 0.5 and " + "fires six times more often on healthy flocks than on sick ones, so " + "the count is not an imprecise reading of the events but an inverted " + "one, and a farm cannot tell those apart. The external comparator is " + "unchanged and is the better of the two โ€” 66.7% sensitivity at 88.4% " + "precision on 51 chickens, roughly a third of events missed โ€” so a " + "quiet result is not a clear result even where the method works. " + "**The capability survives, because the signal does**: frozen CLAP " + "embeddings on the identical clips, on a split where no recording " + "node appears on both sides, reach 0.5651 against a 0.5182 majority " + "baseline, and 0.6032 against 0.3621 over three classes. That is a " + "method failure, not a capability failure, which is exactly the " + "distinction ยง36 turns on. **Never presented as continuous " + "surveillance.**", + data_needed="An event-labelled set: the reference work annotated 763 " + "sneezes across 480 minutes from 51 birds. Bowen " + "University's 346-file Nigerian set (139 healthy, 121 " + "unhealthy, 86 noise, CC BY 4.0) is a starting point but " + "carries bird-level labels only.", + evidence=( + ("Nigerian poultry vocalisation set, 346 files, CC BY 4.0", + "https://data.mendeley.com/datasets/zp4nf2dxbh/1"), + ("51 chickens, 763 sneezes: SE 66.7%, precision 88.4%", + "https://doi.org/10.1016/j.compag.2018.12.028"), + ), + superseded=Superseded( + "needs_labelled_data", + "Closer than the cattle version, and there is a Nigerian dataset " + "to start from.", + "Not overturned โ€” re-sequenced. ยง26 asks for the zero-training " + "audio stack to be tried before a labelling budget is committed, " + "so this is a spike now and a data question afterwards. The " + "event-labelled set is still what a trained detector would need.", + )), + _d("poultry_fecal", _A, + "One of the strongest early capabilities (ยง13). DINOv3 embeddings " + "over the existing labelled sets, nearest-neighbour retrieval, then a " + "structured multimodal review โ€” no training run in the path.", + "Two CC BY 4.0 datasets exist, collected in Tanzania on ordinary " + "smartphones, one of them PCR-validated. The published 98% is carried " + "by the three common classes: Newcastle is 376 of 6,812 images, and a " + "model that never predicts it still scores 94.5%. Reported Newcastle " + "recall is 62.7%. Newcastle is also notifiable, so the same " + "constraint as `cattle_skin` applies to what may be said on screen.", + "Experimental, flock-level, over 4-6 samples: 'Elevated GI-health " + "risk', not a diagnosis from one dropping. **The published 98% is not " + "the number to quote** โ€” a model that never predicts Newcastle still " + "scores 94.5% on that set, and reported Newcastle recall is 62.7%, so " + "more than a third of Newcastle cases would be missed. Newcastle is " + "notifiable, so it is named only as a qualified visual pattern and " + "escalates to a vet. **Coccidiosis may be named as a visual pattern " + "and Newcastle may not, and the reason is recorded rather than " + "assumed**: ยง13 names the coccidiosis wording and names nothing " + "comparable for Newcastle; coccidiosis is one of the three common " + "classes carrying that set's headline accuracy while Newcastle is " + "376 of 6,812 at 62.7% recall; and Newcastle is notifiable in " + "Nigeria while coccidiosis is not. Neither may be *diagnosed* โ€” " + "`coccidiosis_diagnosis` is forbidden alongside " + "`newcastle_diagnosis`, so the real line is between an appearance " + "and a conclusion, not between two diseases.", + data_needed="Newcastle images specifically. The class is 5.5% of the " + "largest public set; a usable screen needs it at " + "something like 20%, which is roughly 1,000 more " + "PCR-confirmed Newcastle droppings.", + evidence=( + ("Machuve et al., 6,812 farm-labelled images, CC BY 4.0", + "https://zenodo.org/records/4628934"), + ("1,255 PCR-validated images, CC BY 4.0", + "https://zenodo.org/records/5801834"), + ("Nigerian set, 14,618 images, binary labels, CC BY 4.0", + "https://data.mendeley.com/datasets/8pnbzpt2k9/1"), + ), + superseded=Superseded( + "needs_labelled_data", + "The best-evidenced poultry feature, and its headline accuracy is " + "not the number to quote.", + "Both halves survive; only the sequencing moved. ยง13 specifies a " + "retrieval pipeline over the datasets that already exist, which " + "needs no new labels to start. The 1,000 PCR-confirmed Newcastle " + "droppings remain what a Newcastle *claim* would cost, and that " + "claim stays rejected until then.", + )), + _d("poultry_inactive_birds", _A, + "Build now as Experimental plus Human Confirmation (ยง15). Use time " + "rather than a single frame: track per-bird motion over a 15-30 " + "second section scan and surface candidates.", + "A dead bird and a sleeping bird are the same photograph. The methods " + "that work add thermal imaging or track the bird over time; the " + "single-frame result on real houses is mAP@0.5 of 80.1% at 79% " + "recall from 2,299 RGB-infrared pairs. The 98% paper composited 19 " + "photographs of dead chickens into 223 empty backgrounds, which " + "measures compositing.", + "Experimental. '5 birds need review', each answered Dead / Sick / " + "Resting / Fine. The honest single-frame comparator is mAP@0.5 of " + "80.1% at 79% recall on real houses โ€” about one in five missed โ€” so " + "the output is a review queue and not a mortality count. The 98% " + "figure in the literature composited 19 photographs into 223 empty " + "backgrounds and measures compositing; it is not quoted. **Never " + "dead-versus-sleeping certainty from one image.**", + data_needed="~2,300 paired frames with roughly 8,000 boxed instances " + "is what the credible study used. On RGB alone, expect " + "less.", + evidence=( + ("2,299 real RGB-IR pairs: mAP@0.5 80.1%, recall 79.0%", + "https://pmc.ncbi.nlm.nih.gov/articles/PMC13072331/"), + ), + superseded=Superseded( + "needs_labelled_data", + "Buildable, and the honest accuracy is about 80%, not the 98% in " + "the literature.", + "ยง15 replaces the classifier with tracking plus an inactivity " + "threshold, which needs no labelled set: the output is a review " + "queue, and the farmer's four answers are the labels. The 2,300 " + "paired frames are what a trained dead-bird detector would need " + "later.", + )), + _d("poultry_weight", _B, + "Build the spike as a sample workflow (ยง23). Ten to twenty held, " + "isolated birds with SAM masks and ARCore or VGGT geometry against " + "published allometric relationships. Not every bird in a flock.", + "The only figure collected on a commercial flock with a held-out set " + "of birds is 7.8% mean relative error, from a Kinect fixed above a " + "house of 48,000. The lab studies reporting R^2 0.98 used 30 birds " + "photographed 2,520 times, and the same group reported 21.5% MAPE on " + "the same animals in a second paper. One recent study reaching 7.27% " + "states in its own text that birds appear in both its training and " + "test sets.", + "Experimental, and a sample statistic: **a mean and a range** over " + "10-20 birds, each estimate shown as a range. The credible " + "comparator is 7.8% mean relative error from a *fixed overhead* " + "Kinect; the lab figures of R^2 0.98 come from 30 birds photographed " + "2,520 times, and the same group reported 21.5% MAPE on those " + "animals elsewhere, so a handheld phone should be expected nearer the " + "worse end. A scale reading verifies it. **Never every bird in a " + "crowded flock from arbitrary video.**\n\n" + "**ยง23 asks for four statistics and only two of them survive this " + "error.** Animap's own simulation, at that same 7.8%: the sample " + "mean is out by 0.33% +/- 3.5 against 0.18% +/- 3.2 with exact " + "weights, which is no difference worth reporting โ€” measurement error " + "averages out of a mean. It squares into a variance, so the " + "coefficient of variation is inflated by +2.40 points at fifteen " + "birds and the derived uniformity reads 6.96 points low, neither of " + "which improves with sample size. Both are refused: see " + "`poultry_uniformity` and ADR 0023. **The range's own inflation was " + "not measured** โ€” the experiment covered the mean and the CV โ€” so " + "keeping it is a judgement about the shape of the claim (a band in " + "kilograms, shown as a band, per ยง38) and not a measurement of it.", + data_needed="Paired image-and-scale readings under a fixed camera. " + "The credible reference used ~13,000 annotated frames " + "against 83 individually weighed birds.", + evidence=( + ("Commercial house, held-out birds: 7.8% mean relative error", + "https://doi.org/10.1016/j.compag.2016.02.011"), + ("Broiler weight set, CDLA-Permissive-1.0", + "https://www.kaggle.com/datasets/lucasheilbuthh/inferring-broiler-chicken-weight"), + ), + superseded=Superseded( + "needs_capture_change", + "Same shape as cattle weight: a fixed overhead camera works, a " + "handheld phone does not.", + "ยง23 concedes the hard case and keeps the easy one: 'Do not " + "attempt weight of every bird in a crowded flock from arbitrary " + "video. Attempt a representative sample-bird workflow.' A held " + "bird against a clear background is a different geometry problem " + "from a bird in a crowd, and the fixed-camera figures do not " + "bound it.", + )), + # **The one capability in this file whose claim was refused rather than + # corrected**, and the one whose disposition moved on Animap's own + # measurement instead of on somebody else's paper. ADR 0023. + _d("poultry_uniformity", _B, + "**Refused as a vision claim (ยง23, ยง36).** The arithmetic was never " + "in doubt and is not what failed โ€” the weights underneath it are. " + "`unsupported_claim`, not `experimental` with an unmet dependency: " + "the input it was waiting for is the input that makes the answer " + "wrong, so waiting was never going to end well.", + "Uniformity is a coefficient of variation over individually weighed " + "birds, and Aviagen's own protocol says to weigh 1% or 100 birds, " + "whichever is larger โ€” 65 birds at CV 8%, 140 at CV 12%, for +/-2% " + "accuracy. That framing survives and is not the blocker. **The " + "blocker is that a measured weight carries its own error into the " + "variance.** `experiments/poultry_weight/` simulated 200 draws at " + "each of six sample sizes and six true CVs, at the best published " + "per-bird error for this method โ€” 7.8% mean relative error, " + "Mortensen et al. 2016, from a *fixed* Kinect depth camera over a " + "commercial house of 48,000, which is a floor on a phone's error and " + "not an estimate of it. Against a commercial flock's own 11-18% " + "spread (Vasdal et al. 2019, 45 Ross 308 flocks, mean 13%), a " + "fifteen-bird sample at a true CV of 12% returns an estimated CV of " + "14.38% โ€” **a bias of +2.40 points**, within two points of the truth " + "40.0% of the time. **More birds does not fix it**: +1.91 at ten, " + "+2.40 at fifteen, +2.24 at twenty, +2.16 at thirty, while the " + "exact-weight control converges towards zero over the same sweep " + "(-0.50, -0.37, -0.01, -0.05). The bias is systematic, not sampling " + "noise.", + "**Nothing.** No uniformity percentage and no coefficient of " + "variation may be shown from camera-estimated weights, and the " + "capability publishes no number at all. Converted through " + "`2*Phi(10/CV) - 1` the CV inflation reads the flock **6.96 points " + "low** at a true CV of 12% โ€” a flock at 60% uniformity reports as " + "53% โ€” and across true CVs of 8-18% at fifteen birds the error runs " + "-1.46 to -13.94 points, **worst where the flock is most uniform**. " + "Uniformity is a decision variable: a farm culls, re-feeds or delays " + "a harvest on it, and every one of those errors runs in the " + "direction that makes a bad flock look acceptable. This is why the " + "answer is not a wider band โ€” `experimental` offers visible " + "uncertainty and a band does not move a centre. **The sample mean " + "and the range survive on `poultry_weight`**: at the same fifteen " + "birds and the same error the mean is out by 0.33% +/- 3.5 against " + "0.18% +/- 3.2 with exact weights, statistically indistinguishable, " + "because error averages out of a mean and squares into a variance. " + "**Uniformity from scale weights is exact and is what ยง23's 'no ML " + "once weights exist' was always true of** โ€” over exact weights the " + "same arithmetic is unbiased and lands within two points of the true " + "CV 63.5% of the time at fifteen birds. It is a derived claim the app " + "computes, not a capability that runs.", + data_needed="A per-bird error well under 4%, and 'under 4%' is not " + "the answer a first pass gives. At a true CV of 12% a 4% " + "error puts the uniformity bias at -0.79 points with " + "fifteen birds, which looks like enough; at thirty it is " + "-1.07 and at fifty -1.88, because the small figure at " + "n=15 is two biases cancelling rather than an error small " + "enough to ignore. The measurement bias alone is about -3 " + "points at 4% and about -9 at 7.8%. **Those 4% figures " + "are the experiment README's working and have no run " + "record** โ€” `config.yaml` declares no 4% arm, so unlike " + "every other figure in this entry they cannot be " + "recomputed from `results/`. Checked independently " + "against the closed form " + "`CV_obs = sqrt(CV^2 + e^2 + CV^2 e^2)`, which puts the " + "4% measurement bias at -2.49 uniformity points rather " + "than -3: the conclusion holds and the README's figure is " + "the more pessimistic. Nothing else is missing: not " + "labels, not compute, not a model.", + evidence=( + ("Aviagen Ross handbook: weigh 1% or 100 birds, individually", + "https://aviagen.com/assets/Tech_Center/Ross_Broiler/Aviagen-ROSS-Broiler-Handbook-EN.pdf"), + ("CV bias +2.40 points and uniformity error -6.96 points at true " + "CV 12%, 15 birds, 7.8% per-bird error; +2.16 at 30 birds", + "experiments/poultry_weight/metrics.json"), + ("The per-bird error the simulation uses: 7.8% mean relative " + "error, 83 held-out broilers, fixed Kinect, commercial house", + "https://doi.org/10.1016/j.compag.2016.02.011"), + ("The flock spread it runs against: uniformity 11-18%, mean 13%, " + "over 45 Ross 308 flocks", + "https://doi.org/10.3382/ps/pez252"), + ), + superseded=Superseded( + "needs_capture_change", + "The best-founded poultry claim available, because the manual " + "method it replaces is already a sample.", + "**This entry has now moved twice, and the second move reverses " + "the first.** The original verdict was about `poultry_weight`'s " + "capture and this entry inherited it. It was then classified " + "`coming_soon`, which an auditor called the weakest " + "classification in the registry โ€” rightly, because `state` is a " + "claim ceiling and availability is `is_runnable`'s to report โ€” " + "so it became `experimental` with an unmet `depends_on`. Every " + "step of that was correct reasoning about the wrong question. " + "**Nobody had asked what the input would be worth when it " + "arrived**, and the sentence above โ€” 'the manual method it " + "replaces is already a sample' โ€” is the error in miniature: it is " + "true, and it compares Animap against the wrong baseline. The " + "manual method is a sample *weighed on a scale*, and the thing " + "that separates it from a camera is not the sampling. It is that " + "one of them has measurement error and the other does not. ยง36 " + "asks for the observable signal to be shown impractical before a " + "capability is called unavailable, and error propagation over a " + "published error floor is that showing โ€” not a failed first " + "model, of which there is none here.", + )), + _d("poultry_footpad", _A, + "Build now as sampling (ยง18). A multimodal model against an explicit " + "0-4 rubric plus DINOv3 reference retrieval, over 10-20 held birds. " + "A slaughter line existing does not make farm sampling pointless.", + "ChickenCheck runs at 12,000-15,000 birds an hour and reaches kappa " + "0.70 against the median human scorer, with the humans themselves at " + "0.33-0.47. On a live bird the only published method stands each hen " + "on a transparent box over an upward-facing camera. A phone " + "photograph of a bird standing in litter does not show the footpad.", + "Experimental. 'Approximate grade: 2 / 4', correctable, aggregated as " + "'20 birds sampled, 4 moderate or worse, 20%'. The word approximate " + "is doing real work: the commercial slaughter-line system reaches " + "kappa 0.70 with a presented, washed foot, and the human scorers it " + "was measured against only reach 0.33-0.47 with each other. A held " + "bird photographed on a farm is a harder capture than either, so no " + "figure is claimed for it. The prevalence is a sample, not the flock.", + evidence=( + ("Slaughterline system, 500 images / 1,000 feet: kappa 0.70", + "https://doi.org/10.1016/j.psj.2020.05.052"), + ), + superseded=Superseded( + "needs_capture_change", + "Solved commercially โ€” on a slaughter line, where the foot " + "arrives presented and washed.", + "ยง18: 'The existence of slaughter-line automation does not make " + "farm sampling pointless.' The capture does change โ€” the farmer " + "holds the bird and photographs the underside of the foot โ€” and " + "that is now the declared protocol. A farm has no slaughter line " + "to use instead.", + )), + # ยง19's four, split into four keys because the capture differs even + # though the model stack does not. See `app/capabilities.py` for the + # argument. They share a disposition shape for the same reason they + # share a stack: the evidence question is identical for all four. + _d("poultry_hock", _A, + "Build now as a visual observation (ยง19). SAM 3.1 for the region, a " + "multimodal model for the description, DINOv3 retrieval where " + "reference images exist.", + "No public hock-lesion dataset was found by the earlier pass or this " + "one, and ยง19 asks for no named diagnosis, so there is nothing here " + "that a benchmark would currently be measuring. The nearest " + "evidenced neighbour is `poultry_footpad`, where a commercial " + "slaughter-line system reaches kappa 0.70 against the median human " + "scorer while the human scorers themselves reach only 0.33-0.47 โ€” " + "which is the honest prior for how well any lesion grade is agreed " + "on, before a phone and a live bird are added.", + "Experimental. 'Moderate visible hock lesion', for the bird in " + "frame. **No accuracy figure exists and none is implied.** Human " + "scorers agree with each other at kappa 0.33-0.47 on the comparable " + "footpad grade, so a severity word is a description and not a score, " + "and one bird is never a flock rate.", + evidence=( + ("Slaughterline system, 500 images / 1,000 feet: kappa 0.70, " + "human scorers 0.33-0.47 โ€” the comparable lesion grade", + "https://doi.org/10.1016/j.psj.2020.05.052"), + )), + _d("poultry_feather", _A, + "Build now as a visual observation (ยง19). Feather coverage is a " + "surface property and a photograph carries it.", + "No public feather-coverage dataset was found, and no accuracy " + "figure is available. There is an argument that the observation " + "suits the method โ€” coverage is an area on a surface, which is what " + "a segmentation mask measures directly, and unlike a lesion grade it " + "asks no scorer to judge severity โ€” but that is reasoning and not " + "evidence, and nothing here has measured it. What the observation " + "cannot carry is cause: reduced coverage on the back and tail is " + "produced by moult, by pecking, by rubbing and by disease alike, and " + "ยง19 asks for none of them to be named.", + "Experimental. 'Reduced feather coverage on back and tail.' Where " + "the coverage is missing is the observation; why it is missing is " + "not claimed, because moult, pecking, rubbing and disease all look " + "the same in a photograph. **No accuracy figure exists and none is " + "implied**, and one bird is never a flock rate.", + evidence=()), + _d("poultry_wound", _A, + "Build now as a visual observation (ยง19), on the same footing as " + "`cattle_wound`: SAM 3.1 mask, multimodal description, and an area " + "when a reference marker is in frame.", + "There is no public poultry wound dataset, exactly as there is no " + "public cattle wound dataset โ€” the entry under `cattle_wound` " + "records that, and the same answer applies. The value is not in the " + "first reading but in the comparison: an area and an appearance " + "recorded today are what make next week's capture mean something. " + "Naming a cause is refused for the same reason as everywhere else in " + "this file.", + "Experimental, and descriptive: 'Open wound visible on left flank', " + "with 'Approximate visible area: 12-16 cmยฒ' when a reference marker " + "is in frame. **No accuracy figure exists and none is implied** โ€” " + "there is no public wound dataset for poultry any more than for " + "cattle. The follow-up comparison is the value. Never a cause.", + evidence=()), + _d("poultry_eye_head", _A, + "Build now as a visual observation (ยง19), escalating rather than " + "naming. Ocular and head signs are where the notifiable diseases " + "show, which changes the wording and not the feasibility.", + "Discharge and swelling around the eye are visible, and they are " + "also among the presenting signs of Newcastle disease and infectious " + "coryza. The constraint is therefore the same one recorded under " + "`poultry_fecal` and `cattle_skin`: Newcastle is notifiable, the " + "published Newcastle recall in the best public dropping dataset is " + "62.7%, and a false positive on a notifiable disease creates " + "obligations that a farmer did not ask a phone to create. No public " + "dataset of poultry ocular signs was found, so there is additionally " + "no figure to quote.", + "Experimental. 'Visible discharge around left eye', plus swelling " + "and location, escalating to veterinary review. **No accuracy figure " + "exists and none is implied.** Nothing is named: these are among the " + "presenting signs of Newcastle disease, which is notifiable, and the " + "best public recall figure for Newcastle anywhere in this file is " + "62.7%. One bird is never a flock rate.", + evidence=( + ("1,255 PCR-validated images, CC BY 4.0 โ€” the nearest validated " + "poultry set, and it is droppings rather than heads", + "https://zenodo.org/records/5801834"), + )), + _d("poultry_heat_stress", _A, + "Build now as behaviour screening (ยง17). A probe measures the house; " + "vision measures the birds' response to it. They are complementary, " + "and the probe cannot tell you the birds are panting.", + "Panting is detectable in RGB โ€” mAP@50 0.927 on 1,000 images โ€” but " + "the demonstration was on caged tiers at 45 degrees, not overhead in " + "a floor house. Heat stress is a house condition, and a temperature " + "and humidity probe measures it directly, continuously, and for the " + "price of a phone call. Build the sensor integration instead.", + "Experimental, in three bands: no obvious visual signs / some " + "heat-associated behaviours / heat-stress-associated behaviour " + "elevated. The only panting figure available โ€” mAP@50 0.927 โ€” was " + "measured on caged tiers at 45 degrees, not overhead in a floor " + "house, so it does not transfer and is not quoted as Animap's. " + "Temperature and humidity are optional inputs, not outputs. **Never " + "an exact physiological claim, and never a fever.**", + evidence=( + ("YOLOv8n on caged tiers, 1,000 images: mAP@50 0.927", + "https://doi.org/10.3390/agriculture14071066"), + ), + superseded=Superseded( + "not_viable", + "Technically workable and commercially pointless. A five-dollar " + "sensor does it better.", + "'Technically workable' was the finding; 'commercially " + "pointless' was a product opinion that ยง17 overrules: 'A " + "temperature sensor measures the environment. Vision measures the " + "birds' response. They are complementary.' The sensor integration " + "is still worth building and is now an optional input here.", + )), + _d("poultry_litter", _A, + "Build now with the claim renamed (ยง16). Litter *condition* is " + "visible in RGB โ€” caking, wet-looking areas, the share of a scanned " + "region affected, where it is worst. Litter *moisture* is not.", + "The target variable is moisture, and moisture has no photometric " + "signature: a dark patch is wet litter, or shadow, or manure. No " + "litter dataset appears in any survey of poultry computer-vision " + "datasets, and 'litter condition' does not occur in a review of 82 " + "YOLO-in-poultry papers. The only work that functions uses UWB radar, " + "in a laboratory.", + "Experimental. 'Caking: High. Wet-looking areas: 18% of scanned " + "region. Worst near drinker line 3.' Every word of that is an " + "appearance claim about a scanned area, because no litter dataset " + "exists in any published survey and there is no accuracy figure to " + "quote. **Never a moisture percentage without a probe** โ€” a dark " + "patch is wet litter, or shadow, or manure, and RGB cannot separate " + "them.", + evidence=(), + superseded=Superseded( + "not_viable", + "Nothing exists, and the thing being asked for is not visible.", + "Right about moisture, wrong about the capability. ยง16: 'Do not " + "claim exact litter moisture from normal RGB. Do build Litter " + "Condition.' Caking, soiling and wet-looking areas are " + "appearances, and appearances are what a photograph carries. " + "`exact_litter_moisture` stays rejected and ยง29 names the probe.", + )), + _d("egg_quality", _A, + "Build now for external quality (ยง20): count, shape, obvious dirt, " + "discolouration and visible damage, from SAM plus OpenCV geometry " + "plus multimodal reasoning. Fine cracks need candling.", + "Geometry from a photograph is easy โ€” 99.4% mAP on 844 images โ€” and " + "mechanical graders already do it for less. Cracks are what a farmer " + "would pay for, and a hairline crack has almost no photometric " + "signature: dedicated hyperspectral work reaches F1 75.5%, while " + "tapping the shell and listening reaches 100%, because the crack " + "changes the resonance and not the picture. Competing against that in " + "RGB is competing against physics.", + "Experimental, for what is on the outside of the shell. Geometry is " + "the well-measured half โ€” 99.4% mAP on 844 images โ€” and it is also " + "the half worth least, so it is reported plainly and not dressed up. " + "**Hairline cracks are not claimed from an ambient photograph**: " + "dedicated hyperspectral work reaches only F1 75.5% while acoustic " + "tapping reaches 100%, because the crack changes the resonance and " + "not the picture. Candling is the route, and it needs a backlight.", + evidence=( + ("Acoustic crack detection outperforms every optical method", + "https://doi.org/10.1016/j.compag.2020.105716"), + ), + superseded=Superseded( + "needs_capture_change", + "Size and shape grading works and is worth little. Crack " + "detection is the valuable half and RGB cannot do it.", + "Both sentences survive and neither justifies removal. ยง20 keeps " + "external quality now and puts fine cracks behind a candling " + "accessory, which is a capture change with a five-dollar answer. " + "`hairline_crack_from_ambient_photo` stays rejected.", + )), + ] +} + + +def get(key: str) -> Disposition | None: + return DISPOSITIONS.get(key) diff --git a/app/identification.py b/app/identification.py new file mode 100644 index 0000000000000000000000000000000000000000..9eb108fd171c6399989142c85303c54bbedaabcc --- /dev/null +++ b/app/identification.py @@ -0,0 +1,699 @@ +"""Matching a capture against a farm's enrolled animals, and refusing to name one. + +Directive ยง6.4, and the counterpart to `app/counting.py`: that module turns boxes +into a claim, this one turns a nearest neighbour into a candidate. Both exist +because the hard part is not the model โ€” it is being honest about the cases the +model cannot carry. + +**The measurement this runner ships against is the open-set one, not the +closed-set one.** Run `8db9e0bd1b30` +(`experiments/cattle_identity/metrics.json`) enrolled 169 animals from the +muzzle268 database and measured, on 789 probes of animals that *were* enrolled: +top-1 0.9772, top-3 0.9937, MRR 0.9853, against a chance rate of 0.005917. That +is the number worth quoting and it is not the number that decides the product. + +On 416 probes of animals nobody enrolled, **the unthresholded false-accept rate +is 1.000**. Every unenrolled animal comes back as somebody, because every query +has a nearest neighbour and nothing about a nearest neighbour knows the right +answer was absent. The two similarity distributions overlap badly โ€” enrolled +probes median 0.9707, unenrolled median 0.9042 with an unenrolled *maximum* of +0.9776 โ€” so the cutoff that admits no impostor at all sits at 0.98 and accepts +**24.08%** of the correct matches. + +`MEASURED_POLICY` is that operating point, and choosing it is the whole design: + +- Three quarters of the time an enrolled animal is photographed, this runner + claims `no_confident_match` **and still returns the ranked candidates**, so a + person sees the names and picks one. That is a worse headline and the same + information. +- The alternative โ€” no threshold, always name the top candidate โ€” is the one + that reads better and puts a neighbour's cow in a farm's records 100% of the + time an unenrolled animal is photographed. + +`ยง6.4`'s confirm step is therefore load-bearing rather than decorative. + +**What carries that requirement to a device, precisely.** This paragraph used to +say `IdentityResult.to_json` states `requires_confirmation: True` on every path. +It does โ€” and this runner never calls it. The served result is +`schemas.InferenceResult`, which has no such field, so **the flag reaches +nobody** and citing it here described a guarantee that was not on the wire. + +What actually crosses is the capability row. `Requirement.HUMAN_CONFIRMATION` is +on the registry entry, travels verbatim to the API, and is published as +`requirements: ["guided_capture", "human_confirmation"]` by `GET /capabilities`, +where a database constraint forces ยง6.4's four buttons to travel beside it. That +is a real mechanism and it is a **different endpoint from the result**: a client +that renders a result without having read `/capabilities` has nothing in the +payload telling it to ask. Closing that gap means a field on `InferenceResult`, +which is a contract change for every capability and is not made here. + +What this runner does guarantee on every path is narrower and worth stating +exactly: no result carries an interpretation, an `observation_confidence`, or a +non-`None` confidence on any observation; `identity_candidate` is emitted only +when the measured policy accepted; and `_result` raises on +`identity_without_confirmation`. A result from here cannot *look* settled. It +relies on the client having read the capability to know it must ask. + +**What is not measured here.** The database is US beef breeds โ€” Angus, Angus x +Hereford, Continental x British crosses. No Nigerian and no zebu animal has been +through this, and a White Fulani is white all over, so the coat-pattern signal +the published re-identification literature leans on is absent for the herds this +product is for. The registry carries `UNVALIDATED_GEOGRAPHY`, which is what stops +`may_be_promoted_to_production` mechanically. + +## The cold-gallery cost, which is a real limit and not a solved problem + +Measured on 2026-08-23 over this service's own HTTP path by +`scripts/verify_identity_wiring.py`, on an Apple M-series laptop, single process, +enrolling five muzzle photographs per animal: + +| Register | Enrolment photographs | Cold request | Per photograph | Warm median | +|---|---|---|---|---| +| 20 animals | 100 | **11.13 s** | 0.11 s | **0.059 s** | +| 60 animals | 300 | **60.35 s** | 0.20 s | **0.07 s** | + +`_VECTOR_CACHE` is what separates the cold column from the warm one, and it only +helps the warm one. **A farm's first identification pays for embedding its whole +register.** + +**Both per-photograph rates are recorded because they disagree**, and the honest +reading is that this is a loaded shared laptop rather than a controlled +measurement โ€” the 0.20 s run had a test suite beside it. Extrapolate on the +slower one; a latency figure from a shared machine is not a latency figure, which +is a rule this repository already writes down in +`experiments/cattle_respiratory/README.md`. At 0.20 s a 169-animal register is +about **169 seconds** cold. + +Neither rate is the 59 ms in `backbones.py`. That is the embed call alone on a +frame somebody already decoded; this path also reads the blob and decodes a JPEG. +An earlier version of this paragraph extrapolated on 59 ms and so put a +169-animal farm at "about a minute" โ€” roughly three times better than measured, +in a paragraph whose entire purpose is to be honest about a limit. + +**Even the small register does not fit the inline job model.** `app/main.py` runs +jobs synchronously, and this service declined to wire an eighteen-second +capability inline at all; eleven seconds for twenty animals is the same order and +a hundred and sixty-nine seconds is not close. So the honest statement is not +"wired for the farm sizes Animap has" โ€” it is that **every** cold request is slow +and only the warm path is fast, so a deployment has to keep the cache warm rather +than treat the cold case as an edge. + +The fix is not a bigger cache. It is for the API to store each enrolment vector +beside its `MediaAsset` and send vectors instead of media ids, which moves the +cost to enrolment time where a person is already waiting. That is a change to the +API's schema and is not made here. + +**Where these numbers come from, since they cite no run record.** They are a +session measurement, not a benchmark, and the distinction matters in a repository +whose whole discipline is that a figure names a run id. The script is +`scripts/verify_identity_wiring.py`; it is reproducible and it has not been +through the experiment harness, so nothing here may be quoted as a measured +accuracy. What it does establish is that the wiring reproduces the benchmark +rather than merely citing it: over 122 held-out probes against a 60-animal +gallery, top-1 similarity ran min 0.8336, median 0.9764, max 0.9930 against the +run record's min 0.8336, median 0.9707, max 0.9930, and rank-1 was right 121 +times out of 122. The accept rate was 41% against the benchmark's 24.08%, which +is the direction a 60-animal gallery moves it against 169. +""" + +from __future__ import annotations + +import threading +from collections import OrderedDict +from uuid import UUID, uuid4 + +import numpy as np +from PIL import Image + +from app.adapters.embedding import DINOV3_SPEC, OnnxEmbeddingAdapter +from app.adapters.embedding.identity import ( + ENROLMENT_VIEWS, + PRIMARY_VIEW, + Embedding, + IdentityIndex, + IndexMismatch, + OpenSetPolicy, +) +from app.capabilities import FORBIDDEN_CLAIMS, Capability +from app.media import MediaRef, MediaStore +from app.providers import ModelArtefact +from app.quality import assess +from app.schemas import ( + EnrolledAnimal, + InferenceLocation, + InferenceRequest, + InferenceResult, + Observation, + QualityCheck, +) + +#: The operating point measured in run `8db9e0bd1b30`, and the only policy this +#: runner ships. +#: +#: `accept_similarity` 0.98 is the lowest cutoff in the published sweep whose +#: false-accept rate is 0.0. `accept_margin` 0.0 is the measured companion and it +#: never rejects anything โ€” a margin can only be non-negative โ€” which is +#: deliberate rather than an oversight: `open_set_margin_sweep` in the same run +#: shows the margin rule buys nothing here. At margin 0.07 it holds the +#: false-accept rate to 0.0072 while accepting 23.45% of correct matches, which +#: is *worse on both axes* than the similarity rule alone. The rule stays wired +#: because a future backbone may separate the two distributions differently, and +#: it is set where the evidence puts it. +#: +#: **Both numbers cite the run that produced them and `OpenSetPolicy` will not +#: accept them otherwise.** Its `__post_init__` raises `UnmeasuredThreshold` on a +#: threshold with no run id, which is what stops a cutoff being tuned by whoever +#: is looking at a demo that morning. +MEASURED_POLICY = OpenSetPolicy.measured( + accept_similarity=0.98, + accept_margin=0.0, + run_id="8db9e0bd1b30", + measured_on="muzzle268-169enrolled", +) + +#: Said on every run, whatever the result, in the shape `CountingProfile` +#: established. ยง6.4's product statement, cut to what a person reads. +STANDING_WARNING = ( + "A suggestion, not a record. Animap proposes an animal and you confirm it โ€” " + "an unconfirmed match is never written to this animal's history." +) + +#: Said whenever candidates are returned at all, because the open-set result is +#: not something a reader can infer from a ranked list. +UNENROLLED_WARNING = ( + "An animal that has never been enrolled will still produce a nearest match. " + "Measured on 416 photographs of unenrolled cattle, every one of them came " + "back as somebody. If this animal is new, register it rather than picking " + "the closest name." +) + +#: How many candidates a result carries. ยง6.4 shows a ranked list rather than one +#: answer, and `evidence_correction.selected_interpretation` is the reason: a +#: farmer picking the second name records *"rank 2 was right"*, which its own +#: docstring calls the highest-value training signal in the table. Three, because +#: `closed_set_top3_accuracy` is 0.9937 and a fourth carries no measurement. +TOP_K = 3 + +#: Enrolment vectors held between requests, keyed by media id and artefact. +#: +#: **Without this the design does not fit the inline job model.** The gallery +#: travels on the request so the service holds no farm state +#: (`schemas.EnrolledAnimal` says why), and the cost of that is re-embedding +#: every enrolled photograph on every identification. At the **measured 0.20 s** +#: per enrolment photograph โ€” end to end, including decode, not the 59 ms embed +#: call `backbones.py` reports on a frame somebody already decoded โ€” a farm with +#: a hundred animals and the five-shot enrolment the accuracy was measured on is +#: 500 photographs, or **about a hundred seconds a request**. That is far past +#: the inline ceiling `app/main.py` describes and far worse than the +#: eighteen-second capability this service declined to wire inline. +#: +#: **A media id is immutable, which is what makes caching it correct rather than +#: merely fast.** `MediaAsset` rows are write-once and the blob behind one is +#: never rewritten, so the same id is the same bytes forever and its vector +#: cannot go stale. The artefact digest is in the key because a re-export with +#: different pooling produces a vector of the same width and a different meaning +#: โ€” the case `IdentityIndex` pins its backbone id to catch โ€” so vectors from two +#: artefacts must never collide here. +#: +#: **The farm id is in the key, and it was not for one commit.** The argument for +#: leaving it out was that nothing can be read from the cache without already +#: holding the media id. A watchdog showed that is not quite the property that +#: matters: `AzureBlobMediaStore` scopes an unpathed lookup to `farm/{farm_id}/`, +#: so *without* a cache a request naming another farm's media id fails to resolve +#: โ€” and *with* one it would hit and score. What leaked was a similarity against +#: a foreign photograph rather than the photograph or the name, and it needed the +#: service token plus a known foreign UUID, so it was narrow. It was also free to +#: close, and a cache must not be the reason a farm boundary that the media store +#: enforces stops being enforced. +_VECTOR_CACHE: OrderedDict[tuple[str, str, str], np.ndarray] = OrderedDict() + +#: Entries kept. 768 float32 values is 3 KB, so this is about 12 MB โ€” small +#: against the container's 4 GiB, and enough for a few hundred animals at five +#: shots each. Least-recently-used is evicted, so the farms being worked today +#: stay warm. +_CACHE_LIMIT = 4096 + +#: `submit()` runs in FastAPI's threadpool, so two captures can be scored at +#: once and an `OrderedDict` is not safe under that on its own. +_CACHE_LOCK = threading.Lock() + + +def cache_clear() -> None: + """Empty the vector cache. For tests, and for a deployment that swaps an + artefact without a restart.""" + with _CACHE_LOCK: + _VECTOR_CACHE.clear() + + +def _cached_vector(key: tuple[str, str, str]) -> np.ndarray | None: + with _CACHE_LOCK: + vector = _VECTOR_CACHE.get(key) + if vector is not None: + _VECTOR_CACHE.move_to_end(key) + return vector + + +def _remember(key: tuple[str, str, str], vector: np.ndarray) -> None: + with _CACHE_LOCK: + _VECTOR_CACHE[key] = vector + _VECTOR_CACHE.move_to_end(key) + while len(_VECTOR_CACHE) > _CACHE_LIMIT: + _VECTOR_CACHE.popitem(last=False) + + +#: Muzzle photographs per animal in the protocol the accuracy was measured under. +#: +#: Run `8db9e0bd1b30` enrolled each animal from five, all under the `muzzle` view +#: name, and `IdentityIndex.candidates` scores an animal by its **best** view โ€” +#: so the five are five chances to match rather than five samples averaged +#: together. An animal enrolled from one photograph is being asked a harder +#: question than the benchmark asked, and measured that way on twelve animals, +#: **0 of 23 held-out queries cleared the 0.98 cutoff**. +#: +#: Used only to report a thin enrolment, never to refuse one. +MEASURED_ENROLMENT_SHOTS = 5 + + +class IdentityRunner: + """Runs `cattle_identity` against one capture and one farm's register.""" + + #: The view a single capture is compared against. ยง6.4 leads with the muzzle + #: and the re-identification measurement is entirely muzzle-to-muzzle, so + #: comparing a muzzle print against a side-body vector would contribute + #: nothing but a chance of a spurious high score. Restricting the pool is + #: what keeps the runtime behaviour inside the measured one. + #: + #: **Nothing checks that the capture actually is a muzzle, and a watchdog was + #: right to call that a gap.** The registry declares `muzzle_not_visible` in + #: `reject_if`, but `app/quality.py` implements resolution, illumination and + #: motion blur and nothing that looks for an anatomical part โ€” so the + #: condition is declared and never evaluated. A caller that sends the five + #: enrolment views in the order the registry lists them sends `front_face` + #: first, and it would be scored against a gallery of muzzles. + #: + #: What that costs is bounded in the safe direction: a front-face photograph + #: is *less* similar to every enrolled muzzle, so it drives the top score down + #: and towards `no_confident_match`. It degrades to a refusal rather than to a + #: wrong name. But it is a refusal for a reason nobody is told, and the fix is + #: a real one โ€” either the capture flow states which view it took, or + #: something evaluates `muzzle_not_visible`. Neither exists today. + query_view = PRIMARY_VIEW + + def run( + self, + *, + request: InferenceRequest, + capability: Capability, + artefact: ModelArtefact, + store: MediaStore, + request_id: UUID | None = None, + ) -> InferenceResult: + request_id = request_id or uuid4() + warnings: list[str] = [STANDING_WARNING] + + if len(request.media_ids) > capability.frames_required: + warnings.append( + f"{len(request.media_ids)} frames were supplied; this capability " + f"reads {capability.frames_required}." + ) + + image = store.open_image(self._ref(request, request.media_ids[0])) + verdict = assess(image) + checks = list(verdict.checks) + + if verdict.blocked: + failure = verdict.first_failure + warnings.append( + failure.detail if failure and failure.detail + else "The capture was not usable." + ) + return self._result( + request=request, capability=capability, artefact=artefact, + request_id=request_id, observations=[], checks=checks, + warnings=warnings, recapture=True, + ) + + adapter = OnnxEmbeddingAdapter( + artefact, DINOV3_SPEC, + input_size=artefact.input_size or 224, + mean=artefact.image_mean or (0.485, 0.456, 0.406), + std=artefact.image_std or (0.229, 0.224, 0.225), + dimensions=artefact.embedding_dimensions or 768, + ).load() + + index = IdentityIndex( + farm_id=str(request.farm_id), + backbone_id=DINOV3_SPEC.adapter_id, + dimensions=adapter.dimensions, + artefact_sha256=artefact.sha256, + ) + names: dict[str, str] = {} + enrolled_views, skipped = self._enrol( + index, adapter, artefact, request, store, names, + ) + warnings.extend(skipped) + + query = self._embed(adapter, artefact, image) + result = index.match( + query, + policy=MEASURED_POLICY, + top_k=TOP_K, + # Compared against muzzles only. See `query_view`. + restrict_to_views=(self.query_view,), + names=names, + ) + + checks.append(self._enrolment_check(index, enrolled_views)) + + observations: list[Observation] = [ + # Emitted on every path, refusals included, for the reason + # `counting.py` emits its grid: a threshold that has to be re-derived + # later is re-derived from stored results or not at all. + Observation( + type="enrolled_animals", + value=float(result.enrolled_animals), + unit="animals", + confidence=None, + ), + Observation( + type="open_set_verified", + value=1.0 if result.open_set_verified else 0.0, + confidence=None, + ), + ] + + top = result.top + if top is not None: + observations.append(Observation( + # **Uncalibrated, and named so.** Nothing mapped cosine + # similarity onto a probability that the animal is Kofi, so this + # is a diagnostic and the release declares no sentence for it โ€” + # exactly as `counting_grid` carries none. ยง37's whole complaint + # is an uncalibrated model score reaching a person as a promise. + type="top_candidate_similarity", + value=round(top.similarity, 4), + confidence=None, + )) + + accepted = result.claim == "identity_candidate" + if not accepted: + observations.append(Observation( + type="no_confident_match", value=None, confidence=None, + )) + + # **The ranked list is emitted on both paths, under different names.** + # `IdentityResult` returns candidates whether or not the policy accepted + # the top one, and dropping them on the refusal path would throw away the + # thing ยง6.4 is shaped around: `evidence_correction.selected_ + # interpretation` records *"rank 2 was right"*, and its own docstring + # calls that the highest-value training signal in the table. At the + # measured operating point three quarters of correct matches are refused, + # so the refusal path is where most of that signal lives โ€” emitting + # nothing there would collect it almost nowhere. + # + # The type differs because the claim differs. `identity_candidate` is + # ยง6.4's *"This looks like Kofi"*; `closest_candidate` is *"not a + # confident match, and these are the nearest on your register"*. One + # observation type carrying both would leave the device deciding which + # sentence to show from a field that does not say, and the release + # declares a separate sentence for each. + # + # ## Exactly one settled name, ever + # + # An accepted run used to publish `identity_candidate` for **all three** + # ranked candidates, so `deploy/semantics` rendered *"This looks like + # Cow 0100"*, *"This looks like Cow 0200"* and *"This looks like Cow + # 0300"* โ€” three settled names for one animal, on one `captured_at`, + # which no ordering on the device can pick between. The policy accepted + # **the top candidate**; it said nothing whatever about ranks 2 and 3. + # + # So rank 1 is the proposal and the rest are what ยง6.4's screen calls + # *"Also considered"* โ€” the same list, under the type that does not + # assert. Picking one of them is `choose_another_animal`, which is a + # correction and the signal the ledger is built to keep. + for position, candidate in enumerate(result.candidates): + settled = accepted and position == 0 + observations.append(Observation( + type="identity_candidate" if settled else "closest_candidate", + # The animal's own name. `identity_candidate` is one of three + # claims in the whole registry that carries free text, and + # `app/capabilities.py` says why: a registry cannot enumerate a + # farm's animals. The value comes from the farm's own register + # via the request, never from this service. + value=candidate.display_name, + # **`confidence` stays None on both paths.** + # `Candidate.confidence` is documented as never set by the + # matcher, and promoting a similarity into it here would defeat + # that in one line. + confidence=None, + )) + + if result.candidates: + warnings.append(UNENROLLED_WARNING) + + warnings.extend(result.warnings) + + if index.unverified_queries: + # Should be zero on this path: every vector is built as an + # `Embedding` below. Reported rather than asserted, because the count + # existing at all is the gap between what the index checks and what + # it would like to. + warnings.append( + f"{index.unverified_queries} vectors were compared without " + f"provenance. Every vector this service builds carries it, so " + f"this is a defect rather than a capture problem." + ) + + return self._result( + request=request, capability=capability, artefact=artefact, + request_id=request_id, observations=observations, checks=checks, + warnings=warnings, + # A refusal here is not a bad photograph. The animal may simply not + # be enrolled, and asking for a recapture would send a worker back to + # a pen to re-photograph a cow the system has never seen. + recapture=verdict.degraded, + ) + + def _ref(self, request: InferenceRequest, media_id: UUID, + object_path: str | None = None) -> MediaRef: + return MediaRef( + media_id=media_id, + farm_id=request.farm_id, + captured_at=request.captured_at, + object_path=object_path or request.path_for(media_id), + ) + + def _embed( + self, adapter: OnnxEmbeddingAdapter, artefact: ModelArtefact, + image: Image.Image, + ) -> Embedding: + """A vector that can prove where it came from. + + Always an `Embedding`, never a bare array. `IdentityIndex._accept` can + only check the *width* of an array, and a watchdog scored a foreign + vector against a `dinov3-vits16` index at similarity 1.0 through exactly + that gap. + """ + return Embedding( + vector=adapter.embed(image), + backbone_id=DINOV3_SPEC.adapter_id, + artefact_sha256=artefact.sha256, + ) + + def _enrol( + self, + index: IdentityIndex, + adapter: OnnxEmbeddingAdapter, + artefact: ModelArtefact, + request: InferenceRequest, + store: MediaStore, + names: dict[str, str], + ) -> tuple[int, list[str]]: + """Build this farm's gallery from the request. Returns views enrolled. + + A gallery entry that cannot be read is **skipped with a warning rather + than failing the job**: one unreadable enrolment photograph out of two + hundred should not stop a farmer identifying an animal, and the animals + that did load are still a gallery. The count that reaches the result is + the count that actually enrolled, so a farm cannot be told it was + compared against more animals than it was. + """ + enrolled_views = 0 + skipped: list[str] = [] + + for entry in request.enrolled: + usable = self._views_for(entry, skipped) + if not usable: + continue + who = entry.display_name or entry.animal_id + stored = 0 + + for view, media_ids in usable.items(): + for position, media_id in enumerate(media_ids): + key = (str(request.farm_id), str(media_id), artefact.sha256) + cached = _cached_vector(key) + if cached is not None: + vector = Embedding( + vector=cached, + backbone_id=DINOV3_SPEC.adapter_id, + artefact_sha256=artefact.sha256, + ) + else: + try: + image = store.open_image(self._ref( + request, media_id, + entry.path_for_view(view, position), + )) + except Exception as exc: # MediaError, or any store error + skipped.append( + f"An enrolled {view} photograph of {who} could " + f"not be read ({exc}), so it was left out of " + f"the comparison." + ) + continue + vector = self._embed(adapter, artefact, image) + _remember(key, vector.vector) + try: + # **One call per photograph, under the one view name.** + # `enrol` takes a mapping of view to vector, so a single + # call can only hold one shot per view โ€” and the measured + # protocol is five muzzles per animal, scored by maximum. + # Calling it per photograph is how those five reach the + # index, and it is what run 8db9e0bd1b30 did. + index.enrol( + entry.animal_id, {view: vector}, + media_ids={view: str(media_id)}, + ) + except (IndexMismatch, ValueError) as exc: + skipped.append( + f"A {view} photograph of {who} was left out of the " + f"comparison: {exc}" + ) + continue + stored += 1 + + if not stored: + continue + names[entry.animal_id] = who + enrolled_views += stored + + return enrolled_views, skipped + + def _views_for( + self, entry: EnrolledAnimal, skipped: list[str] + ) -> dict[str, list[UUID]]: + """The views of one animal this runner will compare against. + + Unknown view names are dropped here with a warning rather than left for + `enrol` to raise on, because one typo in one animal's record should cost + that animal's view and not the whole request. + """ + usable: dict[str, list[UUID]] = {} + for view, media_ids in entry.views.items(): + if view not in ENROLMENT_VIEWS: + skipped.append( + f"{entry.display_name or entry.animal_id} has a view called " + f"{view!r}, which is not one of the five ยง6.4 enrols " + f"({', '.join(ENROLMENT_VIEWS)}). It was ignored." + ) + continue + if media_ids: + usable[view] = list(media_ids) + return usable + + def _enrolment_check( + self, index: IdentityIndex, enrolled_views: int + ) -> QualityCheck: + """Whether this farm had anything to compare the capture against. + + A quality check rather than a warning, because the answer is about the + *farm's register* rather than about the photograph, and a capture flow + that can distinguish the two can say *"register this animal"* instead of + *"take it again"*. + """ + muzzles = sum(1 for v in index.views if v.view == self.query_view) + animals = len(index.animal_ids) + if muzzles: + detail = ( + f"{animals} animals enrolled, {enrolled_views} views, " + f"{muzzles} of them muzzles." + ) + thin = [ + a for a in index.animal_ids + if sum(1 for v in index.views + if v.animal_id == a and v.view == self.query_view) + < MEASURED_ENROLMENT_SHOTS + ] + if thin: + # **Reported, never a refusal.** A thin enrolment still matches, + # and one good muzzle photograph is worth more than a refused + # enrolment โ€” `IdentityIndex.enrol` takes that position and this + # agrees with it. What it is not is the protocol the accuracy was + # measured under, and a farm whose animals never quite match + # deserves to know the reason is its register rather than its + # camera. + detail += ( + f" {len(thin)} of them carry fewer than " + f"{MEASURED_ENROLMENT_SHOTS} muzzle photographs, which is " + f"the enrolment the accuracy was measured on." + ) + return QualityCheck(check="gallery", passed=True, detail=detail) + return QualityCheck( + check="gallery", + passed=False, + detail=( + f"No enrolled {self.query_view} to compare against. " + f"{len(index.animal_ids)} animals were sent and none carries " + f"the view this capability matches on." + ), + ) + + def _result( + self, *, request, capability, artefact, request_id, observations, + checks, warnings, recapture, + ) -> InferenceResult: + forbidden = [o.type for o in observations if o.type in FORBIDDEN_CLAIMS] + if forbidden: + raise ValueError( + f"{capability.key} tried to emit a forbidden claim: {forbidden}" + ) + banned = set(capability.acquisition.forbidden_claims) + offending = [o.type for o in observations if o.type in banned] + if offending: + # `identity_without_confirmation` is the one thing ยง6.4 forbids, and + # it is checked here as well as in the registry because this runner + # is the only thing that could emit it. + raise ValueError( + f"{capability.key} tried to emit {offending}, which its own " + f"registry entry forbids by name." + ) + + return InferenceResult( + request_id=request_id, + capability_key=capability.key, + model_id=artefact.model_id, + model_version=artefact.version, + inference_location=InferenceLocation.REMOTE, + subject_type=request.subject_type, + subject_id=request.subject_id, + observations=observations, + # **No interpretation, and none is possible.** A name is a fact about + # the register, not a judgement about the animal, and there is + # nothing for a vet to review. `observation_confidence` is likewise + # None: the three-level label would be read as a confidence in the + # name, and nothing calibrated one. + interpretations=[], + observation_confidence=None, + interpretation_confidence=None, + quality_checks=checks, + warnings=warnings, + recommended_recapture=recapture, + ) + + +#: Capabilities with an implemented adapter, in the shape `app/counting.py` +#: publishes. `app/main.py` merges the two: a capability with a validated +#: artefact and no entry in either returns 501 rather than a plausible result. +RUNNERS: dict[str, IdentityRunner] = { + "cattle_identity": IdentityRunner(), +} diff --git a/app/main.py b/app/main.py new file mode 100644 index 0000000000000000000000000000000000000000..edc9613225b5cdecb6a0677a0804549444f3bde3 --- /dev/null +++ b/app/main.py @@ -0,0 +1,597 @@ +"""The inference service. + +Brief ยง30. FastAPI, and deliberately small: submit a job, ask about a job, list +what the service can actually do. + +The service's most important property is what it refuses. A capability with no +validated artefact returns `unavailable` โ€” not a plausible-looking result, not a +placeholder, and not an error. `unavailable` is the honest answer for +twenty-five of the twenty-eight capabilities today, and the API's job is to say +so plainly enough that a client cannot mistake it for anything else. + +**`state` and `runnable` are different questions and the client must render +both** (ADR 0021). `state` is the strongest claim a capability is entitled to +make once its stack is wired; `runnable` is whether an artefact and an adapter +exist today. Most capabilities are `experimental` and `runnable: false`, which +means *"this is a real feature and it is not switched on yet"* โ€” not *"this +works."* A client that renders `state` alone will overclaim. + +Three run for real. `cattle_detection` and `poultry_count` execute a checksummed +YOLOX artefact against the referenced media and return a result built from what +the model actually produced (ADR 0018). `cattle_identity` embeds the capture +with a checksummed DINOv3 artefact and ranks it against the enrolled animals the +request carries (`app/identification.py`). + +**Twenty-five is not a backlog of twenty-five equal items**, and reading it as +one is how this service gets misreported. Several of the unwired capabilities +have complete benchmarks whose result is that the method does not work: +`cattle_gait` separates lame from sound at AUC 0.3117 (p = 0.91) and +`poultry_respiratory` at AUC 0.4141, both at or below chance. Those are measured +negatives, not missing work, and registering either would wire a screen that is +anti-correlated with the thing it screens for. + +Jobs run inline. A frame that settles at the first grid is well under a second on +a CPU; a dense one that runs all three grids takes a few seconds (ADR 0019). A +queue becomes worth its moving parts when a capability arrives that takes tens of +seconds, and none does. + +**Two endpoints are open and two are not.** `/health` and `/capabilities` carry +no farm data and the platform's probes need `/health`, so both are unauthenticated. +`/jobs` runs a model against a farm's photographs, so it needs a bearer token +whenever one is configured. +""" + +from __future__ import annotations + +import hmac +import logging +import os +from uuid import UUID, uuid4 + +from fastapi import Depends, FastAPI, Header, HTTPException +from pydantic import BaseModel + +from app import dispositions +from app.capabilities import ( + REGISTRY, + Capability, + CapabilityState, + Requirement, + licence_exposure, +) +from app.counting import RUNNERS as _COUNTING_RUNNERS +from app.detectors import DetectorError +from app.identification import RUNNERS as _IDENTITY_RUNNERS +from app.media import MediaError, build_store +from app.providers import ( + CompositeProvider, + HostedModelProvider, + LocalArtefactProvider, +) +from app.reasoning import RUNNERS as _REASONING_RUNNERS +from app.schemas import InferenceRequest, JobState, JobStatus + +logger = logging.getLogger(__name__) + +app = FastAPI( + title="Animap inference", + version="0.1.0", + summary="Runs validated livestock models, and refuses to invent results.", +) + +#: Both kinds of capability, asked in order. +#: +#: `LocalArtefactProvider` serves the three with a checksummed file on disk; +#: `HostedModelProvider` serves the sixteen whose model is somebody else's and +#: which can never have a file. The sets are disjoint and `CompositeProvider` +#: says why that is a rule rather than a coincidence. +provider = CompositeProvider(LocalArtefactProvider(), HostedModelProvider()) +media = build_store() + +#: Every capability with an implemented adapter, from both modules that hold +#: one. Merged here rather than in either, so neither has to import the other and +#: the set `/health` counts is the set `/jobs` dispatches on. +#: +#: **A key in exactly one of them.** The two dicts are disjoint by construction โ€” +#: counting owns the two detector capabilities, identification owns +#: `cattle_identity` โ€” and a collision would silently take whichever was merged +#: last, so it is refused rather than resolved. +_MODULES = { + "counting": _COUNTING_RUNNERS, + "identification": _IDENTITY_RUNNERS, + "reasoning": _REASONING_RUNNERS, +} +_SEEN: dict[str, str] = {} +for _module, _runners in _MODULES.items(): + for _key in _runners: + if _key in _SEEN: + raise RuntimeError( + f"{_SEEN[_key]} and {_module} both claim {_key}. One capability " + f"has one runner, and a silent winner here is a capability " + f"answered by a model nobody chose." + ) + _SEEN[_key] = _module + +RUNNERS: dict[str, object] = { + **_COUNTING_RUNNERS, **_IDENTITY_RUNNERS, **_REASONING_RUNNERS, +} + +#: In-memory for now. Jobs are re-submittable and carry no record of their own โ€” +#: the durable state lives in the API's `model_runs`, and a lost job means a +#: retry rather than lost data. A queue replaces this when a model exists to run. +_jobs: dict[UUID, JobStatus] = {} + + +def _configured_token() -> str: + """Read at call time, not import time, so a test can set it and so a secret + rotation takes effect on restart rather than needing a rebuild.""" + return os.environ.get("ANIMAP_INFERENCE_TOKEN", "") + + +def require_token(authorization: str = Header(default="")) -> None: + """Bearer auth on the endpoints that touch a farm's media. + + Absent `ANIMAP_INFERENCE_TOKEN` this is a no-op, which is what lets the + tests and a local uvicorn run without ceremony. That default is only safe + because it is *visible*: `/health` reports `authenticated`, so a deployment + that reached the internet without a token says so to anyone who asks, + including the person checking it after a deploy. + """ + expected = _configured_token() + if not expected: + return + + scheme, _, presented = authorization.partition(" ") + # Constant-time, because a token compared with `==` leaks its prefix to + # anyone willing to time a few thousand requests. + if scheme.lower() != "bearer" or not hmac.compare_digest(presented, expected): + raise HTTPException(401, "A valid bearer token is required.") + + +class HealthResponse(BaseModel): + status: str + capabilities_registered: int + capabilities_runnable: int + artefacts_loaded: int + #: Capabilities that have both an artefact and an adapter. The gap between + #: this and `capabilities_runnable` is the set that returns 501. + adapters_implemented: int + #: Where captures are read from. Reported because a service that is healthy + #: and pointed at an empty local directory fails every job for a reason no + #: probe would otherwise show. + media_provider: str + #: The licence of every artefact that is actually loaded. A deployment that + #: has picked up a copyleft model is a deployment in breach, so it is visible + #: from outside rather than only in a log (ADR 0017). + artefact_licenses: list[str] + #: Whether `/jobs` requires a bearer token. False on a public deployment is + #: a misconfiguration, and this is what makes it findable without reading + #: the Container App's environment. + authenticated: bool + + +class SupersededView(BaseModel): + """The verdict a capability used to carry, and why it moved (ADR 0021). + + Served rather than kept internal so that a reframing nobody agrees with is + arguable from outside this service. + """ + + verdict: str + summary: str + reframed_because: str + + +class DispositionView(BaseModel): + """The evidence behind a capability, and what the product may say about it. + + Served alongside the state because a client that can only render `Coming + soon` has nothing to say when a farmer asks *when*. `stated_uncertainty` is + the line a result screen shows beside a number (directive ยง37). + """ + + group: str + summary: str + blocker: str + stated_uncertainty: str + data_needed: str | None + evidence: list[str] + superseded: SupersededView | None + + +class QuantityView(BaseModel): + """What one claim's observation value may carry, and on whose authority. + + `basis` is published rather than kept internal because the three kinds are + not equally strong: a `scale` bound is what the unit means, and a + `guardrail` is an engineering judgement with no measurement behind it. A + client that shows a farmer a refused reading should be able to say which. + """ + + claim: str + #: `null` when the claim carries no quantity at all. + unit: str | None + carries_number: bool + minimum: float | None + maximum: float | None + step: float | None + basis: str | None + why: str + + +class OutputView(BaseModel): + type: str + unit: str | None + show_range: bool + #: The range outside which a value is broken rather than merely surprising, + #: and the granularity the rubric supports. All `null` for the twelve + #: capabilities whose output carries no number. See `OutputSpec` for what + #: these are and, more importantly, what they are not: they bound the + #: impossible, and say nothing about where a real answer usually falls. + #: + #: **Enforced as well as published**, which they were not for one commit: + #: `app/adapters/claims.py::schema_for` and `check_numeric_bounds` both read + #: them, so a client that pre-checks against these gets the same answer the + #: service does. + plausible_min: float | None + plausible_max: float | None + step: float | None + #: Which of `allowed_claims` report the quantity the three fields above + #: bound. Empty for a categorical output. A client showing an observation + #: knows from this whether the bounds apply to it โ€” a footpad grade is 0 to + #: 4, and the sampled prevalence beside it is a percentage. + measured_claims: list[str] + #: Every claim in the vocabulary and what number it may carry, which is what + #: the three fields above could not say. They bound one quantity, so an + #: observation on any other claim went unbounded โ€” 73 of the registry's 93, + #: and the route a lameness score took to a farm. + #: + #: `carries_number: false` means a figure beside this claim is the defect, + #: not a finer reading of it, and the service refuses every number there. The + #: claim still carries a word: `Observation.value` takes a string, and + #: 'moderate' or 'left flank' is what such a claim is for. + quantities: list[QuantityView] + + +class AcquisitionView(BaseModel): + """Directive ยง34. How this capability's signal is acquired, and what may be + claimed from it. The Android capture flow reads this.""" + + protocol: str + modality: list[str] + output: OutputView + minimum_capture_seconds: int | None + preferred_capture_seconds: int | None + minimum_distance_m: float | None + minimum_samples: int | None + preferred_samples: int | None + required_views: list[str] + optional_inputs: list[str] + confirmation_options: list[str] + escalation: str | None + #: Workflows this capability must never hold up (ยง6.6). + never_blocks: list[str] + reject_if: list[str] + #: What a model may emit. This is the closed vocabulary a reasoner's schema + #: is built from, so a client rendering it is showing what Animap can say. + allowed_claims: list[str] + forbidden_claims: list[str] + #: Quantities the result screen may show that no model produces โ€” the app + #: computes them from a capability output plus the farm's own records. + #: `poultry_count` is the only holder, carrying ยง6.3's third quantity. + derived_claims: list[str] + #: **What a model may say in words a farmer reads** (ADR 0024). `evidence` + #: used to be free text; it is a closed enum of these phrases now, per + #: capability, exactly as `allowed_claims` is. The strings are reader-facing + #: rather than identifiers, so a client renders them as they are โ€” and a + #: client that pre-checks a response against this list gets the same answer + #: the service does. + #: + #: Empty for `poultry_uniformity`, which may claim nothing (ADR 0023). + allowed_evidence: list[str] + #: What a model may say about the **capture** rather than about the animal. + #: A shared set plus one phrase per condition in `reject_if`, so the + #: rejection a client already renders and the sentence a farmer reads for it + #: cannot drift apart. + allowed_limits: list[str] + + +class CapabilityView(BaseModel): + key: str + species: str + #: The single-word spelling the API's `capabilities` table stores. The full + #: list is `acquisition.modality`. + modality: str + state: CapabilityState + #: Any combination of `guided_capture`, `human_confirmation`, + #: `hardware_required`, `fixed_installation` โ€” or none. Sorted, so a client + #: can compare two responses. + requirements: list[str] + #: What a farmer gets with no signal. Nineteen capabilities are `required` + #: because their first stage is a hosted model โ€” which is a hosting fact, + #: never a reason for a lower `state`. + connectivity: str + #: Whether this could plausibly move to the phone later. An engineering + #: property, not a promise and not a badge. + on_device_candidate: bool + output_kind: str + depends_on: list[str] + #: The runtimes this capability's declared stack leans on, named as + #: `app/adapters/licences.py` names them. + model_stack: list[str] + #: Runtimes in that stack Animap may not currently serve, cannot fetch + #: unattended, or has never checked the terms of. **Never a reason for a + #: lower `state`** โ€” a licence is an attribute of a model, not a property of + #: a capability. Empty for most. + licence_exposure: list[str] + acquisition: AcquisitionView + #: Whether a result can be produced **today**. Independent of `state`. + runnable: bool + reason: str + disposition: DispositionView | None + + +@app.get("/health", response_model=HealthResponse) +def health() -> HealthResponse: + runnable = [c for c in REGISTRY.values() if provider.can_run(c)] + return HealthResponse( + status="ok", + capabilities_registered=len(REGISTRY), + capabilities_runnable=len(runnable), + artefacts_loaded=len(provider.artefacts), + adapters_implemented=sum(1 for c in runnable if c.key in RUNNERS), + media_provider=media.provider, + artefact_licenses=sorted({a.license for a in provider.artefacts.values()}), + authenticated=bool(_configured_token()), + ) + + +@app.get("/capabilities", response_model=list[CapabilityView]) +def capabilities() -> list[CapabilityView]: + """What the service can do, what it cannot, and what it may say either way. + + The single source of what a user is told about a capability. A client shows + `state` and `runnable` together, `acquisition` to drive the capture, and + `disposition.stated_uncertainty` beside any number. + """ + # Computed once for the whole response rather than per capability: it reads + # the adapters' licence ledger, and 28 lookups of the same table is waste. + exposure = licence_exposure() + return [_view(c, exposure.get(c.key, [])) for c in REGISTRY.values()] + + +@app.post("/jobs", response_model=JobStatus, status_code=202, + dependencies=[Depends(require_token)]) +def submit(request: InferenceRequest) -> JobStatus: + capability = REGISTRY.get(request.capability_key) + if capability is None: + raise HTTPException(404, f"Unknown capability: {request.capability_key}") + + if not provider.can_run(capability): + # Not an error. The capture is already saved on the device and in the + # API; this only says no interpretation is available for it yet. + status = JobStatus( + job_id=uuid4(), + state=JobState.UNAVAILABLE, + capability_key=capability.key, + detail=_unavailable_reason(capability), + ) + _jobs[status.job_id] = status + return status + + runner = RUNNERS.get(capability.key) + if runner is None: + # An artefact without an adapter stays unimplemented rather than + # stubbed: an adapter that returns something plausible is precisely the + # failure ADR 0005 exists to prevent. + raise HTTPException( + 501, + f"{capability.key} has a validated artefact but no adapter is " + f"implemented for it yet.", + ) + + artefact = provider.artefact_for(capability) + job_id = uuid4() + try: + result = runner.run( + request=request, + capability=capability, + artefact=artefact, + store=media, + request_id=job_id, + ) + except MediaError as exc: + # The referenced media is missing or unreadable. The capture itself is + # safe on the device, so this is a job to retry, not data to discard. + status = JobStatus( + job_id=job_id, state=JobState.FAILED, + capability_key=capability.key, detail=str(exc), + ) + except (DetectorError, ValueError) as exc: + logger.exception("%s failed on %s", capability.key, request.media_ids[0]) + status = JobStatus( + job_id=job_id, state=JobState.FAILED, + capability_key=capability.key, + detail=f"The model could not be run: {exc}", + ) + else: + status = JobStatus( + job_id=job_id, state=JobState.COMPLETE, + capability_key=capability.key, result=result, + ) + + _jobs[job_id] = status + return status + + +@app.get("/jobs/{job_id}", response_model=JobStatus, + dependencies=[Depends(require_token)]) +def job(job_id: UUID) -> JobStatus: + status = _jobs.get(job_id) + if status is None: + raise HTTPException(404, "No such job.") + return status + + +def _view(c: Capability, exposure: list[str] | None = None) -> CapabilityView: + runnable = provider.can_run(c) + return CapabilityView( + key=c.key, + species=c.species, + modality=c.modality, + state=c.state, + requirements=sorted(r.value for r in c.requirements), + connectivity=c.connectivity.value, + on_device_candidate=c.on_device_candidate, + output_kind=c.output_kind, + depends_on=list(c.depends_on), + model_stack=list(c.model_stack), + licence_exposure=list(exposure or []), + acquisition=_acquisition_view(c), + runnable=runnable, + reason="" if runnable else _unavailable_reason(c), + disposition=_disposition_view(c.key), + ) + + +def _acquisition_view(c: Capability) -> AcquisitionView: + a = c.acquisition + return AcquisitionView( + protocol=a.protocol, + modality=[m.value for m in a.modality], + output=OutputView( + type=a.output.type, unit=a.output.unit, show_range=a.output.show_range, + plausible_min=a.output.plausible_min, + plausible_max=a.output.plausible_max, + step=a.output.step, + measured_claims=list(a.output.measured_claims), + quantities=[ + QuantityView( + claim=q.claim, unit=q.unit, carries_number=q.carries_number, + minimum=q.minimum, maximum=q.maximum, step=q.step, + basis=q.basis.value if q.basis else None, why=q.why, + ) + for q in a.output.quantities + ], + ), + minimum_capture_seconds=a.minimum_capture_seconds, + preferred_capture_seconds=a.preferred_capture_seconds, + minimum_distance_m=a.minimum_distance_m, + minimum_samples=a.minimum_samples, + preferred_samples=a.preferred_samples, + required_views=list(a.required_views), + optional_inputs=list(a.optional_inputs), + confirmation_options=list(a.confirmation_options), + escalation=a.escalation, + never_blocks=list(a.never_blocks), + reject_if=list(a.reject_if), + allowed_claims=list(a.allowed_claims), + forbidden_claims=list(a.forbidden_claims), + derived_claims=list(a.derived_claims), + allowed_evidence=list(a.evidence_phrases), + allowed_limits=list(a.limit_phrases), + ) + + +def _disposition_view(key: str) -> DispositionView | None: + disposition = dispositions.get(key) + if disposition is None: + return None + superseded = disposition.superseded + return DispositionView( + group=disposition.group.value, + summary=disposition.summary, + blocker=disposition.blocker, + stated_uncertainty=disposition.stated_uncertainty, + data_needed=disposition.data_needed, + evidence=[f"{s.claim} โ€” {s.url}" for s in disposition.evidence], + superseded=None if superseded is None else SupersededView( + verdict=superseded.verdict, + summary=superseded.summary, + reframed_because=superseded.reframed_because, + ), + ) + + +def _unavailable_reason(c: Capability) -> str: + """Why a capability cannot run today, in words a client can show a person. + + **The distinction this string used to draw has moved into `state`.** It once + had to separate "late" from "never", because six capabilities were + `coming_soon` in the registry and `not_viable` in the research, and a client + rendering `status` alone showed the same `Coming soon` for both. After + ADR 0021 nothing is `not_viable`: the claims that were rejected are rejected + as claims, in `capabilities.REJECTED_CLAIMS`, and every capability survives + in a corrected form. + + What is left to say is narrower and more useful โ€” this feature is real, the + model behind it is not installed on this deployment yet, and here is what it + is waiting for. + """ + if c.state == CapabilityState.UNSUPPORTED_CLAIM: + # **One capability holds this state**: `poultry_uniformity`, since + # ADR 0023. The branch existed so that one arriving could not be + # rendered as merely late, and the arrival showed it was half a branch. + # + # ยง39 says not to change a capability to "Not planned", and a flat + # refusal here would do exactly that to a claim that survives in another + # form. So the two cases are split. A capability whose corrected form is + # something the app computes says what that is; only a claim with + # nothing behind it at all reads as unplanned. + if c.acquisition.derived_claims: + return ( + "The capture is saved. Animap will not work this out from the " + "capture this capability declares โ€” measured against the real " + "answer it is wrong by more than the number is worth, and by " + "more birds rather than fewer. What survives is computed from " + "entered values instead: " + + ", ".join(c.acquisition.derived_claims) + + "." + ) + return ( + "This is not planned. The claim behind it is not scientifically " + "supportable in any capture protocol." + ) + # Checked before `state`, not inside a `coming_soon` branch. A capability + # waiting on another capability is waiting whatever its own state says. + # + # **Nothing declares a dependency today.** `poultry_uniformity` was the only + # one and ADR 0023 removed it, because the input it waited on is the input + # that makes its answer wrong โ€” a dependency that resolves into a refusal is + # a worse pointer than none. + if c.depends_on: + return ( + "The capture is saved. This needs " + + ", ".join(c.depends_on) + + " first โ€” there is nothing to compute until those results exist." + ) + if c.state == CapabilityState.COMING_SOON: + return "Not built yet. The engineering path exists and nobody has walked it." + if Requirement.HARDWARE_REQUIRED in c.requirements: + return "This capability needs compatible hardware." + if Requirement.FIXED_INSTALLATION in c.requirements: + return "This capability needs a permanently placed camera or microphone." + artefact = provider.artefact_for(c) + if artefact is None: + return ( + "The capture is saved. No validated model for this capability is " + "installed on this deployment yet, so the analysis is still to come." + ) + # **These two were one string, and it named the wrong thing.** The fallback + # read "The model artefact is present but has not been validated." for + # `cattle_identity`, whose artefact is present *and* validated โ€” it passes + # its checksum, `discover()` loads it, and its licence is in + # `/health.artefact_licenses`. What was missing was on the capability, not on + # the model: the registry entry named no `model_provider`, so + # `Capability.is_runnable` was False. A reason that sends somebody to inspect + # a model card when the model card is fine costs an afternoon, and + # `DEPLOY.md` had to carry a paragraph warning readers not to believe it. + if not artefact.is_validated: + return ( + "The model artefact is installed but its card attests nothing about " + "whether it works, so this deployment will not run it." + ) + return ( + "The capture is saved. The model for this capability is installed and " + "checksummed, and the capability is not switched on in this build yet." + ) diff --git a/app/media.py b/app/media.py new file mode 100644 index 0000000000000000000000000000000000000000..29535989bdcdc7f7b7521ae4c91689afea3454b4 --- /dev/null +++ b/app/media.py @@ -0,0 +1,271 @@ +"""Turning a media reference into pixels. + +An inference request names media by id, never by path or URL, because the id is +the only thing the device, the API and this service all agree on (ADR 0011). +This module is the one place that turns an id into bytes. + +Two stores, chosen by `ANIMAP_MEDIA_PROVIDER`: + +**`local`** reads a directory of `` files. It is what the tests +and the evaluation harness run against, and what a developer uses with no cloud +account. + +**`azure_blob`** reads the container the API uploads to (ADR 0012), with the +managed identity the Container App already runs as. No account key, no SAS to +mint, nothing stored โ€” the same credential-free posture as +`services/api/apps/evidence/storage.py`. + +## Finding a blob from an id + +The API writes `farm/{farm_id}/{yyyy}/{mm}/{asset_id}.{ext}`, partitioned by the +month of **upload**, and an inference request carries the month of **capture**. +Those are usually the same month and are not guaranteed to be: a capture made +offline on the 31st can upload on the 1st, which is exactly the case an +offline-first product has to survive. + +So resolution is two-stage. It probes the handful of paths the convention +predicts, which answers in one HEAD for almost every request, and falls back to +listing the farm's prefix, which is slower and always right. Neither stage +guesses: a blob that is not there produces `MediaError`, never a substitute. + +The API sends the `object_path` it already stores, as `media_paths` positionally +beside `media_ids` (`services/api/apps/evidence/inference/queue.py`), so a +request from the API skips both stages โ€” `MediaRef.object_path` takes it and +issues one `GET`. **This paragraph used to say the API did not call this service +at all, which stopped being true and stayed here.** + +The two stages still run for any caller that sends no path, so the convention is +still a contract, and it is asserted in `tests/test_media.py` against the path +builder in `services/api`. +""" + +from __future__ import annotations + +import io +import os +from dataclasses import dataclass +from datetime import datetime, timedelta, timezone +from pathlib import Path +from uuid import UUID + +from PIL import Image, UnidentifiedImageError + +DEFAULT_ROOT = Path(__file__).resolve().parent.parent / "media" + +#: A capture from a phone is a few megabytes. Anything an order of magnitude +#: past that is a mistake or an attack, and decoding it first is the wrong way +#: to find out. +MAX_BYTES = 40 * 1024 * 1024 + +_SUFFIXES = (".jpg", ".jpeg", ".png", ".webp") + +#: How far either side of the capture month to probe before falling back to a +#: listing. One month covers an upload that crossed a boundary; more than that +#: is a device that was offline for weeks, which the listing handles. +_MONTH_WINDOW = 1 + + +class MediaError(RuntimeError): + """The media could not be found, read, or decoded.""" + + +@dataclass(frozen=True) +class MediaRef: + """A media id and whatever is known about where it was filed. + + `farm_id` and `captured_at` are what make a blob lookup a probe rather than a + scan, and both are already on every inference request. `object_path` is the + exact answer when a caller has it. + """ + + media_id: UUID + farm_id: UUID | None = None + captured_at: datetime | None = None + object_path: str | None = None + + +def _decode(data: bytes, ref: MediaRef) -> Image.Image: + if len(data) > MAX_BYTES: + raise MediaError( + f"{ref.media_id} is {len(data) / 1e6:.0f} MB, past the " + f"{MAX_BYTES / 1e6:.0f} MB limit for a single capture." + ) + try: + image = Image.open(io.BytesIO(data)) + image.load() + except (UnidentifiedImageError, OSError) as exc: + raise MediaError(f"{ref.media_id} is not a readable image: {exc}") from exc + return image + + +class LocalMediaStore: + """Resolves media ids under one root directory. + + Files are named ``. The id comes from the request, so the + lookup is by exact stem and never by a caller-supplied path fragment โ€” a + request must not be able to reach outside the root by asking nicely. + """ + + provider = "local" + + def __init__(self, root: Path | None = None) -> None: + if root is not None: + self.root = Path(root) + else: + self.root = Path(os.environ.get("ANIMAP_MEDIA_ROOT", DEFAULT_ROOT)) + + def path_for(self, media_id: UUID) -> Path: + for suffix in _SUFFIXES: + candidate = self.root / f"{media_id}{suffix}" + if candidate.is_file(): + return candidate + raise MediaError( + f"No media file for {media_id} under {self.root}. The capture may not " + f"have finished uploading yet." + ) + + def open_image(self, ref: MediaRef | UUID) -> Image.Image: + ref = ref if isinstance(ref, MediaRef) else MediaRef(media_id=ref) + path = self.path_for(ref.media_id) + size = path.stat().st_size + if size > MAX_BYTES: + raise MediaError( + f"{ref.media_id} is {size / 1e6:.0f} MB, past the " + f"{MAX_BYTES / 1e6:.0f} MB limit for a single capture." + ) + return _decode(path.read_bytes(), ref) + + +def candidate_paths(ref: MediaRef) -> list[str]: + """The blob paths the API's convention predicts for this reference. + + Ordered so the likeliest is first: the capture month, then its neighbours. + Empty when there is not enough on the request to predict anything, which + sends the caller straight to the listing. + """ + if ref.farm_id is None: + return [] + + when = ref.captured_at or datetime.now(timezone.utc) + months = [] + for offset in range(-_MONTH_WINDOW, _MONTH_WINDOW + 1): + # Day 15 keeps the arithmetic inside the month whatever its length, so a + # capture on the 31st does not skip February. + anchor = when.replace(day=15) + timedelta(days=31 * offset) + months.append((anchor.year, anchor.month)) + + ordered = sorted(set(months), key=lambda m: abs( + (m[0] - when.year) * 12 + m[1] - when.month + )) + return [ + f"farm/{ref.farm_id}/{year:04d}/{month:02d}/{ref.media_id}{suffix}" + for year, month in ordered + for suffix in _SUFFIXES + ] + + +class AzureBlobMediaStore: + """Reads the container the API uploads captures to. + + The credential is `DefaultAzureCredential`, which resolves to the Container + App's managed identity in production and to the developer's `az` login + locally โ€” the same code path in both, and no secret in either. + """ + + provider = "azure_blob" + + def __init__(self, account: str | None = None, container: str | None = None) -> None: + from azure.identity import DefaultAzureCredential + from azure.storage.blob import BlobServiceClient + + self.account = account or os.environ.get("AZURE_STORAGE_ACCOUNT", "") + self.container = container or os.environ.get("AZURE_STORAGE_CONTAINER", "media") + if not self.account: + raise MediaError( + "AZURE_STORAGE_ACCOUNT is not set, so there is no container to " + "read captures from." + ) + self.endpoint = f"https://{self.account}.blob.core.windows.net" + self.client = BlobServiceClient( + self.endpoint, credential=DefaultAzureCredential() + ) + + def _download(self, path: str) -> bytes | None: + from azure.core.exceptions import ResourceNotFoundError + + blob = self.client.get_blob_client(self.container, path) + try: + properties = blob.get_blob_properties() + # Checked before the body is pulled, so an oversized object costs one + # HEAD rather than a 400 MB download into this process's memory. + if properties.size > MAX_BYTES: + raise MediaError( + f"{path} is {properties.size / 1e6:.0f} MB, past the " + f"{MAX_BYTES / 1e6:.0f} MB limit for a single capture." + ) + return blob.download_blob().readall() + except ResourceNotFoundError: + return None + + def _by_listing(self, ref: MediaRef) -> bytes | None: + """The slow, always-correct path: scan the farm's own prefix. + + Scoped to one farm so a lookup never enumerates another tenant's media, + and so the cost is bounded by one farm's history rather than the + container's. + """ + if ref.farm_id is None: + return None + container = self.client.get_container_client(self.container) + stems = {f"{ref.media_id}{suffix}" for suffix in _SUFFIXES} + for blob in container.list_blobs(name_starts_with=f"farm/{ref.farm_id}/"): + if blob.name.rsplit("/", 1)[-1] in stems: + return self._download(blob.name) + return None + + def open_image(self, ref: MediaRef | UUID) -> Image.Image: + ref = ref if isinstance(ref, MediaRef) else MediaRef(media_id=ref) + + if ref.object_path: + data = self._download(ref.object_path) + if data is None: + raise MediaError( + f"No blob at {ref.object_path} in {self.container}." + ) + return _decode(data, ref) + + for path in candidate_paths(ref): + data = self._download(path) + if data is not None: + return _decode(data, ref) + + data = self._by_listing(ref) + if data is None: + raise MediaError( + f"No media object for {ref.media_id} under farm/{ref.farm_id}/ in " + f"{self.account}/{self.container}. The capture may not have " + f"finished uploading yet." + ) + return _decode(data, ref) + + +def build_store(): + """The store this deployment is configured for. + + Defaults to `local`, so a developer who sets nothing gets the filesystem + rather than an authentication error against a storage account they may not + have. + """ + provider = os.environ.get("ANIMAP_MEDIA_PROVIDER", "local") + if provider == "azure_blob": + return AzureBlobMediaStore() + if provider == "local": + return LocalMediaStore() + raise MediaError( + f"No media provider named {provider!r}. Use `local` or `azure_blob`." + ) + + +#: Kept as the name the rest of the service imports, so a call site says what it +#: means rather than which cloud is behind it. +MediaStore = LocalMediaStore diff --git a/app/providers.py b/app/providers.py new file mode 100644 index 0000000000000000000000000000000000000000..d4b44706603566801f48abddb47f34a816f070a9 --- /dev/null +++ b/app/providers.py @@ -0,0 +1,701 @@ +"""Model adapters, and the governance around them. + +ADR 0005 and brief ยง30. A model file is only usable once it can account for +itself: explicit source, licence, checksum, model card, version, and validation +notes. Nothing is downloaded at runtime. + +This is not bureaucracy. A livestock record that informs a treatment or a sale +has to be reconstructable months later โ€” "which model said this animal weighed +386 kg, and had anyone checked it worked in Nigeria?" is a question the system +must be able to answer, and it cannot if artefacts arrive anonymously. +""" + +from __future__ import annotations + +import hashlib +import json +import logging +import os +from dataclasses import dataclass +from pathlib import Path + +from app.adapters import fingerprints, licence_policy, licences +from app.capabilities import UNVALIDATED_GEOGRAPHY, Capability + +logger = logging.getLogger(__name__) + +#: Read in blocks rather than whole. `yolox_x` is 396 MB and this runs on a +#: 2 vCPU / 4 GiB container at start-up, next to onnxruntime's arenas. +_DIGEST_CHUNK = 1 << 20 + +MODELS_DIR = Path(__file__).resolve().parent.parent / "models" + +#: Every field a model card must carry before its artefact may be loaded. +REQUIRED_CARD_FIELDS = ( + "model_id", + "version", + "capability_key", + "source", + "license", + "sha256", + "validation_notes", + "geography_validated", +) + + +class ArtefactError(RuntimeError): + """An artefact is missing, unaccounted for, or does not match its card.""" + + +#: Licences Animap may not serve from. Copyleft that reaches a network service, +#: and non-commercial terms that a commercial product cannot satisfy at all. +#: +#: ADR 0017 records what AGPL-3.0 actually says, which is narrower than this +#: service used to claim: ยง13's source-offer duty is conditioned on *modifying* +#: the program, and ยง0 puts network interaction outside "convey". The reason +#: these are refused anyway is not that breach is certain โ€” it is that the +#: vendor's published position asserts it, the question is unsettled, and the +#: measured cost of avoiding it is roughly nil. +#: +#: The list is broader than AGPL because the failure it guards against is "an +#: artefact whose terms this product cannot meet", and a Non-Commercial dataset +#: licence fails that just as completely. The Bristol cattle sets are the live +#: example (`app/dispositions.py`). +DISALLOWED_LICENSES = frozenset({ + "AGPL-3.0", "AGPL-3.0-only", "AGPL-3.0-or-later", + "GPL-3.0", "GPL-3.0-only", "GPL-3.0-or-later", "GPL-2.0", + "SSPL-1.0", + "CC-BY-NC-4.0", "CC-BY-NC-SA-4.0", "CC BY-NC 4.0", + "Non-Commercial Government Licence", +}) + +#: Kept as the old name because `is_copyleft` reads better at the call site and +#: the AGPL case is still the one this exists for. +NETWORK_COPYLEFT_LICENSES = DISALLOWED_LICENSES + +#: Runtimes that can only load an artefact whose terms Animap may not serve +#: under, whatever the card claims about it. +#: +#: **This used to be described as "the check that matters", and it was not.** A +#: card declares its own licence, so the named threat โ€” somebody editing a card +#: โ€” is exactly the case a self-declaration cannot catch: write +#: `"license": "Apache-2.0"` over a path to `yolo11m.pt` and the licence gate +#: waves it through. The fix was to read `runtime` instead. But `runtime` is +#: also a card field, read eleven lines below a paragraph arguing that the card +#: cannot be trusted, so editing `runtime` rather than `license` defeated it in +#: one line and served AGPL weights as `cattle_identity` under Apache-2.0. +#: +#: **What actually binds a licence to an artefact is +#: `app/adapters/fingerprints.py`**, which reads the file's own structure and +#: says which model it holds. `load_card` runs it, `ModelArtefact.effective_runtime` +#: prefers what the bytes say over what the card claims, and every licence +#: decision below keys off that. This frozenset and the table in +#: `app/adapters/licences.py` both still run, because a layered refusal costs +#: nothing and one of the three may be edited wrongly. +#: +#: The adapter stays in the tree for `evaluation/`, which is the evidence behind +#: ADR 0017. It is unreachable from `discover()`, which is the only path that +#: answers a farmer's request. +DISALLOWED_RUNTIMES = frozenset({"ultralytics"}) + + +@dataclass(frozen=True) +class ModelArtefact: + model_id: str + version: str + capability_key: str + source: str + license: str + sha256: str + validation_notes: str + geography_validated: list[str] + path: Path + #: Which adapter reads this file, **as the card claims**. Optional on the + #: card because governance cares about provenance and the runtime is an + #: implementation detail โ€” but a card without it cannot be run, only + #: inspected. Read `effective_runtime` for the answer that is not a claim. + runtime: str | None = None + #: What the artefact's own bytes say it is. `None` when `load_card` was + #: given no file to read, which happens only in tests. + identity: fingerprints.Identification | None = None + #: How an embedding backbone's input is prepared, **read off the card rather + #: than hard-coded at the call site**. + #: + #: `backbones.preprocess` writes out torchvision's eval transform, and its + #: own docstring says getting it wrong "does not raise โ€” it quietly returns + #: worse vectors". A resize or a normalisation constant that disagrees with + #: the export produces vectors of the right width and the wrong meaning, + #: which is the same silent failure `IdentityIndex` pins the backbone id to + #: catch. So the numbers travel with the artefact they belong to. + #: + #: `None` for every artefact that is not an embedding backbone โ€” YOLOX cards + #: carry none of these โ€” and `OnnxEmbeddingAdapter` keeps its own defaults for + #: that case. `load()` shape-checks the result against the graph either way, + #: so a card that disagrees with its own `.onnx` fails loudly. + input_size: int | None = None + image_mean: tuple[float, float, float] | None = None + image_std: tuple[float, float, float] | None = None + embedding_dimensions: int | None = None + + @property + def observed_runtime(self) -> str | None: + """The runtime the bytes identify, or `None` if they identify none.""" + return self.identity.runtime if self.identity else None + + @property + def refuted_by(self) -> list[str]: + """Which properties of its own claimed runtime this file fails. + + Empty unless the card names a runtime that **has** a fingerprint of the + kind the file was read as, and the bytes fail it. See + `fingerprints.Identification.refutes` for why that case had to be split + out of "unidentified". + """ + if self.identity is None: + return [] + return self.identity.refutes(self.runtime) + + @property + def effective_runtime(self) -> str | None: + """Which loader really runs, preferring evidence over assertion. + + **This is the field every licence decision should read.** The card's + `runtime` is a claim by whoever wrote the card; the fingerprint is a + property of the bytes. When they disagree the bytes win, because the + bytes are what onnxruntime will load and what a farmer's result will + come out of. + + Falls back to the card when nothing recognised the file. That is not a + weakening โ€” an unfingerprinted artefact's identity rested on its card + before this existed and still does, and `identity_conflict` reports the + difference between *"checked and agreed"* and *"not checked"*. + + **It does not fall back to a claim the bytes refute.** That fallback was + the hole: `observed_runtime or self.runtime` treats *"no fingerprint + matched"* as *"nothing is known"*, so an Ultralytics export with its + initializers renamed inherited its card's `yolox-onnx` and every licence + decision below was answered about YOLOX. `None` here is what makes + `licences.gate` refuse it, under both policies, as an integrity failure + rather than a licence one โ€” which is the distinction that module's own + docstring draws and the reason the founder's "drop nothing for a licence + right now" instruction does not reach this case. + """ + if self.observed_runtime: + return self.observed_runtime + return None if self.refuted_by else self.runtime + + @property + def identity_conflict(self) -> str: + """Why the bytes and the card disagree, or an empty string. + + Empty covers two different states on purpose โ€” agreement, and nothing + to compare โ€” because a caller is deciding whether to refuse and both + answers are "do not refuse on this". `identity.detail` is where the + difference is legible. + + There are two ways to disagree and they read differently. Either the + bytes matched some *other* fingerprint, and the report can name what the + file actually is; or nothing matched at all but the card's own claim was + testable and failed, and the report can only say what the file is not. + The second is still a conflict, and it is the one that was missing. + + **Naming what the file is comes first**, because it is the stronger + statement and the one a licence can be attached to. Only when nothing + matched does the report fall back to saying what was disproved. + """ + observed = self.observed_runtime + if observed is not None and self.runtime and observed != self.runtime: + declared_licence = licences.RUNTIME_LICENCES.get(self.runtime) + real_licence = licences.RUNTIME_LICENCES.get(observed) + return ( + f"The card says runtime {self.runtime!r} " + f"({declared_licence.licence if declared_licence else 'unrecorded'}), " + f"but {self.path.name} is structurally {observed!r} " + f"({real_licence.licence if real_licence else 'unrecorded'}). " + f"{self.identity.detail if self.identity else ''} A sha256 proves " + f"these are the reviewed bytes; it does not prove what the bytes " + f"are, and this does." + ) + refuted = self.refuted_by + if observed is None and refuted: + declared_licence = licences.RUNTIME_LICENCES.get(self.runtime or "") + return ( + f"The card says runtime {self.runtime!r} " + f"({declared_licence.licence if declared_licence else 'unrecorded'}), " + f"and {self.path.name} fails {len(refuted)} of that runtime's " + f"structural properties: {'; '.join(refuted)}. No other " + f"fingerprint matched either, so what this file *is* is unknown " + f"โ€” but the card's claim about it has been disproved, and an " + f"artefact nobody can identify may not be served under a licence " + f"borrowed from a model it is not." + ) + return "" + + @property + def is_copyleft(self) -> bool: + """Whether the card's *declared* licence is one Animap may not serve. + + A self-declaration, and therefore not a control on its own โ€” see + `effective_runtime` for the answer that does not depend on the card + telling the truth. + """ + return self.license in DISALLOWED_LICENSES + + @property + def uses_disallowed_runtime(self) -> bool: + """Keyed to the observed runtime, so editing the card does not clear it. + + This is the check ADR 0017 added and the one the original exploit was + aimed at. Reading `effective_runtime` rather than `runtime` is what + makes it hold: a card claiming `dinov3-onnx` over `yolo11m.pt` is + `ultralytics` here, because the file's pickle names + `ultralytics.nn.tasks.DetectionModel` and nothing else can load it. + """ + return (self.effective_runtime or "") in DISALLOWED_RUNTIMES + + @property + def is_validated(self) -> bool: + """Validation notes must say something. + + An empty string is the tell that a card was filled in to get past the + loader rather than because anyone checked the model. + """ + return bool(self.validation_notes.strip()) + + +def load_card(card_path: Path) -> ModelArtefact: + card = json.loads(card_path.read_text()) + + missing = [f for f in REQUIRED_CARD_FIELDS if not card.get(f)] + if missing: + raise ArtefactError( + f"{card_path.name} is missing required fields: {', '.join(missing)}. " + f"An artefact that cannot account for itself is not usable." + ) + + artefact_path = (card_path.parent / card.get("artefact", "")).resolve() + + # Cards may point sideways โ€” several capabilities share one detector, and + # storing the file twice would mean two things to keep in step. They may not + # point *out*, because a card is a reviewed artefact reference and not a way + # to load whatever is on the box. + root = card_path.parent.parent.resolve() + if not artefact_path.is_relative_to(root): + raise ArtefactError( + f"{card_path.name} names an artefact outside {root.name}/." + ) + + if not artefact_path.is_file(): + raise ArtefactError(f"{card_path.name} names an artefact that is not present.") + + digest = _digest(artefact_path) + if digest != card["sha256"]: + raise ArtefactError( + f"{artefact_path.name} does not match the checksum on its card. " + f"Expected {card['sha256'][:12]}โ€ฆ, found {digest[:12]}โ€ฆ" + ) + + # Read *after* the checksum, because there is no point asking what a file is + # when it is not the file that was reviewed. A structure this does not + # recognise is recorded as unidentified and is not an error โ€” see + # `fingerprints.identify`. + identity = fingerprints.identify(artefact_path) + + return ModelArtefact( + model_id=card["model_id"], + version=card["version"], + capability_key=card["capability_key"], + source=card["source"], + license=card["license"], + sha256=card["sha256"], + validation_notes=card["validation_notes"], + geography_validated=list(card["geography_validated"]), + path=artefact_path, + runtime=card.get("runtime"), + identity=identity, + input_size=card.get("input_size"), + image_mean=_triple(card.get("image_mean")), + image_std=_triple(card.get("image_std")), + embedding_dimensions=card.get("embedding_dimensions"), + ) + + +def _triple(value) -> tuple[float, float, float] | None: + """A three-channel normalisation constant, or `None`. + + Refuses a malformed one rather than passing it through, because a mean of + the wrong length reaches `backbones.preprocess` as a numpy broadcast error + at request time โ€” out of a farmer's capture โ€” instead of at start-up where + the card is being read. + """ + if value is None: + return None + if not isinstance(value, (list, tuple)) or len(value) != 3: + raise ArtefactError( + f"An image_mean or image_std must be three numbers, one per " + f"channel, and this card carries {value!r}." + ) + return tuple(float(v) for v in value) + + +def _digest(path: Path) -> str: + sha = hashlib.sha256() + with path.open("rb") as handle: + while chunk := handle.read(_DIGEST_CHUNK): + sha.update(chunk) + return sha.hexdigest() + + +def _enforcing() -> bool: + """Whether a licence problem stops a load right now. + + Every licence refusal in `discover()` routes through this, so the posture is + one setting โ€” `ANIMAP_LICENCE_POLICY`, defined in + `app/adapters/licence_policy.py` โ€” and not three independent checks that + have to be found and changed separately. + """ + return licence_policy.current() is licence_policy.LicencePolicy.ENFORCE + + +def _true_licence(artefact: ModelArtefact) -> str: + """The licence to write in the ledger, which is not always the card's. + + When the bytes identify a runtime, the runtime table's licence is the fact + and the card's `license` is a claim about it. Recording the claim would put + `DINOv3 License` beside a runtime of `ultralytics`, which reads as a + contradiction a person has to unpick rather than as a finding. + """ + real = licences.RUNTIME_LICENCES.get(artefact.effective_runtime or "") + return real.licence if real else artefact.license + + +def _record_identity(artefact: ModelArtefact, what: str) -> bool: + """Handle a disagreement between an artefact's bytes and its card. + + Returns True when the load should stop. Under the default policy it never + does โ€” the founder's instruction is that nothing is dropped on a governance + question right now โ€” and what the caller gets instead is a ledger entry + naming the licence the *bytes* arrive under. + + **That substitution is the point of the whole check.** Before it, a card + editing `runtime` from `ultralytics` to `dinov3-onnx` produced a load that + was recorded, if it was recorded at all, as a bespoke Meta licence. It is + AGPL-3.0. A ledger that answers *"what did we ship that we should not + have?"* with the wrong licence is worse than no ledger, because somebody + will believe it. + """ + conflict = artefact.identity_conflict + if not conflict: + if artefact.identity is not None and not artefact.identity.identified: + # Not a failure and not a pass. Logged at INFO because a listing of + # every unfingerprinted model is a to-do list, not an incident. + logger.info( + "%s: %s", what, + artefact.identity.detail, + ) + return False + + if artefact.observed_runtime is None and artefact.refuted_by: + # **Stops the load under both identity policies, and that is not the + # founder's "drop nothing" instruction being ignored.** `record` mode + # exists so a governance exception stays findable afterwards, and its + # whole value is that the entry it writes is true. Here nothing matched, + # so there is no licence to name: the only honest record would say + # "loaded something we cannot identify under terms we cannot name", + # which is not an exception anyone can act on. Refusing costs one + # capability and a log line that says exactly what to fix. + logger.error("Refusing %s: %s", what, conflict) + return True + + real = licences.RUNTIME_LICENCES.get(artefact.observed_runtime or "") + refuse = ( + licence_policy.identity_policy() is licence_policy.IdentityPolicy.REFUSE + ) + if refuse: + logger.error("Refusing %s: %s", what, conflict) + return True + + licence_policy.record_exception( + what=what, + # Recorded under what the file *is*, never under what it claimed. + runtime=artefact.observed_runtime or "", + licence=real.licence if real else "UNRECORDED", + reason=conflict, + source_url=real.source_url if real else "", + ) + return False + + +def discover() -> dict[str, ModelArtefact]: + """Every validated artefact currently present, keyed by capability. + + A card that fails its checks is logged and skipped rather than raised โ€” one + bad artefact must not take the whole service down, and the capability it + belongs to simply stays unavailable, which is the honest outcome. + + **Every licence decision below reads `effective_runtime`**, which prefers + what the bytes are over what the card says they are. That is the difference + between this and every previous version of the gate. + """ + found: dict[str, ModelArtefact] = {} + if not MODELS_DIR.is_dir(): + return found + + for card_path in sorted(MODELS_DIR.glob("*/model_card.json")): + try: + artefact = load_card(card_path) + except (ArtefactError, json.JSONDecodeError) as exc: + logger.error("Ignoring %s: %s", card_path.parent.name, exc) + continue + if not artefact.is_validated: + logger.error( + "Ignoring %s: validation_notes is empty, so nothing attests that " + "this model works.", card_path.parent.name, + ) + continue + if _record_identity( + artefact, f"{card_path.parent.name}/{artefact.model_id}" + ): + continue + if artefact.is_copyleft: + # Refused under the default policy, recorded under `record`. The + # capability going unavailable is noticed; a log line on a Tuesday + # is not (ADR 0017) โ€” which is why `record` mode logs at ERROR and + # keeps the fact rather than merely warning. + message = ( + f"{artefact.model_id} declares licence {artefact.license}, " + f"which Animap may not serve from." + ) + if _enforcing(): + logger.error( + "Refusing %s: %s Move the capability to a permissively " + "licensed artefact.", card_path.parent.name, message, + ) + continue + licence_policy.record_exception( + what=f"{card_path.parent.name}/{artefact.model_id}", + runtime=artefact.effective_runtime or "", licence=artefact.license, + reason=message, + ) + if artefact.uses_disallowed_runtime and _enforcing(): + # The check the licence field cannot make, keyed to what the bytes + # are. A card that declares Apache-2.0 over a path to an AGPL + # checkpoint passes the test above; a card that also rewrites + # `runtime` used to pass this one, and now does not, because + # `uses_disallowed_runtime` reads `effective_runtime`. + logger.error( + "Refusing %s: runtime %r may not be served, whatever the card " + "declares about the licence (ADR 0017). It exists for " + "evaluation/ only.", + card_path.parent.name, artefact.effective_runtime, + ) + continue + if artefact.uses_disallowed_runtime: + licence_policy.record_exception( + what=f"{card_path.parent.name}/{artefact.model_id}", + runtime=artefact.effective_runtime or "", + licence=_true_licence(artefact), + reason="Runtime is on the hard-coded refusal list (ADR 0017).", + ) + + # The same idea, generalised and applied to every runtime rather than to + # one name. `licences.gate` refuses three things the checks above do not: + # a runtime nobody has recorded a licence for, a runtime whose real + # licence is one Animap may not serve, and โ€” the new one โ€” a card whose + # declared licence *disagrees* with what that runtime actually loads. + # + # It is handed `effective_runtime`, so on a mislabelled artefact it is + # asked about the model that is really there. That also means the + # card-versus-runtime disagreement it raises on becomes a three-way + # check: the card's licence, the card's runtime, and the bytes. + try: + licences.gate( + artefact.effective_runtime, artefact.license, + what=f"{card_path.parent.name}/{artefact.model_id}", + ) + except licences.LicenceRefused as refusal: + logger.error("Refusing %s: %s", card_path.parent.name, refusal) + continue + + found[artefact.capability_key] = artefact + return found + + +class InferenceProvider: + """Runs a capability, or explains why it cannot. + + `run` is deliberately absent from the base class. A provider with a default + implementation would be a provider that can return something without a model, + and that is the one thing this service must not do. + """ + + def can_run(self, capability: Capability) -> bool: + raise NotImplementedError + + +class LocalArtefactProvider(InferenceProvider): + """Serves capabilities backed by a validated artefact on disk. + + Two capabilities have one as of ADR 0018 โ€” `cattle_detection` and + `poultry_count`, both YOLOX-m. The other nineteen have none, and `can_run` + returning False for them is the correct state rather than a gap. + """ + + def __init__(self, artefacts: dict[str, ModelArtefact] | None = None): + self.artefacts = artefacts if artefacts is not None else discover() + + def can_run(self, capability: Capability) -> bool: + artefact = self.artefacts.get(capability.key) + return artefact is not None and artefact.is_validated and capability.is_runnable + + def artefact_for(self, capability: Capability) -> ModelArtefact | None: + return self.artefacts.get(capability.key) + + +class HostedModelProvider(InferenceProvider): + """Serves capabilities whose model is somebody else's, reached over HTTP. + + ## Why this is a provider and not a special case in `LocalArtefactProvider` + + `can_run` there asks one question โ€” is there a validated file on disk โ€” and + for these sixteen the answer is permanently no, and correctly so. There is + no artefact to checksum because there are no bytes: directive ยง4 asks for a + hosted visual reasoner precisely where no public dataset exists to train + one. Bending the local provider to answer True without a file would make + `is_validated` mean two different things depending on the capability, and + that check is the thing standing between a card somebody filled in and a + model somebody validated. + + So the two providers answer for disjoint sets and `main` asks both. + + ## The artefact this returns, and why it names the live model + + `InferenceResult` requires `model_id` and `model_version`, and the API + refuses a result whose pair does not equal the run's โ€” *"register a release + for the artefact that actually ran"*. A hosted model has no sha256, so what + identifies it is its name, and that is read **at call time** from + `ANIMAP_MULTIMODAL_MODEL` rather than frozen at import. + + That has a consequence worth stating plainly: **rotating the model refuses + every queued run of these capabilities until a release is registered for the + new one.** That is the correct behaviour and not a hazard. A different model + is different evidence, `evidence_correction` is a training set built on + knowing which model said what, and a silent swap would put two models' + answers in one column. Re-registering is a management command โ€” neither a + rebuild nor a redeploy โ€” so a rotation still costs a restart and a + `register_capability_release`, exactly as `multimodal.py` intends. + + `capability.model_provider` stays the stable string `hosted-multimodal`. It + is what makes `is_runnable` true and what the Android app copies out of + `capabilities.json`, and it must not change when a model does โ€” a device + would otherwise need a rebuild to keep a capability visible. + """ + + #: What `sha256` says when there are no bytes to hash. + #: + #: Not an empty string and not a fake digest. A reader of a stored result + #: asking "which file was this" gets an answer that says there was no file, + #: rather than a blank that reads as a missing field or sixty-four zeroes + #: that read as a checksum. + NO_LOCAL_BYTES = "hosted:no-local-artefact" + + def __init__(self, keys: frozenset[str] | None = None): + # Imported here rather than at module scope: `reasoning` imports + # `providers` for `ModelArtefact`, and the pair would not load. + if keys is None: + from app.reasoning import RUBRICS + keys = frozenset(RUBRICS) + self.keys = keys + + def _configured_model(self) -> str | None: + provider = os.environ.get("ANIMAP_MULTIMODAL_PROVIDER", "").strip() + model = os.environ.get("ANIMAP_MULTIMODAL_MODEL", "").strip() + if not provider or not model: + return None + return f"{provider}/{model}" + + def can_run(self, capability: Capability) -> bool: + """True when this capability is hosted, declared runnable, and reachable. + + The third clause is the one that keeps `/health` honest. Without a key + the adapter is not ready and every one of these answers `unavailable` + with a reason, which is a deployment that says what it is missing rather + than one that fails sixteen jobs to find out. + """ + from app.adapters.multimodal import HostedMultimodalAdapter + from app.adapters.transports import transport_from_env + from app.reasoning import HOSTED_PROVIDER + + if capability.key not in self.keys: + return False + if capability.model_provider != HOSTED_PROVIDER: + return False + if not capability.is_runnable: + return False + if self._configured_model() is None: + return False + # Built the way `adapters/registry.py` builds it, so `/health`'s + # listing and this answer cannot disagree about whether the vendor is + # reachable. A bare adapter has no transport and is never ready. + return HostedMultimodalAdapter( + transport=transport_from_env() + ).availability().ready + + def artefact_for(self, capability: Capability) -> ModelArtefact | None: + model = self._configured_model() + if capability.key not in self.keys or model is None: + return None + return ModelArtefact( + model_id=model, + # The rubric's version, not the vendor's. A materially changed + # question is different evidence from the same model, and this is + # what a release row records it as. Bumped by hand in `reasoning`. + version="1", + capability_key=capability.key, + source="hosted", + license="vendor-terms", + sha256=self.NO_LOCAL_BYTES, + validation_notes=( + "No local artefact and no measurement. This capability is " + "answered by a hosted visual reasoner, is `experimental`, and " + "requires human confirmation โ€” see app/reasoning.py." + ), + geography_validated=[UNVALIDATED_GEOGRAPHY], + path=Path("hosted"), + runtime="http", + ) + + +class CompositeProvider(InferenceProvider): + """Every provider, asked in order, first answer wins. + + The two sets are disjoint โ€” one is capabilities with a file, the other is + capabilities that can never have one โ€” so "first answer wins" resolves + nothing today and is a rule rather than a tiebreak. `_OVERLAP` in `main` + refuses a collision on the runner side for the same reason. + """ + + def __init__(self, *providers: InferenceProvider): + self.providers = providers + + @property + def artefacts(self) -> dict[str, ModelArtefact]: + """Only the ones on disk. `/health` reports this as `artefacts_loaded`, + and a hosted model is not an artefact anybody loaded.""" + merged: dict[str, ModelArtefact] = {} + for provider in self.providers: + merged.update(getattr(provider, "artefacts", {})) + return merged + + def can_run(self, capability: Capability) -> bool: + return any(p.can_run(capability) for p in self.providers) + + def artefact_for(self, capability: Capability) -> ModelArtefact | None: + for provider in self.providers: + if provider.can_run(capability): + return provider.artefact_for(capability) + for provider in self.providers: + found = provider.artefact_for(capability) + if found is not None: + return found + return None diff --git a/app/quality.py b/app/quality.py new file mode 100644 index 0000000000000000000000000000000000000000..03eca8a79648a3b791ea579f4cd7b7a6bc9cec05 --- /dev/null +++ b/app/quality.py @@ -0,0 +1,147 @@ +"""The quality gate. + +A capture that is too dark, too blurred or too small does not produce a wrong +answer โ€” it produces a confident wrong answer, which is worse. The gate runs +before the model and decides one of three things: + +- **pass** โ€” run the model, report the confidence the model gives. +- **degrade** โ€” run the model, but the result is capped at `low` confidence and + the device is asked for a better frame. Usable evidence, not trustworthy + evidence. +- **block** โ€” do not run the model at all. Nothing a detector says about a black + rectangle is worth storing. + +Every threshold below came from measuring the committed fixtures and +deliberately degraded copies of them. The numbers are in the comments so the +next person can re-derive them instead of guessing what "too dark" meant. +""" + +from __future__ import annotations + +from dataclasses import dataclass + +import numpy as np +from PIL import Image + +from app.schemas import QualityCheck + +#: The metric image is downscaled to this before anything is measured. Laplacian +#: variance scales with resolution, so a 2,675 px barn photo and a 396 px yard +#: photo are otherwise not comparable and the blur threshold means nothing. +_METRIC_SIDE = 512 + +#: Mean luma. Measured: the three fixtures sit at 87โ€“136; the same images at 20% +#: brightness sit at 17โ€“27. A frame below this has lost the shadow detail a +#: detector needs on dark-coated cattle. +MIN_MEAN_LUMA = 35.0 + +#: Share of pixels at or above 250. Measured: fixtures 0.000โ€“0.011, the same +#: images at 3ร— brightness 0.45โ€“0.49. Mean luma alone does not catch a blown +#: frame, because a bright sky plus a dark animal averages out to nothing. +MAX_BLOWN_FRACTION = 0.25 + +#: Variance of the Laplacian, the standard focus proxy. Measured: fixtures +#: 584โ€“2,192; the same images under a 6 px Gaussian blur 1.9โ€“20.9. The gap is +#: wide enough that this threshold does not need to be precise. +MIN_FOCUS_VARIANCE = 80.0 + +#: Short side in pixels. Below this the detector's 640 px input is upscaling +#: more than it is reading, so the result is a soft failure rather than a hard +#: one: still worth running, not worth trusting. +MIN_SHORT_SIDE = 480 + +#: Below this there is nothing to detect at any distance, and running the model +#: is a waste of a result row. +UNUSABLE_SHORT_SIDE = 120 + + +@dataclass(frozen=True) +class QualityVerdict: + checks: list[QualityCheck] + #: No model may run. The capture is still saved; only the interpretation is + #: withheld. + blocked: bool + #: The model may run but the result must not be presented as reliable. + degraded: bool + + @property + def first_failure(self) -> QualityCheck | None: + return next((c for c in self.checks if not c.passed), None) + + +def _metrics(image: Image.Image) -> tuple[float, float, float]: + grey = image.convert("L") + grey.thumbnail((_METRIC_SIDE, _METRIC_SIDE), Image.BILINEAR) + array = np.asarray(grey, dtype=np.float32) + + mean_luma = float(array.mean()) + blown = float((array >= 250.0).mean()) + + # 4-neighbour Laplacian by slicing. A convolution library would be a + # dependency for four additions. + laplacian = ( + -4.0 * array[1:-1, 1:-1] + + array[:-2, 1:-1] + + array[2:, 1:-1] + + array[1:-1, :-2] + + array[1:-1, 2:] + ) + return mean_luma, blown, float(laplacian.var()) + + +def assess(image: Image.Image) -> QualityVerdict: + """Judge one frame. + + The `detail` strings are shown to a worker standing in a paddock, so they + say what to do differently rather than reporting a metric. + """ + width, height = image.size + short_side = min(width, height) + mean_luma, blown, focus = _metrics(image) + + checks: list[QualityCheck] = [] + blocked = False + degraded = False + + if short_side < UNUSABLE_SHORT_SIDE: + checks.append(QualityCheck( + check="resolution", passed=False, + detail=f"The frame is {width}ร—{height}. There is not enough of it to read.", + )) + blocked = True + elif short_side < MIN_SHORT_SIDE: + checks.append(QualityCheck( + check="resolution", passed=False, + detail=f"The frame is {width}ร—{height}. Capture at a higher resolution " + f"for a result you can rely on.", + )) + degraded = True + else: + checks.append(QualityCheck(check="resolution", passed=True)) + + if mean_luma < MIN_MEAN_LUMA: + checks.append(QualityCheck( + check="illumination", passed=False, + detail="Too dark. Move into better light, or wait for the sun.", + )) + blocked = True + elif blown > MAX_BLOWN_FRACTION: + checks.append(QualityCheck( + check="illumination", passed=False, + detail="Too bright โ€” most of the frame is washed out. Turn away from " + "the sun.", + )) + blocked = True + else: + checks.append(QualityCheck(check="illumination", passed=True)) + + if focus < MIN_FOCUS_VARIANCE: + checks.append(QualityCheck( + check="motion_blur", passed=False, + detail="The frame is blurred. Hold still and capture again.", + )) + blocked = True + else: + checks.append(QualityCheck(check="motion_blur", passed=True)) + + return QualityVerdict(checks=checks, blocked=blocked, degraded=degraded) diff --git a/app/reasoning.py b/app/reasoning.py new file mode 100644 index 0000000000000000000000000000000000000000..581756998bd290f83ce3a466e61549dc5306cf70 --- /dev/null +++ b/app/reasoning.py @@ -0,0 +1,645 @@ +"""The capabilities a hosted multimodal model answers, and the runner behind them. + +Sixteen capabilities in the registry declare an RGB photograph, a closed claim +vocabulary and human confirmation, and have no local artefact and no possible +one โ€” there is no public dataset for a cattle wound, a Nigerian hoof or a +footpad lesion, which is directive ยง4's whole argument for a visual reasoner +here. `app/adapters/multimodal.py` holds the contract and +`app/adapters/transports/` the HTTP; this module is what connects them to a +capability key so `/jobs` can dispatch on one. + +## One runner, sixteen capabilities + +`claims.schema_for` builds the same envelope for every capability in the +registry โ€” `claims`, `observations`, `interpretations`, `confidence`, +`evidence`, `limits`, plus `range` and `best_estimate` where the output is +ranged. The vocabulary inside it differs per capability and the *shape* does +not, so one mapping from that envelope to `InferenceResult` serves all of them +and a seventeenth capability is a rubric rather than a module. + +**What differs per capability is the question**, and that is what `RUBRICS` +holds. `SYSTEM_RULES` in `multimodal.py` already carries the safety posture โ€” +observation separated from interpretation, no unqualified disease name, no +invented field, a null in preference to a guess โ€” so a rubric here says only +what to look at and what this particular capability may not conclude. Repeating +the general rules in sixteen places is how they drift. + +## Why the constraints are written into the rubric and not only the schema + +The schema refuses a bad answer. The rubric is what stops the model producing +one, and for several of these the difference is the product's credibility rather +than a validation error: + + * **`cattle_skin` must never name a disease.** Lumpy skin disease and + foot-and-mouth are notifiable under Nigeria's Animal Diseases (Control) Act + 1988 s.8(1). A farm acting on an unqualified name from a photograph is a + reporting obligation triggered by a guess. + * **`cattle_age_dentition` gives a band and never a chronological age.** + Teeth carry an eruption stage; a birth year is not in the photograph. + * **`cattle_sex` must never block registration.** It is a suggestion beside a + field the person fills in. + * **`poultry_litter` may describe condition and never moisture.** Caking and + friability are visible; water content is not, and it is the number a farm + would act on. + * **`egg_quality` reads external quality only.** Cracks need candling โ€” RGB + cannot see them โ€” and cracks are the valuable half, so the limit is stated + rather than left for a farm to discover. + * **`poultry_footpad` reports a sampled prevalence, never a house figure.** + +## What this module cannot do, and what has to happen instead + +Everything downstream of the answer checks that it is *well formed*: the schema +is closed, the claim is in the vocabulary, the sentence is qualified, the number +is in range. **Nothing here can catch an answer that is well formed and wrong.** +A hosted model that confidently reads a healthy footpad as grade 3 produces a +response that passes every gate in this repository. + +So the measurement in `experiments/` is not optional polish for these sixteen, +it is the control. Until one exists, what stands between the model and a farm +is: `experimental` on all sixteen, `requires_review` on every interpretation +this runner builds, and `STANDING_WARNING` on every result. + +**`HUMAN_CONFIRMATION` is not among them, and an earlier draft of this file said +it was.** Six of the sixteen carry it โ€” `cattle_age_dentition`, `cattle_bcs`, +`cattle_breed`, `cattle_sex`, `poultry_fecal`, `poultry_footpad`. **Ten do +not**, and the ten include `cattle_skin` and both wound capabilities: + + cattle_feces cattle_hoof cattle_skin cattle_wound + egg_quality poultry_eye_head poultry_feather poultry_hock + poultry_litter poultry_wound + +The registry's reason is consistent: those ten declare an **observation** output +โ€” a description of the picture โ€” and the directive requires confirmation where a +capability makes a *judgement*. `requires_review` on every interpretation is that +line drawn one level down, and it is the line this runner enforces. + +Whether an unmeasured hosted model's "description" of a lesion is an observation +in that sense is a product question and not this module's to settle. It is +written here because the alternative is a reader assuming, as this file once +asserted, that all sixteen are behind a confirm button. +""" + +from __future__ import annotations + +import logging +from uuid import UUID, uuid4 + +from app.adapters import claims +from app.adapters.base import AdapterError +from app.adapters.claims import ContractViolation +from app.adapters.multimodal import HostedMultimodalAdapter +from app.adapters.transports import transport_from_env +from app.capabilities import Capability +from app.media import MediaRef, MediaStore +from app.providers import ModelArtefact +from app.quality import assess +from app.schemas import ( + InferenceLocation, + InferenceResult, + Interpretation, + Observation, +) + +logger = logging.getLogger(__name__) + +#: What `Capability.model_provider` says for a capability answered over HTTP. +#: +#: A stable declaration rather than the model's name. The name of the model that +#: actually ran is a fact about a *run* and travels on the result โ€” see +#: `providers.HostedModelProvider`, which reads it at call time so a rotation is +#: a restart rather than a rebuild, and so the release row records which model +#: produced which evidence. +HOSTED_PROVIDER = "hosted-multimodal" + +#: The rubric for each capability, keyed by capability key. +#: +#: Read against each entry's `notes` in `app/capabilities.py`, which carry the +#: directive section and the constraint. Where the two could disagree the +#: registry wins and this is wrong. +RUBRICS: dict[str, str] = { + # -- cattle ----------------------------------------------------------- + "cattle_bcs": ( + "Score the body condition of the animal on the 1-5 scale, where 1 is " + "emaciated and 5 is obese. Look at the hooks, the pins, the tail head " + "and the ribs. Give a range covering the plausible scores, not a point " + "value: a single view of one animal does not support a precise score. " + "If the animal is rugged, muddy, heavily haired or standing so the " + "hindquarter is hidden, say so in `limits` and widen the range." + ), + "cattle_age_dentition": ( + "Count the permanent incisors that have erupted in the lower jaw and " + "give the age BAND that eruption stage indicates. Report the count as " + "an observation and the band as the claim.\n\n" + "Teeth carry an eruption stage and not a birth year. Never state a " + "chronological age, an age in months, or a year of birth โ€” the band is " + "the whole answer. If the mouth is not open, the lower incisors are not " + "in frame, or wear makes the count ambiguous, set the count null and say " + "which in `limits`.\n\n" + "**This capability reads teeth and nothing else.** If the lower " + "incisors are not visible in the photograph, refuse it โ€” claim " + "nothing and choose the limit that says so. Do not fall back on body " + "size, frame, horn length or general appearance: those support a guess " + "about age and this capability's whole claim is that its answer came " + "from dentition. A band derived from the animal's build, published " + "under `age_band`, is a different reading wearing this one's name." + ), + "cattle_breed": ( + "Say which breed the animal's phenotype most resembles, from coat " + "colour and pattern, horn shape, hump, dewlap, ear carriage and frame.\n\n" + "**Not forcing a breed is a required answer and not a failure.** Most " + "cattle on a Nigerian smallholding are crossbred, and a crossbred animal " + "must be allowed to stay crossbred: where the animal does not sit " + "cleanly in one breed, claim `crossbred_or_uncertain` rather than the " + "nearest name. Where it does, phrase it as a resemblance โ€” a " + "'White-Fulani-like phenotype' โ€” because a photograph supports a " + "phenotype and not a pedigree." + ), + "cattle_sex": ( + "Say which sex the animal's visible anatomy suggests โ€” udder, scrotum, " + "sheath, frame and horn shape.\n\n" + "This is a suggestion beside a field a person fills in, and it must " + "never read as settled. If the relevant anatomy is not visible, or the " + "animal is young enough that the frame is uninformative, set the claim " + "null and say so in `limits`. An unclear answer costs nothing here; a " + "confident wrong one is written onto an animal's record." + ), + "cattle_wound": ( + "Describe what is visible at the wound: its approximate size relative to " + "the animal, whether the edges are open or closed, whether there is " + "discharge, swelling, or fly strike, and how fresh it appears.\n\n" + "**Triage only.** There is no public dataset of livestock wounds, so " + "this is a description that helps somebody decide whether to look, and " + "never a grade, a stage, a healing time or a cause. Never say what " + "caused the wound. Where the wound looks deep, contaminated, near a " + "joint or eye, or extensive, recommend veterinary review." + ), + "cattle_skin": ( + "Describe the skin: whether there are nodules and roughly how many and " + "how widely spread, whether there is hair loss, crusting, scaling, " + "thickening or discharge, and which body regions are affected.\n\n" + "**Claim what you can actually see, not the vaguest thing that fits.** " + "If there are raised nodules, claim `nodular_lesions_visible` โ€” that is " + "the finding a farmer can act on. Keep `abnormal_skin_pattern` for skin " + "that is clearly wrong in some other way, and do not add it beside the " + "nodule claim as padding. Measured on 282 photographs, the nodule claim " + "is right 98 times in 100 when it is made and the general one is right " + "71; a specific claim is worth more to a farm and it is also the more " + "accurate one.\n\n" + "**Never name a disease.** Lumpy skin disease and foot-and-mouth are " + "notifiable under Nigeria's Animal Diseases (Control) Act 1988 s.8(1), " + "and a name from a photograph is a legal obligation triggered by a " + "guess. Say what the skin looks like and let the vet name it." + ), + "cattle_hoof": ( + "Describe the hoof: overgrowth of the toe or heel, uneven wear, " + "separation at the white line, visible cracks, swelling above the " + "coronet, and any discharge or smell of the interdigital space that is " + "visible as wetness or matting.\n\n" + "A hoof read through mud or dung is a photograph of mud. If the foot is " + "not lifted, not washed, or the sole is not in frame, say so in `limits` " + "rather than reading through it." + ), + "cattle_feces": ( + "Describe the dropping: its consistency on a firm-to-liquid scale, its " + "colour, and whether undigested fibre, mucus, blood or visible worm " + "segments are present.\n\n" + "Consistency reflects the current diet as much as health, so describe " + "what is there and let a person read it against what the animal is " + "eating. Never name a parasite species from a photograph of a dropping." + ), + # -- poultry ---------------------------------------------------------- + "poultry_fecal": ( + "Describe each dropping visible in the sample: its consistency, its " + "colour, and whether blood, mucus, undigested feed or unusually watery " + "content is present. Report how many droppings you could actually " + "assess as an observation.\n\n" + "This is a screen over a sample and never a house diagnosis. Describe " + "the droppings and let a person decide; where blood is visible, " + "recommend review rather than naming coccidiosis or any other disease." + ), + "poultry_footpad": ( + "Grade the footpad on the 0-4 pododermatitis rubric: 0 is a clean " + "unblemished pad, 4 is a large deep lesion with necrosis. Report the " + "grade for each bird you can assess and how many birds you assessed.\n\n" + "**A sampled prevalence, never a house figure.** The birds in these " + "images are the sample; say how many they were and never extrapolate to " + "the flock. If a pad is obscured by litter or the bird is not held so " + "the pad faces the camera, exclude it and say so." + ), + "poultry_hock": ( + "Describe the hock: whether there is discolouration, a burn, a scab, " + "swelling or ulceration, and roughly how severe. Report how many birds " + "you assessed.\n\n" + "As with the footpad, this is a sample and not a house figure. Hock " + "burn and footpad lesions often travel together but are separate " + "captures and separate readings; do not infer one from the other." + ), + "poultry_feather": ( + "Describe feather cover over the back and rump: whether coverage is " + "complete, thinning or bare, whether there are broken shafts, and " + "whether any bare skin shows pecking damage as distinct from moult.\n\n" + "Report what the cover looks like. Do not attribute a cause โ€” feather " + "loss from pecking, moult, rubbing and nutrition look alike in one " + "photograph, and naming a cause is what turns a description into a " + "management decision nobody checked." + ), + "poultry_wound": ( + "Describe what is visible at the wound: approximate size, whether the " + "skin is broken, whether there is bleeding, scabbing or discharge, and " + "whether the surrounding feathers are damaged.\n\n" + "**Triage only**, as for cattle. Never state a cause and never a healing " + "time. A bird with an open bleeding wound in a populated house needs " + "separating, and saying so is the useful output." + ), + "poultry_eye_head": ( + "Describe the head and eyes: whether the eyes are clear or show " + "discharge, swelling or closure, whether there is facial or sinus " + "swelling, whether the comb and wattles are of normal colour or are " + "pale, blue or discoloured, and whether nasal discharge is visible.\n\n" + "Comb colour and facial swelling are the two findings a farm acts on " + "fastest and the two most often over-read from one photograph. Describe " + "them plainly, name no disease, and recommend review where several " + "findings appear together." + ), + "poultry_litter": ( + "Describe the litter: whether the surface is loose and friable or caked " + "and compacted, whether there is visible capping, and how much of the " + "photographed area is affected.\n\n" + "**Condition is visible. Moisture is not, and must never be claimed.** " + "Caking correlates with moisture and is not a measurement of it, and " + "moisture percentage is the number a farm would act on. Describe the " + "surface; a farm measures water with a meter." + ), + "egg_quality": ( + "Describe the external quality of the eggs visible: shell cleanliness, " + "visible soiling, shape abnormality, ridging or roughness of the shell, " + "and obvious breakage. Report how many eggs you could assess.\n\n" + "**External quality only, and say so.** A crack that has not broken the " + "shell membrane is invisible to a camera and needs candling, and cracks " + "are the valuable half of egg quality โ€” so a clean-looking tray here is " + "not a tray without cracks. State that limit rather than leaving a farm " + "to infer it." + ), +} + +#: Said on every run of every capability here, whatever the answer. +#: +#: Not a hedge for its own sake. These sixteen have no measurement โ€” no +#: `experiments/` run has put real photographs through any of them โ€” and a +#: hosted model's confident wrong answer is well-formed, so the thing that +#: protects a farm is a person looking. `HUMAN_CONFIRMATION` makes that +#: structural and this makes it visible. +#: Appended to every rubric. The one instruction that applies to all sixteen. +#: +#: **Written because a measurement showed it was needed.** On `cattle_skin` the +#: model reached for the vague claim more than twice as often as the specific +#: one, and the specific one was the more accurate of the two โ€” 97.9% against +#: 71.4%. A model hedging toward the general claim is not being careful; it is +#: producing a sentence a farm cannot act on, and being more wrong while it does. +BE_SPECIFIC = ( + "Prefer the most specific claim the picture actually supports. A precise " + "finding is what a farmer can act on, and a vague one costs them a decision. " + "Do not add a general claim beside a specific one as a hedge โ€” if you can " + "see the specific thing, claim only that. If you genuinely cannot tell, " + "claim nothing and say why in `limits`; that is a better answer than a " + "claim nobody can use." +) + +STANDING_WARNING = ( + "Read by an experimental visual model, and not measured on livestock like " + "these. Treat it as a second opinion on the photograph and confirm it." +) + + +class ReasoningRunner: + """Runs one hosted-multimodal capability against one capture. + + Constructed per capability key so `RUNNERS` maps a key to a runner exactly + as `counting` and `identification` do, and so the rubric is bound at import + rather than looked up on every request. + """ + + def __init__(self, capability_key: str, adapter: HostedMultimodalAdapter | None = None): + self.capability_key = capability_key + self.rubric = RUBRICS[capability_key] + # Injectable for the tests, which supply an adapter with a stub + # transport. `None` means build one from the environment at call time, + # which is what keeps a key rotation a restart. + self._adapter = adapter + + def _vocabulary_note(self, capability: Capability) -> str: + """The words and array bounds the schema knows and cannot send. + + ## Why this exists at all + + Two controls in `claims.schema_for` do not survive the trip to a + provider, and the model then breaks them in the obvious way: + + * **`ClaimQuantity.values`.** `check_observation_values` refuses an + observation whose `value` is a word the claim did not declare, and + the schema has nowhere to put those words โ€” `value` is + `number | string | null`, and structured outputs refuses an `enum` + beside a union type. So the model is told `body_condition_band` may + carry a word and never which words, and the first real call answered + `"Appropriate"` and then `"Thin to Appropriate"`. Both are refused, + both are the model doing its best with what it was given, and both + cost a vendor call. + * **`maxItems`.** `_for_structured_output` strips it because the API + refuses it, which leaves every array unbounded. The first real call + also produced an answer truncated at the token ceiling โ€” a runaway + list โ€” and a truncated answer is a whole capture refused. + + `claims.enforce` still catches both, so this is not a control. It is what + stops the model failing a control it was never shown, which is the + difference between a capability that works and one that retries. + + **Generated from the registry, never typed.** A word list written here + would be a second copy of `ClaimQuantity.values` and would drift from + the check that enforces it. + """ + contract = claims.schema_for(capability) + output = capability.acquisition.output + lines: list[str] = [] + + exact, named, wordless = [], [], [] + for quantity in (output.quantities or ()): + permitted, free_reason = output.words_for(quantity.claim) + if free_reason is not None: + named.append(quantity.claim) + elif permitted: + offered = ", ".join(f'"{w}"' for w in permitted) + exact.append(f" - {quantity.claim}: {offered}") + else: + wordless.append(quantity.claim) + + if exact: + lines.append( + "Where an observation's `value` is a word it must be exactly " + "one of the words listed for that claim โ€” lower case, as " + "written, not a phrase and not a pair joined by 'to'. If the " + "picture sits between two of them choose the nearer one and " + "widen the range instead.\n" + "\n".join(exact) + ) + if named: + lines.append( + f"These claims carry a NAME and nothing else: " + f"{', '.join(named)}. A name is at most {claims._NAME_WORDS} " + f"words and {claims._NAME_LENGTH} characters, with no digit and " + f"no punctuation beyond a hyphen or an apostrophe โ€” " + f'"White Fulani", not a sentence describing one. Put the ' + f"reasoning in `evidence`, which is a closed list, and never " + f"here." + ) + if wordless: + lines.append( + f"These claims carry NO word: {', '.join(wordless)}. The claim " + f"itself is the finding. Their `value` is a number or null, " + f"never text." + ) + + bounds = [] + for field, spec in (contract.get("properties") or {}).items(): + ceiling = spec.get("maxItems") if isinstance(spec, dict) else None + if isinstance(ceiling, int): + bounds.append(f"{field} at most {ceiling}") + if bounds: + lines.append( + "Length limits, which the schema cannot express and which are " + "checked after you answer: " + "; ".join(bounds) + "." + ) + + if output.show_range: + lines.append( + "`range` is two DIFFERENT numbers, low then high. A pair like " + "[0, 0] is an exact figure wearing a range's clothes and the " + "whole answer is discarded for it.\n\n" + "If what this capability measures is NOT VISIBLE in the " + "picture, refuse: choose the limit that says the subject is not " + "in frame, claim nothing, and let a person take it again. Do " + "not answer with zeros, and do not answer with the widest band " + "the scale allows either โ€” a band from one end of the scale to " + "the other is not a cautious reading, it is a refusal a farmer " + "has to decode, and it reaches them looking like a measurement. " + "Widen a band because the picture is poor; refuse because the " + "subject is absent. They are different answers." + ) + + return "\n\n".join(lines) + + def _reasoner(self) -> HostedMultimodalAdapter: + if self._adapter is not None: + return self._adapter + return HostedMultimodalAdapter(transport=transport_from_env()).load() + + def run( + self, + *, + request, + capability: Capability, + artefact: ModelArtefact, + store: MediaStore, + request_id: UUID | None = None, + ) -> InferenceResult: + request_id = request_id or uuid4() + warnings: list[str] = [STANDING_WARNING] + + images = [] + for media_id in request.media_ids: + images.append(store.open_image(MediaRef( + media_id=media_id, + farm_id=request.farm_id, + captured_at=request.captured_at, + object_path=request.path_for(media_id), + ))) + + # **The quality gate runs before the call, not after it.** A blurred or + # dark frame is refused here for free; sending it costs a vendor call + # and returns a confident reading of a photograph nobody could read. + verdict = assess(images[0]) + checks = list(verdict.checks) + if verdict.blocked: + return self._result( + request=request, capability=capability, artefact=artefact, + request_id=request_id, observations=[], interpretations=[], + confidence=None, checks=checks, + warnings=warnings + [ + "The photograph was not clear enough to read, so nothing was " + "sent to the model and nothing is claimed." + ], + recapture=True, + ) + + try: + note = self._vocabulary_note(capability) + rubric = f"{self.rubric}\n\n{BE_SPECIFIC}" + response = self._reasoner().reason( + images, capability=capability, + rubric=f"{rubric}\n\n{note}" if note else rubric, + ) + except ContractViolation as exc: + # **Answered, and the answer may not be published.** Distinct from + # the transport failure below, and the distinction decides whether + # the run is retried. + # + # The model produced something; it broke the schema, the vocabulary, + # a numeric bound or the forbidden-claims gate. Re-asking the same + # photograph asks the same question of the same model and gets the + # same answer, so raising here would queue a capture that fails + # identically for ever โ€” `MAX_ATTEMPTS` eventually stops it, at the + # cost of that many vendor calls. + # + # The commonest cause is an out-of-protocol capture. `cattle_wound` + # is a `lesion_close_up`; photograph a whole herd with it and the + # honest answer is that there is no wound, which the contract has no + # way to express โ€” ยง7 requires a band from a capability that + # declares one, and `[0, 0]` is an exact figure wearing a band's + # clothes. Both rules are right and together they leave the model + # nowhere to stand, so the capture is refused rather than the rules + # bent. + # + # A farm gets a recapture prompt, nothing is claimed, and the reason + # is on the run. + logger.info( + "%s could not be published for %s: %s", + capability.key, request.media_ids[0], exc, + ) + return self._result( + request=request, capability=capability, artefact=artefact, + request_id=request_id, observations=[], interpretations=[], + confidence=None, checks=checks, + warnings=warnings + [ + "This photograph did not support a reading this capability " + "may publish, so nothing is claimed for it. Take it again " + "the way the capture screen describes.", + ], + recapture=True, + ) + except AdapterError as exc: + # **Not answered.** A timeout, a refused key, a 5xx, a truncated + # body. `/jobs` turns a ValueError into a FAILED job which the API + # keeps queued, and that is right here: the question was never put, + # so asking it again is a different attempt rather than the same + # one. + raise ValueError(str(exc)) from exc + + warnings.extend(response.warnings) + return self._from_envelope( + parsed=response.parsed, request=request, capability=capability, + artefact=artefact, request_id=request_id, checks=checks, + warnings=warnings, + ) + + def _from_envelope( + self, *, parsed: dict, request, capability, artefact, request_id, + checks, warnings, + ) -> InferenceResult: + """`claims.schema_for`'s envelope as an `InferenceResult`. + + Everything here has already been through `claims.parse_strict` and + `claims.enforce`, so the vocabulary, the bounds and the forbidden-claim + gate have all run. What is left is a transcription. + """ + observations = [ + Observation( + type=str(item["type"]), + value=item.get("value"), + unit=item.get("unit"), + confidence=item.get("confidence"), + ) + for item in (parsed.get("observations") or []) + if isinstance(item, dict) and item.get("type") + ] + + # A ranged capability carries its answer in `range`, and the range is + # the answer โ€” `best_estimate` is emitted beside it and never instead of + # it, because ยง7's whole complaint is a point value standing in for a + # band. + band = parsed.get("range") + if isinstance(band, list) and len(band) == 2: + output_type = capability.acquisition.output.type + observations.append(Observation( + type=output_type, value=float(band[0]), unit="range_low", + confidence=None, + )) + observations.append(Observation( + type=output_type, value=float(band[1]), unit="range_high", + confidence=None, + )) + + interpretations = [ + Interpretation( + label=str(item["label"]), + confidence=item.get("confidence"), + # Always true, and the schema pins it to `const: true`. A + # hosted reading a person has not seen is not a record. + requires_review=True, + ) + for item in (parsed.get("interpretations") or []) + if isinstance(item, dict) and item.get("label") + ] + + # The closed lists travel as warnings rather than as claims. They are + # what the model was allowed to say *about its own answer*, and a farm + # reading "One view only" beside a band is the point of them. + for limit in (parsed.get("limits") or []): + if isinstance(limit, str) and limit.strip(): + warnings.append(limit.strip()) + + return self._result( + request=request, capability=capability, artefact=artefact, + request_id=request_id, observations=observations, + interpretations=interpretations, + confidence=parsed.get("confidence"), + checks=checks, warnings=warnings, recapture=False, + ) + + def _result( + self, *, request, capability, artefact, request_id, observations, + interpretations, confidence, checks, warnings, recapture, + ) -> InferenceResult: + banned = set(capability.acquisition.forbidden_claims) + offending = sorted( + {o.type for o in observations if o.type in banned} + | {i.label for i in interpretations if i.label in banned} + ) + if offending: + # `claims.enforce` already refuses these, and this refuses them + # again on the way out. The gate that matters is the one closest to + # the thing being protected, and a runner is the last place a + # forbidden claim could be introduced. + raise ValueError( + f"{capability.key} tried to emit {offending}, which its own " + f"registry entry forbids by name." + ) + + return InferenceResult( + request_id=request_id, + capability_key=capability.key, + model_id=artefact.model_id, + model_version=artefact.version, + inference_location=InferenceLocation.REMOTE, + subject_type=request.subject_type, + subject_id=request.subject_id, + observations=observations, + interpretations=interpretations, + observation_confidence=confidence, + # **Never set, on any path.** The three-level label beside an + # interpretation would read as a confidence in the interpretation, + # and nothing has calibrated one โ€” these sixteen have no measurement + # at all. `requires_review` is the honest field and it is always + # true. + interpretation_confidence=None, + quality_checks=checks, + warnings=warnings, + recommended_recapture=recapture, + ) + + +#: Every capability a hosted multimodal model answers. +#: +#: Derived from `RUBRICS` rather than listed twice: a capability with a rubric +#: and no runner would be silently unavailable, and a runner with no rubric +#: cannot be constructed. +RUNNERS: dict[str, ReasoningRunner] = { + key: ReasoningRunner(key) for key in RUBRICS +} diff --git a/app/schemas.py b/app/schemas.py new file mode 100644 index 0000000000000000000000000000000000000000..3a02e83aa56a52bba29ef077f34c4c5579feba81 --- /dev/null +++ b/app/schemas.py @@ -0,0 +1,250 @@ +"""The inference contract. + +docs/data-model.md ยง6, brief ยง29, ADR 0005. + +Two separations are load-bearing here and neither is negotiable: + +**Observation is not interpretation.** What the model saw and what that means are +different claims with different confidences. The prototype's own copy is the +reference โ€” `Not a diagnosis. A vet confirms this.` โ€” and the product must never +turn `raised_nodules` at 0.94 into a diagnosis because the number looked high. + +**A result names the artefact that produced it.** `model_id` and `model_version` +are required, not optional, because a result whose provenance cannot be +reconstructed is not evidence. The API refuses to store a run without a matching +`capability_releases` row, so a result that cannot name its artefact has nowhere +to go. +""" + +from __future__ import annotations + +from datetime import datetime, timezone +from enum import Enum +from typing import Literal +from uuid import UUID, uuid4 + +from pydantic import BaseModel, ConfigDict, Field + + +class ConfidenceLabel(str, Enum): + """Three levels, matching the prototype's vocabulary. + + Deliberately coarse. A raw score invites false precision in a UI, and the + prototype never shows one โ€” it shows `High confidence`, `Medium confidence`, + `Low confidence`. + """ + + LOW = "low" + MEDIUM = "medium" + HIGH = "high" + + +class InferenceLocation(str, Enum): + """Where a run executed. + + There is deliberately **no `simulated` member**. A development result has no + model artefact, therefore no `capability_releases` row to reference, therefore + its insert fails on a foreign key in the API. Adding a value here would give + a fabricated result somewhere to live (ADR 0005). + """ + + ON_DEVICE = "on_device" + REMOTE = "remote" + + +class JobState(str, Enum): + QUEUED = "queued" + RUNNING = "running" + COMPLETE = "complete" + FAILED = "failed" + #: The capability exists but has no validated model behind it. Not an error โ€” + #: the honest answer, and the one most capabilities give today. + UNAVAILABLE = "unavailable" + + +class QualityCheck(BaseModel): + """Why a capture was or was not usable. + + Surfaced so a recapture prompt can say something specific โ€” the prototype's + `Move farther away`, `Too dark`, `Keep the entire animal in frame` โ€” rather + than failing silently and leaving a worker to guess. + """ + + model_config = ConfigDict(frozen=True) + + check: str = Field(description="e.g. framing, illumination, motion_blur, occlusion") + passed: bool + detail: str | None = None + + +class Observation(BaseModel): + """What the model saw. A measurable fact, not a judgement.""" + + model_config = ConfigDict(frozen=True) + + type: str = Field(description="e.g. raised_nodules, permanent_incisor_count") + value: float | str | None = None + unit: str | None = None + confidence: float | None = Field(default=None, ge=0.0, le=1.0) + + +class Interpretation(BaseModel): + """What the observations might mean. Always hedged, never a diagnosis. + + `label` uses the prototype's own vocabulary of hedges โ€” `lsd_associated_pattern`, + not `lsd`. The distinction is the product's credibility: a photograph can + support a pattern, and only a vet can confirm a disease. + """ + + model_config = ConfigDict(frozen=True) + + label: str + confidence: float | None = Field(default=None, ge=0.0, le=1.0) + requires_review: bool = True + + +class EnrolledAnimal(BaseModel): + """One animal already on this farm's register, as media to compare against. + + **`cattle_identity` is the only capability that needs an input other than the + capture**, and this is it. Matching a muzzle means comparing it with the + muzzles already enrolled, so the request carries the gallery rather than the + service holding one. + + Stateless on purpose, and the reason is farm scoping. + `IdentityIndex`'s own docstring is blunt that its `farm_id` is *"a label, not + a mechanism"* โ€” nothing in `enrol` or `candidates` reads it โ€” and that what + really keeps one farm's animals away from another's is that the caller builds + one index per farm. A gallery the service cached would be a second copy of + that decision, in the process furthest from the farm the request names, and + the failure mode is answering *"which animal is this"* with a neighbour's + cow. The API already scopes every read to a farm, so the caller sends what + that farm has and the service never accumulates a gallery it could confuse. + + It costs an embedding pass over the gallery on every request. For the tens to + low hundreds of animals a farm holds that is the same arithmetic + `IdentityIndex` was sized for, and it is the price of not holding state. + """ + + model_config = ConfigDict(frozen=True) + + #: This farm's own id for the animal, written into + #: `evidence_correction.selected_interpretation` when a person picks it. + animal_id: str + #: The name a screen shows. Supplied rather than derived from `animal_id`, + #: for the reason `Candidate.display_name` gives: rendering a key as copy is + #: how an id reaches a person as a word. Falls back to `animal_id` when the + #: caller sends none, which is visible in the result rather than silent. + display_name: str = "" + #: `view name -> the media ids enrolled under it`, for the five views ยง6.4 + #: names. A view name outside `identity.ENROLMENT_VIEWS` is dropped with a + #: warning rather than stored, because a typo enrols cleanly and then never + #: matches. + #: + #: **A list per view, not one id, and that is the measurement's shape rather + #: than a convenience.** Run `8db9e0bd1b30` enrolled each of its 169 animals + #: from *five muzzle photographs*, all under the one view name, and + #: `IdentityIndex.candidates` scores an animal by its **best** view โ€” so five + #: shots of one muzzle are five chances to match and that is where top-1 + #: 0.9772 comes from. + #: + #: This field held a single id for one commit, which quietly enrolled one + #: muzzle per animal and shipped a capability weaker than its own benchmark. + #: Measured on the muzzle268 database with twelve animals enrolled that way, + #: **0 of 23 held-out queries cleared the 0.98 cutoff** against a measured + #: true-accept rate of 0.2408 โ€” the capability could not have named an animal + #: at all. A farm enrolling one photograph per view gets that behaviour and + #: should be told to add more, which is what `missing_views` and the gallery + #: quality check are for. + views: dict[str, list[UUID]] = Field(default_factory=dict) + #: Object-storage paths for `views`, keyed the same way and positionally + #: matched within each view. The same hint `media_paths` is for the capture, + #: and worth sending for the same reason. + view_paths: dict[str, list[str]] = Field(default_factory=dict) + + def path_for_view(self, view: str, index: int) -> str | None: + """The stored path for one enrolled photograph, if the caller sent one.""" + paths = self.view_paths.get(view) or [] + return paths[index] if index < len(paths) else None + + +class InferenceRequest(BaseModel): + capability_key: str + subject_type: Literal["animal", "herd", "flock_cycle", "house", "case"] + subject_id: UUID + farm_id: UUID + media_ids: list[UUID] = Field(min_length=1) + captured_at: datetime + device_model: str | None = None + + #: Where each media id lives in object storage, positionally matched to + #: `media_ids`. Optional, and worth sending. + #: + #: The API already stores `MediaAsset.object_path`, and passing it turns a + #: blob lookup from "probe the paths the naming convention predicts, then + #: scan the farm's prefix" into a single request. Without it the service + #: still finds the object โ€” the convention is stable and the fallback is + #: exhaustive โ€” it just does more work to do it (ADR 0019). + #: + #: A path is a hint about *location*, never about authorisation. The caller + #: has already checked that the acting profile may read the farm this + #: request names. + media_paths: list[str] | None = None + + #: This farm's enrolled animals, for the one capability that compares the + #: capture against something other than itself. Empty for every other + #: capability, and empty for a farm that has enrolled nobody yet โ€” which + #: `cattle_identity` reports as *"nothing to compare against"* rather than as + #: an error, because a first capture on a new farm is the ordinary case. + enrolled: list[EnrolledAnimal] = Field(default_factory=list) + + def path_for(self, media_id: UUID) -> str | None: + """The stored path for one media id, if the caller sent one.""" + if not self.media_paths: + return None + index = self.media_ids.index(media_id) + return ( + self.media_paths[index] if index < len(self.media_paths) else None + ) + + +class InferenceResult(BaseModel): + """A completed run. + + `model_id` and `model_version` are required. A result that cannot say what + produced it is not evidence, and the API will refuse to store it. + """ + + model_config = ConfigDict(frozen=True) + + request_id: UUID + capability_key: str + model_id: str + model_version: str + inference_location: InferenceLocation + + subject_type: str + subject_id: UUID + + observations: list[Observation] = Field(default_factory=list) + interpretations: list[Interpretation] = Field(default_factory=list) + + observation_confidence: ConfidenceLabel | None = None + interpretation_confidence: ConfidenceLabel | None = None + + quality_checks: list[QualityCheck] = Field(default_factory=list) + warnings: list[str] = Field(default_factory=list) + + #: Set when the capture was not good enough to interpret. The device should + #: prompt rather than store a result nobody should act on. + recommended_recapture: bool = False + + created_at: datetime = Field(default_factory=lambda: datetime.now(timezone.utc)) + + +class JobStatus(BaseModel): + job_id: UUID = Field(default_factory=uuid4) + state: JobState + capability_key: str + detail: str | None = None + result: InferenceResult | None = None diff --git a/app/tiling.py b/app/tiling.py new file mode 100644 index 0000000000000000000000000000000000000000..399a75b01e9f490f6fb8adf9d2334ef32513af15 --- /dev/null +++ b/app/tiling.py @@ -0,0 +1,244 @@ +"""Counting a frame at several resolutions, and noticing when the count runs away. + +A detector reads a fixed 640 px input. Hand it a 4,000 px photograph of a paddock +and every animal is downsampled before the network sees it; hand it one ninth of +that photograph and each animal arrives four times larger. So the same detector +returns a different count depending on how the frame is cut up, and **the way it +changes as you cut finer is the measurement that matters**. + +Two things fall out of that, and the second is the important one. + +**The count gets better.** Slicing a frame into overlapping tiles and detecting +in each one recovers animals that whole-frame inference loses to downsampling. +This is the standard trick for small objects in large images. + +**The count says whether it can be trusted.** In a frame the detector can +actually read, the count stops moving: a paddock with a dozen cattle returns +about a dozen at one tile, at four, and at nine, because there was nothing left +to find. In a broiler house it never stops moving โ€” every finer cut finds more +birds, because there are always more birds hidden behind the ones in front. + +That is the difference between a count and a sample, measured rather than +guessed. The old guard asked whether the boxes it *had* were small, which is a +question about the animals the detector found and says nothing about the ones it +missed โ€” on a shed of a thousand birds it saw twenty large foreground birds, +concluded the frame was sparse, and published twenty. +""" + +from __future__ import annotations + +from dataclasses import dataclass + +from PIL import Image + +from app.detectors.base import Detection, Detector + +#: How much neighbouring tiles overlap, as a share of tile size. An animal +#: sitting exactly on a cut would otherwise be two half-animals, each too +#: partial to detect. Cross-tile NMS then removes the duplicates the overlap +#: creates. +TILE_OVERLAP = 0.20 + +#: IoU above which two boxes are the same animal. Looser than the within-tile +#: NMS threshold, because the same animal seen in two tiles is cropped +#: differently in each and the boxes never align exactly. +MERGE_IOU = 0.55 + +#: Intersection over the *smaller* box's area, above which the smaller box is a +#: part of the larger one rather than a second animal. +#: +#: This is the threshold that makes tiling safe, and leaving it out is how the +#: first attempt turned one cow into three. A cow filling the frame is cut into +#: quarters by a 2x2 grid, and the detector obligingly finds a cow in each +#: quarter; those four quarter-boxes barely overlap *each other*, so IoU keeps +#: all four. Each is almost entirely inside the whole-frame box, so containment +#: removes them. +#: +#: 0.85 rather than something lower because two animals standing one behind the +#: other genuinely overlap: on the evaluation set the near animal's box covered +#: up to three quarters of the far animal's. Merging those would trade a +#: duplicate for a lost animal. +MERGE_CONTAINMENT = 0.85 + +#: Tile grids, coarse to fine. 1 is whole-frame. Stopping at 3 is a cost +#: decision: 1 + 4 + 9 inferences already takes seconds on a CPU, and a frame +#: still finding new animals at 3x3 is a shed โ€” the answer there is that no +#: count exists, not that a fourth grid would find it. +LEVELS: tuple[int, ...] = (1, 2, 3) + + +@dataclass(frozen=True) +class Level: + """Everything found at this grid **and every coarser one**. + + Accumulating rather than replacing is what makes the comparison between + levels mean something: each level is a superset, so a level that adds + nothing new is a level that found nothing new, and the count can only rise. + It is also what keeps the whole-frame box of a large animal in the pool to + absorb the fragments a fine grid makes of it. + """ + + grid: int + detections: list[Detection] + + @property + def count(self) -> int: + return len(self.detections) + + +def _crops(size: tuple[int, int], grid: int) -> list[tuple[int, int, int, int]]: + width, height = size + if grid == 1: + return [(0, 0, width, height)] + + step_x, step_y = width / grid, height / grid + pad_x, pad_y = step_x * TILE_OVERLAP, step_y * TILE_OVERLAP + boxes = [] + for row in range(grid): + for column in range(grid): + x0 = max(0, int(column * step_x - pad_x)) + y0 = max(0, int(row * step_y - pad_y)) + x1 = min(width, int((column + 1) * step_x + pad_x)) + y1 = min(height, int((row + 1) * step_y + pad_y)) + boxes.append((x0, y0, x1, y1)) + return boxes + + +def _overlaps( + a: tuple[float, float, float, float], b: tuple[float, float, float, float] +) -> tuple[float, float]: + """`(IoU, intersection over the smaller area)` for two boxes.""" + ax0, ay0, ax1, ay1 = a + bx0, by0, bx1, by1 = b + x0, y0 = max(ax0, bx0), max(ay0, by0) + x1, y1 = min(ax1, bx1), min(ay1, by1) + overlap = max(0.0, x1 - x0) * max(0.0, y1 - y0) + if overlap <= 0.0: + return 0.0, 0.0 + area_a = (ax1 - ax0) * (ay1 - ay0) + area_b = (bx1 - bx0) * (by1 - by0) + union = area_a + area_b - overlap + smaller = min(area_a, area_b) + return ( + overlap / union if union > 0 else 0.0, + overlap / smaller if smaller > 0 else 0.0, + ) + + +def _merge(detections: list[Detection], frame_area: float) -> list[Detection]: + """One animal, one box, whichever tile found it. + + **Largest box first**, which is the ordering the containment rule needs: the + whole animal has to be in the kept set before its fragments are tested + against it. Score order โ€” the usual choice for NMS โ€” would let a confident + fragment claim the animal and leave its siblings unmatched. + + Boxes come back in frame coordinates, and `area_fraction` is recomputed + against the whole frame: a bird covering a quarter of its tile covers a + thirty-sixth of the picture, and everything downstream reasons about the + picture. + """ + def area(d: Detection) -> float: + x0, y0, x1, y1 = d.box + return (x1 - x0) * (y1 - y0) + + kept: list[Detection] = [] + for detection in sorted(detections, key=area, reverse=True): + duplicate = False + for other in kept: + if other.label != detection.label: + continue + iou, containment = _overlaps(other.box, detection.box) + if iou > MERGE_IOU or containment > MERGE_CONTAINMENT: + duplicate = True + break + if duplicate: + continue + kept.append( + Detection( + label=detection.label, + score=detection.score, + box=detection.box, + area_fraction=area(detection) / frame_area, + ) + ) + kept.sort(key=lambda d: d.score, reverse=True) + return kept + + +def _raw(detector: Detector, image: Image.Image, grid: int) -> list[Detection]: + """Every box one grid produced, in frame coordinates, unmerged.""" + width, height = image.size + gathered: list[Detection] = [] + for x0, y0, x1, y1 in _crops((width, height), grid): + tile = image if grid == 1 else image.crop((x0, y0, x1, y1)) + for detection in detector.detect(tile): + tx0, ty0, tx1, ty1 = detection.box + gathered.append( + Detection( + label=detection.label, + score=detection.score, + box=(tx0 + x0, ty0 + y0, tx1 + x0, ty1 + y0), + # Recomputed by `_merge`; a tile-relative fraction here + # would be wrong by the square of the grid. + area_fraction=detection.area_fraction, + ) + ) + return gathered + + +def detect_at(detector: Detector, image: Image.Image, grid: int) -> list[Detection]: + """Run one grid on its own. Used by the tests and by nothing else.""" + width, height = image.size + return _merge(_raw(detector, image, grid), float(width * height)) + + +def pyramid( + detector: Detector, + image: Image.Image, + subject_classes: tuple[str, ...], + growth_tolerance: float, + levels: tuple[int, ...] = LEVELS, +) -> list[Level]: + """Count at successively finer grids, stopping as soon as the count settles. + + Returns every level that was run, coarsest first, each holding the merged + result of every grid up to and including its own. The caller decides what + the sequence means; this function only refuses to spend inferences it does + not need โ€” a frame that has settled is not going to unsettle, and the common + case is a farmer photographing six animals. + """ + frame_area = float(image.size[0] * image.size[1]) + gathered: list[Detection] = [] + results: list[Level] = [] + + for grid in levels: + gathered.extend(_raw(detector, image, grid)) + results.append(Level(grid=grid, detections=_merge(gathered, frame_area))) + if len(results) >= 2 and converged( + results[-2], results[-1], subject_classes, growth_tolerance + ): + break + return results + + +def subject_count(level: Level, subject_classes: tuple[str, ...]) -> int: + return sum(1 for d in level.detections if d.label in subject_classes) + + +def converged( + coarser: Level, finer: Level, subject_classes: tuple[str, ...], tolerance: float +) -> bool: + """Whether cutting the frame finer stopped finding new animals. + + Growth is measured against the coarser count, so it is a proportion rather + than a difference: three more animals out of six means the frame was not + read, three more out of sixty means it was. + """ + before = subject_count(coarser, subject_classes) + after = subject_count(finer, subject_classes) + if before == 0: + # Nothing at the coarse grid. Converged only if the finer grid agrees, + # otherwise the coarse pass simply could not see the animals. + return after == 0 + return (after - before) / before <= tolerance diff --git a/app/training/__init__.py b/app/training/__init__.py new file mode 100644 index 0000000000000000000000000000000000000000..675ae014a2bd5a327c8bdc0418669b469c1b22b6 --- /dev/null +++ b/app/training/__init__.py @@ -0,0 +1,25 @@ +"""The data contract for the models that do not exist yet. + +`cattle_weight` and `cattle_bcs` cannot be built from a pretrained detector. +They need labelled Nigerian data that nobody has collected. This package is what +turns that from an intention into a task with a definition of done: the record +format the labelling has to produce, the rules that decide whether a record is +ground truth or merely an opinion, and the numbers a trained model has to beat +before its capability moves off `coming_soon`. + +There is deliberately no trainer here. A training loop written before the data +exists is a guess about the data. +""" + +from app.training.gates import PROMOTION_GATES, PromotionGate +from app.training.schema import BcsSample, BcsScore, FrameRef, ScaleReference, WeightSample + +__all__ = [ + "PROMOTION_GATES", + "PromotionGate", + "BcsSample", + "BcsScore", + "FrameRef", + "ScaleReference", + "WeightSample", +] diff --git a/app/training/gates.py b/app/training/gates.py new file mode 100644 index 0000000000000000000000000000000000000000..a1c09f6285a8e307960a91db4e0a70ca0730b40e --- /dev/null +++ b/app/training/gates.py @@ -0,0 +1,94 @@ +"""What a model has to prove before its capability is promoted. + +Held as data so `validate.py` can check a dataset against it and a test can +assert it, rather than as a paragraph in a document that gets read once. + +These thresholds are the team's, not a standard. Each one is set by asking what +error would change a decision a farmer makes, because a model that is more +accurate than that is useful and one that is less accurate than that is worse +than the farmer's own eye. +""" + +from __future__ import annotations + +from dataclasses import dataclass + + +@dataclass(frozen=True) +class PromotionGate: + capability_key: str + #: Ground-truth samples. Records that fail `is_ground_truth` do not count. + min_ground_truth_samples: int + min_animals: int + #: Farms, not photographs. Ten thousand images from one farm teach a model + #: that farm's lighting, breed and handling, and nothing about the next one. + min_farms: int + min_breeds: int + #: Every bucket must hold at least this share of the set, so a model cannot + #: pass by being accurate only where the data is dense. + min_share_per_bucket: float + holdout: str + metric: str + threshold: str + rationale: str + #: What has to be recorded on the card before the artefact may load at all. + geography_required: str = "NG, with the state named" + + +CATTLE_WEIGHT = PromotionGate( + capability_key="cattle_weight", + min_ground_truth_samples=1200, + min_animals=800, + min_farms=5, + min_breeds=3, + min_share_per_bucket=0.05, + holdout=( + "Two entire farms, held out before any training run and never used for " + "tuning. Splitting by image, or even by animal, leaks the farm's camera, " + "light and handling into the evaluation and flatters the result." + ), + metric="Mean absolute percentage error against a weighbridge or platform scale", + threshold="<= 7% overall, and <= 10% within every 100 kg band", + rationale=( + "A trader's offer moves on tens of kilograms. At 7% a 300 kg steer is " + "estimated within 21 kg, which is inside the spread of two experienced " + "eyes and useful for triage. Past 10% the estimate is worse than the " + "farmer's own judgement and the feature costs credibility rather than " + "earning it. The per-band clause exists because calves and mature bulls " + "are the two places a single regression fails first." + ), +) + +CATTLE_BCS = PromotionGate( + capability_key="cattle_bcs", + min_ground_truth_samples=2000, + min_animals=1500, + min_farms=5, + min_breeds=3, + #: Nine half-point steps, so an even split is 11%. Scores of 1.0 and 5.0 are + #: genuinely rare in a working herd, which is exactly why they have to be + #: collected deliberately rather than waited for. + min_share_per_bucket=0.03, + holdout=( + "Two entire farms, plus at least one scorer whose scores appear only in " + "the evaluation set. A model that has learned one technician's habits is " + "not a model of body condition." + ), + metric=( + "Quadratic weighted kappa against the multi-scorer consensus, and " + "accuracy within half a point" + ), + threshold="QWK >= 0.70 and within-half-a-point accuracy >= 85%", + rationale=( + "Body condition drives a decision in half-point steps: supplement, " + "breed, cull. Plain accuracy is the wrong metric because it scores a " + "3.0-for-3.5 miss the same as a 3.0-for-5.0 miss. QWK punishes the " + "distance. 0.70 is roughly the agreement two trained human scorers reach " + "with each other, so it is the point at which the model stops being the " + "weakest scorer in the room." + ), +) + +PROMOTION_GATES: dict[str, PromotionGate] = { + gate.capability_key: gate for gate in (CATTLE_WEIGHT, CATTLE_BCS) +} diff --git a/app/training/schema.py b/app/training/schema.py new file mode 100644 index 0000000000000000000000000000000000000000..df5c3448859b07b7ecee2fcb17a1da9730a35b71 --- /dev/null +++ b/app/training/schema.py @@ -0,0 +1,233 @@ +"""What a labelled sample has to contain. + +One JSON object per line, one file per capability. The models here are strict on +purpose: a dataset is easy to collect badly and expensive to re-collect, and +almost every failure is a field somebody left out in the field and nobody +noticed until training. + +The distinction that matters most is `is_ground_truth`. A record can be +perfectly well formed and still not be evidence โ€” a girth-tape reading is an +estimate produced by the same morphometric relationship the model is trying to +learn, so training a weight model on tape readings teaches it to reproduce a +formula rather than to predict a weight. Those records are allowed in, and +excluded from the evaluation set. +""" + +from __future__ import annotations + +from datetime import date +from typing import Literal + +from pydantic import BaseModel, ConfigDict, Field, field_validator, model_validator + +View = Literal["side", "rear", "front", "top", "oblique"] + +#: Scale readings that are measurements. Everything else is an estimate. +TRUE_SCALES = frozenset({"weighbridge", "platform_scale"}) + +#: A weight and a photograph taken further apart than this are not the same +#: animal state. Cattle gain and lose several kilograms of gut fill in a day. +MAX_WEIGHING_GAP_HOURS = 24.0 + +#: Body condition is scored in half points. A dataset with 3.7 in it was scored +#: by somebody using a different scale. +BCS_STEPS = tuple(round(1.0 + 0.5 * i, 1) for i in range(9)) + + +class FrameRef(BaseModel): + """One photograph, and enough geometry to know what it is a photograph of.""" + + model_config = ConfigDict(frozen=True) + + path: str + view: View + #: Without these, two frames of the same animal at different distances are + #: indistinguishable to a model that has to infer size. + camera_height_cm: float | None = Field(default=None, gt=0) + subject_distance_m: float | None = Field(default=None, gt=0) + + +class ScaleReference(BaseModel): + """The object in frame that makes a pixel mean a centimetre. + + A photograph carries no scale. Without a reference of known length in the + same plane as the animal, heart girth in pixels is not convertible to heart + girth in centimetres, and every weight the model produces is a guess about + how far away the camera was. + """ + + model_config = ConfigDict(frozen=True) + + kind: Literal[ + "marker_board", "chest_band", "calibration_rod", "known_gate_width", + "depth_sensor", + ] + length_cm: float = Field(gt=0) + #: The reference has to be in the frames the measurement is taken from, not + #: merely somewhere in the capture set. + visible_in_views: list[View] = Field(min_length=1) + + +class WeightSample(BaseModel): + """One animal, photographed and weighed. + + Weight estimation from a photograph is regression on morphometrics โ€” heart + girth and body length, recovered from keypoints or a segmentation mask, then + converted to a scale. Every field here exists because one of those three + steps needs it. + """ + + model_config = ConfigDict(frozen=True) + + sample_id: str + farm_id: str + animal_id: str + captured_at: date + + frames: list[FrameRef] = Field(min_length=2) + scale_reference: ScaleReference + + weight_kg: float = Field(gt=20, lt=1200) + scale_type: Literal[ + "weighbridge", "platform_scale", "girth_tape", "visual_estimate", + ] + hours_between_capture_and_weighing: float = Field(ge=0) + + breed: str + sex: Literal["male", "female"] + age_months: int | None = Field(default=None, ge=0, le=360) + + #: Optional, and worth collecting. A tape measurement of the same animal + #: lets the pipeline be evaluated in two halves โ€” did the keypoints recover + #: the girth, and did the regression convert it โ€” instead of only end to end. + heart_girth_cm: float | None = Field(default=None, gt=0) + body_length_cm: float | None = Field(default=None, gt=0) + + @model_validator(mode="after") + def _needs_a_side_and_a_rear_view(self) -> WeightSample: + views = {f.view for f in self.frames} + missing = {"side", "rear"} - views + if missing: + raise ValueError( + f"{self.sample_id}: missing {sorted(missing)} view(s). Side gives " + f"body length, rear gives width; neither alone gives volume." + ) + return self + + @model_validator(mode="after") + def _scale_reference_must_be_in_a_measured_view(self) -> WeightSample: + if not set(self.scale_reference.visible_in_views) & {"side", "rear"}: + raise ValueError( + f"{self.sample_id}: the scale reference is not visible in the side " + f"or rear view, so it cannot scale the measurement." + ) + return self + + @property + def is_ground_truth(self) -> bool: + return ( + self.scale_type in TRUE_SCALES + and self.hours_between_capture_and_weighing <= MAX_WEIGHING_GAP_HOURS + ) + + @property + def weight_band(self) -> str: + """100 kg bands. Coverage per band is what stops a model that only + works on the middle of the range from passing on an overall average.""" + if self.weight_kg < 100: + return "<100" + if self.weight_kg >= 500: + return ">=500" + lower = int(self.weight_kg // 100) * 100 + return f"{lower}-{lower + 100}" + + +class BcsScore(BaseModel): + """One person's score, and what qualifies them to give it.""" + + model_config = ConfigDict(frozen=True) + + scorer_id: str + score: float + credential: Literal["veterinarian", "trained_technician", "farmer"] + + @field_validator("score") + @classmethod + def _half_points_only(cls, value: float) -> float: + if round(value, 1) not in BCS_STEPS: + raise ValueError( + f"{value} is not a body condition score. The scale is 1 to 5 in " + f"half points." + ) + return round(value, 1) + + +class BcsSample(BaseModel): + """One animal, scored by more than one person. + + Body condition is ordinal regression, and its ground truth is a human + judgement with real disagreement in it. A single scorer's opinion is a + label with unknown error, so the schema requires at least two and the + validator measures how far apart they were. + """ + + model_config = ConfigDict(frozen=True) + + sample_id: str + farm_id: str + animal_id: str + captured_at: date + + frames: list[FrameRef] = Field(min_length=1) + scores: list[BcsScore] = Field(min_length=2) + + breed: str + sex: Literal["male", "female"] + age_months: int | None = Field(default=None, ge=0, le=360) + + @model_validator(mode="after") + def _needs_a_rear_view(self) -> BcsSample: + if "rear" not in {f.view for f in self.frames}: + raise ValueError( + f"{self.sample_id}: no rear view. Tailhead and pin bones are where " + f"body condition is read; a side view alone confuses condition " + f"with frame size." + ) + return self + + @model_validator(mode="after") + def _scorers_must_be_distinct(self) -> BcsSample: + ids = [s.scorer_id for s in self.scores] + if len(set(ids)) != len(ids): + raise ValueError(f"{self.sample_id}: the same scorer appears twice.") + return self + + @property + def consensus(self) -> float: + """The median score, breaking an even split downwards. + + Two scorers who say 3.0 and 3.5 average to 3.25, which is not a body + condition score. Rounding has to go one way, and down is the safe + direction: calling an animal leaner than it is prompts somebody to look + at it, and calling it fatter than it is hides a thin one. + """ + ordered = sorted(s.score for s in self.scores) + middle = len(ordered) // 2 + if len(ordered) % 2: + return ordered[middle] + return ordered[middle - 1] + + @property + def spread(self) -> float: + scores = [s.score for s in self.scores] + return max(scores) - min(scores) + + @property + def is_ground_truth(self) -> bool: + """Two qualified scorers who agree within one point. + + A wider spread is not a bad sample โ€” it is a sample nobody can grade, + and using it as evaluation truth measures the scorers, not the model. + """ + qualified = [s for s in self.scores if s.credential != "farmer"] + return len(qualified) >= 2 and self.spread <= 1.0 diff --git a/app/training/validate.py b/app/training/validate.py new file mode 100644 index 0000000000000000000000000000000000000000..942fdec09cf21d0931fc0e9f5ff8a69baa7b62eb --- /dev/null +++ b/app/training/validate.py @@ -0,0 +1,120 @@ +"""Check a labelling run against the gate before anyone trains on it. + + .venv/bin/python -m app.training.validate cattle_weight data/weight.jsonl + .venv/bin/python -m app.training.validate cattle_bcs data/bcs.jsonl + +What this can tell you is whether the **data** could support a model that +passes: enough ground truth, enough farms, enough breeds, and coverage across +the range rather than a pile in the middle. What it cannot tell you is whether a +model trained on it hits the metric โ€” that needs the model. Exit code 0 means +"worth training on", never "ready to promote". + +Run it while collection is still happening. A dataset that turns out to be four +hundred animals from one farm is recoverable in month two and not in month six. +""" + +from __future__ import annotations + +import argparse +import json +import sys +from collections import Counter +from pathlib import Path + +from pydantic import ValidationError + +from app.training.gates import PROMOTION_GATES, PromotionGate +from app.training.schema import BcsSample, WeightSample + +_MODELS = {"cattle_weight": WeightSample, "cattle_bcs": BcsSample} + + +def _bucket(sample) -> str: + if isinstance(sample, WeightSample): + return sample.weight_band + return f"bcs_{sample.consensus:.1f}" + + +def _load(path: Path, model) -> tuple[list, list[str]]: + samples, problems = [], [] + for number, line in enumerate(path.read_text().splitlines(), start=1): + line = line.strip() + if not line or line.startswith("#"): + continue + try: + samples.append(model.model_validate(json.loads(line))) + except (json.JSONDecodeError, ValidationError) as exc: + first = str(exc).splitlines()[-1].strip() + problems.append(f"line {number}: {first}") + return samples, problems + + +def report(samples: list, gate: PromotionGate) -> list[tuple[bool, str]]: + truth = [s for s in samples if s.is_ground_truth] + farms = {s.farm_id for s in truth} + animals = {s.animal_id for s in truth} + breeds = {s.breed for s in truth} + buckets = Counter(_bucket(s) for s in truth) + total = len(truth) or 1 + + thin = sorted( + b for b, n in buckets.items() if n / total < gate.min_share_per_bucket + ) + + lines = [ + (len(truth) >= gate.min_ground_truth_samples, + f"ground-truth samples: {len(truth)} of {gate.min_ground_truth_samples} " + f"({len(samples) - len(truth)} excluded as estimates or unusable)"), + (len(animals) >= gate.min_animals, + f"distinct animals: {len(animals)} of {gate.min_animals}"), + (len(farms) >= gate.min_farms, + f"farms: {len(farms)} of {gate.min_farms}"), + (len(breeds) >= gate.min_breeds, + f"breeds: {len(breeds)} of {gate.min_breeds} โ€” {', '.join(sorted(breeds)) or 'none'}"), + (not thin, + f"buckets under {gate.min_share_per_bucket:.0%}: " + f"{', '.join(thin) if thin else 'none'}"), + (len(farms) >= gate.min_farms + 2, + f"farms to spare for the held-out split: {max(len(farms) - gate.min_farms, 0)} of 2"), + ] + return lines + + +def main(argv: list[str] | None = None) -> int: + parser = argparse.ArgumentParser(description=__doc__) + parser.add_argument("capability", choices=sorted(_MODELS)) + parser.add_argument("manifest", type=Path) + args = parser.parse_args(argv) + + gate = PROMOTION_GATES[args.capability] + samples, problems = _load(args.manifest, _MODELS[args.capability]) + + print(f"{args.manifest} โ€” {args.capability}") + if problems: + print(f"\n {len(problems)} record(s) rejected:") + for problem in problems[:20]: + print(f" โœ— {problem}") + if len(problems) > 20: + print(f" โ€ฆ and {len(problems) - 20} more") + + print() + lines = report(samples, gate) + for passed, text in lines: + print(f" {'โœ“' if passed else 'โœ—'} {text}") + + print(f"\n held-out split: {gate.holdout}") + print(f" metric to beat: {gate.metric}") + print(f" {gate.threshold}") + + ready = all(passed for passed, _ in lines) and not problems + verdict = ( + "The data could support a model that passes. Train, then measure." + if ready + else "Not enough data yet. Keep collecting." + ) + print(f"\n {verdict}") + return 0 if ready else 1 + + +if __name__ == "__main__": + sys.exit(main()) diff --git a/models/README.md b/models/README.md new file mode 100644 index 0000000000000000000000000000000000000000..30a21ce4ce97be8f92173dee8b46d8366edeef01 --- /dev/null +++ b/models/README.md @@ -0,0 +1,183 @@ +# Model artefacts + +**Three capabilities have a model behind them.** `cattle_detection` and +`poultry_count` run a checksummed YOLOX-m ONNX graph (ADR 0017, ADR 0018), and +`cattle_identity` has a frozen DINOv2-small embedding backbone (ADR 0020) โ€” a +backbone, not an identity model, and the capability stays `coming_soon` because +nothing has tested it on the same animal twice. The rest have nothing, and the +service says so rather than inventing a result. + +``` +models/ + _artefacts/ + yolox_m-0.1.1rc0.onnx the shipped detector, shared by two capabilities + dinov3_vits16-onnx-1.onnx frozen embeddings, locally exported + cattle_detection/ + model_card.json + poultry_count/ + model_card.json + cattle_identity/ + model_card.json + alternates/ + yolox_onnx/ Apache-2.0, kept working, never registered + model_card.json + yolox_s-0.1.1rc0.onnx + dinov2_embedding/ Apache-2.0 fallback; measurably worse, kept working + model_card.json + dinov2_small-onnx-1.onnx + megadescriptor/ CC-BY-NC-4.0. Installed to be measured, never served + model_card.json + megadescriptor_l_384-onnx-1.onnx + miewid/ no licence at all. Same: measured, never served + model_card.json + miewid_msv3-onnx-1.onnx +``` + +**Two of those alternates are weights Animap may not serve.** They are on disk +because a licence stopped being a reason to leave a model unmeasured โ€” directive +ยง40.2 asked for MegaDescriptor against DINOv3 and the honest answer needed the +benchmark, not an argument about why it could not be run. Nothing about the terms +changed: `app/adapters/licences.py` still reports both as non-servable, +`registry.refused()` still names them, and they load only under +`ANIMAP_LICENCE_POLICY=record`, which logs every load at ERROR and keeps it. +Under the default `enforce` policy they are refused and their adapters report +unavailable. + +An alternate keeps its artefact **inside its own directory**. `load_card` refuses +a path that climbs out of the card's grandparent, which is what stops a card +being a way to load whatever is on the box. + +Weights are not in git. Cards are โ€” a governance record nobody can see in a diff +is not one. + +## Installing them + +```bash +.venv/bin/python scripts/install_models.py # fetch what the cards name +.venv/bin/python scripts/install_models.py --check # verify, never fetch +``` + +**Some artefacts have no URL to fetch.** An ONNX export of a published +checkpoint is not a file the publisher hosts, so its card carries `produced_by` +naming the script that makes it, and `install_models.py` refuses to download +rather than writing a model-card web page into a `.onnx` file: + +```bash +.venv/bin/pip install -r requirements-export.txt # build-time only, never shipped +.venv/bin/python scripts/export_embedding.py +``` + +The export is checked against torch before it is written and is +byte-reproducible, so the sha256 on a card pins the exact bytes that were +validated. **It is reproducible from the same exporter code, not from the +weights alone** โ€” renaming a wrapper attribute changes the graph's node names and +therefore the checksum, without changing a single weight. + +The script reads the committed cards, downloads what each one names, and refuses +anything whose sha256 does not match. It has no list of its own, so there is +nothing to drift out of date. **The service never calls it.** A model that is not +on disk leaves its capability unavailable. + +## Permissive licences only + +**A card naming AGPL-3.0 is refused at load, not warned about.** `discover()` +skips it and the capability stays unavailable. + +**And a card is not trusted about its own licence.** `app/adapters/licences.py` +holds what each *runtime* really loads under, read from a primary source and +dated, and `discover()` refuses a card whose declared licence disagrees with it. +That closes the exploit ADR 0017 records โ€” `"license": "Apache-2.0"` written over +a path to copyleft weights โ€” without depending on somebody having hard-coded the +right runtime name. It also catches the version of the trap that arrives from +upstream: `BVRA/MegaDescriptor-L-384/config.json` declares `"license": "mit"` +inside the checkpoint config while the repository is CC-BY-NC-4.0. Ultralytics YOLO was the shipped +detector until ADR 0017 and is now not installed at all; read +[`docs/adr/0017-ultralytics-licence.md`](../../../docs/adr/0017-ultralytics-licence.md) +before reaching for it again. + +Four mechanisms hold that position, because it is the failure that is cheap now +and expensive later: `requirements.txt` omits the package, `.dockerignore` +excludes `*.pt`, the `Dockerfile` fails the build if one arrives anyway, and +`providers.discover()` refuses the card. `GET /health` reports +`artefact_licenses`, so a deployment that picked one up says so from outside. + +## Adding one + +Create a directory named for the capability, containing a `model_card.json`. +Every field below is required; a card missing any of them is refused at load, and +the capability stays unavailable: + +```json +{ + "model_id": "animap-cattle-weight", + "version": "0.1.0", + "capability_key": "cattle_weight", + "runtime": "ultralytics", + "artefact": "../_artefacts/model.pt", + "source": "https://โ€ฆ where it came from, exactly", + "license": "the licence, and it must permit this use", + "sha256": "checksum of the artefact file", + "validation_notes": "What was tested, on what data, with what result.", + "geography_validated": ["NG"] +} +``` + +### Why each field is enforced + +**`source` and `license`** โ€” a model of unknown origin cannot be shipped, and a +licence that forbids commercial use is a legal problem discovered too late. + +**`sha256`** โ€” verified on every load. The artefact on disk must be the one that +was validated, not one swapped in afterwards. + +**`validation_notes`** โ€” must be non-empty, and whitespace does not count. This +is the field that says somebody checked the model works. A card can otherwise be +filled in purely to get past the loader. + +**`geography_validated`** โ€” a body-condition model trained on European dairy +cattle is not validated for White Fulani in Kaduna. Recording where it was tested +is what stops that being assumed. The sentinel `global-coco` means "validated +nowhere, only pretrained", and `Capability.may_be_enabled` refuses to promote a +capability past `beta` while it is present. + +**`runtime`** โ€” not required by governance, because governance is about +provenance. But a card without it can be inspected and not run: `detectors.build` +refuses rather than defaulting, since a default would let a typo change which +licensed model produced a farmer's result. + +### Where the artefact may live + +The `artefact` path may point sideways โ€” several capabilities share one detector, +and storing the file twice would mean two things to keep in step. It may not +point out of the models tree; `load_card` refuses that. A card under +`alternates/` is two levels down and outside the discovery glob, so an alternate +can never silently become the model that answered a request. + +## What is not allowed + +- **No runtime downloads.** The service never fetches a model. An artefact + arrives through a reviewed change, or it does not arrive. +- **No binaries in git.** `.gitignore` excludes `*.tflite`, `*.onnx`, `*.pt` and + `*.pth`, and everything in this directory except the READMEs and the cards. +- **No placeholder adapters.** A capability with a validated artefact but no + implemented adapter returns `501`, not a plausible-looking result. An adapter + that returns something believable is precisely the failure ADR 0005 exists to + prevent. + +## Promoting a capability + +The path from nothing to trusted, per ADR 0006: + +1. Register the capability โ€” already done for all 21. +2. Install a validated artefact here. +3. Implement its adapter and check the output against this contract. +4. Move `coming_soon` โ†’ `beta`. The app shows a Beta badge; results are stored + with their confidence, and nothing acts on them automatically. +5. Collect corrections. `corrections.selected_interpretation_id` captures when + the model's second-ranked candidate was the right one, which is the highest + value training signal the schema produces. +6. Promote to `enabled` when the corrections say it earns it. + +Step 6 is a decision about evidence, not a configuration change โ€” and today it is +also blocked mechanically, because both beta capabilities carry the +`global-coco` sentinel. diff --git a/models/_artefacts/yolox_m-0.1.1rc0.onnx b/models/_artefacts/yolox_m-0.1.1rc0.onnx new file mode 100644 index 0000000000000000000000000000000000000000..337bcebf334691622b7f00e204464490a09c8273 --- /dev/null +++ b/models/_artefacts/yolox_m-0.1.1rc0.onnx @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:21ff6cfdeb53b013bac2249599e55f00bff3cfdfdab37ed7a4620818c1d15b3f +size 101259744 diff --git a/models/cattle_detection/model_card.json b/models/cattle_detection/model_card.json new file mode 100644 index 0000000000000000000000000000000000000000..3c93b040971f7a407c707833e9e0014f0fb8f21c --- /dev/null +++ b/models/cattle_detection/model_card.json @@ -0,0 +1,27 @@ +{ + "model_id": "yolox-m-coco", + "version": "0.1.1rc0", + "capability_key": "cattle_detection", + "runtime": "yolox-onnx", + "artefact": "../_artefacts/yolox_m-0.1.1rc0.onnx", + "source": "https://github.com/Megvii-BaseDetection/YOLOX/releases/download/0.1.1rc0/yolox_m.onnx", + "license": "Apache-2.0", + "license_source": "https://github.com/Megvii-BaseDetection/YOLOX/blob/main/LICENSE", + "license_obligation": "None that reaches this service. Apache-2.0 requires the licence text and attribution to travel with redistributed copies; it imposes no copyleft and no network clause. Note what is NOT stated: Megvii publishes no separate licence for the released ONNX weights \u2014 the README, the 0.1.1rc0 release notes and the ONNXRuntime demo docs contain no mention of licensing at all. The repository's Apache-2.0 is read as covering the artefacts the repository distributes. That is an inference, and it is recorded as one in docs/adr/0017-ultralytics-licence.md.", + "sha256": "21ff6cfdeb53b013bac2249599e55f00bff3cfdfdab37ed7a4620818c1d15b3f", + "size_bytes": 101259744, + "training_data": "COCO 2017 train, 80 classes. Class 19 is `cow`; no cattle breed, sex or condition is distinguished. No Nigerian data of any kind.", + "validation_notes": "Measured 2026-08-20 by evaluation/run.py against evaluation/dataset.json \u2014 29 cattle photographs from Wikimedia Commons under CC BY-SA, CC0 and public-domain terms, 11 of them Nigerian. The figures are the service's, not the detector's: every image passes through MediaStore, the quality gate and DetectionCountRunner, so a frame the guard withholds counts against coverage rather than being excluded. Over the 18 exactly-labelled cattle frames: 16 counts published (coverage 0.889), MAE 0.62 animals, MAPE 11.1%, signed bias 0.00, within one animal on 87.5%. All 4 uncountable cattle frames were correctly refused a number, and no uncountable frame in the whole 61-image set leaked a count. No cattle record was affected by the manifest relabelling of 2026-08-20 (see the poultry card); these cattle figures are unchanged by it. On the 8 exactly-labelled Nigerian frames specifically: coverage 0.75, MAE 0.5, MAPE 6.83%. The ground truth is one non-expert annotator's count of the animals visible in each frame, checked by a second tiled pass; that is the weakest part of this measurement and no care in the method removes it. See docs/adr/0018-counting-model.md.", + "geography_validated": [ + "global-coco" + ], + "known_limits": [ + "Reports animals visible in one frame. Never the herd size.", + "Undercounts under occlusion, which is most of a real paddock. Measured signed bias on cattle is 0.00 over 16 frames, so over- and under-counts happened to balance on this set; that is not a claim that it is unbiased.", + "Does not distinguish cattle from buffalo, and can confuse distant cattle with horses and sheep.", + "Reports no number at all when the count keeps rising as the frame is read more finely, or when it exceeds 20 animals \u2014 the largest count ever checked against ground truth.", + "Validated on 18 exactly-labelled frames. That is a small set, it was labelled by one non-expert, and no licensed cattle-counting benchmark exists to check it against: the Bristol datasets are Non-Commercial and WAID carries no licence at all.", + "Two of the three countable frames the guard wrongly withholds are Nigerian, so the cost of refusing is not evenly spread across the geography the product is for." + ], + "alternate_runtime": "models/alternates/yolox_onnx \u2014 YOLOX-s, Apache-2.0, a third of the size and roughly twice as fast, at materially worse coverage. Swap by changing `runtime` and `artefact` here." +} diff --git a/models/poultry_count/model_card.json b/models/poultry_count/model_card.json new file mode 100644 index 0000000000000000000000000000000000000000..8f3d34bcff8ee62ff9fea421219ed58c4473b9d6 --- /dev/null +++ b/models/poultry_count/model_card.json @@ -0,0 +1,28 @@ +{ + "model_id": "yolox-m-coco", + "version": "0.1.1rc0", + "capability_key": "poultry_count", + "runtime": "yolox-onnx", + "artefact": "../_artefacts/yolox_m-0.1.1rc0.onnx", + "source": "https://github.com/Megvii-BaseDetection/YOLOX/releases/download/0.1.1rc0/yolox_m.onnx", + "license": "Apache-2.0", + "license_source": "https://github.com/Megvii-BaseDetection/YOLOX/blob/main/LICENSE", + "license_obligation": "None that reaches this service. Apache-2.0 requires the licence text and attribution to travel with redistributed copies; it imposes no copyleft and no network clause. Megvii publishes no separate licence for the released ONNX weights; the repository's Apache-2.0 is read as covering them, which is an inference recorded in docs/adr/0017-ultralytics-licence.md.", + "sha256": "21ff6cfdeb53b013bac2249599e55f00bff3cfdfdab37ed7a4620818c1d15b3f", + "size_bytes": 101259744, + "training_data": "COCO 2017 train, 80 classes. Class 14 is `bird`, and it is dominated by wild birds \u2014 this model has never been trained on poultry. No Nigerian data of any kind.", + "validation_notes": "Measured 2026-08-20 by evaluation/run.py on two sets. (1) evaluation/dataset.json, 32 poultry photographs from Wikimedia Commons: over the 16 exactly-labelled frames, 15 counts published (coverage 0.938), MAE 1.80 birds, MAPE 20.72%, signed bias -1.80, within one bird on 73.3%. All 16 uncountable frames were correctly refused a number, and no uncountable frame in the whole 61-image set leaked a count. **These figures are worse than a first draft of this card claimed (MAE 1.15, MAPE 17.36%).** That draft rested on 13 exactly-labelled frames; an audit of the manifest against its own agreement rule found three poultry records labelled `uncertain` that the rule makes `exact` \u2014 two with identical annotation passes \u2014 and all three are frames this model handles badly. They were relabelled and the numbers here are the result. (2) PIO, 452 annotated frames from two commercial broiler houses (CC BY 4.0, doi 10.5281/zenodo.16686320) \u2014 the dense regime this capability must never guess at. There the detector found nothing at all in over half the frames and the guard withheld a number on essentially all the rest; that is the intended behaviour and it is why this capability is documented as sparse-only. The worst individual results are 6 birds returned for 13 and 11 for 16, so accuracy degrades steadily above roughly ten birds in frame. The ground truth in set (1) is one non-expert annotator's count checked by a tiled second pass. See docs/adr/0018-counting-model.md.", + "geography_validated": [ + "global-coco" + ], + "known_limits": [ + "A count of birds visible in one frame. Never the flock population, and never a house reconciliation.", + "Sparse flocks only. Detection is the wrong method for a commercial house \u2014 the boxes overlap and NMS merges what is left. The answer there is a density head, which is not built.", + "Undercounts, and increasingly so with flock size. Measured signed bias is -1.80 birds on a set whose frames hold 16 birds or fewer; the worst single result is 6 returned for a human count of 13.", + "Works best below about ten birds in frame. Above that, treat the number as an indication.", + "Reports no number at all when the count keeps rising as the frame is read more finely, or when it exceeds 20 birds.", + "COCO's `bird` class is wild birds. Breed, age and sex are not distinguished, and no poultry-specific training has been done.", + "The largest verified count is 16 birds. Nothing above that has been checked, and above 20 no number is reported." + ], + "alternate_runtime": "models/alternates/yolox_onnx \u2014 YOLOX-s, Apache-2.0. Swap by changing `runtime` and `artefact` here." +} diff --git a/requirements.txt b/requirements.txt new file mode 100644 index 0000000000000000000000000000000000000000..0ef3af4bddb86d9dcfb5a12822f865d76650e45b --- /dev/null +++ b/requirements.txt @@ -0,0 +1,50 @@ +# Production dependencies. **Permissive licences only.** +# +# Ultralytics is deliberately absent. It is AGPL-3.0, and while the strict text +# of ยง13 is narrower than this file once claimed โ€” the source-offer duty is +# conditioned on *modifying* the program โ€” the vendor's published position is +# that a closed-source SaaS needs an Enterprise licence, and the measured cost of +# not having the argument is roughly nil (ADR 0017). +# `requirements-agpl.txt` installs it for anyone reproducing the comparison in +# `evaluation/`, and nothing in the deployment reads that file. + +fastapi>=0.115 +uvicorn[standard]>=0.32 +pydantic>=2.9 + +# Media handling, and the metrics behind the quality gate. +pillow>=10.4 +numpy>=1.26 + +# The shipped detector backend: YOLOX-m through onnxruntime, Apache-2.0. +# No torch, so the image is a few hundred megabytes rather than a few gigabytes, +# and the service starts in under a second instead of half a minute. +onnxruntime>=1.20 + +# TLS roots for scripts/install_models.py. A python.org build on macOS ships no +# CA store, and the alternative to certifi is turning verification off while +# downloading a binary that will run against farm photographs. +certifi>=2024.7 + +# Reading captures out of the container the API uploads them to (ADR 0012). +# `azure-identity` is what makes that credential-free: the Container App's +# managed identity, the same posture as services/api, with no account key to +# store or rotate. Both are unused when ANIMAP_MEDIA_PROVIDER is `local`. +azure-storage-blob>=12.23 +azure-identity>=1.19 + +# The hosted multimodal reasoner, when a deployment has chosen one. +# +# **Imported inside the call, never at module scope.** `app/adapters/transports` +# keeps its provider table as factories so that a deployment which has chosen no +# provider pays nothing for this โ€” the container runs on CPU with no torch and +# starts in under a second, and that is worth keeping. +# +# The dependency is listed rather than optional because the image is built once +# for every deployment and an install that succeeds locally and fails in the one +# environment that has a key is the worst place to find out. +anthropic>=1.0 + +# Hugging Face Space only. See space/space_app.py. +gradio==5.49.1 +spaces diff --git a/scripts/install_models.py b/scripts/install_models.py new file mode 100644 index 0000000000000000000000000000000000000000..62d680d6a7ab06ea423c30d59f127dd3af1f0390 --- /dev/null +++ b/scripts/install_models.py @@ -0,0 +1,143 @@ +"""Fetch the artefacts the committed model cards describe. + +Run by a person, once, before the service starts. **Not** a runtime download: +the service never calls this, and a model that is not on disk leaves its +capability unavailable rather than triggering a fetch (brief ยง30, ADR 0005). + +The cards are the input, not this file. Each one already names its source URL +and the sha256 of the bytes that were reviewed, so this script has no list of +its own to drift out of date โ€” it reads what review approved and refuses +anything else. + + .venv/bin/python scripts/install_models.py + .venv/bin/python scripts/install_models.py --check # verify, never fetch +""" + +from __future__ import annotations + +import argparse +import hashlib +import json +import ssl +import sys +import urllib.request +from pathlib import Path + +MODELS_DIR = Path(__file__).resolve().parent.parent / "models" +CHUNK = 1 << 20 +TIMEOUT_SECONDS = 300 + + +def _digest(path: Path) -> str: + sha = hashlib.sha256() + with path.open("rb") as handle: + while chunk := handle.read(CHUNK): + sha.update(chunk) + return sha.hexdigest() + + +def _ssl_context() -> ssl.SSLContext: + """Verify certificates, using certifi's bundle where the platform has none. + + A python.org build on macOS ships no CA store, so `urlopen` fails on every + HTTPS URL. Disabling verification would be the quick fix and the wrong one: + this script's whole job is fetching a binary that will later run against + farm photographs. + """ + try: + import certifi + except ImportError: + return ssl.create_default_context() + return ssl.create_default_context(cafile=certifi.where()) + + +def _download(url: str, destination: Path) -> None: + destination.parent.mkdir(parents=True, exist_ok=True) + # Downloaded beside the target and renamed, so an interrupted fetch cannot + # leave a truncated file that looks installed. + partial = destination.with_suffix(destination.suffix + ".partial") + request = urllib.request.Request(url, headers={"User-Agent": "animap-inference"}) + with urllib.request.urlopen( + request, timeout=TIMEOUT_SECONDS, context=_ssl_context() + ) as response, partial.open("wb") as handle: + while chunk := response.read(CHUNK): + handle.write(chunk) + partial.replace(destination) + + +def main(argv: list[str] | None = None) -> int: + parser = argparse.ArgumentParser(description=__doc__) + parser.add_argument("--check", action="store_true", + help="Report what is missing or mismatched; download nothing.") + args = parser.parse_args(argv) + + # Deeper than `discover()` looks on purpose. Cards under `alternates/` are + # installed and verified but never registered, so the backend that keeps the + # licence decision reversible stays runnable without ever being able to + # answer a request by accident. + cards = sorted(MODELS_DIR.glob("**/model_card.json")) + if not cards: + print(f"No model cards under {MODELS_DIR}.") + return 1 + + failures = 0 + seen: dict[Path, str] = {} + + for card_path in cards: + card = json.loads(card_path.read_text()) + target = (card_path.parent / card["artefact"]).resolve() + expected = card["sha256"] + label = f"{card_path.parent.name} โ†’ {target.name}" + + # Several capabilities share one detector. Two cards naming the same + # file with different checksums is a review error, and the download + # would silently make one of them right. + if target in seen and seen[target] != expected: + print(f" โœ— {label}: two cards claim different checksums for this file.") + failures += 1 + continue + seen[target] = expected + + if target.is_file(): + actual = _digest(target) + if actual == expected: + print(f" โœ“ {label}") + continue + print(f" โœ— {label}: on disk is {actual[:12]}โ€ฆ, card says {expected[:12]}โ€ฆ") + failures += 1 + continue + + if args.check: + print(f" โœ— {label}: not installed.") + failures += 1 + continue + + # Some artefacts have no URL to fetch because they are produced here โ€” + # an ONNX export of a published checkpoint is not a file the publisher + # hosts. `source` still records where the weights came from, because + # that is the provenance question; `produced_by` records how the bytes + # on disk were made from them. Downloading `source` would write a model + # card page into a .onnx file, so this refuses instead. + producer = card.get("produced_by") + if producer: + print(f" โœ— {label}: not installed, and it is a local export rather " + f"than a download. Run: {producer}") + failures += 1 + continue + + print(f" โ€ฆ {label}: fetching {card['source']}") + _download(card["source"], target) + actual = _digest(target) + if actual != expected: + target.unlink() + print(f" โœ— {label}: downloaded {actual[:12]}โ€ฆ, card says {expected[:12]}โ€ฆ. " + f"Removed.") + failures += 1 + continue + print(f" โœ“ {label} ({target.stat().st_size / 1e6:.1f} MB, {card['license']})") + + return 1 if failures else 0 + + +if __name__ == "__main__": + sys.exit(main()) diff --git a/space/FIXTURES.md b/space/FIXTURES.md new file mode 100644 index 0000000000000000000000000000000000000000..ce23f62be0af5d96483a08c17f724eaec174bea5 --- /dev/null +++ b/space/FIXTURES.md @@ -0,0 +1,39 @@ +# The two captures the Space serves + +`LocalMediaStore` resolves a media id to `/`, so a +fixture's filename *is* its media id. Both ids are `uuid5` of the slug under the +Space's own namespace, which means they are reproducible from the manifest and +stable across rebuilds โ€” a caller can hard-code one and it will still be there +next month. + + NAMESPACE = uuid5(NAMESPACE_URL, + "https://huggingface.co/spaces/bluman1/animap-inference/fixtures") + media_id = uuid5(NAMESPACE, slug) + +**Neither image is stored in the Animap repository**, for the reason +`.gitignore` gives for the other 59: the Commons frames are not Animap's work to +redistribute in git, and `evaluation/fetch.py` rebuilds them from their +manifest. `space/publish.py` holds two slugs, resolves each to a file in +`evaluation/images/`, checks its sha256 against `evaluation/dataset.json`, and +writes it into the Space under the media id above. A frame that has drifted, or +that carries a licence with an obligation attached, stops the publish. + +| Media id | Slug | Species | Human count | Licence | Attribution | +|---|---|---|---|---|---| +| `aa5e8481-8be6-509d-b1fa-f1a178c7cda0` | `cattle_ng_gombe_farm` | cattle | 7 | CC0 | Abdsomod, [Commons](https://commons.wikimedia.org/wiki/File%3ACows_farming_in_gombe_state_04.jpg) | +| `89f41a99-e419-5777-8009-f8a7ca8c4cfb` | `poultry_chickens_in_line` | poultry | 5 | Public domain | GaylaLin, [Commons](https://commons.wikimedia.org/wiki/File%3AChickens-in-line1.jpg) | + +**Public-domain terms on both, and that is the selection rule rather than a +coincidence.** The Space is public, so shipping a frame is redistributing it; +CC0 and public domain are the two rows in the evaluation manifest that carry no +attribution obligation to discharge on a surface nobody has built yet. Twenty +other cattle frames score better as demonstrations and every one of them is +CC BY-SA. + +The counts are one non-expert annotator's, checked by a tiled second pass โ€” the +same ground truth the model cards call the weakest part of their measurement. +They are here so a reader can tell a plausible answer from a wrong one, not as a +target the service is graded against. + +**These are demonstration captures, not test data.** Nothing asserts against +them; `evaluation/run.py` is what measures the detector, over all 61 frames. diff --git a/space/fixtures/89f41a99-e419-5777-8009-f8a7ca8c4cfb.jpg b/space/fixtures/89f41a99-e419-5777-8009-f8a7ca8c4cfb.jpg new file mode 100644 index 0000000000000000000000000000000000000000..94a512b8eb41a1e5c3bf3f0e63001b2f86033047 --- /dev/null +++ b/space/fixtures/89f41a99-e419-5777-8009-f8a7ca8c4cfb.jpg @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:0e195faf2bc3641d30ee7dc8f95ae59a0097c7c4d6db4779388bf53806779bc7 +size 648491 diff --git a/space/fixtures/aa5e8481-8be6-509d-b1fa-f1a178c7cda0.jpg b/space/fixtures/aa5e8481-8be6-509d-b1fa-f1a178c7cda0.jpg new file mode 100644 index 0000000000000000000000000000000000000000..2a606bfd5d319296416084e506b1513fd99c1ffb --- /dev/null +++ b/space/fixtures/aa5e8481-8be6-509d-b1fa-f1a178c7cda0.jpg @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:05df026b26768ca46df764974cae4987ba07e304bb328f544970e7f40da21f87 +size 396360 diff --git a/space_app.py b/space_app.py new file mode 100644 index 0000000000000000000000000000000000000000..8de488222427eb400cfe6f2f87497b4ee17329d9 --- /dev/null +++ b/space_app.py @@ -0,0 +1,248 @@ +"""The Space entry point: the Animap service, plus a page a person can use. + +Hugging Face runs this file, published to the Space root as `space_app.py`. + +**Not `app.py`.** The service's own package is `app/` and sits at that same +root; a module of the same name beside it is a collision Python resolves in +favour of the package, so an entry point called `app.py` cannot be imported by +name. The first staging was called that, and `import app` returned the package. It exists because **ZeroGPU is Gradio-SDK only** โ€” +the Docker Space this replaced ran the identical Azure image and could not be +given a GPU at any price, which is the whole reason CountGD has never been +measured end to end. + +## What is served, and at which path + + / the Gradio page โ€” upload a photograph, pick a capability + /health the service's own, unauthenticated + /capabilities the published contract + /jobs bearer token, exactly as on Azure + +`app.main:app` is mounted whole rather than reimplemented, so the endpoints a +caller already scripts against keep working and the quality gate, the counting +guard and the observation vocabulary are the same code Azure runs. Gradio is +mounted **into** FastAPI rather than the other way round, because the API is the +product and the page is a demonstration of it. + +## `@spaces.GPU`, and the honest state of it + +ZeroGPU allocates a GPU for the duration of a decorated call and releases it +after. `_run_capability` is decorated, so any capability that reaches for CUDA +gets one. + +**Nothing reaches for CUDA today, and that is worth saying plainly rather than +implying a speed-up nobody will see.** The two runnable artefacts here are +YOLOX-m and DINOv3, both ONNX, both executed by `onnxruntime` on CPU. What this +file buys is the *ability* to be given a GPU, which is the prerequisite for the +one capability that needs one: CountGD gets MAE **14.84** on broiler houses +against the deployed detector's **156.80**, and it is a PyTorch model that has +never been runnable anywhere in this project. The decorator is here so that +landing CountGD is a model change and not another SDK migration. + +The decorator is documented as effect-free off ZeroGPU, so the same file runs +locally and on a CPU Space. + +## What this Space still is not + +**Not the production media path.** Azure reads captures from `animapmedia` +through the Container App's managed identity. A Space has no managed identity, +so this serves `ANIMAP_MEDIA_PROVIDER=local` against two public-domain frames +and no farm data reaches it. That was true of the Docker Space and it is true +here; changing SDK changes nothing about it. + +**Not a licence-gate weakening.** The Docker build failed if an AGPL runtime +arrived. A Gradio Space has no Dockerfile to fail, so the gate that matters is +the runtime one that was always there: `providers.discover()` refuses to serve a +capability whose artefact fingerprints as a copyleft runtime, and `/health` +publishes `artefact_licenses` so a deployment in breach is visible from outside. +`scripts/install_models.py --check` is run below at start-up for the same +reason โ€” verify, never fetch. +""" + +from __future__ import annotations + +import os +import subprocess +import sys +from pathlib import Path + +HERE = Path(__file__).resolve().parent + +# **Before the service is imported.** `providers.discover()` reads the artefacts +# at import time, and a checksum that disagrees with its card should stop the +# Space rather than be discovered by a farm's job. On Docker this was a build +# step; a Gradio Space has no build step, so it is the first thing that runs. +_check = subprocess.run( + [sys.executable, "scripts/install_models.py", "--check"], + cwd=HERE, capture_output=True, text=True, +) +print(_check.stdout or "", flush=True) +if _check.returncode != 0: + print(_check.stderr, file=sys.stderr, flush=True) + raise SystemExit( + "An artefact does not match its model card. Nothing is served: a " + "capability answering from an artefact nobody verified is the one " + "thing this service must not do." + ) + +import gradio as gr # noqa: E402 +import uvicorn # noqa: E402 + +from app.capabilities import REGISTRY # noqa: E402 +from app.main import RUNNERS, app as service, media, provider # noqa: E402 + +try: + import spaces +except ImportError: # pragma: no cover - only present on a ZeroGPU Space + class _Spaces: + """A no-op stand-in, so this file runs unchanged off ZeroGPU. + + Handles both spellings โ€” bare `@spaces.GPU` and called + `@spaces.GPU(duration=60)` โ€” because the real decorator does and a + stand-in that only handled one would break the local run it exists for. + """ + + @staticmethod + def GPU(*args, **kwargs): + if args and callable(args[0]): + return args[0] + + def decorate(fn): + return fn + + return decorate + + spaces = _Spaces() + + +#: How long one call may hold a GPU. +#: +#: Sixty is ZeroGPU's default and comfortably over the measured worst case: a +#: dense frame runs three detection grids in a few seconds. It is deliberately +#: not raised "to be safe" โ€” a shorter declared duration improves queue priority +#: for everybody, and a capability that genuinely needs longer should say so +#: when it lands. +GPU_SECONDS = 60 + + +@spaces.GPU(duration=GPU_SECONDS) +def _run_capability(capability_key: str, image): + """One capability against one uploaded image, inside a GPU allocation. + + The decorated boundary is here rather than deeper because ZeroGPU allocates + per call: wrapping the whole job means one allocation for a whole answer, + where wrapping an inner tensor op would mean many. + """ + import datetime + import uuid + + from app.schemas import InferenceRequest + + capability = REGISTRY[capability_key] + runner = RUNNERS[capability_key] + + class _Store: + def open_image(self, ref): + return image + + request = InferenceRequest( + capability_key=capability_key, + farm_id=uuid.UUID(int=2), + subject_type="animal" if capability.species == "cattle" else "flock_cycle", + subject_id=uuid.UUID(int=1), + media_ids=[uuid.uuid4()], + captured_at=datetime.datetime.now(datetime.timezone.utc), + ) + return runner.run( + request=request, capability=capability, + artefact=provider.artefact_for(capability), store=_Store(), + request_id=uuid.uuid4(), + ) + + +def _describe(capability_key: str, image): + """The page's handler. Returns what the service returned, and its caveats.""" + if image is None: + return "Upload a photograph first.", {} + + capability = REGISTRY.get(capability_key) + if capability is None or not provider.can_run(capability): + return ( + f"**{capability_key}** cannot run here. A capability with no " + f"verified artefact and no configured model answers `unavailable` " + f"rather than a placeholder.", + {}, + ) + + try: + result = _run_capability(capability_key, image) + except Exception as exc: # a refusal is an answer; a crash is not + return f"The run failed: `{type(exc).__name__}: {exc}`", {} + + if result.recommended_recapture and not result.observations: + headline = "**Nothing is claimed for this photograph.** Take it again." + else: + rows = [ + f"- `{o.type}` = **{o.value}**" + (f" ({o.unit})" if o.unit else "") + for o in result.observations + ] + headline = "\n".join(rows) or "_No observation was produced._" + + caveats = "\n".join(f"> {w}" for w in result.warnings) + return f"{headline}\n\n{caveats}", result.model_dump(mode="json") + + +_RUNNABLE = sorted(k for k in RUNNERS if provider.can_run(REGISTRY[k])) + +with gr.Blocks(title="Animap inference") as page: + gr.Markdown( + "# Animap inference\n" + "Livestock models that refuse to invent a result. Its most important " + "property is what it **refuses**: a capability with no verified " + "artefact behind it says so rather than returning a plausible number.\n\n" + "**Read the caveats under the answer before you read the answer.** " + "Every capability here is `experimental`, and the hosted ones have " + "never been measured on a Nigerian herd.\n\n" + f"Media provider: `{media.provider}` โ€” two public-domain frames. " + "No farm data reaches this Space." + ) + with gr.Row(): + with gr.Column(): + choice = gr.Dropdown( + choices=_RUNNABLE or ["nothing is runnable here"], + value=(_RUNNABLE[0] if _RUNNABLE else None), + label="Capability", + ) + photo = gr.Image(type="pil", label="Photograph") + go = gr.Button("Read it", variant="primary") + with gr.Column(): + answer = gr.Markdown(label="What it said") + raw = gr.JSON(label="The result, whole") + + go.click(_describe, inputs=[choice, photo], outputs=[answer, raw]) + +# Gradio mounted into FastAPI, not the reverse: `/jobs` and `/health` are the +# product and this page is a demonstration of them. +app = gr.mount_gradio_app(service, page, path="/") + +# **This process must serve, and must block.** Both halves were learned from a +# failed deploy. +# +# Hugging Face runs this file and expects it to keep running. The first Gradio +# publish called `uvicorn.run` and died on +# +# ERROR: [Errno 98] error while attempting to bind on address +# ('0.0.0.0', 7861): address already in use +# +# so the second removed the call on a Space โ€” and the Space then imported +# cleanly, logged "new /", and **exited**, which HF reports as a runtime error +# with no error in the log. Every line looked healthy; there was simply no +# server. +# +# The port was the actual fault. `PORT` was 7861 and something already held it; +# Gradio's convention, and what the Space's proxy forwards to, is +# `GRADIO_SERVER_PORT` defaulting to 7860. +uvicorn.run( + app, + host="0.0.0.0", + port=int(os.environ.get("GRADIO_SERVER_PORT", 7860)), +)