Critic-Gated Retrieval Extension

The retrieval module is optional. It does not change core CIL generation or DoVLA training unless a user explicitly wraps inference with retrieval.

Goal

At inference time, retrieve same-state or similar-state CIL exemplars to condition a policy. The retriever can combine observation-language similarity with predicted utility or effect relevance, instead of using nearest-neighbor similarity alone.

Package Layout

retrieval/embeddings.py: deterministic toy observation-language and record embeddings.
retrieval/index.py: in-memory embedding index over CIL records or groups.
retrieval/retriever.py: retrieval modes, critic-gated ranking, and policy wrapper.
retrieval/prompting.py: compact prompt/table rendering for retrieved exemplars.
retrieval/eval.py: retrieval baseline metrics.

Retrieval Roles

The retriever attempts to return:

positive successful exemplar
near-miss failure exemplar
recovery or partial-success exemplar

When a role is unavailable, it fills remaining slots with the strongest relevant neighbors.

Modes

no_retrieval
nearest_neighbor
success_only
success_failure_contrastive
critic_gated

critic_gated uses critic.score_atom(...) when a TransferCritic-style critic and context are provided. Otherwise it falls back to reward/effect relevance from the CIL record.

Inference Wrapper

RetrievalConditionedPolicyWrapper exposes:

policy(obs, instruction, retrieved_examples=None)

If the wrapped model supports forward_policy(..., retrieved_examples=...), retrieved examples are passed through. Otherwise the wrapper calls the ordinary policy method and stores the retrieved examples for inspection.

Metrics

The lightweight retrieval evaluator reports:

causalstress_success: fraction of queries with a successful positive exemplar
instruction_controllability: fraction with success/failure contrastive support
near_miss_robustness: fraction with a near-miss failure exemplar
retrieval_coverage: fraction with at least one retrieved exemplar

These are retrieval diagnostics, not substitutes for policy rollout evaluation.