Harness catalog
Dimension legend
| Code | Dimension | Values |
|---|---|---|
| L | Advanced lexical retrieval | 0 = exact/regex only; 1 = BM25 plus fuzzy retrieval |
| S | Syntax representation | 0 = raw lines; 1 = Tree-sitter symbols and chunks |
| D | Dense retrieval | 0 = off; 1 = code embeddings |
| G | Structural graph expansion | 0, 1, or 2 dependency hops |
| Q | Query policy | one-shot or iterative reformulation |
| I | Tool interface | unified search tool or specialized tools |
| P | Context packing | ranked snippets, skeletons, whole files, or role summaries |
Exact path, substring, and regex search remains available in H000-H015 and H020. It is removed only from explicitly labeled controls. When two or more of L, S, and D are active, their ranked lists use reciprocal-rank fusion (RRF).
The 21 initial treatments
| ID | L | S | D | G | Q | I | P | Purpose |
|---|---|---|---|---|---|---|---|---|
| H000 | 0 | 0 | 0 | 0 | one-shot | unified | snippets | Exact/regex baseline |
| H001 | 1 | 0 | 0 | 0 | one-shot | unified | snippets | Lexical main effect |
| H002 | 0 | 1 | 0 | 0 | one-shot | unified | snippets | Syntax main effect |
| H003 | 0 | 0 | 1 | 0 | one-shot | unified | snippets | Dense main effect |
| H004 | 1 | 1 | 0 | 0 | one-shot | unified | snippets | Lexical × syntax |
| H005 | 1 | 0 | 1 | 0 | one-shot | unified | snippets | Lexical × dense |
| H006 | 0 | 1 | 1 | 0 | one-shot | unified | snippets | Syntax × dense |
| H007 | 1 | 1 | 1 | 0 | one-shot | unified | snippets | Full retrieval, no graph |
| H008 | 1 | 1 | 1 | 1 | one-shot | unified | snippets | One-hop graph reference |
| H009 | 1 | 1 | 1 | 2 | one-shot | unified | snippets | Two-hop graph |
| H010 | 1 | 1 | 1 | 1 | iterative | unified | snippets | Iterative-query effect |
| H011 | 1 | 1 | 1 | 1 | one-shot | specialized | snippets | Interface effect |
| H012 | 1 | 1 | 1 | 1 | iterative | specialized | snippets | Query × interface |
| H013 | 1 | 1 | 1 | 1 | one-shot | unified | skeletons | Skeleton packing |
| H014 | 1 | 1 | 1 | 1 | one-shot | unified | whole files | Whole-file packing |
| H015 | 1 | 1 | 1 | 1 | one-shot | unified | role summaries | Summary packing |
| H016 | – | – | – | – | one-shot | unified | – | No-search negative control |
| H017 | – | – | – | – | one-shot | unified | random | Token-matched random-context control |
| H018 | – | – | – | – | one-shot | unified | gold files | Oracle-file upper bound |
| H019 | – | – | – | – | one-shot | unified | gold functions | Oracle-function upper bound |
| H020 | 1 | 1 | 1 | 1 | iterative | unified | snippets | Adaptive source router |
The unrestricted seven-dimension Cartesian product has 384 combinations. It is not the primary design because most cells would answer no new clean causal question while greatly multiplying model cost. The 21 treatments form focused factorial and ablation blocks. If later evidence justifies another combination, it receives a new file and ID.
Both a full configuration hash and a causal treatment hash are computed. The latter excludes the ID, name, and description so that relabeling cannot conceal a duplicate treatment.