artifacts/patent_phase0_sample50_ai_review.md

Patent Phase-0 sample-50 AI review

Follow-up to artifacts/patent_phase0_zero_yield_investigation.md.

This run increased the retrieval target from the earlier observed 10 patents to 50 patents and applied an AI review pass to the high-attention candidate spans. No Poucher targets, model-ready targets, training data, or publish artifacts were changed.

Commands

python scripts/patent_substantivity_phase0.py \
  --sample-size 50 \
  --pages-per-query 8 \
  --delay 0.2 \
  --timeout 30 \
  --retrieval-source auto \
  --sample-out data/patent_phase0_sample_50.jsonl \
  --candidates-out data/patent_substantivity_phase0_candidates_50.jsonl \
  --report-out artifacts/patent_substantivity_phase0_report_50.md

jq -c 'select(.confidence >= 0.7 and ((.compound_ref_verbatim != null) or (.resolved_cas != null)))' \
  data/patent_substantivity_phase0_candidates_50.jsonl \
  > data/patent_substantivity_phase0_candidates_50_high_attention.jsonl

python scripts/review_patent_substantivity_tuples.py \
  --candidates data/patent_substantivity_phase0_candidates_50_high_attention.jsonl \
  --tuples-out data/patent_substantivity_reviewed_tuples_50_high_attention_hermes.jsonl \
  --audit-out artifacts/patent_substantivity_tuple_review_50_high_attention_hermes.jsonl \
  --summary-out artifacts/patent_substantivity_tuple_review_50_high_attention_hermes.json \
  --md-out artifacts/patent_substantivity_tuple_review_50_high_attention_hermes.md \
  --backend hermes \
  --batch-size 5 \
  --sleep 0.1 \
  --timeout 180

An Anthropic review attempt was also made against all 191 candidate spans, but all calls failed with account credit errors. The successful AI review below used the Hermes backend.

Expanded retrieval

• Sample patents fetched: 50.
• Retrieval status: ok.
• Patents with numeric keyword spans: 36.
• Numeric keyword spans: 191.
• High-attention spans: 35.
• Spans with printed CAS references: 0/191.
• Spans with printed compound/example reference or CAS: 35/191.

Construct inventory:

• fragrance_intensity: 68.
• longevity: 42.
• odor_intensity: 37.
• substantivity: 16.
• headspace_concentration_or_intensity: 15.
• duration: 9.
• persistence: 2.
• unknown_numeric_measure: 1.
• retention: 1.

AI review result

• Candidate file reviewed: data/patent_substantivity_phase0_candidates_50_high_attention.jsonl.
• Backend: Hermes.
• Processed spans: 35.
• Accepted single-molecule measured tuples: 0.
• Accepted tuples with Poucher CAS overlap: 0.
• Failures: 0.

The high-attention AI review rejected all spans. The main rejection pattern was not a downstream extraction bug; it was that the stronger-looking spans still described composition-level examples, fragrance mixtures, fixative/modulator comparisons, analytical method setup, or procedural timepoints rather than one named molecule with one measured substantivity/longevity/tenacity outcome.

Current verdict

Increasing sample size improved retrieval coverage substantially, but did not produce a usable Poucher-aligned patent tuple set. The bottleneck remains source targeting: the current Google Patents query/fetch path mostly surfaces broad formula examples, methods, figure captions, and mixture claims.

Recommended next retrieval change is to target table-bearing patents directly: query and fetch examples/tables where a named material row and a measured intensity, persistence, residual percentage, or evaporation profile are present in the same table. The current span-level probe should not be merged into training data.