File size: 3,557 Bytes
83db774 | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 | # Patent Phase-0 sample-50 AI review
Follow-up to `artifacts/patent_phase0_zero_yield_investigation.md`.
This run increased the retrieval target from the earlier observed 10 patents to
50 patents and applied an AI review pass to the high-attention candidate spans.
No Poucher targets, model-ready targets, training data, or publish artifacts were
changed.
## Commands
```text
python scripts/patent_substantivity_phase0.py \
--sample-size 50 \
--pages-per-query 8 \
--delay 0.2 \
--timeout 30 \
--retrieval-source auto \
--sample-out data/patent_phase0_sample_50.jsonl \
--candidates-out data/patent_substantivity_phase0_candidates_50.jsonl \
--report-out artifacts/patent_substantivity_phase0_report_50.md
jq -c 'select(.confidence >= 0.7 and ((.compound_ref_verbatim != null) or (.resolved_cas != null)))' \
data/patent_substantivity_phase0_candidates_50.jsonl \
> data/patent_substantivity_phase0_candidates_50_high_attention.jsonl
python scripts/review_patent_substantivity_tuples.py \
--candidates data/patent_substantivity_phase0_candidates_50_high_attention.jsonl \
--tuples-out data/patent_substantivity_reviewed_tuples_50_high_attention_hermes.jsonl \
--audit-out artifacts/patent_substantivity_tuple_review_50_high_attention_hermes.jsonl \
--summary-out artifacts/patent_substantivity_tuple_review_50_high_attention_hermes.json \
--md-out artifacts/patent_substantivity_tuple_review_50_high_attention_hermes.md \
--backend hermes \
--batch-size 5 \
--sleep 0.1 \
--timeout 180
```
An Anthropic review attempt was also made against all 191 candidate spans, but
all calls failed with account credit errors. The successful AI review below used
the Hermes backend.
## Expanded retrieval
- Sample patents fetched: 50.
- Retrieval status: ok.
- Patents with numeric keyword spans: 36.
- Numeric keyword spans: 191.
- High-attention spans: 35.
- Spans with printed CAS references: 0/191.
- Spans with printed compound/example reference or CAS: 35/191.
Construct inventory:
- `fragrance_intensity`: 68.
- `longevity`: 42.
- `odor_intensity`: 37.
- `substantivity`: 16.
- `headspace_concentration_or_intensity`: 15.
- `duration`: 9.
- `persistence`: 2.
- `unknown_numeric_measure`: 1.
- `retention`: 1.
## AI review result
- Candidate file reviewed: `data/patent_substantivity_phase0_candidates_50_high_attention.jsonl`.
- Backend: Hermes.
- Processed spans: 35.
- Accepted single-molecule measured tuples: 0.
- Accepted tuples with Poucher CAS overlap: 0.
- Failures: 0.
The high-attention AI review rejected all spans. The main rejection pattern was
not a downstream extraction bug; it was that the stronger-looking spans still
described composition-level examples, fragrance mixtures, fixative/modulator
comparisons, analytical method setup, or procedural timepoints rather than one
named molecule with one measured substantivity/longevity/tenacity outcome.
## Current verdict
Increasing sample size improved retrieval coverage substantially, but did not
produce a usable Poucher-aligned patent tuple set. The bottleneck remains source
targeting: the current Google Patents query/fetch path mostly surfaces broad
formula examples, methods, figure captions, and mixture claims.
Recommended next retrieval change is to target table-bearing patents directly:
query and fetch examples/tables where a named material row and a measured
intensity, persistence, residual percentage, or evaporation profile are present
in the same table. The current span-level probe should not be merged into
training data.
|