production: harden catalog tenants analytics and widget install

fix(availability): trust schema.org marker over loose first-phrase stock scan

Crawler: fix product over-classification, --clear seen-cache, WooCommerce URLs

crawl: drop storefront utility pages (cart/checkout/account) — gate quarantine fix

fix: ship docs-only crawler mode

chunker: fix 90% content loss on long docs pages + structure-aware prose Mode 3

agentfactory: universal docs-DB gate + answer-agnostic docs-fact extraction

Universal structured-catalog query engine + ingestion persistence fix

fix: tsc_pk gate residuals at source + in-place scrub for existing chunks

fix: tsc_pk gate residuals — entity-decode headings, drop price<=0, 4-char card titles

fix: tsc_pk gate residuals + auto-crawl overhaul (httpx-first, breaker, categories)

fix: deterministic product bounds answers

fix: authority_price (og/JSON-LD) end-to-end + rank-anchor stoplists + budget bounds

feat: universal junk remover — corpus-frequency boilerplate filter + og:title/JSON-LD title authority

feat: category propagation from crawl graph - products inherit listing-page categories

fix: universal junk identifier for product extraction - structure over content

feat: offline pipeline validator + extract refactor — no more crawl-to-test

feat: deterministic range/review/count retrieval + cart-widget scrub

fix: strip dash-site-suffix titles ('X – Babyfy' → 'X')

fix: HTML-entity titles and price-fragment titles

fix: zero prices excluded from extraction fallback and price ranking

fix: slug-matching title head is final authority in product extraction

fix: detail pages with related-item carousels no longer flip to catalog

fix: price fallback scores title-price distance — prose amounts lose to header price

fix: word-boundary on rs/pkr price patterns

debug: chunker diagnostics + isolate per-page chunker failures

feat: £/€ currency support across crawl + retrieval price logic

fix: category pages misclassified as product pages poisoned retrieval

fix: universal product crawl + price-ranking retrieval

crawler: retry queue + 429 backoff, robots.txt Disallow, config-driven docs paths

Fix Bug1/2/3: product gate, description multiline, IDK threshold

feat(crawl): multi-layer URL provenance, docs-safe short-page threshold, title hint fix

fix(crawl): reduce docs overmatch + richer per-URL audit

feat(crawl): docs classification + unified discovery audit

Merge remote-tracking branch 'hf/main'

fix: crawler overhaul, retrieval fixes, IDK price bypass

fix(crawler): improve docs titles and eligibility

fix(crawler): make docs pages stay docs

fix(crawler): keep docs pages and lower save threshold

fix(crawler): use product price capture regex

fix(crawler): import product price capture regex

chore(crawler): nudge rebuild for title fix

fix(crawler): prioritize price-tail product titles

fix(crawler): extract product titles from price tails

fix(crawler): treat site chrome as generic titles

fix: tighten store titles and ranking answers

fix: sharpen store titles and price ranking

fix(crawler): repair bullet-list chunking

fix(crawler): recognize singular product URLs

fix(crawler): classify catalog listings separately