fix: firebase limit on document size causing cache failures

fix: reduce frequency of screenshot if possible

refactor: reorganize features (#37)

fix: process not quitting on errors

fix: allocating incorrect max value due to missing parentheses (#26)

fix: explicitly reject non http protocols

feat: block media resources to improve speed

fix: catch turndown errors

chore: turn up concurrency a bit base on analysis

fix: expose publishedTime if possible

fix: url wrong normalization

fix: keep url details

fix: favor nominal url over real url

chore: increase max instances to target 10k concurrent requests

feat: fallback to google archive (#16)

fix: give expireAt for image cache

feat: add image captioning (#6)

fix: catch parse error

fix: clean broken markdown

feat: clean broken markdown

chore: rename url2text to reader

puppeteer stealth

fix: use gpt bot UA

fix

fix

fix

fix

fix

fix

fix

fix

fix

chore: rename url2text to reader

chore: clean code

fix

fix

gfm

fix

fix

wip

wip

wip