search: new branch using serper as provider

fix: curl response headers

curl-impersonate: add zstd compression options

puppeteer: tweak the ua a bit

puppeteer: ditch puppeteer-stealth and use the real stable chrome

fix: expect malformed url in iframes

style: prefer const for originalSrc

fix: image in summary

fix: unhandled rejection case

fix

fix: compressed response from curl

deployment: dedicated server script for cloud-run (#1139)

fix: content suffix for markdown respond format

remove more attrs in readerlm preprocessing

fix: tweak readerlm

fix: tweak readerlm parameters

fix: readerlm repetition_penalty

fix: readerlm params

fix: lm and related options

fix: curl with errors

feat: new lm engine

feat: gemini to replace blip2 (#1129)

feat: domain profile (#1127)

feat(crawl): viewport options

feat: expose setViewport to page script

fix: pdf upload in multipart

fix: potential gfm performance issue

feat: opt out gfm/table

fix: img turndown rules

fix: tricks applied by puppeteer-extra-plugin-stealth

fix: img with srcset only

fix: poorly transformed detection

feat: image retention config

fix: potential invalid html

fix: there may be invalid root doc

fix: deps and bugs

fix: crawling IP url

fix: expect cookie encoding issue

feat: compound response

feat: script injecting and tools

fix: turndown rules

fix: headers in overridden request

fix: salvaging with google cache does not work anymore

fix: walk around locale setting bug

fix: detect poorly transformed contents

revert: domain cannot be un-doomed due to google function wrapper

fix: force process quit on firebase issue

feat: expand shadow dom

fix: remove link url escaping

fix: pdf mode and google web cache