style: remove unused import

fix: update code and docstrings to remove the old google referer logic

build: pump up deps and browsers versions

Merge branch 'dev' into fix/google-referrer-spoof

fix: return bare Google URL in referer instead of search query

- Add pre-compiled XPath text selector

fix: Selector.get_all_text() doesn't get all text #167

fix(type hints): use correct import for Python < 3.12 (Fixes #163 )

build: pump up the version

fix(browsers): make flag concatenation type-safe

fix(stealth): improve stealth mode by removing unnecessary scripts

fix(mcp): make mcp use less tokens by striping useless tags

fix(solver): Solve CF faster and handle websites that show captcha twice

build: pump version up and the deps

fix: complete MCP schema validation for remaining bare array types

Merge branch 'dev' into fix/mcp-schema-auth-cookies

fix: make MCP get schemas validator-safe

fix: improve regex precision for Cloudflare challenge detection

fix(checkpoint): correct error message

fix(spiders): corrections and removal of dead code

style: docstrings corrections for accuracy

fix: make browser fetchers type hints always present

fix: use more accurate naming for proxy rotation strategy

fix(parser)!: Optimize parser for repeated operations

style(spiders): Lower the default number of concurrent requests

fix: add typed overloads to Selectors.get() for proper default type inference

build: pump up the version status

style: expose `ProxyRotator` class to the public API

style: Use shorter and more accurate naming for constants

build: pump up version and deps

style: Fix all mypy errors and add type hints to untyped function bodies

feat(browsers): Add option to block requests to specific domains

style(parser): Improve the type hint for `find_by_text` and `find_by_regex`

fix(response): Force the body of the response to always be bytes

feat(parser)!: Make all selection return selector objects by default

feat(proxy control): Force a proxy at request level at any given point

feat(spiders/fetchers): Adding proxy rotation logic and change retry logic

fix(browsers): Improve stealth and speed by adjusting flags

fix(browsers): disable autoplay for all browsers

feat(parser): replacing `tldextract` with `tld` library

fix(requests): return the cached fingerprint on second call

feat(spiders/requests): Use a reliable method for fingerprint generation

style: update post requests type hints to be more accurate

build: Add w3lib to deps

feat(spiders): Make graceful shutdown always enabled

fix(spiders): handle errors with manual pause during stream mode

fix(spiders): fix a bug with session stats counter

fix(spiders): Fix for when the pausing system has large queue

feat(spiders): Add pause/resume system for crawls

fix(browsers): solving an issues with leaving playwright loop open when cdp connection fails