Scrapling / scrapling /engines /_browsers

Commit History

fix: update code and docstrings to remove the old google referer logic
a28879b

Karim shoair commited on

fix(browsers): make flag concatenation type-safe
814b9a6

Karim shoair Christian Rost commited on

fix(stealth): improve stealth mode by removing unnecessary scripts
cf06b6e

Karim shoair commited on

fix(solver): Solve CF faster and handle websites that show captcha twice
446b9d5

Karim shoair commited on

fix: improve regex precision for Cloudflare challenge detection
979ef6f

RinZ27 commited on

fix: make browser fetchers type hints always present
7d2ea72

Karim shoair commited on

style: Use shorter and more accurate naming for constants
c2dbf4c

Karim shoair commited on

style: Fix all mypy errors and add type hints to untyped function bodies
31c2447

Karim shoair commited on

feat(browsers): Add option to block requests to specific domains
47dd985

Karim shoair commited on

feat(proxy control): Force a proxy at request level at any given point
32daccc

Karim shoair commited on

feat(spiders/fetchers): Adding proxy rotation logic and change retry logic
ed96cdc

Karim shoair commited on

style: update post requests type hints to be more accurate
c3c5ac6

Karim shoair commited on

fix(browsers): solving an issues with leaving playwright loop open when cdp connection fails
8ff23b3

Karim shoair commited on

feat(browsers): Add option to retry tabs that gives errors
024cbba

Karim shoair commited on

fix(browsers): Close pages that give error
8209e61

Karim shoair commited on

fix(browsers): solve a bug with setting referer on the request level
c7e573a

Karim shoair commited on

fix: add missing line
fff357e

Karim shoair commited on

refactor: internal API changes to be easily used as indicators for spiders
1803348

Karim shoair commited on

docs: change docstring style for some methods
daaad4e

Karim shoair commited on

fix(async stealth): wait the coroutine of the bounding box
1f369fd

Karim shoair commited on

build: update deps
7e1bee4

Karim shoair commited on

docs: shorten docstrings for some functions
1812d2b

Karim shoair commited on

fix: remove forgotten unused code
a7cc20d

Karim shoair commited on

fix: update the cookies type hint for the mcp server
d391239

Karim shoair commited on

fix: remove old proxy logic
f62e5ea

Karim shoair commited on

fix: update shell signatures for the autocompletion
05b2188

Karim shoair commited on

refactor(fetchers)!: Replace Camoufox with patchright and many optimizations
ee2299e

Karim shoair commited on

fix: improve the handling of useragent in chrome fetcher
123011a

Karim shoair commited on

docs: add docs for `timezone_id` argument
17e25c2

Karim shoair commited on

feat(dynamicSession): add the option to set timezone of the browser
1d52f39

Karim shoair commited on

refactor(fetchers): rename internal api
a17a010

Karim shoair commited on

fix(fetchers): better handling for infinite `network_idle`
c64a120

Karim shoair commited on

refactor(browser fetchers): Make all the type hints dynamic + Faster validation
c908f33

Karim shoair commited on

fix(validators): Fix overriding session values with request ones
4836205

Karim shoair commited on

pref(requests): Optimizing memory by specifiying slots
624e46f

Karim shoair commited on

refactor(requests): Make all the type hints dynamic
a68a70b

Karim shoair commited on

fix: added `raw_response` to separate collected response from rendered response
2784aa7

Karim shoair commited on

fix(cloudflare): Limit the waiting period for websites that captcha doesn't disappear after solving
c14cfb8

Karim shoair commited on

refactor(validation): All browser based fetchers are ~8-15% faster now in most cases
8bf21c0

Karim shoair commited on

feat(DynamicSession): New option to add extra browser flags
b6969b2

Karim shoair commited on

refactor(fetchers): Less duplicated code and better handling for unstable websites
6f2d7b6

Karim shoair commited on

fix: Addressing the issue of collecting response after `page_action` in #100
d765ef3

Karim shoair commited on

fix(solver): Improve the logic a bit
e6bbd0d

Karim shoair commited on

feat(Dynamic fetcher): Add `additional_args` option to customize browser as in StealthyFetcher
0de8025

Karim shoair commited on

feat(fetchers): Add argument to control the user data directory
c61a805

Karim shoair commited on

fix(CF solver): further improve the detection
76845e2

Karim shoair commited on

fix(fetchers): More accurate collection of the final response
0de3dd1

Karim shoair commited on

fix(camoufox): Fix a typo in the async solver
696a6a3

Karim shoair commited on

refactor: Making the requests logic better and cleaner
906b02a

Karim shoair commited on

refactor: Making all the codebase acceptable by PyRight
e5ecf76

Karim shoair commited on