Karim shoair commited on
Commit ·
24ea5ca
1
Parent(s): 6290d23
docs: update requests page
Browse files- docs/fetching/static.md +6 -5
docs/fetching/static.md
CHANGED
|
@@ -25,7 +25,7 @@ All methods for making requests here share some arguments, so let's discuss them
|
|
| 25 |
- **timeout**: The number of seconds to wait for each request to be finished. **Defaults to 30 seconds**.
|
| 26 |
- **retries**: The number of retries that the fetcher will do for failed requests. **Defaults to three retries**.
|
| 27 |
- **retry_delay**: Number of seconds to wait between retry attempts. **Defaults to 1 second**.
|
| 28 |
-
- **impersonate**: Impersonate specific browsers' TLS fingerprints. Accepts browser strings like `"chrome110"`, `"firefox102"`, `"safari15_5"` to use specific versions or `"chrome"`, `"firefox"`, `"safari"`, `"edge"` to automatically use the latest version available. This makes your requests appear
|
| 29 |
- **http3**: Use HTTP/3 protocol for requests. **Defaults to False**. It might be problematic if used with `impersonate`.
|
| 30 |
- **cookies**: Cookies to use in the request. Can be a dictionary of `name→value` or a list of dictionaries.
|
| 31 |
- **proxy**: As the name implies, the proxy for this request is used to route all traffic (HTTP and HTTPS). The format accepted here is `http://username:password@localhost:8030`.
|
|
@@ -39,14 +39,15 @@ All methods for making requests here share some arguments, so let's discuss them
|
|
| 39 |
|
| 40 |
> Note: <br/>
|
| 41 |
> 1. The currently available browsers to impersonate are (`"edge"`, `"chrome"`, `"chrome_android"`, `"safari"`, `"safari_beta"`, `"safari_ios"`, `"safari_ios_beta"`, `"firefox"`, `"tor"`)<br/>
|
| 42 |
-
> 2. The available browsers to impersonate
|
|
|
|
| 43 |
|
| 44 |
Other than this, for further customization, you can pass any arguments that `curl_cffi` supports for any method if that method doesn't already support it.
|
| 45 |
|
| 46 |
### HTTP Methods
|
| 47 |
There are additional arguments for each method, depending on the method, such as `params` for GET requests and `data`/`json` for POST/PUT/DELETE requests.
|
| 48 |
|
| 49 |
-
Examples are the best way to explain this
|
| 50 |
|
| 51 |
> Hence: `OPTIONS` and `HEAD` methods are not supported.
|
| 52 |
#### GET
|
|
@@ -166,7 +167,7 @@ And for asynchronous requests, it's a small adjustment
|
|
| 166 |
|
| 167 |
## Session Management
|
| 168 |
|
| 169 |
-
For making multiple requests with the same configuration, use the `FetcherSession` class. It can be used in both synchronous and asynchronous code without issue; the class detects and changes the session type
|
| 170 |
|
| 171 |
The `FetcherSession` class can accept nearly all the arguments that the methods can take, which enables you to specify a config for the entire session and later choose a different config for one of the requests effortlessly, as you will see in the following examples.
|
| 172 |
|
|
@@ -181,7 +182,7 @@ with FetcherSession(
|
|
| 181 |
timeout=30,
|
| 182 |
retries=3
|
| 183 |
) as session:
|
| 184 |
-
# Make multiple requests with the same settings
|
| 185 |
page1 = session.get('https://scrapling.requestcatcher.com/get')
|
| 186 |
page2 = session.post('https://scrapling.requestcatcher.com/post', data={'key': 'value'})
|
| 187 |
page3 = session.get('https://api.github.com/events')
|
|
|
|
| 25 |
- **timeout**: The number of seconds to wait for each request to be finished. **Defaults to 30 seconds**.
|
| 26 |
- **retries**: The number of retries that the fetcher will do for failed requests. **Defaults to three retries**.
|
| 27 |
- **retry_delay**: Number of seconds to wait between retry attempts. **Defaults to 1 second**.
|
| 28 |
+
- **impersonate**: Impersonate specific browsers' TLS fingerprints. Accepts browser strings or a list of them like `"chrome110"`, `"firefox102"`, `"safari15_5"` to use specific versions or `"chrome"`, `"firefox"`, `"safari"`, `"edge"` to automatically use the latest version available. This makes your requests appear to come from real browsers at the TLS level. If you pass it a list of strings, it will choose a random one with each request. **Defaults to the latest available Chrome version.**
|
| 29 |
- **http3**: Use HTTP/3 protocol for requests. **Defaults to False**. It might be problematic if used with `impersonate`.
|
| 30 |
- **cookies**: Cookies to use in the request. Can be a dictionary of `name→value` or a list of dictionaries.
|
| 31 |
- **proxy**: As the name implies, the proxy for this request is used to route all traffic (HTTP and HTTPS). The format accepted here is `http://username:password@localhost:8030`.
|
|
|
|
| 39 |
|
| 40 |
> Note: <br/>
|
| 41 |
> 1. The currently available browsers to impersonate are (`"edge"`, `"chrome"`, `"chrome_android"`, `"safari"`, `"safari_beta"`, `"safari_ios"`, `"safari_ios_beta"`, `"firefox"`, `"tor"`)<br/>
|
| 42 |
+
> 2. The available browsers to impersonate, along with their corresponding versions, are automatically displayed in the argument autocompletion and updated with each `curl_cffi` update.<br/>
|
| 43 |
+
> 3. If any of the arguments `impersonate` or `stealthy_headers` are enabled, the fetchers will automatically generate real browser headers that match the browser version used.
|
| 44 |
|
| 45 |
Other than this, for further customization, you can pass any arguments that `curl_cffi` supports for any method if that method doesn't already support it.
|
| 46 |
|
| 47 |
### HTTP Methods
|
| 48 |
There are additional arguments for each method, depending on the method, such as `params` for GET requests and `data`/`json` for POST/PUT/DELETE requests.
|
| 49 |
|
| 50 |
+
Examples are the best way to explain this:
|
| 51 |
|
| 52 |
> Hence: `OPTIONS` and `HEAD` methods are not supported.
|
| 53 |
#### GET
|
|
|
|
| 167 |
|
| 168 |
## Session Management
|
| 169 |
|
| 170 |
+
For making multiple requests with the same configuration, use the `FetcherSession` class. It can be used in both synchronous and asynchronous code without issue; the class automatically detects and changes the session type, without requiring a different import.
|
| 171 |
|
| 172 |
The `FetcherSession` class can accept nearly all the arguments that the methods can take, which enables you to specify a config for the entire session and later choose a different config for one of the requests effortlessly, as you will see in the following examples.
|
| 173 |
|
|
|
|
| 182 |
timeout=30,
|
| 183 |
retries=3
|
| 184 |
) as session:
|
| 185 |
+
# Make multiple requests with the same settings and the same cookies
|
| 186 |
page1 = session.get('https://scrapling.requestcatcher.com/get')
|
| 187 |
page2 = session.post('https://scrapling.requestcatcher.com/post', data={'key': 'value'})
|
| 188 |
page3 = session.get('https://api.github.com/events')
|