Update README with proper YAML configuration
Browse files
README.md
CHANGED
|
@@ -9,199 +9,67 @@ app_file: app.py
|
|
| 9 |
pinned: false
|
| 10 |
---
|
| 11 |
|
| 12 |
-
# Browser Automation Tool
|
| 13 |
-
|
| 14 |
-
A comprehensive web scraping and browser automation platform - an alternative to browserbase.com. This Hugging Face Space provides powerful tools for web data extraction, screenshot capture, form automation, and multi-URL scraping.
|
| 15 |
-
|
| 16 |
-
## Features ๐
|
| 17 |
-
|
| 18 |
-
### ๐ Single URL Analysis
|
| 19 |
-
- **Screenshot Capture**: Take high-quality screenshots of any webpage
|
| 20 |
-
- **Data Extraction**: Extract text, links, images, forms, and custom elements
|
| 21 |
-
- **Custom Selectors**: Use CSS selectors to extract specific data
|
| 22 |
-
- **Headless/Headed Mode**: Choose between invisible or visible browser operation
|
| 23 |
-
|
| 24 |
-
### ๐ Multiple URLs Scraping
|
| 25 |
-
- **Concurrent Scraping**: Process multiple URLs simultaneously
|
| 26 |
-
- **Configurable Workers**: Control the number of concurrent processes
|
| 27 |
-
- **Batch Processing**: Extract data from entire lists of URLs
|
| 28 |
-
- **Structured Output**: Get organized results in JSON format
|
| 29 |
-
|
| 30 |
-
### ๐ Form Automation
|
| 31 |
-
- **Smart Form Detection**: Automatically detect form fields
|
| 32 |
-
- **Automated Filling**: Fill forms with provided data
|
| 33 |
-
- **Field Type Recognition**: Handle text, email, password, textarea fields
|
| 34 |
-
- **Form Submission**: Submit forms and capture responses
|
| 35 |
-
|
| 36 |
-
### ๐ฎ Interactive Mode
|
| 37 |
-
- **Real-time Screenshot**: Live view of browser activity
|
| 38 |
-
- **Console Output**: Monitor JavaScript console in real-time
|
| 39 |
-
- **Error Handling**: Graceful handling of failed requests
|
| 40 |
-
- **Progress Tracking**: Visual feedback for long operations
|
| 41 |
-
|
| 42 |
-
## How to Use ๐ก
|
| 43 |
-
|
| 44 |
-
### 1. Single URL Analysis
|
| 45 |
-
1. Enter a URL in the "Single URL Analysis" tab
|
| 46 |
-
2. Choose screenshot mode (full page or viewport)
|
| 47 |
-
3. Set wait time for page loading
|
| 48 |
-
4. Click "Run Analysis" and wait for results
|
| 49 |
-
|
| 50 |
-
### 2. Multiple URLs Scraping
|
| 51 |
-
1. Select the "Multiple URLs Scraping" tab
|
| 52 |
-
2. Paste multiple URLs (one per line)
|
| 53 |
-
3. Configure number of workers (1-10)
|
| 54 |
-
4. Set wait time per URL
|
| 55 |
-
5. Click "Start Scraping" and monitor progress
|
| 56 |
-
|
| 57 |
-
### 3. Form Automation
|
| 58 |
-
1. Go to the "Form Automation" tab
|
| 59 |
-
2. Enter target URL and form data
|
| 60 |
-
3. Specify CSS selectors (or use auto-detection)
|
| 61 |
-
4. Click "Fill and Submit" to automate the process
|
| 62 |
-
|
| 63 |
-
### 4. Live Console
|
| 64 |
-
1. Navigate to "Live Console" tab
|
| 65 |
-
2. Enter any URL to monitor
|
| 66 |
-
3. Watch real-time browser console output
|
| 67 |
-
4. See JavaScript errors and logs as they happen
|
| 68 |
-
|
| 69 |
-
## Supported Features ๐ ๏ธ
|
| 70 |
-
|
| 71 |
-
### Browser Capabilities
|
| 72 |
-
- **Headless Chrome**: Fast, efficient browser automation
|
| 73 |
-
- **Customizable Viewport**: Set specific screen dimensions
|
| 74 |
-
- **JavaScript Execution**: Full JS support for dynamic content
|
| 75 |
-
- **Error Recovery**: Automatic retry mechanisms
|
| 76 |
-
- **Resource Management**: Optimized memory and CPU usage
|
| 77 |
-
|
| 78 |
-
### Data Extraction
|
| 79 |
-
- **Text Content**: All visible and hidden text
|
| 80 |
-
- **Links**: Internal and external links
|
| 81 |
-
- **Images**: All images with source URLs
|
| 82 |
-
- **Forms**: All form elements with attributes
|
| 83 |
-
- **Custom Data**: User-defined CSS selectors
|
| 84 |
-
|
| 85 |
-
### Output Formats
|
| 86 |
-
- **JSON**: Structured data with metadata
|
| 87 |
-
- **Raw Text**: Clean text content
|
| 88 |
-
- **Screenshot**: Base64 encoded images
|
| 89 |
-
- **CSV**: Spreadsheet-compatible format
|
| 90 |
-
- **HTML**: Full page HTML source
|
| 91 |
-
|
| 92 |
-
## Technical Details ๐ง
|
| 93 |
-
|
| 94 |
-
### Performance
|
| 95 |
-
- **Concurrent Processing**: Multiple URLs processed in parallel
|
| 96 |
-
- **Rate Limiting**: Built-in delays to respect websites
|
| 97 |
-
- **Memory Optimization**: Efficient resource management
|
| 98 |
-
- **Error Resilience**: Continues processing despite individual failures
|
| 99 |
-
|
| 100 |
-
### Security
|
| 101 |
-
- **No Data Storage**: All processing happens in memory
|
| 102 |
-
- **Temporary Files**: Screenshots and data cleared after use
|
| 103 |
-
- **Secure Communication**: HTTPS-only external requests
|
| 104 |
-
- **Input Sanitization**: URL and data validation
|
| 105 |
-
|
| 106 |
-
### Limitations
|
| 107 |
-
- **Rate Limits**: Built-in delays to avoid overloading websites
|
| 108 |
-
- **JavaScript-Heavy Sites**: May require additional wait time
|
| 109 |
-
- **Captcha Protection**: Cannot bypass CAPTCHA or bot detection
|
| 110 |
-
- **Authentication**: No built-in login/session management
|
| 111 |
-
|
| 112 |
-
## API Usage ๐
|
| 113 |
-
|
| 114 |
-
For developers who want to use this tool programmatically, you can import the main functions:
|
| 115 |
|
| 116 |
-
|
| 117 |
-
from app import BrowserAutomationTool
|
| 118 |
-
|
| 119 |
-
# Initialize the tool
|
| 120 |
-
tool = BrowserAutomationTool()
|
| 121 |
-
|
| 122 |
-
# Capture screenshot
|
| 123 |
-
screenshot = tool.capture_screenshot(
|
| 124 |
-
url="https://example.com",
|
| 125 |
-
full_page=True,
|
| 126 |
-
wait_time=3
|
| 127 |
-
)
|
| 128 |
-
|
| 129 |
-
# Extract data
|
| 130 |
-
data = tool.scrape_website(
|
| 131 |
-
url="https://example.com",
|
| 132 |
-
selectors=["h1", "p", "a"]
|
| 133 |
-
)
|
| 134 |
-
|
| 135 |
-
# Batch scraping
|
| 136 |
-
results = tool.scrape_multiple_urls(
|
| 137 |
-
urls=["https://site1.com", "https://site2.com"],
|
| 138 |
-
max_workers=2,
|
| 139 |
-
wait_time=2
|
| 140 |
-
)
|
| 141 |
-
```
|
| 142 |
|
| 143 |
-
##
|
| 144 |
|
| 145 |
-
|
| 146 |
-
|
| 147 |
-
|
| 148 |
-
|
| 149 |
-
|
| 150 |
-
|
| 151 |
-
|
| 152 |
|
| 153 |
-
##
|
| 154 |
-
```
|
| 155 |
-
URLs:
|
| 156 |
-
- https://amazon.com/product/123
|
| 157 |
-
- https://ebay.com/product/456
|
| 158 |
-
- https://walmart.com/product/789
|
| 159 |
-
Extraction: Price, title, availability
|
| 160 |
-
Expected Output: Comparative pricing data
|
| 161 |
-
```
|
| 162 |
|
| 163 |
-
###
|
| 164 |
-
|
| 165 |
-
URL
|
| 166 |
-
|
| 167 |
-
Action: Fill and submit contact form
|
| 168 |
-
Expected Output: Success confirmation
|
| 169 |
-
```
|
| 170 |
|
| 171 |
-
##
|
| 172 |
-
|
| 173 |
-
|
| 174 |
-
|
| 175 |
-
|
| 176 |
-
|
| 177 |
-
|
| 178 |
-
|
| 179 |
-
#
|
| 180 |
-
|
| 181 |
-
|
| 182 |
-
|
| 183 |
-
|
| 184 |
-
|
| 185 |
-
|
| 186 |
-
|
| 187 |
-
#
|
| 188 |
-
-
|
| 189 |
-
|
| 190 |
-
|
| 191 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
| 192 |
|
| 193 |
-
##
|
| 194 |
-
This is an open-source tool. Feel free to:
|
| 195 |
-
- Report bugs and issues
|
| 196 |
-
- Suggest new features
|
| 197 |
-
- Submit improvements
|
| 198 |
-
- Share your use cases
|
| 199 |
|
| 200 |
-
###
|
| 201 |
-
|
|
|
|
|
|
|
| 202 |
|
| 203 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 204 |
|
| 205 |
-
|
| 206 |
|
| 207 |
-
*
|
|
|
|
| 9 |
pinned: false
|
| 10 |
---
|
| 11 |
|
| 12 |
+
# Browser Automation Tool
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 13 |
|
| 14 |
+
A comprehensive browser automation platform that combines the power of Gradio for web UI and FastAPI for programmatic access. This tool enables seamless web scraping, browser automation, and integration with AI agents.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 15 |
|
| 16 |
+
## Features
|
| 17 |
|
| 18 |
+
- ๐ **Web Browser Automation**: Control Chrome/Chromium browsers programmatically
|
| 19 |
+
- ๐ธ **Screenshot Capture**: Take screenshots of any webpage
|
| 20 |
+
- ๐ **Web Scraping**: Extract data from HTML content
|
| 21 |
+
- ๐ค **AI Agent Integration**: Compatible with Model Context Protocol (MCP)
|
| 22 |
+
- โก **Real-time Streaming**: Server-Sent Events (SSE) for live updates
|
| 23 |
+
- ๐ **RESTful API**: FastAPI backend for programmatic access
|
| 24 |
+
- ๐ฑ **Web Interface**: User-friendly Gradio UI
|
| 25 |
|
| 26 |
+
## Quick Start
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 27 |
|
| 28 |
+
### Using the Web Interface
|
| 29 |
+
1. Visit the Space URL above
|
| 30 |
+
2. Enter a URL to navigate
|
| 31 |
+
3. Use built-in tools for automation
|
|
|
|
|
|
|
|
|
|
| 32 |
|
| 33 |
+
### Using the API
|
| 34 |
+
```python
|
| 35 |
+
import requests
|
| 36 |
+
|
| 37 |
+
# Get available sessions
|
| 38 |
+
response = requests.get('https://your-space-url/api/sessions')
|
| 39 |
+
sessions = response.json()
|
| 40 |
+
|
| 41 |
+
# Create a new browser session
|
| 42 |
+
session_data = {
|
| 43 |
+
'headless': True,
|
| 44 |
+
'window_size': {'width': 1920, 'height': 1080}
|
| 45 |
+
}
|
| 46 |
+
response = requests.post('https://your-space-url/api/sessions', json=session_data)
|
| 47 |
+
session_id = response.json()['session_id']
|
| 48 |
+
|
| 49 |
+
# Navigate to a website
|
| 50 |
+
requests.post(f'https://your-space-url/api/sessions/{{session_id}}/navigate',
|
| 51 |
+
json={{'url': 'https://example.com'}})
|
| 52 |
+
|
| 53 |
+
# Take a screenshot
|
| 54 |
+
response = requests.get(f'https://your-space-url/api/sessions/{{session_id}}/screenshot')
|
| 55 |
+
with open('screenshot.png', 'wb') as f:
|
| 56 |
+
f.write(response.content)
|
| 57 |
+
```
|
| 58 |
|
| 59 |
+
## API Endpoints
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 60 |
|
| 61 |
+
### Session Management
|
| 62 |
+
- `GET /api/sessions` - List all active sessions
|
| 63 |
+
- `POST /api/sessions` - Create new browser session
|
| 64 |
+
- `DELETE /api/sessions/{{session_id}}` - Close session
|
| 65 |
|
| 66 |
+
### Browser Control
|
| 67 |
+
- `POST /api/sessions/{{session_id}}/navigate` - Navigate to URL
|
| 68 |
+
- `GET /api/sessions/{{session_id}}/screenshot` - Take screenshot
|
| 69 |
+
- `POST /api/sessions/{{session_id}}/click` - Click element
|
| 70 |
+
- `POST /api/sessions/{{session_id}}/type` - Type text
|
| 71 |
+
- `GET /api/sessions/{{session_id}}/get_page_source` - Get HTML
|
| 72 |
|
| 73 |
+
## Author
|
| 74 |
|
| 75 |
+
**MiniMax Agent** - Advanced AI automation tools
|