sheikhcoders commited on
Commit
a70da23
ยท
verified ยท
1 Parent(s): 48ec8b8

Update README with proper YAML configuration

Browse files
Files changed (1) hide show
  1. README.md +53 -185
README.md CHANGED
@@ -9,199 +9,67 @@ app_file: app.py
9
  pinned: false
10
  ---
11
 
12
- # Browser Automation Tool ๐ŸŒ
13
-
14
- A comprehensive web scraping and browser automation platform - an alternative to browserbase.com. This Hugging Face Space provides powerful tools for web data extraction, screenshot capture, form automation, and multi-URL scraping.
15
-
16
- ## Features ๐Ÿš€
17
-
18
- ### ๐Ÿ” Single URL Analysis
19
- - **Screenshot Capture**: Take high-quality screenshots of any webpage
20
- - **Data Extraction**: Extract text, links, images, forms, and custom elements
21
- - **Custom Selectors**: Use CSS selectors to extract specific data
22
- - **Headless/Headed Mode**: Choose between invisible or visible browser operation
23
-
24
- ### ๐Ÿ“Š Multiple URLs Scraping
25
- - **Concurrent Scraping**: Process multiple URLs simultaneously
26
- - **Configurable Workers**: Control the number of concurrent processes
27
- - **Batch Processing**: Extract data from entire lists of URLs
28
- - **Structured Output**: Get organized results in JSON format
29
-
30
- ### ๐Ÿ“‹ Form Automation
31
- - **Smart Form Detection**: Automatically detect form fields
32
- - **Automated Filling**: Fill forms with provided data
33
- - **Field Type Recognition**: Handle text, email, password, textarea fields
34
- - **Form Submission**: Submit forms and capture responses
35
-
36
- ### ๐ŸŽฎ Interactive Mode
37
- - **Real-time Screenshot**: Live view of browser activity
38
- - **Console Output**: Monitor JavaScript console in real-time
39
- - **Error Handling**: Graceful handling of failed requests
40
- - **Progress Tracking**: Visual feedback for long operations
41
-
42
- ## How to Use ๐Ÿ’ก
43
-
44
- ### 1. Single URL Analysis
45
- 1. Enter a URL in the "Single URL Analysis" tab
46
- 2. Choose screenshot mode (full page or viewport)
47
- 3. Set wait time for page loading
48
- 4. Click "Run Analysis" and wait for results
49
-
50
- ### 2. Multiple URLs Scraping
51
- 1. Select the "Multiple URLs Scraping" tab
52
- 2. Paste multiple URLs (one per line)
53
- 3. Configure number of workers (1-10)
54
- 4. Set wait time per URL
55
- 5. Click "Start Scraping" and monitor progress
56
-
57
- ### 3. Form Automation
58
- 1. Go to the "Form Automation" tab
59
- 2. Enter target URL and form data
60
- 3. Specify CSS selectors (or use auto-detection)
61
- 4. Click "Fill and Submit" to automate the process
62
-
63
- ### 4. Live Console
64
- 1. Navigate to "Live Console" tab
65
- 2. Enter any URL to monitor
66
- 3. Watch real-time browser console output
67
- 4. See JavaScript errors and logs as they happen
68
-
69
- ## Supported Features ๐Ÿ› ๏ธ
70
-
71
- ### Browser Capabilities
72
- - **Headless Chrome**: Fast, efficient browser automation
73
- - **Customizable Viewport**: Set specific screen dimensions
74
- - **JavaScript Execution**: Full JS support for dynamic content
75
- - **Error Recovery**: Automatic retry mechanisms
76
- - **Resource Management**: Optimized memory and CPU usage
77
-
78
- ### Data Extraction
79
- - **Text Content**: All visible and hidden text
80
- - **Links**: Internal and external links
81
- - **Images**: All images with source URLs
82
- - **Forms**: All form elements with attributes
83
- - **Custom Data**: User-defined CSS selectors
84
-
85
- ### Output Formats
86
- - **JSON**: Structured data with metadata
87
- - **Raw Text**: Clean text content
88
- - **Screenshot**: Base64 encoded images
89
- - **CSV**: Spreadsheet-compatible format
90
- - **HTML**: Full page HTML source
91
-
92
- ## Technical Details ๐Ÿ”ง
93
-
94
- ### Performance
95
- - **Concurrent Processing**: Multiple URLs processed in parallel
96
- - **Rate Limiting**: Built-in delays to respect websites
97
- - **Memory Optimization**: Efficient resource management
98
- - **Error Resilience**: Continues processing despite individual failures
99
-
100
- ### Security
101
- - **No Data Storage**: All processing happens in memory
102
- - **Temporary Files**: Screenshots and data cleared after use
103
- - **Secure Communication**: HTTPS-only external requests
104
- - **Input Sanitization**: URL and data validation
105
-
106
- ### Limitations
107
- - **Rate Limits**: Built-in delays to avoid overloading websites
108
- - **JavaScript-Heavy Sites**: May require additional wait time
109
- - **Captcha Protection**: Cannot bypass CAPTCHA or bot detection
110
- - **Authentication**: No built-in login/session management
111
-
112
- ## API Usage ๐Ÿ
113
-
114
- For developers who want to use this tool programmatically, you can import the main functions:
115
 
116
- ```python
117
- from app import BrowserAutomationTool
118
-
119
- # Initialize the tool
120
- tool = BrowserAutomationTool()
121
-
122
- # Capture screenshot
123
- screenshot = tool.capture_screenshot(
124
- url="https://example.com",
125
- full_page=True,
126
- wait_time=3
127
- )
128
-
129
- # Extract data
130
- data = tool.scrape_website(
131
- url="https://example.com",
132
- selectors=["h1", "p", "a"]
133
- )
134
-
135
- # Batch scraping
136
- results = tool.scrape_multiple_urls(
137
- urls=["https://site1.com", "https://site2.com"],
138
- max_workers=2,
139
- wait_time=2
140
- )
141
- ```
142
 
143
- ## Examples ๐Ÿ“š
144
 
145
- ### Example 1: News Website Analysis
146
- ```
147
- URL: https://news.ycombinator.com
148
- Screenshot: Full page
149
- Extraction: Titles, links, comments
150
- Expected Output: List of current news stories with engagement metrics
151
- ```
152
 
153
- ### Example 2: E-commerce Price Comparison
154
- ```
155
- URLs:
156
- - https://amazon.com/product/123
157
- - https://ebay.com/product/456
158
- - https://walmart.com/product/789
159
- Extraction: Price, title, availability
160
- Expected Output: Comparative pricing data
161
- ```
162
 
163
- ### Example 3: Form Automation
164
- ```
165
- URL: https://example.com/contact
166
- Form Data: name="John Doe", email="john@example.com"
167
- Action: Fill and submit contact form
168
- Expected Output: Success confirmation
169
- ```
170
 
171
- ## Troubleshooting ๐Ÿ”
172
-
173
- ### Common Issues
174
- - **"Page took too long to load"**: Increase wait time, check URL accessibility
175
- - **"Element not found"**: Verify CSS selector, check if page uses dynamic loading
176
- - **"Screenshot failed"**: Ensure URL is accessible, check for popup blockers
177
- - **"Scraping timeout"**: Reduce number of workers, increase timeout values
178
-
179
- ### Performance Tips
180
- - **For large lists**: Start with fewer workers and increase gradually
181
- - **For slow sites**: Increase wait time between requests
182
- - **For dynamic content**: Use JavaScript execution wait
183
- - **For memory issues**: Process URLs in smaller batches
184
-
185
- ## Support & Contributing ๐Ÿ’ฌ
186
-
187
- ### Getting Help
188
- - **Check the examples** in the interface for common use cases
189
- - **Verify your URLs** are publicly accessible
190
- - **Test with simpler pages** first to debug issues
191
- - **Check the console output** for error messages
 
 
 
 
192
 
193
- ### Contributing
194
- This is an open-source tool. Feel free to:
195
- - Report bugs and issues
196
- - Suggest new features
197
- - Submit improvements
198
- - Share your use cases
199
 
200
- ### License
201
- MIT License - Feel free to use this tool in your projects.
 
 
202
 
203
- ---
 
 
 
 
 
204
 
205
- **Made with โค๏ธ for the developer community**
206
 
207
- *Alternative to browserbase.com - Bringing powerful browser automation to everyone.*
 
9
  pinned: false
10
  ---
11
 
12
+ # Browser Automation Tool
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
13
 
14
+ A comprehensive browser automation platform that combines the power of Gradio for web UI and FastAPI for programmatic access. This tool enables seamless web scraping, browser automation, and integration with AI agents.
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
15
 
16
+ ## Features
17
 
18
+ - ๐ŸŒ **Web Browser Automation**: Control Chrome/Chromium browsers programmatically
19
+ - ๐Ÿ“ธ **Screenshot Capture**: Take screenshots of any webpage
20
+ - ๐Ÿ” **Web Scraping**: Extract data from HTML content
21
+ - ๐Ÿค– **AI Agent Integration**: Compatible with Model Context Protocol (MCP)
22
+ - โšก **Real-time Streaming**: Server-Sent Events (SSE) for live updates
23
+ - ๐Ÿš€ **RESTful API**: FastAPI backend for programmatic access
24
+ - ๐Ÿ“ฑ **Web Interface**: User-friendly Gradio UI
25
 
26
+ ## Quick Start
 
 
 
 
 
 
 
 
27
 
28
+ ### Using the Web Interface
29
+ 1. Visit the Space URL above
30
+ 2. Enter a URL to navigate
31
+ 3. Use built-in tools for automation
 
 
 
32
 
33
+ ### Using the API
34
+ ```python
35
+ import requests
36
+
37
+ # Get available sessions
38
+ response = requests.get('https://your-space-url/api/sessions')
39
+ sessions = response.json()
40
+
41
+ # Create a new browser session
42
+ session_data = {
43
+ 'headless': True,
44
+ 'window_size': {'width': 1920, 'height': 1080}
45
+ }
46
+ response = requests.post('https://your-space-url/api/sessions', json=session_data)
47
+ session_id = response.json()['session_id']
48
+
49
+ # Navigate to a website
50
+ requests.post(f'https://your-space-url/api/sessions/{{session_id}}/navigate',
51
+ json={{'url': 'https://example.com'}})
52
+
53
+ # Take a screenshot
54
+ response = requests.get(f'https://your-space-url/api/sessions/{{session_id}}/screenshot')
55
+ with open('screenshot.png', 'wb') as f:
56
+ f.write(response.content)
57
+ ```
58
 
59
+ ## API Endpoints
 
 
 
 
 
60
 
61
+ ### Session Management
62
+ - `GET /api/sessions` - List all active sessions
63
+ - `POST /api/sessions` - Create new browser session
64
+ - `DELETE /api/sessions/{{session_id}}` - Close session
65
 
66
+ ### Browser Control
67
+ - `POST /api/sessions/{{session_id}}/navigate` - Navigate to URL
68
+ - `GET /api/sessions/{{session_id}}/screenshot` - Take screenshot
69
+ - `POST /api/sessions/{{session_id}}/click` - Click element
70
+ - `POST /api/sessions/{{session_id}}/type` - Type text
71
+ - `GET /api/sessions/{{session_id}}/get_page_source` - Get HTML
72
 
73
+ ## Author
74
 
75
+ **MiniMax Agent** - Advanced AI automation tools