| --- |
| title: Juskeo Crawler |
| emoji: 🕷️ |
| colorFrom: green |
| colorTo: yellow |
| sdk: docker |
| app_file: main.py |
| pinned: false |
| --- |
| |
| # Juskeo Deep Crawler — HF Space |
|
|
| Webhook-based crawler. Crawls a website, then POSTs results back to the web URL. |
| No data stored on HF Space. All data lives on the Juskeo web server. |
|
|
| ## Deploy |
|
|
| Push this folder to a HF Space with **Docker** runtime. |
| Space API: `https://USERNAME-SPACE.hf.space` |
|
|
| ## How it works |
|
|
| User → Juskeo Dashboard → api/api.php → HF Space Crawler → Crawls website → POSTs results to api.php webhook → Data stored in data/ folder |
|
|
| ## API (called by Juskeo api.php only) |
|
|
| | Param | Description | |
| |-------|-------------| |
| | `url` | Target website to crawl | |
| | `webhook` | URL to POST results to | |
| | `username` | Username for the crawl | |
|
|
| ## Local Dev |
|
|
| ```bash |
| pip install -r requirements.txt |
| uvicorn main:app --reload --port 7860 |
| ``` |
|
|
