Krixter commited on
Commit
82e4571
Β·
verified Β·
1 Parent(s): d126a36

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +5 -33
README.md CHANGED
@@ -1,38 +1,10 @@
1
  ---
2
- title: Novel Scraper
3
  emoji: πŸ“š
4
- colorFrom: indigo
5
- colorTo: purple
6
  sdk: gradio
7
- sdk_version: 4.44.1
8
  app_file: app.py
9
  pinned: false
10
- ---
11
-
12
- # Novel Scraper
13
-
14
- Scrapes **every novel's title + description** from:
15
- - **novelbin.com** β€” HTML scraping with BeautifulSoup
16
- - **webnovel.com** β€” Public JSON API (no JS rendering needed)
17
-
18
- ## Output files (in `output/` folder)
19
- | File | Contents |
20
- |------|----------|
21
- | `novelbin_novels.csv` | novelbin data as CSV |
22
- | `novelbin_novels.jsonl` | novelbin data as JSONL |
23
- | `webnovel_novels.csv` | webnovel data as CSV |
24
- | `webnovel_novels.jsonl` | webnovel data as JSONL |
25
- | `scraper.log` | Full log of the run |
26
- | `progress.json` | Resumable checkpoint |
27
-
28
- ## Features
29
- - βœ… **Resumable** β€” restart the Space and it picks up where it left off
30
- - βœ… **Rate-limited** β€” polite delays to avoid getting blocked
31
- - βœ… **Retry logic** β€” 5 retries with exponential backoff per failed request
32
- - βœ… **Dual output** β€” CSV for spreadsheets, JSONL for pipelines
33
- - βœ… **Live log** β€” watch progress in real-time in the UI
34
-
35
- ## Usage
36
- 1. Click **β–Ά Start Scraper**
37
- 2. Click **πŸ”„ Refresh Status** any time to see the log
38
- 3. When done, download files from the **Download Output Files** section
 
1
  ---
2
+ title: Novelhall Scraper
3
  emoji: πŸ“š
4
+ colorFrom: blue
5
+ colorTo: indigo
6
  sdk: gradio
7
+ python_version: 3.12
8
  app_file: app.py
9
  pinned: false
10
+ ---