Spaces:
Sleeping
Sleeping
Commit ·
2a77859
1
Parent(s): cdb8eff
Docs: Update README
Browse files- README.md +8 -0
- tests/test_scrape_districts_by_category.py +2 -2
README.md
CHANGED
|
@@ -60,10 +60,18 @@ The app will open automatically in your browser at `http://127.0.0.1:7861`.
|
|
| 60 |
You can also run the tools directly via the terminal if you don't want to use the UI.
|
| 61 |
|
| 62 |
### Step 1: Main Scrape
|
|
|
|
| 63 |
```powershell
|
| 64 |
python -m pytest tests/test_scrape_districts_by_category.py -v -s --state "GOA"
|
| 65 |
```
|
| 66 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 67 |
### Step 2: Retry Failures
|
| 68 |
```powershell
|
| 69 |
python -m pytest tests/test_retry_districts_by_category.py -v -s --state "GOA"
|
|
|
|
| 60 |
You can also run the tools directly via the terminal if you don't want to use the UI.
|
| 61 |
|
| 62 |
### Step 1: Main Scrape
|
| 63 |
+
By default, running the script in the terminal will fetch data for all 6 target categories.
|
| 64 |
```powershell
|
| 65 |
python -m pytest tests/test_scrape_districts_by_category.py -v -s --state "GOA"
|
| 66 |
```
|
| 67 |
|
| 68 |
+
**Filter by Specific Categories:**
|
| 69 |
+
If you want to only fetch data for specific categories (e.g., just category 5, or 3 and 5), you can set the `KYS_TARGET_CATEGORIES` environment variable before the command:
|
| 70 |
+
```powershell
|
| 71 |
+
$env:KYS_TARGET_CATEGORIES="5"; python -m pytest tests/test_scrape_districts_by_category.py -v -s --state "GOA"
|
| 72 |
+
```
|
| 73 |
+
*(You can pass multiple IDs separated by commas: `"3,5"`)*
|
| 74 |
+
|
| 75 |
### Step 2: Retry Failures
|
| 76 |
```powershell
|
| 77 |
python -m pytest tests/test_retry_districts_by_category.py -v -s --state "GOA"
|
tests/test_scrape_districts_by_category.py
CHANGED
|
@@ -607,7 +607,7 @@ def test_scrape_schools_by_district_category(
|
|
| 607 |
else:
|
| 608 |
log.info("All combos collected successfully — no captcha failures!")
|
| 609 |
|
| 610 |
-
assert success_count > 0, (
|
| 611 |
-
f"No
|
| 612 |
f"Check OCR accuracy and network connectivity. Failed: {failed_count}"
|
| 613 |
)
|
|
|
|
| 607 |
else:
|
| 608 |
log.info("All combos collected successfully — no captcha failures!")
|
| 609 |
|
| 610 |
+
assert (success_count + no_data_count) > 0, (
|
| 611 |
+
f"No valid responses collected! "
|
| 612 |
f"Check OCR accuracy and network connectivity. Failed: {failed_count}"
|
| 613 |
)
|