musaw commited on
Commit
574cd8c
·
1 Parent(s): 16953e6

Add repeatable resource cycle runbook and one-command script

Browse files
README.md CHANGED
@@ -37,6 +37,7 @@ This repository now has a machine-readable and searchable resource pipeline:
37
  - Search data export: [docs/search/resources.json](docs/search/resources.json)
38
  - Full index docs: [docs/resource_catalog.md](docs/resource_catalog.md)
39
  - Automation docs: [docs/resource_automation.md](docs/resource_automation.md)
 
40
 
41
  ## How New Resources Are Added
42
 
@@ -55,6 +56,9 @@ The process is semi-automatic:
55
  - Run `python scripts/generate_resource_views.py`
56
  - Commit generated updates (`resources/*/README.md` and `docs/search/resources.json`).
57
 
 
 
 
58
  This prevents low-confidence links from being merged directly while still automating discovery.
59
 
60
  ## Quickstart
 
37
  - Search data export: [docs/search/resources.json](docs/search/resources.json)
38
  - Full index docs: [docs/resource_catalog.md](docs/resource_catalog.md)
39
  - Automation docs: [docs/resource_automation.md](docs/resource_automation.md)
40
+ - Repeatable runbook: [docs/resource_cycle_runbook.md](docs/resource_cycle_runbook.md)
41
 
42
  ## How New Resources Are Added
43
 
 
56
  - Run `python scripts/generate_resource_views.py`
57
  - Commit generated updates (`resources/*/README.md` and `docs/search/resources.json`).
58
 
59
+ Shortcut wrapper:
60
+ - Run `python scripts/run_resource_cycle.py --limit 25`
61
+
62
  This prevents low-confidence links from being merged directly while still automating discovery.
63
 
64
  ## Quickstart
docs/README.md CHANGED
@@ -20,6 +20,7 @@ This folder is the main documentation entry point for contributors.
20
  - Platforms and publish flow: [platforms.md](platforms.md)
21
  - GitHub operations: [github_operations.md](github_operations.md)
22
  - Resource automation: [resource_automation.md](resource_automation.md)
 
23
 
24
  ## Resource tracking
25
  - Master resource index: [resource_catalog.md](resource_catalog.md)
 
20
  - Platforms and publish flow: [platforms.md](platforms.md)
21
  - GitHub operations: [github_operations.md](github_operations.md)
22
  - Resource automation: [resource_automation.md](resource_automation.md)
23
+ - Resource cycle runbook: [resource_cycle_runbook.md](resource_cycle_runbook.md)
24
 
25
  ## Resource tracking
26
  - Master resource index: [resource_catalog.md](resource_catalog.md)
docs/resource_automation.md CHANGED
@@ -17,6 +17,7 @@ This repository uses a semi-automated process to keep Pashto resources current w
17
  - Validate catalog: `python scripts/validate_resource_catalog.py`
18
  - Generate markdown and search index: `python scripts/generate_resource_views.py`
19
  - Sync new candidates: `python scripts/sync_resources.py --limit 20`
 
20
 
21
  ## GitHub Actions
22
  - CI (`.github/workflows/ci.yml`) enforces:
@@ -34,3 +35,6 @@ This repository uses a semi-automated process to keep Pashto resources current w
34
  - `python scripts/validate_resource_catalog.py`
35
  - `python scripts/generate_resource_views.py`
36
  5. Commit and open PR.
 
 
 
 
17
  - Validate catalog: `python scripts/validate_resource_catalog.py`
18
  - Generate markdown and search index: `python scripts/generate_resource_views.py`
19
  - Sync new candidates: `python scripts/sync_resources.py --limit 20`
20
+ - Full run wrapper: `python scripts/run_resource_cycle.py --limit 25`
21
 
22
  ## GitHub Actions
23
  - CI (`.github/workflows/ci.yml`) enforces:
 
35
  - `python scripts/validate_resource_catalog.py`
36
  - `python scripts/generate_resource_views.py`
37
  5. Commit and open PR.
38
+
39
+ ## Runbook
40
+ - Reusable process guide: [resource_cycle_runbook.md](resource_cycle_runbook.md)
docs/resource_cycle_runbook.md ADDED
@@ -0,0 +1,43 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ # Resource Cycle Runbook
2
+
3
+ Use this runbook whenever you want to repeat the resource update process without re-explaining it.
4
+
5
+ ## Weekly automation (already enabled)
6
+ - Workflow: [../.github/workflows/resource_sync.yml](../.github/workflows/resource_sync.yml)
7
+ - Schedule: every Monday (UTC) via GitHub Actions cron.
8
+ - Output: updates [../resources/catalog/pending_candidates.json](../resources/catalog/pending_candidates.json) and opens a review PR.
9
+
10
+ ## Manual run (single command)
11
+ Run from repository root:
12
+
13
+ ```bash
14
+ python scripts/run_resource_cycle.py --limit 25
15
+ ```
16
+
17
+ What it executes:
18
+ 1. `python scripts/sync_resources.py --limit 25`
19
+ 2. `python scripts/validate_resource_catalog.py`
20
+ 3. `python scripts/generate_resource_views.py`
21
+ 4. `python scripts/check_links.py`
22
+ 5. `python -m pytest -q`
23
+
24
+ ## Discovery-only mode
25
+ If you only want fresh candidates:
26
+
27
+ ```bash
28
+ python scripts/run_resource_cycle.py --discover-only --limit 25
29
+ ```
30
+
31
+ ## Promotion step (manual review)
32
+ After discovery, promote only approved resources:
33
+ 1. Open [../resources/catalog/pending_candidates.json](../resources/catalog/pending_candidates.json).
34
+ 2. Copy selected entries into [../resources/catalog/resources.json](../resources/catalog/resources.json).
35
+ 3. Ensure unique `id` and valid evidence fields.
36
+ 4. Re-run:
37
+ - `python scripts/run_resource_cycle.py --skip-pytest`
38
+ 5. Commit and push.
39
+
40
+ ## Guardrails
41
+ - Do not auto-promote candidates without evidence and license review.
42
+ - Keep `status: verified` only for reviewed entries.
43
+ - Generated files must be committed after catalog updates.
scripts/README.md CHANGED
@@ -8,6 +8,7 @@ Automation scripts for quality checks, resource catalog validation, and search i
8
  - `validate_resource_catalog.py`: validate `resources/catalog/resources.json`.
9
  - `generate_resource_views.py`: generate `resources/*/README.md`, `resources/README.md`, and `docs/search/resources.json` from the catalog.
10
  - `sync_resources.py`: collect new candidate Pashto resources from public endpoints into `resources/catalog/pending_candidates.json`.
 
11
 
12
  ## Usage
13
 
@@ -31,6 +32,16 @@ Sync candidate resources for maintainer review:
31
  python scripts/sync_resources.py --limit 20
32
  ```
33
 
 
 
 
 
 
 
 
 
 
 
34
  Check markdown links format:
35
  ```bash
36
  python scripts/check_links.py
 
8
  - `validate_resource_catalog.py`: validate `resources/catalog/resources.json`.
9
  - `generate_resource_views.py`: generate `resources/*/README.md`, `resources/README.md`, and `docs/search/resources.json` from the catalog.
10
  - `sync_resources.py`: collect new candidate Pashto resources from public endpoints into `resources/catalog/pending_candidates.json`.
11
+ - `run_resource_cycle.py`: run the full repeatable resource cycle with one command.
12
 
13
  ## Usage
14
 
 
32
  python scripts/sync_resources.py --limit 20
33
  ```
34
 
35
+ Run full repeatable cycle:
36
+ ```bash
37
+ python scripts/run_resource_cycle.py --limit 25
38
+ ```
39
+
40
+ Run discovery only:
41
+ ```bash
42
+ python scripts/run_resource_cycle.py --discover-only --limit 25
43
+ ```
44
+
45
  Check markdown links format:
46
  ```bash
47
  python scripts/check_links.py
scripts/run_resource_cycle.py ADDED
@@ -0,0 +1,69 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ """Run the Pashto resource update cycle in a repeatable way.
2
+
3
+ This script is a command wrapper around existing resource scripts so maintainers
4
+ do not need to remember the full sequence.
5
+
6
+ Usage:
7
+ python scripts/run_resource_cycle.py
8
+ python scripts/run_resource_cycle.py --limit 30
9
+ python scripts/run_resource_cycle.py --skip-pytest
10
+ python scripts/run_resource_cycle.py --discover-only
11
+ """
12
+
13
+ from __future__ import annotations
14
+
15
+ import argparse
16
+ import shlex
17
+ import subprocess
18
+ import sys
19
+ from pathlib import Path
20
+
21
+
22
+ def _run(command: list[str], cwd: Path) -> int:
23
+ print(f"[run] {shlex.join(command)}")
24
+ completed = subprocess.run(command, cwd=str(cwd), check=False)
25
+ return completed.returncode
26
+
27
+
28
+ def main() -> int:
29
+ parser = argparse.ArgumentParser()
30
+ parser.add_argument("--limit", type=int, default=25, help="Candidate fetch limit per source")
31
+ parser.add_argument("--skip-pytest", action="store_true", help="Skip pytest step")
32
+ parser.add_argument("--discover-only", action="store_true", help="Only sync candidates and stop")
33
+ args = parser.parse_args()
34
+
35
+ repo_root = Path(__file__).resolve().parents[1]
36
+ steps: list[list[str]] = [
37
+ ["python", "scripts/sync_resources.py", "--limit", str(args.limit)],
38
+ ]
39
+
40
+ if not args.discover_only:
41
+ steps.extend(
42
+ [
43
+ ["python", "scripts/validate_resource_catalog.py"],
44
+ ["python", "scripts/generate_resource_views.py"],
45
+ ["python", "scripts/check_links.py"],
46
+ ]
47
+ )
48
+ if not args.skip_pytest:
49
+ steps.append(["python", "-m", "pytest", "-q"])
50
+
51
+ for command in steps:
52
+ code = _run(command, repo_root)
53
+ if code != 0:
54
+ print(f"[fail] Step failed with exit code {code}")
55
+ return code
56
+
57
+ print("[ok] Resource cycle completed")
58
+ if args.discover_only:
59
+ print(
60
+ "Next: review resources/catalog/pending_candidates.json and promote approved "
61
+ "entries into resources/catalog/resources.json."
62
+ )
63
+ else:
64
+ print("Next: commit updated catalog/generated files and push.")
65
+ return 0
66
+
67
+
68
+ if __name__ == "__main__":
69
+ raise SystemExit(main())