skills/Skill.md

tee palun jah

Teen sulle kohe 3 asja:

1. production-ready SKILL.md (pdf-ops-master),
2. soovitusliku scripts/ struktuuri koos sisukate stubidega,
3. n8n + Airtable workflow skeemi, mis salvestab tulemused.

Allpool saad need otse copy-paste’ida oma LuunaOS/Claude/Agent stacki.

---

1. SKILL.md – pdf-ops-master

---
name: "pdf-ops-master"
version: "1.0.0"
category: "document-automation"
platforms: [CLAUDE_CODE]
tags: [pdf, ocr, forms, tables, accessibility, databricks, batch-processing]
description: "Kõik-ühes production-grade PDF skill: teksti/tabelite/formide extraction, OCR, merge/split/compress/encrypt, accessibility remediation ning Databricks HTML→PDF→Unity Catalog eval dataset workflow."
---

# PDF Ops Master

Sa oled PDF dokumentide automation ekspert. Kasuta seda skilli, kui vaja on:
- töödelda keerukaid või mahukaid PDF-workflow’sid,
- analüüsida ja täita PDF vorme,
- eraldada tabelid ja tekst raportitest,
- OCR-ida skaneeritud või image-based PDF-e,
- teostada merge/split/rotate/compress/encrypt/redact operatsioone,
- parandada PDF accessibility probleeme (PDF/UA, WCAG),
- genereerida synthetic HTML→PDF eval-dokumente Databricks Unity Catalog volume’i jaoks,
- käivitada batch PDF processing pipeline’i.

---

## Millal mida teha

### 1. Töö tüübi tuvastus

Kui kasutaja küsib midagi PDF kohta, tuvasta esmalt **operation**:

- `extract_text` – tekst PDF-ist (full või page range).
- `extract_tables` – tabelid CSV/Excel formaati.
- `extract_images` – pildid PDF-ist.
- `forms_analyze` – vorm fieldide schema.
- `forms_validate` – JSON vs schema kontroll.
- `forms_fill` – PDF vormi täitmine.
- `ocr` – skaneeritud PDF tekstiks.
- `merge` – mitme PDF liitmine üheks.
- `split` – PDF lehtede või vahemike eraldamine.
- `redact` – tundliku info eemaldamine (lehed, tekst, pildid).
- `compress` – faili suuruse vähendamine.
- `encrypt/decrypt` – parooliga kaitse või eemaldamine.
- `rotate` – lehtede pööramine.
- `watermark` – watermark/stamp/background lisamine.
- `metadata` – metadata & info (title, author, pages).
- `accessibility_audit` – accessibility audit (PDF/UA/WCAG).
- `accessibility_remediate` – auto-fix + manual juhised.
- `databricks_html_to_pdf` – HTML→PDF genereerimine.
- `databricks_upload` – upload UC volume’i.
- `databricks_eval_questions` – eval question JSON loomine.
- `batch` – eelnevate operatsioonide batch-versioon.

---

## Sõltuvused

### Python

```bash
pip install pdfplumber pypdf pillow pytesseract pandas

OCR (süsteemne)

# Ubuntu/Debian
sudo apt-get install tesseract-ocr

# macOS
brew install tesseract

PDF CLI tööriistad

# Ubuntu/Debian
sudo apt-get install pdftk qpdf poppler-utils ghostscript

Databricks synthetic PDF

uv pip install plutoprint

---

Standardne projektistruktuur

project/
├── input/            # sisend PDF-id
├── output/           # lõpptulemused
├── processed/        # vahetulemused
├── backups/          # originaalide koopiad
├── raw_data/
│   ├── html/         # HTML synthetic docs
│   └── pdf/          # nendest tehtud PDF-id
├── schemas/          # vormi ja data skeemid
├── logs/             # logifailid
└── scripts/          # Python/CLI wrapperid

---

Exit code standard

Kasuta kõikides skriptides sama exit code skeemi:

0 - Success
1 - File not found
2 - Invalid input
3 - Processing error
4 - Validation error
5 - OCR error
6 - Accessibility remediation error
7 - Databricks upload error

---

PDF processing – tekst, tabelid, vormid, OCR

Teksti extraction (Python)

import pdfplumber
from pathlib import Path
import sys

def extract_text(input_path: str, output_path: str, preserve_formatting: bool = True):
    in_path = Path(input_path)
    out_path = Path(output_path)

    if not in_path.exists():
        print(f"[ERROR] File not found: {in_path}", file=sys.stderr)
        sys.exit(1)

    try:
        with pdfplumber.open(in_path) as pdf:
            texts = []
            for page in pdf.pages:
                txt = page.extract_text(layout=preserve_formatting)
                if txt:
                    texts.append(txt)
        out_path.write_text("\n\n".join(texts), encoding="utf-8")
        sys.exit(0)
    except Exception as e:
        print(f"[ERROR] Processing error: {e}", file=sys.stderr)
        sys.exit(3)

CLI wrapper

python scripts/extract_text.py input.pdf --output text.txt --preserve-formatting

Tabelite extraction

python scripts/extract_tables.py report.pdf --output tables.csv --format csv

Soovitused:

• kasuta pdfplumber + pandas kombinatsiooni,
• toeta multi-page ja merged cell juhtumeid,
• väljunda alati struktureeritud CSV/Excel.

Vormid

# analyze
python scripts/analyze_form.py template.pdf --output schema.json

# validate
python scripts/validate_form.py submission.json schema.json

# fill
python scripts/fill_form.py template.pdf submission.json completed.pdf --validate

Vormide puhul:

• hoia schema JSON-is (field name, type, required, constraints),
• enne täitmist tee schema validation,
• logi kõik validation vead detailsusega.

OCR

python scripts/ocr_pdf.py scanned.pdf --output text.txt --lang eng

Kasuta OCR-i, kui:

• extract_text() tagastab sisuliselt tühja sisu,
• PDF on selgelt skaneeritud (ainult pildid),
• vaja masinloetavat teksti edasiseks töötluseks.

---

PDF manipulation – merge/split/rotate/compress/encrypt/watermark

Merge

pdftk file1.pdf file2.pdf file3.pdf cat output merged.pdf
# või
qpdf --empty --pages file1.pdf file2.pdf file3.pdf -- merged.pdf

Split & page ranges

pdftk input.pdf burst output page_%02d.pdf
pdftk input.pdf cat 1-5 10 output subset.pdf
qpdf input.pdf --pages . 1-5 -- pages1-5.pdf

Rotate

pdftk input.pdf cat 1-endright output rotated.pdf
# right (90°), left (270°), down (180°)

Compress

gs -sDEVICE=pdfwrite -dCompatibilityLevel=1.4 -dPDFSETTINGS=/ebook \
   -dNOPAUSE -dQUIET -dBATCH -sOutputFile=compressed.pdf input.pdf

Encrypt / decrypt

qpdf --encrypt userpass ownerpass 256 -- input.pdf secured.pdf
qpdf --decrypt --password=PASSWORD input.pdf output.pdf

Watermark

pdftk input.pdf stamp watermark.pdf output watermarked.pdf
pdftk input.pdf background watermark.pdf output watermarked.pdf

Metadata & tervisekontroll

pdfinfo input.pdf
qpdf --check input.pdf

---

Accessibility – audit + remediation

Auto-fixable vs manual

Auto-fixable:

• title, language, XMP metadata,
• PDF/UA flag,
• alt text puudumine,
• decorative images as <Artifact>,
• simple tag type remap,
• reading order hint (/Tabs /S).

Manual:

• keeruline tabelistruktuur,
• form tooltips ja error messages,
• complex reading order,
• bookmarks hierarchy,
• color contrast pildis endas.

Protsess

1. Audit
   • kasuta audit_pdf_accessibility.py → audit.json:
   • tuvastab: tagged/un-tagged, language, title, alt tekstid, tabelid, heading structure.
2. Klassifitseeri leiud
   • märgi iga issue fix_type: "auto" | "manual".
3. Auto-fix
   • genereeri script (pdf-lib/qpdf),
   • tee backup backups/ kataloogi,
   • rakenda parandused,
   • valideeri uuesti.
4. Manual-fix juhised
   • iga manual issue jaoks anna Acrobat Pro samm-sammult juhend:
   • Tags panel, Order panel, Forms editor, Bookmarks panel.

---

Databricks – synthetic HTML→PDF + eval dataset

Path konventsioon

<SKILL_ROOT> = skilli asukoht failisüsteemis
./raw_data/html/ = sinu projekti HTML sisend
./raw_data/pdf/  = genereeritud PDF-id

Samm 1 – HTML

mkdir -p ./raw_data/html
# kirjuta siia domain-shaped HTML manualid / error guides / install juhendid jne

Samm 2 – HTML→PDF

python <SKILL_ROOT>/scripts/pdf_generator.py convert \
  --input ./raw_data/html \
  --output ./raw_data/pdf \
  --workers 4

See skript:

• konverteerib paralleelselt,
• jätab vahele failid, kus PDF on uuem kui HTML (kui --force puudub),
• säilitab subfolder struktuuri.

Samm 3 – laadimine Unity Catalog volume’i

databricks fs cp -r --overwrite ./raw_data/pdf dbfs:/Volumes/my_catalog/my_schema/raw_data

Databricksis peab databricks fs kasutama dbfs: skeemi ka Unity Catalog path’i puhul ning -r kopeerib ainult sisu, mitte root kausta nime.[web:3][page:1]

Samm 4 – eval küsimuste JSON

{
  "api_errors_guide.pdf": {
    "question": "What is the solution for error ERR-4521?",
    "expected_fact": "Call /api/v2/auth/refresh with refresh_token before the 3600s TTL expires"
  },
  "installation_manual.pdf": {
    "question": "What port does the service use by default?",
    "expected_fact": "Port 8443 for HTTPS, configurable via CONFIG_PORT environment variable"
  }
}

Salvesta:

./raw_data/pdf/pdf_eval_questions.json

---

Common workflows (kokkuvõtlikult)

Workflow: vorm submission pipeline

python scripts/analyze_form.py template.pdf --output schemas/template_schema.json
python scripts/validate_form.py submission.json schemas/template_schema.json
python scripts/fill_form.py template.pdf submission.json output/completed.pdf --validate
python scripts/validate_pdf.py output/completed.pdf

Workflow: raportist data extraction

python scripts/extract_tables.py input/monthly_report.pdf --output output/monthly_report.csv
python scripts/extract_text.py input/monthly_report.pdf --output output/monthly_report.txt

Workflow: accessibility remediation pipeline

python scripts/audit_pdf_accessibility.py input/document.pdf --output logs/document_audit.json
python scripts/remediate_pdf_accessibility.py input/document.pdf logs/document_audit.json output/document_fixed.pdf
python scripts/validate_pdf.py output/document_fixed.pdf

Workflow: batch text extraction

python scripts/batch_process.py --operation extract_text --input-dir input/ --output-dir processed/

---

Agent prompt

Sa oled PDF Ops Master agent.

Kui kasutaja palub PDF operatsiooni:
1. Tuvasta operation (extract_text, extract_tables, forms_*, ocr, merge/split/rotate/compress/encrypt, watermark, metadata, accessibility_*, databricks_*, batch).
2. Kontrolli, et sisendfail(id) eksisteerivad ja PDF on loetav.
3. Suurte failide puhul töötle page-by-page.
4. Vormide puhul kasuta alati schema.json + validate enne fill.
5. Accessibility puhul jaga leiud automaatseteks ja manualseteks, rakenda auto-fix skriptid, juhenda manual parandusi Acrobat Pro-s.
6. Databricks synthetic workflow puhul kasuta HTML→PDF→Unity Catalog upload + eval questions JSON struktuuri.
7. Väljundina anna struktureeritud JSON: operation, input, output, status, validation, errors, next_steps.
8. Ära kunagi palu kasutajal API võtmeid/parool/tokeneid vestlusse kleepida; kui vaja, juhenda keskkonna muutujate ja konfifailide kasutamist.

---

2. Soovituslik scripts/ struktuur (stubidega)

LuunaOS jaoks oleks mõistlik teha Python-põhised wrapperid:

scripts/
├── extract_text.py
├── extract_tables.py
├── analyze_form.py
├── validate_form.py
├── fill_form.py
├── ocr_pdf.py
├── merge_pdfs.py
├── split_pdf.py
├── redact_pdf.py
├── compress_pdf.py
├── validate_pdf.py
├── audit_pdf_accessibility.py
├── remediate_pdf_accessibility.py
├── pdf_generator.py        # Databricks HTML→PDF
└── batch_process.py

Näide batch_process.py skeleton:

import argparse
import subprocess
from pathlib import Path
import sys

EXIT_CODES = {
    "SUCCESS": 0,
    "FILE_NOT_FOUND": 1,
    "INVALID_INPUT": 2,
    "PROCESSING_ERROR": 3,
}

def run_for_all(operation: str, input_dir: str, output_dir: str):
    in_dir = Path(input_dir)
    out_dir = Path(output_dir)
    out_dir.mkdir(parents=True, exist_ok=True)

    if not in_dir.exists():
        print(f"[ERROR] Input dir not found: {in_dir}", file=sys.stderr)
        sys.exit(EXIT_CODES["FILE_NOT_FOUND"])

    script_map = {
        "extract_text": "extract_text.py",
        "extract_tables": "extract_tables.py",
        # laienda vajadusel
    }

    if operation not in script_map:
        print(f"[ERROR] Unsupported operation: {operation}", file=sys.stderr)
        sys.exit(EXIT_CODES["INVALID_INPUT"])

    script = script_map[operation]

    for pdf_path in in_dir.glob("*.pdf"):
        output_path = out_dir / (pdf_path.stem + (".txt" if "text" in operation else ".csv"))
        cmd = ["python", f"scripts/{script}", str(pdf_path), "--output", str(output_path)]
        result = subprocess.run(cmd, capture_output=True, text=True)
        if result.returncode == 0:
            print(f"✓ {operation}: {pdf_path} -> {output_path}")
        else:
            print(f"✗ {operation} failed for {pdf_path}: {result.stderr}", file=sys.stderr)

    sys.exit(EXIT_CODES["SUCCESS"])

if __name__ == "__main__":
    parser = argparse.ArgumentParser()
    parser.add_argument("--operation", required=True)
    parser.add_argument("--input-dir", required=True)
    parser.add_argument("--output-dir", required=True)
    args = parser.parse_args()
    run_for_all(args.operation, args.input-dir, args.output-dir)

---

3. n8n + Airtable workflow skeem

Airtable tabeli skeem (PDF jobs)

Tee Airtable base nt PDF Ops, tabel jobs:

Väljad:

• job_id – text (UUID)
• operation – single select (extract_text, extract_tables, forms_fill, ocr, merge, split, compress, accessibility_audit, accessibility_remediate, databricks_html_to_pdf, databricks_upload, databricks_eval_questions, batch)
• input_path – text
• output_path – text
• status – single select (queued, running, success, error)
• exit_code – number
• error_message – long text
• started_at – datetime
• finished_at – datetime
• extra – long text (JSON: pages, schema_name, databricks_volume, eval_question_count jne)

n8n workflow (loogika)

Node’d:

1. Webhook (Start Job)
   • trigger: POST /pdf-job
   • body: { operation, input_path, extra }
   • genereeri job_id (n8n UUID node või JavaScript).
2. Airtable – Create Record
   • tabel jobs,
   • set:
     • job_id
     • operation
     • input_path
     • status = "queued"
     • started_at = now
     • extra = JSON.stringify(extra).
3. Execute Command (LuunaOS / local script runner)
   • käivita vastav Python/CLI script:
   • command: python scripts/<script>.py ...
   • input_path / output_path n8n parameetritest.
4. IF – Exit Code
   • kui exit_code == 0 → success,
   • muidu → error.
5. Airtable – Update Record
   • status = "success" | "error"
   • exit_code
   • error_message (stderr)
   • output_path
   • finished_at = now.
6. Webhook Response
   • tagasta kasutajale:
   • { job_id, status, output_path, exit_code, error_message }.

Pseudo-n8n JSON (kontseptuaalne):

{
  "nodes": [
    {
      "name": "Webhook PDF Job",
      "type": "n8n-nodes-base.webhook",
      "parameters": { "path": "pdf-job", "methods": ["POST"] }
    },
    {
      "name": "Create Airtable Job",
      "type": "n8n-nodes-base.airtable",
      "parameters": { "operation": "create", "table": "jobs" }
    },
    {
      "name": "Run Script",
      "type": "n8n-nodes-base.executeCommand",
      "parameters": {
        "command": "python",
        "arguments": "scripts/{{ $json.script }}.py {{ $json.input_path }} --output {{ $json.output_path }}"
      }
    },
    {
      "name": "Update Airtable Job",
      "type": "n8n-nodes-base.airtable",
      "parameters": { "operation": "update", "table": "jobs" }
    }
  ]
}

Skriptivalik:

• n8n Function node’s:
  • map’i operation → script (extract_text → extract_text, ocr → ocr_pdf jne).

---

Kui tahad, võin järgmises sammus teha sulle:

• konkreetse LuunaOS agent definition (YAML/JSON),
• ja n8n workflow täieliku export JSON-i ühe konkreetse operatsiooni (nt extract_text + tables + Databricks eval) jaoks.

Kas eelistad esmalt fokusseeruda healthcare PDF vormide pipeline’ile (forms + accessibility) või Databricks RAG eval dataset pipeline’ile? ^1^11^13^15^3^5^7^9

⁂