Spaces:
Sleeping
Sleeping
File size: 1,917 Bytes
532d429 bfe9493 | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 | # Totem: Smoke Signal v0.1
Picture-book OCR & Codex extraction pipeline.
## Folder Structure
```
smoke_signal/
├── source_pdfs/ # Immutable — never overwrite source PDFs
├── manifest/ # source_manifest.csv, run logs
├── renders/ # Page PNG/TIFF renders from PDF
├── ocr_raw/ # Raw OCR outputs per page/region
├── regions/ # Region detection JSON, bounding boxes
├── review/ # Human review queue CSVs and corrections
├── exports/ # Codex-ready JSONL/CSV/Markdown exports
├── logs/ # Per-run timestamped logs
├── configs/ # Frozen config files per run
├── schemas/ # JSON schemas for all output types
├── reports/ # Validation and batch reports
└── scripts/ # Pipeline scripts
```
## Quickstart
```bash
# 1. Install dependencies
pip install -r requirements.txt
# 2. Add PDFs to source_pdfs/ folder
# 3. Register and hash all sources
python scripts/01_register_sources.py
# 4. Review manifest/source_manifest.csv and set rights_class for each book
# 5. Run PDF profiler (Stage 3)
# python scripts/02_profile_pdfs.py ← coming in next stage
```
## Control Principles
| Rule | Description |
|------|-------------|
| No silent acquisition | Every source file must be logged with rights/provenance/hash before extraction |
| No OCR soup in Codex | Low-confidence pages must be flagged before influencing fingerprints |
| No extractor logic drift | Changes to scoring/labels/thresholds require a versioned issue |
| Page-level traceability | Every exported line maps back to book ID, page, region, and extraction method |
## Naming Convention
- Files: `BOOKID_page_REGIONID.ext`
- Batches: `SS-BATCH-001`, `SS-BATCH-002`, ...
- Configs: `ss_ocr_config_v0.1.json`
- Schemas: `schema_book_manifest_v1.json`
# push test
|