Upload folder using huggingface_hub

.gitattributes

@@ -33,3 +33,18 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
 *.zip filter=lfs diff=lfs merge=lfs -text
 *.zst filter=lfs diff=lfs merge=lfs -text
 *tfevents* filter=lfs diff=lfs merge=lfs -text
+Code_files/__pycache__/USLaP_Engine.cpython-314.pyc filter=lfs diff=lfs merge=lfs -text
+Code_files/__pycache__/amr_aql.cpython-314.pyc filter=lfs diff=lfs merge=lfs -text
+Code_files/__pycache__/uslap_quf.cpython-314.pyc filter=lfs diff=lfs merge=lfs -text
+Code_files/backups/uslap_database_v3_pre_aa_migration_20260329_1306.db filter=lfs diff=lfs merge=lfs -text
+Code_files/backups/uslap_v3_backup_20260316_035513.db filter=lfs diff=lfs merge=lfs -text
+Code_files/backups/uslap_v3_backup_20260316_040701.db filter=lfs diff=lfs merge=lfs -text
+Code_files/backups/uslap_v3_backup_20260316_162512.db filter=lfs diff=lfs merge=lfs -text
+Code_files/backups/uslap_v3_backup_20260318_063949.db filter=lfs diff=lfs merge=lfs -text
+Code_files/backups/uslap_v3_backup_20260327_134853.db filter=lfs diff=lfs merge=lfs -text
+Code_files/backups/uslap_v3_pre_strip_20260328_073602.db filter=lfs diff=lfs merge=lfs -text
+Code_files/uslap_database.db filter=lfs diff=lfs merge=lfs -text
+Code_files/uslap_database_v3.db filter=lfs diff=lfs merge=lfs -text
+Code_files/uslap_database_v3_OLD.db filter=lfs diff=lfs merge=lfs -text
+Code_files/uslap_database_v3_pre_v4.db filter=lfs diff=lfs merge=lfs -text
+Code_files/uslap_lattice.db filter=lfs diff=lfs merge=lfs -text

Code_files/.db_sync_state

@@ -0,0 +1 @@
+4fa516ae7aff11b6e221bd1f65739df4

Code_files/.uslap_init_lock

@@ -0,0 +1 @@
+

Code_files/CONSOLIDATION_ISSUES.md

@@ -0,0 +1,75 @@
+# Schema Consolidation — Issues Found (2026-03-27)
+
+## Status: BLOCKED — needs fixes before re-run
+
+Backup: `backups/v3_pre_consolidation_20260327_1643.db` (85MB)
+DB restored to pre-consolidation state. All other session work intact.
+
+## Blocker 1: PK Conflicts on RU Mirror Tables
+
+RU tables use the SAME primary keys as EN tables (they're translations, not new data):
+- `a2_имена_аллаха`: allah_id 1-99 = same as `names_of_allah` allah_id 1-99
+- `a4_производные`: deriv_id = same as `a4_derivatives` deriv_id
+- `a5_перекрёстные_ссылки`: xref_id = same as `a5_cross_refs` xref_id
+
+**Fix options:**
+1. **UPDATE existing rows** — add RU content to the EN row (e.g., add `ru_meaning` column to `names_of_allah`). Preserves PK.
+2. **Offset PKs** — insert RU rows with PK + 100000 offset. Avoids conflict but breaks ID meaning.
+3. **Separate lang column** — generate NEW integer PKs for RU rows, add `lang='RU'` column. Original RU PK stored in `orig_ru_id` column.
+
+**Recommended: Option 1** for Names of Allah (same 99 names, just add RU fields). **Option 3** for derivatives/cross-refs (genuinely different data rows).
+
+## Blocker 2: Orphaned Views
+
+Several views reference tables that don't exist or have been renamed:
+- `m1_phonetic_shifts` → references `phonetic_shifts` (doesn't exist — data is in `shift_lookup`)
+- `a3_quran_refs` → is a VIEW, not a table
+- `a6_country_names` → is a VIEW, not a table
+- `a1_записи` → is a VIEW (data already in `entries`)
+- `a1_entries` → is a VIEW
+
+**Fix:** Drop orphaned views BEFORE dropping triggers. Current script drops triggers first, which causes ALTER TABLE to fail when it touches a table referenced by a view.
+
+**Correct order:**
+1. Save all triggers + views (SQL)
+2. Drop ALL views
+3. Drop ALL triggers
+4. Run migration
+5. Recreate views (new definitions)
+6. Recreate triggers (only for surviving tables)
+
+## Blocker 3: UNIQUE Constraints from Hardening
+
+`harden_v4_schema.py` added UNIQUE indexes:
+- `uq_entries_en_root` on `entries(en_term, root_id)`
+- `uq_bitig_orig2` on `bitig_a1_entries(orig2_term, root_letters)`
+- `uq_eu_lang_term` on `european_a1_entries(lang, term)`
+- `uq_lat_term` on `latin_a1_entries(lat_term)`
+- `uq_roots_letters` on `roots(root_letters)`
+
+These may block RU data insertion if values collide. Need to check each before INSERT.
+
+## Migration Script
+
+`consolidate_v5_clean.py` — handles Phases 1-3 but needs the above fixes.
+`consolidate_schema_v5.py` — original version, same issues.
+
+## What Was Completed This Session
+
+1. Domain-specific QUF (12 lattice layers) — 97% pass, 102K rows, 27 tables
+2. Extended QUF to 130 remaining tables — 40% pass
+3. 4 new AMR AI modules (jism, hisab, tarikh, istakhbarat) — all with domain QUF colours
+4. Schema hardening (indexes, views, health check)
+5. amr_lawh.py QUF filtering wired
+6. Automated backup script created
+7. Banned term "theological" removed from all code
+8. 12-layer lattice architecture defined (replaces 8 academic categories)
+
+## Next Session: Consolidation
+
+1. Fix Blocker 1: per-table PK strategy (UPDATE for names, new PKs for derivatives)
+2. Fix Blocker 2: drop views BEFORE triggers
+3. Fix Blocker 3: handle UNIQUE constraints
+4. Re-run consolidation
+5. Update code references (amr_jism.py, uslap_quf.py, uslap_handler.py, etc.)
+6. Re-run domain QUF on consolidated structure

Code_files/MIGRATION_GUIDE.md

@@ -0,0 +1,418 @@
+# USLaP Database Migration Guide
+
+**بِسْمِ اللَّهِ الرَّحْمَٰنِ الرَّحِيمِ**
+
+This guide provides step-by-step instructions for migrating data from the Excel master file to the full USLaP SQLite relational database. The migration creates a normalized schema designed for world-vocabulary scale (target: 25,000+ entries, 1M+ searchable objects).
+
+## Overview
+
+The migration process:
+1. Creates a fresh SQLite database with the full relational schema (`uslap_lattice.db`)
+2. Reads data from structured Excel sheets (skips the consolidated echo sheet)
+3. Normalizes data into proper relational tables with foreign key relationships
+4. Registers the `extract_consonants()` Python UDF for phonetic search
+5. Generates the `word_fingerprints` table for O(log n) cluster expansion
+6. Creates backups of any existing databases
+7. Verifies data integrity and foreign key constraints
+
+## Prerequisites
+
+### Required Files
+- `USLaP_Final_Data_Consolidated_Master_v3.xlsx` – Master Excel file (in workplace root)
+- `create_uslap_db.sql` – Complete SQLite schema (in `Code_files/`)
+- `migrate_to_sqlite.py` – Migration script (in `Code_files/`)
+- `USLaP_Engine.py` – Contains consonant extraction logic (in `Code_files/`)
+
+### Python Dependencies
+```bash
+pip install openpyxl
+```
+The script requires Python 3.6+ and the `openpyxl` library for reading Excel files. All other dependencies are in the Python standard library.
+
+## Running the Migration
+
+### Step 1: Verify File Locations
+Ensure all files are in the correct locations:
+```
+USLaP workplace/
+├── USLaP_Final_Data_Consolidated_Master_v3.xlsx
+└── Code_files/
+    ├── create_uslap_db.sql
+    ├── migrate_to_sqlite.py
+    ├── USLaP_Engine.py
+    └── [existing .db files]
+```
+
+### Step 2: Run the Migration Script
+From the `USLaP workplace` directory, run:
+```bash
+cd "/Users/mmsetubal/Documents/USLaP workplace"
+python3 "Code_files/migrate_to_sqlite.py"
+```
+
+**Important Notes:**
+- The script will automatically create a backup of any existing `uslap_lattice.db` file
+- Migration may take 1-2 minutes depending on Excel file size
+- All operations are wrapped in a transaction; on failure, the database is rolled back
+
+### Step 3: Monitor Migration Output
+The script provides real-time progress:
+```
+══════════════════════════════════════════════════════════════════════
+  USLaP Migration: Excel → SQLite Relational Database
+  بِسْمِ اللَّهِ الرَّحْمَٰنِ الرَّحِيمِ
+══════════════════════════════════════════════════════════════════════
+
+📖 Loading Excel file: USLaP_Final_Data_Consolidated_Master_v3.xlsx
+  Found 62 sheets
+
+🗄️  Creating database: Code_files/uslap_lattice.db
+  Executing schema...
+  Registering extract_consonants() UDF...
+
+📊 Migrating data...
+  Migrating A1_ENTRIES...
+    Migrated 59 entries
+  Migrating A1_ЗАПИСИ (Russian entries)...
+    Migrated 0 entries
+  [Additional sheets...]
+
+📈 Migration Statistics:
+────────────────────────────────────────
+  Total entries: 59
+  Total roots: 0
+  Child entries: 3
+  Word fingerprints: 6
+  Engine queue items: 0
+
+🔍 Verifying foreign key constraints...
+✓ All foreign key constraints satisfied
+
+══════════════════════════════════════════════════════════════════════
+✅ MIGRATION COMPLETED SUCCESSFULLY
+✅ Database: Code_files/uslap_lattice.db
+══════════════════════════════════════════════════════════════════════
+```
+
+## Verification Steps
+
+### Verify Database Structure
+After migration, verify all tables were created:
+```bash
+sqlite3 "Code_files/uslap_lattice.db" ".tables"
+```
+
+Expected output (30+ tables):
+```
+child_entries          decay_levels          operation_cycles
+child_entry_links      derivatives           operators
+cluster_cache          detection_patterns    operator_aliases
+country_names          engine_queue          operation_codes
+cross_refs             entries               phonetic_mappings
+change_log             events                phonetic_shifts
+dp_codes               host_civilizations    quran_refs
+intel_reports          languages             roots
+names_of_allah         networks              scholars
+nt_codes               op_codes              script_corridors
+qur_verification       session_index         sync_status
+word_fingerprints
+```
+
+### Verify Row Counts
+Run the verification queries in the database:
+```sql
+-- Check entry counts
+SELECT 'entries' as tbl, COUNT(*) FROM entries
+UNION ALL SELECT 'roots', COUNT(*) FROM roots
+UNION ALL SELECT 'derivatives', COUNT(*) FROM derivatives
+UNION ALL SELECT 'cross_refs', COUNT(*) FROM cross_refs
+UNION ALL SELECT 'child_entries', COUNT(*) FROM child_entries
+UNION ALL SELECT 'word_fingerprints', COUNT(*) FROM word_fingerprints;
+
+-- Check FK integrity
+PRAGMA integrity_check;
+PRAGMA foreign_key_check;
+
+-- Check fingerprint coverage
+SELECT COUNT(DISTINCT entry_id) as entries_with_fingerprints FROM word_fingerprints;
+```
+
+**Expected Results:**
+- `entries`: ~390–404 entries total across all languages
+- `roots`: ~200 unique roots
+- `derivatives`: ~632 word forms
+- `cross_refs`: ~155 relationships
+- `child_entries`: ~12 operational intelligence entries
+- `word_fingerprints`: ~1,400+ entries (covers all searchable terms)
+
+### Test the UDF Function
+Verify the `extract_consonants()` function works:
+```bash
+sqlite3 "Code_files/uslap_lattice.db" "SELECT extract_consonants('example')"
+```
+Expected output: `xmpl`
+
+### Test Basic Queries
+```sql
+-- Get entries with high confidence scores
+SELECT entry_id, en_term, score FROM entries WHERE score >= 8 ORDER BY score DESC LIMIT 10;
+
+-- Test phonetic search via word_fingerprints
+SELECT wf.raw_word, wf.consonant_skeleton, e.en_term, e.score
+FROM word_fingerprints wf
+LEFT JOIN entries e ON wf.entry_id = e.entry_id
+WHERE wf.consonant_skeleton = extract_consonants('test')
+LIMIT 5;
+
+-- View operational intelligence (CHILD schema)
+SELECT child_id, shell_name, operation_role, parent_op FROM child_entries;
+```
+
+## Database Schema Details
+
+### Critical Tables for Engine Operations
+
+#### 1. `word_fingerprints` – Phonetic Search Index
+The most critical table for performance. Enables O(log n) cluster expansion via the composite index on `(consonant_skeleton, language)`.
+
+**Triggers:** Automatic population when entries/derivatives/child entries are inserted or updated.
+
+**Index:** `idx_fingerprints_lookup` enables instant phonetic matching.
+
+#### 2. `engine_queue` – Write Conflict Prevention
+Prevents direct writes from the engine to core tables. All proposed changes go through this queue for user approval via the Oversight Dashboard.
+
+**Purpose:** Maintains Excel as the primary write interface while enabling engine proposals.
+
+#### 3. `session_index` – Engine Session Tracking
+Tracks every engine run with performance metrics and error logging.
+
+#### 4. `child_entries` & `child_entry_links` – Operational Intelligence
+CHILD schema integration links operational intelligence (SLV, SQLB, RUS entries) with main A1 entries.
+
+### Foreign Key Enforcement
+Foreign keys are strictly enforced (`PRAGMA foreign_keys = ON`). The migration script temporarily disables them during data insertion to avoid constraint violations, then re-enables and verifies all constraints.
+
+## Python UDF: `extract_consonants()`
+
+### Registration
+The UDF is automatically registered on every database connection via:
+```python
+conn.create_function("extract_consonants", 1, extract_consonants)
+```
+
+**CRITICAL:** Must be registered BEFORE any INSERT operations or trigger execution.
+
+### Implementation
+The function extracts consonant skeletons from words:
+- Removes vowels (a, e, i, o, u)
+- Handles digraphs (sh, ch, gh, th, ph, wh, qu) as single units
+- Normalizes to lowercase
+- Returns empty string for null/empty input
+
+**Source:** Logic matches `PhoneticReversal.extract_consonants()` in `USLaP_Engine.py`.
+
+### SQL Triggers Using the UDF
+The schema includes triggers that automatically populate `word_fingerprints`:
+```sql
+CREATE TRIGGER update_fingerprints_on_entry_insert
+AFTER INSERT ON entries
+BEGIN
+    INSERT INTO word_fingerprints (entry_id, language, raw_word, consonant_skeleton)
+    SELECT NEW.entry_id, 'en', NEW.en_term, extract_consonants(NEW.en_term)
+    WHERE NEW.en_term IS NOT NULL AND NEW.en_term != '';
+    -- ... similar for ru_term, fa_term, ar_word
+END;
+```
+
+## Excel ↔ Database Sync Strategy
+
+### Current Architecture
+- **Excel is primary write interface:** All user-facing writes go through Excel
+- **Database is read/query layer:** Engine reads from database, proposes changes via queue
+- **Sync direction:** Excel → Database (one-way during migration)
+
+### Maintaining Sync
+After migration, keep databases in sync:
+
+1. **Engine proposals:** When USLaP_Engine.py detects new patterns, it writes to `engine_queue`
+2. **User approval:** User reviews proposals in Oversight Dashboard (`USLaP_Oversight_Dashboard.html`)
+3. **Approved changes:** User applies approved changes to Excel manually
+4. **Re-sync:** Run migration script periodically to update database with Excel changes
+
+### Migration Script Updates
+The `migrate_to_sqlite.py` script can be re-run at any time. It will:
+1. Create timestamped backup of existing database
+2. Start fresh with current Excel data
+3. Preserve any `engine_queue` items that haven't been processed
+
+## Switching USLaP_Engine.py to SQLite Reads
+
+### Current State
+`USLaP_Engine.py` currently reads directly from Excel via openpyxl.
+
+### Target State
+Update `USLaP_Engine.py` to use the database access layer (`db_access_layer.py`) for:
+1. Entry lookups (instead of reading Excel sheets)
+2. Phonetic search (using `word_fingerprints` table)
+3. Cluster expansion (O(log n) via indexed searches)
+4. Queue operations (proposing changes via `engine_queue`)
+
+### Implementation Steps
+1. Import `db_access_layer` module
+2. Replace Excel reading with database queries:
+   ```python
+   # Old: reading from Excel
+   # New: using database
+   from db_access_layer import search_word, PhoneticSearchOperations
+   
+   results = search_word("example")
+   similar = PhoneticSearchOperations.find_similar_words("example", conn)
+   ```
+3. Update cluster expansion to use `word_fingerprints` index
+4. Route all proposed changes through `engine_queue` instead of direct writes
+
+### Performance Benefits
+- **Phonetic search:** O(log n) vs O(n) linear scan
+- **Cluster expansion:** Instant via pre-computed fingerprints
+- **Memory usage:** Database queries vs loading entire Excel file
+- **Concurrency:** Multiple engine sessions can query simultaneously
+
+## Troubleshooting
+
+### Common Issues
+
+#### 1. "Excel file not found"
+**Solution:** Ensure `USLaP_Final_Data_Consolidated_Master_v3.xlsx` is in the workplace root directory.
+
+#### 2. "Schema file not found"
+**Solution:** Ensure `create_uslap_db.sql` is in `Code_files/` directory.
+
+#### 3. Foreign key constraint violations
+**Solution:** The migration script temporarily disables foreign keys during insertion. If errors persist:
+```bash
+sqlite3 "Code_files/uslap_lattice.db" "PRAGMA foreign_key_check"
+```
+Check for circular dependencies or missing reference data.
+
+#### 4. UDF not registered
+**Solution:** Ensure `extract_consonants()` is registered before any inserts. The migration script does this automatically. For custom connections, use:
+```python
+from migrate_to_sqlite import extract_consonants
+conn.create_function("extract_consonants", 1, extract_consonants)
+```
+
+#### 5. Low row counts after migration
+**Possible causes:**
+- Excel sheet names don't match expected names
+- Header row detection failed
+- Data is in unexpected format
+
+**Debug:** Run the migration script with additional print statements or examine the Excel sheet structure.
+
+### Recovery Procedures
+
+#### Database Corruption
+If the database becomes corrupted:
+1. Restore from latest backup in `Code_files/backups/`
+2. Or re-run migration script (creates fresh database)
+
+#### Failed Migration
+If migration fails mid-process:
+1. Script automatically rolls back transaction
+2. Old database remains unchanged (if backup was created)
+3. Check error output for specific issue
+4. Fix underlying problem (Excel format, disk space, permissions)
+5. Re-run migration
+
+#### Data Loss Prevention
+- Migration always creates timestamped backups
+- Excel master file remains unchanged (read-only during migration)
+- Transaction rollback on any error
+
+## Performance Optimization
+
+### Index Usage
+The schema includes optimal indexes for:
+1. **Phonetic search:** `idx_fingerprints_lookup` on `(consonant_skeleton, language)`
+2. **Root-based queries:** `idx_entries_root` on `entries(root_id)`
+3. **Score sorting:** `idx_entries_score` on `entries(score DESC)`
+4. **Full-text search:** FTS5 virtual table `entries_fts`
+
+### Query Patterns
+For best performance:
+- Use `word_fingerprints` for phonetic searches
+- Use `entries_fts` for full-text keyword searches
+- Use `cluster_cache` for repeated expansion of same roots
+- Limit results with `LIMIT` clauses for UI responsiveness
+
+### Scaling Considerations
+The schema is designed for 1M+ searchable objects. At that scale:
+- Consider increasing SQLite cache size: `PRAGMA cache_size = -2000;` (2GB)
+- Use WAL mode for concurrent reads: `PRAGMA journal_mode = WAL;`
+- Regular VACUUM to maintain performance: `VACUUM;`
+
+## Appendix A: Migration Script Details
+
+### Sheets Migrated
+The script reads from these structured sheets only:
+- `A1_ENTRIES` – English entries
+- `A1_ЗАПИСИ` – Russian entries  
+- `PERSIAN_A1_MADĀKHIL` – Persian entries
+- `BITIG_A1_ENTRIES` – ORIG2/Turkic entries
+- `CHILD_SCHEMA` – Operational intelligence
+- `A4_DERIVATIVES` – Word forms
+- `A5_CROSS_REFS` – Entry relationships
+- `A3_QURAN_REFS` – Verse references
+- `M1_PHONETIC_SHIFTS` – Phonetic mechanism
+- `M2_DETECTION_PATTERNS` – Detection patterns
+- `M4_NETWORKS` – Network definitions
+- `M3_SCHOLARS` – Scholar biographies
+- `M5_QUR_VERIFICATION` – Qur'an verification
+
+### Sheets Skipped
+- `EXCEL_DATA_CONSOLIDATED` – Echo sheet (not a primary source)
+- Various protocol, correction, and warning sheets
+
+### Data Flow
+1. Read Excel sheet → Clean column names → Map to schema → Insert
+2. Extract unique roots from entries → Create `roots` table entries
+3. Triggers automatically create `word_fingerprints`
+4. Verify foreign key integrity
+5. Commit transaction
+
+## Appendix B: Database Access Layer
+
+The `db_access_layer.py` module provides:
+- `DatabaseConnection` – Context manager for connections
+- `EntryOperations`, `RootOperations` – CRUD operations
+- `PhoneticSearchOperations` – O(log n) cluster expansion
+- `EngineQueueOperations`, `SessionOperations` – Engine control
+- `AnalyticsOperations` – Statistics and analysis
+- High-level API functions: `search_word()`, `add_new_entry()`, `run_engine_session()`
+
+### Example Usage
+```python
+from db_access_layer import search_word, get_connection, EntryOperations
+
+# High-level search
+results = search_word("example")
+print(f"Found {len(results['exact_matches'])} exact matches")
+
+# Direct operations
+with get_connection() as conn:
+    entries = EntryOperations.get_high_score_entries(conn, min_score=8)
+    print(f"High-score entries: {len(entries)}")
+```
+
+## Support
+
+For issues with migration:
+1. Check error messages in console output
+2. Verify file permissions and locations
+3. Ensure Excel file isn't open in another program
+4. Check Python version and openpyxl installation
+
+To report bugs or request enhancements, use the project's issue tracking system.
+
+**وَاللَّهُ أَعْلَمُ**

Code_files/SESSION_44_PROMPT.md

@@ -0,0 +1,127 @@
+# SESSION 44 — Canon الواح Disease↔Cure Addressing System
+
+## بِسْمِ اللَّهِ الرَّحْمَٰنِ الرَّحِيمِ
+
+## MANDATORY FIRST ACTION
+```bash
+python3 Code_files/uslap_session_init.py
+```
+
+---
+
+## WHERE SESSION 43 LEFT OFF
+
+### Canon Book 2 — EXTRACTION COMPLETE
+- **278 drugs** in `canon_materia_medica` table (pages 282-442, Wellcome MS Arab 155, Isfahan 1632 CE)
+- **Introduction** (pages 262-281) extracted — methodology, الواح tables, mizaj system, taste encoding
+- Pipeline: Gemini Flash targeted extraction ($0.41) → abjad disambiguator → handler.write_entry()
+- Total API cost: ~$0.50 for entire Book 2
+
+### Discovery: الواح (Tablets) = Disease↔Cure Addressing System
+
+Ibn Sina's Introduction (p279) contains 11 الواح (computational tablets):
+
+```
+لوح الاورام والبثور      (Tumours/Pustules)     abjad=1068  mod7=4
+لوح الجراح والقروح       (Wounds/Ulcers)        abjad=638   mod7=1
+لوح آلات المفاصل         (Joint Apparatus)       abjad=748   mod7=6
+لوح أعضاء العين          (Eye Organs)            abjad=1077  mod7=6
+لوح أعضاء الرأس          (Head Organs)           abjad=1208  mod7=4
+لوح أعضاء النفض          (Excretion Organs)      abjad=1877  mod7=1
+لوح أعضاء الصدر          (Chest Organs)          abjad=1241  mod7=2
+لوح أعضاء المعدة         (Stomach Organs)        abjad=1061  mod7=4
+لوح أعضاء الكبد          (Liver Organs)          abjad=973   mod7=0
+لوح الزينة               (Cosmetics/Skin)        abjad=142   mod7=2
+لوح السموم               (Poisons/Antidotes)     abjad=221   mod7=4
+```
+
+### Discovery: Drugs address الواح through rational ratios
+
+Cross-referencing 278 drugs against 11 الواح found:
+- **14 drugs** at **11:5 SEED ratio (≈2.2)** with their target لوح
+- **29 drugs** at **7:5 KERNEL ratio (≈1.4)** with their target لوح
+- **451 drugs** with **SUM÷7** relationship
+- السكر (Sugar) dual-addresses: 2.2 with الزينة AND 1.4 with السموم
+- التين (Fig) / السموم = 2.222 — near-exact Falaq seed
+
+### Discovery: Abjad = built-in error-correction code
+
+Every Arabic rasm (undotted skeleton) family has ALL DISTINCT abjad values. Gaps between confused letter pairs: 48 (ب↔ن) to 930 (غ↔ع). This means the abjad system IS an error-correction checksum for Arabic text — the dots are visual disambiguation, the abjad values are mathematical disambiguation.
+
+### Discovery: Salah names encode computational structure
+
+```
+فجر   = 283   ف-ج-ر   SEPARATION+PRESS+MOVEMENT     (dawn)
+ظهر   = 1105  ظ-ه-ر   HEAVY_CONTACT+BREATH+MOVEMENT (midday, ÷5)
+عصر   = 360   ع-ص-ر   DEPTH+STREAM+MOVEMENT         (afternoon, ÷5, = CIRCLE)
+مغرب  = 1242  غ-ر-ب   SCRAPE+MOVEMENT+CLOSURE       (sunset, = 6 × بارد)
+عشاء  = 371   ع-ش-ي   DEPTH+SPREADING+CONTRACTION   (night, ÷7, NO ر)
+```
+
+---
+
+## SESSION 44 PRIORITIES
+
+### 1. Map FULL drug↔لوح relationships from body text
+Each drug entry in the MS has subcategory headings (الجراح, القروح, المفاصل, أعضاء العين, etc.) that specify WHICH الواح the drug addresses. Use Gemini Flash to extract these subcategory lists for each drug → build a drug→الواح mapping table.
+
+### 2. Compute drug↔disease abjad ratios
+For each drug→لوح pair confirmed from the MS text:
+- Compute drug_abjad / lawh_abjad
+- Check if ratio = 11/5 (2.2), 7/5 (1.4), 22/7 (π), or other rational from {3,5,7}
+- Build a ratio distribution table across ALL 278 drugs × their target الواح
+
+### 3. Taste encoding verification
+p269 maps: taste → mizaj (الحلاوة + كثيف → حار, المرارة → حار, etc.)
+- Extract mizaj (temperament) data for all 278 drugs from body text
+- Verify: does the drug's TASTE correctly predict its mizaj through the p269 table?
+- Compute: does the mizaj classification map to the correct لوح through abjad ratios?
+
+### 4. The hadith pair: داء (5) ↔ شفاء (381)
+- مرض root = 1040, شفي root = 390, ratio = 8/3
+- For each drug: compute drug_abjad / مرض_abjad and drug_abjad / شفي_abjad
+- Check if any drugs hit exact rational ratios with the disease/cure roots
+
+### 5. 2.2 in plant architecture
+formula_ratios RT0152-RT0158: plants exhibit 11:5 = 2.2 ratio internally.
+- Do the 14 drugs that hit 2.2 ratio with their لوح also have plant origins?
+- Is the 2.2 ratio the plant's INTERNAL code that matches the لوح it treats?
+
+---
+
+## KEY FILES
+
+| File | What it does |
+|------|-------------|
+| `Code_files/canon_materia_medica` (DB table) | 278 drugs with abjad, page, letter_section |
+| `Code_files/canon_ocr/introduction/` | 20 pages of introduction text + table scan |
+| `Code_files/canon_ocr/drug_extractions/` | Full extraction results + scored drugs |
+| `Code_files/ocr_dot_disambiguator.py` | Rasm→dotted Arabic + abjad checksum engine |
+| `Code_files/canon_deepseek_ocr.py` | OpenRouter pipeline (works with Gemini Flash) |
+| `Code_files/canon_book2_progress.txt` | Full session log with all findings |
+| `Documentation/USLAP_FULL_ARCHITECTURE.md` | Updated 2026-04-13 with Canon section |
+
+## OPENROUTER KEY
+```
+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
+```
+Gemini Flash: $0.0005/query. Budget for Session 44: ~$1 covers 2000 queries.
+
+## HF PRO
+Account: uslap. Training Space paused (uslap/canon-ocr-trainer). Training dataset at uslap/canon-ocr-train. Resume if needed but abjad approach may be sufficient.
+
+---
+
+## THE QUESTION SESSION 44 MUST ANSWER
+
+The hadith says: ما أنزل الله داء إلا أنزل له شفاء — "Allah did not send down a disease except that He sent down a cure for it."
+
+If this is computational (not metaphorical), then:
+- Each disease has an abjad address (from its root)
+- Each cure/drug has an abjad address (from its name)
+- The addressing system maps disease→cure through rational ratios from {3,5,7}
+- The الواح are the routing table
+- The mizaj (temperament) is the protocol
+- The taste is the input interface
+
+**Prove or disprove this from the MS data. Zero weights. DB only.**

Code_files/SESSION_47_HANDOFF.md

@@ -0,0 +1,143 @@
+# بِسْمِ اللَّهِ الرَّحْمَٰنِ الرَّحِيمِ
+
+# Session 47 Handoff — al-Biruni Abjad Disambiguator Complete
+
+**Date:** 2026-04-13  
+**DB state at close:** EN=3260, Roots=3320, Triggers=302, Diwan=8328
+
+---
+
+## What Session 47 accomplished
+
+### 1. Abjad Disambiguator — COMPLETE (`Code_files/biruni_abjad_disambiguate.py`)
+
+Built a full structural-constraint disambiguator for the 4 OCR'd abjad tables. No weights — every fill derives from the MS table's own logic.
+
+**Within-row passes (6):**
+- Pass 1: Direct OCR read
+- Pass 2: Sequential step-1 fill (degree column counting up/down)
+- Pass 3: Multi-cell linear gap fill
+- Pass 4: Alternating 0/ل(30) fill (minutes column) — per-row
+- Pass 5: Red-ink `[R]` prefix strip
+- Pass 6: Final bounded-pair re-check after updates
+
+**Cross-row column passes (5):**
+- CONSTANT: all known values identical → fill any remaining [?]
+- DOMINANT: 70%+ same value (e.g., ع=70 for all العراق cities)
+- ALTERNATING: {0, 30} by row parity — catches minutes column cross-strip
+- LOCAL LINEAR: for each [?], find nearest known above/below in same column; fill only when `diff % gap == 0` (integer step). Handles step-1 sequential, repeat-2 (half-degree), step-2, any integer step.
+- COMPLEMENT PAIRS: detects column pairs summing to a constant (mirror symmetry — degree_R + degree_L = 90 for sine table halves)
+
+**Results — structural ceiling, no weights:**
+
+| Table | Cells | OCR-num | Seq+Col | Unknown | Coverage |
+|-------|-------|---------|---------|---------|----------|
+| Sine (f102-107) | 3,637 | 2,142 | 311 | 1,184 | 67% |
+| Shadow (f112) | 1,743 | 1,132 | 98 | 513 | 70% |
+| Ascension (f131-136) | 3,929 | 2,335 | 295 | 1,299 | 66% |
+| City (f168-175) | 3,571 | 1,553 | 295 | 1,723 | 51% |
+| **GRAND** | **12,880** | **7,162** | **999** | **4,719** | **63%** |
+
+**Why city table is 51%:** Each city row has 4 numeric columns + 1-2 prose columns (place names, region names). Prose is correctly read by OCR but not expressible as abjad numbers → UNRESOLVED in the abjad layer. Actual numeric-cell coverage is higher.
+
+**Why 63% is the structural ceiling:** The 4,719 remaining unknowns split:
+- ~1,900: genuine OCR failures where Gemini returned [?] and no column constraint brackets them
+- ~2,800: correctly-read label text cells (Arabic prose, region names) counted as UNRESOLVED in abjad layer
+
+**Disambiguated output files (all in `Code_files/biruni_ms/abjad_ocr/`):**
+- `sine_table_ocr_disambiguated.json`
+- `shadow_table_ocr_disambiguated.json`
+- `ascension_table_ocr_disambiguated.json`
+- `city_table_ocr_disambiguated.json`
+
+Each cell has `(original_ocr, resolved_value, confidence_label)` — confidence labels include: OCR, OCR_RED, SEQUENCE, MULTI_GAP, ALTERNATE, CONSTANT, STEP2, PASS6_SEQ, COL_CONSTANT, COL_DOMINANT, COL_ALTERNATE, COL_LOCAL_CONST, COL_LINEAR, COL_HALFDEG, COL_COMPLEMENT, UNRESOLVED.
+
+---
+
+### 2. Supporting fixes completed in this session
+
+- **BL-HEB Hebrew block** added to `amr_dereference_audit.py` — catches U+0590-U+05FF in any output field, BL-HEB reference
+- **QUF PRIMARY_SOURCE path** fixed in `amr_istakhbarat.py` — arabic_text + source_ms + edition_page ≥ 3 → MEDIUM (was blocking 272 entries)
+- **43 AA-concept entries** written (entries 3200-3242) across Maqalat 2-11
+
+---
+
+## What is NOT yet done
+
+### Priority 1 — Parse disambiguated JSON → DB science tables
+
+The 4 `*_disambiguated.json` files contain the resolved abjad values but have NOT yet been written to the actual DB science tables. Four tables exist in DB:
+- `biruni_sine_table`
+- `biruni_shadow_table`
+- `biruni_ascension_table`
+- `biruni_city_coordinates`
+
+Only a small number of reference rows were written manually (14 sine, 17 shadow, 11 ascension, 35 city). The bulk of OCR + disambiguated data is still only in JSON.
+
+**Next step:** Write a `biruni_parse_ocr_to_db.py` script that:
+1. Reads each `*_disambiguated.json`
+2. Maps cells to the correct column (degree, minutes, sin_deg, sin_min, etc.) based on column position
+3. Inserts rows with confidence labels into the DB tables
+4. Skips UNRESOLVED cells (leave NULL in DB)
+
+Column order to use (confirmed from calibration):
+- Sine table: col0=degree, col1=minutes, col2=sin_deg, col3=sin_min, col4=sin_sec, col5=sin_thirds
+- Shadow table: same structure but shadow values
+- City table: col0=lon_deg, col1=lon_min, col2=lat_deg, col3=lat_min
+
+### Priority 2 — Re-OCR high-UNRESOLVED strips
+
+The disambiguated JSON flags which strips have the most UNRESOLVED cells. Re-run those specific strips with a more targeted Gemini prompt focusing on the specific cell format expected. Use `--table sine` with strip-level targeting.
+
+OpenRouter key was: `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`
+
+### Priority 3 — 3 missing roots
+
+- ن-ق-ط (point, nuqta) — check Quranic tokens, add if found
+- ز-و-ي (angle) — check Quranic tokens
+- د-ق-ق (minute, daqiqa) — check Quranic tokens
+
+### Priority 4 — 2 blocked vocab entries
+
+- خ-ط-ط (line, khatt): 0 Quranic tokens — blocked by QUF, pending root review
+- ه-ل-ل (crescent, hilal): 0 Quranic tokens in current DB — blocked
+
+### Priority 5 — f165-167 download
+
+Still missing from Gallica. Gallica returns 500/429 on these folios. Manual download or retry with longer delays.
+
+### Priority 6 — al-Athar al-Baqiya science extraction
+
+BnF Arabe 1489 (175 folios) downloaded as PDF. ~30-40 pages of genuine science (calendar computation + astronomical tables) identified. Operator king-list insertions (12 pages) identified and documented. Genuine science not yet extracted to DB.
+
+---
+
+## Architecture reminder
+
+```
+biruni_abjad_ocr.py          ← Gemini Flash OCR → *_ocr.json
+biruni_abjad_disambiguate.py ← structural constraints → *_disambiguated.json
+biruni_parse_ocr_to_db.py    ← [NOT YET WRITTEN] → DB science tables
+```
+
+## Key technical note: abjad authenticity filter
+
+Three numeral systems encountered across MSS:
+1. AA abjad (ا=1, ب=2...ص=90) — genuine science, pre-operator
+2. Bitig word-numerals (bir/eki/üç) — genuine ORIG2
+3. Positional ١٢٣ (eastern Arabic-Indic digits) — operator insertion marker
+
+**Check notation FIRST when opening any MS.** Positional = operator's tradition betraying the operator.
+
+---
+
+## DB state at session close
+
+```
+entries:     EN=3260 (43 Biruni entries 3200-3242, all QUF=TRUE)
+roots:       3320
+diwan_roots: 8328
+triggers:    302 (173 contamination, 25 QUF, 9 auto-index, 6 diwan)
+QUF:         entries 331/3242 (10%), roots 3259/3320 (98%)
+MS registry: ms_id=5 (BnF Arabe 6840, al-Qanun al-Masudi, 502 AH)
+```

Code_files/__pycache__/USLaP_Engine.cpython-314.pyc

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:f29784b512170a15be4743f680d85060345474c3fba8b815223c2bc8c4430f06
+size 232088

Code_files/__pycache__/amr_aql.cpython-314.pyc

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:f2e9206f7196ea6089db983af921786b368b04b4d302794ac05f51c4a60a141f
+size 107662