# Indra OCR — Engineering Investigation (v3.1.1, package 0.3.2) Investigator: OCR_INDRA workspace, 2026-10-17. Method: every claim below is backed by (a) model artifacts inspected bit-for-bit (`models/indra-ocr-v3.1.1/`, Hub revision `v3.1.1`) or (b) source at a cited file:line in the cloned repo (`indra/`, commit `3d773ae`, tag v0.3.2). Verification scripts live in `tools/indra_eval/` and are re-runnable. Status legend: **[V]** verified with evidence here, **[B]** blocked (needs the remote machine `khalti-t4`), **[X]** closed as refuted/inaccurate vs the earlier report `INDRA_OCR_NEPALI_FEASIBILITY.md`. --- ## 0. Verdict **The recognizer is trainable in principle and Nepali-ready in vocabulary, but fine-tuning is a reimplementation project: the repo ships no training code and no training weights for anything but the recognizer.** - Zero-shot evaluation (Experiment A) is *ready today* — the adapter (`tools/indra_eval/eval_indra.py`) runs Indra end-to-end on CPU and scores CER; it needs only the frozen Nepali eval sets from the remote machine [B]. - The CTC recognizer (`rec.onnx`) covers 100% of Devanagari (all 128 assigned codepoints U+0900–U+097F verified in the vocab) — no head surgery needed [V]. - In a same-crop head-to-head under our environment, the ~1.24 M-param Indra recognizer scored **CER 3.49%** on a synthetic Nepali page vs the deployed PP-OCRv5-Mobile Devanagari recognizer's published behavior on similar synthetic material (to be re-measured on identical crops) [V smoke / B]. - Fine-tuning only the recognizer with `model.safetensors` as init is the *only* supported adaptation path: the artifacts ship trainable weights for `rec/` exclusively; det/order/orient/math exist only as ONNX graphs with no accompanying state dict [V]. --- ## 1. What exactly is the ~1M-param recognizer? [V] **Answer: a MobileNetV3-style 1-D CNN trunk with padding-masked SE blocks and a flat CTC head — 1,236,366 parameters exactly, no RNN, no attention.** Evidence: `tools/indra_eval/inspect_rec_nodes.py` (201-node dump of `rec.onnx`), `tools/indra_eval/inspect_rec_mask.py` (intermediate-tensor run), `tools/indra_eval/inspect_models.py` (safetensors ↔ ONNX initializer cross-check: 108 tensors, 1,236,366 F32 params, byte-identical set). ### 1.1 Tensor flow (all verified from the graph) ``` input x [B,1,32,W] f32 ink map (0=paper, 1=ink; see §1.3) input widths [B] int64 (true crop widths BEFORE padding) 0 Conv(k=32x4, stride=32x4, 1→128, pad 0) "collapse height, stride time 4" 2 Squeeze axis 2 → [B,128,T] with T = W/4 1 BN(eps 1e-5, stem-only BatchNormalization) → HardSwish 43 Mul × Cast_4 (padding mask, see §1.2) → trunk input 10 × { dw Conv [128,1,9] g=128 pad 4/4 expand Conv 1×1 128→256 → HardSwish SE: Mul(mask) → ReduceSum(t) ←/→ ReduceSum(mask) = valid count Div (mean over VALID timesteps, Clip min 1) fc1 256→64 → Relu → fc2 64→256 → Sigmoid → gate Mul project Conv 1×1 256→128 Add residual → Mul(mask) } ← mask re-applied per block 188 Conv 1×1 128→1678 (head) → Transpose [B,T,1678] 191 ArgMax → ids [B,T] 190/193..200 softmax-max (logsumexp-free: ReduceMax, Exp, ReduceSum, Recip) → conf [B,T] 27 Clip(lengths) → lengths [B] ``` Op census (from the node dump): 52 Conv, 11 HardSwish, 10 Relu, 10 Sigmoid (SE gates), 32 Mul, 12 ReduceSum, 1 BatchNormalization, 1 ArgMax — plus glue. The parameters: 10 identical residual blocks (dw 128×1×9 + bias, expand 256×128, project 128×256, SE 64×256 + 256×64) ≈ 118 k/block + stem 128×32×4 + BN 4×128 + head 1678×128 ≈ 1,236,366 total [V]. ### 1.2 The padding mask `Cast_4` — semantics now fully pinned [V, closes a [U]] `Cast_4 = (arange(W/4) < lengths).float()` of shape `[B,1,W']`, built in-graph from: - nodes 3–27: `lengths = floor((widths+3)/4) = ceil(widths/4)`, clipped to `W/4` (Clip max = `Shape(x)[2] / 4`... clipped to the feature length); numeric table: `widths 1,2,3,4 → 1; 5 → 2; 9 → 3; 65 → 17; 129 → 32 (= W/4, clipped); 200 → 32`. - nodes 31–40: `Range(0, W/4) < lengths` (broadcast `[1,W'] < [B,1]`) → `Cast` f32 twice (PyTorch export artifact of `unsqueeze().to(f32)`). The mask is consumed **three ways** (this closes the earlier "[U] what does Cast_4 do?"): 1. **post-stem zeroing** (node 43) and **per-block output re-zeroing** (nodes 43, 61, 75, …, 187): the trunk never carries energy into padded timesteps — proven numerically: with ink drawn across the whole 128-column input but `widths=64`, final-block features at t ≥ 16 are *exactly 0.0* (max |f| = 0.0) while the same pixels unmasked (widths=128) reach |f|=15.6; 2. **padding-aware SE pooling** (each block: `ReduceSum(HW × mask) / max(ReduceSum(mask), 1)` → channel means over VALID timesteps only; the mask sum is the divisor, so padded columns cannot dilute the squeeze stats); 3. the **head reads masked features** (node 187→188), so at padded timesteps the graph emits whatever the head computes on a zero vector — measured: argmax = 0 (blank) with conf ≈ 0.78. **`ids/conf` at t ≥ lengths are NOT special-cased; callers must truncate by `lengths`.** `pipeline.py:297` does exactly this (`n = int(lengths[k])`). ### 1.3 Time stride is 4, not 8 [X — closes a wrong claim] `rec_preproc.py:24` says `WIDTH_MULTIPLE = 8 # widths are padded to a multiple of this (timestep stride)`. The graph contradicts the parenthetical: the stem Conv strides are `[32,4]`, giving **one timestep per 4 input columns** (`T = W/4`, `lengths = ceil(widths/4)`), and padding to a multiple of 8 is about the W dimension of the input only (it makes every 2 timesteps aligned, a vectorization nicety). Verified numerically: `T = 32` for `W = 128`, and `lengths` equals `ceil(widths/4)` at every measured value. The comment is wrong as written; the code is right. ### 1.4 Output contract [V — closes a [U]] Graph outputs are exactly `ids [B,T] int64`, `conf [B,T] f32`, `lengths [B] int64` (argmax + softmax max prob + valid count computed in-graph). `pipeline.py:294` consumes them positionally as `best, pmax, lengths`, then: - truncates to `n = int(lengths[k])` (:297), - CTC-greedy collapse: keep ≠ blank(0) and skip repeats (:299), - confidence = mean `pmax` over kept tokens (:315), - **drops the whole line if conf < `MIN_CONF = 0.5`** (:60, :316–319). Any retrained recognizer exported to ONNX must reproduce this contract (argmax + softmax-max + lengths on the same output names/order), or `pipeline.py:294` and the x-positions step (which maps CTC peak timesteps back to page x via the crop scale, :295–306) must be updated. ### 1.5 Recognizer input preprocessing (the shard that fine-tuning must clone) `rec_preproc.py` — `crop_line` (:27): expand box by `PAD_Y=0.2 × h` vertically and `PAD_X=0.3 × h` horizontally (:22–23), restrict to page, resize so height = `LINE_HEIGHT = 32` keeping aspect (`out_w = max(WIDTH_MULTIPLE, min(max_width=4096, round(cw*32/ch)))`), producing an ink map where paper = 0. `pad_batch` (:56) right-pads W to a multiple of 8 in the batch; `width_batches` (:69) groups crops so W-dim batch padding is bounded. The TRAINER uses the same crop per its docstring (:5–10 "expand … resize … ink map") — the docstring states train/infer share this shard, which is why zero-shot works at all and why fine-tuning must reimplement `crop_line` + `pad_batch` bit-exactly. ## 2. What handles Devanagari? [V] **Only `rec.onnx` reads Devanagari — and its vocabulary already includes every assigned Devanagari codepoint.** - `vocab.py:170` `Vocab.multi()` → 1,677 tokens + blank class 0 (:3, :207), head = 1,678 classes. The `meta.json` vocab string is exactly this vocab (`tools/indra_eval/inspect_vocab.py`). - Coverage: **128/128 assigned codepoints in U+0900–U+097F** — 80 Lo letters, 11 Mc vowel signs, 23 Mn marks incl. ़ (U+093C nukta) and the conjunct virama क्, plus ॉ (U+0950), ँ । (U+0964), ॥ (U+0965), digits ०–९ (U+0966–096F), ₹ (U+20B9). Bengali block also present (96 codepoints). - The recognizer is routed by two *fixed* guards, not the network: - `script.onnx` (hanzi binary classifier, sigmoid scalar; `pipeline.py:274–289`) sends hanzi lines to `rec_cjk.onnx` — irrelevant for Nepali pages (all-Devanagari pages never trigger it; the classifier is line-level and trained on hanzi). - `vocab.canonical` (:42) NFC-normalizes; `Vocab.encode` (:212) drops out-of-vocab codepoints — fine-tuning data must be pre-normalized the same way or the loss silently degrades (the trainer's data pipeline is closed; safest is to clone `canonical`/`encode`). - Confirmed counter-evidence collected in the smoke run: ़ (nukta) is often *dropped contextually* at 32 px height (ऱ and र differ only by a 1–2 px dot); ₹ was read as र. Both are training-distribution issues, not capability gaps — exactly the kind of thing fine-tuning would fix (Experiment B). ## 3. Component map: trainable / fixed-but-replaceable / fixed [V] | Component | Where weights come from | Trainable via shipped artifacts? | Replaceability | |---|---|---|---| | `rec.onnx` recognizer | `model.safetensors` (108 tensors, all `rec/`) | **YES** — the only shipped state dict; init for a torch CTC retrain (Experiment B) | head/vocab identical → drop-in for +1,678 classes | | `det.onnx` line detector | ONNX only | NO state dict shipped | could retrain in torch (targets spec exists: `det_post.encode_targets`, `det_post.py:48` — `core logit + top/bottom distances` per line, `CORE_FRAC=0.4`, `CORE_X_SHRINK=0.15`) — but training code is closed → high effort | | `order.onnx` reading order/layout | ONNX only | NO | same: `order_post.label_order`/`decode_*` reveal supervision (10 block classes) but no weights | | `orient.onnx` page rotation | ONNX only | NO | trivial to replace with a torch/tlclassifier or heuristic | | `mask.onnx` CenterMask | NOT SHIPPED in v3.1.1 (`pipeline.py:201–205` optional) | — | absent here; `crop_lines(gray, boxes, 32, None)` runs w/o it (verified) | | `script.onnx` + `rec_cjk.onnx` | ONNX only | NO | optional; would be excluded from a Devanagari build | | `math*.onnx` (4 files) | NOT shipped unless `math=True` (`models.py:22–44`) | — | out of scope | | geometry/deskew/layout postproc | pure Python (`pipeline.py:37–149`, `layout_post.py`) | fixed code, no training needed | replaceable with own code paths | | text-layer (PDF) naive path | `textlayer.py` | fixed code | irrelevant to scans/Nepali notes | **The decisive asymmetry:** Google ships a *recognizer-only* training starting point. Fine-tuning anything else means either (a) training from scratch (det targets spec'd, but no reference weights/recipe) or (b) leaving det/order frozen ONNX and only exercising the rec head — that's Experiment B; both are documented in `INDRA_EXPERIMENTS.md`. ## 4. Integration architecture (how a page becomes text) [V] `pipeline.py:456–470` (`OrtOCR.__call__`): 1. `upright()`: optional `orient.onnx` turn above `ORIENT_MIN = 0.95` (:54) + skew estimate/deskew up to 5° (:62–63). 2. `detect()`: `det.onnx` on a page resized long-side to detector input, threshold → line boxes (uses `det_post.page_to_input` / `decode`; `ADAPTIVE_MIN_H=9px` re-detect pass at a larger scale, :37–40). 3. `recognize()` (:264–320): CenterMask if present (not shipped); per-line `crop_line` → `pad_batch` → rec session, CTC collapse & conf gating (§1.2/§1.4). 4. line splitting at wide gaps (`SPLIT_GAP=3.0`, `CELL_GAP=1.8`, `GUTTER_GAP=1.6`, :44–47) and gutter detection (:342). 5. `reading_order()` on `order.onnx` (`feats [1,N,16]`, outputs pairwise salience `a`, block class `cls` [10 classes], same-block `same`, table t-heads `tab`) then layout (`layout_post.py: page_blocks`, `page_markdown`). MIN_CONF=0.5 means **a busy page with low-quality Devanagari can silently emit empty lines** in Experiment A — compare-on-GT-boxes (rec-lines mode) sidesteps this entirely, which is why Experiment A does both full-pipeline and rec-lines scoring. ## 5. Licensing [V] - **Model weights** (`models/indra-ocr-v3.1.1/LICENSE`): **AI PUBS OPEN RAIL-M (MODIFIED), v0.1, 2023-03-02**. Section II ¶36 states verbatim: *"Use may include creating any content with, fine-tuning, updating, running, training, evaluating and/or re-parametrizing the Model"* — **fine-tuning is explicitly licensed**. ¶5 attaches Attachment A use-based restrictions (standard OpenRAIL misuse list — weapons/illegal/defamation; OCR and document processing is untouched) and requires passing the same restrictions to distributees; ¶4 permits hosting/redistribution with notice. Derivative weights (fine-tuned checkpoints, distillations) fall under the same license. - **Code** (`indra/LICENSE`): **Apache-2.0** — reuse, modification, redistribution of the inference package are standard. - Practical consequence: our fine-tuned recognizer (Experiment B) is legally a "Derivative of the Model" under OpenRAIL-M — internal deployment and evaluation are unrestricted; redistribution requires keeping the license + restriction notice. Full matrix incl. Nepali datasets: `§12` of `INDRA_EXPERIMENTS.md` (dataset side **[B]** — needs `docs/DATA_LICENSING.md` on the remote machine). ## 6. Closing the prior report's [U] items | Prior unknown | Status | Evidence | |---|---|---| | exact recognizer arch (RNN? CNN?) | **[V] + [X]**: pure CNN+CTC, NO RNN/attention | §1.1, node dump | | timestep stride | **[X]**: 4, not 8 | §1.3 (graph constants + inc.timings) | | output contract (best/pmax docstrings vs solo head) | **[V]**: `ids`,`conf`,`lengths` names; truncation mandatory | §1.4 | | safetensors ↔ ONNX mapping (esp. depthwise `onnx::Conv_369…397`) | **[V]**: 108 tensors = 108 initializers, 1,236,366 params both ways; dw weights are `rec/onnx::Conv_369` [128,1,9] + `…370` [128] bias pairs | `inspect_models.py` | | Devanagari coverage claim | **[V]**: 128/128 codepoints in vocab | `inspect_vocab.py` | | What re-export must reproduce | **[V]**: mask + per-block re-zeroing + SE-valid-norm + decode tail; `lengths = ceil(widths/4)` clip W/4 | §1.2, `inspect_rec_mask.py` | | Confidence semantics of padded steps | **[X]**: head-on-zeros ≈ blank conf 0.78; caller truncates | §1.2(3) | | vocab[token i+1] ↔ blank 0 | **[V]** | `vocab.py:3,207,241–242` | | rec input normalization | **[V]**: ink map only (0 paper, 1 ink) per `crop_line`, no mean/std | `rec_preproc.py:1–10` | | enables fine-tune? | **[V] yes for rec only** (weights exist; no training code; CTC contract reconstructed) | §3, `§B` doc | | full pipeline runs on Nepali test image | **[V] smoke**: synthetic shaped page, 10/10 lines, CER 2.6–3.5% | `results/smoke/` | | what differs in real (non-synth) Nepali | **[B]** frozen remote sets + baseline | Experiment A | | trained det/order weights for fine-tune | **[V] not shipped** (ONNX only) | §3 | ## 7. Open items (not closed by this investigation) 1. **[B]** Zero-shot numbers on the frozen Nepali eval sets (needs `khalti-t4`). 2. **[B]** Head-to-head vs PP-OCRv5 Mobile Devanagari *on the same crops* — the remote machine holds the paddle recognizer + eval harness. The local smoke CER of 2.6–3.5% is on a synthetic page with a 32 px font and is *not* a benchmark. 3. **[B]** Licensing matrix final values need `docs/DATA_LICENSING.md` and `docs/DEVANAGARI_RESOURCES_INVENTORY.md` (remote). 4. FX-detector quality on Devanagari (qubee endpoints, aggressively slanted conjuncts) is unknowable without the det model being exercised on real scans — that's part of Experiment A full-pipeline. 5. The *root* training repo (referenced by name in the closed code) is not accessible; Archived recipe unknowns: batch sizes, LR schedule, data mix. ## 8. Reproduce everything ```bash cd /mnt/c/Users/Dell/Desktop/OCR_INDRA # or wherever the repo is PY=/tmp/opencode/indra-lib # pip --target deps export PYTHONPATH="$PWD/indra:$PY" python3 tools/indra_eval/inspect_vocab.py # vocab + Devanagari coverage python3 tools/indra_eval/inspect_models.py # graph I/O + param count python3 tools/indra_eval/inspect_rec_nodes.py # full node dump + smoke run python3 tools/indra_eval/inspect_rec_mask.py # mask semantics + lengths table python3 tools/indra_eval/make_synth_page.py results/smoke # synth page + GT python3 tools/indra_eval/eval_indra.py predict results/smoke/synth_nepali_page.png \ --boxes results/smoke/synth_nepali_gt.jsonl --jsonl results/smoke/rec_only.jsonl python3 tools/indra_eval/eval_indra.py predict results/smoke/synth_nepali_page.png \ --jsonl results/smoke/full_run.jsonl --print-text python3 tools/indra_eval/eval_indra.py cer results/smoke/synth_nepali_record.jsonl \ results/smoke/rec_only.jsonl --detail results/smoke/cer_rec_only.json --rec-only python3 tools/indra_eval/eval_indra.py cer results/smoke/synth_nepali_record.jsonl \ results/smoke/full_run.jsonl --detail results/smoke/cer_full.json ``` Observed (this machine, WSL, CPU, 8 cores, 3 GB RAM): rec-only CER 3.49% / WER 15.79%; full-pipeline CER 2.62% / WER 13.16%; 10/10 lines found; mean_conf 0.962–0.963. Timings on this box are meaningless (det ≈ 18.6 min — machine-starved; bench on the T4).