Download OCR_INDRA/docs/INDRA_INVESTIGATION.md from prosperious/ocr-projects-archive: direct link, hf CLI and curl.
- Browser
- Download file 17.6 kB
-
https://huggingface.co/prosperious/ocr-projects-archive/resolve/main/OCR_INDRA/docs/INDRA_INVESTIGATION.md
- Command line
-
hf download hf://prosperious/ocr-projects-archive/OCR_INDRA/docs/INDRA_INVESTIGATION.md
-
curl -L -o INDRA_INVESTIGATION.md https://huggingface.co/prosperious/ocr-projects-archive/resolve/main/OCR_INDRA/docs/INDRA_INVESTIGATION.md
Indra OCR β Engineering Investigation (v3.1.1, package 0.3.2)
Investigator: OCR_INDRA workspace, 2026-10-17.
Method: every claim below is backed by (a) model artifacts inspected bit-for-bit
(models/indra-ocr-v3.1.1/, Hub revision v3.1.1) or (b) source at a cited
file:line in the cloned repo (indra/, commit 3d773ae, tag v0.3.2).
Verification scripts live in tools/indra_eval/ and are re-runnable.
Status legend: [V] verified with evidence here, [B] blocked (needs the
remote machine khalti-t4), [X] closed as refuted/inaccurate vs the
earlier report INDRA_OCR_NEPALI_FEASIBILITY.md.
0. Verdict
The recognizer is trainable in principle and Nepali-ready in vocabulary, but fine-tuning is a reimplementation project: the repo ships no training code and no training weights for anything but the recognizer.
- Zero-shot evaluation (Experiment A) is ready today β the adapter
(
tools/indra_eval/eval_indra.py) runs Indra end-to-end on CPU and scores CER; it needs only the frozen Nepali eval sets from the remote machine [B]. - The CTC recognizer (
rec.onnx) covers 100% of Devanagari (all 128 assigned codepoints U+0900βU+097F verified in the vocab) β no head surgery needed [V]. - In a same-crop head-to-head under our environment, the ~1.24 M-param Indra recognizer scored CER 3.49% on a synthetic Nepali page vs the deployed PP-OCRv5-Mobile Devanagari recognizer's published behavior on similar synthetic material (to be re-measured on identical crops) [V smoke / B].
- Fine-tuning only the recognizer with
model.safetensorsas init is the only supported adaptation path: the artifacts ship trainable weights forrec/exclusively; det/order/orient/math exist only as ONNX graphs with no accompanying state dict [V].
1. What exactly is the ~1M-param recognizer? [V]
Answer: a MobileNetV3-style 1-D CNN trunk with padding-masked SE blocks and a flat CTC head β 1,236,366 parameters exactly, no RNN, no attention.
Evidence: tools/indra_eval/inspect_rec_nodes.py (201-node dump of
rec.onnx), tools/indra_eval/inspect_rec_mask.py (intermediate-tensor run),
tools/indra_eval/inspect_models.py (safetensors β ONNX initializer
cross-check: 108 tensors, 1,236,366 F32 params, byte-identical set).
1.1 Tensor flow (all verified from the graph)
input x [B,1,32,W] f32 ink map (0=paper, 1=ink; see Β§1.3)
input widths [B] int64 (true crop widths BEFORE padding)
0 Conv(k=32x4, stride=32x4, 1β128, pad 0) "collapse height, stride time 4"
2 Squeeze axis 2 β [B,128,T] with T = W/4
1 BN(eps 1e-5, stem-only BatchNormalization) β HardSwish
43 Mul Γ Cast_4 (padding mask, see Β§1.2) β trunk input
10 Γ { dw Conv [128,1,9] g=128 pad 4/4
expand Conv 1Γ1 128β256 β HardSwish
SE: Mul(mask) β ReduceSum(t) β/β ReduceSum(mask) = valid count
Div (mean over VALID timesteps, Clip min 1)
fc1 256β64 β Relu β fc2 64β256 β Sigmoid β gate Mul
project Conv 1Γ1 256β128
Add residual β Mul(mask) } β mask re-applied per block
188 Conv 1Γ1 128β1678 (head) β Transpose [B,T,1678]
191 ArgMax β ids [B,T]
190/193..200 softmax-max (logsumexp-free: ReduceMax, Exp, ReduceSum, Recip)
β conf [B,T]
27 Clip(lengths) β lengths [B]
Op census (from the node dump): 52 Conv, 11 HardSwish, 10 Relu, 10 Sigmoid (SE gates), 32 Mul, 12 ReduceSum, 1 BatchNormalization, 1 ArgMax β plus glue. The parameters: 10 identical residual blocks (dw 128Γ1Γ9 + bias, expand 256Γ128, project 128Γ256, SE 64Γ256 + 256Γ64) β 118 k/block + stem 128Γ32Γ4 + BN 4Γ128 + head 1678Γ128 β 1,236,366 total [V].
1.2 The padding mask Cast_4 β semantics now fully pinned [V, closes a [U]]
Cast_4 = (arange(W/4) < lengths).float() of shape [B,1,W'], built in-graph
from:
- nodes 3β27:
lengths = floor((widths+3)/4) = ceil(widths/4), clipped toW/4(Clip max =Shape(x)[2] / 4... clipped to the feature length); numeric table:widths 1,2,3,4 β 1; 5 β 2; 9 β 3; 65 β 17; 129 β 32 (= W/4, clipped); 200 β 32. - nodes 31β40:
Range(0, W/4) < lengths(broadcast[1,W'] < [B,1]) βCastf32 twice (PyTorch export artifact ofunsqueeze().to(f32)).
The mask is consumed three ways (this closes the earlier "[U] what does Cast_4 do?"):
- post-stem zeroing (node 43) and per-block output re-zeroing
(nodes 43, 61, 75, β¦, 187): the trunk never carries energy into padded
timesteps β proven numerically: with ink drawn across the whole 128-column
input but
widths=64, final-block features at t β₯ 16 are exactly 0.0 (max |f| = 0.0) while the same pixels unmasked (widths=128) reach |f|=15.6; - padding-aware SE pooling (each block:
ReduceSum(HW Γ mask) / max(ReduceSum(mask), 1)β channel means over VALID timesteps only; the mask sum is the divisor, so padded columns cannot dilute the squeeze stats); - the head reads masked features (node 187β188), so at padded timesteps
the graph emits whatever the head computes on a zero vector β measured:
argmax = 0 (blank) with conf β 0.78.
ids/confat t β₯ lengths are NOT special-cased; callers must truncate bylengths.pipeline.py:297does exactly this (n = int(lengths[k])).
1.3 Time stride is 4, not 8 [X β closes a wrong claim]
rec_preproc.py:24 says WIDTH_MULTIPLE = 8 # widths are padded to a multiple of this (timestep stride). The graph contradicts the parenthetical:
the stem Conv strides are [32,4], giving one timestep per 4 input
columns (T = W/4, lengths = ceil(widths/4)), and padding to a multiple
of 8 is about the W dimension of the input only (it makes every 2 timesteps
aligned, a vectorization nicety). Verified numerically: T = 32 for
W = 128, and lengths equals ceil(widths/4) at every measured value.
The comment is wrong as written; the code is right.
1.4 Output contract [V β closes a [U]]
Graph outputs are exactly ids [B,T] int64, conf [B,T] f32,
lengths [B] int64 (argmax + softmax max prob + valid count computed
in-graph). pipeline.py:294 consumes them positionally as
best, pmax, lengths, then:
- truncates to
n = int(lengths[k])(:297), - CTC-greedy collapse: keep β blank(0) and skip repeats (:299),
- confidence = mean
pmaxover kept tokens (:315), - drops the whole line if conf <
MIN_CONF = 0.5(:60, :316β319).
Any retrained recognizer exported to ONNX must reproduce this contract
(argmax + softmax-max + lengths on the same output names/order), or
pipeline.py:294 and the x-positions step (which maps CTC peak timesteps
back to page x via the crop scale, :295β306) must be updated.
1.5 Recognizer input preprocessing (the shard that fine-tuning must clone)
rec_preproc.py β crop_line (:27): expand box by PAD_Y=0.2 Γ h vertically
and PAD_X=0.3 Γ h horizontally (:22β23), restrict to page, resize so
height = LINE_HEIGHT = 32 keeping aspect (out_w = max(WIDTH_MULTIPLE, min(max_width=4096, round(cw*32/ch)))), producing an ink map where paper = 0.
pad_batch (:56) right-pads W to a multiple of 8 in the batch; width_batches
(:69) groups crops so W-dim batch padding is bounded. The TRAINER uses the
same crop per its docstring (:5β10 "expand β¦ resize β¦ ink map") β the docstring
states train/infer share this shard, which is why zero-shot works at all and
why fine-tuning must reimplement crop_line + pad_batch bit-exactly.
2. What handles Devanagari? [V]
Only rec.onnx reads Devanagari β and its vocabulary already includes
every assigned Devanagari codepoint.
vocab.py:170Vocab.multi()β 1,677 tokens + blank class 0 (:3, :207), head = 1,678 classes. Themeta.jsonvocab string is exactly this vocab (tools/indra_eval/inspect_vocab.py).- Coverage: 128/128 assigned codepoints in U+0900βU+097F β 80 Lo letters, 11 Mc vowel signs, 23 Mn marks incl. ΰ€Ό (U+093C nukta) and the conjunct virama ΰ€ΰ₯, plus ΰ₯ (U+0950), ΰ€ ΰ₯€ (U+0964), ΰ₯₯ (U+0965), digits ΰ₯¦βΰ₯― (U+0966β096F), βΉ (U+20B9). Bengali block also present (96 codepoints).
- The recognizer is routed by two fixed guards, not the network:
script.onnx(hanzi binary classifier, sigmoid scalar;pipeline.py:274β289) sends hanzi lines torec_cjk.onnxβ irrelevant for Nepali pages (all-Devanagari pages never trigger it; the classifier is line-level and trained on hanzi).vocab.canonical(:42) NFC-normalizes;Vocab.encode(:212) drops out-of-vocab codepoints β fine-tuning data must be pre-normalized the same way or the loss silently degrades (the trainer's data pipeline is closed; safest is to clonecanonical/encode).
- Confirmed counter-evidence collected in the smoke run: ΰ€Ό (nukta) is often dropped contextually at 32 px height (ΰ€°ΰ€Ό and ΰ€° differ only by a 1β2 px dot); βΉ was read as ΰ€°. Both are training-distribution issues, not capability gaps β exactly the kind of thing fine-tuning would fix (Experiment B).
3. Component map: trainable / fixed-but-replaceable / fixed [V]
| Component | Where weights come from | Trainable via shipped artifacts? | Replaceability |
|---|---|---|---|
rec.onnx recognizer |
model.safetensors (108 tensors, all rec/) |
YES β the only shipped state dict; init for a torch CTC retrain (Experiment B) | head/vocab identical β drop-in for +1,678 classes |
det.onnx line detector |
ONNX only | NO state dict shipped | could retrain in torch (targets spec exists: det_post.encode_targets, det_post.py:48 β core logit + top/bottom distances per line, CORE_FRAC=0.4, CORE_X_SHRINK=0.15) β but training code is closed β high effort |
order.onnx reading order/layout |
ONNX only | NO | same: order_post.label_order/decode_* reveal supervision (10 block classes) but no weights |
orient.onnx page rotation |
ONNX only | NO | trivial to replace with a torch/tlclassifier or heuristic |
mask.onnx CenterMask |
NOT SHIPPED in v3.1.1 (pipeline.py:201β205 optional) |
β | absent here; crop_lines(gray, boxes, 32, None) runs w/o it (verified) |
script.onnx + rec_cjk.onnx |
ONNX only | NO | optional; would be excluded from a Devanagari build |
math*.onnx (4 files) |
NOT shipped unless math=True (models.py:22β44) |
β | out of scope |
| geometry/deskew/layout postproc | pure Python (pipeline.py:37β149, layout_post.py) |
fixed code, no training needed | replaceable with own code paths |
| text-layer (PDF) naive path | textlayer.py |
fixed code | irrelevant to scans/Nepali notes |
The decisive asymmetry: Google ships a recognizer-only training starting
point. Fine-tuning anything else means either (a) training from scratch (det
targets spec'd, but no reference weights/recipe) or (b) leaving det/order
frozen ONNX and only exercising the rec head β that's Experiment B; both are
documented in INDRA_EXPERIMENTS.md.
4. Integration architecture (how a page becomes text) [V]
pipeline.py:456β470 (OrtOCR.__call__):
upright(): optionalorient.onnxturn aboveORIENT_MIN = 0.95(:54) + skew estimate/deskew up to 5Β° (:62β63).detect():det.onnxon a page resized long-side to detector input, threshold β line boxes (usesdet_post.page_to_input/decode;ADAPTIVE_MIN_H=9pxre-detect pass at a larger scale, :37β40).recognize()(:264β320): CenterMask if present (not shipped); per-linecrop_lineβpad_batchβ rec session, CTC collapse & conf gating (Β§1.2/Β§1.4).- line splitting at wide gaps (
SPLIT_GAP=3.0,CELL_GAP=1.8,GUTTER_GAP=1.6, :44β47) and gutter detection (:342). reading_order()onorder.onnx(feats [1,N,16], outputs pairwise saliencea, block classcls[10 classes], same-blocksame, table t-headstab) then layout (layout_post.py: page_blocks,page_markdown).
MIN_CONF=0.5 means a busy page with low-quality Devanagari can silently emit empty lines in Experiment A β compare-on-GT-boxes (rec-lines mode) sidesteps this entirely, which is why Experiment A does both full-pipeline and rec-lines scoring.
5. Licensing [V]
- Model weights (
models/indra-ocr-v3.1.1/LICENSE): AI PUBS OPEN RAIL-M (MODIFIED), v0.1, 2023-03-02. Section II ΒΆ36 states verbatim: "Use may include creating any content with, fine-tuning, updating, running, training, evaluating and/or re-parametrizing the Model" β fine-tuning is explicitly licensed. ΒΆ5 attaches Attachment A use-based restrictions (standard OpenRAIL misuse list β weapons/illegal/defamation; OCR and document processing is untouched) and requires passing the same restrictions to distributees; ΒΆ4 permits hosting/redistribution with notice. Derivative weights (fine-tuned checkpoints, distillations) fall under the same license. - Code (
indra/LICENSE): Apache-2.0 β reuse, modification, redistribution of the inference package are standard. - Practical consequence: our fine-tuned recognizer (Experiment B) is legally a
"Derivative of the Model" under OpenRAIL-M β internal deployment and
evaluation are unrestricted; redistribution requires keeping the license +
restriction notice. Full matrix incl. Nepali datasets:
Β§12ofINDRA_EXPERIMENTS.md(dataset side [B] β needsdocs/DATA_LICENSING.mdon the remote machine).
6. Closing the prior report's [U] items
| Prior unknown | Status | Evidence |
|---|---|---|
| exact recognizer arch (RNN? CNN?) | [V] + [X]: pure CNN+CTC, NO RNN/attention | Β§1.1, node dump |
| timestep stride | [X]: 4, not 8 | Β§1.3 (graph constants + inc.timings) |
| output contract (best/pmax docstrings vs solo head) | [V]: ids,conf,lengths names; truncation mandatory |
Β§1.4 |
safetensors β ONNX mapping (esp. depthwise onnx::Conv_369β¦397) |
[V]: 108 tensors = 108 initializers, 1,236,366 params both ways; dw weights are rec/onnx::Conv_369 [128,1,9] + β¦370 [128] bias pairs |
inspect_models.py |
| Devanagari coverage claim | [V]: 128/128 codepoints in vocab | inspect_vocab.py |
| What re-export must reproduce | [V]: mask + per-block re-zeroing + SE-valid-norm + decode tail; lengths = ceil(widths/4) clip W/4 |
Β§1.2, inspect_rec_mask.py |
| Confidence semantics of padded steps | [X]: head-on-zeros β blank conf 0.78; caller truncates | Β§1.2(3) |
| vocab[token i+1] β blank 0 | [V] | vocab.py:3,207,241β242 |
| rec input normalization | [V]: ink map only (0 paper, 1 ink) per crop_line, no mean/std |
rec_preproc.py:1β10 |
| enables fine-tune? | [V] yes for rec only (weights exist; no training code; CTC contract reconstructed) | Β§3, Β§B doc |
| full pipeline runs on Nepali test image | [V] smoke: synthetic shaped page, 10/10 lines, CER 2.6β3.5% | results/smoke/ |
| what differs in real (non-synth) Nepali | [B] frozen remote sets + baseline | Experiment A |
| trained det/order weights for fine-tune | [V] not shipped (ONNX only) | Β§3 |
7. Open items (not closed by this investigation)
- [B] Zero-shot numbers on the frozen Nepali eval sets (needs
khalti-t4). - [B] Head-to-head vs PP-OCRv5 Mobile Devanagari on the same crops β the remote machine holds the paddle recognizer + eval harness. The local smoke CER of 2.6β3.5% is on a synthetic page with a 32 px font and is not a benchmark.
- [B] Licensing matrix final values need
docs/DATA_LICENSING.mdanddocs/DEVANAGARI_RESOURCES_INVENTORY.md(remote). - FX-detector quality on Devanagari (qubee endpoints, aggressively slanted conjuncts) is unknowable without the det model being exercised on real scans β that's part of Experiment A full-pipeline.
- The root training repo (referenced by name in the closed code) is not accessible; Archived recipe unknowns: batch sizes, LR schedule, data mix.
8. Reproduce everything
cd /mnt/c/Users/Dell/Desktop/OCR_INDRA # or wherever the repo is
PY=/tmp/opencode/indra-lib # pip --target deps
export PYTHONPATH="$PWD/indra:$PY"
python3 tools/indra_eval/inspect_vocab.py # vocab + Devanagari coverage
python3 tools/indra_eval/inspect_models.py # graph I/O + param count
python3 tools/indra_eval/inspect_rec_nodes.py # full node dump + smoke run
python3 tools/indra_eval/inspect_rec_mask.py # mask semantics + lengths table
python3 tools/indra_eval/make_synth_page.py results/smoke # synth page + GT
python3 tools/indra_eval/eval_indra.py predict results/smoke/synth_nepali_page.png \
--boxes results/smoke/synth_nepali_gt.jsonl --jsonl results/smoke/rec_only.jsonl
python3 tools/indra_eval/eval_indra.py predict results/smoke/synth_nepali_page.png \
--jsonl results/smoke/full_run.jsonl --print-text
python3 tools/indra_eval/eval_indra.py cer results/smoke/synth_nepali_record.jsonl \
results/smoke/rec_only.jsonl --detail results/smoke/cer_rec_only.json --rec-only
python3 tools/indra_eval/eval_indra.py cer results/smoke/synth_nepali_record.jsonl \
results/smoke/full_run.jsonl --detail results/smoke/cer_full.json
Observed (this machine, WSL, CPU, 8 cores, 3 GB RAM): rec-only CER 3.49% / WER 15.79%; full-pipeline CER 2.62% / WER 13.16%; 10/10 lines found; mean_conf 0.962β0.963. Timings on this box are meaningless (det β 18.6 min β machine-starved; bench on the T4).