Sync runtime size checks

README.md

@@ -106,7 +106,7 @@ and **47× smaller** than IndicXlit with its rescoring dictionary — while bein
 
 |  |  |
 |---|---|
-| 🪶 **Extremely small** | 0.57 MB core + 3.97 MB dictionary pack = 4.53 MB gzipped (8.68 MB on disk); 989K parameters, 4-bit weights. Fits in a keyboard app, a browser extension or a web page. |
+| 🪶 **Extremely small** | 0.57 MB core + 3.97 MB dictionary pack = 4.53 MB gzipped (8.69 MB on disk); 989K parameters, 4-bit weights. Fits in a keyboard app, a browser extension or a web page. |
 | 🎯 **Accurate** | With the dictionary pack: 70.37% top-1 / 87.66% top-3 on the Aksharantar Hindi test set (9,991 words); 87.55% word accuracy on Dakshina sentences. Core-only numbers are in [Variants](#variants-core-only-vs-core--dictionary). |
 | 🧠 **Context-aware** | A compact bigram language model picks the right word from its neighbours: the same *main* becomes मैं or में. |
 | 📥 **Starts fast, degrades gracefully** | The core is enough to start typing; the dictionary pack streams in after and is cached. If the pack cannot be fetched, the runtime keeps working with the core alone. |
@@ -209,8 +209,8 @@ Inference runs on your users’ devices, so there are no per-request API fees an
 
 <table>
 <tr><td align="center" valign="top" width="33%"><h2>70.4%</h2><b>top-1 — higher than IndicXlit (model only) (59.2%)</b><br/><sub>at 5× smaller · Aksharantar Hindi test</sub></td><td align="center" valign="top" width="33%"><h2>70.4%</h2><b>top-1 — higher than IndicXlit (full package) (69.3%)</b><br/><sub>at 47× smaller · Aksharantar Hindi test</sub></td><td align="center" valign="top" width="33%"><h2>8.1 pts/MB</h2><b>highest top-1 accuracy per MB among compared models</b><br/><sub>next: IndicXlit (model only) at 1.28 pts/MB</sub></td></tr>
-<tr><td align="center" valign="top" width="33%"><h2>87.7%</h2><b>right word in the first 3 suggestions — vs 59.2% top-1 for IndicXlit (model only)</b><br/><sub>Aksharantar Hindi test</sub></td><td align="center" valign="top" width="33%"><h2>87.7%</h2><b>right word in the first 3 suggestions — vs 69.3% top-1 for IndicXlit (full package)</b><br/><sub>Aksharantar Hindi test</sub></td><td align="center" valign="top" width="33%"><h2>5×</h2><b>smaller than IndicXlit (model only)</b><br/><sub>8.68 MB vs 46.0 MB on disk (fp32 weights)</sub></td></tr>
-<tr><td align="center" valign="top" width="33%"><h2>47×</h2><b>smaller than IndicXlit (full package)</b><br/><sub>8.68 MB vs 404 MB on disk (fp32 weights + 358 MB rescoring dictionary)</sub></td><td align="center" valign="top" width="33%"><h2>219.7 ms</h2><b>per word (p50) in the browser</b><br/><sub>an 8-vCPU Intel Xeon cloud VM (headless Chrome 156, Linux)</sub></td><td align="center" valign="top" width="33%"><h2>4.53 MB</h2><b>complete download</b><br/><sub>core + 1.5M-word dictionary pack · 989K parameters</sub></td></tr>
+<tr><td align="center" valign="top" width="33%"><h2>87.7%</h2><b>right word in the first 3 suggestions — vs 59.2% top-1 for IndicXlit (model only)</b><br/><sub>Aksharantar Hindi test</sub></td><td align="center" valign="top" width="33%"><h2>87.7%</h2><b>right word in the first 3 suggestions — vs 69.3% top-1 for IndicXlit (full package)</b><br/><sub>Aksharantar Hindi test</sub></td><td align="center" valign="top" width="33%"><h2>5×</h2><b>smaller than IndicXlit (model only)</b><br/><sub>8.69 MB vs 46.0 MB on disk (fp32 weights)</sub></td></tr>
+<tr><td align="center" valign="top" width="33%"><h2>47×</h2><b>smaller than IndicXlit (full package)</b><br/><sub>8.69 MB vs 404 MB on disk (fp32 weights + 358 MB rescoring dictionary)</sub></td><td align="center" valign="top" width="33%"><h2>219.7 ms</h2><b>per word (p50) in the browser</b><br/><sub>an 8-vCPU Intel Xeon cloud VM (headless Chrome 156, Linux)</sub></td><td align="center" valign="top" width="33%"><h2>4.53 MB</h2><b>complete download</b><br/><sub>core + 1.5M-word dictionary pack · 989K parameters</sub></td></tr>
 <tr><td align="center" valign="top" width="33%"><h2>0.57 MB</h2><b>to start typing</b><br/><sub>the core works on its own (53.4% top-1 without the pack); the dictionary pack streams in after and is cached</sub></td></tr>
 </table>
 
@@ -218,7 +218,7 @@ Inference runs on your users’ devices, so there are no per-request API fees an
 
 | # | System | On disk | Top-1 % | Top-1 pts per MB |
 |---|---|---|---|---|
-| 1 | **AksharaLP-1M** | 8.68 MB | 70.37 | **8.10** |
+| 1 | **AksharaLP-1M** | 8.69 MB | 70.37 | **8.10** |
 | 2 | IndicXlit (model only) | 46.0 MB | 59.16 | 1.28 |
 | 3 | IndicXlit (full package) | 404 MB | 69.33 | 0.17 |
 
@@ -288,13 +288,13 @@ What you download:
 |---|---|---|---|
 | `xlit.q4` | core: transliteration Transformer, 4-bit weights | 589 KB | 544 KB |
 | `params.json` | core: reranker weights + decoding settings | 12 KB | 5 KB |
-| `aksharalp.mjs` | core: JavaScript runtime (zero dependencies) | 52 KB | 17 KB |
+| `aksharalp.mjs` | core: JavaScript runtime (zero dependencies) | 52 KB | 18 KB |
 | **Core total** | works on its own | **0.65 MB** | **0.57 MB** |
 | `dict/lex.bin` | dictionary pack: Hindi lexicon (trie + unigram scores) | 5,647 KB | 3,137 KB |
 | `dict/lm.bin` | dictionary pack: word-bigram language model, 4-bit | 2,373 KB | 826 KB |
 | `dict/params.json` | dictionary pack: reranker weights + decoding settings used with the pack | 12 KB | 5 KB |
 | **Dictionary pack total** | fetched on first use, then cached | **8.03 MB** | **3.97 MB** |
-| **Complete package** | core + pack | **8.68 MB** | **4.53 MB** |
+| **Complete package** | core + pack | **8.69 MB** | **4.53 MB** |
 
 For comparison, IndicXlit’s fp32 model weights are 46.0 MB and its official rescoring dictionary adds another 358 MB.
 General-purpose LLMs that can transliterate are larger still, with hundreds of millions of parameters or more, and typically need a server or a GPU.
@@ -319,8 +319,8 @@ smaller dictionary were chosen on the validation set only. The 200K-word pack is
 | Variant | Shipped | Download (gz) | On disk | Test top-1 % <sub>[95% CI]</sub> | top-3 % | top-8 % | Val top-1 % <sub>(2-fold CV)</sub> | Test top-1 % with test words removed from the dictionary | Dakshina sentences word acc % |
 |---|---|---|---|---|---|---|---|---|---|
 | **Core only** | yes | 0.57 MB | 0.65 MB | 53.36 <sub>[52.36, 54.32]</sub> | 74.52 | 83.81 | 53.92 | — | 65.35 |
-| Core + 200K-word dictionary | no (measured only) | 1.13 MB | 1.75 MB | 68.38 <sub>[67.48, 69.30]</sub> | 85.70 | 91.58 | 67.35 | 33.33 | 86.89 |
-| **Core + 1.5M-word dictionary pack** | yes | 4.53 MB | 8.68 MB | 70.37 <sub>[69.50, 71.26]</sub> | 87.66 | 92.79 | 69.30 | 19.65 | 87.55 |
+| Core + 200K-word dictionary | no (measured only) | 1.13 MB | 1.76 MB | 68.38 <sub>[67.48, 69.30]</sub> | 85.70 | 91.58 | 67.35 | 33.33 | 86.89 |
+| **Core + 1.5M-word dictionary pack** | yes | 4.53 MB | 8.69 MB | 70.37 <sub>[69.50, 71.26]</sub> | 87.66 | 92.79 | 69.30 | 19.65 | 87.55 |
 
 <sub>The “test words removed” column re-runs the same system with every Aksharantar test reference word deleted from the dictionary, so the dictionary cannot
 simply look up the answer; it shows how much of the gain comes from dictionary coverage (see the caveat below). Dakshina sentences: 4,976 romanized sentences.</sub>

aksharalp.min.mjs

@@ -16,7 +16,7 @@
  * Copyright 2026 Sourojit Dhua. Licensed under the Apache License, Version 2.0; commercial use permitted.
  * Attribution required (Apache-2.0 §4(d)): keep this notice and the NOTICE file in redistributions and derivatives:
  *   "Transliteration by AksharaLP (Sourojit Dhua) — https://huggingface.co/sourojitdotcom/AksharaLP-1M"
- */const zt=0,Rt=1,Tt=2,Dt=3,Ot=32,Ut=132,qt=-1/0,ct=b=>{const s=b.toLowerCase().slice(0,31),t=[];for(const e of s){const n=e.charCodeAt(0);t.push(e.length===1&&n>=97&&n<=122?n-97+4:3)}return t.push(2),t},lt=b=>{const s=b.codePointAt(0);return s>=2304&&s<2432?s-2304+4:-1},ht=b=>{const s=[];for(const t of[...b].slice(0,30)){const e=lt(t);s.push(e<0?3:e)}return s.push(2),s},ut=b=>{let s="";for(const t of b)t>3&&(s+=String.fromCharCode(2304+t-4));return s};function H(b){const s=b&32768?-1:1,t=b>>10&31,e=b&1023;return t===0?s*e*5960464477539063e-23:t===31?e?NaN:s*(1/0):s*(1+e/1024)*Math.pow(2,t-15)}function ft(b){const s=new Uint8Array(b),e=new DataView(s.buffer,s.byteOffset,s.byteLength).getUint32(0,!0),n=JSON.parse(new TextDecoder().decode(s.subarray(4,4+e))),o=s.subarray(4+e);return{meta:n,data:o,dv:new DataView(o.buffer,o.byteOffset,o.byteLength)}}const dt=b=>b.reduce((s,t)=>s*t,1),j=(b,s,t)=>{const e=new Float32Array(t);for(let n=0;n<t;n++)e[n]=H(b.getUint16(s+2*n,!0));return e};function X(b){const{meta:s,data:t,dv:e}=ft(b),n={};for(const o of s.tensors){const[c,a,r,l]=o,f=dt(a);let i;if(r==="f16")i=j(e,l,f);else if(r==="i8"){const u=a[0],d=f/u,p=j(e,l+f,u);i=new Float32Array(f);for(let m=0;m<u;m++)for(let _=0;_<d;_++){const h=t[l+m*d+_]<<24>>24;i[m*d+_]=h*p[m]}}else{const u=r.length>1?parseInt(r.slice(1)):s.bits,d=s.G,p=o[4],[m,_]=a,h=Math.ceil(_/d),w=j(e,l+p,m*h);i=new Float32Array(f);for(let g=0;g<f;g++){let y;if(u===4){const x=t[l+(g>>1)];y=(g&1?x&15:x>>4)-8}else y=t[l+g]<<24>>24;const A=g/_|0,k=g-A*_;i[g]=y*w[A*h+(k/d|0)]}}n[c]={shape:a,data:i}}return{cfg:s.cfg,extra:s.extra||{},T:n}}function F(b,s,t,e,n,o,c,a){for(let r=0;r<a;r++){let l=e?e[r]:0;const f=r*c;for(let i=0;i<c;i++)l+=t[f+i]*b[s+i];n[o+r]=l}}function D(b,s,t,e,n,o,c){let a=0;for(let f=0;f<c;f++)a+=b[s+f];a/=c;let r=0;for(let f=0;f<c;f++){const i=b[s+f]-a;r+=i*i}r/=c;const l=1/Math.sqrt(r+1e-5);for(let f=0;f<c;f++)n[o+f]=(b[s+f]-a)*l*t[f]+e[f]}function pt(b){const s=Math.abs(b),t=1/(1+.5*s),e=t*Math.exp(-s*s-1.26551223+t*(1.00002368+t*(.37409196+t*(.09678418+t*(-.18628806+t*(.27886807+t*(-1.13520398+t*(1.48851587+t*(-.82215223+t*.17087277)))))))));return b>=0?1-e:e-1}const wt=b=>.5*b*(1+pt(b/Math.SQRT2)),Z=b=>1/(1+Math.exp(-b));class gt{constructor(s){const t=s.cfg;this.d=t.d,this.h=t.h,this.dk=t.d/t.h,this.ff=t.ff,this.ne=t.enc,this.nd=t.dec;const e=r=>{const l=s.T[r];if(!l)throw new Error("missing tensor "+r);return l.data};this.semb=e("semb.weight"),this.temb=e("temb.weight"),this.spos=e("spos.weight"),this.tpos=e("tpos.weight");const n=r=>({q:e(r+".q.weight"),qb:e(r+".q.bias"),kv:e(r+".kv.weight"),kvb:e(r+".kv.bias"),o:e(r+".o.weight"),ob:e(r+".o.bias")}),o=r=>({g:e(r+".weight"),b:e(r+".bias")}),c=r=>({w1:e(r+".ff.0.weight"),b1:e(r+".ff.0.bias"),w2:e(r+".ff.2.weight"),b2:e(r+".ff.2.bias")});this.enc=[];for(let r=0;r<t.enc;r++){const l="enc."+r;this.enc.push({n1:o(l+".n1"),att:n(l+".att"),n2:o(l+".n2"),ff:c(l)})}this.dec=[];for(let r=0;r<t.dec;r++){const l="dec."+r;this.dec.push({n1:o(l+".n1"),sa:n(l+".sa"),n2:o(l+".n2"),ca:n(l+".ca"),n3:o(l+".n3"),ff:c(l)})}this.en=o("en"),this.dn=o("dn"),this.scale=Math.sqrt(t.d),this.nParams=Object.values(s.T).reduce((r,l)=>r+l.data.length,0);const a=this.d;this._y=new Float32Array(a),this._a=new Float32Array(a),this._f=new Float32Array(this.ff),this._q=new Float32Array(a),this._kv=new Float32Array(2*a),this._sc=new Float64Array(32)}_ffn(s,t,e,n){const o=this.d,c=this._y,a=this._f,r=this._a;D(t,e,n.g,n.b,c,0,o),F(c,0,s.w1,s.b1,a,0,o,this.ff);for(let l=0;l<this.ff;l++)a[l]=wt(a[l]);F(a,0,s.w2,s.b2,r,0,this.ff,o);for(let l=0;l<o;l++)t[e+l]+=r[l]}_attend(s,t,e,n,o){const{h:c,dk:a,d:r}=this,l=n<=32?this._sc:new Float64Array(n),f=1/Math.sqrt(a);for(let i=0;i<c;i++){let u=-1/0;for(let p=0;p<n;p++){let m=0;const _=e+p*2*r+i*a;for(let h=0;h<a;h++)m+=s[i*a+h]*t[_+h];m*=f,l[p]=m,m>u&&(u=m)}let d=0;for(let p=0;p<n;p++)l[p]=Math.exp(l[p]-u),d+=l[p];for(let p=0;p<a;p++){let m=0;for(let _=0;_<n;_++)m+=l[_]*t[e+_*2*r+r+i*a+p];o[i*a+p]=m/d}}}encode(s){const t=this.d,e=s.length,n=new Float32Array(e*t),o=new Float32Array(e*t),c=new Float32Array(e*2*t),a=new Float32Array(t),r=this._a;for(let i=0;i<e;i++)for(let u=0;u<t;u++)n[i*t+u]=this.semb[s[i]*t+u]*this.scale+this.spos[i*t+u];for(const i of this.enc){for(let d=0;d<e;d++)D(n,d*t,i.n1.g,i.n1.b,o,d*t,t),F(o,d*t,i.att.kv,i.att.kvb,c,d*2*t,t,2*t);const u=new Float32Array(n);for(let d=0;d<e;d++){F(o,d*t,i.att.q,i.att.qb,this._q,0,t,t),this._attend(this._q,c,0,e,a),F(a,0,i.att.o,i.att.ob,r,0,t,t);for(let p=0;p<t;p++)u[d*t+p]+=r[p]}for(let d=0;d<e;d++)this._ffn(i.ff,u,d*t,i.n2);n.set(u)}const l=new Float32Array(e*t);for(let i=0;i<e;i++)D(n,i*t,this.en.g,this.en.b,l,i*t,t);const f=this.dec.map(i=>{const u=new Float32Array(e*2*t);for(let d=0;d<e;d++)F(l,d*t,i.ca.kv,i.ca.kvb,u,d*2*t,t,2*t);return u});return{S:e,cross:f}}newCache(s){const t=this.d;return this.dec.map(()=>new Float32Array(s*32*2*t))}step(s,t,e,n,o,c){const a=this.d,r=new Float32Array(a),l=this._y,f=new Float32Array(a),i=this._a;for(const u of e){for(let h=0;h<a;h++)r[h]=this.temb[n[u]*a+h]*this.scale+this.tpos[o*a+h];for(let h=0;h<this.nd;h++){const w=this.dec[h],g=t[h],y=u*32*2*a;D(r,0,w.n1.g,w.n1.b,l,0,a),F(l,0,w.sa.kv,w.sa.kvb,g,y+o*2*a,a,2*a),F(l,0,w.sa.q,w.sa.qb,this._q,0,a,a),this._attend(this._q,g,y,o+1,f),F(f,0,w.sa.o,w.sa.ob,i,0,a,a);for(let A=0;A<a;A++)r[A]+=i[A];D(r,0,w.n2.g,w.n2.b,l,0,a),F(l,0,w.ca.q,w.ca.qb,this._q,0,a,a),this._attend(this._q,s.cross[h],0,s.S,f),F(f,0,w.ca.o,w.ca.ob,i,0,a,a);for(let A=0;A<a;A++)r[A]+=i[A];this._ffn(w.ff,r,0,w.n3)}D(r,0,this.dn.g,this.dn.b,l,0,a);const d=c[u];let p=-1/0;for(let h=0;h<132;h++){let w=0;for(let g=0;g<a;g++)w+=l[g]*this.temb[h*a+g];d[h]=w,w>p&&(p=w)}let m=0;for(let h=0;h<132;h++)m+=Math.exp(d[h]-p);const _=p+Math.log(m);for(let h=0;h<132;h++)d[h]-=_}}beam(s,t,{beam:e=8,nbest:n=8,trie:o=null}={}){const c=this.d,a=this.newCache(e),r=Array.from({length:e},()=>new Float64Array(132));let l=new Array(e).fill(-1/0);l[0]=0;let f=new Array(e).fill(1),i=Array.from({length:e},()=>[]),u=o?new Array(e).fill(0):null;const d=[],p=Math.min(31,t+8),m=new Float32Array(64*c);for(let h=0;h<p;h++){const w=[];for(let v=0;v<e;v++)l[v]>-1/0&&w.push(v);this.step(s,a,w,f,h,r);const g=[];for(const v of w){const E=r[v];let R=null;if(o){const M=u[v];if(M===null)continue;R=o.allowed(M)}if(R)for(const M of R)M!==0&&M!==1&&M!==3&&g.push([l[v]+E[M],v,M]);else{g.push([l[v]+E[2],v,2]);for(let M=4;M<132;M++)g.push([l[v]+E[M],v,M])}}g.sort((v,E)=>E[0]-v[0]||v[1]*132+v[2]-(E[1]*132+E[2]));const y=g.slice(0,2*e),A=new Array(e).fill(-1/0),k=new Array(e).fill(2),x=new Array(e).fill(0),N=o?new Array(e).fill(null):null;let S=0;for(const[v,E,R]of y)if(v>-1/0){if(R===2){d.length<n&&d.push([ut(i[E]),v]);continue}S<e&&(A[S]=v,k[S]=R,x[S]=E,o&&(N[S]=o.child(u[E],R)),S++)}let z=!0;if(d.length>=n&&(S===0||A[0]<Math.min(...d.map(v=>v[1])))?z=!1:S||(z=!1),!z)break;for(const v of a){const E=(h+1)*2*c,R=[];for(let M=0;M<S;M++){const W=x[M]*32*2*c;R.push(v.slice(W,W+E))}for(let M=0;M<S;M++)v.set(R[M],M*32*2*c)}i=Array.from({length:e},(v,E)=>E<S?i[x[E]].concat([k[E]]):[]),l=A,f=k,u=N}const _=new Map;for(const[h,w]of d)(!_.has(h)||w>_.get(h))&&_.set(h,w);return[..._.entries()].sort((h,w)=>w[1]-h[1]).slice(0,n)}scorePair(s,t){const e=ht(t),n=this.newCache(1),o=[new Float64Array(132)];let c=[1],a=0;for(let r=0;r<e.length;r++)this.step(s,n,[0],c,r,o),a+=o[0][e[r]],c=[e[r]];return a}}function C(b,s,t){const e=new Uint32Array(s);for(let n=0;n<s;n++){let o=0,c=0,a;do a=b[t++],o+=(a&127)*2**c,c+=7;while(a>=128);e[n]=o}return[e,t]}const V=(b,s)=>{const t=new Uint8Array(s);for(let e=0;e<s;e++)t[e]=e&1?b[e>>1]&15:b[e>>1]>>4;return t};class mt{constructor(s,t){if(!s){this.words=[],this.sortedWords=[],this.uni=new Float32Array(0),this.N=0,this.unk=0,this.alpha=!0,this.hasLM=!1;return}const e=new Uint8Array(s),n=new DataView(e.buffer,e.byteOffset,e.byteLength),o=[],c=(r,l,f)=>{let i=[],u=0,d=0;for(;o.length<f;){const p=r?r[d++]:l[u++],m=i.slice(0,p);for(;;){const _=l[u++],h=_&127;if(h!==127&&m.push(h),_&128)break}o.push(String.fromCharCode(...m.map(_=>2304+_))),i=m}};if(e[0]===76&&e[1]===88&&e[2]===52&&e[3]===0){const r=n.getUint32(4,!0),l=n.getUint32(8,!0),f=n.getUint32(16,!0);this.unk=n.getFloat32(12,!0);let i=20;const u=new Float32Array(f);for(let p=0;p<f;p++)u[p]=n.getFloat32(i+4*p,!0);i+=4*f;const d=V(e.subarray(i,i+(r+1>>1)),r);i+=r+1>>1,this.uni=new Float32Array(r);for(let p=0;p<r;p++)this.uni[p]=u[d[p]];c(e.subarray(i,i+r),e.subarray(i+r,i+r+l),r),this.N=r,this.alpha=!0}else{const r=n.getUint32(0,!0),l=n.getUint32(4,!0),f=n.getUint32(8,!0);this.unk=n.getFloat32(12,!0);let[i,u]=C(e,l,16);const d=[];for(let m=0;m<l;m++)d.push(H(n.getUint16(u+2*m,!0)));u+=2*l,this.uni=new Float32Array(r);let p=0;for(let m=0;m<l;m++)for(let _=0;_<i[m];_++)this.uni[p++]=d[m];c(null,e.subarray(u,u+f),r),this.N=r,this.alpha=!1}this.words=o;let a=this.alpha;for(let r=1;a&&r<o.length;r++)o[r-1]<o[r]||(a=!1);a||(this.map=new Map,o.forEach((r,l)=>this.map.set(r,l))),this.sortedWords=a?o:[...new Set(o)].sort(),this.hasLM=!1,t&&this._loadLM(new Uint8Array(t))}id(s){if(this.map){const n=this.map.get(s);return n===void 0?null:n}let t=0,e=this.words.length;for(;t<e;){const n=t+e>>1;this.words[n]<s?t=n+1:e=n}return t<this.words.length&&this.words[t]===s?t:null}_loadLM(s){const t=new DataView(s.buffer,s.byteOffset,s.byteLength),e=this.N,n=t.getUint32(4,!0),o=t.getUint32(16,!0);let c=20;const a=[],r=[];for(let m=0;m<16;m++)a.push(t.getFloat32(c+4*m,!0));c+=64;for(let m=0;m<16;m++)r.push(t.getFloat32(c+4*m,!0));c+=64;let l,f;[l,c]=C(s,e+1,c),[f,c]=C(s,n,c);const i=V(s.subarray(c,c+(n+1>>1)),n);c+=n+1>>1;const u=V(s.subarray(c,c+(o+1>>1)),o);this.off=new Uint32Array(e+2),this.succ=new Uint32Array(n),this.bq=i,this.cb=Float32Array.from(a),this.bow=new Float32Array(e+1);let d=0,p=0;for(let m=0;m<=e;m++){this.off[m]=d;const _=l[m];if(!_)continue;this.bow[m]=r[u[p++]];let h=-1;for(let w=0;w<_;w++)h=h<0?f[d]:h+f[d]+1,this.succ[d++]=h}this.off[e+1]=d,this.hasLM=n>0}unigram(s){return s===null?this.unk:s===this.N?Math.log(.04):this.uni[s]}logp(s,t){const e=s===null?this.N:s;if(t!==null&&this.hasLM){let n=this.off[e],o=this.off[e+1];for(;n<o;){const c=n+o>>1;this.succ[c]<t?n=c+1:o=c}if(n<this.off[e+1]&&this.succ[n]===t)return this.cb[this.bq[n]]}return(this.hasLM?this.bow[e]:0)+this.unigram(t)}}class bt{constructor(s){this.w=s,this.nodes=[[0,s.length,0]],this.ids=new Map([["0,0,"+s.length,0]]),this.cache=new Map,this.allow=new Map}_bisect(s,t,e){const n=this.w;for(;t<e;){const o=t+e>>1;n[o]<s?t=o+1:e=o}return t}children(s){let t=this.cache.get(s);if(t)return t;const[e,n,o]=this.nodes[s],c=this.w;t=new Map;let a=e;for(a<n&&c[a].length===o&&a++;a<n;){const r=c[a].charCodeAt(o),l=c[a].slice(0,o),f=this._bisect(l+String.fromCharCode(r+1),a,n),i=a+","+(o+1)+","+f;let u=this.ids.get(i);u===void 0&&(u=this.nodes.length,this.nodes.push([a,f,o+1]),this.ids.set(i,u)),t.set(r-2304+4,u),a=f}return this.cache.size>4e5&&(this.cache.clear(),this.allow.clear()),this.cache.set(s,t),t}final(s){const[t,,e]=this.nodes[s];return t<this.w.length&&this.w[t].length===e}allowed(s){let t=this.allow.get(s);return t||(t=[...this.children(s).keys()],this.final(s)&&t.push(2),this.allow.set(s,t)),t}child(s,t){return this.children(s).get(t)}}const G={};[..."abcdefghijklmnopqrstuvwxyz'"].forEach((b,s)=>G[b]=s+3);const L=16,yt=["english","romanized_hindi","hinglish"],T=/^[A-Za-z]+(?:'[A-Za-z]+)?$/,O=/^[\u0900-\u0963\u0971-\u097F]+$/;function _t(b,s){let t=2166136261;for(const e of new TextEncoder().encode(b.toLowerCase()))t=Math.imul(t^e,16777619)>>>0;return t%s}class At{constructor(s){this.cfg=s.cfg,this.x=s.extra||{};const t=e=>s.T[e]&&s.T[e].data;Object.assign(this,{ce:t("ce.weight"),c1:t("c1.weight"),c1b:t("c1.bias"),c2:t("c2.weight"),c2b:t("c2.bias"),wemb:t("wemb.weight"),wih:t("gru.weight_ih_l0"),whh:t("gru.weight_hh_l0"),bih:t("gru.bias_ih_l0"),bhh:t("gru.bias_hh_l0"),wihR:t("gru.weight_ih_l0_reverse"),whhR:t("gru.weight_hh_l0_reverse"),bihR:t("gru.bias_ih_l0_reverse"),bhhR:t("gru.bias_hh_l0_reverse"),tw:t("tok.weight"),tb:t("tok.bias"),s0:t("sent.0.weight"),s0b:t("sent.0.bias"),s2:t("sent.2.weight"),s2b:t("sent.2.bias")}),this.nParams=Object.values(s.T).reduce((e,n)=>e+n.data.length,0)}_wvec(s){const{ce:t,cf:e}=this.cfg,n=new Array(L).fill(0),o=[...s.toLowerCase()].slice(0,L-2);n[0]=1,o.forEach((d,p)=>n[p+1]=G[d]??2),n[o.length+1]=2;const c=new Float32Array(t*L);for(let d=0;d<L;d++)for(let p=0;p<t;p++)c[p*L+d]=this.ce[n[d]*t+p];const a=(d,p,m,_)=>{const h=new Float32Array(e*L);for(let w=0;w<e;w++)for(let g=0;g<L;g++){let y=_[w];for(let A=0;A<p;A++)for(let k=0;k<3;k++){const x=g+k-1;x>=0&&x<L&&(y+=m[(w*p+A)*3+k]*d[A*L+x])}h[w*L+g]=y>0?y:0}return h},r=a(c,t,this.c1,this.c1b),l=a(r,e,this.c2,this.c2b),f=this.cfg.wh,i=this.cfg.wd,u=new Float32Array(e+(f?i:0));for(let d=0;d<e;d++){let p=-1/0;for(let m=0;m<L;m++){const _=n[m]===0?-1e4:l[d*L+m]+r[d*L+m];_>p&&(p=_)}u[d]=p}if(f){const d=_t(s,f);for(let p=0;p<i;p++)u[e+p]=this.wemb[d*i+p]}return u}_gru(s,t,e,n,o,c){const a=this.cfg.hid,r=s[0].length,l=s.length,f=new Array(l);let i=new Float32Array(a);const u=new Float32Array(3*a),d=new Float32Array(3*a);for(let p=0;p<l;p++){const m=c?l-1-p:p;F(s[m],0,t,n,u,0,r,3*a),F(i,0,e,o,d,0,a,3*a);const _=new Float32Array(a);for(let h=0;h<a;h++){const w=Z(u[h]+d[h]),g=Z(u[a+h]+d[a+h]),y=Math.tanh(u[2*a+h]+w*d[2*a+h]);_[h]=(1-g)*y+g*i[h]}i=_,f[m]=i}return f}run(s){if(!s.length)return{pen:[],ps:null};const t=this.cfg.hid,e=s.map(g=>this._wvec(g)),n=e.length,o=e[0].length,c=this._gru(e,this.wih,this.whh,this.bih,this.bhh,!1),a=this._gru(e,this.wihR,this.whhR,this.bihR,this.bhhR,!0),r=[],l=new Float32Array(2*t),f=new Float32Array(2*t).fill(-1e4),i=new Float32Array(2*t+o),u=new Float32Array(2);for(let g=0;g<n;g++){for(let x=0;x<t;x++)i[x]=c[g][x],i[t+x]=a[g][x];i.set(e[g],2*t),F(i,0,this.tw,this.tb,u,0,2*t+o,2);const y=Math.max(u[0],u[1]),A=Math.exp(u[0]-y),k=Math.exp(u[1]-y);r.push(A/(A+k));for(let x=0;x<2*t;x++)l[x]+=i[x]/n,i[x]>f[x]&&(f[x]=i[x])}const d=new Float32Array(4*t+1);d.set(l,0),d.set(f,2*t),d[4*t]=r.reduce((g,y)=>g+y,0)/n;const p=new Float32Array(32);F(d,0,this.s0,this.s0b,p,0,4*t+1,32);for(let g=0;g<32;g++)p[g]=Math.max(0,p[g]);const m=new Float32Array(3);F(p,0,this.s2,this.s2b,m,0,32,3);const _=Math.max(...m),h=[...m].map(g=>Math.exp(g-_)),w=h.reduce((g,y)=>g+y,0);return{pen:r,ps:h.map(g=>g/w)}}tag(s){const t=s.filter(c=>T.test(c)),{pen:e}=this.run(t);let n=0;const o=this.x.tok_th??.5;return s.map(c=>O.test(c)?"DEV":T.test(c)?e[n++]>=o?"EN":"HI":"OTHER")}mode(s){const t=s.filter(r=>O.test(r)).length,e=s.filter(r=>T.test(r)).length;if(t&&!e)return"devanagari";if(t&&e)return t/(t+e)>=(this.x.dev_frac??.5)?"devanagari":"mixed_script";if(!e)return"english";const{ps:n}=this.run(s.filter(r=>T.test(r))),o=this.x.mode_bias||[0,0,0];let c=0,a=-1/0;return n.forEach((r,l)=>{const f=Math.log(r+1e-9)+o[l];f>a&&(a=f,c=l)}),yt[c]}}const $=/(https?:\/\/\S+|www\.\S+|[\p{L}\p{N}_.+-]+@[\p{L}\p{N}_-]+\.[\p{L}\p{N}_.]+|[@#][\p{L}\p{N}_]+|[A-Za-z]+(?:'[A-Za-z]+)?|[\u0900-\u0963\u0971-\u097F]+|[0-9\u0966-\u096F]+(?:[.,:][0-9\u0966-\u096F]+)*|[\u0964\u0965]|[^\sA-Za-z0-9\u0900-\u097F])/gu,J=/^[A-Za-z]+$/,xt=/^[b-df-hj-np-tv-z]{2,7}$/,K=Object.fromEntries([..."abcdefghijklmnopqrstuvwxyz"].map((b,s)=>[b,"\u090F \u092C\u0940 \u0938\u0940 \u0921\u0940 \u0908 \u090F\u092B \u091C\u0940 \u090F\u091A \u0906\u0908 \u091C\u0947 \u0915\u0947 \u090F\u0932 \u090F\u092E \u090F\u0928 \u0913 \u092A\u0940 \u0915\u094D\u092F\u0942 \u0906\u0930 \u090F\u0938 \u091F\u0940 \u092F\u0942 \u0935\u0940 \u0921\u092C\u094D\u0932\u094D\u092F\u0942 \u090F\u0915\u094D\u0938 \u0935\u093E\u0908 \u091C\u0947\u0921".split(" ")[s]])),kt={...K,f:"\u090F\u092B\u093C",z:"\u091C\u093C\u0947\u0921"},vt=[["\u0949","\u094B"],["\u094B","\u0949"],["\u093E","\u0949"],["\u0949","\u093E"],["\u0903",""],["\u092B\u093C","\u092B"],["\u092B","\u092B\u093C"],["\u091C\u093C","\u091C"],["\u091C","\u091C\u093C"],["\u0901","\u0902"],["\u0902","\u0901"]],jt=["sc","scn","inlex","uni","free0","infree","incon","aw","visarga","nukta","chandra","acro","variant","frank","skel","skmatch","lev","sc_rel","scn_rel","uni_rel","lenr","visr","hfin","acro_nv"],Mt=new Set(`whatsapp google meet paytm amazon flipkart youtube instagram facebook gmail excel uber ola swiggy zomato
+ */const Rt=0,Tt=1,Dt=2,Ot=3,Ut=32,qt=132,jt=-1/0,lt=b=>{const s=b.toLowerCase().slice(0,31),t=[];for(const e of s){const n=e.charCodeAt(0);t.push(e.length===1&&n>=97&&n<=122?n-97+4:3)}return t.push(2),t},ht=b=>{const s=b.codePointAt(0);return s>=2304&&s<2432?s-2304+4:-1},ut=b=>{const s=[];for(const t of[...b].slice(0,30)){const e=ht(t);s.push(e<0?3:e)}return s.push(2),s},ft=b=>{let s="";for(const t of b)t>3&&(s+=String.fromCharCode(2304+t-4));return s};function Z(b){const s=b&32768?-1:1,t=b>>10&31,e=b&1023;return t===0?s*e*5960464477539063e-23:t===31?e?NaN:s*(1/0):s*(1+e/1024)*Math.pow(2,t-15)}function dt(b){const s=new Uint8Array(b),e=new DataView(s.buffer,s.byteOffset,s.byteLength).getUint32(0,!0),n=JSON.parse(new TextDecoder().decode(s.subarray(4,4+e))),i=s.subarray(4+e);return{meta:n,data:i,dv:new DataView(i.buffer,i.byteOffset,i.byteLength)}}const pt=b=>b.reduce((s,t)=>s*t,1),j=(b,s,t)=>{const e=new Float32Array(t);for(let n=0;n<t;n++)e[n]=Z(b.getUint16(s+2*n,!0));return e};function H(b){const{meta:s,data:t,dv:e}=dt(b),n={};for(const i of s.tensors){const[c,a,o,l]=i,d=pt(a);let r;if(o==="f16")r=j(e,l,d);else if(o==="i8"){const u=a[0],f=d/u,p=j(e,l+d,u);r=new Float32Array(d);for(let m=0;m<u;m++)for(let _=0;_<f;_++){const h=t[l+m*f+_]<<24>>24;r[m*f+_]=h*p[m]}}else{const u=o.length>1?parseInt(o.slice(1)):s.bits,f=s.G,p=i[4],[m,_]=a,h=Math.ceil(_/f),w=j(e,l+p,m*h);r=new Float32Array(d);for(let g=0;g<d;g++){let y;if(u===4){const x=t[l+(g>>1)];y=(g&1?x&15:x>>4)-8}else y=t[l+g]<<24>>24;const A=g/_|0,k=g-A*_;r[g]=y*w[A*h+(k/f|0)]}}n[c]={shape:a,data:r}}return{cfg:s.cfg,extra:s.extra||{},T:n}}function F(b,s,t,e,n,i,c,a){for(let o=0;o<a;o++){let l=e?e[o]:0;const d=o*c;for(let r=0;r<c;r++)l+=t[d+r]*b[s+r];n[i+o]=l}}function D(b,s,t,e,n,i,c){let a=0;for(let d=0;d<c;d++)a+=b[s+d];a/=c;let o=0;for(let d=0;d<c;d++){const r=b[s+d]-a;o+=r*r}o/=c;const l=1/Math.sqrt(o+1e-5);for(let d=0;d<c;d++)n[i+d]=(b[s+d]-a)*l*t[d]+e[d]}function wt(b){const s=Math.abs(b),t=1/(1+.5*s),e=t*Math.exp(-s*s-1.26551223+t*(1.00002368+t*(.37409196+t*(.09678418+t*(-.18628806+t*(.27886807+t*(-1.13520398+t*(1.48851587+t*(-.82215223+t*.17087277)))))))));return b>=0?1-e:e-1}const gt=b=>.5*b*(1+wt(b/Math.SQRT2)),X=b=>1/(1+Math.exp(-b));class mt{constructor(s){const t=s.cfg;this.d=t.d,this.h=t.h,this.dk=t.d/t.h,this.ff=t.ff,this.ne=t.enc,this.nd=t.dec;const e=o=>{const l=s.T[o];if(!l)throw new Error("missing tensor "+o);return l.data};this.semb=e("semb.weight"),this.temb=e("temb.weight"),this.spos=e("spos.weight"),this.tpos=e("tpos.weight");const n=o=>({q:e(o+".q.weight"),qb:e(o+".q.bias"),kv:e(o+".kv.weight"),kvb:e(o+".kv.bias"),o:e(o+".o.weight"),ob:e(o+".o.bias")}),i=o=>({g:e(o+".weight"),b:e(o+".bias")}),c=o=>({w1:e(o+".ff.0.weight"),b1:e(o+".ff.0.bias"),w2:e(o+".ff.2.weight"),b2:e(o+".ff.2.bias")});this.enc=[];for(let o=0;o<t.enc;o++){const l="enc."+o;this.enc.push({n1:i(l+".n1"),att:n(l+".att"),n2:i(l+".n2"),ff:c(l)})}this.dec=[];for(let o=0;o<t.dec;o++){const l="dec."+o;this.dec.push({n1:i(l+".n1"),sa:n(l+".sa"),n2:i(l+".n2"),ca:n(l+".ca"),n3:i(l+".n3"),ff:c(l)})}this.en=i("en"),this.dn=i("dn"),this.scale=Math.sqrt(t.d),this.nParams=Object.values(s.T).reduce((o,l)=>o+l.data.length,0);const a=this.d;this._y=new Float32Array(a),this._a=new Float32Array(a),this._f=new Float32Array(this.ff),this._q=new Float32Array(a),this._kv=new Float32Array(2*a),this._sc=new Float64Array(32)}_ffn(s,t,e,n){const i=this.d,c=this._y,a=this._f,o=this._a;D(t,e,n.g,n.b,c,0,i),F(c,0,s.w1,s.b1,a,0,i,this.ff);for(let l=0;l<this.ff;l++)a[l]=gt(a[l]);F(a,0,s.w2,s.b2,o,0,this.ff,i);for(let l=0;l<i;l++)t[e+l]+=o[l]}_attend(s,t,e,n,i){const{h:c,dk:a,d:o}=this,l=n<=32?this._sc:new Float64Array(n),d=1/Math.sqrt(a);for(let r=0;r<c;r++){let u=-1/0;for(let p=0;p<n;p++){let m=0;const _=e+p*2*o+r*a;for(let h=0;h<a;h++)m+=s[r*a+h]*t[_+h];m*=d,l[p]=m,m>u&&(u=m)}let f=0;for(let p=0;p<n;p++)l[p]=Math.exp(l[p]-u),f+=l[p];for(let p=0;p<a;p++){let m=0;for(let _=0;_<n;_++)m+=l[_]*t[e+_*2*o+o+r*a+p];i[r*a+p]=m/f}}}encode(s){const t=this.d,e=s.length,n=new Float32Array(e*t),i=new Float32Array(e*t),c=new Float32Array(e*2*t),a=new Float32Array(t),o=this._a;for(let r=0;r<e;r++)for(let u=0;u<t;u++)n[r*t+u]=this.semb[s[r]*t+u]*this.scale+this.spos[r*t+u];for(const r of this.enc){for(let f=0;f<e;f++)D(n,f*t,r.n1.g,r.n1.b,i,f*t,t),F(i,f*t,r.att.kv,r.att.kvb,c,f*2*t,t,2*t);const u=new Float32Array(n);for(let f=0;f<e;f++){F(i,f*t,r.att.q,r.att.qb,this._q,0,t,t),this._attend(this._q,c,0,e,a),F(a,0,r.att.o,r.att.ob,o,0,t,t);for(let p=0;p<t;p++)u[f*t+p]+=o[p]}for(let f=0;f<e;f++)this._ffn(r.ff,u,f*t,r.n2);n.set(u)}const l=new Float32Array(e*t);for(let r=0;r<e;r++)D(n,r*t,this.en.g,this.en.b,l,r*t,t);const d=this.dec.map(r=>{const u=new Float32Array(e*2*t);for(let f=0;f<e;f++)F(l,f*t,r.ca.kv,r.ca.kvb,u,f*2*t,t,2*t);return u});return{S:e,cross:d}}newCache(s){const t=this.d;return this.dec.map(()=>new Float32Array(s*32*2*t))}step(s,t,e,n,i,c){const a=this.d,o=new Float32Array(a),l=this._y,d=new Float32Array(a),r=this._a;for(const u of e){for(let h=0;h<a;h++)o[h]=this.temb[n[u]*a+h]*this.scale+this.tpos[i*a+h];for(let h=0;h<this.nd;h++){const w=this.dec[h],g=t[h],y=u*32*2*a;D(o,0,w.n1.g,w.n1.b,l,0,a),F(l,0,w.sa.kv,w.sa.kvb,g,y+i*2*a,a,2*a),F(l,0,w.sa.q,w.sa.qb,this._q,0,a,a),this._attend(this._q,g,y,i+1,d),F(d,0,w.sa.o,w.sa.ob,r,0,a,a);for(let A=0;A<a;A++)o[A]+=r[A];D(o,0,w.n2.g,w.n2.b,l,0,a),F(l,0,w.ca.q,w.ca.qb,this._q,0,a,a),this._attend(this._q,s.cross[h],0,s.S,d),F(d,0,w.ca.o,w.ca.ob,r,0,a,a);for(let A=0;A<a;A++)o[A]+=r[A];this._ffn(w.ff,o,0,w.n3)}D(o,0,this.dn.g,this.dn.b,l,0,a);const f=c[u];let p=-1/0;for(let h=0;h<132;h++){let w=0;for(let g=0;g<a;g++)w+=l[g]*this.temb[h*a+g];f[h]=w,w>p&&(p=w)}let m=0;for(let h=0;h<132;h++)m+=Math.exp(f[h]-p);const _=p+Math.log(m);for(let h=0;h<132;h++)f[h]-=_}}beam(s,t,{beam:e=8,nbest:n=8,trie:i=null}={}){const c=this.d,a=this.newCache(e),o=Array.from({length:e},()=>new Float64Array(132));let l=new Array(e).fill(-1/0);l[0]=0;let d=new Array(e).fill(1),r=Array.from({length:e},()=>[]),u=i?new Array(e).fill(0):null;const f=[],p=Math.min(31,t+8),m=new Float32Array(64*c);for(let h=0;h<p;h++){const w=[];for(let v=0;v<e;v++)l[v]>-1/0&&w.push(v);this.step(s,a,w,d,h,o);const g=[];for(const v of w){const E=o[v];let R=null;if(i){const M=u[v];if(M===null)continue;R=i.allowed(M)}if(R)for(const M of R)M!==0&&M!==1&&M!==3&&g.push([l[v]+E[M],v,M]);else{g.push([l[v]+E[2],v,2]);for(let M=4;M<132;M++)g.push([l[v]+E[M],v,M])}}g.sort((v,E)=>E[0]-v[0]||v[1]*132+v[2]-(E[1]*132+E[2]));const y=g.slice(0,2*e),A=new Array(e).fill(-1/0),k=new Array(e).fill(2),x=new Array(e).fill(0),N=i?new Array(e).fill(null):null;let S=0;for(const[v,E,R]of y)if(v>-1/0){if(R===2){f.length<n&&f.push([ft(r[E]),v]);continue}S<e&&(A[S]=v,k[S]=R,x[S]=E,i&&(N[S]=i.child(u[E],R)),S++)}let z=!0;if(f.length>=n&&(S===0||A[0]<Math.min(...f.map(v=>v[1])))?z=!1:S||(z=!1),!z)break;for(const v of a){const E=(h+1)*2*c,R=[];for(let M=0;M<S;M++){const W=x[M]*32*2*c;R.push(v.slice(W,W+E))}for(let M=0;M<S;M++)v.set(R[M],M*32*2*c)}r=Array.from({length:e},(v,E)=>E<S?r[x[E]].concat([k[E]]):[]),l=A,d=k,u=N}const _=new Map;for(const[h,w]of f)(!_.has(h)||w>_.get(h))&&_.set(h,w);return[..._.entries()].sort((h,w)=>w[1]-h[1]).slice(0,n)}scorePair(s,t){const e=ut(t),n=this.newCache(1),i=[new Float64Array(132)];let c=[1],a=0;for(let o=0;o<e.length;o++)this.step(s,n,[0],c,o,i),a+=i[0][e[o]],c=[e[o]];return a}}function C(b,s,t){const e=new Uint32Array(s);for(let n=0;n<s;n++){let i=0,c=0,a;do a=b[t++],i+=(a&127)*2**c,c+=7;while(a>=128);e[n]=i}return[e,t]}const V=(b,s)=>{const t=new Uint8Array(s);for(let e=0;e<s;e++)t[e]=e&1?b[e>>1]&15:b[e>>1]>>4;return t};class bt{constructor(s,t){if(!s){this.words=[],this.sortedWords=[],this.uni=new Float32Array(0),this.N=0,this.unk=0,this.alpha=!0,this.hasLM=!1;return}const e=new Uint8Array(s),n=new DataView(e.buffer,e.byteOffset,e.byteLength),i=[],c=(o,l,d)=>{let r=[],u=0,f=0;for(;i.length<d;){const p=o?o[f++]:l[u++],m=r.slice(0,p);for(;;){const _=l[u++],h=_&127;if(h!==127&&m.push(h),_&128)break}i.push(String.fromCharCode(...m.map(_=>2304+_))),r=m}};if(e[0]===76&&e[1]===88&&e[2]===52&&e[3]===0){const o=n.getUint32(4,!0),l=n.getUint32(8,!0),d=n.getUint32(16,!0);this.unk=n.getFloat32(12,!0);let r=20;const u=new Float32Array(d);for(let p=0;p<d;p++)u[p]=n.getFloat32(r+4*p,!0);r+=4*d;const f=V(e.subarray(r,r+(o+1>>1)),o);r+=o+1>>1,this.uni=new Float32Array(o);for(let p=0;p<o;p++)this.uni[p]=u[f[p]];c(e.subarray(r,r+o),e.subarray(r+o,r+o+l),o),this.N=o,this.alpha=!0}else{const o=n.getUint32(0,!0),l=n.getUint32(4,!0),d=n.getUint32(8,!0);this.unk=n.getFloat32(12,!0);let[r,u]=C(e,l,16);const f=[];for(let m=0;m<l;m++)f.push(Z(n.getUint16(u+2*m,!0)));u+=2*l,this.uni=new Float32Array(o);let p=0;for(let m=0;m<l;m++)for(let _=0;_<r[m];_++)this.uni[p++]=f[m];c(null,e.subarray(u,u+d),o),this.N=o,this.alpha=!1}this.words=i;let a=this.alpha;for(let o=1;a&&o<i.length;o++)i[o-1]<i[o]||(a=!1);a||(this.map=new Map,i.forEach((o,l)=>this.map.set(o,l))),this.sortedWords=a?i:[...new Set(i)].sort(),this.hasLM=!1,t&&this._loadLM(new Uint8Array(t))}id(s){if(this.map){const n=this.map.get(s);return n===void 0?null:n}let t=0,e=this.words.length;for(;t<e;){const n=t+e>>1;this.words[n]<s?t=n+1:e=n}return t<this.words.length&&this.words[t]===s?t:null}_loadLM(s){const t=new DataView(s.buffer,s.byteOffset,s.byteLength),e=this.N,n=t.getUint32(4,!0),i=t.getUint32(16,!0);let c=20;const a=[],o=[];for(let m=0;m<16;m++)a.push(t.getFloat32(c+4*m,!0));c+=64;for(let m=0;m<16;m++)o.push(t.getFloat32(c+4*m,!0));c+=64;let l,d;[l,c]=C(s,e+1,c),[d,c]=C(s,n,c);const r=V(s.subarray(c,c+(n+1>>1)),n);c+=n+1>>1;const u=V(s.subarray(c,c+(i+1>>1)),i);this.off=new Uint32Array(e+2),this.succ=new Uint32Array(n),this.bq=r,this.cb=Float32Array.from(a),this.bow=new Float32Array(e+1);let f=0,p=0;for(let m=0;m<=e;m++){this.off[m]=f;const _=l[m];if(!_)continue;this.bow[m]=o[u[p++]];let h=-1;for(let w=0;w<_;w++)h=h<0?d[f]:h+d[f]+1,this.succ[f++]=h}this.off[e+1]=f,this.hasLM=n>0}unigram(s){return s===null?this.unk:s===this.N?Math.log(.04):this.uni[s]}logp(s,t){const e=s===null?this.N:s;if(t!==null&&this.hasLM){let n=this.off[e],i=this.off[e+1];for(;n<i;){const c=n+i>>1;this.succ[c]<t?n=c+1:i=c}if(n<this.off[e+1]&&this.succ[n]===t)return this.cb[this.bq[n]]}return(this.hasLM?this.bow[e]:0)+this.unigram(t)}}class yt{constructor(s){this.w=s,this.nodes=[[0,s.length,0]],this.ids=new Map([["0,0,"+s.length,0]]),this.cache=new Map,this.allow=new Map}_bisect(s,t,e){const n=this.w;for(;t<e;){const i=t+e>>1;n[i]<s?t=i+1:e=i}return t}children(s){let t=this.cache.get(s);if(t)return t;const[e,n,i]=this.nodes[s],c=this.w;t=new Map;let a=e;for(a<n&&c[a].length===i&&a++;a<n;){const o=c[a].charCodeAt(i),l=c[a].slice(0,i),d=this._bisect(l+String.fromCharCode(o+1),a,n),r=a+","+(i+1)+","+d;let u=this.ids.get(r);u===void 0&&(u=this.nodes.length,this.nodes.push([a,d,i+1]),this.ids.set(r,u)),t.set(o-2304+4,u),a=d}return this.cache.size>4e5&&(this.cache.clear(),this.allow.clear()),this.cache.set(s,t),t}final(s){const[t,,e]=this.nodes[s];return t<this.w.length&&this.w[t].length===e}allowed(s){let t=this.allow.get(s);return t||(t=[...this.children(s).keys()],this.final(s)&&t.push(2),this.allow.set(s,t)),t}child(s,t){return this.children(s).get(t)}}const $={};[..."abcdefghijklmnopqrstuvwxyz'"].forEach((b,s)=>$[b]=s+3);const L=16,_t=["english","romanized_hindi","hinglish"],T=/^[A-Za-z]+(?:'[A-Za-z]+)?$/,O=/^[\u0900-\u0963\u0971-\u097F]+$/;function At(b,s){let t=2166136261;for(const e of new TextEncoder().encode(b.toLowerCase()))t=Math.imul(t^e,16777619)>>>0;return t%s}class xt{constructor(s){this.cfg=s.cfg,this.x=s.extra||{};const t=e=>s.T[e]&&s.T[e].data;Object.assign(this,{ce:t("ce.weight"),c1:t("c1.weight"),c1b:t("c1.bias"),c2:t("c2.weight"),c2b:t("c2.bias"),wemb:t("wemb.weight"),wih:t("gru.weight_ih_l0"),whh:t("gru.weight_hh_l0"),bih:t("gru.bias_ih_l0"),bhh:t("gru.bias_hh_l0"),wihR:t("gru.weight_ih_l0_reverse"),whhR:t("gru.weight_hh_l0_reverse"),bihR:t("gru.bias_ih_l0_reverse"),bhhR:t("gru.bias_hh_l0_reverse"),tw:t("tok.weight"),tb:t("tok.bias"),s0:t("sent.0.weight"),s0b:t("sent.0.bias"),s2:t("sent.2.weight"),s2b:t("sent.2.bias")}),this.nParams=Object.values(s.T).reduce((e,n)=>e+n.data.length,0)}_wvec(s){const{ce:t,cf:e}=this.cfg,n=new Array(L).fill(0),i=[...s.toLowerCase()].slice(0,L-2);n[0]=1,i.forEach((f,p)=>n[p+1]=$[f]??2),n[i.length+1]=2;const c=new Float32Array(t*L);for(let f=0;f<L;f++)for(let p=0;p<t;p++)c[p*L+f]=this.ce[n[f]*t+p];const a=(f,p,m,_)=>{const h=new Float32Array(e*L);for(let w=0;w<e;w++)for(let g=0;g<L;g++){let y=_[w];for(let A=0;A<p;A++)for(let k=0;k<3;k++){const x=g+k-1;x>=0&&x<L&&(y+=m[(w*p+A)*3+k]*f[A*L+x])}h[w*L+g]=y>0?y:0}return h},o=a(c,t,this.c1,this.c1b),l=a(o,e,this.c2,this.c2b),d=this.cfg.wh,r=this.cfg.wd,u=new Float32Array(e+(d?r:0));for(let f=0;f<e;f++){let p=-1/0;for(let m=0;m<L;m++){const _=n[m]===0?-1e4:l[f*L+m]+o[f*L+m];_>p&&(p=_)}u[f]=p}if(d){const f=At(s,d);for(let p=0;p<r;p++)u[e+p]=this.wemb[f*r+p]}return u}_gru(s,t,e,n,i,c){const a=this.cfg.hid,o=s[0].length,l=s.length,d=new Array(l);let r=new Float32Array(a);const u=new Float32Array(3*a),f=new Float32Array(3*a);for(let p=0;p<l;p++){const m=c?l-1-p:p;F(s[m],0,t,n,u,0,o,3*a),F(r,0,e,i,f,0,a,3*a);const _=new Float32Array(a);for(let h=0;h<a;h++){const w=X(u[h]+f[h]),g=X(u[a+h]+f[a+h]),y=Math.tanh(u[2*a+h]+w*f[2*a+h]);_[h]=(1-g)*y+g*r[h]}r=_,d[m]=r}return d}run(s){if(!s.length)return{pen:[],ps:null};const t=this.cfg.hid,e=s.map(g=>this._wvec(g)),n=e.length,i=e[0].length,c=this._gru(e,this.wih,this.whh,this.bih,this.bhh,!1),a=this._gru(e,this.wihR,this.whhR,this.bihR,this.bhhR,!0),o=[],l=new Float32Array(2*t),d=new Float32Array(2*t).fill(-1e4),r=new Float32Array(2*t+i),u=new Float32Array(2);for(let g=0;g<n;g++){for(let x=0;x<t;x++)r[x]=c[g][x],r[t+x]=a[g][x];r.set(e[g],2*t),F(r,0,this.tw,this.tb,u,0,2*t+i,2);const y=Math.max(u[0],u[1]),A=Math.exp(u[0]-y),k=Math.exp(u[1]-y);o.push(A/(A+k));for(let x=0;x<2*t;x++)l[x]+=r[x]/n,r[x]>d[x]&&(d[x]=r[x])}const f=new Float32Array(4*t+1);f.set(l,0),f.set(d,2*t),f[4*t]=o.reduce((g,y)=>g+y,0)/n;const p=new Float32Array(32);F(f,0,this.s0,this.s0b,p,0,4*t+1,32);for(let g=0;g<32;g++)p[g]=Math.max(0,p[g]);const m=new Float32Array(3);F(p,0,this.s2,this.s2b,m,0,32,3);const _=Math.max(...m),h=[...m].map(g=>Math.exp(g-_)),w=h.reduce((g,y)=>g+y,0);return{pen:o,ps:h.map(g=>g/w)}}tag(s){const t=s.filter(c=>T.test(c)),{pen:e}=this.run(t);let n=0;const i=this.x.tok_th??.5;return s.map(c=>O.test(c)?"DEV":T.test(c)?e[n++]>=i?"EN":"HI":"OTHER")}mode(s){const t=s.filter(o=>O.test(o)).length,e=s.filter(o=>T.test(o)).length;if(t&&!e)return"devanagari";if(t&&e)return t/(t+e)>=(this.x.dev_frac??.5)?"devanagari":"mixed_script";if(!e)return"english";const{ps:n}=this.run(s.filter(o=>T.test(o))),i=this.x.mode_bias||[0,0,0];let c=0,a=-1/0;return n.forEach((o,l)=>{const d=Math.log(o+1e-9)+i[l];d>a&&(a=d,c=l)}),_t[c]}}const G=/(https?:\/\/\S+|www\.\S+|[\p{L}\p{N}_.+-]+@[\p{L}\p{N}_-]+\.[\p{L}\p{N}_.]+|[@#][\p{L}\p{N}_]+|[A-Za-z]+(?:'[A-Za-z]+)?|[\u0900-\u0963\u0971-\u097F]+|[0-9\u0966-\u096F]+(?:[.,:][0-9\u0966-\u096F]+)*|[\u0964\u0965]|[^\sA-Za-z0-9\u0900-\u097F])/gu,J=/^[A-Za-z]+$/,kt=/^[b-df-hj-np-tv-z]{2,7}$/,K=Object.fromEntries([..."abcdefghijklmnopqrstuvwxyz"].map((b,s)=>[b,"\u090F \u092C\u0940 \u0938\u0940 \u0921\u0940 \u0908 \u090F\u092B \u091C\u0940 \u090F\u091A \u0906\u0908 \u091C\u0947 \u0915\u0947 \u090F\u0932 \u090F\u092E \u090F\u0928 \u0913 \u092A\u0940 \u0915\u094D\u092F\u0942 \u0906\u0930 \u090F\u0938 \u091F\u0940 \u092F\u0942 \u0935\u0940 \u0921\u092C\u094D\u0932\u094D\u092F\u0942 \u090F\u0915\u094D\u0938 \u0935\u093E\u0908 \u091C\u0947\u0921".split(" ")[s]])),vt={...K,f:"\u090F\u092B\u093C",z:"\u091C\u093C\u0947\u0921"},Mt=[["\u0949","\u094B"],["\u094B","\u0949"],["\u093E","\u0949"],["\u0949","\u093E"],["\u0903",""],["\u092B\u093C","\u092B"],["\u092B","\u092B\u093C"],["\u091C\u093C","\u091C"],["\u091C","\u091C\u093C"],["\u0901","\u0902"],["\u0902","\u0901"]],Ct=["sc","scn","inlex","uni","free0","infree","incon","aw","visarga","nukta","chandra","acro","variant","frank","skel","skmatch","lev","sc_rel","scn_rel","uni_rel","lenr","visr","hfin","acro_nv"],Et=new Set(`whatsapp google meet paytm amazon flipkart youtube instagram facebook gmail excel uber ola swiggy zomato
 netflix linkedin github python java jira slack zoom iphone android windows microsoft apple samsung twitter telegram
-chrome firefox safari office365 outlook teams spotify hotstar jio airtel phonepe gpay`.split(/\s+/)),I=/^(https?:\/\/|www\.|[@#])|@.+\.|_|\d.*[A-Za-z]|[A-Za-z].*\d|^[a-z]+[A-Z]/,Et=b=>/[A-Z]/.test(b)&&b===b.toUpperCase(),Q={beam:8,nbest:8,cbeam:8,variants:!0,acro:!0,w:null,mu:null,sd:null,w_lm:.8,oov_pen_s:4,w_ch_s:1,use_lm:!0,skel:!1,skel_k:200,skel_m:16,skel_wu:.03,visr:!1,acro_v:!1},Y=b=>{const s=new Map;for(const t of b.split(`
-`)){const e=t.indexOf("	");e>0&&s.set(t.slice(0,e).trim(),t.slice(e+1).trim())}return s},q=()=>typeof performance<"u"?performance.now():Date.now();async function P(b,s,t){const e=b.endsWith("/")?b+s:b+"/"+s,n=await t(e);if(!n.ok)throw new Error(`fetch ${e}: ${n.status}`);return s.endsWith(".gz")&&typeof DecompressionStream<"u"?new Uint8Array(await new Response(n.body.pipeThrough(new DecompressionStream("gzip"))).arrayBuffer()):new Uint8Array(await n.arrayBuffer())}const Nt={\u0915:"K",\u0916:"K",\u0917:"G",\u0918:"G",\u091A:"C",\u091B:"C",\u091C:"J",\u091D:"J",\u091F:"T",\u0920:"T",\u0924:"T",\u0925:"T",\u0921:"D",\u0922:"D",\u0926:"D",\u0927:"D",\u0919:"N",\u091E:"N",\u0923:"N",\u0928:"N",\u0929:"N",\u092E:"N","\u0902":"N","\u0901":"N",\u092A:"P",\u092B:"P",\u092C:"B",\u092D:"B",\u0930:"R",\u0931:"R",\u090B:"R","\u0943":"R",\u0932:"L",\u0933:"L",\u0935:"V",\u0936:"S",\u0937:"S",\u0938:"S",\u0939:"H","\u0903":"H"},tt={k:"K",q:"K",g:"G",j:"J",z:"J",t:"T",d:"D",n:"N",m:"N",p:"P",f:"P",b:"B",r:"R",l:"L",v:"V",w:"V",s:"S",h:"H"},St="bcdgjkpstz",et={\u0915:"k",\u0916:"kh",\u0917:"g",\u0918:"gh",\u0919:"n",\u091A:"ch",\u091B:"chh",\u091C:"j",\u091D:"jh",\u091E:"n",\u091F:"t",\u0920:"th",\u0921:"d",\u0922:"dh",\u0923:"n",\u0924:"t",\u0925:"th",\u0926:"d",\u0927:"dh",\u0928:"n",\u092A:"p",\u092B:"ph",\u092C:"b",\u092D:"bh",\u092E:"m",\u092F:"y",\u0930:"r",\u0932:"l",\u0935:"v",\u0936:"sh",\u0937:"sh",\u0938:"s",\u0939:"h",\u0933:"l"},st={\u0905:"a",\u0906:"aa",\u0907:"i",\u0908:"ee",\u0909:"u",\u090A:"oo",\u090B:"ri",\u090F:"e",\u0910:"ai",\u0913:"o",\u0914:"au",\u0911:"o"},B={"\u093E":"a","\u093F":"i","\u0940":"ee","\u0941":"u","\u0942":"oo","\u0943":"ri","\u0947":"e","\u0948":"ai","\u094B":"o","\u094C":"au","\u0949":"o","\u0945":"e"},Ft={k:"q",kh:"kh",g:"g",j:"z",ph:"f",d:"r",dh:"rh"};function nt(b){let s="";for(const t of b){const e=Nt[t];e&&s[s.length-1]!==e&&(s+=e)}return s}function it(b,s=4){b=b.toLowerCase();let t=[""],e=0;const n=b.length;for(;e<n;){const o=b[e];let c=1,a;if((o==="t"||o==="s")&&b.startsWith("ion",e+1))a=o==="t"?["SN","TN"]:["SN"],c=4;else if(o==="c"){const l=e+1<n?b[e+1]:"";if(l==="h")for(a=["C","K"],c=2;e+c<n&&(b[e+c]==="c"||b[e+c]==="h");)c++;else l==="e"||l==="i"||l==="y"?a=["S","K"]:l==="k"?(a=["K"],c=2):a=["K","S"]}else if(o==="x")a=["KS"];else if(St.includes(o)&&e+1<n&&b[e+1]==="h")for(a=[tt[o]||""],c=2;e+c<n&&b[e+c]==="h";)c++;else a=[tt[o]||""];const r=[];for(const l of t)for(const f of a){let i=l;for(const u of f)(!i||i[i.length-1]!==u)&&(i+=u);r.push(i)}t=[...new Set(r)].slice(0,s),e+=c}return t}function ot(b){const s=[],t=[...b],e=t.length;for(let n=0;n<e;n++){const o=t[n];if(et[o]!==void 0){s.push(et[o]);let c=n+1<e?t[n+1]:"";c==="\u093C"&&(c=n+2<e?t[n+2]:""),c&&B[c]===void 0&&c!=="\u094D"&&n+1<e-1&&s.push("a")}else if(st[o]!==void 0)s.push(st[o]);else if(B[o]!==void 0)s.push(B[o]);else if(o==="\u0902"||o==="\u0901")s.push("n");else if(o==="\u0903")s.push("h");else if(o==="\u093C"&&s.length){const c=s[s.length-1];s[s.length-1]=Ft[c]??c}}return s.join("")}function rt(b,s){b.length<s.length&&([b,s]=[s,b]);let t=Array.from({length:s.length+1},(e,n)=>n);for(let e=1;e<=b.length;e++){const n=[e];for(let o=1;o<=s.length;o++)n.push(Math.min(t[o]+1,n[o-1]+1,t[o-1]+(b[e-1]!==s[o-1]?1:0)));t=n}return t[s.length]}class Lt{constructor(s){const t=new Map,e=s.words.length;for(let n=0;n<e;n++){const o=nt(s.words[n]);if(o){let c=t.get(o);c||t.set(o,c=[]),c.push(n)}}for(const n of t.values())n.sort((o,c)=>s.uni[c]-s.uni[o]||o-c);this.b=t}lookup(s,t=24){const e=[],n=new Set;for(const o of it(s)){const c=this.b.get(o);if(c)for(const a of c.slice(0,t))n.has(a)||(n.add(a),e.push(a))}return e}}const at=/[aeiou]h$/;class U{static async load(s,{fetch:t=globalThis.fetch,onProgress:e,dict:n=!0,cache:o="aksharalp-dict"}={}){const c=q(),a=g=>new TextDecoder().decode(g);let r;try{r=JSON.parse(a(await P(s,"manifest.json",t)))}catch{r={files:{model:"xlit.q4",lex:"lex.bin",lm:"lm.bin",params:"params.json",lid:"lid.int8",loan:"loan.tsv",natural:"natural_hi.tsv"}}}const l=r.files,f=async(g,y)=>{if(!g)return null;try{const A=await P(s,g,t);return e&&e(g),A}catch(A){if(y)return null;throw A}},[i,u,d,p,m,_,h]=await Promise.all([f(l.model),f(l.lex,!0),f(l.lm,!0),f(l.params,!0),f(l.lid,!0),f(l.loan,!0),f(l.natural,!0)]),w=new U({model:i,lex:u,lm:d,params:p?JSON.parse(a(p)):{},lid:m,loan:_?a(_):null,natural:h?a(h):null,manifest:r});return w.base=s,w.fetchImpl=t,n&&r.packs&&r.packs.dict&&await w.loadDict({onProgress:e,cache:o}),w.loadMs=q()-c,w}async loadDict({onProgress:s,cache:t="aksharalp-dict"}={}){const e=this.manifest&&this.manifest.packs&&this.manifest.packs.dict;if(!e)return!1;const n=e.dir.endsWith("/")?e.dir:e.dir+"/",o=this.fetchImpl||globalThis.fetch,c=t&&typeof caches<"u",a=async r=>{if(!r)return null;const l=e.sizes&&e.sizes[r]&&e.sizes[r].sha256||"";if(c)try{const i=new URL(n+r,new URL(this.base.endsWith("/")?this.base:this.base+"/",globalThis.location&&globalThis.location.href)).href,u=await caches.open(t),d=i+"?v="+l.slice(0,16),p=await u.match(d);if(p)return s&&s(r,"cache"),new Uint8Array(await p.arrayBuffer());const m=await o(i);if(!m.ok)throw new Error(`fetch ${i}: ${m.status}`);const _=new Uint8Array(await m.arrayBuffer());try{await u.put(d,new Response(_))}catch{}return s&&s(r,"network"),_}catch(i){if(String(i).includes("fetch "))throw i}const f=await P(this.base,n+r,o);return s&&s(r,"network"),f};try{const[r,l,f]=await Promise.all([a(e.files.lex),a(e.files.lm),a(e.files.params)]);return this.attachDict({lex:r,lm:l,params:f?JSON.parse(new TextDecoder().decode(f)):{}}),this.dictInfo={name:e.name||"dict",words:this.lm.N,bytes:e.bytes,gz:e.gz},!0}catch(r){return this.dictError=String(r&&r.message||r),!1}}attachDict({lex:s,lm:t=null,params:e={}}){this.p={...Q,...this.coreParams,...e},this._setLex(s,t),this.cache=new Map,this.hasDict=this.lm.N>0}_setLex(s,t){this.lm=new mt(s,t&&this.p.use_lm?t:null),this.lm.N||(this.p.cbeam=0,this.p.skel=!1);const e=this.lm.sortedWords,n=e.length&&e[0]===""?e.slice(1):e;this.trie=this.p.cbeam?new bt(n):null,this.skix=this.p.skel?new Lt(this.lm):null}constructor({model:s,lex:t=null,lm:e=null,params:n={},lid:o=null,loan:c=null,natural:a=null,manifest:r=null}){this.coreParams=n,this.p={...Q,...n},this.manifest=r,this.model=new gt(X(s)),this._setLex(t,e),this.hasDict=this.lm.N>0,this.dictInfo=null,this.dictError=null,this.lid=o?new At(X(o)):null,this.loan=c?Y(c):new Map,this.natural=a?Y(a):new Map,this.cache=new Map}info(){const s=this.manifest&&this.manifest.sizes?this.manifest.sizes:{},t=Object.values(this.manifest?this.manifest.files:{}).filter(e=>e&&s[e]);return{params:this.model.nParams+(this.lid?this.lid.nParams:0),xlitParams:this.model.nParams,lidParams:this.lid?this.lid.nParams:0,lexWords:this.lm.N,bytes:t.reduce((e,n)=>e+s[n].bytes,0),gz:t.reduce((e,n)=>e+s[n].gz,0),hasLID:!!this.lid,hasLM:this.lm.hasLM,hasDict:this.hasDict,dict:this.dictInfo,dictError:this.dictError,hasLoan:this.loan.size>0,hasNatural:this.natural.size>0}}candFeats(s){const t=s.toLowerCase(),e=this.cache.get(t);if(e)return e;const n=this.p,o=this.model,c=ct(t),a=o.encode(c),r=c.length,l=o.beam(a,r,{beam:n.beam,nbest:n.nbest}),f=n.cbeam&&this.trie?o.beam(a,r,{beam:n.beam,nbest:n.nbest,trie:this.trie}):[],i=new Map;l.forEach(([h,w],g)=>{h&&i.set(h,{sc:w,infree:1,frank:g,free0:+(g===0),beamEntry:1})});for(const[h,w]of f)h&&(i.has(h)||i.set(h,{sc:w,infree:0,frank:n.nbest,free0:0,beamEntry:1}),i.get(h).incon=1);if(n.variants){const h=[...i.keys()].sort((w,g)=>i.get(g).sc-i.get(w).sc).slice(0,4);for(const w of h)for(const[g,y]of vt)if(w.includes(g)){const A=w.split(g).join(y);!i.has(A)&&this.lm.id(A)!==null&&i.set(A,{sc:o.scorePair(a,A),infree:0,frank:n.nbest,free0:0,variant:1})}}if(n.acro&&(xt.test(t)||n.acro_v&&t.length>=2&&t.length<=4)){const h=[...t].map(w=>K[w]).join("");i.has(h)?i.get(h).acro=1:i.set(h,{sc:o.scorePair(a,h),infree:0,frank:n.nbest,free0:0,acro:1})}if(n.visr&&at.test(t)){const h=[...i.keys()].filter(w=>i.get(w).beamEntry).sort((w,g)=>i.get(g).sc-i.get(w).sc).slice(0,2);for(const w of h){const g=w.endsWith("\u0939")?w.slice(0,-1)+"\u0903":w.endsWith("\u0903")?null:w+"\u0903";g&&!i.has(g)&&i.set(g,{sc:o.scorePair(a,g),infree:0,frank:n.nbest,free0:0,visr:1})}}if(this.skix){const h=t.length+1,w=this.lm.words,g=this.lm.uni,y=this.skix.lookup(t,n.skel_k).map(A=>[A,rt(t,ot(w[A]))/h-n.skel_wu*g[A]]).sort((A,k)=>A[1]-k[1]).slice(0,n.skel_m);for(const[A]of y){const k=w[A];i.has(k)?i.get(k).skel=1:i.set(k,{sc:o.scorePair(a,k),infree:0,frank:n.nbest,free0:0,skel:1})}}const u=[],d=new Set(it(t)),p=t.length+1,m=at.test(t),_=!/[aeiou]/.test(t);for(const[h,w]of i){const g=this.lm.id(h),y=[...h].length;u.push([h,[w.sc,w.sc/(y+1),+(g!==null),g!==null?this.lm.unigram(g):0,w.free0||0,w.infree||0,w.incon||0,+(h.includes("\u0949")||h.includes("\u0911")),+h.includes("\u0903"),+h.includes("\u093C"),+h.includes("\u0901"),w.acro||0,w.variant||0,(w.frank??n.nbest)/n.nbest,w.skel||0,+d.has(nt(h)),rt(t,ot(h))/p,0,0,0,y/p,w.visr||0,+(m&&h.endsWith("\u0903")),+(!!w.acro&&_)],g])}if(u.length){let h=-1/0,w=-1/0,g=null;for(const[,y]of u)h=Math.max(h,y[0]),w=Math.max(w,y[1]),y[2]&&(g=g===null?y[3]:Math.max(g,y[3]));for(const[,y]of u)y[17]=y[0]-h,y[18]=y[1]-w,y[19]=y[2]?y[3]-(g??0):0}return this.cache.size>2e4&&this.cache.clear(),this.cache.set(t,u),u}score(s){const t=this.p;if(!t.w&&!t.w1)return s[0]+.2*(s[2]?s[3]:-25);const e=t.mu.length,n=new Array(e);for(let c=0;c<e;c++)n[c]=(s[c]-t.mu[c])/t.sd[c];if(t.w1){let c=t.b2||0;for(let a=0;a<t.w1.length;a++){const r=t.w1[a];let l=t.b1[a];for(let f=0;f<e;f++)l+=r[f]*n[f];c+=t.w2[a]*Math.tanh(l)}return c}let o=0;for(let c=0;c<Math.min(e,t.w.length);c++)o+=t.w[c]*n[c];return o}word(s,t=8){return J.test(s)?this.candFeats(s).map(([n,o])=>({text:n,score:this.score(o)})).sort((n,o)=>o.score-n.score).slice(0,t):[{text:s,score:0}]}decodeSlots(s,t=!0){const e=this.p,n=this.lm,o=[];for(const[f,i]of s)if(f==="x"){let u=this.candFeats(i).map(([d,p,m])=>[d,this.score(p),m]);if(u.length){const d=Math.max(...u.map(m=>m[1])),p=d+Math.log(u.reduce((m,_)=>m+Math.exp(_[1]-d),0));u=u.map(([m,_,h])=>[m,e.w_ch_s*(_-p),h]).sort((m,_)=>_[1]-m[1]).slice(0,12)}o.push(u.length?u:[[i,0,null]])}else{const u=O.test(i)?n.id(i)??-1:-2;o.push([[i,0,u]])}const c=o.map(f=>f.map(([i,u])=>[i,u]));if(!t||!e.use_lm||!n.hasLM)return{out:o.map(f=>f.reduce((i,u)=>u[1]>i[1]?u:i)[0]),lat:c};let a=[[0,null,[]]];for(const f of o){if(f[0][2]===-2&&f.length===1){a=a.map(([u,d,p])=>[u,d,p.concat([f[0][0]])]);continue}const i=new Map;for(const[u,d,p]of a)for(const[m,_,h]of f){const w=h===null||h===-1?null:h,g=u+_+e.w_lm*(n.logp(d,w)-(w===null?e.oov_pen_s:0));(!i.has(m)||g>i.get(m)[0])&&i.set(m,[g,w,p.concat([m])])}a=[...i.values()].sort((u,d)=>d[0]-u[0]).slice(0,16)}let r=null,l=-1/0;for(const f of a){const i=f[0]+e.w_lm*n.logp(f[1],n.N);i>l&&(l=i,r=f)}return{out:r[2],lat:c}}convertPlain(s){const t=[...s.matchAll($)].map(c=>[c.index,c.index+c[0].length,c[0]]),{out:e}=this.decodeSlots(t.map(([,,c])=>J.test(c)?["x",c]:["fix",c]));e.length&&e[e.length-1]==="."&&(e[e.length-1]="\u0964");let n="",o=0;return t.forEach(([c,a],r)=>{n+=s.slice(o,c)+e[r],o=a}),n+s.slice(o)}transliterate(s,{mode:t="conversational",k:e=5,overrides:n={},context:o=!0}={}){const c=q(),a=t==="strict"?"strict":"conversational",r=[...s.matchAll($)].map(y=>[y.index,y.index+y[0].length,y[0]]),l=r.map(y=>y[2]),f=this.lid?this.lid.tag(l):l.map(y=>O.test(y)?"DEV":T.test(y)?this.loan.has(y.toLowerCase())?"EN":"HI":"OTHER"),i=[],u=[];let d=!1;for(let y=0;y<l.length;y++){const A=l[y],k=f[y],x=l[y+1]||"",N=A.toLowerCase(),S=n[r[y][0]];if(S!==void 0){i.push(["fix",S]),u.push("user"),d=!1;continue}if(!T.test(A)||I.test(A)||x==="("){A==="."&&(y===l.length-1||/^[A-Z]/.test(l[y+1]||""))?i.push(["fix","\u0964"]):i.push(["fix",A]),u.push(O.test(A)?"devanagari":I.test(A)?"keep":"punct"),d=!1;continue}const z=Mt.has(N)||d&&/^[A-Z]/.test(A);if(d=z,a==="strict"&&z){i.push(["fix",A]),u.push("brand");continue}if(t==="natural"&&k==="EN"&&this.natural.has(N)){i.push(["fix",this.natural.get(N)]),u.push("glossary");continue}if(Et(A)&&A.length>=2&&A.length<=5&&!this.loan.has(N)){i.push(["fix",[...N].map(v=>kt[v]).join("")]),u.push("acronym");continue}if((k==="EN"||z)&&this.loan.has(N)){i.push(["fix",this.loan.get(N)]),u.push("loanword");continue}i.push(["x",A]),u.push("model")}const{out:p,lat:m}=this.decodeSlots(i,o);let _="",h=0;r.forEach(([y,A],k)=>{_+=s.slice(h,y)+p[k],h=A}),_=(_+s.slice(h)).trim();const w=r.map(([y,A,k],x)=>{let N=[];if(i[x][0]==="x"){const z=m[x][0]?m[x][0][1]:0;N=m[x].slice(0,e).map(([v,E])=>({text:v,score:Math.exp(E-z)}))}else if(T.test(k)&&!I.test(k)){N=[{text:p[x],score:1}];for(const z of this.word(k,e))z.text!==p[x]&&N.push({text:z.text,score:null});p[x]!==k&&N.push({text:k,score:null}),N=N.slice(0,e)}const S=T.test(k)||O.test(k);return{text:k,start:y,end:A,output:p[x],lang:S?f[x]:"OTHER",source:u[x],candidates:N}});let g=null;if(this.lid&&l.length)try{g=this.lid.mode(l)}catch{g=null}return{output:_,tokens:w,sentence:g,ms:q()-c}}}async function Ct(b,s={}){const t=await import("node:fs/promises"),e=await import("node:path"),n=async o=>{try{const c=await t.readFile(e.resolve(o));return new Response(c)}catch{return new Response(null,{status:404})}};return U.load(b,{fetch:n,cache:!1,...s})}var Vt=U;export{U as AksharaLP,U as BhashaCore,jt as FEATS,At as LID,mt as LexLM,$ as TOKEN_RE,gt as XlitModel,Vt as default,ot as devRom,nt as devSkel,rt as lev,Ct as loadFromDir,X as parseWeights,it as romSkels};
+chrome firefox safari office365 outlook teams spotify hotstar jio airtel phonepe gpay`.split(/\s+/)),I=/^(https?:\/\/|www\.|[@#])|@.+\.|_|\d.*[A-Za-z]|[A-Za-z].*\d|^[a-z]+[A-Z]/,Nt=b=>/[A-Z]/.test(b)&&b===b.toUpperCase(),Q={beam:8,nbest:8,cbeam:8,variants:!0,acro:!0,w:null,mu:null,sd:null,w_lm:.8,oov_pen_s:4,w_ch_s:1,use_lm:!0,skel:!1,skel_k:200,skel_m:16,skel_wu:.03,visr:!1,acro_v:!1},Y=b=>{const s=new Map;for(const t of b.split(`
+`)){const e=t.indexOf("	");e>0&&s.set(t.slice(0,e).trim(),t.slice(e+1).trim())}return s},q=()=>typeof performance<"u"?performance.now():Date.now(),tt=(b,s,t)=>{if(s&&b.length!==s)throw new Error(`fetch ${t}: got ${b.length} bytes, manifest says ${s}`);return b};async function P(b,s,t,e=0){const n=b.endsWith("/")?b+s:b+"/"+s,i=await t(n);if(!i.ok)throw new Error(`fetch ${n}: ${i.status}`);return s.endsWith(".gz")&&typeof DecompressionStream<"u"?new Uint8Array(await new Response(i.body.pipeThrough(new DecompressionStream("gzip"))).arrayBuffer()):tt(new Uint8Array(await i.arrayBuffer()),e,n)}const St={\u0915:"K",\u0916:"K",\u0917:"G",\u0918:"G",\u091A:"C",\u091B:"C",\u091C:"J",\u091D:"J",\u091F:"T",\u0920:"T",\u0924:"T",\u0925:"T",\u0921:"D",\u0922:"D",\u0926:"D",\u0927:"D",\u0919:"N",\u091E:"N",\u0923:"N",\u0928:"N",\u0929:"N",\u092E:"N","\u0902":"N","\u0901":"N",\u092A:"P",\u092B:"P",\u092C:"B",\u092D:"B",\u0930:"R",\u0931:"R",\u090B:"R","\u0943":"R",\u0932:"L",\u0933:"L",\u0935:"V",\u0936:"S",\u0937:"S",\u0938:"S",\u0939:"H","\u0903":"H"},et={k:"K",q:"K",g:"G",j:"J",z:"J",t:"T",d:"D",n:"N",m:"N",p:"P",f:"P",b:"B",r:"R",l:"L",v:"V",w:"V",s:"S",h:"H"},Ft="bcdgjkpstz",st={\u0915:"k",\u0916:"kh",\u0917:"g",\u0918:"gh",\u0919:"n",\u091A:"ch",\u091B:"chh",\u091C:"j",\u091D:"jh",\u091E:"n",\u091F:"t",\u0920:"th",\u0921:"d",\u0922:"dh",\u0923:"n",\u0924:"t",\u0925:"th",\u0926:"d",\u0927:"dh",\u0928:"n",\u092A:"p",\u092B:"ph",\u092C:"b",\u092D:"bh",\u092E:"m",\u092F:"y",\u0930:"r",\u0932:"l",\u0935:"v",\u0936:"sh",\u0937:"sh",\u0938:"s",\u0939:"h",\u0933:"l"},nt={\u0905:"a",\u0906:"aa",\u0907:"i",\u0908:"ee",\u0909:"u",\u090A:"oo",\u090B:"ri",\u090F:"e",\u0910:"ai",\u0913:"o",\u0914:"au",\u0911:"o"},B={"\u093E":"a","\u093F":"i","\u0940":"ee","\u0941":"u","\u0942":"oo","\u0943":"ri","\u0947":"e","\u0948":"ai","\u094B":"o","\u094C":"au","\u0949":"o","\u0945":"e"},Lt={k:"q",kh:"kh",g:"g",j:"z",ph:"f",d:"r",dh:"rh"};function it(b){let s="";for(const t of b){const e=St[t];e&&s[s.length-1]!==e&&(s+=e)}return s}function ot(b,s=4){b=b.toLowerCase();let t=[""],e=0;const n=b.length;for(;e<n;){const i=b[e];let c=1,a;if((i==="t"||i==="s")&&b.startsWith("ion",e+1))a=i==="t"?["SN","TN"]:["SN"],c=4;else if(i==="c"){const l=e+1<n?b[e+1]:"";if(l==="h")for(a=["C","K"],c=2;e+c<n&&(b[e+c]==="c"||b[e+c]==="h");)c++;else l==="e"||l==="i"||l==="y"?a=["S","K"]:l==="k"?(a=["K"],c=2):a=["K","S"]}else if(i==="x")a=["KS"];else if(Ft.includes(i)&&e+1<n&&b[e+1]==="h")for(a=[et[i]||""],c=2;e+c<n&&b[e+c]==="h";)c++;else a=[et[i]||""];const o=[];for(const l of t)for(const d of a){let r=l;for(const u of d)(!r||r[r.length-1]!==u)&&(r+=u);o.push(r)}t=[...new Set(o)].slice(0,s),e+=c}return t}function rt(b){const s=[],t=[...b],e=t.length;for(let n=0;n<e;n++){const i=t[n];if(st[i]!==void 0){s.push(st[i]);let c=n+1<e?t[n+1]:"";c==="\u093C"&&(c=n+2<e?t[n+2]:""),c&&B[c]===void 0&&c!=="\u094D"&&n+1<e-1&&s.push("a")}else if(nt[i]!==void 0)s.push(nt[i]);else if(B[i]!==void 0)s.push(B[i]);else if(i==="\u0902"||i==="\u0901")s.push("n");else if(i==="\u0903")s.push("h");else if(i==="\u093C"&&s.length){const c=s[s.length-1];s[s.length-1]=Lt[c]??c}}return s.join("")}function at(b,s){b.length<s.length&&([b,s]=[s,b]);let t=Array.from({length:s.length+1},(e,n)=>n);for(let e=1;e<=b.length;e++){const n=[e];for(let i=1;i<=s.length;i++)n.push(Math.min(t[i]+1,n[i-1]+1,t[i-1]+(b[e-1]!==s[i-1]?1:0)));t=n}return t[s.length]}class zt{constructor(s){const t=new Map,e=s.words.length;for(let n=0;n<e;n++){const i=it(s.words[n]);if(i){let c=t.get(i);c||t.set(i,c=[]),c.push(n)}}for(const n of t.values())n.sort((i,c)=>s.uni[c]-s.uni[i]||i-c);this.b=t}lookup(s,t=24){const e=[],n=new Set;for(const i of ot(s)){const c=this.b.get(i);if(c)for(const a of c.slice(0,t))n.has(a)||(n.add(a),e.push(a))}return e}}const ct=/[aeiou]h$/;class U{static async load(s,{fetch:t=globalThis.fetch,onProgress:e,dict:n=!0,cache:i="aksharalp-dict"}={}){const c=q(),a=y=>new TextDecoder().decode(y);let o;try{o=JSON.parse(a(await P(s,"manifest.json",t)))}catch{o={files:{model:"xlit.q4",lex:"lex.bin",lm:"lm.bin",params:"params.json",lid:"lid.int8",loan:"loan.tsv",natural:"natural_hi.tsv"}}}const l=o.files,d=o.sizes||{},r=async(y,A)=>{if(!y)return null;try{const k=await P(s,y,t,d[y]&&d[y].bytes);return e&&e(y),k}catch(k){if(A)return null;throw k}},[u,f,p,m,_,h,w]=await Promise.all([r(l.model),r(l.lex,!0),r(l.lm,!0),r(l.params,!0),r(l.lid,!0),r(l.loan,!0),r(l.natural,!0)]),g=new U({model:u,lex:f,lm:p,params:m?JSON.parse(a(m)):{},lid:_,loan:h?a(h):null,natural:w?a(w):null,manifest:o});return g.base=s,g.fetchImpl=t,n&&o.packs&&o.packs.dict&&await g.loadDict({onProgress:e,cache:i}),g.loadMs=q()-c,g}async loadDict({onProgress:s,cache:t="aksharalp-dict"}={}){const e=this.manifest&&this.manifest.packs&&this.manifest.packs.dict;if(!e)return!1;const n=e.dir.endsWith("/")?e.dir:e.dir+"/",i=this.fetchImpl||globalThis.fetch,c=t&&typeof caches<"u",a=async o=>{if(!o)return null;const l=e.sizes&&e.sizes[o]&&e.sizes[o].sha256||"",d=e.sizes&&e.sizes[o]&&e.sizes[o].bytes||0;if(c)try{const u=new URL(n+o,new URL(this.base.endsWith("/")?this.base:this.base+"/",globalThis.location&&globalThis.location.href)).href,f=await caches.open(t),p=u+"?v="+l.slice(0,16),m=await f.match(p);if(m){const w=new Uint8Array(await m.arrayBuffer());if(!d||w.length===d)return s&&s(o,"cache"),w;await f.delete(p)}const _=await i(u);if(!_.ok)throw new Error(`fetch ${u}: ${_.status}`);const h=tt(new Uint8Array(await _.arrayBuffer()),d,u);try{await f.put(p,new Response(h))}catch{}return s&&s(o,"network"),h}catch(u){if(String(u).includes("fetch "))throw u}const r=await P(this.base,n+o,i,d);return s&&s(o,"network"),r};try{const[o,l,d]=await Promise.all([a(e.files.lex),a(e.files.lm),a(e.files.params)]);return this.attachDict({lex:o,lm:l,params:d?JSON.parse(new TextDecoder().decode(d)):{}}),this.dictInfo={name:e.name||"dict",words:this.lm.N,bytes:e.bytes,gz:e.gz},!0}catch(o){return this.dictError=String(o&&o.message||o),!1}}attachDict({lex:s,lm:t=null,params:e={}}){this.p={...Q,...this.coreParams,...e},this._setLex(s,t),this.cache=new Map,this.hasDict=this.lm.N>0}_setLex(s,t){this.lm=new bt(s,t&&this.p.use_lm?t:null),this.lm.N||(this.p.cbeam=0,this.p.skel=!1);const e=this.lm.sortedWords,n=e.length&&e[0]===""?e.slice(1):e;this.trie=this.p.cbeam?new yt(n):null,this.skix=this.p.skel?new zt(this.lm):null}constructor({model:s,lex:t=null,lm:e=null,params:n={},lid:i=null,loan:c=null,natural:a=null,manifest:o=null}){this.coreParams=n,this.p={...Q,...n},this.manifest=o,this.model=new mt(H(s)),this._setLex(t,e),this.hasDict=this.lm.N>0,this.dictInfo=null,this.dictError=null,this.lid=i?new xt(H(i)):null,this.loan=c?Y(c):new Map,this.natural=a?Y(a):new Map,this.cache=new Map}info(){const s=this.manifest&&this.manifest.sizes?this.manifest.sizes:{},t=Object.values(this.manifest?this.manifest.files:{}).filter(e=>e&&s[e]);return{params:this.model.nParams+(this.lid?this.lid.nParams:0),xlitParams:this.model.nParams,lidParams:this.lid?this.lid.nParams:0,lexWords:this.lm.N,bytes:t.reduce((e,n)=>e+s[n].bytes,0),gz:t.reduce((e,n)=>e+s[n].gz,0),hasLID:!!this.lid,hasLM:this.lm.hasLM,hasDict:this.hasDict,dict:this.dictInfo,dictError:this.dictError,hasLoan:this.loan.size>0,hasNatural:this.natural.size>0}}candFeats(s){const t=s.toLowerCase(),e=this.cache.get(t);if(e)return e;const n=this.p,i=this.model,c=lt(t),a=i.encode(c),o=c.length,l=i.beam(a,o,{beam:n.beam,nbest:n.nbest}),d=n.cbeam&&this.trie?i.beam(a,o,{beam:n.beam,nbest:n.nbest,trie:this.trie}):[],r=new Map;l.forEach(([h,w],g)=>{h&&r.set(h,{sc:w,infree:1,frank:g,free0:+(g===0),beamEntry:1})});for(const[h,w]of d)h&&(r.has(h)||r.set(h,{sc:w,infree:0,frank:n.nbest,free0:0,beamEntry:1}),r.get(h).incon=1);if(n.variants){const h=[...r.keys()].sort((w,g)=>r.get(g).sc-r.get(w).sc).slice(0,4);for(const w of h)for(const[g,y]of Mt)if(w.includes(g)){const A=w.split(g).join(y);!r.has(A)&&this.lm.id(A)!==null&&r.set(A,{sc:i.scorePair(a,A),infree:0,frank:n.nbest,free0:0,variant:1})}}if(n.acro&&(kt.test(t)||n.acro_v&&t.length>=2&&t.length<=4)){const h=[...t].map(w=>K[w]).join("");r.has(h)?r.get(h).acro=1:r.set(h,{sc:i.scorePair(a,h),infree:0,frank:n.nbest,free0:0,acro:1})}if(n.visr&&ct.test(t)){const h=[...r.keys()].filter(w=>r.get(w).beamEntry).sort((w,g)=>r.get(g).sc-r.get(w).sc).slice(0,2);for(const w of h){const g=w.endsWith("\u0939")?w.slice(0,-1)+"\u0903":w.endsWith("\u0903")?null:w+"\u0903";g&&!r.has(g)&&r.set(g,{sc:i.scorePair(a,g),infree:0,frank:n.nbest,free0:0,visr:1})}}if(this.skix){const h=t.length+1,w=this.lm.words,g=this.lm.uni,y=this.skix.lookup(t,n.skel_k).map(A=>[A,at(t,rt(w[A]))/h-n.skel_wu*g[A]]).sort((A,k)=>A[1]-k[1]).slice(0,n.skel_m);for(const[A]of y){const k=w[A];r.has(k)?r.get(k).skel=1:r.set(k,{sc:i.scorePair(a,k),infree:0,frank:n.nbest,free0:0,skel:1})}}const u=[],f=new Set(ot(t)),p=t.length+1,m=ct.test(t),_=!/[aeiou]/.test(t);for(const[h,w]of r){const g=this.lm.id(h),y=[...h].length;u.push([h,[w.sc,w.sc/(y+1),+(g!==null),g!==null?this.lm.unigram(g):0,w.free0||0,w.infree||0,w.incon||0,+(h.includes("\u0949")||h.includes("\u0911")),+h.includes("\u0903"),+h.includes("\u093C"),+h.includes("\u0901"),w.acro||0,w.variant||0,(w.frank??n.nbest)/n.nbest,w.skel||0,+f.has(it(h)),at(t,rt(h))/p,0,0,0,y/p,w.visr||0,+(m&&h.endsWith("\u0903")),+(!!w.acro&&_)],g])}if(u.length){let h=-1/0,w=-1/0,g=null;for(const[,y]of u)h=Math.max(h,y[0]),w=Math.max(w,y[1]),y[2]&&(g=g===null?y[3]:Math.max(g,y[3]));for(const[,y]of u)y[17]=y[0]-h,y[18]=y[1]-w,y[19]=y[2]?y[3]-(g??0):0}return this.cache.size>2e4&&this.cache.clear(),this.cache.set(t,u),u}score(s){const t=this.p;if(!t.w&&!t.w1)return s[0]+.2*(s[2]?s[3]:-25);const e=t.mu.length,n=new Array(e);for(let c=0;c<e;c++)n[c]=(s[c]-t.mu[c])/t.sd[c];if(t.w1){let c=t.b2||0;for(let a=0;a<t.w1.length;a++){const o=t.w1[a];let l=t.b1[a];for(let d=0;d<e;d++)l+=o[d]*n[d];c+=t.w2[a]*Math.tanh(l)}return c}let i=0;for(let c=0;c<Math.min(e,t.w.length);c++)i+=t.w[c]*n[c];return i}word(s,t=8){return J.test(s)?this.candFeats(s).map(([n,i])=>({text:n,score:this.score(i)})).sort((n,i)=>i.score-n.score).slice(0,t):[{text:s,score:0}]}decodeSlots(s,t=!0){const e=this.p,n=this.lm,i=[];for(const[d,r]of s)if(d==="x"){let u=this.candFeats(r).map(([f,p,m])=>[f,this.score(p),m]);if(u.length){const f=Math.max(...u.map(m=>m[1])),p=f+Math.log(u.reduce((m,_)=>m+Math.exp(_[1]-f),0));u=u.map(([m,_,h])=>[m,e.w_ch_s*(_-p),h]).sort((m,_)=>_[1]-m[1]).slice(0,12)}i.push(u.length?u:[[r,0,null]])}else{const u=O.test(r)?n.id(r)??-1:-2;i.push([[r,0,u]])}const c=i.map(d=>d.map(([r,u])=>[r,u]));if(!t||!e.use_lm||!n.hasLM)return{out:i.map(d=>d.reduce((r,u)=>u[1]>r[1]?u:r)[0]),lat:c};let a=[[0,null,[]]];for(const d of i){if(d[0][2]===-2&&d.length===1){a=a.map(([u,f,p])=>[u,f,p.concat([d[0][0]])]);continue}const r=new Map;for(const[u,f,p]of a)for(const[m,_,h]of d){const w=h===null||h===-1?null:h,g=u+_+e.w_lm*(n.logp(f,w)-(w===null?e.oov_pen_s:0));(!r.has(m)||g>r.get(m)[0])&&r.set(m,[g,w,p.concat([m])])}a=[...r.values()].sort((u,f)=>f[0]-u[0]).slice(0,16)}let o=null,l=-1/0;for(const d of a){const r=d[0]+e.w_lm*n.logp(d[1],n.N);r>l&&(l=r,o=d)}return{out:o[2],lat:c}}convertPlain(s){const t=[...s.matchAll(G)].map(c=>[c.index,c.index+c[0].length,c[0]]),{out:e}=this.decodeSlots(t.map(([,,c])=>J.test(c)?["x",c]:["fix",c]));e.length&&e[e.length-1]==="."&&(e[e.length-1]="\u0964");let n="",i=0;return t.forEach(([c,a],o)=>{n+=s.slice(i,c)+e[o],i=a}),n+s.slice(i)}transliterate(s,{mode:t="conversational",k:e=5,overrides:n={},context:i=!0}={}){const c=q(),a=t==="strict"?"strict":"conversational",o=[...s.matchAll(G)].map(y=>[y.index,y.index+y[0].length,y[0]]),l=o.map(y=>y[2]),d=this.lid?this.lid.tag(l):l.map(y=>O.test(y)?"DEV":T.test(y)?this.loan.has(y.toLowerCase())?"EN":"HI":"OTHER"),r=[],u=[];let f=!1;for(let y=0;y<l.length;y++){const A=l[y],k=d[y],x=l[y+1]||"",N=A.toLowerCase(),S=n[o[y][0]];if(S!==void 0){r.push(["fix",S]),u.push("user"),f=!1;continue}if(!T.test(A)||I.test(A)||x==="("){A==="."&&(y===l.length-1||/^[A-Z]/.test(l[y+1]||""))?r.push(["fix","\u0964"]):r.push(["fix",A]),u.push(O.test(A)?"devanagari":I.test(A)?"keep":"punct"),f=!1;continue}const z=Et.has(N)||f&&/^[A-Z]/.test(A);if(f=z,a==="strict"&&z){r.push(["fix",A]),u.push("brand");continue}if(t==="natural"&&k==="EN"&&this.natural.has(N)){r.push(["fix",this.natural.get(N)]),u.push("glossary");continue}if(Nt(A)&&A.length>=2&&A.length<=5&&!this.loan.has(N)){r.push(["fix",[...N].map(v=>vt[v]).join("")]),u.push("acronym");continue}if((k==="EN"||z)&&this.loan.has(N)){r.push(["fix",this.loan.get(N)]),u.push("loanword");continue}r.push(["x",A]),u.push("model")}const{out:p,lat:m}=this.decodeSlots(r,i);let _="",h=0;o.forEach(([y,A],k)=>{_+=s.slice(h,y)+p[k],h=A}),_=(_+s.slice(h)).trim();const w=o.map(([y,A,k],x)=>{let N=[];if(r[x][0]==="x"){const z=m[x][0]?m[x][0][1]:0;N=m[x].slice(0,e).map(([v,E])=>({text:v,score:Math.exp(E-z)}))}else if(T.test(k)&&!I.test(k)){N=[{text:p[x],score:1}];for(const z of this.word(k,e))z.text!==p[x]&&N.push({text:z.text,score:null});p[x]!==k&&N.push({text:k,score:null}),N=N.slice(0,e)}const S=T.test(k)||O.test(k);return{text:k,start:y,end:A,output:p[x],lang:S?d[x]:"OTHER",source:u[x],candidates:N}});let g=null;if(this.lid&&l.length)try{g=this.lid.mode(l)}catch{g=null}return{output:_,tokens:w,sentence:g,ms:q()-c}}}async function Vt(b,s={}){const t=await import("node:fs/promises"),e=await import("node:path"),n=async i=>{try{const c=await t.readFile(e.resolve(i));return new Response(c)}catch{return new Response(null,{status:404})}};return U.load(b,{fetch:n,cache:!1,...s})}var It=U;export{U as AksharaLP,U as BhashaCore,Ct as FEATS,xt as LID,bt as LexLM,G as TOKEN_RE,mt as XlitModel,It as default,rt as devRom,it as devSkel,at as lev,Vt as loadFromDir,H as parseWeights,ot as romSkels};

aksharalp.mjs

@@ -405,11 +405,14 @@ const DEFAULTS = { beam: 8, nbest: 8, cbeam: 8, variants: true, acro: true, w: n
 const parseTSV = (txt) => { const m = new Map(); for (const l of txt.split("\n")) { const i = l.indexOf("\t"); if (i > 0) m.set(l.slice(0, i).trim(), l.slice(i + 1).trim()); } return m; };
 const now = () => (typeof performance !== "undefined" ? performance.now() : Date.now());
 
-async function fetchBytes(base, name, fetchImpl) {
+// Sizes are checked against the manifest, never against Content-Length (a CDN redirect or an auth error page has its
+// own, unrelated length); a wrong-size body (e.g. an HTML/plain-text error page served with 200) is rejected.
+const checkSize = (b, expect, url) => { if (expect && b.length !== expect) throw new Error(`fetch ${url}: got ${b.length} bytes, manifest says ${expect}`); return b; };
+async function fetchBytes(base, name, fetchImpl, expect = 0) {
   const url = base.endsWith("/") ? base + name : base + "/" + name;
   const r = await fetchImpl(url); if (!r.ok) throw new Error(`fetch ${url}: ${r.status}`);
   if (name.endsWith(".gz") && typeof DecompressionStream !== "undefined") return new Uint8Array(await new Response(r.body.pipeThrough(new DecompressionStream("gzip"))).arrayBuffer());
-  return new Uint8Array(await r.arrayBuffer());
+  return checkSize(new Uint8Array(await r.arrayBuffer()), expect, url);
 }
 
 // ------------------------------------------------------------------ consonant-skeleton index (bhc/skel.py; tables generated from it)
@@ -492,7 +495,7 @@ export class BhashaCore {
   static async load(base, { fetch: fetchImpl = globalThis.fetch, onProgress, dict = true, cache = "aksharalp-dict" } = {}) {
     const t0 = now(); const txt = (b) => new TextDecoder().decode(b);
     let man; try { man = JSON.parse(txt(await fetchBytes(base, "manifest.json", fetchImpl))); } catch { man = { files: { model: "xlit.q4", lex: "lex.bin", lm: "lm.bin", params: "params.json", lid: "lid.int8", loan: "loan.tsv", natural: "natural_hi.tsv" } }; }
-    const F = man.files, get = async (n, opt) => { if (!n) return null; try { const b = await fetchBytes(base, n, fetchImpl); onProgress && onProgress(n); return b; } catch (e) { if (opt) return null; throw e; } };
+    const F = man.files, SZ = man.sizes || {}, get = async (n, opt) => { if (!n) return null; try { const b = await fetchBytes(base, n, fetchImpl, SZ[n] && SZ[n].bytes); onProgress && onProgress(n); return b; } catch (e) { if (opt) return null; throw e; } };
     const [mb, lexb, lmb, pb, lidb, loanb, natb] = await Promise.all([get(F.model), get(F.lex, true), get(F.lm, true), get(F.params, true), get(F.lid, true), get(F.loan, true), get(F.natural, true)]);
     const e = new BhashaCore({ model: mb, lex: lexb, lm: lmb, params: pb ? JSON.parse(txt(pb)) : {}, lid: lidb, loan: loanb ? txt(loanb) : null, natural: natb ? txt(natb) : null, manifest: man });
     e.base = base; e.fetchImpl = fetchImpl;
@@ -507,18 +510,18 @@ export class BhashaCore {
     const useCache = cache && typeof caches !== "undefined";
     const get = async (n) => {
       if (!n) return null;
-      const ver = (pk.sizes && pk.sizes[n] && pk.sizes[n].sha256) || "";
+      const ver = (pk.sizes && pk.sizes[n] && pk.sizes[n].sha256) || "", expect = (pk.sizes && pk.sizes[n] && pk.sizes[n].bytes) || 0;
       if (useCache) {
         try {
           const url = new URL(dir + n, new URL(this.base.endsWith("/") ? this.base : this.base + "/", globalThis.location && globalThis.location.href)).href;
           const c = await caches.open(cache), key = url + "?v=" + ver.slice(0, 16), hit = await c.match(key);
-          if (hit) { onProgress && onProgress(n, "cache"); return new Uint8Array(await hit.arrayBuffer()); }
+          if (hit) { const hb = new Uint8Array(await hit.arrayBuffer()); if (!expect || hb.length === expect) { onProgress && onProgress(n, "cache"); return hb; } await c.delete(key); }
           const r = await fetchImpl(url); if (!r.ok) throw new Error(`fetch ${url}: ${r.status}`);
-          const b = new Uint8Array(await r.arrayBuffer()); try { await c.put(key, new Response(b)); } catch (_) {}
+          const b = checkSize(new Uint8Array(await r.arrayBuffer()), expect, url); try { await c.put(key, new Response(b)); } catch (_) {}   // stored as a fresh, non-redirected response
           onProgress && onProgress(n, "network"); return b;
         } catch (e) { if (String(e).includes("fetch ")) throw e; }
       }
-      const b = await fetchBytes(this.base, dir + n, fetchImpl); onProgress && onProgress(n, "network"); return b;
+      const b = await fetchBytes(this.base, dir + n, fetchImpl, expect); onProgress && onProgress(n, "network"); return b;
     };
     try {
       const [lexb, lmb, pb] = await Promise.all([get(pk.files.lex), get(pk.files.lm), get(pk.files.params)]);