[
    {
        "paper_id": "1704.00864.json",
        "image_id": "figure_2",
        "image_path": [
            "/home/yz979/scratch/chengye/scimolmo/test-images-tables/dataset_imagestables/1704.00864/images/lih.eps"
        ],
        "caption": "Initiator error convergence for the five lowest energy states of LiH in an aug-cc-pVQZ basis, at an internuclear distance of $1.5957$\\AA~as the number of walkers in each distribution is increased. Results are shifted relative to their values at the largest walker population considered, therefore approximately representing the initiator error. (a) Dipole moments. (b) Transition dipole moments from the ground state. (c) Energy calculated from a trial estimator, $E_{\\textrm{Trial}}$. (d) Energy calculated from the RDM estimator, $E_{\\textrm{RDM}}$. $N_w$ denotes the number of walkers for \\emph{each} state and replica sampled. Simulations were typically averaged over 5 simulations to obtain error bars.",
        "classify": "Chart",
        "section_info": "5 Results\n\\section{Results}\n\\label{sec:results}\n\nAs an initial test of these ideas, we consider the calculation of dipole moments, transition dipole moments, and oscillator strengths for low-lying states of small diatomic molecules. These quantities are of great importance for understanding various properties of molecular systems. The oscillator strength in particular is required to explain optical spectra, as it determines the probabilities of absorption and emission of photons coupling different electronic states. Nonetheless, dipole moments are challenging to calculate accurately, even for small molecules, because they are very sensitive to the quality of the wave function and single-particle basis set used, generally requiring many diffuse orbitals for an accurate description, with far greater basis set sensitivity than the energy\\cite{Green1974}.\n\nWe therefore begin by considering the LiH and BH molecules in aug-cc-pVDZ, aug-cc-pVTZ and aug-cc-pVQZ, containing $32$, $69$  and $126$ spatial orbitals respectively. The aug-cc-pVQZ basis 2-RDM was unobtainable in the previous RDM implementation, despite the small molecular size. We then consider the MgO molecule in an aug-cc-pVDZ basis set. We note that while the calculation of dipole moments only requires the 1-RDM, for these calculations we obtain the 1-RDM by contracting the 2-RDM, which we also use to calculate the energy using the estimator\n\\begin{equation}\n(E_{\\textrm{RDM}})_n = \\frac{ \\textrm{Tr} \\big[ \\hat{H} \\; \\hat{\\Gamma}^n \\big] }{ \\textrm{Tr} \\big[ \\hat{\\Gamma}^n \\big] }.\n\\label{eq:rdm_energy}\n\\end{equation}\nTherefore, the following is a good test of the newly-introduced ideas, as well as providing further insight into the effect of the initiator adaptation for different estimators and excited states.\n\nThe dipole moment for the state $|\\Phi^n\\ket$ is defined by\n\\begin{equation}\n\\bs{\\mu}_{n} = \\sum_{pq} \\gamma_{p,q}^{n} \\bra p | \\hat{\\bs{r}} | q \\ket.\n\\end{equation}\nwhile a transition dipole moment, $\\bs{t}_{nm}$, is defined by Eq.~(\\ref{eq:trans_dip_mom}), and the corresponding oscillator strength by\n\\begin{equation}\nf_{nm} = \\frac{2}{3} \\Delta E_{nm} |\\bs{t}_{nm}|^2,\n\\end{equation}\nfor an energy gap of $\\Delta E_{nm}$ between states $|\\Phi^n\\ket$ and $|\\Phi^m\\ket$.\n\n\\begin{figure*}[t!]\n\\includegraphics{lih.eps}\n\\caption{Initiator error convergence for the five lowest energy states of LiH in an aug-cc-pVQZ basis, at an internuclear distance of $1.5957$\\AA~as the number of walkers in each distribution is increased. Results are shifted relative to their values at the largest walker population considered, therefore approximately representing the initiator error. (a) Dipole moments. (b) Transition dipole moments from the ground state. (c) Energy calculated from a trial estimator, $E_{\\textrm{Trial}}$. (d) Energy calculated from the RDM estimator, $E_{\\textrm{RDM}}$. $N_w$ denotes the number of walkers for \\emph{each} state and replica sampled. Simulations were typically averaged over 5 simulations to obtain error bars.}\n\\label{fig:lih_init}\n\\end{figure*}\n\n\\begin{table*}[t]\n\\begin{center}{\\footnotesize\n\\begin{tabular}{@{\\extracolsep{4pt}}lccccc@{}}\n\\hline\n\\hline\nBasis & State $n$ & Energy gap ($\\Delta E_{0n}$) & Dipole moment ($\\mu_n$) & Transition dipole moment ($t_{0n}$) & Oscillator strength ($f_{0n}$) \\\\\n\\hline\naug-cc-pVDZ & 0 $\\;$ (${}^1\\Sigma^+$)  & -            & -2.3251372(2) & -            & -           \\\\\n            & 1 $\\;$ (${}^1\\Sigma^+$)  & 0.130434(1)  &  2.01947(4)   & 0.965189(7)  & 0.081007(1) \\\\\n            & 2 $\\;$ (${}^1\\Sigma^+$)  & 0.2149799(6) & -3.3543(9)    & 0.37471(1)   & 0.020123(1) \\\\\n            & 3 $\\;$ (${}^1\\Sigma^+$)  & 0.229077(4)  &  5.0832(8)    & 0.09126(8)   & 0.001271(2) \\\\\n            & 4 $\\;$ (${}^1\\Sigma^+$)  & 0.246350(3)  & -0.2958(3)    & 0.56074(2)   & 0.051639(4) \\\\\n\\hline\naug-cc-pVTZ & 0 $\\;$ (${}^1\\Sigma^+$) & -            & -2.306440(9)  & -            & -           \\\\\n            & 1 $\\;$ (${}^1\\Sigma^+$) & 0.132458(3)  &  2.02541(7)   & 0.93538(2)   & 0.077262(4) \\\\\n            & 2 $\\;$ (${}^1\\Sigma^+$) & 0.216705(6)  & -3.794(1)     & 0.41146(2)   & 0.024459(2) \\\\\n            & 3 $\\;$ (${}^1\\Sigma^+$) & 0.230621(2)  &  5.533(1)     & 0.07042(8)   & 0.000762(2) \\\\\n            & 4 $\\;$ (${}^1\\Sigma^+$) & 0.246520(2)  & -0.6235(7)    & 0.693170(7)  & 0.078966(2) \\\\\n\\hline\naug-cc-pVQZ & 0 $\\;$ (${}^1\\Sigma^+$) & -            & -2.30168(3)   & -            & -           \\\\\n            & 1 $\\;$ (${}^1\\Sigma^+$) & 0.132943(7)  &  2.0188(1)    & 0.92658(4)   & 0.076093(7) \\\\\n            & 2 $\\;$ (${}^1\\Sigma^+$) & 0.217616(7)  & -3.696(2)     & 0.3984(1)    & 0.02303(1)  \\\\\n            & 3 $\\;$ (${}^1\\Sigma^+$) & 0.231229(2)  &  6.211(2)     & 0.1083(2)    & 0.001809(6) \\\\\n            & 4 $\\;$ (${}^1\\Sigma^+$) & 0.242846(9)  & -1.998(2)     & 0.6201(5)    & 0.06224(9)  \\\\\n\\hline\n\\hline\n\\end{tabular}\n}\n\\caption{Final converged estimates for the LiH molecule at an internuclear distance of $1.5957$\\AA. Results are for the five lowest energy states in the $A_1$ irrep of the $C_{2v}$ point group, with $M_S=0$ and $S=\\textrm{even}$ quantum numbers (which happen to all be ${}^1\\Sigma^+$ states). $n=0$ refers to the ground state, $n>1$ to excited states. Numbers in parentheses denote stochastic error, not initiator error. Energy gaps ($\\Delta E_{1n}$) were calculated using RDM-based energy estimates, Eq.~(\\ref{eq:rdm_energy}). Integrals were generated using the PySCF program\\cite{pyscf}. In the small aug-cc-pVDZ, all results were verified against exact FCI results obtained from PySCF (not shown here).}\n\\label{tab:lih}\n\\end{center}\n\\end{table*}\n\nFor all simulations, the intial restricted Hartree--Fock (RHF) calculation was performed by PySCF\\cite{pyscf}. Integrals from PySCF were then passed to our FCIQMC program, \\url{NECI}, for the main calculation, which output one and two body density matrices. These were then contracted with integrals from PySCF to calculate final dipole moment estimates. Energy estimates were calculated on-the-fly in \\url{NECI}.\n\nThe five lowest energy states were calculated for LiH and BH, and the four lowest states of MgO, considering only states with $M_s=0$ and using the $A_1$ irreducible representation (irrep) of the $C_{2v}$ point group. Also, time-reversal symmetrized functions\\cite{Smeyers1973} were used as the many-particle basis states, therefore restricting the total spin quantum number, $S$, to be even, and thus removing triplet states. In all cases, the FCIQMC simulation time step was varied in the initial iterations so as to prevent ``bloom'' events, where many walkers can be created in a single spawning event (which often leads to large initiator error).\n\nWe also note that in generating excitations for the walker spawning step, we use an approach that greatly improves efficiency compared to the uniform sampling used in early FCIQMC results\\cite{Booth2009}. In this approach, the pair of orbital labels from which electrons are excited, $(i,j)$, are chosen uniformly, while the orbitals excited to, $(a,b)$, are selected with probabilities drawn from a Cauchy-Schwarz distribution, namely $p(ab|ij) \\propto \\sqrt{\\langle ia|ia \\rangle \\langle jb|jb \\rangle}$.\\cite{Smart_unpublished} Another approach to select connections efficiently was considered by Holmes \\emph{et. al.}\\cite{Holmes2016}, but not used here.\n\nAll simulations used the semi-stochastic adaptation to reduce stochastic errors\\cite{Petruzielo2012, Blunt2015}. For the LiH molecule the deterministic space consisted of all configurations up to and including double excitations from the Hartree--Fock determinant. For the BH and MgO molecules the deterministic space was formed from the $10^4$ most populated configurations across all wave functions sampled, once the simulations were deemed to have largely converged, using the approach described in Ref.~(\\onlinecite{Blunt2015}).\n\n\\subsection{LiH}\n\nSimulations on LiH were performed using between $1.25 \\times 10^4$ and $10^6$ walkers per simulation (i.e., for each state and replica sampled), in order to converge initiator error for all states. Density matrices were typically averaged over $10^5$ iterations, once convergence was deemed to have been reached for all states and all estimators. These entire simulations were then repeated five times with different initial RNG seeds, and the results averaged in order to calculate error estimates.\n\nFigure~\\ref{fig:lih_init} shows initiator convergence for LiH in the aug-cc-pVQZ basis set, for the lowest five energy eigenstates, and for four different estimators: dipole moments, transition dipoles moments, and energies calculated from both the RDM-based energy estimator, Eq.~(\\ref{eq:rdm_energy}), and from a trial wave function-projected estimator:\n\\begin{equation}\n(E_{\\textrm{Trial}})_n = \\frac{ \\bra \\Psi_{\\textrm{Trial}}^n | \\hat{H} | \\Psi^n \\ket }{ \\bra \\Psi_{\\textrm{Trial}}^n | \\Psi^n \\ket }.\n\\label{eq:trial_energy}\n\\end{equation}\nHere, $| \\Psi_{\\textrm{Trial}}^n \\ket$ is a trial wave function designed to have a large overlap with the exact state $| \\Phi^n \\ket$. We have discussed the use of such trial wave function estimators in excited-state FCIQMC in Ref.~(\\onlinecite{Blunt2015_3}). To generate $| \\Psi_{\\textrm{Trial}}^n \\ket$, we calculate the configuration interaction singles and doubles (CISD) wave functions for the lowest fifteen energy states. Then, once convergence of all FCIQMC simulations is deemed to have been reached, we assign each simulation one trial wave function by choosing the CISD solution with the largest overlap in each case. The reason for obtaining more CISD solutions than FCIQMC simulations is that CISD solutions can have a different energy ordering to FCI solutions. Averaging of each $E_{\\textrm{Trial}}$ estimate was performed from roughly the same point that RDM sampling began, and so both RDM and trial energy estimates are obtained from a similar number of iterations, usually $10^5$.\n\nThe initiator-FCIQMC estimates in Figure~\\ref{fig:lih_init} are all plotted relative to their values at the largest walker population considered, $N_{w}=10^6$. Here, convergence has been largely reached in all cases, and so the figures effectively plot initiator error against walker population. Reassuringly, initiator error in energy estimates is incredibly small for both estimators and for all states. Indeed, the largest error at the smallest walker population tested is less than $\\sim 0.5$ m$E_\\textrm{h}$ for $E_{\\textrm{Trial}}$.\n\nInterestingly, initiator error in $E_{\\textrm{RDM}}$ is much smaller than in $E_{\\textrm{Trial}}$. This is a trend that we have often observed, although exceptions do occur (and in the limit of an exact $| \\Psi_{\\textrm{Trial}}^n \\ket$, the initiator error is zero). Initiator error in the $E_{\\textrm{RDM}}$ energies are variational in all cases within stochastic errors, while it is not strictly enforced (though common) for this to also be the case for $E_{\\textrm{Trial}}$. For RDM-based energy estimates, this variationality is effectively ensured by the Hylleraas-Undheim-McDonald theorem\\cite{Hylleraas1930, McDonald1933}, which is expected to approximately hold for FCIQMC-sampled wave functions. Initiator error is larger for excited states, as previously observed\\cite{Blunt2015_3}. This is expected due to the more multi-configurational nature of excited states. It remains to be seen whether orbital optimization can increase this rate of convergence for excited states. Random errors however are larger in the RDM-based energy estimates, which is expected due to the fact that two uncorrelated simulations (from the two replicas) contribute to this quantity. However, error bars are extremely small in all cases here, always being smaller than $10^{-2}$ m$E_{\\textrm{h}}$.\n\n\\begin{figure*}[t!]\n\\includegraphics{bh.eps}\n\\caption{Initiator convergence for the five lowest energy states of BH in an aug-cc-pVTZ basis, at an internuclear distance of $1.2324$\\AA. Results are shifted relative to their values at the largest walker population considered, therefore approximately representing the initiator error. (a) Dipole moments. (b) Transition dipole moments from the ground state. (c) Energy calculated from a trial estimator, $E_{\\textrm{Trial}}$. (d) Energy calculated from the RDM estimator, $E_{\\textrm{RDM}}$. $N_w$ denotes the number of walkers for \\emph{each} state and replica sampled. Simulations were typically averaged over 5 simulations to obtain error bars.}\n\\label{fig:bh_init}\n\\end{figure*}\n\n\\begin{table*}[t]\n\\begin{center}{\\footnotesize\n\\begin{tabular}{@{\\extracolsep{4pt}}lccccc@{}}\n\\hline\n\\hline\nBasis & State $n$ & Energy gap ($\\Delta E_{0n}$) & Dipole moment ($\\mu_n$) & Transition dipole moment ($t_{0n}$) & Oscillator strength ($f_{0n}$) \\\\\n\\hline\naug-cc-pVDZ & 0 $\\;$ (${}^1\\Sigma^+$)     & -           &  0.528082(7) & -           & -           \\\\\n            & 1 $\\;$ ($\\; {}^1\\Delta \\;$) & 0.216230(3) & -0.18983(3)  & 0.0         & 0.0         \\\\\n            & 2 $\\;$ (${}^1\\Sigma^+$)     & 0.23727(1)  & -1.4146(5)   & 0.93478(3)  & 0.13822(1)  \\\\\n            & 3 $\\;$ (${}^1\\Sigma^+$)     & 0.257587(4) & -0.3219(3)   & 0.2102(1)   & 0.007590(9) \\\\\n            & 4 $\\;$ (${}^1\\Sigma^+$)     & 0.282665(1) &  3.5459(1)   & 0.44725(4)  & 0.037696(7) \\\\\n\\hline\naug-cc-pVTZ & 0 $\\;$ (${}^1\\Sigma^+$)     & -           &  0.54561(2) & -          & -           \\\\\n            & 1 $\\;$ ($\\; {}^1\\Delta \\;$) & 0.211482(6) & -0.19271(7) & 0.0        & 0.0         \\\\\n            & 2 $\\;$ (${}^1\\Sigma^+$)     & 0.238668(8) & -1.2943(5)  & 0.88508(5) & 0.12464(1)  \\\\\n            & 3 $\\;$ (${}^1\\Sigma^+$)     & 0.253574(4) & -0.4973(6)  & 0.1454(2)  & 0.00358(1)  \\\\\n            & 4 $\\;$ (${}^1\\Sigma^+$)     & 0.283481(2) &  3.4088(2)  & 0.35740(7) & 0.024141(9) \\\\\n\\hline\naug-cc-pVQZ & 0 $\\;$ (${}^1\\Sigma^+$)     & -          &  0.54914(6) & -         & -          \\\\\n            & 1 $\\;$ ($\\; {}^1\\Delta \\;$) & 0.21059(2) & -0.1968(3)  & 0.0       & 0.0        \\\\\n            & 2 $\\;$ (${}^1\\Sigma^+$)     & 0.23876(3) & -1.268(3)   & 0.8704(3) & 0.1206(1)  \\\\\n            & 3 $\\;$ (${}^1\\Sigma^+$)     & 0.25261(3) & -0.504(3)   & 0.139(1)  & 0.00327(7) \\\\\n            & 4 $\\;$ (${}^1\\Sigma^+$)     & 0.28289(1) &  3.2889(9)  & 0.3138(1) & 0.01857(2) \\\\\n\\hline\n\\hline\n\\end{tabular}\n}\n\\caption{Final converged estimates for the BH molecule at an internuclear distance of $1.2324$\\AA. Results are for the five lowest energy states in the $A_1$ irrep of the $C_{2v}$ point group, with $M_S=0$ and $S=\\textrm{even}$ quantum numbers. $n=0$ refers to the ground state, $n>1$ to excited states. Numbers in parentheses denote stochastic error, not initiator error. Energy gaps ($\\Delta E_{1n}$) were calculated using RDM-based energy estimates, Eq.~(\\ref{eq:rdm_energy}). Integrals were generated using the PySCF program\\cite{pyscf}.}\n\\label{tab:bh}\n\\end{center}\n\\end{table*}\n\nThe calculation of dipole moments provides a more interesting test, due to their greater dependence on more highly-excited determinants and diffuse single-particle orbitals. The relative initiator error is much larger, particularly for certain excited states (i.e. $\\mu_2$ and $\\mu_3$). The transition dipole moments considered involve transitions from the ground ($n=0$) state to excited ($n>1$) states. Because they always involve the ground state, it is to be expected that they have smaller relative initiator and stochastic error, compared to the corresponding non-transition dipole moment (i.e. $t_{0n}$ compared to $\\mu_n$). This expectation is borne out in the results, with initiator and stochastic error in $t_{0n}$ often being $\\sim 5$ times smaller than for $\\mu_n$. For the calculation of dipole moments from FCIQMC-sampled RDMs, relative stochastic errors are clearly much larger than for energies, and so the use of the semi-stochastic adaptation is of great importance here, whereas its use can be somewhat unnecessary in small ground-state energy calculations.\n\nClearly, the accurate calculation of dipole moments is more challenging than energies, requiring larger walker populations to obtain similar relative errors. However, this is not uniquely a feature of the initiator approximation in FCIQMC, but is equally true in other approximate methods, where properties such as the dipole moment are far more sensitive to the basis set and quality of the wavefunction than ground state energetics. That we are able to observe systematic converge of these quantities, with respect to a single simulation parameter, is reassuring.\n\nTable~\\ref{tab:lih} gives final results for the aug-cc-pV$X$Z basis sets, with $X=2,3,4$. Results in the small $X=2$ basis were fully converged at the smallest walker populations considered, $N_w = 1.25 \\times 10^4$, as confirmed by comparison to FCI results from the PySCF program. As expected, dipole moments vary quite substantially with basis set, particularly for the second, third and fourth excited states, demonstrating the importance of large basis sets with diffuse functions. Errors in brackets denote stochastic error bars, not initiator error, which is larger. However, given the careful convergence of initiator error, as shown in Figure~\\ref{fig:lih_init}, we expect dipole moments to be converged to around $10^{-3}e a_0$ in most cases, and energies to be converged \\emph{substantially} beyond chemical accuracy.\n\n\\subsection{BH}\n\nFigure~\\ref{fig:bh_init} shows results for BH in the aug-cc-pVTZ basis set and at an internuclear distance of $1.2324$\\AA, demonstrating similar initiator convergence plots to those in Figure~\\ref{fig:lih_init}. Here, results used between $1.25 \\times 10^4$ and $2 \\times 10^6$ walkers per simulation. RDM estimators and $E_{\\textrm{Trial}}$ were averaged over $5 \\times 10^4$ iterations, once convergence was achieved for all states and estimators. Here, instead of using CISD solutions as trial wave functions for $E_{\\textrm{Trial}}$, a slightly different approach was used: a ``trial space'' was defined as consisting of the $2 \\times 10^3$ most populated configurations across all simulations, once convergence had been approximately reached. Trial wave functions were then obtained as the eigenstates of $\\hat{H}$ within this subspace. This is similar to the approach to generate the deterministic space, as described above\\cite{Blunt2015}, and allows important basis states to be picked, while allowing an inexpensive calculation to determine each $| \\Psi_{\\textrm{Trial}}^n \\ket$.\n\nResults contain the same features as observed for LiH. Initiator error in the energy estimates are extremely small in all cases, particularly for estimates obtained from contraction of the RDM, and initiator convergence always occurs variationally. Stochastic error bars are larger for $E_{\\textrm{RDM}}$, as well as for excited states, but always extremely small. For dipole moments, similar trends also occur. Initiator and stochastic relative errors for the dipole moment are very small for the ground and first excited states ($\\mu_0$ and $\\mu_1$) and for the corresponding transition dipole moment ($t_{01}$) even at small walker populations. However, results for higher excited states contain larger errors, although we once again observe that errors in $t_{0n}$ are smaller than errors in $\\mu_n$ for each $n$, presumably because of the involvement of the ground state, which is well converged at lower walker populations, in each of the transition dipole moments considered.\n\nTable~\\ref{tab:bh} shows final results in aug-cc-pV$X$Z basis sets, for $X=2,3,4$. Results for $X=2$ used $2 \\times 10^5$ walkers per simulation, while results for $X=3$ and $X=4$ results used $2 \\times 10^6$ walkers per simulation. The expected strong dependence of dipole moments on the basis set is once again observed. This is particularly true for the second, third and fourth excited states ($n=2,3,4$). We note that these three states also contained the largest initiator error at small walker populations, as seen in Figure~\\ref{fig:bh_init}. This is probably not a coincidence, since the initiator approximation will inevitably result in a poorer description of highly excited regions of the wave function, presumably including excitations into high-energy diffuse functions, which appear important for accurate calculation of dipole moments for these particular states. Despite larger initiator error than for energy estimates, there is still a substantial undersampling of the space here, using $2 \\times 10^6$ walkers for a space size of $\\sim 7 \\times 10^9$ for the aug-cc-pVQZ basis, even for this small molecule, with benefits of Monte Carlo sampling typically increasing with system size.\n\n\\subsection{MgO}\n\n\\begin{figure*}[t!]\n\\includegraphics{mgo.eps}\n\\caption{Initiator convergence for dipole moments (left) and energies (right), for MgO in an aug-cc-pVDZ basis set, at an internuclear distance of $1.749$\\AA, and with 4 core electrons frozen. The four lowest-energy states are considered in the $A_1$ irrep of $C_{2v}$ and with $S=\\textrm{even}$ enforced (all ${}^1\\Sigma^+$ states). Energies are calculated from both RDM ($E_{\\textrm{RDM}}$) and trial wave function ($E_{\\textrm{Trial}}$) based estimates, and become equal to good accuracy at large walker number, $N_w$. Dipole moments appear mostly converged at $N_w=3.2 \\times 10^7$, except for $\\mu_1$. Error bars are only available for $N_w < 10^6$, but are small by this point and should only decrease in magnitude for larger walker populations.}\n\\label{fig:mgo_init}\n\\end{figure*}\n\n\\begin{table*}[t]\n\\begin{center}{\\footnotesize\n\\begin{tabular}{@{\\extracolsep{4pt}}c|ccc|ccc@{}}\n\\hline\n\\hline\nState $n$  & \\multicolumn{3}{c|}{ Energy/$E_{\\textrm{h}}$ } & \\multicolumn{3}{c}{ Dipole moment ($\\mu_n$) /$ea_0$ } \\\\\n\\hline\n & CCSD & CCSDT & FCIQMC & CCSD & CCSDT & FCIQMC \\\\\n\\hline\n0 $\\;$ (${}^1\\Sigma^+$) $\\;$ &  -274.632  &  -274.651  &  -274.654 &  2.590  &  2.398  &  2.382 \\\\\n1 $\\;$ (${}^1\\Sigma^+$) $\\;$ &  -274.531  &  -274.559  &  -274.564 &  1.811  &  2.008  &  2.289 \\\\\n2 $\\;$ (${}^1\\Sigma^+$) $\\;$ &  -274.480  &  -274.514  &  -274.517 &  0.297  &  0.847  &  1.154 \\\\\n3 $\\;$ (${}^1\\Sigma^+$) $\\;$ &  -274.440  &  -274.478  &  -274.480 & -0.366  &  0.529  &  1.198 \\\\\n\\hline\n\\hline\n\\end{tabular}\n}\n\\caption{Energies and dipole moments for MgO in an aug-cc-pVDZ basis set, at an internuclear distance of $1.749$\\AA, and with 4 core electrons frozen at the Hartree--Fock level. The four lowest-energy states are considered in the $A_1$ irrep of $C_{2v}$ and with $S=\\textrm{even}$ enforced (all ${}^1\\Sigma^+$ states). Error bars on FCIQMC results are not given, but are smaller than the order to which results are presented. FCIQMC energies are taken from the RDM-based estimates, $E_{\\textrm{RDM}}$. CCSD and CCSDT values were obtained from NWChem\\cite{NWChem}.}\n\\label{tab:mgo}\n\\end{center}\n\\end{table*}\n\nTo study a more challenging problem, we consider the calculation of energies and dipole moments for the MgO molecule, at its ground state equilibrium separation of $1.749$\\AA, and with 4 core electrons frozen at the Hartree--Fock level. Thus, a total of 16 electrons are correlated in $48$ spatial orbitals. Enforcing $M_s=0$, using the $A_1$ irrep of the $C_{2v}$ point group, and working with time-reversal symmetrized functions\\cite{Smeyers1973} (to enforce $S=\\textrm{even}$), results in a space size of roughly $1.8 \\times 10^{16}$ basis functions. This is a large space, particularly given the challenges of converging initiator error in excited-state dipole moments, as seen already.\n\nFigure~\\ref{fig:mgo_init} presents initiator convergence for walker populations (per state and per replica), $N_w$, ranging from $2.5 \\times 10^4$ to $3.2 \\times 10^7$. The ground state and first three excited states are calculated. For $N_w \\le 4 \\times 10^5$, error bars are calculated by averaging over 5 repeated calculations with varying RNG seeds. Due to the expensive nature of calculations, repeats were not performed for $N_w > 4 \\times 10^5$, and so error bars were not obtained. However, these error bars should mostly only decrease with increasing $N_w$, and are already small at $N_w = 4 \\times 10^5$. Therefore, at the largest walker populations considered, stochastic error should be much smaller than initiator error.\n\nInitiator profiles of both $E_{\\textrm{RDM}}$ and $E_{\\textrm{Trial}}$ estimators are presented in Figure~\\ref{fig:mgo_init}. At convergence, these should clearly become equal. By $N_w = 3.2 \\times 10^7$, this is the case to much better than $1$m$E_\\textrm{h}$ accuracy. As previously found, convergence is monotonic in all cases and $E_{\\textrm{RDM}}$ usually results in smaller initiator error.\n\nConvergence of dipole moments is also shown. Here, relative initiator error is once again larger than for energies, and convergence is non-monotonic. Because of this non-monotonic behavior, combined with the challenging nature of the system, our confidence in the accurate convergence of these values is somewhat less than for LiH and BH results. We cannot rule out the possibility of sudden further convergence at higher $N_w$ values. However we believe any significant deviations unlikely, although it is clear that $\\mu_1$ in particular is not fully converged on the scale shown.\n\nTable~\\ref{tab:mgo} presents FCIQMC energies and dipole moments, using $N_w = 3.2 \\times 10^7$, and with energies taken from the $E_{\\textrm{RDM}}$ estimator. For comparison, coupled cluster results are shown, using both singles and doubles (CCSD) and singles, doubles and triples (CCSDT). These values were calculated using NWChem package\\cite{NWChem}, with the equation-of-motion (EOM-CCSD and EOM-CCSDT) variants used for excited states. As expected, energies obtained from CCSDT are accurate compared to FCIQMC values, even for excited states. Meanwhile, dipole moments show greater differences, particularly for the $n=3$ state. For this state, EOM-CCSD and EOM-CCSDT values also greatly differ, with a flipped dipole moment resulting from EOM-CCSD. These results are consistent with those observed in FCIQMC in regions of large initiator error, that the relative error in dipole moments is much greater than in energies. We again expect that this is primarily due to the increased dependence on highly-excited determinants, and such configurations have particularly large amplitudes in excited states. CCSD and CCSDT appear to be unable to describe the wave function with sufficient accuracy in this region of configuration space, for this system, and for these challenging states.\n\n5.1 LiH\n\\subsection{LiH}\n\nSimulations on LiH were performed using between $1.25 \\times 10^4$ and $10^6$ walkers per simulation (i.e., for each state and replica sampled), in order to converge initiator error for all states. Density matrices were typically averaged over $10^5$ iterations, once convergence was deemed to have been reached for all states and all estimators. These entire simulations were then repeated five times with different initial RNG seeds, and the results averaged in order to calculate error estimates.\n\nFigure~\\ref{fig:lih_init} shows initiator convergence for LiH in the aug-cc-pVQZ basis set, for the lowest five energy eigenstates, and for four different estimators: dipole moments, transition dipoles moments, and energies calculated from both the RDM-based energy estimator, Eq.~(\\ref{eq:rdm_energy}), and from a trial wave function-projected estimator:\n\\begin{equation}\n(E_{\\textrm{Trial}})_n = \\frac{ \\bra \\Psi_{\\textrm{Trial}}^n | \\hat{H} | \\Psi^n \\ket }{ \\bra \\Psi_{\\textrm{Trial}}^n | \\Psi^n \\ket }.\n\\label{eq:trial_energy}\n\\end{equation}\nHere, $| \\Psi_{\\textrm{Trial}}^n \\ket$ is a trial wave function designed to have a large overlap with the exact state $| \\Phi^n \\ket$. We have discussed the use of such trial wave function estimators in excited-state FCIQMC in Ref.~(\\onlinecite{Blunt2015_3}). To generate $| \\Psi_{\\textrm{Trial}}^n \\ket$, we calculate the configuration interaction singles and doubles (CISD) wave functions for the lowest fifteen energy states. Then, once convergence of all FCIQMC simulations is deemed to have been reached, we assign each simulation one trial wave function by choosing the CISD solution with the largest overlap in each case. The reason for obtaining more CISD solutions than FCIQMC simulations is that CISD solutions can have a different energy ordering to FCI solutions. Averaging of each $E_{\\textrm{Trial}}$ estimate was performed from roughly the same point that RDM sampling began, and so both RDM and trial energy estimates are obtained from a similar number of iterations, usually $10^5$.\n\nThe initiator-FCIQMC estimates in Figure~\\ref{fig:lih_init} are all plotted relative to their values at the largest walker population considered, $N_{w}=10^6$. Here, convergence has been largely reached in all cases, and so the figures effectively plot initiator error against walker population. Reassuringly, initiator error in energy estimates is incredibly small for both estimators and for all states. Indeed, the largest error at the smallest walker population tested is less than $\\sim 0.5$ m$E_\\textrm{h}$ for $E_{\\textrm{Trial}}$.\n\nInterestingly, initiator error in $E_{\\textrm{RDM}}$ is much smaller than in $E_{\\textrm{Trial}}$. This is a trend that we have often observed, although exceptions do occur (and in the limit of an exact $| \\Psi_{\\textrm{Trial}}^n \\ket$, the initiator error is zero). Initiator error in the $E_{\\textrm{RDM}}$ energies are variational in all cases within stochastic errors, while it is not strictly enforced (though common) for this to also be the case for $E_{\\textrm{Trial}}$. For RDM-based energy estimates, this variationality is effectively ensured by the Hylleraas-Undheim-McDonald theorem\\cite{Hylleraas1930, McDonald1933}, which is expected to approximately hold for FCIQMC-sampled wave functions. Initiator error is larger for excited states, as previously observed\\cite{Blunt2015_3}. This is expected due to the more multi-configurational nature of excited states. It remains to be seen whether orbital optimization can increase this rate of convergence for excited states. Random errors however are larger in the RDM-based energy estimates, which is expected due to the fact that two uncorrelated simulations (from the two replicas) contribute to this quantity. However, error bars are extremely small in all cases here, always being smaller than $10^{-2}$ m$E_{\\textrm{h}}$.\n\n\\begin{figure*}[t!]\n\\includegraphics{bh.eps}\n\\caption{Initiator convergence for the five lowest energy states of BH in an aug-cc-pVTZ basis, at an internuclear distance of $1.2324$\\AA. Results are shifted relative to their values at the largest walker population considered, therefore approximately representing the initiator error. (a) Dipole moments. (b) Transition dipole moments from the ground state. (c) Energy calculated from a trial estimator, $E_{\\textrm{Trial}}$. (d) Energy calculated from the RDM estimator, $E_{\\textrm{RDM}}$. $N_w$ denotes the number of walkers for \\emph{each} state and replica sampled. Simulations were typically averaged over 5 simulations to obtain error bars.}\n\\label{fig:bh_init}\n\\end{figure*}\n\n\\begin{table*}[t]\n\\begin{center}{\\footnotesize\n\\begin{tabular}{@{\\extracolsep{4pt}}lccccc@{}}\n\\hline\n\\hline\nBasis & State $n$ & Energy gap ($\\Delta E_{0n}$) & Dipole moment ($\\mu_n$) & Transition dipole moment ($t_{0n}$) & Oscillator strength ($f_{0n}$) \\\\\n\\hline\naug-cc-pVDZ & 0 $\\;$ (${}^1\\Sigma^+$)     & -           &  0.528082(7) & -           & -           \\\\\n            & 1 $\\;$ ($\\; {}^1\\Delta \\;$) & 0.216230(3) & -0.18983(3)  & 0.0         & 0.0         \\\\\n            & 2 $\\;$ (${}^1\\Sigma^+$)     & 0.23727(1)  & -1.4146(5)   & 0.93478(3)  & 0.13822(1)  \\\\\n            & 3 $\\;$ (${}^1\\Sigma^+$)     & 0.257587(4) & -0.3219(3)   & 0.2102(1)   & 0.007590(9) \\\\\n            & 4 $\\;$ (${}^1\\Sigma^+$)     & 0.282665(1) &  3.5459(1)   & 0.44725(4)  & 0.037696(7) \\\\\n\\hline\naug-cc-pVTZ & 0 $\\;$ (${}^1\\Sigma^+$)     & -           &  0.54561(2) & -          & -           \\\\\n            & 1 $\\;$ ($\\; {}^1\\Delta \\;$) & 0.211482(6) & -0.19271(7) & 0.0        & 0.0         \\\\\n            & 2 $\\;$ (${}^1\\Sigma^+$)     & 0.238668(8) & -1.2943(5)  & 0.88508(5) & 0.12464(1)  \\\\\n            & 3 $\\;$ (${}^1\\Sigma^+$)     & 0.253574(4) & -0.4973(6)  & 0.1454(2)  & 0.00358(1)  \\\\\n            & 4 $\\;$ (${}^1\\Sigma^+$)     & 0.283481(2) &  3.4088(2)  & 0.35740(7) & 0.024141(9) \\\\\n\\hline\naug-cc-pVQZ & 0 $\\;$ (${}^1\\Sigma^+$)     & -          &  0.54914(6) & -         & -          \\\\\n            & 1 $\\;$ ($\\; {}^1\\Delta \\;$) & 0.21059(2) & -0.1968(3)  & 0.0       & 0.0        \\\\\n            & 2 $\\;$ (${}^1\\Sigma^+$)     & 0.23876(3) & -1.268(3)   & 0.8704(3) & 0.1206(1)  \\\\\n            & 3 $\\;$ (${}^1\\Sigma^+$)     & 0.25261(3) & -0.504(3)   & 0.139(1)  & 0.00327(7) \\\\\n            & 4 $\\;$ (${}^1\\Sigma^+$)     & 0.28289(1) &  3.2889(9)  & 0.3138(1) & 0.01857(2) \\\\\n\\hline\n\\hline\n\\end{tabular}\n}\n\\caption{Final converged estimates for the BH molecule at an internuclear distance of $1.2324$\\AA. Results are for the five lowest energy states in the $A_1$ irrep of the $C_{2v}$ point group, with $M_S=0$ and $S=\\textrm{even}$ quantum numbers. $n=0$ refers to the ground state, $n>1$ to excited states. Numbers in parentheses denote stochastic error, not initiator error. Energy gaps ($\\Delta E_{1n}$) were calculated using RDM-based energy estimates, Eq.~(\\ref{eq:rdm_energy}). Integrals were generated using the PySCF program\\cite{pyscf}.}\n\\label{tab:bh}\n\\end{center}\n\\end{table*}\n\nThe calculation of dipole moments provides a more interesting test, due to their greater dependence on more highly-excited determinants and diffuse single-particle orbitals. The relative initiator error is much larger, particularly for certain excited states (i.e. $\\mu_2$ and $\\mu_3$). The transition dipole moments considered involve transitions from the ground ($n=0$) state to excited ($n>1$) states. Because they always involve the ground state, it is to be expected that they have smaller relative initiator and stochastic error, compared to the corresponding non-transition dipole moment (i.e. $t_{0n}$ compared to $\\mu_n$). This expectation is borne out in the results, with initiator and stochastic error in $t_{0n}$ often being $\\sim 5$ times smaller than for $\\mu_n$. For the calculation of dipole moments from FCIQMC-sampled RDMs, relative stochastic errors are clearly much larger than for energies, and so the use of the semi-stochastic adaptation is of great importance here, whereas its use can be somewhat unnecessary in small ground-state energy calculations.\n\nClearly, the accurate calculation of dipole moments is more challenging than energies, requiring larger walker populations to obtain similar relative errors. However, this is not uniquely a feature of the initiator approximation in FCIQMC, but is equally true in other approximate methods, where properties such as the dipole moment are far more sensitive to the basis set and quality of the wavefunction than ground state energetics. That we are able to observe systematic converge of these quantities, with respect to a single simulation parameter, is reassuring.\n\nTable~\\ref{tab:lih} gives final results for the aug-cc-pV$X$Z basis sets, with $X=2,3,4$. Results in the small $X=2$ basis were fully converged at the smallest walker populations considered, $N_w = 1.25 \\times 10^4$, as confirmed by comparison to FCI results from the PySCF program. As expected, dipole moments vary quite substantially with basis set, particularly for the second, third and fourth excited states, demonstrating the importance of large basis sets with diffuse functions. Errors in brackets denote stochastic error bars, not initiator error, which is larger. However, given the careful convergence of initiator error, as shown in Figure~\\ref{fig:lih_init}, we expect dipole moments to be converged to around $10^{-3}e a_0$ in most cases, and energies to be converged \\emph{substantially} beyond chemical accuracy.\n\n5.2 BH\n\\subsection{BH}\n\nFigure~\\ref{fig:bh_init} shows results for BH in the aug-cc-pVTZ basis set and at an internuclear distance of $1.2324$\\AA, demonstrating similar initiator convergence plots to those in Figure~\\ref{fig:lih_init}. Here, results used between $1.25 \\times 10^4$ and $2 \\times 10^6$ walkers per simulation. RDM estimators and $E_{\\textrm{Trial}}$ were averaged over $5 \\times 10^4$ iterations, once convergence was achieved for all states and estimators. Here, instead of using CISD solutions as trial wave functions for $E_{\\textrm{Trial}}$, a slightly different approach was used: a ``trial space'' was defined as consisting of the $2 \\times 10^3$ most populated configurations across all simulations, once convergence had been approximately reached. Trial wave functions were then obtained as the eigenstates of $\\hat{H}$ within this subspace. This is similar to the approach to generate the deterministic space, as described above\\cite{Blunt2015}, and allows important basis states to be picked, while allowing an inexpensive calculation to determine each $| \\Psi_{\\textrm{Trial}}^n \\ket$.\n\nResults contain the same features as observed for LiH. Initiator error in the energy estimates are extremely small in all cases, particularly for estimates obtained from contraction of the RDM, and initiator convergence always occurs variationally. Stochastic error bars are larger for $E_{\\textrm{RDM}}$, as well as for excited states, but always extremely small. For dipole moments, similar trends also occur. Initiator and stochastic relative errors for the dipole moment are very small for the ground and first excited states ($\\mu_0$ and $\\mu_1$) and for the corresponding transition dipole moment ($t_{01}$) even at small walker populations. However, results for higher excited states contain larger errors, although we once again observe that errors in $t_{0n}$ are smaller than errors in $\\mu_n$ for each $n$, presumably because of the involvement of the ground state, which is well converged at lower walker populations, in each of the transition dipole moments considered.\n\nTable~\\ref{tab:bh} shows final results in aug-cc-pV$X$Z basis sets, for $X=2,3,4$. Results for $X=2$ used $2 \\times 10^5$ walkers per simulation, while results for $X=3$ and $X=4$ results used $2 \\times 10^6$ walkers per simulation. The expected strong dependence of dipole moments on the basis set is once again observed. This is particularly true for the second, third and fourth excited states ($n=2,3,4$). We note that these three states also contained the largest initiator error at small walker populations, as seen in Figure~\\ref{fig:bh_init}. This is probably not a coincidence, since the initiator approximation will inevitably result in a poorer description of highly excited regions of the wave function, presumably including excitations into high-energy diffuse functions, which appear important for accurate calculation of dipole moments for these particular states. Despite larger initiator error than for energy estimates, there is still a substantial undersampling of the space here, using $2 \\times 10^6$ walkers for a space size of $\\sim 7 \\times 10^9$ for the aug-cc-pVQZ basis, even for this small molecule, with benefits of Monte Carlo sampling typically increasing with system size.\n\n",
        "label": "fig:lih_init",
        "Descriptive_question1": "What is the label of the x-axis in figure_2?",
        "Descriptive_question2": "Which panel in figure_2 represents dipole moments?",
        "Reasoning_question1": "How does the initiator error in energy estimates using the RDM estimator compare to the trial estimator in figure_2 across different walker populations?",
        "Reasoning_question2": "What trend can be observed in the convergence of dipole moments for excited states compared to the ground state in figure_2 as the number of walkers increases?",
        "Descriptive_answer1": "$N_w$",
        "Descriptive_answer2": "(a)",
        "Reasoning_answer1": "Observing figure_2, the initiator error in energy estimates calculated from the RDM estimator is consistently smaller across all walker populations compared to the trial estimator. As the number of walkers increases, both estimators converge towards zero initiator error, but the RDM estimator demonstrates a faster and more stable convergence with smaller errors, highlighting its improved accuracy and reliability in these simulations.",
        "Reasoning_answer2": "Figure_2 shows that dipole moments for excited states (n > 0) exhibit larger initiator errors and slower convergence compared to the ground state dipole moment as the number of walkers increases. The initiator error decreases with increasing walker population, but the excited states require significantly larger numbers of walkers to reach convergence similar to the ground state, reflecting the more multi-configurational nature and sensitivity of excited-state dipole moments."
    },
    {
        "paper_id": "1704.00864.json",
        "image_id": "figure_3",
        "image_path": [
            "/home/yz979/scratch/chengye/scimolmo/test-images-tables/dataset_imagestables/1704.00864/images/bh.eps"
        ],
        "caption": "Initiator convergence for the five lowest energy states of BH in an aug-cc-pVTZ basis, at an internuclear distance of $1.2324$\\AA. Results are shifted relative to their values at the largest walker population considered, therefore approximately representing the initiator error. (a) Dipole moments. (b) Transition dipole moments from the ground state. (c) Energy calculated from a trial estimator, $E_{\\textrm{Trial}}$. (d) Energy calculated from the RDM estimator, $E_{\\textrm{RDM}}$. $N_w$ denotes the number of walkers for \\emph{each} state and replica sampled. Simulations were typically averaged over 5 simulations to obtain error bars.",
        "classify": "Chart",
        "section_info": "5 Results\n\\section{Results}\n\\label{sec:results}\n\nAs an initial test of these ideas, we consider the calculation of dipole moments, transition dipole moments, and oscillator strengths for low-lying states of small diatomic molecules. These quantities are of great importance for understanding various properties of molecular systems. The oscillator strength in particular is required to explain optical spectra, as it determines the probabilities of absorption and emission of photons coupling different electronic states. Nonetheless, dipole moments are challenging to calculate accurately, even for small molecules, because they are very sensitive to the quality of the wave function and single-particle basis set used, generally requiring many diffuse orbitals for an accurate description, with far greater basis set sensitivity than the energy\\cite{Green1974}.\n\nWe therefore begin by considering the LiH and BH molecules in aug-cc-pVDZ, aug-cc-pVTZ and aug-cc-pVQZ, containing $32$, $69$  and $126$ spatial orbitals respectively. The aug-cc-pVQZ basis 2-RDM was unobtainable in the previous RDM implementation, despite the small molecular size. We then consider the MgO molecule in an aug-cc-pVDZ basis set. We note that while the calculation of dipole moments only requires the 1-RDM, for these calculations we obtain the 1-RDM by contracting the 2-RDM, which we also use to calculate the energy using the estimator\n\\begin{equation}\n(E_{\\textrm{RDM}})_n = \\frac{ \\textrm{Tr} \\big[ \\hat{H} \\; \\hat{\\Gamma}^n \\big] }{ \\textrm{Tr} \\big[ \\hat{\\Gamma}^n \\big] }.\n\\label{eq:rdm_energy}\n\\end{equation}\nTherefore, the following is a good test of the newly-introduced ideas, as well as providing further insight into the effect of the initiator adaptation for different estimators and excited states.\n\nThe dipole moment for the state $|\\Phi^n\\ket$ is defined by\n\\begin{equation}\n\\bs{\\mu}_{n} = \\sum_{pq} \\gamma_{p,q}^{n} \\bra p | \\hat{\\bs{r}} | q \\ket.\n\\end{equation}\nwhile a transition dipole moment, $\\bs{t}_{nm}$, is defined by Eq.~(\\ref{eq:trans_dip_mom}), and the corresponding oscillator strength by\n\\begin{equation}\nf_{nm} = \\frac{2}{3} \\Delta E_{nm} |\\bs{t}_{nm}|^2,\n\\end{equation}\nfor an energy gap of $\\Delta E_{nm}$ between states $|\\Phi^n\\ket$ and $|\\Phi^m\\ket$.\n\n\\begin{figure*}[t!]\n\\includegraphics{lih.eps}\n\\caption{Initiator error convergence for the five lowest energy states of LiH in an aug-cc-pVQZ basis, at an internuclear distance of $1.5957$\\AA~as the number of walkers in each distribution is increased. Results are shifted relative to their values at the largest walker population considered, therefore approximately representing the initiator error. (a) Dipole moments. (b) Transition dipole moments from the ground state. (c) Energy calculated from a trial estimator, $E_{\\textrm{Trial}}$. (d) Energy calculated from the RDM estimator, $E_{\\textrm{RDM}}$. $N_w$ denotes the number of walkers for \\emph{each} state and replica sampled. Simulations were typically averaged over 5 simulations to obtain error bars.}\n\\label{fig:lih_init}\n\\end{figure*}\n\n\\begin{table*}[t]\n\\begin{center}{\\footnotesize\n\\begin{tabular}{@{\\extracolsep{4pt}}lccccc@{}}\n\\hline\n\\hline\nBasis & State $n$ & Energy gap ($\\Delta E_{0n}$) & Dipole moment ($\\mu_n$) & Transition dipole moment ($t_{0n}$) & Oscillator strength ($f_{0n}$) \\\\\n\\hline\naug-cc-pVDZ & 0 $\\;$ (${}^1\\Sigma^+$)  & -            & -2.3251372(2) & -            & -           \\\\\n            & 1 $\\;$ (${}^1\\Sigma^+$)  & 0.130434(1)  &  2.01947(4)   & 0.965189(7)  & 0.081007(1) \\\\\n            & 2 $\\;$ (${}^1\\Sigma^+$)  & 0.2149799(6) & -3.3543(9)    & 0.37471(1)   & 0.020123(1) \\\\\n            & 3 $\\;$ (${}^1\\Sigma^+$)  & 0.229077(4)  &  5.0832(8)    & 0.09126(8)   & 0.001271(2) \\\\\n            & 4 $\\;$ (${}^1\\Sigma^+$)  & 0.246350(3)  & -0.2958(3)    & 0.56074(2)   & 0.051639(4) \\\\\n\\hline\naug-cc-pVTZ & 0 $\\;$ (${}^1\\Sigma^+$) & -            & -2.306440(9)  & -            & -           \\\\\n            & 1 $\\;$ (${}^1\\Sigma^+$) & 0.132458(3)  &  2.02541(7)   & 0.93538(2)   & 0.077262(4) \\\\\n            & 2 $\\;$ (${}^1\\Sigma^+$) & 0.216705(6)  & -3.794(1)     & 0.41146(2)   & 0.024459(2) \\\\\n            & 3 $\\;$ (${}^1\\Sigma^+$) & 0.230621(2)  &  5.533(1)     & 0.07042(8)   & 0.000762(2) \\\\\n            & 4 $\\;$ (${}^1\\Sigma^+$) & 0.246520(2)  & -0.6235(7)    & 0.693170(7)  & 0.078966(2) \\\\\n\\hline\naug-cc-pVQZ & 0 $\\;$ (${}^1\\Sigma^+$) & -            & -2.30168(3)   & -            & -           \\\\\n            & 1 $\\;$ (${}^1\\Sigma^+$) & 0.132943(7)  &  2.0188(1)    & 0.92658(4)   & 0.076093(7) \\\\\n            & 2 $\\;$ (${}^1\\Sigma^+$) & 0.217616(7)  & -3.696(2)     & 0.3984(1)    & 0.02303(1)  \\\\\n            & 3 $\\;$ (${}^1\\Sigma^+$) & 0.231229(2)  &  6.211(2)     & 0.1083(2)    & 0.001809(6) \\\\\n            & 4 $\\;$ (${}^1\\Sigma^+$) & 0.242846(9)  & -1.998(2)     & 0.6201(5)    & 0.06224(9)  \\\\\n\\hline\n\\hline\n\\end{tabular}\n}\n\\caption{Final converged estimates for the LiH molecule at an internuclear distance of $1.5957$\\AA. Results are for the five lowest energy states in the $A_1$ irrep of the $C_{2v}$ point group, with $M_S=0$ and $S=\\textrm{even}$ quantum numbers (which happen to all be ${}^1\\Sigma^+$ states). $n=0$ refers to the ground state, $n>1$ to excited states. Numbers in parentheses denote stochastic error, not initiator error. Energy gaps ($\\Delta E_{1n}$) were calculated using RDM-based energy estimates, Eq.~(\\ref{eq:rdm_energy}). Integrals were generated using the PySCF program\\cite{pyscf}. In the small aug-cc-pVDZ, all results were verified against exact FCI results obtained from PySCF (not shown here).}\n\\label{tab:lih}\n\\end{center}\n\\end{table*}\n\nFor all simulations, the intial restricted Hartree--Fock (RHF) calculation was performed by PySCF\\cite{pyscf}. Integrals from PySCF were then passed to our FCIQMC program, \\url{NECI}, for the main calculation, which output one and two body density matrices. These were then contracted with integrals from PySCF to calculate final dipole moment estimates. Energy estimates were calculated on-the-fly in \\url{NECI}.\n\nThe five lowest energy states were calculated for LiH and BH, and the four lowest states of MgO, considering only states with $M_s=0$ and using the $A_1$ irreducible representation (irrep) of the $C_{2v}$ point group. Also, time-reversal symmetrized functions\\cite{Smeyers1973} were used as the many-particle basis states, therefore restricting the total spin quantum number, $S$, to be even, and thus removing triplet states. In all cases, the FCIQMC simulation time step was varied in the initial iterations so as to prevent ``bloom'' events, where many walkers can be created in a single spawning event (which often leads to large initiator error).\n\nWe also note that in generating excitations for the walker spawning step, we use an approach that greatly improves efficiency compared to the uniform sampling used in early FCIQMC results\\cite{Booth2009}. In this approach, the pair of orbital labels from which electrons are excited, $(i,j)$, are chosen uniformly, while the orbitals excited to, $(a,b)$, are selected with probabilities drawn from a Cauchy-Schwarz distribution, namely $p(ab|ij) \\propto \\sqrt{\\langle ia|ia \\rangle \\langle jb|jb \\rangle}$.\\cite{Smart_unpublished} Another approach to select connections efficiently was considered by Holmes \\emph{et. al.}\\cite{Holmes2016}, but not used here.\n\nAll simulations used the semi-stochastic adaptation to reduce stochastic errors\\cite{Petruzielo2012, Blunt2015}. For the LiH molecule the deterministic space consisted of all configurations up to and including double excitations from the Hartree--Fock determinant. For the BH and MgO molecules the deterministic space was formed from the $10^4$ most populated configurations across all wave functions sampled, once the simulations were deemed to have largely converged, using the approach described in Ref.~(\\onlinecite{Blunt2015}).\n\n\\subsection{LiH}\n\nSimulations on LiH were performed using between $1.25 \\times 10^4$ and $10^6$ walkers per simulation (i.e., for each state and replica sampled), in order to converge initiator error for all states. Density matrices were typically averaged over $10^5$ iterations, once convergence was deemed to have been reached for all states and all estimators. These entire simulations were then repeated five times with different initial RNG seeds, and the results averaged in order to calculate error estimates.\n\nFigure~\\ref{fig:lih_init} shows initiator convergence for LiH in the aug-cc-pVQZ basis set, for the lowest five energy eigenstates, and for four different estimators: dipole moments, transition dipoles moments, and energies calculated from both the RDM-based energy estimator, Eq.~(\\ref{eq:rdm_energy}), and from a trial wave function-projected estimator:\n\\begin{equation}\n(E_{\\textrm{Trial}})_n = \\frac{ \\bra \\Psi_{\\textrm{Trial}}^n | \\hat{H} | \\Psi^n \\ket }{ \\bra \\Psi_{\\textrm{Trial}}^n | \\Psi^n \\ket }.\n\\label{eq:trial_energy}\n\\end{equation}\nHere, $| \\Psi_{\\textrm{Trial}}^n \\ket$ is a trial wave function designed to have a large overlap with the exact state $| \\Phi^n \\ket$. We have discussed the use of such trial wave function estimators in excited-state FCIQMC in Ref.~(\\onlinecite{Blunt2015_3}). To generate $| \\Psi_{\\textrm{Trial}}^n \\ket$, we calculate the configuration interaction singles and doubles (CISD) wave functions for the lowest fifteen energy states. Then, once convergence of all FCIQMC simulations is deemed to have been reached, we assign each simulation one trial wave function by choosing the CISD solution with the largest overlap in each case. The reason for obtaining more CISD solutions than FCIQMC simulations is that CISD solutions can have a different energy ordering to FCI solutions. Averaging of each $E_{\\textrm{Trial}}$ estimate was performed from roughly the same point that RDM sampling began, and so both RDM and trial energy estimates are obtained from a similar number of iterations, usually $10^5$.\n\nThe initiator-FCIQMC estimates in Figure~\\ref{fig:lih_init} are all plotted relative to their values at the largest walker population considered, $N_{w}=10^6$. Here, convergence has been largely reached in all cases, and so the figures effectively plot initiator error against walker population. Reassuringly, initiator error in energy estimates is incredibly small for both estimators and for all states. Indeed, the largest error at the smallest walker population tested is less than $\\sim 0.5$ m$E_\\textrm{h}$ for $E_{\\textrm{Trial}}$.\n\nInterestingly, initiator error in $E_{\\textrm{RDM}}$ is much smaller than in $E_{\\textrm{Trial}}$. This is a trend that we have often observed, although exceptions do occur (and in the limit of an exact $| \\Psi_{\\textrm{Trial}}^n \\ket$, the initiator error is zero). Initiator error in the $E_{\\textrm{RDM}}$ energies are variational in all cases within stochastic errors, while it is not strictly enforced (though common) for this to also be the case for $E_{\\textrm{Trial}}$. For RDM-based energy estimates, this variationality is effectively ensured by the Hylleraas-Undheim-McDonald theorem\\cite{Hylleraas1930, McDonald1933}, which is expected to approximately hold for FCIQMC-sampled wave functions. Initiator error is larger for excited states, as previously observed\\cite{Blunt2015_3}. This is expected due to the more multi-configurational nature of excited states. It remains to be seen whether orbital optimization can increase this rate of convergence for excited states. Random errors however are larger in the RDM-based energy estimates, which is expected due to the fact that two uncorrelated simulations (from the two replicas) contribute to this quantity. However, error bars are extremely small in all cases here, always being smaller than $10^{-2}$ m$E_{\\textrm{h}}$.\n\n\\begin{figure*}[t!]\n\\includegraphics{bh.eps}\n\\caption{Initiator convergence for the five lowest energy states of BH in an aug-cc-pVTZ basis, at an internuclear distance of $1.2324$\\AA. Results are shifted relative to their values at the largest walker population considered, therefore approximately representing the initiator error. (a) Dipole moments. (b) Transition dipole moments from the ground state. (c) Energy calculated from a trial estimator, $E_{\\textrm{Trial}}$. (d) Energy calculated from the RDM estimator, $E_{\\textrm{RDM}}$. $N_w$ denotes the number of walkers for \\emph{each} state and replica sampled. Simulations were typically averaged over 5 simulations to obtain error bars.}\n\\label{fig:bh_init}\n\\end{figure*}\n\n\\begin{table*}[t]\n\\begin{center}{\\footnotesize\n\\begin{tabular}{@{\\extracolsep{4pt}}lccccc@{}}\n\\hline\n\\hline\nBasis & State $n$ & Energy gap ($\\Delta E_{0n}$) & Dipole moment ($\\mu_n$) & Transition dipole moment ($t_{0n}$) & Oscillator strength ($f_{0n}$) \\\\\n\\hline\naug-cc-pVDZ & 0 $\\;$ (${}^1\\Sigma^+$)     & -           &  0.528082(7) & -           & -           \\\\\n            & 1 $\\;$ ($\\; {}^1\\Delta \\;$) & 0.216230(3) & -0.18983(3)  & 0.0         & 0.0         \\\\\n            & 2 $\\;$ (${}^1\\Sigma^+$)     & 0.23727(1)  & -1.4146(5)   & 0.93478(3)  & 0.13822(1)  \\\\\n            & 3 $\\;$ (${}^1\\Sigma^+$)     & 0.257587(4) & -0.3219(3)   & 0.2102(1)   & 0.007590(9) \\\\\n            & 4 $\\;$ (${}^1\\Sigma^+$)     & 0.282665(1) &  3.5459(1)   & 0.44725(4)  & 0.037696(7) \\\\\n\\hline\naug-cc-pVTZ & 0 $\\;$ (${}^1\\Sigma^+$)     & -           &  0.54561(2) & -          & -           \\\\\n            & 1 $\\;$ ($\\; {}^1\\Delta \\;$) & 0.211482(6) & -0.19271(7) & 0.0        & 0.0         \\\\\n            & 2 $\\;$ (${}^1\\Sigma^+$)     & 0.238668(8) & -1.2943(5)  & 0.88508(5) & 0.12464(1)  \\\\\n            & 3 $\\;$ (${}^1\\Sigma^+$)     & 0.253574(4) & -0.4973(6)  & 0.1454(2)  & 0.00358(1)  \\\\\n            & 4 $\\;$ (${}^1\\Sigma^+$)     & 0.283481(2) &  3.4088(2)  & 0.35740(7) & 0.024141(9) \\\\\n\\hline\naug-cc-pVQZ & 0 $\\;$ (${}^1\\Sigma^+$)     & -          &  0.54914(6) & -         & -          \\\\\n            & 1 $\\;$ ($\\; {}^1\\Delta \\;$) & 0.21059(2) & -0.1968(3)  & 0.0       & 0.0        \\\\\n            & 2 $\\;$ (${}^1\\Sigma^+$)     & 0.23876(3) & -1.268(3)   & 0.8704(3) & 0.1206(1)  \\\\\n            & 3 $\\;$ (${}^1\\Sigma^+$)     & 0.25261(3) & -0.504(3)   & 0.139(1)  & 0.00327(7) \\\\\n            & 4 $\\;$ (${}^1\\Sigma^+$)     & 0.28289(1) &  3.2889(9)  & 0.3138(1) & 0.01857(2) \\\\\n\\hline\n\\hline\n\\end{tabular}\n}\n\\caption{Final converged estimates for the BH molecule at an internuclear distance of $1.2324$\\AA. Results are for the five lowest energy states in the $A_1$ irrep of the $C_{2v}$ point group, with $M_S=0$ and $S=\\textrm{even}$ quantum numbers. $n=0$ refers to the ground state, $n>1$ to excited states. Numbers in parentheses denote stochastic error, not initiator error. Energy gaps ($\\Delta E_{1n}$) were calculated using RDM-based energy estimates, Eq.~(\\ref{eq:rdm_energy}). Integrals were generated using the PySCF program\\cite{pyscf}.}\n\\label{tab:bh}\n\\end{center}\n\\end{table*}\n\nThe calculation of dipole moments provides a more interesting test, due to their greater dependence on more highly-excited determinants and diffuse single-particle orbitals. The relative initiator error is much larger, particularly for certain excited states (i.e. $\\mu_2$ and $\\mu_3$). The transition dipole moments considered involve transitions from the ground ($n=0$) state to excited ($n>1$) states. Because they always involve the ground state, it is to be expected that they have smaller relative initiator and stochastic error, compared to the corresponding non-transition dipole moment (i.e. $t_{0n}$ compared to $\\mu_n$). This expectation is borne out in the results, with initiator and stochastic error in $t_{0n}$ often being $\\sim 5$ times smaller than for $\\mu_n$. For the calculation of dipole moments from FCIQMC-sampled RDMs, relative stochastic errors are clearly much larger than for energies, and so the use of the semi-stochastic adaptation is of great importance here, whereas its use can be somewhat unnecessary in small ground-state energy calculations.\n\nClearly, the accurate calculation of dipole moments is more challenging than energies, requiring larger walker populations to obtain similar relative errors. However, this is not uniquely a feature of the initiator approximation in FCIQMC, but is equally true in other approximate methods, where properties such as the dipole moment are far more sensitive to the basis set and quality of the wavefunction than ground state energetics. That we are able to observe systematic converge of these quantities, with respect to a single simulation parameter, is reassuring.\n\nTable~\\ref{tab:lih} gives final results for the aug-cc-pV$X$Z basis sets, with $X=2,3,4$. Results in the small $X=2$ basis were fully converged at the smallest walker populations considered, $N_w = 1.25 \\times 10^4$, as confirmed by comparison to FCI results from the PySCF program. As expected, dipole moments vary quite substantially with basis set, particularly for the second, third and fourth excited states, demonstrating the importance of large basis sets with diffuse functions. Errors in brackets denote stochastic error bars, not initiator error, which is larger. However, given the careful convergence of initiator error, as shown in Figure~\\ref{fig:lih_init}, we expect dipole moments to be converged to around $10^{-3}e a_0$ in most cases, and energies to be converged \\emph{substantially} beyond chemical accuracy.\n\n\\subsection{BH}\n\nFigure~\\ref{fig:bh_init} shows results for BH in the aug-cc-pVTZ basis set and at an internuclear distance of $1.2324$\\AA, demonstrating similar initiator convergence plots to those in Figure~\\ref{fig:lih_init}. Here, results used between $1.25 \\times 10^4$ and $2 \\times 10^6$ walkers per simulation. RDM estimators and $E_{\\textrm{Trial}}$ were averaged over $5 \\times 10^4$ iterations, once convergence was achieved for all states and estimators. Here, instead of using CISD solutions as trial wave functions for $E_{\\textrm{Trial}}$, a slightly different approach was used: a ``trial space'' was defined as consisting of the $2 \\times 10^3$ most populated configurations across all simulations, once convergence had been approximately reached. Trial wave functions were then obtained as the eigenstates of $\\hat{H}$ within this subspace. This is similar to the approach to generate the deterministic space, as described above\\cite{Blunt2015}, and allows important basis states to be picked, while allowing an inexpensive calculation to determine each $| \\Psi_{\\textrm{Trial}}^n \\ket$.\n\nResults contain the same features as observed for LiH. Initiator error in the energy estimates are extremely small in all cases, particularly for estimates obtained from contraction of the RDM, and initiator convergence always occurs variationally. Stochastic error bars are larger for $E_{\\textrm{RDM}}$, as well as for excited states, but always extremely small. For dipole moments, similar trends also occur. Initiator and stochastic relative errors for the dipole moment are very small for the ground and first excited states ($\\mu_0$ and $\\mu_1$) and for the corresponding transition dipole moment ($t_{01}$) even at small walker populations. However, results for higher excited states contain larger errors, although we once again observe that errors in $t_{0n}$ are smaller than errors in $\\mu_n$ for each $n$, presumably because of the involvement of the ground state, which is well converged at lower walker populations, in each of the transition dipole moments considered.\n\nTable~\\ref{tab:bh} shows final results in aug-cc-pV$X$Z basis sets, for $X=2,3,4$. Results for $X=2$ used $2 \\times 10^5$ walkers per simulation, while results for $X=3$ and $X=4$ results used $2 \\times 10^6$ walkers per simulation. The expected strong dependence of dipole moments on the basis set is once again observed. This is particularly true for the second, third and fourth excited states ($n=2,3,4$). We note that these three states also contained the largest initiator error at small walker populations, as seen in Figure~\\ref{fig:bh_init}. This is probably not a coincidence, since the initiator approximation will inevitably result in a poorer description of highly excited regions of the wave function, presumably including excitations into high-energy diffuse functions, which appear important for accurate calculation of dipole moments for these particular states. Despite larger initiator error than for energy estimates, there is still a substantial undersampling of the space here, using $2 \\times 10^6$ walkers for a space size of $\\sim 7 \\times 10^9$ for the aug-cc-pVQZ basis, even for this small molecule, with benefits of Monte Carlo sampling typically increasing with system size.\n\n\\subsection{MgO}\n\n\\begin{figure*}[t!]\n\\includegraphics{mgo.eps}\n\\caption{Initiator convergence for dipole moments (left) and energies (right), for MgO in an aug-cc-pVDZ basis set, at an internuclear distance of $1.749$\\AA, and with 4 core electrons frozen. The four lowest-energy states are considered in the $A_1$ irrep of $C_{2v}$ and with $S=\\textrm{even}$ enforced (all ${}^1\\Sigma^+$ states). Energies are calculated from both RDM ($E_{\\textrm{RDM}}$) and trial wave function ($E_{\\textrm{Trial}}$) based estimates, and become equal to good accuracy at large walker number, $N_w$. Dipole moments appear mostly converged at $N_w=3.2 \\times 10^7$, except for $\\mu_1$. Error bars are only available for $N_w < 10^6$, but are small by this point and should only decrease in magnitude for larger walker populations.}\n\\label{fig:mgo_init}\n\\end{figure*}\n\n\\begin{table*}[t]\n\\begin{center}{\\footnotesize\n\\begin{tabular}{@{\\extracolsep{4pt}}c|ccc|ccc@{}}\n\\hline\n\\hline\nState $n$  & \\multicolumn{3}{c|}{ Energy/$E_{\\textrm{h}}$ } & \\multicolumn{3}{c}{ Dipole moment ($\\mu_n$) /$ea_0$ } \\\\\n\\hline\n & CCSD & CCSDT & FCIQMC & CCSD & CCSDT & FCIQMC \\\\\n\\hline\n0 $\\;$ (${}^1\\Sigma^+$) $\\;$ &  -274.632  &  -274.651  &  -274.654 &  2.590  &  2.398  &  2.382 \\\\\n1 $\\;$ (${}^1\\Sigma^+$) $\\;$ &  -274.531  &  -274.559  &  -274.564 &  1.811  &  2.008  &  2.289 \\\\\n2 $\\;$ (${}^1\\Sigma^+$) $\\;$ &  -274.480  &  -274.514  &  -274.517 &  0.297  &  0.847  &  1.154 \\\\\n3 $\\;$ (${}^1\\Sigma^+$) $\\;$ &  -274.440  &  -274.478  &  -274.480 & -0.366  &  0.529  &  1.198 \\\\\n\\hline\n\\hline\n\\end{tabular}\n}\n\\caption{Energies and dipole moments for MgO in an aug-cc-pVDZ basis set, at an internuclear distance of $1.749$\\AA, and with 4 core electrons frozen at the Hartree--Fock level. The four lowest-energy states are considered in the $A_1$ irrep of $C_{2v}$ and with $S=\\textrm{even}$ enforced (all ${}^1\\Sigma^+$ states). Error bars on FCIQMC results are not given, but are smaller than the order to which results are presented. FCIQMC energies are taken from the RDM-based estimates, $E_{\\textrm{RDM}}$. CCSD and CCSDT values were obtained from NWChem\\cite{NWChem}.}\n\\label{tab:mgo}\n\\end{center}\n\\end{table*}\n\nTo study a more challenging problem, we consider the calculation of energies and dipole moments for the MgO molecule, at its ground state equilibrium separation of $1.749$\\AA, and with 4 core electrons frozen at the Hartree--Fock level. Thus, a total of 16 electrons are correlated in $48$ spatial orbitals. Enforcing $M_s=0$, using the $A_1$ irrep of the $C_{2v}$ point group, and working with time-reversal symmetrized functions\\cite{Smeyers1973} (to enforce $S=\\textrm{even}$), results in a space size of roughly $1.8 \\times 10^{16}$ basis functions. This is a large space, particularly given the challenges of converging initiator error in excited-state dipole moments, as seen already.\n\nFigure~\\ref{fig:mgo_init} presents initiator convergence for walker populations (per state and per replica), $N_w$, ranging from $2.5 \\times 10^4$ to $3.2 \\times 10^7$. The ground state and first three excited states are calculated. For $N_w \\le 4 \\times 10^5$, error bars are calculated by averaging over 5 repeated calculations with varying RNG seeds. Due to the expensive nature of calculations, repeats were not performed for $N_w > 4 \\times 10^5$, and so error bars were not obtained. However, these error bars should mostly only decrease with increasing $N_w$, and are already small at $N_w = 4 \\times 10^5$. Therefore, at the largest walker populations considered, stochastic error should be much smaller than initiator error.\n\nInitiator profiles of both $E_{\\textrm{RDM}}$ and $E_{\\textrm{Trial}}$ estimators are presented in Figure~\\ref{fig:mgo_init}. At convergence, these should clearly become equal. By $N_w = 3.2 \\times 10^7$, this is the case to much better than $1$m$E_\\textrm{h}$ accuracy. As previously found, convergence is monotonic in all cases and $E_{\\textrm{RDM}}$ usually results in smaller initiator error.\n\nConvergence of dipole moments is also shown. Here, relative initiator error is once again larger than for energies, and convergence is non-monotonic. Because of this non-monotonic behavior, combined with the challenging nature of the system, our confidence in the accurate convergence of these values is somewhat less than for LiH and BH results. We cannot rule out the possibility of sudden further convergence at higher $N_w$ values. However we believe any significant deviations unlikely, although it is clear that $\\mu_1$ in particular is not fully converged on the scale shown.\n\nTable~\\ref{tab:mgo} presents FCIQMC energies and dipole moments, using $N_w = 3.2 \\times 10^7$, and with energies taken from the $E_{\\textrm{RDM}}$ estimator. For comparison, coupled cluster results are shown, using both singles and doubles (CCSD) and singles, doubles and triples (CCSDT). These values were calculated using NWChem package\\cite{NWChem}, with the equation-of-motion (EOM-CCSD and EOM-CCSDT) variants used for excited states. As expected, energies obtained from CCSDT are accurate compared to FCIQMC values, even for excited states. Meanwhile, dipole moments show greater differences, particularly for the $n=3$ state. For this state, EOM-CCSD and EOM-CCSDT values also greatly differ, with a flipped dipole moment resulting from EOM-CCSD. These results are consistent with those observed in FCIQMC in regions of large initiator error, that the relative error in dipole moments is much greater than in energies. We again expect that this is primarily due to the increased dependence on highly-excited determinants, and such configurations have particularly large amplitudes in excited states. CCSD and CCSDT appear to be unable to describe the wave function with sufficient accuracy in this region of configuration space, for this system, and for these challenging states.\n\n5.2 BH\n\\subsection{BH}\n\nFigure~\\ref{fig:bh_init} shows results for BH in the aug-cc-pVTZ basis set and at an internuclear distance of $1.2324$\\AA, demonstrating similar initiator convergence plots to those in Figure~\\ref{fig:lih_init}. Here, results used between $1.25 \\times 10^4$ and $2 \\times 10^6$ walkers per simulation. RDM estimators and $E_{\\textrm{Trial}}$ were averaged over $5 \\times 10^4$ iterations, once convergence was achieved for all states and estimators. Here, instead of using CISD solutions as trial wave functions for $E_{\\textrm{Trial}}$, a slightly different approach was used: a ``trial space'' was defined as consisting of the $2 \\times 10^3$ most populated configurations across all simulations, once convergence had been approximately reached. Trial wave functions were then obtained as the eigenstates of $\\hat{H}$ within this subspace. This is similar to the approach to generate the deterministic space, as described above\\cite{Blunt2015}, and allows important basis states to be picked, while allowing an inexpensive calculation to determine each $| \\Psi_{\\textrm{Trial}}^n \\ket$.\n\nResults contain the same features as observed for LiH. Initiator error in the energy estimates are extremely small in all cases, particularly for estimates obtained from contraction of the RDM, and initiator convergence always occurs variationally. Stochastic error bars are larger for $E_{\\textrm{RDM}}$, as well as for excited states, but always extremely small. For dipole moments, similar trends also occur. Initiator and stochastic relative errors for the dipole moment are very small for the ground and first excited states ($\\mu_0$ and $\\mu_1$) and for the corresponding transition dipole moment ($t_{01}$) even at small walker populations. However, results for higher excited states contain larger errors, although we once again observe that errors in $t_{0n}$ are smaller than errors in $\\mu_n$ for each $n$, presumably because of the involvement of the ground state, which is well converged at lower walker populations, in each of the transition dipole moments considered.\n\nTable~\\ref{tab:bh} shows final results in aug-cc-pV$X$Z basis sets, for $X=2,3,4$. Results for $X=2$ used $2 \\times 10^5$ walkers per simulation, while results for $X=3$ and $X=4$ results used $2 \\times 10^6$ walkers per simulation. The expected strong dependence of dipole moments on the basis set is once again observed. This is particularly true for the second, third and fourth excited states ($n=2,3,4$). We note that these three states also contained the largest initiator error at small walker populations, as seen in Figure~\\ref{fig:bh_init}. This is probably not a coincidence, since the initiator approximation will inevitably result in a poorer description of highly excited regions of the wave function, presumably including excitations into high-energy diffuse functions, which appear important for accurate calculation of dipole moments for these particular states. Despite larger initiator error than for energy estimates, there is still a substantial undersampling of the space here, using $2 \\times 10^6$ walkers for a space size of $\\sim 7 \\times 10^9$ for the aug-cc-pVQZ basis, even for this small molecule, with benefits of Monte Carlo sampling typically increasing with system size.\n\n",
        "label": "fig:bh_init",
        "Descriptive_question1": "What is the internuclear distance mentioned for BH in figure_3?",
        "Descriptive_question2": "What basis set is used for the BH molecule in figure_3?",
        "Reasoning_question1": "How does the initiator error convergence for dipole moments in figure_3 compare between the ground state and higher excited states for BH?",
        "Reasoning_question2": "What trend can be observed in the energy estimates using RDM and Trial estimators in figure_3 as the number of walkers increases for BH?",
        "Descriptive_answer1": "1.2324\\AA",
        "Descriptive_answer2": "aug-cc-pVTZ",
        "Reasoning_answer1": "The initiator error convergence for dipole moments in figure_3 shows that the ground state dipole moment ($\\mu_0$) has very small initiator error and converges quickly with increasing walker number. In contrast, higher excited states exhibit larger initiator errors and slower convergence. This is likely due to the more complex, multi-configurational nature of excited states that require larger walker populations for accurate sampling, reflecting greater sensitivity to the initiator approximation.",
        "Reasoning_answer2": "As the number of walkers increases in figure_3 for BH, the energy estimates from both RDM and Trial estimators converge monotonically and variationally towards the same value, with the RDM estimator generally exhibiting smaller initiator errors. This trend indicates that the two estimators yield consistent energies at sufficiently large walker populations, with the RDM estimator often providing more accurate and stable energy estimates during convergence."
    },
    {
        "paper_id": "1704.00864.json",
        "image_id": "figure_4",
        "image_path": [
            "/home/yz979/scratch/chengye/scimolmo/test-images-tables/dataset_imagestables/1704.00864/images/mgo.eps"
        ],
        "caption": "Initiator convergence for dipole moments (left) and energies (right), for MgO in an aug-cc-pVDZ basis set, at an internuclear distance of $1.749$\\AA, and with 4 core electrons frozen. The four lowest-energy states are considered in the $A_1$ irrep of $C_{2v}$ and with $S=\\textrm{even}$ enforced (all ${}^1\\Sigma^+$ states). Energies are calculated from both RDM ($E_{\\textrm{RDM}}$) and trial wave function ($E_{\\textrm{Trial}}$) based estimates, and become equal to good accuracy at large walker number, $N_w$. Dipole moments appear mostly converged at $N_w=3.2 \\times 10^7$, except for $\\mu_1$. Error bars are only available for $N_w < 10^6$, but are small by this point and should only decrease in magnitude for larger walker populations.",
        "classify": "Chart",
        "section_info": "5 Results\n\\section{Results}\n\\label{sec:results}\n\nAs an initial test of these ideas, we consider the calculation of dipole moments, transition dipole moments, and oscillator strengths for low-lying states of small diatomic molecules. These quantities are of great importance for understanding various properties of molecular systems. The oscillator strength in particular is required to explain optical spectra, as it determines the probabilities of absorption and emission of photons coupling different electronic states. Nonetheless, dipole moments are challenging to calculate accurately, even for small molecules, because they are very sensitive to the quality of the wave function and single-particle basis set used, generally requiring many diffuse orbitals for an accurate description, with far greater basis set sensitivity than the energy\\cite{Green1974}.\n\nWe therefore begin by considering the LiH and BH molecules in aug-cc-pVDZ, aug-cc-pVTZ and aug-cc-pVQZ, containing $32$, $69$  and $126$ spatial orbitals respectively. The aug-cc-pVQZ basis 2-RDM was unobtainable in the previous RDM implementation, despite the small molecular size. We then consider the MgO molecule in an aug-cc-pVDZ basis set. We note that while the calculation of dipole moments only requires the 1-RDM, for these calculations we obtain the 1-RDM by contracting the 2-RDM, which we also use to calculate the energy using the estimator\n\\begin{equation}\n(E_{\\textrm{RDM}})_n = \\frac{ \\textrm{Tr} \\big[ \\hat{H} \\; \\hat{\\Gamma}^n \\big] }{ \\textrm{Tr} \\big[ \\hat{\\Gamma}^n \\big] }.\n\\label{eq:rdm_energy}\n\\end{equation}\nTherefore, the following is a good test of the newly-introduced ideas, as well as providing further insight into the effect of the initiator adaptation for different estimators and excited states.\n\nThe dipole moment for the state $|\\Phi^n\\ket$ is defined by\n\\begin{equation}\n\\bs{\\mu}_{n} = \\sum_{pq} \\gamma_{p,q}^{n} \\bra p | \\hat{\\bs{r}} | q \\ket.\n\\end{equation}\nwhile a transition dipole moment, $\\bs{t}_{nm}$, is defined by Eq.~(\\ref{eq:trans_dip_mom}), and the corresponding oscillator strength by\n\\begin{equation}\nf_{nm} = \\frac{2}{3} \\Delta E_{nm} |\\bs{t}_{nm}|^2,\n\\end{equation}\nfor an energy gap of $\\Delta E_{nm}$ between states $|\\Phi^n\\ket$ and $|\\Phi^m\\ket$.\n\n\\begin{figure*}[t!]\n\\includegraphics{lih.eps}\n\\caption{Initiator error convergence for the five lowest energy states of LiH in an aug-cc-pVQZ basis, at an internuclear distance of $1.5957$\\AA~as the number of walkers in each distribution is increased. Results are shifted relative to their values at the largest walker population considered, therefore approximately representing the initiator error. (a) Dipole moments. (b) Transition dipole moments from the ground state. (c) Energy calculated from a trial estimator, $E_{\\textrm{Trial}}$. (d) Energy calculated from the RDM estimator, $E_{\\textrm{RDM}}$. $N_w$ denotes the number of walkers for \\emph{each} state and replica sampled. Simulations were typically averaged over 5 simulations to obtain error bars.}\n\\label{fig:lih_init}\n\\end{figure*}\n\n\\begin{table*}[t]\n\\begin{center}{\\footnotesize\n\\begin{tabular}{@{\\extracolsep{4pt}}lccccc@{}}\n\\hline\n\\hline\nBasis & State $n$ & Energy gap ($\\Delta E_{0n}$) & Dipole moment ($\\mu_n$) & Transition dipole moment ($t_{0n}$) & Oscillator strength ($f_{0n}$) \\\\\n\\hline\naug-cc-pVDZ & 0 $\\;$ (${}^1\\Sigma^+$)  & -            & -2.3251372(2) & -            & -           \\\\\n            & 1 $\\;$ (${}^1\\Sigma^+$)  & 0.130434(1)  &  2.01947(4)   & 0.965189(7)  & 0.081007(1) \\\\\n            & 2 $\\;$ (${}^1\\Sigma^+$)  & 0.2149799(6) & -3.3543(9)    & 0.37471(1)   & 0.020123(1) \\\\\n            & 3 $\\;$ (${}^1\\Sigma^+$)  & 0.229077(4)  &  5.0832(8)    & 0.09126(8)   & 0.001271(2) \\\\\n            & 4 $\\;$ (${}^1\\Sigma^+$)  & 0.246350(3)  & -0.2958(3)    & 0.56074(2)   & 0.051639(4) \\\\\n\\hline\naug-cc-pVTZ & 0 $\\;$ (${}^1\\Sigma^+$) & -            & -2.306440(9)  & -            & -           \\\\\n            & 1 $\\;$ (${}^1\\Sigma^+$) & 0.132458(3)  &  2.02541(7)   & 0.93538(2)   & 0.077262(4) \\\\\n            & 2 $\\;$ (${}^1\\Sigma^+$) & 0.216705(6)  & -3.794(1)     & 0.41146(2)   & 0.024459(2) \\\\\n            & 3 $\\;$ (${}^1\\Sigma^+$) & 0.230621(2)  &  5.533(1)     & 0.07042(8)   & 0.000762(2) \\\\\n            & 4 $\\;$ (${}^1\\Sigma^+$) & 0.246520(2)  & -0.6235(7)    & 0.693170(7)  & 0.078966(2) \\\\\n\\hline\naug-cc-pVQZ & 0 $\\;$ (${}^1\\Sigma^+$) & -            & -2.30168(3)   & -            & -           \\\\\n            & 1 $\\;$ (${}^1\\Sigma^+$) & 0.132943(7)  &  2.0188(1)    & 0.92658(4)   & 0.076093(7) \\\\\n            & 2 $\\;$ (${}^1\\Sigma^+$) & 0.217616(7)  & -3.696(2)     & 0.3984(1)    & 0.02303(1)  \\\\\n            & 3 $\\;$ (${}^1\\Sigma^+$) & 0.231229(2)  &  6.211(2)     & 0.1083(2)    & 0.001809(6) \\\\\n            & 4 $\\;$ (${}^1\\Sigma^+$) & 0.242846(9)  & -1.998(2)     & 0.6201(5)    & 0.06224(9)  \\\\\n\\hline\n\\hline\n\\end{tabular}\n}\n\\caption{Final converged estimates for the LiH molecule at an internuclear distance of $1.5957$\\AA. Results are for the five lowest energy states in the $A_1$ irrep of the $C_{2v}$ point group, with $M_S=0$ and $S=\\textrm{even}$ quantum numbers (which happen to all be ${}^1\\Sigma^+$ states). $n=0$ refers to the ground state, $n>1$ to excited states. Numbers in parentheses denote stochastic error, not initiator error. Energy gaps ($\\Delta E_{1n}$) were calculated using RDM-based energy estimates, Eq.~(\\ref{eq:rdm_energy}). Integrals were generated using the PySCF program\\cite{pyscf}. In the small aug-cc-pVDZ, all results were verified against exact FCI results obtained from PySCF (not shown here).}\n\\label{tab:lih}\n\\end{center}\n\\end{table*}\n\nFor all simulations, the intial restricted Hartree--Fock (RHF) calculation was performed by PySCF\\cite{pyscf}. Integrals from PySCF were then passed to our FCIQMC program, \\url{NECI}, for the main calculation, which output one and two body density matrices. These were then contracted with integrals from PySCF to calculate final dipole moment estimates. Energy estimates were calculated on-the-fly in \\url{NECI}.\n\nThe five lowest energy states were calculated for LiH and BH, and the four lowest states of MgO, considering only states with $M_s=0$ and using the $A_1$ irreducible representation (irrep) of the $C_{2v}$ point group. Also, time-reversal symmetrized functions\\cite{Smeyers1973} were used as the many-particle basis states, therefore restricting the total spin quantum number, $S$, to be even, and thus removing triplet states. In all cases, the FCIQMC simulation time step was varied in the initial iterations so as to prevent ``bloom'' events, where many walkers can be created in a single spawning event (which often leads to large initiator error).\n\nWe also note that in generating excitations for the walker spawning step, we use an approach that greatly improves efficiency compared to the uniform sampling used in early FCIQMC results\\cite{Booth2009}. In this approach, the pair of orbital labels from which electrons are excited, $(i,j)$, are chosen uniformly, while the orbitals excited to, $(a,b)$, are selected with probabilities drawn from a Cauchy-Schwarz distribution, namely $p(ab|ij) \\propto \\sqrt{\\langle ia|ia \\rangle \\langle jb|jb \\rangle}$.\\cite{Smart_unpublished} Another approach to select connections efficiently was considered by Holmes \\emph{et. al.}\\cite{Holmes2016}, but not used here.\n\nAll simulations used the semi-stochastic adaptation to reduce stochastic errors\\cite{Petruzielo2012, Blunt2015}. For the LiH molecule the deterministic space consisted of all configurations up to and including double excitations from the Hartree--Fock determinant. For the BH and MgO molecules the deterministic space was formed from the $10^4$ most populated configurations across all wave functions sampled, once the simulations were deemed to have largely converged, using the approach described in Ref.~(\\onlinecite{Blunt2015}).\n\n\\subsection{LiH}\n\nSimulations on LiH were performed using between $1.25 \\times 10^4$ and $10^6$ walkers per simulation (i.e., for each state and replica sampled), in order to converge initiator error for all states. Density matrices were typically averaged over $10^5$ iterations, once convergence was deemed to have been reached for all states and all estimators. These entire simulations were then repeated five times with different initial RNG seeds, and the results averaged in order to calculate error estimates.\n\nFigure~\\ref{fig:lih_init} shows initiator convergence for LiH in the aug-cc-pVQZ basis set, for the lowest five energy eigenstates, and for four different estimators: dipole moments, transition dipoles moments, and energies calculated from both the RDM-based energy estimator, Eq.~(\\ref{eq:rdm_energy}), and from a trial wave function-projected estimator:\n\\begin{equation}\n(E_{\\textrm{Trial}})_n = \\frac{ \\bra \\Psi_{\\textrm{Trial}}^n | \\hat{H} | \\Psi^n \\ket }{ \\bra \\Psi_{\\textrm{Trial}}^n | \\Psi^n \\ket }.\n\\label{eq:trial_energy}\n\\end{equation}\nHere, $| \\Psi_{\\textrm{Trial}}^n \\ket$ is a trial wave function designed to have a large overlap with the exact state $| \\Phi^n \\ket$. We have discussed the use of such trial wave function estimators in excited-state FCIQMC in Ref.~(\\onlinecite{Blunt2015_3}). To generate $| \\Psi_{\\textrm{Trial}}^n \\ket$, we calculate the configuration interaction singles and doubles (CISD) wave functions for the lowest fifteen energy states. Then, once convergence of all FCIQMC simulations is deemed to have been reached, we assign each simulation one trial wave function by choosing the CISD solution with the largest overlap in each case. The reason for obtaining more CISD solutions than FCIQMC simulations is that CISD solutions can have a different energy ordering to FCI solutions. Averaging of each $E_{\\textrm{Trial}}$ estimate was performed from roughly the same point that RDM sampling began, and so both RDM and trial energy estimates are obtained from a similar number of iterations, usually $10^5$.\n\nThe initiator-FCIQMC estimates in Figure~\\ref{fig:lih_init} are all plotted relative to their values at the largest walker population considered, $N_{w}=10^6$. Here, convergence has been largely reached in all cases, and so the figures effectively plot initiator error against walker population. Reassuringly, initiator error in energy estimates is incredibly small for both estimators and for all states. Indeed, the largest error at the smallest walker population tested is less than $\\sim 0.5$ m$E_\\textrm{h}$ for $E_{\\textrm{Trial}}$.\n\nInterestingly, initiator error in $E_{\\textrm{RDM}}$ is much smaller than in $E_{\\textrm{Trial}}$. This is a trend that we have often observed, although exceptions do occur (and in the limit of an exact $| \\Psi_{\\textrm{Trial}}^n \\ket$, the initiator error is zero). Initiator error in the $E_{\\textrm{RDM}}$ energies are variational in all cases within stochastic errors, while it is not strictly enforced (though common) for this to also be the case for $E_{\\textrm{Trial}}$. For RDM-based energy estimates, this variationality is effectively ensured by the Hylleraas-Undheim-McDonald theorem\\cite{Hylleraas1930, McDonald1933}, which is expected to approximately hold for FCIQMC-sampled wave functions. Initiator error is larger for excited states, as previously observed\\cite{Blunt2015_3}. This is expected due to the more multi-configurational nature of excited states. It remains to be seen whether orbital optimization can increase this rate of convergence for excited states. Random errors however are larger in the RDM-based energy estimates, which is expected due to the fact that two uncorrelated simulations (from the two replicas) contribute to this quantity. However, error bars are extremely small in all cases here, always being smaller than $10^{-2}$ m$E_{\\textrm{h}}$.\n\n\\begin{figure*}[t!]\n\\includegraphics{bh.eps}\n\\caption{Initiator convergence for the five lowest energy states of BH in an aug-cc-pVTZ basis, at an internuclear distance of $1.2324$\\AA. Results are shifted relative to their values at the largest walker population considered, therefore approximately representing the initiator error. (a) Dipole moments. (b) Transition dipole moments from the ground state. (c) Energy calculated from a trial estimator, $E_{\\textrm{Trial}}$. (d) Energy calculated from the RDM estimator, $E_{\\textrm{RDM}}$. $N_w$ denotes the number of walkers for \\emph{each} state and replica sampled. Simulations were typically averaged over 5 simulations to obtain error bars.}\n\\label{fig:bh_init}\n\\end{figure*}\n\n\\begin{table*}[t]\n\\begin{center}{\\footnotesize\n\\begin{tabular}{@{\\extracolsep{4pt}}lccccc@{}}\n\\hline\n\\hline\nBasis & State $n$ & Energy gap ($\\Delta E_{0n}$) & Dipole moment ($\\mu_n$) & Transition dipole moment ($t_{0n}$) & Oscillator strength ($f_{0n}$) \\\\\n\\hline\naug-cc-pVDZ & 0 $\\;$ (${}^1\\Sigma^+$)     & -           &  0.528082(7) & -           & -           \\\\\n            & 1 $\\;$ ($\\; {}^1\\Delta \\;$) & 0.216230(3) & -0.18983(3)  & 0.0         & 0.0         \\\\\n            & 2 $\\;$ (${}^1\\Sigma^+$)     & 0.23727(1)  & -1.4146(5)   & 0.93478(3)  & 0.13822(1)  \\\\\n            & 3 $\\;$ (${}^1\\Sigma^+$)     & 0.257587(4) & -0.3219(3)   & 0.2102(1)   & 0.007590(9) \\\\\n            & 4 $\\;$ (${}^1\\Sigma^+$)     & 0.282665(1) &  3.5459(1)   & 0.44725(4)  & 0.037696(7) \\\\\n\\hline\naug-cc-pVTZ & 0 $\\;$ (${}^1\\Sigma^+$)     & -           &  0.54561(2) & -          & -           \\\\\n            & 1 $\\;$ ($\\; {}^1\\Delta \\;$) & 0.211482(6) & -0.19271(7) & 0.0        & 0.0         \\\\\n            & 2 $\\;$ (${}^1\\Sigma^+$)     & 0.238668(8) & -1.2943(5)  & 0.88508(5) & 0.12464(1)  \\\\\n            & 3 $\\;$ (${}^1\\Sigma^+$)     & 0.253574(4) & -0.4973(6)  & 0.1454(2)  & 0.00358(1)  \\\\\n            & 4 $\\;$ (${}^1\\Sigma^+$)     & 0.283481(2) &  3.4088(2)  & 0.35740(7) & 0.024141(9) \\\\\n\\hline\naug-cc-pVQZ & 0 $\\;$ (${}^1\\Sigma^+$)     & -          &  0.54914(6) & -         & -          \\\\\n            & 1 $\\;$ ($\\; {}^1\\Delta \\;$) & 0.21059(2) & -0.1968(3)  & 0.0       & 0.0        \\\\\n            & 2 $\\;$ (${}^1\\Sigma^+$)     & 0.23876(3) & -1.268(3)   & 0.8704(3) & 0.1206(1)  \\\\\n            & 3 $\\;$ (${}^1\\Sigma^+$)     & 0.25261(3) & -0.504(3)   & 0.139(1)  & 0.00327(7) \\\\\n            & 4 $\\;$ (${}^1\\Sigma^+$)     & 0.28289(1) &  3.2889(9)  & 0.3138(1) & 0.01857(2) \\\\\n\\hline\n\\hline\n\\end{tabular}\n}\n\\caption{Final converged estimates for the BH molecule at an internuclear distance of $1.2324$\\AA. Results are for the five lowest energy states in the $A_1$ irrep of the $C_{2v}$ point group, with $M_S=0$ and $S=\\textrm{even}$ quantum numbers. $n=0$ refers to the ground state, $n>1$ to excited states. Numbers in parentheses denote stochastic error, not initiator error. Energy gaps ($\\Delta E_{1n}$) were calculated using RDM-based energy estimates, Eq.~(\\ref{eq:rdm_energy}). Integrals were generated using the PySCF program\\cite{pyscf}.}\n\\label{tab:bh}\n\\end{center}\n\\end{table*}\n\nThe calculation of dipole moments provides a more interesting test, due to their greater dependence on more highly-excited determinants and diffuse single-particle orbitals. The relative initiator error is much larger, particularly for certain excited states (i.e. $\\mu_2$ and $\\mu_3$). The transition dipole moments considered involve transitions from the ground ($n=0$) state to excited ($n>1$) states. Because they always involve the ground state, it is to be expected that they have smaller relative initiator and stochastic error, compared to the corresponding non-transition dipole moment (i.e. $t_{0n}$ compared to $\\mu_n$). This expectation is borne out in the results, with initiator and stochastic error in $t_{0n}$ often being $\\sim 5$ times smaller than for $\\mu_n$. For the calculation of dipole moments from FCIQMC-sampled RDMs, relative stochastic errors are clearly much larger than for energies, and so the use of the semi-stochastic adaptation is of great importance here, whereas its use can be somewhat unnecessary in small ground-state energy calculations.\n\nClearly, the accurate calculation of dipole moments is more challenging than energies, requiring larger walker populations to obtain similar relative errors. However, this is not uniquely a feature of the initiator approximation in FCIQMC, but is equally true in other approximate methods, where properties such as the dipole moment are far more sensitive to the basis set and quality of the wavefunction than ground state energetics. That we are able to observe systematic converge of these quantities, with respect to a single simulation parameter, is reassuring.\n\nTable~\\ref{tab:lih} gives final results for the aug-cc-pV$X$Z basis sets, with $X=2,3,4$. Results in the small $X=2$ basis were fully converged at the smallest walker populations considered, $N_w = 1.25 \\times 10^4$, as confirmed by comparison to FCI results from the PySCF program. As expected, dipole moments vary quite substantially with basis set, particularly for the second, third and fourth excited states, demonstrating the importance of large basis sets with diffuse functions. Errors in brackets denote stochastic error bars, not initiator error, which is larger. However, given the careful convergence of initiator error, as shown in Figure~\\ref{fig:lih_init}, we expect dipole moments to be converged to around $10^{-3}e a_0$ in most cases, and energies to be converged \\emph{substantially} beyond chemical accuracy.\n\n\\subsection{BH}\n\nFigure~\\ref{fig:bh_init} shows results for BH in the aug-cc-pVTZ basis set and at an internuclear distance of $1.2324$\\AA, demonstrating similar initiator convergence plots to those in Figure~\\ref{fig:lih_init}. Here, results used between $1.25 \\times 10^4$ and $2 \\times 10^6$ walkers per simulation. RDM estimators and $E_{\\textrm{Trial}}$ were averaged over $5 \\times 10^4$ iterations, once convergence was achieved for all states and estimators. Here, instead of using CISD solutions as trial wave functions for $E_{\\textrm{Trial}}$, a slightly different approach was used: a ``trial space'' was defined as consisting of the $2 \\times 10^3$ most populated configurations across all simulations, once convergence had been approximately reached. Trial wave functions were then obtained as the eigenstates of $\\hat{H}$ within this subspace. This is similar to the approach to generate the deterministic space, as described above\\cite{Blunt2015}, and allows important basis states to be picked, while allowing an inexpensive calculation to determine each $| \\Psi_{\\textrm{Trial}}^n \\ket$.\n\nResults contain the same features as observed for LiH. Initiator error in the energy estimates are extremely small in all cases, particularly for estimates obtained from contraction of the RDM, and initiator convergence always occurs variationally. Stochastic error bars are larger for $E_{\\textrm{RDM}}$, as well as for excited states, but always extremely small. For dipole moments, similar trends also occur. Initiator and stochastic relative errors for the dipole moment are very small for the ground and first excited states ($\\mu_0$ and $\\mu_1$) and for the corresponding transition dipole moment ($t_{01}$) even at small walker populations. However, results for higher excited states contain larger errors, although we once again observe that errors in $t_{0n}$ are smaller than errors in $\\mu_n$ for each $n$, presumably because of the involvement of the ground state, which is well converged at lower walker populations, in each of the transition dipole moments considered.\n\nTable~\\ref{tab:bh} shows final results in aug-cc-pV$X$Z basis sets, for $X=2,3,4$. Results for $X=2$ used $2 \\times 10^5$ walkers per simulation, while results for $X=3$ and $X=4$ results used $2 \\times 10^6$ walkers per simulation. The expected strong dependence of dipole moments on the basis set is once again observed. This is particularly true for the second, third and fourth excited states ($n=2,3,4$). We note that these three states also contained the largest initiator error at small walker populations, as seen in Figure~\\ref{fig:bh_init}. This is probably not a coincidence, since the initiator approximation will inevitably result in a poorer description of highly excited regions of the wave function, presumably including excitations into high-energy diffuse functions, which appear important for accurate calculation of dipole moments for these particular states. Despite larger initiator error than for energy estimates, there is still a substantial undersampling of the space here, using $2 \\times 10^6$ walkers for a space size of $\\sim 7 \\times 10^9$ for the aug-cc-pVQZ basis, even for this small molecule, with benefits of Monte Carlo sampling typically increasing with system size.\n\n\\subsection{MgO}\n\n\\begin{figure*}[t!]\n\\includegraphics{mgo.eps}\n\\caption{Initiator convergence for dipole moments (left) and energies (right), for MgO in an aug-cc-pVDZ basis set, at an internuclear distance of $1.749$\\AA, and with 4 core electrons frozen. The four lowest-energy states are considered in the $A_1$ irrep of $C_{2v}$ and with $S=\\textrm{even}$ enforced (all ${}^1\\Sigma^+$ states). Energies are calculated from both RDM ($E_{\\textrm{RDM}}$) and trial wave function ($E_{\\textrm{Trial}}$) based estimates, and become equal to good accuracy at large walker number, $N_w$. Dipole moments appear mostly converged at $N_w=3.2 \\times 10^7$, except for $\\mu_1$. Error bars are only available for $N_w < 10^6$, but are small by this point and should only decrease in magnitude for larger walker populations.}\n\\label{fig:mgo_init}\n\\end{figure*}\n\n\\begin{table*}[t]\n\\begin{center}{\\footnotesize\n\\begin{tabular}{@{\\extracolsep{4pt}}c|ccc|ccc@{}}\n\\hline\n\\hline\nState $n$  & \\multicolumn{3}{c|}{ Energy/$E_{\\textrm{h}}$ } & \\multicolumn{3}{c}{ Dipole moment ($\\mu_n$) /$ea_0$ } \\\\\n\\hline\n & CCSD & CCSDT & FCIQMC & CCSD & CCSDT & FCIQMC \\\\\n\\hline\n0 $\\;$ (${}^1\\Sigma^+$) $\\;$ &  -274.632  &  -274.651  &  -274.654 &  2.590  &  2.398  &  2.382 \\\\\n1 $\\;$ (${}^1\\Sigma^+$) $\\;$ &  -274.531  &  -274.559  &  -274.564 &  1.811  &  2.008  &  2.289 \\\\\n2 $\\;$ (${}^1\\Sigma^+$) $\\;$ &  -274.480  &  -274.514  &  -274.517 &  0.297  &  0.847  &  1.154 \\\\\n3 $\\;$ (${}^1\\Sigma^+$) $\\;$ &  -274.440  &  -274.478  &  -274.480 & -0.366  &  0.529  &  1.198 \\\\\n\\hline\n\\hline\n\\end{tabular}\n}\n\\caption{Energies and dipole moments for MgO in an aug-cc-pVDZ basis set, at an internuclear distance of $1.749$\\AA, and with 4 core electrons frozen at the Hartree--Fock level. The four lowest-energy states are considered in the $A_1$ irrep of $C_{2v}$ and with $S=\\textrm{even}$ enforced (all ${}^1\\Sigma^+$ states). Error bars on FCIQMC results are not given, but are smaller than the order to which results are presented. FCIQMC energies are taken from the RDM-based estimates, $E_{\\textrm{RDM}}$. CCSD and CCSDT values were obtained from NWChem\\cite{NWChem}.}\n\\label{tab:mgo}\n\\end{center}\n\\end{table*}\n\nTo study a more challenging problem, we consider the calculation of energies and dipole moments for the MgO molecule, at its ground state equilibrium separation of $1.749$\\AA, and with 4 core electrons frozen at the Hartree--Fock level. Thus, a total of 16 electrons are correlated in $48$ spatial orbitals. Enforcing $M_s=0$, using the $A_1$ irrep of the $C_{2v}$ point group, and working with time-reversal symmetrized functions\\cite{Smeyers1973} (to enforce $S=\\textrm{even}$), results in a space size of roughly $1.8 \\times 10^{16}$ basis functions. This is a large space, particularly given the challenges of converging initiator error in excited-state dipole moments, as seen already.\n\nFigure~\\ref{fig:mgo_init} presents initiator convergence for walker populations (per state and per replica), $N_w$, ranging from $2.5 \\times 10^4$ to $3.2 \\times 10^7$. The ground state and first three excited states are calculated. For $N_w \\le 4 \\times 10^5$, error bars are calculated by averaging over 5 repeated calculations with varying RNG seeds. Due to the expensive nature of calculations, repeats were not performed for $N_w > 4 \\times 10^5$, and so error bars were not obtained. However, these error bars should mostly only decrease with increasing $N_w$, and are already small at $N_w = 4 \\times 10^5$. Therefore, at the largest walker populations considered, stochastic error should be much smaller than initiator error.\n\nInitiator profiles of both $E_{\\textrm{RDM}}$ and $E_{\\textrm{Trial}}$ estimators are presented in Figure~\\ref{fig:mgo_init}. At convergence, these should clearly become equal. By $N_w = 3.2 \\times 10^7$, this is the case to much better than $1$m$E_\\textrm{h}$ accuracy. As previously found, convergence is monotonic in all cases and $E_{\\textrm{RDM}}$ usually results in smaller initiator error.\n\nConvergence of dipole moments is also shown. Here, relative initiator error is once again larger than for energies, and convergence is non-monotonic. Because of this non-monotonic behavior, combined with the challenging nature of the system, our confidence in the accurate convergence of these values is somewhat less than for LiH and BH results. We cannot rule out the possibility of sudden further convergence at higher $N_w$ values. However we believe any significant deviations unlikely, although it is clear that $\\mu_1$ in particular is not fully converged on the scale shown.\n\nTable~\\ref{tab:mgo} presents FCIQMC energies and dipole moments, using $N_w = 3.2 \\times 10^7$, and with energies taken from the $E_{\\textrm{RDM}}$ estimator. For comparison, coupled cluster results are shown, using both singles and doubles (CCSD) and singles, doubles and triples (CCSDT). These values were calculated using NWChem package\\cite{NWChem}, with the equation-of-motion (EOM-CCSD and EOM-CCSDT) variants used for excited states. As expected, energies obtained from CCSDT are accurate compared to FCIQMC values, even for excited states. Meanwhile, dipole moments show greater differences, particularly for the $n=3$ state. For this state, EOM-CCSD and EOM-CCSDT values also greatly differ, with a flipped dipole moment resulting from EOM-CCSD. These results are consistent with those observed in FCIQMC in regions of large initiator error, that the relative error in dipole moments is much greater than in energies. We again expect that this is primarily due to the increased dependence on highly-excited determinants, and such configurations have particularly large amplitudes in excited states. CCSD and CCSDT appear to be unable to describe the wave function with sufficient accuracy in this region of configuration space, for this system, and for these challenging states.\n\n5.3 MgO\n\\subsection{MgO}\n\n\\begin{figure*}[t!]\n\\includegraphics{mgo.eps}\n\\caption{Initiator convergence for dipole moments (left) and energies (right), for MgO in an aug-cc-pVDZ basis set, at an internuclear distance of $1.749$\\AA, and with 4 core electrons frozen. The four lowest-energy states are considered in the $A_1$ irrep of $C_{2v}$ and with $S=\\textrm{even}$ enforced (all ${}^1\\Sigma^+$ states). Energies are calculated from both RDM ($E_{\\textrm{RDM}}$) and trial wave function ($E_{\\textrm{Trial}}$) based estimates, and become equal to good accuracy at large walker number, $N_w$. Dipole moments appear mostly converged at $N_w=3.2 \\times 10^7$, except for $\\mu_1$. Error bars are only available for $N_w < 10^6$, but are small by this point and should only decrease in magnitude for larger walker populations.}\n\\label{fig:mgo_init}\n\\end{figure*}\n\n\\begin{table*}[t]\n\\begin{center}{\\footnotesize\n\\begin{tabular}{@{\\extracolsep{4pt}}c|ccc|ccc@{}}\n\\hline\n\\hline\nState $n$  & \\multicolumn{3}{c|}{ Energy/$E_{\\textrm{h}}$ } & \\multicolumn{3}{c}{ Dipole moment ($\\mu_n$) /$ea_0$ } \\\\\n\\hline\n & CCSD & CCSDT & FCIQMC & CCSD & CCSDT & FCIQMC \\\\\n\\hline\n0 $\\;$ (${}^1\\Sigma^+$) $\\;$ &  -274.632  &  -274.651  &  -274.654 &  2.590  &  2.398  &  2.382 \\\\\n1 $\\;$ (${}^1\\Sigma^+$) $\\;$ &  -274.531  &  -274.559  &  -274.564 &  1.811  &  2.008  &  2.289 \\\\\n2 $\\;$ (${}^1\\Sigma^+$) $\\;$ &  -274.480  &  -274.514  &  -274.517 &  0.297  &  0.847  &  1.154 \\\\\n3 $\\;$ (${}^1\\Sigma^+$) $\\;$ &  -274.440  &  -274.478  &  -274.480 & -0.366  &  0.529  &  1.198 \\\\\n\\hline\n\\hline\n\\end{tabular}\n}\n\\caption{Energies and dipole moments for MgO in an aug-cc-pVDZ basis set, at an internuclear distance of $1.749$\\AA, and with 4 core electrons frozen at the Hartree--Fock level. The four lowest-energy states are considered in the $A_1$ irrep of $C_{2v}$ and with $S=\\textrm{even}$ enforced (all ${}^1\\Sigma^+$ states). Error bars on FCIQMC results are not given, but are smaller than the order to which results are presented. FCIQMC energies are taken from the RDM-based estimates, $E_{\\textrm{RDM}}$. CCSD and CCSDT values were obtained from NWChem\\cite{NWChem}.}\n\\label{tab:mgo}\n\\end{center}\n\\end{table*}\n\nTo study a more challenging problem, we consider the calculation of energies and dipole moments for the MgO molecule, at its ground state equilibrium separation of $1.749$\\AA, and with 4 core electrons frozen at the Hartree--Fock level. Thus, a total of 16 electrons are correlated in $48$ spatial orbitals. Enforcing $M_s=0$, using the $A_1$ irrep of the $C_{2v}$ point group, and working with time-reversal symmetrized functions\\cite{Smeyers1973} (to enforce $S=\\textrm{even}$), results in a space size of roughly $1.8 \\times 10^{16}$ basis functions. This is a large space, particularly given the challenges of converging initiator error in excited-state dipole moments, as seen already.\n\nFigure~\\ref{fig:mgo_init} presents initiator convergence for walker populations (per state and per replica), $N_w$, ranging from $2.5 \\times 10^4$ to $3.2 \\times 10^7$. The ground state and first three excited states are calculated. For $N_w \\le 4 \\times 10^5$, error bars are calculated by averaging over 5 repeated calculations with varying RNG seeds. Due to the expensive nature of calculations, repeats were not performed for $N_w > 4 \\times 10^5$, and so error bars were not obtained. However, these error bars should mostly only decrease with increasing $N_w$, and are already small at $N_w = 4 \\times 10^5$. Therefore, at the largest walker populations considered, stochastic error should be much smaller than initiator error.\n\nInitiator profiles of both $E_{\\textrm{RDM}}$ and $E_{\\textrm{Trial}}$ estimators are presented in Figure~\\ref{fig:mgo_init}. At convergence, these should clearly become equal. By $N_w = 3.2 \\times 10^7$, this is the case to much better than $1$m$E_\\textrm{h}$ accuracy. As previously found, convergence is monotonic in all cases and $E_{\\textrm{RDM}}$ usually results in smaller initiator error.\n\nConvergence of dipole moments is also shown. Here, relative initiator error is once again larger than for energies, and convergence is non-monotonic. Because of this non-monotonic behavior, combined with the challenging nature of the system, our confidence in the accurate convergence of these values is somewhat less than for LiH and BH results. We cannot rule out the possibility of sudden further convergence at higher $N_w$ values. However we believe any significant deviations unlikely, although it is clear that $\\mu_1$ in particular is not fully converged on the scale shown.\n\nTable~\\ref{tab:mgo} presents FCIQMC energies and dipole moments, using $N_w = 3.2 \\times 10^7$, and with energies taken from the $E_{\\textrm{RDM}}$ estimator. For comparison, coupled cluster results are shown, using both singles and doubles (CCSD) and singles, doubles and triples (CCSDT). These values were calculated using NWChem package\\cite{NWChem}, with the equation-of-motion (EOM-CCSD and EOM-CCSDT) variants used for excited states. As expected, energies obtained from CCSDT are accurate compared to FCIQMC values, even for excited states. Meanwhile, dipole moments show greater differences, particularly for the $n=3$ state. For this state, EOM-CCSD and EOM-CCSDT values also greatly differ, with a flipped dipole moment resulting from EOM-CCSD. These results are consistent with those observed in FCIQMC in regions of large initiator error, that the relative error in dipole moments is much greater than in energies. We again expect that this is primarily due to the increased dependence on highly-excited determinants, and such configurations have particularly large amplitudes in excited states. CCSD and CCSDT appear to be unable to describe the wave function with sufficient accuracy in this region of configuration space, for this system, and for these challenging states.\n\n",
        "label": "fig:mgo_init",
        "Descriptive_question1": "What is the walker number at which dipole moments appear mostly converged in figure_4?",
        "Descriptive_question2": "How many core electrons are frozen in the MgO simulation shown in figure_4?",
        "Reasoning_question1": "Why does the dipole moment for state mu_1 appear not fully converged compared to other states in figure_4, even at high walker numbers?",
        "Reasoning_question2": "What can be inferred about the accuracy of energy estimates from RDM and Trial estimators in figure_4 as walker numbers increase?",
        "Descriptive_answer1": "3.2 × 10^7",
        "Descriptive_answer2": "4",
        "Reasoning_answer1": "The dipole moment for state μ₁ exhibits non-monotonic and slower convergence behavior, indicating that this particular state has a more complex electronic structure or is more sensitive to the initiator error. This may be due to the higher multi-configurational character or greater dependence on highly-excited determinants in this state, making it more challenging to accurately sample with the given walker numbers, even at the highest populations tested.",
        "Reasoning_answer2": "As walker populations increase, energy estimates from both RDM (E_{RDM}) and Trial wave function (E_{Trial}) estimators converge monotonically towards each other, with differences less than 1 millihartree at the largest walker numbers. This suggests that both estimators become accurate and consistent, with E_{RDM} typically showing smaller initiator errors and monotonic convergence, indicating improved reliability at large walker numbers."
    },
    {
        "paper_id": "1704.01111.json",
        "image_id": "figure_2",
        "image_path": [
            "/home/yz979/scratch/chengye/scimolmo/test-images-tables/dataset_imagestables/1704.01111/images/constQ_figure.eps"
        ],
        "caption": "\\label{constQ} Constant momentum slices taken on the MAPS spectrometer with E$_{i}$=75 meV on Fe$_{1.057(7)}$Te.  $(a-c)$ show scans and one dimensional cuts along the H direction and $(d-f)$ show scans along the K direction at 5 and 100 K.  The constant energy cuts in panels $(c)$ and $(f)$ were done at 10 $\\pm$ 2 meV.",
        "classify": "Chart",
        "section_info": "3 x=0.057(7) - Collinear magnetism and stripy fluctuations\n\\section{x=0.057(7) - Collinear magnetism and stripy fluctuations}\n\nWe first discuss the temperature dependent magnetic dynamics in the collinear phase of the Fe$_{1+x}$Te phase diagram by studying single crystals of Fe$_{1.057(7)}$Te.  Fe$_{1.057(7)}$Te is placed on the iron deficient side of the phase diagram shown in Fig. \\ref{structures} and has a first order transition at 75 K to a collinear magnetic phase accompanied by a structural transition from a tetragonal (space group $P4/nmm$) to monoclinic (space group $P2_{1}/m$) unit cell.  We first show how the magnetic fluctuations change in the (H,K) plane as a function of temperature and then study the anisotropy of these temperature dependent fluctuations using polarized neutrons.  \n\n\\begin{figure}[t]\n\\includegraphics[width=8.8cm]{constQ_figure.eps}\n\\caption{\\label{constQ} Constant momentum slices taken on the MAPS spectrometer with E$_{i}$=75 meV on Fe$_{1.057(7)}$Te.  $(a-c)$ show scans and one dimensional cuts along the H direction and $(d-f)$ show scans along the K direction at 5 and 100 K.  The constant energy cuts in panels $(c)$ and $(f)$ were done at 10 $\\pm$ 2 meV.}\n\\end{figure}\n\n\\begin{figure}[t]\n\\includegraphics[width=9.5cm]{constE_figure.eps}\n\\caption{\\label{constE} Constant energy slices taken on the MAPS spectrometer with E$_{i}$=75 meV and k$_{i}$ aligned along the $c$ axis.  $(a-c)$ displays slices at E=10 $\\pm$ 2 meV and $(d-f)$ illustrate scans at 30 $\\pm$ 5 meV.  Temperatures of 5, 70, and 100 K are shown for each energy transfer.}\n\\end{figure}\n\nFigure \\ref{constQ} shows constant momentum slices taken near (H,K)=(0.5,0) at 5 K, below the transition to collinear magnetic order, both along the H and K directions taken on the MAPS chopper spectrometer with E$_{i}$=75 meV.  Unlike the case of Se doped Fe$_{1+x}$Te$_{1-y}$Se$_{y}$ where the magnetic fluctuations are peaked near the ($\\pi$, $\\pi$) position, in parent Fe$_{1+x}$Te, the magnetic correlations are peaked near ($\\pi$, 0).~\\cite{Chi11:84}  As previously published, the low temperature magnetic fluctuations are strongly correlated along both the $a$ and $b$ directions and become one dimensional at higher energy transfers in excess of $\\sim$ 30 meV.~\\cite{Stock14:90}    This is confirmed in the constant momentum slices in panels $(a)$ and $(d)$ and the corresponding cuts in panels $(c)$ and $(f)$ where the magnetic fluctuations are strongly correlated in momentum along both the H and K directions at E=10 meV.  A considerable broadening occurs at high temperatures of 100 K, however, the fluctuations remain anisotropic in momentum at this temperature as illustrated in constant momentum slices in panels $(b)$ and $(e)$ and also cuts $(c)$ and $(f)$ taken at E=10 $\\pm$ 2 meV.\n\nFigure \\ref{constE} displays constant energy slices at low energy transfers of 10 $\\pm$ 2 meV (panesl $a-c$) and also 30 $\\pm$ 5 meV (panels $d-f$).  The data are also from the MAPS spectrometer with E$_{i}$=75 meV.  At low temperatures and low energies displayed in panel $(c)$, a constant energy map shows that the scattering is well correlated in both the $a$ and $b$ directions.  At 70 K (panel $b$) close to the first order transition to collinear order, the results discussed above is further confirmed showing broadened, yet still anisotropic correlations.  At 100 K (well above T$_{N}$), however as illustrated in panel $(a)$, the scattering becomes more isotropic being broader along $b$ yet there is still a clear anisotropy in the correlations along $a$ and $b$.  At higher energies (E=30 $\\pm$ 5 meV displayed in panels $d-f$), a different picture emerges with the magnetic fluctuations being more elongated along the K direction at 5 K indicative of one dimensional fluctuations.  At higher temperatures of 70 K, the magnetic correlations become isotropic along the H and K directions with the scattering forming nearly a ring in momentum at 100 K.\n\nAs noted previously in a high energy neutron scattering study~\\cite{Stock14:90} as a function of interstitial iron concentration, the magnetic excitations extend up to at $\\sim$ 200 meV and this is also confirmed by two-magnon results using Raman.~\\cite{Okazaki11:83}  Within error of $\\pm$ 15 \\%, we observe no temperature dependence to the integrated intensity at 5, 70, and 100 K integrating over energy transfers up to 50 meV.  While the analysis is sensitive to how the elastic line is treated, the increase in spectral weight in the inelastic channel is accounted for by the loss of spectral weight at the magnetic Bragg position within error.   This contrasts with some previous studies on Fe$_{1+x}$Te (Ref.\\onlinecite{Zal11:107}), however we emphasize that our measurements are performed on a different sample which is located at a different point in the magnetic and structural phase diagram drawn in Fig. \\ref{structures}.  We have also discussed possible sources of error due to low-energy phonons in the supplementary information in Ref. \\onlinecite{Stock14:90}.  In the collinear phase of Fe$_{1.057(7)}$Te, we therefore do not observe evidence of a spin transition, but rather a re-distribution of spectral weight from the elastic line to the inelastic position and also throughout the Brillouin zone as a function of temperature. \n\nThe constant energy and momentum cuts in Figs. \\ref{constQ} and \\ref{constE} illustrate that the fluctuations become considerably broadened in momentum and energy crossing the Neel transition (T$_{N}$=75 K).  Fig. \\ref{constQ} panels $(c)$ and $(f)$ show that the magnetic fluctuations remain peaked around K=0 and H=0.5, however at high temperatures of 100 K above the first order magnetic and structural transition, the magnetic fluctuations at 10 meV are slightly displaced in H to lower values away from the commensurate H=0.5 position.   The nature of these incommensurate fluctuations will be discussed in more detail below.  It is interesting to note that while the magnetic fluctuations become considerably broadened at high temperatures, they do remain very anisotropic in the (H,K) plane as illustrated in Fig. \\ref{constE} panel $(a)$ which is at 100 K, well above the Neel transition temperature.  Gaussian fits to the data produce an anisotropy in momentum with widths of $\\xi_{a}$/$\\xi_{b}$=1.85 $\\pm$ 0.10 at 100 K.  Therefore, the high temperature low energy fluctuations in Fe$_{1.057(7)}$Te are anisotropic in momentum, despite the tetragonal shape of the lattice and the equivalence of the $a$ and $b$ directions.  However, these fluctuations centered around the ($\\pi$, 0) position do preserve the C$_{4}$ symmetry of the lattice and should be distinguished from the ``nematic\" phase fluctuations identified in the ``122\" pnictides at high temperatures.~\\cite{Lu14:345,Fernandes14:10}  The anisotropy around the ($\\pi$,0) position may reflect the underlying Fermi surface~\\cite{Subedi08:78} as suggested to explain a similar anisotropy in the magnetic fluctuations in iron based pnictides.~\\cite{Park10:82,Graser09:11,Lu14:345} \n\n\\begin{figure}[t]\n\\includegraphics[width=9.3cm]{elastic_mag.eps}\n\\caption{\\label{pol_elastic} Polarization analysis of the elastic (0.5, 0, 1.5) magnetic Bragg peak showing spin-flip (open circles) and non-spin-flip (filled circles) scattering with the incident beam of neutrons polarized along the X, Y, and Z directions as defined in the main text.  The peak in the non spin-flip channel in panel $(a)$ is the result of incomplete polarization of the neutron beam and is defined by the flipping ratio.} \n\\end{figure}\n\nWe now investigate the polarization of the magnetic fluctuations as a function of temperature using polarized neutron scattering obtained at the 4F1 triple-axis spectrometer.  Figure \\ref{pol_elastic} illustrates scans through the low temperature elastic magnetic Bragg peak at (0.5, 0, 1.5).  Spin-flip (open circles) and non spin-flip (filled circles) are illustrated for the neutron beam polarized along the X (defined as parallel $\\vec{Q}$), Y (perpendicular to $\\vec{Q}$, but within the horizontal (H0L) scattering plane), and Z (perpendicular to the $\\vec{Q}$ and perpendicular to the horizontal scattering plane).  Panel $(a)$ shows that the dominant cross section is in the spin-flip channel, as expected for magnetic scattering, with the feed-through measured in the non-spin-flip channel the result of incomplete polarization characterized by the flipping ratio discussed above in the experimental section.   Scans with the polarization along Y indicate a strong spin-flip cross section indicating that the magnetic moment is oriented out of the scattering plane.  This is confirmed by scans with the neutron polarization oriented along Z which show a dominant cross section in the non-spin-flip channel.  Polarization analysis along the Y and Z directions confirm that the magnetic moments are aligned along the $b$ axis, perpendicular to the (H0L) scattering plane chosen for the 4F1 polarized experiments.   This result is consistent with previous powder diffraction and single crystal neutron diffraction reported for the iron deficient side of the Fe$_{1+x}$Te phase diagram.\n\n\\begin{figure}[t]\n\\includegraphics[width=9.0cm]{polarized_cuts.eps}\n\\caption{\\label{pol_cuts} Constant energy scans with polarization analysis at low temperatures at E=8.5 meV (panels $a-c$) and near the Neel transition at E=0.5 meV.  Given that the ordered magnetic moment is aligned along the $b$ axis, the polarized scans illustrated in $(a-c)$ show the dominant magnetic cross section is transverse to the magnetic moment direction.  This is contrasted with panels $(d-f)$ which illustrate a magnetic cross section predominately polarized along the $b$-axis.  The high temperature scattering also appears a $\\vec{Q}_{0}$=($\\sim$ 0.45,0,0.5) which is contrasted with the commensurate magnetic scattering at low temperatures (illustrated by the dotted line).}\n\\end{figure}\n\nHaving reviewed the magnetic structure at low temperatures with elastic neutron scattering with polarization analysis, we now discuss the polarization of the low temperature spin fluctuations.  The low temperature magnetic dynamics in Fe$_{1.057(7)}$Te are gapped for this particular iron concentration~\\cite{Stock11:84,Stock14:90} as shown in Fig. \\ref{constQ}.  In Fig. \\ref{pol_cuts} panels $(a-c)$, we investigate the polarization of these fluctuations at an energy transfer of E=8.5 meV, above the energy gap.  Panel $(a)$ shows the total magnetic cross section as probed in the spin-flip channel with the neutron beam polarized along X.   Panel $(b)$ illustrates the same scan, but now with the neutron beam polarized along the Z direction (perpendicular to $\\vec{Q}$ and the horizontal (H0L) scattering plane utilized on 4F1).  Given the geometry of the spectrometer and sample, this corresponds to the $b$ axis of the sample.  The intensity measured in this channel is, within error, equal to the total magnetic cross section measured in panel $(a)$ with the neutron beam polarized along X.  A small spin-flip cross section is measured with the beam polarized along Y.  This scan is sensitive to spin fluctuations along the $b$ axis of the material and parallel to the low temperature ordered magnetic moment direction.  Given the statistics, it is not clear if this is statistically significant given the flipping ratio.  The main result found in the polarization analysis in panels $(a-c)$ is that the dominant magnetic cross section at E=8.5 meV is transverse to the ordered magnetic moment direction at low temperatures.  We therefore conclude that the low energy spin fluctuations in Fe$_{1.057(7)}$Te are the result of localized spin fluctuations similar to spin-waves in an ordered antiferromagnet.  \n\nFigure \\ref{pol_cuts} $(d-f)$ show polarization analysis at E=0.5 meV of the low energy fluctuations at 70 K near the N\\'eel temperature (T$_{N}$=75 K).     As noted previously~\\cite{Parshall12:85}, these fluctuations are incommensurate at H$\\sim$ 0.45 and this is highlighted by the vertical dashed line at the commensurate H=0.5 position in Fig. \\ref{pol_cuts}.  Panel $(d)$ shows the total magnetic cross section with the neutron beam polarized along $\\vec{Q}$, defined as the X direction.  Panel $(e)$ shows a weaker cross section corresponding to fluctuations perpendicular to the $b$ axis of the sample (the low temperature ordered magnetic moment direction), however, a larger cross section is found in panel $(f)$ with the Y-polarized neutrons.  This analysis suggests a dominant fraction of the neutron cross section at 70 K corresponding to fluctuations polarized along the $b$ axis which are longitudinal fluctuations parallel to the low temperature ordered magnetic moment.  \n\n\\begin{figure}[t]\n\\includegraphics[width=9.0cm]{energy_depend.eps}\n\\caption{\\label{pol_energy} Polarization analysis of the incommensurate fluctuations near the Neel transition with neutrons polarized along the Y and Z directions as defined in the main text.  $(a)$ and $(b)$ illustrate an anisotropy in the fluctuations at 1.0 meV as evidenced by different intensities in the two channels.  At 2.0 meV (panels $c-d$) the fluctuations are isotropic with equal spectral weight in both polarization channels.  The vertical dashed line indicates the (0.5,0,0.5) position highlighting the fact that the high temperature spin fluctuations are incommensurate.}\n\\end{figure}\n\nFigure \\ref{pol_energy} illustrates the energy dependence of the incommensurate fluctuations critical to collinear Neel ordering.  Panels $(a,b)$ show polarization analysis at an energy transfer of 1.0 meV and panels $(c,d)$ at 2.0 meV.  The Y-polarized spin-flip channel is sensitive to fluctuations along the $b$ axis and the Z-polarized channel is sensitive to fluctuations transverse, or perpendicular, to $b$.  An anisotropy is observable at 1.0 meV, however at higher energy transfers of 2.0 meV (panels $c,d$), the excitations are isotropic within error with equal weight residing in the Y and Z polarized spin-flip channels.  This shows that the low energy incommensurate fluctuations are primarily longitudinal in nature, at higher energy transfers the fluctuations become more isotropic.\n\n\\begin{figure}[t]\n\\includegraphics[width=9.0cm]{temp_depend.eps}\n\\caption{\\label{pol_temp}  The temperature, energy, and polarization dependence of the magnetic fluctuations at $\\vec{Q}$=(0.45, 0 , 0.5).  The data was taken on the polarized cold triple-axis spectrometer 4F1.  Panel $(a)$ shows and energy scan with X and Y polarized neutrons at 70 K illustrating that the anisotropy develops between the two channels at low energy transfers.  $(b)$ shows the same constant-Q scan at 100 K illustrating that the fluctuations are isotropic, within error, at this temperature for all energy transfers investigated.  $(c)$ illustrates a temperature scan with E=0.5 meV and $\\vec{Q}$=(0.45,0.5,0.5) for Y and Z polarized neutrons.  The anisotropy between the two channels develops near T$_{N}$.}\n\\end{figure}\n\nFigure \\ref{pol_temp} illustrates background corrected temperature and energy scans for the incommensurate magnetic fluctuations peaked at (0.45, 0, 0.5).  Panel $(a)$ and $(b)$ display constant momentum cuts.  At 70 K, near the N\\'eel temperature, a significant difference develops between the Y and Z polarized channels at low energy transfers below $\\sim$ 1 meV.    At higher temperatures of 100 K displayed in panel $(b)$, the two channels for neutrons polarized along Y and Z have equal intensities within error indicating isotropic fluctuations at all energy transfers studied.  This is expected for a paramagnet at temperatures well above the ordering temperature.   The temperature dependence of the magnetic fluctuations at E=0.5 meV and with $\\vec{Q}$=(0.45, 0, 0.5) is displayed in panel $(c)$ where it is seen that a large difference between the spin-flip channels with Y and Z polarized neutrons is present near and below T$_{N}$.  At high temperatures the two channels are equal within error.\n\nThe polarized neutron scattering results demonstrate anisotropic spin fluctuations which develop near T$_{N}$ in Fe$_{1.057(7)}$Te.  This is evidenced in the difference seen between the Y and Z polarization channels in Figs. \\ref{pol_cuts}, \\ref{pol_energy}, and \\ref{pol_temp} discussed above.  If the magnetic fluctuations were isotropic, the intensity in these two spin-flip channels would be equal and ${1\\over 2}$ the intensity when the neutron beam is polarized along $\\vec{Q}$ as observed in magnets in the paramagnetic region at high temperatures as shown in Refs. \\onlinecite{Wicksted84:30,Ishikawa85:31}.   These anisotropic fluctuations are preferentially polarized along the $b$ axis which is parallel to the low temperature ordered magnetic moment.  However, these fluctuations are located at an incommensurate wave vector of $\\vec{q}_{0}$=($\\sim$ 0.45, 0, 0.5) and are distinct from the low temperature commensurate magnetic order and the fluctuations associated with this order which occurs at (0.5, 0, 0.5).  This indicates that these high temperature fluctuations are associated with a competing phases.  The polarization and also the wavevector are the same as the collinear spin-density wave reviewed above for Fe$_{0.124(5)}$Te.  We therefore conclude that this magnetic density wave phase competes with collinear and commensurate order in the Fe$_{1+x}$Te phase diagram.\n\n",
        "label": "constQ",
        "Descriptive_question1": "What is the energy transfer value for the constant energy cuts in panels (c) and (f) of figure_2?",
        "Descriptive_question2": "At what temperatures are the scans along the H and K directions shown in figure_2?",
        "Reasoning_question1": "How does the anisotropy of magnetic fluctuations in figure_2 change with temperature, particularly between 5 K and 100 K?",
        "Reasoning_question2": "What can be inferred about the correlation of magnetic fluctuations along the H and K directions at 10 meV in figure_2, and how does this vary with temperature?",
        "Descriptive_answer1": "10 meV",
        "Descriptive_answer2": "5 K and 100 K",
        "Reasoning_answer1": "At 5 K, the magnetic fluctuations shown in figure_2 are strongly correlated and anisotropic along both the H and K directions, indicating well-defined magnetic correlations in both momentum directions. However, at 100 K, there is noticeable broadening of the fluctuations, indicating reduced correlation lengths. Despite this broadening, the fluctuations remain anisotropic in momentum space, as evident from the slices and cuts presented. Therefore, with increasing temperature from 5 K to 100 K, the magnetic fluctuations become broader but maintain their anisotropic character in the (H,K) plane.",
        "Reasoning_answer2": "At 10 meV, figure_2 illustrates that at low temperature (5 K), magnetic fluctuations are strongly correlated along both the H and K directions, showing anisotropic peaks near the wavevector (0.5, 0). As the temperature increases to 100 K, these correlations broaden, indicating a reduction in the sharpness of magnetic fluctuations. Yet, the fluctuations remain anisotropic despite the broadening, implying that while thermal agitation reduces correlation strength, the directional dependence along H and K persists. This suggests that the magnetic fluctuations maintain their characteristic momentum space anisotropy even at temperatures well above the magnetic ordering temperature."
    },
    {
        "paper_id": "1704.01111.json",
        "image_id": "figure_3",
        "image_path": [
            "/home/yz979/scratch/chengye/scimolmo/test-images-tables/dataset_imagestables/1704.01111/images/constE_figure.eps"
        ],
        "caption": "\\label{constE} Constant energy slices taken on the MAPS spectrometer with E$_{i}$=75 meV and k$_{i}$ aligned along the $c$ axis.  $(a-c)$ displays slices at E=10 $\\pm$ 2 meV and $(d-f)$ illustrate scans at 30 $\\pm$ 5 meV.  Temperatures of 5, 70, and 100 K are shown for each energy transfer.",
        "classify": "Chart",
        "section_info": "3 x=0.057(7) - Collinear magnetism and stripy fluctuations\n\\section{x=0.057(7) - Collinear magnetism and stripy fluctuations}\n\nWe first discuss the temperature dependent magnetic dynamics in the collinear phase of the Fe$_{1+x}$Te phase diagram by studying single crystals of Fe$_{1.057(7)}$Te.  Fe$_{1.057(7)}$Te is placed on the iron deficient side of the phase diagram shown in Fig. \\ref{structures} and has a first order transition at 75 K to a collinear magnetic phase accompanied by a structural transition from a tetragonal (space group $P4/nmm$) to monoclinic (space group $P2_{1}/m$) unit cell.  We first show how the magnetic fluctuations change in the (H,K) plane as a function of temperature and then study the anisotropy of these temperature dependent fluctuations using polarized neutrons.  \n\n\\begin{figure}[t]\n\\includegraphics[width=8.8cm]{constQ_figure.eps}\n\\caption{\\label{constQ} Constant momentum slices taken on the MAPS spectrometer with E$_{i}$=75 meV on Fe$_{1.057(7)}$Te.  $(a-c)$ show scans and one dimensional cuts along the H direction and $(d-f)$ show scans along the K direction at 5 and 100 K.  The constant energy cuts in panels $(c)$ and $(f)$ were done at 10 $\\pm$ 2 meV.}\n\\end{figure}\n\n\\begin{figure}[t]\n\\includegraphics[width=9.5cm]{constE_figure.eps}\n\\caption{\\label{constE} Constant energy slices taken on the MAPS spectrometer with E$_{i}$=75 meV and k$_{i}$ aligned along the $c$ axis.  $(a-c)$ displays slices at E=10 $\\pm$ 2 meV and $(d-f)$ illustrate scans at 30 $\\pm$ 5 meV.  Temperatures of 5, 70, and 100 K are shown for each energy transfer.}\n\\end{figure}\n\nFigure \\ref{constQ} shows constant momentum slices taken near (H,K)=(0.5,0) at 5 K, below the transition to collinear magnetic order, both along the H and K directions taken on the MAPS chopper spectrometer with E$_{i}$=75 meV.  Unlike the case of Se doped Fe$_{1+x}$Te$_{1-y}$Se$_{y}$ where the magnetic fluctuations are peaked near the ($\\pi$, $\\pi$) position, in parent Fe$_{1+x}$Te, the magnetic correlations are peaked near ($\\pi$, 0).~\\cite{Chi11:84}  As previously published, the low temperature magnetic fluctuations are strongly correlated along both the $a$ and $b$ directions and become one dimensional at higher energy transfers in excess of $\\sim$ 30 meV.~\\cite{Stock14:90}    This is confirmed in the constant momentum slices in panels $(a)$ and $(d)$ and the corresponding cuts in panels $(c)$ and $(f)$ where the magnetic fluctuations are strongly correlated in momentum along both the H and K directions at E=10 meV.  A considerable broadening occurs at high temperatures of 100 K, however, the fluctuations remain anisotropic in momentum at this temperature as illustrated in constant momentum slices in panels $(b)$ and $(e)$ and also cuts $(c)$ and $(f)$ taken at E=10 $\\pm$ 2 meV.\n\nFigure \\ref{constE} displays constant energy slices at low energy transfers of 10 $\\pm$ 2 meV (panesl $a-c$) and also 30 $\\pm$ 5 meV (panels $d-f$).  The data are also from the MAPS spectrometer with E$_{i}$=75 meV.  At low temperatures and low energies displayed in panel $(c)$, a constant energy map shows that the scattering is well correlated in both the $a$ and $b$ directions.  At 70 K (panel $b$) close to the first order transition to collinear order, the results discussed above is further confirmed showing broadened, yet still anisotropic correlations.  At 100 K (well above T$_{N}$), however as illustrated in panel $(a)$, the scattering becomes more isotropic being broader along $b$ yet there is still a clear anisotropy in the correlations along $a$ and $b$.  At higher energies (E=30 $\\pm$ 5 meV displayed in panels $d-f$), a different picture emerges with the magnetic fluctuations being more elongated along the K direction at 5 K indicative of one dimensional fluctuations.  At higher temperatures of 70 K, the magnetic correlations become isotropic along the H and K directions with the scattering forming nearly a ring in momentum at 100 K.\n\nAs noted previously in a high energy neutron scattering study~\\cite{Stock14:90} as a function of interstitial iron concentration, the magnetic excitations extend up to at $\\sim$ 200 meV and this is also confirmed by two-magnon results using Raman.~\\cite{Okazaki11:83}  Within error of $\\pm$ 15 \\%, we observe no temperature dependence to the integrated intensity at 5, 70, and 100 K integrating over energy transfers up to 50 meV.  While the analysis is sensitive to how the elastic line is treated, the increase in spectral weight in the inelastic channel is accounted for by the loss of spectral weight at the magnetic Bragg position within error.   This contrasts with some previous studies on Fe$_{1+x}$Te (Ref.\\onlinecite{Zal11:107}), however we emphasize that our measurements are performed on a different sample which is located at a different point in the magnetic and structural phase diagram drawn in Fig. \\ref{structures}.  We have also discussed possible sources of error due to low-energy phonons in the supplementary information in Ref. \\onlinecite{Stock14:90}.  In the collinear phase of Fe$_{1.057(7)}$Te, we therefore do not observe evidence of a spin transition, but rather a re-distribution of spectral weight from the elastic line to the inelastic position and also throughout the Brillouin zone as a function of temperature. \n\nThe constant energy and momentum cuts in Figs. \\ref{constQ} and \\ref{constE} illustrate that the fluctuations become considerably broadened in momentum and energy crossing the Neel transition (T$_{N}$=75 K).  Fig. \\ref{constQ} panels $(c)$ and $(f)$ show that the magnetic fluctuations remain peaked around K=0 and H=0.5, however at high temperatures of 100 K above the first order magnetic and structural transition, the magnetic fluctuations at 10 meV are slightly displaced in H to lower values away from the commensurate H=0.5 position.   The nature of these incommensurate fluctuations will be discussed in more detail below.  It is interesting to note that while the magnetic fluctuations become considerably broadened at high temperatures, they do remain very anisotropic in the (H,K) plane as illustrated in Fig. \\ref{constE} panel $(a)$ which is at 100 K, well above the Neel transition temperature.  Gaussian fits to the data produce an anisotropy in momentum with widths of $\\xi_{a}$/$\\xi_{b}$=1.85 $\\pm$ 0.10 at 100 K.  Therefore, the high temperature low energy fluctuations in Fe$_{1.057(7)}$Te are anisotropic in momentum, despite the tetragonal shape of the lattice and the equivalence of the $a$ and $b$ directions.  However, these fluctuations centered around the ($\\pi$, 0) position do preserve the C$_{4}$ symmetry of the lattice and should be distinguished from the ``nematic\" phase fluctuations identified in the ``122\" pnictides at high temperatures.~\\cite{Lu14:345,Fernandes14:10}  The anisotropy around the ($\\pi$,0) position may reflect the underlying Fermi surface~\\cite{Subedi08:78} as suggested to explain a similar anisotropy in the magnetic fluctuations in iron based pnictides.~\\cite{Park10:82,Graser09:11,Lu14:345} \n\n\\begin{figure}[t]\n\\includegraphics[width=9.3cm]{elastic_mag.eps}\n\\caption{\\label{pol_elastic} Polarization analysis of the elastic (0.5, 0, 1.5) magnetic Bragg peak showing spin-flip (open circles) and non-spin-flip (filled circles) scattering with the incident beam of neutrons polarized along the X, Y, and Z directions as defined in the main text.  The peak in the non spin-flip channel in panel $(a)$ is the result of incomplete polarization of the neutron beam and is defined by the flipping ratio.} \n\\end{figure}\n\nWe now investigate the polarization of the magnetic fluctuations as a function of temperature using polarized neutron scattering obtained at the 4F1 triple-axis spectrometer.  Figure \\ref{pol_elastic} illustrates scans through the low temperature elastic magnetic Bragg peak at (0.5, 0, 1.5).  Spin-flip (open circles) and non spin-flip (filled circles) are illustrated for the neutron beam polarized along the X (defined as parallel $\\vec{Q}$), Y (perpendicular to $\\vec{Q}$, but within the horizontal (H0L) scattering plane), and Z (perpendicular to the $\\vec{Q}$ and perpendicular to the horizontal scattering plane).  Panel $(a)$ shows that the dominant cross section is in the spin-flip channel, as expected for magnetic scattering, with the feed-through measured in the non-spin-flip channel the result of incomplete polarization characterized by the flipping ratio discussed above in the experimental section.   Scans with the polarization along Y indicate a strong spin-flip cross section indicating that the magnetic moment is oriented out of the scattering plane.  This is confirmed by scans with the neutron polarization oriented along Z which show a dominant cross section in the non-spin-flip channel.  Polarization analysis along the Y and Z directions confirm that the magnetic moments are aligned along the $b$ axis, perpendicular to the (H0L) scattering plane chosen for the 4F1 polarized experiments.   This result is consistent with previous powder diffraction and single crystal neutron diffraction reported for the iron deficient side of the Fe$_{1+x}$Te phase diagram.\n\n\\begin{figure}[t]\n\\includegraphics[width=9.0cm]{polarized_cuts.eps}\n\\caption{\\label{pol_cuts} Constant energy scans with polarization analysis at low temperatures at E=8.5 meV (panels $a-c$) and near the Neel transition at E=0.5 meV.  Given that the ordered magnetic moment is aligned along the $b$ axis, the polarized scans illustrated in $(a-c)$ show the dominant magnetic cross section is transverse to the magnetic moment direction.  This is contrasted with panels $(d-f)$ which illustrate a magnetic cross section predominately polarized along the $b$-axis.  The high temperature scattering also appears a $\\vec{Q}_{0}$=($\\sim$ 0.45,0,0.5) which is contrasted with the commensurate magnetic scattering at low temperatures (illustrated by the dotted line).}\n\\end{figure}\n\nHaving reviewed the magnetic structure at low temperatures with elastic neutron scattering with polarization analysis, we now discuss the polarization of the low temperature spin fluctuations.  The low temperature magnetic dynamics in Fe$_{1.057(7)}$Te are gapped for this particular iron concentration~\\cite{Stock11:84,Stock14:90} as shown in Fig. \\ref{constQ}.  In Fig. \\ref{pol_cuts} panels $(a-c)$, we investigate the polarization of these fluctuations at an energy transfer of E=8.5 meV, above the energy gap.  Panel $(a)$ shows the total magnetic cross section as probed in the spin-flip channel with the neutron beam polarized along X.   Panel $(b)$ illustrates the same scan, but now with the neutron beam polarized along the Z direction (perpendicular to $\\vec{Q}$ and the horizontal (H0L) scattering plane utilized on 4F1).  Given the geometry of the spectrometer and sample, this corresponds to the $b$ axis of the sample.  The intensity measured in this channel is, within error, equal to the total magnetic cross section measured in panel $(a)$ with the neutron beam polarized along X.  A small spin-flip cross section is measured with the beam polarized along Y.  This scan is sensitive to spin fluctuations along the $b$ axis of the material and parallel to the low temperature ordered magnetic moment direction.  Given the statistics, it is not clear if this is statistically significant given the flipping ratio.  The main result found in the polarization analysis in panels $(a-c)$ is that the dominant magnetic cross section at E=8.5 meV is transverse to the ordered magnetic moment direction at low temperatures.  We therefore conclude that the low energy spin fluctuations in Fe$_{1.057(7)}$Te are the result of localized spin fluctuations similar to spin-waves in an ordered antiferromagnet.  \n\nFigure \\ref{pol_cuts} $(d-f)$ show polarization analysis at E=0.5 meV of the low energy fluctuations at 70 K near the N\\'eel temperature (T$_{N}$=75 K).     As noted previously~\\cite{Parshall12:85}, these fluctuations are incommensurate at H$\\sim$ 0.45 and this is highlighted by the vertical dashed line at the commensurate H=0.5 position in Fig. \\ref{pol_cuts}.  Panel $(d)$ shows the total magnetic cross section with the neutron beam polarized along $\\vec{Q}$, defined as the X direction.  Panel $(e)$ shows a weaker cross section corresponding to fluctuations perpendicular to the $b$ axis of the sample (the low temperature ordered magnetic moment direction), however, a larger cross section is found in panel $(f)$ with the Y-polarized neutrons.  This analysis suggests a dominant fraction of the neutron cross section at 70 K corresponding to fluctuations polarized along the $b$ axis which are longitudinal fluctuations parallel to the low temperature ordered magnetic moment.  \n\n\\begin{figure}[t]\n\\includegraphics[width=9.0cm]{energy_depend.eps}\n\\caption{\\label{pol_energy} Polarization analysis of the incommensurate fluctuations near the Neel transition with neutrons polarized along the Y and Z directions as defined in the main text.  $(a)$ and $(b)$ illustrate an anisotropy in the fluctuations at 1.0 meV as evidenced by different intensities in the two channels.  At 2.0 meV (panels $c-d$) the fluctuations are isotropic with equal spectral weight in both polarization channels.  The vertical dashed line indicates the (0.5,0,0.5) position highlighting the fact that the high temperature spin fluctuations are incommensurate.}\n\\end{figure}\n\nFigure \\ref{pol_energy} illustrates the energy dependence of the incommensurate fluctuations critical to collinear Neel ordering.  Panels $(a,b)$ show polarization analysis at an energy transfer of 1.0 meV and panels $(c,d)$ at 2.0 meV.  The Y-polarized spin-flip channel is sensitive to fluctuations along the $b$ axis and the Z-polarized channel is sensitive to fluctuations transverse, or perpendicular, to $b$.  An anisotropy is observable at 1.0 meV, however at higher energy transfers of 2.0 meV (panels $c,d$), the excitations are isotropic within error with equal weight residing in the Y and Z polarized spin-flip channels.  This shows that the low energy incommensurate fluctuations are primarily longitudinal in nature, at higher energy transfers the fluctuations become more isotropic.\n\n\\begin{figure}[t]\n\\includegraphics[width=9.0cm]{temp_depend.eps}\n\\caption{\\label{pol_temp}  The temperature, energy, and polarization dependence of the magnetic fluctuations at $\\vec{Q}$=(0.45, 0 , 0.5).  The data was taken on the polarized cold triple-axis spectrometer 4F1.  Panel $(a)$ shows and energy scan with X and Y polarized neutrons at 70 K illustrating that the anisotropy develops between the two channels at low energy transfers.  $(b)$ shows the same constant-Q scan at 100 K illustrating that the fluctuations are isotropic, within error, at this temperature for all energy transfers investigated.  $(c)$ illustrates a temperature scan with E=0.5 meV and $\\vec{Q}$=(0.45,0.5,0.5) for Y and Z polarized neutrons.  The anisotropy between the two channels develops near T$_{N}$.}\n\\end{figure}\n\nFigure \\ref{pol_temp} illustrates background corrected temperature and energy scans for the incommensurate magnetic fluctuations peaked at (0.45, 0, 0.5).  Panel $(a)$ and $(b)$ display constant momentum cuts.  At 70 K, near the N\\'eel temperature, a significant difference develops between the Y and Z polarized channels at low energy transfers below $\\sim$ 1 meV.    At higher temperatures of 100 K displayed in panel $(b)$, the two channels for neutrons polarized along Y and Z have equal intensities within error indicating isotropic fluctuations at all energy transfers studied.  This is expected for a paramagnet at temperatures well above the ordering temperature.   The temperature dependence of the magnetic fluctuations at E=0.5 meV and with $\\vec{Q}$=(0.45, 0, 0.5) is displayed in panel $(c)$ where it is seen that a large difference between the spin-flip channels with Y and Z polarized neutrons is present near and below T$_{N}$.  At high temperatures the two channels are equal within error.\n\nThe polarized neutron scattering results demonstrate anisotropic spin fluctuations which develop near T$_{N}$ in Fe$_{1.057(7)}$Te.  This is evidenced in the difference seen between the Y and Z polarization channels in Figs. \\ref{pol_cuts}, \\ref{pol_energy}, and \\ref{pol_temp} discussed above.  If the magnetic fluctuations were isotropic, the intensity in these two spin-flip channels would be equal and ${1\\over 2}$ the intensity when the neutron beam is polarized along $\\vec{Q}$ as observed in magnets in the paramagnetic region at high temperatures as shown in Refs. \\onlinecite{Wicksted84:30,Ishikawa85:31}.   These anisotropic fluctuations are preferentially polarized along the $b$ axis which is parallel to the low temperature ordered magnetic moment.  However, these fluctuations are located at an incommensurate wave vector of $\\vec{q}_{0}$=($\\sim$ 0.45, 0, 0.5) and are distinct from the low temperature commensurate magnetic order and the fluctuations associated with this order which occurs at (0.5, 0, 0.5).  This indicates that these high temperature fluctuations are associated with a competing phases.  The polarization and also the wavevector are the same as the collinear spin-density wave reviewed above for Fe$_{0.124(5)}$Te.  We therefore conclude that this magnetic density wave phase competes with collinear and commensurate order in the Fe$_{1+x}$Te phase diagram.\n\n",
        "label": "constE",
        "Descriptive_question1": "What energy transfer is shown in panels (a-c) of figure_3?",
        "Descriptive_question2": "At what temperature is the slice in panel (a) of figure_3 taken?",
        "Reasoning_question1": "How does the anisotropy of magnetic fluctuations in figure_3 change with temperature at an energy transfer of 10 ± 2 meV?",
        "Reasoning_question2": "What can be inferred about the dimensional nature of magnetic fluctuations at 5 K in figure_3 when comparing energy transfers of 10 ± 2 meV and 30 ± 5 meV?",
        "Descriptive_answer1": "10 ± 2 meV",
        "Descriptive_answer2": "100 K",
        "Reasoning_answer1": "At 10 ± 2 meV energy transfer, the magnetic fluctuations display anisotropic correlations in momentum space across all temperatures measured, including 5 K, 70 K, and 100 K. While at low temperatures (e.g., 5 K) the fluctuations are well correlated and anisotropic, even at 100 K (above the Neel temperature), the fluctuations remain anisotropic, though broader. This indicates that the anisotropy weakens but persists as temperature increases in this energy range.",
        "Reasoning_answer2": "At 5 K, comparing energy transfers of 10 ± 2 meV and 30 ± 5 meV reveals a dimensional crossover of magnetic fluctuations. At 10 meV, fluctuations are strongly correlated along both H and K directions, showing two-dimensional character. However, at 30 meV, the fluctuations become elongated specifically along the K direction, indicative of one-dimensional fluctuations. Therefore, with increasing energy transfer at low temperatures, the magnetic fluctuations transition from two-dimensional to more one-dimensional character."
    },
    {
        "paper_id": "1704.01111.json",
        "image_id": "figure_4",
        "image_path": [
            "/home/yz979/scratch/chengye/scimolmo/test-images-tables/dataset_imagestables/1704.01111/images/elastic_mag.eps"
        ],
        "caption": "\\label{pol_elastic} Polarization analysis of the elastic (0.5, 0, 1.5) magnetic Bragg peak showing spin-flip (open circles) and non-spin-flip (filled circles) scattering with the incident beam of neutrons polarized along the X, Y, and Z directions as defined in the main text.  The peak in the non spin-flip channel in panel $(a)$ is the result of incomplete polarization of the neutron beam and is defined by the flipping ratio.",
        "classify": "Chart",
        "section_info": "3 x=0.057(7) - Collinear magnetism and stripy fluctuations\n\\section{x=0.057(7) - Collinear magnetism and stripy fluctuations}\n\nWe first discuss the temperature dependent magnetic dynamics in the collinear phase of the Fe$_{1+x}$Te phase diagram by studying single crystals of Fe$_{1.057(7)}$Te.  Fe$_{1.057(7)}$Te is placed on the iron deficient side of the phase diagram shown in Fig. \\ref{structures} and has a first order transition at 75 K to a collinear magnetic phase accompanied by a structural transition from a tetragonal (space group $P4/nmm$) to monoclinic (space group $P2_{1}/m$) unit cell.  We first show how the magnetic fluctuations change in the (H,K) plane as a function of temperature and then study the anisotropy of these temperature dependent fluctuations using polarized neutrons.  \n\n\\begin{figure}[t]\n\\includegraphics[width=8.8cm]{constQ_figure.eps}\n\\caption{\\label{constQ} Constant momentum slices taken on the MAPS spectrometer with E$_{i}$=75 meV on Fe$_{1.057(7)}$Te.  $(a-c)$ show scans and one dimensional cuts along the H direction and $(d-f)$ show scans along the K direction at 5 and 100 K.  The constant energy cuts in panels $(c)$ and $(f)$ were done at 10 $\\pm$ 2 meV.}\n\\end{figure}\n\n\\begin{figure}[t]\n\\includegraphics[width=9.5cm]{constE_figure.eps}\n\\caption{\\label{constE} Constant energy slices taken on the MAPS spectrometer with E$_{i}$=75 meV and k$_{i}$ aligned along the $c$ axis.  $(a-c)$ displays slices at E=10 $\\pm$ 2 meV and $(d-f)$ illustrate scans at 30 $\\pm$ 5 meV.  Temperatures of 5, 70, and 100 K are shown for each energy transfer.}\n\\end{figure}\n\nFigure \\ref{constQ} shows constant momentum slices taken near (H,K)=(0.5,0) at 5 K, below the transition to collinear magnetic order, both along the H and K directions taken on the MAPS chopper spectrometer with E$_{i}$=75 meV.  Unlike the case of Se doped Fe$_{1+x}$Te$_{1-y}$Se$_{y}$ where the magnetic fluctuations are peaked near the ($\\pi$, $\\pi$) position, in parent Fe$_{1+x}$Te, the magnetic correlations are peaked near ($\\pi$, 0).~\\cite{Chi11:84}  As previously published, the low temperature magnetic fluctuations are strongly correlated along both the $a$ and $b$ directions and become one dimensional at higher energy transfers in excess of $\\sim$ 30 meV.~\\cite{Stock14:90}    This is confirmed in the constant momentum slices in panels $(a)$ and $(d)$ and the corresponding cuts in panels $(c)$ and $(f)$ where the magnetic fluctuations are strongly correlated in momentum along both the H and K directions at E=10 meV.  A considerable broadening occurs at high temperatures of 100 K, however, the fluctuations remain anisotropic in momentum at this temperature as illustrated in constant momentum slices in panels $(b)$ and $(e)$ and also cuts $(c)$ and $(f)$ taken at E=10 $\\pm$ 2 meV.\n\nFigure \\ref{constE} displays constant energy slices at low energy transfers of 10 $\\pm$ 2 meV (panesl $a-c$) and also 30 $\\pm$ 5 meV (panels $d-f$).  The data are also from the MAPS spectrometer with E$_{i}$=75 meV.  At low temperatures and low energies displayed in panel $(c)$, a constant energy map shows that the scattering is well correlated in both the $a$ and $b$ directions.  At 70 K (panel $b$) close to the first order transition to collinear order, the results discussed above is further confirmed showing broadened, yet still anisotropic correlations.  At 100 K (well above T$_{N}$), however as illustrated in panel $(a)$, the scattering becomes more isotropic being broader along $b$ yet there is still a clear anisotropy in the correlations along $a$ and $b$.  At higher energies (E=30 $\\pm$ 5 meV displayed in panels $d-f$), a different picture emerges with the magnetic fluctuations being more elongated along the K direction at 5 K indicative of one dimensional fluctuations.  At higher temperatures of 70 K, the magnetic correlations become isotropic along the H and K directions with the scattering forming nearly a ring in momentum at 100 K.\n\nAs noted previously in a high energy neutron scattering study~\\cite{Stock14:90} as a function of interstitial iron concentration, the magnetic excitations extend up to at $\\sim$ 200 meV and this is also confirmed by two-magnon results using Raman.~\\cite{Okazaki11:83}  Within error of $\\pm$ 15 \\%, we observe no temperature dependence to the integrated intensity at 5, 70, and 100 K integrating over energy transfers up to 50 meV.  While the analysis is sensitive to how the elastic line is treated, the increase in spectral weight in the inelastic channel is accounted for by the loss of spectral weight at the magnetic Bragg position within error.   This contrasts with some previous studies on Fe$_{1+x}$Te (Ref.\\onlinecite{Zal11:107}), however we emphasize that our measurements are performed on a different sample which is located at a different point in the magnetic and structural phase diagram drawn in Fig. \\ref{structures}.  We have also discussed possible sources of error due to low-energy phonons in the supplementary information in Ref. \\onlinecite{Stock14:90}.  In the collinear phase of Fe$_{1.057(7)}$Te, we therefore do not observe evidence of a spin transition, but rather a re-distribution of spectral weight from the elastic line to the inelastic position and also throughout the Brillouin zone as a function of temperature. \n\nThe constant energy and momentum cuts in Figs. \\ref{constQ} and \\ref{constE} illustrate that the fluctuations become considerably broadened in momentum and energy crossing the Neel transition (T$_{N}$=75 K).  Fig. \\ref{constQ} panels $(c)$ and $(f)$ show that the magnetic fluctuations remain peaked around K=0 and H=0.5, however at high temperatures of 100 K above the first order magnetic and structural transition, the magnetic fluctuations at 10 meV are slightly displaced in H to lower values away from the commensurate H=0.5 position.   The nature of these incommensurate fluctuations will be discussed in more detail below.  It is interesting to note that while the magnetic fluctuations become considerably broadened at high temperatures, they do remain very anisotropic in the (H,K) plane as illustrated in Fig. \\ref{constE} panel $(a)$ which is at 100 K, well above the Neel transition temperature.  Gaussian fits to the data produce an anisotropy in momentum with widths of $\\xi_{a}$/$\\xi_{b}$=1.85 $\\pm$ 0.10 at 100 K.  Therefore, the high temperature low energy fluctuations in Fe$_{1.057(7)}$Te are anisotropic in momentum, despite the tetragonal shape of the lattice and the equivalence of the $a$ and $b$ directions.  However, these fluctuations centered around the ($\\pi$, 0) position do preserve the C$_{4}$ symmetry of the lattice and should be distinguished from the ``nematic\" phase fluctuations identified in the ``122\" pnictides at high temperatures.~\\cite{Lu14:345,Fernandes14:10}  The anisotropy around the ($\\pi$,0) position may reflect the underlying Fermi surface~\\cite{Subedi08:78} as suggested to explain a similar anisotropy in the magnetic fluctuations in iron based pnictides.~\\cite{Park10:82,Graser09:11,Lu14:345} \n\n\\begin{figure}[t]\n\\includegraphics[width=9.3cm]{elastic_mag.eps}\n\\caption{\\label{pol_elastic} Polarization analysis of the elastic (0.5, 0, 1.5) magnetic Bragg peak showing spin-flip (open circles) and non-spin-flip (filled circles) scattering with the incident beam of neutrons polarized along the X, Y, and Z directions as defined in the main text.  The peak in the non spin-flip channel in panel $(a)$ is the result of incomplete polarization of the neutron beam and is defined by the flipping ratio.} \n\\end{figure}\n\nWe now investigate the polarization of the magnetic fluctuations as a function of temperature using polarized neutron scattering obtained at the 4F1 triple-axis spectrometer.  Figure \\ref{pol_elastic} illustrates scans through the low temperature elastic magnetic Bragg peak at (0.5, 0, 1.5).  Spin-flip (open circles) and non spin-flip (filled circles) are illustrated for the neutron beam polarized along the X (defined as parallel $\\vec{Q}$), Y (perpendicular to $\\vec{Q}$, but within the horizontal (H0L) scattering plane), and Z (perpendicular to the $\\vec{Q}$ and perpendicular to the horizontal scattering plane).  Panel $(a)$ shows that the dominant cross section is in the spin-flip channel, as expected for magnetic scattering, with the feed-through measured in the non-spin-flip channel the result of incomplete polarization characterized by the flipping ratio discussed above in the experimental section.   Scans with the polarization along Y indicate a strong spin-flip cross section indicating that the magnetic moment is oriented out of the scattering plane.  This is confirmed by scans with the neutron polarization oriented along Z which show a dominant cross section in the non-spin-flip channel.  Polarization analysis along the Y and Z directions confirm that the magnetic moments are aligned along the $b$ axis, perpendicular to the (H0L) scattering plane chosen for the 4F1 polarized experiments.   This result is consistent with previous powder diffraction and single crystal neutron diffraction reported for the iron deficient side of the Fe$_{1+x}$Te phase diagram.\n\n\\begin{figure}[t]\n\\includegraphics[width=9.0cm]{polarized_cuts.eps}\n\\caption{\\label{pol_cuts} Constant energy scans with polarization analysis at low temperatures at E=8.5 meV (panels $a-c$) and near the Neel transition at E=0.5 meV.  Given that the ordered magnetic moment is aligned along the $b$ axis, the polarized scans illustrated in $(a-c)$ show the dominant magnetic cross section is transverse to the magnetic moment direction.  This is contrasted with panels $(d-f)$ which illustrate a magnetic cross section predominately polarized along the $b$-axis.  The high temperature scattering also appears a $\\vec{Q}_{0}$=($\\sim$ 0.45,0,0.5) which is contrasted with the commensurate magnetic scattering at low temperatures (illustrated by the dotted line).}\n\\end{figure}\n\nHaving reviewed the magnetic structure at low temperatures with elastic neutron scattering with polarization analysis, we now discuss the polarization of the low temperature spin fluctuations.  The low temperature magnetic dynamics in Fe$_{1.057(7)}$Te are gapped for this particular iron concentration~\\cite{Stock11:84,Stock14:90} as shown in Fig. \\ref{constQ}.  In Fig. \\ref{pol_cuts} panels $(a-c)$, we investigate the polarization of these fluctuations at an energy transfer of E=8.5 meV, above the energy gap.  Panel $(a)$ shows the total magnetic cross section as probed in the spin-flip channel with the neutron beam polarized along X.   Panel $(b)$ illustrates the same scan, but now with the neutron beam polarized along the Z direction (perpendicular to $\\vec{Q}$ and the horizontal (H0L) scattering plane utilized on 4F1).  Given the geometry of the spectrometer and sample, this corresponds to the $b$ axis of the sample.  The intensity measured in this channel is, within error, equal to the total magnetic cross section measured in panel $(a)$ with the neutron beam polarized along X.  A small spin-flip cross section is measured with the beam polarized along Y.  This scan is sensitive to spin fluctuations along the $b$ axis of the material and parallel to the low temperature ordered magnetic moment direction.  Given the statistics, it is not clear if this is statistically significant given the flipping ratio.  The main result found in the polarization analysis in panels $(a-c)$ is that the dominant magnetic cross section at E=8.5 meV is transverse to the ordered magnetic moment direction at low temperatures.  We therefore conclude that the low energy spin fluctuations in Fe$_{1.057(7)}$Te are the result of localized spin fluctuations similar to spin-waves in an ordered antiferromagnet.  \n\nFigure \\ref{pol_cuts} $(d-f)$ show polarization analysis at E=0.5 meV of the low energy fluctuations at 70 K near the N\\'eel temperature (T$_{N}$=75 K).     As noted previously~\\cite{Parshall12:85}, these fluctuations are incommensurate at H$\\sim$ 0.45 and this is highlighted by the vertical dashed line at the commensurate H=0.5 position in Fig. \\ref{pol_cuts}.  Panel $(d)$ shows the total magnetic cross section with the neutron beam polarized along $\\vec{Q}$, defined as the X direction.  Panel $(e)$ shows a weaker cross section corresponding to fluctuations perpendicular to the $b$ axis of the sample (the low temperature ordered magnetic moment direction), however, a larger cross section is found in panel $(f)$ with the Y-polarized neutrons.  This analysis suggests a dominant fraction of the neutron cross section at 70 K corresponding to fluctuations polarized along the $b$ axis which are longitudinal fluctuations parallel to the low temperature ordered magnetic moment.  \n\n\\begin{figure}[t]\n\\includegraphics[width=9.0cm]{energy_depend.eps}\n\\caption{\\label{pol_energy} Polarization analysis of the incommensurate fluctuations near the Neel transition with neutrons polarized along the Y and Z directions as defined in the main text.  $(a)$ and $(b)$ illustrate an anisotropy in the fluctuations at 1.0 meV as evidenced by different intensities in the two channels.  At 2.0 meV (panels $c-d$) the fluctuations are isotropic with equal spectral weight in both polarization channels.  The vertical dashed line indicates the (0.5,0,0.5) position highlighting the fact that the high temperature spin fluctuations are incommensurate.}\n\\end{figure}\n\nFigure \\ref{pol_energy} illustrates the energy dependence of the incommensurate fluctuations critical to collinear Neel ordering.  Panels $(a,b)$ show polarization analysis at an energy transfer of 1.0 meV and panels $(c,d)$ at 2.0 meV.  The Y-polarized spin-flip channel is sensitive to fluctuations along the $b$ axis and the Z-polarized channel is sensitive to fluctuations transverse, or perpendicular, to $b$.  An anisotropy is observable at 1.0 meV, however at higher energy transfers of 2.0 meV (panels $c,d$), the excitations are isotropic within error with equal weight residing in the Y and Z polarized spin-flip channels.  This shows that the low energy incommensurate fluctuations are primarily longitudinal in nature, at higher energy transfers the fluctuations become more isotropic.\n\n\\begin{figure}[t]\n\\includegraphics[width=9.0cm]{temp_depend.eps}\n\\caption{\\label{pol_temp}  The temperature, energy, and polarization dependence of the magnetic fluctuations at $\\vec{Q}$=(0.45, 0 , 0.5).  The data was taken on the polarized cold triple-axis spectrometer 4F1.  Panel $(a)$ shows and energy scan with X and Y polarized neutrons at 70 K illustrating that the anisotropy develops between the two channels at low energy transfers.  $(b)$ shows the same constant-Q scan at 100 K illustrating that the fluctuations are isotropic, within error, at this temperature for all energy transfers investigated.  $(c)$ illustrates a temperature scan with E=0.5 meV and $\\vec{Q}$=(0.45,0.5,0.5) for Y and Z polarized neutrons.  The anisotropy between the two channels develops near T$_{N}$.}\n\\end{figure}\n\nFigure \\ref{pol_temp} illustrates background corrected temperature and energy scans for the incommensurate magnetic fluctuations peaked at (0.45, 0, 0.5).  Panel $(a)$ and $(b)$ display constant momentum cuts.  At 70 K, near the N\\'eel temperature, a significant difference develops between the Y and Z polarized channels at low energy transfers below $\\sim$ 1 meV.    At higher temperatures of 100 K displayed in panel $(b)$, the two channels for neutrons polarized along Y and Z have equal intensities within error indicating isotropic fluctuations at all energy transfers studied.  This is expected for a paramagnet at temperatures well above the ordering temperature.   The temperature dependence of the magnetic fluctuations at E=0.5 meV and with $\\vec{Q}$=(0.45, 0, 0.5) is displayed in panel $(c)$ where it is seen that a large difference between the spin-flip channels with Y and Z polarized neutrons is present near and below T$_{N}$.  At high temperatures the two channels are equal within error.\n\nThe polarized neutron scattering results demonstrate anisotropic spin fluctuations which develop near T$_{N}$ in Fe$_{1.057(7)}$Te.  This is evidenced in the difference seen between the Y and Z polarization channels in Figs. \\ref{pol_cuts}, \\ref{pol_energy}, and \\ref{pol_temp} discussed above.  If the magnetic fluctuations were isotropic, the intensity in these two spin-flip channels would be equal and ${1\\over 2}$ the intensity when the neutron beam is polarized along $\\vec{Q}$ as observed in magnets in the paramagnetic region at high temperatures as shown in Refs. \\onlinecite{Wicksted84:30,Ishikawa85:31}.   These anisotropic fluctuations are preferentially polarized along the $b$ axis which is parallel to the low temperature ordered magnetic moment.  However, these fluctuations are located at an incommensurate wave vector of $\\vec{q}_{0}$=($\\sim$ 0.45, 0, 0.5) and are distinct from the low temperature commensurate magnetic order and the fluctuations associated with this order which occurs at (0.5, 0, 0.5).  This indicates that these high temperature fluctuations are associated with a competing phases.  The polarization and also the wavevector are the same as the collinear spin-density wave reviewed above for Fe$_{0.124(5)}$Te.  We therefore conclude that this magnetic density wave phase competes with collinear and commensurate order in the Fe$_{1+x}$Te phase diagram.\n\n",
        "label": "pol_elastic",
        "Descriptive_question1": "What type of scattering is represented by open circles in figure_4?",
        "Descriptive_question2": "In which panel of figure_4 is the peak in the non-spin-flip channel observed?",
        "Reasoning_question1": "Why is there a peak in the non-spin-flip channel in figure_4, and how does it relate to the neutron beam's polarization?",
        "Reasoning_question2": "What can be inferred about the orientation of the magnetic moments in figure_4 based on the spin-flip and non-spin-flip scattering data across different polarization directions?",
        "Descriptive_answer1": "Spin-flip",
        "Descriptive_answer2": "a",
        "Reasoning_answer1": "The peak in the non-spin-flip channel observed in panel (a) of figure_4 is due to incomplete polarization of the neutron beam. This means that some neutrons are not perfectly polarized along the intended direction, leading to some leakage or feed-through of magnetic scattering intensity into the non-spin-flip channel, which would ideally only contain nuclear scattering. The flipping ratio defines this imperfection, so the observed peak arises from experimental limitations in neutron beam polarization rather than intrinsic sample properties.",
        "Reasoning_answer2": "The data in figure_4 shows that the dominant cross section is in the spin-flip channel when neutrons are polarized along X (parallel to Q), and the spin-flip scattering is strong with polarization along Y, indicating magnetic moments are oriented out of the scattering plane (H0L). Complementary scans with polarization along Z show dominant non-spin-flip scattering, confirming that the magnetic moments are aligned along the b axis, perpendicular to the (H0L) plane. Thus, the magnetic moments are oriented along the b axis based on the spin-flip and non-spin-flip scattering patterns across the different neutron polarization directions."
    },
    {
        "paper_id": "1704.01111.json",
        "image_id": "figure_5",
        "image_path": [
            "/home/yz979/scratch/chengye/scimolmo/test-images-tables/dataset_imagestables/1704.01111/images/polarized_cuts.eps"
        ],
        "caption": "\\label{pol_cuts} Constant energy scans with polarization analysis at low temperatures at E=8.5 meV (panels $a-c$) and near the Neel transition at E=0.5 meV.  Given that the ordered magnetic moment is aligned along the $b$ axis, the polarized scans illustrated in $(a-c)$ show the dominant magnetic cross section is transverse to the magnetic moment direction.  This is contrasted with panels $(d-f)$ which illustrate a magnetic cross section predominately polarized along the $b$-axis.  The high temperature scattering also appears a $\\vec{Q}_{0}$=($\\sim$ 0.45,0,0.5) which is contrasted with the commensurate magnetic scattering at low temperatures (illustrated by the dotted line).",
        "classify": "Chart",
        "section_info": "3 x=0.057(7) - Collinear magnetism and stripy fluctuations\n\\section{x=0.057(7) - Collinear magnetism and stripy fluctuations}\n\nWe first discuss the temperature dependent magnetic dynamics in the collinear phase of the Fe$_{1+x}$Te phase diagram by studying single crystals of Fe$_{1.057(7)}$Te.  Fe$_{1.057(7)}$Te is placed on the iron deficient side of the phase diagram shown in Fig. \\ref{structures} and has a first order transition at 75 K to a collinear magnetic phase accompanied by a structural transition from a tetragonal (space group $P4/nmm$) to monoclinic (space group $P2_{1}/m$) unit cell.  We first show how the magnetic fluctuations change in the (H,K) plane as a function of temperature and then study the anisotropy of these temperature dependent fluctuations using polarized neutrons.  \n\n\\begin{figure}[t]\n\\includegraphics[width=8.8cm]{constQ_figure.eps}\n\\caption{\\label{constQ} Constant momentum slices taken on the MAPS spectrometer with E$_{i}$=75 meV on Fe$_{1.057(7)}$Te.  $(a-c)$ show scans and one dimensional cuts along the H direction and $(d-f)$ show scans along the K direction at 5 and 100 K.  The constant energy cuts in panels $(c)$ and $(f)$ were done at 10 $\\pm$ 2 meV.}\n\\end{figure}\n\n\\begin{figure}[t]\n\\includegraphics[width=9.5cm]{constE_figure.eps}\n\\caption{\\label{constE} Constant energy slices taken on the MAPS spectrometer with E$_{i}$=75 meV and k$_{i}$ aligned along the $c$ axis.  $(a-c)$ displays slices at E=10 $\\pm$ 2 meV and $(d-f)$ illustrate scans at 30 $\\pm$ 5 meV.  Temperatures of 5, 70, and 100 K are shown for each energy transfer.}\n\\end{figure}\n\nFigure \\ref{constQ} shows constant momentum slices taken near (H,K)=(0.5,0) at 5 K, below the transition to collinear magnetic order, both along the H and K directions taken on the MAPS chopper spectrometer with E$_{i}$=75 meV.  Unlike the case of Se doped Fe$_{1+x}$Te$_{1-y}$Se$_{y}$ where the magnetic fluctuations are peaked near the ($\\pi$, $\\pi$) position, in parent Fe$_{1+x}$Te, the magnetic correlations are peaked near ($\\pi$, 0).~\\cite{Chi11:84}  As previously published, the low temperature magnetic fluctuations are strongly correlated along both the $a$ and $b$ directions and become one dimensional at higher energy transfers in excess of $\\sim$ 30 meV.~\\cite{Stock14:90}    This is confirmed in the constant momentum slices in panels $(a)$ and $(d)$ and the corresponding cuts in panels $(c)$ and $(f)$ where the magnetic fluctuations are strongly correlated in momentum along both the H and K directions at E=10 meV.  A considerable broadening occurs at high temperatures of 100 K, however, the fluctuations remain anisotropic in momentum at this temperature as illustrated in constant momentum slices in panels $(b)$ and $(e)$ and also cuts $(c)$ and $(f)$ taken at E=10 $\\pm$ 2 meV.\n\nFigure \\ref{constE} displays constant energy slices at low energy transfers of 10 $\\pm$ 2 meV (panesl $a-c$) and also 30 $\\pm$ 5 meV (panels $d-f$).  The data are also from the MAPS spectrometer with E$_{i}$=75 meV.  At low temperatures and low energies displayed in panel $(c)$, a constant energy map shows that the scattering is well correlated in both the $a$ and $b$ directions.  At 70 K (panel $b$) close to the first order transition to collinear order, the results discussed above is further confirmed showing broadened, yet still anisotropic correlations.  At 100 K (well above T$_{N}$), however as illustrated in panel $(a)$, the scattering becomes more isotropic being broader along $b$ yet there is still a clear anisotropy in the correlations along $a$ and $b$.  At higher energies (E=30 $\\pm$ 5 meV displayed in panels $d-f$), a different picture emerges with the magnetic fluctuations being more elongated along the K direction at 5 K indicative of one dimensional fluctuations.  At higher temperatures of 70 K, the magnetic correlations become isotropic along the H and K directions with the scattering forming nearly a ring in momentum at 100 K.\n\nAs noted previously in a high energy neutron scattering study~\\cite{Stock14:90} as a function of interstitial iron concentration, the magnetic excitations extend up to at $\\sim$ 200 meV and this is also confirmed by two-magnon results using Raman.~\\cite{Okazaki11:83}  Within error of $\\pm$ 15 \\%, we observe no temperature dependence to the integrated intensity at 5, 70, and 100 K integrating over energy transfers up to 50 meV.  While the analysis is sensitive to how the elastic line is treated, the increase in spectral weight in the inelastic channel is accounted for by the loss of spectral weight at the magnetic Bragg position within error.   This contrasts with some previous studies on Fe$_{1+x}$Te (Ref.\\onlinecite{Zal11:107}), however we emphasize that our measurements are performed on a different sample which is located at a different point in the magnetic and structural phase diagram drawn in Fig. \\ref{structures}.  We have also discussed possible sources of error due to low-energy phonons in the supplementary information in Ref. \\onlinecite{Stock14:90}.  In the collinear phase of Fe$_{1.057(7)}$Te, we therefore do not observe evidence of a spin transition, but rather a re-distribution of spectral weight from the elastic line to the inelastic position and also throughout the Brillouin zone as a function of temperature. \n\nThe constant energy and momentum cuts in Figs. \\ref{constQ} and \\ref{constE} illustrate that the fluctuations become considerably broadened in momentum and energy crossing the Neel transition (T$_{N}$=75 K).  Fig. \\ref{constQ} panels $(c)$ and $(f)$ show that the magnetic fluctuations remain peaked around K=0 and H=0.5, however at high temperatures of 100 K above the first order magnetic and structural transition, the magnetic fluctuations at 10 meV are slightly displaced in H to lower values away from the commensurate H=0.5 position.   The nature of these incommensurate fluctuations will be discussed in more detail below.  It is interesting to note that while the magnetic fluctuations become considerably broadened at high temperatures, they do remain very anisotropic in the (H,K) plane as illustrated in Fig. \\ref{constE} panel $(a)$ which is at 100 K, well above the Neel transition temperature.  Gaussian fits to the data produce an anisotropy in momentum with widths of $\\xi_{a}$/$\\xi_{b}$=1.85 $\\pm$ 0.10 at 100 K.  Therefore, the high temperature low energy fluctuations in Fe$_{1.057(7)}$Te are anisotropic in momentum, despite the tetragonal shape of the lattice and the equivalence of the $a$ and $b$ directions.  However, these fluctuations centered around the ($\\pi$, 0) position do preserve the C$_{4}$ symmetry of the lattice and should be distinguished from the ``nematic\" phase fluctuations identified in the ``122\" pnictides at high temperatures.~\\cite{Lu14:345,Fernandes14:10}  The anisotropy around the ($\\pi$,0) position may reflect the underlying Fermi surface~\\cite{Subedi08:78} as suggested to explain a similar anisotropy in the magnetic fluctuations in iron based pnictides.~\\cite{Park10:82,Graser09:11,Lu14:345} \n\n\\begin{figure}[t]\n\\includegraphics[width=9.3cm]{elastic_mag.eps}\n\\caption{\\label{pol_elastic} Polarization analysis of the elastic (0.5, 0, 1.5) magnetic Bragg peak showing spin-flip (open circles) and non-spin-flip (filled circles) scattering with the incident beam of neutrons polarized along the X, Y, and Z directions as defined in the main text.  The peak in the non spin-flip channel in panel $(a)$ is the result of incomplete polarization of the neutron beam and is defined by the flipping ratio.} \n\\end{figure}\n\nWe now investigate the polarization of the magnetic fluctuations as a function of temperature using polarized neutron scattering obtained at the 4F1 triple-axis spectrometer.  Figure \\ref{pol_elastic} illustrates scans through the low temperature elastic magnetic Bragg peak at (0.5, 0, 1.5).  Spin-flip (open circles) and non spin-flip (filled circles) are illustrated for the neutron beam polarized along the X (defined as parallel $\\vec{Q}$), Y (perpendicular to $\\vec{Q}$, but within the horizontal (H0L) scattering plane), and Z (perpendicular to the $\\vec{Q}$ and perpendicular to the horizontal scattering plane).  Panel $(a)$ shows that the dominant cross section is in the spin-flip channel, as expected for magnetic scattering, with the feed-through measured in the non-spin-flip channel the result of incomplete polarization characterized by the flipping ratio discussed above in the experimental section.   Scans with the polarization along Y indicate a strong spin-flip cross section indicating that the magnetic moment is oriented out of the scattering plane.  This is confirmed by scans with the neutron polarization oriented along Z which show a dominant cross section in the non-spin-flip channel.  Polarization analysis along the Y and Z directions confirm that the magnetic moments are aligned along the $b$ axis, perpendicular to the (H0L) scattering plane chosen for the 4F1 polarized experiments.   This result is consistent with previous powder diffraction and single crystal neutron diffraction reported for the iron deficient side of the Fe$_{1+x}$Te phase diagram.\n\n\\begin{figure}[t]\n\\includegraphics[width=9.0cm]{polarized_cuts.eps}\n\\caption{\\label{pol_cuts} Constant energy scans with polarization analysis at low temperatures at E=8.5 meV (panels $a-c$) and near the Neel transition at E=0.5 meV.  Given that the ordered magnetic moment is aligned along the $b$ axis, the polarized scans illustrated in $(a-c)$ show the dominant magnetic cross section is transverse to the magnetic moment direction.  This is contrasted with panels $(d-f)$ which illustrate a magnetic cross section predominately polarized along the $b$-axis.  The high temperature scattering also appears a $\\vec{Q}_{0}$=($\\sim$ 0.45,0,0.5) which is contrasted with the commensurate magnetic scattering at low temperatures (illustrated by the dotted line).}\n\\end{figure}\n\nHaving reviewed the magnetic structure at low temperatures with elastic neutron scattering with polarization analysis, we now discuss the polarization of the low temperature spin fluctuations.  The low temperature magnetic dynamics in Fe$_{1.057(7)}$Te are gapped for this particular iron concentration~\\cite{Stock11:84,Stock14:90} as shown in Fig. \\ref{constQ}.  In Fig. \\ref{pol_cuts} panels $(a-c)$, we investigate the polarization of these fluctuations at an energy transfer of E=8.5 meV, above the energy gap.  Panel $(a)$ shows the total magnetic cross section as probed in the spin-flip channel with the neutron beam polarized along X.   Panel $(b)$ illustrates the same scan, but now with the neutron beam polarized along the Z direction (perpendicular to $\\vec{Q}$ and the horizontal (H0L) scattering plane utilized on 4F1).  Given the geometry of the spectrometer and sample, this corresponds to the $b$ axis of the sample.  The intensity measured in this channel is, within error, equal to the total magnetic cross section measured in panel $(a)$ with the neutron beam polarized along X.  A small spin-flip cross section is measured with the beam polarized along Y.  This scan is sensitive to spin fluctuations along the $b$ axis of the material and parallel to the low temperature ordered magnetic moment direction.  Given the statistics, it is not clear if this is statistically significant given the flipping ratio.  The main result found in the polarization analysis in panels $(a-c)$ is that the dominant magnetic cross section at E=8.5 meV is transverse to the ordered magnetic moment direction at low temperatures.  We therefore conclude that the low energy spin fluctuations in Fe$_{1.057(7)}$Te are the result of localized spin fluctuations similar to spin-waves in an ordered antiferromagnet.  \n\nFigure \\ref{pol_cuts} $(d-f)$ show polarization analysis at E=0.5 meV of the low energy fluctuations at 70 K near the N\\'eel temperature (T$_{N}$=75 K).     As noted previously~\\cite{Parshall12:85}, these fluctuations are incommensurate at H$\\sim$ 0.45 and this is highlighted by the vertical dashed line at the commensurate H=0.5 position in Fig. \\ref{pol_cuts}.  Panel $(d)$ shows the total magnetic cross section with the neutron beam polarized along $\\vec{Q}$, defined as the X direction.  Panel $(e)$ shows a weaker cross section corresponding to fluctuations perpendicular to the $b$ axis of the sample (the low temperature ordered magnetic moment direction), however, a larger cross section is found in panel $(f)$ with the Y-polarized neutrons.  This analysis suggests a dominant fraction of the neutron cross section at 70 K corresponding to fluctuations polarized along the $b$ axis which are longitudinal fluctuations parallel to the low temperature ordered magnetic moment.  \n\n\\begin{figure}[t]\n\\includegraphics[width=9.0cm]{energy_depend.eps}\n\\caption{\\label{pol_energy} Polarization analysis of the incommensurate fluctuations near the Neel transition with neutrons polarized along the Y and Z directions as defined in the main text.  $(a)$ and $(b)$ illustrate an anisotropy in the fluctuations at 1.0 meV as evidenced by different intensities in the two channels.  At 2.0 meV (panels $c-d$) the fluctuations are isotropic with equal spectral weight in both polarization channels.  The vertical dashed line indicates the (0.5,0,0.5) position highlighting the fact that the high temperature spin fluctuations are incommensurate.}\n\\end{figure}\n\nFigure \\ref{pol_energy} illustrates the energy dependence of the incommensurate fluctuations critical to collinear Neel ordering.  Panels $(a,b)$ show polarization analysis at an energy transfer of 1.0 meV and panels $(c,d)$ at 2.0 meV.  The Y-polarized spin-flip channel is sensitive to fluctuations along the $b$ axis and the Z-polarized channel is sensitive to fluctuations transverse, or perpendicular, to $b$.  An anisotropy is observable at 1.0 meV, however at higher energy transfers of 2.0 meV (panels $c,d$), the excitations are isotropic within error with equal weight residing in the Y and Z polarized spin-flip channels.  This shows that the low energy incommensurate fluctuations are primarily longitudinal in nature, at higher energy transfers the fluctuations become more isotropic.\n\n\\begin{figure}[t]\n\\includegraphics[width=9.0cm]{temp_depend.eps}\n\\caption{\\label{pol_temp}  The temperature, energy, and polarization dependence of the magnetic fluctuations at $\\vec{Q}$=(0.45, 0 , 0.5).  The data was taken on the polarized cold triple-axis spectrometer 4F1.  Panel $(a)$ shows and energy scan with X and Y polarized neutrons at 70 K illustrating that the anisotropy develops between the two channels at low energy transfers.  $(b)$ shows the same constant-Q scan at 100 K illustrating that the fluctuations are isotropic, within error, at this temperature for all energy transfers investigated.  $(c)$ illustrates a temperature scan with E=0.5 meV and $\\vec{Q}$=(0.45,0.5,0.5) for Y and Z polarized neutrons.  The anisotropy between the two channels develops near T$_{N}$.}\n\\end{figure}\n\nFigure \\ref{pol_temp} illustrates background corrected temperature and energy scans for the incommensurate magnetic fluctuations peaked at (0.45, 0, 0.5).  Panel $(a)$ and $(b)$ display constant momentum cuts.  At 70 K, near the N\\'eel temperature, a significant difference develops between the Y and Z polarized channels at low energy transfers below $\\sim$ 1 meV.    At higher temperatures of 100 K displayed in panel $(b)$, the two channels for neutrons polarized along Y and Z have equal intensities within error indicating isotropic fluctuations at all energy transfers studied.  This is expected for a paramagnet at temperatures well above the ordering temperature.   The temperature dependence of the magnetic fluctuations at E=0.5 meV and with $\\vec{Q}$=(0.45, 0, 0.5) is displayed in panel $(c)$ where it is seen that a large difference between the spin-flip channels with Y and Z polarized neutrons is present near and below T$_{N}$.  At high temperatures the two channels are equal within error.\n\nThe polarized neutron scattering results demonstrate anisotropic spin fluctuations which develop near T$_{N}$ in Fe$_{1.057(7)}$Te.  This is evidenced in the difference seen between the Y and Z polarization channels in Figs. \\ref{pol_cuts}, \\ref{pol_energy}, and \\ref{pol_temp} discussed above.  If the magnetic fluctuations were isotropic, the intensity in these two spin-flip channels would be equal and ${1\\over 2}$ the intensity when the neutron beam is polarized along $\\vec{Q}$ as observed in magnets in the paramagnetic region at high temperatures as shown in Refs. \\onlinecite{Wicksted84:30,Ishikawa85:31}.   These anisotropic fluctuations are preferentially polarized along the $b$ axis which is parallel to the low temperature ordered magnetic moment.  However, these fluctuations are located at an incommensurate wave vector of $\\vec{q}_{0}$=($\\sim$ 0.45, 0, 0.5) and are distinct from the low temperature commensurate magnetic order and the fluctuations associated with this order which occurs at (0.5, 0, 0.5).  This indicates that these high temperature fluctuations are associated with a competing phases.  The polarization and also the wavevector are the same as the collinear spin-density wave reviewed above for Fe$_{0.124(5)}$Te.  We therefore conclude that this magnetic density wave phase competes with collinear and commensurate order in the Fe$_{1+x}$Te phase diagram.\n\n",
        "label": "pol_cuts",
        "Descriptive_question1": "What energy value is associated with the low temperature scans in panels a-c of figure_5?",
        "Descriptive_question2": "At what approximate H value does the high temperature scattering appear in figure_5?",
        "Reasoning_question1": "Why do the polarized scans in panels a-c of figure_5 show a dominant magnetic cross section transverse to the magnetic moment direction?",
        "Reasoning_question2": "How does the polarization of the magnetic cross section in panels d-f of figure_5 near the Neel transition differ from that in panels a-c at low temperatures, and what does this suggest about the nature of the fluctuations?",
        "Descriptive_answer1": "8.5 meV",
        "Descriptive_answer2": "~0.45",
        "Reasoning_answer1": "The polarized scans in panels a-c at low temperatures (E=8.5 meV) show the dominant magnetic cross section transverse to the ordered magnetic moment direction because the magnetic fluctuations behave like conventional spin waves in an ordered antiferromagnet, where fluctuations are primarily perpendicular to the ordered moment. This is consistent with localized spin fluctuations, and polarization analysis confirms that the cross section transverse to the b-axis (the moment direction) dominates.",
        "Reasoning_answer2": "Near the Neel transition in panels d-f at E=0.5 meV, the magnetic cross section is predominantly polarized along the b-axis, parallel to the ordered moment direction, unlike the transverse fluctuations seen at low temperatures. This suggests that near the transition temperature, fluctuations become longitudinal (parallel to the moment direction) and are associated with incommensurate wave vectors, indicative of a competing spin-density wave phase rather than conventional spin-wave-like fluctuations."
    },
    {
        "paper_id": "1704.01111.json",
        "image_id": "figure_6",
        "image_path": [
            "/home/yz979/scratch/chengye/scimolmo/test-images-tables/dataset_imagestables/1704.01111/images/energy_depend.eps"
        ],
        "caption": "\\label{pol_energy} Polarization analysis of the incommensurate fluctuations near the Neel transition with neutrons polarized along the Y and Z directions as defined in the main text.  $(a)$ and $(b)$ illustrate an anisotropy in the fluctuations at 1.0 meV as evidenced by different intensities in the two channels.  At 2.0 meV (panels $c-d$) the fluctuations are isotropic with equal spectral weight in both polarization channels.  The vertical dashed line indicates the (0.5,0,0.5) position highlighting the fact that the high temperature spin fluctuations are incommensurate.",
        "classify": "Chart",
        "section_info": "3 x=0.057(7) - Collinear magnetism and stripy fluctuations\n\\section{x=0.057(7) - Collinear magnetism and stripy fluctuations}\n\nWe first discuss the temperature dependent magnetic dynamics in the collinear phase of the Fe$_{1+x}$Te phase diagram by studying single crystals of Fe$_{1.057(7)}$Te.  Fe$_{1.057(7)}$Te is placed on the iron deficient side of the phase diagram shown in Fig. \\ref{structures} and has a first order transition at 75 K to a collinear magnetic phase accompanied by a structural transition from a tetragonal (space group $P4/nmm$) to monoclinic (space group $P2_{1}/m$) unit cell.  We first show how the magnetic fluctuations change in the (H,K) plane as a function of temperature and then study the anisotropy of these temperature dependent fluctuations using polarized neutrons.  \n\n\\begin{figure}[t]\n\\includegraphics[width=8.8cm]{constQ_figure.eps}\n\\caption{\\label{constQ} Constant momentum slices taken on the MAPS spectrometer with E$_{i}$=75 meV on Fe$_{1.057(7)}$Te.  $(a-c)$ show scans and one dimensional cuts along the H direction and $(d-f)$ show scans along the K direction at 5 and 100 K.  The constant energy cuts in panels $(c)$ and $(f)$ were done at 10 $\\pm$ 2 meV.}\n\\end{figure}\n\n\\begin{figure}[t]\n\\includegraphics[width=9.5cm]{constE_figure.eps}\n\\caption{\\label{constE} Constant energy slices taken on the MAPS spectrometer with E$_{i}$=75 meV and k$_{i}$ aligned along the $c$ axis.  $(a-c)$ displays slices at E=10 $\\pm$ 2 meV and $(d-f)$ illustrate scans at 30 $\\pm$ 5 meV.  Temperatures of 5, 70, and 100 K are shown for each energy transfer.}\n\\end{figure}\n\nFigure \\ref{constQ} shows constant momentum slices taken near (H,K)=(0.5,0) at 5 K, below the transition to collinear magnetic order, both along the H and K directions taken on the MAPS chopper spectrometer with E$_{i}$=75 meV.  Unlike the case of Se doped Fe$_{1+x}$Te$_{1-y}$Se$_{y}$ where the magnetic fluctuations are peaked near the ($\\pi$, $\\pi$) position, in parent Fe$_{1+x}$Te, the magnetic correlations are peaked near ($\\pi$, 0).~\\cite{Chi11:84}  As previously published, the low temperature magnetic fluctuations are strongly correlated along both the $a$ and $b$ directions and become one dimensional at higher energy transfers in excess of $\\sim$ 30 meV.~\\cite{Stock14:90}    This is confirmed in the constant momentum slices in panels $(a)$ and $(d)$ and the corresponding cuts in panels $(c)$ and $(f)$ where the magnetic fluctuations are strongly correlated in momentum along both the H and K directions at E=10 meV.  A considerable broadening occurs at high temperatures of 100 K, however, the fluctuations remain anisotropic in momentum at this temperature as illustrated in constant momentum slices in panels $(b)$ and $(e)$ and also cuts $(c)$ and $(f)$ taken at E=10 $\\pm$ 2 meV.\n\nFigure \\ref{constE} displays constant energy slices at low energy transfers of 10 $\\pm$ 2 meV (panesl $a-c$) and also 30 $\\pm$ 5 meV (panels $d-f$).  The data are also from the MAPS spectrometer with E$_{i}$=75 meV.  At low temperatures and low energies displayed in panel $(c)$, a constant energy map shows that the scattering is well correlated in both the $a$ and $b$ directions.  At 70 K (panel $b$) close to the first order transition to collinear order, the results discussed above is further confirmed showing broadened, yet still anisotropic correlations.  At 100 K (well above T$_{N}$), however as illustrated in panel $(a)$, the scattering becomes more isotropic being broader along $b$ yet there is still a clear anisotropy in the correlations along $a$ and $b$.  At higher energies (E=30 $\\pm$ 5 meV displayed in panels $d-f$), a different picture emerges with the magnetic fluctuations being more elongated along the K direction at 5 K indicative of one dimensional fluctuations.  At higher temperatures of 70 K, the magnetic correlations become isotropic along the H and K directions with the scattering forming nearly a ring in momentum at 100 K.\n\nAs noted previously in a high energy neutron scattering study~\\cite{Stock14:90} as a function of interstitial iron concentration, the magnetic excitations extend up to at $\\sim$ 200 meV and this is also confirmed by two-magnon results using Raman.~\\cite{Okazaki11:83}  Within error of $\\pm$ 15 \\%, we observe no temperature dependence to the integrated intensity at 5, 70, and 100 K integrating over energy transfers up to 50 meV.  While the analysis is sensitive to how the elastic line is treated, the increase in spectral weight in the inelastic channel is accounted for by the loss of spectral weight at the magnetic Bragg position within error.   This contrasts with some previous studies on Fe$_{1+x}$Te (Ref.\\onlinecite{Zal11:107}), however we emphasize that our measurements are performed on a different sample which is located at a different point in the magnetic and structural phase diagram drawn in Fig. \\ref{structures}.  We have also discussed possible sources of error due to low-energy phonons in the supplementary information in Ref. \\onlinecite{Stock14:90}.  In the collinear phase of Fe$_{1.057(7)}$Te, we therefore do not observe evidence of a spin transition, but rather a re-distribution of spectral weight from the elastic line to the inelastic position and also throughout the Brillouin zone as a function of temperature. \n\nThe constant energy and momentum cuts in Figs. \\ref{constQ} and \\ref{constE} illustrate that the fluctuations become considerably broadened in momentum and energy crossing the Neel transition (T$_{N}$=75 K).  Fig. \\ref{constQ} panels $(c)$ and $(f)$ show that the magnetic fluctuations remain peaked around K=0 and H=0.5, however at high temperatures of 100 K above the first order magnetic and structural transition, the magnetic fluctuations at 10 meV are slightly displaced in H to lower values away from the commensurate H=0.5 position.   The nature of these incommensurate fluctuations will be discussed in more detail below.  It is interesting to note that while the magnetic fluctuations become considerably broadened at high temperatures, they do remain very anisotropic in the (H,K) plane as illustrated in Fig. \\ref{constE} panel $(a)$ which is at 100 K, well above the Neel transition temperature.  Gaussian fits to the data produce an anisotropy in momentum with widths of $\\xi_{a}$/$\\xi_{b}$=1.85 $\\pm$ 0.10 at 100 K.  Therefore, the high temperature low energy fluctuations in Fe$_{1.057(7)}$Te are anisotropic in momentum, despite the tetragonal shape of the lattice and the equivalence of the $a$ and $b$ directions.  However, these fluctuations centered around the ($\\pi$, 0) position do preserve the C$_{4}$ symmetry of the lattice and should be distinguished from the ``nematic\" phase fluctuations identified in the ``122\" pnictides at high temperatures.~\\cite{Lu14:345,Fernandes14:10}  The anisotropy around the ($\\pi$,0) position may reflect the underlying Fermi surface~\\cite{Subedi08:78} as suggested to explain a similar anisotropy in the magnetic fluctuations in iron based pnictides.~\\cite{Park10:82,Graser09:11,Lu14:345} \n\n\\begin{figure}[t]\n\\includegraphics[width=9.3cm]{elastic_mag.eps}\n\\caption{\\label{pol_elastic} Polarization analysis of the elastic (0.5, 0, 1.5) magnetic Bragg peak showing spin-flip (open circles) and non-spin-flip (filled circles) scattering with the incident beam of neutrons polarized along the X, Y, and Z directions as defined in the main text.  The peak in the non spin-flip channel in panel $(a)$ is the result of incomplete polarization of the neutron beam and is defined by the flipping ratio.} \n\\end{figure}\n\nWe now investigate the polarization of the magnetic fluctuations as a function of temperature using polarized neutron scattering obtained at the 4F1 triple-axis spectrometer.  Figure \\ref{pol_elastic} illustrates scans through the low temperature elastic magnetic Bragg peak at (0.5, 0, 1.5).  Spin-flip (open circles) and non spin-flip (filled circles) are illustrated for the neutron beam polarized along the X (defined as parallel $\\vec{Q}$), Y (perpendicular to $\\vec{Q}$, but within the horizontal (H0L) scattering plane), and Z (perpendicular to the $\\vec{Q}$ and perpendicular to the horizontal scattering plane).  Panel $(a)$ shows that the dominant cross section is in the spin-flip channel, as expected for magnetic scattering, with the feed-through measured in the non-spin-flip channel the result of incomplete polarization characterized by the flipping ratio discussed above in the experimental section.   Scans with the polarization along Y indicate a strong spin-flip cross section indicating that the magnetic moment is oriented out of the scattering plane.  This is confirmed by scans with the neutron polarization oriented along Z which show a dominant cross section in the non-spin-flip channel.  Polarization analysis along the Y and Z directions confirm that the magnetic moments are aligned along the $b$ axis, perpendicular to the (H0L) scattering plane chosen for the 4F1 polarized experiments.   This result is consistent with previous powder diffraction and single crystal neutron diffraction reported for the iron deficient side of the Fe$_{1+x}$Te phase diagram.\n\n\\begin{figure}[t]\n\\includegraphics[width=9.0cm]{polarized_cuts.eps}\n\\caption{\\label{pol_cuts} Constant energy scans with polarization analysis at low temperatures at E=8.5 meV (panels $a-c$) and near the Neel transition at E=0.5 meV.  Given that the ordered magnetic moment is aligned along the $b$ axis, the polarized scans illustrated in $(a-c)$ show the dominant magnetic cross section is transverse to the magnetic moment direction.  This is contrasted with panels $(d-f)$ which illustrate a magnetic cross section predominately polarized along the $b$-axis.  The high temperature scattering also appears a $\\vec{Q}_{0}$=($\\sim$ 0.45,0,0.5) which is contrasted with the commensurate magnetic scattering at low temperatures (illustrated by the dotted line).}\n\\end{figure}\n\nHaving reviewed the magnetic structure at low temperatures with elastic neutron scattering with polarization analysis, we now discuss the polarization of the low temperature spin fluctuations.  The low temperature magnetic dynamics in Fe$_{1.057(7)}$Te are gapped for this particular iron concentration~\\cite{Stock11:84,Stock14:90} as shown in Fig. \\ref{constQ}.  In Fig. \\ref{pol_cuts} panels $(a-c)$, we investigate the polarization of these fluctuations at an energy transfer of E=8.5 meV, above the energy gap.  Panel $(a)$ shows the total magnetic cross section as probed in the spin-flip channel with the neutron beam polarized along X.   Panel $(b)$ illustrates the same scan, but now with the neutron beam polarized along the Z direction (perpendicular to $\\vec{Q}$ and the horizontal (H0L) scattering plane utilized on 4F1).  Given the geometry of the spectrometer and sample, this corresponds to the $b$ axis of the sample.  The intensity measured in this channel is, within error, equal to the total magnetic cross section measured in panel $(a)$ with the neutron beam polarized along X.  A small spin-flip cross section is measured with the beam polarized along Y.  This scan is sensitive to spin fluctuations along the $b$ axis of the material and parallel to the low temperature ordered magnetic moment direction.  Given the statistics, it is not clear if this is statistically significant given the flipping ratio.  The main result found in the polarization analysis in panels $(a-c)$ is that the dominant magnetic cross section at E=8.5 meV is transverse to the ordered magnetic moment direction at low temperatures.  We therefore conclude that the low energy spin fluctuations in Fe$_{1.057(7)}$Te are the result of localized spin fluctuations similar to spin-waves in an ordered antiferromagnet.  \n\nFigure \\ref{pol_cuts} $(d-f)$ show polarization analysis at E=0.5 meV of the low energy fluctuations at 70 K near the N\\'eel temperature (T$_{N}$=75 K).     As noted previously~\\cite{Parshall12:85}, these fluctuations are incommensurate at H$\\sim$ 0.45 and this is highlighted by the vertical dashed line at the commensurate H=0.5 position in Fig. \\ref{pol_cuts}.  Panel $(d)$ shows the total magnetic cross section with the neutron beam polarized along $\\vec{Q}$, defined as the X direction.  Panel $(e)$ shows a weaker cross section corresponding to fluctuations perpendicular to the $b$ axis of the sample (the low temperature ordered magnetic moment direction), however, a larger cross section is found in panel $(f)$ with the Y-polarized neutrons.  This analysis suggests a dominant fraction of the neutron cross section at 70 K corresponding to fluctuations polarized along the $b$ axis which are longitudinal fluctuations parallel to the low temperature ordered magnetic moment.  \n\n\\begin{figure}[t]\n\\includegraphics[width=9.0cm]{energy_depend.eps}\n\\caption{\\label{pol_energy} Polarization analysis of the incommensurate fluctuations near the Neel transition with neutrons polarized along the Y and Z directions as defined in the main text.  $(a)$ and $(b)$ illustrate an anisotropy in the fluctuations at 1.0 meV as evidenced by different intensities in the two channels.  At 2.0 meV (panels $c-d$) the fluctuations are isotropic with equal spectral weight in both polarization channels.  The vertical dashed line indicates the (0.5,0,0.5) position highlighting the fact that the high temperature spin fluctuations are incommensurate.}\n\\end{figure}\n\nFigure \\ref{pol_energy} illustrates the energy dependence of the incommensurate fluctuations critical to collinear Neel ordering.  Panels $(a,b)$ show polarization analysis at an energy transfer of 1.0 meV and panels $(c,d)$ at 2.0 meV.  The Y-polarized spin-flip channel is sensitive to fluctuations along the $b$ axis and the Z-polarized channel is sensitive to fluctuations transverse, or perpendicular, to $b$.  An anisotropy is observable at 1.0 meV, however at higher energy transfers of 2.0 meV (panels $c,d$), the excitations are isotropic within error with equal weight residing in the Y and Z polarized spin-flip channels.  This shows that the low energy incommensurate fluctuations are primarily longitudinal in nature, at higher energy transfers the fluctuations become more isotropic.\n\n\\begin{figure}[t]\n\\includegraphics[width=9.0cm]{temp_depend.eps}\n\\caption{\\label{pol_temp}  The temperature, energy, and polarization dependence of the magnetic fluctuations at $\\vec{Q}$=(0.45, 0 , 0.5).  The data was taken on the polarized cold triple-axis spectrometer 4F1.  Panel $(a)$ shows and energy scan with X and Y polarized neutrons at 70 K illustrating that the anisotropy develops between the two channels at low energy transfers.  $(b)$ shows the same constant-Q scan at 100 K illustrating that the fluctuations are isotropic, within error, at this temperature for all energy transfers investigated.  $(c)$ illustrates a temperature scan with E=0.5 meV and $\\vec{Q}$=(0.45,0.5,0.5) for Y and Z polarized neutrons.  The anisotropy between the two channels develops near T$_{N}$.}\n\\end{figure}\n\nFigure \\ref{pol_temp} illustrates background corrected temperature and energy scans for the incommensurate magnetic fluctuations peaked at (0.45, 0, 0.5).  Panel $(a)$ and $(b)$ display constant momentum cuts.  At 70 K, near the N\\'eel temperature, a significant difference develops between the Y and Z polarized channels at low energy transfers below $\\sim$ 1 meV.    At higher temperatures of 100 K displayed in panel $(b)$, the two channels for neutrons polarized along Y and Z have equal intensities within error indicating isotropic fluctuations at all energy transfers studied.  This is expected for a paramagnet at temperatures well above the ordering temperature.   The temperature dependence of the magnetic fluctuations at E=0.5 meV and with $\\vec{Q}$=(0.45, 0, 0.5) is displayed in panel $(c)$ where it is seen that a large difference between the spin-flip channels with Y and Z polarized neutrons is present near and below T$_{N}$.  At high temperatures the two channels are equal within error.\n\nThe polarized neutron scattering results demonstrate anisotropic spin fluctuations which develop near T$_{N}$ in Fe$_{1.057(7)}$Te.  This is evidenced in the difference seen between the Y and Z polarization channels in Figs. \\ref{pol_cuts}, \\ref{pol_energy}, and \\ref{pol_temp} discussed above.  If the magnetic fluctuations were isotropic, the intensity in these two spin-flip channels would be equal and ${1\\over 2}$ the intensity when the neutron beam is polarized along $\\vec{Q}$ as observed in magnets in the paramagnetic region at high temperatures as shown in Refs. \\onlinecite{Wicksted84:30,Ishikawa85:31}.   These anisotropic fluctuations are preferentially polarized along the $b$ axis which is parallel to the low temperature ordered magnetic moment.  However, these fluctuations are located at an incommensurate wave vector of $\\vec{q}_{0}$=($\\sim$ 0.45, 0, 0.5) and are distinct from the low temperature commensurate magnetic order and the fluctuations associated with this order which occurs at (0.5, 0, 0.5).  This indicates that these high temperature fluctuations are associated with a competing phases.  The polarization and also the wavevector are the same as the collinear spin-density wave reviewed above for Fe$_{0.124(5)}$Te.  We therefore conclude that this magnetic density wave phase competes with collinear and commensurate order in the Fe$_{1+x}$Te phase diagram.\n\n",
        "label": "pol_energy",
        "Descriptive_question1": "What energy transfer is analyzed in panels (a) and (b) of figure_6?",
        "Descriptive_question2": "What position is indicated by the vertical dashed line in figure_6?",
        "Reasoning_question1": "What does the difference in intensities between the Y and Z polarization channels at 1.0 meV in figure_6 suggest about the nature of the magnetic fluctuations near the Neel transition?",
        "Reasoning_question2": "How does the isotropy of fluctuations at 2.0 meV in figure_6 compare to those at 1.0 meV, and what might this imply about the energy dependence of magnetic anisotropy?",
        "Descriptive_answer1": "1.0 meV",
        "Descriptive_answer2": "(0.5,0,0.5)",
        "Reasoning_answer1": "In panels (a) and (b) of figure_6, there is a clear anisotropy in intensities between the Y and Z polarized neutron scattering channels at 1.0 meV. Since the Y polarization corresponds to fluctuations along the b axis and Z corresponds to fluctuations perpendicular to b, the differing intensities imply that the magnetic fluctuations near the Neel transition are predominantly longitudinal, i.e., polarized along the b axis rather than isotropic. This anisotropy reveals that low-energy fluctuations are directionally dependent and likely influenced by the ordered moment direction near T_N.",
        "Reasoning_answer2": "At 2.0 meV, panels (c) and (d) show equal spectral weight in both the Y and Z polarization channels, indicating isotropic magnetic fluctuations at this higher energy. Comparing this to the anisotropic fluctuations at 1.0 meV, it suggests that the magnetic anisotropy diminishes with increasing energy transfer. This implies that low-energy magnetic excitations near the Neel transition have a strong directional dependence, but as energy increases beyond 1.0 meV, fluctuations become more isotropic and directionally uniform, reflecting an energy-dependent crossover in magnetic fluctuation character."
    },
    {
        "paper_id": "1704.01111.json",
        "image_id": "figure_7",
        "image_path": [
            "/home/yz979/scratch/chengye/scimolmo/test-images-tables/dataset_imagestables/1704.01111/images/temp_depend.eps"
        ],
        "caption": "\\label{pol_temp}  The temperature, energy, and polarization dependence of the magnetic fluctuations at $\\vec{Q}$=(0.45, 0 , 0.5).  The data was taken on the polarized cold triple-axis spectrometer 4F1.  Panel $(a)$ shows and energy scan with X and Y polarized neutrons at 70 K illustrating that the anisotropy develops between the two channels at low energy transfers.  $(b)$ shows the same constant-Q scan at 100 K illustrating that the fluctuations are isotropic, within error, at this temperature for all energy transfers investigated.  $(c)$ illustrates a temperature scan with E=0.5 meV and $\\vec{Q}$=(0.45,0.5,0.5) for Y and Z polarized neutrons.  The anisotropy between the two channels develops near T$_{N}$.",
        "classify": "Chart",
        "section_info": "3 x=0.057(7) - Collinear magnetism and stripy fluctuations\n\\section{x=0.057(7) - Collinear magnetism and stripy fluctuations}\n\nWe first discuss the temperature dependent magnetic dynamics in the collinear phase of the Fe$_{1+x}$Te phase diagram by studying single crystals of Fe$_{1.057(7)}$Te.  Fe$_{1.057(7)}$Te is placed on the iron deficient side of the phase diagram shown in Fig. \\ref{structures} and has a first order transition at 75 K to a collinear magnetic phase accompanied by a structural transition from a tetragonal (space group $P4/nmm$) to monoclinic (space group $P2_{1}/m$) unit cell.  We first show how the magnetic fluctuations change in the (H,K) plane as a function of temperature and then study the anisotropy of these temperature dependent fluctuations using polarized neutrons.  \n\n\\begin{figure}[t]\n\\includegraphics[width=8.8cm]{constQ_figure.eps}\n\\caption{\\label{constQ} Constant momentum slices taken on the MAPS spectrometer with E$_{i}$=75 meV on Fe$_{1.057(7)}$Te.  $(a-c)$ show scans and one dimensional cuts along the H direction and $(d-f)$ show scans along the K direction at 5 and 100 K.  The constant energy cuts in panels $(c)$ and $(f)$ were done at 10 $\\pm$ 2 meV.}\n\\end{figure}\n\n\\begin{figure}[t]\n\\includegraphics[width=9.5cm]{constE_figure.eps}\n\\caption{\\label{constE} Constant energy slices taken on the MAPS spectrometer with E$_{i}$=75 meV and k$_{i}$ aligned along the $c$ axis.  $(a-c)$ displays slices at E=10 $\\pm$ 2 meV and $(d-f)$ illustrate scans at 30 $\\pm$ 5 meV.  Temperatures of 5, 70, and 100 K are shown for each energy transfer.}\n\\end{figure}\n\nFigure \\ref{constQ} shows constant momentum slices taken near (H,K)=(0.5,0) at 5 K, below the transition to collinear magnetic order, both along the H and K directions taken on the MAPS chopper spectrometer with E$_{i}$=75 meV.  Unlike the case of Se doped Fe$_{1+x}$Te$_{1-y}$Se$_{y}$ where the magnetic fluctuations are peaked near the ($\\pi$, $\\pi$) position, in parent Fe$_{1+x}$Te, the magnetic correlations are peaked near ($\\pi$, 0).~\\cite{Chi11:84}  As previously published, the low temperature magnetic fluctuations are strongly correlated along both the $a$ and $b$ directions and become one dimensional at higher energy transfers in excess of $\\sim$ 30 meV.~\\cite{Stock14:90}    This is confirmed in the constant momentum slices in panels $(a)$ and $(d)$ and the corresponding cuts in panels $(c)$ and $(f)$ where the magnetic fluctuations are strongly correlated in momentum along both the H and K directions at E=10 meV.  A considerable broadening occurs at high temperatures of 100 K, however, the fluctuations remain anisotropic in momentum at this temperature as illustrated in constant momentum slices in panels $(b)$ and $(e)$ and also cuts $(c)$ and $(f)$ taken at E=10 $\\pm$ 2 meV.\n\nFigure \\ref{constE} displays constant energy slices at low energy transfers of 10 $\\pm$ 2 meV (panesl $a-c$) and also 30 $\\pm$ 5 meV (panels $d-f$).  The data are also from the MAPS spectrometer with E$_{i}$=75 meV.  At low temperatures and low energies displayed in panel $(c)$, a constant energy map shows that the scattering is well correlated in both the $a$ and $b$ directions.  At 70 K (panel $b$) close to the first order transition to collinear order, the results discussed above is further confirmed showing broadened, yet still anisotropic correlations.  At 100 K (well above T$_{N}$), however as illustrated in panel $(a)$, the scattering becomes more isotropic being broader along $b$ yet there is still a clear anisotropy in the correlations along $a$ and $b$.  At higher energies (E=30 $\\pm$ 5 meV displayed in panels $d-f$), a different picture emerges with the magnetic fluctuations being more elongated along the K direction at 5 K indicative of one dimensional fluctuations.  At higher temperatures of 70 K, the magnetic correlations become isotropic along the H and K directions with the scattering forming nearly a ring in momentum at 100 K.\n\nAs noted previously in a high energy neutron scattering study~\\cite{Stock14:90} as a function of interstitial iron concentration, the magnetic excitations extend up to at $\\sim$ 200 meV and this is also confirmed by two-magnon results using Raman.~\\cite{Okazaki11:83}  Within error of $\\pm$ 15 \\%, we observe no temperature dependence to the integrated intensity at 5, 70, and 100 K integrating over energy transfers up to 50 meV.  While the analysis is sensitive to how the elastic line is treated, the increase in spectral weight in the inelastic channel is accounted for by the loss of spectral weight at the magnetic Bragg position within error.   This contrasts with some previous studies on Fe$_{1+x}$Te (Ref.\\onlinecite{Zal11:107}), however we emphasize that our measurements are performed on a different sample which is located at a different point in the magnetic and structural phase diagram drawn in Fig. \\ref{structures}.  We have also discussed possible sources of error due to low-energy phonons in the supplementary information in Ref. \\onlinecite{Stock14:90}.  In the collinear phase of Fe$_{1.057(7)}$Te, we therefore do not observe evidence of a spin transition, but rather a re-distribution of spectral weight from the elastic line to the inelastic position and also throughout the Brillouin zone as a function of temperature. \n\nThe constant energy and momentum cuts in Figs. \\ref{constQ} and \\ref{constE} illustrate that the fluctuations become considerably broadened in momentum and energy crossing the Neel transition (T$_{N}$=75 K).  Fig. \\ref{constQ} panels $(c)$ and $(f)$ show that the magnetic fluctuations remain peaked around K=0 and H=0.5, however at high temperatures of 100 K above the first order magnetic and structural transition, the magnetic fluctuations at 10 meV are slightly displaced in H to lower values away from the commensurate H=0.5 position.   The nature of these incommensurate fluctuations will be discussed in more detail below.  It is interesting to note that while the magnetic fluctuations become considerably broadened at high temperatures, they do remain very anisotropic in the (H,K) plane as illustrated in Fig. \\ref{constE} panel $(a)$ which is at 100 K, well above the Neel transition temperature.  Gaussian fits to the data produce an anisotropy in momentum with widths of $\\xi_{a}$/$\\xi_{b}$=1.85 $\\pm$ 0.10 at 100 K.  Therefore, the high temperature low energy fluctuations in Fe$_{1.057(7)}$Te are anisotropic in momentum, despite the tetragonal shape of the lattice and the equivalence of the $a$ and $b$ directions.  However, these fluctuations centered around the ($\\pi$, 0) position do preserve the C$_{4}$ symmetry of the lattice and should be distinguished from the ``nematic\" phase fluctuations identified in the ``122\" pnictides at high temperatures.~\\cite{Lu14:345,Fernandes14:10}  The anisotropy around the ($\\pi$,0) position may reflect the underlying Fermi surface~\\cite{Subedi08:78} as suggested to explain a similar anisotropy in the magnetic fluctuations in iron based pnictides.~\\cite{Park10:82,Graser09:11,Lu14:345} \n\n\\begin{figure}[t]\n\\includegraphics[width=9.3cm]{elastic_mag.eps}\n\\caption{\\label{pol_elastic} Polarization analysis of the elastic (0.5, 0, 1.5) magnetic Bragg peak showing spin-flip (open circles) and non-spin-flip (filled circles) scattering with the incident beam of neutrons polarized along the X, Y, and Z directions as defined in the main text.  The peak in the non spin-flip channel in panel $(a)$ is the result of incomplete polarization of the neutron beam and is defined by the flipping ratio.} \n\\end{figure}\n\nWe now investigate the polarization of the magnetic fluctuations as a function of temperature using polarized neutron scattering obtained at the 4F1 triple-axis spectrometer.  Figure \\ref{pol_elastic} illustrates scans through the low temperature elastic magnetic Bragg peak at (0.5, 0, 1.5).  Spin-flip (open circles) and non spin-flip (filled circles) are illustrated for the neutron beam polarized along the X (defined as parallel $\\vec{Q}$), Y (perpendicular to $\\vec{Q}$, but within the horizontal (H0L) scattering plane), and Z (perpendicular to the $\\vec{Q}$ and perpendicular to the horizontal scattering plane).  Panel $(a)$ shows that the dominant cross section is in the spin-flip channel, as expected for magnetic scattering, with the feed-through measured in the non-spin-flip channel the result of incomplete polarization characterized by the flipping ratio discussed above in the experimental section.   Scans with the polarization along Y indicate a strong spin-flip cross section indicating that the magnetic moment is oriented out of the scattering plane.  This is confirmed by scans with the neutron polarization oriented along Z which show a dominant cross section in the non-spin-flip channel.  Polarization analysis along the Y and Z directions confirm that the magnetic moments are aligned along the $b$ axis, perpendicular to the (H0L) scattering plane chosen for the 4F1 polarized experiments.   This result is consistent with previous powder diffraction and single crystal neutron diffraction reported for the iron deficient side of the Fe$_{1+x}$Te phase diagram.\n\n\\begin{figure}[t]\n\\includegraphics[width=9.0cm]{polarized_cuts.eps}\n\\caption{\\label{pol_cuts} Constant energy scans with polarization analysis at low temperatures at E=8.5 meV (panels $a-c$) and near the Neel transition at E=0.5 meV.  Given that the ordered magnetic moment is aligned along the $b$ axis, the polarized scans illustrated in $(a-c)$ show the dominant magnetic cross section is transverse to the magnetic moment direction.  This is contrasted with panels $(d-f)$ which illustrate a magnetic cross section predominately polarized along the $b$-axis.  The high temperature scattering also appears a $\\vec{Q}_{0}$=($\\sim$ 0.45,0,0.5) which is contrasted with the commensurate magnetic scattering at low temperatures (illustrated by the dotted line).}\n\\end{figure}\n\nHaving reviewed the magnetic structure at low temperatures with elastic neutron scattering with polarization analysis, we now discuss the polarization of the low temperature spin fluctuations.  The low temperature magnetic dynamics in Fe$_{1.057(7)}$Te are gapped for this particular iron concentration~\\cite{Stock11:84,Stock14:90} as shown in Fig. \\ref{constQ}.  In Fig. \\ref{pol_cuts} panels $(a-c)$, we investigate the polarization of these fluctuations at an energy transfer of E=8.5 meV, above the energy gap.  Panel $(a)$ shows the total magnetic cross section as probed in the spin-flip channel with the neutron beam polarized along X.   Panel $(b)$ illustrates the same scan, but now with the neutron beam polarized along the Z direction (perpendicular to $\\vec{Q}$ and the horizontal (H0L) scattering plane utilized on 4F1).  Given the geometry of the spectrometer and sample, this corresponds to the $b$ axis of the sample.  The intensity measured in this channel is, within error, equal to the total magnetic cross section measured in panel $(a)$ with the neutron beam polarized along X.  A small spin-flip cross section is measured with the beam polarized along Y.  This scan is sensitive to spin fluctuations along the $b$ axis of the material and parallel to the low temperature ordered magnetic moment direction.  Given the statistics, it is not clear if this is statistically significant given the flipping ratio.  The main result found in the polarization analysis in panels $(a-c)$ is that the dominant magnetic cross section at E=8.5 meV is transverse to the ordered magnetic moment direction at low temperatures.  We therefore conclude that the low energy spin fluctuations in Fe$_{1.057(7)}$Te are the result of localized spin fluctuations similar to spin-waves in an ordered antiferromagnet.  \n\nFigure \\ref{pol_cuts} $(d-f)$ show polarization analysis at E=0.5 meV of the low energy fluctuations at 70 K near the N\\'eel temperature (T$_{N}$=75 K).     As noted previously~\\cite{Parshall12:85}, these fluctuations are incommensurate at H$\\sim$ 0.45 and this is highlighted by the vertical dashed line at the commensurate H=0.5 position in Fig. \\ref{pol_cuts}.  Panel $(d)$ shows the total magnetic cross section with the neutron beam polarized along $\\vec{Q}$, defined as the X direction.  Panel $(e)$ shows a weaker cross section corresponding to fluctuations perpendicular to the $b$ axis of the sample (the low temperature ordered magnetic moment direction), however, a larger cross section is found in panel $(f)$ with the Y-polarized neutrons.  This analysis suggests a dominant fraction of the neutron cross section at 70 K corresponding to fluctuations polarized along the $b$ axis which are longitudinal fluctuations parallel to the low temperature ordered magnetic moment.  \n\n\\begin{figure}[t]\n\\includegraphics[width=9.0cm]{energy_depend.eps}\n\\caption{\\label{pol_energy} Polarization analysis of the incommensurate fluctuations near the Neel transition with neutrons polarized along the Y and Z directions as defined in the main text.  $(a)$ and $(b)$ illustrate an anisotropy in the fluctuations at 1.0 meV as evidenced by different intensities in the two channels.  At 2.0 meV (panels $c-d$) the fluctuations are isotropic with equal spectral weight in both polarization channels.  The vertical dashed line indicates the (0.5,0,0.5) position highlighting the fact that the high temperature spin fluctuations are incommensurate.}\n\\end{figure}\n\nFigure \\ref{pol_energy} illustrates the energy dependence of the incommensurate fluctuations critical to collinear Neel ordering.  Panels $(a,b)$ show polarization analysis at an energy transfer of 1.0 meV and panels $(c,d)$ at 2.0 meV.  The Y-polarized spin-flip channel is sensitive to fluctuations along the $b$ axis and the Z-polarized channel is sensitive to fluctuations transverse, or perpendicular, to $b$.  An anisotropy is observable at 1.0 meV, however at higher energy transfers of 2.0 meV (panels $c,d$), the excitations are isotropic within error with equal weight residing in the Y and Z polarized spin-flip channels.  This shows that the low energy incommensurate fluctuations are primarily longitudinal in nature, at higher energy transfers the fluctuations become more isotropic.\n\n\\begin{figure}[t]\n\\includegraphics[width=9.0cm]{temp_depend.eps}\n\\caption{\\label{pol_temp}  The temperature, energy, and polarization dependence of the magnetic fluctuations at $\\vec{Q}$=(0.45, 0 , 0.5).  The data was taken on the polarized cold triple-axis spectrometer 4F1.  Panel $(a)$ shows and energy scan with X and Y polarized neutrons at 70 K illustrating that the anisotropy develops between the two channels at low energy transfers.  $(b)$ shows the same constant-Q scan at 100 K illustrating that the fluctuations are isotropic, within error, at this temperature for all energy transfers investigated.  $(c)$ illustrates a temperature scan with E=0.5 meV and $\\vec{Q}$=(0.45,0.5,0.5) for Y and Z polarized neutrons.  The anisotropy between the two channels develops near T$_{N}$.}\n\\end{figure}\n\nFigure \\ref{pol_temp} illustrates background corrected temperature and energy scans for the incommensurate magnetic fluctuations peaked at (0.45, 0, 0.5).  Panel $(a)$ and $(b)$ display constant momentum cuts.  At 70 K, near the N\\'eel temperature, a significant difference develops between the Y and Z polarized channels at low energy transfers below $\\sim$ 1 meV.    At higher temperatures of 100 K displayed in panel $(b)$, the two channels for neutrons polarized along Y and Z have equal intensities within error indicating isotropic fluctuations at all energy transfers studied.  This is expected for a paramagnet at temperatures well above the ordering temperature.   The temperature dependence of the magnetic fluctuations at E=0.5 meV and with $\\vec{Q}$=(0.45, 0, 0.5) is displayed in panel $(c)$ where it is seen that a large difference between the spin-flip channels with Y and Z polarized neutrons is present near and below T$_{N}$.  At high temperatures the two channels are equal within error.\n\nThe polarized neutron scattering results demonstrate anisotropic spin fluctuations which develop near T$_{N}$ in Fe$_{1.057(7)}$Te.  This is evidenced in the difference seen between the Y and Z polarization channels in Figs. \\ref{pol_cuts}, \\ref{pol_energy}, and \\ref{pol_temp} discussed above.  If the magnetic fluctuations were isotropic, the intensity in these two spin-flip channels would be equal and ${1\\over 2}$ the intensity when the neutron beam is polarized along $\\vec{Q}$ as observed in magnets in the paramagnetic region at high temperatures as shown in Refs. \\onlinecite{Wicksted84:30,Ishikawa85:31}.   These anisotropic fluctuations are preferentially polarized along the $b$ axis which is parallel to the low temperature ordered magnetic moment.  However, these fluctuations are located at an incommensurate wave vector of $\\vec{q}_{0}$=($\\sim$ 0.45, 0, 0.5) and are distinct from the low temperature commensurate magnetic order and the fluctuations associated with this order which occurs at (0.5, 0, 0.5).  This indicates that these high temperature fluctuations are associated with a competing phases.  The polarization and also the wavevector are the same as the collinear spin-density wave reviewed above for Fe$_{0.124(5)}$Te.  We therefore conclude that this magnetic density wave phase competes with collinear and commensurate order in the Fe$_{1+x}$Te phase diagram.\n\n",
        "label": "pol_temp",
        "Descriptive_question1": "What temperature is shown in panel (a) of figure_7?",
        "Descriptive_question2": "What energy transfer value is used in the temperature scan in panel (c) of figure_7?",
        "Reasoning_question1": "Why do the magnetic fluctuations show anisotropy between the Y and Z polarized channels at 70 K but not at 100 K in figure_7?",
        "Reasoning_question2": "How does the development of anisotropy near T_N in figure_7 panel (c) relate to the transition to collinear magnetic order in Fe_{1.057(7)}Te?",
        "Descriptive_answer1": "70 K",
        "Descriptive_answer2": "0.5 meV",
        "Reasoning_answer1": "At 70 K, which is near the Neel temperature T_N (75 K), the system is transitioning from a paramagnetic to an ordered magnetic state. This proximity to T_N causes the magnetic fluctuations to become anisotropic, favoring certain spin directions (notably the b axis), resulting in a difference in intensity between the Y and Z polarized neutron channels. By contrast, at 100 K, well above T_N, the system is in a paramagnetic state where the magnetic fluctuations are isotropic, so Y and Z polarized channels show equal intensities within experimental error.",
        "Reasoning_answer2": "Figure 7 panel (c) shows the temperature-dependent magnetic fluctuations at E=0.5 meV. The anisotropy between the Y and Z polarization channels develops near T_N, indicating that as the system approaches the antiferromagnetic transition, spin fluctuations become preferentially aligned along the b axis, the direction of the ordered moments below T_N. This suggests that the onset of anisotropy in fluctuations is a precursor to, and related to, the establishment of collinear magnetic order in Fe_{1.057(7)}Te."
    },
    {
        "paper_id": "1704.01111.json",
        "image_id": "figure_8",
        "image_path": [
            "/home/yz979/scratch/chengye/scimolmo/test-images-tables/dataset_imagestables/1704.01111/images/refinement.eps"
        ],
        "caption": "\\label{refinement} Results from single magnetic structure refinement on a single crystal of Fe$_{1.141(5)}$Te performed on the HB-3A diffractometer.  $(a)$ shows the results of a refinement where the interstitial iron moment sizes was allowed to vary while $(b)$ shows a refinement where they were constrained to be equal.  For model $(a)$ the refined size of the Fe moment in the FeTe layers is 2.01(2) $\\mu_{B}$ and 3.5(5) $\\mu_{B}$ on the interstitial site.  In model $(b)$, the iron moments in both the FeTe layers and interstitial sites were constrained to be equal giving a refined moment size of 2.1(1) $\\mu_{B}$.",
        "classify": "Chart",
        "section_info": "4 The magnetic structure in x=0.141(5) - Helical magnetism and ordered interstitial iron sites\n\\section{The magnetic structure in x=0.141(5) - Helical magnetism and ordered interstitial iron sites}\n\nHaving discussed the competition between and localized collinear magnetism and spin density wave phase in iron deficient Fe$_{1+x}$Te for $x$ less than $\\sim$ 0.12, we now discuss single crystal neutron diffraction for large concentrations of interstitial iron where helical magnetic order has been previously observed.   \n\nOwing to the presence of ferromagnetic iron oxide near the surface of the single crystal, polarized experiments on large interstitial iron concentrations were not successful.  Therefore we pursued single crystal unpolarized measurements on HB-3A (Oak Ridge).  \n\nLarge concentrations of interstitial iron have been found to result in semiconducting or poorly metallic behavior over a broad temperature range.~\\cite{Rodriguez13:88}  Further transport studies on superconducting samples of Fe$_{1+x}$Te$_{1-y}$(Se,S)$_{y}$ found evidence for interstitial iron even causing charge localization.~\\cite{Liu09:80}  Here we use single crystal neutron diffraction to investigate the magnetism on the interstitial iron site in Fe$_{1.141(5)}$Te.\n\n\\begin{figure}[t]\n\\includegraphics[width=8.5cm]{refinement.eps}\n\\caption{\\label{refinement} Results from single magnetic structure refinement on a single crystal of Fe$_{1.141(5)}$Te performed on the HB-3A diffractometer.  $(a)$ shows the results of a refinement where the interstitial iron moment sizes was allowed to vary while $(b)$ shows a refinement where they were constrained to be equal.  For model $(a)$ the refined size of the Fe moment in the FeTe layers is 2.01(2) $\\mu_{B}$ and 3.5(5) $\\mu_{B}$ on the interstitial site.  In model $(b)$, the iron moments in both the FeTe layers and interstitial sites were constrained to be equal giving a refined moment size of 2.1(1) $\\mu_{B}$.}\n\\end{figure}\n\nResults of a single crystal refinement for Fe$_{1.141(5)}$Te is illustrated in Fig. \\ref{refinement} which plots $|F_{cal}|^{2}$ as a function of $|F_{obs}|^{2}$ with the $R$-factor listed for each fit.  This particular concentration of interstitial iron is placed beyond the Lifshitz point separating collinear and helical magnetism.  Two models are shown in Fig. \\ref{refinement}, the first where the interstitial iron moment size was allowed to vary independently of the moment size in the FeTe layers and the second where both were constrained to be equal.   The first model (panel $a$) refines to 2.01(2) $\\mu_{B}$ and 3.5(5) $\\mu_{B}$ respectively for iron in the FeTe layers and interstitial sites respectively.   The constrained model (panel $b$) refines to 2.1(1) $\\mu_{B}$.  \n\nThe refined helical magnetic structure in Fe$_{1.141(5)}$Te is different to the collinear phase found for smaller interstitial iron concentrations.  It is also different to the helical phase in FeAs~\\cite{Rodriguez11:2,Segawa09:78,Selte69:23} which displays a noncollinear spin density wave with the spin amplitude along the $b$ axis direction larger than the $a$ direction.  The magnetic iron moments refine to a uniform helical magnetic structure.  The refinement also illustrates that the interstitial sites are fully ordered with a moment size that is comparable, and larger within error, to ordered magnetic moments within the FeTe layers. We note that while powder diffraction results indicated a substantial magnetic moment on the interstitial site, the single crystal results presented here confirm this result along with the fact that the interstitial site follows the same magnetic structure as the FeTe layers.  \n\n",
        "label": "refinement",
        "Descriptive_question1": "What is the refined moment size of Fe in the FeTe layers in model (a) of figure_8?",
        "Descriptive_question2": "What is the constrained moment size in model (b) of figure_8?",
        "Reasoning_question1": "Why might the interstitial iron moment size in model (a) of figure_8 be larger than the Fe moment in the FeTe layers?",
        "Reasoning_question2": "How does the difference in refinement approaches between model (a) and model (b) in figure_8 affect the interpretation of magnetic moment sizes?",
        "Descriptive_answer1": "2.01(2) μ_B",
        "Descriptive_answer2": "2.1(1) μ_B",
        "Reasoning_answer1": "In model (a), the interstitial iron moment size is larger (3.5(5) μ_B) compared to the Fe moment in the FeTe layers (2.01(2) μ_B). This could be because the interstitial iron atoms have a different local environment or electronic structure, leading to a higher degree of magnetic ordering and a larger moment. Additionally, since interstitial sites may have fewer neighboring atoms, the localized magnetic moments could be less quenched, resulting in an enhanced moment size.",
        "Reasoning_answer2": "Model (a) allows the interstitial iron moments to vary independently from the FeTe layers, capturing potential differences in their magnetic behavior, resulting in distinct moment sizes (2.01(2) μ_B vs 3.5(5) μ_B). Model (b), by constraining the moments to be equal, simplifies the model and yields a single refined moment (2.1(1) μ_B) representing an average. This constraint might mask the real differences and nuances in magnetic moments on different sites, potentially leading to less accurate representation of the magnetic structure."
    },
    {
        "paper_id": "1704.01111.json",
        "image_id": "figure_9",
        "image_path": [
            "/home/yz979/scratch/chengye/scimolmo/test-images-tables/dataset_imagestables/1704.01111/images/prop_vector.eps"
        ],
        "caption": "\\label{prop_vector} $(a)$  Elastic energy slice taken on the MACS spectrometer.  $(b)$ displays a cut with L=0.5 illustrating the incommensurate wavevector.  The smaller peak at lower H results from the inclusion of a magnetic helical phase with a larger interstitial iron concentration.~\\cite{Rodriguez11:84}  $(c-d)$ show cuts taken on the WAND diffractometer illustrating the propagation vector.  The filled circles are symmetrized data and displayed to illustrate the relative positions of the magnetic scattering.",
        "classify": "Chart",
        "section_info": "5 $x$=0.124(5)- Spin density wave and search for charge wave\n\\section{$x$=0.124(5)- Spin density wave and search for charge wave}\n\n\\begin{figure}[t]\n\\includegraphics[width=8.8cm]{prop_vector.eps}\n\\caption{\\label{prop_vector} $(a)$  Elastic energy slice taken on the MACS spectrometer.  $(b)$ displays a cut with L=0.5 illustrating the incommensurate wavevector.  The smaller peak at lower H results from the inclusion of a magnetic helical phase with a larger interstitial iron concentration.~\\cite{Rodriguez11:84}  $(c-d)$ show cuts taken on the WAND diffractometer illustrating the propagation vector.  The filled circles are symmetrized data and displayed to illustrate the relative positions of the magnetic scattering.}\n\\end{figure}\n\nWe now discuss the magnetic properties at the border between collinear and helical order in the Fe$_{1+x}$Te phase diagram by presenting neutron diffraction data on a single crystal of Fe$_{1.124(5)}$Te.\n\nMeasurements of the elastic neutron cross section at the border between collinear antiferromagnetism and the helical phase were done using a single crystal of Fe$_{1.124(5)}$Te.   Figure \\ref{prop_vector} $(a)$ illustrates a constant energy slice at the elastic position taken on the MACS cold triple axis spectrometer at 2 K.  Panel $(b)$ displays a cut along the H direction illustrating the incommensurate wavevector at $q_{0}$=0.46 $\\pm$ 0.01 along the H direction based on fits to a Lorentzian squared lineshape.  Within experimental error, the peak is commensurate along L being positioned at L=0.5 and no observable evidence of second harmonics at 2$q_{0}$ are observable in the data (within 2\\% of the peak height at $q_{0}$=0.46 $\\pm$ 0.01).  As discussed previously in Ref. \\onlinecite{Rodriguez11:84}, the static magnetism corresponding to this peak is short-range along the $a$ axis evidenced by a broader than resolution lineshape along the H direction.  The lineshape is resolution limited along $c$ corresponding to long range order along L.  An analysis based on polarized neutrons found that the magnetic structure is polarized along the $b$ axis in contrast to the helical order for Fe$_{1+x}$Te samples on the iron rich side of the phase diagram.  The magnetic structure at $x$=0.124(5) therefore corresponds to a collinear spin-density wave phase.  A second peak is observed at a lower $q$ position and as discussed in Refs. \\onlinecite{Rodriguez11:84,Rodriguez13:88} based on a polarized neutron analysis, this corresponds to a small inclusion of a helical phase with a larger interstitial iron concentration.  \n\nAs discussed above in the introduction, there have been several reports of magnetism at this incommensurate wave vector, even in superconducting samples doped with Se.  However, it is not clear from the limited momentum range if the peak is incommensurate with respect to the nuclear positions or to the antiferromagnetic H=0.5 point as might be expected based on analogies with cuprates.  Figure \\ref{prop_vector}, panels $(c)$ and $(d)$, show cuts at L=0.5 and L=1.5 taken on WAND where the combined thermal neutron wavelengths and broad detector coverage allow us to study the magnetism over a broad range of momentum transfer.  The cuts prove the result reported previously that the propagation vector is $q_{0}$=0.46 $\\pm$ 0.01 and is incommensurate with respect to the nuclear positions.  This contrasts with some studies that have stated that the propagation vector is taken as (0.5-$\\delta$,0,0.5)~\\cite{Wen12:86} and is only clear in the current data set given the broad momentum coverage afforded by the WAND diffractometer.   \n\n\\begin{figure}[t]\n\\includegraphics[width=9.5cm]{overview_mag.eps}\n\\caption{\\label{wand_mag} Elastic momentum slices obtained from the WAND diffractometer.  $(a)$ and $(b)$ show scans obtained at 4 K and 80 K in the spin density wave and paramagnetic phase respectively. $(c)$ shows a cut along the L direction with H=0.46 $\\pm$ 0.01 r.l.u. showing the half integer commensurate nature of this scattering and also that it follows the expected decay of intensity based on the Fe$^{2+}$ form factor.  Backgrounds with an empty can have been obtained at both temperatures and subtracted.}\n\\end{figure}\n\nFigure \\ref{wand_mag} illustrates an extensive reciprocal space map at 4 K (in the magnetically ordered state) and also at high temperatures of 80 K where Fe$_{1.124(5)}$Te is paramagnetic.   A series of magnetic superlattice peaks are clearly observed at H$\\sim$ 0.46, 1.54 and 2.45 r.l.u. at T=4 K, but absent at high temperatures of 80 K, confirming the magnetic origin.  This is highlighted by the yellow ellipse at H $\\sim$ 0.46 at both temperatures.  Panel $(c)$ plots an L scan at H=0.46 $\\pm$ 0.01 r.l.u. showing that the magnetic peaks appear at the commensurate half integer positions along L and also that the intensity decays with the expected Fe$^{2+}$ form factor.  This is consistent with dipolar selection rules for the intensity based on localized magnetic moments pointing along the $b$ axis.\n\nOur WAND results are not consistent with suggestions of antiphase boundaries separating locally ordered collinear states.~\\cite{Mazin09:5}  While such a structure can produce scattering at incommensurate positions, as observed in stripe phases of nickelates~\\cite{Tranquada96:54} and also cuprates~\\cite{Waki00:61,Waki99:60,Stock04:69,Stock10:82}, it fails to model both the incommensurate wavevector and the lack of higher harmonics that would be associated with a sharp uniaxial boundary.  It has recently been proposed that the structure maybe understood in terms of solitons~\\cite{Materne15:115}, however the magnetic structure proposed would produce a $c$ and $a$ axis component to the scattering in our previous polarized neutron diffraction studies of this compound.  This contradicts the data which is consistent with a component only along the $c$-axis.  We therefore conclude that the short-range static antiferromagnetism observed near interstitial iron concentrations of $x\\sim$ 0.12 is more consistent with a spin density wave where the magnitude of the spin varies along the direction of propagation.  \n\nUsing the wide momentum coverage on WAND, we have also searched for any charge density wave that may accompany this spin density wave.  The single crystal momentum maps in Fig. \\ref{wand_mag} $(a)$ and $(b)$ display no observable superlattice peaks that may be associated with a charge density wave.  A small peak is observed near (2.39 $\\pm$ 0.05, 0, 0$\\pm$ 0.05), however this peak is present at both 4 K and 80 K and is not observable near any other primary nuclear Bragg peak measured in our reciprocal space mapping.  The peak is illustrated in Fig. \\ref{wand_nuc}.  While a coupling between strain or charge and magnetism is expected,~\\cite{Turner09:80} as displayed in Cr metal~\\cite{Fawcett88:60}, the charge density wave peak intensity is in proportion to the spin density wave and should be at harmonic of the primary wavevector which is not the case here.~\\cite{Pynn76:13,Kotani76:41,Kotani78:44}   \n\nThe lack of consistency with what has been discussed in relation to coupled spin and charge density waves and the wave vector imply that this peak is not associated with a charge density wave.  From an experimental viewpoint, the peak is also suspicious given the large tails from the (200) peak likely originating from strong bragg scattering feeding through the collimators.~\\cite{Shirane:book}  A similar structure can be seen near (004) which is also a strong nuclear Bragg peak.  The inconsistency between different Bragg positions and also the correlation with strong nuclear Bragg lead us to conclude that the peak is likely spurious.  These small weak peaks are likely due to secondary scattering from the aluminium window located on the multi wire detector system.\n\n\\begin{figure}[t]\n\\includegraphics[width=9.5cm]{overview_nuc.eps}\n\\caption{\\label{wand_nuc} Momentum slices from WAND of the ``superlattice\" nuclear peak observed at (2.39 $\\pm$ 0.05, 0, 0$\\pm$ 0.05).  $(a)$ and $(c)$ show the peak to be present at 4 and 80 K, in the spin density and paramagnetic phase. $(b)$ and $(d)$ show cuts through the peak establishing its position in momentum. The lack of temperature dependence and inconsistency in the position near different nuclear Bragg peaks leads to believe this peak is spurious.}\n\\end{figure}\n \n",
        "label": "prop_vector",
        "Descriptive_question1": "What is the value of the incommensurate wavevector along the H direction in figure_9?",
        "Descriptive_question2": "At what temperature was the elastic energy slice taken on the MACS spectrometer in figure_9?",
        "Reasoning_question1": "What does the presence of a smaller peak at lower H in figure_9 suggest about the magnetic phases in the sample?",
        "Reasoning_question2": "How does the data from the WAND diffractometer in figure_9 clarify the nature of the propagation vector compared to previous studies?",
        "Descriptive_answer1": "0.46",
        "Descriptive_answer2": "2 K",
        "Reasoning_answer1": "The smaller peak at lower H indicates the presence of a minor magnetic helical phase with a larger interstitial iron concentration coexisting with the main spin density wave phase. This suggests phase separation or inhomogeneity in the sample's magnetic order, where one phase is the collinear spin-density wave and the other is helical magnetism due to iron content variations.",
        "Reasoning_answer2": "The WAND data, with broad momentum coverage, shows that the propagation vector is incommensurate with respect to the nuclear positions, at q_0 = 0.46 ± 0.01, rather than being just slightly offset from the commensurate position H=0.5 as some previous works suggested (0.5-δ). This clarifies that the propagation vector is truly incommensurate, ruling out interpretations that treat it as a small deviation from antiferromagnetic order, thanks to the wide reciprocal space mapping possible with WAND."
    },
    {
        "paper_id": "1704.01111.json",
        "image_id": "figure_10",
        "image_path": [
            "/home/yz979/scratch/chengye/scimolmo/test-images-tables/dataset_imagestables/1704.01111/images/overview_mag.eps"
        ],
        "caption": "\\label{wand_mag} Elastic momentum slices obtained from the WAND diffractometer.  $(a)$ and $(b)$ show scans obtained at 4 K and 80 K in the spin density wave and paramagnetic phase respectively. $(c)$ shows a cut along the L direction with H=0.46 $\\pm$ 0.01 r.l.u. showing the half integer commensurate nature of this scattering and also that it follows the expected decay of intensity based on the Fe$^{2+}$ form factor.  Backgrounds with an empty can have been obtained at both temperatures and subtracted.",
        "classify": "Chart",
        "section_info": "5 $x$=0.124(5)- Spin density wave and search for charge wave\n\\section{$x$=0.124(5)- Spin density wave and search for charge wave}\n\n\\begin{figure}[t]\n\\includegraphics[width=8.8cm]{prop_vector.eps}\n\\caption{\\label{prop_vector} $(a)$  Elastic energy slice taken on the MACS spectrometer.  $(b)$ displays a cut with L=0.5 illustrating the incommensurate wavevector.  The smaller peak at lower H results from the inclusion of a magnetic helical phase with a larger interstitial iron concentration.~\\cite{Rodriguez11:84}  $(c-d)$ show cuts taken on the WAND diffractometer illustrating the propagation vector.  The filled circles are symmetrized data and displayed to illustrate the relative positions of the magnetic scattering.}\n\\end{figure}\n\nWe now discuss the magnetic properties at the border between collinear and helical order in the Fe$_{1+x}$Te phase diagram by presenting neutron diffraction data on a single crystal of Fe$_{1.124(5)}$Te.\n\nMeasurements of the elastic neutron cross section at the border between collinear antiferromagnetism and the helical phase were done using a single crystal of Fe$_{1.124(5)}$Te.   Figure \\ref{prop_vector} $(a)$ illustrates a constant energy slice at the elastic position taken on the MACS cold triple axis spectrometer at 2 K.  Panel $(b)$ displays a cut along the H direction illustrating the incommensurate wavevector at $q_{0}$=0.46 $\\pm$ 0.01 along the H direction based on fits to a Lorentzian squared lineshape.  Within experimental error, the peak is commensurate along L being positioned at L=0.5 and no observable evidence of second harmonics at 2$q_{0}$ are observable in the data (within 2\\% of the peak height at $q_{0}$=0.46 $\\pm$ 0.01).  As discussed previously in Ref. \\onlinecite{Rodriguez11:84}, the static magnetism corresponding to this peak is short-range along the $a$ axis evidenced by a broader than resolution lineshape along the H direction.  The lineshape is resolution limited along $c$ corresponding to long range order along L.  An analysis based on polarized neutrons found that the magnetic structure is polarized along the $b$ axis in contrast to the helical order for Fe$_{1+x}$Te samples on the iron rich side of the phase diagram.  The magnetic structure at $x$=0.124(5) therefore corresponds to a collinear spin-density wave phase.  A second peak is observed at a lower $q$ position and as discussed in Refs. \\onlinecite{Rodriguez11:84,Rodriguez13:88} based on a polarized neutron analysis, this corresponds to a small inclusion of a helical phase with a larger interstitial iron concentration.  \n\nAs discussed above in the introduction, there have been several reports of magnetism at this incommensurate wave vector, even in superconducting samples doped with Se.  However, it is not clear from the limited momentum range if the peak is incommensurate with respect to the nuclear positions or to the antiferromagnetic H=0.5 point as might be expected based on analogies with cuprates.  Figure \\ref{prop_vector}, panels $(c)$ and $(d)$, show cuts at L=0.5 and L=1.5 taken on WAND where the combined thermal neutron wavelengths and broad detector coverage allow us to study the magnetism over a broad range of momentum transfer.  The cuts prove the result reported previously that the propagation vector is $q_{0}$=0.46 $\\pm$ 0.01 and is incommensurate with respect to the nuclear positions.  This contrasts with some studies that have stated that the propagation vector is taken as (0.5-$\\delta$,0,0.5)~\\cite{Wen12:86} and is only clear in the current data set given the broad momentum coverage afforded by the WAND diffractometer.   \n\n\\begin{figure}[t]\n\\includegraphics[width=9.5cm]{overview_mag.eps}\n\\caption{\\label{wand_mag} Elastic momentum slices obtained from the WAND diffractometer.  $(a)$ and $(b)$ show scans obtained at 4 K and 80 K in the spin density wave and paramagnetic phase respectively. $(c)$ shows a cut along the L direction with H=0.46 $\\pm$ 0.01 r.l.u. showing the half integer commensurate nature of this scattering and also that it follows the expected decay of intensity based on the Fe$^{2+}$ form factor.  Backgrounds with an empty can have been obtained at both temperatures and subtracted.}\n\\end{figure}\n\nFigure \\ref{wand_mag} illustrates an extensive reciprocal space map at 4 K (in the magnetically ordered state) and also at high temperatures of 80 K where Fe$_{1.124(5)}$Te is paramagnetic.   A series of magnetic superlattice peaks are clearly observed at H$\\sim$ 0.46, 1.54 and 2.45 r.l.u. at T=4 K, but absent at high temperatures of 80 K, confirming the magnetic origin.  This is highlighted by the yellow ellipse at H $\\sim$ 0.46 at both temperatures.  Panel $(c)$ plots an L scan at H=0.46 $\\pm$ 0.01 r.l.u. showing that the magnetic peaks appear at the commensurate half integer positions along L and also that the intensity decays with the expected Fe$^{2+}$ form factor.  This is consistent with dipolar selection rules for the intensity based on localized magnetic moments pointing along the $b$ axis.\n\nOur WAND results are not consistent with suggestions of antiphase boundaries separating locally ordered collinear states.~\\cite{Mazin09:5}  While such a structure can produce scattering at incommensurate positions, as observed in stripe phases of nickelates~\\cite{Tranquada96:54} and also cuprates~\\cite{Waki00:61,Waki99:60,Stock04:69,Stock10:82}, it fails to model both the incommensurate wavevector and the lack of higher harmonics that would be associated with a sharp uniaxial boundary.  It has recently been proposed that the structure maybe understood in terms of solitons~\\cite{Materne15:115}, however the magnetic structure proposed would produce a $c$ and $a$ axis component to the scattering in our previous polarized neutron diffraction studies of this compound.  This contradicts the data which is consistent with a component only along the $c$-axis.  We therefore conclude that the short-range static antiferromagnetism observed near interstitial iron concentrations of $x\\sim$ 0.12 is more consistent with a spin density wave where the magnitude of the spin varies along the direction of propagation.  \n\nUsing the wide momentum coverage on WAND, we have also searched for any charge density wave that may accompany this spin density wave.  The single crystal momentum maps in Fig. \\ref{wand_mag} $(a)$ and $(b)$ display no observable superlattice peaks that may be associated with a charge density wave.  A small peak is observed near (2.39 $\\pm$ 0.05, 0, 0$\\pm$ 0.05), however this peak is present at both 4 K and 80 K and is not observable near any other primary nuclear Bragg peak measured in our reciprocal space mapping.  The peak is illustrated in Fig. \\ref{wand_nuc}.  While a coupling between strain or charge and magnetism is expected,~\\cite{Turner09:80} as displayed in Cr metal~\\cite{Fawcett88:60}, the charge density wave peak intensity is in proportion to the spin density wave and should be at harmonic of the primary wavevector which is not the case here.~\\cite{Pynn76:13,Kotani76:41,Kotani78:44}   \n\nThe lack of consistency with what has been discussed in relation to coupled spin and charge density waves and the wave vector imply that this peak is not associated with a charge density wave.  From an experimental viewpoint, the peak is also suspicious given the large tails from the (200) peak likely originating from strong bragg scattering feeding through the collimators.~\\cite{Shirane:book}  A similar structure can be seen near (004) which is also a strong nuclear Bragg peak.  The inconsistency between different Bragg positions and also the correlation with strong nuclear Bragg lead us to conclude that the peak is likely spurious.  These small weak peaks are likely due to secondary scattering from the aluminium window located on the multi wire detector system.\n\n\\begin{figure}[t]\n\\includegraphics[width=9.5cm]{overview_nuc.eps}\n\\caption{\\label{wand_nuc} Momentum slices from WAND of the ``superlattice\" nuclear peak observed at (2.39 $\\pm$ 0.05, 0, 0$\\pm$ 0.05).  $(a)$ and $(c)$ show the peak to be present at 4 and 80 K, in the spin density and paramagnetic phase. $(b)$ and $(d)$ show cuts through the peak establishing its position in momentum. The lack of temperature dependence and inconsistency in the position near different nuclear Bragg peaks leads to believe this peak is spurious.}\n\\end{figure}\n \n",
        "label": "wand_mag",
        "Descriptive_question1": "What temperature is shown in panel (a) of figure_10?",
        "Descriptive_question2": "What is the H value in the L direction cut shown in panel (c) of figure_10?",
        "Reasoning_question1": "What does the absence of magnetic superlattice peaks at 80 K in figure_10 suggest about the magnetic state of Fe_{1.124(5)}Te at this temperature compared to 4 K?",
        "Reasoning_question2": "How does the intensity decay along the L direction in panel (c) of figure_10 support the conclusion about the magnetic moment direction in Fe_{1.124(5)}Te?",
        "Descriptive_answer1": "4 K",
        "Descriptive_answer2": "0.46 \\pm 0.01 r.l.u.",
        "Reasoning_answer1": "The presence of magnetic superlattice peaks at 4 K and their absence at 80 K indicate that Fe_{1.124(5)}Te is magnetically ordered at low temperature (4 K), but transitions to a paramagnetic state at higher temperature (80 K), where magnetic order disappears. This change reflects the loss of static magnetic order as temperature increases above the ordering temperature.",
        "Reasoning_answer2": "The magnetic intensity decays along L according to the expected Fe^{2+} form factor, which implies that the magnetic scattering follows dipolar selection rules consistent with moments oriented along the b axis. If spins pointed in other directions, the L dependence would deviate from this form factor. Therefore, the observed decay of intensity along L supports the conclusion that magnetic moments are aligned along the b axis."
    },
    {
        "paper_id": "1704.01111.json",
        "image_id": "figure_11",
        "image_path": [
            "/home/yz979/scratch/chengye/scimolmo/test-images-tables/dataset_imagestables/1704.01111/images/overview_nuc.eps"
        ],
        "caption": "\\label{wand_nuc} Momentum slices from WAND of the ``superlattice\" nuclear peak observed at (2.39 $\\pm$ 0.05, 0, 0$\\pm$ 0.05).  $(a)$ and $(c)$ show the peak to be present at 4 and 80 K, in the spin density and paramagnetic phase. $(b)$ and $(d)$ show cuts through the peak establishing its position in momentum. The lack of temperature dependence and inconsistency in the position near different nuclear Bragg peaks leads to believe this peak is spurious.",
        "classify": "Chart",
        "section_info": "5 $x$=0.124(5)- Spin density wave and search for charge wave\n\\section{$x$=0.124(5)- Spin density wave and search for charge wave}\n\n\\begin{figure}[t]\n\\includegraphics[width=8.8cm]{prop_vector.eps}\n\\caption{\\label{prop_vector} $(a)$  Elastic energy slice taken on the MACS spectrometer.  $(b)$ displays a cut with L=0.5 illustrating the incommensurate wavevector.  The smaller peak at lower H results from the inclusion of a magnetic helical phase with a larger interstitial iron concentration.~\\cite{Rodriguez11:84}  $(c-d)$ show cuts taken on the WAND diffractometer illustrating the propagation vector.  The filled circles are symmetrized data and displayed to illustrate the relative positions of the magnetic scattering.}\n\\end{figure}\n\nWe now discuss the magnetic properties at the border between collinear and helical order in the Fe$_{1+x}$Te phase diagram by presenting neutron diffraction data on a single crystal of Fe$_{1.124(5)}$Te.\n\nMeasurements of the elastic neutron cross section at the border between collinear antiferromagnetism and the helical phase were done using a single crystal of Fe$_{1.124(5)}$Te.   Figure \\ref{prop_vector} $(a)$ illustrates a constant energy slice at the elastic position taken on the MACS cold triple axis spectrometer at 2 K.  Panel $(b)$ displays a cut along the H direction illustrating the incommensurate wavevector at $q_{0}$=0.46 $\\pm$ 0.01 along the H direction based on fits to a Lorentzian squared lineshape.  Within experimental error, the peak is commensurate along L being positioned at L=0.5 and no observable evidence of second harmonics at 2$q_{0}$ are observable in the data (within 2\\% of the peak height at $q_{0}$=0.46 $\\pm$ 0.01).  As discussed previously in Ref. \\onlinecite{Rodriguez11:84}, the static magnetism corresponding to this peak is short-range along the $a$ axis evidenced by a broader than resolution lineshape along the H direction.  The lineshape is resolution limited along $c$ corresponding to long range order along L.  An analysis based on polarized neutrons found that the magnetic structure is polarized along the $b$ axis in contrast to the helical order for Fe$_{1+x}$Te samples on the iron rich side of the phase diagram.  The magnetic structure at $x$=0.124(5) therefore corresponds to a collinear spin-density wave phase.  A second peak is observed at a lower $q$ position and as discussed in Refs. \\onlinecite{Rodriguez11:84,Rodriguez13:88} based on a polarized neutron analysis, this corresponds to a small inclusion of a helical phase with a larger interstitial iron concentration.  \n\nAs discussed above in the introduction, there have been several reports of magnetism at this incommensurate wave vector, even in superconducting samples doped with Se.  However, it is not clear from the limited momentum range if the peak is incommensurate with respect to the nuclear positions or to the antiferromagnetic H=0.5 point as might be expected based on analogies with cuprates.  Figure \\ref{prop_vector}, panels $(c)$ and $(d)$, show cuts at L=0.5 and L=1.5 taken on WAND where the combined thermal neutron wavelengths and broad detector coverage allow us to study the magnetism over a broad range of momentum transfer.  The cuts prove the result reported previously that the propagation vector is $q_{0}$=0.46 $\\pm$ 0.01 and is incommensurate with respect to the nuclear positions.  This contrasts with some studies that have stated that the propagation vector is taken as (0.5-$\\delta$,0,0.5)~\\cite{Wen12:86} and is only clear in the current data set given the broad momentum coverage afforded by the WAND diffractometer.   \n\n\\begin{figure}[t]\n\\includegraphics[width=9.5cm]{overview_mag.eps}\n\\caption{\\label{wand_mag} Elastic momentum slices obtained from the WAND diffractometer.  $(a)$ and $(b)$ show scans obtained at 4 K and 80 K in the spin density wave and paramagnetic phase respectively. $(c)$ shows a cut along the L direction with H=0.46 $\\pm$ 0.01 r.l.u. showing the half integer commensurate nature of this scattering and also that it follows the expected decay of intensity based on the Fe$^{2+}$ form factor.  Backgrounds with an empty can have been obtained at both temperatures and subtracted.}\n\\end{figure}\n\nFigure \\ref{wand_mag} illustrates an extensive reciprocal space map at 4 K (in the magnetically ordered state) and also at high temperatures of 80 K where Fe$_{1.124(5)}$Te is paramagnetic.   A series of magnetic superlattice peaks are clearly observed at H$\\sim$ 0.46, 1.54 and 2.45 r.l.u. at T=4 K, but absent at high temperatures of 80 K, confirming the magnetic origin.  This is highlighted by the yellow ellipse at H $\\sim$ 0.46 at both temperatures.  Panel $(c)$ plots an L scan at H=0.46 $\\pm$ 0.01 r.l.u. showing that the magnetic peaks appear at the commensurate half integer positions along L and also that the intensity decays with the expected Fe$^{2+}$ form factor.  This is consistent with dipolar selection rules for the intensity based on localized magnetic moments pointing along the $b$ axis.\n\nOur WAND results are not consistent with suggestions of antiphase boundaries separating locally ordered collinear states.~\\cite{Mazin09:5}  While such a structure can produce scattering at incommensurate positions, as observed in stripe phases of nickelates~\\cite{Tranquada96:54} and also cuprates~\\cite{Waki00:61,Waki99:60,Stock04:69,Stock10:82}, it fails to model both the incommensurate wavevector and the lack of higher harmonics that would be associated with a sharp uniaxial boundary.  It has recently been proposed that the structure maybe understood in terms of solitons~\\cite{Materne15:115}, however the magnetic structure proposed would produce a $c$ and $a$ axis component to the scattering in our previous polarized neutron diffraction studies of this compound.  This contradicts the data which is consistent with a component only along the $c$-axis.  We therefore conclude that the short-range static antiferromagnetism observed near interstitial iron concentrations of $x\\sim$ 0.12 is more consistent with a spin density wave where the magnitude of the spin varies along the direction of propagation.  \n\nUsing the wide momentum coverage on WAND, we have also searched for any charge density wave that may accompany this spin density wave.  The single crystal momentum maps in Fig. \\ref{wand_mag} $(a)$ and $(b)$ display no observable superlattice peaks that may be associated with a charge density wave.  A small peak is observed near (2.39 $\\pm$ 0.05, 0, 0$\\pm$ 0.05), however this peak is present at both 4 K and 80 K and is not observable near any other primary nuclear Bragg peak measured in our reciprocal space mapping.  The peak is illustrated in Fig. \\ref{wand_nuc}.  While a coupling between strain or charge and magnetism is expected,~\\cite{Turner09:80} as displayed in Cr metal~\\cite{Fawcett88:60}, the charge density wave peak intensity is in proportion to the spin density wave and should be at harmonic of the primary wavevector which is not the case here.~\\cite{Pynn76:13,Kotani76:41,Kotani78:44}   \n\nThe lack of consistency with what has been discussed in relation to coupled spin and charge density waves and the wave vector imply that this peak is not associated with a charge density wave.  From an experimental viewpoint, the peak is also suspicious given the large tails from the (200) peak likely originating from strong bragg scattering feeding through the collimators.~\\cite{Shirane:book}  A similar structure can be seen near (004) which is also a strong nuclear Bragg peak.  The inconsistency between different Bragg positions and also the correlation with strong nuclear Bragg lead us to conclude that the peak is likely spurious.  These small weak peaks are likely due to secondary scattering from the aluminium window located on the multi wire detector system.\n\n\\begin{figure}[t]\n\\includegraphics[width=9.5cm]{overview_nuc.eps}\n\\caption{\\label{wand_nuc} Momentum slices from WAND of the ``superlattice\" nuclear peak observed at (2.39 $\\pm$ 0.05, 0, 0$\\pm$ 0.05).  $(a)$ and $(c)$ show the peak to be present at 4 and 80 K, in the spin density and paramagnetic phase. $(b)$ and $(d)$ show cuts through the peak establishing its position in momentum. The lack of temperature dependence and inconsistency in the position near different nuclear Bragg peaks leads to believe this peak is spurious.}\n\\end{figure}\n \n",
        "label": "wand_nuc",
        "Descriptive_question1": "At what temperature is the peak shown in panel (a) of figure_11 observed?",
        "Descriptive_question2": "What is the H coordinate of the nuclear peak in figure_11?",
        "Reasoning_question1": "Why might the nuclear peak in figure_11 be considered spurious based on the temperature data provided?",
        "Reasoning_question2": "How does the position of the peak in figure_11 relative to different nuclear Bragg peaks influence the interpretation of its origin?",
        "Descriptive_answer1": "4 K",
        "Descriptive_answer2": "2.39",
        "Reasoning_answer1": "The peak is observed at both 4 K (spin density wave phase) and 80 K (paramagnetic phase) without any temperature dependence, which is unusual for a true charge density wave peak that would typically show temperature dependence linked to magnetic ordering. This temperature invariance suggests the peak is likely not related to intrinsic charge ordering and is therefore considered spurious.",
        "Reasoning_answer2": "The inconsistency in the peak position near different primary nuclear Bragg peaks implies the observed peak does not correspond to a genuine charge density wave or magnetic order, which would be expected to appear consistently relative to these Bragg positions. Instead, this inconsistency supports the conclusion that the peak arises from artifacts such as secondary scattering or instrumental effects, leading to its interpretation as spurious."
    },
    {
        "paper_id": "1812.04217.json",
        "image_id": "figure_4",
        "image_path": [
            "/home/yz979/scratch/chengye/scimolmo/test-images-tables/dataset_imagestables/1812.04217/images/compare_u0_draft.eps"
        ],
        "caption": "\nLeft panel: Comparison between the exact solution and the result obtained by analytical continuation\nfor the real and imaginary parts of the OTOC $\\langle c_\\sigma^\\dagger(t)c_\\sigma(0), c_\\sigma^\\dagger(t)c_\\sigma(0)\\rangle$ \nin the noninteracting case ($U=0, \\beta\\hbar=50$) of the single-orbital Hubbard model.\nRight panel: Comparison of the long-time behavior for the modulus $|\\langle c_\\sigma^\\dagger(t)c_\\sigma(0), c_\\sigma^\\dagger(t)c_\\sigma(0)\\rangle|$. \nThe exact result shows a power-law decay ($\\sim 1/t^3$). \n",
        "classify": "Chart",
        "section_info": "4 Numerical results for the Hubbard model\n\\section{Numerical results for the Hubbard model}\n\\label{numerical results}\n\n\\subsection{Observables and numerical procedure}\n\nWe consider the single-orbital, two-orbital, and three-orbital Hubbard models on an infinitely-connected Bethe lattice, which can be solved exactly within DMFT \\cite{GeorgesKotliarKrauthRozenberg1996}. In this case, the noninteracting density of states is semicircular with bandwidth $4v_\\ast$, for which there exists a simplified DMFT self-consistency condition between the hybridization function $\\Delta_{\\alpha\\sigma}$ of the DMFT impurity problem and the local (impurity) Green's function $G_{\\alpha\\sigma}$: \n\n\\begin{equation}\n\\Delta_{\\alpha\\sigma}(\\tau)=v_\\ast^2 G_{\\alpha\\sigma}(\\tau), \n\\label{semicirc}\n\\end{equation}\n\nwith $\\alpha$ the orbital and $\\sigma$ the spin index. We will use $v_\\ast$ as the unit of energy and measure time in units of $\\hbar/v_\\ast$. \n\nThree different types of imaginary-time four-point functions $C_{(AB)^2}^M(\\tau)$\nof the form of Eq.~(\\ref{imaginary-time OTOC})\nwith $(\\hat A, \\hat B)=(c_\\sigma^\\dagger, c_\\sigma), (\\hat n_\\sigma, \\hat n_\\sigma)$, and $(\\hat n, \\hat n)$\nare calculated in the interval $0\\le \\tau\\le \\frac{\\beta\\hbar}{2}$: \n\\begin{eqnarray}\nC_{(c^\\dagger_\\sigma c_\\sigma)^2}^M(\\tau) &=& -\\langle c^\\dagger_\\sigma(\\tau+\\tfrac{\\beta\\hbar}{2}) c_\\sigma(\\tfrac{\\beta\\hbar}{2}) c^\\dagger_\\sigma(\\tau) c_\\sigma(0) \\rangle, \\label{fermion_otoc}\\\\\nC_{(n_\\sigma n_\\sigma)^2}^M(\\tau) &=& -\\langle \\hat n_\\sigma(\\tau+\\tfrac{\\beta\\hbar}{2})\\hat n_\\sigma(\\tfrac{\\beta\\hbar}{2})\\hat n_\\sigma(\\tau)\\hat n_\\sigma(0) \\rangle, \\label{otoc_nsns}\\\\\nC_{(nn)^2}^M(\\tau) &=& -\\langle \\hat n(\\tau+\\tfrac{\\beta\\hbar}{2})\\hat n(\\tfrac{\\beta\\hbar}{2})\\hat n(\\tau)\\hat n(0) \\rangle,\\label{otoc_nn}\n\\end{eqnarray}\n\nwhere $c_\\sigma^\\dagger$ ($c_\\sigma$) are the fermionic creation (annihilation) operators for spin $\\sigma$ (and orbital $\\alpha=1$ in the multi-orbital case), $\\hat n_\\sigma=c^\\dagger_\\sigma c_\\sigma$ is the corresponding spin-dependent density operator, and $\\hat n=\\hat n_\\uparrow + \\hat n_\\downarrow$ the total density operator. Note that in all the three cases above we have $\\hat B=\\hat A^\\dagger$. \nWe measure these local correlation functions in the impurity model using a hybridization expansion continuous-time Monte Carlo algorithm (CT-HYB) \\cite{Werner2006}. In this algorithm, the \ntwo-particle\nGreen's functions of the type (\\ref{fermion_otoc}) can be measured by removing two hybridization lines, i.e., from the elements of the inverse hybridization matrix, while the density-density correlation functions can be easily measured either by insertion of density operators (matrix formalism) \\cite{Werner2006Kondo} or by reading off the occupation of the orbitals at the four time points in the segment implementation \\cite{Werner2006}. We use the latter algorithm since we consider only density-density interactions. \nThe values at the end-points $\\tau=0$ and $\\tau=\\frac{\\beta\\hbar}{2}$ reduce to standard density-density correlation functions, which in the case of Eq.~(\\ref{fermion_otoc}) are measured separately. \n\nWe perform the analytic continuation to the real-frequency axis using the Maximum Entropy method \\cite{Jarrell1996,Lewin_maxent} with a bosonic kernel. This yields the imaginary part of the retarded correlator \n$-\\frac{1}{\\pi}\\text{Im}\\, C^R_{(AA^\\dagger)^2}(\\omega)=\\mathscr A_{(AA^\\dagger)^2}(\\omega)$.  \nWhen $\\hat B=\\hat A^\\dagger=\\hat A$ (which is the case for $\\hat A=\\hat n_\\sigma, \\hat n$), we have $C_{(AA)^2}^R(\\omega)^\\ast=C_{(AA)^2}^R(-\\omega)$. At half filling, the particle-hole symmetry furthermore ensures $C_{c_\\sigma^\\dagger c_\\sigma}^R(\\omega)^\\ast=C_{c_\\sigma^\\dagger c_\\sigma}^R(-\\omega)$.\nThus, in all the cases considered in this study, the retarded OTOC has the symmetry property $C_{(AA^\\dagger)^2}^R(\\omega)^\\ast=C_{(AA^\\dagger)^2}^R(-\\omega)$. Using this, as well as the out-of-time-order fluctuation-dissipation\ntheorem discussed in the previous section, we can obtain the real-time OTOC\n$\\langle \\hat A(t)\\hat A^\\dagger(0), \\hat A(t)\\hat A^\\dagger(0)\\rangle$ from the following inverse Fourier transformation, \n\\begin{align}{\\rm Re}\\langle \\hat A(t)\\hat A^\\dagger(0), \\hat A(t)\\hat A^\\dagger(0)\\rangle\n&=\n{\\rm Re}\n\\int_0^\\infty d\\omega\\, e^{-i\\omega t}\\coth\\left(\\frac{\\beta\\hbar\\omega}{4}\\right)\n\\left(-\\frac{1}{\\pi}{\\rm Im}\\, C_{(AA^\\dagger)^2}^R(\\omega)\\right),\n\\\\\n{\\rm Im}\\langle \\hat A(t)\\hat A^\\dagger(0), \\hat A(t)\\hat A^\\dagger(0)\\rangle\n&=\n{\\rm Im}\\int_0^{\\infty} d\\omega\\, e^{-i\\omega t} \\left(-\\frac{1}{\\pi}{\\rm Im}\\, C_{(AA^\\dagger)^2}^R(\\omega)\\right),\n\\end{align}\nfor $\\hat A=c_\\sigma^\\dagger, \\hat n_\\sigma$, and $\\hat n$.\n\n\\begin{figure}[b]\n\\begin{center}\n\\includegraphics[angle=-90, width=0.48\\columnwidth]{compare_u0_draft.eps}\\hfill\n\\includegraphics[angle=-90, width=0.48\\columnwidth]{compare_u0_longtime_modulus.eps}\n\\caption{\nLeft panel: Comparison between the exact solution and the result obtained by analytical continuation\nfor the real and imaginary parts of the OTOC $\\langle c_\\sigma^\\dagger(t)c_\\sigma(0), c_\\sigma^\\dagger(t)c_\\sigma(0)\\rangle$ \nin the noninteracting case ($U=0, \\beta\\hbar=50$) of the single-orbital Hubbard model.\nRight panel: Comparison of the long-time behavior for the modulus $|\\langle c_\\sigma^\\dagger(t)c_\\sigma(0), c_\\sigma^\\dagger(t)c_\\sigma(0)\\rangle|$. \nThe exact result shows a power-law decay ($\\sim 1/t^3$). \n}\n\\label{fig:test_u0}\n\\end{center}\n\\end{figure}\n\nAs a check of our procedure, we first compare the results for the noninteracting model,\nfor which an exact solution of the OTOC $\\langle c_\\sigma^\\dagger(t)c_\\sigma(0), c_\\sigma^\\dagger(t)c_\\sigma(0)\\rangle$ \nis available \\cite{TsujiWernerUeda2017}.\nIt is a nontrivial task to measure the noninteracting two-particle Green's function in CT-HYB, \nand to analytically continue \n$C_{(c^\\dagger_\\sigma c_\\sigma)^2}^M(\\tau)$ \nby the Maximum Entropy method.  \nThe results for the inverse temperature $\\beta\\hbar=50$ are plotted in Fig.~\\ref{fig:test_u0}. \nOne can see that the OTOC oscillates for about one cycle, and quickly decays to zero.\nThe analytically continued data show a good agreement with the exact solution. \nIn particular, for short times (up to about two inverse hoppings) the dynamics is accurately reproduced, while deviations appear at longer times. \nIt is known that for the noninteracting fermion system the OTOC decays as a power law at long time ($\\sim 1/t^3$) \\cite{TsujiWernerUeda2017}.\nThe analytical continuation method cannot reproduce the details of the oscillations at intermediate or long times, but it roughly captures the long-time decay, which is controlled by low-frequency spectral features. \nUsually, analytic continuation is unreliable for high frequency components, because high-frequency information is suppressed \nby the kernel $K(\\tau,\\omega)$ in the transformation from the spectral function $A_O(\\omega)$ to the (bosonic) Matsubara correlation function $O(\\tau)$ [$O(\\tau)=\\int_{-\\infty}^{\\infty} d\\omega\\, K(\\omega,\\tau) A_O(\\omega)$ with $K(\\tau,\\omega)=e^{-\\tau\\omega}/(1-e^{-\\beta\\omega})$]. On the other hand, the low frequency components can be rather accurately determined. As a result, in Fig. 4 we more or less recover the low-frequency features of the time-dependent correlation function, while the details of rapid oscillations are not captured. The accurate result at short times can be understood from the fact that this region is close to the imaginary time axis in the complex time plane, where the original QMC data are available.\nAt higher temperatures, the Maximum Entropy method becomes less reliable, so that the resulting OTOCs are expected to be less accurate.  \n\n\n\\subsection{Single-orbital Hubbard model}\n\\label{sec:hubbard}\n\nIn this section, we calculate the OTOCs (\\ref{fermion_otoc})-(\\ref{otoc_nn}) for the interacting single-orbital Hubbard model with the Hamiltonian \n\\begin{align}\nH\n&=\n-v\\sum_{\\langle i,j\\rangle\\sigma} (c^\\dagger_{i\\sigma}c_{j\\sigma}+ \\text{h.c.})\n-\\mu\\sum_i \\hat n_i+U\\sum_i \\hat n_{i\\uparrow}\\hat n_{i\\downarrow}, \n\\end{align}\nwhere $v$ is the hopping amplitude, $\\langle i,j\\rangle$ represents the nearest-neighbor site pairs, \n$\\mu$ is the chemical potential, and $U$ is the on-site interaction.\nIn this section, we focus on the half-filled case (i.e., $\\mu=U/2$). The DMFT solution for the simplified self-consistency (\\ref{semicirc}) is the exact solution for an infinitely connected Bethe lattice \\cite{GeorgesKotliarKrauthRozenberg1996}.\nLet us briefly recall the paramagnetic phase diagram for the single-orbital Hubbard model on this lattice \\cite{Bluemer_thesis}. At half-filling, there is a metal-insulator crossover at high temperature and a first-order Mott transition below a temperature corresponding to $\\beta\\hbar\\approx 17$ with a coexistence region between $U_{c1}$ and $U_{c2}$. The finite-temperature critical endpoint is at $U\\approx 4.7$, while the zero-temperature Mott transition occurs at $U=U_{c2}\\approx 5.6$. \nAt low temperature, the self-energy shows the Fermi liquid behavior [${\\rm Im}\\,\\Sigma(\\omega)\\sim \\omega^2$] \nin the weakly correlated metallic phase, \nwhile it shows an insulating behavior [${\\rm Im}\\,\\Sigma(\\omega)\\sim \\delta(\\omega)$] in the Mott phase\n\\cite{GeorgesKotliarKrauthRozenberg1996}. \nIn the correlated metallic phase, as the temperature is increased, deviations from the Fermi liquid behavior become apparent, \nbut an ${\\rm Im}\\,\\Sigma(\\omega)\\sim \\sqrt{\\omega}$ scaling as in the SYK model is not observed. \n\nIn the following, we will start the DMFT iterations from the noninteracting solution, which means that the finite-temperature Mott transition occurs at $U_{c2}$.\nFrom now on, we set $\\hbar=1$. \n\n\\begin{figure}[t]\n\\begin{center}\n\\includegraphics[angle=-90, width=0.49\\columnwidth]{plot_otoc_b50_Re.eps}\n\\includegraphics[angle=-90, width=0.49\\columnwidth]{plot_otoc_b50_Im.eps}\n\\includegraphics[angle=-90, width=0.49\\columnwidth]{plot_coth_b50_appendix.eps}\n\\includegraphics[angle=-90, width=0.49\\columnwidth]{plot_otoc_b50_modulus.eps}\n\\caption{\nTop panels: Real and imaginary parts of the OTOC $\\langle c^\\dagger_\\sigma(t)c_\\sigma(0), c^\\dagger_\\sigma(t)c_\\sigma(0)\\rangle$ for the single-orbital Hubbard model with $\\beta=50$ at half filling. The thin lines correspond to metallic solutions, while the thick line ($U=6$) corresponds to a Mott insulating solution. \nBottom left panel: Spectral function $\\mathscr A_{(c_\\sigma^\\dagger c_\\sigma)^2}(\\omega)=-\\frac{1}{\\pi}{\\rm Im}\\,C_{(c^\\dagger_\\sigma c_\\sigma)^2}^R(\\omega)$ multiplied by $\\coth\\big(\\frac{\\beta\\omega}{4}\\big)$. \nBottom right panel: Modulus of the OTOC $|\\langle c^\\dagger_\\sigma(t)c_\\sigma(0), c^\\dagger_\\sigma(t)c_\\sigma(0)\\rangle|$ (normalized at $t=0$) on a log-log scale. The dashed line is the result for $U=0$.\n}\n\\label{fig:results_fermion}\n\\end{center}\n\\end{figure}\n\n\n\nIn Fig.~\\ref{fig:results_fermion}, we show the results of $\\langle c^\\dagger_\\sigma(t)c_\\sigma(0), c^\\dagger_\\sigma(t)c_\\sigma(0)\\rangle$ for the interacting single-orbital Hubbard model at $\\beta=50$. \nThe top panels present the real and imaginary parts of the OTOC. One can see that the oscillations become more pronounced as one increases the interaction. In the bottom left panel of Fig.~\\ref{fig:results_fermion},\nwe show the corresponding spectral function $\\mathscr A_{(c_\\sigma^\\dagger c_\\sigma)^2}(\\omega)=-\\frac{1}{\\pi}{\\rm Im}\\,C_{(c^\\dagger_\\sigma c_\\sigma)^2}^R(\\omega)$ multiplied by $\\coth\\big(\\frac{\\beta\\omega}{4}\\big)$.\nThe Mott transition, which occurs between $U=5$ and $U=6$ at $\\beta=50$, manifests itself by the opening of a gap in the spectral function $\\mathscr A_{(c_\\sigma^\\dagger c_\\sigma)^2(\\omega)}$. This translates into more weakly damped oscillations in the real-time evolution in the Mott phase.  \nThe bottom right panel of Fig.~\\ref{fig:results_fermion} plots the modulus of the OTOC (normalized at time $t=0$, i.e., $|\\langle c^\\dagger_\\sigma(t)c_\\sigma(0), c^\\dagger_\\sigma(t)c_\\sigma(0)\\rangle|/|\\langle c^\\dagger_\\sigma(0)c_\\sigma(0), c^\\dagger_\\sigma(0)c_\\sigma(0)\\rangle|$) on a log-log scale. \nDue to the oscillations and the limited accuracy of the analytic continuation procedure, it is hard to clearly identify the nature of the long-time decay of the OTOC,\nbut the results indicate that the OTOC decays much faster (possibly exponentially) in the Mott phase than in the metallic phase. This is consistent with the results for the spectral functions of the OTOC. The comparison to the noninteracting case (dashed curve in the bottom right panel of Fig.~\\ref{fig:results_fermion}) suggests that the correlated metallic phase has a similar (possibly power-law) decay behavior of the OTOC as in the noninteracting case.\n\n\n\\begin{figure}[t]\n\\begin{center}\n\\includegraphics[angle=-90, width=0.49\\columnwidth]{plot_otoc_b50_nn_Re.eps}\n\\includegraphics[angle=-90, width=0.49\\columnwidth]{plot_otoc_b50_nn_Im.eps}\n\\includegraphics[angle=-90, width=0.49\\columnwidth]{plot_coth_b50_nn_appendix.eps}\n\\includegraphics[angle=-90, width=0.49\\columnwidth]{plot_otoc_b50_nn_modulus.eps}\n\\caption{\nTop panels: Real and imaginary parts of the OTOC $\\langle \\hat n(t)\\hat n(0), \\hat n(t)\\hat n(0)\\rangle - 1$ for the single-orbital Hubbard model with $\\beta=50$ at half filling. The thin lines correspond to metallic solutions, \nwhile the thick line ($U=6$) corresponds to a Mott insulating solution. \nBottom left panel: Spectral function $\\mathscr A_{(nn)^2}(\\omega)=-\\frac{1}{\\pi}{\\rm Im}\\,C_{(n n)^2}^R(\\omega)$ multiplied by $\\coth\\big(\\frac{\\beta\\omega}{4}\\big)$.\nBottom right panel: Modulus of the OTOC $|\\langle \\hat n(t)\\hat n(0), \\hat n(t)\\hat n(0)\\rangle - 1|$ (normalized at $t=0$) on a log-log scale. \n}\n\\label{fig:results_nn}\n\\end{center}\n\\end{figure}\n\n\nIn Fig.~\\ref{fig:results_nn}, we plot the OTOC $\\langle \\hat n(t)\\hat n(0), \\hat n(t)\\hat n(0)\\rangle$\nfor the single-orbital Hubbard model in a similar manner as in Fig.~\\ref{fig:results_fermion}.\nSince at half filling this correlation function approaches $\\langle \\hat n\\rangle^4=1$ at long times\nand at sufficiently low temperature,\nwe perform the analytical continuation procedure for $\\langle \\hat n(t)\\hat n(0), \\hat n(t)\\hat n(0)\\rangle-1$. \nThe top panels of Fig.~\\ref{fig:results_nn} show the real and imaginary parts of this shifted OTOC. \nContrary to the case of $\\langle c_\\sigma^\\dagger(t)c_\\sigma(0), c_\\sigma^\\dagger(t)c_\\sigma(0)\\rangle$, \nthe amplitude of the OTOC $\\langle \\hat n(t)\\hat n(0), \\hat n(t)\\hat n(0)\\rangle-1$ is suppressed as one increases the interaction, which reflects the reduced charge fluctuations in the Mott insulator.\nThe bottom left panel in Fig.~\\ref{fig:results_nn} shows the analytically continued spectral function $\\mathscr A_{(nn)^2}(\\omega)=-\\frac{1}{\\pi}{\\rm Im}\\, C_{(nn)^2}^R(\\omega)$ multiplied by $\\coth\\big(\\frac{\\beta\\omega}{4}\\big)$. \nAgain we notice the opening of a gap in the Mott phase, and a corresponding shift of the spectral weight to higher energies. This results in more rapid oscillations of the OTOC in the Mott state.\nThe bottom right panel in Fig.~\\ref{fig:results_nn} plots the modulus of the OTOC $|\\langle \\hat n(t)\\hat n(0), \\hat n(t)\\hat n(0)\\rangle-1|$ (normalized at $t=0$) on a log-log scale.\nThe long-time behavior of the OTOC is consistent with a power-law decay in the metallic phase, and an exponential decay in the Mott insulating phase.\n\n\n\\begin{figure}[t]\n\\begin{center}\n\\includegraphics[angle=-90, width=0.49\\columnwidth]{plot_otoc_nsns_b50_draft_Re.eps}\n\\includegraphics[angle=-90, width=0.49\\columnwidth]{plot_otoc_nsns_b50_draft_Im.eps}\n\\includegraphics[angle=-90, width=0.49\\columnwidth]{plot_coth_b50_nsns_inset.eps}\n\\includegraphics[angle=-90, width=0.49\\columnwidth]{plot_otoc_nsns_b50_draft_modulus_zoom.eps}\n\\caption{\nTop panels: Real and imaginary parts of the OTOC $\\langle \\hat n_\\sigma(t)\\hat n_\\sigma(0), \\hat n_\\sigma(t)\\hat n_\\sigma(0)\\rangle  - \\langle \\hat n_\\sigma\\rangle^4$ for the single-orbital Hubbard model with $\\beta=50$ at half filling. All the colored lines correspond to metallic solutions. \nBottom left panel: Spectral function $\\mathscr A_{(n_\\sigma n_\\sigma)^2}(\\omega)=-\\frac{1}{\\pi}{\\rm Im}\\,C_{(n_\\sigma n_\\sigma)^2}^R(\\omega)$ multiplied by $\\coth\\big(\\frac{\\beta\\omega}{4}\\big)$. In the inset, we plot the imaginary-time four-point function $\\langle \\hat n_\\sigma(\\tau)\\hat n_\\sigma(0), \\hat n_\\sigma(\\tau)\\hat n_\\sigma(0)\\rangle-\\langle \\hat n_\\sigma\\rangle^4$, with the black line corresponding to the Mott insulating solution at $U=6$.  \nBottom right panel: Modulus of the OTOC $|\\langle \\hat n_\\sigma(t)\\hat n_\\sigma(0), \\hat n_\\sigma(t)\\hat n_\\sigma(0)\\rangle  - \\langle \\hat n_\\sigma\\rangle^4|$ (normalized at $t=0$) on a log-log scale.\n}\n\\label{fig:results_nsns}\n\\end{center}\n\\end{figure}\n\nFigure~\\ref{fig:results_nsns} \nshows the results of the OTOC $\\langle \\hat n_\\sigma(t)\\hat n_\\sigma(0), \\hat n_\\sigma(t)\\hat n_\\sigma(0)\\rangle$\nfor the single-orbital Hubbard model. At half filling, this correlation function approaches $\\langle \\hat n_\\sigma\\rangle^4=\\frac{1}{16}$ in the long-time limit and at sufficiently low temperature, \nso that we perform the analytical continuation procedure for $\\langle \\hat n_\\sigma(t)\\hat n_\\sigma(0), \\hat n_\\sigma(t)\\hat n_\\sigma(0)\\rangle-\\langle \\hat n_\\sigma\\rangle^4$. \nThe top panels of Fig.~\\ref{fig:results_nsns} show the real and imaginary parts of \nthis shifted OTOC. \nHere, we only present results for the metallic phase, because resolving the very sharp low-energy feature of the OTOC spectrum in the half-filled Mott state is challenging. One can see that coherent oscillations are not observed for this type of OTOC, and that the incoherent part becomes dominant compared to the other two OTOCs shown in Figs.~\\ref{fig:results_fermion} and \\ref{fig:results_nn}. The amplitude of the OTOC $\\langle \\hat n_\\sigma(t)\\hat n_\\sigma(0), \\hat n_\\sigma(t)\\hat n_\\sigma(0)\\rangle-\\langle \\hat n_\\sigma\\rangle^4$ is enhanced as $U$ increases. As we argue below, this is related to the enhancement of spin fluctuations \nand local moment formation close to the Mott transition.\n\nThe bottom left panel in Fig.~\\ref{fig:results_nsns} plots the analytically continued spectral function $\\mathscr A_{(n_\\sigma n_\\sigma)^2}(\\omega)=-\\frac{1}{\\pi}{\\rm Im}\\, C_{(n_\\sigma n_\\sigma)^2}^R(\\omega)$ multiplied by $\\coth\\big(\\frac{\\beta\\omega}{4}\\big)$. \nWe can see that the spectral weight is concentrated in the low-energy region as we approach the Mott transition point.\nThe inset in the bottom left panel of Fig.~\\ref{fig:results_nsns} shows the imaginary-time four-point function $\\langle \\hat n_\\sigma(\\tau)\\hat n_\\sigma(0), \\hat n_\\sigma(\\tau)\\hat n_\\sigma(0)\\rangle-\\langle \\hat n_\\sigma\\rangle^4$, with the black line corresponding to a Mott insulating solution ($U=6$).  \nFor $U\\gtrsim 4$, the imaginary-time four-point function does not decay to zero but remains relatively large near $\\tau=\\frac{\\beta}{4}$. \nThis behavior is reminiscent of the spin-freezing physics seen in the correlated Hund metal phase of multi-orbital Hubbard models, where the dynamical spin correlation function $\\langle \\hat S_z(\\tau)\\hat S_z(0)\\rangle$ is trapped at a finite value at long $\\tau$ \\cite{Werner2008}. \nIn fact, the OTOC $\\langle \\hat n_\\sigma(\\tau)\\hat n_\\sigma(0), \\hat n_\\sigma(\\tau)\\hat n_\\sigma(0)\\rangle$\nmeasures spin correlations (in addition to charge correlations) through $\\hat n_\\sigma=\\frac{1}{2}\\hat n+\\sigma\\hat S_z$.\nPhysically, the trapping of $\\langle \\hat S_z(\\tau)\\hat S_z(0)\\rangle$ signals the formation of frozen local magnetic moments.\nIn the metal-insulator crossover region, the scattering induced by these magnetic moments results in incoherent metal states. Intuitively, one might expect fast scrambling in the corresponding regions of the phase diagram. \nAlthough there is some resemblance with the spin freezing crossover,\nthe self-energy does not exhibit a square-root frequency dependence in the single-orbital case, and a spin-freezing crossover in the sense of Ref.~\\cite{Werner2008} does not exist.  \n\nThe bottom right panel of Fig.~\\ref{fig:results_nsns} plots the modulus of the OTOC $|\\langle \\hat n_\\sigma(t)\\hat n_\\sigma(0), \\hat n_\\sigma(t)\\hat n_\\sigma(0)\\rangle-\\langle \\hat n_\\sigma\\rangle^4|$ (normalized at $t=0$) on a log-log scale. \nThe dynamics of $\\langle \\hat n_\\sigma(t)\\hat n_\\sigma(0), \\hat n_\\sigma(t)\\hat n_\\sigma(0)\\rangle$ is quite distinct from that of $\\langle \\hat n(t)\\hat n(0), \\hat n(t)\\hat n(0)\\rangle$. As the Mott transition is approached, we observe a slow-down in the decay of the modulus, \nwhich indicates the presence of slowly fluctuating local moments. \nThe long-time behavior may be consistent with a power-law, although slow oscillations make it difficult to determine the long-time asymptotic form from the numerics. \n\n\n\\begin{figure}[t]\n\\begin{center}\n\\includegraphics[angle=-90, width=0.49\\columnwidth]{plot_fitnsns_b2.eps}\n\\includegraphics[angle=-90, width=0.49\\columnwidth]{plot_coeffc_draft.eps}\n\\caption{\nLeft panel: \nThe imaginary-time four-point function $\\langle \\hat n_\\sigma(\\tau)\\hat n_\\sigma(0), \\hat n_\\sigma(\\tau)\\hat n_\\sigma(0)\\rangle$, the squared density-density correlation function $\\langle \\hat n_\\sigma(\\tau)\\hat n_\\sigma(0)\\rangle^2$,\nand their difference for the single-orbital Hubbard model with $U=5.5$ and $\\beta=50$.\nThe dashed curve shows a fit of the difference with the function $f_\\text{fit}(\\tau)=a+b\\tau+c\\tau^2$ in the interval $\\tau/\\beta \\in [0: 0.2]$.\nRight panel: Fitting coefficient $c$ as a function of $U$ for indicated values of $\\beta$.\n\n}\n\\label{fig:coeffs}\n\\end{center}\n\\end{figure}\n\nFor large enough $\\beta$ and small enough $\\tau$, the OTOC (\\ref{otoc_nsns}) factorizes into $\\langle \\hat n_\\sigma(\\tau)\\hat n_\\sigma(0), \\hat n_\\sigma(\\tau)\\hat n_\\sigma(0)\\rangle \\approx \\langle \\hat n_\\sigma(\\tau)\\hat n_\\sigma(0)\\rangle^2$. \nOn the real-time axis, this corresponds to the decoupling $\\langle \\hat n_\\sigma(t)\\hat n_\\sigma(0), \\hat n_\\sigma(t)\\hat n_\\sigma(0)\\rangle \\approx \\langle \\hat n_\\sigma(t)\\hat n_\\sigma(0)\\rangle^2$, that is, \nthe OTOC is reduced to a product of ordinary density-density correlation functions.\nIn order to see whether nontrivial correlations are captured beyond the decoupled form by the OTOC function, \nwe consider the difference $\\langle \\hat n_\\sigma(\\tau)\\hat n_\\sigma(0), \\hat n_\\sigma(\\tau)\\hat n_\\sigma(0)\\rangle-\\langle \\hat n_\\sigma(\\tau)\\hat n_\\sigma(0)\\rangle^2$ and fit the short-time behavior of this function to $f_\\text{fit}(\\tau)=a+b\\tau+c\\tau^2$, as illustrated in the left panel of Fig.~\\ref{fig:coeffs}. The coefficients $b$ and $c$ serve as  indicators of the nontrivial correlations that cannot be attributed to the decoupled form of the OTOC.\nThe coefficient $c$ is plotted as a function of $U$ and for different $\\beta$ in the right hand panel. The coefficient $b$ (not shown) exhibits a similar trend. We notice that the OTOC picks up nontrival correlations near the metal-insulator transition, and in particular at \nintermediate\ntemperatures, while the Mott phase shows no such correlations. The crossover region  \nassociated with the emergence of local moments \ncorresponds, roughly, to the interaction range where the nontrivial correlations start to become significant. \n\n\n\\subsection{Two-orbital Hubbard model}\n\\label{sec:two-orbital hubbard}\n\nIn this section, we investigate OTOCs in the two-orbital Hubbard model with Hund coupling $J>0$ \nand density-density interactions. (Three-orbital results are presented in Appendix~\\ref{sec:3orbital}.) \nThe Hamiltonian is given by\n\\begin{align}\nH\n&=\n-v\\sum_{\\langle i,j\\rangle\\alpha\\sigma} (c^\\dagger_{i\\alpha\\sigma}c_{j\\alpha\\sigma}+ \\text{h.c.})\n-\\mu\\sum_{i\\alpha\\sigma} \\hat n_{i\\alpha\\sigma}\n+U\\sum_{i\\alpha} \\hat n_{i\\alpha\\uparrow}\\hat n_{i\\alpha\\downarrow}\n+(U-2J)\\sum_{i\\sigma} \\hat n_{i1\\sigma}\\hat n_{i2\\bar\\sigma}\n\\notag\n\\\\\n&\\quad\n+(U-3J)\\sum_{i\\sigma} \\hat n_{i1\\sigma}\\hat n_{i2\\sigma},\n\\end{align}\n\nwith $U$ being the intra-orbital interaction $U$, $U-2J$ the inter-orbital antiparallel-spin interaction, and $U-3J$ the inter-orbital parallel-spin interaction. This is the simplest model that shows a crossover from a spin-frozen metal to a Fermi-liquid metal as one dopes the half-filled Mott insulator, with the self-energy scaling as $\\text{Im}\\Sigma(\\omega_n)\\sim\\sqrt{\\omega_n}$ over a significant energy range in the crossover regime \\cite{Hafermann2012}.\nThe sketch of the phase diagram is shown in Fig.~\\ref{fig:illustration}. \nAs we have seen in the previous section, the spin-related OTOC of the type (\\ref{otoc_nsns}) is relevant for the analysis of  the spin-freezing crossover, so that we concentrate on this OTOC here.\nIn the following calculations, we choose $U=8$, $J=U/4$, and compute $\\langle \\hat n_\\sigma(t)\\hat n_\\sigma(0), \\hat n_\\sigma(t)\\hat n_\\sigma(0)\\rangle - \\langle n_\\sigma \\rangle^4$ \nas a function of filling at $\\beta=50$ (dashed lines in Fig.~\\ref{fig:illustration}). \n\n\n\\begin{figure}[t]\n\\begin{center}\n\\includegraphics[angle=-90, width=0.49\\columnwidth]{plot_qp_draft.eps}\\hfill\n\\includegraphics[angle=-90, width=0.49\\columnwidth]{plot_fit_2orbital.eps}\n\\caption{\nLeft panel: Imaginary part of the self-energy as a function of Matsubara frequency for the two-orbital Hubbard model with $U=8$, $J=U/4$, and $\\beta=50$ on a log-log scale, with the dashed line indicating $\\sqrt{\\omega_n}$ behavior. \nRight panel: The coefficient $c$ extracted from\nfitting $\\langle \\hat n_\\sigma(\\tau)\\hat n_\\sigma(0), \\hat n_\\sigma(\\tau)\\hat n_\\sigma(0)\\rangle-\\langle \\hat n_\\sigma(\\tau)\\hat n_\\sigma(0)\\rangle^2$ with a function $f_\\text{fit}(\\tau)=a+b\\tau+c\\tau^2$\nin the interval $\\tau/\\beta\\in [0: 0.05]$ for the two-orbital Hubbard model. \nThe spin-freezing crossover occurs roughly at the filling $n_\\sigma\\approx 0.28$ (blue curve in the left panel and blue arrow in the right panel). \n}\n\\label{fig:self}\n\\end{center}\n\\end{figure}\n\n\nThe left panel of Fig.~\\ref{fig:self} plots the imaginary part of the self-energy as a function of Matsubara frequency in order to identify the filling corresponding to the spin-freezing crossover \\cite{Werner2008}. An approximate square-root scaling is observed over a wide range of frequencies near $n_\\sigma\\approx 0.28$ (blue line). In the right panel of Fig.~\\ref{fig:self}, we present the coefficient $c$ obtained from a similar analysis as presented in Fig.~\\ref{fig:coeffs}, but with a fitting range $\\tau/\\beta\\in [0 : 0.05]$. Again, we find that the OTOC $\\langle \\hat n_\\sigma(t)\\hat n_\\sigma(0), \\hat n_\\sigma(t)\\hat n_\\sigma(0)\\rangle - \\langle \\hat n_\\sigma\\rangle^4$ detects nontrivial correlations in the spin-freezing crossover (blue arrow) and spin-frozen metal regime, but not in the Fermi liquid or the Mott insulating phase.\n\n\n\\begin{figure}[t]\n\\begin{center}\n\\includegraphics[angle=-90, width=0.49\\columnwidth]{plot_otoc_2orbital_Re.eps}\n\\includegraphics[angle=-90, width=0.486\\columnwidth]{plot_otoc_2orbital_Im.eps}\n\\includegraphics[angle=-90, width=0.484\\columnwidth]{plot_spectra_coth_2orbital_inset.eps}\n\\includegraphics[angle=-90, width=0.486\\columnwidth]{plot_otoc_2orbital_modulus_inset.eps}\n\\caption{\nTop panels: Real and imaginary parts of the OTOC $\\langle \\hat n_\\sigma(t)\\hat n_\\sigma(0), \\hat n_\\sigma(t)\\hat n_\\sigma(0)\\rangle  - \\langle \\hat n_\\sigma\\rangle^4$ for the two-orbital Hubbard model with $U=8$, $J=U/4$, $\\beta=50$ and indicated fillings. \nBottom left panel: Spectral function $\\mathscr A_{(n_\\sigma n_\\sigma)^2}(\\omega)=-\\frac{1}{\\pi}{\\rm Im}\\,C_{(n_\\sigma n_\\sigma)^2}^R(\\omega)$ multiplied by $\\coth\\big(\\frac{\\beta\\omega}{4}\\big)$. In the inset, we plot the imaginary-time four-point function $\\langle \\hat n_\\sigma(\\tau)\\hat n_\\sigma(0), \\hat n_\\sigma(\\tau)\\hat n_\\sigma(0)\\rangle-\\langle \\hat n_\\sigma\\rangle^4$. \nBottom right panel: Modulus of the OTOC $|\\langle \\hat n_\\sigma(t)\\hat n_\\sigma(0), \\hat n_\\sigma(t)\\hat n_\\sigma(0)\\rangle  - \\langle \\hat n_\\sigma\\rangle^4|$ (normalized at $t=0$) on a log-log scale.\nIn the inset of the bottom right panel, we illustrate the approximate data collapse obtained by considering the spin-freezing crossover regimes for different values of $J$. \nIn all the panels, the thick blue curves represent the results for the spin-freezing crossover region. \n}\n\\label{fig:2orbital}\n\\end{center}\n\\end{figure}\n\n\nThe top panels of Fig.~\\ref{fig:2orbital} show the real and imaginary parts of the OTOC\n$\\langle \\hat n_\\sigma(t)\\hat n_\\sigma(0), \\hat n_\\sigma(t)\\hat n_\\sigma(0)\\rangle-\\langle \\hat n_\\sigma\\rangle^4$\nfor the two-orbital Hubbard model. In the spin-frozen phase ($n_\\sigma \\gtrsim 0.3$), both components exhibit \nhighly incoherent oscillations, which get suppressed as the filling is reduced. As one moves across the crossover point ($n_\\sigma \\sim 0.28$),\nthe oscillations completely disappear, and the real part of the OTOC starts to overshoot to the negative side\nin the Fermi liquid regime.\nThe bottom left panel in Fig.~\\ref{fig:2orbital} presents the analytically continued spectral function $\\mathscr A_{(n_\\sigma n_\\sigma)^2}(\\omega)=-\\frac{1}{\\pi}{\\rm Im}\\,C_{(n_\\sigma n_\\sigma)^2}^R(\\omega)$ multiplied by $\\coth\\big(\\frac{\\beta\\omega}{4}\\big)$. \nIn the spin-frozen regime ($n_\\sigma \\gtrsim 0.3$) a sharp peak appears at low energy, which originates from the saturation of the imaginary-time four point function $\\langle \\hat n_\\sigma(\\tau)\\hat n_\\sigma(0), \\hat n_\\sigma(\\tau)\\hat n_\\sigma(0)\\rangle$ at large $\\tau$ (see the inset of the bottom left panel of Fig.~\\ref{fig:2orbital}). This low-energy peak represents the slow dynamics of the frozen spins, and translates into a slow long-time decay of $|\\langle \\hat n_\\sigma(t)\\hat n_\\sigma(0), \\hat n_\\sigma(t)\\hat n_\\sigma(0)\\rangle-\\langle \\hat n_\\sigma\\rangle^4|$,\nas shown in the bottom right panel of Fig.~\\ref{fig:2orbital}\non a log-log scale. Again, it is difficult to clearly resolve the long-time behavior of the OTOC due to the limited\naccuracy of the analytic continuation, but in the spin-freezing crossover regime (thick blue curves)\nthe decay of the OTOC is consistent with a power law $\\sim 1/t^{1.5}$ at least for $2\\lesssim t \\lesssim 5$. \nThe shorter-time behavior ($0.5\\lesssim t \\lesssim 2$) is instead well fitted by an exponential decay $\\sim e^{-\\alpha t}$ with a decay constant $\\alpha=0.78$. \nThis exponent exhibits\na strong dependence on the Hund coupling. \n\nIn fact, the OTOCs in the spin-freezing regime for $J=U/4$ and $J=U/6$ can be approximately collapsed by plotting them as a function of $tJ$ (see the inset in the bottom right panel of Fig.~\\ref{fig:2orbital}), which indicates that the Hund coupling is the parameter which controls the dynamics in this regime. \nThis is distinct from the Lyapunov behavior in which the exponential decay scales with $t/\\beta$ \\cite{Bagrets2017}.\nA possible reason is that we are still far from the large $N$ limit (where $N$ corresponds to the number of orbitals times the number of spin degrees of freedom) so that the large $N$ behavior \nsuch as the Lyapunov growth is not observed in our calculations. \nIn fact, the strong dependence of the time scale on $J$ and weak dependence on $\\beta$ is consistent\nwith the behavior of the finite $N$ SYK model in Ref.~\\cite{FuSachdev2016}.\nIn the spin-frozen regime ($n_\\sigma\\gtrsim 0.3$) the decay is much slower than near the crossover point, while in the Fermi liquid regime the decay is accelerated, approaching the free fermion behavior of $t^{-3}$\nas shown in Fig.~\\ref{analytic continuation}. \nQualitatively similar results for this OTOC are obtained \nfor the three-orbital Hubbard model (see Appendix~\\ref{sec:3orbital}).\n\nThe time dependence of the modulus of \n$\\langle \\hat n_\\sigma(\\tau)\\hat n_\\sigma(0), \\hat n_\\sigma(\\tau)\\hat n_\\sigma(0)\\rangle-\\langle \\hat n_\\sigma\\rangle^4$ \nin the spin-freezing crossover regime of multi-orbital Hubbard models (exponential decay crossing over into a power-law) bears a close resemblance to a (different) OTOC for the SYK model discussed in Ref.~\\cite{Bagrets2017}. \nAlso, a recent study of yet another type of OTOCs for the finite-$N$ SYK model found a qualitatively similar decay as observed here in the spin-freezing crossover regime \\cite{FuSachdev2016}. In the following section, we will make the connection to the SYK dynamics more quantitative.   \n\n\n\\subsection{Comparison to the SKY model}\n\\label{sec:SYK}\n\nIn this section, we compare the results of the multi-orbital Hubbard models to the SYK model.\nWe consider the particle-hole symmetric SYK model of complex fermions \\cite{FuSachdev2016},\nwhose Hamiltonian is given by\n\\begin{align}\nH\n&=\n\\frac{1}{(2N)^{3/2}} \\sum_{i,j,k,l=1}^N \nJ_{ij;kl} (c_i^\\dagger c_j^\\dagger c_k c_l\n+\\delta_{ik}n c_j^\\dagger c_l\n-\\delta_{il}n c_j^\\dagger c_k\n-\\delta_{jk}n c_i^\\dagger c_l\n+\\delta_{jl}n c_i^\\dagger c_k).\n\\end{align}\nThe coupling constant $J_{ij;kl}$ is a gaussian random variable, satisfying $J_{ij;kl}=-J_{ji;kl}=-J_{ij;lk}=J_{kl;ij}^\\ast$, and\n\\begin{align}\n\\overline{({\\rm Re}\\, J_{ij;kl})^2}\n&=\n\\begin{cases}\nJ_{\\rm SYK}^2/2 & (i,j)\\neq (k,l) \\\\\nJ_{\\rm SYK}^2 & (i,j)=(k,l)\n\\end{cases},\n\\\\\n\\overline{({\\rm Im}\\, J_{ij;kl})^2}\n&=\n\\begin{cases}\nJ_{\\rm SYK}^2/2 & (i,j)\\neq (k,l) \\\\\n0 & (i,j)=(k,l)\n\\end{cases},\n\\end{align}\nwhere the overline represents an average over each realization of $J_{ij;kl}$. The parameter \n$J_{\\rm SYK}$ ($J_{\\rm SYK}^{-1}$) defines the unit of energy (time) in the following calculations. \nThe system is particle-hole symmetric when\n$n=0.5$, which fixes the total number of fermions as $N^{-1}\\sum_{i=1}^N \\langle c_i^\\dagger c_i\\rangle=0.5$.\nWe take the particle-hole symmetric form of the SYK model because it has been well studied previously \\cite{FuSachdev2016} and avoids the effect of the filling drift.\nWe numerically solve the model by exact diagonalization for finite $N(\\le 12)$, and evaluate the density-density OTOC\n$\\overline{\\langle \\hat n_i(t)\\hat n_i(0), \\hat n_i(t)\\hat n_i(0)\\rangle-\\langle \\hat n_i\\rangle^4}$\nand its imaginary-time counterpart.\n\n\\begin{figure}[t]\n\\begin{center}\n\n\\includegraphics[angle=-90,width=0.49\\columnwidth]{plot_fitnsns_syk.eps}\n\n\\includegraphics[angle=-90,width=0.49\\columnwidth]{plot_coeffc_syk.eps}\n\\caption{\nLeft panel: \nThe imaginary-time four-point function $\\overline{\\langle \\hat n_i(\\tau)\\hat n_i(0), \\hat n_i(\\tau)\\hat n_i(0)\\rangle}$, the density-density correlation function $\\overline{\\langle \\hat n_i(\\tau)\\hat n_i(0)\\rangle^2}$,\nand their difference for the SYK model with $N=12$ and $\\beta J_{\\rm SYK}=10$.\nThe dashed curve shows a fit of the difference with a function $f_\\text{fit}(\\tau)=a+b\\tau+c\\tau^2$ in the interval $\\tau/\\beta \\in [0: 0.2]$. \nRight panel: Fitting coefficient $c$ as a function of $T/J_{\\rm SYK}$.\n}\n\\label{fig:SYK fit}\n\\end{center}\n\\end{figure}\n\nIn the left panel of Fig.~\\ref{fig:SYK fit}, we plot the imaginary-time four-point function \n$\\overline{\\langle \\hat n_i(\\tau)\\hat n_i(0), \\hat n_i(\\tau)\\hat n_i(0)\\rangle}$\nfor the SYK model with $N=12$ and $\\beta J_{\\rm SYK}=10$. \nAt low temperature, we expect that $\\overline{\\langle \\hat n_i(\\tau)\\hat n_i(0), \\hat n_i(\\tau)\\hat n_i(0)\\rangle}$ is decoupled into $\\overline{\\langle \\hat n_i(\\tau)\\hat n_i(0)\\rangle^2}$.\nTo see whether nontrivial correlations beyond the decoupled form are present, we fit the difference\n$\\overline{\\langle \\hat n_i(\\tau)\\hat n_i(0), \\hat n_i(\\tau)\\hat n_i(0)\\rangle}\n-\\overline{\\langle \\hat n_i(\\tau)\\hat n_i(0)\\rangle^2}$ with a function $f_{\\rm fit}(\\tau)=a+b\\tau+c\\tau^2$\nin the interval $\\tau/\\beta\\in[0:0.2]$, in the same manner as for the Hubbard models.\nThe obtained coefficient $c$ is plotted as a function of the temperature $T$ in the right panel of\nFig.~\\ref{fig:SYK fit}. One can see that nontrivial correlations exist in a wide range of temperature.\nEspecially, they are enhanced in the intermediate temperature regime. This is because in the zero-temperature limit\nthe four-point function is decoupled as \n$\\langle \\hat n_i(\\tau)\\hat n_i(0), \\hat n_i(\\tau)\\hat n_i(0)\\rangle \\approx \\langle \\hat n_i(\\tau)\\hat n_i(0)\\rangle^2$, while in the high-temperature limit ($\\beta\\to 0$) the imaginary-time dependence is washed out.\n\n\n\n\n\\begin{figure}[t]\n\\begin{center}\n\\includegraphics[angle=-90, width=0.49\\columnwidth]{plot_syk_Re.eps}\n\\includegraphics[angle=-90, width=0.49\\columnwidth]{plot_syk_Im.eps}\n\\includegraphics[angle=-90, width=0.49\\columnwidth]{plot_syk_modulus-log.eps}\n\\includegraphics[angle=-90, width=0.49\\columnwidth]{plot_syk_modulus.eps}\n\\caption{\nThe density-density OTOC $\\overline{\\langle \\hat n_i(t)\\hat n_i(0), \\hat n_i(t)\\hat n_i(0)\\rangle-\\langle \\hat n_i\\rangle^4}$ for the particle-hole symmetric SYK model averaged over $10^2$ samples with $\\beta J_{\\rm SYK}=100$ and various $N$. \nThe top left and right panels show the real and imaginary parts of the OTOC,\nwhile the bottom left and right panels show the log and log-log plots of the modulus of the OTOC (normalized at $t=0$).\nThe dashed line shows a power-law decay $\\propto t^{-1.6}$ for comparison.}\n\\label{fig:SYK}\n\\end{center}\n\\end{figure}\n\nIn Fig.~\\ref{fig:SYK}, we show the numerical results of the density-density OTOC for $\\beta J_{\\rm SYK}=100$.\nThe real part of the OTOC rapidly drops from the initial value within the time scale of $tJ_{\\rm SYK}\\sim 2$,\nand slowly decays to zero in the long time limit.\nAs one increases $N$, the oscillations that appear at longer time tend to be suppressed.\nOne can see that the result for $N=12$ in the top panels of Fig.~\\ref{fig:SYK} closely resembles the OTOC in \nthe spin-freezing crossover regime of the two-orbital Hubbard model shown\nby the blue curve in the top panels of Fig.~\\ref{fig:2orbital},\nwhile it differs from the single-orbital result shown in the top panels of Fig.~\\ref{fig:results_nsns}.\nDue to the limitation of our calculations to small system sizes ($N\\le 12$), we do not clearly observe\nan exponential growth ($\\sim c_0-c_1 e^{\\lambda t}$)  at short times or an exponential decay ($\\sim e^{-\\alpha t}$) \nat intermediate times. \nHowever, we find that for $tJ_{\\rm SYK}\\gtrsim 2$ the OTOC decays\napproximately in a power law $\\propto t^{-\\gamma}$ with $\\gamma=1.6$ (see bottom right panel in Fig.~\\ref{fig:SYK}). \nThe time interval exhibiting the power-law decay becomes longer as we increase $N$.  \nThe temperature dependence of the OTOC for the SYK model is shown in Fig.~\\ref{fig:SYK-T}.\nThe time scale of the initial drop is more or less independent of the temperature, while \nthe power-law-like decay is only visible in the low-temperature regime.\n\n\\begin{figure}[t]\n\\begin{center}\n\\includegraphics[angle=-90, width=0.49\\columnwidth]{plot_syk_beta_re.eps}\n\\includegraphics[angle=-90, width=0.49\\columnwidth]{plot_syk_beta_im.eps}\n\\includegraphics[angle=-90, width=0.49\\columnwidth]{plot_syk_beta_modulus-log.eps}\n\\includegraphics[angle=-90, width=0.49\\columnwidth]{plot_syk_beta_modulus.eps}\n\\caption{\nTemperature dependence of the density-density OTOC\n$\\overline{\\langle \\hat n_i(t)\\hat n_i(0), \\hat n_i(t)\\hat n_i(0)\\rangle-\\langle \\hat n_i\\rangle^4}$ for the particle-hole symmetric SYK model averaged over $10^2$ samples with $N=12$.\nThe top left and right panels show the real and imaginary parts of the OTOC,\nwhile the bottom left and right panels show the log and log-log plots of the modulus of the OTOC.\nThe dashed line shows a power-law decay $\\propto t^{-1.6}$ for comparison.}\n\\label{fig:SYK-T}\n\\end{center}\n\\end{figure}\n\nIn Ref.~\\onlinecite{Werner2018}, it has been argued that the spin-freezing crossover regime of multi-orbital Hubbard models is effectively described by the SYK model in which $J_{\\rm SYK}$ is replaced by the Hund coupling $J$. The results in Fig.~\\ref{fig:2orbital} are for the two-orbital Hubbard model with $U=8$, $J/U=1/4$ and $\\beta=50$, corresponding to $\\beta J=100$, which means that it is meaningful to directly compare the blue lines in Fig.~\\ref{fig:2orbital} with the blue lines in Figs.~\\ref{fig:SYK} and \\ref{fig:SYK-T}. Not only is the qualitative agreement remarkable, even the power-law exponent measured for the two-orbital Hubbard model ($\\gamma=1.5$) is very close to the exponent extracted from the finite-$N$ SYK calculation. \nThe power-law decay in the three-orbital case is approximately $1/t^{1.75}$ (see Appendix~\\ref{sec:3orbital}), and thus also in semi-quantitative agreement with the SYK model behavior. \nThis nontrivial result provides further support for the identification of the spin-freezing crossover regime of multi-orbital Hubbard systems with an SYK strange metal. \n\nIn the large-$N$ and low-$T$ limit of the SYK model, we expect that the power-law behavior approaches $1/t^2$, since in this limit the OTOC is decoupled as $\\langle \\hat n_i(t)\\hat n_i(0), \\hat n_i(t)\\hat n_i(0)\\rangle\\approx \\langle \\hat n_i(t)\\hat n_i(0)\\rangle^2$ and the decay of $\\langle \\hat n_i(t)\\hat n_i(0)\\rangle$ is dominated by what corresponds to the slow spin relaxation $\\langle \\hat S_z(t)\\hat S_z(0)\\rangle\\sim 1/t$ in the Sachdev-Ye model.\n\n\n\n\n\n4.1 Observables and numerical procedure\n\\subsection{Observables and numerical procedure}\n\nWe consider the single-orbital, two-orbital, and three-orbital Hubbard models on an infinitely-connected Bethe lattice, which can be solved exactly within DMFT \\cite{GeorgesKotliarKrauthRozenberg1996}. In this case, the noninteracting density of states is semicircular with bandwidth $4v_\\ast$, for which there exists a simplified DMFT self-consistency condition between the hybridization function $\\Delta_{\\alpha\\sigma}$ of the DMFT impurity problem and the local (impurity) Green's function $G_{\\alpha\\sigma}$: \n\n\\begin{equation}\n\\Delta_{\\alpha\\sigma}(\\tau)=v_\\ast^2 G_{\\alpha\\sigma}(\\tau), \n\\label{semicirc}\n\\end{equation}\n\nwith $\\alpha$ the orbital and $\\sigma$ the spin index. We will use $v_\\ast$ as the unit of energy and measure time in units of $\\hbar/v_\\ast$. \n\nThree different types of imaginary-time four-point functions $C_{(AB)^2}^M(\\tau)$\nof the form of Eq.~(\\ref{imaginary-time OTOC})\nwith $(\\hat A, \\hat B)=(c_\\sigma^\\dagger, c_\\sigma), (\\hat n_\\sigma, \\hat n_\\sigma)$, and $(\\hat n, \\hat n)$\nare calculated in the interval $0\\le \\tau\\le \\frac{\\beta\\hbar}{2}$: \n\\begin{eqnarray}\nC_{(c^\\dagger_\\sigma c_\\sigma)^2}^M(\\tau) &=& -\\langle c^\\dagger_\\sigma(\\tau+\\tfrac{\\beta\\hbar}{2}) c_\\sigma(\\tfrac{\\beta\\hbar}{2}) c^\\dagger_\\sigma(\\tau) c_\\sigma(0) \\rangle, \\label{fermion_otoc}\\\\\nC_{(n_\\sigma n_\\sigma)^2}^M(\\tau) &=& -\\langle \\hat n_\\sigma(\\tau+\\tfrac{\\beta\\hbar}{2})\\hat n_\\sigma(\\tfrac{\\beta\\hbar}{2})\\hat n_\\sigma(\\tau)\\hat n_\\sigma(0) \\rangle, \\label{otoc_nsns}\\\\\nC_{(nn)^2}^M(\\tau) &=& -\\langle \\hat n(\\tau+\\tfrac{\\beta\\hbar}{2})\\hat n(\\tfrac{\\beta\\hbar}{2})\\hat n(\\tau)\\hat n(0) \\rangle,\\label{otoc_nn}\n\\end{eqnarray}\n\nwhere $c_\\sigma^\\dagger$ ($c_\\sigma$) are the fermionic creation (annihilation) operators for spin $\\sigma$ (and orbital $\\alpha=1$ in the multi-orbital case), $\\hat n_\\sigma=c^\\dagger_\\sigma c_\\sigma$ is the corresponding spin-dependent density operator, and $\\hat n=\\hat n_\\uparrow + \\hat n_\\downarrow$ the total density operator. Note that in all the three cases above we have $\\hat B=\\hat A^\\dagger$. \nWe measure these local correlation functions in the impurity model using a hybridization expansion continuous-time Monte Carlo algorithm (CT-HYB) \\cite{Werner2006}. In this algorithm, the \ntwo-particle\nGreen's functions of the type (\\ref{fermion_otoc}) can be measured by removing two hybridization lines, i.e., from the elements of the inverse hybridization matrix, while the density-density correlation functions can be easily measured either by insertion of density operators (matrix formalism) \\cite{Werner2006Kondo} or by reading off the occupation of the orbitals at the four time points in the segment implementation \\cite{Werner2006}. We use the latter algorithm since we consider only density-density interactions. \nThe values at the end-points $\\tau=0$ and $\\tau=\\frac{\\beta\\hbar}{2}$ reduce to standard density-density correlation functions, which in the case of Eq.~(\\ref{fermion_otoc}) are measured separately. \n\nWe perform the analytic continuation to the real-frequency axis using the Maximum Entropy method \\cite{Jarrell1996,Lewin_maxent} with a bosonic kernel. This yields the imaginary part of the retarded correlator \n$-\\frac{1}{\\pi}\\text{Im}\\, C^R_{(AA^\\dagger)^2}(\\omega)=\\mathscr A_{(AA^\\dagger)^2}(\\omega)$.  \nWhen $\\hat B=\\hat A^\\dagger=\\hat A$ (which is the case for $\\hat A=\\hat n_\\sigma, \\hat n$), we have $C_{(AA)^2}^R(\\omega)^\\ast=C_{(AA)^2}^R(-\\omega)$. At half filling, the particle-hole symmetry furthermore ensures $C_{c_\\sigma^\\dagger c_\\sigma}^R(\\omega)^\\ast=C_{c_\\sigma^\\dagger c_\\sigma}^R(-\\omega)$.\nThus, in all the cases considered in this study, the retarded OTOC has the symmetry property $C_{(AA^\\dagger)^2}^R(\\omega)^\\ast=C_{(AA^\\dagger)^2}^R(-\\omega)$. Using this, as well as the out-of-time-order fluctuation-dissipation\ntheorem discussed in the previous section, we can obtain the real-time OTOC\n$\\langle \\hat A(t)\\hat A^\\dagger(0), \\hat A(t)\\hat A^\\dagger(0)\\rangle$ from the following inverse Fourier transformation, \n\\begin{align}{\\rm Re}\\langle \\hat A(t)\\hat A^\\dagger(0), \\hat A(t)\\hat A^\\dagger(0)\\rangle\n&=\n{\\rm Re}\n\\int_0^\\infty d\\omega\\, e^{-i\\omega t}\\coth\\left(\\frac{\\beta\\hbar\\omega}{4}\\right)\n\\left(-\\frac{1}{\\pi}{\\rm Im}\\, C_{(AA^\\dagger)^2}^R(\\omega)\\right),\n\\\\\n{\\rm Im}\\langle \\hat A(t)\\hat A^\\dagger(0), \\hat A(t)\\hat A^\\dagger(0)\\rangle\n&=\n{\\rm Im}\\int_0^{\\infty} d\\omega\\, e^{-i\\omega t} \\left(-\\frac{1}{\\pi}{\\rm Im}\\, C_{(AA^\\dagger)^2}^R(\\omega)\\right),\n\\end{align}\nfor $\\hat A=c_\\sigma^\\dagger, \\hat n_\\sigma$, and $\\hat n$.\n\n\\begin{figure}[b]\n\\begin{center}\n\\includegraphics[angle=-90, width=0.48\\columnwidth]{compare_u0_draft.eps}\\hfill\n\\includegraphics[angle=-90, width=0.48\\columnwidth]{compare_u0_longtime_modulus.eps}\n\\caption{\nLeft panel: Comparison between the exact solution and the result obtained by analytical continuation\nfor the real and imaginary parts of the OTOC $\\langle c_\\sigma^\\dagger(t)c_\\sigma(0), c_\\sigma^\\dagger(t)c_\\sigma(0)\\rangle$ \nin the noninteracting case ($U=0, \\beta\\hbar=50$) of the single-orbital Hubbard model.\nRight panel: Comparison of the long-time behavior for the modulus $|\\langle c_\\sigma^\\dagger(t)c_\\sigma(0), c_\\sigma^\\dagger(t)c_\\sigma(0)\\rangle|$. \nThe exact result shows a power-law decay ($\\sim 1/t^3$). \n}\n\\label{fig:test_u0}\n\\end{center}\n\\end{figure}\n\nAs a check of our procedure, we first compare the results for the noninteracting model,\nfor which an exact solution of the OTOC $\\langle c_\\sigma^\\dagger(t)c_\\sigma(0), c_\\sigma^\\dagger(t)c_\\sigma(0)\\rangle$ \nis available \\cite{TsujiWernerUeda2017}.\nIt is a nontrivial task to measure the noninteracting two-particle Green's function in CT-HYB, \nand to analytically continue \n$C_{(c^\\dagger_\\sigma c_\\sigma)^2}^M(\\tau)$ \nby the Maximum Entropy method.  \nThe results for the inverse temperature $\\beta\\hbar=50$ are plotted in Fig.~\\ref{fig:test_u0}. \nOne can see that the OTOC oscillates for about one cycle, and quickly decays to zero.\nThe analytically continued data show a good agreement with the exact solution. \nIn particular, for short times (up to about two inverse hoppings) the dynamics is accurately reproduced, while deviations appear at longer times. \nIt is known that for the noninteracting fermion system the OTOC decays as a power law at long time ($\\sim 1/t^3$) \\cite{TsujiWernerUeda2017}.\nThe analytical continuation method cannot reproduce the details of the oscillations at intermediate or long times, but it roughly captures the long-time decay, which is controlled by low-frequency spectral features. \nUsually, analytic continuation is unreliable for high frequency components, because high-frequency information is suppressed \nby the kernel $K(\\tau,\\omega)$ in the transformation from the spectral function $A_O(\\omega)$ to the (bosonic) Matsubara correlation function $O(\\tau)$ [$O(\\tau)=\\int_{-\\infty}^{\\infty} d\\omega\\, K(\\omega,\\tau) A_O(\\omega)$ with $K(\\tau,\\omega)=e^{-\\tau\\omega}/(1-e^{-\\beta\\omega})$]. On the other hand, the low frequency components can be rather accurately determined. As a result, in Fig. 4 we more or less recover the low-frequency features of the time-dependent correlation function, while the details of rapid oscillations are not captured. The accurate result at short times can be understood from the fact that this region is close to the imaginary time axis in the complex time plane, where the original QMC data are available.\nAt higher temperatures, the Maximum Entropy method becomes less reliable, so that the resulting OTOCs are expected to be less accurate.  \n\n\n",
        "label": "fig:test_u0",
        "Descriptive_question1": "What is the value of the interaction strength U in figure_4 for the noninteracting case?",
        "Descriptive_question2": "What type of decay is shown by the exact result in the right panel of figure_4?",
        "Reasoning_question1": "How does the analytical continuation method compare to the exact solution in capturing the long-time behavior of the OTOC modulus in figure_4?",
        "Reasoning_question2": "What can be inferred about the accuracy of the analytical continuation method for short versus long times in figure_4?",
        "Descriptive_answer1": "U=0",
        "Descriptive_answer2": "power-law decay",
        "Reasoning_answer1": "The analytical continuation method roughly captures the long-time decay behavior of the OTOC modulus, which follows a power-law decay of approximately 1/t^3, comparable to the exact solution, although it does not reproduce the detailed oscillations at intermediate or long times.",
        "Reasoning_answer2": "The analytical continuation method accurately reproduces the short-time dynamics of the OTOC up to about two inverse hoppings because this time region is close to the imaginary time axis where QMC data are available; however, at longer times, deviations appear and the method fails to capture the detailed oscillations due to loss of high-frequency information during analytic continuation."
    },
    {
        "paper_id": "1812.04217.json",
        "image_id": "figure_5",
        "image_path": [
            "/home/yz979/scratch/chengye/scimolmo/test-images-tables/dataset_imagestables/1812.04217/images/plot_otoc_b50_Re.eps"
        ],
        "caption": "\nTop panels: Real and imaginary parts of the OTOC $\\langle c^\\dagger_\\sigma(t)c_\\sigma(0), c^\\dagger_\\sigma(t)c_\\sigma(0)\\rangle$ for the single-orbital Hubbard model with $\\beta=50$ at half filling. The thin lines correspond to metallic solutions, while the thick line ($U=6$) corresponds to a Mott insulating solution. \nBottom left panel: Spectral function $\\mathscr A_{(c_\\sigma^\\dagger c_\\sigma)^2}(\\omega)=-\\frac{1}{\\pi}{\\rm Im}\\,C_{(c^\\dagger_\\sigma c_\\sigma)^2}^R(\\omega)$ multiplied by $\\coth\\big(\\frac{\\beta\\omega}{4}\\big)$. \nBottom right panel: Modulus of the OTOC $|\\langle c^\\dagger_\\sigma(t)c_\\sigma(0), c^\\dagger_\\sigma(t)c_\\sigma(0)\\rangle|$ (normalized at $t=0$) on a log-log scale. The dashed line is the result for $U=0$.\n",
        "classify": "Chart",
        "section_info": "4 Numerical results for the Hubbard model\n\\section{Numerical results for the Hubbard model}\n\\label{numerical results}\n\n\\subsection{Observables and numerical procedure}\n\nWe consider the single-orbital, two-orbital, and three-orbital Hubbard models on an infinitely-connected Bethe lattice, which can be solved exactly within DMFT \\cite{GeorgesKotliarKrauthRozenberg1996}. In this case, the noninteracting density of states is semicircular with bandwidth $4v_\\ast$, for which there exists a simplified DMFT self-consistency condition between the hybridization function $\\Delta_{\\alpha\\sigma}$ of the DMFT impurity problem and the local (impurity) Green's function $G_{\\alpha\\sigma}$: \n\n\\begin{equation}\n\\Delta_{\\alpha\\sigma}(\\tau)=v_\\ast^2 G_{\\alpha\\sigma}(\\tau), \n\\label{semicirc}\n\\end{equation}\n\nwith $\\alpha$ the orbital and $\\sigma$ the spin index. We will use $v_\\ast$ as the unit of energy and measure time in units of $\\hbar/v_\\ast$. \n\nThree different types of imaginary-time four-point functions $C_{(AB)^2}^M(\\tau)$\nof the form of Eq.~(\\ref{imaginary-time OTOC})\nwith $(\\hat A, \\hat B)=(c_\\sigma^\\dagger, c_\\sigma), (\\hat n_\\sigma, \\hat n_\\sigma)$, and $(\\hat n, \\hat n)$\nare calculated in the interval $0\\le \\tau\\le \\frac{\\beta\\hbar}{2}$: \n\\begin{eqnarray}\nC_{(c^\\dagger_\\sigma c_\\sigma)^2}^M(\\tau) &=& -\\langle c^\\dagger_\\sigma(\\tau+\\tfrac{\\beta\\hbar}{2}) c_\\sigma(\\tfrac{\\beta\\hbar}{2}) c^\\dagger_\\sigma(\\tau) c_\\sigma(0) \\rangle, \\label{fermion_otoc}\\\\\nC_{(n_\\sigma n_\\sigma)^2}^M(\\tau) &=& -\\langle \\hat n_\\sigma(\\tau+\\tfrac{\\beta\\hbar}{2})\\hat n_\\sigma(\\tfrac{\\beta\\hbar}{2})\\hat n_\\sigma(\\tau)\\hat n_\\sigma(0) \\rangle, \\label{otoc_nsns}\\\\\nC_{(nn)^2}^M(\\tau) &=& -\\langle \\hat n(\\tau+\\tfrac{\\beta\\hbar}{2})\\hat n(\\tfrac{\\beta\\hbar}{2})\\hat n(\\tau)\\hat n(0) \\rangle,\\label{otoc_nn}\n\\end{eqnarray}\n\nwhere $c_\\sigma^\\dagger$ ($c_\\sigma$) are the fermionic creation (annihilation) operators for spin $\\sigma$ (and orbital $\\alpha=1$ in the multi-orbital case), $\\hat n_\\sigma=c^\\dagger_\\sigma c_\\sigma$ is the corresponding spin-dependent density operator, and $\\hat n=\\hat n_\\uparrow + \\hat n_\\downarrow$ the total density operator. Note that in all the three cases above we have $\\hat B=\\hat A^\\dagger$. \nWe measure these local correlation functions in the impurity model using a hybridization expansion continuous-time Monte Carlo algorithm (CT-HYB) \\cite{Werner2006}. In this algorithm, the \ntwo-particle\nGreen's functions of the type (\\ref{fermion_otoc}) can be measured by removing two hybridization lines, i.e., from the elements of the inverse hybridization matrix, while the density-density correlation functions can be easily measured either by insertion of density operators (matrix formalism) \\cite{Werner2006Kondo} or by reading off the occupation of the orbitals at the four time points in the segment implementation \\cite{Werner2006}. We use the latter algorithm since we consider only density-density interactions. \nThe values at the end-points $\\tau=0$ and $\\tau=\\frac{\\beta\\hbar}{2}$ reduce to standard density-density correlation functions, which in the case of Eq.~(\\ref{fermion_otoc}) are measured separately. \n\nWe perform the analytic continuation to the real-frequency axis using the Maximum Entropy method \\cite{Jarrell1996,Lewin_maxent} with a bosonic kernel. This yields the imaginary part of the retarded correlator \n$-\\frac{1}{\\pi}\\text{Im}\\, C^R_{(AA^\\dagger)^2}(\\omega)=\\mathscr A_{(AA^\\dagger)^2}(\\omega)$.  \nWhen $\\hat B=\\hat A^\\dagger=\\hat A$ (which is the case for $\\hat A=\\hat n_\\sigma, \\hat n$), we have $C_{(AA)^2}^R(\\omega)^\\ast=C_{(AA)^2}^R(-\\omega)$. At half filling, the particle-hole symmetry furthermore ensures $C_{c_\\sigma^\\dagger c_\\sigma}^R(\\omega)^\\ast=C_{c_\\sigma^\\dagger c_\\sigma}^R(-\\omega)$.\nThus, in all the cases considered in this study, the retarded OTOC has the symmetry property $C_{(AA^\\dagger)^2}^R(\\omega)^\\ast=C_{(AA^\\dagger)^2}^R(-\\omega)$. Using this, as well as the out-of-time-order fluctuation-dissipation\ntheorem discussed in the previous section, we can obtain the real-time OTOC\n$\\langle \\hat A(t)\\hat A^\\dagger(0), \\hat A(t)\\hat A^\\dagger(0)\\rangle$ from the following inverse Fourier transformation, \n\\begin{align}{\\rm Re}\\langle \\hat A(t)\\hat A^\\dagger(0), \\hat A(t)\\hat A^\\dagger(0)\\rangle\n&=\n{\\rm Re}\n\\int_0^\\infty d\\omega\\, e^{-i\\omega t}\\coth\\left(\\frac{\\beta\\hbar\\omega}{4}\\right)\n\\left(-\\frac{1}{\\pi}{\\rm Im}\\, C_{(AA^\\dagger)^2}^R(\\omega)\\right),\n\\\\\n{\\rm Im}\\langle \\hat A(t)\\hat A^\\dagger(0), \\hat A(t)\\hat A^\\dagger(0)\\rangle\n&=\n{\\rm Im}\\int_0^{\\infty} d\\omega\\, e^{-i\\omega t} \\left(-\\frac{1}{\\pi}{\\rm Im}\\, C_{(AA^\\dagger)^2}^R(\\omega)\\right),\n\\end{align}\nfor $\\hat A=c_\\sigma^\\dagger, \\hat n_\\sigma$, and $\\hat n$.\n\n\\begin{figure}[b]\n\\begin{center}\n\\includegraphics[angle=-90, width=0.48\\columnwidth]{compare_u0_draft.eps}\\hfill\n\\includegraphics[angle=-90, width=0.48\\columnwidth]{compare_u0_longtime_modulus.eps}\n\\caption{\nLeft panel: Comparison between the exact solution and the result obtained by analytical continuation\nfor the real and imaginary parts of the OTOC $\\langle c_\\sigma^\\dagger(t)c_\\sigma(0), c_\\sigma^\\dagger(t)c_\\sigma(0)\\rangle$ \nin the noninteracting case ($U=0, \\beta\\hbar=50$) of the single-orbital Hubbard model.\nRight panel: Comparison of the long-time behavior for the modulus $|\\langle c_\\sigma^\\dagger(t)c_\\sigma(0), c_\\sigma^\\dagger(t)c_\\sigma(0)\\rangle|$. \nThe exact result shows a power-law decay ($\\sim 1/t^3$). \n}\n\\label{fig:test_u0}\n\\end{center}\n\\end{figure}\n\nAs a check of our procedure, we first compare the results for the noninteracting model,\nfor which an exact solution of the OTOC $\\langle c_\\sigma^\\dagger(t)c_\\sigma(0), c_\\sigma^\\dagger(t)c_\\sigma(0)\\rangle$ \nis available \\cite{TsujiWernerUeda2017}.\nIt is a nontrivial task to measure the noninteracting two-particle Green's function in CT-HYB, \nand to analytically continue \n$C_{(c^\\dagger_\\sigma c_\\sigma)^2}^M(\\tau)$ \nby the Maximum Entropy method.  \nThe results for the inverse temperature $\\beta\\hbar=50$ are plotted in Fig.~\\ref{fig:test_u0}. \nOne can see that the OTOC oscillates for about one cycle, and quickly decays to zero.\nThe analytically continued data show a good agreement with the exact solution. \nIn particular, for short times (up to about two inverse hoppings) the dynamics is accurately reproduced, while deviations appear at longer times. \nIt is known that for the noninteracting fermion system the OTOC decays as a power law at long time ($\\sim 1/t^3$) \\cite{TsujiWernerUeda2017}.\nThe analytical continuation method cannot reproduce the details of the oscillations at intermediate or long times, but it roughly captures the long-time decay, which is controlled by low-frequency spectral features. \nUsually, analytic continuation is unreliable for high frequency components, because high-frequency information is suppressed \nby the kernel $K(\\tau,\\omega)$ in the transformation from the spectral function $A_O(\\omega)$ to the (bosonic) Matsubara correlation function $O(\\tau)$ [$O(\\tau)=\\int_{-\\infty}^{\\infty} d\\omega\\, K(\\omega,\\tau) A_O(\\omega)$ with $K(\\tau,\\omega)=e^{-\\tau\\omega}/(1-e^{-\\beta\\omega})$]. On the other hand, the low frequency components can be rather accurately determined. As a result, in Fig. 4 we more or less recover the low-frequency features of the time-dependent correlation function, while the details of rapid oscillations are not captured. The accurate result at short times can be understood from the fact that this region is close to the imaginary time axis in the complex time plane, where the original QMC data are available.\nAt higher temperatures, the Maximum Entropy method becomes less reliable, so that the resulting OTOCs are expected to be less accurate.  \n\n\n\\subsection{Single-orbital Hubbard model}\n\\label{sec:hubbard}\n\nIn this section, we calculate the OTOCs (\\ref{fermion_otoc})-(\\ref{otoc_nn}) for the interacting single-orbital Hubbard model with the Hamiltonian \n\\begin{align}\nH\n&=\n-v\\sum_{\\langle i,j\\rangle\\sigma} (c^\\dagger_{i\\sigma}c_{j\\sigma}+ \\text{h.c.})\n-\\mu\\sum_i \\hat n_i+U\\sum_i \\hat n_{i\\uparrow}\\hat n_{i\\downarrow}, \n\\end{align}\nwhere $v$ is the hopping amplitude, $\\langle i,j\\rangle$ represents the nearest-neighbor site pairs, \n$\\mu$ is the chemical potential, and $U$ is the on-site interaction.\nIn this section, we focus on the half-filled case (i.e., $\\mu=U/2$). The DMFT solution for the simplified self-consistency (\\ref{semicirc}) is the exact solution for an infinitely connected Bethe lattice \\cite{GeorgesKotliarKrauthRozenberg1996}.\nLet us briefly recall the paramagnetic phase diagram for the single-orbital Hubbard model on this lattice \\cite{Bluemer_thesis}. At half-filling, there is a metal-insulator crossover at high temperature and a first-order Mott transition below a temperature corresponding to $\\beta\\hbar\\approx 17$ with a coexistence region between $U_{c1}$ and $U_{c2}$. The finite-temperature critical endpoint is at $U\\approx 4.7$, while the zero-temperature Mott transition occurs at $U=U_{c2}\\approx 5.6$. \nAt low temperature, the self-energy shows the Fermi liquid behavior [${\\rm Im}\\,\\Sigma(\\omega)\\sim \\omega^2$] \nin the weakly correlated metallic phase, \nwhile it shows an insulating behavior [${\\rm Im}\\,\\Sigma(\\omega)\\sim \\delta(\\omega)$] in the Mott phase\n\\cite{GeorgesKotliarKrauthRozenberg1996}. \nIn the correlated metallic phase, as the temperature is increased, deviations from the Fermi liquid behavior become apparent, \nbut an ${\\rm Im}\\,\\Sigma(\\omega)\\sim \\sqrt{\\omega}$ scaling as in the SYK model is not observed. \n\nIn the following, we will start the DMFT iterations from the noninteracting solution, which means that the finite-temperature Mott transition occurs at $U_{c2}$.\nFrom now on, we set $\\hbar=1$. \n\n\\begin{figure}[t]\n\\begin{center}\n\\includegraphics[angle=-90, width=0.49\\columnwidth]{plot_otoc_b50_Re.eps}\n\\includegraphics[angle=-90, width=0.49\\columnwidth]{plot_otoc_b50_Im.eps}\n\\includegraphics[angle=-90, width=0.49\\columnwidth]{plot_coth_b50_appendix.eps}\n\\includegraphics[angle=-90, width=0.49\\columnwidth]{plot_otoc_b50_modulus.eps}\n\\caption{\nTop panels: Real and imaginary parts of the OTOC $\\langle c^\\dagger_\\sigma(t)c_\\sigma(0), c^\\dagger_\\sigma(t)c_\\sigma(0)\\rangle$ for the single-orbital Hubbard model with $\\beta=50$ at half filling. The thin lines correspond to metallic solutions, while the thick line ($U=6$) corresponds to a Mott insulating solution. \nBottom left panel: Spectral function $\\mathscr A_{(c_\\sigma^\\dagger c_\\sigma)^2}(\\omega)=-\\frac{1}{\\pi}{\\rm Im}\\,C_{(c^\\dagger_\\sigma c_\\sigma)^2}^R(\\omega)$ multiplied by $\\coth\\big(\\frac{\\beta\\omega}{4}\\big)$. \nBottom right panel: Modulus of the OTOC $|\\langle c^\\dagger_\\sigma(t)c_\\sigma(0), c^\\dagger_\\sigma(t)c_\\sigma(0)\\rangle|$ (normalized at $t=0$) on a log-log scale. The dashed line is the result for $U=0$.\n}\n\\label{fig:results_fermion}\n\\end{center}\n\\end{figure}\n\n\n\nIn Fig.~\\ref{fig:results_fermion}, we show the results of $\\langle c^\\dagger_\\sigma(t)c_\\sigma(0), c^\\dagger_\\sigma(t)c_\\sigma(0)\\rangle$ for the interacting single-orbital Hubbard model at $\\beta=50$. \nThe top panels present the real and imaginary parts of the OTOC. One can see that the oscillations become more pronounced as one increases the interaction. In the bottom left panel of Fig.~\\ref{fig:results_fermion},\nwe show the corresponding spectral function $\\mathscr A_{(c_\\sigma^\\dagger c_\\sigma)^2}(\\omega)=-\\frac{1}{\\pi}{\\rm Im}\\,C_{(c^\\dagger_\\sigma c_\\sigma)^2}^R(\\omega)$ multiplied by $\\coth\\big(\\frac{\\beta\\omega}{4}\\big)$.\nThe Mott transition, which occurs between $U=5$ and $U=6$ at $\\beta=50$, manifests itself by the opening of a gap in the spectral function $\\mathscr A_{(c_\\sigma^\\dagger c_\\sigma)^2(\\omega)}$. This translates into more weakly damped oscillations in the real-time evolution in the Mott phase.  \nThe bottom right panel of Fig.~\\ref{fig:results_fermion} plots the modulus of the OTOC (normalized at time $t=0$, i.e., $|\\langle c^\\dagger_\\sigma(t)c_\\sigma(0), c^\\dagger_\\sigma(t)c_\\sigma(0)\\rangle|/|\\langle c^\\dagger_\\sigma(0)c_\\sigma(0), c^\\dagger_\\sigma(0)c_\\sigma(0)\\rangle|$) on a log-log scale. \nDue to the oscillations and the limited accuracy of the analytic continuation procedure, it is hard to clearly identify the nature of the long-time decay of the OTOC,\nbut the results indicate that the OTOC decays much faster (possibly exponentially) in the Mott phase than in the metallic phase. This is consistent with the results for the spectral functions of the OTOC. The comparison to the noninteracting case (dashed curve in the bottom right panel of Fig.~\\ref{fig:results_fermion}) suggests that the correlated metallic phase has a similar (possibly power-law) decay behavior of the OTOC as in the noninteracting case.\n\n\n\\begin{figure}[t]\n\\begin{center}\n\\includegraphics[angle=-90, width=0.49\\columnwidth]{plot_otoc_b50_nn_Re.eps}\n\\includegraphics[angle=-90, width=0.49\\columnwidth]{plot_otoc_b50_nn_Im.eps}\n\\includegraphics[angle=-90, width=0.49\\columnwidth]{plot_coth_b50_nn_appendix.eps}\n\\includegraphics[angle=-90, width=0.49\\columnwidth]{plot_otoc_b50_nn_modulus.eps}\n\\caption{\nTop panels: Real and imaginary parts of the OTOC $\\langle \\hat n(t)\\hat n(0), \\hat n(t)\\hat n(0)\\rangle - 1$ for the single-orbital Hubbard model with $\\beta=50$ at half filling. The thin lines correspond to metallic solutions, \nwhile the thick line ($U=6$) corresponds to a Mott insulating solution. \nBottom left panel: Spectral function $\\mathscr A_{(nn)^2}(\\omega)=-\\frac{1}{\\pi}{\\rm Im}\\,C_{(n n)^2}^R(\\omega)$ multiplied by $\\coth\\big(\\frac{\\beta\\omega}{4}\\big)$.\nBottom right panel: Modulus of the OTOC $|\\langle \\hat n(t)\\hat n(0), \\hat n(t)\\hat n(0)\\rangle - 1|$ (normalized at $t=0$) on a log-log scale. \n}\n\\label{fig:results_nn}\n\\end{center}\n\\end{figure}\n\n\nIn Fig.~\\ref{fig:results_nn}, we plot the OTOC $\\langle \\hat n(t)\\hat n(0), \\hat n(t)\\hat n(0)\\rangle$\nfor the single-orbital Hubbard model in a similar manner as in Fig.~\\ref{fig:results_fermion}.\nSince at half filling this correlation function approaches $\\langle \\hat n\\rangle^4=1$ at long times\nand at sufficiently low temperature,\nwe perform the analytical continuation procedure for $\\langle \\hat n(t)\\hat n(0), \\hat n(t)\\hat n(0)\\rangle-1$. \nThe top panels of Fig.~\\ref{fig:results_nn} show the real and imaginary parts of this shifted OTOC. \nContrary to the case of $\\langle c_\\sigma^\\dagger(t)c_\\sigma(0), c_\\sigma^\\dagger(t)c_\\sigma(0)\\rangle$, \nthe amplitude of the OTOC $\\langle \\hat n(t)\\hat n(0), \\hat n(t)\\hat n(0)\\rangle-1$ is suppressed as one increases the interaction, which reflects the reduced charge fluctuations in the Mott insulator.\nThe bottom left panel in Fig.~\\ref{fig:results_nn} shows the analytically continued spectral function $\\mathscr A_{(nn)^2}(\\omega)=-\\frac{1}{\\pi}{\\rm Im}\\, C_{(nn)^2}^R(\\omega)$ multiplied by $\\coth\\big(\\frac{\\beta\\omega}{4}\\big)$. \nAgain we notice the opening of a gap in the Mott phase, and a corresponding shift of the spectral weight to higher energies. This results in more rapid oscillations of the OTOC in the Mott state.\nThe bottom right panel in Fig.~\\ref{fig:results_nn} plots the modulus of the OTOC $|\\langle \\hat n(t)\\hat n(0), \\hat n(t)\\hat n(0)\\rangle-1|$ (normalized at $t=0$) on a log-log scale.\nThe long-time behavior of the OTOC is consistent with a power-law decay in the metallic phase, and an exponential decay in the Mott insulating phase.\n\n\n\\begin{figure}[t]\n\\begin{center}\n\\includegraphics[angle=-90, width=0.49\\columnwidth]{plot_otoc_nsns_b50_draft_Re.eps}\n\\includegraphics[angle=-90, width=0.49\\columnwidth]{plot_otoc_nsns_b50_draft_Im.eps}\n\\includegraphics[angle=-90, width=0.49\\columnwidth]{plot_coth_b50_nsns_inset.eps}\n\\includegraphics[angle=-90, width=0.49\\columnwidth]{plot_otoc_nsns_b50_draft_modulus_zoom.eps}\n\\caption{\nTop panels: Real and imaginary parts of the OTOC $\\langle \\hat n_\\sigma(t)\\hat n_\\sigma(0), \\hat n_\\sigma(t)\\hat n_\\sigma(0)\\rangle  - \\langle \\hat n_\\sigma\\rangle^4$ for the single-orbital Hubbard model with $\\beta=50$ at half filling. All the colored lines correspond to metallic solutions. \nBottom left panel: Spectral function $\\mathscr A_{(n_\\sigma n_\\sigma)^2}(\\omega)=-\\frac{1}{\\pi}{\\rm Im}\\,C_{(n_\\sigma n_\\sigma)^2}^R(\\omega)$ multiplied by $\\coth\\big(\\frac{\\beta\\omega}{4}\\big)$. In the inset, we plot the imaginary-time four-point function $\\langle \\hat n_\\sigma(\\tau)\\hat n_\\sigma(0), \\hat n_\\sigma(\\tau)\\hat n_\\sigma(0)\\rangle-\\langle \\hat n_\\sigma\\rangle^4$, with the black line corresponding to the Mott insulating solution at $U=6$.  \nBottom right panel: Modulus of the OTOC $|\\langle \\hat n_\\sigma(t)\\hat n_\\sigma(0), \\hat n_\\sigma(t)\\hat n_\\sigma(0)\\rangle  - \\langle \\hat n_\\sigma\\rangle^4|$ (normalized at $t=0$) on a log-log scale.\n}\n\\label{fig:results_nsns}\n\\end{center}\n\\end{figure}\n\nFigure~\\ref{fig:results_nsns} \nshows the results of the OTOC $\\langle \\hat n_\\sigma(t)\\hat n_\\sigma(0), \\hat n_\\sigma(t)\\hat n_\\sigma(0)\\rangle$\nfor the single-orbital Hubbard model. At half filling, this correlation function approaches $\\langle \\hat n_\\sigma\\rangle^4=\\frac{1}{16}$ in the long-time limit and at sufficiently low temperature, \nso that we perform the analytical continuation procedure for $\\langle \\hat n_\\sigma(t)\\hat n_\\sigma(0), \\hat n_\\sigma(t)\\hat n_\\sigma(0)\\rangle-\\langle \\hat n_\\sigma\\rangle^4$. \nThe top panels of Fig.~\\ref{fig:results_nsns} show the real and imaginary parts of \nthis shifted OTOC. \nHere, we only present results for the metallic phase, because resolving the very sharp low-energy feature of the OTOC spectrum in the half-filled Mott state is challenging. One can see that coherent oscillations are not observed for this type of OTOC, and that the incoherent part becomes dominant compared to the other two OTOCs shown in Figs.~\\ref{fig:results_fermion} and \\ref{fig:results_nn}. The amplitude of the OTOC $\\langle \\hat n_\\sigma(t)\\hat n_\\sigma(0), \\hat n_\\sigma(t)\\hat n_\\sigma(0)\\rangle-\\langle \\hat n_\\sigma\\rangle^4$ is enhanced as $U$ increases. As we argue below, this is related to the enhancement of spin fluctuations \nand local moment formation close to the Mott transition.\n\nThe bottom left panel in Fig.~\\ref{fig:results_nsns} plots the analytically continued spectral function $\\mathscr A_{(n_\\sigma n_\\sigma)^2}(\\omega)=-\\frac{1}{\\pi}{\\rm Im}\\, C_{(n_\\sigma n_\\sigma)^2}^R(\\omega)$ multiplied by $\\coth\\big(\\frac{\\beta\\omega}{4}\\big)$. \nWe can see that the spectral weight is concentrated in the low-energy region as we approach the Mott transition point.\nThe inset in the bottom left panel of Fig.~\\ref{fig:results_nsns} shows the imaginary-time four-point function $\\langle \\hat n_\\sigma(\\tau)\\hat n_\\sigma(0), \\hat n_\\sigma(\\tau)\\hat n_\\sigma(0)\\rangle-\\langle \\hat n_\\sigma\\rangle^4$, with the black line corresponding to a Mott insulating solution ($U=6$).  \nFor $U\\gtrsim 4$, the imaginary-time four-point function does not decay to zero but remains relatively large near $\\tau=\\frac{\\beta}{4}$. \nThis behavior is reminiscent of the spin-freezing physics seen in the correlated Hund metal phase of multi-orbital Hubbard models, where the dynamical spin correlation function $\\langle \\hat S_z(\\tau)\\hat S_z(0)\\rangle$ is trapped at a finite value at long $\\tau$ \\cite{Werner2008}. \nIn fact, the OTOC $\\langle \\hat n_\\sigma(\\tau)\\hat n_\\sigma(0), \\hat n_\\sigma(\\tau)\\hat n_\\sigma(0)\\rangle$\nmeasures spin correlations (in addition to charge correlations) through $\\hat n_\\sigma=\\frac{1}{2}\\hat n+\\sigma\\hat S_z$.\nPhysically, the trapping of $\\langle \\hat S_z(\\tau)\\hat S_z(0)\\rangle$ signals the formation of frozen local magnetic moments.\nIn the metal-insulator crossover region, the scattering induced by these magnetic moments results in incoherent metal states. Intuitively, one might expect fast scrambling in the corresponding regions of the phase diagram. \nAlthough there is some resemblance with the spin freezing crossover,\nthe self-energy does not exhibit a square-root frequency dependence in the single-orbital case, and a spin-freezing crossover in the sense of Ref.~\\cite{Werner2008} does not exist.  \n\nThe bottom right panel of Fig.~\\ref{fig:results_nsns} plots the modulus of the OTOC $|\\langle \\hat n_\\sigma(t)\\hat n_\\sigma(0), \\hat n_\\sigma(t)\\hat n_\\sigma(0)\\rangle-\\langle \\hat n_\\sigma\\rangle^4|$ (normalized at $t=0$) on a log-log scale. \nThe dynamics of $\\langle \\hat n_\\sigma(t)\\hat n_\\sigma(0), \\hat n_\\sigma(t)\\hat n_\\sigma(0)\\rangle$ is quite distinct from that of $\\langle \\hat n(t)\\hat n(0), \\hat n(t)\\hat n(0)\\rangle$. As the Mott transition is approached, we observe a slow-down in the decay of the modulus, \nwhich indicates the presence of slowly fluctuating local moments. \nThe long-time behavior may be consistent with a power-law, although slow oscillations make it difficult to determine the long-time asymptotic form from the numerics. \n\n\n\\begin{figure}[t]\n\\begin{center}\n\\includegraphics[angle=-90, width=0.49\\columnwidth]{plot_fitnsns_b2.eps}\n\\includegraphics[angle=-90, width=0.49\\columnwidth]{plot_coeffc_draft.eps}\n\\caption{\nLeft panel: \nThe imaginary-time four-point function $\\langle \\hat n_\\sigma(\\tau)\\hat n_\\sigma(0), \\hat n_\\sigma(\\tau)\\hat n_\\sigma(0)\\rangle$, the squared density-density correlation function $\\langle \\hat n_\\sigma(\\tau)\\hat n_\\sigma(0)\\rangle^2$,\nand their difference for the single-orbital Hubbard model with $U=5.5$ and $\\beta=50$.\nThe dashed curve shows a fit of the difference with the function $f_\\text{fit}(\\tau)=a+b\\tau+c\\tau^2$ in the interval $\\tau/\\beta \\in [0: 0.2]$.\nRight panel: Fitting coefficient $c$ as a function of $U$ for indicated values of $\\beta$.\n\n}\n\\label{fig:coeffs}\n\\end{center}\n\\end{figure}\n\nFor large enough $\\beta$ and small enough $\\tau$, the OTOC (\\ref{otoc_nsns}) factorizes into $\\langle \\hat n_\\sigma(\\tau)\\hat n_\\sigma(0), \\hat n_\\sigma(\\tau)\\hat n_\\sigma(0)\\rangle \\approx \\langle \\hat n_\\sigma(\\tau)\\hat n_\\sigma(0)\\rangle^2$. \nOn the real-time axis, this corresponds to the decoupling $\\langle \\hat n_\\sigma(t)\\hat n_\\sigma(0), \\hat n_\\sigma(t)\\hat n_\\sigma(0)\\rangle \\approx \\langle \\hat n_\\sigma(t)\\hat n_\\sigma(0)\\rangle^2$, that is, \nthe OTOC is reduced to a product of ordinary density-density correlation functions.\nIn order to see whether nontrivial correlations are captured beyond the decoupled form by the OTOC function, \nwe consider the difference $\\langle \\hat n_\\sigma(\\tau)\\hat n_\\sigma(0), \\hat n_\\sigma(\\tau)\\hat n_\\sigma(0)\\rangle-\\langle \\hat n_\\sigma(\\tau)\\hat n_\\sigma(0)\\rangle^2$ and fit the short-time behavior of this function to $f_\\text{fit}(\\tau)=a+b\\tau+c\\tau^2$, as illustrated in the left panel of Fig.~\\ref{fig:coeffs}. The coefficients $b$ and $c$ serve as  indicators of the nontrivial correlations that cannot be attributed to the decoupled form of the OTOC.\nThe coefficient $c$ is plotted as a function of $U$ and for different $\\beta$ in the right hand panel. The coefficient $b$ (not shown) exhibits a similar trend. We notice that the OTOC picks up nontrival correlations near the metal-insulator transition, and in particular at \nintermediate\ntemperatures, while the Mott phase shows no such correlations. The crossover region  \nassociated with the emergence of local moments \ncorresponds, roughly, to the interaction range where the nontrivial correlations start to become significant. \n\n\n\\subsection{Two-orbital Hubbard model}\n\\label{sec:two-orbital hubbard}\n\nIn this section, we investigate OTOCs in the two-orbital Hubbard model with Hund coupling $J>0$ \nand density-density interactions. (Three-orbital results are presented in Appendix~\\ref{sec:3orbital}.) \nThe Hamiltonian is given by\n\\begin{align}\nH\n&=\n-v\\sum_{\\langle i,j\\rangle\\alpha\\sigma} (c^\\dagger_{i\\alpha\\sigma}c_{j\\alpha\\sigma}+ \\text{h.c.})\n-\\mu\\sum_{i\\alpha\\sigma} \\hat n_{i\\alpha\\sigma}\n+U\\sum_{i\\alpha} \\hat n_{i\\alpha\\uparrow}\\hat n_{i\\alpha\\downarrow}\n+(U-2J)\\sum_{i\\sigma} \\hat n_{i1\\sigma}\\hat n_{i2\\bar\\sigma}\n\\notag\n\\\\\n&\\quad\n+(U-3J)\\sum_{i\\sigma} \\hat n_{i1\\sigma}\\hat n_{i2\\sigma},\n\\end{align}\n\nwith $U$ being the intra-orbital interaction $U$, $U-2J$ the inter-orbital antiparallel-spin interaction, and $U-3J$ the inter-orbital parallel-spin interaction. This is the simplest model that shows a crossover from a spin-frozen metal to a Fermi-liquid metal as one dopes the half-filled Mott insulator, with the self-energy scaling as $\\text{Im}\\Sigma(\\omega_n)\\sim\\sqrt{\\omega_n}$ over a significant energy range in the crossover regime \\cite{Hafermann2012}.\nThe sketch of the phase diagram is shown in Fig.~\\ref{fig:illustration}. \nAs we have seen in the previous section, the spin-related OTOC of the type (\\ref{otoc_nsns}) is relevant for the analysis of  the spin-freezing crossover, so that we concentrate on this OTOC here.\nIn the following calculations, we choose $U=8$, $J=U/4$, and compute $\\langle \\hat n_\\sigma(t)\\hat n_\\sigma(0), \\hat n_\\sigma(t)\\hat n_\\sigma(0)\\rangle - \\langle n_\\sigma \\rangle^4$ \nas a function of filling at $\\beta=50$ (dashed lines in Fig.~\\ref{fig:illustration}). \n\n\n\\begin{figure}[t]\n\\begin{center}\n\\includegraphics[angle=-90, width=0.49\\columnwidth]{plot_qp_draft.eps}\\hfill\n\\includegraphics[angle=-90, width=0.49\\columnwidth]{plot_fit_2orbital.eps}\n\\caption{\nLeft panel: Imaginary part of the self-energy as a function of Matsubara frequency for the two-orbital Hubbard model with $U=8$, $J=U/4$, and $\\beta=50$ on a log-log scale, with the dashed line indicating $\\sqrt{\\omega_n}$ behavior. \nRight panel: The coefficient $c$ extracted from\nfitting $\\langle \\hat n_\\sigma(\\tau)\\hat n_\\sigma(0), \\hat n_\\sigma(\\tau)\\hat n_\\sigma(0)\\rangle-\\langle \\hat n_\\sigma(\\tau)\\hat n_\\sigma(0)\\rangle^2$ with a function $f_\\text{fit}(\\tau)=a+b\\tau+c\\tau^2$\nin the interval $\\tau/\\beta\\in [0: 0.05]$ for the two-orbital Hubbard model. \nThe spin-freezing crossover occurs roughly at the filling $n_\\sigma\\approx 0.28$ (blue curve in the left panel and blue arrow in the right panel). \n}\n\\label{fig:self}\n\\end{center}\n\\end{figure}\n\n\nThe left panel of Fig.~\\ref{fig:self} plots the imaginary part of the self-energy as a function of Matsubara frequency in order to identify the filling corresponding to the spin-freezing crossover \\cite{Werner2008}. An approximate square-root scaling is observed over a wide range of frequencies near $n_\\sigma\\approx 0.28$ (blue line). In the right panel of Fig.~\\ref{fig:self}, we present the coefficient $c$ obtained from a similar analysis as presented in Fig.~\\ref{fig:coeffs}, but with a fitting range $\\tau/\\beta\\in [0 : 0.05]$. Again, we find that the OTOC $\\langle \\hat n_\\sigma(t)\\hat n_\\sigma(0), \\hat n_\\sigma(t)\\hat n_\\sigma(0)\\rangle - \\langle \\hat n_\\sigma\\rangle^4$ detects nontrivial correlations in the spin-freezing crossover (blue arrow) and spin-frozen metal regime, but not in the Fermi liquid or the Mott insulating phase.\n\n\n\\begin{figure}[t]\n\\begin{center}\n\\includegraphics[angle=-90, width=0.49\\columnwidth]{plot_otoc_2orbital_Re.eps}\n\\includegraphics[angle=-90, width=0.486\\columnwidth]{plot_otoc_2orbital_Im.eps}\n\\includegraphics[angle=-90, width=0.484\\columnwidth]{plot_spectra_coth_2orbital_inset.eps}\n\\includegraphics[angle=-90, width=0.486\\columnwidth]{plot_otoc_2orbital_modulus_inset.eps}\n\\caption{\nTop panels: Real and imaginary parts of the OTOC $\\langle \\hat n_\\sigma(t)\\hat n_\\sigma(0), \\hat n_\\sigma(t)\\hat n_\\sigma(0)\\rangle  - \\langle \\hat n_\\sigma\\rangle^4$ for the two-orbital Hubbard model with $U=8$, $J=U/4$, $\\beta=50$ and indicated fillings. \nBottom left panel: Spectral function $\\mathscr A_{(n_\\sigma n_\\sigma)^2}(\\omega)=-\\frac{1}{\\pi}{\\rm Im}\\,C_{(n_\\sigma n_\\sigma)^2}^R(\\omega)$ multiplied by $\\coth\\big(\\frac{\\beta\\omega}{4}\\big)$. In the inset, we plot the imaginary-time four-point function $\\langle \\hat n_\\sigma(\\tau)\\hat n_\\sigma(0), \\hat n_\\sigma(\\tau)\\hat n_\\sigma(0)\\rangle-\\langle \\hat n_\\sigma\\rangle^4$. \nBottom right panel: Modulus of the OTOC $|\\langle \\hat n_\\sigma(t)\\hat n_\\sigma(0), \\hat n_\\sigma(t)\\hat n_\\sigma(0)\\rangle  - \\langle \\hat n_\\sigma\\rangle^4|$ (normalized at $t=0$) on a log-log scale.\nIn the inset of the bottom right panel, we illustrate the approximate data collapse obtained by considering the spin-freezing crossover regimes for different values of $J$. \nIn all the panels, the thick blue curves represent the results for the spin-freezing crossover region. \n}\n\\label{fig:2orbital}\n\\end{center}\n\\end{figure}\n\n\nThe top panels of Fig.~\\ref{fig:2orbital} show the real and imaginary parts of the OTOC\n$\\langle \\hat n_\\sigma(t)\\hat n_\\sigma(0), \\hat n_\\sigma(t)\\hat n_\\sigma(0)\\rangle-\\langle \\hat n_\\sigma\\rangle^4$\nfor the two-orbital Hubbard model. In the spin-frozen phase ($n_\\sigma \\gtrsim 0.3$), both components exhibit \nhighly incoherent oscillations, which get suppressed as the filling is reduced. As one moves across the crossover point ($n_\\sigma \\sim 0.28$),\nthe oscillations completely disappear, and the real part of the OTOC starts to overshoot to the negative side\nin the Fermi liquid regime.\nThe bottom left panel in Fig.~\\ref{fig:2orbital} presents the analytically continued spectral function $\\mathscr A_{(n_\\sigma n_\\sigma)^2}(\\omega)=-\\frac{1}{\\pi}{\\rm Im}\\,C_{(n_\\sigma n_\\sigma)^2}^R(\\omega)$ multiplied by $\\coth\\big(\\frac{\\beta\\omega}{4}\\big)$. \nIn the spin-frozen regime ($n_\\sigma \\gtrsim 0.3$) a sharp peak appears at low energy, which originates from the saturation of the imaginary-time four point function $\\langle \\hat n_\\sigma(\\tau)\\hat n_\\sigma(0), \\hat n_\\sigma(\\tau)\\hat n_\\sigma(0)\\rangle$ at large $\\tau$ (see the inset of the bottom left panel of Fig.~\\ref{fig:2orbital}). This low-energy peak represents the slow dynamics of the frozen spins, and translates into a slow long-time decay of $|\\langle \\hat n_\\sigma(t)\\hat n_\\sigma(0), \\hat n_\\sigma(t)\\hat n_\\sigma(0)\\rangle-\\langle \\hat n_\\sigma\\rangle^4|$,\nas shown in the bottom right panel of Fig.~\\ref{fig:2orbital}\non a log-log scale. Again, it is difficult to clearly resolve the long-time behavior of the OTOC due to the limited\naccuracy of the analytic continuation, but in the spin-freezing crossover regime (thick blue curves)\nthe decay of the OTOC is consistent with a power law $\\sim 1/t^{1.5}$ at least for $2\\lesssim t \\lesssim 5$. \nThe shorter-time behavior ($0.5\\lesssim t \\lesssim 2$) is instead well fitted by an exponential decay $\\sim e^{-\\alpha t}$ with a decay constant $\\alpha=0.78$. \nThis exponent exhibits\na strong dependence on the Hund coupling. \n\nIn fact, the OTOCs in the spin-freezing regime for $J=U/4$ and $J=U/6$ can be approximately collapsed by plotting them as a function of $tJ$ (see the inset in the bottom right panel of Fig.~\\ref{fig:2orbital}), which indicates that the Hund coupling is the parameter which controls the dynamics in this regime. \nThis is distinct from the Lyapunov behavior in which the exponential decay scales with $t/\\beta$ \\cite{Bagrets2017}.\nA possible reason is that we are still far from the large $N$ limit (where $N$ corresponds to the number of orbitals times the number of spin degrees of freedom) so that the large $N$ behavior \nsuch as the Lyapunov growth is not observed in our calculations. \nIn fact, the strong dependence of the time scale on $J$ and weak dependence on $\\beta$ is consistent\nwith the behavior of the finite $N$ SYK model in Ref.~\\cite{FuSachdev2016}.\nIn the spin-frozen regime ($n_\\sigma\\gtrsim 0.3$) the decay is much slower than near the crossover point, while in the Fermi liquid regime the decay is accelerated, approaching the free fermion behavior of $t^{-3}$\nas shown in Fig.~\\ref{analytic continuation}. \nQualitatively similar results for this OTOC are obtained \nfor the three-orbital Hubbard model (see Appendix~\\ref{sec:3orbital}).\n\nThe time dependence of the modulus of \n$\\langle \\hat n_\\sigma(\\tau)\\hat n_\\sigma(0), \\hat n_\\sigma(\\tau)\\hat n_\\sigma(0)\\rangle-\\langle \\hat n_\\sigma\\rangle^4$ \nin the spin-freezing crossover regime of multi-orbital Hubbard models (exponential decay crossing over into a power-law) bears a close resemblance to a (different) OTOC for the SYK model discussed in Ref.~\\cite{Bagrets2017}. \nAlso, a recent study of yet another type of OTOCs for the finite-$N$ SYK model found a qualitatively similar decay as observed here in the spin-freezing crossover regime \\cite{FuSachdev2016}. In the following section, we will make the connection to the SYK dynamics more quantitative.   \n\n\n\\subsection{Comparison to the SKY model}\n\\label{sec:SYK}\n\nIn this section, we compare the results of the multi-orbital Hubbard models to the SYK model.\nWe consider the particle-hole symmetric SYK model of complex fermions \\cite{FuSachdev2016},\nwhose Hamiltonian is given by\n\\begin{align}\nH\n&=\n\\frac{1}{(2N)^{3/2}} \\sum_{i,j,k,l=1}^N \nJ_{ij;kl} (c_i^\\dagger c_j^\\dagger c_k c_l\n+\\delta_{ik}n c_j^\\dagger c_l\n-\\delta_{il}n c_j^\\dagger c_k\n-\\delta_{jk}n c_i^\\dagger c_l\n+\\delta_{jl}n c_i^\\dagger c_k).\n\\end{align}\nThe coupling constant $J_{ij;kl}$ is a gaussian random variable, satisfying $J_{ij;kl}=-J_{ji;kl}=-J_{ij;lk}=J_{kl;ij}^\\ast$, and\n\\begin{align}\n\\overline{({\\rm Re}\\, J_{ij;kl})^2}\n&=\n\\begin{cases}\nJ_{\\rm SYK}^2/2 & (i,j)\\neq (k,l) \\\\\nJ_{\\rm SYK}^2 & (i,j)=(k,l)\n\\end{cases},\n\\\\\n\\overline{({\\rm Im}\\, J_{ij;kl})^2}\n&=\n\\begin{cases}\nJ_{\\rm SYK}^2/2 & (i,j)\\neq (k,l) \\\\\n0 & (i,j)=(k,l)\n\\end{cases},\n\\end{align}\nwhere the overline represents an average over each realization of $J_{ij;kl}$. The parameter \n$J_{\\rm SYK}$ ($J_{\\rm SYK}^{-1}$) defines the unit of energy (time) in the following calculations. \nThe system is particle-hole symmetric when\n$n=0.5$, which fixes the total number of fermions as $N^{-1}\\sum_{i=1}^N \\langle c_i^\\dagger c_i\\rangle=0.5$.\nWe take the particle-hole symmetric form of the SYK model because it has been well studied previously \\cite{FuSachdev2016} and avoids the effect of the filling drift.\nWe numerically solve the model by exact diagonalization for finite $N(\\le 12)$, and evaluate the density-density OTOC\n$\\overline{\\langle \\hat n_i(t)\\hat n_i(0), \\hat n_i(t)\\hat n_i(0)\\rangle-\\langle \\hat n_i\\rangle^4}$\nand its imaginary-time counterpart.\n\n\\begin{figure}[t]\n\\begin{center}\n\n\\includegraphics[angle=-90,width=0.49\\columnwidth]{plot_fitnsns_syk.eps}\n\n\\includegraphics[angle=-90,width=0.49\\columnwidth]{plot_coeffc_syk.eps}\n\\caption{\nLeft panel: \nThe imaginary-time four-point function $\\overline{\\langle \\hat n_i(\\tau)\\hat n_i(0), \\hat n_i(\\tau)\\hat n_i(0)\\rangle}$, the density-density correlation function $\\overline{\\langle \\hat n_i(\\tau)\\hat n_i(0)\\rangle^2}$,\nand their difference for the SYK model with $N=12$ and $\\beta J_{\\rm SYK}=10$.\nThe dashed curve shows a fit of the difference with a function $f_\\text{fit}(\\tau)=a+b\\tau+c\\tau^2$ in the interval $\\tau/\\beta \\in [0: 0.2]$. \nRight panel: Fitting coefficient $c$ as a function of $T/J_{\\rm SYK}$.\n}\n\\label{fig:SYK fit}\n\\end{center}\n\\end{figure}\n\nIn the left panel of Fig.~\\ref{fig:SYK fit}, we plot the imaginary-time four-point function \n$\\overline{\\langle \\hat n_i(\\tau)\\hat n_i(0), \\hat n_i(\\tau)\\hat n_i(0)\\rangle}$\nfor the SYK model with $N=12$ and $\\beta J_{\\rm SYK}=10$. \nAt low temperature, we expect that $\\overline{\\langle \\hat n_i(\\tau)\\hat n_i(0), \\hat n_i(\\tau)\\hat n_i(0)\\rangle}$ is decoupled into $\\overline{\\langle \\hat n_i(\\tau)\\hat n_i(0)\\rangle^2}$.\nTo see whether nontrivial correlations beyond the decoupled form are present, we fit the difference\n$\\overline{\\langle \\hat n_i(\\tau)\\hat n_i(0), \\hat n_i(\\tau)\\hat n_i(0)\\rangle}\n-\\overline{\\langle \\hat n_i(\\tau)\\hat n_i(0)\\rangle^2}$ with a function $f_{\\rm fit}(\\tau)=a+b\\tau+c\\tau^2$\nin the interval $\\tau/\\beta\\in[0:0.2]$, in the same manner as for the Hubbard models.\nThe obtained coefficient $c$ is plotted as a function of the temperature $T$ in the right panel of\nFig.~\\ref{fig:SYK fit}. One can see that nontrivial correlations exist in a wide range of temperature.\nEspecially, they are enhanced in the intermediate temperature regime. This is because in the zero-temperature limit\nthe four-point function is decoupled as \n$\\langle \\hat n_i(\\tau)\\hat n_i(0), \\hat n_i(\\tau)\\hat n_i(0)\\rangle \\approx \\langle \\hat n_i(\\tau)\\hat n_i(0)\\rangle^2$, while in the high-temperature limit ($\\beta\\to 0$) the imaginary-time dependence is washed out.\n\n\n\n\n\\begin{figure}[t]\n\\begin{center}\n\\includegraphics[angle=-90, width=0.49\\columnwidth]{plot_syk_Re.eps}\n\\includegraphics[angle=-90, width=0.49\\columnwidth]{plot_syk_Im.eps}\n\\includegraphics[angle=-90, width=0.49\\columnwidth]{plot_syk_modulus-log.eps}\n\\includegraphics[angle=-90, width=0.49\\columnwidth]{plot_syk_modulus.eps}\n\\caption{\nThe density-density OTOC $\\overline{\\langle \\hat n_i(t)\\hat n_i(0), \\hat n_i(t)\\hat n_i(0)\\rangle-\\langle \\hat n_i\\rangle^4}$ for the particle-hole symmetric SYK model averaged over $10^2$ samples with $\\beta J_{\\rm SYK}=100$ and various $N$. \nThe top left and right panels show the real and imaginary parts of the OTOC,\nwhile the bottom left and right panels show the log and log-log plots of the modulus of the OTOC (normalized at $t=0$).\nThe dashed line shows a power-law decay $\\propto t^{-1.6}$ for comparison.}\n\\label{fig:SYK}\n\\end{center}\n\\end{figure}\n\nIn Fig.~\\ref{fig:SYK}, we show the numerical results of the density-density OTOC for $\\beta J_{\\rm SYK}=100$.\nThe real part of the OTOC rapidly drops from the initial value within the time scale of $tJ_{\\rm SYK}\\sim 2$,\nand slowly decays to zero in the long time limit.\nAs one increases $N$, the oscillations that appear at longer time tend to be suppressed.\nOne can see that the result for $N=12$ in the top panels of Fig.~\\ref{fig:SYK} closely resembles the OTOC in \nthe spin-freezing crossover regime of the two-orbital Hubbard model shown\nby the blue curve in the top panels of Fig.~\\ref{fig:2orbital},\nwhile it differs from the single-orbital result shown in the top panels of Fig.~\\ref{fig:results_nsns}.\nDue to the limitation of our calculations to small system sizes ($N\\le 12$), we do not clearly observe\nan exponential growth ($\\sim c_0-c_1 e^{\\lambda t}$)  at short times or an exponential decay ($\\sim e^{-\\alpha t}$) \nat intermediate times. \nHowever, we find that for $tJ_{\\rm SYK}\\gtrsim 2$ the OTOC decays\napproximately in a power law $\\propto t^{-\\gamma}$ with $\\gamma=1.6$ (see bottom right panel in Fig.~\\ref{fig:SYK}). \nThe time interval exhibiting the power-law decay becomes longer as we increase $N$.  \nThe temperature dependence of the OTOC for the SYK model is shown in Fig.~\\ref{fig:SYK-T}.\nThe time scale of the initial drop is more or less independent of the temperature, while \nthe power-law-like decay is only visible in the low-temperature regime.\n\n\\begin{figure}[t]\n\\begin{center}\n\\includegraphics[angle=-90, width=0.49\\columnwidth]{plot_syk_beta_re.eps}\n\\includegraphics[angle=-90, width=0.49\\columnwidth]{plot_syk_beta_im.eps}\n\\includegraphics[angle=-90, width=0.49\\columnwidth]{plot_syk_beta_modulus-log.eps}\n\\includegraphics[angle=-90, width=0.49\\columnwidth]{plot_syk_beta_modulus.eps}\n\\caption{\nTemperature dependence of the density-density OTOC\n$\\overline{\\langle \\hat n_i(t)\\hat n_i(0), \\hat n_i(t)\\hat n_i(0)\\rangle-\\langle \\hat n_i\\rangle^4}$ for the particle-hole symmetric SYK model averaged over $10^2$ samples with $N=12$.\nThe top left and right panels show the real and imaginary parts of the OTOC,\nwhile the bottom left and right panels show the log and log-log plots of the modulus of the OTOC.\nThe dashed line shows a power-law decay $\\propto t^{-1.6}$ for comparison.}\n\\label{fig:SYK-T}\n\\end{center}\n\\end{figure}\n\nIn Ref.~\\onlinecite{Werner2018}, it has been argued that the spin-freezing crossover regime of multi-orbital Hubbard models is effectively described by the SYK model in which $J_{\\rm SYK}$ is replaced by the Hund coupling $J$. The results in Fig.~\\ref{fig:2orbital} are for the two-orbital Hubbard model with $U=8$, $J/U=1/4$ and $\\beta=50$, corresponding to $\\beta J=100$, which means that it is meaningful to directly compare the blue lines in Fig.~\\ref{fig:2orbital} with the blue lines in Figs.~\\ref{fig:SYK} and \\ref{fig:SYK-T}. Not only is the qualitative agreement remarkable, even the power-law exponent measured for the two-orbital Hubbard model ($\\gamma=1.5$) is very close to the exponent extracted from the finite-$N$ SYK calculation. \nThe power-law decay in the three-orbital case is approximately $1/t^{1.75}$ (see Appendix~\\ref{sec:3orbital}), and thus also in semi-quantitative agreement with the SYK model behavior. \nThis nontrivial result provides further support for the identification of the spin-freezing crossover regime of multi-orbital Hubbard systems with an SYK strange metal. \n\nIn the large-$N$ and low-$T$ limit of the SYK model, we expect that the power-law behavior approaches $1/t^2$, since in this limit the OTOC is decoupled as $\\langle \\hat n_i(t)\\hat n_i(0), \\hat n_i(t)\\hat n_i(0)\\rangle\\approx \\langle \\hat n_i(t)\\hat n_i(0)\\rangle^2$ and the decay of $\\langle \\hat n_i(t)\\hat n_i(0)\\rangle$ is dominated by what corresponds to the slow spin relaxation $\\langle \\hat S_z(t)\\hat S_z(0)\\rangle\\sim 1/t$ in the Sachdev-Ye model.\n\n\n\n\n\n4.2 Single-orbital Hubbard model\n\\subsection{Single-orbital Hubbard model}\n\\label{sec:hubbard}\n\nIn this section, we calculate the OTOCs (\\ref{fermion_otoc})-(\\ref{otoc_nn}) for the interacting single-orbital Hubbard model with the Hamiltonian \n\\begin{align}\nH\n&=\n-v\\sum_{\\langle i,j\\rangle\\sigma} (c^\\dagger_{i\\sigma}c_{j\\sigma}+ \\text{h.c.})\n-\\mu\\sum_i \\hat n_i+U\\sum_i \\hat n_{i\\uparrow}\\hat n_{i\\downarrow}, \n\\end{align}\nwhere $v$ is the hopping amplitude, $\\langle i,j\\rangle$ represents the nearest-neighbor site pairs, \n$\\mu$ is the chemical potential, and $U$ is the on-site interaction.\nIn this section, we focus on the half-filled case (i.e., $\\mu=U/2$). The DMFT solution for the simplified self-consistency (\\ref{semicirc}) is the exact solution for an infinitely connected Bethe lattice \\cite{GeorgesKotliarKrauthRozenberg1996}.\nLet us briefly recall the paramagnetic phase diagram for the single-orbital Hubbard model on this lattice \\cite{Bluemer_thesis}. At half-filling, there is a metal-insulator crossover at high temperature and a first-order Mott transition below a temperature corresponding to $\\beta\\hbar\\approx 17$ with a coexistence region between $U_{c1}$ and $U_{c2}$. The finite-temperature critical endpoint is at $U\\approx 4.7$, while the zero-temperature Mott transition occurs at $U=U_{c2}\\approx 5.6$. \nAt low temperature, the self-energy shows the Fermi liquid behavior [${\\rm Im}\\,\\Sigma(\\omega)\\sim \\omega^2$] \nin the weakly correlated metallic phase, \nwhile it shows an insulating behavior [${\\rm Im}\\,\\Sigma(\\omega)\\sim \\delta(\\omega)$] in the Mott phase\n\\cite{GeorgesKotliarKrauthRozenberg1996}. \nIn the correlated metallic phase, as the temperature is increased, deviations from the Fermi liquid behavior become apparent, \nbut an ${\\rm Im}\\,\\Sigma(\\omega)\\sim \\sqrt{\\omega}$ scaling as in the SYK model is not observed. \n\nIn the following, we will start the DMFT iterations from the noninteracting solution, which means that the finite-temperature Mott transition occurs at $U_{c2}$.\nFrom now on, we set $\\hbar=1$. \n\n\\begin{figure}[t]\n\\begin{center}\n\\includegraphics[angle=-90, width=0.49\\columnwidth]{plot_otoc_b50_Re.eps}\n\\includegraphics[angle=-90, width=0.49\\columnwidth]{plot_otoc_b50_Im.eps}\n\\includegraphics[angle=-90, width=0.49\\columnwidth]{plot_coth_b50_appendix.eps}\n\\includegraphics[angle=-90, width=0.49\\columnwidth]{plot_otoc_b50_modulus.eps}\n\\caption{\nTop panels: Real and imaginary parts of the OTOC $\\langle c^\\dagger_\\sigma(t)c_\\sigma(0), c^\\dagger_\\sigma(t)c_\\sigma(0)\\rangle$ for the single-orbital Hubbard model with $\\beta=50$ at half filling. The thin lines correspond to metallic solutions, while the thick line ($U=6$) corresponds to a Mott insulating solution. \nBottom left panel: Spectral function $\\mathscr A_{(c_\\sigma^\\dagger c_\\sigma)^2}(\\omega)=-\\frac{1}{\\pi}{\\rm Im}\\,C_{(c^\\dagger_\\sigma c_\\sigma)^2}^R(\\omega)$ multiplied by $\\coth\\big(\\frac{\\beta\\omega}{4}\\big)$. \nBottom right panel: Modulus of the OTOC $|\\langle c^\\dagger_\\sigma(t)c_\\sigma(0), c^\\dagger_\\sigma(t)c_\\sigma(0)\\rangle|$ (normalized at $t=0$) on a log-log scale. The dashed line is the result for $U=0$.\n}\n\\label{fig:results_fermion}\n\\end{center}\n\\end{figure}\n\n\n\nIn Fig.~\\ref{fig:results_fermion}, we show the results of $\\langle c^\\dagger_\\sigma(t)c_\\sigma(0), c^\\dagger_\\sigma(t)c_\\sigma(0)\\rangle$ for the interacting single-orbital Hubbard model at $\\beta=50$. \nThe top panels present the real and imaginary parts of the OTOC. One can see that the oscillations become more pronounced as one increases the interaction. In the bottom left panel of Fig.~\\ref{fig:results_fermion},\nwe show the corresponding spectral function $\\mathscr A_{(c_\\sigma^\\dagger c_\\sigma)^2}(\\omega)=-\\frac{1}{\\pi}{\\rm Im}\\,C_{(c^\\dagger_\\sigma c_\\sigma)^2}^R(\\omega)$ multiplied by $\\coth\\big(\\frac{\\beta\\omega}{4}\\big)$.\nThe Mott transition, which occurs between $U=5$ and $U=6$ at $\\beta=50$, manifests itself by the opening of a gap in the spectral function $\\mathscr A_{(c_\\sigma^\\dagger c_\\sigma)^2(\\omega)}$. This translates into more weakly damped oscillations in the real-time evolution in the Mott phase.  \nThe bottom right panel of Fig.~\\ref{fig:results_fermion} plots the modulus of the OTOC (normalized at time $t=0$, i.e., $|\\langle c^\\dagger_\\sigma(t)c_\\sigma(0), c^\\dagger_\\sigma(t)c_\\sigma(0)\\rangle|/|\\langle c^\\dagger_\\sigma(0)c_\\sigma(0), c^\\dagger_\\sigma(0)c_\\sigma(0)\\rangle|$) on a log-log scale. \nDue to the oscillations and the limited accuracy of the analytic continuation procedure, it is hard to clearly identify the nature of the long-time decay of the OTOC,\nbut the results indicate that the OTOC decays much faster (possibly exponentially) in the Mott phase than in the metallic phase. This is consistent with the results for the spectral functions of the OTOC. The comparison to the noninteracting case (dashed curve in the bottom right panel of Fig.~\\ref{fig:results_fermion}) suggests that the correlated metallic phase has a similar (possibly power-law) decay behavior of the OTOC as in the noninteracting case.\n\n\n\\begin{figure}[t]\n\\begin{center}\n\\includegraphics[angle=-90, width=0.49\\columnwidth]{plot_otoc_b50_nn_Re.eps}\n\\includegraphics[angle=-90, width=0.49\\columnwidth]{plot_otoc_b50_nn_Im.eps}\n\\includegraphics[angle=-90, width=0.49\\columnwidth]{plot_coth_b50_nn_appendix.eps}\n\\includegraphics[angle=-90, width=0.49\\columnwidth]{plot_otoc_b50_nn_modulus.eps}\n\\caption{\nTop panels: Real and imaginary parts of the OTOC $\\langle \\hat n(t)\\hat n(0), \\hat n(t)\\hat n(0)\\rangle - 1$ for the single-orbital Hubbard model with $\\beta=50$ at half filling. The thin lines correspond to metallic solutions, \nwhile the thick line ($U=6$) corresponds to a Mott insulating solution. \nBottom left panel: Spectral function $\\mathscr A_{(nn)^2}(\\omega)=-\\frac{1}{\\pi}{\\rm Im}\\,C_{(n n)^2}^R(\\omega)$ multiplied by $\\coth\\big(\\frac{\\beta\\omega}{4}\\big)$.\nBottom right panel: Modulus of the OTOC $|\\langle \\hat n(t)\\hat n(0), \\hat n(t)\\hat n(0)\\rangle - 1|$ (normalized at $t=0$) on a log-log scale. \n}\n\\label{fig:results_nn}\n\\end{center}\n\\end{figure}\n\n\nIn Fig.~\\ref{fig:results_nn}, we plot the OTOC $\\langle \\hat n(t)\\hat n(0), \\hat n(t)\\hat n(0)\\rangle$\nfor the single-orbital Hubbard model in a similar manner as in Fig.~\\ref{fig:results_fermion}.\nSince at half filling this correlation function approaches $\\langle \\hat n\\rangle^4=1$ at long times\nand at sufficiently low temperature,\nwe perform the analytical continuation procedure for $\\langle \\hat n(t)\\hat n(0), \\hat n(t)\\hat n(0)\\rangle-1$. \nThe top panels of Fig.~\\ref{fig:results_nn} show the real and imaginary parts of this shifted OTOC. \nContrary to the case of $\\langle c_\\sigma^\\dagger(t)c_\\sigma(0), c_\\sigma^\\dagger(t)c_\\sigma(0)\\rangle$, \nthe amplitude of the OTOC $\\langle \\hat n(t)\\hat n(0), \\hat n(t)\\hat n(0)\\rangle-1$ is suppressed as one increases the interaction, which reflects the reduced charge fluctuations in the Mott insulator.\nThe bottom left panel in Fig.~\\ref{fig:results_nn} shows the analytically continued spectral function $\\mathscr A_{(nn)^2}(\\omega)=-\\frac{1}{\\pi}{\\rm Im}\\, C_{(nn)^2}^R(\\omega)$ multiplied by $\\coth\\big(\\frac{\\beta\\omega}{4}\\big)$. \nAgain we notice the opening of a gap in the Mott phase, and a corresponding shift of the spectral weight to higher energies. This results in more rapid oscillations of the OTOC in the Mott state.\nThe bottom right panel in Fig.~\\ref{fig:results_nn} plots the modulus of the OTOC $|\\langle \\hat n(t)\\hat n(0), \\hat n(t)\\hat n(0)\\rangle-1|$ (normalized at $t=0$) on a log-log scale.\nThe long-time behavior of the OTOC is consistent with a power-law decay in the metallic phase, and an exponential decay in the Mott insulating phase.\n\n\n\\begin{figure}[t]\n\\begin{center}\n\\includegraphics[angle=-90, width=0.49\\columnwidth]{plot_otoc_nsns_b50_draft_Re.eps}\n\\includegraphics[angle=-90, width=0.49\\columnwidth]{plot_otoc_nsns_b50_draft_Im.eps}\n\\includegraphics[angle=-90, width=0.49\\columnwidth]{plot_coth_b50_nsns_inset.eps}\n\\includegraphics[angle=-90, width=0.49\\columnwidth]{plot_otoc_nsns_b50_draft_modulus_zoom.eps}\n\\caption{\nTop panels: Real and imaginary parts of the OTOC $\\langle \\hat n_\\sigma(t)\\hat n_\\sigma(0), \\hat n_\\sigma(t)\\hat n_\\sigma(0)\\rangle  - \\langle \\hat n_\\sigma\\rangle^4$ for the single-orbital Hubbard model with $\\beta=50$ at half filling. All the colored lines correspond to metallic solutions. \nBottom left panel: Spectral function $\\mathscr A_{(n_\\sigma n_\\sigma)^2}(\\omega)=-\\frac{1}{\\pi}{\\rm Im}\\,C_{(n_\\sigma n_\\sigma)^2}^R(\\omega)$ multiplied by $\\coth\\big(\\frac{\\beta\\omega}{4}\\big)$. In the inset, we plot the imaginary-time four-point function $\\langle \\hat n_\\sigma(\\tau)\\hat n_\\sigma(0), \\hat n_\\sigma(\\tau)\\hat n_\\sigma(0)\\rangle-\\langle \\hat n_\\sigma\\rangle^4$, with the black line corresponding to the Mott insulating solution at $U=6$.  \nBottom right panel: Modulus of the OTOC $|\\langle \\hat n_\\sigma(t)\\hat n_\\sigma(0), \\hat n_\\sigma(t)\\hat n_\\sigma(0)\\rangle  - \\langle \\hat n_\\sigma\\rangle^4|$ (normalized at $t=0$) on a log-log scale.\n}\n\\label{fig:results_nsns}\n\\end{center}\n\\end{figure}\n\nFigure~\\ref{fig:results_nsns} \nshows the results of the OTOC $\\langle \\hat n_\\sigma(t)\\hat n_\\sigma(0), \\hat n_\\sigma(t)\\hat n_\\sigma(0)\\rangle$\nfor the single-orbital Hubbard model. At half filling, this correlation function approaches $\\langle \\hat n_\\sigma\\rangle^4=\\frac{1}{16}$ in the long-time limit and at sufficiently low temperature, \nso that we perform the analytical continuation procedure for $\\langle \\hat n_\\sigma(t)\\hat n_\\sigma(0), \\hat n_\\sigma(t)\\hat n_\\sigma(0)\\rangle-\\langle \\hat n_\\sigma\\rangle^4$. \nThe top panels of Fig.~\\ref{fig:results_nsns} show the real and imaginary parts of \nthis shifted OTOC. \nHere, we only present results for the metallic phase, because resolving the very sharp low-energy feature of the OTOC spectrum in the half-filled Mott state is challenging. One can see that coherent oscillations are not observed for this type of OTOC, and that the incoherent part becomes dominant compared to the other two OTOCs shown in Figs.~\\ref{fig:results_fermion} and \\ref{fig:results_nn}. The amplitude of the OTOC $\\langle \\hat n_\\sigma(t)\\hat n_\\sigma(0), \\hat n_\\sigma(t)\\hat n_\\sigma(0)\\rangle-\\langle \\hat n_\\sigma\\rangle^4$ is enhanced as $U$ increases. As we argue below, this is related to the enhancement of spin fluctuations \nand local moment formation close to the Mott transition.\n\nThe bottom left panel in Fig.~\\ref{fig:results_nsns} plots the analytically continued spectral function $\\mathscr A_{(n_\\sigma n_\\sigma)^2}(\\omega)=-\\frac{1}{\\pi}{\\rm Im}\\, C_{(n_\\sigma n_\\sigma)^2}^R(\\omega)$ multiplied by $\\coth\\big(\\frac{\\beta\\omega}{4}\\big)$. \nWe can see that the spectral weight is concentrated in the low-energy region as we approach the Mott transition point.\nThe inset in the bottom left panel of Fig.~\\ref{fig:results_nsns} shows the imaginary-time four-point function $\\langle \\hat n_\\sigma(\\tau)\\hat n_\\sigma(0), \\hat n_\\sigma(\\tau)\\hat n_\\sigma(0)\\rangle-\\langle \\hat n_\\sigma\\rangle^4$, with the black line corresponding to a Mott insulating solution ($U=6$).  \nFor $U\\gtrsim 4$, the imaginary-time four-point function does not decay to zero but remains relatively large near $\\tau=\\frac{\\beta}{4}$. \nThis behavior is reminiscent of the spin-freezing physics seen in the correlated Hund metal phase of multi-orbital Hubbard models, where the dynamical spin correlation function $\\langle \\hat S_z(\\tau)\\hat S_z(0)\\rangle$ is trapped at a finite value at long $\\tau$ \\cite{Werner2008}. \nIn fact, the OTOC $\\langle \\hat n_\\sigma(\\tau)\\hat n_\\sigma(0), \\hat n_\\sigma(\\tau)\\hat n_\\sigma(0)\\rangle$\nmeasures spin correlations (in addition to charge correlations) through $\\hat n_\\sigma=\\frac{1}{2}\\hat n+\\sigma\\hat S_z$.\nPhysically, the trapping of $\\langle \\hat S_z(\\tau)\\hat S_z(0)\\rangle$ signals the formation of frozen local magnetic moments.\nIn the metal-insulator crossover region, the scattering induced by these magnetic moments results in incoherent metal states. Intuitively, one might expect fast scrambling in the corresponding regions of the phase diagram. \nAlthough there is some resemblance with the spin freezing crossover,\nthe self-energy does not exhibit a square-root frequency dependence in the single-orbital case, and a spin-freezing crossover in the sense of Ref.~\\cite{Werner2008} does not exist.  \n\nThe bottom right panel of Fig.~\\ref{fig:results_nsns} plots the modulus of the OTOC $|\\langle \\hat n_\\sigma(t)\\hat n_\\sigma(0), \\hat n_\\sigma(t)\\hat n_\\sigma(0)\\rangle-\\langle \\hat n_\\sigma\\rangle^4|$ (normalized at $t=0$) on a log-log scale. \nThe dynamics of $\\langle \\hat n_\\sigma(t)\\hat n_\\sigma(0), \\hat n_\\sigma(t)\\hat n_\\sigma(0)\\rangle$ is quite distinct from that of $\\langle \\hat n(t)\\hat n(0), \\hat n(t)\\hat n(0)\\rangle$. As the Mott transition is approached, we observe a slow-down in the decay of the modulus, \nwhich indicates the presence of slowly fluctuating local moments. \nThe long-time behavior may be consistent with a power-law, although slow oscillations make it difficult to determine the long-time asymptotic form from the numerics. \n\n\n\\begin{figure}[t]\n\\begin{center}\n\\includegraphics[angle=-90, width=0.49\\columnwidth]{plot_fitnsns_b2.eps}\n\\includegraphics[angle=-90, width=0.49\\columnwidth]{plot_coeffc_draft.eps}\n\\caption{\nLeft panel: \nThe imaginary-time four-point function $\\langle \\hat n_\\sigma(\\tau)\\hat n_\\sigma(0), \\hat n_\\sigma(\\tau)\\hat n_\\sigma(0)\\rangle$, the squared density-density correlation function $\\langle \\hat n_\\sigma(\\tau)\\hat n_\\sigma(0)\\rangle^2$,\nand their difference for the single-orbital Hubbard model with $U=5.5$ and $\\beta=50$.\nThe dashed curve shows a fit of the difference with the function $f_\\text{fit}(\\tau)=a+b\\tau+c\\tau^2$ in the interval $\\tau/\\beta \\in [0: 0.2]$.\nRight panel: Fitting coefficient $c$ as a function of $U$ for indicated values of $\\beta$.\n\n}\n\\label{fig:coeffs}\n\\end{center}\n\\end{figure}\n\nFor large enough $\\beta$ and small enough $\\tau$, the OTOC (\\ref{otoc_nsns}) factorizes into $\\langle \\hat n_\\sigma(\\tau)\\hat n_\\sigma(0), \\hat n_\\sigma(\\tau)\\hat n_\\sigma(0)\\rangle \\approx \\langle \\hat n_\\sigma(\\tau)\\hat n_\\sigma(0)\\rangle^2$. \nOn the real-time axis, this corresponds to the decoupling $\\langle \\hat n_\\sigma(t)\\hat n_\\sigma(0), \\hat n_\\sigma(t)\\hat n_\\sigma(0)\\rangle \\approx \\langle \\hat n_\\sigma(t)\\hat n_\\sigma(0)\\rangle^2$, that is, \nthe OTOC is reduced to a product of ordinary density-density correlation functions.\nIn order to see whether nontrivial correlations are captured beyond the decoupled form by the OTOC function, \nwe consider the difference $\\langle \\hat n_\\sigma(\\tau)\\hat n_\\sigma(0), \\hat n_\\sigma(\\tau)\\hat n_\\sigma(0)\\rangle-\\langle \\hat n_\\sigma(\\tau)\\hat n_\\sigma(0)\\rangle^2$ and fit the short-time behavior of this function to $f_\\text{fit}(\\tau)=a+b\\tau+c\\tau^2$, as illustrated in the left panel of Fig.~\\ref{fig:coeffs}. The coefficients $b$ and $c$ serve as  indicators of the nontrivial correlations that cannot be attributed to the decoupled form of the OTOC.\nThe coefficient $c$ is plotted as a function of $U$ and for different $\\beta$ in the right hand panel. The coefficient $b$ (not shown) exhibits a similar trend. We notice that the OTOC picks up nontrival correlations near the metal-insulator transition, and in particular at \nintermediate\ntemperatures, while the Mott phase shows no such correlations. The crossover region  \nassociated with the emergence of local moments \ncorresponds, roughly, to the interaction range where the nontrivial correlations start to become significant. \n\n\n",
        "label": "fig:results_fermion",
        "Descriptive_question1": "What is the value of the interaction strength U for the Mott insulating solution shown as a thick line in the top panels of figure_5?",
        "Descriptive_question2": "Which line style represents the noninteracting case (U=0) in the bottom right panel of figure_5?",
        "Reasoning_question1": "How does the increase in interaction strength U affect the oscillations in the real and imaginary parts of the OTOC in the top panels of figure_5, and what might this indicate about the system's behavior?",
        "Reasoning_question2": "Comparing the spectral function in the bottom left panel of figure_5 between metallic and Mott insulating solutions, what can be inferred about the impact of the Mott transition on the system's dynamics?",
        "Descriptive_answer1": "6",
        "Descriptive_answer2": "Dashed line",
        "Reasoning_answer1": "As U increases, the oscillations in both the real and imaginary parts of the OTOC become more pronounced, indicating stronger local correlations and the presence of a Mott insulating phase with gapped excitations. This suggests that increasing interaction strength leads to more coherent oscillations and slower decay dynamics due to the opening of an energy gap.",
        "Reasoning_answer2": "The spectral function for the Mott insulating solution shows a clear gap opening compared to the metallic solutions, which have spectral weight near zero frequency. This gap indicates suppressed low-energy excitations in the Mott phase, resulting in more weakly damped oscillations and slower decay of the OTOC, reflecting the insulating nature and reduced charge fluctuations in the system after the Mott transition."
    },
    {
        "paper_id": "1812.04217.json",
        "image_id": "figure_6",
        "image_path": [
            "/home/yz979/scratch/chengye/scimolmo/test-images-tables/dataset_imagestables/1812.04217/images/plot_otoc_b50_nn_Re.eps"
        ],
        "caption": "\nTop panels: Real and imaginary parts of the OTOC $\\langle \\hat n(t)\\hat n(0), \\hat n(t)\\hat n(0)\\rangle - 1$ for the single-orbital Hubbard model with $\\beta=50$ at half filling. The thin lines correspond to metallic solutions, \nwhile the thick line ($U=6$) corresponds to a Mott insulating solution. \nBottom left panel: Spectral function $\\mathscr A_{(nn)^2}(\\omega)=-\\frac{1}{\\pi}{\\rm Im}\\,C_{(n n)^2}^R(\\omega)$ multiplied by $\\coth\\big(\\frac{\\beta\\omega}{4}\\big)$.\nBottom right panel: Modulus of the OTOC $|\\langle \\hat n(t)\\hat n(0), \\hat n(t)\\hat n(0)\\rangle - 1|$ (normalized at $t=0$) on a log-log scale. \n",
        "classify": "Chart",
        "section_info": "4 Numerical results for the Hubbard model\n\\section{Numerical results for the Hubbard model}\n\\label{numerical results}\n\n\\subsection{Observables and numerical procedure}\n\nWe consider the single-orbital, two-orbital, and three-orbital Hubbard models on an infinitely-connected Bethe lattice, which can be solved exactly within DMFT \\cite{GeorgesKotliarKrauthRozenberg1996}. In this case, the noninteracting density of states is semicircular with bandwidth $4v_\\ast$, for which there exists a simplified DMFT self-consistency condition between the hybridization function $\\Delta_{\\alpha\\sigma}$ of the DMFT impurity problem and the local (impurity) Green's function $G_{\\alpha\\sigma}$: \n\n\\begin{equation}\n\\Delta_{\\alpha\\sigma}(\\tau)=v_\\ast^2 G_{\\alpha\\sigma}(\\tau), \n\\label{semicirc}\n\\end{equation}\n\nwith $\\alpha$ the orbital and $\\sigma$ the spin index. We will use $v_\\ast$ as the unit of energy and measure time in units of $\\hbar/v_\\ast$. \n\nThree different types of imaginary-time four-point functions $C_{(AB)^2}^M(\\tau)$\nof the form of Eq.~(\\ref{imaginary-time OTOC})\nwith $(\\hat A, \\hat B)=(c_\\sigma^\\dagger, c_\\sigma), (\\hat n_\\sigma, \\hat n_\\sigma)$, and $(\\hat n, \\hat n)$\nare calculated in the interval $0\\le \\tau\\le \\frac{\\beta\\hbar}{2}$: \n\\begin{eqnarray}\nC_{(c^\\dagger_\\sigma c_\\sigma)^2}^M(\\tau) &=& -\\langle c^\\dagger_\\sigma(\\tau+\\tfrac{\\beta\\hbar}{2}) c_\\sigma(\\tfrac{\\beta\\hbar}{2}) c^\\dagger_\\sigma(\\tau) c_\\sigma(0) \\rangle, \\label{fermion_otoc}\\\\\nC_{(n_\\sigma n_\\sigma)^2}^M(\\tau) &=& -\\langle \\hat n_\\sigma(\\tau+\\tfrac{\\beta\\hbar}{2})\\hat n_\\sigma(\\tfrac{\\beta\\hbar}{2})\\hat n_\\sigma(\\tau)\\hat n_\\sigma(0) \\rangle, \\label{otoc_nsns}\\\\\nC_{(nn)^2}^M(\\tau) &=& -\\langle \\hat n(\\tau+\\tfrac{\\beta\\hbar}{2})\\hat n(\\tfrac{\\beta\\hbar}{2})\\hat n(\\tau)\\hat n(0) \\rangle,\\label{otoc_nn}\n\\end{eqnarray}\n\nwhere $c_\\sigma^\\dagger$ ($c_\\sigma$) are the fermionic creation (annihilation) operators for spin $\\sigma$ (and orbital $\\alpha=1$ in the multi-orbital case), $\\hat n_\\sigma=c^\\dagger_\\sigma c_\\sigma$ is the corresponding spin-dependent density operator, and $\\hat n=\\hat n_\\uparrow + \\hat n_\\downarrow$ the total density operator. Note that in all the three cases above we have $\\hat B=\\hat A^\\dagger$. \nWe measure these local correlation functions in the impurity model using a hybridization expansion continuous-time Monte Carlo algorithm (CT-HYB) \\cite{Werner2006}. In this algorithm, the \ntwo-particle\nGreen's functions of the type (\\ref{fermion_otoc}) can be measured by removing two hybridization lines, i.e., from the elements of the inverse hybridization matrix, while the density-density correlation functions can be easily measured either by insertion of density operators (matrix formalism) \\cite{Werner2006Kondo} or by reading off the occupation of the orbitals at the four time points in the segment implementation \\cite{Werner2006}. We use the latter algorithm since we consider only density-density interactions. \nThe values at the end-points $\\tau=0$ and $\\tau=\\frac{\\beta\\hbar}{2}$ reduce to standard density-density correlation functions, which in the case of Eq.~(\\ref{fermion_otoc}) are measured separately. \n\nWe perform the analytic continuation to the real-frequency axis using the Maximum Entropy method \\cite{Jarrell1996,Lewin_maxent} with a bosonic kernel. This yields the imaginary part of the retarded correlator \n$-\\frac{1}{\\pi}\\text{Im}\\, C^R_{(AA^\\dagger)^2}(\\omega)=\\mathscr A_{(AA^\\dagger)^2}(\\omega)$.  \nWhen $\\hat B=\\hat A^\\dagger=\\hat A$ (which is the case for $\\hat A=\\hat n_\\sigma, \\hat n$), we have $C_{(AA)^2}^R(\\omega)^\\ast=C_{(AA)^2}^R(-\\omega)$. At half filling, the particle-hole symmetry furthermore ensures $C_{c_\\sigma^\\dagger c_\\sigma}^R(\\omega)^\\ast=C_{c_\\sigma^\\dagger c_\\sigma}^R(-\\omega)$.\nThus, in all the cases considered in this study, the retarded OTOC has the symmetry property $C_{(AA^\\dagger)^2}^R(\\omega)^\\ast=C_{(AA^\\dagger)^2}^R(-\\omega)$. Using this, as well as the out-of-time-order fluctuation-dissipation\ntheorem discussed in the previous section, we can obtain the real-time OTOC\n$\\langle \\hat A(t)\\hat A^\\dagger(0), \\hat A(t)\\hat A^\\dagger(0)\\rangle$ from the following inverse Fourier transformation, \n\\begin{align}{\\rm Re}\\langle \\hat A(t)\\hat A^\\dagger(0), \\hat A(t)\\hat A^\\dagger(0)\\rangle\n&=\n{\\rm Re}\n\\int_0^\\infty d\\omega\\, e^{-i\\omega t}\\coth\\left(\\frac{\\beta\\hbar\\omega}{4}\\right)\n\\left(-\\frac{1}{\\pi}{\\rm Im}\\, C_{(AA^\\dagger)^2}^R(\\omega)\\right),\n\\\\\n{\\rm Im}\\langle \\hat A(t)\\hat A^\\dagger(0), \\hat A(t)\\hat A^\\dagger(0)\\rangle\n&=\n{\\rm Im}\\int_0^{\\infty} d\\omega\\, e^{-i\\omega t} \\left(-\\frac{1}{\\pi}{\\rm Im}\\, C_{(AA^\\dagger)^2}^R(\\omega)\\right),\n\\end{align}\nfor $\\hat A=c_\\sigma^\\dagger, \\hat n_\\sigma$, and $\\hat n$.\n\n\\begin{figure}[b]\n\\begin{center}\n\\includegraphics[angle=-90, width=0.48\\columnwidth]{compare_u0_draft.eps}\\hfill\n\\includegraphics[angle=-90, width=0.48\\columnwidth]{compare_u0_longtime_modulus.eps}\n\\caption{\nLeft panel: Comparison between the exact solution and the result obtained by analytical continuation\nfor the real and imaginary parts of the OTOC $\\langle c_\\sigma^\\dagger(t)c_\\sigma(0), c_\\sigma^\\dagger(t)c_\\sigma(0)\\rangle$ \nin the noninteracting case ($U=0, \\beta\\hbar=50$) of the single-orbital Hubbard model.\nRight panel: Comparison of the long-time behavior for the modulus $|\\langle c_\\sigma^\\dagger(t)c_\\sigma(0), c_\\sigma^\\dagger(t)c_\\sigma(0)\\rangle|$. \nThe exact result shows a power-law decay ($\\sim 1/t^3$). \n}\n\\label{fig:test_u0}\n\\end{center}\n\\end{figure}\n\nAs a check of our procedure, we first compare the results for the noninteracting model,\nfor which an exact solution of the OTOC $\\langle c_\\sigma^\\dagger(t)c_\\sigma(0), c_\\sigma^\\dagger(t)c_\\sigma(0)\\rangle$ \nis available \\cite{TsujiWernerUeda2017}.\nIt is a nontrivial task to measure the noninteracting two-particle Green's function in CT-HYB, \nand to analytically continue \n$C_{(c^\\dagger_\\sigma c_\\sigma)^2}^M(\\tau)$ \nby the Maximum Entropy method.  \nThe results for the inverse temperature $\\beta\\hbar=50$ are plotted in Fig.~\\ref{fig:test_u0}. \nOne can see that the OTOC oscillates for about one cycle, and quickly decays to zero.\nThe analytically continued data show a good agreement with the exact solution. \nIn particular, for short times (up to about two inverse hoppings) the dynamics is accurately reproduced, while deviations appear at longer times. \nIt is known that for the noninteracting fermion system the OTOC decays as a power law at long time ($\\sim 1/t^3$) \\cite{TsujiWernerUeda2017}.\nThe analytical continuation method cannot reproduce the details of the oscillations at intermediate or long times, but it roughly captures the long-time decay, which is controlled by low-frequency spectral features. \nUsually, analytic continuation is unreliable for high frequency components, because high-frequency information is suppressed \nby the kernel $K(\\tau,\\omega)$ in the transformation from the spectral function $A_O(\\omega)$ to the (bosonic) Matsubara correlation function $O(\\tau)$ [$O(\\tau)=\\int_{-\\infty}^{\\infty} d\\omega\\, K(\\omega,\\tau) A_O(\\omega)$ with $K(\\tau,\\omega)=e^{-\\tau\\omega}/(1-e^{-\\beta\\omega})$]. On the other hand, the low frequency components can be rather accurately determined. As a result, in Fig. 4 we more or less recover the low-frequency features of the time-dependent correlation function, while the details of rapid oscillations are not captured. The accurate result at short times can be understood from the fact that this region is close to the imaginary time axis in the complex time plane, where the original QMC data are available.\nAt higher temperatures, the Maximum Entropy method becomes less reliable, so that the resulting OTOCs are expected to be less accurate.  \n\n\n\\subsection{Single-orbital Hubbard model}\n\\label{sec:hubbard}\n\nIn this section, we calculate the OTOCs (\\ref{fermion_otoc})-(\\ref{otoc_nn}) for the interacting single-orbital Hubbard model with the Hamiltonian \n\\begin{align}\nH\n&=\n-v\\sum_{\\langle i,j\\rangle\\sigma} (c^\\dagger_{i\\sigma}c_{j\\sigma}+ \\text{h.c.})\n-\\mu\\sum_i \\hat n_i+U\\sum_i \\hat n_{i\\uparrow}\\hat n_{i\\downarrow}, \n\\end{align}\nwhere $v$ is the hopping amplitude, $\\langle i,j\\rangle$ represents the nearest-neighbor site pairs, \n$\\mu$ is the chemical potential, and $U$ is the on-site interaction.\nIn this section, we focus on the half-filled case (i.e., $\\mu=U/2$). The DMFT solution for the simplified self-consistency (\\ref{semicirc}) is the exact solution for an infinitely connected Bethe lattice \\cite{GeorgesKotliarKrauthRozenberg1996}.\nLet us briefly recall the paramagnetic phase diagram for the single-orbital Hubbard model on this lattice \\cite{Bluemer_thesis}. At half-filling, there is a metal-insulator crossover at high temperature and a first-order Mott transition below a temperature corresponding to $\\beta\\hbar\\approx 17$ with a coexistence region between $U_{c1}$ and $U_{c2}$. The finite-temperature critical endpoint is at $U\\approx 4.7$, while the zero-temperature Mott transition occurs at $U=U_{c2}\\approx 5.6$. \nAt low temperature, the self-energy shows the Fermi liquid behavior [${\\rm Im}\\,\\Sigma(\\omega)\\sim \\omega^2$] \nin the weakly correlated metallic phase, \nwhile it shows an insulating behavior [${\\rm Im}\\,\\Sigma(\\omega)\\sim \\delta(\\omega)$] in the Mott phase\n\\cite{GeorgesKotliarKrauthRozenberg1996}. \nIn the correlated metallic phase, as the temperature is increased, deviations from the Fermi liquid behavior become apparent, \nbut an ${\\rm Im}\\,\\Sigma(\\omega)\\sim \\sqrt{\\omega}$ scaling as in the SYK model is not observed. \n\nIn the following, we will start the DMFT iterations from the noninteracting solution, which means that the finite-temperature Mott transition occurs at $U_{c2}$.\nFrom now on, we set $\\hbar=1$. \n\n\\begin{figure}[t]\n\\begin{center}\n\\includegraphics[angle=-90, width=0.49\\columnwidth]{plot_otoc_b50_Re.eps}\n\\includegraphics[angle=-90, width=0.49\\columnwidth]{plot_otoc_b50_Im.eps}\n\\includegraphics[angle=-90, width=0.49\\columnwidth]{plot_coth_b50_appendix.eps}\n\\includegraphics[angle=-90, width=0.49\\columnwidth]{plot_otoc_b50_modulus.eps}\n\\caption{\nTop panels: Real and imaginary parts of the OTOC $\\langle c^\\dagger_\\sigma(t)c_\\sigma(0), c^\\dagger_\\sigma(t)c_\\sigma(0)\\rangle$ for the single-orbital Hubbard model with $\\beta=50$ at half filling. The thin lines correspond to metallic solutions, while the thick line ($U=6$) corresponds to a Mott insulating solution. \nBottom left panel: Spectral function $\\mathscr A_{(c_\\sigma^\\dagger c_\\sigma)^2}(\\omega)=-\\frac{1}{\\pi}{\\rm Im}\\,C_{(c^\\dagger_\\sigma c_\\sigma)^2}^R(\\omega)$ multiplied by $\\coth\\big(\\frac{\\beta\\omega}{4}\\big)$. \nBottom right panel: Modulus of the OTOC $|\\langle c^\\dagger_\\sigma(t)c_\\sigma(0), c^\\dagger_\\sigma(t)c_\\sigma(0)\\rangle|$ (normalized at $t=0$) on a log-log scale. The dashed line is the result for $U=0$.\n}\n\\label{fig:results_fermion}\n\\end{center}\n\\end{figure}\n\n\n\nIn Fig.~\\ref{fig:results_fermion}, we show the results of $\\langle c^\\dagger_\\sigma(t)c_\\sigma(0), c^\\dagger_\\sigma(t)c_\\sigma(0)\\rangle$ for the interacting single-orbital Hubbard model at $\\beta=50$. \nThe top panels present the real and imaginary parts of the OTOC. One can see that the oscillations become more pronounced as one increases the interaction. In the bottom left panel of Fig.~\\ref{fig:results_fermion},\nwe show the corresponding spectral function $\\mathscr A_{(c_\\sigma^\\dagger c_\\sigma)^2}(\\omega)=-\\frac{1}{\\pi}{\\rm Im}\\,C_{(c^\\dagger_\\sigma c_\\sigma)^2}^R(\\omega)$ multiplied by $\\coth\\big(\\frac{\\beta\\omega}{4}\\big)$.\nThe Mott transition, which occurs between $U=5$ and $U=6$ at $\\beta=50$, manifests itself by the opening of a gap in the spectral function $\\mathscr A_{(c_\\sigma^\\dagger c_\\sigma)^2(\\omega)}$. This translates into more weakly damped oscillations in the real-time evolution in the Mott phase.  \nThe bottom right panel of Fig.~\\ref{fig:results_fermion} plots the modulus of the OTOC (normalized at time $t=0$, i.e., $|\\langle c^\\dagger_\\sigma(t)c_\\sigma(0), c^\\dagger_\\sigma(t)c_\\sigma(0)\\rangle|/|\\langle c^\\dagger_\\sigma(0)c_\\sigma(0), c^\\dagger_\\sigma(0)c_\\sigma(0)\\rangle|$) on a log-log scale. \nDue to the oscillations and the limited accuracy of the analytic continuation procedure, it is hard to clearly identify the nature of the long-time decay of the OTOC,\nbut the results indicate that the OTOC decays much faster (possibly exponentially) in the Mott phase than in the metallic phase. This is consistent with the results for the spectral functions of the OTOC. The comparison to the noninteracting case (dashed curve in the bottom right panel of Fig.~\\ref{fig:results_fermion}) suggests that the correlated metallic phase has a similar (possibly power-law) decay behavior of the OTOC as in the noninteracting case.\n\n\n\\begin{figure}[t]\n\\begin{center}\n\\includegraphics[angle=-90, width=0.49\\columnwidth]{plot_otoc_b50_nn_Re.eps}\n\\includegraphics[angle=-90, width=0.49\\columnwidth]{plot_otoc_b50_nn_Im.eps}\n\\includegraphics[angle=-90, width=0.49\\columnwidth]{plot_coth_b50_nn_appendix.eps}\n\\includegraphics[angle=-90, width=0.49\\columnwidth]{plot_otoc_b50_nn_modulus.eps}\n\\caption{\nTop panels: Real and imaginary parts of the OTOC $\\langle \\hat n(t)\\hat n(0), \\hat n(t)\\hat n(0)\\rangle - 1$ for the single-orbital Hubbard model with $\\beta=50$ at half filling. The thin lines correspond to metallic solutions, \nwhile the thick line ($U=6$) corresponds to a Mott insulating solution. \nBottom left panel: Spectral function $\\mathscr A_{(nn)^2}(\\omega)=-\\frac{1}{\\pi}{\\rm Im}\\,C_{(n n)^2}^R(\\omega)$ multiplied by $\\coth\\big(\\frac{\\beta\\omega}{4}\\big)$.\nBottom right panel: Modulus of the OTOC $|\\langle \\hat n(t)\\hat n(0), \\hat n(t)\\hat n(0)\\rangle - 1|$ (normalized at $t=0$) on a log-log scale. \n}\n\\label{fig:results_nn}\n\\end{center}\n\\end{figure}\n\n\nIn Fig.~\\ref{fig:results_nn}, we plot the OTOC $\\langle \\hat n(t)\\hat n(0), \\hat n(t)\\hat n(0)\\rangle$\nfor the single-orbital Hubbard model in a similar manner as in Fig.~\\ref{fig:results_fermion}.\nSince at half filling this correlation function approaches $\\langle \\hat n\\rangle^4=1$ at long times\nand at sufficiently low temperature,\nwe perform the analytical continuation procedure for $\\langle \\hat n(t)\\hat n(0), \\hat n(t)\\hat n(0)\\rangle-1$. \nThe top panels of Fig.~\\ref{fig:results_nn} show the real and imaginary parts of this shifted OTOC. \nContrary to the case of $\\langle c_\\sigma^\\dagger(t)c_\\sigma(0), c_\\sigma^\\dagger(t)c_\\sigma(0)\\rangle$, \nthe amplitude of the OTOC $\\langle \\hat n(t)\\hat n(0), \\hat n(t)\\hat n(0)\\rangle-1$ is suppressed as one increases the interaction, which reflects the reduced charge fluctuations in the Mott insulator.\nThe bottom left panel in Fig.~\\ref{fig:results_nn} shows the analytically continued spectral function $\\mathscr A_{(nn)^2}(\\omega)=-\\frac{1}{\\pi}{\\rm Im}\\, C_{(nn)^2}^R(\\omega)$ multiplied by $\\coth\\big(\\frac{\\beta\\omega}{4}\\big)$. \nAgain we notice the opening of a gap in the Mott phase, and a corresponding shift of the spectral weight to higher energies. This results in more rapid oscillations of the OTOC in the Mott state.\nThe bottom right panel in Fig.~\\ref{fig:results_nn} plots the modulus of the OTOC $|\\langle \\hat n(t)\\hat n(0), \\hat n(t)\\hat n(0)\\rangle-1|$ (normalized at $t=0$) on a log-log scale.\nThe long-time behavior of the OTOC is consistent with a power-law decay in the metallic phase, and an exponential decay in the Mott insulating phase.\n\n\n\\begin{figure}[t]\n\\begin{center}\n\\includegraphics[angle=-90, width=0.49\\columnwidth]{plot_otoc_nsns_b50_draft_Re.eps}\n\\includegraphics[angle=-90, width=0.49\\columnwidth]{plot_otoc_nsns_b50_draft_Im.eps}\n\\includegraphics[angle=-90, width=0.49\\columnwidth]{plot_coth_b50_nsns_inset.eps}\n\\includegraphics[angle=-90, width=0.49\\columnwidth]{plot_otoc_nsns_b50_draft_modulus_zoom.eps}\n\\caption{\nTop panels: Real and imaginary parts of the OTOC $\\langle \\hat n_\\sigma(t)\\hat n_\\sigma(0), \\hat n_\\sigma(t)\\hat n_\\sigma(0)\\rangle  - \\langle \\hat n_\\sigma\\rangle^4$ for the single-orbital Hubbard model with $\\beta=50$ at half filling. All the colored lines correspond to metallic solutions. \nBottom left panel: Spectral function $\\mathscr A_{(n_\\sigma n_\\sigma)^2}(\\omega)=-\\frac{1}{\\pi}{\\rm Im}\\,C_{(n_\\sigma n_\\sigma)^2}^R(\\omega)$ multiplied by $\\coth\\big(\\frac{\\beta\\omega}{4}\\big)$. In the inset, we plot the imaginary-time four-point function $\\langle \\hat n_\\sigma(\\tau)\\hat n_\\sigma(0), \\hat n_\\sigma(\\tau)\\hat n_\\sigma(0)\\rangle-\\langle \\hat n_\\sigma\\rangle^4$, with the black line corresponding to the Mott insulating solution at $U=6$.  \nBottom right panel: Modulus of the OTOC $|\\langle \\hat n_\\sigma(t)\\hat n_\\sigma(0), \\hat n_\\sigma(t)\\hat n_\\sigma(0)\\rangle  - \\langle \\hat n_\\sigma\\rangle^4|$ (normalized at $t=0$) on a log-log scale.\n}\n\\label{fig:results_nsns}\n\\end{center}\n\\end{figure}\n\nFigure~\\ref{fig:results_nsns} \nshows the results of the OTOC $\\langle \\hat n_\\sigma(t)\\hat n_\\sigma(0), \\hat n_\\sigma(t)\\hat n_\\sigma(0)\\rangle$\nfor the single-orbital Hubbard model. At half filling, this correlation function approaches $\\langle \\hat n_\\sigma\\rangle^4=\\frac{1}{16}$ in the long-time limit and at sufficiently low temperature, \nso that we perform the analytical continuation procedure for $\\langle \\hat n_\\sigma(t)\\hat n_\\sigma(0), \\hat n_\\sigma(t)\\hat n_\\sigma(0)\\rangle-\\langle \\hat n_\\sigma\\rangle^4$. \nThe top panels of Fig.~\\ref{fig:results_nsns} show the real and imaginary parts of \nthis shifted OTOC. \nHere, we only present results for the metallic phase, because resolving the very sharp low-energy feature of the OTOC spectrum in the half-filled Mott state is challenging. One can see that coherent oscillations are not observed for this type of OTOC, and that the incoherent part becomes dominant compared to the other two OTOCs shown in Figs.~\\ref{fig:results_fermion} and \\ref{fig:results_nn}. The amplitude of the OTOC $\\langle \\hat n_\\sigma(t)\\hat n_\\sigma(0), \\hat n_\\sigma(t)\\hat n_\\sigma(0)\\rangle-\\langle \\hat n_\\sigma\\rangle^4$ is enhanced as $U$ increases. As we argue below, this is related to the enhancement of spin fluctuations \nand local moment formation close to the Mott transition.\n\nThe bottom left panel in Fig.~\\ref{fig:results_nsns} plots the analytically continued spectral function $\\mathscr A_{(n_\\sigma n_\\sigma)^2}(\\omega)=-\\frac{1}{\\pi}{\\rm Im}\\, C_{(n_\\sigma n_\\sigma)^2}^R(\\omega)$ multiplied by $\\coth\\big(\\frac{\\beta\\omega}{4}\\big)$. \nWe can see that the spectral weight is concentrated in the low-energy region as we approach the Mott transition point.\nThe inset in the bottom left panel of Fig.~\\ref{fig:results_nsns} shows the imaginary-time four-point function $\\langle \\hat n_\\sigma(\\tau)\\hat n_\\sigma(0), \\hat n_\\sigma(\\tau)\\hat n_\\sigma(0)\\rangle-\\langle \\hat n_\\sigma\\rangle^4$, with the black line corresponding to a Mott insulating solution ($U=6$).  \nFor $U\\gtrsim 4$, the imaginary-time four-point function does not decay to zero but remains relatively large near $\\tau=\\frac{\\beta}{4}$. \nThis behavior is reminiscent of the spin-freezing physics seen in the correlated Hund metal phase of multi-orbital Hubbard models, where the dynamical spin correlation function $\\langle \\hat S_z(\\tau)\\hat S_z(0)\\rangle$ is trapped at a finite value at long $\\tau$ \\cite{Werner2008}. \nIn fact, the OTOC $\\langle \\hat n_\\sigma(\\tau)\\hat n_\\sigma(0), \\hat n_\\sigma(\\tau)\\hat n_\\sigma(0)\\rangle$\nmeasures spin correlations (in addition to charge correlations) through $\\hat n_\\sigma=\\frac{1}{2}\\hat n+\\sigma\\hat S_z$.\nPhysically, the trapping of $\\langle \\hat S_z(\\tau)\\hat S_z(0)\\rangle$ signals the formation of frozen local magnetic moments.\nIn the metal-insulator crossover region, the scattering induced by these magnetic moments results in incoherent metal states. Intuitively, one might expect fast scrambling in the corresponding regions of the phase diagram. \nAlthough there is some resemblance with the spin freezing crossover,\nthe self-energy does not exhibit a square-root frequency dependence in the single-orbital case, and a spin-freezing crossover in the sense of Ref.~\\cite{Werner2008} does not exist.  \n\nThe bottom right panel of Fig.~\\ref{fig:results_nsns} plots the modulus of the OTOC $|\\langle \\hat n_\\sigma(t)\\hat n_\\sigma(0), \\hat n_\\sigma(t)\\hat n_\\sigma(0)\\rangle-\\langle \\hat n_\\sigma\\rangle^4|$ (normalized at $t=0$) on a log-log scale. \nThe dynamics of $\\langle \\hat n_\\sigma(t)\\hat n_\\sigma(0), \\hat n_\\sigma(t)\\hat n_\\sigma(0)\\rangle$ is quite distinct from that of $\\langle \\hat n(t)\\hat n(0), \\hat n(t)\\hat n(0)\\rangle$. As the Mott transition is approached, we observe a slow-down in the decay of the modulus, \nwhich indicates the presence of slowly fluctuating local moments. \nThe long-time behavior may be consistent with a power-law, although slow oscillations make it difficult to determine the long-time asymptotic form from the numerics. \n\n\n\\begin{figure}[t]\n\\begin{center}\n\\includegraphics[angle=-90, width=0.49\\columnwidth]{plot_fitnsns_b2.eps}\n\\includegraphics[angle=-90, width=0.49\\columnwidth]{plot_coeffc_draft.eps}\n\\caption{\nLeft panel: \nThe imaginary-time four-point function $\\langle \\hat n_\\sigma(\\tau)\\hat n_\\sigma(0), \\hat n_\\sigma(\\tau)\\hat n_\\sigma(0)\\rangle$, the squared density-density correlation function $\\langle \\hat n_\\sigma(\\tau)\\hat n_\\sigma(0)\\rangle^2$,\nand their difference for the single-orbital Hubbard model with $U=5.5$ and $\\beta=50$.\nThe dashed curve shows a fit of the difference with the function $f_\\text{fit}(\\tau)=a+b\\tau+c\\tau^2$ in the interval $\\tau/\\beta \\in [0: 0.2]$.\nRight panel: Fitting coefficient $c$ as a function of $U$ for indicated values of $\\beta$.\n\n}\n\\label{fig:coeffs}\n\\end{center}\n\\end{figure}\n\nFor large enough $\\beta$ and small enough $\\tau$, the OTOC (\\ref{otoc_nsns}) factorizes into $\\langle \\hat n_\\sigma(\\tau)\\hat n_\\sigma(0), \\hat n_\\sigma(\\tau)\\hat n_\\sigma(0)\\rangle \\approx \\langle \\hat n_\\sigma(\\tau)\\hat n_\\sigma(0)\\rangle^2$. \nOn the real-time axis, this corresponds to the decoupling $\\langle \\hat n_\\sigma(t)\\hat n_\\sigma(0), \\hat n_\\sigma(t)\\hat n_\\sigma(0)\\rangle \\approx \\langle \\hat n_\\sigma(t)\\hat n_\\sigma(0)\\rangle^2$, that is, \nthe OTOC is reduced to a product of ordinary density-density correlation functions.\nIn order to see whether nontrivial correlations are captured beyond the decoupled form by the OTOC function, \nwe consider the difference $\\langle \\hat n_\\sigma(\\tau)\\hat n_\\sigma(0), \\hat n_\\sigma(\\tau)\\hat n_\\sigma(0)\\rangle-\\langle \\hat n_\\sigma(\\tau)\\hat n_\\sigma(0)\\rangle^2$ and fit the short-time behavior of this function to $f_\\text{fit}(\\tau)=a+b\\tau+c\\tau^2$, as illustrated in the left panel of Fig.~\\ref{fig:coeffs}. The coefficients $b$ and $c$ serve as  indicators of the nontrivial correlations that cannot be attributed to the decoupled form of the OTOC.\nThe coefficient $c$ is plotted as a function of $U$ and for different $\\beta$ in the right hand panel. The coefficient $b$ (not shown) exhibits a similar trend. We notice that the OTOC picks up nontrival correlations near the metal-insulator transition, and in particular at \nintermediate\ntemperatures, while the Mott phase shows no such correlations. The crossover region  \nassociated with the emergence of local moments \ncorresponds, roughly, to the interaction range where the nontrivial correlations start to become significant. \n\n\n\\subsection{Two-orbital Hubbard model}\n\\label{sec:two-orbital hubbard}\n\nIn this section, we investigate OTOCs in the two-orbital Hubbard model with Hund coupling $J>0$ \nand density-density interactions. (Three-orbital results are presented in Appendix~\\ref{sec:3orbital}.) \nThe Hamiltonian is given by\n\\begin{align}\nH\n&=\n-v\\sum_{\\langle i,j\\rangle\\alpha\\sigma} (c^\\dagger_{i\\alpha\\sigma}c_{j\\alpha\\sigma}+ \\text{h.c.})\n-\\mu\\sum_{i\\alpha\\sigma} \\hat n_{i\\alpha\\sigma}\n+U\\sum_{i\\alpha} \\hat n_{i\\alpha\\uparrow}\\hat n_{i\\alpha\\downarrow}\n+(U-2J)\\sum_{i\\sigma} \\hat n_{i1\\sigma}\\hat n_{i2\\bar\\sigma}\n\\notag\n\\\\\n&\\quad\n+(U-3J)\\sum_{i\\sigma} \\hat n_{i1\\sigma}\\hat n_{i2\\sigma},\n\\end{align}\n\nwith $U$ being the intra-orbital interaction $U$, $U-2J$ the inter-orbital antiparallel-spin interaction, and $U-3J$ the inter-orbital parallel-spin interaction. This is the simplest model that shows a crossover from a spin-frozen metal to a Fermi-liquid metal as one dopes the half-filled Mott insulator, with the self-energy scaling as $\\text{Im}\\Sigma(\\omega_n)\\sim\\sqrt{\\omega_n}$ over a significant energy range in the crossover regime \\cite{Hafermann2012}.\nThe sketch of the phase diagram is shown in Fig.~\\ref{fig:illustration}. \nAs we have seen in the previous section, the spin-related OTOC of the type (\\ref{otoc_nsns}) is relevant for the analysis of  the spin-freezing crossover, so that we concentrate on this OTOC here.\nIn the following calculations, we choose $U=8$, $J=U/4$, and compute $\\langle \\hat n_\\sigma(t)\\hat n_\\sigma(0), \\hat n_\\sigma(t)\\hat n_\\sigma(0)\\rangle - \\langle n_\\sigma \\rangle^4$ \nas a function of filling at $\\beta=50$ (dashed lines in Fig.~\\ref{fig:illustration}). \n\n\n\\begin{figure}[t]\n\\begin{center}\n\\includegraphics[angle=-90, width=0.49\\columnwidth]{plot_qp_draft.eps}\\hfill\n\\includegraphics[angle=-90, width=0.49\\columnwidth]{plot_fit_2orbital.eps}\n\\caption{\nLeft panel: Imaginary part of the self-energy as a function of Matsubara frequency for the two-orbital Hubbard model with $U=8$, $J=U/4$, and $\\beta=50$ on a log-log scale, with the dashed line indicating $\\sqrt{\\omega_n}$ behavior. \nRight panel: The coefficient $c$ extracted from\nfitting $\\langle \\hat n_\\sigma(\\tau)\\hat n_\\sigma(0), \\hat n_\\sigma(\\tau)\\hat n_\\sigma(0)\\rangle-\\langle \\hat n_\\sigma(\\tau)\\hat n_\\sigma(0)\\rangle^2$ with a function $f_\\text{fit}(\\tau)=a+b\\tau+c\\tau^2$\nin the interval $\\tau/\\beta\\in [0: 0.05]$ for the two-orbital Hubbard model. \nThe spin-freezing crossover occurs roughly at the filling $n_\\sigma\\approx 0.28$ (blue curve in the left panel and blue arrow in the right panel). \n}\n\\label{fig:self}\n\\end{center}\n\\end{figure}\n\n\nThe left panel of Fig.~\\ref{fig:self} plots the imaginary part of the self-energy as a function of Matsubara frequency in order to identify the filling corresponding to the spin-freezing crossover \\cite{Werner2008}. An approximate square-root scaling is observed over a wide range of frequencies near $n_\\sigma\\approx 0.28$ (blue line). In the right panel of Fig.~\\ref{fig:self}, we present the coefficient $c$ obtained from a similar analysis as presented in Fig.~\\ref{fig:coeffs}, but with a fitting range $\\tau/\\beta\\in [0 : 0.05]$. Again, we find that the OTOC $\\langle \\hat n_\\sigma(t)\\hat n_\\sigma(0), \\hat n_\\sigma(t)\\hat n_\\sigma(0)\\rangle - \\langle \\hat n_\\sigma\\rangle^4$ detects nontrivial correlations in the spin-freezing crossover (blue arrow) and spin-frozen metal regime, but not in the Fermi liquid or the Mott insulating phase.\n\n\n\\begin{figure}[t]\n\\begin{center}\n\\includegraphics[angle=-90, width=0.49\\columnwidth]{plot_otoc_2orbital_Re.eps}\n\\includegraphics[angle=-90, width=0.486\\columnwidth]{plot_otoc_2orbital_Im.eps}\n\\includegraphics[angle=-90, width=0.484\\columnwidth]{plot_spectra_coth_2orbital_inset.eps}\n\\includegraphics[angle=-90, width=0.486\\columnwidth]{plot_otoc_2orbital_modulus_inset.eps}\n\\caption{\nTop panels: Real and imaginary parts of the OTOC $\\langle \\hat n_\\sigma(t)\\hat n_\\sigma(0), \\hat n_\\sigma(t)\\hat n_\\sigma(0)\\rangle  - \\langle \\hat n_\\sigma\\rangle^4$ for the two-orbital Hubbard model with $U=8$, $J=U/4$, $\\beta=50$ and indicated fillings. \nBottom left panel: Spectral function $\\mathscr A_{(n_\\sigma n_\\sigma)^2}(\\omega)=-\\frac{1}{\\pi}{\\rm Im}\\,C_{(n_\\sigma n_\\sigma)^2}^R(\\omega)$ multiplied by $\\coth\\big(\\frac{\\beta\\omega}{4}\\big)$. In the inset, we plot the imaginary-time four-point function $\\langle \\hat n_\\sigma(\\tau)\\hat n_\\sigma(0), \\hat n_\\sigma(\\tau)\\hat n_\\sigma(0)\\rangle-\\langle \\hat n_\\sigma\\rangle^4$. \nBottom right panel: Modulus of the OTOC $|\\langle \\hat n_\\sigma(t)\\hat n_\\sigma(0), \\hat n_\\sigma(t)\\hat n_\\sigma(0)\\rangle  - \\langle \\hat n_\\sigma\\rangle^4|$ (normalized at $t=0$) on a log-log scale.\nIn the inset of the bottom right panel, we illustrate the approximate data collapse obtained by considering the spin-freezing crossover regimes for different values of $J$. \nIn all the panels, the thick blue curves represent the results for the spin-freezing crossover region. \n}\n\\label{fig:2orbital}\n\\end{center}\n\\end{figure}\n\n\nThe top panels of Fig.~\\ref{fig:2orbital} show the real and imaginary parts of the OTOC\n$\\langle \\hat n_\\sigma(t)\\hat n_\\sigma(0), \\hat n_\\sigma(t)\\hat n_\\sigma(0)\\rangle-\\langle \\hat n_\\sigma\\rangle^4$\nfor the two-orbital Hubbard model. In the spin-frozen phase ($n_\\sigma \\gtrsim 0.3$), both components exhibit \nhighly incoherent oscillations, which get suppressed as the filling is reduced. As one moves across the crossover point ($n_\\sigma \\sim 0.28$),\nthe oscillations completely disappear, and the real part of the OTOC starts to overshoot to the negative side\nin the Fermi liquid regime.\nThe bottom left panel in Fig.~\\ref{fig:2orbital} presents the analytically continued spectral function $\\mathscr A_{(n_\\sigma n_\\sigma)^2}(\\omega)=-\\frac{1}{\\pi}{\\rm Im}\\,C_{(n_\\sigma n_\\sigma)^2}^R(\\omega)$ multiplied by $\\coth\\big(\\frac{\\beta\\omega}{4}\\big)$. \nIn the spin-frozen regime ($n_\\sigma \\gtrsim 0.3$) a sharp peak appears at low energy, which originates from the saturation of the imaginary-time four point function $\\langle \\hat n_\\sigma(\\tau)\\hat n_\\sigma(0), \\hat n_\\sigma(\\tau)\\hat n_\\sigma(0)\\rangle$ at large $\\tau$ (see the inset of the bottom left panel of Fig.~\\ref{fig:2orbital}). This low-energy peak represents the slow dynamics of the frozen spins, and translates into a slow long-time decay of $|\\langle \\hat n_\\sigma(t)\\hat n_\\sigma(0), \\hat n_\\sigma(t)\\hat n_\\sigma(0)\\rangle-\\langle \\hat n_\\sigma\\rangle^4|$,\nas shown in the bottom right panel of Fig.~\\ref{fig:2orbital}\non a log-log scale. Again, it is difficult to clearly resolve the long-time behavior of the OTOC due to the limited\naccuracy of the analytic continuation, but in the spin-freezing crossover regime (thick blue curves)\nthe decay of the OTOC is consistent with a power law $\\sim 1/t^{1.5}$ at least for $2\\lesssim t \\lesssim 5$. \nThe shorter-time behavior ($0.5\\lesssim t \\lesssim 2$) is instead well fitted by an exponential decay $\\sim e^{-\\alpha t}$ with a decay constant $\\alpha=0.78$. \nThis exponent exhibits\na strong dependence on the Hund coupling. \n\nIn fact, the OTOCs in the spin-freezing regime for $J=U/4$ and $J=U/6$ can be approximately collapsed by plotting them as a function of $tJ$ (see the inset in the bottom right panel of Fig.~\\ref{fig:2orbital}), which indicates that the Hund coupling is the parameter which controls the dynamics in this regime. \nThis is distinct from the Lyapunov behavior in which the exponential decay scales with $t/\\beta$ \\cite{Bagrets2017}.\nA possible reason is that we are still far from the large $N$ limit (where $N$ corresponds to the number of orbitals times the number of spin degrees of freedom) so that the large $N$ behavior \nsuch as the Lyapunov growth is not observed in our calculations. \nIn fact, the strong dependence of the time scale on $J$ and weak dependence on $\\beta$ is consistent\nwith the behavior of the finite $N$ SYK model in Ref.~\\cite{FuSachdev2016}.\nIn the spin-frozen regime ($n_\\sigma\\gtrsim 0.3$) the decay is much slower than near the crossover point, while in the Fermi liquid regime the decay is accelerated, approaching the free fermion behavior of $t^{-3}$\nas shown in Fig.~\\ref{analytic continuation}. \nQualitatively similar results for this OTOC are obtained \nfor the three-orbital Hubbard model (see Appendix~\\ref{sec:3orbital}).\n\nThe time dependence of the modulus of \n$\\langle \\hat n_\\sigma(\\tau)\\hat n_\\sigma(0), \\hat n_\\sigma(\\tau)\\hat n_\\sigma(0)\\rangle-\\langle \\hat n_\\sigma\\rangle^4$ \nin the spin-freezing crossover regime of multi-orbital Hubbard models (exponential decay crossing over into a power-law) bears a close resemblance to a (different) OTOC for the SYK model discussed in Ref.~\\cite{Bagrets2017}. \nAlso, a recent study of yet another type of OTOCs for the finite-$N$ SYK model found a qualitatively similar decay as observed here in the spin-freezing crossover regime \\cite{FuSachdev2016}. In the following section, we will make the connection to the SYK dynamics more quantitative.   \n\n\n\\subsection{Comparison to the SKY model}\n\\label{sec:SYK}\n\nIn this section, we compare the results of the multi-orbital Hubbard models to the SYK model.\nWe consider the particle-hole symmetric SYK model of complex fermions \\cite{FuSachdev2016},\nwhose Hamiltonian is given by\n\\begin{align}\nH\n&=\n\\frac{1}{(2N)^{3/2}} \\sum_{i,j,k,l=1}^N \nJ_{ij;kl} (c_i^\\dagger c_j^\\dagger c_k c_l\n+\\delta_{ik}n c_j^\\dagger c_l\n-\\delta_{il}n c_j^\\dagger c_k\n-\\delta_{jk}n c_i^\\dagger c_l\n+\\delta_{jl}n c_i^\\dagger c_k).\n\\end{align}\nThe coupling constant $J_{ij;kl}$ is a gaussian random variable, satisfying $J_{ij;kl}=-J_{ji;kl}=-J_{ij;lk}=J_{kl;ij}^\\ast$, and\n\\begin{align}\n\\overline{({\\rm Re}\\, J_{ij;kl})^2}\n&=\n\\begin{cases}\nJ_{\\rm SYK}^2/2 & (i,j)\\neq (k,l) \\\\\nJ_{\\rm SYK}^2 & (i,j)=(k,l)\n\\end{cases},\n\\\\\n\\overline{({\\rm Im}\\, J_{ij;kl})^2}\n&=\n\\begin{cases}\nJ_{\\rm SYK}^2/2 & (i,j)\\neq (k,l) \\\\\n0 & (i,j)=(k,l)\n\\end{cases},\n\\end{align}\nwhere the overline represents an average over each realization of $J_{ij;kl}$. The parameter \n$J_{\\rm SYK}$ ($J_{\\rm SYK}^{-1}$) defines the unit of energy (time) in the following calculations. \nThe system is particle-hole symmetric when\n$n=0.5$, which fixes the total number of fermions as $N^{-1}\\sum_{i=1}^N \\langle c_i^\\dagger c_i\\rangle=0.5$.\nWe take the particle-hole symmetric form of the SYK model because it has been well studied previously \\cite{FuSachdev2016} and avoids the effect of the filling drift.\nWe numerically solve the model by exact diagonalization for finite $N(\\le 12)$, and evaluate the density-density OTOC\n$\\overline{\\langle \\hat n_i(t)\\hat n_i(0), \\hat n_i(t)\\hat n_i(0)\\rangle-\\langle \\hat n_i\\rangle^4}$\nand its imaginary-time counterpart.\n\n\\begin{figure}[t]\n\\begin{center}\n\n\\includegraphics[angle=-90,width=0.49\\columnwidth]{plot_fitnsns_syk.eps}\n\n\\includegraphics[angle=-90,width=0.49\\columnwidth]{plot_coeffc_syk.eps}\n\\caption{\nLeft panel: \nThe imaginary-time four-point function $\\overline{\\langle \\hat n_i(\\tau)\\hat n_i(0), \\hat n_i(\\tau)\\hat n_i(0)\\rangle}$, the density-density correlation function $\\overline{\\langle \\hat n_i(\\tau)\\hat n_i(0)\\rangle^2}$,\nand their difference for the SYK model with $N=12$ and $\\beta J_{\\rm SYK}=10$.\nThe dashed curve shows a fit of the difference with a function $f_\\text{fit}(\\tau)=a+b\\tau+c\\tau^2$ in the interval $\\tau/\\beta \\in [0: 0.2]$. \nRight panel: Fitting coefficient $c$ as a function of $T/J_{\\rm SYK}$.\n}\n\\label{fig:SYK fit}\n\\end{center}\n\\end{figure}\n\nIn the left panel of Fig.~\\ref{fig:SYK fit}, we plot the imaginary-time four-point function \n$\\overline{\\langle \\hat n_i(\\tau)\\hat n_i(0), \\hat n_i(\\tau)\\hat n_i(0)\\rangle}$\nfor the SYK model with $N=12$ and $\\beta J_{\\rm SYK}=10$. \nAt low temperature, we expect that $\\overline{\\langle \\hat n_i(\\tau)\\hat n_i(0), \\hat n_i(\\tau)\\hat n_i(0)\\rangle}$ is decoupled into $\\overline{\\langle \\hat n_i(\\tau)\\hat n_i(0)\\rangle^2}$.\nTo see whether nontrivial correlations beyond the decoupled form are present, we fit the difference\n$\\overline{\\langle \\hat n_i(\\tau)\\hat n_i(0), \\hat n_i(\\tau)\\hat n_i(0)\\rangle}\n-\\overline{\\langle \\hat n_i(\\tau)\\hat n_i(0)\\rangle^2}$ with a function $f_{\\rm fit}(\\tau)=a+b\\tau+c\\tau^2$\nin the interval $\\tau/\\beta\\in[0:0.2]$, in the same manner as for the Hubbard models.\nThe obtained coefficient $c$ is plotted as a function of the temperature $T$ in the right panel of\nFig.~\\ref{fig:SYK fit}. One can see that nontrivial correlations exist in a wide range of temperature.\nEspecially, they are enhanced in the intermediate temperature regime. This is because in the zero-temperature limit\nthe four-point function is decoupled as \n$\\langle \\hat n_i(\\tau)\\hat n_i(0), \\hat n_i(\\tau)\\hat n_i(0)\\rangle \\approx \\langle \\hat n_i(\\tau)\\hat n_i(0)\\rangle^2$, while in the high-temperature limit ($\\beta\\to 0$) the imaginary-time dependence is washed out.\n\n\n\n\n\\begin{figure}[t]\n\\begin{center}\n\\includegraphics[angle=-90, width=0.49\\columnwidth]{plot_syk_Re.eps}\n\\includegraphics[angle=-90, width=0.49\\columnwidth]{plot_syk_Im.eps}\n\\includegraphics[angle=-90, width=0.49\\columnwidth]{plot_syk_modulus-log.eps}\n\\includegraphics[angle=-90, width=0.49\\columnwidth]{plot_syk_modulus.eps}\n\\caption{\nThe density-density OTOC $\\overline{\\langle \\hat n_i(t)\\hat n_i(0), \\hat n_i(t)\\hat n_i(0)\\rangle-\\langle \\hat n_i\\rangle^4}$ for the particle-hole symmetric SYK model averaged over $10^2$ samples with $\\beta J_{\\rm SYK}=100$ and various $N$. \nThe top left and right panels show the real and imaginary parts of the OTOC,\nwhile the bottom left and right panels show the log and log-log plots of the modulus of the OTOC (normalized at $t=0$).\nThe dashed line shows a power-law decay $\\propto t^{-1.6}$ for comparison.}\n\\label{fig:SYK}\n\\end{center}\n\\end{figure}\n\nIn Fig.~\\ref{fig:SYK}, we show the numerical results of the density-density OTOC for $\\beta J_{\\rm SYK}=100$.\nThe real part of the OTOC rapidly drops from the initial value within the time scale of $tJ_{\\rm SYK}\\sim 2$,\nand slowly decays to zero in the long time limit.\nAs one increases $N$, the oscillations that appear at longer time tend to be suppressed.\nOne can see that the result for $N=12$ in the top panels of Fig.~\\ref{fig:SYK} closely resembles the OTOC in \nthe spin-freezing crossover regime of the two-orbital Hubbard model shown\nby the blue curve in the top panels of Fig.~\\ref{fig:2orbital},\nwhile it differs from the single-orbital result shown in the top panels of Fig.~\\ref{fig:results_nsns}.\nDue to the limitation of our calculations to small system sizes ($N\\le 12$), we do not clearly observe\nan exponential growth ($\\sim c_0-c_1 e^{\\lambda t}$)  at short times or an exponential decay ($\\sim e^{-\\alpha t}$) \nat intermediate times. \nHowever, we find that for $tJ_{\\rm SYK}\\gtrsim 2$ the OTOC decays\napproximately in a power law $\\propto t^{-\\gamma}$ with $\\gamma=1.6$ (see bottom right panel in Fig.~\\ref{fig:SYK}). \nThe time interval exhibiting the power-law decay becomes longer as we increase $N$.  \nThe temperature dependence of the OTOC for the SYK model is shown in Fig.~\\ref{fig:SYK-T}.\nThe time scale of the initial drop is more or less independent of the temperature, while \nthe power-law-like decay is only visible in the low-temperature regime.\n\n\\begin{figure}[t]\n\\begin{center}\n\\includegraphics[angle=-90, width=0.49\\columnwidth]{plot_syk_beta_re.eps}\n\\includegraphics[angle=-90, width=0.49\\columnwidth]{plot_syk_beta_im.eps}\n\\includegraphics[angle=-90, width=0.49\\columnwidth]{plot_syk_beta_modulus-log.eps}\n\\includegraphics[angle=-90, width=0.49\\columnwidth]{plot_syk_beta_modulus.eps}\n\\caption{\nTemperature dependence of the density-density OTOC\n$\\overline{\\langle \\hat n_i(t)\\hat n_i(0), \\hat n_i(t)\\hat n_i(0)\\rangle-\\langle \\hat n_i\\rangle^4}$ for the particle-hole symmetric SYK model averaged over $10^2$ samples with $N=12$.\nThe top left and right panels show the real and imaginary parts of the OTOC,\nwhile the bottom left and right panels show the log and log-log plots of the modulus of the OTOC.\nThe dashed line shows a power-law decay $\\propto t^{-1.6}$ for comparison.}\n\\label{fig:SYK-T}\n\\end{center}\n\\end{figure}\n\nIn Ref.~\\onlinecite{Werner2018}, it has been argued that the spin-freezing crossover regime of multi-orbital Hubbard models is effectively described by the SYK model in which $J_{\\rm SYK}$ is replaced by the Hund coupling $J$. The results in Fig.~\\ref{fig:2orbital} are for the two-orbital Hubbard model with $U=8$, $J/U=1/4$ and $\\beta=50$, corresponding to $\\beta J=100$, which means that it is meaningful to directly compare the blue lines in Fig.~\\ref{fig:2orbital} with the blue lines in Figs.~\\ref{fig:SYK} and \\ref{fig:SYK-T}. Not only is the qualitative agreement remarkable, even the power-law exponent measured for the two-orbital Hubbard model ($\\gamma=1.5$) is very close to the exponent extracted from the finite-$N$ SYK calculation. \nThe power-law decay in the three-orbital case is approximately $1/t^{1.75}$ (see Appendix~\\ref{sec:3orbital}), and thus also in semi-quantitative agreement with the SYK model behavior. \nThis nontrivial result provides further support for the identification of the spin-freezing crossover regime of multi-orbital Hubbard systems with an SYK strange metal. \n\nIn the large-$N$ and low-$T$ limit of the SYK model, we expect that the power-law behavior approaches $1/t^2$, since in this limit the OTOC is decoupled as $\\langle \\hat n_i(t)\\hat n_i(0), \\hat n_i(t)\\hat n_i(0)\\rangle\\approx \\langle \\hat n_i(t)\\hat n_i(0)\\rangle^2$ and the decay of $\\langle \\hat n_i(t)\\hat n_i(0)\\rangle$ is dominated by what corresponds to the slow spin relaxation $\\langle \\hat S_z(t)\\hat S_z(0)\\rangle\\sim 1/t$ in the Sachdev-Ye model.\n\n\n\n\n\n4.2 Single-orbital Hubbard model\n\\subsection{Single-orbital Hubbard model}\n\\label{sec:hubbard}\n\nIn this section, we calculate the OTOCs (\\ref{fermion_otoc})-(\\ref{otoc_nn}) for the interacting single-orbital Hubbard model with the Hamiltonian \n\\begin{align}\nH\n&=\n-v\\sum_{\\langle i,j\\rangle\\sigma} (c^\\dagger_{i\\sigma}c_{j\\sigma}+ \\text{h.c.})\n-\\mu\\sum_i \\hat n_i+U\\sum_i \\hat n_{i\\uparrow}\\hat n_{i\\downarrow}, \n\\end{align}\nwhere $v$ is the hopping amplitude, $\\langle i,j\\rangle$ represents the nearest-neighbor site pairs, \n$\\mu$ is the chemical potential, and $U$ is the on-site interaction.\nIn this section, we focus on the half-filled case (i.e., $\\mu=U/2$). The DMFT solution for the simplified self-consistency (\\ref{semicirc}) is the exact solution for an infinitely connected Bethe lattice \\cite{GeorgesKotliarKrauthRozenberg1996}.\nLet us briefly recall the paramagnetic phase diagram for the single-orbital Hubbard model on this lattice \\cite{Bluemer_thesis}. At half-filling, there is a metal-insulator crossover at high temperature and a first-order Mott transition below a temperature corresponding to $\\beta\\hbar\\approx 17$ with a coexistence region between $U_{c1}$ and $U_{c2}$. The finite-temperature critical endpoint is at $U\\approx 4.7$, while the zero-temperature Mott transition occurs at $U=U_{c2}\\approx 5.6$. \nAt low temperature, the self-energy shows the Fermi liquid behavior [${\\rm Im}\\,\\Sigma(\\omega)\\sim \\omega^2$] \nin the weakly correlated metallic phase, \nwhile it shows an insulating behavior [${\\rm Im}\\,\\Sigma(\\omega)\\sim \\delta(\\omega)$] in the Mott phase\n\\cite{GeorgesKotliarKrauthRozenberg1996}. \nIn the correlated metallic phase, as the temperature is increased, deviations from the Fermi liquid behavior become apparent, \nbut an ${\\rm Im}\\,\\Sigma(\\omega)\\sim \\sqrt{\\omega}$ scaling as in the SYK model is not observed. \n\nIn the following, we will start the DMFT iterations from the noninteracting solution, which means that the finite-temperature Mott transition occurs at $U_{c2}$.\nFrom now on, we set $\\hbar=1$. \n\n\\begin{figure}[t]\n\\begin{center}\n\\includegraphics[angle=-90, width=0.49\\columnwidth]{plot_otoc_b50_Re.eps}\n\\includegraphics[angle=-90, width=0.49\\columnwidth]{plot_otoc_b50_Im.eps}\n\\includegraphics[angle=-90, width=0.49\\columnwidth]{plot_coth_b50_appendix.eps}\n\\includegraphics[angle=-90, width=0.49\\columnwidth]{plot_otoc_b50_modulus.eps}\n\\caption{\nTop panels: Real and imaginary parts of the OTOC $\\langle c^\\dagger_\\sigma(t)c_\\sigma(0), c^\\dagger_\\sigma(t)c_\\sigma(0)\\rangle$ for the single-orbital Hubbard model with $\\beta=50$ at half filling. The thin lines correspond to metallic solutions, while the thick line ($U=6$) corresponds to a Mott insulating solution. \nBottom left panel: Spectral function $\\mathscr A_{(c_\\sigma^\\dagger c_\\sigma)^2}(\\omega)=-\\frac{1}{\\pi}{\\rm Im}\\,C_{(c^\\dagger_\\sigma c_\\sigma)^2}^R(\\omega)$ multiplied by $\\coth\\big(\\frac{\\beta\\omega}{4}\\big)$. \nBottom right panel: Modulus of the OTOC $|\\langle c^\\dagger_\\sigma(t)c_\\sigma(0), c^\\dagger_\\sigma(t)c_\\sigma(0)\\rangle|$ (normalized at $t=0$) on a log-log scale. The dashed line is the result for $U=0$.\n}\n\\label{fig:results_fermion}\n\\end{center}\n\\end{figure}\n\n\n\nIn Fig.~\\ref{fig:results_fermion}, we show the results of $\\langle c^\\dagger_\\sigma(t)c_\\sigma(0), c^\\dagger_\\sigma(t)c_\\sigma(0)\\rangle$ for the interacting single-orbital Hubbard model at $\\beta=50$. \nThe top panels present the real and imaginary parts of the OTOC. One can see that the oscillations become more pronounced as one increases the interaction. In the bottom left panel of Fig.~\\ref{fig:results_fermion},\nwe show the corresponding spectral function $\\mathscr A_{(c_\\sigma^\\dagger c_\\sigma)^2}(\\omega)=-\\frac{1}{\\pi}{\\rm Im}\\,C_{(c^\\dagger_\\sigma c_\\sigma)^2}^R(\\omega)$ multiplied by $\\coth\\big(\\frac{\\beta\\omega}{4}\\big)$.\nThe Mott transition, which occurs between $U=5$ and $U=6$ at $\\beta=50$, manifests itself by the opening of a gap in the spectral function $\\mathscr A_{(c_\\sigma^\\dagger c_\\sigma)^2(\\omega)}$. This translates into more weakly damped oscillations in the real-time evolution in the Mott phase.  \nThe bottom right panel of Fig.~\\ref{fig:results_fermion} plots the modulus of the OTOC (normalized at time $t=0$, i.e., $|\\langle c^\\dagger_\\sigma(t)c_\\sigma(0), c^\\dagger_\\sigma(t)c_\\sigma(0)\\rangle|/|\\langle c^\\dagger_\\sigma(0)c_\\sigma(0), c^\\dagger_\\sigma(0)c_\\sigma(0)\\rangle|$) on a log-log scale. \nDue to the oscillations and the limited accuracy of the analytic continuation procedure, it is hard to clearly identify the nature of the long-time decay of the OTOC,\nbut the results indicate that the OTOC decays much faster (possibly exponentially) in the Mott phase than in the metallic phase. This is consistent with the results for the spectral functions of the OTOC. The comparison to the noninteracting case (dashed curve in the bottom right panel of Fig.~\\ref{fig:results_fermion}) suggests that the correlated metallic phase has a similar (possibly power-law) decay behavior of the OTOC as in the noninteracting case.\n\n\n\\begin{figure}[t]\n\\begin{center}\n\\includegraphics[angle=-90, width=0.49\\columnwidth]{plot_otoc_b50_nn_Re.eps}\n\\includegraphics[angle=-90, width=0.49\\columnwidth]{plot_otoc_b50_nn_Im.eps}\n\\includegraphics[angle=-90, width=0.49\\columnwidth]{plot_coth_b50_nn_appendix.eps}\n\\includegraphics[angle=-90, width=0.49\\columnwidth]{plot_otoc_b50_nn_modulus.eps}\n\\caption{\nTop panels: Real and imaginary parts of the OTOC $\\langle \\hat n(t)\\hat n(0), \\hat n(t)\\hat n(0)\\rangle - 1$ for the single-orbital Hubbard model with $\\beta=50$ at half filling. The thin lines correspond to metallic solutions, \nwhile the thick line ($U=6$) corresponds to a Mott insulating solution. \nBottom left panel: Spectral function $\\mathscr A_{(nn)^2}(\\omega)=-\\frac{1}{\\pi}{\\rm Im}\\,C_{(n n)^2}^R(\\omega)$ multiplied by $\\coth\\big(\\frac{\\beta\\omega}{4}\\big)$.\nBottom right panel: Modulus of the OTOC $|\\langle \\hat n(t)\\hat n(0), \\hat n(t)\\hat n(0)\\rangle - 1|$ (normalized at $t=0$) on a log-log scale. \n}\n\\label{fig:results_nn}\n\\end{center}\n\\end{figure}\n\n\nIn Fig.~\\ref{fig:results_nn}, we plot the OTOC $\\langle \\hat n(t)\\hat n(0), \\hat n(t)\\hat n(0)\\rangle$\nfor the single-orbital Hubbard model in a similar manner as in Fig.~\\ref{fig:results_fermion}.\nSince at half filling this correlation function approaches $\\langle \\hat n\\rangle^4=1$ at long times\nand at sufficiently low temperature,\nwe perform the analytical continuation procedure for $\\langle \\hat n(t)\\hat n(0), \\hat n(t)\\hat n(0)\\rangle-1$. \nThe top panels of Fig.~\\ref{fig:results_nn} show the real and imaginary parts of this shifted OTOC. \nContrary to the case of $\\langle c_\\sigma^\\dagger(t)c_\\sigma(0), c_\\sigma^\\dagger(t)c_\\sigma(0)\\rangle$, \nthe amplitude of the OTOC $\\langle \\hat n(t)\\hat n(0), \\hat n(t)\\hat n(0)\\rangle-1$ is suppressed as one increases the interaction, which reflects the reduced charge fluctuations in the Mott insulator.\nThe bottom left panel in Fig.~\\ref{fig:results_nn} shows the analytically continued spectral function $\\mathscr A_{(nn)^2}(\\omega)=-\\frac{1}{\\pi}{\\rm Im}\\, C_{(nn)^2}^R(\\omega)$ multiplied by $\\coth\\big(\\frac{\\beta\\omega}{4}\\big)$. \nAgain we notice the opening of a gap in the Mott phase, and a corresponding shift of the spectral weight to higher energies. This results in more rapid oscillations of the OTOC in the Mott state.\nThe bottom right panel in Fig.~\\ref{fig:results_nn} plots the modulus of the OTOC $|\\langle \\hat n(t)\\hat n(0), \\hat n(t)\\hat n(0)\\rangle-1|$ (normalized at $t=0$) on a log-log scale.\nThe long-time behavior of the OTOC is consistent with a power-law decay in the metallic phase, and an exponential decay in the Mott insulating phase.\n\n\n\\begin{figure}[t]\n\\begin{center}\n\\includegraphics[angle=-90, width=0.49\\columnwidth]{plot_otoc_nsns_b50_draft_Re.eps}\n\\includegraphics[angle=-90, width=0.49\\columnwidth]{plot_otoc_nsns_b50_draft_Im.eps}\n\\includegraphics[angle=-90, width=0.49\\columnwidth]{plot_coth_b50_nsns_inset.eps}\n\\includegraphics[angle=-90, width=0.49\\columnwidth]{plot_otoc_nsns_b50_draft_modulus_zoom.eps}\n\\caption{\nTop panels: Real and imaginary parts of the OTOC $\\langle \\hat n_\\sigma(t)\\hat n_\\sigma(0), \\hat n_\\sigma(t)\\hat n_\\sigma(0)\\rangle  - \\langle \\hat n_\\sigma\\rangle^4$ for the single-orbital Hubbard model with $\\beta=50$ at half filling. All the colored lines correspond to metallic solutions. \nBottom left panel: Spectral function $\\mathscr A_{(n_\\sigma n_\\sigma)^2}(\\omega)=-\\frac{1}{\\pi}{\\rm Im}\\,C_{(n_\\sigma n_\\sigma)^2}^R(\\omega)$ multiplied by $\\coth\\big(\\frac{\\beta\\omega}{4}\\big)$. In the inset, we plot the imaginary-time four-point function $\\langle \\hat n_\\sigma(\\tau)\\hat n_\\sigma(0), \\hat n_\\sigma(\\tau)\\hat n_\\sigma(0)\\rangle-\\langle \\hat n_\\sigma\\rangle^4$, with the black line corresponding to the Mott insulating solution at $U=6$.  \nBottom right panel: Modulus of the OTOC $|\\langle \\hat n_\\sigma(t)\\hat n_\\sigma(0), \\hat n_\\sigma(t)\\hat n_\\sigma(0)\\rangle  - \\langle \\hat n_\\sigma\\rangle^4|$ (normalized at $t=0$) on a log-log scale.\n}\n\\label{fig:results_nsns}\n\\end{center}\n\\end{figure}\n\nFigure~\\ref{fig:results_nsns} \nshows the results of the OTOC $\\langle \\hat n_\\sigma(t)\\hat n_\\sigma(0), \\hat n_\\sigma(t)\\hat n_\\sigma(0)\\rangle$\nfor the single-orbital Hubbard model. At half filling, this correlation function approaches $\\langle \\hat n_\\sigma\\rangle^4=\\frac{1}{16}$ in the long-time limit and at sufficiently low temperature, \nso that we perform the analytical continuation procedure for $\\langle \\hat n_\\sigma(t)\\hat n_\\sigma(0), \\hat n_\\sigma(t)\\hat n_\\sigma(0)\\rangle-\\langle \\hat n_\\sigma\\rangle^4$. \nThe top panels of Fig.~\\ref{fig:results_nsns} show the real and imaginary parts of \nthis shifted OTOC. \nHere, we only present results for the metallic phase, because resolving the very sharp low-energy feature of the OTOC spectrum in the half-filled Mott state is challenging. One can see that coherent oscillations are not observed for this type of OTOC, and that the incoherent part becomes dominant compared to the other two OTOCs shown in Figs.~\\ref{fig:results_fermion} and \\ref{fig:results_nn}. The amplitude of the OTOC $\\langle \\hat n_\\sigma(t)\\hat n_\\sigma(0), \\hat n_\\sigma(t)\\hat n_\\sigma(0)\\rangle-\\langle \\hat n_\\sigma\\rangle^4$ is enhanced as $U$ increases. As we argue below, this is related to the enhancement of spin fluctuations \nand local moment formation close to the Mott transition.\n\nThe bottom left panel in Fig.~\\ref{fig:results_nsns} plots the analytically continued spectral function $\\mathscr A_{(n_\\sigma n_\\sigma)^2}(\\omega)=-\\frac{1}{\\pi}{\\rm Im}\\, C_{(n_\\sigma n_\\sigma)^2}^R(\\omega)$ multiplied by $\\coth\\big(\\frac{\\beta\\omega}{4}\\big)$. \nWe can see that the spectral weight is concentrated in the low-energy region as we approach the Mott transition point.\nThe inset in the bottom left panel of Fig.~\\ref{fig:results_nsns} shows the imaginary-time four-point function $\\langle \\hat n_\\sigma(\\tau)\\hat n_\\sigma(0), \\hat n_\\sigma(\\tau)\\hat n_\\sigma(0)\\rangle-\\langle \\hat n_\\sigma\\rangle^4$, with the black line corresponding to a Mott insulating solution ($U=6$).  \nFor $U\\gtrsim 4$, the imaginary-time four-point function does not decay to zero but remains relatively large near $\\tau=\\frac{\\beta}{4}$. \nThis behavior is reminiscent of the spin-freezing physics seen in the correlated Hund metal phase of multi-orbital Hubbard models, where the dynamical spin correlation function $\\langle \\hat S_z(\\tau)\\hat S_z(0)\\rangle$ is trapped at a finite value at long $\\tau$ \\cite{Werner2008}. \nIn fact, the OTOC $\\langle \\hat n_\\sigma(\\tau)\\hat n_\\sigma(0), \\hat n_\\sigma(\\tau)\\hat n_\\sigma(0)\\rangle$\nmeasures spin correlations (in addition to charge correlations) through $\\hat n_\\sigma=\\frac{1}{2}\\hat n+\\sigma\\hat S_z$.\nPhysically, the trapping of $\\langle \\hat S_z(\\tau)\\hat S_z(0)\\rangle$ signals the formation of frozen local magnetic moments.\nIn the metal-insulator crossover region, the scattering induced by these magnetic moments results in incoherent metal states. Intuitively, one might expect fast scrambling in the corresponding regions of the phase diagram. \nAlthough there is some resemblance with the spin freezing crossover,\nthe self-energy does not exhibit a square-root frequency dependence in the single-orbital case, and a spin-freezing crossover in the sense of Ref.~\\cite{Werner2008} does not exist.  \n\nThe bottom right panel of Fig.~\\ref{fig:results_nsns} plots the modulus of the OTOC $|\\langle \\hat n_\\sigma(t)\\hat n_\\sigma(0), \\hat n_\\sigma(t)\\hat n_\\sigma(0)\\rangle-\\langle \\hat n_\\sigma\\rangle^4|$ (normalized at $t=0$) on a log-log scale. \nThe dynamics of $\\langle \\hat n_\\sigma(t)\\hat n_\\sigma(0), \\hat n_\\sigma(t)\\hat n_\\sigma(0)\\rangle$ is quite distinct from that of $\\langle \\hat n(t)\\hat n(0), \\hat n(t)\\hat n(0)\\rangle$. As the Mott transition is approached, we observe a slow-down in the decay of the modulus, \nwhich indicates the presence of slowly fluctuating local moments. \nThe long-time behavior may be consistent with a power-law, although slow oscillations make it difficult to determine the long-time asymptotic form from the numerics. \n\n\n\\begin{figure}[t]\n\\begin{center}\n\\includegraphics[angle=-90, width=0.49\\columnwidth]{plot_fitnsns_b2.eps}\n\\includegraphics[angle=-90, width=0.49\\columnwidth]{plot_coeffc_draft.eps}\n\\caption{\nLeft panel: \nThe imaginary-time four-point function $\\langle \\hat n_\\sigma(\\tau)\\hat n_\\sigma(0), \\hat n_\\sigma(\\tau)\\hat n_\\sigma(0)\\rangle$, the squared density-density correlation function $\\langle \\hat n_\\sigma(\\tau)\\hat n_\\sigma(0)\\rangle^2$,\nand their difference for the single-orbital Hubbard model with $U=5.5$ and $\\beta=50$.\nThe dashed curve shows a fit of the difference with the function $f_\\text{fit}(\\tau)=a+b\\tau+c\\tau^2$ in the interval $\\tau/\\beta \\in [0: 0.2]$.\nRight panel: Fitting coefficient $c$ as a function of $U$ for indicated values of $\\beta$.\n\n}\n\\label{fig:coeffs}\n\\end{center}\n\\end{figure}\n\nFor large enough $\\beta$ and small enough $\\tau$, the OTOC (\\ref{otoc_nsns}) factorizes into $\\langle \\hat n_\\sigma(\\tau)\\hat n_\\sigma(0), \\hat n_\\sigma(\\tau)\\hat n_\\sigma(0)\\rangle \\approx \\langle \\hat n_\\sigma(\\tau)\\hat n_\\sigma(0)\\rangle^2$. \nOn the real-time axis, this corresponds to the decoupling $\\langle \\hat n_\\sigma(t)\\hat n_\\sigma(0), \\hat n_\\sigma(t)\\hat n_\\sigma(0)\\rangle \\approx \\langle \\hat n_\\sigma(t)\\hat n_\\sigma(0)\\rangle^2$, that is, \nthe OTOC is reduced to a product of ordinary density-density correlation functions.\nIn order to see whether nontrivial correlations are captured beyond the decoupled form by the OTOC function, \nwe consider the difference $\\langle \\hat n_\\sigma(\\tau)\\hat n_\\sigma(0), \\hat n_\\sigma(\\tau)\\hat n_\\sigma(0)\\rangle-\\langle \\hat n_\\sigma(\\tau)\\hat n_\\sigma(0)\\rangle^2$ and fit the short-time behavior of this function to $f_\\text{fit}(\\tau)=a+b\\tau+c\\tau^2$, as illustrated in the left panel of Fig.~\\ref{fig:coeffs}. The coefficients $b$ and $c$ serve as  indicators of the nontrivial correlations that cannot be attributed to the decoupled form of the OTOC.\nThe coefficient $c$ is plotted as a function of $U$ and for different $\\beta$ in the right hand panel. The coefficient $b$ (not shown) exhibits a similar trend. We notice that the OTOC picks up nontrival correlations near the metal-insulator transition, and in particular at \nintermediate\ntemperatures, while the Mott phase shows no such correlations. The crossover region  \nassociated with the emergence of local moments \ncorresponds, roughly, to the interaction range where the nontrivial correlations start to become significant. \n\n\n",
        "label": "fig:results_nn",
        "Descriptive_question1": "What is the value of beta used in the single-orbital Hubbard model shown in figure_6?",
        "Descriptive_question2": "What does the thick line represent in the top panels of figure_6?",
        "Reasoning_question1": "How does the amplitude of the OTOC in figure_6 change with increasing interaction strength, and what does this indicate about charge fluctuations in the Mott insulator?",
        "Reasoning_question2": "What can be inferred about the long-time decay behavior of the OTOC in the metallic versus the Mott insulating phase from the bottom right panel of figure_6?",
        "Descriptive_answer1": "β=50",
        "Descriptive_answer2": "Mott insulating solution",
        "Reasoning_answer1": "In figure_6, as the interaction strength U increases, particularly up to U=6, the amplitude of the OTOC ⟨n(t)n(0), n(t)n(0)⟩ - 1 decreases. This suppression of amplitude with increasing interaction strength indicates a reduction in charge fluctuations, which is characteristic of the Mott insulating phase where electrons become localized and charge fluctuations are suppressed.",
        "Reasoning_answer2": "From the bottom right panel of figure_6, the modulus of the OTOC normalized at t=0 shows a decay over time. In the metallic phase (thin lines), the decay is consistent with a power-law trend, indicating slower decay and sustained correlations over longer times. In contrast, in the Mott insulating phase (thick line, U=6), the decay is more rapid and appears exponential, reflecting faster loss of correlation. Thus, the long-time decay of the OTOC is slower and power-law-like in the metallic phase and faster, possibly exponential, in the Mott insulator phase."
    },
    {
        "paper_id": "1812.04217.json",
        "image_id": "figure_7",
        "image_path": [
            "/home/yz979/scratch/chengye/scimolmo/test-images-tables/dataset_imagestables/1812.04217/images/plot_otoc_nsns_b50_draft_Re.eps"
        ],
        "caption": "\nTop panels: Real and imaginary parts of the OTOC $\\langle \\hat n_\\sigma(t)\\hat n_\\sigma(0), \\hat n_\\sigma(t)\\hat n_\\sigma(0)\\rangle  - \\langle \\hat n_\\sigma\\rangle^4$ for the single-orbital Hubbard model with $\\beta=50$ at half filling. All the colored lines correspond to metallic solutions. \nBottom left panel: Spectral function $\\mathscr A_{(n_\\sigma n_\\sigma)^2}(\\omega)=-\\frac{1}{\\pi}{\\rm Im}\\,C_{(n_\\sigma n_\\sigma)^2}^R(\\omega)$ multiplied by $\\coth\\big(\\frac{\\beta\\omega}{4}\\big)$. In the inset, we plot the imaginary-time four-point function $\\langle \\hat n_\\sigma(\\tau)\\hat n_\\sigma(0), \\hat n_\\sigma(\\tau)\\hat n_\\sigma(0)\\rangle-\\langle \\hat n_\\sigma\\rangle^4$, with the black line corresponding to the Mott insulating solution at $U=6$.  \nBottom right panel: Modulus of the OTOC $|\\langle \\hat n_\\sigma(t)\\hat n_\\sigma(0), \\hat n_\\sigma(t)\\hat n_\\sigma(0)\\rangle  - \\langle \\hat n_\\sigma\\rangle^4|$ (normalized at $t=0$) on a log-log scale.\n",
        "classify": "Chart",
        "section_info": "4 Numerical results for the Hubbard model\n\\section{Numerical results for the Hubbard model}\n\\label{numerical results}\n\n\\subsection{Observables and numerical procedure}\n\nWe consider the single-orbital, two-orbital, and three-orbital Hubbard models on an infinitely-connected Bethe lattice, which can be solved exactly within DMFT \\cite{GeorgesKotliarKrauthRozenberg1996}. In this case, the noninteracting density of states is semicircular with bandwidth $4v_\\ast$, for which there exists a simplified DMFT self-consistency condition between the hybridization function $\\Delta_{\\alpha\\sigma}$ of the DMFT impurity problem and the local (impurity) Green's function $G_{\\alpha\\sigma}$: \n\n\\begin{equation}\n\\Delta_{\\alpha\\sigma}(\\tau)=v_\\ast^2 G_{\\alpha\\sigma}(\\tau), \n\\label{semicirc}\n\\end{equation}\n\nwith $\\alpha$ the orbital and $\\sigma$ the spin index. We will use $v_\\ast$ as the unit of energy and measure time in units of $\\hbar/v_\\ast$. \n\nThree different types of imaginary-time four-point functions $C_{(AB)^2}^M(\\tau)$\nof the form of Eq.~(\\ref{imaginary-time OTOC})\nwith $(\\hat A, \\hat B)=(c_\\sigma^\\dagger, c_\\sigma), (\\hat n_\\sigma, \\hat n_\\sigma)$, and $(\\hat n, \\hat n)$\nare calculated in the interval $0\\le \\tau\\le \\frac{\\beta\\hbar}{2}$: \n\\begin{eqnarray}\nC_{(c^\\dagger_\\sigma c_\\sigma)^2}^M(\\tau) &=& -\\langle c^\\dagger_\\sigma(\\tau+\\tfrac{\\beta\\hbar}{2}) c_\\sigma(\\tfrac{\\beta\\hbar}{2}) c^\\dagger_\\sigma(\\tau) c_\\sigma(0) \\rangle, \\label{fermion_otoc}\\\\\nC_{(n_\\sigma n_\\sigma)^2}^M(\\tau) &=& -\\langle \\hat n_\\sigma(\\tau+\\tfrac{\\beta\\hbar}{2})\\hat n_\\sigma(\\tfrac{\\beta\\hbar}{2})\\hat n_\\sigma(\\tau)\\hat n_\\sigma(0) \\rangle, \\label{otoc_nsns}\\\\\nC_{(nn)^2}^M(\\tau) &=& -\\langle \\hat n(\\tau+\\tfrac{\\beta\\hbar}{2})\\hat n(\\tfrac{\\beta\\hbar}{2})\\hat n(\\tau)\\hat n(0) \\rangle,\\label{otoc_nn}\n\\end{eqnarray}\n\nwhere $c_\\sigma^\\dagger$ ($c_\\sigma$) are the fermionic creation (annihilation) operators for spin $\\sigma$ (and orbital $\\alpha=1$ in the multi-orbital case), $\\hat n_\\sigma=c^\\dagger_\\sigma c_\\sigma$ is the corresponding spin-dependent density operator, and $\\hat n=\\hat n_\\uparrow + \\hat n_\\downarrow$ the total density operator. Note that in all the three cases above we have $\\hat B=\\hat A^\\dagger$. \nWe measure these local correlation functions in the impurity model using a hybridization expansion continuous-time Monte Carlo algorithm (CT-HYB) \\cite{Werner2006}. In this algorithm, the \ntwo-particle\nGreen's functions of the type (\\ref{fermion_otoc}) can be measured by removing two hybridization lines, i.e., from the elements of the inverse hybridization matrix, while the density-density correlation functions can be easily measured either by insertion of density operators (matrix formalism) \\cite{Werner2006Kondo} or by reading off the occupation of the orbitals at the four time points in the segment implementation \\cite{Werner2006}. We use the latter algorithm since we consider only density-density interactions. \nThe values at the end-points $\\tau=0$ and $\\tau=\\frac{\\beta\\hbar}{2}$ reduce to standard density-density correlation functions, which in the case of Eq.~(\\ref{fermion_otoc}) are measured separately. \n\nWe perform the analytic continuation to the real-frequency axis using the Maximum Entropy method \\cite{Jarrell1996,Lewin_maxent} with a bosonic kernel. This yields the imaginary part of the retarded correlator \n$-\\frac{1}{\\pi}\\text{Im}\\, C^R_{(AA^\\dagger)^2}(\\omega)=\\mathscr A_{(AA^\\dagger)^2}(\\omega)$.  \nWhen $\\hat B=\\hat A^\\dagger=\\hat A$ (which is the case for $\\hat A=\\hat n_\\sigma, \\hat n$), we have $C_{(AA)^2}^R(\\omega)^\\ast=C_{(AA)^2}^R(-\\omega)$. At half filling, the particle-hole symmetry furthermore ensures $C_{c_\\sigma^\\dagger c_\\sigma}^R(\\omega)^\\ast=C_{c_\\sigma^\\dagger c_\\sigma}^R(-\\omega)$.\nThus, in all the cases considered in this study, the retarded OTOC has the symmetry property $C_{(AA^\\dagger)^2}^R(\\omega)^\\ast=C_{(AA^\\dagger)^2}^R(-\\omega)$. Using this, as well as the out-of-time-order fluctuation-dissipation\ntheorem discussed in the previous section, we can obtain the real-time OTOC\n$\\langle \\hat A(t)\\hat A^\\dagger(0), \\hat A(t)\\hat A^\\dagger(0)\\rangle$ from the following inverse Fourier transformation, \n\\begin{align}{\\rm Re}\\langle \\hat A(t)\\hat A^\\dagger(0), \\hat A(t)\\hat A^\\dagger(0)\\rangle\n&=\n{\\rm Re}\n\\int_0^\\infty d\\omega\\, e^{-i\\omega t}\\coth\\left(\\frac{\\beta\\hbar\\omega}{4}\\right)\n\\left(-\\frac{1}{\\pi}{\\rm Im}\\, C_{(AA^\\dagger)^2}^R(\\omega)\\right),\n\\\\\n{\\rm Im}\\langle \\hat A(t)\\hat A^\\dagger(0), \\hat A(t)\\hat A^\\dagger(0)\\rangle\n&=\n{\\rm Im}\\int_0^{\\infty} d\\omega\\, e^{-i\\omega t} \\left(-\\frac{1}{\\pi}{\\rm Im}\\, C_{(AA^\\dagger)^2}^R(\\omega)\\right),\n\\end{align}\nfor $\\hat A=c_\\sigma^\\dagger, \\hat n_\\sigma$, and $\\hat n$.\n\n\\begin{figure}[b]\n\\begin{center}\n\\includegraphics[angle=-90, width=0.48\\columnwidth]{compare_u0_draft.eps}\\hfill\n\\includegraphics[angle=-90, width=0.48\\columnwidth]{compare_u0_longtime_modulus.eps}\n\\caption{\nLeft panel: Comparison between the exact solution and the result obtained by analytical continuation\nfor the real and imaginary parts of the OTOC $\\langle c_\\sigma^\\dagger(t)c_\\sigma(0), c_\\sigma^\\dagger(t)c_\\sigma(0)\\rangle$ \nin the noninteracting case ($U=0, \\beta\\hbar=50$) of the single-orbital Hubbard model.\nRight panel: Comparison of the long-time behavior for the modulus $|\\langle c_\\sigma^\\dagger(t)c_\\sigma(0), c_\\sigma^\\dagger(t)c_\\sigma(0)\\rangle|$. \nThe exact result shows a power-law decay ($\\sim 1/t^3$). \n}\n\\label{fig:test_u0}\n\\end{center}\n\\end{figure}\n\nAs a check of our procedure, we first compare the results for the noninteracting model,\nfor which an exact solution of the OTOC $\\langle c_\\sigma^\\dagger(t)c_\\sigma(0), c_\\sigma^\\dagger(t)c_\\sigma(0)\\rangle$ \nis available \\cite{TsujiWernerUeda2017}.\nIt is a nontrivial task to measure the noninteracting two-particle Green's function in CT-HYB, \nand to analytically continue \n$C_{(c^\\dagger_\\sigma c_\\sigma)^2}^M(\\tau)$ \nby the Maximum Entropy method.  \nThe results for the inverse temperature $\\beta\\hbar=50$ are plotted in Fig.~\\ref{fig:test_u0}. \nOne can see that the OTOC oscillates for about one cycle, and quickly decays to zero.\nThe analytically continued data show a good agreement with the exact solution. \nIn particular, for short times (up to about two inverse hoppings) the dynamics is accurately reproduced, while deviations appear at longer times. \nIt is known that for the noninteracting fermion system the OTOC decays as a power law at long time ($\\sim 1/t^3$) \\cite{TsujiWernerUeda2017}.\nThe analytical continuation method cannot reproduce the details of the oscillations at intermediate or long times, but it roughly captures the long-time decay, which is controlled by low-frequency spectral features. \nUsually, analytic continuation is unreliable for high frequency components, because high-frequency information is suppressed \nby the kernel $K(\\tau,\\omega)$ in the transformation from the spectral function $A_O(\\omega)$ to the (bosonic) Matsubara correlation function $O(\\tau)$ [$O(\\tau)=\\int_{-\\infty}^{\\infty} d\\omega\\, K(\\omega,\\tau) A_O(\\omega)$ with $K(\\tau,\\omega)=e^{-\\tau\\omega}/(1-e^{-\\beta\\omega})$]. On the other hand, the low frequency components can be rather accurately determined. As a result, in Fig. 4 we more or less recover the low-frequency features of the time-dependent correlation function, while the details of rapid oscillations are not captured. The accurate result at short times can be understood from the fact that this region is close to the imaginary time axis in the complex time plane, where the original QMC data are available.\nAt higher temperatures, the Maximum Entropy method becomes less reliable, so that the resulting OTOCs are expected to be less accurate.  \n\n\n\\subsection{Single-orbital Hubbard model}\n\\label{sec:hubbard}\n\nIn this section, we calculate the OTOCs (\\ref{fermion_otoc})-(\\ref{otoc_nn}) for the interacting single-orbital Hubbard model with the Hamiltonian \n\\begin{align}\nH\n&=\n-v\\sum_{\\langle i,j\\rangle\\sigma} (c^\\dagger_{i\\sigma}c_{j\\sigma}+ \\text{h.c.})\n-\\mu\\sum_i \\hat n_i+U\\sum_i \\hat n_{i\\uparrow}\\hat n_{i\\downarrow}, \n\\end{align}\nwhere $v$ is the hopping amplitude, $\\langle i,j\\rangle$ represents the nearest-neighbor site pairs, \n$\\mu$ is the chemical potential, and $U$ is the on-site interaction.\nIn this section, we focus on the half-filled case (i.e., $\\mu=U/2$). The DMFT solution for the simplified self-consistency (\\ref{semicirc}) is the exact solution for an infinitely connected Bethe lattice \\cite{GeorgesKotliarKrauthRozenberg1996}.\nLet us briefly recall the paramagnetic phase diagram for the single-orbital Hubbard model on this lattice \\cite{Bluemer_thesis}. At half-filling, there is a metal-insulator crossover at high temperature and a first-order Mott transition below a temperature corresponding to $\\beta\\hbar\\approx 17$ with a coexistence region between $U_{c1}$ and $U_{c2}$. The finite-temperature critical endpoint is at $U\\approx 4.7$, while the zero-temperature Mott transition occurs at $U=U_{c2}\\approx 5.6$. \nAt low temperature, the self-energy shows the Fermi liquid behavior [${\\rm Im}\\,\\Sigma(\\omega)\\sim \\omega^2$] \nin the weakly correlated metallic phase, \nwhile it shows an insulating behavior [${\\rm Im}\\,\\Sigma(\\omega)\\sim \\delta(\\omega)$] in the Mott phase\n\\cite{GeorgesKotliarKrauthRozenberg1996}. \nIn the correlated metallic phase, as the temperature is increased, deviations from the Fermi liquid behavior become apparent, \nbut an ${\\rm Im}\\,\\Sigma(\\omega)\\sim \\sqrt{\\omega}$ scaling as in the SYK model is not observed. \n\nIn the following, we will start the DMFT iterations from the noninteracting solution, which means that the finite-temperature Mott transition occurs at $U_{c2}$.\nFrom now on, we set $\\hbar=1$. \n\n\\begin{figure}[t]\n\\begin{center}\n\\includegraphics[angle=-90, width=0.49\\columnwidth]{plot_otoc_b50_Re.eps}\n\\includegraphics[angle=-90, width=0.49\\columnwidth]{plot_otoc_b50_Im.eps}\n\\includegraphics[angle=-90, width=0.49\\columnwidth]{plot_coth_b50_appendix.eps}\n\\includegraphics[angle=-90, width=0.49\\columnwidth]{plot_otoc_b50_modulus.eps}\n\\caption{\nTop panels: Real and imaginary parts of the OTOC $\\langle c^\\dagger_\\sigma(t)c_\\sigma(0), c^\\dagger_\\sigma(t)c_\\sigma(0)\\rangle$ for the single-orbital Hubbard model with $\\beta=50$ at half filling. The thin lines correspond to metallic solutions, while the thick line ($U=6$) corresponds to a Mott insulating solution. \nBottom left panel: Spectral function $\\mathscr A_{(c_\\sigma^\\dagger c_\\sigma)^2}(\\omega)=-\\frac{1}{\\pi}{\\rm Im}\\,C_{(c^\\dagger_\\sigma c_\\sigma)^2}^R(\\omega)$ multiplied by $\\coth\\big(\\frac{\\beta\\omega}{4}\\big)$. \nBottom right panel: Modulus of the OTOC $|\\langle c^\\dagger_\\sigma(t)c_\\sigma(0), c^\\dagger_\\sigma(t)c_\\sigma(0)\\rangle|$ (normalized at $t=0$) on a log-log scale. The dashed line is the result for $U=0$.\n}\n\\label{fig:results_fermion}\n\\end{center}\n\\end{figure}\n\n\n\nIn Fig.~\\ref{fig:results_fermion}, we show the results of $\\langle c^\\dagger_\\sigma(t)c_\\sigma(0), c^\\dagger_\\sigma(t)c_\\sigma(0)\\rangle$ for the interacting single-orbital Hubbard model at $\\beta=50$. \nThe top panels present the real and imaginary parts of the OTOC. One can see that the oscillations become more pronounced as one increases the interaction. In the bottom left panel of Fig.~\\ref{fig:results_fermion},\nwe show the corresponding spectral function $\\mathscr A_{(c_\\sigma^\\dagger c_\\sigma)^2}(\\omega)=-\\frac{1}{\\pi}{\\rm Im}\\,C_{(c^\\dagger_\\sigma c_\\sigma)^2}^R(\\omega)$ multiplied by $\\coth\\big(\\frac{\\beta\\omega}{4}\\big)$.\nThe Mott transition, which occurs between $U=5$ and $U=6$ at $\\beta=50$, manifests itself by the opening of a gap in the spectral function $\\mathscr A_{(c_\\sigma^\\dagger c_\\sigma)^2(\\omega)}$. This translates into more weakly damped oscillations in the real-time evolution in the Mott phase.  \nThe bottom right panel of Fig.~\\ref{fig:results_fermion} plots the modulus of the OTOC (normalized at time $t=0$, i.e., $|\\langle c^\\dagger_\\sigma(t)c_\\sigma(0), c^\\dagger_\\sigma(t)c_\\sigma(0)\\rangle|/|\\langle c^\\dagger_\\sigma(0)c_\\sigma(0), c^\\dagger_\\sigma(0)c_\\sigma(0)\\rangle|$) on a log-log scale. \nDue to the oscillations and the limited accuracy of the analytic continuation procedure, it is hard to clearly identify the nature of the long-time decay of the OTOC,\nbut the results indicate that the OTOC decays much faster (possibly exponentially) in the Mott phase than in the metallic phase. This is consistent with the results for the spectral functions of the OTOC. The comparison to the noninteracting case (dashed curve in the bottom right panel of Fig.~\\ref{fig:results_fermion}) suggests that the correlated metallic phase has a similar (possibly power-law) decay behavior of the OTOC as in the noninteracting case.\n\n\n\\begin{figure}[t]\n\\begin{center}\n\\includegraphics[angle=-90, width=0.49\\columnwidth]{plot_otoc_b50_nn_Re.eps}\n\\includegraphics[angle=-90, width=0.49\\columnwidth]{plot_otoc_b50_nn_Im.eps}\n\\includegraphics[angle=-90, width=0.49\\columnwidth]{plot_coth_b50_nn_appendix.eps}\n\\includegraphics[angle=-90, width=0.49\\columnwidth]{plot_otoc_b50_nn_modulus.eps}\n\\caption{\nTop panels: Real and imaginary parts of the OTOC $\\langle \\hat n(t)\\hat n(0), \\hat n(t)\\hat n(0)\\rangle - 1$ for the single-orbital Hubbard model with $\\beta=50$ at half filling. The thin lines correspond to metallic solutions, \nwhile the thick line ($U=6$) corresponds to a Mott insulating solution. \nBottom left panel: Spectral function $\\mathscr A_{(nn)^2}(\\omega)=-\\frac{1}{\\pi}{\\rm Im}\\,C_{(n n)^2}^R(\\omega)$ multiplied by $\\coth\\big(\\frac{\\beta\\omega}{4}\\big)$.\nBottom right panel: Modulus of the OTOC $|\\langle \\hat n(t)\\hat n(0), \\hat n(t)\\hat n(0)\\rangle - 1|$ (normalized at $t=0$) on a log-log scale. \n}\n\\label{fig:results_nn}\n\\end{center}\n\\end{figure}\n\n\nIn Fig.~\\ref{fig:results_nn}, we plot the OTOC $\\langle \\hat n(t)\\hat n(0), \\hat n(t)\\hat n(0)\\rangle$\nfor the single-orbital Hubbard model in a similar manner as in Fig.~\\ref{fig:results_fermion}.\nSince at half filling this correlation function approaches $\\langle \\hat n\\rangle^4=1$ at long times\nand at sufficiently low temperature,\nwe perform the analytical continuation procedure for $\\langle \\hat n(t)\\hat n(0), \\hat n(t)\\hat n(0)\\rangle-1$. \nThe top panels of Fig.~\\ref{fig:results_nn} show the real and imaginary parts of this shifted OTOC. \nContrary to the case of $\\langle c_\\sigma^\\dagger(t)c_\\sigma(0), c_\\sigma^\\dagger(t)c_\\sigma(0)\\rangle$, \nthe amplitude of the OTOC $\\langle \\hat n(t)\\hat n(0), \\hat n(t)\\hat n(0)\\rangle-1$ is suppressed as one increases the interaction, which reflects the reduced charge fluctuations in the Mott insulator.\nThe bottom left panel in Fig.~\\ref{fig:results_nn} shows the analytically continued spectral function $\\mathscr A_{(nn)^2}(\\omega)=-\\frac{1}{\\pi}{\\rm Im}\\, C_{(nn)^2}^R(\\omega)$ multiplied by $\\coth\\big(\\frac{\\beta\\omega}{4}\\big)$. \nAgain we notice the opening of a gap in the Mott phase, and a corresponding shift of the spectral weight to higher energies. This results in more rapid oscillations of the OTOC in the Mott state.\nThe bottom right panel in Fig.~\\ref{fig:results_nn} plots the modulus of the OTOC $|\\langle \\hat n(t)\\hat n(0), \\hat n(t)\\hat n(0)\\rangle-1|$ (normalized at $t=0$) on a log-log scale.\nThe long-time behavior of the OTOC is consistent with a power-law decay in the metallic phase, and an exponential decay in the Mott insulating phase.\n\n\n\\begin{figure}[t]\n\\begin{center}\n\\includegraphics[angle=-90, width=0.49\\columnwidth]{plot_otoc_nsns_b50_draft_Re.eps}\n\\includegraphics[angle=-90, width=0.49\\columnwidth]{plot_otoc_nsns_b50_draft_Im.eps}\n\\includegraphics[angle=-90, width=0.49\\columnwidth]{plot_coth_b50_nsns_inset.eps}\n\\includegraphics[angle=-90, width=0.49\\columnwidth]{plot_otoc_nsns_b50_draft_modulus_zoom.eps}\n\\caption{\nTop panels: Real and imaginary parts of the OTOC $\\langle \\hat n_\\sigma(t)\\hat n_\\sigma(0), \\hat n_\\sigma(t)\\hat n_\\sigma(0)\\rangle  - \\langle \\hat n_\\sigma\\rangle^4$ for the single-orbital Hubbard model with $\\beta=50$ at half filling. All the colored lines correspond to metallic solutions. \nBottom left panel: Spectral function $\\mathscr A_{(n_\\sigma n_\\sigma)^2}(\\omega)=-\\frac{1}{\\pi}{\\rm Im}\\,C_{(n_\\sigma n_\\sigma)^2}^R(\\omega)$ multiplied by $\\coth\\big(\\frac{\\beta\\omega}{4}\\big)$. In the inset, we plot the imaginary-time four-point function $\\langle \\hat n_\\sigma(\\tau)\\hat n_\\sigma(0), \\hat n_\\sigma(\\tau)\\hat n_\\sigma(0)\\rangle-\\langle \\hat n_\\sigma\\rangle^4$, with the black line corresponding to the Mott insulating solution at $U=6$.  \nBottom right panel: Modulus of the OTOC $|\\langle \\hat n_\\sigma(t)\\hat n_\\sigma(0), \\hat n_\\sigma(t)\\hat n_\\sigma(0)\\rangle  - \\langle \\hat n_\\sigma\\rangle^4|$ (normalized at $t=0$) on a log-log scale.\n}\n\\label{fig:results_nsns}\n\\end{center}\n\\end{figure}\n\nFigure~\\ref{fig:results_nsns} \nshows the results of the OTOC $\\langle \\hat n_\\sigma(t)\\hat n_\\sigma(0), \\hat n_\\sigma(t)\\hat n_\\sigma(0)\\rangle$\nfor the single-orbital Hubbard model. At half filling, this correlation function approaches $\\langle \\hat n_\\sigma\\rangle^4=\\frac{1}{16}$ in the long-time limit and at sufficiently low temperature, \nso that we perform the analytical continuation procedure for $\\langle \\hat n_\\sigma(t)\\hat n_\\sigma(0), \\hat n_\\sigma(t)\\hat n_\\sigma(0)\\rangle-\\langle \\hat n_\\sigma\\rangle^4$. \nThe top panels of Fig.~\\ref{fig:results_nsns} show the real and imaginary parts of \nthis shifted OTOC. \nHere, we only present results for the metallic phase, because resolving the very sharp low-energy feature of the OTOC spectrum in the half-filled Mott state is challenging. One can see that coherent oscillations are not observed for this type of OTOC, and that the incoherent part becomes dominant compared to the other two OTOCs shown in Figs.~\\ref{fig:results_fermion} and \\ref{fig:results_nn}. The amplitude of the OTOC $\\langle \\hat n_\\sigma(t)\\hat n_\\sigma(0), \\hat n_\\sigma(t)\\hat n_\\sigma(0)\\rangle-\\langle \\hat n_\\sigma\\rangle^4$ is enhanced as $U$ increases. As we argue below, this is related to the enhancement of spin fluctuations \nand local moment formation close to the Mott transition.\n\nThe bottom left panel in Fig.~\\ref{fig:results_nsns} plots the analytically continued spectral function $\\mathscr A_{(n_\\sigma n_\\sigma)^2}(\\omega)=-\\frac{1}{\\pi}{\\rm Im}\\, C_{(n_\\sigma n_\\sigma)^2}^R(\\omega)$ multiplied by $\\coth\\big(\\frac{\\beta\\omega}{4}\\big)$. \nWe can see that the spectral weight is concentrated in the low-energy region as we approach the Mott transition point.\nThe inset in the bottom left panel of Fig.~\\ref{fig:results_nsns} shows the imaginary-time four-point function $\\langle \\hat n_\\sigma(\\tau)\\hat n_\\sigma(0), \\hat n_\\sigma(\\tau)\\hat n_\\sigma(0)\\rangle-\\langle \\hat n_\\sigma\\rangle^4$, with the black line corresponding to a Mott insulating solution ($U=6$).  \nFor $U\\gtrsim 4$, the imaginary-time four-point function does not decay to zero but remains relatively large near $\\tau=\\frac{\\beta}{4}$. \nThis behavior is reminiscent of the spin-freezing physics seen in the correlated Hund metal phase of multi-orbital Hubbard models, where the dynamical spin correlation function $\\langle \\hat S_z(\\tau)\\hat S_z(0)\\rangle$ is trapped at a finite value at long $\\tau$ \\cite{Werner2008}. \nIn fact, the OTOC $\\langle \\hat n_\\sigma(\\tau)\\hat n_\\sigma(0), \\hat n_\\sigma(\\tau)\\hat n_\\sigma(0)\\rangle$\nmeasures spin correlations (in addition to charge correlations) through $\\hat n_\\sigma=\\frac{1}{2}\\hat n+\\sigma\\hat S_z$.\nPhysically, the trapping of $\\langle \\hat S_z(\\tau)\\hat S_z(0)\\rangle$ signals the formation of frozen local magnetic moments.\nIn the metal-insulator crossover region, the scattering induced by these magnetic moments results in incoherent metal states. Intuitively, one might expect fast scrambling in the corresponding regions of the phase diagram. \nAlthough there is some resemblance with the spin freezing crossover,\nthe self-energy does not exhibit a square-root frequency dependence in the single-orbital case, and a spin-freezing crossover in the sense of Ref.~\\cite{Werner2008} does not exist.  \n\nThe bottom right panel of Fig.~\\ref{fig:results_nsns} plots the modulus of the OTOC $|\\langle \\hat n_\\sigma(t)\\hat n_\\sigma(0), \\hat n_\\sigma(t)\\hat n_\\sigma(0)\\rangle-\\langle \\hat n_\\sigma\\rangle^4|$ (normalized at $t=0$) on a log-log scale. \nThe dynamics of $\\langle \\hat n_\\sigma(t)\\hat n_\\sigma(0), \\hat n_\\sigma(t)\\hat n_\\sigma(0)\\rangle$ is quite distinct from that of $\\langle \\hat n(t)\\hat n(0), \\hat n(t)\\hat n(0)\\rangle$. As the Mott transition is approached, we observe a slow-down in the decay of the modulus, \nwhich indicates the presence of slowly fluctuating local moments. \nThe long-time behavior may be consistent with a power-law, although slow oscillations make it difficult to determine the long-time asymptotic form from the numerics. \n\n\n\\begin{figure}[t]\n\\begin{center}\n\\includegraphics[angle=-90, width=0.49\\columnwidth]{plot_fitnsns_b2.eps}\n\\includegraphics[angle=-90, width=0.49\\columnwidth]{plot_coeffc_draft.eps}\n\\caption{\nLeft panel: \nThe imaginary-time four-point function $\\langle \\hat n_\\sigma(\\tau)\\hat n_\\sigma(0), \\hat n_\\sigma(\\tau)\\hat n_\\sigma(0)\\rangle$, the squared density-density correlation function $\\langle \\hat n_\\sigma(\\tau)\\hat n_\\sigma(0)\\rangle^2$,\nand their difference for the single-orbital Hubbard model with $U=5.5$ and $\\beta=50$.\nThe dashed curve shows a fit of the difference with the function $f_\\text{fit}(\\tau)=a+b\\tau+c\\tau^2$ in the interval $\\tau/\\beta \\in [0: 0.2]$.\nRight panel: Fitting coefficient $c$ as a function of $U$ for indicated values of $\\beta$.\n\n}\n\\label{fig:coeffs}\n\\end{center}\n\\end{figure}\n\nFor large enough $\\beta$ and small enough $\\tau$, the OTOC (\\ref{otoc_nsns}) factorizes into $\\langle \\hat n_\\sigma(\\tau)\\hat n_\\sigma(0), \\hat n_\\sigma(\\tau)\\hat n_\\sigma(0)\\rangle \\approx \\langle \\hat n_\\sigma(\\tau)\\hat n_\\sigma(0)\\rangle^2$. \nOn the real-time axis, this corresponds to the decoupling $\\langle \\hat n_\\sigma(t)\\hat n_\\sigma(0), \\hat n_\\sigma(t)\\hat n_\\sigma(0)\\rangle \\approx \\langle \\hat n_\\sigma(t)\\hat n_\\sigma(0)\\rangle^2$, that is, \nthe OTOC is reduced to a product of ordinary density-density correlation functions.\nIn order to see whether nontrivial correlations are captured beyond the decoupled form by the OTOC function, \nwe consider the difference $\\langle \\hat n_\\sigma(\\tau)\\hat n_\\sigma(0), \\hat n_\\sigma(\\tau)\\hat n_\\sigma(0)\\rangle-\\langle \\hat n_\\sigma(\\tau)\\hat n_\\sigma(0)\\rangle^2$ and fit the short-time behavior of this function to $f_\\text{fit}(\\tau)=a+b\\tau+c\\tau^2$, as illustrated in the left panel of Fig.~\\ref{fig:coeffs}. The coefficients $b$ and $c$ serve as  indicators of the nontrivial correlations that cannot be attributed to the decoupled form of the OTOC.\nThe coefficient $c$ is plotted as a function of $U$ and for different $\\beta$ in the right hand panel. The coefficient $b$ (not shown) exhibits a similar trend. We notice that the OTOC picks up nontrival correlations near the metal-insulator transition, and in particular at \nintermediate\ntemperatures, while the Mott phase shows no such correlations. The crossover region  \nassociated with the emergence of local moments \ncorresponds, roughly, to the interaction range where the nontrivial correlations start to become significant. \n\n\n\\subsection{Two-orbital Hubbard model}\n\\label{sec:two-orbital hubbard}\n\nIn this section, we investigate OTOCs in the two-orbital Hubbard model with Hund coupling $J>0$ \nand density-density interactions. (Three-orbital results are presented in Appendix~\\ref{sec:3orbital}.) \nThe Hamiltonian is given by\n\\begin{align}\nH\n&=\n-v\\sum_{\\langle i,j\\rangle\\alpha\\sigma} (c^\\dagger_{i\\alpha\\sigma}c_{j\\alpha\\sigma}+ \\text{h.c.})\n-\\mu\\sum_{i\\alpha\\sigma} \\hat n_{i\\alpha\\sigma}\n+U\\sum_{i\\alpha} \\hat n_{i\\alpha\\uparrow}\\hat n_{i\\alpha\\downarrow}\n+(U-2J)\\sum_{i\\sigma} \\hat n_{i1\\sigma}\\hat n_{i2\\bar\\sigma}\n\\notag\n\\\\\n&\\quad\n+(U-3J)\\sum_{i\\sigma} \\hat n_{i1\\sigma}\\hat n_{i2\\sigma},\n\\end{align}\n\nwith $U$ being the intra-orbital interaction $U$, $U-2J$ the inter-orbital antiparallel-spin interaction, and $U-3J$ the inter-orbital parallel-spin interaction. This is the simplest model that shows a crossover from a spin-frozen metal to a Fermi-liquid metal as one dopes the half-filled Mott insulator, with the self-energy scaling as $\\text{Im}\\Sigma(\\omega_n)\\sim\\sqrt{\\omega_n}$ over a significant energy range in the crossover regime \\cite{Hafermann2012}.\nThe sketch of the phase diagram is shown in Fig.~\\ref{fig:illustration}. \nAs we have seen in the previous section, the spin-related OTOC of the type (\\ref{otoc_nsns}) is relevant for the analysis of  the spin-freezing crossover, so that we concentrate on this OTOC here.\nIn the following calculations, we choose $U=8$, $J=U/4$, and compute $\\langle \\hat n_\\sigma(t)\\hat n_\\sigma(0), \\hat n_\\sigma(t)\\hat n_\\sigma(0)\\rangle - \\langle n_\\sigma \\rangle^4$ \nas a function of filling at $\\beta=50$ (dashed lines in Fig.~\\ref{fig:illustration}). \n\n\n\\begin{figure}[t]\n\\begin{center}\n\\includegraphics[angle=-90, width=0.49\\columnwidth]{plot_qp_draft.eps}\\hfill\n\\includegraphics[angle=-90, width=0.49\\columnwidth]{plot_fit_2orbital.eps}\n\\caption{\nLeft panel: Imaginary part of the self-energy as a function of Matsubara frequency for the two-orbital Hubbard model with $U=8$, $J=U/4$, and $\\beta=50$ on a log-log scale, with the dashed line indicating $\\sqrt{\\omega_n}$ behavior. \nRight panel: The coefficient $c$ extracted from\nfitting $\\langle \\hat n_\\sigma(\\tau)\\hat n_\\sigma(0), \\hat n_\\sigma(\\tau)\\hat n_\\sigma(0)\\rangle-\\langle \\hat n_\\sigma(\\tau)\\hat n_\\sigma(0)\\rangle^2$ with a function $f_\\text{fit}(\\tau)=a+b\\tau+c\\tau^2$\nin the interval $\\tau/\\beta\\in [0: 0.05]$ for the two-orbital Hubbard model. \nThe spin-freezing crossover occurs roughly at the filling $n_\\sigma\\approx 0.28$ (blue curve in the left panel and blue arrow in the right panel). \n}\n\\label{fig:self}\n\\end{center}\n\\end{figure}\n\n\nThe left panel of Fig.~\\ref{fig:self} plots the imaginary part of the self-energy as a function of Matsubara frequency in order to identify the filling corresponding to the spin-freezing crossover \\cite{Werner2008}. An approximate square-root scaling is observed over a wide range of frequencies near $n_\\sigma\\approx 0.28$ (blue line). In the right panel of Fig.~\\ref{fig:self}, we present the coefficient $c$ obtained from a similar analysis as presented in Fig.~\\ref{fig:coeffs}, but with a fitting range $\\tau/\\beta\\in [0 : 0.05]$. Again, we find that the OTOC $\\langle \\hat n_\\sigma(t)\\hat n_\\sigma(0), \\hat n_\\sigma(t)\\hat n_\\sigma(0)\\rangle - \\langle \\hat n_\\sigma\\rangle^4$ detects nontrivial correlations in the spin-freezing crossover (blue arrow) and spin-frozen metal regime, but not in the Fermi liquid or the Mott insulating phase.\n\n\n\\begin{figure}[t]\n\\begin{center}\n\\includegraphics[angle=-90, width=0.49\\columnwidth]{plot_otoc_2orbital_Re.eps}\n\\includegraphics[angle=-90, width=0.486\\columnwidth]{plot_otoc_2orbital_Im.eps}\n\\includegraphics[angle=-90, width=0.484\\columnwidth]{plot_spectra_coth_2orbital_inset.eps}\n\\includegraphics[angle=-90, width=0.486\\columnwidth]{plot_otoc_2orbital_modulus_inset.eps}\n\\caption{\nTop panels: Real and imaginary parts of the OTOC $\\langle \\hat n_\\sigma(t)\\hat n_\\sigma(0), \\hat n_\\sigma(t)\\hat n_\\sigma(0)\\rangle  - \\langle \\hat n_\\sigma\\rangle^4$ for the two-orbital Hubbard model with $U=8$, $J=U/4$, $\\beta=50$ and indicated fillings. \nBottom left panel: Spectral function $\\mathscr A_{(n_\\sigma n_\\sigma)^2}(\\omega)=-\\frac{1}{\\pi}{\\rm Im}\\,C_{(n_\\sigma n_\\sigma)^2}^R(\\omega)$ multiplied by $\\coth\\big(\\frac{\\beta\\omega}{4}\\big)$. In the inset, we plot the imaginary-time four-point function $\\langle \\hat n_\\sigma(\\tau)\\hat n_\\sigma(0), \\hat n_\\sigma(\\tau)\\hat n_\\sigma(0)\\rangle-\\langle \\hat n_\\sigma\\rangle^4$. \nBottom right panel: Modulus of the OTOC $|\\langle \\hat n_\\sigma(t)\\hat n_\\sigma(0), \\hat n_\\sigma(t)\\hat n_\\sigma(0)\\rangle  - \\langle \\hat n_\\sigma\\rangle^4|$ (normalized at $t=0$) on a log-log scale.\nIn the inset of the bottom right panel, we illustrate the approximate data collapse obtained by considering the spin-freezing crossover regimes for different values of $J$. \nIn all the panels, the thick blue curves represent the results for the spin-freezing crossover region. \n}\n\\label{fig:2orbital}\n\\end{center}\n\\end{figure}\n\n\nThe top panels of Fig.~\\ref{fig:2orbital} show the real and imaginary parts of the OTOC\n$\\langle \\hat n_\\sigma(t)\\hat n_\\sigma(0), \\hat n_\\sigma(t)\\hat n_\\sigma(0)\\rangle-\\langle \\hat n_\\sigma\\rangle^4$\nfor the two-orbital Hubbard model. In the spin-frozen phase ($n_\\sigma \\gtrsim 0.3$), both components exhibit \nhighly incoherent oscillations, which get suppressed as the filling is reduced. As one moves across the crossover point ($n_\\sigma \\sim 0.28$),\nthe oscillations completely disappear, and the real part of the OTOC starts to overshoot to the negative side\nin the Fermi liquid regime.\nThe bottom left panel in Fig.~\\ref{fig:2orbital} presents the analytically continued spectral function $\\mathscr A_{(n_\\sigma n_\\sigma)^2}(\\omega)=-\\frac{1}{\\pi}{\\rm Im}\\,C_{(n_\\sigma n_\\sigma)^2}^R(\\omega)$ multiplied by $\\coth\\big(\\frac{\\beta\\omega}{4}\\big)$. \nIn the spin-frozen regime ($n_\\sigma \\gtrsim 0.3$) a sharp peak appears at low energy, which originates from the saturation of the imaginary-time four point function $\\langle \\hat n_\\sigma(\\tau)\\hat n_\\sigma(0), \\hat n_\\sigma(\\tau)\\hat n_\\sigma(0)\\rangle$ at large $\\tau$ (see the inset of the bottom left panel of Fig.~\\ref{fig:2orbital}). This low-energy peak represents the slow dynamics of the frozen spins, and translates into a slow long-time decay of $|\\langle \\hat n_\\sigma(t)\\hat n_\\sigma(0), \\hat n_\\sigma(t)\\hat n_\\sigma(0)\\rangle-\\langle \\hat n_\\sigma\\rangle^4|$,\nas shown in the bottom right panel of Fig.~\\ref{fig:2orbital}\non a log-log scale. Again, it is difficult to clearly resolve the long-time behavior of the OTOC due to the limited\naccuracy of the analytic continuation, but in the spin-freezing crossover regime (thick blue curves)\nthe decay of the OTOC is consistent with a power law $\\sim 1/t^{1.5}$ at least for $2\\lesssim t \\lesssim 5$. \nThe shorter-time behavior ($0.5\\lesssim t \\lesssim 2$) is instead well fitted by an exponential decay $\\sim e^{-\\alpha t}$ with a decay constant $\\alpha=0.78$. \nThis exponent exhibits\na strong dependence on the Hund coupling. \n\nIn fact, the OTOCs in the spin-freezing regime for $J=U/4$ and $J=U/6$ can be approximately collapsed by plotting them as a function of $tJ$ (see the inset in the bottom right panel of Fig.~\\ref{fig:2orbital}), which indicates that the Hund coupling is the parameter which controls the dynamics in this regime. \nThis is distinct from the Lyapunov behavior in which the exponential decay scales with $t/\\beta$ \\cite{Bagrets2017}.\nA possible reason is that we are still far from the large $N$ limit (where $N$ corresponds to the number of orbitals times the number of spin degrees of freedom) so that the large $N$ behavior \nsuch as the Lyapunov growth is not observed in our calculations. \nIn fact, the strong dependence of the time scale on $J$ and weak dependence on $\\beta$ is consistent\nwith the behavior of the finite $N$ SYK model in Ref.~\\cite{FuSachdev2016}.\nIn the spin-frozen regime ($n_\\sigma\\gtrsim 0.3$) the decay is much slower than near the crossover point, while in the Fermi liquid regime the decay is accelerated, approaching the free fermion behavior of $t^{-3}$\nas shown in Fig.~\\ref{analytic continuation}. \nQualitatively similar results for this OTOC are obtained \nfor the three-orbital Hubbard model (see Appendix~\\ref{sec:3orbital}).\n\nThe time dependence of the modulus of \n$\\langle \\hat n_\\sigma(\\tau)\\hat n_\\sigma(0), \\hat n_\\sigma(\\tau)\\hat n_\\sigma(0)\\rangle-\\langle \\hat n_\\sigma\\rangle^4$ \nin the spin-freezing crossover regime of multi-orbital Hubbard models (exponential decay crossing over into a power-law) bears a close resemblance to a (different) OTOC for the SYK model discussed in Ref.~\\cite{Bagrets2017}. \nAlso, a recent study of yet another type of OTOCs for the finite-$N$ SYK model found a qualitatively similar decay as observed here in the spin-freezing crossover regime \\cite{FuSachdev2016}. In the following section, we will make the connection to the SYK dynamics more quantitative.   \n\n\n\\subsection{Comparison to the SKY model}\n\\label{sec:SYK}\n\nIn this section, we compare the results of the multi-orbital Hubbard models to the SYK model.\nWe consider the particle-hole symmetric SYK model of complex fermions \\cite{FuSachdev2016},\nwhose Hamiltonian is given by\n\\begin{align}\nH\n&=\n\\frac{1}{(2N)^{3/2}} \\sum_{i,j,k,l=1}^N \nJ_{ij;kl} (c_i^\\dagger c_j^\\dagger c_k c_l\n+\\delta_{ik}n c_j^\\dagger c_l\n-\\delta_{il}n c_j^\\dagger c_k\n-\\delta_{jk}n c_i^\\dagger c_l\n+\\delta_{jl}n c_i^\\dagger c_k).\n\\end{align}\nThe coupling constant $J_{ij;kl}$ is a gaussian random variable, satisfying $J_{ij;kl}=-J_{ji;kl}=-J_{ij;lk}=J_{kl;ij}^\\ast$, and\n\\begin{align}\n\\overline{({\\rm Re}\\, J_{ij;kl})^2}\n&=\n\\begin{cases}\nJ_{\\rm SYK}^2/2 & (i,j)\\neq (k,l) \\\\\nJ_{\\rm SYK}^2 & (i,j)=(k,l)\n\\end{cases},\n\\\\\n\\overline{({\\rm Im}\\, J_{ij;kl})^2}\n&=\n\\begin{cases}\nJ_{\\rm SYK}^2/2 & (i,j)\\neq (k,l) \\\\\n0 & (i,j)=(k,l)\n\\end{cases},\n\\end{align}\nwhere the overline represents an average over each realization of $J_{ij;kl}$. The parameter \n$J_{\\rm SYK}$ ($J_{\\rm SYK}^{-1}$) defines the unit of energy (time) in the following calculations. \nThe system is particle-hole symmetric when\n$n=0.5$, which fixes the total number of fermions as $N^{-1}\\sum_{i=1}^N \\langle c_i^\\dagger c_i\\rangle=0.5$.\nWe take the particle-hole symmetric form of the SYK model because it has been well studied previously \\cite{FuSachdev2016} and avoids the effect of the filling drift.\nWe numerically solve the model by exact diagonalization for finite $N(\\le 12)$, and evaluate the density-density OTOC\n$\\overline{\\langle \\hat n_i(t)\\hat n_i(0), \\hat n_i(t)\\hat n_i(0)\\rangle-\\langle \\hat n_i\\rangle^4}$\nand its imaginary-time counterpart.\n\n\\begin{figure}[t]\n\\begin{center}\n\n\\includegraphics[angle=-90,width=0.49\\columnwidth]{plot_fitnsns_syk.eps}\n\n\\includegraphics[angle=-90,width=0.49\\columnwidth]{plot_coeffc_syk.eps}\n\\caption{\nLeft panel: \nThe imaginary-time four-point function $\\overline{\\langle \\hat n_i(\\tau)\\hat n_i(0), \\hat n_i(\\tau)\\hat n_i(0)\\rangle}$, the density-density correlation function $\\overline{\\langle \\hat n_i(\\tau)\\hat n_i(0)\\rangle^2}$,\nand their difference for the SYK model with $N=12$ and $\\beta J_{\\rm SYK}=10$.\nThe dashed curve shows a fit of the difference with a function $f_\\text{fit}(\\tau)=a+b\\tau+c\\tau^2$ in the interval $\\tau/\\beta \\in [0: 0.2]$. \nRight panel: Fitting coefficient $c$ as a function of $T/J_{\\rm SYK}$.\n}\n\\label{fig:SYK fit}\n\\end{center}\n\\end{figure}\n\nIn the left panel of Fig.~\\ref{fig:SYK fit}, we plot the imaginary-time four-point function \n$\\overline{\\langle \\hat n_i(\\tau)\\hat n_i(0), \\hat n_i(\\tau)\\hat n_i(0)\\rangle}$\nfor the SYK model with $N=12$ and $\\beta J_{\\rm SYK}=10$. \nAt low temperature, we expect that $\\overline{\\langle \\hat n_i(\\tau)\\hat n_i(0), \\hat n_i(\\tau)\\hat n_i(0)\\rangle}$ is decoupled into $\\overline{\\langle \\hat n_i(\\tau)\\hat n_i(0)\\rangle^2}$.\nTo see whether nontrivial correlations beyond the decoupled form are present, we fit the difference\n$\\overline{\\langle \\hat n_i(\\tau)\\hat n_i(0), \\hat n_i(\\tau)\\hat n_i(0)\\rangle}\n-\\overline{\\langle \\hat n_i(\\tau)\\hat n_i(0)\\rangle^2}$ with a function $f_{\\rm fit}(\\tau)=a+b\\tau+c\\tau^2$\nin the interval $\\tau/\\beta\\in[0:0.2]$, in the same manner as for the Hubbard models.\nThe obtained coefficient $c$ is plotted as a function of the temperature $T$ in the right panel of\nFig.~\\ref{fig:SYK fit}. One can see that nontrivial correlations exist in a wide range of temperature.\nEspecially, they are enhanced in the intermediate temperature regime. This is because in the zero-temperature limit\nthe four-point function is decoupled as \n$\\langle \\hat n_i(\\tau)\\hat n_i(0), \\hat n_i(\\tau)\\hat n_i(0)\\rangle \\approx \\langle \\hat n_i(\\tau)\\hat n_i(0)\\rangle^2$, while in the high-temperature limit ($\\beta\\to 0$) the imaginary-time dependence is washed out.\n\n\n\n\n\\begin{figure}[t]\n\\begin{center}\n\\includegraphics[angle=-90, width=0.49\\columnwidth]{plot_syk_Re.eps}\n\\includegraphics[angle=-90, width=0.49\\columnwidth]{plot_syk_Im.eps}\n\\includegraphics[angle=-90, width=0.49\\columnwidth]{plot_syk_modulus-log.eps}\n\\includegraphics[angle=-90, width=0.49\\columnwidth]{plot_syk_modulus.eps}\n\\caption{\nThe density-density OTOC $\\overline{\\langle \\hat n_i(t)\\hat n_i(0), \\hat n_i(t)\\hat n_i(0)\\rangle-\\langle \\hat n_i\\rangle^4}$ for the particle-hole symmetric SYK model averaged over $10^2$ samples with $\\beta J_{\\rm SYK}=100$ and various $N$. \nThe top left and right panels show the real and imaginary parts of the OTOC,\nwhile the bottom left and right panels show the log and log-log plots of the modulus of the OTOC (normalized at $t=0$).\nThe dashed line shows a power-law decay $\\propto t^{-1.6}$ for comparison.}\n\\label{fig:SYK}\n\\end{center}\n\\end{figure}\n\nIn Fig.~\\ref{fig:SYK}, we show the numerical results of the density-density OTOC for $\\beta J_{\\rm SYK}=100$.\nThe real part of the OTOC rapidly drops from the initial value within the time scale of $tJ_{\\rm SYK}\\sim 2$,\nand slowly decays to zero in the long time limit.\nAs one increases $N$, the oscillations that appear at longer time tend to be suppressed.\nOne can see that the result for $N=12$ in the top panels of Fig.~\\ref{fig:SYK} closely resembles the OTOC in \nthe spin-freezing crossover regime of the two-orbital Hubbard model shown\nby the blue curve in the top panels of Fig.~\\ref{fig:2orbital},\nwhile it differs from the single-orbital result shown in the top panels of Fig.~\\ref{fig:results_nsns}.\nDue to the limitation of our calculations to small system sizes ($N\\le 12$), we do not clearly observe\nan exponential growth ($\\sim c_0-c_1 e^{\\lambda t}$)  at short times or an exponential decay ($\\sim e^{-\\alpha t}$) \nat intermediate times. \nHowever, we find that for $tJ_{\\rm SYK}\\gtrsim 2$ the OTOC decays\napproximately in a power law $\\propto t^{-\\gamma}$ with $\\gamma=1.6$ (see bottom right panel in Fig.~\\ref{fig:SYK}). \nThe time interval exhibiting the power-law decay becomes longer as we increase $N$.  \nThe temperature dependence of the OTOC for the SYK model is shown in Fig.~\\ref{fig:SYK-T}.\nThe time scale of the initial drop is more or less independent of the temperature, while \nthe power-law-like decay is only visible in the low-temperature regime.\n\n\\begin{figure}[t]\n\\begin{center}\n\\includegraphics[angle=-90, width=0.49\\columnwidth]{plot_syk_beta_re.eps}\n\\includegraphics[angle=-90, width=0.49\\columnwidth]{plot_syk_beta_im.eps}\n\\includegraphics[angle=-90, width=0.49\\columnwidth]{plot_syk_beta_modulus-log.eps}\n\\includegraphics[angle=-90, width=0.49\\columnwidth]{plot_syk_beta_modulus.eps}\n\\caption{\nTemperature dependence of the density-density OTOC\n$\\overline{\\langle \\hat n_i(t)\\hat n_i(0), \\hat n_i(t)\\hat n_i(0)\\rangle-\\langle \\hat n_i\\rangle^4}$ for the particle-hole symmetric SYK model averaged over $10^2$ samples with $N=12$.\nThe top left and right panels show the real and imaginary parts of the OTOC,\nwhile the bottom left and right panels show the log and log-log plots of the modulus of the OTOC.\nThe dashed line shows a power-law decay $\\propto t^{-1.6}$ for comparison.}\n\\label{fig:SYK-T}\n\\end{center}\n\\end{figure}\n\nIn Ref.~\\onlinecite{Werner2018}, it has been argued that the spin-freezing crossover regime of multi-orbital Hubbard models is effectively described by the SYK model in which $J_{\\rm SYK}$ is replaced by the Hund coupling $J$. The results in Fig.~\\ref{fig:2orbital} are for the two-orbital Hubbard model with $U=8$, $J/U=1/4$ and $\\beta=50$, corresponding to $\\beta J=100$, which means that it is meaningful to directly compare the blue lines in Fig.~\\ref{fig:2orbital} with the blue lines in Figs.~\\ref{fig:SYK} and \\ref{fig:SYK-T}. Not only is the qualitative agreement remarkable, even the power-law exponent measured for the two-orbital Hubbard model ($\\gamma=1.5$) is very close to the exponent extracted from the finite-$N$ SYK calculation. \nThe power-law decay in the three-orbital case is approximately $1/t^{1.75}$ (see Appendix~\\ref{sec:3orbital}), and thus also in semi-quantitative agreement with the SYK model behavior. \nThis nontrivial result provides further support for the identification of the spin-freezing crossover regime of multi-orbital Hubbard systems with an SYK strange metal. \n\nIn the large-$N$ and low-$T$ limit of the SYK model, we expect that the power-law behavior approaches $1/t^2$, since in this limit the OTOC is decoupled as $\\langle \\hat n_i(t)\\hat n_i(0), \\hat n_i(t)\\hat n_i(0)\\rangle\\approx \\langle \\hat n_i(t)\\hat n_i(0)\\rangle^2$ and the decay of $\\langle \\hat n_i(t)\\hat n_i(0)\\rangle$ is dominated by what corresponds to the slow spin relaxation $\\langle \\hat S_z(t)\\hat S_z(0)\\rangle\\sim 1/t$ in the Sachdev-Ye model.\n\n\n\n\n\n4.2 Single-orbital Hubbard model\n\\subsection{Single-orbital Hubbard model}\n\\label{sec:hubbard}\n\nIn this section, we calculate the OTOCs (\\ref{fermion_otoc})-(\\ref{otoc_nn}) for the interacting single-orbital Hubbard model with the Hamiltonian \n\\begin{align}\nH\n&=\n-v\\sum_{\\langle i,j\\rangle\\sigma} (c^\\dagger_{i\\sigma}c_{j\\sigma}+ \\text{h.c.})\n-\\mu\\sum_i \\hat n_i+U\\sum_i \\hat n_{i\\uparrow}\\hat n_{i\\downarrow}, \n\\end{align}\nwhere $v$ is the hopping amplitude, $\\langle i,j\\rangle$ represents the nearest-neighbor site pairs, \n$\\mu$ is the chemical potential, and $U$ is the on-site interaction.\nIn this section, we focus on the half-filled case (i.e., $\\mu=U/2$). The DMFT solution for the simplified self-consistency (\\ref{semicirc}) is the exact solution for an infinitely connected Bethe lattice \\cite{GeorgesKotliarKrauthRozenberg1996}.\nLet us briefly recall the paramagnetic phase diagram for the single-orbital Hubbard model on this lattice \\cite{Bluemer_thesis}. At half-filling, there is a metal-insulator crossover at high temperature and a first-order Mott transition below a temperature corresponding to $\\beta\\hbar\\approx 17$ with a coexistence region between $U_{c1}$ and $U_{c2}$. The finite-temperature critical endpoint is at $U\\approx 4.7$, while the zero-temperature Mott transition occurs at $U=U_{c2}\\approx 5.6$. \nAt low temperature, the self-energy shows the Fermi liquid behavior [${\\rm Im}\\,\\Sigma(\\omega)\\sim \\omega^2$] \nin the weakly correlated metallic phase, \nwhile it shows an insulating behavior [${\\rm Im}\\,\\Sigma(\\omega)\\sim \\delta(\\omega)$] in the Mott phase\n\\cite{GeorgesKotliarKrauthRozenberg1996}. \nIn the correlated metallic phase, as the temperature is increased, deviations from the Fermi liquid behavior become apparent, \nbut an ${\\rm Im}\\,\\Sigma(\\omega)\\sim \\sqrt{\\omega}$ scaling as in the SYK model is not observed. \n\nIn the following, we will start the DMFT iterations from the noninteracting solution, which means that the finite-temperature Mott transition occurs at $U_{c2}$.\nFrom now on, we set $\\hbar=1$. \n\n\\begin{figure}[t]\n\\begin{center}\n\\includegraphics[angle=-90, width=0.49\\columnwidth]{plot_otoc_b50_Re.eps}\n\\includegraphics[angle=-90, width=0.49\\columnwidth]{plot_otoc_b50_Im.eps}\n\\includegraphics[angle=-90, width=0.49\\columnwidth]{plot_coth_b50_appendix.eps}\n\\includegraphics[angle=-90, width=0.49\\columnwidth]{plot_otoc_b50_modulus.eps}\n\\caption{\nTop panels: Real and imaginary parts of the OTOC $\\langle c^\\dagger_\\sigma(t)c_\\sigma(0), c^\\dagger_\\sigma(t)c_\\sigma(0)\\rangle$ for the single-orbital Hubbard model with $\\beta=50$ at half filling. The thin lines correspond to metallic solutions, while the thick line ($U=6$) corresponds to a Mott insulating solution. \nBottom left panel: Spectral function $\\mathscr A_{(c_\\sigma^\\dagger c_\\sigma)^2}(\\omega)=-\\frac{1}{\\pi}{\\rm Im}\\,C_{(c^\\dagger_\\sigma c_\\sigma)^2}^R(\\omega)$ multiplied by $\\coth\\big(\\frac{\\beta\\omega}{4}\\big)$. \nBottom right panel: Modulus of the OTOC $|\\langle c^\\dagger_\\sigma(t)c_\\sigma(0), c^\\dagger_\\sigma(t)c_\\sigma(0)\\rangle|$ (normalized at $t=0$) on a log-log scale. The dashed line is the result for $U=0$.\n}\n\\label{fig:results_fermion}\n\\end{center}\n\\end{figure}\n\n\n\nIn Fig.~\\ref{fig:results_fermion}, we show the results of $\\langle c^\\dagger_\\sigma(t)c_\\sigma(0), c^\\dagger_\\sigma(t)c_\\sigma(0)\\rangle$ for the interacting single-orbital Hubbard model at $\\beta=50$. \nThe top panels present the real and imaginary parts of the OTOC. One can see that the oscillations become more pronounced as one increases the interaction. In the bottom left panel of Fig.~\\ref{fig:results_fermion},\nwe show the corresponding spectral function $\\mathscr A_{(c_\\sigma^\\dagger c_\\sigma)^2}(\\omega)=-\\frac{1}{\\pi}{\\rm Im}\\,C_{(c^\\dagger_\\sigma c_\\sigma)^2}^R(\\omega)$ multiplied by $\\coth\\big(\\frac{\\beta\\omega}{4}\\big)$.\nThe Mott transition, which occurs between $U=5$ and $U=6$ at $\\beta=50$, manifests itself by the opening of a gap in the spectral function $\\mathscr A_{(c_\\sigma^\\dagger c_\\sigma)^2(\\omega)}$. This translates into more weakly damped oscillations in the real-time evolution in the Mott phase.  \nThe bottom right panel of Fig.~\\ref{fig:results_fermion} plots the modulus of the OTOC (normalized at time $t=0$, i.e., $|\\langle c^\\dagger_\\sigma(t)c_\\sigma(0), c^\\dagger_\\sigma(t)c_\\sigma(0)\\rangle|/|\\langle c^\\dagger_\\sigma(0)c_\\sigma(0), c^\\dagger_\\sigma(0)c_\\sigma(0)\\rangle|$) on a log-log scale. \nDue to the oscillations and the limited accuracy of the analytic continuation procedure, it is hard to clearly identify the nature of the long-time decay of the OTOC,\nbut the results indicate that the OTOC decays much faster (possibly exponentially) in the Mott phase than in the metallic phase. This is consistent with the results for the spectral functions of the OTOC. The comparison to the noninteracting case (dashed curve in the bottom right panel of Fig.~\\ref{fig:results_fermion}) suggests that the correlated metallic phase has a similar (possibly power-law) decay behavior of the OTOC as in the noninteracting case.\n\n\n\\begin{figure}[t]\n\\begin{center}\n\\includegraphics[angle=-90, width=0.49\\columnwidth]{plot_otoc_b50_nn_Re.eps}\n\\includegraphics[angle=-90, width=0.49\\columnwidth]{plot_otoc_b50_nn_Im.eps}\n\\includegraphics[angle=-90, width=0.49\\columnwidth]{plot_coth_b50_nn_appendix.eps}\n\\includegraphics[angle=-90, width=0.49\\columnwidth]{plot_otoc_b50_nn_modulus.eps}\n\\caption{\nTop panels: Real and imaginary parts of the OTOC $\\langle \\hat n(t)\\hat n(0), \\hat n(t)\\hat n(0)\\rangle - 1$ for the single-orbital Hubbard model with $\\beta=50$ at half filling. The thin lines correspond to metallic solutions, \nwhile the thick line ($U=6$) corresponds to a Mott insulating solution. \nBottom left panel: Spectral function $\\mathscr A_{(nn)^2}(\\omega)=-\\frac{1}{\\pi}{\\rm Im}\\,C_{(n n)^2}^R(\\omega)$ multiplied by $\\coth\\big(\\frac{\\beta\\omega}{4}\\big)$.\nBottom right panel: Modulus of the OTOC $|\\langle \\hat n(t)\\hat n(0), \\hat n(t)\\hat n(0)\\rangle - 1|$ (normalized at $t=0$) on a log-log scale. \n}\n\\label{fig:results_nn}\n\\end{center}\n\\end{figure}\n\n\nIn Fig.~\\ref{fig:results_nn}, we plot the OTOC $\\langle \\hat n(t)\\hat n(0), \\hat n(t)\\hat n(0)\\rangle$\nfor the single-orbital Hubbard model in a similar manner as in Fig.~\\ref{fig:results_fermion}.\nSince at half filling this correlation function approaches $\\langle \\hat n\\rangle^4=1$ at long times\nand at sufficiently low temperature,\nwe perform the analytical continuation procedure for $\\langle \\hat n(t)\\hat n(0), \\hat n(t)\\hat n(0)\\rangle-1$. \nThe top panels of Fig.~\\ref{fig:results_nn} show the real and imaginary parts of this shifted OTOC. \nContrary to the case of $\\langle c_\\sigma^\\dagger(t)c_\\sigma(0), c_\\sigma^\\dagger(t)c_\\sigma(0)\\rangle$, \nthe amplitude of the OTOC $\\langle \\hat n(t)\\hat n(0), \\hat n(t)\\hat n(0)\\rangle-1$ is suppressed as one increases the interaction, which reflects the reduced charge fluctuations in the Mott insulator.\nThe bottom left panel in Fig.~\\ref{fig:results_nn} shows the analytically continued spectral function $\\mathscr A_{(nn)^2}(\\omega)=-\\frac{1}{\\pi}{\\rm Im}\\, C_{(nn)^2}^R(\\omega)$ multiplied by $\\coth\\big(\\frac{\\beta\\omega}{4}\\big)$. \nAgain we notice the opening of a gap in the Mott phase, and a corresponding shift of the spectral weight to higher energies. This results in more rapid oscillations of the OTOC in the Mott state.\nThe bottom right panel in Fig.~\\ref{fig:results_nn} plots the modulus of the OTOC $|\\langle \\hat n(t)\\hat n(0), \\hat n(t)\\hat n(0)\\rangle-1|$ (normalized at $t=0$) on a log-log scale.\nThe long-time behavior of the OTOC is consistent with a power-law decay in the metallic phase, and an exponential decay in the Mott insulating phase.\n\n\n\\begin{figure}[t]\n\\begin{center}\n\\includegraphics[angle=-90, width=0.49\\columnwidth]{plot_otoc_nsns_b50_draft_Re.eps}\n\\includegraphics[angle=-90, width=0.49\\columnwidth]{plot_otoc_nsns_b50_draft_Im.eps}\n\\includegraphics[angle=-90, width=0.49\\columnwidth]{plot_coth_b50_nsns_inset.eps}\n\\includegraphics[angle=-90, width=0.49\\columnwidth]{plot_otoc_nsns_b50_draft_modulus_zoom.eps}\n\\caption{\nTop panels: Real and imaginary parts of the OTOC $\\langle \\hat n_\\sigma(t)\\hat n_\\sigma(0), \\hat n_\\sigma(t)\\hat n_\\sigma(0)\\rangle  - \\langle \\hat n_\\sigma\\rangle^4$ for the single-orbital Hubbard model with $\\beta=50$ at half filling. All the colored lines correspond to metallic solutions. \nBottom left panel: Spectral function $\\mathscr A_{(n_\\sigma n_\\sigma)^2}(\\omega)=-\\frac{1}{\\pi}{\\rm Im}\\,C_{(n_\\sigma n_\\sigma)^2}^R(\\omega)$ multiplied by $\\coth\\big(\\frac{\\beta\\omega}{4}\\big)$. In the inset, we plot the imaginary-time four-point function $\\langle \\hat n_\\sigma(\\tau)\\hat n_\\sigma(0), \\hat n_\\sigma(\\tau)\\hat n_\\sigma(0)\\rangle-\\langle \\hat n_\\sigma\\rangle^4$, with the black line corresponding to the Mott insulating solution at $U=6$.  \nBottom right panel: Modulus of the OTOC $|\\langle \\hat n_\\sigma(t)\\hat n_\\sigma(0), \\hat n_\\sigma(t)\\hat n_\\sigma(0)\\rangle  - \\langle \\hat n_\\sigma\\rangle^4|$ (normalized at $t=0$) on a log-log scale.\n}\n\\label{fig:results_nsns}\n\\end{center}\n\\end{figure}\n\nFigure~\\ref{fig:results_nsns} \nshows the results of the OTOC $\\langle \\hat n_\\sigma(t)\\hat n_\\sigma(0), \\hat n_\\sigma(t)\\hat n_\\sigma(0)\\rangle$\nfor the single-orbital Hubbard model. At half filling, this correlation function approaches $\\langle \\hat n_\\sigma\\rangle^4=\\frac{1}{16}$ in the long-time limit and at sufficiently low temperature, \nso that we perform the analytical continuation procedure for $\\langle \\hat n_\\sigma(t)\\hat n_\\sigma(0), \\hat n_\\sigma(t)\\hat n_\\sigma(0)\\rangle-\\langle \\hat n_\\sigma\\rangle^4$. \nThe top panels of Fig.~\\ref{fig:results_nsns} show the real and imaginary parts of \nthis shifted OTOC. \nHere, we only present results for the metallic phase, because resolving the very sharp low-energy feature of the OTOC spectrum in the half-filled Mott state is challenging. One can see that coherent oscillations are not observed for this type of OTOC, and that the incoherent part becomes dominant compared to the other two OTOCs shown in Figs.~\\ref{fig:results_fermion} and \\ref{fig:results_nn}. The amplitude of the OTOC $\\langle \\hat n_\\sigma(t)\\hat n_\\sigma(0), \\hat n_\\sigma(t)\\hat n_\\sigma(0)\\rangle-\\langle \\hat n_\\sigma\\rangle^4$ is enhanced as $U$ increases. As we argue below, this is related to the enhancement of spin fluctuations \nand local moment formation close to the Mott transition.\n\nThe bottom left panel in Fig.~\\ref{fig:results_nsns} plots the analytically continued spectral function $\\mathscr A_{(n_\\sigma n_\\sigma)^2}(\\omega)=-\\frac{1}{\\pi}{\\rm Im}\\, C_{(n_\\sigma n_\\sigma)^2}^R(\\omega)$ multiplied by $\\coth\\big(\\frac{\\beta\\omega}{4}\\big)$. \nWe can see that the spectral weight is concentrated in the low-energy region as we approach the Mott transition point.\nThe inset in the bottom left panel of Fig.~\\ref{fig:results_nsns} shows the imaginary-time four-point function $\\langle \\hat n_\\sigma(\\tau)\\hat n_\\sigma(0), \\hat n_\\sigma(\\tau)\\hat n_\\sigma(0)\\rangle-\\langle \\hat n_\\sigma\\rangle^4$, with the black line corresponding to a Mott insulating solution ($U=6$).  \nFor $U\\gtrsim 4$, the imaginary-time four-point function does not decay to zero but remains relatively large near $\\tau=\\frac{\\beta}{4}$. \nThis behavior is reminiscent of the spin-freezing physics seen in the correlated Hund metal phase of multi-orbital Hubbard models, where the dynamical spin correlation function $\\langle \\hat S_z(\\tau)\\hat S_z(0)\\rangle$ is trapped at a finite value at long $\\tau$ \\cite{Werner2008}. \nIn fact, the OTOC $\\langle \\hat n_\\sigma(\\tau)\\hat n_\\sigma(0), \\hat n_\\sigma(\\tau)\\hat n_\\sigma(0)\\rangle$\nmeasures spin correlations (in addition to charge correlations) through $\\hat n_\\sigma=\\frac{1}{2}\\hat n+\\sigma\\hat S_z$.\nPhysically, the trapping of $\\langle \\hat S_z(\\tau)\\hat S_z(0)\\rangle$ signals the formation of frozen local magnetic moments.\nIn the metal-insulator crossover region, the scattering induced by these magnetic moments results in incoherent metal states. Intuitively, one might expect fast scrambling in the corresponding regions of the phase diagram. \nAlthough there is some resemblance with the spin freezing crossover,\nthe self-energy does not exhibit a square-root frequency dependence in the single-orbital case, and a spin-freezing crossover in the sense of Ref.~\\cite{Werner2008} does not exist.  \n\nThe bottom right panel of Fig.~\\ref{fig:results_nsns} plots the modulus of the OTOC $|\\langle \\hat n_\\sigma(t)\\hat n_\\sigma(0), \\hat n_\\sigma(t)\\hat n_\\sigma(0)\\rangle-\\langle \\hat n_\\sigma\\rangle^4|$ (normalized at $t=0$) on a log-log scale. \nThe dynamics of $\\langle \\hat n_\\sigma(t)\\hat n_\\sigma(0), \\hat n_\\sigma(t)\\hat n_\\sigma(0)\\rangle$ is quite distinct from that of $\\langle \\hat n(t)\\hat n(0), \\hat n(t)\\hat n(0)\\rangle$. As the Mott transition is approached, we observe a slow-down in the decay of the modulus, \nwhich indicates the presence of slowly fluctuating local moments. \nThe long-time behavior may be consistent with a power-law, although slow oscillations make it difficult to determine the long-time asymptotic form from the numerics. \n\n\n\\begin{figure}[t]\n\\begin{center}\n\\includegraphics[angle=-90, width=0.49\\columnwidth]{plot_fitnsns_b2.eps}\n\\includegraphics[angle=-90, width=0.49\\columnwidth]{plot_coeffc_draft.eps}\n\\caption{\nLeft panel: \nThe imaginary-time four-point function $\\langle \\hat n_\\sigma(\\tau)\\hat n_\\sigma(0), \\hat n_\\sigma(\\tau)\\hat n_\\sigma(0)\\rangle$, the squared density-density correlation function $\\langle \\hat n_\\sigma(\\tau)\\hat n_\\sigma(0)\\rangle^2$,\nand their difference for the single-orbital Hubbard model with $U=5.5$ and $\\beta=50$.\nThe dashed curve shows a fit of the difference with the function $f_\\text{fit}(\\tau)=a+b\\tau+c\\tau^2$ in the interval $\\tau/\\beta \\in [0: 0.2]$.\nRight panel: Fitting coefficient $c$ as a function of $U$ for indicated values of $\\beta$.\n\n}\n\\label{fig:coeffs}\n\\end{center}\n\\end{figure}\n\nFor large enough $\\beta$ and small enough $\\tau$, the OTOC (\\ref{otoc_nsns}) factorizes into $\\langle \\hat n_\\sigma(\\tau)\\hat n_\\sigma(0), \\hat n_\\sigma(\\tau)\\hat n_\\sigma(0)\\rangle \\approx \\langle \\hat n_\\sigma(\\tau)\\hat n_\\sigma(0)\\rangle^2$. \nOn the real-time axis, this corresponds to the decoupling $\\langle \\hat n_\\sigma(t)\\hat n_\\sigma(0), \\hat n_\\sigma(t)\\hat n_\\sigma(0)\\rangle \\approx \\langle \\hat n_\\sigma(t)\\hat n_\\sigma(0)\\rangle^2$, that is, \nthe OTOC is reduced to a product of ordinary density-density correlation functions.\nIn order to see whether nontrivial correlations are captured beyond the decoupled form by the OTOC function, \nwe consider the difference $\\langle \\hat n_\\sigma(\\tau)\\hat n_\\sigma(0), \\hat n_\\sigma(\\tau)\\hat n_\\sigma(0)\\rangle-\\langle \\hat n_\\sigma(\\tau)\\hat n_\\sigma(0)\\rangle^2$ and fit the short-time behavior of this function to $f_\\text{fit}(\\tau)=a+b\\tau+c\\tau^2$, as illustrated in the left panel of Fig.~\\ref{fig:coeffs}. The coefficients $b$ and $c$ serve as  indicators of the nontrivial correlations that cannot be attributed to the decoupled form of the OTOC.\nThe coefficient $c$ is plotted as a function of $U$ and for different $\\beta$ in the right hand panel. The coefficient $b$ (not shown) exhibits a similar trend. We notice that the OTOC picks up nontrival correlations near the metal-insulator transition, and in particular at \nintermediate\ntemperatures, while the Mott phase shows no such correlations. The crossover region  \nassociated with the emergence of local moments \ncorresponds, roughly, to the interaction range where the nontrivial correlations start to become significant. \n\n\n4.4 Comparison to the SKY model\n\\subsection{Comparison to the SKY model}\n\\label{sec:SYK}\n\nIn this section, we compare the results of the multi-orbital Hubbard models to the SYK model.\nWe consider the particle-hole symmetric SYK model of complex fermions \\cite{FuSachdev2016},\nwhose Hamiltonian is given by\n\\begin{align}\nH\n&=\n\\frac{1}{(2N)^{3/2}} \\sum_{i,j,k,l=1}^N \nJ_{ij;kl} (c_i^\\dagger c_j^\\dagger c_k c_l\n+\\delta_{ik}n c_j^\\dagger c_l\n-\\delta_{il}n c_j^\\dagger c_k\n-\\delta_{jk}n c_i^\\dagger c_l\n+\\delta_{jl}n c_i^\\dagger c_k).\n\\end{align}\nThe coupling constant $J_{ij;kl}$ is a gaussian random variable, satisfying $J_{ij;kl}=-J_{ji;kl}=-J_{ij;lk}=J_{kl;ij}^\\ast$, and\n\\begin{align}\n\\overline{({\\rm Re}\\, J_{ij;kl})^2}\n&=\n\\begin{cases}\nJ_{\\rm SYK}^2/2 & (i,j)\\neq (k,l) \\\\\nJ_{\\rm SYK}^2 & (i,j)=(k,l)\n\\end{cases},\n\\\\\n\\overline{({\\rm Im}\\, J_{ij;kl})^2}\n&=\n\\begin{cases}\nJ_{\\rm SYK}^2/2 & (i,j)\\neq (k,l) \\\\\n0 & (i,j)=(k,l)\n\\end{cases},\n\\end{align}\nwhere the overline represents an average over each realization of $J_{ij;kl}$. The parameter \n$J_{\\rm SYK}$ ($J_{\\rm SYK}^{-1}$) defines the unit of energy (time) in the following calculations. \nThe system is particle-hole symmetric when\n$n=0.5$, which fixes the total number of fermions as $N^{-1}\\sum_{i=1}^N \\langle c_i^\\dagger c_i\\rangle=0.5$.\nWe take the particle-hole symmetric form of the SYK model because it has been well studied previously \\cite{FuSachdev2016} and avoids the effect of the filling drift.\nWe numerically solve the model by exact diagonalization for finite $N(\\le 12)$, and evaluate the density-density OTOC\n$\\overline{\\langle \\hat n_i(t)\\hat n_i(0), \\hat n_i(t)\\hat n_i(0)\\rangle-\\langle \\hat n_i\\rangle^4}$\nand its imaginary-time counterpart.\n\n\\begin{figure}[t]\n\\begin{center}\n\n\\includegraphics[angle=-90,width=0.49\\columnwidth]{plot_fitnsns_syk.eps}\n\n\\includegraphics[angle=-90,width=0.49\\columnwidth]{plot_coeffc_syk.eps}\n\\caption{\nLeft panel: \nThe imaginary-time four-point function $\\overline{\\langle \\hat n_i(\\tau)\\hat n_i(0), \\hat n_i(\\tau)\\hat n_i(0)\\rangle}$, the density-density correlation function $\\overline{\\langle \\hat n_i(\\tau)\\hat n_i(0)\\rangle^2}$,\nand their difference for the SYK model with $N=12$ and $\\beta J_{\\rm SYK}=10$.\nThe dashed curve shows a fit of the difference with a function $f_\\text{fit}(\\tau)=a+b\\tau+c\\tau^2$ in the interval $\\tau/\\beta \\in [0: 0.2]$. \nRight panel: Fitting coefficient $c$ as a function of $T/J_{\\rm SYK}$.\n}\n\\label{fig:SYK fit}\n\\end{center}\n\\end{figure}\n\nIn the left panel of Fig.~\\ref{fig:SYK fit}, we plot the imaginary-time four-point function \n$\\overline{\\langle \\hat n_i(\\tau)\\hat n_i(0), \\hat n_i(\\tau)\\hat n_i(0)\\rangle}$\nfor the SYK model with $N=12$ and $\\beta J_{\\rm SYK}=10$. \nAt low temperature, we expect that $\\overline{\\langle \\hat n_i(\\tau)\\hat n_i(0), \\hat n_i(\\tau)\\hat n_i(0)\\rangle}$ is decoupled into $\\overline{\\langle \\hat n_i(\\tau)\\hat n_i(0)\\rangle^2}$.\nTo see whether nontrivial correlations beyond the decoupled form are present, we fit the difference\n$\\overline{\\langle \\hat n_i(\\tau)\\hat n_i(0), \\hat n_i(\\tau)\\hat n_i(0)\\rangle}\n-\\overline{\\langle \\hat n_i(\\tau)\\hat n_i(0)\\rangle^2}$ with a function $f_{\\rm fit}(\\tau)=a+b\\tau+c\\tau^2$\nin the interval $\\tau/\\beta\\in[0:0.2]$, in the same manner as for the Hubbard models.\nThe obtained coefficient $c$ is plotted as a function of the temperature $T$ in the right panel of\nFig.~\\ref{fig:SYK fit}. One can see that nontrivial correlations exist in a wide range of temperature.\nEspecially, they are enhanced in the intermediate temperature regime. This is because in the zero-temperature limit\nthe four-point function is decoupled as \n$\\langle \\hat n_i(\\tau)\\hat n_i(0), \\hat n_i(\\tau)\\hat n_i(0)\\rangle \\approx \\langle \\hat n_i(\\tau)\\hat n_i(0)\\rangle^2$, while in the high-temperature limit ($\\beta\\to 0$) the imaginary-time dependence is washed out.\n\n\n\n\n\\begin{figure}[t]\n\\begin{center}\n\\includegraphics[angle=-90, width=0.49\\columnwidth]{plot_syk_Re.eps}\n\\includegraphics[angle=-90, width=0.49\\columnwidth]{plot_syk_Im.eps}\n\\includegraphics[angle=-90, width=0.49\\columnwidth]{plot_syk_modulus-log.eps}\n\\includegraphics[angle=-90, width=0.49\\columnwidth]{plot_syk_modulus.eps}\n\\caption{\nThe density-density OTOC $\\overline{\\langle \\hat n_i(t)\\hat n_i(0), \\hat n_i(t)\\hat n_i(0)\\rangle-\\langle \\hat n_i\\rangle^4}$ for the particle-hole symmetric SYK model averaged over $10^2$ samples with $\\beta J_{\\rm SYK}=100$ and various $N$. \nThe top left and right panels show the real and imaginary parts of the OTOC,\nwhile the bottom left and right panels show the log and log-log plots of the modulus of the OTOC (normalized at $t=0$).\nThe dashed line shows a power-law decay $\\propto t^{-1.6}$ for comparison.}\n\\label{fig:SYK}\n\\end{center}\n\\end{figure}\n\nIn Fig.~\\ref{fig:SYK}, we show the numerical results of the density-density OTOC for $\\beta J_{\\rm SYK}=100$.\nThe real part of the OTOC rapidly drops from the initial value within the time scale of $tJ_{\\rm SYK}\\sim 2$,\nand slowly decays to zero in the long time limit.\nAs one increases $N$, the oscillations that appear at longer time tend to be suppressed.\nOne can see that the result for $N=12$ in the top panels of Fig.~\\ref{fig:SYK} closely resembles the OTOC in \nthe spin-freezing crossover regime of the two-orbital Hubbard model shown\nby the blue curve in the top panels of Fig.~\\ref{fig:2orbital},\nwhile it differs from the single-orbital result shown in the top panels of Fig.~\\ref{fig:results_nsns}.\nDue to the limitation of our calculations to small system sizes ($N\\le 12$), we do not clearly observe\nan exponential growth ($\\sim c_0-c_1 e^{\\lambda t}$)  at short times or an exponential decay ($\\sim e^{-\\alpha t}$) \nat intermediate times. \nHowever, we find that for $tJ_{\\rm SYK}\\gtrsim 2$ the OTOC decays\napproximately in a power law $\\propto t^{-\\gamma}$ with $\\gamma=1.6$ (see bottom right panel in Fig.~\\ref{fig:SYK}). \nThe time interval exhibiting the power-law decay becomes longer as we increase $N$.  \nThe temperature dependence of the OTOC for the SYK model is shown in Fig.~\\ref{fig:SYK-T}.\nThe time scale of the initial drop is more or less independent of the temperature, while \nthe power-law-like decay is only visible in the low-temperature regime.\n\n\\begin{figure}[t]\n\\begin{center}\n\\includegraphics[angle=-90, width=0.49\\columnwidth]{plot_syk_beta_re.eps}\n\\includegraphics[angle=-90, width=0.49\\columnwidth]{plot_syk_beta_im.eps}\n\\includegraphics[angle=-90, width=0.49\\columnwidth]{plot_syk_beta_modulus-log.eps}\n\\includegraphics[angle=-90, width=0.49\\columnwidth]{plot_syk_beta_modulus.eps}\n\\caption{\nTemperature dependence of the density-density OTOC\n$\\overline{\\langle \\hat n_i(t)\\hat n_i(0), \\hat n_i(t)\\hat n_i(0)\\rangle-\\langle \\hat n_i\\rangle^4}$ for the particle-hole symmetric SYK model averaged over $10^2$ samples with $N=12$.\nThe top left and right panels show the real and imaginary parts of the OTOC,\nwhile the bottom left and right panels show the log and log-log plots of the modulus of the OTOC.\nThe dashed line shows a power-law decay $\\propto t^{-1.6}$ for comparison.}\n\\label{fig:SYK-T}\n\\end{center}\n\\end{figure}\n\nIn Ref.~\\onlinecite{Werner2018}, it has been argued that the spin-freezing crossover regime of multi-orbital Hubbard models is effectively described by the SYK model in which $J_{\\rm SYK}$ is replaced by the Hund coupling $J$. The results in Fig.~\\ref{fig:2orbital} are for the two-orbital Hubbard model with $U=8$, $J/U=1/4$ and $\\beta=50$, corresponding to $\\beta J=100$, which means that it is meaningful to directly compare the blue lines in Fig.~\\ref{fig:2orbital} with the blue lines in Figs.~\\ref{fig:SYK} and \\ref{fig:SYK-T}. Not only is the qualitative agreement remarkable, even the power-law exponent measured for the two-orbital Hubbard model ($\\gamma=1.5$) is very close to the exponent extracted from the finite-$N$ SYK calculation. \nThe power-law decay in the three-orbital case is approximately $1/t^{1.75}$ (see Appendix~\\ref{sec:3orbital}), and thus also in semi-quantitative agreement with the SYK model behavior. \nThis nontrivial result provides further support for the identification of the spin-freezing crossover regime of multi-orbital Hubbard systems with an SYK strange metal. \n\nIn the large-$N$ and low-$T$ limit of the SYK model, we expect that the power-law behavior approaches $1/t^2$, since in this limit the OTOC is decoupled as $\\langle \\hat n_i(t)\\hat n_i(0), \\hat n_i(t)\\hat n_i(0)\\rangle\\approx \\langle \\hat n_i(t)\\hat n_i(0)\\rangle^2$ and the decay of $\\langle \\hat n_i(t)\\hat n_i(0)\\rangle$ is dominated by what corresponds to the slow spin relaxation $\\langle \\hat S_z(t)\\hat S_z(0)\\rangle\\sim 1/t$ in the Sachdev-Ye model.\n\n\n\n\n\n",
        "label": "fig:results_nsns",
        "Descriptive_question1": "What is the value of beta used in the simulations shown in figure_7?",
        "Descriptive_question2": "What do all the colored lines in the top panels of figure_7 represent?",
        "Reasoning_question1": "Why is there an enhancement in the amplitude of the OTOC as U increases in figure_7, and how does this relate to physical phenomena near the Mott transition?",
        "Reasoning_question2": "How does the concentration of spectral weight in the low-energy region in the bottom left panel of figure_7 affect the dynamics of the OTOC as the Mott transition is approached?",
        "Descriptive_answer1": "50",
        "Descriptive_answer2": "metallic solutions",
        "Reasoning_answer1": "The amplitude enhancement of the OTOC with increasing U reflects stronger spin fluctuations and local moment formation near the Mott transition. As U increases, electron correlations become stronger, leading to the formation of local magnetic moments and slowing down of spin dynamics, which enhances the correlations captured by the OTOC amplitude.",
        "Reasoning_answer2": "The concentration of spectral weight in the low-energy region signifies slow dynamics and the presence of low-energy excitations near the Mott transition. This leads to a slower decay and more persistent correlations in the real-time evolution of the OTOC, indicating slow fluctuating local moments characterizing the crossover to the Mott insulating state."
    },
    {
        "paper_id": "1812.04217.json",
        "image_id": "figure_8",
        "image_path": [
            "/home/yz979/scratch/chengye/scimolmo/test-images-tables/dataset_imagestables/1812.04217/images/plot_fitnsns_b2.eps"
        ],
        "caption": "\nLeft panel: \nThe imaginary-time four-point function $\\langle \\hat n_\\sigma(\\tau)\\hat n_\\sigma(0), \\hat n_\\sigma(\\tau)\\hat n_\\sigma(0)\\rangle$, the squared density-density correlation function $\\langle \\hat n_\\sigma(\\tau)\\hat n_\\sigma(0)\\rangle^2$,\nand their difference for the single-orbital Hubbard model with $U=5.5$ and $\\beta=50$.\nThe dashed curve shows a fit of the difference with the function $f_\\text{fit}(\\tau)=a+b\\tau+c\\tau^2$ in the interval $\\tau/\\beta \\in [0: 0.2]$.\nRight panel: Fitting coefficient $c$ as a function of $U$ for indicated values of $\\beta$.\n\n",
        "classify": "Chart",
        "section_info": "4 Numerical results for the Hubbard model\n\\section{Numerical results for the Hubbard model}\n\\label{numerical results}\n\n\\subsection{Observables and numerical procedure}\n\nWe consider the single-orbital, two-orbital, and three-orbital Hubbard models on an infinitely-connected Bethe lattice, which can be solved exactly within DMFT \\cite{GeorgesKotliarKrauthRozenberg1996}. In this case, the noninteracting density of states is semicircular with bandwidth $4v_\\ast$, for which there exists a simplified DMFT self-consistency condition between the hybridization function $\\Delta_{\\alpha\\sigma}$ of the DMFT impurity problem and the local (impurity) Green's function $G_{\\alpha\\sigma}$: \n\n\\begin{equation}\n\\Delta_{\\alpha\\sigma}(\\tau)=v_\\ast^2 G_{\\alpha\\sigma}(\\tau), \n\\label{semicirc}\n\\end{equation}\n\nwith $\\alpha$ the orbital and $\\sigma$ the spin index. We will use $v_\\ast$ as the unit of energy and measure time in units of $\\hbar/v_\\ast$. \n\nThree different types of imaginary-time four-point functions $C_{(AB)^2}^M(\\tau)$\nof the form of Eq.~(\\ref{imaginary-time OTOC})\nwith $(\\hat A, \\hat B)=(c_\\sigma^\\dagger, c_\\sigma), (\\hat n_\\sigma, \\hat n_\\sigma)$, and $(\\hat n, \\hat n)$\nare calculated in the interval $0\\le \\tau\\le \\frac{\\beta\\hbar}{2}$: \n\\begin{eqnarray}\nC_{(c^\\dagger_\\sigma c_\\sigma)^2}^M(\\tau) &=& -\\langle c^\\dagger_\\sigma(\\tau+\\tfrac{\\beta\\hbar}{2}) c_\\sigma(\\tfrac{\\beta\\hbar}{2}) c^\\dagger_\\sigma(\\tau) c_\\sigma(0) \\rangle, \\label{fermion_otoc}\\\\\nC_{(n_\\sigma n_\\sigma)^2}^M(\\tau) &=& -\\langle \\hat n_\\sigma(\\tau+\\tfrac{\\beta\\hbar}{2})\\hat n_\\sigma(\\tfrac{\\beta\\hbar}{2})\\hat n_\\sigma(\\tau)\\hat n_\\sigma(0) \\rangle, \\label{otoc_nsns}\\\\\nC_{(nn)^2}^M(\\tau) &=& -\\langle \\hat n(\\tau+\\tfrac{\\beta\\hbar}{2})\\hat n(\\tfrac{\\beta\\hbar}{2})\\hat n(\\tau)\\hat n(0) \\rangle,\\label{otoc_nn}\n\\end{eqnarray}\n\nwhere $c_\\sigma^\\dagger$ ($c_\\sigma$) are the fermionic creation (annihilation) operators for spin $\\sigma$ (and orbital $\\alpha=1$ in the multi-orbital case), $\\hat n_\\sigma=c^\\dagger_\\sigma c_\\sigma$ is the corresponding spin-dependent density operator, and $\\hat n=\\hat n_\\uparrow + \\hat n_\\downarrow$ the total density operator. Note that in all the three cases above we have $\\hat B=\\hat A^\\dagger$. \nWe measure these local correlation functions in the impurity model using a hybridization expansion continuous-time Monte Carlo algorithm (CT-HYB) \\cite{Werner2006}. In this algorithm, the \ntwo-particle\nGreen's functions of the type (\\ref{fermion_otoc}) can be measured by removing two hybridization lines, i.e., from the elements of the inverse hybridization matrix, while the density-density correlation functions can be easily measured either by insertion of density operators (matrix formalism) \\cite{Werner2006Kondo} or by reading off the occupation of the orbitals at the four time points in the segment implementation \\cite{Werner2006}. We use the latter algorithm since we consider only density-density interactions. \nThe values at the end-points $\\tau=0$ and $\\tau=\\frac{\\beta\\hbar}{2}$ reduce to standard density-density correlation functions, which in the case of Eq.~(\\ref{fermion_otoc}) are measured separately. \n\nWe perform the analytic continuation to the real-frequency axis using the Maximum Entropy method \\cite{Jarrell1996,Lewin_maxent} with a bosonic kernel. This yields the imaginary part of the retarded correlator \n$-\\frac{1}{\\pi}\\text{Im}\\, C^R_{(AA^\\dagger)^2}(\\omega)=\\mathscr A_{(AA^\\dagger)^2}(\\omega)$.  \nWhen $\\hat B=\\hat A^\\dagger=\\hat A$ (which is the case for $\\hat A=\\hat n_\\sigma, \\hat n$), we have $C_{(AA)^2}^R(\\omega)^\\ast=C_{(AA)^2}^R(-\\omega)$. At half filling, the particle-hole symmetry furthermore ensures $C_{c_\\sigma^\\dagger c_\\sigma}^R(\\omega)^\\ast=C_{c_\\sigma^\\dagger c_\\sigma}^R(-\\omega)$.\nThus, in all the cases considered in this study, the retarded OTOC has the symmetry property $C_{(AA^\\dagger)^2}^R(\\omega)^\\ast=C_{(AA^\\dagger)^2}^R(-\\omega)$. Using this, as well as the out-of-time-order fluctuation-dissipation\ntheorem discussed in the previous section, we can obtain the real-time OTOC\n$\\langle \\hat A(t)\\hat A^\\dagger(0), \\hat A(t)\\hat A^\\dagger(0)\\rangle$ from the following inverse Fourier transformation, \n\\begin{align}{\\rm Re}\\langle \\hat A(t)\\hat A^\\dagger(0), \\hat A(t)\\hat A^\\dagger(0)\\rangle\n&=\n{\\rm Re}\n\\int_0^\\infty d\\omega\\, e^{-i\\omega t}\\coth\\left(\\frac{\\beta\\hbar\\omega}{4}\\right)\n\\left(-\\frac{1}{\\pi}{\\rm Im}\\, C_{(AA^\\dagger)^2}^R(\\omega)\\right),\n\\\\\n{\\rm Im}\\langle \\hat A(t)\\hat A^\\dagger(0), \\hat A(t)\\hat A^\\dagger(0)\\rangle\n&=\n{\\rm Im}\\int_0^{\\infty} d\\omega\\, e^{-i\\omega t} \\left(-\\frac{1}{\\pi}{\\rm Im}\\, C_{(AA^\\dagger)^2}^R(\\omega)\\right),\n\\end{align}\nfor $\\hat A=c_\\sigma^\\dagger, \\hat n_\\sigma$, and $\\hat n$.\n\n\\begin{figure}[b]\n\\begin{center}\n\\includegraphics[angle=-90, width=0.48\\columnwidth]{compare_u0_draft.eps}\\hfill\n\\includegraphics[angle=-90, width=0.48\\columnwidth]{compare_u0_longtime_modulus.eps}\n\\caption{\nLeft panel: Comparison between the exact solution and the result obtained by analytical continuation\nfor the real and imaginary parts of the OTOC $\\langle c_\\sigma^\\dagger(t)c_\\sigma(0), c_\\sigma^\\dagger(t)c_\\sigma(0)\\rangle$ \nin the noninteracting case ($U=0, \\beta\\hbar=50$) of the single-orbital Hubbard model.\nRight panel: Comparison of the long-time behavior for the modulus $|\\langle c_\\sigma^\\dagger(t)c_\\sigma(0), c_\\sigma^\\dagger(t)c_\\sigma(0)\\rangle|$. \nThe exact result shows a power-law decay ($\\sim 1/t^3$). \n}\n\\label{fig:test_u0}\n\\end{center}\n\\end{figure}\n\nAs a check of our procedure, we first compare the results for the noninteracting model,\nfor which an exact solution of the OTOC $\\langle c_\\sigma^\\dagger(t)c_\\sigma(0), c_\\sigma^\\dagger(t)c_\\sigma(0)\\rangle$ \nis available \\cite{TsujiWernerUeda2017}.\nIt is a nontrivial task to measure the noninteracting two-particle Green's function in CT-HYB, \nand to analytically continue \n$C_{(c^\\dagger_\\sigma c_\\sigma)^2}^M(\\tau)$ \nby the Maximum Entropy method.  \nThe results for the inverse temperature $\\beta\\hbar=50$ are plotted in Fig.~\\ref{fig:test_u0}. \nOne can see that the OTOC oscillates for about one cycle, and quickly decays to zero.\nThe analytically continued data show a good agreement with the exact solution. \nIn particular, for short times (up to about two inverse hoppings) the dynamics is accurately reproduced, while deviations appear at longer times. \nIt is known that for the noninteracting fermion system the OTOC decays as a power law at long time ($\\sim 1/t^3$) \\cite{TsujiWernerUeda2017}.\nThe analytical continuation method cannot reproduce the details of the oscillations at intermediate or long times, but it roughly captures the long-time decay, which is controlled by low-frequency spectral features. \nUsually, analytic continuation is unreliable for high frequency components, because high-frequency information is suppressed \nby the kernel $K(\\tau,\\omega)$ in the transformation from the spectral function $A_O(\\omega)$ to the (bosonic) Matsubara correlation function $O(\\tau)$ [$O(\\tau)=\\int_{-\\infty}^{\\infty} d\\omega\\, K(\\omega,\\tau) A_O(\\omega)$ with $K(\\tau,\\omega)=e^{-\\tau\\omega}/(1-e^{-\\beta\\omega})$]. On the other hand, the low frequency components can be rather accurately determined. As a result, in Fig. 4 we more or less recover the low-frequency features of the time-dependent correlation function, while the details of rapid oscillations are not captured. The accurate result at short times can be understood from the fact that this region is close to the imaginary time axis in the complex time plane, where the original QMC data are available.\nAt higher temperatures, the Maximum Entropy method becomes less reliable, so that the resulting OTOCs are expected to be less accurate.  \n\n\n\\subsection{Single-orbital Hubbard model}\n\\label{sec:hubbard}\n\nIn this section, we calculate the OTOCs (\\ref{fermion_otoc})-(\\ref{otoc_nn}) for the interacting single-orbital Hubbard model with the Hamiltonian \n\\begin{align}\nH\n&=\n-v\\sum_{\\langle i,j\\rangle\\sigma} (c^\\dagger_{i\\sigma}c_{j\\sigma}+ \\text{h.c.})\n-\\mu\\sum_i \\hat n_i+U\\sum_i \\hat n_{i\\uparrow}\\hat n_{i\\downarrow}, \n\\end{align}\nwhere $v$ is the hopping amplitude, $\\langle i,j\\rangle$ represents the nearest-neighbor site pairs, \n$\\mu$ is the chemical potential, and $U$ is the on-site interaction.\nIn this section, we focus on the half-filled case (i.e., $\\mu=U/2$). The DMFT solution for the simplified self-consistency (\\ref{semicirc}) is the exact solution for an infinitely connected Bethe lattice \\cite{GeorgesKotliarKrauthRozenberg1996}.\nLet us briefly recall the paramagnetic phase diagram for the single-orbital Hubbard model on this lattice \\cite{Bluemer_thesis}. At half-filling, there is a metal-insulator crossover at high temperature and a first-order Mott transition below a temperature corresponding to $\\beta\\hbar\\approx 17$ with a coexistence region between $U_{c1}$ and $U_{c2}$. The finite-temperature critical endpoint is at $U\\approx 4.7$, while the zero-temperature Mott transition occurs at $U=U_{c2}\\approx 5.6$. \nAt low temperature, the self-energy shows the Fermi liquid behavior [${\\rm Im}\\,\\Sigma(\\omega)\\sim \\omega^2$] \nin the weakly correlated metallic phase, \nwhile it shows an insulating behavior [${\\rm Im}\\,\\Sigma(\\omega)\\sim \\delta(\\omega)$] in the Mott phase\n\\cite{GeorgesKotliarKrauthRozenberg1996}. \nIn the correlated metallic phase, as the temperature is increased, deviations from the Fermi liquid behavior become apparent, \nbut an ${\\rm Im}\\,\\Sigma(\\omega)\\sim \\sqrt{\\omega}$ scaling as in the SYK model is not observed. \n\nIn the following, we will start the DMFT iterations from the noninteracting solution, which means that the finite-temperature Mott transition occurs at $U_{c2}$.\nFrom now on, we set $\\hbar=1$. \n\n\\begin{figure}[t]\n\\begin{center}\n\\includegraphics[angle=-90, width=0.49\\columnwidth]{plot_otoc_b50_Re.eps}\n\\includegraphics[angle=-90, width=0.49\\columnwidth]{plot_otoc_b50_Im.eps}\n\\includegraphics[angle=-90, width=0.49\\columnwidth]{plot_coth_b50_appendix.eps}\n\\includegraphics[angle=-90, width=0.49\\columnwidth]{plot_otoc_b50_modulus.eps}\n\\caption{\nTop panels: Real and imaginary parts of the OTOC $\\langle c^\\dagger_\\sigma(t)c_\\sigma(0), c^\\dagger_\\sigma(t)c_\\sigma(0)\\rangle$ for the single-orbital Hubbard model with $\\beta=50$ at half filling. The thin lines correspond to metallic solutions, while the thick line ($U=6$) corresponds to a Mott insulating solution. \nBottom left panel: Spectral function $\\mathscr A_{(c_\\sigma^\\dagger c_\\sigma)^2}(\\omega)=-\\frac{1}{\\pi}{\\rm Im}\\,C_{(c^\\dagger_\\sigma c_\\sigma)^2}^R(\\omega)$ multiplied by $\\coth\\big(\\frac{\\beta\\omega}{4}\\big)$. \nBottom right panel: Modulus of the OTOC $|\\langle c^\\dagger_\\sigma(t)c_\\sigma(0), c^\\dagger_\\sigma(t)c_\\sigma(0)\\rangle|$ (normalized at $t=0$) on a log-log scale. The dashed line is the result for $U=0$.\n}\n\\label{fig:results_fermion}\n\\end{center}\n\\end{figure}\n\n\n\nIn Fig.~\\ref{fig:results_fermion}, we show the results of $\\langle c^\\dagger_\\sigma(t)c_\\sigma(0), c^\\dagger_\\sigma(t)c_\\sigma(0)\\rangle$ for the interacting single-orbital Hubbard model at $\\beta=50$. \nThe top panels present the real and imaginary parts of the OTOC. One can see that the oscillations become more pronounced as one increases the interaction. In the bottom left panel of Fig.~\\ref{fig:results_fermion},\nwe show the corresponding spectral function $\\mathscr A_{(c_\\sigma^\\dagger c_\\sigma)^2}(\\omega)=-\\frac{1}{\\pi}{\\rm Im}\\,C_{(c^\\dagger_\\sigma c_\\sigma)^2}^R(\\omega)$ multiplied by $\\coth\\big(\\frac{\\beta\\omega}{4}\\big)$.\nThe Mott transition, which occurs between $U=5$ and $U=6$ at $\\beta=50$, manifests itself by the opening of a gap in the spectral function $\\mathscr A_{(c_\\sigma^\\dagger c_\\sigma)^2(\\omega)}$. This translates into more weakly damped oscillations in the real-time evolution in the Mott phase.  \nThe bottom right panel of Fig.~\\ref{fig:results_fermion} plots the modulus of the OTOC (normalized at time $t=0$, i.e., $|\\langle c^\\dagger_\\sigma(t)c_\\sigma(0), c^\\dagger_\\sigma(t)c_\\sigma(0)\\rangle|/|\\langle c^\\dagger_\\sigma(0)c_\\sigma(0), c^\\dagger_\\sigma(0)c_\\sigma(0)\\rangle|$) on a log-log scale. \nDue to the oscillations and the limited accuracy of the analytic continuation procedure, it is hard to clearly identify the nature of the long-time decay of the OTOC,\nbut the results indicate that the OTOC decays much faster (possibly exponentially) in the Mott phase than in the metallic phase. This is consistent with the results for the spectral functions of the OTOC. The comparison to the noninteracting case (dashed curve in the bottom right panel of Fig.~\\ref{fig:results_fermion}) suggests that the correlated metallic phase has a similar (possibly power-law) decay behavior of the OTOC as in the noninteracting case.\n\n\n\\begin{figure}[t]\n\\begin{center}\n\\includegraphics[angle=-90, width=0.49\\columnwidth]{plot_otoc_b50_nn_Re.eps}\n\\includegraphics[angle=-90, width=0.49\\columnwidth]{plot_otoc_b50_nn_Im.eps}\n\\includegraphics[angle=-90, width=0.49\\columnwidth]{plot_coth_b50_nn_appendix.eps}\n\\includegraphics[angle=-90, width=0.49\\columnwidth]{plot_otoc_b50_nn_modulus.eps}\n\\caption{\nTop panels: Real and imaginary parts of the OTOC $\\langle \\hat n(t)\\hat n(0), \\hat n(t)\\hat n(0)\\rangle - 1$ for the single-orbital Hubbard model with $\\beta=50$ at half filling. The thin lines correspond to metallic solutions, \nwhile the thick line ($U=6$) corresponds to a Mott insulating solution. \nBottom left panel: Spectral function $\\mathscr A_{(nn)^2}(\\omega)=-\\frac{1}{\\pi}{\\rm Im}\\,C_{(n n)^2}^R(\\omega)$ multiplied by $\\coth\\big(\\frac{\\beta\\omega}{4}\\big)$.\nBottom right panel: Modulus of the OTOC $|\\langle \\hat n(t)\\hat n(0), \\hat n(t)\\hat n(0)\\rangle - 1|$ (normalized at $t=0$) on a log-log scale. \n}\n\\label{fig:results_nn}\n\\end{center}\n\\end{figure}\n\n\nIn Fig.~\\ref{fig:results_nn}, we plot the OTOC $\\langle \\hat n(t)\\hat n(0), \\hat n(t)\\hat n(0)\\rangle$\nfor the single-orbital Hubbard model in a similar manner as in Fig.~\\ref{fig:results_fermion}.\nSince at half filling this correlation function approaches $\\langle \\hat n\\rangle^4=1$ at long times\nand at sufficiently low temperature,\nwe perform the analytical continuation procedure for $\\langle \\hat n(t)\\hat n(0), \\hat n(t)\\hat n(0)\\rangle-1$. \nThe top panels of Fig.~\\ref{fig:results_nn} show the real and imaginary parts of this shifted OTOC. \nContrary to the case of $\\langle c_\\sigma^\\dagger(t)c_\\sigma(0), c_\\sigma^\\dagger(t)c_\\sigma(0)\\rangle$, \nthe amplitude of the OTOC $\\langle \\hat n(t)\\hat n(0), \\hat n(t)\\hat n(0)\\rangle-1$ is suppressed as one increases the interaction, which reflects the reduced charge fluctuations in the Mott insulator.\nThe bottom left panel in Fig.~\\ref{fig:results_nn} shows the analytically continued spectral function $\\mathscr A_{(nn)^2}(\\omega)=-\\frac{1}{\\pi}{\\rm Im}\\, C_{(nn)^2}^R(\\omega)$ multiplied by $\\coth\\big(\\frac{\\beta\\omega}{4}\\big)$. \nAgain we notice the opening of a gap in the Mott phase, and a corresponding shift of the spectral weight to higher energies. This results in more rapid oscillations of the OTOC in the Mott state.\nThe bottom right panel in Fig.~\\ref{fig:results_nn} plots the modulus of the OTOC $|\\langle \\hat n(t)\\hat n(0), \\hat n(t)\\hat n(0)\\rangle-1|$ (normalized at $t=0$) on a log-log scale.\nThe long-time behavior of the OTOC is consistent with a power-law decay in the metallic phase, and an exponential decay in the Mott insulating phase.\n\n\n\\begin{figure}[t]\n\\begin{center}\n\\includegraphics[angle=-90, width=0.49\\columnwidth]{plot_otoc_nsns_b50_draft_Re.eps}\n\\includegraphics[angle=-90, width=0.49\\columnwidth]{plot_otoc_nsns_b50_draft_Im.eps}\n\\includegraphics[angle=-90, width=0.49\\columnwidth]{plot_coth_b50_nsns_inset.eps}\n\\includegraphics[angle=-90, width=0.49\\columnwidth]{plot_otoc_nsns_b50_draft_modulus_zoom.eps}\n\\caption{\nTop panels: Real and imaginary parts of the OTOC $\\langle \\hat n_\\sigma(t)\\hat n_\\sigma(0), \\hat n_\\sigma(t)\\hat n_\\sigma(0)\\rangle  - \\langle \\hat n_\\sigma\\rangle^4$ for the single-orbital Hubbard model with $\\beta=50$ at half filling. All the colored lines correspond to metallic solutions. \nBottom left panel: Spectral function $\\mathscr A_{(n_\\sigma n_\\sigma)^2}(\\omega)=-\\frac{1}{\\pi}{\\rm Im}\\,C_{(n_\\sigma n_\\sigma)^2}^R(\\omega)$ multiplied by $\\coth\\big(\\frac{\\beta\\omega}{4}\\big)$. In the inset, we plot the imaginary-time four-point function $\\langle \\hat n_\\sigma(\\tau)\\hat n_\\sigma(0), \\hat n_\\sigma(\\tau)\\hat n_\\sigma(0)\\rangle-\\langle \\hat n_\\sigma\\rangle^4$, with the black line corresponding to the Mott insulating solution at $U=6$.  \nBottom right panel: Modulus of the OTOC $|\\langle \\hat n_\\sigma(t)\\hat n_\\sigma(0), \\hat n_\\sigma(t)\\hat n_\\sigma(0)\\rangle  - \\langle \\hat n_\\sigma\\rangle^4|$ (normalized at $t=0$) on a log-log scale.\n}\n\\label{fig:results_nsns}\n\\end{center}\n\\end{figure}\n\nFigure~\\ref{fig:results_nsns} \nshows the results of the OTOC $\\langle \\hat n_\\sigma(t)\\hat n_\\sigma(0), \\hat n_\\sigma(t)\\hat n_\\sigma(0)\\rangle$\nfor the single-orbital Hubbard model. At half filling, this correlation function approaches $\\langle \\hat n_\\sigma\\rangle^4=\\frac{1}{16}$ in the long-time limit and at sufficiently low temperature, \nso that we perform the analytical continuation procedure for $\\langle \\hat n_\\sigma(t)\\hat n_\\sigma(0), \\hat n_\\sigma(t)\\hat n_\\sigma(0)\\rangle-\\langle \\hat n_\\sigma\\rangle^4$. \nThe top panels of Fig.~\\ref{fig:results_nsns} show the real and imaginary parts of \nthis shifted OTOC. \nHere, we only present results for the metallic phase, because resolving the very sharp low-energy feature of the OTOC spectrum in the half-filled Mott state is challenging. One can see that coherent oscillations are not observed for this type of OTOC, and that the incoherent part becomes dominant compared to the other two OTOCs shown in Figs.~\\ref{fig:results_fermion} and \\ref{fig:results_nn}. The amplitude of the OTOC $\\langle \\hat n_\\sigma(t)\\hat n_\\sigma(0), \\hat n_\\sigma(t)\\hat n_\\sigma(0)\\rangle-\\langle \\hat n_\\sigma\\rangle^4$ is enhanced as $U$ increases. As we argue below, this is related to the enhancement of spin fluctuations \nand local moment formation close to the Mott transition.\n\nThe bottom left panel in Fig.~\\ref{fig:results_nsns} plots the analytically continued spectral function $\\mathscr A_{(n_\\sigma n_\\sigma)^2}(\\omega)=-\\frac{1}{\\pi}{\\rm Im}\\, C_{(n_\\sigma n_\\sigma)^2}^R(\\omega)$ multiplied by $\\coth\\big(\\frac{\\beta\\omega}{4}\\big)$. \nWe can see that the spectral weight is concentrated in the low-energy region as we approach the Mott transition point.\nThe inset in the bottom left panel of Fig.~\\ref{fig:results_nsns} shows the imaginary-time four-point function $\\langle \\hat n_\\sigma(\\tau)\\hat n_\\sigma(0), \\hat n_\\sigma(\\tau)\\hat n_\\sigma(0)\\rangle-\\langle \\hat n_\\sigma\\rangle^4$, with the black line corresponding to a Mott insulating solution ($U=6$).  \nFor $U\\gtrsim 4$, the imaginary-time four-point function does not decay to zero but remains relatively large near $\\tau=\\frac{\\beta}{4}$. \nThis behavior is reminiscent of the spin-freezing physics seen in the correlated Hund metal phase of multi-orbital Hubbard models, where the dynamical spin correlation function $\\langle \\hat S_z(\\tau)\\hat S_z(0)\\rangle$ is trapped at a finite value at long $\\tau$ \\cite{Werner2008}. \nIn fact, the OTOC $\\langle \\hat n_\\sigma(\\tau)\\hat n_\\sigma(0), \\hat n_\\sigma(\\tau)\\hat n_\\sigma(0)\\rangle$\nmeasures spin correlations (in addition to charge correlations) through $\\hat n_\\sigma=\\frac{1}{2}\\hat n+\\sigma\\hat S_z$.\nPhysically, the trapping of $\\langle \\hat S_z(\\tau)\\hat S_z(0)\\rangle$ signals the formation of frozen local magnetic moments.\nIn the metal-insulator crossover region, the scattering induced by these magnetic moments results in incoherent metal states. Intuitively, one might expect fast scrambling in the corresponding regions of the phase diagram. \nAlthough there is some resemblance with the spin freezing crossover,\nthe self-energy does not exhibit a square-root frequency dependence in the single-orbital case, and a spin-freezing crossover in the sense of Ref.~\\cite{Werner2008} does not exist.  \n\nThe bottom right panel of Fig.~\\ref{fig:results_nsns} plots the modulus of the OTOC $|\\langle \\hat n_\\sigma(t)\\hat n_\\sigma(0), \\hat n_\\sigma(t)\\hat n_\\sigma(0)\\rangle-\\langle \\hat n_\\sigma\\rangle^4|$ (normalized at $t=0$) on a log-log scale. \nThe dynamics of $\\langle \\hat n_\\sigma(t)\\hat n_\\sigma(0), \\hat n_\\sigma(t)\\hat n_\\sigma(0)\\rangle$ is quite distinct from that of $\\langle \\hat n(t)\\hat n(0), \\hat n(t)\\hat n(0)\\rangle$. As the Mott transition is approached, we observe a slow-down in the decay of the modulus, \nwhich indicates the presence of slowly fluctuating local moments. \nThe long-time behavior may be consistent with a power-law, although slow oscillations make it difficult to determine the long-time asymptotic form from the numerics. \n\n\n\\begin{figure}[t]\n\\begin{center}\n\\includegraphics[angle=-90, width=0.49\\columnwidth]{plot_fitnsns_b2.eps}\n\\includegraphics[angle=-90, width=0.49\\columnwidth]{plot_coeffc_draft.eps}\n\\caption{\nLeft panel: \nThe imaginary-time four-point function $\\langle \\hat n_\\sigma(\\tau)\\hat n_\\sigma(0), \\hat n_\\sigma(\\tau)\\hat n_\\sigma(0)\\rangle$, the squared density-density correlation function $\\langle \\hat n_\\sigma(\\tau)\\hat n_\\sigma(0)\\rangle^2$,\nand their difference for the single-orbital Hubbard model with $U=5.5$ and $\\beta=50$.\nThe dashed curve shows a fit of the difference with the function $f_\\text{fit}(\\tau)=a+b\\tau+c\\tau^2$ in the interval $\\tau/\\beta \\in [0: 0.2]$.\nRight panel: Fitting coefficient $c$ as a function of $U$ for indicated values of $\\beta$.\n\n}\n\\label{fig:coeffs}\n\\end{center}\n\\end{figure}\n\nFor large enough $\\beta$ and small enough $\\tau$, the OTOC (\\ref{otoc_nsns}) factorizes into $\\langle \\hat n_\\sigma(\\tau)\\hat n_\\sigma(0), \\hat n_\\sigma(\\tau)\\hat n_\\sigma(0)\\rangle \\approx \\langle \\hat n_\\sigma(\\tau)\\hat n_\\sigma(0)\\rangle^2$. \nOn the real-time axis, this corresponds to the decoupling $\\langle \\hat n_\\sigma(t)\\hat n_\\sigma(0), \\hat n_\\sigma(t)\\hat n_\\sigma(0)\\rangle \\approx \\langle \\hat n_\\sigma(t)\\hat n_\\sigma(0)\\rangle^2$, that is, \nthe OTOC is reduced to a product of ordinary density-density correlation functions.\nIn order to see whether nontrivial correlations are captured beyond the decoupled form by the OTOC function, \nwe consider the difference $\\langle \\hat n_\\sigma(\\tau)\\hat n_\\sigma(0), \\hat n_\\sigma(\\tau)\\hat n_\\sigma(0)\\rangle-\\langle \\hat n_\\sigma(\\tau)\\hat n_\\sigma(0)\\rangle^2$ and fit the short-time behavior of this function to $f_\\text{fit}(\\tau)=a+b\\tau+c\\tau^2$, as illustrated in the left panel of Fig.~\\ref{fig:coeffs}. The coefficients $b$ and $c$ serve as  indicators of the nontrivial correlations that cannot be attributed to the decoupled form of the OTOC.\nThe coefficient $c$ is plotted as a function of $U$ and for different $\\beta$ in the right hand panel. The coefficient $b$ (not shown) exhibits a similar trend. We notice that the OTOC picks up nontrival correlations near the metal-insulator transition, and in particular at \nintermediate\ntemperatures, while the Mott phase shows no such correlations. The crossover region  \nassociated with the emergence of local moments \ncorresponds, roughly, to the interaction range where the nontrivial correlations start to become significant. \n\n\n\\subsection{Two-orbital Hubbard model}\n\\label{sec:two-orbital hubbard}\n\nIn this section, we investigate OTOCs in the two-orbital Hubbard model with Hund coupling $J>0$ \nand density-density interactions. (Three-orbital results are presented in Appendix~\\ref{sec:3orbital}.) \nThe Hamiltonian is given by\n\\begin{align}\nH\n&=\n-v\\sum_{\\langle i,j\\rangle\\alpha\\sigma} (c^\\dagger_{i\\alpha\\sigma}c_{j\\alpha\\sigma}+ \\text{h.c.})\n-\\mu\\sum_{i\\alpha\\sigma} \\hat n_{i\\alpha\\sigma}\n+U\\sum_{i\\alpha} \\hat n_{i\\alpha\\uparrow}\\hat n_{i\\alpha\\downarrow}\n+(U-2J)\\sum_{i\\sigma} \\hat n_{i1\\sigma}\\hat n_{i2\\bar\\sigma}\n\\notag\n\\\\\n&\\quad\n+(U-3J)\\sum_{i\\sigma} \\hat n_{i1\\sigma}\\hat n_{i2\\sigma},\n\\end{align}\n\nwith $U$ being the intra-orbital interaction $U$, $U-2J$ the inter-orbital antiparallel-spin interaction, and $U-3J$ the inter-orbital parallel-spin interaction. This is the simplest model that shows a crossover from a spin-frozen metal to a Fermi-liquid metal as one dopes the half-filled Mott insulator, with the self-energy scaling as $\\text{Im}\\Sigma(\\omega_n)\\sim\\sqrt{\\omega_n}$ over a significant energy range in the crossover regime \\cite{Hafermann2012}.\nThe sketch of the phase diagram is shown in Fig.~\\ref{fig:illustration}. \nAs we have seen in the previous section, the spin-related OTOC of the type (\\ref{otoc_nsns}) is relevant for the analysis of  the spin-freezing crossover, so that we concentrate on this OTOC here.\nIn the following calculations, we choose $U=8$, $J=U/4$, and compute $\\langle \\hat n_\\sigma(t)\\hat n_\\sigma(0), \\hat n_\\sigma(t)\\hat n_\\sigma(0)\\rangle - \\langle n_\\sigma \\rangle^4$ \nas a function of filling at $\\beta=50$ (dashed lines in Fig.~\\ref{fig:illustration}). \n\n\n\\begin{figure}[t]\n\\begin{center}\n\\includegraphics[angle=-90, width=0.49\\columnwidth]{plot_qp_draft.eps}\\hfill\n\\includegraphics[angle=-90, width=0.49\\columnwidth]{plot_fit_2orbital.eps}\n\\caption{\nLeft panel: Imaginary part of the self-energy as a function of Matsubara frequency for the two-orbital Hubbard model with $U=8$, $J=U/4$, and $\\beta=50$ on a log-log scale, with the dashed line indicating $\\sqrt{\\omega_n}$ behavior. \nRight panel: The coefficient $c$ extracted from\nfitting $\\langle \\hat n_\\sigma(\\tau)\\hat n_\\sigma(0), \\hat n_\\sigma(\\tau)\\hat n_\\sigma(0)\\rangle-\\langle \\hat n_\\sigma(\\tau)\\hat n_\\sigma(0)\\rangle^2$ with a function $f_\\text{fit}(\\tau)=a+b\\tau+c\\tau^2$\nin the interval $\\tau/\\beta\\in [0: 0.05]$ for the two-orbital Hubbard model. \nThe spin-freezing crossover occurs roughly at the filling $n_\\sigma\\approx 0.28$ (blue curve in the left panel and blue arrow in the right panel). \n}\n\\label{fig:self}\n\\end{center}\n\\end{figure}\n\n\nThe left panel of Fig.~\\ref{fig:self} plots the imaginary part of the self-energy as a function of Matsubara frequency in order to identify the filling corresponding to the spin-freezing crossover \\cite{Werner2008}. An approximate square-root scaling is observed over a wide range of frequencies near $n_\\sigma\\approx 0.28$ (blue line). In the right panel of Fig.~\\ref{fig:self}, we present the coefficient $c$ obtained from a similar analysis as presented in Fig.~\\ref{fig:coeffs}, but with a fitting range $\\tau/\\beta\\in [0 : 0.05]$. Again, we find that the OTOC $\\langle \\hat n_\\sigma(t)\\hat n_\\sigma(0), \\hat n_\\sigma(t)\\hat n_\\sigma(0)\\rangle - \\langle \\hat n_\\sigma\\rangle^4$ detects nontrivial correlations in the spin-freezing crossover (blue arrow) and spin-frozen metal regime, but not in the Fermi liquid or the Mott insulating phase.\n\n\n\\begin{figure}[t]\n\\begin{center}\n\\includegraphics[angle=-90, width=0.49\\columnwidth]{plot_otoc_2orbital_Re.eps}\n\\includegraphics[angle=-90, width=0.486\\columnwidth]{plot_otoc_2orbital_Im.eps}\n\\includegraphics[angle=-90, width=0.484\\columnwidth]{plot_spectra_coth_2orbital_inset.eps}\n\\includegraphics[angle=-90, width=0.486\\columnwidth]{plot_otoc_2orbital_modulus_inset.eps}\n\\caption{\nTop panels: Real and imaginary parts of the OTOC $\\langle \\hat n_\\sigma(t)\\hat n_\\sigma(0), \\hat n_\\sigma(t)\\hat n_\\sigma(0)\\rangle  - \\langle \\hat n_\\sigma\\rangle^4$ for the two-orbital Hubbard model with $U=8$, $J=U/4$, $\\beta=50$ and indicated fillings. \nBottom left panel: Spectral function $\\mathscr A_{(n_\\sigma n_\\sigma)^2}(\\omega)=-\\frac{1}{\\pi}{\\rm Im}\\,C_{(n_\\sigma n_\\sigma)^2}^R(\\omega)$ multiplied by $\\coth\\big(\\frac{\\beta\\omega}{4}\\big)$. In the inset, we plot the imaginary-time four-point function $\\langle \\hat n_\\sigma(\\tau)\\hat n_\\sigma(0), \\hat n_\\sigma(\\tau)\\hat n_\\sigma(0)\\rangle-\\langle \\hat n_\\sigma\\rangle^4$. \nBottom right panel: Modulus of the OTOC $|\\langle \\hat n_\\sigma(t)\\hat n_\\sigma(0), \\hat n_\\sigma(t)\\hat n_\\sigma(0)\\rangle  - \\langle \\hat n_\\sigma\\rangle^4|$ (normalized at $t=0$) on a log-log scale.\nIn the inset of the bottom right panel, we illustrate the approximate data collapse obtained by considering the spin-freezing crossover regimes for different values of $J$. \nIn all the panels, the thick blue curves represent the results for the spin-freezing crossover region. \n}\n\\label{fig:2orbital}\n\\end{center}\n\\end{figure}\n\n\nThe top panels of Fig.~\\ref{fig:2orbital} show the real and imaginary parts of the OTOC\n$\\langle \\hat n_\\sigma(t)\\hat n_\\sigma(0), \\hat n_\\sigma(t)\\hat n_\\sigma(0)\\rangle-\\langle \\hat n_\\sigma\\rangle^4$\nfor the two-orbital Hubbard model. In the spin-frozen phase ($n_\\sigma \\gtrsim 0.3$), both components exhibit \nhighly incoherent oscillations, which get suppressed as the filling is reduced. As one moves across the crossover point ($n_\\sigma \\sim 0.28$),\nthe oscillations completely disappear, and the real part of the OTOC starts to overshoot to the negative side\nin the Fermi liquid regime.\nThe bottom left panel in Fig.~\\ref{fig:2orbital} presents the analytically continued spectral function $\\mathscr A_{(n_\\sigma n_\\sigma)^2}(\\omega)=-\\frac{1}{\\pi}{\\rm Im}\\,C_{(n_\\sigma n_\\sigma)^2}^R(\\omega)$ multiplied by $\\coth\\big(\\frac{\\beta\\omega}{4}\\big)$. \nIn the spin-frozen regime ($n_\\sigma \\gtrsim 0.3$) a sharp peak appears at low energy, which originates from the saturation of the imaginary-time four point function $\\langle \\hat n_\\sigma(\\tau)\\hat n_\\sigma(0), \\hat n_\\sigma(\\tau)\\hat n_\\sigma(0)\\rangle$ at large $\\tau$ (see the inset of the bottom left panel of Fig.~\\ref{fig:2orbital}). This low-energy peak represents the slow dynamics of the frozen spins, and translates into a slow long-time decay of $|\\langle \\hat n_\\sigma(t)\\hat n_\\sigma(0), \\hat n_\\sigma(t)\\hat n_\\sigma(0)\\rangle-\\langle \\hat n_\\sigma\\rangle^4|$,\nas shown in the bottom right panel of Fig.~\\ref{fig:2orbital}\non a log-log scale. Again, it is difficult to clearly resolve the long-time behavior of the OTOC due to the limited\naccuracy of the analytic continuation, but in the spin-freezing crossover regime (thick blue curves)\nthe decay of the OTOC is consistent with a power law $\\sim 1/t^{1.5}$ at least for $2\\lesssim t \\lesssim 5$. \nThe shorter-time behavior ($0.5\\lesssim t \\lesssim 2$) is instead well fitted by an exponential decay $\\sim e^{-\\alpha t}$ with a decay constant $\\alpha=0.78$. \nThis exponent exhibits\na strong dependence on the Hund coupling. \n\nIn fact, the OTOCs in the spin-freezing regime for $J=U/4$ and $J=U/6$ can be approximately collapsed by plotting them as a function of $tJ$ (see the inset in the bottom right panel of Fig.~\\ref{fig:2orbital}), which indicates that the Hund coupling is the parameter which controls the dynamics in this regime. \nThis is distinct from the Lyapunov behavior in which the exponential decay scales with $t/\\beta$ \\cite{Bagrets2017}.\nA possible reason is that we are still far from the large $N$ limit (where $N$ corresponds to the number of orbitals times the number of spin degrees of freedom) so that the large $N$ behavior \nsuch as the Lyapunov growth is not observed in our calculations. \nIn fact, the strong dependence of the time scale on $J$ and weak dependence on $\\beta$ is consistent\nwith the behavior of the finite $N$ SYK model in Ref.~\\cite{FuSachdev2016}.\nIn the spin-frozen regime ($n_\\sigma\\gtrsim 0.3$) the decay is much slower than near the crossover point, while in the Fermi liquid regime the decay is accelerated, approaching the free fermion behavior of $t^{-3}$\nas shown in Fig.~\\ref{analytic continuation}. \nQualitatively similar results for this OTOC are obtained \nfor the three-orbital Hubbard model (see Appendix~\\ref{sec:3orbital}).\n\nThe time dependence of the modulus of \n$\\langle \\hat n_\\sigma(\\tau)\\hat n_\\sigma(0), \\hat n_\\sigma(\\tau)\\hat n_\\sigma(0)\\rangle-\\langle \\hat n_\\sigma\\rangle^4$ \nin the spin-freezing crossover regime of multi-orbital Hubbard models (exponential decay crossing over into a power-law) bears a close resemblance to a (different) OTOC for the SYK model discussed in Ref.~\\cite{Bagrets2017}. \nAlso, a recent study of yet another type of OTOCs for the finite-$N$ SYK model found a qualitatively similar decay as observed here in the spin-freezing crossover regime \\cite{FuSachdev2016}. In the following section, we will make the connection to the SYK dynamics more quantitative.   \n\n\n\\subsection{Comparison to the SKY model}\n\\label{sec:SYK}\n\nIn this section, we compare the results of the multi-orbital Hubbard models to the SYK model.\nWe consider the particle-hole symmetric SYK model of complex fermions \\cite{FuSachdev2016},\nwhose Hamiltonian is given by\n\\begin{align}\nH\n&=\n\\frac{1}{(2N)^{3/2}} \\sum_{i,j,k,l=1}^N \nJ_{ij;kl} (c_i^\\dagger c_j^\\dagger c_k c_l\n+\\delta_{ik}n c_j^\\dagger c_l\n-\\delta_{il}n c_j^\\dagger c_k\n-\\delta_{jk}n c_i^\\dagger c_l\n+\\delta_{jl}n c_i^\\dagger c_k).\n\\end{align}\nThe coupling constant $J_{ij;kl}$ is a gaussian random variable, satisfying $J_{ij;kl}=-J_{ji;kl}=-J_{ij;lk}=J_{kl;ij}^\\ast$, and\n\\begin{align}\n\\overline{({\\rm Re}\\, J_{ij;kl})^2}\n&=\n\\begin{cases}\nJ_{\\rm SYK}^2/2 & (i,j)\\neq (k,l) \\\\\nJ_{\\rm SYK}^2 & (i,j)=(k,l)\n\\end{cases},\n\\\\\n\\overline{({\\rm Im}\\, J_{ij;kl})^2}\n&=\n\\begin{cases}\nJ_{\\rm SYK}^2/2 & (i,j)\\neq (k,l) \\\\\n0 & (i,j)=(k,l)\n\\end{cases},\n\\end{align}\nwhere the overline represents an average over each realization of $J_{ij;kl}$. The parameter \n$J_{\\rm SYK}$ ($J_{\\rm SYK}^{-1}$) defines the unit of energy (time) in the following calculations. \nThe system is particle-hole symmetric when\n$n=0.5$, which fixes the total number of fermions as $N^{-1}\\sum_{i=1}^N \\langle c_i^\\dagger c_i\\rangle=0.5$.\nWe take the particle-hole symmetric form of the SYK model because it has been well studied previously \\cite{FuSachdev2016} and avoids the effect of the filling drift.\nWe numerically solve the model by exact diagonalization for finite $N(\\le 12)$, and evaluate the density-density OTOC\n$\\overline{\\langle \\hat n_i(t)\\hat n_i(0), \\hat n_i(t)\\hat n_i(0)\\rangle-\\langle \\hat n_i\\rangle^4}$\nand its imaginary-time counterpart.\n\n\\begin{figure}[t]\n\\begin{center}\n\n\\includegraphics[angle=-90,width=0.49\\columnwidth]{plot_fitnsns_syk.eps}\n\n\\includegraphics[angle=-90,width=0.49\\columnwidth]{plot_coeffc_syk.eps}\n\\caption{\nLeft panel: \nThe imaginary-time four-point function $\\overline{\\langle \\hat n_i(\\tau)\\hat n_i(0), \\hat n_i(\\tau)\\hat n_i(0)\\rangle}$, the density-density correlation function $\\overline{\\langle \\hat n_i(\\tau)\\hat n_i(0)\\rangle^2}$,\nand their difference for the SYK model with $N=12$ and $\\beta J_{\\rm SYK}=10$.\nThe dashed curve shows a fit of the difference with a function $f_\\text{fit}(\\tau)=a+b\\tau+c\\tau^2$ in the interval $\\tau/\\beta \\in [0: 0.2]$. \nRight panel: Fitting coefficient $c$ as a function of $T/J_{\\rm SYK}$.\n}\n\\label{fig:SYK fit}\n\\end{center}\n\\end{figure}\n\nIn the left panel of Fig.~\\ref{fig:SYK fit}, we plot the imaginary-time four-point function \n$\\overline{\\langle \\hat n_i(\\tau)\\hat n_i(0), \\hat n_i(\\tau)\\hat n_i(0)\\rangle}$\nfor the SYK model with $N=12$ and $\\beta J_{\\rm SYK}=10$. \nAt low temperature, we expect that $\\overline{\\langle \\hat n_i(\\tau)\\hat n_i(0), \\hat n_i(\\tau)\\hat n_i(0)\\rangle}$ is decoupled into $\\overline{\\langle \\hat n_i(\\tau)\\hat n_i(0)\\rangle^2}$.\nTo see whether nontrivial correlations beyond the decoupled form are present, we fit the difference\n$\\overline{\\langle \\hat n_i(\\tau)\\hat n_i(0), \\hat n_i(\\tau)\\hat n_i(0)\\rangle}\n-\\overline{\\langle \\hat n_i(\\tau)\\hat n_i(0)\\rangle^2}$ with a function $f_{\\rm fit}(\\tau)=a+b\\tau+c\\tau^2$\nin the interval $\\tau/\\beta\\in[0:0.2]$, in the same manner as for the Hubbard models.\nThe obtained coefficient $c$ is plotted as a function of the temperature $T$ in the right panel of\nFig.~\\ref{fig:SYK fit}. One can see that nontrivial correlations exist in a wide range of temperature.\nEspecially, they are enhanced in the intermediate temperature regime. This is because in the zero-temperature limit\nthe four-point function is decoupled as \n$\\langle \\hat n_i(\\tau)\\hat n_i(0), \\hat n_i(\\tau)\\hat n_i(0)\\rangle \\approx \\langle \\hat n_i(\\tau)\\hat n_i(0)\\rangle^2$, while in the high-temperature limit ($\\beta\\to 0$) the imaginary-time dependence is washed out.\n\n\n\n\n\\begin{figure}[t]\n\\begin{center}\n\\includegraphics[angle=-90, width=0.49\\columnwidth]{plot_syk_Re.eps}\n\\includegraphics[angle=-90, width=0.49\\columnwidth]{plot_syk_Im.eps}\n\\includegraphics[angle=-90, width=0.49\\columnwidth]{plot_syk_modulus-log.eps}\n\\includegraphics[angle=-90, width=0.49\\columnwidth]{plot_syk_modulus.eps}\n\\caption{\nThe density-density OTOC $\\overline{\\langle \\hat n_i(t)\\hat n_i(0), \\hat n_i(t)\\hat n_i(0)\\rangle-\\langle \\hat n_i\\rangle^4}$ for the particle-hole symmetric SYK model averaged over $10^2$ samples with $\\beta J_{\\rm SYK}=100$ and various $N$. \nThe top left and right panels show the real and imaginary parts of the OTOC,\nwhile the bottom left and right panels show the log and log-log plots of the modulus of the OTOC (normalized at $t=0$).\nThe dashed line shows a power-law decay $\\propto t^{-1.6}$ for comparison.}\n\\label{fig:SYK}\n\\end{center}\n\\end{figure}\n\nIn Fig.~\\ref{fig:SYK}, we show the numerical results of the density-density OTOC for $\\beta J_{\\rm SYK}=100$.\nThe real part of the OTOC rapidly drops from the initial value within the time scale of $tJ_{\\rm SYK}\\sim 2$,\nand slowly decays to zero in the long time limit.\nAs one increases $N$, the oscillations that appear at longer time tend to be suppressed.\nOne can see that the result for $N=12$ in the top panels of Fig.~\\ref{fig:SYK} closely resembles the OTOC in \nthe spin-freezing crossover regime of the two-orbital Hubbard model shown\nby the blue curve in the top panels of Fig.~\\ref{fig:2orbital},\nwhile it differs from the single-orbital result shown in the top panels of Fig.~\\ref{fig:results_nsns}.\nDue to the limitation of our calculations to small system sizes ($N\\le 12$), we do not clearly observe\nan exponential growth ($\\sim c_0-c_1 e^{\\lambda t}$)  at short times or an exponential decay ($\\sim e^{-\\alpha t}$) \nat intermediate times. \nHowever, we find that for $tJ_{\\rm SYK}\\gtrsim 2$ the OTOC decays\napproximately in a power law $\\propto t^{-\\gamma}$ with $\\gamma=1.6$ (see bottom right panel in Fig.~\\ref{fig:SYK}). \nThe time interval exhibiting the power-law decay becomes longer as we increase $N$.  \nThe temperature dependence of the OTOC for the SYK model is shown in Fig.~\\ref{fig:SYK-T}.\nThe time scale of the initial drop is more or less independent of the temperature, while \nthe power-law-like decay is only visible in the low-temperature regime.\n\n\\begin{figure}[t]\n\\begin{center}\n\\includegraphics[angle=-90, width=0.49\\columnwidth]{plot_syk_beta_re.eps}\n\\includegraphics[angle=-90, width=0.49\\columnwidth]{plot_syk_beta_im.eps}\n\\includegraphics[angle=-90, width=0.49\\columnwidth]{plot_syk_beta_modulus-log.eps}\n\\includegraphics[angle=-90, width=0.49\\columnwidth]{plot_syk_beta_modulus.eps}\n\\caption{\nTemperature dependence of the density-density OTOC\n$\\overline{\\langle \\hat n_i(t)\\hat n_i(0), \\hat n_i(t)\\hat n_i(0)\\rangle-\\langle \\hat n_i\\rangle^4}$ for the particle-hole symmetric SYK model averaged over $10^2$ samples with $N=12$.\nThe top left and right panels show the real and imaginary parts of the OTOC,\nwhile the bottom left and right panels show the log and log-log plots of the modulus of the OTOC.\nThe dashed line shows a power-law decay $\\propto t^{-1.6}$ for comparison.}\n\\label{fig:SYK-T}\n\\end{center}\n\\end{figure}\n\nIn Ref.~\\onlinecite{Werner2018}, it has been argued that the spin-freezing crossover regime of multi-orbital Hubbard models is effectively described by the SYK model in which $J_{\\rm SYK}$ is replaced by the Hund coupling $J$. The results in Fig.~\\ref{fig:2orbital} are for the two-orbital Hubbard model with $U=8$, $J/U=1/4$ and $\\beta=50$, corresponding to $\\beta J=100$, which means that it is meaningful to directly compare the blue lines in Fig.~\\ref{fig:2orbital} with the blue lines in Figs.~\\ref{fig:SYK} and \\ref{fig:SYK-T}. Not only is the qualitative agreement remarkable, even the power-law exponent measured for the two-orbital Hubbard model ($\\gamma=1.5$) is very close to the exponent extracted from the finite-$N$ SYK calculation. \nThe power-law decay in the three-orbital case is approximately $1/t^{1.75}$ (see Appendix~\\ref{sec:3orbital}), and thus also in semi-quantitative agreement with the SYK model behavior. \nThis nontrivial result provides further support for the identification of the spin-freezing crossover regime of multi-orbital Hubbard systems with an SYK strange metal. \n\nIn the large-$N$ and low-$T$ limit of the SYK model, we expect that the power-law behavior approaches $1/t^2$, since in this limit the OTOC is decoupled as $\\langle \\hat n_i(t)\\hat n_i(0), \\hat n_i(t)\\hat n_i(0)\\rangle\\approx \\langle \\hat n_i(t)\\hat n_i(0)\\rangle^2$ and the decay of $\\langle \\hat n_i(t)\\hat n_i(0)\\rangle$ is dominated by what corresponds to the slow spin relaxation $\\langle \\hat S_z(t)\\hat S_z(0)\\rangle\\sim 1/t$ in the Sachdev-Ye model.\n\n\n\n\n\n4.2 Single-orbital Hubbard model\n\\subsection{Single-orbital Hubbard model}\n\\label{sec:hubbard}\n\nIn this section, we calculate the OTOCs (\\ref{fermion_otoc})-(\\ref{otoc_nn}) for the interacting single-orbital Hubbard model with the Hamiltonian \n\\begin{align}\nH\n&=\n-v\\sum_{\\langle i,j\\rangle\\sigma} (c^\\dagger_{i\\sigma}c_{j\\sigma}+ \\text{h.c.})\n-\\mu\\sum_i \\hat n_i+U\\sum_i \\hat n_{i\\uparrow}\\hat n_{i\\downarrow}, \n\\end{align}\nwhere $v$ is the hopping amplitude, $\\langle i,j\\rangle$ represents the nearest-neighbor site pairs, \n$\\mu$ is the chemical potential, and $U$ is the on-site interaction.\nIn this section, we focus on the half-filled case (i.e., $\\mu=U/2$). The DMFT solution for the simplified self-consistency (\\ref{semicirc}) is the exact solution for an infinitely connected Bethe lattice \\cite{GeorgesKotliarKrauthRozenberg1996}.\nLet us briefly recall the paramagnetic phase diagram for the single-orbital Hubbard model on this lattice \\cite{Bluemer_thesis}. At half-filling, there is a metal-insulator crossover at high temperature and a first-order Mott transition below a temperature corresponding to $\\beta\\hbar\\approx 17$ with a coexistence region between $U_{c1}$ and $U_{c2}$. The finite-temperature critical endpoint is at $U\\approx 4.7$, while the zero-temperature Mott transition occurs at $U=U_{c2}\\approx 5.6$. \nAt low temperature, the self-energy shows the Fermi liquid behavior [${\\rm Im}\\,\\Sigma(\\omega)\\sim \\omega^2$] \nin the weakly correlated metallic phase, \nwhile it shows an insulating behavior [${\\rm Im}\\,\\Sigma(\\omega)\\sim \\delta(\\omega)$] in the Mott phase\n\\cite{GeorgesKotliarKrauthRozenberg1996}. \nIn the correlated metallic phase, as the temperature is increased, deviations from the Fermi liquid behavior become apparent, \nbut an ${\\rm Im}\\,\\Sigma(\\omega)\\sim \\sqrt{\\omega}$ scaling as in the SYK model is not observed. \n\nIn the following, we will start the DMFT iterations from the noninteracting solution, which means that the finite-temperature Mott transition occurs at $U_{c2}$.\nFrom now on, we set $\\hbar=1$. \n\n\\begin{figure}[t]\n\\begin{center}\n\\includegraphics[angle=-90, width=0.49\\columnwidth]{plot_otoc_b50_Re.eps}\n\\includegraphics[angle=-90, width=0.49\\columnwidth]{plot_otoc_b50_Im.eps}\n\\includegraphics[angle=-90, width=0.49\\columnwidth]{plot_coth_b50_appendix.eps}\n\\includegraphics[angle=-90, width=0.49\\columnwidth]{plot_otoc_b50_modulus.eps}\n\\caption{\nTop panels: Real and imaginary parts of the OTOC $\\langle c^\\dagger_\\sigma(t)c_\\sigma(0), c^\\dagger_\\sigma(t)c_\\sigma(0)\\rangle$ for the single-orbital Hubbard model with $\\beta=50$ at half filling. The thin lines correspond to metallic solutions, while the thick line ($U=6$) corresponds to a Mott insulating solution. \nBottom left panel: Spectral function $\\mathscr A_{(c_\\sigma^\\dagger c_\\sigma)^2}(\\omega)=-\\frac{1}{\\pi}{\\rm Im}\\,C_{(c^\\dagger_\\sigma c_\\sigma)^2}^R(\\omega)$ multiplied by $\\coth\\big(\\frac{\\beta\\omega}{4}\\big)$. \nBottom right panel: Modulus of the OTOC $|\\langle c^\\dagger_\\sigma(t)c_\\sigma(0), c^\\dagger_\\sigma(t)c_\\sigma(0)\\rangle|$ (normalized at $t=0$) on a log-log scale. The dashed line is the result for $U=0$.\n}\n\\label{fig:results_fermion}\n\\end{center}\n\\end{figure}\n\n\n\nIn Fig.~\\ref{fig:results_fermion}, we show the results of $\\langle c^\\dagger_\\sigma(t)c_\\sigma(0), c^\\dagger_\\sigma(t)c_\\sigma(0)\\rangle$ for the interacting single-orbital Hubbard model at $\\beta=50$. \nThe top panels present the real and imaginary parts of the OTOC. One can see that the oscillations become more pronounced as one increases the interaction. In the bottom left panel of Fig.~\\ref{fig:results_fermion},\nwe show the corresponding spectral function $\\mathscr A_{(c_\\sigma^\\dagger c_\\sigma)^2}(\\omega)=-\\frac{1}{\\pi}{\\rm Im}\\,C_{(c^\\dagger_\\sigma c_\\sigma)^2}^R(\\omega)$ multiplied by $\\coth\\big(\\frac{\\beta\\omega}{4}\\big)$.\nThe Mott transition, which occurs between $U=5$ and $U=6$ at $\\beta=50$, manifests itself by the opening of a gap in the spectral function $\\mathscr A_{(c_\\sigma^\\dagger c_\\sigma)^2(\\omega)}$. This translates into more weakly damped oscillations in the real-time evolution in the Mott phase.  \nThe bottom right panel of Fig.~\\ref{fig:results_fermion} plots the modulus of the OTOC (normalized at time $t=0$, i.e., $|\\langle c^\\dagger_\\sigma(t)c_\\sigma(0), c^\\dagger_\\sigma(t)c_\\sigma(0)\\rangle|/|\\langle c^\\dagger_\\sigma(0)c_\\sigma(0), c^\\dagger_\\sigma(0)c_\\sigma(0)\\rangle|$) on a log-log scale. \nDue to the oscillations and the limited accuracy of the analytic continuation procedure, it is hard to clearly identify the nature of the long-time decay of the OTOC,\nbut the results indicate that the OTOC decays much faster (possibly exponentially) in the Mott phase than in the metallic phase. This is consistent with the results for the spectral functions of the OTOC. The comparison to the noninteracting case (dashed curve in the bottom right panel of Fig.~\\ref{fig:results_fermion}) suggests that the correlated metallic phase has a similar (possibly power-law) decay behavior of the OTOC as in the noninteracting case.\n\n\n\\begin{figure}[t]\n\\begin{center}\n\\includegraphics[angle=-90, width=0.49\\columnwidth]{plot_otoc_b50_nn_Re.eps}\n\\includegraphics[angle=-90, width=0.49\\columnwidth]{plot_otoc_b50_nn_Im.eps}\n\\includegraphics[angle=-90, width=0.49\\columnwidth]{plot_coth_b50_nn_appendix.eps}\n\\includegraphics[angle=-90, width=0.49\\columnwidth]{plot_otoc_b50_nn_modulus.eps}\n\\caption{\nTop panels: Real and imaginary parts of the OTOC $\\langle \\hat n(t)\\hat n(0), \\hat n(t)\\hat n(0)\\rangle - 1$ for the single-orbital Hubbard model with $\\beta=50$ at half filling. The thin lines correspond to metallic solutions, \nwhile the thick line ($U=6$) corresponds to a Mott insulating solution. \nBottom left panel: Spectral function $\\mathscr A_{(nn)^2}(\\omega)=-\\frac{1}{\\pi}{\\rm Im}\\,C_{(n n)^2}^R(\\omega)$ multiplied by $\\coth\\big(\\frac{\\beta\\omega}{4}\\big)$.\nBottom right panel: Modulus of the OTOC $|\\langle \\hat n(t)\\hat n(0), \\hat n(t)\\hat n(0)\\rangle - 1|$ (normalized at $t=0$) on a log-log scale. \n}\n\\label{fig:results_nn}\n\\end{center}\n\\end{figure}\n\n\nIn Fig.~\\ref{fig:results_nn}, we plot the OTOC $\\langle \\hat n(t)\\hat n(0), \\hat n(t)\\hat n(0)\\rangle$\nfor the single-orbital Hubbard model in a similar manner as in Fig.~\\ref{fig:results_fermion}.\nSince at half filling this correlation function approaches $\\langle \\hat n\\rangle^4=1$ at long times\nand at sufficiently low temperature,\nwe perform the analytical continuation procedure for $\\langle \\hat n(t)\\hat n(0), \\hat n(t)\\hat n(0)\\rangle-1$. \nThe top panels of Fig.~\\ref{fig:results_nn} show the real and imaginary parts of this shifted OTOC. \nContrary to the case of $\\langle c_\\sigma^\\dagger(t)c_\\sigma(0), c_\\sigma^\\dagger(t)c_\\sigma(0)\\rangle$, \nthe amplitude of the OTOC $\\langle \\hat n(t)\\hat n(0), \\hat n(t)\\hat n(0)\\rangle-1$ is suppressed as one increases the interaction, which reflects the reduced charge fluctuations in the Mott insulator.\nThe bottom left panel in Fig.~\\ref{fig:results_nn} shows the analytically continued spectral function $\\mathscr A_{(nn)^2}(\\omega)=-\\frac{1}{\\pi}{\\rm Im}\\, C_{(nn)^2}^R(\\omega)$ multiplied by $\\coth\\big(\\frac{\\beta\\omega}{4}\\big)$. \nAgain we notice the opening of a gap in the Mott phase, and a corresponding shift of the spectral weight to higher energies. This results in more rapid oscillations of the OTOC in the Mott state.\nThe bottom right panel in Fig.~\\ref{fig:results_nn} plots the modulus of the OTOC $|\\langle \\hat n(t)\\hat n(0), \\hat n(t)\\hat n(0)\\rangle-1|$ (normalized at $t=0$) on a log-log scale.\nThe long-time behavior of the OTOC is consistent with a power-law decay in the metallic phase, and an exponential decay in the Mott insulating phase.\n\n\n\\begin{figure}[t]\n\\begin{center}\n\\includegraphics[angle=-90, width=0.49\\columnwidth]{plot_otoc_nsns_b50_draft_Re.eps}\n\\includegraphics[angle=-90, width=0.49\\columnwidth]{plot_otoc_nsns_b50_draft_Im.eps}\n\\includegraphics[angle=-90, width=0.49\\columnwidth]{plot_coth_b50_nsns_inset.eps}\n\\includegraphics[angle=-90, width=0.49\\columnwidth]{plot_otoc_nsns_b50_draft_modulus_zoom.eps}\n\\caption{\nTop panels: Real and imaginary parts of the OTOC $\\langle \\hat n_\\sigma(t)\\hat n_\\sigma(0), \\hat n_\\sigma(t)\\hat n_\\sigma(0)\\rangle  - \\langle \\hat n_\\sigma\\rangle^4$ for the single-orbital Hubbard model with $\\beta=50$ at half filling. All the colored lines correspond to metallic solutions. \nBottom left panel: Spectral function $\\mathscr A_{(n_\\sigma n_\\sigma)^2}(\\omega)=-\\frac{1}{\\pi}{\\rm Im}\\,C_{(n_\\sigma n_\\sigma)^2}^R(\\omega)$ multiplied by $\\coth\\big(\\frac{\\beta\\omega}{4}\\big)$. In the inset, we plot the imaginary-time four-point function $\\langle \\hat n_\\sigma(\\tau)\\hat n_\\sigma(0), \\hat n_\\sigma(\\tau)\\hat n_\\sigma(0)\\rangle-\\langle \\hat n_\\sigma\\rangle^4$, with the black line corresponding to the Mott insulating solution at $U=6$.  \nBottom right panel: Modulus of the OTOC $|\\langle \\hat n_\\sigma(t)\\hat n_\\sigma(0), \\hat n_\\sigma(t)\\hat n_\\sigma(0)\\rangle  - \\langle \\hat n_\\sigma\\rangle^4|$ (normalized at $t=0$) on a log-log scale.\n}\n\\label{fig:results_nsns}\n\\end{center}\n\\end{figure}\n\nFigure~\\ref{fig:results_nsns} \nshows the results of the OTOC $\\langle \\hat n_\\sigma(t)\\hat n_\\sigma(0), \\hat n_\\sigma(t)\\hat n_\\sigma(0)\\rangle$\nfor the single-orbital Hubbard model. At half filling, this correlation function approaches $\\langle \\hat n_\\sigma\\rangle^4=\\frac{1}{16}$ in the long-time limit and at sufficiently low temperature, \nso that we perform the analytical continuation procedure for $\\langle \\hat n_\\sigma(t)\\hat n_\\sigma(0), \\hat n_\\sigma(t)\\hat n_\\sigma(0)\\rangle-\\langle \\hat n_\\sigma\\rangle^4$. \nThe top panels of Fig.~\\ref{fig:results_nsns} show the real and imaginary parts of \nthis shifted OTOC. \nHere, we only present results for the metallic phase, because resolving the very sharp low-energy feature of the OTOC spectrum in the half-filled Mott state is challenging. One can see that coherent oscillations are not observed for this type of OTOC, and that the incoherent part becomes dominant compared to the other two OTOCs shown in Figs.~\\ref{fig:results_fermion} and \\ref{fig:results_nn}. The amplitude of the OTOC $\\langle \\hat n_\\sigma(t)\\hat n_\\sigma(0), \\hat n_\\sigma(t)\\hat n_\\sigma(0)\\rangle-\\langle \\hat n_\\sigma\\rangle^4$ is enhanced as $U$ increases. As we argue below, this is related to the enhancement of spin fluctuations \nand local moment formation close to the Mott transition.\n\nThe bottom left panel in Fig.~\\ref{fig:results_nsns} plots the analytically continued spectral function $\\mathscr A_{(n_\\sigma n_\\sigma)^2}(\\omega)=-\\frac{1}{\\pi}{\\rm Im}\\, C_{(n_\\sigma n_\\sigma)^2}^R(\\omega)$ multiplied by $\\coth\\big(\\frac{\\beta\\omega}{4}\\big)$. \nWe can see that the spectral weight is concentrated in the low-energy region as we approach the Mott transition point.\nThe inset in the bottom left panel of Fig.~\\ref{fig:results_nsns} shows the imaginary-time four-point function $\\langle \\hat n_\\sigma(\\tau)\\hat n_\\sigma(0), \\hat n_\\sigma(\\tau)\\hat n_\\sigma(0)\\rangle-\\langle \\hat n_\\sigma\\rangle^4$, with the black line corresponding to a Mott insulating solution ($U=6$).  \nFor $U\\gtrsim 4$, the imaginary-time four-point function does not decay to zero but remains relatively large near $\\tau=\\frac{\\beta}{4}$. \nThis behavior is reminiscent of the spin-freezing physics seen in the correlated Hund metal phase of multi-orbital Hubbard models, where the dynamical spin correlation function $\\langle \\hat S_z(\\tau)\\hat S_z(0)\\rangle$ is trapped at a finite value at long $\\tau$ \\cite{Werner2008}. \nIn fact, the OTOC $\\langle \\hat n_\\sigma(\\tau)\\hat n_\\sigma(0), \\hat n_\\sigma(\\tau)\\hat n_\\sigma(0)\\rangle$\nmeasures spin correlations (in addition to charge correlations) through $\\hat n_\\sigma=\\frac{1}{2}\\hat n+\\sigma\\hat S_z$.\nPhysically, the trapping of $\\langle \\hat S_z(\\tau)\\hat S_z(0)\\rangle$ signals the formation of frozen local magnetic moments.\nIn the metal-insulator crossover region, the scattering induced by these magnetic moments results in incoherent metal states. Intuitively, one might expect fast scrambling in the corresponding regions of the phase diagram. \nAlthough there is some resemblance with the spin freezing crossover,\nthe self-energy does not exhibit a square-root frequency dependence in the single-orbital case, and a spin-freezing crossover in the sense of Ref.~\\cite{Werner2008} does not exist.  \n\nThe bottom right panel of Fig.~\\ref{fig:results_nsns} plots the modulus of the OTOC $|\\langle \\hat n_\\sigma(t)\\hat n_\\sigma(0), \\hat n_\\sigma(t)\\hat n_\\sigma(0)\\rangle-\\langle \\hat n_\\sigma\\rangle^4|$ (normalized at $t=0$) on a log-log scale. \nThe dynamics of $\\langle \\hat n_\\sigma(t)\\hat n_\\sigma(0), \\hat n_\\sigma(t)\\hat n_\\sigma(0)\\rangle$ is quite distinct from that of $\\langle \\hat n(t)\\hat n(0), \\hat n(t)\\hat n(0)\\rangle$. As the Mott transition is approached, we observe a slow-down in the decay of the modulus, \nwhich indicates the presence of slowly fluctuating local moments. \nThe long-time behavior may be consistent with a power-law, although slow oscillations make it difficult to determine the long-time asymptotic form from the numerics. \n\n\n\\begin{figure}[t]\n\\begin{center}\n\\includegraphics[angle=-90, width=0.49\\columnwidth]{plot_fitnsns_b2.eps}\n\\includegraphics[angle=-90, width=0.49\\columnwidth]{plot_coeffc_draft.eps}\n\\caption{\nLeft panel: \nThe imaginary-time four-point function $\\langle \\hat n_\\sigma(\\tau)\\hat n_\\sigma(0), \\hat n_\\sigma(\\tau)\\hat n_\\sigma(0)\\rangle$, the squared density-density correlation function $\\langle \\hat n_\\sigma(\\tau)\\hat n_\\sigma(0)\\rangle^2$,\nand their difference for the single-orbital Hubbard model with $U=5.5$ and $\\beta=50$.\nThe dashed curve shows a fit of the difference with the function $f_\\text{fit}(\\tau)=a+b\\tau+c\\tau^2$ in the interval $\\tau/\\beta \\in [0: 0.2]$.\nRight panel: Fitting coefficient $c$ as a function of $U$ for indicated values of $\\beta$.\n\n}\n\\label{fig:coeffs}\n\\end{center}\n\\end{figure}\n\nFor large enough $\\beta$ and small enough $\\tau$, the OTOC (\\ref{otoc_nsns}) factorizes into $\\langle \\hat n_\\sigma(\\tau)\\hat n_\\sigma(0), \\hat n_\\sigma(\\tau)\\hat n_\\sigma(0)\\rangle \\approx \\langle \\hat n_\\sigma(\\tau)\\hat n_\\sigma(0)\\rangle^2$. \nOn the real-time axis, this corresponds to the decoupling $\\langle \\hat n_\\sigma(t)\\hat n_\\sigma(0), \\hat n_\\sigma(t)\\hat n_\\sigma(0)\\rangle \\approx \\langle \\hat n_\\sigma(t)\\hat n_\\sigma(0)\\rangle^2$, that is, \nthe OTOC is reduced to a product of ordinary density-density correlation functions.\nIn order to see whether nontrivial correlations are captured beyond the decoupled form by the OTOC function, \nwe consider the difference $\\langle \\hat n_\\sigma(\\tau)\\hat n_\\sigma(0), \\hat n_\\sigma(\\tau)\\hat n_\\sigma(0)\\rangle-\\langle \\hat n_\\sigma(\\tau)\\hat n_\\sigma(0)\\rangle^2$ and fit the short-time behavior of this function to $f_\\text{fit}(\\tau)=a+b\\tau+c\\tau^2$, as illustrated in the left panel of Fig.~\\ref{fig:coeffs}. The coefficients $b$ and $c$ serve as  indicators of the nontrivial correlations that cannot be attributed to the decoupled form of the OTOC.\nThe coefficient $c$ is plotted as a function of $U$ and for different $\\beta$ in the right hand panel. The coefficient $b$ (not shown) exhibits a similar trend. We notice that the OTOC picks up nontrival correlations near the metal-insulator transition, and in particular at \nintermediate\ntemperatures, while the Mott phase shows no such correlations. The crossover region  \nassociated with the emergence of local moments \ncorresponds, roughly, to the interaction range where the nontrivial correlations start to become significant. \n\n\n4.3 Two-orbital Hubbard model\n\\subsection{Two-orbital Hubbard model}\n\\label{sec:two-orbital hubbard}\n\nIn this section, we investigate OTOCs in the two-orbital Hubbard model with Hund coupling $J>0$ \nand density-density interactions. (Three-orbital results are presented in Appendix~\\ref{sec:3orbital}.) \nThe Hamiltonian is given by\n\\begin{align}\nH\n&=\n-v\\sum_{\\langle i,j\\rangle\\alpha\\sigma} (c^\\dagger_{i\\alpha\\sigma}c_{j\\alpha\\sigma}+ \\text{h.c.})\n-\\mu\\sum_{i\\alpha\\sigma} \\hat n_{i\\alpha\\sigma}\n+U\\sum_{i\\alpha} \\hat n_{i\\alpha\\uparrow}\\hat n_{i\\alpha\\downarrow}\n+(U-2J)\\sum_{i\\sigma} \\hat n_{i1\\sigma}\\hat n_{i2\\bar\\sigma}\n\\notag\n\\\\\n&\\quad\n+(U-3J)\\sum_{i\\sigma} \\hat n_{i1\\sigma}\\hat n_{i2\\sigma},\n\\end{align}\n\nwith $U$ being the intra-orbital interaction $U$, $U-2J$ the inter-orbital antiparallel-spin interaction, and $U-3J$ the inter-orbital parallel-spin interaction. This is the simplest model that shows a crossover from a spin-frozen metal to a Fermi-liquid metal as one dopes the half-filled Mott insulator, with the self-energy scaling as $\\text{Im}\\Sigma(\\omega_n)\\sim\\sqrt{\\omega_n}$ over a significant energy range in the crossover regime \\cite{Hafermann2012}.\nThe sketch of the phase diagram is shown in Fig.~\\ref{fig:illustration}. \nAs we have seen in the previous section, the spin-related OTOC of the type (\\ref{otoc_nsns}) is relevant for the analysis of  the spin-freezing crossover, so that we concentrate on this OTOC here.\nIn the following calculations, we choose $U=8$, $J=U/4$, and compute $\\langle \\hat n_\\sigma(t)\\hat n_\\sigma(0), \\hat n_\\sigma(t)\\hat n_\\sigma(0)\\rangle - \\langle n_\\sigma \\rangle^4$ \nas a function of filling at $\\beta=50$ (dashed lines in Fig.~\\ref{fig:illustration}). \n\n\n\\begin{figure}[t]\n\\begin{center}\n\\includegraphics[angle=-90, width=0.49\\columnwidth]{plot_qp_draft.eps}\\hfill\n\\includegraphics[angle=-90, width=0.49\\columnwidth]{plot_fit_2orbital.eps}\n\\caption{\nLeft panel: Imaginary part of the self-energy as a function of Matsubara frequency for the two-orbital Hubbard model with $U=8$, $J=U/4$, and $\\beta=50$ on a log-log scale, with the dashed line indicating $\\sqrt{\\omega_n}$ behavior. \nRight panel: The coefficient $c$ extracted from\nfitting $\\langle \\hat n_\\sigma(\\tau)\\hat n_\\sigma(0), \\hat n_\\sigma(\\tau)\\hat n_\\sigma(0)\\rangle-\\langle \\hat n_\\sigma(\\tau)\\hat n_\\sigma(0)\\rangle^2$ with a function $f_\\text{fit}(\\tau)=a+b\\tau+c\\tau^2$\nin the interval $\\tau/\\beta\\in [0: 0.05]$ for the two-orbital Hubbard model. \nThe spin-freezing crossover occurs roughly at the filling $n_\\sigma\\approx 0.28$ (blue curve in the left panel and blue arrow in the right panel). \n}\n\\label{fig:self}\n\\end{center}\n\\end{figure}\n\n\nThe left panel of Fig.~\\ref{fig:self} plots the imaginary part of the self-energy as a function of Matsubara frequency in order to identify the filling corresponding to the spin-freezing crossover \\cite{Werner2008}. An approximate square-root scaling is observed over a wide range of frequencies near $n_\\sigma\\approx 0.28$ (blue line). In the right panel of Fig.~\\ref{fig:self}, we present the coefficient $c$ obtained from a similar analysis as presented in Fig.~\\ref{fig:coeffs}, but with a fitting range $\\tau/\\beta\\in [0 : 0.05]$. Again, we find that the OTOC $\\langle \\hat n_\\sigma(t)\\hat n_\\sigma(0), \\hat n_\\sigma(t)\\hat n_\\sigma(0)\\rangle - \\langle \\hat n_\\sigma\\rangle^4$ detects nontrivial correlations in the spin-freezing crossover (blue arrow) and spin-frozen metal regime, but not in the Fermi liquid or the Mott insulating phase.\n\n\n\\begin{figure}[t]\n\\begin{center}\n\\includegraphics[angle=-90, width=0.49\\columnwidth]{plot_otoc_2orbital_Re.eps}\n\\includegraphics[angle=-90, width=0.486\\columnwidth]{plot_otoc_2orbital_Im.eps}\n\\includegraphics[angle=-90, width=0.484\\columnwidth]{plot_spectra_coth_2orbital_inset.eps}\n\\includegraphics[angle=-90, width=0.486\\columnwidth]{plot_otoc_2orbital_modulus_inset.eps}\n\\caption{\nTop panels: Real and imaginary parts of the OTOC $\\langle \\hat n_\\sigma(t)\\hat n_\\sigma(0), \\hat n_\\sigma(t)\\hat n_\\sigma(0)\\rangle  - \\langle \\hat n_\\sigma\\rangle^4$ for the two-orbital Hubbard model with $U=8$, $J=U/4$, $\\beta=50$ and indicated fillings. \nBottom left panel: Spectral function $\\mathscr A_{(n_\\sigma n_\\sigma)^2}(\\omega)=-\\frac{1}{\\pi}{\\rm Im}\\,C_{(n_\\sigma n_\\sigma)^2}^R(\\omega)$ multiplied by $\\coth\\big(\\frac{\\beta\\omega}{4}\\big)$. In the inset, we plot the imaginary-time four-point function $\\langle \\hat n_\\sigma(\\tau)\\hat n_\\sigma(0), \\hat n_\\sigma(\\tau)\\hat n_\\sigma(0)\\rangle-\\langle \\hat n_\\sigma\\rangle^4$. \nBottom right panel: Modulus of the OTOC $|\\langle \\hat n_\\sigma(t)\\hat n_\\sigma(0), \\hat n_\\sigma(t)\\hat n_\\sigma(0)\\rangle  - \\langle \\hat n_\\sigma\\rangle^4|$ (normalized at $t=0$) on a log-log scale.\nIn the inset of the bottom right panel, we illustrate the approximate data collapse obtained by considering the spin-freezing crossover regimes for different values of $J$. \nIn all the panels, the thick blue curves represent the results for the spin-freezing crossover region. \n}\n\\label{fig:2orbital}\n\\end{center}\n\\end{figure}\n\n\nThe top panels of Fig.~\\ref{fig:2orbital} show the real and imaginary parts of the OTOC\n$\\langle \\hat n_\\sigma(t)\\hat n_\\sigma(0), \\hat n_\\sigma(t)\\hat n_\\sigma(0)\\rangle-\\langle \\hat n_\\sigma\\rangle^4$\nfor the two-orbital Hubbard model. In the spin-frozen phase ($n_\\sigma \\gtrsim 0.3$), both components exhibit \nhighly incoherent oscillations, which get suppressed as the filling is reduced. As one moves across the crossover point ($n_\\sigma \\sim 0.28$),\nthe oscillations completely disappear, and the real part of the OTOC starts to overshoot to the negative side\nin the Fermi liquid regime.\nThe bottom left panel in Fig.~\\ref{fig:2orbital} presents the analytically continued spectral function $\\mathscr A_{(n_\\sigma n_\\sigma)^2}(\\omega)=-\\frac{1}{\\pi}{\\rm Im}\\,C_{(n_\\sigma n_\\sigma)^2}^R(\\omega)$ multiplied by $\\coth\\big(\\frac{\\beta\\omega}{4}\\big)$. \nIn the spin-frozen regime ($n_\\sigma \\gtrsim 0.3$) a sharp peak appears at low energy, which originates from the saturation of the imaginary-time four point function $\\langle \\hat n_\\sigma(\\tau)\\hat n_\\sigma(0), \\hat n_\\sigma(\\tau)\\hat n_\\sigma(0)\\rangle$ at large $\\tau$ (see the inset of the bottom left panel of Fig.~\\ref{fig:2orbital}). This low-energy peak represents the slow dynamics of the frozen spins, and translates into a slow long-time decay of $|\\langle \\hat n_\\sigma(t)\\hat n_\\sigma(0), \\hat n_\\sigma(t)\\hat n_\\sigma(0)\\rangle-\\langle \\hat n_\\sigma\\rangle^4|$,\nas shown in the bottom right panel of Fig.~\\ref{fig:2orbital}\non a log-log scale. Again, it is difficult to clearly resolve the long-time behavior of the OTOC due to the limited\naccuracy of the analytic continuation, but in the spin-freezing crossover regime (thick blue curves)\nthe decay of the OTOC is consistent with a power law $\\sim 1/t^{1.5}$ at least for $2\\lesssim t \\lesssim 5$. \nThe shorter-time behavior ($0.5\\lesssim t \\lesssim 2$) is instead well fitted by an exponential decay $\\sim e^{-\\alpha t}$ with a decay constant $\\alpha=0.78$. \nThis exponent exhibits\na strong dependence on the Hund coupling. \n\nIn fact, the OTOCs in the spin-freezing regime for $J=U/4$ and $J=U/6$ can be approximately collapsed by plotting them as a function of $tJ$ (see the inset in the bottom right panel of Fig.~\\ref{fig:2orbital}), which indicates that the Hund coupling is the parameter which controls the dynamics in this regime. \nThis is distinct from the Lyapunov behavior in which the exponential decay scales with $t/\\beta$ \\cite{Bagrets2017}.\nA possible reason is that we are still far from the large $N$ limit (where $N$ corresponds to the number of orbitals times the number of spin degrees of freedom) so that the large $N$ behavior \nsuch as the Lyapunov growth is not observed in our calculations. \nIn fact, the strong dependence of the time scale on $J$ and weak dependence on $\\beta$ is consistent\nwith the behavior of the finite $N$ SYK model in Ref.~\\cite{FuSachdev2016}.\nIn the spin-frozen regime ($n_\\sigma\\gtrsim 0.3$) the decay is much slower than near the crossover point, while in the Fermi liquid regime the decay is accelerated, approaching the free fermion behavior of $t^{-3}$\nas shown in Fig.~\\ref{analytic continuation}. \nQualitatively similar results for this OTOC are obtained \nfor the three-orbital Hubbard model (see Appendix~\\ref{sec:3orbital}).\n\nThe time dependence of the modulus of \n$\\langle \\hat n_\\sigma(\\tau)\\hat n_\\sigma(0), \\hat n_\\sigma(\\tau)\\hat n_\\sigma(0)\\rangle-\\langle \\hat n_\\sigma\\rangle^4$ \nin the spin-freezing crossover regime of multi-orbital Hubbard models (exponential decay crossing over into a power-law) bears a close resemblance to a (different) OTOC for the SYK model discussed in Ref.~\\cite{Bagrets2017}. \nAlso, a recent study of yet another type of OTOCs for the finite-$N$ SYK model found a qualitatively similar decay as observed here in the spin-freezing crossover regime \\cite{FuSachdev2016}. In the following section, we will make the connection to the SYK dynamics more quantitative.   \n\n\n",
        "label": "fig:coeffs",
        "Descriptive_question1": "What is the value of U used in the left panel of figure_8?",
        "Descriptive_question2": "What is the value of β shown in the left panel of figure_8?",
        "Reasoning_question1": "How does the fitting coefficient c vary with U in the right panel of figure_8, and what might this indicate about the metal-insulator transition?",
        "Reasoning_question2": "Why does the difference between the imaginary-time four-point function and the squared density-density correlation function in the left panel of figure_8 require a quadratic fit, and what does this suggest about the underlying correlations?",
        "Descriptive_answer1": "5.5",
        "Descriptive_answer2": "50",
        "Reasoning_answer1": "The fitting coefficient c initially increases as U increases, showing a peak around the metal-insulator transition region, then decreases in the Mott insulating phase. This indicates that nontrivial correlations beyond the decoupled form become significant near the metal-insulator transition, reflecting enhanced fluctuations or local moment formation, while in the insulating phase such correlations diminish.",
        "Reasoning_answer2": "The difference between the four-point function and squared density-density correlation cannot be captured by a linear term alone and necessitates a quadratic fit, implying that the nontrivial correlations beyond simple factorization have a non-linear time dependence at short imaginary times. This suggests the presence of subtle dynamical correlations and fluctuations emerging near the metal-insulator transition beyond what is captured by factorized correlations."
    },
    {
        "paper_id": "1812.04217.json",
        "image_id": "figure_9",
        "image_path": [
            "/home/yz979/scratch/chengye/scimolmo/test-images-tables/dataset_imagestables/1812.04217/images/plot_qp_draft.eps"
        ],
        "caption": "\nLeft panel: Imaginary part of the self-energy as a function of Matsubara frequency for the two-orbital Hubbard model with $U=8$, $J=U/4$, and $\\beta=50$ on a log-log scale, with the dashed line indicating $\\sqrt{\\omega_n}$ behavior. \nRight panel: The coefficient $c$ extracted from\nfitting $\\langle \\hat n_\\sigma(\\tau)\\hat n_\\sigma(0), \\hat n_\\sigma(\\tau)\\hat n_\\sigma(0)\\rangle-\\langle \\hat n_\\sigma(\\tau)\\hat n_\\sigma(0)\\rangle^2$ with a function $f_\\text{fit}(\\tau)=a+b\\tau+c\\tau^2$\nin the interval $\\tau/\\beta\\in [0: 0.05]$ for the two-orbital Hubbard model. \nThe spin-freezing crossover occurs roughly at the filling $n_\\sigma\\approx 0.28$ (blue curve in the left panel and blue arrow in the right panel). \n",
        "classify": "Chart",
        "section_info": "4 Numerical results for the Hubbard model\n\\section{Numerical results for the Hubbard model}\n\\label{numerical results}\n\n\\subsection{Observables and numerical procedure}\n\nWe consider the single-orbital, two-orbital, and three-orbital Hubbard models on an infinitely-connected Bethe lattice, which can be solved exactly within DMFT \\cite{GeorgesKotliarKrauthRozenberg1996}. In this case, the noninteracting density of states is semicircular with bandwidth $4v_\\ast$, for which there exists a simplified DMFT self-consistency condition between the hybridization function $\\Delta_{\\alpha\\sigma}$ of the DMFT impurity problem and the local (impurity) Green's function $G_{\\alpha\\sigma}$: \n\n\\begin{equation}\n\\Delta_{\\alpha\\sigma}(\\tau)=v_\\ast^2 G_{\\alpha\\sigma}(\\tau), \n\\label{semicirc}\n\\end{equation}\n\nwith $\\alpha$ the orbital and $\\sigma$ the spin index. We will use $v_\\ast$ as the unit of energy and measure time in units of $\\hbar/v_\\ast$. \n\nThree different types of imaginary-time four-point functions $C_{(AB)^2}^M(\\tau)$\nof the form of Eq.~(\\ref{imaginary-time OTOC})\nwith $(\\hat A, \\hat B)=(c_\\sigma^\\dagger, c_\\sigma), (\\hat n_\\sigma, \\hat n_\\sigma)$, and $(\\hat n, \\hat n)$\nare calculated in the interval $0\\le \\tau\\le \\frac{\\beta\\hbar}{2}$: \n\\begin{eqnarray}\nC_{(c^\\dagger_\\sigma c_\\sigma)^2}^M(\\tau) &=& -\\langle c^\\dagger_\\sigma(\\tau+\\tfrac{\\beta\\hbar}{2}) c_\\sigma(\\tfrac{\\beta\\hbar}{2}) c^\\dagger_\\sigma(\\tau) c_\\sigma(0) \\rangle, \\label{fermion_otoc}\\\\\nC_{(n_\\sigma n_\\sigma)^2}^M(\\tau) &=& -\\langle \\hat n_\\sigma(\\tau+\\tfrac{\\beta\\hbar}{2})\\hat n_\\sigma(\\tfrac{\\beta\\hbar}{2})\\hat n_\\sigma(\\tau)\\hat n_\\sigma(0) \\rangle, \\label{otoc_nsns}\\\\\nC_{(nn)^2}^M(\\tau) &=& -\\langle \\hat n(\\tau+\\tfrac{\\beta\\hbar}{2})\\hat n(\\tfrac{\\beta\\hbar}{2})\\hat n(\\tau)\\hat n(0) \\rangle,\\label{otoc_nn}\n\\end{eqnarray}\n\nwhere $c_\\sigma^\\dagger$ ($c_\\sigma$) are the fermionic creation (annihilation) operators for spin $\\sigma$ (and orbital $\\alpha=1$ in the multi-orbital case), $\\hat n_\\sigma=c^\\dagger_\\sigma c_\\sigma$ is the corresponding spin-dependent density operator, and $\\hat n=\\hat n_\\uparrow + \\hat n_\\downarrow$ the total density operator. Note that in all the three cases above we have $\\hat B=\\hat A^\\dagger$. \nWe measure these local correlation functions in the impurity model using a hybridization expansion continuous-time Monte Carlo algorithm (CT-HYB) \\cite{Werner2006}. In this algorithm, the \ntwo-particle\nGreen's functions of the type (\\ref{fermion_otoc}) can be measured by removing two hybridization lines, i.e., from the elements of the inverse hybridization matrix, while the density-density correlation functions can be easily measured either by insertion of density operators (matrix formalism) \\cite{Werner2006Kondo} or by reading off the occupation of the orbitals at the four time points in the segment implementation \\cite{Werner2006}. We use the latter algorithm since we consider only density-density interactions. \nThe values at the end-points $\\tau=0$ and $\\tau=\\frac{\\beta\\hbar}{2}$ reduce to standard density-density correlation functions, which in the case of Eq.~(\\ref{fermion_otoc}) are measured separately. \n\nWe perform the analytic continuation to the real-frequency axis using the Maximum Entropy method \\cite{Jarrell1996,Lewin_maxent} with a bosonic kernel. This yields the imaginary part of the retarded correlator \n$-\\frac{1}{\\pi}\\text{Im}\\, C^R_{(AA^\\dagger)^2}(\\omega)=\\mathscr A_{(AA^\\dagger)^2}(\\omega)$.  \nWhen $\\hat B=\\hat A^\\dagger=\\hat A$ (which is the case for $\\hat A=\\hat n_\\sigma, \\hat n$), we have $C_{(AA)^2}^R(\\omega)^\\ast=C_{(AA)^2}^R(-\\omega)$. At half filling, the particle-hole symmetry furthermore ensures $C_{c_\\sigma^\\dagger c_\\sigma}^R(\\omega)^\\ast=C_{c_\\sigma^\\dagger c_\\sigma}^R(-\\omega)$.\nThus, in all the cases considered in this study, the retarded OTOC has the symmetry property $C_{(AA^\\dagger)^2}^R(\\omega)^\\ast=C_{(AA^\\dagger)^2}^R(-\\omega)$. Using this, as well as the out-of-time-order fluctuation-dissipation\ntheorem discussed in the previous section, we can obtain the real-time OTOC\n$\\langle \\hat A(t)\\hat A^\\dagger(0), \\hat A(t)\\hat A^\\dagger(0)\\rangle$ from the following inverse Fourier transformation, \n\\begin{align}{\\rm Re}\\langle \\hat A(t)\\hat A^\\dagger(0), \\hat A(t)\\hat A^\\dagger(0)\\rangle\n&=\n{\\rm Re}\n\\int_0^\\infty d\\omega\\, e^{-i\\omega t}\\coth\\left(\\frac{\\beta\\hbar\\omega}{4}\\right)\n\\left(-\\frac{1}{\\pi}{\\rm Im}\\, C_{(AA^\\dagger)^2}^R(\\omega)\\right),\n\\\\\n{\\rm Im}\\langle \\hat A(t)\\hat A^\\dagger(0), \\hat A(t)\\hat A^\\dagger(0)\\rangle\n&=\n{\\rm Im}\\int_0^{\\infty} d\\omega\\, e^{-i\\omega t} \\left(-\\frac{1}{\\pi}{\\rm Im}\\, C_{(AA^\\dagger)^2}^R(\\omega)\\right),\n\\end{align}\nfor $\\hat A=c_\\sigma^\\dagger, \\hat n_\\sigma$, and $\\hat n$.\n\n\\begin{figure}[b]\n\\begin{center}\n\\includegraphics[angle=-90, width=0.48\\columnwidth]{compare_u0_draft.eps}\\hfill\n\\includegraphics[angle=-90, width=0.48\\columnwidth]{compare_u0_longtime_modulus.eps}\n\\caption{\nLeft panel: Comparison between the exact solution and the result obtained by analytical continuation\nfor the real and imaginary parts of the OTOC $\\langle c_\\sigma^\\dagger(t)c_\\sigma(0), c_\\sigma^\\dagger(t)c_\\sigma(0)\\rangle$ \nin the noninteracting case ($U=0, \\beta\\hbar=50$) of the single-orbital Hubbard model.\nRight panel: Comparison of the long-time behavior for the modulus $|\\langle c_\\sigma^\\dagger(t)c_\\sigma(0), c_\\sigma^\\dagger(t)c_\\sigma(0)\\rangle|$. \nThe exact result shows a power-law decay ($\\sim 1/t^3$). \n}\n\\label{fig:test_u0}\n\\end{center}\n\\end{figure}\n\nAs a check of our procedure, we first compare the results for the noninteracting model,\nfor which an exact solution of the OTOC $\\langle c_\\sigma^\\dagger(t)c_\\sigma(0), c_\\sigma^\\dagger(t)c_\\sigma(0)\\rangle$ \nis available \\cite{TsujiWernerUeda2017}.\nIt is a nontrivial task to measure the noninteracting two-particle Green's function in CT-HYB, \nand to analytically continue \n$C_{(c^\\dagger_\\sigma c_\\sigma)^2}^M(\\tau)$ \nby the Maximum Entropy method.  \nThe results for the inverse temperature $\\beta\\hbar=50$ are plotted in Fig.~\\ref{fig:test_u0}. \nOne can see that the OTOC oscillates for about one cycle, and quickly decays to zero.\nThe analytically continued data show a good agreement with the exact solution. \nIn particular, for short times (up to about two inverse hoppings) the dynamics is accurately reproduced, while deviations appear at longer times. \nIt is known that for the noninteracting fermion system the OTOC decays as a power law at long time ($\\sim 1/t^3$) \\cite{TsujiWernerUeda2017}.\nThe analytical continuation method cannot reproduce the details of the oscillations at intermediate or long times, but it roughly captures the long-time decay, which is controlled by low-frequency spectral features. \nUsually, analytic continuation is unreliable for high frequency components, because high-frequency information is suppressed \nby the kernel $K(\\tau,\\omega)$ in the transformation from the spectral function $A_O(\\omega)$ to the (bosonic) Matsubara correlation function $O(\\tau)$ [$O(\\tau)=\\int_{-\\infty}^{\\infty} d\\omega\\, K(\\omega,\\tau) A_O(\\omega)$ with $K(\\tau,\\omega)=e^{-\\tau\\omega}/(1-e^{-\\beta\\omega})$]. On the other hand, the low frequency components can be rather accurately determined. As a result, in Fig. 4 we more or less recover the low-frequency features of the time-dependent correlation function, while the details of rapid oscillations are not captured. The accurate result at short times can be understood from the fact that this region is close to the imaginary time axis in the complex time plane, where the original QMC data are available.\nAt higher temperatures, the Maximum Entropy method becomes less reliable, so that the resulting OTOCs are expected to be less accurate.  \n\n\n\\subsection{Single-orbital Hubbard model}\n\\label{sec:hubbard}\n\nIn this section, we calculate the OTOCs (\\ref{fermion_otoc})-(\\ref{otoc_nn}) for the interacting single-orbital Hubbard model with the Hamiltonian \n\\begin{align}\nH\n&=\n-v\\sum_{\\langle i,j\\rangle\\sigma} (c^\\dagger_{i\\sigma}c_{j\\sigma}+ \\text{h.c.})\n-\\mu\\sum_i \\hat n_i+U\\sum_i \\hat n_{i\\uparrow}\\hat n_{i\\downarrow}, \n\\end{align}\nwhere $v$ is the hopping amplitude, $\\langle i,j\\rangle$ represents the nearest-neighbor site pairs, \n$\\mu$ is the chemical potential, and $U$ is the on-site interaction.\nIn this section, we focus on the half-filled case (i.e., $\\mu=U/2$). The DMFT solution for the simplified self-consistency (\\ref{semicirc}) is the exact solution for an infinitely connected Bethe lattice \\cite{GeorgesKotliarKrauthRozenberg1996}.\nLet us briefly recall the paramagnetic phase diagram for the single-orbital Hubbard model on this lattice \\cite{Bluemer_thesis}. At half-filling, there is a metal-insulator crossover at high temperature and a first-order Mott transition below a temperature corresponding to $\\beta\\hbar\\approx 17$ with a coexistence region between $U_{c1}$ and $U_{c2}$. The finite-temperature critical endpoint is at $U\\approx 4.7$, while the zero-temperature Mott transition occurs at $U=U_{c2}\\approx 5.6$. \nAt low temperature, the self-energy shows the Fermi liquid behavior [${\\rm Im}\\,\\Sigma(\\omega)\\sim \\omega^2$] \nin the weakly correlated metallic phase, \nwhile it shows an insulating behavior [${\\rm Im}\\,\\Sigma(\\omega)\\sim \\delta(\\omega)$] in the Mott phase\n\\cite{GeorgesKotliarKrauthRozenberg1996}. \nIn the correlated metallic phase, as the temperature is increased, deviations from the Fermi liquid behavior become apparent, \nbut an ${\\rm Im}\\,\\Sigma(\\omega)\\sim \\sqrt{\\omega}$ scaling as in the SYK model is not observed. \n\nIn the following, we will start the DMFT iterations from the noninteracting solution, which means that the finite-temperature Mott transition occurs at $U_{c2}$.\nFrom now on, we set $\\hbar=1$. \n\n\\begin{figure}[t]\n\\begin{center}\n\\includegraphics[angle=-90, width=0.49\\columnwidth]{plot_otoc_b50_Re.eps}\n\\includegraphics[angle=-90, width=0.49\\columnwidth]{plot_otoc_b50_Im.eps}\n\\includegraphics[angle=-90, width=0.49\\columnwidth]{plot_coth_b50_appendix.eps}\n\\includegraphics[angle=-90, width=0.49\\columnwidth]{plot_otoc_b50_modulus.eps}\n\\caption{\nTop panels: Real and imaginary parts of the OTOC $\\langle c^\\dagger_\\sigma(t)c_\\sigma(0), c^\\dagger_\\sigma(t)c_\\sigma(0)\\rangle$ for the single-orbital Hubbard model with $\\beta=50$ at half filling. The thin lines correspond to metallic solutions, while the thick line ($U=6$) corresponds to a Mott insulating solution. \nBottom left panel: Spectral function $\\mathscr A_{(c_\\sigma^\\dagger c_\\sigma)^2}(\\omega)=-\\frac{1}{\\pi}{\\rm Im}\\,C_{(c^\\dagger_\\sigma c_\\sigma)^2}^R(\\omega)$ multiplied by $\\coth\\big(\\frac{\\beta\\omega}{4}\\big)$. \nBottom right panel: Modulus of the OTOC $|\\langle c^\\dagger_\\sigma(t)c_\\sigma(0), c^\\dagger_\\sigma(t)c_\\sigma(0)\\rangle|$ (normalized at $t=0$) on a log-log scale. The dashed line is the result for $U=0$.\n}\n\\label{fig:results_fermion}\n\\end{center}\n\\end{figure}\n\n\n\nIn Fig.~\\ref{fig:results_fermion}, we show the results of $\\langle c^\\dagger_\\sigma(t)c_\\sigma(0), c^\\dagger_\\sigma(t)c_\\sigma(0)\\rangle$ for the interacting single-orbital Hubbard model at $\\beta=50$. \nThe top panels present the real and imaginary parts of the OTOC. One can see that the oscillations become more pronounced as one increases the interaction. In the bottom left panel of Fig.~\\ref{fig:results_fermion},\nwe show the corresponding spectral function $\\mathscr A_{(c_\\sigma^\\dagger c_\\sigma)^2}(\\omega)=-\\frac{1}{\\pi}{\\rm Im}\\,C_{(c^\\dagger_\\sigma c_\\sigma)^2}^R(\\omega)$ multiplied by $\\coth\\big(\\frac{\\beta\\omega}{4}\\big)$.\nThe Mott transition, which occurs between $U=5$ and $U=6$ at $\\beta=50$, manifests itself by the opening of a gap in the spectral function $\\mathscr A_{(c_\\sigma^\\dagger c_\\sigma)^2(\\omega)}$. This translates into more weakly damped oscillations in the real-time evolution in the Mott phase.  \nThe bottom right panel of Fig.~\\ref{fig:results_fermion} plots the modulus of the OTOC (normalized at time $t=0$, i.e., $|\\langle c^\\dagger_\\sigma(t)c_\\sigma(0), c^\\dagger_\\sigma(t)c_\\sigma(0)\\rangle|/|\\langle c^\\dagger_\\sigma(0)c_\\sigma(0), c^\\dagger_\\sigma(0)c_\\sigma(0)\\rangle|$) on a log-log scale. \nDue to the oscillations and the limited accuracy of the analytic continuation procedure, it is hard to clearly identify the nature of the long-time decay of the OTOC,\nbut the results indicate that the OTOC decays much faster (possibly exponentially) in the Mott phase than in the metallic phase. This is consistent with the results for the spectral functions of the OTOC. The comparison to the noninteracting case (dashed curve in the bottom right panel of Fig.~\\ref{fig:results_fermion}) suggests that the correlated metallic phase has a similar (possibly power-law) decay behavior of the OTOC as in the noninteracting case.\n\n\n\\begin{figure}[t]\n\\begin{center}\n\\includegraphics[angle=-90, width=0.49\\columnwidth]{plot_otoc_b50_nn_Re.eps}\n\\includegraphics[angle=-90, width=0.49\\columnwidth]{plot_otoc_b50_nn_Im.eps}\n\\includegraphics[angle=-90, width=0.49\\columnwidth]{plot_coth_b50_nn_appendix.eps}\n\\includegraphics[angle=-90, width=0.49\\columnwidth]{plot_otoc_b50_nn_modulus.eps}\n\\caption{\nTop panels: Real and imaginary parts of the OTOC $\\langle \\hat n(t)\\hat n(0), \\hat n(t)\\hat n(0)\\rangle - 1$ for the single-orbital Hubbard model with $\\beta=50$ at half filling. The thin lines correspond to metallic solutions, \nwhile the thick line ($U=6$) corresponds to a Mott insulating solution. \nBottom left panel: Spectral function $\\mathscr A_{(nn)^2}(\\omega)=-\\frac{1}{\\pi}{\\rm Im}\\,C_{(n n)^2}^R(\\omega)$ multiplied by $\\coth\\big(\\frac{\\beta\\omega}{4}\\big)$.\nBottom right panel: Modulus of the OTOC $|\\langle \\hat n(t)\\hat n(0), \\hat n(t)\\hat n(0)\\rangle - 1|$ (normalized at $t=0$) on a log-log scale. \n}\n\\label{fig:results_nn}\n\\end{center}\n\\end{figure}\n\n\nIn Fig.~\\ref{fig:results_nn}, we plot the OTOC $\\langle \\hat n(t)\\hat n(0), \\hat n(t)\\hat n(0)\\rangle$\nfor the single-orbital Hubbard model in a similar manner as in Fig.~\\ref{fig:results_fermion}.\nSince at half filling this correlation function approaches $\\langle \\hat n\\rangle^4=1$ at long times\nand at sufficiently low temperature,\nwe perform the analytical continuation procedure for $\\langle \\hat n(t)\\hat n(0), \\hat n(t)\\hat n(0)\\rangle-1$. \nThe top panels of Fig.~\\ref{fig:results_nn} show the real and imaginary parts of this shifted OTOC. \nContrary to the case of $\\langle c_\\sigma^\\dagger(t)c_\\sigma(0), c_\\sigma^\\dagger(t)c_\\sigma(0)\\rangle$, \nthe amplitude of the OTOC $\\langle \\hat n(t)\\hat n(0), \\hat n(t)\\hat n(0)\\rangle-1$ is suppressed as one increases the interaction, which reflects the reduced charge fluctuations in the Mott insulator.\nThe bottom left panel in Fig.~\\ref{fig:results_nn} shows the analytically continued spectral function $\\mathscr A_{(nn)^2}(\\omega)=-\\frac{1}{\\pi}{\\rm Im}\\, C_{(nn)^2}^R(\\omega)$ multiplied by $\\coth\\big(\\frac{\\beta\\omega}{4}\\big)$. \nAgain we notice the opening of a gap in the Mott phase, and a corresponding shift of the spectral weight to higher energies. This results in more rapid oscillations of the OTOC in the Mott state.\nThe bottom right panel in Fig.~\\ref{fig:results_nn} plots the modulus of the OTOC $|\\langle \\hat n(t)\\hat n(0), \\hat n(t)\\hat n(0)\\rangle-1|$ (normalized at $t=0$) on a log-log scale.\nThe long-time behavior of the OTOC is consistent with a power-law decay in the metallic phase, and an exponential decay in the Mott insulating phase.\n\n\n\\begin{figure}[t]\n\\begin{center}\n\\includegraphics[angle=-90, width=0.49\\columnwidth]{plot_otoc_nsns_b50_draft_Re.eps}\n\\includegraphics[angle=-90, width=0.49\\columnwidth]{plot_otoc_nsns_b50_draft_Im.eps}\n\\includegraphics[angle=-90, width=0.49\\columnwidth]{plot_coth_b50_nsns_inset.eps}\n\\includegraphics[angle=-90, width=0.49\\columnwidth]{plot_otoc_nsns_b50_draft_modulus_zoom.eps}\n\\caption{\nTop panels: Real and imaginary parts of the OTOC $\\langle \\hat n_\\sigma(t)\\hat n_\\sigma(0), \\hat n_\\sigma(t)\\hat n_\\sigma(0)\\rangle  - \\langle \\hat n_\\sigma\\rangle^4$ for the single-orbital Hubbard model with $\\beta=50$ at half filling. All the colored lines correspond to metallic solutions. \nBottom left panel: Spectral function $\\mathscr A_{(n_\\sigma n_\\sigma)^2}(\\omega)=-\\frac{1}{\\pi}{\\rm Im}\\,C_{(n_\\sigma n_\\sigma)^2}^R(\\omega)$ multiplied by $\\coth\\big(\\frac{\\beta\\omega}{4}\\big)$. In the inset, we plot the imaginary-time four-point function $\\langle \\hat n_\\sigma(\\tau)\\hat n_\\sigma(0), \\hat n_\\sigma(\\tau)\\hat n_\\sigma(0)\\rangle-\\langle \\hat n_\\sigma\\rangle^4$, with the black line corresponding to the Mott insulating solution at $U=6$.  \nBottom right panel: Modulus of the OTOC $|\\langle \\hat n_\\sigma(t)\\hat n_\\sigma(0), \\hat n_\\sigma(t)\\hat n_\\sigma(0)\\rangle  - \\langle \\hat n_\\sigma\\rangle^4|$ (normalized at $t=0$) on a log-log scale.\n}\n\\label{fig:results_nsns}\n\\end{center}\n\\end{figure}\n\nFigure~\\ref{fig:results_nsns} \nshows the results of the OTOC $\\langle \\hat n_\\sigma(t)\\hat n_\\sigma(0), \\hat n_\\sigma(t)\\hat n_\\sigma(0)\\rangle$\nfor the single-orbital Hubbard model. At half filling, this correlation function approaches $\\langle \\hat n_\\sigma\\rangle^4=\\frac{1}{16}$ in the long-time limit and at sufficiently low temperature, \nso that we perform the analytical continuation procedure for $\\langle \\hat n_\\sigma(t)\\hat n_\\sigma(0), \\hat n_\\sigma(t)\\hat n_\\sigma(0)\\rangle-\\langle \\hat n_\\sigma\\rangle^4$. \nThe top panels of Fig.~\\ref{fig:results_nsns} show the real and imaginary parts of \nthis shifted OTOC. \nHere, we only present results for the metallic phase, because resolving the very sharp low-energy feature of the OTOC spectrum in the half-filled Mott state is challenging. One can see that coherent oscillations are not observed for this type of OTOC, and that the incoherent part becomes dominant compared to the other two OTOCs shown in Figs.~\\ref{fig:results_fermion} and \\ref{fig:results_nn}. The amplitude of the OTOC $\\langle \\hat n_\\sigma(t)\\hat n_\\sigma(0), \\hat n_\\sigma(t)\\hat n_\\sigma(0)\\rangle-\\langle \\hat n_\\sigma\\rangle^4$ is enhanced as $U$ increases. As we argue below, this is related to the enhancement of spin fluctuations \nand local moment formation close to the Mott transition.\n\nThe bottom left panel in Fig.~\\ref{fig:results_nsns} plots the analytically continued spectral function $\\mathscr A_{(n_\\sigma n_\\sigma)^2}(\\omega)=-\\frac{1}{\\pi}{\\rm Im}\\, C_{(n_\\sigma n_\\sigma)^2}^R(\\omega)$ multiplied by $\\coth\\big(\\frac{\\beta\\omega}{4}\\big)$. \nWe can see that the spectral weight is concentrated in the low-energy region as we approach the Mott transition point.\nThe inset in the bottom left panel of Fig.~\\ref{fig:results_nsns} shows the imaginary-time four-point function $\\langle \\hat n_\\sigma(\\tau)\\hat n_\\sigma(0), \\hat n_\\sigma(\\tau)\\hat n_\\sigma(0)\\rangle-\\langle \\hat n_\\sigma\\rangle^4$, with the black line corresponding to a Mott insulating solution ($U=6$).  \nFor $U\\gtrsim 4$, the imaginary-time four-point function does not decay to zero but remains relatively large near $\\tau=\\frac{\\beta}{4}$. \nThis behavior is reminiscent of the spin-freezing physics seen in the correlated Hund metal phase of multi-orbital Hubbard models, where the dynamical spin correlation function $\\langle \\hat S_z(\\tau)\\hat S_z(0)\\rangle$ is trapped at a finite value at long $\\tau$ \\cite{Werner2008}. \nIn fact, the OTOC $\\langle \\hat n_\\sigma(\\tau)\\hat n_\\sigma(0), \\hat n_\\sigma(\\tau)\\hat n_\\sigma(0)\\rangle$\nmeasures spin correlations (in addition to charge correlations) through $\\hat n_\\sigma=\\frac{1}{2}\\hat n+\\sigma\\hat S_z$.\nPhysically, the trapping of $\\langle \\hat S_z(\\tau)\\hat S_z(0)\\rangle$ signals the formation of frozen local magnetic moments.\nIn the metal-insulator crossover region, the scattering induced by these magnetic moments results in incoherent metal states. Intuitively, one might expect fast scrambling in the corresponding regions of the phase diagram. \nAlthough there is some resemblance with the spin freezing crossover,\nthe self-energy does not exhibit a square-root frequency dependence in the single-orbital case, and a spin-freezing crossover in the sense of Ref.~\\cite{Werner2008} does not exist.  \n\nThe bottom right panel of Fig.~\\ref{fig:results_nsns} plots the modulus of the OTOC $|\\langle \\hat n_\\sigma(t)\\hat n_\\sigma(0), \\hat n_\\sigma(t)\\hat n_\\sigma(0)\\rangle-\\langle \\hat n_\\sigma\\rangle^4|$ (normalized at $t=0$) on a log-log scale. \nThe dynamics of $\\langle \\hat n_\\sigma(t)\\hat n_\\sigma(0), \\hat n_\\sigma(t)\\hat n_\\sigma(0)\\rangle$ is quite distinct from that of $\\langle \\hat n(t)\\hat n(0), \\hat n(t)\\hat n(0)\\rangle$. As the Mott transition is approached, we observe a slow-down in the decay of the modulus, \nwhich indicates the presence of slowly fluctuating local moments. \nThe long-time behavior may be consistent with a power-law, although slow oscillations make it difficult to determine the long-time asymptotic form from the numerics. \n\n\n\\begin{figure}[t]\n\\begin{center}\n\\includegraphics[angle=-90, width=0.49\\columnwidth]{plot_fitnsns_b2.eps}\n\\includegraphics[angle=-90, width=0.49\\columnwidth]{plot_coeffc_draft.eps}\n\\caption{\nLeft panel: \nThe imaginary-time four-point function $\\langle \\hat n_\\sigma(\\tau)\\hat n_\\sigma(0), \\hat n_\\sigma(\\tau)\\hat n_\\sigma(0)\\rangle$, the squared density-density correlation function $\\langle \\hat n_\\sigma(\\tau)\\hat n_\\sigma(0)\\rangle^2$,\nand their difference for the single-orbital Hubbard model with $U=5.5$ and $\\beta=50$.\nThe dashed curve shows a fit of the difference with the function $f_\\text{fit}(\\tau)=a+b\\tau+c\\tau^2$ in the interval $\\tau/\\beta \\in [0: 0.2]$.\nRight panel: Fitting coefficient $c$ as a function of $U$ for indicated values of $\\beta$.\n\n}\n\\label{fig:coeffs}\n\\end{center}\n\\end{figure}\n\nFor large enough $\\beta$ and small enough $\\tau$, the OTOC (\\ref{otoc_nsns}) factorizes into $\\langle \\hat n_\\sigma(\\tau)\\hat n_\\sigma(0), \\hat n_\\sigma(\\tau)\\hat n_\\sigma(0)\\rangle \\approx \\langle \\hat n_\\sigma(\\tau)\\hat n_\\sigma(0)\\rangle^2$. \nOn the real-time axis, this corresponds to the decoupling $\\langle \\hat n_\\sigma(t)\\hat n_\\sigma(0), \\hat n_\\sigma(t)\\hat n_\\sigma(0)\\rangle \\approx \\langle \\hat n_\\sigma(t)\\hat n_\\sigma(0)\\rangle^2$, that is, \nthe OTOC is reduced to a product of ordinary density-density correlation functions.\nIn order to see whether nontrivial correlations are captured beyond the decoupled form by the OTOC function, \nwe consider the difference $\\langle \\hat n_\\sigma(\\tau)\\hat n_\\sigma(0), \\hat n_\\sigma(\\tau)\\hat n_\\sigma(0)\\rangle-\\langle \\hat n_\\sigma(\\tau)\\hat n_\\sigma(0)\\rangle^2$ and fit the short-time behavior of this function to $f_\\text{fit}(\\tau)=a+b\\tau+c\\tau^2$, as illustrated in the left panel of Fig.~\\ref{fig:coeffs}. The coefficients $b$ and $c$ serve as  indicators of the nontrivial correlations that cannot be attributed to the decoupled form of the OTOC.\nThe coefficient $c$ is plotted as a function of $U$ and for different $\\beta$ in the right hand panel. The coefficient $b$ (not shown) exhibits a similar trend. We notice that the OTOC picks up nontrival correlations near the metal-insulator transition, and in particular at \nintermediate\ntemperatures, while the Mott phase shows no such correlations. The crossover region  \nassociated with the emergence of local moments \ncorresponds, roughly, to the interaction range where the nontrivial correlations start to become significant. \n\n\n\\subsection{Two-orbital Hubbard model}\n\\label{sec:two-orbital hubbard}\n\nIn this section, we investigate OTOCs in the two-orbital Hubbard model with Hund coupling $J>0$ \nand density-density interactions. (Three-orbital results are presented in Appendix~\\ref{sec:3orbital}.) \nThe Hamiltonian is given by\n\\begin{align}\nH\n&=\n-v\\sum_{\\langle i,j\\rangle\\alpha\\sigma} (c^\\dagger_{i\\alpha\\sigma}c_{j\\alpha\\sigma}+ \\text{h.c.})\n-\\mu\\sum_{i\\alpha\\sigma} \\hat n_{i\\alpha\\sigma}\n+U\\sum_{i\\alpha} \\hat n_{i\\alpha\\uparrow}\\hat n_{i\\alpha\\downarrow}\n+(U-2J)\\sum_{i\\sigma} \\hat n_{i1\\sigma}\\hat n_{i2\\bar\\sigma}\n\\notag\n\\\\\n&\\quad\n+(U-3J)\\sum_{i\\sigma} \\hat n_{i1\\sigma}\\hat n_{i2\\sigma},\n\\end{align}\n\nwith $U$ being the intra-orbital interaction $U$, $U-2J$ the inter-orbital antiparallel-spin interaction, and $U-3J$ the inter-orbital parallel-spin interaction. This is the simplest model that shows a crossover from a spin-frozen metal to a Fermi-liquid metal as one dopes the half-filled Mott insulator, with the self-energy scaling as $\\text{Im}\\Sigma(\\omega_n)\\sim\\sqrt{\\omega_n}$ over a significant energy range in the crossover regime \\cite{Hafermann2012}.\nThe sketch of the phase diagram is shown in Fig.~\\ref{fig:illustration}. \nAs we have seen in the previous section, the spin-related OTOC of the type (\\ref{otoc_nsns}) is relevant for the analysis of  the spin-freezing crossover, so that we concentrate on this OTOC here.\nIn the following calculations, we choose $U=8$, $J=U/4$, and compute $\\langle \\hat n_\\sigma(t)\\hat n_\\sigma(0), \\hat n_\\sigma(t)\\hat n_\\sigma(0)\\rangle - \\langle n_\\sigma \\rangle^4$ \nas a function of filling at $\\beta=50$ (dashed lines in Fig.~\\ref{fig:illustration}). \n\n\n\\begin{figure}[t]\n\\begin{center}\n\\includegraphics[angle=-90, width=0.49\\columnwidth]{plot_qp_draft.eps}\\hfill\n\\includegraphics[angle=-90, width=0.49\\columnwidth]{plot_fit_2orbital.eps}\n\\caption{\nLeft panel: Imaginary part of the self-energy as a function of Matsubara frequency for the two-orbital Hubbard model with $U=8$, $J=U/4$, and $\\beta=50$ on a log-log scale, with the dashed line indicating $\\sqrt{\\omega_n}$ behavior. \nRight panel: The coefficient $c$ extracted from\nfitting $\\langle \\hat n_\\sigma(\\tau)\\hat n_\\sigma(0), \\hat n_\\sigma(\\tau)\\hat n_\\sigma(0)\\rangle-\\langle \\hat n_\\sigma(\\tau)\\hat n_\\sigma(0)\\rangle^2$ with a function $f_\\text{fit}(\\tau)=a+b\\tau+c\\tau^2$\nin the interval $\\tau/\\beta\\in [0: 0.05]$ for the two-orbital Hubbard model. \nThe spin-freezing crossover occurs roughly at the filling $n_\\sigma\\approx 0.28$ (blue curve in the left panel and blue arrow in the right panel). \n}\n\\label{fig:self}\n\\end{center}\n\\end{figure}\n\n\nThe left panel of Fig.~\\ref{fig:self} plots the imaginary part of the self-energy as a function of Matsubara frequency in order to identify the filling corresponding to the spin-freezing crossover \\cite{Werner2008}. An approximate square-root scaling is observed over a wide range of frequencies near $n_\\sigma\\approx 0.28$ (blue line). In the right panel of Fig.~\\ref{fig:self}, we present the coefficient $c$ obtained from a similar analysis as presented in Fig.~\\ref{fig:coeffs}, but with a fitting range $\\tau/\\beta\\in [0 : 0.05]$. Again, we find that the OTOC $\\langle \\hat n_\\sigma(t)\\hat n_\\sigma(0), \\hat n_\\sigma(t)\\hat n_\\sigma(0)\\rangle - \\langle \\hat n_\\sigma\\rangle^4$ detects nontrivial correlations in the spin-freezing crossover (blue arrow) and spin-frozen metal regime, but not in the Fermi liquid or the Mott insulating phase.\n\n\n\\begin{figure}[t]\n\\begin{center}\n\\includegraphics[angle=-90, width=0.49\\columnwidth]{plot_otoc_2orbital_Re.eps}\n\\includegraphics[angle=-90, width=0.486\\columnwidth]{plot_otoc_2orbital_Im.eps}\n\\includegraphics[angle=-90, width=0.484\\columnwidth]{plot_spectra_coth_2orbital_inset.eps}\n\\includegraphics[angle=-90, width=0.486\\columnwidth]{plot_otoc_2orbital_modulus_inset.eps}\n\\caption{\nTop panels: Real and imaginary parts of the OTOC $\\langle \\hat n_\\sigma(t)\\hat n_\\sigma(0), \\hat n_\\sigma(t)\\hat n_\\sigma(0)\\rangle  - \\langle \\hat n_\\sigma\\rangle^4$ for the two-orbital Hubbard model with $U=8$, $J=U/4$, $\\beta=50$ and indicated fillings. \nBottom left panel: Spectral function $\\mathscr A_{(n_\\sigma n_\\sigma)^2}(\\omega)=-\\frac{1}{\\pi}{\\rm Im}\\,C_{(n_\\sigma n_\\sigma)^2}^R(\\omega)$ multiplied by $\\coth\\big(\\frac{\\beta\\omega}{4}\\big)$. In the inset, we plot the imaginary-time four-point function $\\langle \\hat n_\\sigma(\\tau)\\hat n_\\sigma(0), \\hat n_\\sigma(\\tau)\\hat n_\\sigma(0)\\rangle-\\langle \\hat n_\\sigma\\rangle^4$. \nBottom right panel: Modulus of the OTOC $|\\langle \\hat n_\\sigma(t)\\hat n_\\sigma(0), \\hat n_\\sigma(t)\\hat n_\\sigma(0)\\rangle  - \\langle \\hat n_\\sigma\\rangle^4|$ (normalized at $t=0$) on a log-log scale.\nIn the inset of the bottom right panel, we illustrate the approximate data collapse obtained by considering the spin-freezing crossover regimes for different values of $J$. \nIn all the panels, the thick blue curves represent the results for the spin-freezing crossover region. \n}\n\\label{fig:2orbital}\n\\end{center}\n\\end{figure}\n\n\nThe top panels of Fig.~\\ref{fig:2orbital} show the real and imaginary parts of the OTOC\n$\\langle \\hat n_\\sigma(t)\\hat n_\\sigma(0), \\hat n_\\sigma(t)\\hat n_\\sigma(0)\\rangle-\\langle \\hat n_\\sigma\\rangle^4$\nfor the two-orbital Hubbard model. In the spin-frozen phase ($n_\\sigma \\gtrsim 0.3$), both components exhibit \nhighly incoherent oscillations, which get suppressed as the filling is reduced. As one moves across the crossover point ($n_\\sigma \\sim 0.28$),\nthe oscillations completely disappear, and the real part of the OTOC starts to overshoot to the negative side\nin the Fermi liquid regime.\nThe bottom left panel in Fig.~\\ref{fig:2orbital} presents the analytically continued spectral function $\\mathscr A_{(n_\\sigma n_\\sigma)^2}(\\omega)=-\\frac{1}{\\pi}{\\rm Im}\\,C_{(n_\\sigma n_\\sigma)^2}^R(\\omega)$ multiplied by $\\coth\\big(\\frac{\\beta\\omega}{4}\\big)$. \nIn the spin-frozen regime ($n_\\sigma \\gtrsim 0.3$) a sharp peak appears at low energy, which originates from the saturation of the imaginary-time four point function $\\langle \\hat n_\\sigma(\\tau)\\hat n_\\sigma(0), \\hat n_\\sigma(\\tau)\\hat n_\\sigma(0)\\rangle$ at large $\\tau$ (see the inset of the bottom left panel of Fig.~\\ref{fig:2orbital}). This low-energy peak represents the slow dynamics of the frozen spins, and translates into a slow long-time decay of $|\\langle \\hat n_\\sigma(t)\\hat n_\\sigma(0), \\hat n_\\sigma(t)\\hat n_\\sigma(0)\\rangle-\\langle \\hat n_\\sigma\\rangle^4|$,\nas shown in the bottom right panel of Fig.~\\ref{fig:2orbital}\non a log-log scale. Again, it is difficult to clearly resolve the long-time behavior of the OTOC due to the limited\naccuracy of the analytic continuation, but in the spin-freezing crossover regime (thick blue curves)\nthe decay of the OTOC is consistent with a power law $\\sim 1/t^{1.5}$ at least for $2\\lesssim t \\lesssim 5$. \nThe shorter-time behavior ($0.5\\lesssim t \\lesssim 2$) is instead well fitted by an exponential decay $\\sim e^{-\\alpha t}$ with a decay constant $\\alpha=0.78$. \nThis exponent exhibits\na strong dependence on the Hund coupling. \n\nIn fact, the OTOCs in the spin-freezing regime for $J=U/4$ and $J=U/6$ can be approximately collapsed by plotting them as a function of $tJ$ (see the inset in the bottom right panel of Fig.~\\ref{fig:2orbital}), which indicates that the Hund coupling is the parameter which controls the dynamics in this regime. \nThis is distinct from the Lyapunov behavior in which the exponential decay scales with $t/\\beta$ \\cite{Bagrets2017}.\nA possible reason is that we are still far from the large $N$ limit (where $N$ corresponds to the number of orbitals times the number of spin degrees of freedom) so that the large $N$ behavior \nsuch as the Lyapunov growth is not observed in our calculations. \nIn fact, the strong dependence of the time scale on $J$ and weak dependence on $\\beta$ is consistent\nwith the behavior of the finite $N$ SYK model in Ref.~\\cite{FuSachdev2016}.\nIn the spin-frozen regime ($n_\\sigma\\gtrsim 0.3$) the decay is much slower than near the crossover point, while in the Fermi liquid regime the decay is accelerated, approaching the free fermion behavior of $t^{-3}$\nas shown in Fig.~\\ref{analytic continuation}. \nQualitatively similar results for this OTOC are obtained \nfor the three-orbital Hubbard model (see Appendix~\\ref{sec:3orbital}).\n\nThe time dependence of the modulus of \n$\\langle \\hat n_\\sigma(\\tau)\\hat n_\\sigma(0), \\hat n_\\sigma(\\tau)\\hat n_\\sigma(0)\\rangle-\\langle \\hat n_\\sigma\\rangle^4$ \nin the spin-freezing crossover regime of multi-orbital Hubbard models (exponential decay crossing over into a power-law) bears a close resemblance to a (different) OTOC for the SYK model discussed in Ref.~\\cite{Bagrets2017}. \nAlso, a recent study of yet another type of OTOCs for the finite-$N$ SYK model found a qualitatively similar decay as observed here in the spin-freezing crossover regime \\cite{FuSachdev2016}. In the following section, we will make the connection to the SYK dynamics more quantitative.   \n\n\n\\subsection{Comparison to the SKY model}\n\\label{sec:SYK}\n\nIn this section, we compare the results of the multi-orbital Hubbard models to the SYK model.\nWe consider the particle-hole symmetric SYK model of complex fermions \\cite{FuSachdev2016},\nwhose Hamiltonian is given by\n\\begin{align}\nH\n&=\n\\frac{1}{(2N)^{3/2}} \\sum_{i,j,k,l=1}^N \nJ_{ij;kl} (c_i^\\dagger c_j^\\dagger c_k c_l\n+\\delta_{ik}n c_j^\\dagger c_l\n-\\delta_{il}n c_j^\\dagger c_k\n-\\delta_{jk}n c_i^\\dagger c_l\n+\\delta_{jl}n c_i^\\dagger c_k).\n\\end{align}\nThe coupling constant $J_{ij;kl}$ is a gaussian random variable, satisfying $J_{ij;kl}=-J_{ji;kl}=-J_{ij;lk}=J_{kl;ij}^\\ast$, and\n\\begin{align}\n\\overline{({\\rm Re}\\, J_{ij;kl})^2}\n&=\n\\begin{cases}\nJ_{\\rm SYK}^2/2 & (i,j)\\neq (k,l) \\\\\nJ_{\\rm SYK}^2 & (i,j)=(k,l)\n\\end{cases},\n\\\\\n\\overline{({\\rm Im}\\, J_{ij;kl})^2}\n&=\n\\begin{cases}\nJ_{\\rm SYK}^2/2 & (i,j)\\neq (k,l) \\\\\n0 & (i,j)=(k,l)\n\\end{cases},\n\\end{align}\nwhere the overline represents an average over each realization of $J_{ij;kl}$. The parameter \n$J_{\\rm SYK}$ ($J_{\\rm SYK}^{-1}$) defines the unit of energy (time) in the following calculations. \nThe system is particle-hole symmetric when\n$n=0.5$, which fixes the total number of fermions as $N^{-1}\\sum_{i=1}^N \\langle c_i^\\dagger c_i\\rangle=0.5$.\nWe take the particle-hole symmetric form of the SYK model because it has been well studied previously \\cite{FuSachdev2016} and avoids the effect of the filling drift.\nWe numerically solve the model by exact diagonalization for finite $N(\\le 12)$, and evaluate the density-density OTOC\n$\\overline{\\langle \\hat n_i(t)\\hat n_i(0), \\hat n_i(t)\\hat n_i(0)\\rangle-\\langle \\hat n_i\\rangle^4}$\nand its imaginary-time counterpart.\n\n\\begin{figure}[t]\n\\begin{center}\n\n\\includegraphics[angle=-90,width=0.49\\columnwidth]{plot_fitnsns_syk.eps}\n\n\\includegraphics[angle=-90,width=0.49\\columnwidth]{plot_coeffc_syk.eps}\n\\caption{\nLeft panel: \nThe imaginary-time four-point function $\\overline{\\langle \\hat n_i(\\tau)\\hat n_i(0), \\hat n_i(\\tau)\\hat n_i(0)\\rangle}$, the density-density correlation function $\\overline{\\langle \\hat n_i(\\tau)\\hat n_i(0)\\rangle^2}$,\nand their difference for the SYK model with $N=12$ and $\\beta J_{\\rm SYK}=10$.\nThe dashed curve shows a fit of the difference with a function $f_\\text{fit}(\\tau)=a+b\\tau+c\\tau^2$ in the interval $\\tau/\\beta \\in [0: 0.2]$. \nRight panel: Fitting coefficient $c$ as a function of $T/J_{\\rm SYK}$.\n}\n\\label{fig:SYK fit}\n\\end{center}\n\\end{figure}\n\nIn the left panel of Fig.~\\ref{fig:SYK fit}, we plot the imaginary-time four-point function \n$\\overline{\\langle \\hat n_i(\\tau)\\hat n_i(0), \\hat n_i(\\tau)\\hat n_i(0)\\rangle}$\nfor the SYK model with $N=12$ and $\\beta J_{\\rm SYK}=10$. \nAt low temperature, we expect that $\\overline{\\langle \\hat n_i(\\tau)\\hat n_i(0), \\hat n_i(\\tau)\\hat n_i(0)\\rangle}$ is decoupled into $\\overline{\\langle \\hat n_i(\\tau)\\hat n_i(0)\\rangle^2}$.\nTo see whether nontrivial correlations beyond the decoupled form are present, we fit the difference\n$\\overline{\\langle \\hat n_i(\\tau)\\hat n_i(0), \\hat n_i(\\tau)\\hat n_i(0)\\rangle}\n-\\overline{\\langle \\hat n_i(\\tau)\\hat n_i(0)\\rangle^2}$ with a function $f_{\\rm fit}(\\tau)=a+b\\tau+c\\tau^2$\nin the interval $\\tau/\\beta\\in[0:0.2]$, in the same manner as for the Hubbard models.\nThe obtained coefficient $c$ is plotted as a function of the temperature $T$ in the right panel of\nFig.~\\ref{fig:SYK fit}. One can see that nontrivial correlations exist in a wide range of temperature.\nEspecially, they are enhanced in the intermediate temperature regime. This is because in the zero-temperature limit\nthe four-point function is decoupled as \n$\\langle \\hat n_i(\\tau)\\hat n_i(0), \\hat n_i(\\tau)\\hat n_i(0)\\rangle \\approx \\langle \\hat n_i(\\tau)\\hat n_i(0)\\rangle^2$, while in the high-temperature limit ($\\beta\\to 0$) the imaginary-time dependence is washed out.\n\n\n\n\n\\begin{figure}[t]\n\\begin{center}\n\\includegraphics[angle=-90, width=0.49\\columnwidth]{plot_syk_Re.eps}\n\\includegraphics[angle=-90, width=0.49\\columnwidth]{plot_syk_Im.eps}\n\\includegraphics[angle=-90, width=0.49\\columnwidth]{plot_syk_modulus-log.eps}\n\\includegraphics[angle=-90, width=0.49\\columnwidth]{plot_syk_modulus.eps}\n\\caption{\nThe density-density OTOC $\\overline{\\langle \\hat n_i(t)\\hat n_i(0), \\hat n_i(t)\\hat n_i(0)\\rangle-\\langle \\hat n_i\\rangle^4}$ for the particle-hole symmetric SYK model averaged over $10^2$ samples with $\\beta J_{\\rm SYK}=100$ and various $N$. \nThe top left and right panels show the real and imaginary parts of the OTOC,\nwhile the bottom left and right panels show the log and log-log plots of the modulus of the OTOC (normalized at $t=0$).\nThe dashed line shows a power-law decay $\\propto t^{-1.6}$ for comparison.}\n\\label{fig:SYK}\n\\end{center}\n\\end{figure}\n\nIn Fig.~\\ref{fig:SYK}, we show the numerical results of the density-density OTOC for $\\beta J_{\\rm SYK}=100$.\nThe real part of the OTOC rapidly drops from the initial value within the time scale of $tJ_{\\rm SYK}\\sim 2$,\nand slowly decays to zero in the long time limit.\nAs one increases $N$, the oscillations that appear at longer time tend to be suppressed.\nOne can see that the result for $N=12$ in the top panels of Fig.~\\ref{fig:SYK} closely resembles the OTOC in \nthe spin-freezing crossover regime of the two-orbital Hubbard model shown\nby the blue curve in the top panels of Fig.~\\ref{fig:2orbital},\nwhile it differs from the single-orbital result shown in the top panels of Fig.~\\ref{fig:results_nsns}.\nDue to the limitation of our calculations to small system sizes ($N\\le 12$), we do not clearly observe\nan exponential growth ($\\sim c_0-c_1 e^{\\lambda t}$)  at short times or an exponential decay ($\\sim e^{-\\alpha t}$) \nat intermediate times. \nHowever, we find that for $tJ_{\\rm SYK}\\gtrsim 2$ the OTOC decays\napproximately in a power law $\\propto t^{-\\gamma}$ with $\\gamma=1.6$ (see bottom right panel in Fig.~\\ref{fig:SYK}). \nThe time interval exhibiting the power-law decay becomes longer as we increase $N$.  \nThe temperature dependence of the OTOC for the SYK model is shown in Fig.~\\ref{fig:SYK-T}.\nThe time scale of the initial drop is more or less independent of the temperature, while \nthe power-law-like decay is only visible in the low-temperature regime.\n\n\\begin{figure}[t]\n\\begin{center}\n\\includegraphics[angle=-90, width=0.49\\columnwidth]{plot_syk_beta_re.eps}\n\\includegraphics[angle=-90, width=0.49\\columnwidth]{plot_syk_beta_im.eps}\n\\includegraphics[angle=-90, width=0.49\\columnwidth]{plot_syk_beta_modulus-log.eps}\n\\includegraphics[angle=-90, width=0.49\\columnwidth]{plot_syk_beta_modulus.eps}\n\\caption{\nTemperature dependence of the density-density OTOC\n$\\overline{\\langle \\hat n_i(t)\\hat n_i(0), \\hat n_i(t)\\hat n_i(0)\\rangle-\\langle \\hat n_i\\rangle^4}$ for the particle-hole symmetric SYK model averaged over $10^2$ samples with $N=12$.\nThe top left and right panels show the real and imaginary parts of the OTOC,\nwhile the bottom left and right panels show the log and log-log plots of the modulus of the OTOC.\nThe dashed line shows a power-law decay $\\propto t^{-1.6}$ for comparison.}\n\\label{fig:SYK-T}\n\\end{center}\n\\end{figure}\n\nIn Ref.~\\onlinecite{Werner2018}, it has been argued that the spin-freezing crossover regime of multi-orbital Hubbard models is effectively described by the SYK model in which $J_{\\rm SYK}$ is replaced by the Hund coupling $J$. The results in Fig.~\\ref{fig:2orbital} are for the two-orbital Hubbard model with $U=8$, $J/U=1/4$ and $\\beta=50$, corresponding to $\\beta J=100$, which means that it is meaningful to directly compare the blue lines in Fig.~\\ref{fig:2orbital} with the blue lines in Figs.~\\ref{fig:SYK} and \\ref{fig:SYK-T}. Not only is the qualitative agreement remarkable, even the power-law exponent measured for the two-orbital Hubbard model ($\\gamma=1.5$) is very close to the exponent extracted from the finite-$N$ SYK calculation. \nThe power-law decay in the three-orbital case is approximately $1/t^{1.75}$ (see Appendix~\\ref{sec:3orbital}), and thus also in semi-quantitative agreement with the SYK model behavior. \nThis nontrivial result provides further support for the identification of the spin-freezing crossover regime of multi-orbital Hubbard systems with an SYK strange metal. \n\nIn the large-$N$ and low-$T$ limit of the SYK model, we expect that the power-law behavior approaches $1/t^2$, since in this limit the OTOC is decoupled as $\\langle \\hat n_i(t)\\hat n_i(0), \\hat n_i(t)\\hat n_i(0)\\rangle\\approx \\langle \\hat n_i(t)\\hat n_i(0)\\rangle^2$ and the decay of $\\langle \\hat n_i(t)\\hat n_i(0)\\rangle$ is dominated by what corresponds to the slow spin relaxation $\\langle \\hat S_z(t)\\hat S_z(0)\\rangle\\sim 1/t$ in the Sachdev-Ye model.\n\n\n\n\n\n4.3 Two-orbital Hubbard model\n\\subsection{Two-orbital Hubbard model}\n\\label{sec:two-orbital hubbard}\n\nIn this section, we investigate OTOCs in the two-orbital Hubbard model with Hund coupling $J>0$ \nand density-density interactions. (Three-orbital results are presented in Appendix~\\ref{sec:3orbital}.) \nThe Hamiltonian is given by\n\\begin{align}\nH\n&=\n-v\\sum_{\\langle i,j\\rangle\\alpha\\sigma} (c^\\dagger_{i\\alpha\\sigma}c_{j\\alpha\\sigma}+ \\text{h.c.})\n-\\mu\\sum_{i\\alpha\\sigma} \\hat n_{i\\alpha\\sigma}\n+U\\sum_{i\\alpha} \\hat n_{i\\alpha\\uparrow}\\hat n_{i\\alpha\\downarrow}\n+(U-2J)\\sum_{i\\sigma} \\hat n_{i1\\sigma}\\hat n_{i2\\bar\\sigma}\n\\notag\n\\\\\n&\\quad\n+(U-3J)\\sum_{i\\sigma} \\hat n_{i1\\sigma}\\hat n_{i2\\sigma},\n\\end{align}\n\nwith $U$ being the intra-orbital interaction $U$, $U-2J$ the inter-orbital antiparallel-spin interaction, and $U-3J$ the inter-orbital parallel-spin interaction. This is the simplest model that shows a crossover from a spin-frozen metal to a Fermi-liquid metal as one dopes the half-filled Mott insulator, with the self-energy scaling as $\\text{Im}\\Sigma(\\omega_n)\\sim\\sqrt{\\omega_n}$ over a significant energy range in the crossover regime \\cite{Hafermann2012}.\nThe sketch of the phase diagram is shown in Fig.~\\ref{fig:illustration}. \nAs we have seen in the previous section, the spin-related OTOC of the type (\\ref{otoc_nsns}) is relevant for the analysis of  the spin-freezing crossover, so that we concentrate on this OTOC here.\nIn the following calculations, we choose $U=8$, $J=U/4$, and compute $\\langle \\hat n_\\sigma(t)\\hat n_\\sigma(0), \\hat n_\\sigma(t)\\hat n_\\sigma(0)\\rangle - \\langle n_\\sigma \\rangle^4$ \nas a function of filling at $\\beta=50$ (dashed lines in Fig.~\\ref{fig:illustration}). \n\n\n\\begin{figure}[t]\n\\begin{center}\n\\includegraphics[angle=-90, width=0.49\\columnwidth]{plot_qp_draft.eps}\\hfill\n\\includegraphics[angle=-90, width=0.49\\columnwidth]{plot_fit_2orbital.eps}\n\\caption{\nLeft panel: Imaginary part of the self-energy as a function of Matsubara frequency for the two-orbital Hubbard model with $U=8$, $J=U/4$, and $\\beta=50$ on a log-log scale, with the dashed line indicating $\\sqrt{\\omega_n}$ behavior. \nRight panel: The coefficient $c$ extracted from\nfitting $\\langle \\hat n_\\sigma(\\tau)\\hat n_\\sigma(0), \\hat n_\\sigma(\\tau)\\hat n_\\sigma(0)\\rangle-\\langle \\hat n_\\sigma(\\tau)\\hat n_\\sigma(0)\\rangle^2$ with a function $f_\\text{fit}(\\tau)=a+b\\tau+c\\tau^2$\nin the interval $\\tau/\\beta\\in [0: 0.05]$ for the two-orbital Hubbard model. \nThe spin-freezing crossover occurs roughly at the filling $n_\\sigma\\approx 0.28$ (blue curve in the left panel and blue arrow in the right panel). \n}\n\\label{fig:self}\n\\end{center}\n\\end{figure}\n\n\nThe left panel of Fig.~\\ref{fig:self} plots the imaginary part of the self-energy as a function of Matsubara frequency in order to identify the filling corresponding to the spin-freezing crossover \\cite{Werner2008}. An approximate square-root scaling is observed over a wide range of frequencies near $n_\\sigma\\approx 0.28$ (blue line). In the right panel of Fig.~\\ref{fig:self}, we present the coefficient $c$ obtained from a similar analysis as presented in Fig.~\\ref{fig:coeffs}, but with a fitting range $\\tau/\\beta\\in [0 : 0.05]$. Again, we find that the OTOC $\\langle \\hat n_\\sigma(t)\\hat n_\\sigma(0), \\hat n_\\sigma(t)\\hat n_\\sigma(0)\\rangle - \\langle \\hat n_\\sigma\\rangle^4$ detects nontrivial correlations in the spin-freezing crossover (blue arrow) and spin-frozen metal regime, but not in the Fermi liquid or the Mott insulating phase.\n\n\n\\begin{figure}[t]\n\\begin{center}\n\\includegraphics[angle=-90, width=0.49\\columnwidth]{plot_otoc_2orbital_Re.eps}\n\\includegraphics[angle=-90, width=0.486\\columnwidth]{plot_otoc_2orbital_Im.eps}\n\\includegraphics[angle=-90, width=0.484\\columnwidth]{plot_spectra_coth_2orbital_inset.eps}\n\\includegraphics[angle=-90, width=0.486\\columnwidth]{plot_otoc_2orbital_modulus_inset.eps}\n\\caption{\nTop panels: Real and imaginary parts of the OTOC $\\langle \\hat n_\\sigma(t)\\hat n_\\sigma(0), \\hat n_\\sigma(t)\\hat n_\\sigma(0)\\rangle  - \\langle \\hat n_\\sigma\\rangle^4$ for the two-orbital Hubbard model with $U=8$, $J=U/4$, $\\beta=50$ and indicated fillings. \nBottom left panel: Spectral function $\\mathscr A_{(n_\\sigma n_\\sigma)^2}(\\omega)=-\\frac{1}{\\pi}{\\rm Im}\\,C_{(n_\\sigma n_\\sigma)^2}^R(\\omega)$ multiplied by $\\coth\\big(\\frac{\\beta\\omega}{4}\\big)$. In the inset, we plot the imaginary-time four-point function $\\langle \\hat n_\\sigma(\\tau)\\hat n_\\sigma(0), \\hat n_\\sigma(\\tau)\\hat n_\\sigma(0)\\rangle-\\langle \\hat n_\\sigma\\rangle^4$. \nBottom right panel: Modulus of the OTOC $|\\langle \\hat n_\\sigma(t)\\hat n_\\sigma(0), \\hat n_\\sigma(t)\\hat n_\\sigma(0)\\rangle  - \\langle \\hat n_\\sigma\\rangle^4|$ (normalized at $t=0$) on a log-log scale.\nIn the inset of the bottom right panel, we illustrate the approximate data collapse obtained by considering the spin-freezing crossover regimes for different values of $J$. \nIn all the panels, the thick blue curves represent the results for the spin-freezing crossover region. \n}\n\\label{fig:2orbital}\n\\end{center}\n\\end{figure}\n\n\nThe top panels of Fig.~\\ref{fig:2orbital} show the real and imaginary parts of the OTOC\n$\\langle \\hat n_\\sigma(t)\\hat n_\\sigma(0), \\hat n_\\sigma(t)\\hat n_\\sigma(0)\\rangle-\\langle \\hat n_\\sigma\\rangle^4$\nfor the two-orbital Hubbard model. In the spin-frozen phase ($n_\\sigma \\gtrsim 0.3$), both components exhibit \nhighly incoherent oscillations, which get suppressed as the filling is reduced. As one moves across the crossover point ($n_\\sigma \\sim 0.28$),\nthe oscillations completely disappear, and the real part of the OTOC starts to overshoot to the negative side\nin the Fermi liquid regime.\nThe bottom left panel in Fig.~\\ref{fig:2orbital} presents the analytically continued spectral function $\\mathscr A_{(n_\\sigma n_\\sigma)^2}(\\omega)=-\\frac{1}{\\pi}{\\rm Im}\\,C_{(n_\\sigma n_\\sigma)^2}^R(\\omega)$ multiplied by $\\coth\\big(\\frac{\\beta\\omega}{4}\\big)$. \nIn the spin-frozen regime ($n_\\sigma \\gtrsim 0.3$) a sharp peak appears at low energy, which originates from the saturation of the imaginary-time four point function $\\langle \\hat n_\\sigma(\\tau)\\hat n_\\sigma(0), \\hat n_\\sigma(\\tau)\\hat n_\\sigma(0)\\rangle$ at large $\\tau$ (see the inset of the bottom left panel of Fig.~\\ref{fig:2orbital}). This low-energy peak represents the slow dynamics of the frozen spins, and translates into a slow long-time decay of $|\\langle \\hat n_\\sigma(t)\\hat n_\\sigma(0), \\hat n_\\sigma(t)\\hat n_\\sigma(0)\\rangle-\\langle \\hat n_\\sigma\\rangle^4|$,\nas shown in the bottom right panel of Fig.~\\ref{fig:2orbital}\non a log-log scale. Again, it is difficult to clearly resolve the long-time behavior of the OTOC due to the limited\naccuracy of the analytic continuation, but in the spin-freezing crossover regime (thick blue curves)\nthe decay of the OTOC is consistent with a power law $\\sim 1/t^{1.5}$ at least for $2\\lesssim t \\lesssim 5$. \nThe shorter-time behavior ($0.5\\lesssim t \\lesssim 2$) is instead well fitted by an exponential decay $\\sim e^{-\\alpha t}$ with a decay constant $\\alpha=0.78$. \nThis exponent exhibits\na strong dependence on the Hund coupling. \n\nIn fact, the OTOCs in the spin-freezing regime for $J=U/4$ and $J=U/6$ can be approximately collapsed by plotting them as a function of $tJ$ (see the inset in the bottom right panel of Fig.~\\ref{fig:2orbital}), which indicates that the Hund coupling is the parameter which controls the dynamics in this regime. \nThis is distinct from the Lyapunov behavior in which the exponential decay scales with $t/\\beta$ \\cite{Bagrets2017}.\nA possible reason is that we are still far from the large $N$ limit (where $N$ corresponds to the number of orbitals times the number of spin degrees of freedom) so that the large $N$ behavior \nsuch as the Lyapunov growth is not observed in our calculations. \nIn fact, the strong dependence of the time scale on $J$ and weak dependence on $\\beta$ is consistent\nwith the behavior of the finite $N$ SYK model in Ref.~\\cite{FuSachdev2016}.\nIn the spin-frozen regime ($n_\\sigma\\gtrsim 0.3$) the decay is much slower than near the crossover point, while in the Fermi liquid regime the decay is accelerated, approaching the free fermion behavior of $t^{-3}$\nas shown in Fig.~\\ref{analytic continuation}. \nQualitatively similar results for this OTOC are obtained \nfor the three-orbital Hubbard model (see Appendix~\\ref{sec:3orbital}).\n\nThe time dependence of the modulus of \n$\\langle \\hat n_\\sigma(\\tau)\\hat n_\\sigma(0), \\hat n_\\sigma(\\tau)\\hat n_\\sigma(0)\\rangle-\\langle \\hat n_\\sigma\\rangle^4$ \nin the spin-freezing crossover regime of multi-orbital Hubbard models (exponential decay crossing over into a power-law) bears a close resemblance to a (different) OTOC for the SYK model discussed in Ref.~\\cite{Bagrets2017}. \nAlso, a recent study of yet another type of OTOCs for the finite-$N$ SYK model found a qualitatively similar decay as observed here in the spin-freezing crossover regime \\cite{FuSachdev2016}. In the following section, we will make the connection to the SYK dynamics more quantitative.   \n\n\n",
        "label": "fig:self",
        "Descriptive_question1": "What is the approximate filling value at which the spin-freezing crossover occurs in figure_9?",
        "Descriptive_question2": "What color is used to highlight the spin-freezing crossover in the left panel of figure_9?",
        "Reasoning_question1": "How does the behavior of the self-energy in the left panel of figure_9 indicate the presence of a spin-freezing crossover at a specific filling value?",
        "Reasoning_question2": "What does the trend in the coefficient c in the right panel of figure_9 suggest about the nature of correlations in the two-orbital Hubbard model across different filling values?",
        "Descriptive_answer1": "0.28",
        "Descriptive_answer2": "blue",
        "Reasoning_answer1": "The left panel of figure_9 shows the imaginary part of the self-energy plotted against Matsubara frequency on a log-log scale. At the filling around 0.28, the self-energy exhibits an approximate square-root dependence on frequency, indicated by the dashed line. This sqrt(omega_n) behavior is characteristic of the spin-freezing crossover regime, signaling non-Fermi liquid behavior associated with slow spin dynamics and local moment formation.",
        "Reasoning_answer2": "The right panel shows the coefficient c extracted from fitting the difference between the four-point correlation function and the squared two-point correlation function. The coefficient c peaks near the filling 0.28, corresponding to the spin-freezing crossover, and decreases away from that point. This trend suggests that nontrivial correlations beyond simple factorization become significant near the crossover, indicating enhanced local moment fluctuations and spin freezing, while correlations weaken in both the Fermi liquid and Mott insulating regimes at other fillings."
    },
    {
        "paper_id": "1812.04217.json",
        "image_id": "figure_10",
        "image_path": [
            "/home/yz979/scratch/chengye/scimolmo/test-images-tables/dataset_imagestables/1812.04217/images/plot_otoc_2orbital_Re.eps"
        ],
        "caption": "\nTop panels: Real and imaginary parts of the OTOC $\\langle \\hat n_\\sigma(t)\\hat n_\\sigma(0), \\hat n_\\sigma(t)\\hat n_\\sigma(0)\\rangle  - \\langle \\hat n_\\sigma\\rangle^4$ for the two-orbital Hubbard model with $U=8$, $J=U/4$, $\\beta=50$ and indicated fillings. \nBottom left panel: Spectral function $\\mathscr A_{(n_\\sigma n_\\sigma)^2}(\\omega)=-\\frac{1}{\\pi}{\\rm Im}\\,C_{(n_\\sigma n_\\sigma)^2}^R(\\omega)$ multiplied by $\\coth\\big(\\frac{\\beta\\omega}{4}\\big)$. In the inset, we plot the imaginary-time four-point function $\\langle \\hat n_\\sigma(\\tau)\\hat n_\\sigma(0), \\hat n_\\sigma(\\tau)\\hat n_\\sigma(0)\\rangle-\\langle \\hat n_\\sigma\\rangle^4$. \nBottom right panel: Modulus of the OTOC $|\\langle \\hat n_\\sigma(t)\\hat n_\\sigma(0), \\hat n_\\sigma(t)\\hat n_\\sigma(0)\\rangle  - \\langle \\hat n_\\sigma\\rangle^4|$ (normalized at $t=0$) on a log-log scale.\nIn the inset of the bottom right panel, we illustrate the approximate data collapse obtained by considering the spin-freezing crossover regimes for different values of $J$. \nIn all the panels, the thick blue curves represent the results for the spin-freezing crossover region. \n",
        "classify": "Chart",
        "section_info": "4 Numerical results for the Hubbard model\n\\section{Numerical results for the Hubbard model}\n\\label{numerical results}\n\n\\subsection{Observables and numerical procedure}\n\nWe consider the single-orbital, two-orbital, and three-orbital Hubbard models on an infinitely-connected Bethe lattice, which can be solved exactly within DMFT \\cite{GeorgesKotliarKrauthRozenberg1996}. In this case, the noninteracting density of states is semicircular with bandwidth $4v_\\ast$, for which there exists a simplified DMFT self-consistency condition between the hybridization function $\\Delta_{\\alpha\\sigma}$ of the DMFT impurity problem and the local (impurity) Green's function $G_{\\alpha\\sigma}$: \n\n\\begin{equation}\n\\Delta_{\\alpha\\sigma}(\\tau)=v_\\ast^2 G_{\\alpha\\sigma}(\\tau), \n\\label{semicirc}\n\\end{equation}\n\nwith $\\alpha$ the orbital and $\\sigma$ the spin index. We will use $v_\\ast$ as the unit of energy and measure time in units of $\\hbar/v_\\ast$. \n\nThree different types of imaginary-time four-point functions $C_{(AB)^2}^M(\\tau)$\nof the form of Eq.~(\\ref{imaginary-time OTOC})\nwith $(\\hat A, \\hat B)=(c_\\sigma^\\dagger, c_\\sigma), (\\hat n_\\sigma, \\hat n_\\sigma)$, and $(\\hat n, \\hat n)$\nare calculated in the interval $0\\le \\tau\\le \\frac{\\beta\\hbar}{2}$: \n\\begin{eqnarray}\nC_{(c^\\dagger_\\sigma c_\\sigma)^2}^M(\\tau) &=& -\\langle c^\\dagger_\\sigma(\\tau+\\tfrac{\\beta\\hbar}{2}) c_\\sigma(\\tfrac{\\beta\\hbar}{2}) c^\\dagger_\\sigma(\\tau) c_\\sigma(0) \\rangle, \\label{fermion_otoc}\\\\\nC_{(n_\\sigma n_\\sigma)^2}^M(\\tau) &=& -\\langle \\hat n_\\sigma(\\tau+\\tfrac{\\beta\\hbar}{2})\\hat n_\\sigma(\\tfrac{\\beta\\hbar}{2})\\hat n_\\sigma(\\tau)\\hat n_\\sigma(0) \\rangle, \\label{otoc_nsns}\\\\\nC_{(nn)^2}^M(\\tau) &=& -\\langle \\hat n(\\tau+\\tfrac{\\beta\\hbar}{2})\\hat n(\\tfrac{\\beta\\hbar}{2})\\hat n(\\tau)\\hat n(0) \\rangle,\\label{otoc_nn}\n\\end{eqnarray}\n\nwhere $c_\\sigma^\\dagger$ ($c_\\sigma$) are the fermionic creation (annihilation) operators for spin $\\sigma$ (and orbital $\\alpha=1$ in the multi-orbital case), $\\hat n_\\sigma=c^\\dagger_\\sigma c_\\sigma$ is the corresponding spin-dependent density operator, and $\\hat n=\\hat n_\\uparrow + \\hat n_\\downarrow$ the total density operator. Note that in all the three cases above we have $\\hat B=\\hat A^\\dagger$. \nWe measure these local correlation functions in the impurity model using a hybridization expansion continuous-time Monte Carlo algorithm (CT-HYB) \\cite{Werner2006}. In this algorithm, the \ntwo-particle\nGreen's functions of the type (\\ref{fermion_otoc}) can be measured by removing two hybridization lines, i.e., from the elements of the inverse hybridization matrix, while the density-density correlation functions can be easily measured either by insertion of density operators (matrix formalism) \\cite{Werner2006Kondo} or by reading off the occupation of the orbitals at the four time points in the segment implementation \\cite{Werner2006}. We use the latter algorithm since we consider only density-density interactions. \nThe values at the end-points $\\tau=0$ and $\\tau=\\frac{\\beta\\hbar}{2}$ reduce to standard density-density correlation functions, which in the case of Eq.~(\\ref{fermion_otoc}) are measured separately. \n\nWe perform the analytic continuation to the real-frequency axis using the Maximum Entropy method \\cite{Jarrell1996,Lewin_maxent} with a bosonic kernel. This yields the imaginary part of the retarded correlator \n$-\\frac{1}{\\pi}\\text{Im}\\, C^R_{(AA^\\dagger)^2}(\\omega)=\\mathscr A_{(AA^\\dagger)^2}(\\omega)$.  \nWhen $\\hat B=\\hat A^\\dagger=\\hat A$ (which is the case for $\\hat A=\\hat n_\\sigma, \\hat n$), we have $C_{(AA)^2}^R(\\omega)^\\ast=C_{(AA)^2}^R(-\\omega)$. At half filling, the particle-hole symmetry furthermore ensures $C_{c_\\sigma^\\dagger c_\\sigma}^R(\\omega)^\\ast=C_{c_\\sigma^\\dagger c_\\sigma}^R(-\\omega)$.\nThus, in all the cases considered in this study, the retarded OTOC has the symmetry property $C_{(AA^\\dagger)^2}^R(\\omega)^\\ast=C_{(AA^\\dagger)^2}^R(-\\omega)$. Using this, as well as the out-of-time-order fluctuation-dissipation\ntheorem discussed in the previous section, we can obtain the real-time OTOC\n$\\langle \\hat A(t)\\hat A^\\dagger(0), \\hat A(t)\\hat A^\\dagger(0)\\rangle$ from the following inverse Fourier transformation, \n\\begin{align}{\\rm Re}\\langle \\hat A(t)\\hat A^\\dagger(0), \\hat A(t)\\hat A^\\dagger(0)\\rangle\n&=\n{\\rm Re}\n\\int_0^\\infty d\\omega\\, e^{-i\\omega t}\\coth\\left(\\frac{\\beta\\hbar\\omega}{4}\\right)\n\\left(-\\frac{1}{\\pi}{\\rm Im}\\, C_{(AA^\\dagger)^2}^R(\\omega)\\right),\n\\\\\n{\\rm Im}\\langle \\hat A(t)\\hat A^\\dagger(0), \\hat A(t)\\hat A^\\dagger(0)\\rangle\n&=\n{\\rm Im}\\int_0^{\\infty} d\\omega\\, e^{-i\\omega t} \\left(-\\frac{1}{\\pi}{\\rm Im}\\, C_{(AA^\\dagger)^2}^R(\\omega)\\right),\n\\end{align}\nfor $\\hat A=c_\\sigma^\\dagger, \\hat n_\\sigma$, and $\\hat n$.\n\n\\begin{figure}[b]\n\\begin{center}\n\\includegraphics[angle=-90, width=0.48\\columnwidth]{compare_u0_draft.eps}\\hfill\n\\includegraphics[angle=-90, width=0.48\\columnwidth]{compare_u0_longtime_modulus.eps}\n\\caption{\nLeft panel: Comparison between the exact solution and the result obtained by analytical continuation\nfor the real and imaginary parts of the OTOC $\\langle c_\\sigma^\\dagger(t)c_\\sigma(0), c_\\sigma^\\dagger(t)c_\\sigma(0)\\rangle$ \nin the noninteracting case ($U=0, \\beta\\hbar=50$) of the single-orbital Hubbard model.\nRight panel: Comparison of the long-time behavior for the modulus $|\\langle c_\\sigma^\\dagger(t)c_\\sigma(0), c_\\sigma^\\dagger(t)c_\\sigma(0)\\rangle|$. \nThe exact result shows a power-law decay ($\\sim 1/t^3$). \n}\n\\label{fig:test_u0}\n\\end{center}\n\\end{figure}\n\nAs a check of our procedure, we first compare the results for the noninteracting model,\nfor which an exact solution of the OTOC $\\langle c_\\sigma^\\dagger(t)c_\\sigma(0), c_\\sigma^\\dagger(t)c_\\sigma(0)\\rangle$ \nis available \\cite{TsujiWernerUeda2017}.\nIt is a nontrivial task to measure the noninteracting two-particle Green's function in CT-HYB, \nand to analytically continue \n$C_{(c^\\dagger_\\sigma c_\\sigma)^2}^M(\\tau)$ \nby the Maximum Entropy method.  \nThe results for the inverse temperature $\\beta\\hbar=50$ are plotted in Fig.~\\ref{fig:test_u0}. \nOne can see that the OTOC oscillates for about one cycle, and quickly decays to zero.\nThe analytically continued data show a good agreement with the exact solution. \nIn particular, for short times (up to about two inverse hoppings) the dynamics is accurately reproduced, while deviations appear at longer times. \nIt is known that for the noninteracting fermion system the OTOC decays as a power law at long time ($\\sim 1/t^3$) \\cite{TsujiWernerUeda2017}.\nThe analytical continuation method cannot reproduce the details of the oscillations at intermediate or long times, but it roughly captures the long-time decay, which is controlled by low-frequency spectral features. \nUsually, analytic continuation is unreliable for high frequency components, because high-frequency information is suppressed \nby the kernel $K(\\tau,\\omega)$ in the transformation from the spectral function $A_O(\\omega)$ to the (bosonic) Matsubara correlation function $O(\\tau)$ [$O(\\tau)=\\int_{-\\infty}^{\\infty} d\\omega\\, K(\\omega,\\tau) A_O(\\omega)$ with $K(\\tau,\\omega)=e^{-\\tau\\omega}/(1-e^{-\\beta\\omega})$]. On the other hand, the low frequency components can be rather accurately determined. As a result, in Fig. 4 we more or less recover the low-frequency features of the time-dependent correlation function, while the details of rapid oscillations are not captured. The accurate result at short times can be understood from the fact that this region is close to the imaginary time axis in the complex time plane, where the original QMC data are available.\nAt higher temperatures, the Maximum Entropy method becomes less reliable, so that the resulting OTOCs are expected to be less accurate.  \n\n\n\\subsection{Single-orbital Hubbard model}\n\\label{sec:hubbard}\n\nIn this section, we calculate the OTOCs (\\ref{fermion_otoc})-(\\ref{otoc_nn}) for the interacting single-orbital Hubbard model with the Hamiltonian \n\\begin{align}\nH\n&=\n-v\\sum_{\\langle i,j\\rangle\\sigma} (c^\\dagger_{i\\sigma}c_{j\\sigma}+ \\text{h.c.})\n-\\mu\\sum_i \\hat n_i+U\\sum_i \\hat n_{i\\uparrow}\\hat n_{i\\downarrow}, \n\\end{align}\nwhere $v$ is the hopping amplitude, $\\langle i,j\\rangle$ represents the nearest-neighbor site pairs, \n$\\mu$ is the chemical potential, and $U$ is the on-site interaction.\nIn this section, we focus on the half-filled case (i.e., $\\mu=U/2$). The DMFT solution for the simplified self-consistency (\\ref{semicirc}) is the exact solution for an infinitely connected Bethe lattice \\cite{GeorgesKotliarKrauthRozenberg1996}.\nLet us briefly recall the paramagnetic phase diagram for the single-orbital Hubbard model on this lattice \\cite{Bluemer_thesis}. At half-filling, there is a metal-insulator crossover at high temperature and a first-order Mott transition below a temperature corresponding to $\\beta\\hbar\\approx 17$ with a coexistence region between $U_{c1}$ and $U_{c2}$. The finite-temperature critical endpoint is at $U\\approx 4.7$, while the zero-temperature Mott transition occurs at $U=U_{c2}\\approx 5.6$. \nAt low temperature, the self-energy shows the Fermi liquid behavior [${\\rm Im}\\,\\Sigma(\\omega)\\sim \\omega^2$] \nin the weakly correlated metallic phase, \nwhile it shows an insulating behavior [${\\rm Im}\\,\\Sigma(\\omega)\\sim \\delta(\\omega)$] in the Mott phase\n\\cite{GeorgesKotliarKrauthRozenberg1996}. \nIn the correlated metallic phase, as the temperature is increased, deviations from the Fermi liquid behavior become apparent, \nbut an ${\\rm Im}\\,\\Sigma(\\omega)\\sim \\sqrt{\\omega}$ scaling as in the SYK model is not observed. \n\nIn the following, we will start the DMFT iterations from the noninteracting solution, which means that the finite-temperature Mott transition occurs at $U_{c2}$.\nFrom now on, we set $\\hbar=1$. \n\n\\begin{figure}[t]\n\\begin{center}\n\\includegraphics[angle=-90, width=0.49\\columnwidth]{plot_otoc_b50_Re.eps}\n\\includegraphics[angle=-90, width=0.49\\columnwidth]{plot_otoc_b50_Im.eps}\n\\includegraphics[angle=-90, width=0.49\\columnwidth]{plot_coth_b50_appendix.eps}\n\\includegraphics[angle=-90, width=0.49\\columnwidth]{plot_otoc_b50_modulus.eps}\n\\caption{\nTop panels: Real and imaginary parts of the OTOC $\\langle c^\\dagger_\\sigma(t)c_\\sigma(0), c^\\dagger_\\sigma(t)c_\\sigma(0)\\rangle$ for the single-orbital Hubbard model with $\\beta=50$ at half filling. The thin lines correspond to metallic solutions, while the thick line ($U=6$) corresponds to a Mott insulating solution. \nBottom left panel: Spectral function $\\mathscr A_{(c_\\sigma^\\dagger c_\\sigma)^2}(\\omega)=-\\frac{1}{\\pi}{\\rm Im}\\,C_{(c^\\dagger_\\sigma c_\\sigma)^2}^R(\\omega)$ multiplied by $\\coth\\big(\\frac{\\beta\\omega}{4}\\big)$. \nBottom right panel: Modulus of the OTOC $|\\langle c^\\dagger_\\sigma(t)c_\\sigma(0), c^\\dagger_\\sigma(t)c_\\sigma(0)\\rangle|$ (normalized at $t=0$) on a log-log scale. The dashed line is the result for $U=0$.\n}\n\\label{fig:results_fermion}\n\\end{center}\n\\end{figure}\n\n\n\nIn Fig.~\\ref{fig:results_fermion}, we show the results of $\\langle c^\\dagger_\\sigma(t)c_\\sigma(0), c^\\dagger_\\sigma(t)c_\\sigma(0)\\rangle$ for the interacting single-orbital Hubbard model at $\\beta=50$. \nThe top panels present the real and imaginary parts of the OTOC. One can see that the oscillations become more pronounced as one increases the interaction. In the bottom left panel of Fig.~\\ref{fig:results_fermion},\nwe show the corresponding spectral function $\\mathscr A_{(c_\\sigma^\\dagger c_\\sigma)^2}(\\omega)=-\\frac{1}{\\pi}{\\rm Im}\\,C_{(c^\\dagger_\\sigma c_\\sigma)^2}^R(\\omega)$ multiplied by $\\coth\\big(\\frac{\\beta\\omega}{4}\\big)$.\nThe Mott transition, which occurs between $U=5$ and $U=6$ at $\\beta=50$, manifests itself by the opening of a gap in the spectral function $\\mathscr A_{(c_\\sigma^\\dagger c_\\sigma)^2(\\omega)}$. This translates into more weakly damped oscillations in the real-time evolution in the Mott phase.  \nThe bottom right panel of Fig.~\\ref{fig:results_fermion} plots the modulus of the OTOC (normalized at time $t=0$, i.e., $|\\langle c^\\dagger_\\sigma(t)c_\\sigma(0), c^\\dagger_\\sigma(t)c_\\sigma(0)\\rangle|/|\\langle c^\\dagger_\\sigma(0)c_\\sigma(0), c^\\dagger_\\sigma(0)c_\\sigma(0)\\rangle|$) on a log-log scale. \nDue to the oscillations and the limited accuracy of the analytic continuation procedure, it is hard to clearly identify the nature of the long-time decay of the OTOC,\nbut the results indicate that the OTOC decays much faster (possibly exponentially) in the Mott phase than in the metallic phase. This is consistent with the results for the spectral functions of the OTOC. The comparison to the noninteracting case (dashed curve in the bottom right panel of Fig.~\\ref{fig:results_fermion}) suggests that the correlated metallic phase has a similar (possibly power-law) decay behavior of the OTOC as in the noninteracting case.\n\n\n\\begin{figure}[t]\n\\begin{center}\n\\includegraphics[angle=-90, width=0.49\\columnwidth]{plot_otoc_b50_nn_Re.eps}\n\\includegraphics[angle=-90, width=0.49\\columnwidth]{plot_otoc_b50_nn_Im.eps}\n\\includegraphics[angle=-90, width=0.49\\columnwidth]{plot_coth_b50_nn_appendix.eps}\n\\includegraphics[angle=-90, width=0.49\\columnwidth]{plot_otoc_b50_nn_modulus.eps}\n\\caption{\nTop panels: Real and imaginary parts of the OTOC $\\langle \\hat n(t)\\hat n(0), \\hat n(t)\\hat n(0)\\rangle - 1$ for the single-orbital Hubbard model with $\\beta=50$ at half filling. The thin lines correspond to metallic solutions, \nwhile the thick line ($U=6$) corresponds to a Mott insulating solution. \nBottom left panel: Spectral function $\\mathscr A_{(nn)^2}(\\omega)=-\\frac{1}{\\pi}{\\rm Im}\\,C_{(n n)^2}^R(\\omega)$ multiplied by $\\coth\\big(\\frac{\\beta\\omega}{4}\\big)$.\nBottom right panel: Modulus of the OTOC $|\\langle \\hat n(t)\\hat n(0), \\hat n(t)\\hat n(0)\\rangle - 1|$ (normalized at $t=0$) on a log-log scale. \n}\n\\label{fig:results_nn}\n\\end{center}\n\\end{figure}\n\n\nIn Fig.~\\ref{fig:results_nn}, we plot the OTOC $\\langle \\hat n(t)\\hat n(0), \\hat n(t)\\hat n(0)\\rangle$\nfor the single-orbital Hubbard model in a similar manner as in Fig.~\\ref{fig:results_fermion}.\nSince at half filling this correlation function approaches $\\langle \\hat n\\rangle^4=1$ at long times\nand at sufficiently low temperature,\nwe perform the analytical continuation procedure for $\\langle \\hat n(t)\\hat n(0), \\hat n(t)\\hat n(0)\\rangle-1$. \nThe top panels of Fig.~\\ref{fig:results_nn} show the real and imaginary parts of this shifted OTOC. \nContrary to the case of $\\langle c_\\sigma^\\dagger(t)c_\\sigma(0), c_\\sigma^\\dagger(t)c_\\sigma(0)\\rangle$, \nthe amplitude of the OTOC $\\langle \\hat n(t)\\hat n(0), \\hat n(t)\\hat n(0)\\rangle-1$ is suppressed as one increases the interaction, which reflects the reduced charge fluctuations in the Mott insulator.\nThe bottom left panel in Fig.~\\ref{fig:results_nn} shows the analytically continued spectral function $\\mathscr A_{(nn)^2}(\\omega)=-\\frac{1}{\\pi}{\\rm Im}\\, C_{(nn)^2}^R(\\omega)$ multiplied by $\\coth\\big(\\frac{\\beta\\omega}{4}\\big)$. \nAgain we notice the opening of a gap in the Mott phase, and a corresponding shift of the spectral weight to higher energies. This results in more rapid oscillations of the OTOC in the Mott state.\nThe bottom right panel in Fig.~\\ref{fig:results_nn} plots the modulus of the OTOC $|\\langle \\hat n(t)\\hat n(0), \\hat n(t)\\hat n(0)\\rangle-1|$ (normalized at $t=0$) on a log-log scale.\nThe long-time behavior of the OTOC is consistent with a power-law decay in the metallic phase, and an exponential decay in the Mott insulating phase.\n\n\n\\begin{figure}[t]\n\\begin{center}\n\\includegraphics[angle=-90, width=0.49\\columnwidth]{plot_otoc_nsns_b50_draft_Re.eps}\n\\includegraphics[angle=-90, width=0.49\\columnwidth]{plot_otoc_nsns_b50_draft_Im.eps}\n\\includegraphics[angle=-90, width=0.49\\columnwidth]{plot_coth_b50_nsns_inset.eps}\n\\includegraphics[angle=-90, width=0.49\\columnwidth]{plot_otoc_nsns_b50_draft_modulus_zoom.eps}\n\\caption{\nTop panels: Real and imaginary parts of the OTOC $\\langle \\hat n_\\sigma(t)\\hat n_\\sigma(0), \\hat n_\\sigma(t)\\hat n_\\sigma(0)\\rangle  - \\langle \\hat n_\\sigma\\rangle^4$ for the single-orbital Hubbard model with $\\beta=50$ at half filling. All the colored lines correspond to metallic solutions. \nBottom left panel: Spectral function $\\mathscr A_{(n_\\sigma n_\\sigma)^2}(\\omega)=-\\frac{1}{\\pi}{\\rm Im}\\,C_{(n_\\sigma n_\\sigma)^2}^R(\\omega)$ multiplied by $\\coth\\big(\\frac{\\beta\\omega}{4}\\big)$. In the inset, we plot the imaginary-time four-point function $\\langle \\hat n_\\sigma(\\tau)\\hat n_\\sigma(0), \\hat n_\\sigma(\\tau)\\hat n_\\sigma(0)\\rangle-\\langle \\hat n_\\sigma\\rangle^4$, with the black line corresponding to the Mott insulating solution at $U=6$.  \nBottom right panel: Modulus of the OTOC $|\\langle \\hat n_\\sigma(t)\\hat n_\\sigma(0), \\hat n_\\sigma(t)\\hat n_\\sigma(0)\\rangle  - \\langle \\hat n_\\sigma\\rangle^4|$ (normalized at $t=0$) on a log-log scale.\n}\n\\label{fig:results_nsns}\n\\end{center}\n\\end{figure}\n\nFigure~\\ref{fig:results_nsns} \nshows the results of the OTOC $\\langle \\hat n_\\sigma(t)\\hat n_\\sigma(0), \\hat n_\\sigma(t)\\hat n_\\sigma(0)\\rangle$\nfor the single-orbital Hubbard model. At half filling, this correlation function approaches $\\langle \\hat n_\\sigma\\rangle^4=\\frac{1}{16}$ in the long-time limit and at sufficiently low temperature, \nso that we perform the analytical continuation procedure for $\\langle \\hat n_\\sigma(t)\\hat n_\\sigma(0), \\hat n_\\sigma(t)\\hat n_\\sigma(0)\\rangle-\\langle \\hat n_\\sigma\\rangle^4$. \nThe top panels of Fig.~\\ref{fig:results_nsns} show the real and imaginary parts of \nthis shifted OTOC. \nHere, we only present results for the metallic phase, because resolving the very sharp low-energy feature of the OTOC spectrum in the half-filled Mott state is challenging. One can see that coherent oscillations are not observed for this type of OTOC, and that the incoherent part becomes dominant compared to the other two OTOCs shown in Figs.~\\ref{fig:results_fermion} and \\ref{fig:results_nn}. The amplitude of the OTOC $\\langle \\hat n_\\sigma(t)\\hat n_\\sigma(0), \\hat n_\\sigma(t)\\hat n_\\sigma(0)\\rangle-\\langle \\hat n_\\sigma\\rangle^4$ is enhanced as $U$ increases. As we argue below, this is related to the enhancement of spin fluctuations \nand local moment formation close to the Mott transition.\n\nThe bottom left panel in Fig.~\\ref{fig:results_nsns} plots the analytically continued spectral function $\\mathscr A_{(n_\\sigma n_\\sigma)^2}(\\omega)=-\\frac{1}{\\pi}{\\rm Im}\\, C_{(n_\\sigma n_\\sigma)^2}^R(\\omega)$ multiplied by $\\coth\\big(\\frac{\\beta\\omega}{4}\\big)$. \nWe can see that the spectral weight is concentrated in the low-energy region as we approach the Mott transition point.\nThe inset in the bottom left panel of Fig.~\\ref{fig:results_nsns} shows the imaginary-time four-point function $\\langle \\hat n_\\sigma(\\tau)\\hat n_\\sigma(0), \\hat n_\\sigma(\\tau)\\hat n_\\sigma(0)\\rangle-\\langle \\hat n_\\sigma\\rangle^4$, with the black line corresponding to a Mott insulating solution ($U=6$).  \nFor $U\\gtrsim 4$, the imaginary-time four-point function does not decay to zero but remains relatively large near $\\tau=\\frac{\\beta}{4}$. \nThis behavior is reminiscent of the spin-freezing physics seen in the correlated Hund metal phase of multi-orbital Hubbard models, where the dynamical spin correlation function $\\langle \\hat S_z(\\tau)\\hat S_z(0)\\rangle$ is trapped at a finite value at long $\\tau$ \\cite{Werner2008}. \nIn fact, the OTOC $\\langle \\hat n_\\sigma(\\tau)\\hat n_\\sigma(0), \\hat n_\\sigma(\\tau)\\hat n_\\sigma(0)\\rangle$\nmeasures spin correlations (in addition to charge correlations) through $\\hat n_\\sigma=\\frac{1}{2}\\hat n+\\sigma\\hat S_z$.\nPhysically, the trapping of $\\langle \\hat S_z(\\tau)\\hat S_z(0)\\rangle$ signals the formation of frozen local magnetic moments.\nIn the metal-insulator crossover region, the scattering induced by these magnetic moments results in incoherent metal states. Intuitively, one might expect fast scrambling in the corresponding regions of the phase diagram. \nAlthough there is some resemblance with the spin freezing crossover,\nthe self-energy does not exhibit a square-root frequency dependence in the single-orbital case, and a spin-freezing crossover in the sense of Ref.~\\cite{Werner2008} does not exist.  \n\nThe bottom right panel of Fig.~\\ref{fig:results_nsns} plots the modulus of the OTOC $|\\langle \\hat n_\\sigma(t)\\hat n_\\sigma(0), \\hat n_\\sigma(t)\\hat n_\\sigma(0)\\rangle-\\langle \\hat n_\\sigma\\rangle^4|$ (normalized at $t=0$) on a log-log scale. \nThe dynamics of $\\langle \\hat n_\\sigma(t)\\hat n_\\sigma(0), \\hat n_\\sigma(t)\\hat n_\\sigma(0)\\rangle$ is quite distinct from that of $\\langle \\hat n(t)\\hat n(0), \\hat n(t)\\hat n(0)\\rangle$. As the Mott transition is approached, we observe a slow-down in the decay of the modulus, \nwhich indicates the presence of slowly fluctuating local moments. \nThe long-time behavior may be consistent with a power-law, although slow oscillations make it difficult to determine the long-time asymptotic form from the numerics. \n\n\n\\begin{figure}[t]\n\\begin{center}\n\\includegraphics[angle=-90, width=0.49\\columnwidth]{plot_fitnsns_b2.eps}\n\\includegraphics[angle=-90, width=0.49\\columnwidth]{plot_coeffc_draft.eps}\n\\caption{\nLeft panel: \nThe imaginary-time four-point function $\\langle \\hat n_\\sigma(\\tau)\\hat n_\\sigma(0), \\hat n_\\sigma(\\tau)\\hat n_\\sigma(0)\\rangle$, the squared density-density correlation function $\\langle \\hat n_\\sigma(\\tau)\\hat n_\\sigma(0)\\rangle^2$,\nand their difference for the single-orbital Hubbard model with $U=5.5$ and $\\beta=50$.\nThe dashed curve shows a fit of the difference with the function $f_\\text{fit}(\\tau)=a+b\\tau+c\\tau^2$ in the interval $\\tau/\\beta \\in [0: 0.2]$.\nRight panel: Fitting coefficient $c$ as a function of $U$ for indicated values of $\\beta$.\n\n}\n\\label{fig:coeffs}\n\\end{center}\n\\end{figure}\n\nFor large enough $\\beta$ and small enough $\\tau$, the OTOC (\\ref{otoc_nsns}) factorizes into $\\langle \\hat n_\\sigma(\\tau)\\hat n_\\sigma(0), \\hat n_\\sigma(\\tau)\\hat n_\\sigma(0)\\rangle \\approx \\langle \\hat n_\\sigma(\\tau)\\hat n_\\sigma(0)\\rangle^2$. \nOn the real-time axis, this corresponds to the decoupling $\\langle \\hat n_\\sigma(t)\\hat n_\\sigma(0), \\hat n_\\sigma(t)\\hat n_\\sigma(0)\\rangle \\approx \\langle \\hat n_\\sigma(t)\\hat n_\\sigma(0)\\rangle^2$, that is, \nthe OTOC is reduced to a product of ordinary density-density correlation functions.\nIn order to see whether nontrivial correlations are captured beyond the decoupled form by the OTOC function, \nwe consider the difference $\\langle \\hat n_\\sigma(\\tau)\\hat n_\\sigma(0), \\hat n_\\sigma(\\tau)\\hat n_\\sigma(0)\\rangle-\\langle \\hat n_\\sigma(\\tau)\\hat n_\\sigma(0)\\rangle^2$ and fit the short-time behavior of this function to $f_\\text{fit}(\\tau)=a+b\\tau+c\\tau^2$, as illustrated in the left panel of Fig.~\\ref{fig:coeffs}. The coefficients $b$ and $c$ serve as  indicators of the nontrivial correlations that cannot be attributed to the decoupled form of the OTOC.\nThe coefficient $c$ is plotted as a function of $U$ and for different $\\beta$ in the right hand panel. The coefficient $b$ (not shown) exhibits a similar trend. We notice that the OTOC picks up nontrival correlations near the metal-insulator transition, and in particular at \nintermediate\ntemperatures, while the Mott phase shows no such correlations. The crossover region  \nassociated with the emergence of local moments \ncorresponds, roughly, to the interaction range where the nontrivial correlations start to become significant. \n\n\n\\subsection{Two-orbital Hubbard model}\n\\label{sec:two-orbital hubbard}\n\nIn this section, we investigate OTOCs in the two-orbital Hubbard model with Hund coupling $J>0$ \nand density-density interactions. (Three-orbital results are presented in Appendix~\\ref{sec:3orbital}.) \nThe Hamiltonian is given by\n\\begin{align}\nH\n&=\n-v\\sum_{\\langle i,j\\rangle\\alpha\\sigma} (c^\\dagger_{i\\alpha\\sigma}c_{j\\alpha\\sigma}+ \\text{h.c.})\n-\\mu\\sum_{i\\alpha\\sigma} \\hat n_{i\\alpha\\sigma}\n+U\\sum_{i\\alpha} \\hat n_{i\\alpha\\uparrow}\\hat n_{i\\alpha\\downarrow}\n+(U-2J)\\sum_{i\\sigma} \\hat n_{i1\\sigma}\\hat n_{i2\\bar\\sigma}\n\\notag\n\\\\\n&\\quad\n+(U-3J)\\sum_{i\\sigma} \\hat n_{i1\\sigma}\\hat n_{i2\\sigma},\n\\end{align}\n\nwith $U$ being the intra-orbital interaction $U$, $U-2J$ the inter-orbital antiparallel-spin interaction, and $U-3J$ the inter-orbital parallel-spin interaction. This is the simplest model that shows a crossover from a spin-frozen metal to a Fermi-liquid metal as one dopes the half-filled Mott insulator, with the self-energy scaling as $\\text{Im}\\Sigma(\\omega_n)\\sim\\sqrt{\\omega_n}$ over a significant energy range in the crossover regime \\cite{Hafermann2012}.\nThe sketch of the phase diagram is shown in Fig.~\\ref{fig:illustration}. \nAs we have seen in the previous section, the spin-related OTOC of the type (\\ref{otoc_nsns}) is relevant for the analysis of  the spin-freezing crossover, so that we concentrate on this OTOC here.\nIn the following calculations, we choose $U=8$, $J=U/4$, and compute $\\langle \\hat n_\\sigma(t)\\hat n_\\sigma(0), \\hat n_\\sigma(t)\\hat n_\\sigma(0)\\rangle - \\langle n_\\sigma \\rangle^4$ \nas a function of filling at $\\beta=50$ (dashed lines in Fig.~\\ref{fig:illustration}). \n\n\n\\begin{figure}[t]\n\\begin{center}\n\\includegraphics[angle=-90, width=0.49\\columnwidth]{plot_qp_draft.eps}\\hfill\n\\includegraphics[angle=-90, width=0.49\\columnwidth]{plot_fit_2orbital.eps}\n\\caption{\nLeft panel: Imaginary part of the self-energy as a function of Matsubara frequency for the two-orbital Hubbard model with $U=8$, $J=U/4$, and $\\beta=50$ on a log-log scale, with the dashed line indicating $\\sqrt{\\omega_n}$ behavior. \nRight panel: The coefficient $c$ extracted from\nfitting $\\langle \\hat n_\\sigma(\\tau)\\hat n_\\sigma(0), \\hat n_\\sigma(\\tau)\\hat n_\\sigma(0)\\rangle-\\langle \\hat n_\\sigma(\\tau)\\hat n_\\sigma(0)\\rangle^2$ with a function $f_\\text{fit}(\\tau)=a+b\\tau+c\\tau^2$\nin the interval $\\tau/\\beta\\in [0: 0.05]$ for the two-orbital Hubbard model. \nThe spin-freezing crossover occurs roughly at the filling $n_\\sigma\\approx 0.28$ (blue curve in the left panel and blue arrow in the right panel). \n}\n\\label{fig:self}\n\\end{center}\n\\end{figure}\n\n\nThe left panel of Fig.~\\ref{fig:self} plots the imaginary part of the self-energy as a function of Matsubara frequency in order to identify the filling corresponding to the spin-freezing crossover \\cite{Werner2008}. An approximate square-root scaling is observed over a wide range of frequencies near $n_\\sigma\\approx 0.28$ (blue line). In the right panel of Fig.~\\ref{fig:self}, we present the coefficient $c$ obtained from a similar analysis as presented in Fig.~\\ref{fig:coeffs}, but with a fitting range $\\tau/\\beta\\in [0 : 0.05]$. Again, we find that the OTOC $\\langle \\hat n_\\sigma(t)\\hat n_\\sigma(0), \\hat n_\\sigma(t)\\hat n_\\sigma(0)\\rangle - \\langle \\hat n_\\sigma\\rangle^4$ detects nontrivial correlations in the spin-freezing crossover (blue arrow) and spin-frozen metal regime, but not in the Fermi liquid or the Mott insulating phase.\n\n\n\\begin{figure}[t]\n\\begin{center}\n\\includegraphics[angle=-90, width=0.49\\columnwidth]{plot_otoc_2orbital_Re.eps}\n\\includegraphics[angle=-90, width=0.486\\columnwidth]{plot_otoc_2orbital_Im.eps}\n\\includegraphics[angle=-90, width=0.484\\columnwidth]{plot_spectra_coth_2orbital_inset.eps}\n\\includegraphics[angle=-90, width=0.486\\columnwidth]{plot_otoc_2orbital_modulus_inset.eps}\n\\caption{\nTop panels: Real and imaginary parts of the OTOC $\\langle \\hat n_\\sigma(t)\\hat n_\\sigma(0), \\hat n_\\sigma(t)\\hat n_\\sigma(0)\\rangle  - \\langle \\hat n_\\sigma\\rangle^4$ for the two-orbital Hubbard model with $U=8$, $J=U/4$, $\\beta=50$ and indicated fillings. \nBottom left panel: Spectral function $\\mathscr A_{(n_\\sigma n_\\sigma)^2}(\\omega)=-\\frac{1}{\\pi}{\\rm Im}\\,C_{(n_\\sigma n_\\sigma)^2}^R(\\omega)$ multiplied by $\\coth\\big(\\frac{\\beta\\omega}{4}\\big)$. In the inset, we plot the imaginary-time four-point function $\\langle \\hat n_\\sigma(\\tau)\\hat n_\\sigma(0), \\hat n_\\sigma(\\tau)\\hat n_\\sigma(0)\\rangle-\\langle \\hat n_\\sigma\\rangle^4$. \nBottom right panel: Modulus of the OTOC $|\\langle \\hat n_\\sigma(t)\\hat n_\\sigma(0), \\hat n_\\sigma(t)\\hat n_\\sigma(0)\\rangle  - \\langle \\hat n_\\sigma\\rangle^4|$ (normalized at $t=0$) on a log-log scale.\nIn the inset of the bottom right panel, we illustrate the approximate data collapse obtained by considering the spin-freezing crossover regimes for different values of $J$. \nIn all the panels, the thick blue curves represent the results for the spin-freezing crossover region. \n}\n\\label{fig:2orbital}\n\\end{center}\n\\end{figure}\n\n\nThe top panels of Fig.~\\ref{fig:2orbital} show the real and imaginary parts of the OTOC\n$\\langle \\hat n_\\sigma(t)\\hat n_\\sigma(0), \\hat n_\\sigma(t)\\hat n_\\sigma(0)\\rangle-\\langle \\hat n_\\sigma\\rangle^4$\nfor the two-orbital Hubbard model. In the spin-frozen phase ($n_\\sigma \\gtrsim 0.3$), both components exhibit \nhighly incoherent oscillations, which get suppressed as the filling is reduced. As one moves across the crossover point ($n_\\sigma \\sim 0.28$),\nthe oscillations completely disappear, and the real part of the OTOC starts to overshoot to the negative side\nin the Fermi liquid regime.\nThe bottom left panel in Fig.~\\ref{fig:2orbital} presents the analytically continued spectral function $\\mathscr A_{(n_\\sigma n_\\sigma)^2}(\\omega)=-\\frac{1}{\\pi}{\\rm Im}\\,C_{(n_\\sigma n_\\sigma)^2}^R(\\omega)$ multiplied by $\\coth\\big(\\frac{\\beta\\omega}{4}\\big)$. \nIn the spin-frozen regime ($n_\\sigma \\gtrsim 0.3$) a sharp peak appears at low energy, which originates from the saturation of the imaginary-time four point function $\\langle \\hat n_\\sigma(\\tau)\\hat n_\\sigma(0), \\hat n_\\sigma(\\tau)\\hat n_\\sigma(0)\\rangle$ at large $\\tau$ (see the inset of the bottom left panel of Fig.~\\ref{fig:2orbital}). This low-energy peak represents the slow dynamics of the frozen spins, and translates into a slow long-time decay of $|\\langle \\hat n_\\sigma(t)\\hat n_\\sigma(0), \\hat n_\\sigma(t)\\hat n_\\sigma(0)\\rangle-\\langle \\hat n_\\sigma\\rangle^4|$,\nas shown in the bottom right panel of Fig.~\\ref{fig:2orbital}\non a log-log scale. Again, it is difficult to clearly resolve the long-time behavior of the OTOC due to the limited\naccuracy of the analytic continuation, but in the spin-freezing crossover regime (thick blue curves)\nthe decay of the OTOC is consistent with a power law $\\sim 1/t^{1.5}$ at least for $2\\lesssim t \\lesssim 5$. \nThe shorter-time behavior ($0.5\\lesssim t \\lesssim 2$) is instead well fitted by an exponential decay $\\sim e^{-\\alpha t}$ with a decay constant $\\alpha=0.78$. \nThis exponent exhibits\na strong dependence on the Hund coupling. \n\nIn fact, the OTOCs in the spin-freezing regime for $J=U/4$ and $J=U/6$ can be approximately collapsed by plotting them as a function of $tJ$ (see the inset in the bottom right panel of Fig.~\\ref{fig:2orbital}), which indicates that the Hund coupling is the parameter which controls the dynamics in this regime. \nThis is distinct from the Lyapunov behavior in which the exponential decay scales with $t/\\beta$ \\cite{Bagrets2017}.\nA possible reason is that we are still far from the large $N$ limit (where $N$ corresponds to the number of orbitals times the number of spin degrees of freedom) so that the large $N$ behavior \nsuch as the Lyapunov growth is not observed in our calculations. \nIn fact, the strong dependence of the time scale on $J$ and weak dependence on $\\beta$ is consistent\nwith the behavior of the finite $N$ SYK model in Ref.~\\cite{FuSachdev2016}.\nIn the spin-frozen regime ($n_\\sigma\\gtrsim 0.3$) the decay is much slower than near the crossover point, while in the Fermi liquid regime the decay is accelerated, approaching the free fermion behavior of $t^{-3}$\nas shown in Fig.~\\ref{analytic continuation}. \nQualitatively similar results for this OTOC are obtained \nfor the three-orbital Hubbard model (see Appendix~\\ref{sec:3orbital}).\n\nThe time dependence of the modulus of \n$\\langle \\hat n_\\sigma(\\tau)\\hat n_\\sigma(0), \\hat n_\\sigma(\\tau)\\hat n_\\sigma(0)\\rangle-\\langle \\hat n_\\sigma\\rangle^4$ \nin the spin-freezing crossover regime of multi-orbital Hubbard models (exponential decay crossing over into a power-law) bears a close resemblance to a (different) OTOC for the SYK model discussed in Ref.~\\cite{Bagrets2017}. \nAlso, a recent study of yet another type of OTOCs for the finite-$N$ SYK model found a qualitatively similar decay as observed here in the spin-freezing crossover regime \\cite{FuSachdev2016}. In the following section, we will make the connection to the SYK dynamics more quantitative.   \n\n\n\\subsection{Comparison to the SKY model}\n\\label{sec:SYK}\n\nIn this section, we compare the results of the multi-orbital Hubbard models to the SYK model.\nWe consider the particle-hole symmetric SYK model of complex fermions \\cite{FuSachdev2016},\nwhose Hamiltonian is given by\n\\begin{align}\nH\n&=\n\\frac{1}{(2N)^{3/2}} \\sum_{i,j,k,l=1}^N \nJ_{ij;kl} (c_i^\\dagger c_j^\\dagger c_k c_l\n+\\delta_{ik}n c_j^\\dagger c_l\n-\\delta_{il}n c_j^\\dagger c_k\n-\\delta_{jk}n c_i^\\dagger c_l\n+\\delta_{jl}n c_i^\\dagger c_k).\n\\end{align}\nThe coupling constant $J_{ij;kl}$ is a gaussian random variable, satisfying $J_{ij;kl}=-J_{ji;kl}=-J_{ij;lk}=J_{kl;ij}^\\ast$, and\n\\begin{align}\n\\overline{({\\rm Re}\\, J_{ij;kl})^2}\n&=\n\\begin{cases}\nJ_{\\rm SYK}^2/2 & (i,j)\\neq (k,l) \\\\\nJ_{\\rm SYK}^2 & (i,j)=(k,l)\n\\end{cases},\n\\\\\n\\overline{({\\rm Im}\\, J_{ij;kl})^2}\n&=\n\\begin{cases}\nJ_{\\rm SYK}^2/2 & (i,j)\\neq (k,l) \\\\\n0 & (i,j)=(k,l)\n\\end{cases},\n\\end{align}\nwhere the overline represents an average over each realization of $J_{ij;kl}$. The parameter \n$J_{\\rm SYK}$ ($J_{\\rm SYK}^{-1}$) defines the unit of energy (time) in the following calculations. \nThe system is particle-hole symmetric when\n$n=0.5$, which fixes the total number of fermions as $N^{-1}\\sum_{i=1}^N \\langle c_i^\\dagger c_i\\rangle=0.5$.\nWe take the particle-hole symmetric form of the SYK model because it has been well studied previously \\cite{FuSachdev2016} and avoids the effect of the filling drift.\nWe numerically solve the model by exact diagonalization for finite $N(\\le 12)$, and evaluate the density-density OTOC\n$\\overline{\\langle \\hat n_i(t)\\hat n_i(0), \\hat n_i(t)\\hat n_i(0)\\rangle-\\langle \\hat n_i\\rangle^4}$\nand its imaginary-time counterpart.\n\n\\begin{figure}[t]\n\\begin{center}\n\n\\includegraphics[angle=-90,width=0.49\\columnwidth]{plot_fitnsns_syk.eps}\n\n\\includegraphics[angle=-90,width=0.49\\columnwidth]{plot_coeffc_syk.eps}\n\\caption{\nLeft panel: \nThe imaginary-time four-point function $\\overline{\\langle \\hat n_i(\\tau)\\hat n_i(0), \\hat n_i(\\tau)\\hat n_i(0)\\rangle}$, the density-density correlation function $\\overline{\\langle \\hat n_i(\\tau)\\hat n_i(0)\\rangle^2}$,\nand their difference for the SYK model with $N=12$ and $\\beta J_{\\rm SYK}=10$.\nThe dashed curve shows a fit of the difference with a function $f_\\text{fit}(\\tau)=a+b\\tau+c\\tau^2$ in the interval $\\tau/\\beta \\in [0: 0.2]$. \nRight panel: Fitting coefficient $c$ as a function of $T/J_{\\rm SYK}$.\n}\n\\label{fig:SYK fit}\n\\end{center}\n\\end{figure}\n\nIn the left panel of Fig.~\\ref{fig:SYK fit}, we plot the imaginary-time four-point function \n$\\overline{\\langle \\hat n_i(\\tau)\\hat n_i(0), \\hat n_i(\\tau)\\hat n_i(0)\\rangle}$\nfor the SYK model with $N=12$ and $\\beta J_{\\rm SYK}=10$. \nAt low temperature, we expect that $\\overline{\\langle \\hat n_i(\\tau)\\hat n_i(0), \\hat n_i(\\tau)\\hat n_i(0)\\rangle}$ is decoupled into $\\overline{\\langle \\hat n_i(\\tau)\\hat n_i(0)\\rangle^2}$.\nTo see whether nontrivial correlations beyond the decoupled form are present, we fit the difference\n$\\overline{\\langle \\hat n_i(\\tau)\\hat n_i(0), \\hat n_i(\\tau)\\hat n_i(0)\\rangle}\n-\\overline{\\langle \\hat n_i(\\tau)\\hat n_i(0)\\rangle^2}$ with a function $f_{\\rm fit}(\\tau)=a+b\\tau+c\\tau^2$\nin the interval $\\tau/\\beta\\in[0:0.2]$, in the same manner as for the Hubbard models.\nThe obtained coefficient $c$ is plotted as a function of the temperature $T$ in the right panel of\nFig.~\\ref{fig:SYK fit}. One can see that nontrivial correlations exist in a wide range of temperature.\nEspecially, they are enhanced in the intermediate temperature regime. This is because in the zero-temperature limit\nthe four-point function is decoupled as \n$\\langle \\hat n_i(\\tau)\\hat n_i(0), \\hat n_i(\\tau)\\hat n_i(0)\\rangle \\approx \\langle \\hat n_i(\\tau)\\hat n_i(0)\\rangle^2$, while in the high-temperature limit ($\\beta\\to 0$) the imaginary-time dependence is washed out.\n\n\n\n\n\\begin{figure}[t]\n\\begin{center}\n\\includegraphics[angle=-90, width=0.49\\columnwidth]{plot_syk_Re.eps}\n\\includegraphics[angle=-90, width=0.49\\columnwidth]{plot_syk_Im.eps}\n\\includegraphics[angle=-90, width=0.49\\columnwidth]{plot_syk_modulus-log.eps}\n\\includegraphics[angle=-90, width=0.49\\columnwidth]{plot_syk_modulus.eps}\n\\caption{\nThe density-density OTOC $\\overline{\\langle \\hat n_i(t)\\hat n_i(0), \\hat n_i(t)\\hat n_i(0)\\rangle-\\langle \\hat n_i\\rangle^4}$ for the particle-hole symmetric SYK model averaged over $10^2$ samples with $\\beta J_{\\rm SYK}=100$ and various $N$. \nThe top left and right panels show the real and imaginary parts of the OTOC,\nwhile the bottom left and right panels show the log and log-log plots of the modulus of the OTOC (normalized at $t=0$).\nThe dashed line shows a power-law decay $\\propto t^{-1.6}$ for comparison.}\n\\label{fig:SYK}\n\\end{center}\n\\end{figure}\n\nIn Fig.~\\ref{fig:SYK}, we show the numerical results of the density-density OTOC for $\\beta J_{\\rm SYK}=100$.\nThe real part of the OTOC rapidly drops from the initial value within the time scale of $tJ_{\\rm SYK}\\sim 2$,\nand slowly decays to zero in the long time limit.\nAs one increases $N$, the oscillations that appear at longer time tend to be suppressed.\nOne can see that the result for $N=12$ in the top panels of Fig.~\\ref{fig:SYK} closely resembles the OTOC in \nthe spin-freezing crossover regime of the two-orbital Hubbard model shown\nby the blue curve in the top panels of Fig.~\\ref{fig:2orbital},\nwhile it differs from the single-orbital result shown in the top panels of Fig.~\\ref{fig:results_nsns}.\nDue to the limitation of our calculations to small system sizes ($N\\le 12$), we do not clearly observe\nan exponential growth ($\\sim c_0-c_1 e^{\\lambda t}$)  at short times or an exponential decay ($\\sim e^{-\\alpha t}$) \nat intermediate times. \nHowever, we find that for $tJ_{\\rm SYK}\\gtrsim 2$ the OTOC decays\napproximately in a power law $\\propto t^{-\\gamma}$ with $\\gamma=1.6$ (see bottom right panel in Fig.~\\ref{fig:SYK}). \nThe time interval exhibiting the power-law decay becomes longer as we increase $N$.  \nThe temperature dependence of the OTOC for the SYK model is shown in Fig.~\\ref{fig:SYK-T}.\nThe time scale of the initial drop is more or less independent of the temperature, while \nthe power-law-like decay is only visible in the low-temperature regime.\n\n\\begin{figure}[t]\n\\begin{center}\n\\includegraphics[angle=-90, width=0.49\\columnwidth]{plot_syk_beta_re.eps}\n\\includegraphics[angle=-90, width=0.49\\columnwidth]{plot_syk_beta_im.eps}\n\\includegraphics[angle=-90, width=0.49\\columnwidth]{plot_syk_beta_modulus-log.eps}\n\\includegraphics[angle=-90, width=0.49\\columnwidth]{plot_syk_beta_modulus.eps}\n\\caption{\nTemperature dependence of the density-density OTOC\n$\\overline{\\langle \\hat n_i(t)\\hat n_i(0), \\hat n_i(t)\\hat n_i(0)\\rangle-\\langle \\hat n_i\\rangle^4}$ for the particle-hole symmetric SYK model averaged over $10^2$ samples with $N=12$.\nThe top left and right panels show the real and imaginary parts of the OTOC,\nwhile the bottom left and right panels show the log and log-log plots of the modulus of the OTOC.\nThe dashed line shows a power-law decay $\\propto t^{-1.6}$ for comparison.}\n\\label{fig:SYK-T}\n\\end{center}\n\\end{figure}\n\nIn Ref.~\\onlinecite{Werner2018}, it has been argued that the spin-freezing crossover regime of multi-orbital Hubbard models is effectively described by the SYK model in which $J_{\\rm SYK}$ is replaced by the Hund coupling $J$. The results in Fig.~\\ref{fig:2orbital} are for the two-orbital Hubbard model with $U=8$, $J/U=1/4$ and $\\beta=50$, corresponding to $\\beta J=100$, which means that it is meaningful to directly compare the blue lines in Fig.~\\ref{fig:2orbital} with the blue lines in Figs.~\\ref{fig:SYK} and \\ref{fig:SYK-T}. Not only is the qualitative agreement remarkable, even the power-law exponent measured for the two-orbital Hubbard model ($\\gamma=1.5$) is very close to the exponent extracted from the finite-$N$ SYK calculation. \nThe power-law decay in the three-orbital case is approximately $1/t^{1.75}$ (see Appendix~\\ref{sec:3orbital}), and thus also in semi-quantitative agreement with the SYK model behavior. \nThis nontrivial result provides further support for the identification of the spin-freezing crossover regime of multi-orbital Hubbard systems with an SYK strange metal. \n\nIn the large-$N$ and low-$T$ limit of the SYK model, we expect that the power-law behavior approaches $1/t^2$, since in this limit the OTOC is decoupled as $\\langle \\hat n_i(t)\\hat n_i(0), \\hat n_i(t)\\hat n_i(0)\\rangle\\approx \\langle \\hat n_i(t)\\hat n_i(0)\\rangle^2$ and the decay of $\\langle \\hat n_i(t)\\hat n_i(0)\\rangle$ is dominated by what corresponds to the slow spin relaxation $\\langle \\hat S_z(t)\\hat S_z(0)\\rangle\\sim 1/t$ in the Sachdev-Ye model.\n\n\n\n\n\n4.3 Two-orbital Hubbard model\n\\subsection{Two-orbital Hubbard model}\n\\label{sec:two-orbital hubbard}\n\nIn this section, we investigate OTOCs in the two-orbital Hubbard model with Hund coupling $J>0$ \nand density-density interactions. (Three-orbital results are presented in Appendix~\\ref{sec:3orbital}.) \nThe Hamiltonian is given by\n\\begin{align}\nH\n&=\n-v\\sum_{\\langle i,j\\rangle\\alpha\\sigma} (c^\\dagger_{i\\alpha\\sigma}c_{j\\alpha\\sigma}+ \\text{h.c.})\n-\\mu\\sum_{i\\alpha\\sigma} \\hat n_{i\\alpha\\sigma}\n+U\\sum_{i\\alpha} \\hat n_{i\\alpha\\uparrow}\\hat n_{i\\alpha\\downarrow}\n+(U-2J)\\sum_{i\\sigma} \\hat n_{i1\\sigma}\\hat n_{i2\\bar\\sigma}\n\\notag\n\\\\\n&\\quad\n+(U-3J)\\sum_{i\\sigma} \\hat n_{i1\\sigma}\\hat n_{i2\\sigma},\n\\end{align}\n\nwith $U$ being the intra-orbital interaction $U$, $U-2J$ the inter-orbital antiparallel-spin interaction, and $U-3J$ the inter-orbital parallel-spin interaction. This is the simplest model that shows a crossover from a spin-frozen metal to a Fermi-liquid metal as one dopes the half-filled Mott insulator, with the self-energy scaling as $\\text{Im}\\Sigma(\\omega_n)\\sim\\sqrt{\\omega_n}$ over a significant energy range in the crossover regime \\cite{Hafermann2012}.\nThe sketch of the phase diagram is shown in Fig.~\\ref{fig:illustration}. \nAs we have seen in the previous section, the spin-related OTOC of the type (\\ref{otoc_nsns}) is relevant for the analysis of  the spin-freezing crossover, so that we concentrate on this OTOC here.\nIn the following calculations, we choose $U=8$, $J=U/4$, and compute $\\langle \\hat n_\\sigma(t)\\hat n_\\sigma(0), \\hat n_\\sigma(t)\\hat n_\\sigma(0)\\rangle - \\langle n_\\sigma \\rangle^4$ \nas a function of filling at $\\beta=50$ (dashed lines in Fig.~\\ref{fig:illustration}). \n\n\n\\begin{figure}[t]\n\\begin{center}\n\\includegraphics[angle=-90, width=0.49\\columnwidth]{plot_qp_draft.eps}\\hfill\n\\includegraphics[angle=-90, width=0.49\\columnwidth]{plot_fit_2orbital.eps}\n\\caption{\nLeft panel: Imaginary part of the self-energy as a function of Matsubara frequency for the two-orbital Hubbard model with $U=8$, $J=U/4$, and $\\beta=50$ on a log-log scale, with the dashed line indicating $\\sqrt{\\omega_n}$ behavior. \nRight panel: The coefficient $c$ extracted from\nfitting $\\langle \\hat n_\\sigma(\\tau)\\hat n_\\sigma(0), \\hat n_\\sigma(\\tau)\\hat n_\\sigma(0)\\rangle-\\langle \\hat n_\\sigma(\\tau)\\hat n_\\sigma(0)\\rangle^2$ with a function $f_\\text{fit}(\\tau)=a+b\\tau+c\\tau^2$\nin the interval $\\tau/\\beta\\in [0: 0.05]$ for the two-orbital Hubbard model. \nThe spin-freezing crossover occurs roughly at the filling $n_\\sigma\\approx 0.28$ (blue curve in the left panel and blue arrow in the right panel). \n}\n\\label{fig:self}\n\\end{center}\n\\end{figure}\n\n\nThe left panel of Fig.~\\ref{fig:self} plots the imaginary part of the self-energy as a function of Matsubara frequency in order to identify the filling corresponding to the spin-freezing crossover \\cite{Werner2008}. An approximate square-root scaling is observed over a wide range of frequencies near $n_\\sigma\\approx 0.28$ (blue line). In the right panel of Fig.~\\ref{fig:self}, we present the coefficient $c$ obtained from a similar analysis as presented in Fig.~\\ref{fig:coeffs}, but with a fitting range $\\tau/\\beta\\in [0 : 0.05]$. Again, we find that the OTOC $\\langle \\hat n_\\sigma(t)\\hat n_\\sigma(0), \\hat n_\\sigma(t)\\hat n_\\sigma(0)\\rangle - \\langle \\hat n_\\sigma\\rangle^4$ detects nontrivial correlations in the spin-freezing crossover (blue arrow) and spin-frozen metal regime, but not in the Fermi liquid or the Mott insulating phase.\n\n\n\\begin{figure}[t]\n\\begin{center}\n\\includegraphics[angle=-90, width=0.49\\columnwidth]{plot_otoc_2orbital_Re.eps}\n\\includegraphics[angle=-90, width=0.486\\columnwidth]{plot_otoc_2orbital_Im.eps}\n\\includegraphics[angle=-90, width=0.484\\columnwidth]{plot_spectra_coth_2orbital_inset.eps}\n\\includegraphics[angle=-90, width=0.486\\columnwidth]{plot_otoc_2orbital_modulus_inset.eps}\n\\caption{\nTop panels: Real and imaginary parts of the OTOC $\\langle \\hat n_\\sigma(t)\\hat n_\\sigma(0), \\hat n_\\sigma(t)\\hat n_\\sigma(0)\\rangle  - \\langle \\hat n_\\sigma\\rangle^4$ for the two-orbital Hubbard model with $U=8$, $J=U/4$, $\\beta=50$ and indicated fillings. \nBottom left panel: Spectral function $\\mathscr A_{(n_\\sigma n_\\sigma)^2}(\\omega)=-\\frac{1}{\\pi}{\\rm Im}\\,C_{(n_\\sigma n_\\sigma)^2}^R(\\omega)$ multiplied by $\\coth\\big(\\frac{\\beta\\omega}{4}\\big)$. In the inset, we plot the imaginary-time four-point function $\\langle \\hat n_\\sigma(\\tau)\\hat n_\\sigma(0), \\hat n_\\sigma(\\tau)\\hat n_\\sigma(0)\\rangle-\\langle \\hat n_\\sigma\\rangle^4$. \nBottom right panel: Modulus of the OTOC $|\\langle \\hat n_\\sigma(t)\\hat n_\\sigma(0), \\hat n_\\sigma(t)\\hat n_\\sigma(0)\\rangle  - \\langle \\hat n_\\sigma\\rangle^4|$ (normalized at $t=0$) on a log-log scale.\nIn the inset of the bottom right panel, we illustrate the approximate data collapse obtained by considering the spin-freezing crossover regimes for different values of $J$. \nIn all the panels, the thick blue curves represent the results for the spin-freezing crossover region. \n}\n\\label{fig:2orbital}\n\\end{center}\n\\end{figure}\n\n\nThe top panels of Fig.~\\ref{fig:2orbital} show the real and imaginary parts of the OTOC\n$\\langle \\hat n_\\sigma(t)\\hat n_\\sigma(0), \\hat n_\\sigma(t)\\hat n_\\sigma(0)\\rangle-\\langle \\hat n_\\sigma\\rangle^4$\nfor the two-orbital Hubbard model. In the spin-frozen phase ($n_\\sigma \\gtrsim 0.3$), both components exhibit \nhighly incoherent oscillations, which get suppressed as the filling is reduced. As one moves across the crossover point ($n_\\sigma \\sim 0.28$),\nthe oscillations completely disappear, and the real part of the OTOC starts to overshoot to the negative side\nin the Fermi liquid regime.\nThe bottom left panel in Fig.~\\ref{fig:2orbital} presents the analytically continued spectral function $\\mathscr A_{(n_\\sigma n_\\sigma)^2}(\\omega)=-\\frac{1}{\\pi}{\\rm Im}\\,C_{(n_\\sigma n_\\sigma)^2}^R(\\omega)$ multiplied by $\\coth\\big(\\frac{\\beta\\omega}{4}\\big)$. \nIn the spin-frozen regime ($n_\\sigma \\gtrsim 0.3$) a sharp peak appears at low energy, which originates from the saturation of the imaginary-time four point function $\\langle \\hat n_\\sigma(\\tau)\\hat n_\\sigma(0), \\hat n_\\sigma(\\tau)\\hat n_\\sigma(0)\\rangle$ at large $\\tau$ (see the inset of the bottom left panel of Fig.~\\ref{fig:2orbital}). This low-energy peak represents the slow dynamics of the frozen spins, and translates into a slow long-time decay of $|\\langle \\hat n_\\sigma(t)\\hat n_\\sigma(0), \\hat n_\\sigma(t)\\hat n_\\sigma(0)\\rangle-\\langle \\hat n_\\sigma\\rangle^4|$,\nas shown in the bottom right panel of Fig.~\\ref{fig:2orbital}\non a log-log scale. Again, it is difficult to clearly resolve the long-time behavior of the OTOC due to the limited\naccuracy of the analytic continuation, but in the spin-freezing crossover regime (thick blue curves)\nthe decay of the OTOC is consistent with a power law $\\sim 1/t^{1.5}$ at least for $2\\lesssim t \\lesssim 5$. \nThe shorter-time behavior ($0.5\\lesssim t \\lesssim 2$) is instead well fitted by an exponential decay $\\sim e^{-\\alpha t}$ with a decay constant $\\alpha=0.78$. \nThis exponent exhibits\na strong dependence on the Hund coupling. \n\nIn fact, the OTOCs in the spin-freezing regime for $J=U/4$ and $J=U/6$ can be approximately collapsed by plotting them as a function of $tJ$ (see the inset in the bottom right panel of Fig.~\\ref{fig:2orbital}), which indicates that the Hund coupling is the parameter which controls the dynamics in this regime. \nThis is distinct from the Lyapunov behavior in which the exponential decay scales with $t/\\beta$ \\cite{Bagrets2017}.\nA possible reason is that we are still far from the large $N$ limit (where $N$ corresponds to the number of orbitals times the number of spin degrees of freedom) so that the large $N$ behavior \nsuch as the Lyapunov growth is not observed in our calculations. \nIn fact, the strong dependence of the time scale on $J$ and weak dependence on $\\beta$ is consistent\nwith the behavior of the finite $N$ SYK model in Ref.~\\cite{FuSachdev2016}.\nIn the spin-frozen regime ($n_\\sigma\\gtrsim 0.3$) the decay is much slower than near the crossover point, while in the Fermi liquid regime the decay is accelerated, approaching the free fermion behavior of $t^{-3}$\nas shown in Fig.~\\ref{analytic continuation}. \nQualitatively similar results for this OTOC are obtained \nfor the three-orbital Hubbard model (see Appendix~\\ref{sec:3orbital}).\n\nThe time dependence of the modulus of \n$\\langle \\hat n_\\sigma(\\tau)\\hat n_\\sigma(0), \\hat n_\\sigma(\\tau)\\hat n_\\sigma(0)\\rangle-\\langle \\hat n_\\sigma\\rangle^4$ \nin the spin-freezing crossover regime of multi-orbital Hubbard models (exponential decay crossing over into a power-law) bears a close resemblance to a (different) OTOC for the SYK model discussed in Ref.~\\cite{Bagrets2017}. \nAlso, a recent study of yet another type of OTOCs for the finite-$N$ SYK model found a qualitatively similar decay as observed here in the spin-freezing crossover regime \\cite{FuSachdev2016}. In the following section, we will make the connection to the SYK dynamics more quantitative.   \n\n\n4.4 Comparison to the SKY model\n\\subsection{Comparison to the SKY model}\n\\label{sec:SYK}\n\nIn this section, we compare the results of the multi-orbital Hubbard models to the SYK model.\nWe consider the particle-hole symmetric SYK model of complex fermions \\cite{FuSachdev2016},\nwhose Hamiltonian is given by\n\\begin{align}\nH\n&=\n\\frac{1}{(2N)^{3/2}} \\sum_{i,j,k,l=1}^N \nJ_{ij;kl} (c_i^\\dagger c_j^\\dagger c_k c_l\n+\\delta_{ik}n c_j^\\dagger c_l\n-\\delta_{il}n c_j^\\dagger c_k\n-\\delta_{jk}n c_i^\\dagger c_l\n+\\delta_{jl}n c_i^\\dagger c_k).\n\\end{align}\nThe coupling constant $J_{ij;kl}$ is a gaussian random variable, satisfying $J_{ij;kl}=-J_{ji;kl}=-J_{ij;lk}=J_{kl;ij}^\\ast$, and\n\\begin{align}\n\\overline{({\\rm Re}\\, J_{ij;kl})^2}\n&=\n\\begin{cases}\nJ_{\\rm SYK}^2/2 & (i,j)\\neq (k,l) \\\\\nJ_{\\rm SYK}^2 & (i,j)=(k,l)\n\\end{cases},\n\\\\\n\\overline{({\\rm Im}\\, J_{ij;kl})^2}\n&=\n\\begin{cases}\nJ_{\\rm SYK}^2/2 & (i,j)\\neq (k,l) \\\\\n0 & (i,j)=(k,l)\n\\end{cases},\n\\end{align}\nwhere the overline represents an average over each realization of $J_{ij;kl}$. The parameter \n$J_{\\rm SYK}$ ($J_{\\rm SYK}^{-1}$) defines the unit of energy (time) in the following calculations. \nThe system is particle-hole symmetric when\n$n=0.5$, which fixes the total number of fermions as $N^{-1}\\sum_{i=1}^N \\langle c_i^\\dagger c_i\\rangle=0.5$.\nWe take the particle-hole symmetric form of the SYK model because it has been well studied previously \\cite{FuSachdev2016} and avoids the effect of the filling drift.\nWe numerically solve the model by exact diagonalization for finite $N(\\le 12)$, and evaluate the density-density OTOC\n$\\overline{\\langle \\hat n_i(t)\\hat n_i(0), \\hat n_i(t)\\hat n_i(0)\\rangle-\\langle \\hat n_i\\rangle^4}$\nand its imaginary-time counterpart.\n\n\\begin{figure}[t]\n\\begin{center}\n\n\\includegraphics[angle=-90,width=0.49\\columnwidth]{plot_fitnsns_syk.eps}\n\n\\includegraphics[angle=-90,width=0.49\\columnwidth]{plot_coeffc_syk.eps}\n\\caption{\nLeft panel: \nThe imaginary-time four-point function $\\overline{\\langle \\hat n_i(\\tau)\\hat n_i(0), \\hat n_i(\\tau)\\hat n_i(0)\\rangle}$, the density-density correlation function $\\overline{\\langle \\hat n_i(\\tau)\\hat n_i(0)\\rangle^2}$,\nand their difference for the SYK model with $N=12$ and $\\beta J_{\\rm SYK}=10$.\nThe dashed curve shows a fit of the difference with a function $f_\\text{fit}(\\tau)=a+b\\tau+c\\tau^2$ in the interval $\\tau/\\beta \\in [0: 0.2]$. \nRight panel: Fitting coefficient $c$ as a function of $T/J_{\\rm SYK}$.\n}\n\\label{fig:SYK fit}\n\\end{center}\n\\end{figure}\n\nIn the left panel of Fig.~\\ref{fig:SYK fit}, we plot the imaginary-time four-point function \n$\\overline{\\langle \\hat n_i(\\tau)\\hat n_i(0), \\hat n_i(\\tau)\\hat n_i(0)\\rangle}$\nfor the SYK model with $N=12$ and $\\beta J_{\\rm SYK}=10$. \nAt low temperature, we expect that $\\overline{\\langle \\hat n_i(\\tau)\\hat n_i(0), \\hat n_i(\\tau)\\hat n_i(0)\\rangle}$ is decoupled into $\\overline{\\langle \\hat n_i(\\tau)\\hat n_i(0)\\rangle^2}$.\nTo see whether nontrivial correlations beyond the decoupled form are present, we fit the difference\n$\\overline{\\langle \\hat n_i(\\tau)\\hat n_i(0), \\hat n_i(\\tau)\\hat n_i(0)\\rangle}\n-\\overline{\\langle \\hat n_i(\\tau)\\hat n_i(0)\\rangle^2}$ with a function $f_{\\rm fit}(\\tau)=a+b\\tau+c\\tau^2$\nin the interval $\\tau/\\beta\\in[0:0.2]$, in the same manner as for the Hubbard models.\nThe obtained coefficient $c$ is plotted as a function of the temperature $T$ in the right panel of\nFig.~\\ref{fig:SYK fit}. One can see that nontrivial correlations exist in a wide range of temperature.\nEspecially, they are enhanced in the intermediate temperature regime. This is because in the zero-temperature limit\nthe four-point function is decoupled as \n$\\langle \\hat n_i(\\tau)\\hat n_i(0), \\hat n_i(\\tau)\\hat n_i(0)\\rangle \\approx \\langle \\hat n_i(\\tau)\\hat n_i(0)\\rangle^2$, while in the high-temperature limit ($\\beta\\to 0$) the imaginary-time dependence is washed out.\n\n\n\n\n\\begin{figure}[t]\n\\begin{center}\n\\includegraphics[angle=-90, width=0.49\\columnwidth]{plot_syk_Re.eps}\n\\includegraphics[angle=-90, width=0.49\\columnwidth]{plot_syk_Im.eps}\n\\includegraphics[angle=-90, width=0.49\\columnwidth]{plot_syk_modulus-log.eps}\n\\includegraphics[angle=-90, width=0.49\\columnwidth]{plot_syk_modulus.eps}\n\\caption{\nThe density-density OTOC $\\overline{\\langle \\hat n_i(t)\\hat n_i(0), \\hat n_i(t)\\hat n_i(0)\\rangle-\\langle \\hat n_i\\rangle^4}$ for the particle-hole symmetric SYK model averaged over $10^2$ samples with $\\beta J_{\\rm SYK}=100$ and various $N$. \nThe top left and right panels show the real and imaginary parts of the OTOC,\nwhile the bottom left and right panels show the log and log-log plots of the modulus of the OTOC (normalized at $t=0$).\nThe dashed line shows a power-law decay $\\propto t^{-1.6}$ for comparison.}\n\\label{fig:SYK}\n\\end{center}\n\\end{figure}\n\nIn Fig.~\\ref{fig:SYK}, we show the numerical results of the density-density OTOC for $\\beta J_{\\rm SYK}=100$.\nThe real part of the OTOC rapidly drops from the initial value within the time scale of $tJ_{\\rm SYK}\\sim 2$,\nand slowly decays to zero in the long time limit.\nAs one increases $N$, the oscillations that appear at longer time tend to be suppressed.\nOne can see that the result for $N=12$ in the top panels of Fig.~\\ref{fig:SYK} closely resembles the OTOC in \nthe spin-freezing crossover regime of the two-orbital Hubbard model shown\nby the blue curve in the top panels of Fig.~\\ref{fig:2orbital},\nwhile it differs from the single-orbital result shown in the top panels of Fig.~\\ref{fig:results_nsns}.\nDue to the limitation of our calculations to small system sizes ($N\\le 12$), we do not clearly observe\nan exponential growth ($\\sim c_0-c_1 e^{\\lambda t}$)  at short times or an exponential decay ($\\sim e^{-\\alpha t}$) \nat intermediate times. \nHowever, we find that for $tJ_{\\rm SYK}\\gtrsim 2$ the OTOC decays\napproximately in a power law $\\propto t^{-\\gamma}$ with $\\gamma=1.6$ (see bottom right panel in Fig.~\\ref{fig:SYK}). \nThe time interval exhibiting the power-law decay becomes longer as we increase $N$.  \nThe temperature dependence of the OTOC for the SYK model is shown in Fig.~\\ref{fig:SYK-T}.\nThe time scale of the initial drop is more or less independent of the temperature, while \nthe power-law-like decay is only visible in the low-temperature regime.\n\n\\begin{figure}[t]\n\\begin{center}\n\\includegraphics[angle=-90, width=0.49\\columnwidth]{plot_syk_beta_re.eps}\n\\includegraphics[angle=-90, width=0.49\\columnwidth]{plot_syk_beta_im.eps}\n\\includegraphics[angle=-90, width=0.49\\columnwidth]{plot_syk_beta_modulus-log.eps}\n\\includegraphics[angle=-90, width=0.49\\columnwidth]{plot_syk_beta_modulus.eps}\n\\caption{\nTemperature dependence of the density-density OTOC\n$\\overline{\\langle \\hat n_i(t)\\hat n_i(0), \\hat n_i(t)\\hat n_i(0)\\rangle-\\langle \\hat n_i\\rangle^4}$ for the particle-hole symmetric SYK model averaged over $10^2$ samples with $N=12$.\nThe top left and right panels show the real and imaginary parts of the OTOC,\nwhile the bottom left and right panels show the log and log-log plots of the modulus of the OTOC.\nThe dashed line shows a power-law decay $\\propto t^{-1.6}$ for comparison.}\n\\label{fig:SYK-T}\n\\end{center}\n\\end{figure}\n\nIn Ref.~\\onlinecite{Werner2018}, it has been argued that the spin-freezing crossover regime of multi-orbital Hubbard models is effectively described by the SYK model in which $J_{\\rm SYK}$ is replaced by the Hund coupling $J$. The results in Fig.~\\ref{fig:2orbital} are for the two-orbital Hubbard model with $U=8$, $J/U=1/4$ and $\\beta=50$, corresponding to $\\beta J=100$, which means that it is meaningful to directly compare the blue lines in Fig.~\\ref{fig:2orbital} with the blue lines in Figs.~\\ref{fig:SYK} and \\ref{fig:SYK-T}. Not only is the qualitative agreement remarkable, even the power-law exponent measured for the two-orbital Hubbard model ($\\gamma=1.5$) is very close to the exponent extracted from the finite-$N$ SYK calculation. \nThe power-law decay in the three-orbital case is approximately $1/t^{1.75}$ (see Appendix~\\ref{sec:3orbital}), and thus also in semi-quantitative agreement with the SYK model behavior. \nThis nontrivial result provides further support for the identification of the spin-freezing crossover regime of multi-orbital Hubbard systems with an SYK strange metal. \n\nIn the large-$N$ and low-$T$ limit of the SYK model, we expect that the power-law behavior approaches $1/t^2$, since in this limit the OTOC is decoupled as $\\langle \\hat n_i(t)\\hat n_i(0), \\hat n_i(t)\\hat n_i(0)\\rangle\\approx \\langle \\hat n_i(t)\\hat n_i(0)\\rangle^2$ and the decay of $\\langle \\hat n_i(t)\\hat n_i(0)\\rangle$ is dominated by what corresponds to the slow spin relaxation $\\langle \\hat S_z(t)\\hat S_z(0)\\rangle\\sim 1/t$ in the Sachdev-Ye model.\n\n\n\n\n\n",
        "label": "fig:2orbital",
        "Descriptive_question1": "What is the value of U used in the two-orbital Hubbard model shown in figure_10?",
        "Descriptive_question2": "What color represents the spin-freezing crossover region in the panels of figure_10?",
        "Reasoning_question1": "How does the decay behavior of the OTOC modulus in figure_10 vary across different filling values in the two-orbital Hubbard model, and what does this suggest about the dynamics in the spin-freezing crossover regime compared to the Fermi liquid regime?",
        "Reasoning_question2": "What can be inferred about the role of Hund coupling J in controlling the dynamics of the OTOC in the spin-freezing regime based on the data collapse shown in the inset of the bottom right panel of figure_10?",
        "Descriptive_answer1": "U=8",
        "Descriptive_answer2": "blue",
        "Reasoning_answer1": "The decay behavior of the OTOC modulus $|\\langle \\hat n_\\sigma(t)\\hat n_\\sigma(0), \\hat n_\\sigma(t)\\hat n_\\sigma(0)\\rangle - \\langle \\hat n_\\sigma \\rangle^4|$ varies with filling in the two-orbital Hubbard model. In the spin-frozen regime (fillings $n_\\sigma \\gtrsim 0.3$), the modulus displays highly incoherent oscillations and a slower decay, consistent with a power-law decay approximately $1/t^{1.5}$ for intermediate times. Near the spin-freezing crossover filling ($n_\\sigma \\sim 0.28$), the decay shows an initial exponential decay transitioning into a power law. In the Fermi liquid regime (lower fillings), the decay accelerates and approaches a faster decay similar to the free fermion case (e.g., $t^{-3}$). This suggests that in the spin-freezing crossover regime, the dynamics are characterized by slow spin fluctuations and moment formation leading to slower OTOC decay, reflecting a non-Fermi-liquid, incoherent metal. In contrast, the Fermi liquid regime shows faster decay characteristic of coherent quasiparticle behavior.",
        "Reasoning_answer2": "The approximate data collapse of the OTOC in the spin-freezing crossover regimes for different values of Hund coupling $J$ (seen in the inset of the bottom right panel) indicates that the Hund coupling sets the natural time scale for the OTOC dynamics in this regime. Specifically, when plotted as a function of $tJ$, the OTOC curves for different $J$ values overlap, implying that the decay rates and time scales of the OTOC are controlled by $J$. This supports the interpretation that the Hund coupling governs the slow spin dynamics and scrambling processes in the spin-freezing crossover, rather than temperature or other energy scales. This behavior differs from the Lyapunov-type scaling ($t/\\beta$) and aligns with finite-$N$ SYK model behavior where coupling sets the dynamics scale."
    },
    {
        "paper_id": "1812.04217.json",
        "image_id": "figure_11",
        "image_path": [
            "/home/yz979/scratch/chengye/scimolmo/test-images-tables/dataset_imagestables/1812.04217/images/plot_fitnsns_syk.eps"
        ],
        "caption": "\nLeft panel: \nThe imaginary-time four-point function $\\overline{\\langle \\hat n_i(\\tau)\\hat n_i(0), \\hat n_i(\\tau)\\hat n_i(0)\\rangle}$, the density-density correlation function $\\overline{\\langle \\hat n_i(\\tau)\\hat n_i(0)\\rangle^2}$,\nand their difference for the SYK model with $N=12$ and $\\beta J_{\\rm SYK}=10$.\nThe dashed curve shows a fit of the difference with a function $f_\\text{fit}(\\tau)=a+b\\tau+c\\tau^2$ in the interval $\\tau/\\beta \\in [0: 0.2]$. \nRight panel: Fitting coefficient $c$ as a function of $T/J_{\\rm SYK}$.\n",
        "classify": "Chart",
        "section_info": "4 Numerical results for the Hubbard model\n\\section{Numerical results for the Hubbard model}\n\\label{numerical results}\n\n\\subsection{Observables and numerical procedure}\n\nWe consider the single-orbital, two-orbital, and three-orbital Hubbard models on an infinitely-connected Bethe lattice, which can be solved exactly within DMFT \\cite{GeorgesKotliarKrauthRozenberg1996}. In this case, the noninteracting density of states is semicircular with bandwidth $4v_\\ast$, for which there exists a simplified DMFT self-consistency condition between the hybridization function $\\Delta_{\\alpha\\sigma}$ of the DMFT impurity problem and the local (impurity) Green's function $G_{\\alpha\\sigma}$: \n\n\\begin{equation}\n\\Delta_{\\alpha\\sigma}(\\tau)=v_\\ast^2 G_{\\alpha\\sigma}(\\tau), \n\\label{semicirc}\n\\end{equation}\n\nwith $\\alpha$ the orbital and $\\sigma$ the spin index. We will use $v_\\ast$ as the unit of energy and measure time in units of $\\hbar/v_\\ast$. \n\nThree different types of imaginary-time four-point functions $C_{(AB)^2}^M(\\tau)$\nof the form of Eq.~(\\ref{imaginary-time OTOC})\nwith $(\\hat A, \\hat B)=(c_\\sigma^\\dagger, c_\\sigma), (\\hat n_\\sigma, \\hat n_\\sigma)$, and $(\\hat n, \\hat n)$\nare calculated in the interval $0\\le \\tau\\le \\frac{\\beta\\hbar}{2}$: \n\\begin{eqnarray}\nC_{(c^\\dagger_\\sigma c_\\sigma)^2}^M(\\tau) &=& -\\langle c^\\dagger_\\sigma(\\tau+\\tfrac{\\beta\\hbar}{2}) c_\\sigma(\\tfrac{\\beta\\hbar}{2}) c^\\dagger_\\sigma(\\tau) c_\\sigma(0) \\rangle, \\label{fermion_otoc}\\\\\nC_{(n_\\sigma n_\\sigma)^2}^M(\\tau) &=& -\\langle \\hat n_\\sigma(\\tau+\\tfrac{\\beta\\hbar}{2})\\hat n_\\sigma(\\tfrac{\\beta\\hbar}{2})\\hat n_\\sigma(\\tau)\\hat n_\\sigma(0) \\rangle, \\label{otoc_nsns}\\\\\nC_{(nn)^2}^M(\\tau) &=& -\\langle \\hat n(\\tau+\\tfrac{\\beta\\hbar}{2})\\hat n(\\tfrac{\\beta\\hbar}{2})\\hat n(\\tau)\\hat n(0) \\rangle,\\label{otoc_nn}\n\\end{eqnarray}\n\nwhere $c_\\sigma^\\dagger$ ($c_\\sigma$) are the fermionic creation (annihilation) operators for spin $\\sigma$ (and orbital $\\alpha=1$ in the multi-orbital case), $\\hat n_\\sigma=c^\\dagger_\\sigma c_\\sigma$ is the corresponding spin-dependent density operator, and $\\hat n=\\hat n_\\uparrow + \\hat n_\\downarrow$ the total density operator. Note that in all the three cases above we have $\\hat B=\\hat A^\\dagger$. \nWe measure these local correlation functions in the impurity model using a hybridization expansion continuous-time Monte Carlo algorithm (CT-HYB) \\cite{Werner2006}. In this algorithm, the \ntwo-particle\nGreen's functions of the type (\\ref{fermion_otoc}) can be measured by removing two hybridization lines, i.e., from the elements of the inverse hybridization matrix, while the density-density correlation functions can be easily measured either by insertion of density operators (matrix formalism) \\cite{Werner2006Kondo} or by reading off the occupation of the orbitals at the four time points in the segment implementation \\cite{Werner2006}. We use the latter algorithm since we consider only density-density interactions. \nThe values at the end-points $\\tau=0$ and $\\tau=\\frac{\\beta\\hbar}{2}$ reduce to standard density-density correlation functions, which in the case of Eq.~(\\ref{fermion_otoc}) are measured separately. \n\nWe perform the analytic continuation to the real-frequency axis using the Maximum Entropy method \\cite{Jarrell1996,Lewin_maxent} with a bosonic kernel. This yields the imaginary part of the retarded correlator \n$-\\frac{1}{\\pi}\\text{Im}\\, C^R_{(AA^\\dagger)^2}(\\omega)=\\mathscr A_{(AA^\\dagger)^2}(\\omega)$.  \nWhen $\\hat B=\\hat A^\\dagger=\\hat A$ (which is the case for $\\hat A=\\hat n_\\sigma, \\hat n$), we have $C_{(AA)^2}^R(\\omega)^\\ast=C_{(AA)^2}^R(-\\omega)$. At half filling, the particle-hole symmetry furthermore ensures $C_{c_\\sigma^\\dagger c_\\sigma}^R(\\omega)^\\ast=C_{c_\\sigma^\\dagger c_\\sigma}^R(-\\omega)$.\nThus, in all the cases considered in this study, the retarded OTOC has the symmetry property $C_{(AA^\\dagger)^2}^R(\\omega)^\\ast=C_{(AA^\\dagger)^2}^R(-\\omega)$. Using this, as well as the out-of-time-order fluctuation-dissipation\ntheorem discussed in the previous section, we can obtain the real-time OTOC\n$\\langle \\hat A(t)\\hat A^\\dagger(0), \\hat A(t)\\hat A^\\dagger(0)\\rangle$ from the following inverse Fourier transformation, \n\\begin{align}{\\rm Re}\\langle \\hat A(t)\\hat A^\\dagger(0), \\hat A(t)\\hat A^\\dagger(0)\\rangle\n&=\n{\\rm Re}\n\\int_0^\\infty d\\omega\\, e^{-i\\omega t}\\coth\\left(\\frac{\\beta\\hbar\\omega}{4}\\right)\n\\left(-\\frac{1}{\\pi}{\\rm Im}\\, C_{(AA^\\dagger)^2}^R(\\omega)\\right),\n\\\\\n{\\rm Im}\\langle \\hat A(t)\\hat A^\\dagger(0), \\hat A(t)\\hat A^\\dagger(0)\\rangle\n&=\n{\\rm Im}\\int_0^{\\infty} d\\omega\\, e^{-i\\omega t} \\left(-\\frac{1}{\\pi}{\\rm Im}\\, C_{(AA^\\dagger)^2}^R(\\omega)\\right),\n\\end{align}\nfor $\\hat A=c_\\sigma^\\dagger, \\hat n_\\sigma$, and $\\hat n$.\n\n\\begin{figure}[b]\n\\begin{center}\n\\includegraphics[angle=-90, width=0.48\\columnwidth]{compare_u0_draft.eps}\\hfill\n\\includegraphics[angle=-90, width=0.48\\columnwidth]{compare_u0_longtime_modulus.eps}\n\\caption{\nLeft panel: Comparison between the exact solution and the result obtained by analytical continuation\nfor the real and imaginary parts of the OTOC $\\langle c_\\sigma^\\dagger(t)c_\\sigma(0), c_\\sigma^\\dagger(t)c_\\sigma(0)\\rangle$ \nin the noninteracting case ($U=0, \\beta\\hbar=50$) of the single-orbital Hubbard model.\nRight panel: Comparison of the long-time behavior for the modulus $|\\langle c_\\sigma^\\dagger(t)c_\\sigma(0), c_\\sigma^\\dagger(t)c_\\sigma(0)\\rangle|$. \nThe exact result shows a power-law decay ($\\sim 1/t^3$). \n}\n\\label{fig:test_u0}\n\\end{center}\n\\end{figure}\n\nAs a check of our procedure, we first compare the results for the noninteracting model,\nfor which an exact solution of the OTOC $\\langle c_\\sigma^\\dagger(t)c_\\sigma(0), c_\\sigma^\\dagger(t)c_\\sigma(0)\\rangle$ \nis available \\cite{TsujiWernerUeda2017}.\nIt is a nontrivial task to measure the noninteracting two-particle Green's function in CT-HYB, \nand to analytically continue \n$C_{(c^\\dagger_\\sigma c_\\sigma)^2}^M(\\tau)$ \nby the Maximum Entropy method.  \nThe results for the inverse temperature $\\beta\\hbar=50$ are plotted in Fig.~\\ref{fig:test_u0}. \nOne can see that the OTOC oscillates for about one cycle, and quickly decays to zero.\nThe analytically continued data show a good agreement with the exact solution. \nIn particular, for short times (up to about two inverse hoppings) the dynamics is accurately reproduced, while deviations appear at longer times. \nIt is known that for the noninteracting fermion system the OTOC decays as a power law at long time ($\\sim 1/t^3$) \\cite{TsujiWernerUeda2017}.\nThe analytical continuation method cannot reproduce the details of the oscillations at intermediate or long times, but it roughly captures the long-time decay, which is controlled by low-frequency spectral features. \nUsually, analytic continuation is unreliable for high frequency components, because high-frequency information is suppressed \nby the kernel $K(\\tau,\\omega)$ in the transformation from the spectral function $A_O(\\omega)$ to the (bosonic) Matsubara correlation function $O(\\tau)$ [$O(\\tau)=\\int_{-\\infty}^{\\infty} d\\omega\\, K(\\omega,\\tau) A_O(\\omega)$ with $K(\\tau,\\omega)=e^{-\\tau\\omega}/(1-e^{-\\beta\\omega})$]. On the other hand, the low frequency components can be rather accurately determined. As a result, in Fig. 4 we more or less recover the low-frequency features of the time-dependent correlation function, while the details of rapid oscillations are not captured. The accurate result at short times can be understood from the fact that this region is close to the imaginary time axis in the complex time plane, where the original QMC data are available.\nAt higher temperatures, the Maximum Entropy method becomes less reliable, so that the resulting OTOCs are expected to be less accurate.  \n\n\n\\subsection{Single-orbital Hubbard model}\n\\label{sec:hubbard}\n\nIn this section, we calculate the OTOCs (\\ref{fermion_otoc})-(\\ref{otoc_nn}) for the interacting single-orbital Hubbard model with the Hamiltonian \n\\begin{align}\nH\n&=\n-v\\sum_{\\langle i,j\\rangle\\sigma} (c^\\dagger_{i\\sigma}c_{j\\sigma}+ \\text{h.c.})\n-\\mu\\sum_i \\hat n_i+U\\sum_i \\hat n_{i\\uparrow}\\hat n_{i\\downarrow}, \n\\end{align}\nwhere $v$ is the hopping amplitude, $\\langle i,j\\rangle$ represents the nearest-neighbor site pairs, \n$\\mu$ is the chemical potential, and $U$ is the on-site interaction.\nIn this section, we focus on the half-filled case (i.e., $\\mu=U/2$). The DMFT solution for the simplified self-consistency (\\ref{semicirc}) is the exact solution for an infinitely connected Bethe lattice \\cite{GeorgesKotliarKrauthRozenberg1996}.\nLet us briefly recall the paramagnetic phase diagram for the single-orbital Hubbard model on this lattice \\cite{Bluemer_thesis}. At half-filling, there is a metal-insulator crossover at high temperature and a first-order Mott transition below a temperature corresponding to $\\beta\\hbar\\approx 17$ with a coexistence region between $U_{c1}$ and $U_{c2}$. The finite-temperature critical endpoint is at $U\\approx 4.7$, while the zero-temperature Mott transition occurs at $U=U_{c2}\\approx 5.6$. \nAt low temperature, the self-energy shows the Fermi liquid behavior [${\\rm Im}\\,\\Sigma(\\omega)\\sim \\omega^2$] \nin the weakly correlated metallic phase, \nwhile it shows an insulating behavior [${\\rm Im}\\,\\Sigma(\\omega)\\sim \\delta(\\omega)$] in the Mott phase\n\\cite{GeorgesKotliarKrauthRozenberg1996}. \nIn the correlated metallic phase, as the temperature is increased, deviations from the Fermi liquid behavior become apparent, \nbut an ${\\rm Im}\\,\\Sigma(\\omega)\\sim \\sqrt{\\omega}$ scaling as in the SYK model is not observed. \n\nIn the following, we will start the DMFT iterations from the noninteracting solution, which means that the finite-temperature Mott transition occurs at $U_{c2}$.\nFrom now on, we set $\\hbar=1$. \n\n\\begin{figure}[t]\n\\begin{center}\n\\includegraphics[angle=-90, width=0.49\\columnwidth]{plot_otoc_b50_Re.eps}\n\\includegraphics[angle=-90, width=0.49\\columnwidth]{plot_otoc_b50_Im.eps}\n\\includegraphics[angle=-90, width=0.49\\columnwidth]{plot_coth_b50_appendix.eps}\n\\includegraphics[angle=-90, width=0.49\\columnwidth]{plot_otoc_b50_modulus.eps}\n\\caption{\nTop panels: Real and imaginary parts of the OTOC $\\langle c^\\dagger_\\sigma(t)c_\\sigma(0), c^\\dagger_\\sigma(t)c_\\sigma(0)\\rangle$ for the single-orbital Hubbard model with $\\beta=50$ at half filling. The thin lines correspond to metallic solutions, while the thick line ($U=6$) corresponds to a Mott insulating solution. \nBottom left panel: Spectral function $\\mathscr A_{(c_\\sigma^\\dagger c_\\sigma)^2}(\\omega)=-\\frac{1}{\\pi}{\\rm Im}\\,C_{(c^\\dagger_\\sigma c_\\sigma)^2}^R(\\omega)$ multiplied by $\\coth\\big(\\frac{\\beta\\omega}{4}\\big)$. \nBottom right panel: Modulus of the OTOC $|\\langle c^\\dagger_\\sigma(t)c_\\sigma(0), c^\\dagger_\\sigma(t)c_\\sigma(0)\\rangle|$ (normalized at $t=0$) on a log-log scale. The dashed line is the result for $U=0$.\n}\n\\label{fig:results_fermion}\n\\end{center}\n\\end{figure}\n\n\n\nIn Fig.~\\ref{fig:results_fermion}, we show the results of $\\langle c^\\dagger_\\sigma(t)c_\\sigma(0), c^\\dagger_\\sigma(t)c_\\sigma(0)\\rangle$ for the interacting single-orbital Hubbard model at $\\beta=50$. \nThe top panels present the real and imaginary parts of the OTOC. One can see that the oscillations become more pronounced as one increases the interaction. In the bottom left panel of Fig.~\\ref{fig:results_fermion},\nwe show the corresponding spectral function $\\mathscr A_{(c_\\sigma^\\dagger c_\\sigma)^2}(\\omega)=-\\frac{1}{\\pi}{\\rm Im}\\,C_{(c^\\dagger_\\sigma c_\\sigma)^2}^R(\\omega)$ multiplied by $\\coth\\big(\\frac{\\beta\\omega}{4}\\big)$.\nThe Mott transition, which occurs between $U=5$ and $U=6$ at $\\beta=50$, manifests itself by the opening of a gap in the spectral function $\\mathscr A_{(c_\\sigma^\\dagger c_\\sigma)^2(\\omega)}$. This translates into more weakly damped oscillations in the real-time evolution in the Mott phase.  \nThe bottom right panel of Fig.~\\ref{fig:results_fermion} plots the modulus of the OTOC (normalized at time $t=0$, i.e., $|\\langle c^\\dagger_\\sigma(t)c_\\sigma(0), c^\\dagger_\\sigma(t)c_\\sigma(0)\\rangle|/|\\langle c^\\dagger_\\sigma(0)c_\\sigma(0), c^\\dagger_\\sigma(0)c_\\sigma(0)\\rangle|$) on a log-log scale. \nDue to the oscillations and the limited accuracy of the analytic continuation procedure, it is hard to clearly identify the nature of the long-time decay of the OTOC,\nbut the results indicate that the OTOC decays much faster (possibly exponentially) in the Mott phase than in the metallic phase. This is consistent with the results for the spectral functions of the OTOC. The comparison to the noninteracting case (dashed curve in the bottom right panel of Fig.~\\ref{fig:results_fermion}) suggests that the correlated metallic phase has a similar (possibly power-law) decay behavior of the OTOC as in the noninteracting case.\n\n\n\\begin{figure}[t]\n\\begin{center}\n\\includegraphics[angle=-90, width=0.49\\columnwidth]{plot_otoc_b50_nn_Re.eps}\n\\includegraphics[angle=-90, width=0.49\\columnwidth]{plot_otoc_b50_nn_Im.eps}\n\\includegraphics[angle=-90, width=0.49\\columnwidth]{plot_coth_b50_nn_appendix.eps}\n\\includegraphics[angle=-90, width=0.49\\columnwidth]{plot_otoc_b50_nn_modulus.eps}\n\\caption{\nTop panels: Real and imaginary parts of the OTOC $\\langle \\hat n(t)\\hat n(0), \\hat n(t)\\hat n(0)\\rangle - 1$ for the single-orbital Hubbard model with $\\beta=50$ at half filling. The thin lines correspond to metallic solutions, \nwhile the thick line ($U=6$) corresponds to a Mott insulating solution. \nBottom left panel: Spectral function $\\mathscr A_{(nn)^2}(\\omega)=-\\frac{1}{\\pi}{\\rm Im}\\,C_{(n n)^2}^R(\\omega)$ multiplied by $\\coth\\big(\\frac{\\beta\\omega}{4}\\big)$.\nBottom right panel: Modulus of the OTOC $|\\langle \\hat n(t)\\hat n(0), \\hat n(t)\\hat n(0)\\rangle - 1|$ (normalized at $t=0$) on a log-log scale. \n}\n\\label{fig:results_nn}\n\\end{center}\n\\end{figure}\n\n\nIn Fig.~\\ref{fig:results_nn}, we plot the OTOC $\\langle \\hat n(t)\\hat n(0), \\hat n(t)\\hat n(0)\\rangle$\nfor the single-orbital Hubbard model in a similar manner as in Fig.~\\ref{fig:results_fermion}.\nSince at half filling this correlation function approaches $\\langle \\hat n\\rangle^4=1$ at long times\nand at sufficiently low temperature,\nwe perform the analytical continuation procedure for $\\langle \\hat n(t)\\hat n(0), \\hat n(t)\\hat n(0)\\rangle-1$. \nThe top panels of Fig.~\\ref{fig:results_nn} show the real and imaginary parts of this shifted OTOC. \nContrary to the case of $\\langle c_\\sigma^\\dagger(t)c_\\sigma(0), c_\\sigma^\\dagger(t)c_\\sigma(0)\\rangle$, \nthe amplitude of the OTOC $\\langle \\hat n(t)\\hat n(0), \\hat n(t)\\hat n(0)\\rangle-1$ is suppressed as one increases the interaction, which reflects the reduced charge fluctuations in the Mott insulator.\nThe bottom left panel in Fig.~\\ref{fig:results_nn} shows the analytically continued spectral function $\\mathscr A_{(nn)^2}(\\omega)=-\\frac{1}{\\pi}{\\rm Im}\\, C_{(nn)^2}^R(\\omega)$ multiplied by $\\coth\\big(\\frac{\\beta\\omega}{4}\\big)$. \nAgain we notice the opening of a gap in the Mott phase, and a corresponding shift of the spectral weight to higher energies. This results in more rapid oscillations of the OTOC in the Mott state.\nThe bottom right panel in Fig.~\\ref{fig:results_nn} plots the modulus of the OTOC $|\\langle \\hat n(t)\\hat n(0), \\hat n(t)\\hat n(0)\\rangle-1|$ (normalized at $t=0$) on a log-log scale.\nThe long-time behavior of the OTOC is consistent with a power-law decay in the metallic phase, and an exponential decay in the Mott insulating phase.\n\n\n\\begin{figure}[t]\n\\begin{center}\n\\includegraphics[angle=-90, width=0.49\\columnwidth]{plot_otoc_nsns_b50_draft_Re.eps}\n\\includegraphics[angle=-90, width=0.49\\columnwidth]{plot_otoc_nsns_b50_draft_Im.eps}\n\\includegraphics[angle=-90, width=0.49\\columnwidth]{plot_coth_b50_nsns_inset.eps}\n\\includegraphics[angle=-90, width=0.49\\columnwidth]{plot_otoc_nsns_b50_draft_modulus_zoom.eps}\n\\caption{\nTop panels: Real and imaginary parts of the OTOC $\\langle \\hat n_\\sigma(t)\\hat n_\\sigma(0), \\hat n_\\sigma(t)\\hat n_\\sigma(0)\\rangle  - \\langle \\hat n_\\sigma\\rangle^4$ for the single-orbital Hubbard model with $\\beta=50$ at half filling. All the colored lines correspond to metallic solutions. \nBottom left panel: Spectral function $\\mathscr A_{(n_\\sigma n_\\sigma)^2}(\\omega)=-\\frac{1}{\\pi}{\\rm Im}\\,C_{(n_\\sigma n_\\sigma)^2}^R(\\omega)$ multiplied by $\\coth\\big(\\frac{\\beta\\omega}{4}\\big)$. In the inset, we plot the imaginary-time four-point function $\\langle \\hat n_\\sigma(\\tau)\\hat n_\\sigma(0), \\hat n_\\sigma(\\tau)\\hat n_\\sigma(0)\\rangle-\\langle \\hat n_\\sigma\\rangle^4$, with the black line corresponding to the Mott insulating solution at $U=6$.  \nBottom right panel: Modulus of the OTOC $|\\langle \\hat n_\\sigma(t)\\hat n_\\sigma(0), \\hat n_\\sigma(t)\\hat n_\\sigma(0)\\rangle  - \\langle \\hat n_\\sigma\\rangle^4|$ (normalized at $t=0$) on a log-log scale.\n}\n\\label{fig:results_nsns}\n\\end{center}\n\\end{figure}\n\nFigure~\\ref{fig:results_nsns} \nshows the results of the OTOC $\\langle \\hat n_\\sigma(t)\\hat n_\\sigma(0), \\hat n_\\sigma(t)\\hat n_\\sigma(0)\\rangle$\nfor the single-orbital Hubbard model. At half filling, this correlation function approaches $\\langle \\hat n_\\sigma\\rangle^4=\\frac{1}{16}$ in the long-time limit and at sufficiently low temperature, \nso that we perform the analytical continuation procedure for $\\langle \\hat n_\\sigma(t)\\hat n_\\sigma(0), \\hat n_\\sigma(t)\\hat n_\\sigma(0)\\rangle-\\langle \\hat n_\\sigma\\rangle^4$. \nThe top panels of Fig.~\\ref{fig:results_nsns} show the real and imaginary parts of \nthis shifted OTOC. \nHere, we only present results for the metallic phase, because resolving the very sharp low-energy feature of the OTOC spectrum in the half-filled Mott state is challenging. One can see that coherent oscillations are not observed for this type of OTOC, and that the incoherent part becomes dominant compared to the other two OTOCs shown in Figs.~\\ref{fig:results_fermion} and \\ref{fig:results_nn}. The amplitude of the OTOC $\\langle \\hat n_\\sigma(t)\\hat n_\\sigma(0), \\hat n_\\sigma(t)\\hat n_\\sigma(0)\\rangle-\\langle \\hat n_\\sigma\\rangle^4$ is enhanced as $U$ increases. As we argue below, this is related to the enhancement of spin fluctuations \nand local moment formation close to the Mott transition.\n\nThe bottom left panel in Fig.~\\ref{fig:results_nsns} plots the analytically continued spectral function $\\mathscr A_{(n_\\sigma n_\\sigma)^2}(\\omega)=-\\frac{1}{\\pi}{\\rm Im}\\, C_{(n_\\sigma n_\\sigma)^2}^R(\\omega)$ multiplied by $\\coth\\big(\\frac{\\beta\\omega}{4}\\big)$. \nWe can see that the spectral weight is concentrated in the low-energy region as we approach the Mott transition point.\nThe inset in the bottom left panel of Fig.~\\ref{fig:results_nsns} shows the imaginary-time four-point function $\\langle \\hat n_\\sigma(\\tau)\\hat n_\\sigma(0), \\hat n_\\sigma(\\tau)\\hat n_\\sigma(0)\\rangle-\\langle \\hat n_\\sigma\\rangle^4$, with the black line corresponding to a Mott insulating solution ($U=6$).  \nFor $U\\gtrsim 4$, the imaginary-time four-point function does not decay to zero but remains relatively large near $\\tau=\\frac{\\beta}{4}$. \nThis behavior is reminiscent of the spin-freezing physics seen in the correlated Hund metal phase of multi-orbital Hubbard models, where the dynamical spin correlation function $\\langle \\hat S_z(\\tau)\\hat S_z(0)\\rangle$ is trapped at a finite value at long $\\tau$ \\cite{Werner2008}. \nIn fact, the OTOC $\\langle \\hat n_\\sigma(\\tau)\\hat n_\\sigma(0), \\hat n_\\sigma(\\tau)\\hat n_\\sigma(0)\\rangle$\nmeasures spin correlations (in addition to charge correlations) through $\\hat n_\\sigma=\\frac{1}{2}\\hat n+\\sigma\\hat S_z$.\nPhysically, the trapping of $\\langle \\hat S_z(\\tau)\\hat S_z(0)\\rangle$ signals the formation of frozen local magnetic moments.\nIn the metal-insulator crossover region, the scattering induced by these magnetic moments results in incoherent metal states. Intuitively, one might expect fast scrambling in the corresponding regions of the phase diagram. \nAlthough there is some resemblance with the spin freezing crossover,\nthe self-energy does not exhibit a square-root frequency dependence in the single-orbital case, and a spin-freezing crossover in the sense of Ref.~\\cite{Werner2008} does not exist.  \n\nThe bottom right panel of Fig.~\\ref{fig:results_nsns} plots the modulus of the OTOC $|\\langle \\hat n_\\sigma(t)\\hat n_\\sigma(0), \\hat n_\\sigma(t)\\hat n_\\sigma(0)\\rangle-\\langle \\hat n_\\sigma\\rangle^4|$ (normalized at $t=0$) on a log-log scale. \nThe dynamics of $\\langle \\hat n_\\sigma(t)\\hat n_\\sigma(0), \\hat n_\\sigma(t)\\hat n_\\sigma(0)\\rangle$ is quite distinct from that of $\\langle \\hat n(t)\\hat n(0), \\hat n(t)\\hat n(0)\\rangle$. As the Mott transition is approached, we observe a slow-down in the decay of the modulus, \nwhich indicates the presence of slowly fluctuating local moments. \nThe long-time behavior may be consistent with a power-law, although slow oscillations make it difficult to determine the long-time asymptotic form from the numerics. \n\n\n\\begin{figure}[t]\n\\begin{center}\n\\includegraphics[angle=-90, width=0.49\\columnwidth]{plot_fitnsns_b2.eps}\n\\includegraphics[angle=-90, width=0.49\\columnwidth]{plot_coeffc_draft.eps}\n\\caption{\nLeft panel: \nThe imaginary-time four-point function $\\langle \\hat n_\\sigma(\\tau)\\hat n_\\sigma(0), \\hat n_\\sigma(\\tau)\\hat n_\\sigma(0)\\rangle$, the squared density-density correlation function $\\langle \\hat n_\\sigma(\\tau)\\hat n_\\sigma(0)\\rangle^2$,\nand their difference for the single-orbital Hubbard model with $U=5.5$ and $\\beta=50$.\nThe dashed curve shows a fit of the difference with the function $f_\\text{fit}(\\tau)=a+b\\tau+c\\tau^2$ in the interval $\\tau/\\beta \\in [0: 0.2]$.\nRight panel: Fitting coefficient $c$ as a function of $U$ for indicated values of $\\beta$.\n\n}\n\\label{fig:coeffs}\n\\end{center}\n\\end{figure}\n\nFor large enough $\\beta$ and small enough $\\tau$, the OTOC (\\ref{otoc_nsns}) factorizes into $\\langle \\hat n_\\sigma(\\tau)\\hat n_\\sigma(0), \\hat n_\\sigma(\\tau)\\hat n_\\sigma(0)\\rangle \\approx \\langle \\hat n_\\sigma(\\tau)\\hat n_\\sigma(0)\\rangle^2$. \nOn the real-time axis, this corresponds to the decoupling $\\langle \\hat n_\\sigma(t)\\hat n_\\sigma(0), \\hat n_\\sigma(t)\\hat n_\\sigma(0)\\rangle \\approx \\langle \\hat n_\\sigma(t)\\hat n_\\sigma(0)\\rangle^2$, that is, \nthe OTOC is reduced to a product of ordinary density-density correlation functions.\nIn order to see whether nontrivial correlations are captured beyond the decoupled form by the OTOC function, \nwe consider the difference $\\langle \\hat n_\\sigma(\\tau)\\hat n_\\sigma(0), \\hat n_\\sigma(\\tau)\\hat n_\\sigma(0)\\rangle-\\langle \\hat n_\\sigma(\\tau)\\hat n_\\sigma(0)\\rangle^2$ and fit the short-time behavior of this function to $f_\\text{fit}(\\tau)=a+b\\tau+c\\tau^2$, as illustrated in the left panel of Fig.~\\ref{fig:coeffs}. The coefficients $b$ and $c$ serve as  indicators of the nontrivial correlations that cannot be attributed to the decoupled form of the OTOC.\nThe coefficient $c$ is plotted as a function of $U$ and for different $\\beta$ in the right hand panel. The coefficient $b$ (not shown) exhibits a similar trend. We notice that the OTOC picks up nontrival correlations near the metal-insulator transition, and in particular at \nintermediate\ntemperatures, while the Mott phase shows no such correlations. The crossover region  \nassociated with the emergence of local moments \ncorresponds, roughly, to the interaction range where the nontrivial correlations start to become significant. \n\n\n\\subsection{Two-orbital Hubbard model}\n\\label{sec:two-orbital hubbard}\n\nIn this section, we investigate OTOCs in the two-orbital Hubbard model with Hund coupling $J>0$ \nand density-density interactions. (Three-orbital results are presented in Appendix~\\ref{sec:3orbital}.) \nThe Hamiltonian is given by\n\\begin{align}\nH\n&=\n-v\\sum_{\\langle i,j\\rangle\\alpha\\sigma} (c^\\dagger_{i\\alpha\\sigma}c_{j\\alpha\\sigma}+ \\text{h.c.})\n-\\mu\\sum_{i\\alpha\\sigma} \\hat n_{i\\alpha\\sigma}\n+U\\sum_{i\\alpha} \\hat n_{i\\alpha\\uparrow}\\hat n_{i\\alpha\\downarrow}\n+(U-2J)\\sum_{i\\sigma} \\hat n_{i1\\sigma}\\hat n_{i2\\bar\\sigma}\n\\notag\n\\\\\n&\\quad\n+(U-3J)\\sum_{i\\sigma} \\hat n_{i1\\sigma}\\hat n_{i2\\sigma},\n\\end{align}\n\nwith $U$ being the intra-orbital interaction $U$, $U-2J$ the inter-orbital antiparallel-spin interaction, and $U-3J$ the inter-orbital parallel-spin interaction. This is the simplest model that shows a crossover from a spin-frozen metal to a Fermi-liquid metal as one dopes the half-filled Mott insulator, with the self-energy scaling as $\\text{Im}\\Sigma(\\omega_n)\\sim\\sqrt{\\omega_n}$ over a significant energy range in the crossover regime \\cite{Hafermann2012}.\nThe sketch of the phase diagram is shown in Fig.~\\ref{fig:illustration}. \nAs we have seen in the previous section, the spin-related OTOC of the type (\\ref{otoc_nsns}) is relevant for the analysis of  the spin-freezing crossover, so that we concentrate on this OTOC here.\nIn the following calculations, we choose $U=8$, $J=U/4$, and compute $\\langle \\hat n_\\sigma(t)\\hat n_\\sigma(0), \\hat n_\\sigma(t)\\hat n_\\sigma(0)\\rangle - \\langle n_\\sigma \\rangle^4$ \nas a function of filling at $\\beta=50$ (dashed lines in Fig.~\\ref{fig:illustration}). \n\n\n\\begin{figure}[t]\n\\begin{center}\n\\includegraphics[angle=-90, width=0.49\\columnwidth]{plot_qp_draft.eps}\\hfill\n\\includegraphics[angle=-90, width=0.49\\columnwidth]{plot_fit_2orbital.eps}\n\\caption{\nLeft panel: Imaginary part of the self-energy as a function of Matsubara frequency for the two-orbital Hubbard model with $U=8$, $J=U/4$, and $\\beta=50$ on a log-log scale, with the dashed line indicating $\\sqrt{\\omega_n}$ behavior. \nRight panel: The coefficient $c$ extracted from\nfitting $\\langle \\hat n_\\sigma(\\tau)\\hat n_\\sigma(0), \\hat n_\\sigma(\\tau)\\hat n_\\sigma(0)\\rangle-\\langle \\hat n_\\sigma(\\tau)\\hat n_\\sigma(0)\\rangle^2$ with a function $f_\\text{fit}(\\tau)=a+b\\tau+c\\tau^2$\nin the interval $\\tau/\\beta\\in [0: 0.05]$ for the two-orbital Hubbard model. \nThe spin-freezing crossover occurs roughly at the filling $n_\\sigma\\approx 0.28$ (blue curve in the left panel and blue arrow in the right panel). \n}\n\\label{fig:self}\n\\end{center}\n\\end{figure}\n\n\nThe left panel of Fig.~\\ref{fig:self} plots the imaginary part of the self-energy as a function of Matsubara frequency in order to identify the filling corresponding to the spin-freezing crossover \\cite{Werner2008}. An approximate square-root scaling is observed over a wide range of frequencies near $n_\\sigma\\approx 0.28$ (blue line). In the right panel of Fig.~\\ref{fig:self}, we present the coefficient $c$ obtained from a similar analysis as presented in Fig.~\\ref{fig:coeffs}, but with a fitting range $\\tau/\\beta\\in [0 : 0.05]$. Again, we find that the OTOC $\\langle \\hat n_\\sigma(t)\\hat n_\\sigma(0), \\hat n_\\sigma(t)\\hat n_\\sigma(0)\\rangle - \\langle \\hat n_\\sigma\\rangle^4$ detects nontrivial correlations in the spin-freezing crossover (blue arrow) and spin-frozen metal regime, but not in the Fermi liquid or the Mott insulating phase.\n\n\n\\begin{figure}[t]\n\\begin{center}\n\\includegraphics[angle=-90, width=0.49\\columnwidth]{plot_otoc_2orbital_Re.eps}\n\\includegraphics[angle=-90, width=0.486\\columnwidth]{plot_otoc_2orbital_Im.eps}\n\\includegraphics[angle=-90, width=0.484\\columnwidth]{plot_spectra_coth_2orbital_inset.eps}\n\\includegraphics[angle=-90, width=0.486\\columnwidth]{plot_otoc_2orbital_modulus_inset.eps}\n\\caption{\nTop panels: Real and imaginary parts of the OTOC $\\langle \\hat n_\\sigma(t)\\hat n_\\sigma(0), \\hat n_\\sigma(t)\\hat n_\\sigma(0)\\rangle  - \\langle \\hat n_\\sigma\\rangle^4$ for the two-orbital Hubbard model with $U=8$, $J=U/4$, $\\beta=50$ and indicated fillings. \nBottom left panel: Spectral function $\\mathscr A_{(n_\\sigma n_\\sigma)^2}(\\omega)=-\\frac{1}{\\pi}{\\rm Im}\\,C_{(n_\\sigma n_\\sigma)^2}^R(\\omega)$ multiplied by $\\coth\\big(\\frac{\\beta\\omega}{4}\\big)$. In the inset, we plot the imaginary-time four-point function $\\langle \\hat n_\\sigma(\\tau)\\hat n_\\sigma(0), \\hat n_\\sigma(\\tau)\\hat n_\\sigma(0)\\rangle-\\langle \\hat n_\\sigma\\rangle^4$. \nBottom right panel: Modulus of the OTOC $|\\langle \\hat n_\\sigma(t)\\hat n_\\sigma(0), \\hat n_\\sigma(t)\\hat n_\\sigma(0)\\rangle  - \\langle \\hat n_\\sigma\\rangle^4|$ (normalized at $t=0$) on a log-log scale.\nIn the inset of the bottom right panel, we illustrate the approximate data collapse obtained by considering the spin-freezing crossover regimes for different values of $J$. \nIn all the panels, the thick blue curves represent the results for the spin-freezing crossover region. \n}\n\\label{fig:2orbital}\n\\end{center}\n\\end{figure}\n\n\nThe top panels of Fig.~\\ref{fig:2orbital} show the real and imaginary parts of the OTOC\n$\\langle \\hat n_\\sigma(t)\\hat n_\\sigma(0), \\hat n_\\sigma(t)\\hat n_\\sigma(0)\\rangle-\\langle \\hat n_\\sigma\\rangle^4$\nfor the two-orbital Hubbard model. In the spin-frozen phase ($n_\\sigma \\gtrsim 0.3$), both components exhibit \nhighly incoherent oscillations, which get suppressed as the filling is reduced. As one moves across the crossover point ($n_\\sigma \\sim 0.28$),\nthe oscillations completely disappear, and the real part of the OTOC starts to overshoot to the negative side\nin the Fermi liquid regime.\nThe bottom left panel in Fig.~\\ref{fig:2orbital} presents the analytically continued spectral function $\\mathscr A_{(n_\\sigma n_\\sigma)^2}(\\omega)=-\\frac{1}{\\pi}{\\rm Im}\\,C_{(n_\\sigma n_\\sigma)^2}^R(\\omega)$ multiplied by $\\coth\\big(\\frac{\\beta\\omega}{4}\\big)$. \nIn the spin-frozen regime ($n_\\sigma \\gtrsim 0.3$) a sharp peak appears at low energy, which originates from the saturation of the imaginary-time four point function $\\langle \\hat n_\\sigma(\\tau)\\hat n_\\sigma(0), \\hat n_\\sigma(\\tau)\\hat n_\\sigma(0)\\rangle$ at large $\\tau$ (see the inset of the bottom left panel of Fig.~\\ref{fig:2orbital}). This low-energy peak represents the slow dynamics of the frozen spins, and translates into a slow long-time decay of $|\\langle \\hat n_\\sigma(t)\\hat n_\\sigma(0), \\hat n_\\sigma(t)\\hat n_\\sigma(0)\\rangle-\\langle \\hat n_\\sigma\\rangle^4|$,\nas shown in the bottom right panel of Fig.~\\ref{fig:2orbital}\non a log-log scale. Again, it is difficult to clearly resolve the long-time behavior of the OTOC due to the limited\naccuracy of the analytic continuation, but in the spin-freezing crossover regime (thick blue curves)\nthe decay of the OTOC is consistent with a power law $\\sim 1/t^{1.5}$ at least for $2\\lesssim t \\lesssim 5$. \nThe shorter-time behavior ($0.5\\lesssim t \\lesssim 2$) is instead well fitted by an exponential decay $\\sim e^{-\\alpha t}$ with a decay constant $\\alpha=0.78$. \nThis exponent exhibits\na strong dependence on the Hund coupling. \n\nIn fact, the OTOCs in the spin-freezing regime for $J=U/4$ and $J=U/6$ can be approximately collapsed by plotting them as a function of $tJ$ (see the inset in the bottom right panel of Fig.~\\ref{fig:2orbital}), which indicates that the Hund coupling is the parameter which controls the dynamics in this regime. \nThis is distinct from the Lyapunov behavior in which the exponential decay scales with $t/\\beta$ \\cite{Bagrets2017}.\nA possible reason is that we are still far from the large $N$ limit (where $N$ corresponds to the number of orbitals times the number of spin degrees of freedom) so that the large $N$ behavior \nsuch as the Lyapunov growth is not observed in our calculations. \nIn fact, the strong dependence of the time scale on $J$ and weak dependence on $\\beta$ is consistent\nwith the behavior of the finite $N$ SYK model in Ref.~\\cite{FuSachdev2016}.\nIn the spin-frozen regime ($n_\\sigma\\gtrsim 0.3$) the decay is much slower than near the crossover point, while in the Fermi liquid regime the decay is accelerated, approaching the free fermion behavior of $t^{-3}$\nas shown in Fig.~\\ref{analytic continuation}. \nQualitatively similar results for this OTOC are obtained \nfor the three-orbital Hubbard model (see Appendix~\\ref{sec:3orbital}).\n\nThe time dependence of the modulus of \n$\\langle \\hat n_\\sigma(\\tau)\\hat n_\\sigma(0), \\hat n_\\sigma(\\tau)\\hat n_\\sigma(0)\\rangle-\\langle \\hat n_\\sigma\\rangle^4$ \nin the spin-freezing crossover regime of multi-orbital Hubbard models (exponential decay crossing over into a power-law) bears a close resemblance to a (different) OTOC for the SYK model discussed in Ref.~\\cite{Bagrets2017}. \nAlso, a recent study of yet another type of OTOCs for the finite-$N$ SYK model found a qualitatively similar decay as observed here in the spin-freezing crossover regime \\cite{FuSachdev2016}. In the following section, we will make the connection to the SYK dynamics more quantitative.   \n\n\n\\subsection{Comparison to the SKY model}\n\\label{sec:SYK}\n\nIn this section, we compare the results of the multi-orbital Hubbard models to the SYK model.\nWe consider the particle-hole symmetric SYK model of complex fermions \\cite{FuSachdev2016},\nwhose Hamiltonian is given by\n\\begin{align}\nH\n&=\n\\frac{1}{(2N)^{3/2}} \\sum_{i,j,k,l=1}^N \nJ_{ij;kl} (c_i^\\dagger c_j^\\dagger c_k c_l\n+\\delta_{ik}n c_j^\\dagger c_l\n-\\delta_{il}n c_j^\\dagger c_k\n-\\delta_{jk}n c_i^\\dagger c_l\n+\\delta_{jl}n c_i^\\dagger c_k).\n\\end{align}\nThe coupling constant $J_{ij;kl}$ is a gaussian random variable, satisfying $J_{ij;kl}=-J_{ji;kl}=-J_{ij;lk}=J_{kl;ij}^\\ast$, and\n\\begin{align}\n\\overline{({\\rm Re}\\, J_{ij;kl})^2}\n&=\n\\begin{cases}\nJ_{\\rm SYK}^2/2 & (i,j)\\neq (k,l) \\\\\nJ_{\\rm SYK}^2 & (i,j)=(k,l)\n\\end{cases},\n\\\\\n\\overline{({\\rm Im}\\, J_{ij;kl})^2}\n&=\n\\begin{cases}\nJ_{\\rm SYK}^2/2 & (i,j)\\neq (k,l) \\\\\n0 & (i,j)=(k,l)\n\\end{cases},\n\\end{align}\nwhere the overline represents an average over each realization of $J_{ij;kl}$. The parameter \n$J_{\\rm SYK}$ ($J_{\\rm SYK}^{-1}$) defines the unit of energy (time) in the following calculations. \nThe system is particle-hole symmetric when\n$n=0.5$, which fixes the total number of fermions as $N^{-1}\\sum_{i=1}^N \\langle c_i^\\dagger c_i\\rangle=0.5$.\nWe take the particle-hole symmetric form of the SYK model because it has been well studied previously \\cite{FuSachdev2016} and avoids the effect of the filling drift.\nWe numerically solve the model by exact diagonalization for finite $N(\\le 12)$, and evaluate the density-density OTOC\n$\\overline{\\langle \\hat n_i(t)\\hat n_i(0), \\hat n_i(t)\\hat n_i(0)\\rangle-\\langle \\hat n_i\\rangle^4}$\nand its imaginary-time counterpart.\n\n\\begin{figure}[t]\n\\begin{center}\n\n\\includegraphics[angle=-90,width=0.49\\columnwidth]{plot_fitnsns_syk.eps}\n\n\\includegraphics[angle=-90,width=0.49\\columnwidth]{plot_coeffc_syk.eps}\n\\caption{\nLeft panel: \nThe imaginary-time four-point function $\\overline{\\langle \\hat n_i(\\tau)\\hat n_i(0), \\hat n_i(\\tau)\\hat n_i(0)\\rangle}$, the density-density correlation function $\\overline{\\langle \\hat n_i(\\tau)\\hat n_i(0)\\rangle^2}$,\nand their difference for the SYK model with $N=12$ and $\\beta J_{\\rm SYK}=10$.\nThe dashed curve shows a fit of the difference with a function $f_\\text{fit}(\\tau)=a+b\\tau+c\\tau^2$ in the interval $\\tau/\\beta \\in [0: 0.2]$. \nRight panel: Fitting coefficient $c$ as a function of $T/J_{\\rm SYK}$.\n}\n\\label{fig:SYK fit}\n\\end{center}\n\\end{figure}\n\nIn the left panel of Fig.~\\ref{fig:SYK fit}, we plot the imaginary-time four-point function \n$\\overline{\\langle \\hat n_i(\\tau)\\hat n_i(0), \\hat n_i(\\tau)\\hat n_i(0)\\rangle}$\nfor the SYK model with $N=12$ and $\\beta J_{\\rm SYK}=10$. \nAt low temperature, we expect that $\\overline{\\langle \\hat n_i(\\tau)\\hat n_i(0), \\hat n_i(\\tau)\\hat n_i(0)\\rangle}$ is decoupled into $\\overline{\\langle \\hat n_i(\\tau)\\hat n_i(0)\\rangle^2}$.\nTo see whether nontrivial correlations beyond the decoupled form are present, we fit the difference\n$\\overline{\\langle \\hat n_i(\\tau)\\hat n_i(0), \\hat n_i(\\tau)\\hat n_i(0)\\rangle}\n-\\overline{\\langle \\hat n_i(\\tau)\\hat n_i(0)\\rangle^2}$ with a function $f_{\\rm fit}(\\tau)=a+b\\tau+c\\tau^2$\nin the interval $\\tau/\\beta\\in[0:0.2]$, in the same manner as for the Hubbard models.\nThe obtained coefficient $c$ is plotted as a function of the temperature $T$ in the right panel of\nFig.~\\ref{fig:SYK fit}. One can see that nontrivial correlations exist in a wide range of temperature.\nEspecially, they are enhanced in the intermediate temperature regime. This is because in the zero-temperature limit\nthe four-point function is decoupled as \n$\\langle \\hat n_i(\\tau)\\hat n_i(0), \\hat n_i(\\tau)\\hat n_i(0)\\rangle \\approx \\langle \\hat n_i(\\tau)\\hat n_i(0)\\rangle^2$, while in the high-temperature limit ($\\beta\\to 0$) the imaginary-time dependence is washed out.\n\n\n\n\n\\begin{figure}[t]\n\\begin{center}\n\\includegraphics[angle=-90, width=0.49\\columnwidth]{plot_syk_Re.eps}\n\\includegraphics[angle=-90, width=0.49\\columnwidth]{plot_syk_Im.eps}\n\\includegraphics[angle=-90, width=0.49\\columnwidth]{plot_syk_modulus-log.eps}\n\\includegraphics[angle=-90, width=0.49\\columnwidth]{plot_syk_modulus.eps}\n\\caption{\nThe density-density OTOC $\\overline{\\langle \\hat n_i(t)\\hat n_i(0), \\hat n_i(t)\\hat n_i(0)\\rangle-\\langle \\hat n_i\\rangle^4}$ for the particle-hole symmetric SYK model averaged over $10^2$ samples with $\\beta J_{\\rm SYK}=100$ and various $N$. \nThe top left and right panels show the real and imaginary parts of the OTOC,\nwhile the bottom left and right panels show the log and log-log plots of the modulus of the OTOC (normalized at $t=0$).\nThe dashed line shows a power-law decay $\\propto t^{-1.6}$ for comparison.}\n\\label{fig:SYK}\n\\end{center}\n\\end{figure}\n\nIn Fig.~\\ref{fig:SYK}, we show the numerical results of the density-density OTOC for $\\beta J_{\\rm SYK}=100$.\nThe real part of the OTOC rapidly drops from the initial value within the time scale of $tJ_{\\rm SYK}\\sim 2$,\nand slowly decays to zero in the long time limit.\nAs one increases $N$, the oscillations that appear at longer time tend to be suppressed.\nOne can see that the result for $N=12$ in the top panels of Fig.~\\ref{fig:SYK} closely resembles the OTOC in \nthe spin-freezing crossover regime of the two-orbital Hubbard model shown\nby the blue curve in the top panels of Fig.~\\ref{fig:2orbital},\nwhile it differs from the single-orbital result shown in the top panels of Fig.~\\ref{fig:results_nsns}.\nDue to the limitation of our calculations to small system sizes ($N\\le 12$), we do not clearly observe\nan exponential growth ($\\sim c_0-c_1 e^{\\lambda t}$)  at short times or an exponential decay ($\\sim e^{-\\alpha t}$) \nat intermediate times. \nHowever, we find that for $tJ_{\\rm SYK}\\gtrsim 2$ the OTOC decays\napproximately in a power law $\\propto t^{-\\gamma}$ with $\\gamma=1.6$ (see bottom right panel in Fig.~\\ref{fig:SYK}). \nThe time interval exhibiting the power-law decay becomes longer as we increase $N$.  \nThe temperature dependence of the OTOC for the SYK model is shown in Fig.~\\ref{fig:SYK-T}.\nThe time scale of the initial drop is more or less independent of the temperature, while \nthe power-law-like decay is only visible in the low-temperature regime.\n\n\\begin{figure}[t]\n\\begin{center}\n\\includegraphics[angle=-90, width=0.49\\columnwidth]{plot_syk_beta_re.eps}\n\\includegraphics[angle=-90, width=0.49\\columnwidth]{plot_syk_beta_im.eps}\n\\includegraphics[angle=-90, width=0.49\\columnwidth]{plot_syk_beta_modulus-log.eps}\n\\includegraphics[angle=-90, width=0.49\\columnwidth]{plot_syk_beta_modulus.eps}\n\\caption{\nTemperature dependence of the density-density OTOC\n$\\overline{\\langle \\hat n_i(t)\\hat n_i(0), \\hat n_i(t)\\hat n_i(0)\\rangle-\\langle \\hat n_i\\rangle^4}$ for the particle-hole symmetric SYK model averaged over $10^2$ samples with $N=12$.\nThe top left and right panels show the real and imaginary parts of the OTOC,\nwhile the bottom left and right panels show the log and log-log plots of the modulus of the OTOC.\nThe dashed line shows a power-law decay $\\propto t^{-1.6}$ for comparison.}\n\\label{fig:SYK-T}\n\\end{center}\n\\end{figure}\n\nIn Ref.~\\onlinecite{Werner2018}, it has been argued that the spin-freezing crossover regime of multi-orbital Hubbard models is effectively described by the SYK model in which $J_{\\rm SYK}$ is replaced by the Hund coupling $J$. The results in Fig.~\\ref{fig:2orbital} are for the two-orbital Hubbard model with $U=8$, $J/U=1/4$ and $\\beta=50$, corresponding to $\\beta J=100$, which means that it is meaningful to directly compare the blue lines in Fig.~\\ref{fig:2orbital} with the blue lines in Figs.~\\ref{fig:SYK} and \\ref{fig:SYK-T}. Not only is the qualitative agreement remarkable, even the power-law exponent measured for the two-orbital Hubbard model ($\\gamma=1.5$) is very close to the exponent extracted from the finite-$N$ SYK calculation. \nThe power-law decay in the three-orbital case is approximately $1/t^{1.75}$ (see Appendix~\\ref{sec:3orbital}), and thus also in semi-quantitative agreement with the SYK model behavior. \nThis nontrivial result provides further support for the identification of the spin-freezing crossover regime of multi-orbital Hubbard systems with an SYK strange metal. \n\nIn the large-$N$ and low-$T$ limit of the SYK model, we expect that the power-law behavior approaches $1/t^2$, since in this limit the OTOC is decoupled as $\\langle \\hat n_i(t)\\hat n_i(0), \\hat n_i(t)\\hat n_i(0)\\rangle\\approx \\langle \\hat n_i(t)\\hat n_i(0)\\rangle^2$ and the decay of $\\langle \\hat n_i(t)\\hat n_i(0)\\rangle$ is dominated by what corresponds to the slow spin relaxation $\\langle \\hat S_z(t)\\hat S_z(0)\\rangle\\sim 1/t$ in the Sachdev-Ye model.\n\n\n\n\n\n4.4 Comparison to the SKY model\n\\subsection{Comparison to the SKY model}\n\\label{sec:SYK}\n\nIn this section, we compare the results of the multi-orbital Hubbard models to the SYK model.\nWe consider the particle-hole symmetric SYK model of complex fermions \\cite{FuSachdev2016},\nwhose Hamiltonian is given by\n\\begin{align}\nH\n&=\n\\frac{1}{(2N)^{3/2}} \\sum_{i,j,k,l=1}^N \nJ_{ij;kl} (c_i^\\dagger c_j^\\dagger c_k c_l\n+\\delta_{ik}n c_j^\\dagger c_l\n-\\delta_{il}n c_j^\\dagger c_k\n-\\delta_{jk}n c_i^\\dagger c_l\n+\\delta_{jl}n c_i^\\dagger c_k).\n\\end{align}\nThe coupling constant $J_{ij;kl}$ is a gaussian random variable, satisfying $J_{ij;kl}=-J_{ji;kl}=-J_{ij;lk}=J_{kl;ij}^\\ast$, and\n\\begin{align}\n\\overline{({\\rm Re}\\, J_{ij;kl})^2}\n&=\n\\begin{cases}\nJ_{\\rm SYK}^2/2 & (i,j)\\neq (k,l) \\\\\nJ_{\\rm SYK}^2 & (i,j)=(k,l)\n\\end{cases},\n\\\\\n\\overline{({\\rm Im}\\, J_{ij;kl})^2}\n&=\n\\begin{cases}\nJ_{\\rm SYK}^2/2 & (i,j)\\neq (k,l) \\\\\n0 & (i,j)=(k,l)\n\\end{cases},\n\\end{align}\nwhere the overline represents an average over each realization of $J_{ij;kl}$. The parameter \n$J_{\\rm SYK}$ ($J_{\\rm SYK}^{-1}$) defines the unit of energy (time) in the following calculations. \nThe system is particle-hole symmetric when\n$n=0.5$, which fixes the total number of fermions as $N^{-1}\\sum_{i=1}^N \\langle c_i^\\dagger c_i\\rangle=0.5$.\nWe take the particle-hole symmetric form of the SYK model because it has been well studied previously \\cite{FuSachdev2016} and avoids the effect of the filling drift.\nWe numerically solve the model by exact diagonalization for finite $N(\\le 12)$, and evaluate the density-density OTOC\n$\\overline{\\langle \\hat n_i(t)\\hat n_i(0), \\hat n_i(t)\\hat n_i(0)\\rangle-\\langle \\hat n_i\\rangle^4}$\nand its imaginary-time counterpart.\n\n\\begin{figure}[t]\n\\begin{center}\n\n\\includegraphics[angle=-90,width=0.49\\columnwidth]{plot_fitnsns_syk.eps}\n\n\\includegraphics[angle=-90,width=0.49\\columnwidth]{plot_coeffc_syk.eps}\n\\caption{\nLeft panel: \nThe imaginary-time four-point function $\\overline{\\langle \\hat n_i(\\tau)\\hat n_i(0), \\hat n_i(\\tau)\\hat n_i(0)\\rangle}$, the density-density correlation function $\\overline{\\langle \\hat n_i(\\tau)\\hat n_i(0)\\rangle^2}$,\nand their difference for the SYK model with $N=12$ and $\\beta J_{\\rm SYK}=10$.\nThe dashed curve shows a fit of the difference with a function $f_\\text{fit}(\\tau)=a+b\\tau+c\\tau^2$ in the interval $\\tau/\\beta \\in [0: 0.2]$. \nRight panel: Fitting coefficient $c$ as a function of $T/J_{\\rm SYK}$.\n}\n\\label{fig:SYK fit}\n\\end{center}\n\\end{figure}\n\nIn the left panel of Fig.~\\ref{fig:SYK fit}, we plot the imaginary-time four-point function \n$\\overline{\\langle \\hat n_i(\\tau)\\hat n_i(0), \\hat n_i(\\tau)\\hat n_i(0)\\rangle}$\nfor the SYK model with $N=12$ and $\\beta J_{\\rm SYK}=10$. \nAt low temperature, we expect that $\\overline{\\langle \\hat n_i(\\tau)\\hat n_i(0), \\hat n_i(\\tau)\\hat n_i(0)\\rangle}$ is decoupled into $\\overline{\\langle \\hat n_i(\\tau)\\hat n_i(0)\\rangle^2}$.\nTo see whether nontrivial correlations beyond the decoupled form are present, we fit the difference\n$\\overline{\\langle \\hat n_i(\\tau)\\hat n_i(0), \\hat n_i(\\tau)\\hat n_i(0)\\rangle}\n-\\overline{\\langle \\hat n_i(\\tau)\\hat n_i(0)\\rangle^2}$ with a function $f_{\\rm fit}(\\tau)=a+b\\tau+c\\tau^2$\nin the interval $\\tau/\\beta\\in[0:0.2]$, in the same manner as for the Hubbard models.\nThe obtained coefficient $c$ is plotted as a function of the temperature $T$ in the right panel of\nFig.~\\ref{fig:SYK fit}. One can see that nontrivial correlations exist in a wide range of temperature.\nEspecially, they are enhanced in the intermediate temperature regime. This is because in the zero-temperature limit\nthe four-point function is decoupled as \n$\\langle \\hat n_i(\\tau)\\hat n_i(0), \\hat n_i(\\tau)\\hat n_i(0)\\rangle \\approx \\langle \\hat n_i(\\tau)\\hat n_i(0)\\rangle^2$, while in the high-temperature limit ($\\beta\\to 0$) the imaginary-time dependence is washed out.\n\n\n\n\n\\begin{figure}[t]\n\\begin{center}\n\\includegraphics[angle=-90, width=0.49\\columnwidth]{plot_syk_Re.eps}\n\\includegraphics[angle=-90, width=0.49\\columnwidth]{plot_syk_Im.eps}\n\\includegraphics[angle=-90, width=0.49\\columnwidth]{plot_syk_modulus-log.eps}\n\\includegraphics[angle=-90, width=0.49\\columnwidth]{plot_syk_modulus.eps}\n\\caption{\nThe density-density OTOC $\\overline{\\langle \\hat n_i(t)\\hat n_i(0), \\hat n_i(t)\\hat n_i(0)\\rangle-\\langle \\hat n_i\\rangle^4}$ for the particle-hole symmetric SYK model averaged over $10^2$ samples with $\\beta J_{\\rm SYK}=100$ and various $N$. \nThe top left and right panels show the real and imaginary parts of the OTOC,\nwhile the bottom left and right panels show the log and log-log plots of the modulus of the OTOC (normalized at $t=0$).\nThe dashed line shows a power-law decay $\\propto t^{-1.6}$ for comparison.}\n\\label{fig:SYK}\n\\end{center}\n\\end{figure}\n\nIn Fig.~\\ref{fig:SYK}, we show the numerical results of the density-density OTOC for $\\beta J_{\\rm SYK}=100$.\nThe real part of the OTOC rapidly drops from the initial value within the time scale of $tJ_{\\rm SYK}\\sim 2$,\nand slowly decays to zero in the long time limit.\nAs one increases $N$, the oscillations that appear at longer time tend to be suppressed.\nOne can see that the result for $N=12$ in the top panels of Fig.~\\ref{fig:SYK} closely resembles the OTOC in \nthe spin-freezing crossover regime of the two-orbital Hubbard model shown\nby the blue curve in the top panels of Fig.~\\ref{fig:2orbital},\nwhile it differs from the single-orbital result shown in the top panels of Fig.~\\ref{fig:results_nsns}.\nDue to the limitation of our calculations to small system sizes ($N\\le 12$), we do not clearly observe\nan exponential growth ($\\sim c_0-c_1 e^{\\lambda t}$)  at short times or an exponential decay ($\\sim e^{-\\alpha t}$) \nat intermediate times. \nHowever, we find that for $tJ_{\\rm SYK}\\gtrsim 2$ the OTOC decays\napproximately in a power law $\\propto t^{-\\gamma}$ with $\\gamma=1.6$ (see bottom right panel in Fig.~\\ref{fig:SYK}). \nThe time interval exhibiting the power-law decay becomes longer as we increase $N$.  \nThe temperature dependence of the OTOC for the SYK model is shown in Fig.~\\ref{fig:SYK-T}.\nThe time scale of the initial drop is more or less independent of the temperature, while \nthe power-law-like decay is only visible in the low-temperature regime.\n\n\\begin{figure}[t]\n\\begin{center}\n\\includegraphics[angle=-90, width=0.49\\columnwidth]{plot_syk_beta_re.eps}\n\\includegraphics[angle=-90, width=0.49\\columnwidth]{plot_syk_beta_im.eps}\n\\includegraphics[angle=-90, width=0.49\\columnwidth]{plot_syk_beta_modulus-log.eps}\n\\includegraphics[angle=-90, width=0.49\\columnwidth]{plot_syk_beta_modulus.eps}\n\\caption{\nTemperature dependence of the density-density OTOC\n$\\overline{\\langle \\hat n_i(t)\\hat n_i(0), \\hat n_i(t)\\hat n_i(0)\\rangle-\\langle \\hat n_i\\rangle^4}$ for the particle-hole symmetric SYK model averaged over $10^2$ samples with $N=12$.\nThe top left and right panels show the real and imaginary parts of the OTOC,\nwhile the bottom left and right panels show the log and log-log plots of the modulus of the OTOC.\nThe dashed line shows a power-law decay $\\propto t^{-1.6}$ for comparison.}\n\\label{fig:SYK-T}\n\\end{center}\n\\end{figure}\n\nIn Ref.~\\onlinecite{Werner2018}, it has been argued that the spin-freezing crossover regime of multi-orbital Hubbard models is effectively described by the SYK model in which $J_{\\rm SYK}$ is replaced by the Hund coupling $J$. The results in Fig.~\\ref{fig:2orbital} are for the two-orbital Hubbard model with $U=8$, $J/U=1/4$ and $\\beta=50$, corresponding to $\\beta J=100$, which means that it is meaningful to directly compare the blue lines in Fig.~\\ref{fig:2orbital} with the blue lines in Figs.~\\ref{fig:SYK} and \\ref{fig:SYK-T}. Not only is the qualitative agreement remarkable, even the power-law exponent measured for the two-orbital Hubbard model ($\\gamma=1.5$) is very close to the exponent extracted from the finite-$N$ SYK calculation. \nThe power-law decay in the three-orbital case is approximately $1/t^{1.75}$ (see Appendix~\\ref{sec:3orbital}), and thus also in semi-quantitative agreement with the SYK model behavior. \nThis nontrivial result provides further support for the identification of the spin-freezing crossover regime of multi-orbital Hubbard systems with an SYK strange metal. \n\nIn the large-$N$ and low-$T$ limit of the SYK model, we expect that the power-law behavior approaches $1/t^2$, since in this limit the OTOC is decoupled as $\\langle \\hat n_i(t)\\hat n_i(0), \\hat n_i(t)\\hat n_i(0)\\rangle\\approx \\langle \\hat n_i(t)\\hat n_i(0)\\rangle^2$ and the decay of $\\langle \\hat n_i(t)\\hat n_i(0)\\rangle$ is dominated by what corresponds to the slow spin relaxation $\\langle \\hat S_z(t)\\hat S_z(0)\\rangle\\sim 1/t$ in the Sachdev-Ye model.\n\n\n\n\n\n",
        "label": "fig:SYK fit",
        "Descriptive_question1": "What is the value of N for the SYK model shown in figure_11?",
        "Descriptive_question2": "What is the value of β J_SYK in figure_11 for the SYK model?",
        "Reasoning_question1": "How does the fitting coefficient c vary with temperature T/J_SYK in figure_11, and what might this indicate about the presence of nontrivial correlations in the SYK model?",
        "Reasoning_question2": "Why might the difference between the imaginary-time four-point function and the density-density correlation function be fitted with a quadratic function in figure_11, and what does this suggest about the short-time behavior of the SYK model?",
        "Descriptive_answer1": "N=12",
        "Descriptive_answer2": "β J_SYK=10",
        "Reasoning_answer1": "The fitting coefficient c, as shown in the right panel of figure_11, varies as a function of temperature T/J_SYK. Specifically, c is nonzero over a range of temperatures and shows enhancement in the intermediate temperature regime. This indicates that nontrivial correlations beyond the simple factorization (decoupling) of the four-point function into squared two-point functions exist at these intermediate temperatures. At very low temperatures, c approaches zero, consistent with the expected decoupling, and at high temperatures, the imaginary-time dependence is washed out. Hence, the behavior of c with temperature reveals the regimes where complex many-body correlations are significant in the SYK model.",
        "Reasoning_answer2": "The difference between the imaginary-time four-point function and the squared density-density correlation function is fitted with a quadratic function f_fit(τ) = a + b τ + c τ^2 over a short interval of τ/β. Fitting with a quadratic reflects capturing the leading order (short-time) deviations from the decoupled form in powers of τ around τ=0. This suggests that at short imaginary times, the four-point function contains nontrivial correlations that can be characterized by a smooth, low-order expansion, representing how these correlations build up as time progresses from zero. This approach captures subtle many-body effects that go beyond simple factorization, indicating complex short-time dynamics in the SYK model."
    },
    {
        "paper_id": "1812.04217.json",
        "image_id": "figure_12",
        "image_path": [
            "/home/yz979/scratch/chengye/scimolmo/test-images-tables/dataset_imagestables/1812.04217/images/plot_syk_Re.eps"
        ],
        "caption": "\nThe density-density OTOC $\\overline{\\langle \\hat n_i(t)\\hat n_i(0), \\hat n_i(t)\\hat n_i(0)\\rangle-\\langle \\hat n_i\\rangle^4}$ for the particle-hole symmetric SYK model averaged over $10^2$ samples with $\\beta J_{\\rm SYK}=100$ and various $N$. \nThe top left and right panels show the real and imaginary parts of the OTOC,\nwhile the bottom left and right panels show the log and log-log plots of the modulus of the OTOC (normalized at $t=0$).\nThe dashed line shows a power-law decay $\\propto t^{-1.6}$ for comparison.",
        "classify": "Chart",
        "section_info": "4 Numerical results for the Hubbard model\n\\section{Numerical results for the Hubbard model}\n\\label{numerical results}\n\n\\subsection{Observables and numerical procedure}\n\nWe consider the single-orbital, two-orbital, and three-orbital Hubbard models on an infinitely-connected Bethe lattice, which can be solved exactly within DMFT \\cite{GeorgesKotliarKrauthRozenberg1996}. In this case, the noninteracting density of states is semicircular with bandwidth $4v_\\ast$, for which there exists a simplified DMFT self-consistency condition between the hybridization function $\\Delta_{\\alpha\\sigma}$ of the DMFT impurity problem and the local (impurity) Green's function $G_{\\alpha\\sigma}$: \n\n\\begin{equation}\n\\Delta_{\\alpha\\sigma}(\\tau)=v_\\ast^2 G_{\\alpha\\sigma}(\\tau), \n\\label{semicirc}\n\\end{equation}\n\nwith $\\alpha$ the orbital and $\\sigma$ the spin index. We will use $v_\\ast$ as the unit of energy and measure time in units of $\\hbar/v_\\ast$. \n\nThree different types of imaginary-time four-point functions $C_{(AB)^2}^M(\\tau)$\nof the form of Eq.~(\\ref{imaginary-time OTOC})\nwith $(\\hat A, \\hat B)=(c_\\sigma^\\dagger, c_\\sigma), (\\hat n_\\sigma, \\hat n_\\sigma)$, and $(\\hat n, \\hat n)$\nare calculated in the interval $0\\le \\tau\\le \\frac{\\beta\\hbar}{2}$: \n\\begin{eqnarray}\nC_{(c^\\dagger_\\sigma c_\\sigma)^2}^M(\\tau) &=& -\\langle c^\\dagger_\\sigma(\\tau+\\tfrac{\\beta\\hbar}{2}) c_\\sigma(\\tfrac{\\beta\\hbar}{2}) c^\\dagger_\\sigma(\\tau) c_\\sigma(0) \\rangle, \\label{fermion_otoc}\\\\\nC_{(n_\\sigma n_\\sigma)^2}^M(\\tau) &=& -\\langle \\hat n_\\sigma(\\tau+\\tfrac{\\beta\\hbar}{2})\\hat n_\\sigma(\\tfrac{\\beta\\hbar}{2})\\hat n_\\sigma(\\tau)\\hat n_\\sigma(0) \\rangle, \\label{otoc_nsns}\\\\\nC_{(nn)^2}^M(\\tau) &=& -\\langle \\hat n(\\tau+\\tfrac{\\beta\\hbar}{2})\\hat n(\\tfrac{\\beta\\hbar}{2})\\hat n(\\tau)\\hat n(0) \\rangle,\\label{otoc_nn}\n\\end{eqnarray}\n\nwhere $c_\\sigma^\\dagger$ ($c_\\sigma$) are the fermionic creation (annihilation) operators for spin $\\sigma$ (and orbital $\\alpha=1$ in the multi-orbital case), $\\hat n_\\sigma=c^\\dagger_\\sigma c_\\sigma$ is the corresponding spin-dependent density operator, and $\\hat n=\\hat n_\\uparrow + \\hat n_\\downarrow$ the total density operator. Note that in all the three cases above we have $\\hat B=\\hat A^\\dagger$. \nWe measure these local correlation functions in the impurity model using a hybridization expansion continuous-time Monte Carlo algorithm (CT-HYB) \\cite{Werner2006}. In this algorithm, the \ntwo-particle\nGreen's functions of the type (\\ref{fermion_otoc}) can be measured by removing two hybridization lines, i.e., from the elements of the inverse hybridization matrix, while the density-density correlation functions can be easily measured either by insertion of density operators (matrix formalism) \\cite{Werner2006Kondo} or by reading off the occupation of the orbitals at the four time points in the segment implementation \\cite{Werner2006}. We use the latter algorithm since we consider only density-density interactions. \nThe values at the end-points $\\tau=0$ and $\\tau=\\frac{\\beta\\hbar}{2}$ reduce to standard density-density correlation functions, which in the case of Eq.~(\\ref{fermion_otoc}) are measured separately. \n\nWe perform the analytic continuation to the real-frequency axis using the Maximum Entropy method \\cite{Jarrell1996,Lewin_maxent} with a bosonic kernel. This yields the imaginary part of the retarded correlator \n$-\\frac{1}{\\pi}\\text{Im}\\, C^R_{(AA^\\dagger)^2}(\\omega)=\\mathscr A_{(AA^\\dagger)^2}(\\omega)$.  \nWhen $\\hat B=\\hat A^\\dagger=\\hat A$ (which is the case for $\\hat A=\\hat n_\\sigma, \\hat n$), we have $C_{(AA)^2}^R(\\omega)^\\ast=C_{(AA)^2}^R(-\\omega)$. At half filling, the particle-hole symmetry furthermore ensures $C_{c_\\sigma^\\dagger c_\\sigma}^R(\\omega)^\\ast=C_{c_\\sigma^\\dagger c_\\sigma}^R(-\\omega)$.\nThus, in all the cases considered in this study, the retarded OTOC has the symmetry property $C_{(AA^\\dagger)^2}^R(\\omega)^\\ast=C_{(AA^\\dagger)^2}^R(-\\omega)$. Using this, as well as the out-of-time-order fluctuation-dissipation\ntheorem discussed in the previous section, we can obtain the real-time OTOC\n$\\langle \\hat A(t)\\hat A^\\dagger(0), \\hat A(t)\\hat A^\\dagger(0)\\rangle$ from the following inverse Fourier transformation, \n\\begin{align}{\\rm Re}\\langle \\hat A(t)\\hat A^\\dagger(0), \\hat A(t)\\hat A^\\dagger(0)\\rangle\n&=\n{\\rm Re}\n\\int_0^\\infty d\\omega\\, e^{-i\\omega t}\\coth\\left(\\frac{\\beta\\hbar\\omega}{4}\\right)\n\\left(-\\frac{1}{\\pi}{\\rm Im}\\, C_{(AA^\\dagger)^2}^R(\\omega)\\right),\n\\\\\n{\\rm Im}\\langle \\hat A(t)\\hat A^\\dagger(0), \\hat A(t)\\hat A^\\dagger(0)\\rangle\n&=\n{\\rm Im}\\int_0^{\\infty} d\\omega\\, e^{-i\\omega t} \\left(-\\frac{1}{\\pi}{\\rm Im}\\, C_{(AA^\\dagger)^2}^R(\\omega)\\right),\n\\end{align}\nfor $\\hat A=c_\\sigma^\\dagger, \\hat n_\\sigma$, and $\\hat n$.\n\n\\begin{figure}[b]\n\\begin{center}\n\\includegraphics[angle=-90, width=0.48\\columnwidth]{compare_u0_draft.eps}\\hfill\n\\includegraphics[angle=-90, width=0.48\\columnwidth]{compare_u0_longtime_modulus.eps}\n\\caption{\nLeft panel: Comparison between the exact solution and the result obtained by analytical continuation\nfor the real and imaginary parts of the OTOC $\\langle c_\\sigma^\\dagger(t)c_\\sigma(0), c_\\sigma^\\dagger(t)c_\\sigma(0)\\rangle$ \nin the noninteracting case ($U=0, \\beta\\hbar=50$) of the single-orbital Hubbard model.\nRight panel: Comparison of the long-time behavior for the modulus $|\\langle c_\\sigma^\\dagger(t)c_\\sigma(0), c_\\sigma^\\dagger(t)c_\\sigma(0)\\rangle|$. \nThe exact result shows a power-law decay ($\\sim 1/t^3$). \n}\n\\label{fig:test_u0}\n\\end{center}\n\\end{figure}\n\nAs a check of our procedure, we first compare the results for the noninteracting model,\nfor which an exact solution of the OTOC $\\langle c_\\sigma^\\dagger(t)c_\\sigma(0), c_\\sigma^\\dagger(t)c_\\sigma(0)\\rangle$ \nis available \\cite{TsujiWernerUeda2017}.\nIt is a nontrivial task to measure the noninteracting two-particle Green's function in CT-HYB, \nand to analytically continue \n$C_{(c^\\dagger_\\sigma c_\\sigma)^2}^M(\\tau)$ \nby the Maximum Entropy method.  \nThe results for the inverse temperature $\\beta\\hbar=50$ are plotted in Fig.~\\ref{fig:test_u0}. \nOne can see that the OTOC oscillates for about one cycle, and quickly decays to zero.\nThe analytically continued data show a good agreement with the exact solution. \nIn particular, for short times (up to about two inverse hoppings) the dynamics is accurately reproduced, while deviations appear at longer times. \nIt is known that for the noninteracting fermion system the OTOC decays as a power law at long time ($\\sim 1/t^3$) \\cite{TsujiWernerUeda2017}.\nThe analytical continuation method cannot reproduce the details of the oscillations at intermediate or long times, but it roughly captures the long-time decay, which is controlled by low-frequency spectral features. \nUsually, analytic continuation is unreliable for high frequency components, because high-frequency information is suppressed \nby the kernel $K(\\tau,\\omega)$ in the transformation from the spectral function $A_O(\\omega)$ to the (bosonic) Matsubara correlation function $O(\\tau)$ [$O(\\tau)=\\int_{-\\infty}^{\\infty} d\\omega\\, K(\\omega,\\tau) A_O(\\omega)$ with $K(\\tau,\\omega)=e^{-\\tau\\omega}/(1-e^{-\\beta\\omega})$]. On the other hand, the low frequency components can be rather accurately determined. As a result, in Fig. 4 we more or less recover the low-frequency features of the time-dependent correlation function, while the details of rapid oscillations are not captured. The accurate result at short times can be understood from the fact that this region is close to the imaginary time axis in the complex time plane, where the original QMC data are available.\nAt higher temperatures, the Maximum Entropy method becomes less reliable, so that the resulting OTOCs are expected to be less accurate.  \n\n\n\\subsection{Single-orbital Hubbard model}\n\\label{sec:hubbard}\n\nIn this section, we calculate the OTOCs (\\ref{fermion_otoc})-(\\ref{otoc_nn}) for the interacting single-orbital Hubbard model with the Hamiltonian \n\\begin{align}\nH\n&=\n-v\\sum_{\\langle i,j\\rangle\\sigma} (c^\\dagger_{i\\sigma}c_{j\\sigma}+ \\text{h.c.})\n-\\mu\\sum_i \\hat n_i+U\\sum_i \\hat n_{i\\uparrow}\\hat n_{i\\downarrow}, \n\\end{align}\nwhere $v$ is the hopping amplitude, $\\langle i,j\\rangle$ represents the nearest-neighbor site pairs, \n$\\mu$ is the chemical potential, and $U$ is the on-site interaction.\nIn this section, we focus on the half-filled case (i.e., $\\mu=U/2$). The DMFT solution for the simplified self-consistency (\\ref{semicirc}) is the exact solution for an infinitely connected Bethe lattice \\cite{GeorgesKotliarKrauthRozenberg1996}.\nLet us briefly recall the paramagnetic phase diagram for the single-orbital Hubbard model on this lattice \\cite{Bluemer_thesis}. At half-filling, there is a metal-insulator crossover at high temperature and a first-order Mott transition below a temperature corresponding to $\\beta\\hbar\\approx 17$ with a coexistence region between $U_{c1}$ and $U_{c2}$. The finite-temperature critical endpoint is at $U\\approx 4.7$, while the zero-temperature Mott transition occurs at $U=U_{c2}\\approx 5.6$. \nAt low temperature, the self-energy shows the Fermi liquid behavior [${\\rm Im}\\,\\Sigma(\\omega)\\sim \\omega^2$] \nin the weakly correlated metallic phase, \nwhile it shows an insulating behavior [${\\rm Im}\\,\\Sigma(\\omega)\\sim \\delta(\\omega)$] in the Mott phase\n\\cite{GeorgesKotliarKrauthRozenberg1996}. \nIn the correlated metallic phase, as the temperature is increased, deviations from the Fermi liquid behavior become apparent, \nbut an ${\\rm Im}\\,\\Sigma(\\omega)\\sim \\sqrt{\\omega}$ scaling as in the SYK model is not observed. \n\nIn the following, we will start the DMFT iterations from the noninteracting solution, which means that the finite-temperature Mott transition occurs at $U_{c2}$.\nFrom now on, we set $\\hbar=1$. \n\n\\begin{figure}[t]\n\\begin{center}\n\\includegraphics[angle=-90, width=0.49\\columnwidth]{plot_otoc_b50_Re.eps}\n\\includegraphics[angle=-90, width=0.49\\columnwidth]{plot_otoc_b50_Im.eps}\n\\includegraphics[angle=-90, width=0.49\\columnwidth]{plot_coth_b50_appendix.eps}\n\\includegraphics[angle=-90, width=0.49\\columnwidth]{plot_otoc_b50_modulus.eps}\n\\caption{\nTop panels: Real and imaginary parts of the OTOC $\\langle c^\\dagger_\\sigma(t)c_\\sigma(0), c^\\dagger_\\sigma(t)c_\\sigma(0)\\rangle$ for the single-orbital Hubbard model with $\\beta=50$ at half filling. The thin lines correspond to metallic solutions, while the thick line ($U=6$) corresponds to a Mott insulating solution. \nBottom left panel: Spectral function $\\mathscr A_{(c_\\sigma^\\dagger c_\\sigma)^2}(\\omega)=-\\frac{1}{\\pi}{\\rm Im}\\,C_{(c^\\dagger_\\sigma c_\\sigma)^2}^R(\\omega)$ multiplied by $\\coth\\big(\\frac{\\beta\\omega}{4}\\big)$. \nBottom right panel: Modulus of the OTOC $|\\langle c^\\dagger_\\sigma(t)c_\\sigma(0), c^\\dagger_\\sigma(t)c_\\sigma(0)\\rangle|$ (normalized at $t=0$) on a log-log scale. The dashed line is the result for $U=0$.\n}\n\\label{fig:results_fermion}\n\\end{center}\n\\end{figure}\n\n\n\nIn Fig.~\\ref{fig:results_fermion}, we show the results of $\\langle c^\\dagger_\\sigma(t)c_\\sigma(0), c^\\dagger_\\sigma(t)c_\\sigma(0)\\rangle$ for the interacting single-orbital Hubbard model at $\\beta=50$. \nThe top panels present the real and imaginary parts of the OTOC. One can see that the oscillations become more pronounced as one increases the interaction. In the bottom left panel of Fig.~\\ref{fig:results_fermion},\nwe show the corresponding spectral function $\\mathscr A_{(c_\\sigma^\\dagger c_\\sigma)^2}(\\omega)=-\\frac{1}{\\pi}{\\rm Im}\\,C_{(c^\\dagger_\\sigma c_\\sigma)^2}^R(\\omega)$ multiplied by $\\coth\\big(\\frac{\\beta\\omega}{4}\\big)$.\nThe Mott transition, which occurs between $U=5$ and $U=6$ at $\\beta=50$, manifests itself by the opening of a gap in the spectral function $\\mathscr A_{(c_\\sigma^\\dagger c_\\sigma)^2(\\omega)}$. This translates into more weakly damped oscillations in the real-time evolution in the Mott phase.  \nThe bottom right panel of Fig.~\\ref{fig:results_fermion} plots the modulus of the OTOC (normalized at time $t=0$, i.e., $|\\langle c^\\dagger_\\sigma(t)c_\\sigma(0), c^\\dagger_\\sigma(t)c_\\sigma(0)\\rangle|/|\\langle c^\\dagger_\\sigma(0)c_\\sigma(0), c^\\dagger_\\sigma(0)c_\\sigma(0)\\rangle|$) on a log-log scale. \nDue to the oscillations and the limited accuracy of the analytic continuation procedure, it is hard to clearly identify the nature of the long-time decay of the OTOC,\nbut the results indicate that the OTOC decays much faster (possibly exponentially) in the Mott phase than in the metallic phase. This is consistent with the results for the spectral functions of the OTOC. The comparison to the noninteracting case (dashed curve in the bottom right panel of Fig.~\\ref{fig:results_fermion}) suggests that the correlated metallic phase has a similar (possibly power-law) decay behavior of the OTOC as in the noninteracting case.\n\n\n\\begin{figure}[t]\n\\begin{center}\n\\includegraphics[angle=-90, width=0.49\\columnwidth]{plot_otoc_b50_nn_Re.eps}\n\\includegraphics[angle=-90, width=0.49\\columnwidth]{plot_otoc_b50_nn_Im.eps}\n\\includegraphics[angle=-90, width=0.49\\columnwidth]{plot_coth_b50_nn_appendix.eps}\n\\includegraphics[angle=-90, width=0.49\\columnwidth]{plot_otoc_b50_nn_modulus.eps}\n\\caption{\nTop panels: Real and imaginary parts of the OTOC $\\langle \\hat n(t)\\hat n(0), \\hat n(t)\\hat n(0)\\rangle - 1$ for the single-orbital Hubbard model with $\\beta=50$ at half filling. The thin lines correspond to metallic solutions, \nwhile the thick line ($U=6$) corresponds to a Mott insulating solution. \nBottom left panel: Spectral function $\\mathscr A_{(nn)^2}(\\omega)=-\\frac{1}{\\pi}{\\rm Im}\\,C_{(n n)^2}^R(\\omega)$ multiplied by $\\coth\\big(\\frac{\\beta\\omega}{4}\\big)$.\nBottom right panel: Modulus of the OTOC $|\\langle \\hat n(t)\\hat n(0), \\hat n(t)\\hat n(0)\\rangle - 1|$ (normalized at $t=0$) on a log-log scale. \n}\n\\label{fig:results_nn}\n\\end{center}\n\\end{figure}\n\n\nIn Fig.~\\ref{fig:results_nn}, we plot the OTOC $\\langle \\hat n(t)\\hat n(0), \\hat n(t)\\hat n(0)\\rangle$\nfor the single-orbital Hubbard model in a similar manner as in Fig.~\\ref{fig:results_fermion}.\nSince at half filling this correlation function approaches $\\langle \\hat n\\rangle^4=1$ at long times\nand at sufficiently low temperature,\nwe perform the analytical continuation procedure for $\\langle \\hat n(t)\\hat n(0), \\hat n(t)\\hat n(0)\\rangle-1$. \nThe top panels of Fig.~\\ref{fig:results_nn} show the real and imaginary parts of this shifted OTOC. \nContrary to the case of $\\langle c_\\sigma^\\dagger(t)c_\\sigma(0), c_\\sigma^\\dagger(t)c_\\sigma(0)\\rangle$, \nthe amplitude of the OTOC $\\langle \\hat n(t)\\hat n(0), \\hat n(t)\\hat n(0)\\rangle-1$ is suppressed as one increases the interaction, which reflects the reduced charge fluctuations in the Mott insulator.\nThe bottom left panel in Fig.~\\ref{fig:results_nn} shows the analytically continued spectral function $\\mathscr A_{(nn)^2}(\\omega)=-\\frac{1}{\\pi}{\\rm Im}\\, C_{(nn)^2}^R(\\omega)$ multiplied by $\\coth\\big(\\frac{\\beta\\omega}{4}\\big)$. \nAgain we notice the opening of a gap in the Mott phase, and a corresponding shift of the spectral weight to higher energies. This results in more rapid oscillations of the OTOC in the Mott state.\nThe bottom right panel in Fig.~\\ref{fig:results_nn} plots the modulus of the OTOC $|\\langle \\hat n(t)\\hat n(0), \\hat n(t)\\hat n(0)\\rangle-1|$ (normalized at $t=0$) on a log-log scale.\nThe long-time behavior of the OTOC is consistent with a power-law decay in the metallic phase, and an exponential decay in the Mott insulating phase.\n\n\n\\begin{figure}[t]\n\\begin{center}\n\\includegraphics[angle=-90, width=0.49\\columnwidth]{plot_otoc_nsns_b50_draft_Re.eps}\n\\includegraphics[angle=-90, width=0.49\\columnwidth]{plot_otoc_nsns_b50_draft_Im.eps}\n\\includegraphics[angle=-90, width=0.49\\columnwidth]{plot_coth_b50_nsns_inset.eps}\n\\includegraphics[angle=-90, width=0.49\\columnwidth]{plot_otoc_nsns_b50_draft_modulus_zoom.eps}\n\\caption{\nTop panels: Real and imaginary parts of the OTOC $\\langle \\hat n_\\sigma(t)\\hat n_\\sigma(0), \\hat n_\\sigma(t)\\hat n_\\sigma(0)\\rangle  - \\langle \\hat n_\\sigma\\rangle^4$ for the single-orbital Hubbard model with $\\beta=50$ at half filling. All the colored lines correspond to metallic solutions. \nBottom left panel: Spectral function $\\mathscr A_{(n_\\sigma n_\\sigma)^2}(\\omega)=-\\frac{1}{\\pi}{\\rm Im}\\,C_{(n_\\sigma n_\\sigma)^2}^R(\\omega)$ multiplied by $\\coth\\big(\\frac{\\beta\\omega}{4}\\big)$. In the inset, we plot the imaginary-time four-point function $\\langle \\hat n_\\sigma(\\tau)\\hat n_\\sigma(0), \\hat n_\\sigma(\\tau)\\hat n_\\sigma(0)\\rangle-\\langle \\hat n_\\sigma\\rangle^4$, with the black line corresponding to the Mott insulating solution at $U=6$.  \nBottom right panel: Modulus of the OTOC $|\\langle \\hat n_\\sigma(t)\\hat n_\\sigma(0), \\hat n_\\sigma(t)\\hat n_\\sigma(0)\\rangle  - \\langle \\hat n_\\sigma\\rangle^4|$ (normalized at $t=0$) on a log-log scale.\n}\n\\label{fig:results_nsns}\n\\end{center}\n\\end{figure}\n\nFigure~\\ref{fig:results_nsns} \nshows the results of the OTOC $\\langle \\hat n_\\sigma(t)\\hat n_\\sigma(0), \\hat n_\\sigma(t)\\hat n_\\sigma(0)\\rangle$\nfor the single-orbital Hubbard model. At half filling, this correlation function approaches $\\langle \\hat n_\\sigma\\rangle^4=\\frac{1}{16}$ in the long-time limit and at sufficiently low temperature, \nso that we perform the analytical continuation procedure for $\\langle \\hat n_\\sigma(t)\\hat n_\\sigma(0), \\hat n_\\sigma(t)\\hat n_\\sigma(0)\\rangle-\\langle \\hat n_\\sigma\\rangle^4$. \nThe top panels of Fig.~\\ref{fig:results_nsns} show the real and imaginary parts of \nthis shifted OTOC. \nHere, we only present results for the metallic phase, because resolving the very sharp low-energy feature of the OTOC spectrum in the half-filled Mott state is challenging. One can see that coherent oscillations are not observed for this type of OTOC, and that the incoherent part becomes dominant compared to the other two OTOCs shown in Figs.~\\ref{fig:results_fermion} and \\ref{fig:results_nn}. The amplitude of the OTOC $\\langle \\hat n_\\sigma(t)\\hat n_\\sigma(0), \\hat n_\\sigma(t)\\hat n_\\sigma(0)\\rangle-\\langle \\hat n_\\sigma\\rangle^4$ is enhanced as $U$ increases. As we argue below, this is related to the enhancement of spin fluctuations \nand local moment formation close to the Mott transition.\n\nThe bottom left panel in Fig.~\\ref{fig:results_nsns} plots the analytically continued spectral function $\\mathscr A_{(n_\\sigma n_\\sigma)^2}(\\omega)=-\\frac{1}{\\pi}{\\rm Im}\\, C_{(n_\\sigma n_\\sigma)^2}^R(\\omega)$ multiplied by $\\coth\\big(\\frac{\\beta\\omega}{4}\\big)$. \nWe can see that the spectral weight is concentrated in the low-energy region as we approach the Mott transition point.\nThe inset in the bottom left panel of Fig.~\\ref{fig:results_nsns} shows the imaginary-time four-point function $\\langle \\hat n_\\sigma(\\tau)\\hat n_\\sigma(0), \\hat n_\\sigma(\\tau)\\hat n_\\sigma(0)\\rangle-\\langle \\hat n_\\sigma\\rangle^4$, with the black line corresponding to a Mott insulating solution ($U=6$).  \nFor $U\\gtrsim 4$, the imaginary-time four-point function does not decay to zero but remains relatively large near $\\tau=\\frac{\\beta}{4}$. \nThis behavior is reminiscent of the spin-freezing physics seen in the correlated Hund metal phase of multi-orbital Hubbard models, where the dynamical spin correlation function $\\langle \\hat S_z(\\tau)\\hat S_z(0)\\rangle$ is trapped at a finite value at long $\\tau$ \\cite{Werner2008}. \nIn fact, the OTOC $\\langle \\hat n_\\sigma(\\tau)\\hat n_\\sigma(0), \\hat n_\\sigma(\\tau)\\hat n_\\sigma(0)\\rangle$\nmeasures spin correlations (in addition to charge correlations) through $\\hat n_\\sigma=\\frac{1}{2}\\hat n+\\sigma\\hat S_z$.\nPhysically, the trapping of $\\langle \\hat S_z(\\tau)\\hat S_z(0)\\rangle$ signals the formation of frozen local magnetic moments.\nIn the metal-insulator crossover region, the scattering induced by these magnetic moments results in incoherent metal states. Intuitively, one might expect fast scrambling in the corresponding regions of the phase diagram. \nAlthough there is some resemblance with the spin freezing crossover,\nthe self-energy does not exhibit a square-root frequency dependence in the single-orbital case, and a spin-freezing crossover in the sense of Ref.~\\cite{Werner2008} does not exist.  \n\nThe bottom right panel of Fig.~\\ref{fig:results_nsns} plots the modulus of the OTOC $|\\langle \\hat n_\\sigma(t)\\hat n_\\sigma(0), \\hat n_\\sigma(t)\\hat n_\\sigma(0)\\rangle-\\langle \\hat n_\\sigma\\rangle^4|$ (normalized at $t=0$) on a log-log scale. \nThe dynamics of $\\langle \\hat n_\\sigma(t)\\hat n_\\sigma(0), \\hat n_\\sigma(t)\\hat n_\\sigma(0)\\rangle$ is quite distinct from that of $\\langle \\hat n(t)\\hat n(0), \\hat n(t)\\hat n(0)\\rangle$. As the Mott transition is approached, we observe a slow-down in the decay of the modulus, \nwhich indicates the presence of slowly fluctuating local moments. \nThe long-time behavior may be consistent with a power-law, although slow oscillations make it difficult to determine the long-time asymptotic form from the numerics. \n\n\n\\begin{figure}[t]\n\\begin{center}\n\\includegraphics[angle=-90, width=0.49\\columnwidth]{plot_fitnsns_b2.eps}\n\\includegraphics[angle=-90, width=0.49\\columnwidth]{plot_coeffc_draft.eps}\n\\caption{\nLeft panel: \nThe imaginary-time four-point function $\\langle \\hat n_\\sigma(\\tau)\\hat n_\\sigma(0), \\hat n_\\sigma(\\tau)\\hat n_\\sigma(0)\\rangle$, the squared density-density correlation function $\\langle \\hat n_\\sigma(\\tau)\\hat n_\\sigma(0)\\rangle^2$,\nand their difference for the single-orbital Hubbard model with $U=5.5$ and $\\beta=50$.\nThe dashed curve shows a fit of the difference with the function $f_\\text{fit}(\\tau)=a+b\\tau+c\\tau^2$ in the interval $\\tau/\\beta \\in [0: 0.2]$.\nRight panel: Fitting coefficient $c$ as a function of $U$ for indicated values of $\\beta$.\n\n}\n\\label{fig:coeffs}\n\\end{center}\n\\end{figure}\n\nFor large enough $\\beta$ and small enough $\\tau$, the OTOC (\\ref{otoc_nsns}) factorizes into $\\langle \\hat n_\\sigma(\\tau)\\hat n_\\sigma(0), \\hat n_\\sigma(\\tau)\\hat n_\\sigma(0)\\rangle \\approx \\langle \\hat n_\\sigma(\\tau)\\hat n_\\sigma(0)\\rangle^2$. \nOn the real-time axis, this corresponds to the decoupling $\\langle \\hat n_\\sigma(t)\\hat n_\\sigma(0), \\hat n_\\sigma(t)\\hat n_\\sigma(0)\\rangle \\approx \\langle \\hat n_\\sigma(t)\\hat n_\\sigma(0)\\rangle^2$, that is, \nthe OTOC is reduced to a product of ordinary density-density correlation functions.\nIn order to see whether nontrivial correlations are captured beyond the decoupled form by the OTOC function, \nwe consider the difference $\\langle \\hat n_\\sigma(\\tau)\\hat n_\\sigma(0), \\hat n_\\sigma(\\tau)\\hat n_\\sigma(0)\\rangle-\\langle \\hat n_\\sigma(\\tau)\\hat n_\\sigma(0)\\rangle^2$ and fit the short-time behavior of this function to $f_\\text{fit}(\\tau)=a+b\\tau+c\\tau^2$, as illustrated in the left panel of Fig.~\\ref{fig:coeffs}. The coefficients $b$ and $c$ serve as  indicators of the nontrivial correlations that cannot be attributed to the decoupled form of the OTOC.\nThe coefficient $c$ is plotted as a function of $U$ and for different $\\beta$ in the right hand panel. The coefficient $b$ (not shown) exhibits a similar trend. We notice that the OTOC picks up nontrival correlations near the metal-insulator transition, and in particular at \nintermediate\ntemperatures, while the Mott phase shows no such correlations. The crossover region  \nassociated with the emergence of local moments \ncorresponds, roughly, to the interaction range where the nontrivial correlations start to become significant. \n\n\n\\subsection{Two-orbital Hubbard model}\n\\label{sec:two-orbital hubbard}\n\nIn this section, we investigate OTOCs in the two-orbital Hubbard model with Hund coupling $J>0$ \nand density-density interactions. (Three-orbital results are presented in Appendix~\\ref{sec:3orbital}.) \nThe Hamiltonian is given by\n\\begin{align}\nH\n&=\n-v\\sum_{\\langle i,j\\rangle\\alpha\\sigma} (c^\\dagger_{i\\alpha\\sigma}c_{j\\alpha\\sigma}+ \\text{h.c.})\n-\\mu\\sum_{i\\alpha\\sigma} \\hat n_{i\\alpha\\sigma}\n+U\\sum_{i\\alpha} \\hat n_{i\\alpha\\uparrow}\\hat n_{i\\alpha\\downarrow}\n+(U-2J)\\sum_{i\\sigma} \\hat n_{i1\\sigma}\\hat n_{i2\\bar\\sigma}\n\\notag\n\\\\\n&\\quad\n+(U-3J)\\sum_{i\\sigma} \\hat n_{i1\\sigma}\\hat n_{i2\\sigma},\n\\end{align}\n\nwith $U$ being the intra-orbital interaction $U$, $U-2J$ the inter-orbital antiparallel-spin interaction, and $U-3J$ the inter-orbital parallel-spin interaction. This is the simplest model that shows a crossover from a spin-frozen metal to a Fermi-liquid metal as one dopes the half-filled Mott insulator, with the self-energy scaling as $\\text{Im}\\Sigma(\\omega_n)\\sim\\sqrt{\\omega_n}$ over a significant energy range in the crossover regime \\cite{Hafermann2012}.\nThe sketch of the phase diagram is shown in Fig.~\\ref{fig:illustration}. \nAs we have seen in the previous section, the spin-related OTOC of the type (\\ref{otoc_nsns}) is relevant for the analysis of  the spin-freezing crossover, so that we concentrate on this OTOC here.\nIn the following calculations, we choose $U=8$, $J=U/4$, and compute $\\langle \\hat n_\\sigma(t)\\hat n_\\sigma(0), \\hat n_\\sigma(t)\\hat n_\\sigma(0)\\rangle - \\langle n_\\sigma \\rangle^4$ \nas a function of filling at $\\beta=50$ (dashed lines in Fig.~\\ref{fig:illustration}). \n\n\n\\begin{figure}[t]\n\\begin{center}\n\\includegraphics[angle=-90, width=0.49\\columnwidth]{plot_qp_draft.eps}\\hfill\n\\includegraphics[angle=-90, width=0.49\\columnwidth]{plot_fit_2orbital.eps}\n\\caption{\nLeft panel: Imaginary part of the self-energy as a function of Matsubara frequency for the two-orbital Hubbard model with $U=8$, $J=U/4$, and $\\beta=50$ on a log-log scale, with the dashed line indicating $\\sqrt{\\omega_n}$ behavior. \nRight panel: The coefficient $c$ extracted from\nfitting $\\langle \\hat n_\\sigma(\\tau)\\hat n_\\sigma(0), \\hat n_\\sigma(\\tau)\\hat n_\\sigma(0)\\rangle-\\langle \\hat n_\\sigma(\\tau)\\hat n_\\sigma(0)\\rangle^2$ with a function $f_\\text{fit}(\\tau)=a+b\\tau+c\\tau^2$\nin the interval $\\tau/\\beta\\in [0: 0.05]$ for the two-orbital Hubbard model. \nThe spin-freezing crossover occurs roughly at the filling $n_\\sigma\\approx 0.28$ (blue curve in the left panel and blue arrow in the right panel). \n}\n\\label{fig:self}\n\\end{center}\n\\end{figure}\n\n\nThe left panel of Fig.~\\ref{fig:self} plots the imaginary part of the self-energy as a function of Matsubara frequency in order to identify the filling corresponding to the spin-freezing crossover \\cite{Werner2008}. An approximate square-root scaling is observed over a wide range of frequencies near $n_\\sigma\\approx 0.28$ (blue line). In the right panel of Fig.~\\ref{fig:self}, we present the coefficient $c$ obtained from a similar analysis as presented in Fig.~\\ref{fig:coeffs}, but with a fitting range $\\tau/\\beta\\in [0 : 0.05]$. Again, we find that the OTOC $\\langle \\hat n_\\sigma(t)\\hat n_\\sigma(0), \\hat n_\\sigma(t)\\hat n_\\sigma(0)\\rangle - \\langle \\hat n_\\sigma\\rangle^4$ detects nontrivial correlations in the spin-freezing crossover (blue arrow) and spin-frozen metal regime, but not in the Fermi liquid or the Mott insulating phase.\n\n\n\\begin{figure}[t]\n\\begin{center}\n\\includegraphics[angle=-90, width=0.49\\columnwidth]{plot_otoc_2orbital_Re.eps}\n\\includegraphics[angle=-90, width=0.486\\columnwidth]{plot_otoc_2orbital_Im.eps}\n\\includegraphics[angle=-90, width=0.484\\columnwidth]{plot_spectra_coth_2orbital_inset.eps}\n\\includegraphics[angle=-90, width=0.486\\columnwidth]{plot_otoc_2orbital_modulus_inset.eps}\n\\caption{\nTop panels: Real and imaginary parts of the OTOC $\\langle \\hat n_\\sigma(t)\\hat n_\\sigma(0), \\hat n_\\sigma(t)\\hat n_\\sigma(0)\\rangle  - \\langle \\hat n_\\sigma\\rangle^4$ for the two-orbital Hubbard model with $U=8$, $J=U/4$, $\\beta=50$ and indicated fillings. \nBottom left panel: Spectral function $\\mathscr A_{(n_\\sigma n_\\sigma)^2}(\\omega)=-\\frac{1}{\\pi}{\\rm Im}\\,C_{(n_\\sigma n_\\sigma)^2}^R(\\omega)$ multiplied by $\\coth\\big(\\frac{\\beta\\omega}{4}\\big)$. In the inset, we plot the imaginary-time four-point function $\\langle \\hat n_\\sigma(\\tau)\\hat n_\\sigma(0), \\hat n_\\sigma(\\tau)\\hat n_\\sigma(0)\\rangle-\\langle \\hat n_\\sigma\\rangle^4$. \nBottom right panel: Modulus of the OTOC $|\\langle \\hat n_\\sigma(t)\\hat n_\\sigma(0), \\hat n_\\sigma(t)\\hat n_\\sigma(0)\\rangle  - \\langle \\hat n_\\sigma\\rangle^4|$ (normalized at $t=0$) on a log-log scale.\nIn the inset of the bottom right panel, we illustrate the approximate data collapse obtained by considering the spin-freezing crossover regimes for different values of $J$. \nIn all the panels, the thick blue curves represent the results for the spin-freezing crossover region. \n}\n\\label{fig:2orbital}\n\\end{center}\n\\end{figure}\n\n\nThe top panels of Fig.~\\ref{fig:2orbital} show the real and imaginary parts of the OTOC\n$\\langle \\hat n_\\sigma(t)\\hat n_\\sigma(0), \\hat n_\\sigma(t)\\hat n_\\sigma(0)\\rangle-\\langle \\hat n_\\sigma\\rangle^4$\nfor the two-orbital Hubbard model. In the spin-frozen phase ($n_\\sigma \\gtrsim 0.3$), both components exhibit \nhighly incoherent oscillations, which get suppressed as the filling is reduced. As one moves across the crossover point ($n_\\sigma \\sim 0.28$),\nthe oscillations completely disappear, and the real part of the OTOC starts to overshoot to the negative side\nin the Fermi liquid regime.\nThe bottom left panel in Fig.~\\ref{fig:2orbital} presents the analytically continued spectral function $\\mathscr A_{(n_\\sigma n_\\sigma)^2}(\\omega)=-\\frac{1}{\\pi}{\\rm Im}\\,C_{(n_\\sigma n_\\sigma)^2}^R(\\omega)$ multiplied by $\\coth\\big(\\frac{\\beta\\omega}{4}\\big)$. \nIn the spin-frozen regime ($n_\\sigma \\gtrsim 0.3$) a sharp peak appears at low energy, which originates from the saturation of the imaginary-time four point function $\\langle \\hat n_\\sigma(\\tau)\\hat n_\\sigma(0), \\hat n_\\sigma(\\tau)\\hat n_\\sigma(0)\\rangle$ at large $\\tau$ (see the inset of the bottom left panel of Fig.~\\ref{fig:2orbital}). This low-energy peak represents the slow dynamics of the frozen spins, and translates into a slow long-time decay of $|\\langle \\hat n_\\sigma(t)\\hat n_\\sigma(0), \\hat n_\\sigma(t)\\hat n_\\sigma(0)\\rangle-\\langle \\hat n_\\sigma\\rangle^4|$,\nas shown in the bottom right panel of Fig.~\\ref{fig:2orbital}\non a log-log scale. Again, it is difficult to clearly resolve the long-time behavior of the OTOC due to the limited\naccuracy of the analytic continuation, but in the spin-freezing crossover regime (thick blue curves)\nthe decay of the OTOC is consistent with a power law $\\sim 1/t^{1.5}$ at least for $2\\lesssim t \\lesssim 5$. \nThe shorter-time behavior ($0.5\\lesssim t \\lesssim 2$) is instead well fitted by an exponential decay $\\sim e^{-\\alpha t}$ with a decay constant $\\alpha=0.78$. \nThis exponent exhibits\na strong dependence on the Hund coupling. \n\nIn fact, the OTOCs in the spin-freezing regime for $J=U/4$ and $J=U/6$ can be approximately collapsed by plotting them as a function of $tJ$ (see the inset in the bottom right panel of Fig.~\\ref{fig:2orbital}), which indicates that the Hund coupling is the parameter which controls the dynamics in this regime. \nThis is distinct from the Lyapunov behavior in which the exponential decay scales with $t/\\beta$ \\cite{Bagrets2017}.\nA possible reason is that we are still far from the large $N$ limit (where $N$ corresponds to the number of orbitals times the number of spin degrees of freedom) so that the large $N$ behavior \nsuch as the Lyapunov growth is not observed in our calculations. \nIn fact, the strong dependence of the time scale on $J$ and weak dependence on $\\beta$ is consistent\nwith the behavior of the finite $N$ SYK model in Ref.~\\cite{FuSachdev2016}.\nIn the spin-frozen regime ($n_\\sigma\\gtrsim 0.3$) the decay is much slower than near the crossover point, while in the Fermi liquid regime the decay is accelerated, approaching the free fermion behavior of $t^{-3}$\nas shown in Fig.~\\ref{analytic continuation}. \nQualitatively similar results for this OTOC are obtained \nfor the three-orbital Hubbard model (see Appendix~\\ref{sec:3orbital}).\n\nThe time dependence of the modulus of \n$\\langle \\hat n_\\sigma(\\tau)\\hat n_\\sigma(0), \\hat n_\\sigma(\\tau)\\hat n_\\sigma(0)\\rangle-\\langle \\hat n_\\sigma\\rangle^4$ \nin the spin-freezing crossover regime of multi-orbital Hubbard models (exponential decay crossing over into a power-law) bears a close resemblance to a (different) OTOC for the SYK model discussed in Ref.~\\cite{Bagrets2017}. \nAlso, a recent study of yet another type of OTOCs for the finite-$N$ SYK model found a qualitatively similar decay as observed here in the spin-freezing crossover regime \\cite{FuSachdev2016}. In the following section, we will make the connection to the SYK dynamics more quantitative.   \n\n\n\\subsection{Comparison to the SKY model}\n\\label{sec:SYK}\n\nIn this section, we compare the results of the multi-orbital Hubbard models to the SYK model.\nWe consider the particle-hole symmetric SYK model of complex fermions \\cite{FuSachdev2016},\nwhose Hamiltonian is given by\n\\begin{align}\nH\n&=\n\\frac{1}{(2N)^{3/2}} \\sum_{i,j,k,l=1}^N \nJ_{ij;kl} (c_i^\\dagger c_j^\\dagger c_k c_l\n+\\delta_{ik}n c_j^\\dagger c_l\n-\\delta_{il}n c_j^\\dagger c_k\n-\\delta_{jk}n c_i^\\dagger c_l\n+\\delta_{jl}n c_i^\\dagger c_k).\n\\end{align}\nThe coupling constant $J_{ij;kl}$ is a gaussian random variable, satisfying $J_{ij;kl}=-J_{ji;kl}=-J_{ij;lk}=J_{kl;ij}^\\ast$, and\n\\begin{align}\n\\overline{({\\rm Re}\\, J_{ij;kl})^2}\n&=\n\\begin{cases}\nJ_{\\rm SYK}^2/2 & (i,j)\\neq (k,l) \\\\\nJ_{\\rm SYK}^2 & (i,j)=(k,l)\n\\end{cases},\n\\\\\n\\overline{({\\rm Im}\\, J_{ij;kl})^2}\n&=\n\\begin{cases}\nJ_{\\rm SYK}^2/2 & (i,j)\\neq (k,l) \\\\\n0 & (i,j)=(k,l)\n\\end{cases},\n\\end{align}\nwhere the overline represents an average over each realization of $J_{ij;kl}$. The parameter \n$J_{\\rm SYK}$ ($J_{\\rm SYK}^{-1}$) defines the unit of energy (time) in the following calculations. \nThe system is particle-hole symmetric when\n$n=0.5$, which fixes the total number of fermions as $N^{-1}\\sum_{i=1}^N \\langle c_i^\\dagger c_i\\rangle=0.5$.\nWe take the particle-hole symmetric form of the SYK model because it has been well studied previously \\cite{FuSachdev2016} and avoids the effect of the filling drift.\nWe numerically solve the model by exact diagonalization for finite $N(\\le 12)$, and evaluate the density-density OTOC\n$\\overline{\\langle \\hat n_i(t)\\hat n_i(0), \\hat n_i(t)\\hat n_i(0)\\rangle-\\langle \\hat n_i\\rangle^4}$\nand its imaginary-time counterpart.\n\n\\begin{figure}[t]\n\\begin{center}\n\n\\includegraphics[angle=-90,width=0.49\\columnwidth]{plot_fitnsns_syk.eps}\n\n\\includegraphics[angle=-90,width=0.49\\columnwidth]{plot_coeffc_syk.eps}\n\\caption{\nLeft panel: \nThe imaginary-time four-point function $\\overline{\\langle \\hat n_i(\\tau)\\hat n_i(0), \\hat n_i(\\tau)\\hat n_i(0)\\rangle}$, the density-density correlation function $\\overline{\\langle \\hat n_i(\\tau)\\hat n_i(0)\\rangle^2}$,\nand their difference for the SYK model with $N=12$ and $\\beta J_{\\rm SYK}=10$.\nThe dashed curve shows a fit of the difference with a function $f_\\text{fit}(\\tau)=a+b\\tau+c\\tau^2$ in the interval $\\tau/\\beta \\in [0: 0.2]$. \nRight panel: Fitting coefficient $c$ as a function of $T/J_{\\rm SYK}$.\n}\n\\label{fig:SYK fit}\n\\end{center}\n\\end{figure}\n\nIn the left panel of Fig.~\\ref{fig:SYK fit}, we plot the imaginary-time four-point function \n$\\overline{\\langle \\hat n_i(\\tau)\\hat n_i(0), \\hat n_i(\\tau)\\hat n_i(0)\\rangle}$\nfor the SYK model with $N=12$ and $\\beta J_{\\rm SYK}=10$. \nAt low temperature, we expect that $\\overline{\\langle \\hat n_i(\\tau)\\hat n_i(0), \\hat n_i(\\tau)\\hat n_i(0)\\rangle}$ is decoupled into $\\overline{\\langle \\hat n_i(\\tau)\\hat n_i(0)\\rangle^2}$.\nTo see whether nontrivial correlations beyond the decoupled form are present, we fit the difference\n$\\overline{\\langle \\hat n_i(\\tau)\\hat n_i(0), \\hat n_i(\\tau)\\hat n_i(0)\\rangle}\n-\\overline{\\langle \\hat n_i(\\tau)\\hat n_i(0)\\rangle^2}$ with a function $f_{\\rm fit}(\\tau)=a+b\\tau+c\\tau^2$\nin the interval $\\tau/\\beta\\in[0:0.2]$, in the same manner as for the Hubbard models.\nThe obtained coefficient $c$ is plotted as a function of the temperature $T$ in the right panel of\nFig.~\\ref{fig:SYK fit}. One can see that nontrivial correlations exist in a wide range of temperature.\nEspecially, they are enhanced in the intermediate temperature regime. This is because in the zero-temperature limit\nthe four-point function is decoupled as \n$\\langle \\hat n_i(\\tau)\\hat n_i(0), \\hat n_i(\\tau)\\hat n_i(0)\\rangle \\approx \\langle \\hat n_i(\\tau)\\hat n_i(0)\\rangle^2$, while in the high-temperature limit ($\\beta\\to 0$) the imaginary-time dependence is washed out.\n\n\n\n\n\\begin{figure}[t]\n\\begin{center}\n\\includegraphics[angle=-90, width=0.49\\columnwidth]{plot_syk_Re.eps}\n\\includegraphics[angle=-90, width=0.49\\columnwidth]{plot_syk_Im.eps}\n\\includegraphics[angle=-90, width=0.49\\columnwidth]{plot_syk_modulus-log.eps}\n\\includegraphics[angle=-90, width=0.49\\columnwidth]{plot_syk_modulus.eps}\n\\caption{\nThe density-density OTOC $\\overline{\\langle \\hat n_i(t)\\hat n_i(0), \\hat n_i(t)\\hat n_i(0)\\rangle-\\langle \\hat n_i\\rangle^4}$ for the particle-hole symmetric SYK model averaged over $10^2$ samples with $\\beta J_{\\rm SYK}=100$ and various $N$. \nThe top left and right panels show the real and imaginary parts of the OTOC,\nwhile the bottom left and right panels show the log and log-log plots of the modulus of the OTOC (normalized at $t=0$).\nThe dashed line shows a power-law decay $\\propto t^{-1.6}$ for comparison.}\n\\label{fig:SYK}\n\\end{center}\n\\end{figure}\n\nIn Fig.~\\ref{fig:SYK}, we show the numerical results of the density-density OTOC for $\\beta J_{\\rm SYK}=100$.\nThe real part of the OTOC rapidly drops from the initial value within the time scale of $tJ_{\\rm SYK}\\sim 2$,\nand slowly decays to zero in the long time limit.\nAs one increases $N$, the oscillations that appear at longer time tend to be suppressed.\nOne can see that the result for $N=12$ in the top panels of Fig.~\\ref{fig:SYK} closely resembles the OTOC in \nthe spin-freezing crossover regime of the two-orbital Hubbard model shown\nby the blue curve in the top panels of Fig.~\\ref{fig:2orbital},\nwhile it differs from the single-orbital result shown in the top panels of Fig.~\\ref{fig:results_nsns}.\nDue to the limitation of our calculations to small system sizes ($N\\le 12$), we do not clearly observe\nan exponential growth ($\\sim c_0-c_1 e^{\\lambda t}$)  at short times or an exponential decay ($\\sim e^{-\\alpha t}$) \nat intermediate times. \nHowever, we find that for $tJ_{\\rm SYK}\\gtrsim 2$ the OTOC decays\napproximately in a power law $\\propto t^{-\\gamma}$ with $\\gamma=1.6$ (see bottom right panel in Fig.~\\ref{fig:SYK}). \nThe time interval exhibiting the power-law decay becomes longer as we increase $N$.  \nThe temperature dependence of the OTOC for the SYK model is shown in Fig.~\\ref{fig:SYK-T}.\nThe time scale of the initial drop is more or less independent of the temperature, while \nthe power-law-like decay is only visible in the low-temperature regime.\n\n\\begin{figure}[t]\n\\begin{center}\n\\includegraphics[angle=-90, width=0.49\\columnwidth]{plot_syk_beta_re.eps}\n\\includegraphics[angle=-90, width=0.49\\columnwidth]{plot_syk_beta_im.eps}\n\\includegraphics[angle=-90, width=0.49\\columnwidth]{plot_syk_beta_modulus-log.eps}\n\\includegraphics[angle=-90, width=0.49\\columnwidth]{plot_syk_beta_modulus.eps}\n\\caption{\nTemperature dependence of the density-density OTOC\n$\\overline{\\langle \\hat n_i(t)\\hat n_i(0), \\hat n_i(t)\\hat n_i(0)\\rangle-\\langle \\hat n_i\\rangle^4}$ for the particle-hole symmetric SYK model averaged over $10^2$ samples with $N=12$.\nThe top left and right panels show the real and imaginary parts of the OTOC,\nwhile the bottom left and right panels show the log and log-log plots of the modulus of the OTOC.\nThe dashed line shows a power-law decay $\\propto t^{-1.6}$ for comparison.}\n\\label{fig:SYK-T}\n\\end{center}\n\\end{figure}\n\nIn Ref.~\\onlinecite{Werner2018}, it has been argued that the spin-freezing crossover regime of multi-orbital Hubbard models is effectively described by the SYK model in which $J_{\\rm SYK}$ is replaced by the Hund coupling $J$. The results in Fig.~\\ref{fig:2orbital} are for the two-orbital Hubbard model with $U=8$, $J/U=1/4$ and $\\beta=50$, corresponding to $\\beta J=100$, which means that it is meaningful to directly compare the blue lines in Fig.~\\ref{fig:2orbital} with the blue lines in Figs.~\\ref{fig:SYK} and \\ref{fig:SYK-T}. Not only is the qualitative agreement remarkable, even the power-law exponent measured for the two-orbital Hubbard model ($\\gamma=1.5$) is very close to the exponent extracted from the finite-$N$ SYK calculation. \nThe power-law decay in the three-orbital case is approximately $1/t^{1.75}$ (see Appendix~\\ref{sec:3orbital}), and thus also in semi-quantitative agreement with the SYK model behavior. \nThis nontrivial result provides further support for the identification of the spin-freezing crossover regime of multi-orbital Hubbard systems with an SYK strange metal. \n\nIn the large-$N$ and low-$T$ limit of the SYK model, we expect that the power-law behavior approaches $1/t^2$, since in this limit the OTOC is decoupled as $\\langle \\hat n_i(t)\\hat n_i(0), \\hat n_i(t)\\hat n_i(0)\\rangle\\approx \\langle \\hat n_i(t)\\hat n_i(0)\\rangle^2$ and the decay of $\\langle \\hat n_i(t)\\hat n_i(0)\\rangle$ is dominated by what corresponds to the slow spin relaxation $\\langle \\hat S_z(t)\\hat S_z(0)\\rangle\\sim 1/t$ in the Sachdev-Ye model.\n\n\n\n\n\n4.4 Comparison to the SKY model\n\\subsection{Comparison to the SKY model}\n\\label{sec:SYK}\n\nIn this section, we compare the results of the multi-orbital Hubbard models to the SYK model.\nWe consider the particle-hole symmetric SYK model of complex fermions \\cite{FuSachdev2016},\nwhose Hamiltonian is given by\n\\begin{align}\nH\n&=\n\\frac{1}{(2N)^{3/2}} \\sum_{i,j,k,l=1}^N \nJ_{ij;kl} (c_i^\\dagger c_j^\\dagger c_k c_l\n+\\delta_{ik}n c_j^\\dagger c_l\n-\\delta_{il}n c_j^\\dagger c_k\n-\\delta_{jk}n c_i^\\dagger c_l\n+\\delta_{jl}n c_i^\\dagger c_k).\n\\end{align}\nThe coupling constant $J_{ij;kl}$ is a gaussian random variable, satisfying $J_{ij;kl}=-J_{ji;kl}=-J_{ij;lk}=J_{kl;ij}^\\ast$, and\n\\begin{align}\n\\overline{({\\rm Re}\\, J_{ij;kl})^2}\n&=\n\\begin{cases}\nJ_{\\rm SYK}^2/2 & (i,j)\\neq (k,l) \\\\\nJ_{\\rm SYK}^2 & (i,j)=(k,l)\n\\end{cases},\n\\\\\n\\overline{({\\rm Im}\\, J_{ij;kl})^2}\n&=\n\\begin{cases}\nJ_{\\rm SYK}^2/2 & (i,j)\\neq (k,l) \\\\\n0 & (i,j)=(k,l)\n\\end{cases},\n\\end{align}\nwhere the overline represents an average over each realization of $J_{ij;kl}$. The parameter \n$J_{\\rm SYK}$ ($J_{\\rm SYK}^{-1}$) defines the unit of energy (time) in the following calculations. \nThe system is particle-hole symmetric when\n$n=0.5$, which fixes the total number of fermions as $N^{-1}\\sum_{i=1}^N \\langle c_i^\\dagger c_i\\rangle=0.5$.\nWe take the particle-hole symmetric form of the SYK model because it has been well studied previously \\cite{FuSachdev2016} and avoids the effect of the filling drift.\nWe numerically solve the model by exact diagonalization for finite $N(\\le 12)$, and evaluate the density-density OTOC\n$\\overline{\\langle \\hat n_i(t)\\hat n_i(0), \\hat n_i(t)\\hat n_i(0)\\rangle-\\langle \\hat n_i\\rangle^4}$\nand its imaginary-time counterpart.\n\n\\begin{figure}[t]\n\\begin{center}\n\n\\includegraphics[angle=-90,width=0.49\\columnwidth]{plot_fitnsns_syk.eps}\n\n\\includegraphics[angle=-90,width=0.49\\columnwidth]{plot_coeffc_syk.eps}\n\\caption{\nLeft panel: \nThe imaginary-time four-point function $\\overline{\\langle \\hat n_i(\\tau)\\hat n_i(0), \\hat n_i(\\tau)\\hat n_i(0)\\rangle}$, the density-density correlation function $\\overline{\\langle \\hat n_i(\\tau)\\hat n_i(0)\\rangle^2}$,\nand their difference for the SYK model with $N=12$ and $\\beta J_{\\rm SYK}=10$.\nThe dashed curve shows a fit of the difference with a function $f_\\text{fit}(\\tau)=a+b\\tau+c\\tau^2$ in the interval $\\tau/\\beta \\in [0: 0.2]$. \nRight panel: Fitting coefficient $c$ as a function of $T/J_{\\rm SYK}$.\n}\n\\label{fig:SYK fit}\n\\end{center}\n\\end{figure}\n\nIn the left panel of Fig.~\\ref{fig:SYK fit}, we plot the imaginary-time four-point function \n$\\overline{\\langle \\hat n_i(\\tau)\\hat n_i(0), \\hat n_i(\\tau)\\hat n_i(0)\\rangle}$\nfor the SYK model with $N=12$ and $\\beta J_{\\rm SYK}=10$. \nAt low temperature, we expect that $\\overline{\\langle \\hat n_i(\\tau)\\hat n_i(0), \\hat n_i(\\tau)\\hat n_i(0)\\rangle}$ is decoupled into $\\overline{\\langle \\hat n_i(\\tau)\\hat n_i(0)\\rangle^2}$.\nTo see whether nontrivial correlations beyond the decoupled form are present, we fit the difference\n$\\overline{\\langle \\hat n_i(\\tau)\\hat n_i(0), \\hat n_i(\\tau)\\hat n_i(0)\\rangle}\n-\\overline{\\langle \\hat n_i(\\tau)\\hat n_i(0)\\rangle^2}$ with a function $f_{\\rm fit}(\\tau)=a+b\\tau+c\\tau^2$\nin the interval $\\tau/\\beta\\in[0:0.2]$, in the same manner as for the Hubbard models.\nThe obtained coefficient $c$ is plotted as a function of the temperature $T$ in the right panel of\nFig.~\\ref{fig:SYK fit}. One can see that nontrivial correlations exist in a wide range of temperature.\nEspecially, they are enhanced in the intermediate temperature regime. This is because in the zero-temperature limit\nthe four-point function is decoupled as \n$\\langle \\hat n_i(\\tau)\\hat n_i(0), \\hat n_i(\\tau)\\hat n_i(0)\\rangle \\approx \\langle \\hat n_i(\\tau)\\hat n_i(0)\\rangle^2$, while in the high-temperature limit ($\\beta\\to 0$) the imaginary-time dependence is washed out.\n\n\n\n\n\\begin{figure}[t]\n\\begin{center}\n\\includegraphics[angle=-90, width=0.49\\columnwidth]{plot_syk_Re.eps}\n\\includegraphics[angle=-90, width=0.49\\columnwidth]{plot_syk_Im.eps}\n\\includegraphics[angle=-90, width=0.49\\columnwidth]{plot_syk_modulus-log.eps}\n\\includegraphics[angle=-90, width=0.49\\columnwidth]{plot_syk_modulus.eps}\n\\caption{\nThe density-density OTOC $\\overline{\\langle \\hat n_i(t)\\hat n_i(0), \\hat n_i(t)\\hat n_i(0)\\rangle-\\langle \\hat n_i\\rangle^4}$ for the particle-hole symmetric SYK model averaged over $10^2$ samples with $\\beta J_{\\rm SYK}=100$ and various $N$. \nThe top left and right panels show the real and imaginary parts of the OTOC,\nwhile the bottom left and right panels show the log and log-log plots of the modulus of the OTOC (normalized at $t=0$).\nThe dashed line shows a power-law decay $\\propto t^{-1.6}$ for comparison.}\n\\label{fig:SYK}\n\\end{center}\n\\end{figure}\n\nIn Fig.~\\ref{fig:SYK}, we show the numerical results of the density-density OTOC for $\\beta J_{\\rm SYK}=100$.\nThe real part of the OTOC rapidly drops from the initial value within the time scale of $tJ_{\\rm SYK}\\sim 2$,\nand slowly decays to zero in the long time limit.\nAs one increases $N$, the oscillations that appear at longer time tend to be suppressed.\nOne can see that the result for $N=12$ in the top panels of Fig.~\\ref{fig:SYK} closely resembles the OTOC in \nthe spin-freezing crossover regime of the two-orbital Hubbard model shown\nby the blue curve in the top panels of Fig.~\\ref{fig:2orbital},\nwhile it differs from the single-orbital result shown in the top panels of Fig.~\\ref{fig:results_nsns}.\nDue to the limitation of our calculations to small system sizes ($N\\le 12$), we do not clearly observe\nan exponential growth ($\\sim c_0-c_1 e^{\\lambda t}$)  at short times or an exponential decay ($\\sim e^{-\\alpha t}$) \nat intermediate times. \nHowever, we find that for $tJ_{\\rm SYK}\\gtrsim 2$ the OTOC decays\napproximately in a power law $\\propto t^{-\\gamma}$ with $\\gamma=1.6$ (see bottom right panel in Fig.~\\ref{fig:SYK}). \nThe time interval exhibiting the power-law decay becomes longer as we increase $N$.  \nThe temperature dependence of the OTOC for the SYK model is shown in Fig.~\\ref{fig:SYK-T}.\nThe time scale of the initial drop is more or less independent of the temperature, while \nthe power-law-like decay is only visible in the low-temperature regime.\n\n\\begin{figure}[t]\n\\begin{center}\n\\includegraphics[angle=-90, width=0.49\\columnwidth]{plot_syk_beta_re.eps}\n\\includegraphics[angle=-90, width=0.49\\columnwidth]{plot_syk_beta_im.eps}\n\\includegraphics[angle=-90, width=0.49\\columnwidth]{plot_syk_beta_modulus-log.eps}\n\\includegraphics[angle=-90, width=0.49\\columnwidth]{plot_syk_beta_modulus.eps}\n\\caption{\nTemperature dependence of the density-density OTOC\n$\\overline{\\langle \\hat n_i(t)\\hat n_i(0), \\hat n_i(t)\\hat n_i(0)\\rangle-\\langle \\hat n_i\\rangle^4}$ for the particle-hole symmetric SYK model averaged over $10^2$ samples with $N=12$.\nThe top left and right panels show the real and imaginary parts of the OTOC,\nwhile the bottom left and right panels show the log and log-log plots of the modulus of the OTOC.\nThe dashed line shows a power-law decay $\\propto t^{-1.6}$ for comparison.}\n\\label{fig:SYK-T}\n\\end{center}\n\\end{figure}\n\nIn Ref.~\\onlinecite{Werner2018}, it has been argued that the spin-freezing crossover regime of multi-orbital Hubbard models is effectively described by the SYK model in which $J_{\\rm SYK}$ is replaced by the Hund coupling $J$. The results in Fig.~\\ref{fig:2orbital} are for the two-orbital Hubbard model with $U=8$, $J/U=1/4$ and $\\beta=50$, corresponding to $\\beta J=100$, which means that it is meaningful to directly compare the blue lines in Fig.~\\ref{fig:2orbital} with the blue lines in Figs.~\\ref{fig:SYK} and \\ref{fig:SYK-T}. Not only is the qualitative agreement remarkable, even the power-law exponent measured for the two-orbital Hubbard model ($\\gamma=1.5$) is very close to the exponent extracted from the finite-$N$ SYK calculation. \nThe power-law decay in the three-orbital case is approximately $1/t^{1.75}$ (see Appendix~\\ref{sec:3orbital}), and thus also in semi-quantitative agreement with the SYK model behavior. \nThis nontrivial result provides further support for the identification of the spin-freezing crossover regime of multi-orbital Hubbard systems with an SYK strange metal. \n\nIn the large-$N$ and low-$T$ limit of the SYK model, we expect that the power-law behavior approaches $1/t^2$, since in this limit the OTOC is decoupled as $\\langle \\hat n_i(t)\\hat n_i(0), \\hat n_i(t)\\hat n_i(0)\\rangle\\approx \\langle \\hat n_i(t)\\hat n_i(0)\\rangle^2$ and the decay of $\\langle \\hat n_i(t)\\hat n_i(0)\\rangle$ is dominated by what corresponds to the slow spin relaxation $\\langle \\hat S_z(t)\\hat S_z(0)\\rangle\\sim 1/t$ in the Sachdev-Ye model.\n\n\n\n\n\n",
        "label": "fig:SYK",
        "Descriptive_question1": "What is the power-law decay exponent shown by the dashed line in figure_12?",
        "Descriptive_question2": "What is the value of βJ_SYK used in figure_12?",
        "Reasoning_question1": "How does the decay behavior of the OTOC in figure_12 change with increasing N, and what might this imply about the system's dynamics?",
        "Reasoning_question2": "Why might the oscillations in the real and imaginary parts of the OTOC in figure_12 be suppressed as N increases, and how does this relate to the SYK model's characteristics?",
        "Descriptive_answer1": "1.6",
        "Descriptive_answer2": "100",
        "Reasoning_answer1": "In figure_12, as N increases, the oscillations in the OTOC's real and imaginary parts become suppressed and the power-law decay behavior extends over a longer time interval. This suggests that larger system size (N) smoothens out finite-size fluctuations and leads to a more well-defined power-law decay of the OTOC, indicating more robust and coherent long-time dynamical behavior characteristic of the SYK model in the large-N limit.",
        "Reasoning_answer2": "The suppression of oscillations in the real and imaginary parts of the OTOC as N increases can be attributed to the self-averaging property of large-N SYK models, where random fluctuations average out across many degrees of freedom. This leads to a smoother decay profile of the OTOC at large N, reflecting the chaotic, strongly-interacting nature of the SYK model, where the dynamics become more universal and less dependent on individual sample fluctuations."
    },
    {
        "paper_id": "1812.04217.json",
        "image_id": "figure_13",
        "image_path": [
            "/home/yz979/scratch/chengye/scimolmo/test-images-tables/dataset_imagestables/1812.04217/images/plot_syk_beta_re.eps"
        ],
        "caption": "\nTemperature dependence of the density-density OTOC\n$\\overline{\\langle \\hat n_i(t)\\hat n_i(0), \\hat n_i(t)\\hat n_i(0)\\rangle-\\langle \\hat n_i\\rangle^4}$ for the particle-hole symmetric SYK model averaged over $10^2$ samples with $N=12$.\nThe top left and right panels show the real and imaginary parts of the OTOC,\nwhile the bottom left and right panels show the log and log-log plots of the modulus of the OTOC.\nThe dashed line shows a power-law decay $\\propto t^{-1.6}$ for comparison.",
        "classify": "Chart",
        "section_info": "4 Numerical results for the Hubbard model\n\\section{Numerical results for the Hubbard model}\n\\label{numerical results}\n\n\\subsection{Observables and numerical procedure}\n\nWe consider the single-orbital, two-orbital, and three-orbital Hubbard models on an infinitely-connected Bethe lattice, which can be solved exactly within DMFT \\cite{GeorgesKotliarKrauthRozenberg1996}. In this case, the noninteracting density of states is semicircular with bandwidth $4v_\\ast$, for which there exists a simplified DMFT self-consistency condition between the hybridization function $\\Delta_{\\alpha\\sigma}$ of the DMFT impurity problem and the local (impurity) Green's function $G_{\\alpha\\sigma}$: \n\n\\begin{equation}\n\\Delta_{\\alpha\\sigma}(\\tau)=v_\\ast^2 G_{\\alpha\\sigma}(\\tau), \n\\label{semicirc}\n\\end{equation}\n\nwith $\\alpha$ the orbital and $\\sigma$ the spin index. We will use $v_\\ast$ as the unit of energy and measure time in units of $\\hbar/v_\\ast$. \n\nThree different types of imaginary-time four-point functions $C_{(AB)^2}^M(\\tau)$\nof the form of Eq.~(\\ref{imaginary-time OTOC})\nwith $(\\hat A, \\hat B)=(c_\\sigma^\\dagger, c_\\sigma), (\\hat n_\\sigma, \\hat n_\\sigma)$, and $(\\hat n, \\hat n)$\nare calculated in the interval $0\\le \\tau\\le \\frac{\\beta\\hbar}{2}$: \n\\begin{eqnarray}\nC_{(c^\\dagger_\\sigma c_\\sigma)^2}^M(\\tau) &=& -\\langle c^\\dagger_\\sigma(\\tau+\\tfrac{\\beta\\hbar}{2}) c_\\sigma(\\tfrac{\\beta\\hbar}{2}) c^\\dagger_\\sigma(\\tau) c_\\sigma(0) \\rangle, \\label{fermion_otoc}\\\\\nC_{(n_\\sigma n_\\sigma)^2}^M(\\tau) &=& -\\langle \\hat n_\\sigma(\\tau+\\tfrac{\\beta\\hbar}{2})\\hat n_\\sigma(\\tfrac{\\beta\\hbar}{2})\\hat n_\\sigma(\\tau)\\hat n_\\sigma(0) \\rangle, \\label{otoc_nsns}\\\\\nC_{(nn)^2}^M(\\tau) &=& -\\langle \\hat n(\\tau+\\tfrac{\\beta\\hbar}{2})\\hat n(\\tfrac{\\beta\\hbar}{2})\\hat n(\\tau)\\hat n(0) \\rangle,\\label{otoc_nn}\n\\end{eqnarray}\n\nwhere $c_\\sigma^\\dagger$ ($c_\\sigma$) are the fermionic creation (annihilation) operators for spin $\\sigma$ (and orbital $\\alpha=1$ in the multi-orbital case), $\\hat n_\\sigma=c^\\dagger_\\sigma c_\\sigma$ is the corresponding spin-dependent density operator, and $\\hat n=\\hat n_\\uparrow + \\hat n_\\downarrow$ the total density operator. Note that in all the three cases above we have $\\hat B=\\hat A^\\dagger$. \nWe measure these local correlation functions in the impurity model using a hybridization expansion continuous-time Monte Carlo algorithm (CT-HYB) \\cite{Werner2006}. In this algorithm, the \ntwo-particle\nGreen's functions of the type (\\ref{fermion_otoc}) can be measured by removing two hybridization lines, i.e., from the elements of the inverse hybridization matrix, while the density-density correlation functions can be easily measured either by insertion of density operators (matrix formalism) \\cite{Werner2006Kondo} or by reading off the occupation of the orbitals at the four time points in the segment implementation \\cite{Werner2006}. We use the latter algorithm since we consider only density-density interactions. \nThe values at the end-points $\\tau=0$ and $\\tau=\\frac{\\beta\\hbar}{2}$ reduce to standard density-density correlation functions, which in the case of Eq.~(\\ref{fermion_otoc}) are measured separately. \n\nWe perform the analytic continuation to the real-frequency axis using the Maximum Entropy method \\cite{Jarrell1996,Lewin_maxent} with a bosonic kernel. This yields the imaginary part of the retarded correlator \n$-\\frac{1}{\\pi}\\text{Im}\\, C^R_{(AA^\\dagger)^2}(\\omega)=\\mathscr A_{(AA^\\dagger)^2}(\\omega)$.  \nWhen $\\hat B=\\hat A^\\dagger=\\hat A$ (which is the case for $\\hat A=\\hat n_\\sigma, \\hat n$), we have $C_{(AA)^2}^R(\\omega)^\\ast=C_{(AA)^2}^R(-\\omega)$. At half filling, the particle-hole symmetry furthermore ensures $C_{c_\\sigma^\\dagger c_\\sigma}^R(\\omega)^\\ast=C_{c_\\sigma^\\dagger c_\\sigma}^R(-\\omega)$.\nThus, in all the cases considered in this study, the retarded OTOC has the symmetry property $C_{(AA^\\dagger)^2}^R(\\omega)^\\ast=C_{(AA^\\dagger)^2}^R(-\\omega)$. Using this, as well as the out-of-time-order fluctuation-dissipation\ntheorem discussed in the previous section, we can obtain the real-time OTOC\n$\\langle \\hat A(t)\\hat A^\\dagger(0), \\hat A(t)\\hat A^\\dagger(0)\\rangle$ from the following inverse Fourier transformation, \n\\begin{align}{\\rm Re}\\langle \\hat A(t)\\hat A^\\dagger(0), \\hat A(t)\\hat A^\\dagger(0)\\rangle\n&=\n{\\rm Re}\n\\int_0^\\infty d\\omega\\, e^{-i\\omega t}\\coth\\left(\\frac{\\beta\\hbar\\omega}{4}\\right)\n\\left(-\\frac{1}{\\pi}{\\rm Im}\\, C_{(AA^\\dagger)^2}^R(\\omega)\\right),\n\\\\\n{\\rm Im}\\langle \\hat A(t)\\hat A^\\dagger(0), \\hat A(t)\\hat A^\\dagger(0)\\rangle\n&=\n{\\rm Im}\\int_0^{\\infty} d\\omega\\, e^{-i\\omega t} \\left(-\\frac{1}{\\pi}{\\rm Im}\\, C_{(AA^\\dagger)^2}^R(\\omega)\\right),\n\\end{align}\nfor $\\hat A=c_\\sigma^\\dagger, \\hat n_\\sigma$, and $\\hat n$.\n\n\\begin{figure}[b]\n\\begin{center}\n\\includegraphics[angle=-90, width=0.48\\columnwidth]{compare_u0_draft.eps}\\hfill\n\\includegraphics[angle=-90, width=0.48\\columnwidth]{compare_u0_longtime_modulus.eps}\n\\caption{\nLeft panel: Comparison between the exact solution and the result obtained by analytical continuation\nfor the real and imaginary parts of the OTOC $\\langle c_\\sigma^\\dagger(t)c_\\sigma(0), c_\\sigma^\\dagger(t)c_\\sigma(0)\\rangle$ \nin the noninteracting case ($U=0, \\beta\\hbar=50$) of the single-orbital Hubbard model.\nRight panel: Comparison of the long-time behavior for the modulus $|\\langle c_\\sigma^\\dagger(t)c_\\sigma(0), c_\\sigma^\\dagger(t)c_\\sigma(0)\\rangle|$. \nThe exact result shows a power-law decay ($\\sim 1/t^3$). \n}\n\\label{fig:test_u0}\n\\end{center}\n\\end{figure}\n\nAs a check of our procedure, we first compare the results for the noninteracting model,\nfor which an exact solution of the OTOC $\\langle c_\\sigma^\\dagger(t)c_\\sigma(0), c_\\sigma^\\dagger(t)c_\\sigma(0)\\rangle$ \nis available \\cite{TsujiWernerUeda2017}.\nIt is a nontrivial task to measure the noninteracting two-particle Green's function in CT-HYB, \nand to analytically continue \n$C_{(c^\\dagger_\\sigma c_\\sigma)^2}^M(\\tau)$ \nby the Maximum Entropy method.  \nThe results for the inverse temperature $\\beta\\hbar=50$ are plotted in Fig.~\\ref{fig:test_u0}. \nOne can see that the OTOC oscillates for about one cycle, and quickly decays to zero.\nThe analytically continued data show a good agreement with the exact solution. \nIn particular, for short times (up to about two inverse hoppings) the dynamics is accurately reproduced, while deviations appear at longer times. \nIt is known that for the noninteracting fermion system the OTOC decays as a power law at long time ($\\sim 1/t^3$) \\cite{TsujiWernerUeda2017}.\nThe analytical continuation method cannot reproduce the details of the oscillations at intermediate or long times, but it roughly captures the long-time decay, which is controlled by low-frequency spectral features. \nUsually, analytic continuation is unreliable for high frequency components, because high-frequency information is suppressed \nby the kernel $K(\\tau,\\omega)$ in the transformation from the spectral function $A_O(\\omega)$ to the (bosonic) Matsubara correlation function $O(\\tau)$ [$O(\\tau)=\\int_{-\\infty}^{\\infty} d\\omega\\, K(\\omega,\\tau) A_O(\\omega)$ with $K(\\tau,\\omega)=e^{-\\tau\\omega}/(1-e^{-\\beta\\omega})$]. On the other hand, the low frequency components can be rather accurately determined. As a result, in Fig. 4 we more or less recover the low-frequency features of the time-dependent correlation function, while the details of rapid oscillations are not captured. The accurate result at short times can be understood from the fact that this region is close to the imaginary time axis in the complex time plane, where the original QMC data are available.\nAt higher temperatures, the Maximum Entropy method becomes less reliable, so that the resulting OTOCs are expected to be less accurate.  \n\n\n\\subsection{Single-orbital Hubbard model}\n\\label{sec:hubbard}\n\nIn this section, we calculate the OTOCs (\\ref{fermion_otoc})-(\\ref{otoc_nn}) for the interacting single-orbital Hubbard model with the Hamiltonian \n\\begin{align}\nH\n&=\n-v\\sum_{\\langle i,j\\rangle\\sigma} (c^\\dagger_{i\\sigma}c_{j\\sigma}+ \\text{h.c.})\n-\\mu\\sum_i \\hat n_i+U\\sum_i \\hat n_{i\\uparrow}\\hat n_{i\\downarrow}, \n\\end{align}\nwhere $v$ is the hopping amplitude, $\\langle i,j\\rangle$ represents the nearest-neighbor site pairs, \n$\\mu$ is the chemical potential, and $U$ is the on-site interaction.\nIn this section, we focus on the half-filled case (i.e., $\\mu=U/2$). The DMFT solution for the simplified self-consistency (\\ref{semicirc}) is the exact solution for an infinitely connected Bethe lattice \\cite{GeorgesKotliarKrauthRozenberg1996}.\nLet us briefly recall the paramagnetic phase diagram for the single-orbital Hubbard model on this lattice \\cite{Bluemer_thesis}. At half-filling, there is a metal-insulator crossover at high temperature and a first-order Mott transition below a temperature corresponding to $\\beta\\hbar\\approx 17$ with a coexistence region between $U_{c1}$ and $U_{c2}$. The finite-temperature critical endpoint is at $U\\approx 4.7$, while the zero-temperature Mott transition occurs at $U=U_{c2}\\approx 5.6$. \nAt low temperature, the self-energy shows the Fermi liquid behavior [${\\rm Im}\\,\\Sigma(\\omega)\\sim \\omega^2$] \nin the weakly correlated metallic phase, \nwhile it shows an insulating behavior [${\\rm Im}\\,\\Sigma(\\omega)\\sim \\delta(\\omega)$] in the Mott phase\n\\cite{GeorgesKotliarKrauthRozenberg1996}. \nIn the correlated metallic phase, as the temperature is increased, deviations from the Fermi liquid behavior become apparent, \nbut an ${\\rm Im}\\,\\Sigma(\\omega)\\sim \\sqrt{\\omega}$ scaling as in the SYK model is not observed. \n\nIn the following, we will start the DMFT iterations from the noninteracting solution, which means that the finite-temperature Mott transition occurs at $U_{c2}$.\nFrom now on, we set $\\hbar=1$. \n\n\\begin{figure}[t]\n\\begin{center}\n\\includegraphics[angle=-90, width=0.49\\columnwidth]{plot_otoc_b50_Re.eps}\n\\includegraphics[angle=-90, width=0.49\\columnwidth]{plot_otoc_b50_Im.eps}\n\\includegraphics[angle=-90, width=0.49\\columnwidth]{plot_coth_b50_appendix.eps}\n\\includegraphics[angle=-90, width=0.49\\columnwidth]{plot_otoc_b50_modulus.eps}\n\\caption{\nTop panels: Real and imaginary parts of the OTOC $\\langle c^\\dagger_\\sigma(t)c_\\sigma(0), c^\\dagger_\\sigma(t)c_\\sigma(0)\\rangle$ for the single-orbital Hubbard model with $\\beta=50$ at half filling. The thin lines correspond to metallic solutions, while the thick line ($U=6$) corresponds to a Mott insulating solution. \nBottom left panel: Spectral function $\\mathscr A_{(c_\\sigma^\\dagger c_\\sigma)^2}(\\omega)=-\\frac{1}{\\pi}{\\rm Im}\\,C_{(c^\\dagger_\\sigma c_\\sigma)^2}^R(\\omega)$ multiplied by $\\coth\\big(\\frac{\\beta\\omega}{4}\\big)$. \nBottom right panel: Modulus of the OTOC $|\\langle c^\\dagger_\\sigma(t)c_\\sigma(0), c^\\dagger_\\sigma(t)c_\\sigma(0)\\rangle|$ (normalized at $t=0$) on a log-log scale. The dashed line is the result for $U=0$.\n}\n\\label{fig:results_fermion}\n\\end{center}\n\\end{figure}\n\n\n\nIn Fig.~\\ref{fig:results_fermion}, we show the results of $\\langle c^\\dagger_\\sigma(t)c_\\sigma(0), c^\\dagger_\\sigma(t)c_\\sigma(0)\\rangle$ for the interacting single-orbital Hubbard model at $\\beta=50$. \nThe top panels present the real and imaginary parts of the OTOC. One can see that the oscillations become more pronounced as one increases the interaction. In the bottom left panel of Fig.~\\ref{fig:results_fermion},\nwe show the corresponding spectral function $\\mathscr A_{(c_\\sigma^\\dagger c_\\sigma)^2}(\\omega)=-\\frac{1}{\\pi}{\\rm Im}\\,C_{(c^\\dagger_\\sigma c_\\sigma)^2}^R(\\omega)$ multiplied by $\\coth\\big(\\frac{\\beta\\omega}{4}\\big)$.\nThe Mott transition, which occurs between $U=5$ and $U=6$ at $\\beta=50$, manifests itself by the opening of a gap in the spectral function $\\mathscr A_{(c_\\sigma^\\dagger c_\\sigma)^2(\\omega)}$. This translates into more weakly damped oscillations in the real-time evolution in the Mott phase.  \nThe bottom right panel of Fig.~\\ref{fig:results_fermion} plots the modulus of the OTOC (normalized at time $t=0$, i.e., $|\\langle c^\\dagger_\\sigma(t)c_\\sigma(0), c^\\dagger_\\sigma(t)c_\\sigma(0)\\rangle|/|\\langle c^\\dagger_\\sigma(0)c_\\sigma(0), c^\\dagger_\\sigma(0)c_\\sigma(0)\\rangle|$) on a log-log scale. \nDue to the oscillations and the limited accuracy of the analytic continuation procedure, it is hard to clearly identify the nature of the long-time decay of the OTOC,\nbut the results indicate that the OTOC decays much faster (possibly exponentially) in the Mott phase than in the metallic phase. This is consistent with the results for the spectral functions of the OTOC. The comparison to the noninteracting case (dashed curve in the bottom right panel of Fig.~\\ref{fig:results_fermion}) suggests that the correlated metallic phase has a similar (possibly power-law) decay behavior of the OTOC as in the noninteracting case.\n\n\n\\begin{figure}[t]\n\\begin{center}\n\\includegraphics[angle=-90, width=0.49\\columnwidth]{plot_otoc_b50_nn_Re.eps}\n\\includegraphics[angle=-90, width=0.49\\columnwidth]{plot_otoc_b50_nn_Im.eps}\n\\includegraphics[angle=-90, width=0.49\\columnwidth]{plot_coth_b50_nn_appendix.eps}\n\\includegraphics[angle=-90, width=0.49\\columnwidth]{plot_otoc_b50_nn_modulus.eps}\n\\caption{\nTop panels: Real and imaginary parts of the OTOC $\\langle \\hat n(t)\\hat n(0), \\hat n(t)\\hat n(0)\\rangle - 1$ for the single-orbital Hubbard model with $\\beta=50$ at half filling. The thin lines correspond to metallic solutions, \nwhile the thick line ($U=6$) corresponds to a Mott insulating solution. \nBottom left panel: Spectral function $\\mathscr A_{(nn)^2}(\\omega)=-\\frac{1}{\\pi}{\\rm Im}\\,C_{(n n)^2}^R(\\omega)$ multiplied by $\\coth\\big(\\frac{\\beta\\omega}{4}\\big)$.\nBottom right panel: Modulus of the OTOC $|\\langle \\hat n(t)\\hat n(0), \\hat n(t)\\hat n(0)\\rangle - 1|$ (normalized at $t=0$) on a log-log scale. \n}\n\\label{fig:results_nn}\n\\end{center}\n\\end{figure}\n\n\nIn Fig.~\\ref{fig:results_nn}, we plot the OTOC $\\langle \\hat n(t)\\hat n(0), \\hat n(t)\\hat n(0)\\rangle$\nfor the single-orbital Hubbard model in a similar manner as in Fig.~\\ref{fig:results_fermion}.\nSince at half filling this correlation function approaches $\\langle \\hat n\\rangle^4=1$ at long times\nand at sufficiently low temperature,\nwe perform the analytical continuation procedure for $\\langle \\hat n(t)\\hat n(0), \\hat n(t)\\hat n(0)\\rangle-1$. \nThe top panels of Fig.~\\ref{fig:results_nn} show the real and imaginary parts of this shifted OTOC. \nContrary to the case of $\\langle c_\\sigma^\\dagger(t)c_\\sigma(0), c_\\sigma^\\dagger(t)c_\\sigma(0)\\rangle$, \nthe amplitude of the OTOC $\\langle \\hat n(t)\\hat n(0), \\hat n(t)\\hat n(0)\\rangle-1$ is suppressed as one increases the interaction, which reflects the reduced charge fluctuations in the Mott insulator.\nThe bottom left panel in Fig.~\\ref{fig:results_nn} shows the analytically continued spectral function $\\mathscr A_{(nn)^2}(\\omega)=-\\frac{1}{\\pi}{\\rm Im}\\, C_{(nn)^2}^R(\\omega)$ multiplied by $\\coth\\big(\\frac{\\beta\\omega}{4}\\big)$. \nAgain we notice the opening of a gap in the Mott phase, and a corresponding shift of the spectral weight to higher energies. This results in more rapid oscillations of the OTOC in the Mott state.\nThe bottom right panel in Fig.~\\ref{fig:results_nn} plots the modulus of the OTOC $|\\langle \\hat n(t)\\hat n(0), \\hat n(t)\\hat n(0)\\rangle-1|$ (normalized at $t=0$) on a log-log scale.\nThe long-time behavior of the OTOC is consistent with a power-law decay in the metallic phase, and an exponential decay in the Mott insulating phase.\n\n\n\\begin{figure}[t]\n\\begin{center}\n\\includegraphics[angle=-90, width=0.49\\columnwidth]{plot_otoc_nsns_b50_draft_Re.eps}\n\\includegraphics[angle=-90, width=0.49\\columnwidth]{plot_otoc_nsns_b50_draft_Im.eps}\n\\includegraphics[angle=-90, width=0.49\\columnwidth]{plot_coth_b50_nsns_inset.eps}\n\\includegraphics[angle=-90, width=0.49\\columnwidth]{plot_otoc_nsns_b50_draft_modulus_zoom.eps}\n\\caption{\nTop panels: Real and imaginary parts of the OTOC $\\langle \\hat n_\\sigma(t)\\hat n_\\sigma(0), \\hat n_\\sigma(t)\\hat n_\\sigma(0)\\rangle  - \\langle \\hat n_\\sigma\\rangle^4$ for the single-orbital Hubbard model with $\\beta=50$ at half filling. All the colored lines correspond to metallic solutions. \nBottom left panel: Spectral function $\\mathscr A_{(n_\\sigma n_\\sigma)^2}(\\omega)=-\\frac{1}{\\pi}{\\rm Im}\\,C_{(n_\\sigma n_\\sigma)^2}^R(\\omega)$ multiplied by $\\coth\\big(\\frac{\\beta\\omega}{4}\\big)$. In the inset, we plot the imaginary-time four-point function $\\langle \\hat n_\\sigma(\\tau)\\hat n_\\sigma(0), \\hat n_\\sigma(\\tau)\\hat n_\\sigma(0)\\rangle-\\langle \\hat n_\\sigma\\rangle^4$, with the black line corresponding to the Mott insulating solution at $U=6$.  \nBottom right panel: Modulus of the OTOC $|\\langle \\hat n_\\sigma(t)\\hat n_\\sigma(0), \\hat n_\\sigma(t)\\hat n_\\sigma(0)\\rangle  - \\langle \\hat n_\\sigma\\rangle^4|$ (normalized at $t=0$) on a log-log scale.\n}\n\\label{fig:results_nsns}\n\\end{center}\n\\end{figure}\n\nFigure~\\ref{fig:results_nsns} \nshows the results of the OTOC $\\langle \\hat n_\\sigma(t)\\hat n_\\sigma(0), \\hat n_\\sigma(t)\\hat n_\\sigma(0)\\rangle$\nfor the single-orbital Hubbard model. At half filling, this correlation function approaches $\\langle \\hat n_\\sigma\\rangle^4=\\frac{1}{16}$ in the long-time limit and at sufficiently low temperature, \nso that we perform the analytical continuation procedure for $\\langle \\hat n_\\sigma(t)\\hat n_\\sigma(0), \\hat n_\\sigma(t)\\hat n_\\sigma(0)\\rangle-\\langle \\hat n_\\sigma\\rangle^4$. \nThe top panels of Fig.~\\ref{fig:results_nsns} show the real and imaginary parts of \nthis shifted OTOC. \nHere, we only present results for the metallic phase, because resolving the very sharp low-energy feature of the OTOC spectrum in the half-filled Mott state is challenging. One can see that coherent oscillations are not observed for this type of OTOC, and that the incoherent part becomes dominant compared to the other two OTOCs shown in Figs.~\\ref{fig:results_fermion} and \\ref{fig:results_nn}. The amplitude of the OTOC $\\langle \\hat n_\\sigma(t)\\hat n_\\sigma(0), \\hat n_\\sigma(t)\\hat n_\\sigma(0)\\rangle-\\langle \\hat n_\\sigma\\rangle^4$ is enhanced as $U$ increases. As we argue below, this is related to the enhancement of spin fluctuations \nand local moment formation close to the Mott transition.\n\nThe bottom left panel in Fig.~\\ref{fig:results_nsns} plots the analytically continued spectral function $\\mathscr A_{(n_\\sigma n_\\sigma)^2}(\\omega)=-\\frac{1}{\\pi}{\\rm Im}\\, C_{(n_\\sigma n_\\sigma)^2}^R(\\omega)$ multiplied by $\\coth\\big(\\frac{\\beta\\omega}{4}\\big)$. \nWe can see that the spectral weight is concentrated in the low-energy region as we approach the Mott transition point.\nThe inset in the bottom left panel of Fig.~\\ref{fig:results_nsns} shows the imaginary-time four-point function $\\langle \\hat n_\\sigma(\\tau)\\hat n_\\sigma(0), \\hat n_\\sigma(\\tau)\\hat n_\\sigma(0)\\rangle-\\langle \\hat n_\\sigma\\rangle^4$, with the black line corresponding to a Mott insulating solution ($U=6$).  \nFor $U\\gtrsim 4$, the imaginary-time four-point function does not decay to zero but remains relatively large near $\\tau=\\frac{\\beta}{4}$. \nThis behavior is reminiscent of the spin-freezing physics seen in the correlated Hund metal phase of multi-orbital Hubbard models, where the dynamical spin correlation function $\\langle \\hat S_z(\\tau)\\hat S_z(0)\\rangle$ is trapped at a finite value at long $\\tau$ \\cite{Werner2008}. \nIn fact, the OTOC $\\langle \\hat n_\\sigma(\\tau)\\hat n_\\sigma(0), \\hat n_\\sigma(\\tau)\\hat n_\\sigma(0)\\rangle$\nmeasures spin correlations (in addition to charge correlations) through $\\hat n_\\sigma=\\frac{1}{2}\\hat n+\\sigma\\hat S_z$.\nPhysically, the trapping of $\\langle \\hat S_z(\\tau)\\hat S_z(0)\\rangle$ signals the formation of frozen local magnetic moments.\nIn the metal-insulator crossover region, the scattering induced by these magnetic moments results in incoherent metal states. Intuitively, one might expect fast scrambling in the corresponding regions of the phase diagram. \nAlthough there is some resemblance with the spin freezing crossover,\nthe self-energy does not exhibit a square-root frequency dependence in the single-orbital case, and a spin-freezing crossover in the sense of Ref.~\\cite{Werner2008} does not exist.  \n\nThe bottom right panel of Fig.~\\ref{fig:results_nsns} plots the modulus of the OTOC $|\\langle \\hat n_\\sigma(t)\\hat n_\\sigma(0), \\hat n_\\sigma(t)\\hat n_\\sigma(0)\\rangle-\\langle \\hat n_\\sigma\\rangle^4|$ (normalized at $t=0$) on a log-log scale. \nThe dynamics of $\\langle \\hat n_\\sigma(t)\\hat n_\\sigma(0), \\hat n_\\sigma(t)\\hat n_\\sigma(0)\\rangle$ is quite distinct from that of $\\langle \\hat n(t)\\hat n(0), \\hat n(t)\\hat n(0)\\rangle$. As the Mott transition is approached, we observe a slow-down in the decay of the modulus, \nwhich indicates the presence of slowly fluctuating local moments. \nThe long-time behavior may be consistent with a power-law, although slow oscillations make it difficult to determine the long-time asymptotic form from the numerics. \n\n\n\\begin{figure}[t]\n\\begin{center}\n\\includegraphics[angle=-90, width=0.49\\columnwidth]{plot_fitnsns_b2.eps}\n\\includegraphics[angle=-90, width=0.49\\columnwidth]{plot_coeffc_draft.eps}\n\\caption{\nLeft panel: \nThe imaginary-time four-point function $\\langle \\hat n_\\sigma(\\tau)\\hat n_\\sigma(0), \\hat n_\\sigma(\\tau)\\hat n_\\sigma(0)\\rangle$, the squared density-density correlation function $\\langle \\hat n_\\sigma(\\tau)\\hat n_\\sigma(0)\\rangle^2$,\nand their difference for the single-orbital Hubbard model with $U=5.5$ and $\\beta=50$.\nThe dashed curve shows a fit of the difference with the function $f_\\text{fit}(\\tau)=a+b\\tau+c\\tau^2$ in the interval $\\tau/\\beta \\in [0: 0.2]$.\nRight panel: Fitting coefficient $c$ as a function of $U$ for indicated values of $\\beta$.\n\n}\n\\label{fig:coeffs}\n\\end{center}\n\\end{figure}\n\nFor large enough $\\beta$ and small enough $\\tau$, the OTOC (\\ref{otoc_nsns}) factorizes into $\\langle \\hat n_\\sigma(\\tau)\\hat n_\\sigma(0), \\hat n_\\sigma(\\tau)\\hat n_\\sigma(0)\\rangle \\approx \\langle \\hat n_\\sigma(\\tau)\\hat n_\\sigma(0)\\rangle^2$. \nOn the real-time axis, this corresponds to the decoupling $\\langle \\hat n_\\sigma(t)\\hat n_\\sigma(0), \\hat n_\\sigma(t)\\hat n_\\sigma(0)\\rangle \\approx \\langle \\hat n_\\sigma(t)\\hat n_\\sigma(0)\\rangle^2$, that is, \nthe OTOC is reduced to a product of ordinary density-density correlation functions.\nIn order to see whether nontrivial correlations are captured beyond the decoupled form by the OTOC function, \nwe consider the difference $\\langle \\hat n_\\sigma(\\tau)\\hat n_\\sigma(0), \\hat n_\\sigma(\\tau)\\hat n_\\sigma(0)\\rangle-\\langle \\hat n_\\sigma(\\tau)\\hat n_\\sigma(0)\\rangle^2$ and fit the short-time behavior of this function to $f_\\text{fit}(\\tau)=a+b\\tau+c\\tau^2$, as illustrated in the left panel of Fig.~\\ref{fig:coeffs}. The coefficients $b$ and $c$ serve as  indicators of the nontrivial correlations that cannot be attributed to the decoupled form of the OTOC.\nThe coefficient $c$ is plotted as a function of $U$ and for different $\\beta$ in the right hand panel. The coefficient $b$ (not shown) exhibits a similar trend. We notice that the OTOC picks up nontrival correlations near the metal-insulator transition, and in particular at \nintermediate\ntemperatures, while the Mott phase shows no such correlations. The crossover region  \nassociated with the emergence of local moments \ncorresponds, roughly, to the interaction range where the nontrivial correlations start to become significant. \n\n\n\\subsection{Two-orbital Hubbard model}\n\\label{sec:two-orbital hubbard}\n\nIn this section, we investigate OTOCs in the two-orbital Hubbard model with Hund coupling $J>0$ \nand density-density interactions. (Three-orbital results are presented in Appendix~\\ref{sec:3orbital}.) \nThe Hamiltonian is given by\n\\begin{align}\nH\n&=\n-v\\sum_{\\langle i,j\\rangle\\alpha\\sigma} (c^\\dagger_{i\\alpha\\sigma}c_{j\\alpha\\sigma}+ \\text{h.c.})\n-\\mu\\sum_{i\\alpha\\sigma} \\hat n_{i\\alpha\\sigma}\n+U\\sum_{i\\alpha} \\hat n_{i\\alpha\\uparrow}\\hat n_{i\\alpha\\downarrow}\n+(U-2J)\\sum_{i\\sigma} \\hat n_{i1\\sigma}\\hat n_{i2\\bar\\sigma}\n\\notag\n\\\\\n&\\quad\n+(U-3J)\\sum_{i\\sigma} \\hat n_{i1\\sigma}\\hat n_{i2\\sigma},\n\\end{align}\n\nwith $U$ being the intra-orbital interaction $U$, $U-2J$ the inter-orbital antiparallel-spin interaction, and $U-3J$ the inter-orbital parallel-spin interaction. This is the simplest model that shows a crossover from a spin-frozen metal to a Fermi-liquid metal as one dopes the half-filled Mott insulator, with the self-energy scaling as $\\text{Im}\\Sigma(\\omega_n)\\sim\\sqrt{\\omega_n}$ over a significant energy range in the crossover regime \\cite{Hafermann2012}.\nThe sketch of the phase diagram is shown in Fig.~\\ref{fig:illustration}. \nAs we have seen in the previous section, the spin-related OTOC of the type (\\ref{otoc_nsns}) is relevant for the analysis of  the spin-freezing crossover, so that we concentrate on this OTOC here.\nIn the following calculations, we choose $U=8$, $J=U/4$, and compute $\\langle \\hat n_\\sigma(t)\\hat n_\\sigma(0), \\hat n_\\sigma(t)\\hat n_\\sigma(0)\\rangle - \\langle n_\\sigma \\rangle^4$ \nas a function of filling at $\\beta=50$ (dashed lines in Fig.~\\ref{fig:illustration}). \n\n\n\\begin{figure}[t]\n\\begin{center}\n\\includegraphics[angle=-90, width=0.49\\columnwidth]{plot_qp_draft.eps}\\hfill\n\\includegraphics[angle=-90, width=0.49\\columnwidth]{plot_fit_2orbital.eps}\n\\caption{\nLeft panel: Imaginary part of the self-energy as a function of Matsubara frequency for the two-orbital Hubbard model with $U=8$, $J=U/4$, and $\\beta=50$ on a log-log scale, with the dashed line indicating $\\sqrt{\\omega_n}$ behavior. \nRight panel: The coefficient $c$ extracted from\nfitting $\\langle \\hat n_\\sigma(\\tau)\\hat n_\\sigma(0), \\hat n_\\sigma(\\tau)\\hat n_\\sigma(0)\\rangle-\\langle \\hat n_\\sigma(\\tau)\\hat n_\\sigma(0)\\rangle^2$ with a function $f_\\text{fit}(\\tau)=a+b\\tau+c\\tau^2$\nin the interval $\\tau/\\beta\\in [0: 0.05]$ for the two-orbital Hubbard model. \nThe spin-freezing crossover occurs roughly at the filling $n_\\sigma\\approx 0.28$ (blue curve in the left panel and blue arrow in the right panel). \n}\n\\label{fig:self}\n\\end{center}\n\\end{figure}\n\n\nThe left panel of Fig.~\\ref{fig:self} plots the imaginary part of the self-energy as a function of Matsubara frequency in order to identify the filling corresponding to the spin-freezing crossover \\cite{Werner2008}. An approximate square-root scaling is observed over a wide range of frequencies near $n_\\sigma\\approx 0.28$ (blue line). In the right panel of Fig.~\\ref{fig:self}, we present the coefficient $c$ obtained from a similar analysis as presented in Fig.~\\ref{fig:coeffs}, but with a fitting range $\\tau/\\beta\\in [0 : 0.05]$. Again, we find that the OTOC $\\langle \\hat n_\\sigma(t)\\hat n_\\sigma(0), \\hat n_\\sigma(t)\\hat n_\\sigma(0)\\rangle - \\langle \\hat n_\\sigma\\rangle^4$ detects nontrivial correlations in the spin-freezing crossover (blue arrow) and spin-frozen metal regime, but not in the Fermi liquid or the Mott insulating phase.\n\n\n\\begin{figure}[t]\n\\begin{center}\n\\includegraphics[angle=-90, width=0.49\\columnwidth]{plot_otoc_2orbital_Re.eps}\n\\includegraphics[angle=-90, width=0.486\\columnwidth]{plot_otoc_2orbital_Im.eps}\n\\includegraphics[angle=-90, width=0.484\\columnwidth]{plot_spectra_coth_2orbital_inset.eps}\n\\includegraphics[angle=-90, width=0.486\\columnwidth]{plot_otoc_2orbital_modulus_inset.eps}\n\\caption{\nTop panels: Real and imaginary parts of the OTOC $\\langle \\hat n_\\sigma(t)\\hat n_\\sigma(0), \\hat n_\\sigma(t)\\hat n_\\sigma(0)\\rangle  - \\langle \\hat n_\\sigma\\rangle^4$ for the two-orbital Hubbard model with $U=8$, $J=U/4$, $\\beta=50$ and indicated fillings. \nBottom left panel: Spectral function $\\mathscr A_{(n_\\sigma n_\\sigma)^2}(\\omega)=-\\frac{1}{\\pi}{\\rm Im}\\,C_{(n_\\sigma n_\\sigma)^2}^R(\\omega)$ multiplied by $\\coth\\big(\\frac{\\beta\\omega}{4}\\big)$. In the inset, we plot the imaginary-time four-point function $\\langle \\hat n_\\sigma(\\tau)\\hat n_\\sigma(0), \\hat n_\\sigma(\\tau)\\hat n_\\sigma(0)\\rangle-\\langle \\hat n_\\sigma\\rangle^4$. \nBottom right panel: Modulus of the OTOC $|\\langle \\hat n_\\sigma(t)\\hat n_\\sigma(0), \\hat n_\\sigma(t)\\hat n_\\sigma(0)\\rangle  - \\langle \\hat n_\\sigma\\rangle^4|$ (normalized at $t=0$) on a log-log scale.\nIn the inset of the bottom right panel, we illustrate the approximate data collapse obtained by considering the spin-freezing crossover regimes for different values of $J$. \nIn all the panels, the thick blue curves represent the results for the spin-freezing crossover region. \n}\n\\label{fig:2orbital}\n\\end{center}\n\\end{figure}\n\n\nThe top panels of Fig.~\\ref{fig:2orbital} show the real and imaginary parts of the OTOC\n$\\langle \\hat n_\\sigma(t)\\hat n_\\sigma(0), \\hat n_\\sigma(t)\\hat n_\\sigma(0)\\rangle-\\langle \\hat n_\\sigma\\rangle^4$\nfor the two-orbital Hubbard model. In the spin-frozen phase ($n_\\sigma \\gtrsim 0.3$), both components exhibit \nhighly incoherent oscillations, which get suppressed as the filling is reduced. As one moves across the crossover point ($n_\\sigma \\sim 0.28$),\nthe oscillations completely disappear, and the real part of the OTOC starts to overshoot to the negative side\nin the Fermi liquid regime.\nThe bottom left panel in Fig.~\\ref{fig:2orbital} presents the analytically continued spectral function $\\mathscr A_{(n_\\sigma n_\\sigma)^2}(\\omega)=-\\frac{1}{\\pi}{\\rm Im}\\,C_{(n_\\sigma n_\\sigma)^2}^R(\\omega)$ multiplied by $\\coth\\big(\\frac{\\beta\\omega}{4}\\big)$. \nIn the spin-frozen regime ($n_\\sigma \\gtrsim 0.3$) a sharp peak appears at low energy, which originates from the saturation of the imaginary-time four point function $\\langle \\hat n_\\sigma(\\tau)\\hat n_\\sigma(0), \\hat n_\\sigma(\\tau)\\hat n_\\sigma(0)\\rangle$ at large $\\tau$ (see the inset of the bottom left panel of Fig.~\\ref{fig:2orbital}). This low-energy peak represents the slow dynamics of the frozen spins, and translates into a slow long-time decay of $|\\langle \\hat n_\\sigma(t)\\hat n_\\sigma(0), \\hat n_\\sigma(t)\\hat n_\\sigma(0)\\rangle-\\langle \\hat n_\\sigma\\rangle^4|$,\nas shown in the bottom right panel of Fig.~\\ref{fig:2orbital}\non a log-log scale. Again, it is difficult to clearly resolve the long-time behavior of the OTOC due to the limited\naccuracy of the analytic continuation, but in the spin-freezing crossover regime (thick blue curves)\nthe decay of the OTOC is consistent with a power law $\\sim 1/t^{1.5}$ at least for $2\\lesssim t \\lesssim 5$. \nThe shorter-time behavior ($0.5\\lesssim t \\lesssim 2$) is instead well fitted by an exponential decay $\\sim e^{-\\alpha t}$ with a decay constant $\\alpha=0.78$. \nThis exponent exhibits\na strong dependence on the Hund coupling. \n\nIn fact, the OTOCs in the spin-freezing regime for $J=U/4$ and $J=U/6$ can be approximately collapsed by plotting them as a function of $tJ$ (see the inset in the bottom right panel of Fig.~\\ref{fig:2orbital}), which indicates that the Hund coupling is the parameter which controls the dynamics in this regime. \nThis is distinct from the Lyapunov behavior in which the exponential decay scales with $t/\\beta$ \\cite{Bagrets2017}.\nA possible reason is that we are still far from the large $N$ limit (where $N$ corresponds to the number of orbitals times the number of spin degrees of freedom) so that the large $N$ behavior \nsuch as the Lyapunov growth is not observed in our calculations. \nIn fact, the strong dependence of the time scale on $J$ and weak dependence on $\\beta$ is consistent\nwith the behavior of the finite $N$ SYK model in Ref.~\\cite{FuSachdev2016}.\nIn the spin-frozen regime ($n_\\sigma\\gtrsim 0.3$) the decay is much slower than near the crossover point, while in the Fermi liquid regime the decay is accelerated, approaching the free fermion behavior of $t^{-3}$\nas shown in Fig.~\\ref{analytic continuation}. \nQualitatively similar results for this OTOC are obtained \nfor the three-orbital Hubbard model (see Appendix~\\ref{sec:3orbital}).\n\nThe time dependence of the modulus of \n$\\langle \\hat n_\\sigma(\\tau)\\hat n_\\sigma(0), \\hat n_\\sigma(\\tau)\\hat n_\\sigma(0)\\rangle-\\langle \\hat n_\\sigma\\rangle^4$ \nin the spin-freezing crossover regime of multi-orbital Hubbard models (exponential decay crossing over into a power-law) bears a close resemblance to a (different) OTOC for the SYK model discussed in Ref.~\\cite{Bagrets2017}. \nAlso, a recent study of yet another type of OTOCs for the finite-$N$ SYK model found a qualitatively similar decay as observed here in the spin-freezing crossover regime \\cite{FuSachdev2016}. In the following section, we will make the connection to the SYK dynamics more quantitative.   \n\n\n\\subsection{Comparison to the SKY model}\n\\label{sec:SYK}\n\nIn this section, we compare the results of the multi-orbital Hubbard models to the SYK model.\nWe consider the particle-hole symmetric SYK model of complex fermions \\cite{FuSachdev2016},\nwhose Hamiltonian is given by\n\\begin{align}\nH\n&=\n\\frac{1}{(2N)^{3/2}} \\sum_{i,j,k,l=1}^N \nJ_{ij;kl} (c_i^\\dagger c_j^\\dagger c_k c_l\n+\\delta_{ik}n c_j^\\dagger c_l\n-\\delta_{il}n c_j^\\dagger c_k\n-\\delta_{jk}n c_i^\\dagger c_l\n+\\delta_{jl}n c_i^\\dagger c_k).\n\\end{align}\nThe coupling constant $J_{ij;kl}$ is a gaussian random variable, satisfying $J_{ij;kl}=-J_{ji;kl}=-J_{ij;lk}=J_{kl;ij}^\\ast$, and\n\\begin{align}\n\\overline{({\\rm Re}\\, J_{ij;kl})^2}\n&=\n\\begin{cases}\nJ_{\\rm SYK}^2/2 & (i,j)\\neq (k,l) \\\\\nJ_{\\rm SYK}^2 & (i,j)=(k,l)\n\\end{cases},\n\\\\\n\\overline{({\\rm Im}\\, J_{ij;kl})^2}\n&=\n\\begin{cases}\nJ_{\\rm SYK}^2/2 & (i,j)\\neq (k,l) \\\\\n0 & (i,j)=(k,l)\n\\end{cases},\n\\end{align}\nwhere the overline represents an average over each realization of $J_{ij;kl}$. The parameter \n$J_{\\rm SYK}$ ($J_{\\rm SYK}^{-1}$) defines the unit of energy (time) in the following calculations. \nThe system is particle-hole symmetric when\n$n=0.5$, which fixes the total number of fermions as $N^{-1}\\sum_{i=1}^N \\langle c_i^\\dagger c_i\\rangle=0.5$.\nWe take the particle-hole symmetric form of the SYK model because it has been well studied previously \\cite{FuSachdev2016} and avoids the effect of the filling drift.\nWe numerically solve the model by exact diagonalization for finite $N(\\le 12)$, and evaluate the density-density OTOC\n$\\overline{\\langle \\hat n_i(t)\\hat n_i(0), \\hat n_i(t)\\hat n_i(0)\\rangle-\\langle \\hat n_i\\rangle^4}$\nand its imaginary-time counterpart.\n\n\\begin{figure}[t]\n\\begin{center}\n\n\\includegraphics[angle=-90,width=0.49\\columnwidth]{plot_fitnsns_syk.eps}\n\n\\includegraphics[angle=-90,width=0.49\\columnwidth]{plot_coeffc_syk.eps}\n\\caption{\nLeft panel: \nThe imaginary-time four-point function $\\overline{\\langle \\hat n_i(\\tau)\\hat n_i(0), \\hat n_i(\\tau)\\hat n_i(0)\\rangle}$, the density-density correlation function $\\overline{\\langle \\hat n_i(\\tau)\\hat n_i(0)\\rangle^2}$,\nand their difference for the SYK model with $N=12$ and $\\beta J_{\\rm SYK}=10$.\nThe dashed curve shows a fit of the difference with a function $f_\\text{fit}(\\tau)=a+b\\tau+c\\tau^2$ in the interval $\\tau/\\beta \\in [0: 0.2]$. \nRight panel: Fitting coefficient $c$ as a function of $T/J_{\\rm SYK}$.\n}\n\\label{fig:SYK fit}\n\\end{center}\n\\end{figure}\n\nIn the left panel of Fig.~\\ref{fig:SYK fit}, we plot the imaginary-time four-point function \n$\\overline{\\langle \\hat n_i(\\tau)\\hat n_i(0), \\hat n_i(\\tau)\\hat n_i(0)\\rangle}$\nfor the SYK model with $N=12$ and $\\beta J_{\\rm SYK}=10$. \nAt low temperature, we expect that $\\overline{\\langle \\hat n_i(\\tau)\\hat n_i(0), \\hat n_i(\\tau)\\hat n_i(0)\\rangle}$ is decoupled into $\\overline{\\langle \\hat n_i(\\tau)\\hat n_i(0)\\rangle^2}$.\nTo see whether nontrivial correlations beyond the decoupled form are present, we fit the difference\n$\\overline{\\langle \\hat n_i(\\tau)\\hat n_i(0), \\hat n_i(\\tau)\\hat n_i(0)\\rangle}\n-\\overline{\\langle \\hat n_i(\\tau)\\hat n_i(0)\\rangle^2}$ with a function $f_{\\rm fit}(\\tau)=a+b\\tau+c\\tau^2$\nin the interval $\\tau/\\beta\\in[0:0.2]$, in the same manner as for the Hubbard models.\nThe obtained coefficient $c$ is plotted as a function of the temperature $T$ in the right panel of\nFig.~\\ref{fig:SYK fit}. One can see that nontrivial correlations exist in a wide range of temperature.\nEspecially, they are enhanced in the intermediate temperature regime. This is because in the zero-temperature limit\nthe four-point function is decoupled as \n$\\langle \\hat n_i(\\tau)\\hat n_i(0), \\hat n_i(\\tau)\\hat n_i(0)\\rangle \\approx \\langle \\hat n_i(\\tau)\\hat n_i(0)\\rangle^2$, while in the high-temperature limit ($\\beta\\to 0$) the imaginary-time dependence is washed out.\n\n\n\n\n\\begin{figure}[t]\n\\begin{center}\n\\includegraphics[angle=-90, width=0.49\\columnwidth]{plot_syk_Re.eps}\n\\includegraphics[angle=-90, width=0.49\\columnwidth]{plot_syk_Im.eps}\n\\includegraphics[angle=-90, width=0.49\\columnwidth]{plot_syk_modulus-log.eps}\n\\includegraphics[angle=-90, width=0.49\\columnwidth]{plot_syk_modulus.eps}\n\\caption{\nThe density-density OTOC $\\overline{\\langle \\hat n_i(t)\\hat n_i(0), \\hat n_i(t)\\hat n_i(0)\\rangle-\\langle \\hat n_i\\rangle^4}$ for the particle-hole symmetric SYK model averaged over $10^2$ samples with $\\beta J_{\\rm SYK}=100$ and various $N$. \nThe top left and right panels show the real and imaginary parts of the OTOC,\nwhile the bottom left and right panels show the log and log-log plots of the modulus of the OTOC (normalized at $t=0$).\nThe dashed line shows a power-law decay $\\propto t^{-1.6}$ for comparison.}\n\\label{fig:SYK}\n\\end{center}\n\\end{figure}\n\nIn Fig.~\\ref{fig:SYK}, we show the numerical results of the density-density OTOC for $\\beta J_{\\rm SYK}=100$.\nThe real part of the OTOC rapidly drops from the initial value within the time scale of $tJ_{\\rm SYK}\\sim 2$,\nand slowly decays to zero in the long time limit.\nAs one increases $N$, the oscillations that appear at longer time tend to be suppressed.\nOne can see that the result for $N=12$ in the top panels of Fig.~\\ref{fig:SYK} closely resembles the OTOC in \nthe spin-freezing crossover regime of the two-orbital Hubbard model shown\nby the blue curve in the top panels of Fig.~\\ref{fig:2orbital},\nwhile it differs from the single-orbital result shown in the top panels of Fig.~\\ref{fig:results_nsns}.\nDue to the limitation of our calculations to small system sizes ($N\\le 12$), we do not clearly observe\nan exponential growth ($\\sim c_0-c_1 e^{\\lambda t}$)  at short times or an exponential decay ($\\sim e^{-\\alpha t}$) \nat intermediate times. \nHowever, we find that for $tJ_{\\rm SYK}\\gtrsim 2$ the OTOC decays\napproximately in a power law $\\propto t^{-\\gamma}$ with $\\gamma=1.6$ (see bottom right panel in Fig.~\\ref{fig:SYK}). \nThe time interval exhibiting the power-law decay becomes longer as we increase $N$.  \nThe temperature dependence of the OTOC for the SYK model is shown in Fig.~\\ref{fig:SYK-T}.\nThe time scale of the initial drop is more or less independent of the temperature, while \nthe power-law-like decay is only visible in the low-temperature regime.\n\n\\begin{figure}[t]\n\\begin{center}\n\\includegraphics[angle=-90, width=0.49\\columnwidth]{plot_syk_beta_re.eps}\n\\includegraphics[angle=-90, width=0.49\\columnwidth]{plot_syk_beta_im.eps}\n\\includegraphics[angle=-90, width=0.49\\columnwidth]{plot_syk_beta_modulus-log.eps}\n\\includegraphics[angle=-90, width=0.49\\columnwidth]{plot_syk_beta_modulus.eps}\n\\caption{\nTemperature dependence of the density-density OTOC\n$\\overline{\\langle \\hat n_i(t)\\hat n_i(0), \\hat n_i(t)\\hat n_i(0)\\rangle-\\langle \\hat n_i\\rangle^4}$ for the particle-hole symmetric SYK model averaged over $10^2$ samples with $N=12$.\nThe top left and right panels show the real and imaginary parts of the OTOC,\nwhile the bottom left and right panels show the log and log-log plots of the modulus of the OTOC.\nThe dashed line shows a power-law decay $\\propto t^{-1.6}$ for comparison.}\n\\label{fig:SYK-T}\n\\end{center}\n\\end{figure}\n\nIn Ref.~\\onlinecite{Werner2018}, it has been argued that the spin-freezing crossover regime of multi-orbital Hubbard models is effectively described by the SYK model in which $J_{\\rm SYK}$ is replaced by the Hund coupling $J$. The results in Fig.~\\ref{fig:2orbital} are for the two-orbital Hubbard model with $U=8$, $J/U=1/4$ and $\\beta=50$, corresponding to $\\beta J=100$, which means that it is meaningful to directly compare the blue lines in Fig.~\\ref{fig:2orbital} with the blue lines in Figs.~\\ref{fig:SYK} and \\ref{fig:SYK-T}. Not only is the qualitative agreement remarkable, even the power-law exponent measured for the two-orbital Hubbard model ($\\gamma=1.5$) is very close to the exponent extracted from the finite-$N$ SYK calculation. \nThe power-law decay in the three-orbital case is approximately $1/t^{1.75}$ (see Appendix~\\ref{sec:3orbital}), and thus also in semi-quantitative agreement with the SYK model behavior. \nThis nontrivial result provides further support for the identification of the spin-freezing crossover regime of multi-orbital Hubbard systems with an SYK strange metal. \n\nIn the large-$N$ and low-$T$ limit of the SYK model, we expect that the power-law behavior approaches $1/t^2$, since in this limit the OTOC is decoupled as $\\langle \\hat n_i(t)\\hat n_i(0), \\hat n_i(t)\\hat n_i(0)\\rangle\\approx \\langle \\hat n_i(t)\\hat n_i(0)\\rangle^2$ and the decay of $\\langle \\hat n_i(t)\\hat n_i(0)\\rangle$ is dominated by what corresponds to the slow spin relaxation $\\langle \\hat S_z(t)\\hat S_z(0)\\rangle\\sim 1/t$ in the Sachdev-Ye model.\n\n\n\n\n\n4.4 Comparison to the SKY model\n\\subsection{Comparison to the SKY model}\n\\label{sec:SYK}\n\nIn this section, we compare the results of the multi-orbital Hubbard models to the SYK model.\nWe consider the particle-hole symmetric SYK model of complex fermions \\cite{FuSachdev2016},\nwhose Hamiltonian is given by\n\\begin{align}\nH\n&=\n\\frac{1}{(2N)^{3/2}} \\sum_{i,j,k,l=1}^N \nJ_{ij;kl} (c_i^\\dagger c_j^\\dagger c_k c_l\n+\\delta_{ik}n c_j^\\dagger c_l\n-\\delta_{il}n c_j^\\dagger c_k\n-\\delta_{jk}n c_i^\\dagger c_l\n+\\delta_{jl}n c_i^\\dagger c_k).\n\\end{align}\nThe coupling constant $J_{ij;kl}$ is a gaussian random variable, satisfying $J_{ij;kl}=-J_{ji;kl}=-J_{ij;lk}=J_{kl;ij}^\\ast$, and\n\\begin{align}\n\\overline{({\\rm Re}\\, J_{ij;kl})^2}\n&=\n\\begin{cases}\nJ_{\\rm SYK}^2/2 & (i,j)\\neq (k,l) \\\\\nJ_{\\rm SYK}^2 & (i,j)=(k,l)\n\\end{cases},\n\\\\\n\\overline{({\\rm Im}\\, J_{ij;kl})^2}\n&=\n\\begin{cases}\nJ_{\\rm SYK}^2/2 & (i,j)\\neq (k,l) \\\\\n0 & (i,j)=(k,l)\n\\end{cases},\n\\end{align}\nwhere the overline represents an average over each realization of $J_{ij;kl}$. The parameter \n$J_{\\rm SYK}$ ($J_{\\rm SYK}^{-1}$) defines the unit of energy (time) in the following calculations. \nThe system is particle-hole symmetric when\n$n=0.5$, which fixes the total number of fermions as $N^{-1}\\sum_{i=1}^N \\langle c_i^\\dagger c_i\\rangle=0.5$.\nWe take the particle-hole symmetric form of the SYK model because it has been well studied previously \\cite{FuSachdev2016} and avoids the effect of the filling drift.\nWe numerically solve the model by exact diagonalization for finite $N(\\le 12)$, and evaluate the density-density OTOC\n$\\overline{\\langle \\hat n_i(t)\\hat n_i(0), \\hat n_i(t)\\hat n_i(0)\\rangle-\\langle \\hat n_i\\rangle^4}$\nand its imaginary-time counterpart.\n\n\\begin{figure}[t]\n\\begin{center}\n\n\\includegraphics[angle=-90,width=0.49\\columnwidth]{plot_fitnsns_syk.eps}\n\n\\includegraphics[angle=-90,width=0.49\\columnwidth]{plot_coeffc_syk.eps}\n\\caption{\nLeft panel: \nThe imaginary-time four-point function $\\overline{\\langle \\hat n_i(\\tau)\\hat n_i(0), \\hat n_i(\\tau)\\hat n_i(0)\\rangle}$, the density-density correlation function $\\overline{\\langle \\hat n_i(\\tau)\\hat n_i(0)\\rangle^2}$,\nand their difference for the SYK model with $N=12$ and $\\beta J_{\\rm SYK}=10$.\nThe dashed curve shows a fit of the difference with a function $f_\\text{fit}(\\tau)=a+b\\tau+c\\tau^2$ in the interval $\\tau/\\beta \\in [0: 0.2]$. \nRight panel: Fitting coefficient $c$ as a function of $T/J_{\\rm SYK}$.\n}\n\\label{fig:SYK fit}\n\\end{center}\n\\end{figure}\n\nIn the left panel of Fig.~\\ref{fig:SYK fit}, we plot the imaginary-time four-point function \n$\\overline{\\langle \\hat n_i(\\tau)\\hat n_i(0), \\hat n_i(\\tau)\\hat n_i(0)\\rangle}$\nfor the SYK model with $N=12$ and $\\beta J_{\\rm SYK}=10$. \nAt low temperature, we expect that $\\overline{\\langle \\hat n_i(\\tau)\\hat n_i(0), \\hat n_i(\\tau)\\hat n_i(0)\\rangle}$ is decoupled into $\\overline{\\langle \\hat n_i(\\tau)\\hat n_i(0)\\rangle^2}$.\nTo see whether nontrivial correlations beyond the decoupled form are present, we fit the difference\n$\\overline{\\langle \\hat n_i(\\tau)\\hat n_i(0), \\hat n_i(\\tau)\\hat n_i(0)\\rangle}\n-\\overline{\\langle \\hat n_i(\\tau)\\hat n_i(0)\\rangle^2}$ with a function $f_{\\rm fit}(\\tau)=a+b\\tau+c\\tau^2$\nin the interval $\\tau/\\beta\\in[0:0.2]$, in the same manner as for the Hubbard models.\nThe obtained coefficient $c$ is plotted as a function of the temperature $T$ in the right panel of\nFig.~\\ref{fig:SYK fit}. One can see that nontrivial correlations exist in a wide range of temperature.\nEspecially, they are enhanced in the intermediate temperature regime. This is because in the zero-temperature limit\nthe four-point function is decoupled as \n$\\langle \\hat n_i(\\tau)\\hat n_i(0), \\hat n_i(\\tau)\\hat n_i(0)\\rangle \\approx \\langle \\hat n_i(\\tau)\\hat n_i(0)\\rangle^2$, while in the high-temperature limit ($\\beta\\to 0$) the imaginary-time dependence is washed out.\n\n\n\n\n\\begin{figure}[t]\n\\begin{center}\n\\includegraphics[angle=-90, width=0.49\\columnwidth]{plot_syk_Re.eps}\n\\includegraphics[angle=-90, width=0.49\\columnwidth]{plot_syk_Im.eps}\n\\includegraphics[angle=-90, width=0.49\\columnwidth]{plot_syk_modulus-log.eps}\n\\includegraphics[angle=-90, width=0.49\\columnwidth]{plot_syk_modulus.eps}\n\\caption{\nThe density-density OTOC $\\overline{\\langle \\hat n_i(t)\\hat n_i(0), \\hat n_i(t)\\hat n_i(0)\\rangle-\\langle \\hat n_i\\rangle^4}$ for the particle-hole symmetric SYK model averaged over $10^2$ samples with $\\beta J_{\\rm SYK}=100$ and various $N$. \nThe top left and right panels show the real and imaginary parts of the OTOC,\nwhile the bottom left and right panels show the log and log-log plots of the modulus of the OTOC (normalized at $t=0$).\nThe dashed line shows a power-law decay $\\propto t^{-1.6}$ for comparison.}\n\\label{fig:SYK}\n\\end{center}\n\\end{figure}\n\nIn Fig.~\\ref{fig:SYK}, we show the numerical results of the density-density OTOC for $\\beta J_{\\rm SYK}=100$.\nThe real part of the OTOC rapidly drops from the initial value within the time scale of $tJ_{\\rm SYK}\\sim 2$,\nand slowly decays to zero in the long time limit.\nAs one increases $N$, the oscillations that appear at longer time tend to be suppressed.\nOne can see that the result for $N=12$ in the top panels of Fig.~\\ref{fig:SYK} closely resembles the OTOC in \nthe spin-freezing crossover regime of the two-orbital Hubbard model shown\nby the blue curve in the top panels of Fig.~\\ref{fig:2orbital},\nwhile it differs from the single-orbital result shown in the top panels of Fig.~\\ref{fig:results_nsns}.\nDue to the limitation of our calculations to small system sizes ($N\\le 12$), we do not clearly observe\nan exponential growth ($\\sim c_0-c_1 e^{\\lambda t}$)  at short times or an exponential decay ($\\sim e^{-\\alpha t}$) \nat intermediate times. \nHowever, we find that for $tJ_{\\rm SYK}\\gtrsim 2$ the OTOC decays\napproximately in a power law $\\propto t^{-\\gamma}$ with $\\gamma=1.6$ (see bottom right panel in Fig.~\\ref{fig:SYK}). \nThe time interval exhibiting the power-law decay becomes longer as we increase $N$.  \nThe temperature dependence of the OTOC for the SYK model is shown in Fig.~\\ref{fig:SYK-T}.\nThe time scale of the initial drop is more or less independent of the temperature, while \nthe power-law-like decay is only visible in the low-temperature regime.\n\n\\begin{figure}[t]\n\\begin{center}\n\\includegraphics[angle=-90, width=0.49\\columnwidth]{plot_syk_beta_re.eps}\n\\includegraphics[angle=-90, width=0.49\\columnwidth]{plot_syk_beta_im.eps}\n\\includegraphics[angle=-90, width=0.49\\columnwidth]{plot_syk_beta_modulus-log.eps}\n\\includegraphics[angle=-90, width=0.49\\columnwidth]{plot_syk_beta_modulus.eps}\n\\caption{\nTemperature dependence of the density-density OTOC\n$\\overline{\\langle \\hat n_i(t)\\hat n_i(0), \\hat n_i(t)\\hat n_i(0)\\rangle-\\langle \\hat n_i\\rangle^4}$ for the particle-hole symmetric SYK model averaged over $10^2$ samples with $N=12$.\nThe top left and right panels show the real and imaginary parts of the OTOC,\nwhile the bottom left and right panels show the log and log-log plots of the modulus of the OTOC.\nThe dashed line shows a power-law decay $\\propto t^{-1.6}$ for comparison.}\n\\label{fig:SYK-T}\n\\end{center}\n\\end{figure}\n\nIn Ref.~\\onlinecite{Werner2018}, it has been argued that the spin-freezing crossover regime of multi-orbital Hubbard models is effectively described by the SYK model in which $J_{\\rm SYK}$ is replaced by the Hund coupling $J$. The results in Fig.~\\ref{fig:2orbital} are for the two-orbital Hubbard model with $U=8$, $J/U=1/4$ and $\\beta=50$, corresponding to $\\beta J=100$, which means that it is meaningful to directly compare the blue lines in Fig.~\\ref{fig:2orbital} with the blue lines in Figs.~\\ref{fig:SYK} and \\ref{fig:SYK-T}. Not only is the qualitative agreement remarkable, even the power-law exponent measured for the two-orbital Hubbard model ($\\gamma=1.5$) is very close to the exponent extracted from the finite-$N$ SYK calculation. \nThe power-law decay in the three-orbital case is approximately $1/t^{1.75}$ (see Appendix~\\ref{sec:3orbital}), and thus also in semi-quantitative agreement with the SYK model behavior. \nThis nontrivial result provides further support for the identification of the spin-freezing crossover regime of multi-orbital Hubbard systems with an SYK strange metal. \n\nIn the large-$N$ and low-$T$ limit of the SYK model, we expect that the power-law behavior approaches $1/t^2$, since in this limit the OTOC is decoupled as $\\langle \\hat n_i(t)\\hat n_i(0), \\hat n_i(t)\\hat n_i(0)\\rangle\\approx \\langle \\hat n_i(t)\\hat n_i(0)\\rangle^2$ and the decay of $\\langle \\hat n_i(t)\\hat n_i(0)\\rangle$ is dominated by what corresponds to the slow spin relaxation $\\langle \\hat S_z(t)\\hat S_z(0)\\rangle\\sim 1/t$ in the Sachdev-Ye model.\n\n\n\n\n\n",
        "label": "fig:SYK-T",
        "Descriptive_question1": "What is the power-law decay exponent shown by the dashed line in figure_13?",
        "Descriptive_question2": "What is the value of N used in the SYK model results shown in figure_13?",
        "Reasoning_question1": "How does the temperature dependence affect the decay behavior of the OTOC modulus in figure_13, particularly in terms of the visibility of power-law decay across different temperature regimes?",
        "Reasoning_question2": "Based on the comparison in figure_13, why might the power-law decay of the OTOC become more pronounced as N increases in the SYK model?",
        "Descriptive_answer1": "1.6",
        "Descriptive_answer2": "12",
        "Reasoning_answer1": "The power-law decay of the OTOC modulus, indicated by the dashed line with an exponent of 1.6, is prominent at low temperatures (e.g., at \\(\\beta J_{\\rm SYK} = 100\\)) as shown in figure_13. As the temperature increases, the power-law decay becomes less visible or washed out because thermal fluctuations obscure the low-frequency correlations responsible for such decay. Thus, the power-law decay is mainly observed in the low-temperature regime, whereas at higher temperatures, the decay behavior deviates from the power-law profile and the OTOC modulus decays more rapidly or differently.",
        "Reasoning_answer2": "As \\(N\\) increases in the SYK model, the system approaches the large-\\(N\\) limit, where fluctuations are suppressed and the OTOC behavior becomes more universal and less affected by finite-size effects. This leads to smoother and more stable power-law decay in the OTOC modulus. In small \\(N\\) systems, finite-size oscillations and discrete energy spectrum effects can mask or disrupt the power-law decay; increasing \\(N\\) reduces these finite-size effects, making the power-law decay more pronounced and persistent over longer time intervals."
    },
    {
        "paper_id": "1812.04217.json",
        "image_id": "figure_14",
        "image_path": [
            "/home/yz979/scratch/chengye/scimolmo/test-images-tables/dataset_imagestables/1812.04217/images/plot_qp_3orbital.eps"
        ],
        "caption": "\nResults for the three-orbital Hubbard model with $U=8$, $J=U/4$, and $\\beta=50$. \nLeft panel: Imaginary part of the self-energy as a function of Matsubara frequency $\\omega_n$.\nThe dashed line corresponds to $\\sqrt{\\omega_n}$. \nThe spin-freezing crossover is located near $n_\\sigma=0.220$.  \nRight panel: The modulus of the OTOC $|\\langle \\hat n_\\sigma(t)\\hat n_\\sigma(0), \\hat n_\\sigma(t)\\hat n_\\sigma(0)\\rangle  - \\langle \\hat n_\\sigma\\rangle^4|$ (normalized at $t=0$) on a log-log scale. In the spin-freezing crossover region, there is an approximate power-law behavior $\\sim 1/t^{1.75}$ indicated by the dashed line.\n",
        "classify": "Chart",
        "section_info": "6 Results for the three-orbital Hubbard model\n\\section{Results for the three-orbital Hubbard model}\n\\label{sec:3orbital}\n\n\\begin{figure}[t]\n\\begin{center}\n\\includegraphics[angle=-90, width=0.505\\columnwidth]{plot_qp_3orbital.eps}\\hfill\n\\includegraphics[angle=-90, width=0.48\\columnwidth]{plot_otoc_nsns_b50_3orbital_modulus.eps}\n\\caption{\nResults for the three-orbital Hubbard model with $U=8$, $J=U/4$, and $\\beta=50$. \nLeft panel: Imaginary part of the self-energy as a function of Matsubara frequency $\\omega_n$.\nThe dashed line corresponds to $\\sqrt{\\omega_n}$. \nThe spin-freezing crossover is located near $n_\\sigma=0.220$.  \nRight panel: The modulus of the OTOC $|\\langle \\hat n_\\sigma(t)\\hat n_\\sigma(0), \\hat n_\\sigma(t)\\hat n_\\sigma(0)\\rangle  - \\langle \\hat n_\\sigma\\rangle^4|$ (normalized at $t=0$) on a log-log scale. In the spin-freezing crossover region, there is an approximate power-law behavior $\\sim 1/t^{1.75}$ indicated by the dashed line.\n}\n\\label{fig:3orbital}\n\\end{center}\n\\end{figure}\n\nWe show the results for the self-energy and the modulus of the OTOC $|\\langle \\hat n_\\sigma(t)\\hat n_\\sigma(0), \\hat n_\\sigma(t)\\hat n_\\sigma(0)\\rangle  - \\langle \\hat n_\\sigma\\rangle^4|$ for the three-orbital Hubbard model\nwith $U=8$, $J=U/4$, and $\\beta=50$ in Fig.~\\ref{fig:3orbital}.  \nThe spin-freezing crossover roughly occurs at the filling $n_\\sigma=0.220$ (blue lines), where we observe an approximate power-law decay $1/t^\\gamma$ of the modulus of the OTOC (\\ref{otoc_nsns}) at longer times, with $\\gamma=1.75$.\n\n\n\n\\bibliography{ref}\n\n\n\\end{document}",
        "label": "fig:3orbital",
        "Descriptive_question1": "What is the value of the spin-freezing crossover filling in figure_14?",
        "Descriptive_question2": "What is the power-law behavior indicated by the dashed line in the right panel of figure_14?",
        "Reasoning_question1": "How does the imaginary part of the self-energy in figure_14 relate to the Matsubara frequency, and what does the comparison to the square root of the frequency suggest about the system's behavior?",
        "Reasoning_question2": "What can be inferred about the dynamics of the three-orbital Hubbard model in the spin-freezing crossover region from the power-law decay observed in the OTOC modulus in figure_14?",
        "Descriptive_answer1": "0.220",
        "Descriptive_answer2": "1/t^{1.75}",
        "Reasoning_answer1": "The imaginary part of the self-energy is plotted against the Matsubara frequency, with a dashed line corresponding to the square root of the frequency (\\sqrt{\\omega_n}). Since the self-energy curve approximates the dashed \\sqrt{\\omega_n} line near the spin-freezing crossover, this suggests non-Fermi liquid behavior consistent with spin-freezing, as typical Fermi liquids would have a different frequency dependence.",
        "Reasoning_answer2": "The OTOC modulus exhibits an approximate power-law decay \\sim 1/t^{1.75} in the spin-freezing crossover region, indicating slow relaxation and persistent correlations characteristic of glassy or frozen spin dynamics. This implies that in the crossover region, the system's dynamics are neither fully ergodic nor localized, showing critical slow dynamics associated with spin freezing."
    },
    {
        "paper_id": "1511.03946.json",
        "image_id": "figure_2",
        "image_path": [
            "/home/yz979/scratch/chengye/scimolmo/test-images-tables/dataset_imagestables/1511.03946/images/figure2.eps"
        ],
        "caption": "Total photoionization cross section measured in Mb on a logarithmic scale as a function of photon energy in eV. All results display the initial ground state, statistically weighted $^2$P$^{\\rm o}$, with $J=3/2$ and $J=1/2$ odd states to all allowed final states. A 10meV gaussian convolution at FWHM is applied to compare directly with experimental resolution for all theoretical calculations. The yellow circles, green circles with error bars, and solid turquoise line are the experimental results, absolute measurements at resonance free regions and theoretical calculations respectively, performed by \\citet{2011PhRvA..84a3413C}. The dashed orange and solid purple lines represent our PBP and DARC3 calculations respectively. \\label{fig:valence}",
        "classify": "Chart",
        "section_info": "4 Results\n\\section{Results}\\label{sec:results}\nBefore embarking on the large scale DARC3 calculation we thought it prudent to investigate first the important properties and characteristics found in the photoionization cross section of Ar {\\sc ii} in its ground state. In Figure \\ref{fig:comparison}\nwe present the total photoionization cross section in Mb on a logarithmic scale as a function of photon energy in eV, from the initial ground Ar {\\sc ii} $^2$P$^{\\rm{o}}_{3/2}$ state to all allowed final states. Three calculations are presented in this figure; both the 209 level DARC1 and its contributions from the 3s$^2$3p$^4$ levels indexed as 1-5 in Table \\ref{tab:energy} and the extended 257 level DARC2 calculation. Clearly Figure \\ref{fig:comparison}\nshows the importance of including at least the first five 3s$^2$3p$^4$ levels of Ar {\\sc iii} in this photoionization calculation. The contributions from these levels dominates the total cross section up to a photon energy of approximately 50eV and all three calculations exhibit excellent agreement up to this point. It is essential, therefore, that an accurate description is achieved for the wavefunction representation of those low-lying levels. Above 50eV the additional levels associated with the more complex DARC1 and DARC2 models come into play and the cross section rises as we move to higher photon energies as more channels become accessible. Interestingly the inclusion of the additional 3s$^2$3p$^3$4d and 3s$^2$3p$^3$5s levels in the DARC2 model has little or no effect on the photoionization cross section produced by the DARC1 model up to 60eV, both datasets showing near perfect agreement. Therefore we do not retain these additional 3s$^2$3p$^3$4d and 3s$^2$3p$^3$5s configurations in our largest DARC3 calculation as can be seen from Table \\ref{tab:calculations}.\n\n\n\n\n\\begin{figure*}\n\\includegraphics[scale=0.6, angle=-90]{figure3.eps}\n\\caption{Photoionization cross section measured in Mb on a logarithmic scale as a function of the photon energy between 27.8-29.2 eV just above threshold. The solid black line is the current statistically weighted, initial ground state, DARC3 calculation against the experimental values from \\citet{2011PhRvA..84a3413C} represented by the yellow circles taken from Figure \\ref{fig:valence}. \\label{fig:valence_zoom}}\n\\end{figure*}\n\n\n\n\n\\begin{figure*}\n\\includegraphics[scale=0.58, angle=-90]{figure4.eps}\n\\caption{Total ground state photoionization cross section measured in Mb as a function of the photon energy between 0-280eV. The transition is from the initial state 3s3p$^6$ $^2$S$_{1/2}$ to all allowed final states from the DARC3 model. \\label{fig:excited}}\n\\end{figure*}\n\n\\subsection{Valence shell photoionization}\nThe only available data currently in the literature for valence shell photoionization of Ar {\\sc ii} up to photon energies of 60eV is performed by \\citet{2011PhRvA..84a3413C}. In this paper both theoretical and experimental cross sections are presented. Absolute cross sections are obtained from the merged beam technique at the Advanced Light Source (ALS) with a spectral resolution of 10meV. It was found that the primary ion beam contained a mixture of both $^2$P$_{3/2}^{\\rm o}$ and $^2$P$_{1/2}^{\\rm o}$ initial states. Hence the total cross section was presented as a statistical weighting of the odd parity $J=3/2$ ground and $J=1/2$ metastable states respectively. The accompanying theoretical cross sections presented by \\citet{2011PhRvA..84a3413C} were evaluated using the Breit-Pauli \\textbf{R}-matrix approach. A total of 48 $LSJ\\pi$ fine-structure levels were included in the wavefunction representation with configurations 3s$^2$3p$^4$, 3s3p$^5$, 3p$^6$ and 3s$^2$3p$^2$3d$^2$. Some important correlation effects are thus omitted from this model such as levels associated with the 3s$^2$3p$^3$3d configuration and those arising from the lower $n=4$ complex. \n\n\n\n\n\\begin{figure}\n\\includegraphics[scale=0.29, angle=-90]{figure7.eps}\n\\caption{The photoionization cross section is presented on a linear scale against photon energy in eV above 261.2eV. The solid black line represents our current DARC3 model convoluted at 140meV FWHM and the dashed black line is the contribution to the cross section from valence shell photoionization of the 3s and 3p. The grey circles are experimental values of \\citet{2012PhRvA..85d3408B} for the single ionization channel and pink circles represent the total contribution. Each 2p$^5$3s$^2$3p$^5$ threshold is represented by an asterisk.\\label{fig:s_and_d}}\n\\end{figure}\n\nIn order to compare with this data we present in Figure \\ref{fig:valence} the total photoionization cross section from the initial $^2$P$^{\\rm{o}}$ ground state of Ar {\\sc ii} statistically weighted to the $J=3/2$ and $J=1/2$ states. We present two of our calculations in the figure, the most sophisticated DARC3 model and, in order to perform a direct comparison with the Breit-Pauli theoretical results of \\citet{2011PhRvA..84a3413C}, the PBP 124 level model outlined in Table \\ref{tab:calculations}. To match experimental resolving power, we convolute our total results with a 10meV gaussian profile at full-width half-maximum (FWHM). In addition, to replicate the target thresholds, we have shifted our threshold values recorded in Table \\ref{tab:energy} to the experimental NIST values where possible, during the diagonalization of the Hamiltonian matrix. The remaining levels not contained in NIST are shifted by an average proportion to each corresponding angular and spin momentum state, which has little effect on the background and is meant only for consistency. This ensures that resonance features are properly positioned with respect to the observed thresholds, making a direct comparison with experiment more meaningful.\n\n\n\n\n\\begin{figure*}\n\\includegraphics[scale=0.6, angle=-90]{figure5.eps}\n\\caption{Photoionization cross section measured in Mb on a logarithmic scale as a function of the photon energy in eV between 250-270eV. The circles are the experimental results from \\citet{2012PhRvA..85d3408B} with error bars included. The solid black line represents our current, DARC3 model results for the statistically weighted initial ground state, convoluted at 140meV at FWHM. \\label{fig:bigL}}\n\\end{figure*}\n\nWe can clearly see in Figure \\ref{fig:valence} that the low energy region just above threshold is completely dominated by 3s$^2$3p$^5$ $\\rightarrow$ 3s$^2$3p$^4nl$ transitions occurring at discrete energies prior to the ejection of an electron. This densely populated region of Rydberg resonances up to approximately 30eV is followed by a steep decline in the photoionization cross section forming the expected Cooper minimum around 45-50eV. This minimum is well known to appear in the spectra of noble gases \\citep{1962PhRv..128..681C}. Above this minimum the cross section rises due to excitations from 3p $\\rightarrow$ 3d transitions, before monotonically decreasing towards zero with increasing photon energy. \n\nExcellent agreement is evident between the 124 level PBP and the 48 level Breit-Pauli calculation of \\citet{2011PhRvA..84a3413C}, for all photon energies up to 60eV. Note that the cross section in Figure \\ref{fig:valence} is plotted on a log scale. Evidently the larger basis expansion of the present PBP evaluation, which includes the 4s, 4p and 4d orbitals, has minimal effect on the resulting photoionization cross section. Both of these Breit-Pauli evaluations, however, underestimate the cross section above roughly 45eV and lie considerably lower than the experimental measurements from ALS. The larger DARC3 evaluation, incorporating 557 fine-structure levels, gives much better agreement with experiment at photon energies above the Cooper minimum. This is partly due to the more substantial calculation, and also a more accurate description of the wavefunctions included. Both techniques in fact are known to reproduce similar results as shown in a study by \\citet{2005JPhB...38.1667B}, showing that the average difference in effective collision strengths for Fe$^{14+}$ to be 6$\\%$ between all transitions considered. The additional levels included, and the Rydberg resonances converging onto their thresholds, have the effect of raising the cross section above 45eV.\n\nIn order to further emphasize the excellent agreement between the DARC3 and the experimental measurements, we zoom in on the photon energy region just above threshold, from 27.8-29.2eV, in Figure \\ref{fig:valence_zoom}. It is clearly evident that the disparities found between theory and experiment in this very narrow energy range are negligible and excellent conformity is achieved. This high level of agreement supports the accuracy of the DARC3 evaluation and we believe that these valence shell photoionization cross sections for the ground state of Ar {\\sc ii} accurately reproduce the experimental spectrum. \n\nIn Figure \\ref{fig:excited} we present the total photoionization cross section for the process defined in Equation \\ref{eq:excited}, photoionization from the lowest excited initial 3s3p$^6\\; ^2$S$_{1/2}$ bound state of Ar {\\sc ii} to all possible allowed final states of Ar {\\sc iii}. These evaluations were carried out using the DARC3 model and present for the first time, cross sections for photoionization from an excited Ar {\\sc ii} state. There are no other theoretical or experimental data with which we can compare in this figure. The cross section is presented as a function of the photon energy in eV which ranges from just above the ionization threshold to beyond the opening of the L-shell thresholds. The photoionization cross section tends towards zero with increasing energy, and it is only due to the inclusion of the additional 10 hole states in the DARC3 model do we witness contributions to the cross section at photon energies between 200-250 eV. \n\n\\subsection{L-shell photoionization}\nCalculations and experiment have been carried out at the L-shell energy region between 250-280eV by \\citet{2012PhRvA..85d3408B} at the SOLEIL facility in France as described in Section \\ref{sec:introduction}. All the results herein have been convoluted with a 140meV Gaussian profile at FWHM to match the spectral resolution of experiment. Similar to the valence shell comparisons, the initial ground state cross section is formed from a statistically weighted average of the contributions from the odd $J=3/2$ and $J=1/2$ partial waves. Due to time of flight between the ion source and interacting region, excited levels can populate the main ion beam. This leads to a possible inclusion of the initial 3s3p$^6$ $^2$S$_{1/2}$ bound state which may also contribute to the total cross section. In Figure \\ref{fig:excited} we have already shown the immediate result of the lowest excited initial bound state transitions arising from the configuration 3s3p$^6$.\n\n\n\n\n\n\\begin{figure}\n\\includegraphics[scale=0.30, angle=-90]{figure6.eps}\n\\caption{The total convoluted FWHM at 140meV photoionization cross section between 254-256eV taken from Figure \\ref{fig:bigL} highlighting the intense resonant peaks. The spectra is broken into the contributions from each dipole allowed symmetry from both initial (middle third) and metastable (bottom third) initial states according to their statistical weighting. These even $J=5/2, 3/2, 1/2$ partial symmetries are the solid turquoise, red and purple lines. The total (top third) summed contribution is presented by the solid black curve and the two dominate resonances are marked by the dashed line. \\label{fig:res_peaks}}\n\\end{figure}\n\nIn order to compare with experiment, we have presented our results against various ionization channels from \\citet{2012PhRvA..85d3408B} in Figure \\ref{fig:s_and_d}. The timescale for Auger decay is much shorter than the time of flight required by the Argon ions after interaction with a photon, and therefore, the single ionization channel from experiment depicts the characteristics of photoionizing a valence electron. We can directly compare with this process in Figure \\ref{fig:s_and_d} by omitting the contribution from the additional 10 target states annotated by asterisks. Both above and during these thresholds we expect a rise in the photoionization cross section as more channels are opened and become accessible. The total result obtained by DARC3 can be compared directly to the combination of both single and double ionization modes of experiment. We have neglected the error bars for both modes in order to visualise the results more clearly.\n\nWe now present in Figure \\ref{fig:bigL} the photoionization cross section, on a linear scale, as a function of incident photon energy in eV across the L-shell threshold range from 250-270eV. Comparisons are made between the present DARC3 cross section and the measurements performed by \\citet{2012PhRvA..85d3408B}. Clearly excellent agreement is evident between theory and experiment across the range considered, as the features and energy positions of the resonance profiles exhibit good agreement. We note that as we have employed orbitals optimized on the valence state photoionization, an energy shift of 7.5eV was required to match the experimental spectra to our current results. The theory clearly predicts this process to a high standard of accuracy and allows us to benchmark the quality of results obtained from experiment.\n\nIn an attempt to investigate the features further, we have broken down the spectrum in Figure \\ref{fig:res_peaks} from the total into each of the allowed, final, even $J$ states $J=1/2$, $J=3/2$ and $J=5/2$. Clearly visible is the intense spike at $\\approx 254.9$ eV which is dominated by transitions of the form, 2p $\\rightarrow$ nd, ns which are engulfed by the convolution. The second strong peak at $\\approx 255.65$ eV is visible mostly through the metastable initial state transition from another strong 2p $\\rightarrow$ nd, J = 3/2 resonance. In reference to Figure \\ref{fig:excited}, the cross section has already reached close towards zero in the photon energy range of interest and therefore any contribution to the total cross section from these initial excited bound states would result in a reduction to the intensity of each resonant state. \n\nThis method of deconstructing the cross section is also important to identify which initial state has been photoionized during the experiment. It is clear however that the strongest profiles are not well isolated and therefore eliminates the possibility to further conduct any analysis on the weighted contributions. We therefore retain the statistical averaging of the ground state as our best result.\n\nAll resonances in this paper were identified using the technique detailed by \\citet{1996JPhB...29.4529Q} and \\citet{1998CoPhC.114..225Q}, which involves an analytic approach complementary to the \\textbf{R}-matrix method. By exploiting multichannel quantum defect theory \\citep{1983RPPh...46..167S}, each resonant state part of a Rydberg series has constant defect, $\\mu$ for each effective $n$ quantum number defined by,\n\\begin{equation}\\label{eq:resonance}\nE_r = E_{n \\rightarrow \\infty} - \\Big[\\frac{Z-N}{n-\\mu}\\Big]^2\n\\end{equation}\nwhere $E_r$ is the resonance energy converging to the target thresholds, $E_{n \\rightarrow \\infty}$. The overlapping nature of the resonant states makes it difficult to accurately evaluate resonance widths and assign each transition taking place. It is possible to deduce that the hole resonant states arise from 2p $\\rightarrow nd, (n+1)s$ transitions for $n\\ge3$, and correspond to the strongest peaks evident in Figure \\ref{fig:res_peaks}.\n\n\n\n\n\n\n\n\n\n\n\n\n\n4.1 Valence shell photoionization\n\\subsection{Valence shell photoionization}\nThe only available data currently in the literature for valence shell photoionization of Ar {\\sc ii} up to photon energies of 60eV is performed by \\citet{2011PhRvA..84a3413C}. In this paper both theoretical and experimental cross sections are presented. Absolute cross sections are obtained from the merged beam technique at the Advanced Light Source (ALS) with a spectral resolution of 10meV. It was found that the primary ion beam contained a mixture of both $^2$P$_{3/2}^{\\rm o}$ and $^2$P$_{1/2}^{\\rm o}$ initial states. Hence the total cross section was presented as a statistical weighting of the odd parity $J=3/2$ ground and $J=1/2$ metastable states respectively. The accompanying theoretical cross sections presented by \\citet{2011PhRvA..84a3413C} were evaluated using the Breit-Pauli \\textbf{R}-matrix approach. A total of 48 $LSJ\\pi$ fine-structure levels were included in the wavefunction representation with configurations 3s$^2$3p$^4$, 3s3p$^5$, 3p$^6$ and 3s$^2$3p$^2$3d$^2$. Some important correlation effects are thus omitted from this model such as levels associated with the 3s$^2$3p$^3$3d configuration and those arising from the lower $n=4$ complex. \n\n\n\n\n\\begin{figure}\n\\includegraphics[scale=0.29, angle=-90]{figure7.eps}\n\\caption{The photoionization cross section is presented on a linear scale against photon energy in eV above 261.2eV. The solid black line represents our current DARC3 model convoluted at 140meV FWHM and the dashed black line is the contribution to the cross section from valence shell photoionization of the 3s and 3p. The grey circles are experimental values of \\citet{2012PhRvA..85d3408B} for the single ionization channel and pink circles represent the total contribution. Each 2p$^5$3s$^2$3p$^5$ threshold is represented by an asterisk.\\label{fig:s_and_d}}\n\\end{figure}\n\nIn order to compare with this data we present in Figure \\ref{fig:valence} the total photoionization cross section from the initial $^2$P$^{\\rm{o}}$ ground state of Ar {\\sc ii} statistically weighted to the $J=3/2$ and $J=1/2$ states. We present two of our calculations in the figure, the most sophisticated DARC3 model and, in order to perform a direct comparison with the Breit-Pauli theoretical results of \\citet{2011PhRvA..84a3413C}, the PBP 124 level model outlined in Table \\ref{tab:calculations}. To match experimental resolving power, we convolute our total results with a 10meV gaussian profile at full-width half-maximum (FWHM). In addition, to replicate the target thresholds, we have shifted our threshold values recorded in Table \\ref{tab:energy} to the experimental NIST values where possible, during the diagonalization of the Hamiltonian matrix. The remaining levels not contained in NIST are shifted by an average proportion to each corresponding angular and spin momentum state, which has little effect on the background and is meant only for consistency. This ensures that resonance features are properly positioned with respect to the observed thresholds, making a direct comparison with experiment more meaningful.\n\n\n\n\n\\begin{figure*}\n\\includegraphics[scale=0.6, angle=-90]{figure5.eps}\n\\caption{Photoionization cross section measured in Mb on a logarithmic scale as a function of the photon energy in eV between 250-270eV. The circles are the experimental results from \\citet{2012PhRvA..85d3408B} with error bars included. The solid black line represents our current, DARC3 model results for the statistically weighted initial ground state, convoluted at 140meV at FWHM. \\label{fig:bigL}}\n\\end{figure*}\n\nWe can clearly see in Figure \\ref{fig:valence} that the low energy region just above threshold is completely dominated by 3s$^2$3p$^5$ $\\rightarrow$ 3s$^2$3p$^4nl$ transitions occurring at discrete energies prior to the ejection of an electron. This densely populated region of Rydberg resonances up to approximately 30eV is followed by a steep decline in the photoionization cross section forming the expected Cooper minimum around 45-50eV. This minimum is well known to appear in the spectra of noble gases \\citep{1962PhRv..128..681C}. Above this minimum the cross section rises due to excitations from 3p $\\rightarrow$ 3d transitions, before monotonically decreasing towards zero with increasing photon energy. \n\nExcellent agreement is evident between the 124 level PBP and the 48 level Breit-Pauli calculation of \\citet{2011PhRvA..84a3413C}, for all photon energies up to 60eV. Note that the cross section in Figure \\ref{fig:valence} is plotted on a log scale. Evidently the larger basis expansion of the present PBP evaluation, which includes the 4s, 4p and 4d orbitals, has minimal effect on the resulting photoionization cross section. Both of these Breit-Pauli evaluations, however, underestimate the cross section above roughly 45eV and lie considerably lower than the experimental measurements from ALS. The larger DARC3 evaluation, incorporating 557 fine-structure levels, gives much better agreement with experiment at photon energies above the Cooper minimum. This is partly due to the more substantial calculation, and also a more accurate description of the wavefunctions included. Both techniques in fact are known to reproduce similar results as shown in a study by \\citet{2005JPhB...38.1667B}, showing that the average difference in effective collision strengths for Fe$^{14+}$ to be 6$\\%$ between all transitions considered. The additional levels included, and the Rydberg resonances converging onto their thresholds, have the effect of raising the cross section above 45eV.\n\nIn order to further emphasize the excellent agreement between the DARC3 and the experimental measurements, we zoom in on the photon energy region just above threshold, from 27.8-29.2eV, in Figure \\ref{fig:valence_zoom}. It is clearly evident that the disparities found between theory and experiment in this very narrow energy range are negligible and excellent conformity is achieved. This high level of agreement supports the accuracy of the DARC3 evaluation and we believe that these valence shell photoionization cross sections for the ground state of Ar {\\sc ii} accurately reproduce the experimental spectrum. \n\nIn Figure \\ref{fig:excited} we present the total photoionization cross section for the process defined in Equation \\ref{eq:excited}, photoionization from the lowest excited initial 3s3p$^6\\; ^2$S$_{1/2}$ bound state of Ar {\\sc ii} to all possible allowed final states of Ar {\\sc iii}. These evaluations were carried out using the DARC3 model and present for the first time, cross sections for photoionization from an excited Ar {\\sc ii} state. There are no other theoretical or experimental data with which we can compare in this figure. The cross section is presented as a function of the photon energy in eV which ranges from just above the ionization threshold to beyond the opening of the L-shell thresholds. The photoionization cross section tends towards zero with increasing energy, and it is only due to the inclusion of the additional 10 hole states in the DARC3 model do we witness contributions to the cross section at photon energies between 200-250 eV. \n\n",
        "label": "fig:valence",
        "Descriptive_question1": "What color represents the experimental results in figure_2?",
        "Descriptive_question2": "What type of line represents the DARC3 calculations in figure_2?",
        "Reasoning_question1": "Why does the photoionization cross section in figure_2 show a steep decline around 45-50 eV, and what is this phenomenon called?",
        "Reasoning_question2": "How does the inclusion of additional levels in the DARC3 model affect the photoionization cross section above 45 eV in figure_2 compared to the PBP and Breit-Pauli calculations?",
        "Descriptive_answer1": "yellow circles",
        "Descriptive_answer2": "solid purple line",
        "Reasoning_answer1": "The steep decline in the photoionization cross section around 45-50 eV, observed in figure_2, corresponds to the Cooper minimum. This is a known phenomenon in noble gases where the cross section decreases sharply due to a node in the radial wavefunction of the involved electronic orbital, leading to destructive interference in the photoionization amplitude, thus reducing the cross section significantly in that energy range.",
        "Reasoning_answer2": "The inclusion of additional levels in the larger DARC3 model, which comprises 557 fine-structure levels, results in a higher photoionization cross section above 45 eV compared to the PBP and Breit-Pauli calculations. This enhancement is due to the more comprehensive wavefunction description and the inclusion of more Rydberg resonances converging on target thresholds, adding contributions that raise the cross section and improve agreement with experimental measurements."
    },
    {
        "paper_id": "1511.03946.json",
        "image_id": "figure_3",
        "image_path": [
            "/home/yz979/scratch/chengye/scimolmo/test-images-tables/dataset_imagestables/1511.03946/images/figure3.eps"
        ],
        "caption": "Photoionization cross section measured in Mb on a logarithmic scale as a function of the photon energy between 27.8-29.2 eV just above threshold. The solid black line is the current statistically weighted, initial ground state, DARC3 calculation against the experimental values from \\citet{2011PhRvA..84a3413C} represented by the yellow circles taken from Figure \\ref{fig:valence}. \\label{fig:valence_zoom}",
        "classify": "Chart",
        "section_info": "4 Results\n\\section{Results}\\label{sec:results}\nBefore embarking on the large scale DARC3 calculation we thought it prudent to investigate first the important properties and characteristics found in the photoionization cross section of Ar {\\sc ii} in its ground state. In Figure \\ref{fig:comparison}\nwe present the total photoionization cross section in Mb on a logarithmic scale as a function of photon energy in eV, from the initial ground Ar {\\sc ii} $^2$P$^{\\rm{o}}_{3/2}$ state to all allowed final states. Three calculations are presented in this figure; both the 209 level DARC1 and its contributions from the 3s$^2$3p$^4$ levels indexed as 1-5 in Table \\ref{tab:energy} and the extended 257 level DARC2 calculation. Clearly Figure \\ref{fig:comparison}\nshows the importance of including at least the first five 3s$^2$3p$^4$ levels of Ar {\\sc iii} in this photoionization calculation. The contributions from these levels dominates the total cross section up to a photon energy of approximately 50eV and all three calculations exhibit excellent agreement up to this point. It is essential, therefore, that an accurate description is achieved for the wavefunction representation of those low-lying levels. Above 50eV the additional levels associated with the more complex DARC1 and DARC2 models come into play and the cross section rises as we move to higher photon energies as more channels become accessible. Interestingly the inclusion of the additional 3s$^2$3p$^3$4d and 3s$^2$3p$^3$5s levels in the DARC2 model has little or no effect on the photoionization cross section produced by the DARC1 model up to 60eV, both datasets showing near perfect agreement. Therefore we do not retain these additional 3s$^2$3p$^3$4d and 3s$^2$3p$^3$5s configurations in our largest DARC3 calculation as can be seen from Table \\ref{tab:calculations}.\n\n\n\n\n\\begin{figure*}\n\\includegraphics[scale=0.6, angle=-90]{figure3.eps}\n\\caption{Photoionization cross section measured in Mb on a logarithmic scale as a function of the photon energy between 27.8-29.2 eV just above threshold. The solid black line is the current statistically weighted, initial ground state, DARC3 calculation against the experimental values from \\citet{2011PhRvA..84a3413C} represented by the yellow circles taken from Figure \\ref{fig:valence}. \\label{fig:valence_zoom}}\n\\end{figure*}\n\n\n\n\n\\begin{figure*}\n\\includegraphics[scale=0.58, angle=-90]{figure4.eps}\n\\caption{Total ground state photoionization cross section measured in Mb as a function of the photon energy between 0-280eV. The transition is from the initial state 3s3p$^6$ $^2$S$_{1/2}$ to all allowed final states from the DARC3 model. \\label{fig:excited}}\n\\end{figure*}\n\n\\subsection{Valence shell photoionization}\nThe only available data currently in the literature for valence shell photoionization of Ar {\\sc ii} up to photon energies of 60eV is performed by \\citet{2011PhRvA..84a3413C}. In this paper both theoretical and experimental cross sections are presented. Absolute cross sections are obtained from the merged beam technique at the Advanced Light Source (ALS) with a spectral resolution of 10meV. It was found that the primary ion beam contained a mixture of both $^2$P$_{3/2}^{\\rm o}$ and $^2$P$_{1/2}^{\\rm o}$ initial states. Hence the total cross section was presented as a statistical weighting of the odd parity $J=3/2$ ground and $J=1/2$ metastable states respectively. The accompanying theoretical cross sections presented by \\citet{2011PhRvA..84a3413C} were evaluated using the Breit-Pauli \\textbf{R}-matrix approach. A total of 48 $LSJ\\pi$ fine-structure levels were included in the wavefunction representation with configurations 3s$^2$3p$^4$, 3s3p$^5$, 3p$^6$ and 3s$^2$3p$^2$3d$^2$. Some important correlation effects are thus omitted from this model such as levels associated with the 3s$^2$3p$^3$3d configuration and those arising from the lower $n=4$ complex. \n\n\n\n\n\\begin{figure}\n\\includegraphics[scale=0.29, angle=-90]{figure7.eps}\n\\caption{The photoionization cross section is presented on a linear scale against photon energy in eV above 261.2eV. The solid black line represents our current DARC3 model convoluted at 140meV FWHM and the dashed black line is the contribution to the cross section from valence shell photoionization of the 3s and 3p. The grey circles are experimental values of \\citet{2012PhRvA..85d3408B} for the single ionization channel and pink circles represent the total contribution. Each 2p$^5$3s$^2$3p$^5$ threshold is represented by an asterisk.\\label{fig:s_and_d}}\n\\end{figure}\n\nIn order to compare with this data we present in Figure \\ref{fig:valence} the total photoionization cross section from the initial $^2$P$^{\\rm{o}}$ ground state of Ar {\\sc ii} statistically weighted to the $J=3/2$ and $J=1/2$ states. We present two of our calculations in the figure, the most sophisticated DARC3 model and, in order to perform a direct comparison with the Breit-Pauli theoretical results of \\citet{2011PhRvA..84a3413C}, the PBP 124 level model outlined in Table \\ref{tab:calculations}. To match experimental resolving power, we convolute our total results with a 10meV gaussian profile at full-width half-maximum (FWHM). In addition, to replicate the target thresholds, we have shifted our threshold values recorded in Table \\ref{tab:energy} to the experimental NIST values where possible, during the diagonalization of the Hamiltonian matrix. The remaining levels not contained in NIST are shifted by an average proportion to each corresponding angular and spin momentum state, which has little effect on the background and is meant only for consistency. This ensures that resonance features are properly positioned with respect to the observed thresholds, making a direct comparison with experiment more meaningful.\n\n\n\n\n\\begin{figure*}\n\\includegraphics[scale=0.6, angle=-90]{figure5.eps}\n\\caption{Photoionization cross section measured in Mb on a logarithmic scale as a function of the photon energy in eV between 250-270eV. The circles are the experimental results from \\citet{2012PhRvA..85d3408B} with error bars included. The solid black line represents our current, DARC3 model results for the statistically weighted initial ground state, convoluted at 140meV at FWHM. \\label{fig:bigL}}\n\\end{figure*}\n\nWe can clearly see in Figure \\ref{fig:valence} that the low energy region just above threshold is completely dominated by 3s$^2$3p$^5$ $\\rightarrow$ 3s$^2$3p$^4nl$ transitions occurring at discrete energies prior to the ejection of an electron. This densely populated region of Rydberg resonances up to approximately 30eV is followed by a steep decline in the photoionization cross section forming the expected Cooper minimum around 45-50eV. This minimum is well known to appear in the spectra of noble gases \\citep{1962PhRv..128..681C}. Above this minimum the cross section rises due to excitations from 3p $\\rightarrow$ 3d transitions, before monotonically decreasing towards zero with increasing photon energy. \n\nExcellent agreement is evident between the 124 level PBP and the 48 level Breit-Pauli calculation of \\citet{2011PhRvA..84a3413C}, for all photon energies up to 60eV. Note that the cross section in Figure \\ref{fig:valence} is plotted on a log scale. Evidently the larger basis expansion of the present PBP evaluation, which includes the 4s, 4p and 4d orbitals, has minimal effect on the resulting photoionization cross section. Both of these Breit-Pauli evaluations, however, underestimate the cross section above roughly 45eV and lie considerably lower than the experimental measurements from ALS. The larger DARC3 evaluation, incorporating 557 fine-structure levels, gives much better agreement with experiment at photon energies above the Cooper minimum. This is partly due to the more substantial calculation, and also a more accurate description of the wavefunctions included. Both techniques in fact are known to reproduce similar results as shown in a study by \\citet{2005JPhB...38.1667B}, showing that the average difference in effective collision strengths for Fe$^{14+}$ to be 6$\\%$ between all transitions considered. The additional levels included, and the Rydberg resonances converging onto their thresholds, have the effect of raising the cross section above 45eV.\n\nIn order to further emphasize the excellent agreement between the DARC3 and the experimental measurements, we zoom in on the photon energy region just above threshold, from 27.8-29.2eV, in Figure \\ref{fig:valence_zoom}. It is clearly evident that the disparities found between theory and experiment in this very narrow energy range are negligible and excellent conformity is achieved. This high level of agreement supports the accuracy of the DARC3 evaluation and we believe that these valence shell photoionization cross sections for the ground state of Ar {\\sc ii} accurately reproduce the experimental spectrum. \n\nIn Figure \\ref{fig:excited} we present the total photoionization cross section for the process defined in Equation \\ref{eq:excited}, photoionization from the lowest excited initial 3s3p$^6\\; ^2$S$_{1/2}$ bound state of Ar {\\sc ii} to all possible allowed final states of Ar {\\sc iii}. These evaluations were carried out using the DARC3 model and present for the first time, cross sections for photoionization from an excited Ar {\\sc ii} state. There are no other theoretical or experimental data with which we can compare in this figure. The cross section is presented as a function of the photon energy in eV which ranges from just above the ionization threshold to beyond the opening of the L-shell thresholds. The photoionization cross section tends towards zero with increasing energy, and it is only due to the inclusion of the additional 10 hole states in the DARC3 model do we witness contributions to the cross section at photon energies between 200-250 eV. \n\n\\subsection{L-shell photoionization}\nCalculations and experiment have been carried out at the L-shell energy region between 250-280eV by \\citet{2012PhRvA..85d3408B} at the SOLEIL facility in France as described in Section \\ref{sec:introduction}. All the results herein have been convoluted with a 140meV Gaussian profile at FWHM to match the spectral resolution of experiment. Similar to the valence shell comparisons, the initial ground state cross section is formed from a statistically weighted average of the contributions from the odd $J=3/2$ and $J=1/2$ partial waves. Due to time of flight between the ion source and interacting region, excited levels can populate the main ion beam. This leads to a possible inclusion of the initial 3s3p$^6$ $^2$S$_{1/2}$ bound state which may also contribute to the total cross section. In Figure \\ref{fig:excited} we have already shown the immediate result of the lowest excited initial bound state transitions arising from the configuration 3s3p$^6$.\n\n\n\n\n\n\\begin{figure}\n\\includegraphics[scale=0.30, angle=-90]{figure6.eps}\n\\caption{The total convoluted FWHM at 140meV photoionization cross section between 254-256eV taken from Figure \\ref{fig:bigL} highlighting the intense resonant peaks. The spectra is broken into the contributions from each dipole allowed symmetry from both initial (middle third) and metastable (bottom third) initial states according to their statistical weighting. These even $J=5/2, 3/2, 1/2$ partial symmetries are the solid turquoise, red and purple lines. The total (top third) summed contribution is presented by the solid black curve and the two dominate resonances are marked by the dashed line. \\label{fig:res_peaks}}\n\\end{figure}\n\nIn order to compare with experiment, we have presented our results against various ionization channels from \\citet{2012PhRvA..85d3408B} in Figure \\ref{fig:s_and_d}. The timescale for Auger decay is much shorter than the time of flight required by the Argon ions after interaction with a photon, and therefore, the single ionization channel from experiment depicts the characteristics of photoionizing a valence electron. We can directly compare with this process in Figure \\ref{fig:s_and_d} by omitting the contribution from the additional 10 target states annotated by asterisks. Both above and during these thresholds we expect a rise in the photoionization cross section as more channels are opened and become accessible. The total result obtained by DARC3 can be compared directly to the combination of both single and double ionization modes of experiment. We have neglected the error bars for both modes in order to visualise the results more clearly.\n\nWe now present in Figure \\ref{fig:bigL} the photoionization cross section, on a linear scale, as a function of incident photon energy in eV across the L-shell threshold range from 250-270eV. Comparisons are made between the present DARC3 cross section and the measurements performed by \\citet{2012PhRvA..85d3408B}. Clearly excellent agreement is evident between theory and experiment across the range considered, as the features and energy positions of the resonance profiles exhibit good agreement. We note that as we have employed orbitals optimized on the valence state photoionization, an energy shift of 7.5eV was required to match the experimental spectra to our current results. The theory clearly predicts this process to a high standard of accuracy and allows us to benchmark the quality of results obtained from experiment.\n\nIn an attempt to investigate the features further, we have broken down the spectrum in Figure \\ref{fig:res_peaks} from the total into each of the allowed, final, even $J$ states $J=1/2$, $J=3/2$ and $J=5/2$. Clearly visible is the intense spike at $\\approx 254.9$ eV which is dominated by transitions of the form, 2p $\\rightarrow$ nd, ns which are engulfed by the convolution. The second strong peak at $\\approx 255.65$ eV is visible mostly through the metastable initial state transition from another strong 2p $\\rightarrow$ nd, J = 3/2 resonance. In reference to Figure \\ref{fig:excited}, the cross section has already reached close towards zero in the photon energy range of interest and therefore any contribution to the total cross section from these initial excited bound states would result in a reduction to the intensity of each resonant state. \n\nThis method of deconstructing the cross section is also important to identify which initial state has been photoionized during the experiment. It is clear however that the strongest profiles are not well isolated and therefore eliminates the possibility to further conduct any analysis on the weighted contributions. We therefore retain the statistical averaging of the ground state as our best result.\n\nAll resonances in this paper were identified using the technique detailed by \\citet{1996JPhB...29.4529Q} and \\citet{1998CoPhC.114..225Q}, which involves an analytic approach complementary to the \\textbf{R}-matrix method. By exploiting multichannel quantum defect theory \\citep{1983RPPh...46..167S}, each resonant state part of a Rydberg series has constant defect, $\\mu$ for each effective $n$ quantum number defined by,\n\\begin{equation}\\label{eq:resonance}\nE_r = E_{n \\rightarrow \\infty} - \\Big[\\frac{Z-N}{n-\\mu}\\Big]^2\n\\end{equation}\nwhere $E_r$ is the resonance energy converging to the target thresholds, $E_{n \\rightarrow \\infty}$. The overlapping nature of the resonant states makes it difficult to accurately evaluate resonance widths and assign each transition taking place. It is possible to deduce that the hole resonant states arise from 2p $\\rightarrow nd, (n+1)s$ transitions for $n\\ge3$, and correspond to the strongest peaks evident in Figure \\ref{fig:res_peaks}.\n\n\n\n\n\n\n\n\n\n\n\n\n\n4.1 Valence shell photoionization\n\\subsection{Valence shell photoionization}\nThe only available data currently in the literature for valence shell photoionization of Ar {\\sc ii} up to photon energies of 60eV is performed by \\citet{2011PhRvA..84a3413C}. In this paper both theoretical and experimental cross sections are presented. Absolute cross sections are obtained from the merged beam technique at the Advanced Light Source (ALS) with a spectral resolution of 10meV. It was found that the primary ion beam contained a mixture of both $^2$P$_{3/2}^{\\rm o}$ and $^2$P$_{1/2}^{\\rm o}$ initial states. Hence the total cross section was presented as a statistical weighting of the odd parity $J=3/2$ ground and $J=1/2$ metastable states respectively. The accompanying theoretical cross sections presented by \\citet{2011PhRvA..84a3413C} were evaluated using the Breit-Pauli \\textbf{R}-matrix approach. A total of 48 $LSJ\\pi$ fine-structure levels were included in the wavefunction representation with configurations 3s$^2$3p$^4$, 3s3p$^5$, 3p$^6$ and 3s$^2$3p$^2$3d$^2$. Some important correlation effects are thus omitted from this model such as levels associated with the 3s$^2$3p$^3$3d configuration and those arising from the lower $n=4$ complex. \n\n\n\n\n\\begin{figure}\n\\includegraphics[scale=0.29, angle=-90]{figure7.eps}\n\\caption{The photoionization cross section is presented on a linear scale against photon energy in eV above 261.2eV. The solid black line represents our current DARC3 model convoluted at 140meV FWHM and the dashed black line is the contribution to the cross section from valence shell photoionization of the 3s and 3p. The grey circles are experimental values of \\citet{2012PhRvA..85d3408B} for the single ionization channel and pink circles represent the total contribution. Each 2p$^5$3s$^2$3p$^5$ threshold is represented by an asterisk.\\label{fig:s_and_d}}\n\\end{figure}\n\nIn order to compare with this data we present in Figure \\ref{fig:valence} the total photoionization cross section from the initial $^2$P$^{\\rm{o}}$ ground state of Ar {\\sc ii} statistically weighted to the $J=3/2$ and $J=1/2$ states. We present two of our calculations in the figure, the most sophisticated DARC3 model and, in order to perform a direct comparison with the Breit-Pauli theoretical results of \\citet{2011PhRvA..84a3413C}, the PBP 124 level model outlined in Table \\ref{tab:calculations}. To match experimental resolving power, we convolute our total results with a 10meV gaussian profile at full-width half-maximum (FWHM). In addition, to replicate the target thresholds, we have shifted our threshold values recorded in Table \\ref{tab:energy} to the experimental NIST values where possible, during the diagonalization of the Hamiltonian matrix. The remaining levels not contained in NIST are shifted by an average proportion to each corresponding angular and spin momentum state, which has little effect on the background and is meant only for consistency. This ensures that resonance features are properly positioned with respect to the observed thresholds, making a direct comparison with experiment more meaningful.\n\n\n\n\n\\begin{figure*}\n\\includegraphics[scale=0.6, angle=-90]{figure5.eps}\n\\caption{Photoionization cross section measured in Mb on a logarithmic scale as a function of the photon energy in eV between 250-270eV. The circles are the experimental results from \\citet{2012PhRvA..85d3408B} with error bars included. The solid black line represents our current, DARC3 model results for the statistically weighted initial ground state, convoluted at 140meV at FWHM. \\label{fig:bigL}}\n\\end{figure*}\n\nWe can clearly see in Figure \\ref{fig:valence} that the low energy region just above threshold is completely dominated by 3s$^2$3p$^5$ $\\rightarrow$ 3s$^2$3p$^4nl$ transitions occurring at discrete energies prior to the ejection of an electron. This densely populated region of Rydberg resonances up to approximately 30eV is followed by a steep decline in the photoionization cross section forming the expected Cooper minimum around 45-50eV. This minimum is well known to appear in the spectra of noble gases \\citep{1962PhRv..128..681C}. Above this minimum the cross section rises due to excitations from 3p $\\rightarrow$ 3d transitions, before monotonically decreasing towards zero with increasing photon energy. \n\nExcellent agreement is evident between the 124 level PBP and the 48 level Breit-Pauli calculation of \\citet{2011PhRvA..84a3413C}, for all photon energies up to 60eV. Note that the cross section in Figure \\ref{fig:valence} is plotted on a log scale. Evidently the larger basis expansion of the present PBP evaluation, which includes the 4s, 4p and 4d orbitals, has minimal effect on the resulting photoionization cross section. Both of these Breit-Pauli evaluations, however, underestimate the cross section above roughly 45eV and lie considerably lower than the experimental measurements from ALS. The larger DARC3 evaluation, incorporating 557 fine-structure levels, gives much better agreement with experiment at photon energies above the Cooper minimum. This is partly due to the more substantial calculation, and also a more accurate description of the wavefunctions included. Both techniques in fact are known to reproduce similar results as shown in a study by \\citet{2005JPhB...38.1667B}, showing that the average difference in effective collision strengths for Fe$^{14+}$ to be 6$\\%$ between all transitions considered. The additional levels included, and the Rydberg resonances converging onto their thresholds, have the effect of raising the cross section above 45eV.\n\nIn order to further emphasize the excellent agreement between the DARC3 and the experimental measurements, we zoom in on the photon energy region just above threshold, from 27.8-29.2eV, in Figure \\ref{fig:valence_zoom}. It is clearly evident that the disparities found between theory and experiment in this very narrow energy range are negligible and excellent conformity is achieved. This high level of agreement supports the accuracy of the DARC3 evaluation and we believe that these valence shell photoionization cross sections for the ground state of Ar {\\sc ii} accurately reproduce the experimental spectrum. \n\nIn Figure \\ref{fig:excited} we present the total photoionization cross section for the process defined in Equation \\ref{eq:excited}, photoionization from the lowest excited initial 3s3p$^6\\; ^2$S$_{1/2}$ bound state of Ar {\\sc ii} to all possible allowed final states of Ar {\\sc iii}. These evaluations were carried out using the DARC3 model and present for the first time, cross sections for photoionization from an excited Ar {\\sc ii} state. There are no other theoretical or experimental data with which we can compare in this figure. The cross section is presented as a function of the photon energy in eV which ranges from just above the ionization threshold to beyond the opening of the L-shell thresholds. The photoionization cross section tends towards zero with increasing energy, and it is only due to the inclusion of the additional 10 hole states in the DARC3 model do we witness contributions to the cross section at photon energies between 200-250 eV. \n\n",
        "label": "fig:valence_zoom",
        "Descriptive_question1": "What is the photon energy range shown in figure_3?",
        "Descriptive_question2": "What color represents the experimental values in figure_3?",
        "Reasoning_question1": "How does the agreement between the DARC3 calculation and experimental values in figure_3 reflect the accuracy of the theoretical model in the photon energy range just above threshold?",
        "Reasoning_question2": "What might explain the negligible disparities between the theoretical DARC3 results and experimental data observed in figure_3 within the energy range of 27.8-29.2 eV?",
        "Descriptive_answer1": "27.8-29.2 eV",
        "Descriptive_answer2": "yellow",
        "Reasoning_answer1": "The excellent conformity between the DARC3 calculation (solid black line) and the experimental values (yellow circles) in Figure 3 within the photon energy range of 27.8-29.2 eV indicates that the theoretical model accurately captures the physical processes involved in valence shell photoionization near threshold. This close agreement suggests that the DARC3 model effectively represents the wavefunctions and resonance structures in this energy domain, validating its precision for predicting cross section behavior just above the ionization threshold.",
        "Reasoning_answer2": "The negligible disparities between the theoretical DARC3 results and experimental data in the 27.8-29.2 eV range can be attributed to several factors. First, the DARC3 model includes a comprehensive set of 557 fine-structure levels, providing an accurate and detailed description of the atomic states involved. Second, the calculation incorporates a statistical weighting of initial ground and metastable states, aligning with the experimental composition of the ion beam. Third, threshold energies were shifted to match experimental NIST values, ensuring proper resonance positioning. Together, these factors contribute to the close match observed, minimizing deviations in this narrow energy interval."
    },
    {
        "paper_id": "1511.03946.json",
        "image_id": "figure_4",
        "image_path": [
            "/home/yz979/scratch/chengye/scimolmo/test-images-tables/dataset_imagestables/1511.03946/images/figure4.eps"
        ],
        "caption": "Total ground state photoionization cross section measured in Mb as a function of the photon energy between 0-280eV. The transition is from the initial state 3s3p$^6$ $^2$S$_{1/2}$ to all allowed final states from the DARC3 model. \\label{fig:excited}",
        "classify": "Chart",
        "section_info": "4 Results\n\\section{Results}\\label{sec:results}\nBefore embarking on the large scale DARC3 calculation we thought it prudent to investigate first the important properties and characteristics found in the photoionization cross section of Ar {\\sc ii} in its ground state. In Figure \\ref{fig:comparison}\nwe present the total photoionization cross section in Mb on a logarithmic scale as a function of photon energy in eV, from the initial ground Ar {\\sc ii} $^2$P$^{\\rm{o}}_{3/2}$ state to all allowed final states. Three calculations are presented in this figure; both the 209 level DARC1 and its contributions from the 3s$^2$3p$^4$ levels indexed as 1-5 in Table \\ref{tab:energy} and the extended 257 level DARC2 calculation. Clearly Figure \\ref{fig:comparison}\nshows the importance of including at least the first five 3s$^2$3p$^4$ levels of Ar {\\sc iii} in this photoionization calculation. The contributions from these levels dominates the total cross section up to a photon energy of approximately 50eV and all three calculations exhibit excellent agreement up to this point. It is essential, therefore, that an accurate description is achieved for the wavefunction representation of those low-lying levels. Above 50eV the additional levels associated with the more complex DARC1 and DARC2 models come into play and the cross section rises as we move to higher photon energies as more channels become accessible. Interestingly the inclusion of the additional 3s$^2$3p$^3$4d and 3s$^2$3p$^3$5s levels in the DARC2 model has little or no effect on the photoionization cross section produced by the DARC1 model up to 60eV, both datasets showing near perfect agreement. Therefore we do not retain these additional 3s$^2$3p$^3$4d and 3s$^2$3p$^3$5s configurations in our largest DARC3 calculation as can be seen from Table \\ref{tab:calculations}.\n\n\n\n\n\\begin{figure*}\n\\includegraphics[scale=0.6, angle=-90]{figure3.eps}\n\\caption{Photoionization cross section measured in Mb on a logarithmic scale as a function of the photon energy between 27.8-29.2 eV just above threshold. The solid black line is the current statistically weighted, initial ground state, DARC3 calculation against the experimental values from \\citet{2011PhRvA..84a3413C} represented by the yellow circles taken from Figure \\ref{fig:valence}. \\label{fig:valence_zoom}}\n\\end{figure*}\n\n\n\n\n\\begin{figure*}\n\\includegraphics[scale=0.58, angle=-90]{figure4.eps}\n\\caption{Total ground state photoionization cross section measured in Mb as a function of the photon energy between 0-280eV. The transition is from the initial state 3s3p$^6$ $^2$S$_{1/2}$ to all allowed final states from the DARC3 model. \\label{fig:excited}}\n\\end{figure*}\n\n\\subsection{Valence shell photoionization}\nThe only available data currently in the literature for valence shell photoionization of Ar {\\sc ii} up to photon energies of 60eV is performed by \\citet{2011PhRvA..84a3413C}. In this paper both theoretical and experimental cross sections are presented. Absolute cross sections are obtained from the merged beam technique at the Advanced Light Source (ALS) with a spectral resolution of 10meV. It was found that the primary ion beam contained a mixture of both $^2$P$_{3/2}^{\\rm o}$ and $^2$P$_{1/2}^{\\rm o}$ initial states. Hence the total cross section was presented as a statistical weighting of the odd parity $J=3/2$ ground and $J=1/2$ metastable states respectively. The accompanying theoretical cross sections presented by \\citet{2011PhRvA..84a3413C} were evaluated using the Breit-Pauli \\textbf{R}-matrix approach. A total of 48 $LSJ\\pi$ fine-structure levels were included in the wavefunction representation with configurations 3s$^2$3p$^4$, 3s3p$^5$, 3p$^6$ and 3s$^2$3p$^2$3d$^2$. Some important correlation effects are thus omitted from this model such as levels associated with the 3s$^2$3p$^3$3d configuration and those arising from the lower $n=4$ complex. \n\n\n\n\n\\begin{figure}\n\\includegraphics[scale=0.29, angle=-90]{figure7.eps}\n\\caption{The photoionization cross section is presented on a linear scale against photon energy in eV above 261.2eV. The solid black line represents our current DARC3 model convoluted at 140meV FWHM and the dashed black line is the contribution to the cross section from valence shell photoionization of the 3s and 3p. The grey circles are experimental values of \\citet{2012PhRvA..85d3408B} for the single ionization channel and pink circles represent the total contribution. Each 2p$^5$3s$^2$3p$^5$ threshold is represented by an asterisk.\\label{fig:s_and_d}}\n\\end{figure}\n\nIn order to compare with this data we present in Figure \\ref{fig:valence} the total photoionization cross section from the initial $^2$P$^{\\rm{o}}$ ground state of Ar {\\sc ii} statistically weighted to the $J=3/2$ and $J=1/2$ states. We present two of our calculations in the figure, the most sophisticated DARC3 model and, in order to perform a direct comparison with the Breit-Pauli theoretical results of \\citet{2011PhRvA..84a3413C}, the PBP 124 level model outlined in Table \\ref{tab:calculations}. To match experimental resolving power, we convolute our total results with a 10meV gaussian profile at full-width half-maximum (FWHM). In addition, to replicate the target thresholds, we have shifted our threshold values recorded in Table \\ref{tab:energy} to the experimental NIST values where possible, during the diagonalization of the Hamiltonian matrix. The remaining levels not contained in NIST are shifted by an average proportion to each corresponding angular and spin momentum state, which has little effect on the background and is meant only for consistency. This ensures that resonance features are properly positioned with respect to the observed thresholds, making a direct comparison with experiment more meaningful.\n\n\n\n\n\\begin{figure*}\n\\includegraphics[scale=0.6, angle=-90]{figure5.eps}\n\\caption{Photoionization cross section measured in Mb on a logarithmic scale as a function of the photon energy in eV between 250-270eV. The circles are the experimental results from \\citet{2012PhRvA..85d3408B} with error bars included. The solid black line represents our current, DARC3 model results for the statistically weighted initial ground state, convoluted at 140meV at FWHM. \\label{fig:bigL}}\n\\end{figure*}\n\nWe can clearly see in Figure \\ref{fig:valence} that the low energy region just above threshold is completely dominated by 3s$^2$3p$^5$ $\\rightarrow$ 3s$^2$3p$^4nl$ transitions occurring at discrete energies prior to the ejection of an electron. This densely populated region of Rydberg resonances up to approximately 30eV is followed by a steep decline in the photoionization cross section forming the expected Cooper minimum around 45-50eV. This minimum is well known to appear in the spectra of noble gases \\citep{1962PhRv..128..681C}. Above this minimum the cross section rises due to excitations from 3p $\\rightarrow$ 3d transitions, before monotonically decreasing towards zero with increasing photon energy. \n\nExcellent agreement is evident between the 124 level PBP and the 48 level Breit-Pauli calculation of \\citet{2011PhRvA..84a3413C}, for all photon energies up to 60eV. Note that the cross section in Figure \\ref{fig:valence} is plotted on a log scale. Evidently the larger basis expansion of the present PBP evaluation, which includes the 4s, 4p and 4d orbitals, has minimal effect on the resulting photoionization cross section. Both of these Breit-Pauli evaluations, however, underestimate the cross section above roughly 45eV and lie considerably lower than the experimental measurements from ALS. The larger DARC3 evaluation, incorporating 557 fine-structure levels, gives much better agreement with experiment at photon energies above the Cooper minimum. This is partly due to the more substantial calculation, and also a more accurate description of the wavefunctions included. Both techniques in fact are known to reproduce similar results as shown in a study by \\citet{2005JPhB...38.1667B}, showing that the average difference in effective collision strengths for Fe$^{14+}$ to be 6$\\%$ between all transitions considered. The additional levels included, and the Rydberg resonances converging onto their thresholds, have the effect of raising the cross section above 45eV.\n\nIn order to further emphasize the excellent agreement between the DARC3 and the experimental measurements, we zoom in on the photon energy region just above threshold, from 27.8-29.2eV, in Figure \\ref{fig:valence_zoom}. It is clearly evident that the disparities found between theory and experiment in this very narrow energy range are negligible and excellent conformity is achieved. This high level of agreement supports the accuracy of the DARC3 evaluation and we believe that these valence shell photoionization cross sections for the ground state of Ar {\\sc ii} accurately reproduce the experimental spectrum. \n\nIn Figure \\ref{fig:excited} we present the total photoionization cross section for the process defined in Equation \\ref{eq:excited}, photoionization from the lowest excited initial 3s3p$^6\\; ^2$S$_{1/2}$ bound state of Ar {\\sc ii} to all possible allowed final states of Ar {\\sc iii}. These evaluations were carried out using the DARC3 model and present for the first time, cross sections for photoionization from an excited Ar {\\sc ii} state. There are no other theoretical or experimental data with which we can compare in this figure. The cross section is presented as a function of the photon energy in eV which ranges from just above the ionization threshold to beyond the opening of the L-shell thresholds. The photoionization cross section tends towards zero with increasing energy, and it is only due to the inclusion of the additional 10 hole states in the DARC3 model do we witness contributions to the cross section at photon energies between 200-250 eV. \n\n\\subsection{L-shell photoionization}\nCalculations and experiment have been carried out at the L-shell energy region between 250-280eV by \\citet{2012PhRvA..85d3408B} at the SOLEIL facility in France as described in Section \\ref{sec:introduction}. All the results herein have been convoluted with a 140meV Gaussian profile at FWHM to match the spectral resolution of experiment. Similar to the valence shell comparisons, the initial ground state cross section is formed from a statistically weighted average of the contributions from the odd $J=3/2$ and $J=1/2$ partial waves. Due to time of flight between the ion source and interacting region, excited levels can populate the main ion beam. This leads to a possible inclusion of the initial 3s3p$^6$ $^2$S$_{1/2}$ bound state which may also contribute to the total cross section. In Figure \\ref{fig:excited} we have already shown the immediate result of the lowest excited initial bound state transitions arising from the configuration 3s3p$^6$.\n\n\n\n\n\n\\begin{figure}\n\\includegraphics[scale=0.30, angle=-90]{figure6.eps}\n\\caption{The total convoluted FWHM at 140meV photoionization cross section between 254-256eV taken from Figure \\ref{fig:bigL} highlighting the intense resonant peaks. The spectra is broken into the contributions from each dipole allowed symmetry from both initial (middle third) and metastable (bottom third) initial states according to their statistical weighting. These even $J=5/2, 3/2, 1/2$ partial symmetries are the solid turquoise, red and purple lines. The total (top third) summed contribution is presented by the solid black curve and the two dominate resonances are marked by the dashed line. \\label{fig:res_peaks}}\n\\end{figure}\n\nIn order to compare with experiment, we have presented our results against various ionization channels from \\citet{2012PhRvA..85d3408B} in Figure \\ref{fig:s_and_d}. The timescale for Auger decay is much shorter than the time of flight required by the Argon ions after interaction with a photon, and therefore, the single ionization channel from experiment depicts the characteristics of photoionizing a valence electron. We can directly compare with this process in Figure \\ref{fig:s_and_d} by omitting the contribution from the additional 10 target states annotated by asterisks. Both above and during these thresholds we expect a rise in the photoionization cross section as more channels are opened and become accessible. The total result obtained by DARC3 can be compared directly to the combination of both single and double ionization modes of experiment. We have neglected the error bars for both modes in order to visualise the results more clearly.\n\nWe now present in Figure \\ref{fig:bigL} the photoionization cross section, on a linear scale, as a function of incident photon energy in eV across the L-shell threshold range from 250-270eV. Comparisons are made between the present DARC3 cross section and the measurements performed by \\citet{2012PhRvA..85d3408B}. Clearly excellent agreement is evident between theory and experiment across the range considered, as the features and energy positions of the resonance profiles exhibit good agreement. We note that as we have employed orbitals optimized on the valence state photoionization, an energy shift of 7.5eV was required to match the experimental spectra to our current results. The theory clearly predicts this process to a high standard of accuracy and allows us to benchmark the quality of results obtained from experiment.\n\nIn an attempt to investigate the features further, we have broken down the spectrum in Figure \\ref{fig:res_peaks} from the total into each of the allowed, final, even $J$ states $J=1/2$, $J=3/2$ and $J=5/2$. Clearly visible is the intense spike at $\\approx 254.9$ eV which is dominated by transitions of the form, 2p $\\rightarrow$ nd, ns which are engulfed by the convolution. The second strong peak at $\\approx 255.65$ eV is visible mostly through the metastable initial state transition from another strong 2p $\\rightarrow$ nd, J = 3/2 resonance. In reference to Figure \\ref{fig:excited}, the cross section has already reached close towards zero in the photon energy range of interest and therefore any contribution to the total cross section from these initial excited bound states would result in a reduction to the intensity of each resonant state. \n\nThis method of deconstructing the cross section is also important to identify which initial state has been photoionized during the experiment. It is clear however that the strongest profiles are not well isolated and therefore eliminates the possibility to further conduct any analysis on the weighted contributions. We therefore retain the statistical averaging of the ground state as our best result.\n\nAll resonances in this paper were identified using the technique detailed by \\citet{1996JPhB...29.4529Q} and \\citet{1998CoPhC.114..225Q}, which involves an analytic approach complementary to the \\textbf{R}-matrix method. By exploiting multichannel quantum defect theory \\citep{1983RPPh...46..167S}, each resonant state part of a Rydberg series has constant defect, $\\mu$ for each effective $n$ quantum number defined by,\n\\begin{equation}\\label{eq:resonance}\nE_r = E_{n \\rightarrow \\infty} - \\Big[\\frac{Z-N}{n-\\mu}\\Big]^2\n\\end{equation}\nwhere $E_r$ is the resonance energy converging to the target thresholds, $E_{n \\rightarrow \\infty}$. The overlapping nature of the resonant states makes it difficult to accurately evaluate resonance widths and assign each transition taking place. It is possible to deduce that the hole resonant states arise from 2p $\\rightarrow nd, (n+1)s$ transitions for $n\\ge3$, and correspond to the strongest peaks evident in Figure \\ref{fig:res_peaks}.\n\n\n\n\n\n\n\n\n\n\n\n\n\n4.1 Valence shell photoionization\n\\subsection{Valence shell photoionization}\nThe only available data currently in the literature for valence shell photoionization of Ar {\\sc ii} up to photon energies of 60eV is performed by \\citet{2011PhRvA..84a3413C}. In this paper both theoretical and experimental cross sections are presented. Absolute cross sections are obtained from the merged beam technique at the Advanced Light Source (ALS) with a spectral resolution of 10meV. It was found that the primary ion beam contained a mixture of both $^2$P$_{3/2}^{\\rm o}$ and $^2$P$_{1/2}^{\\rm o}$ initial states. Hence the total cross section was presented as a statistical weighting of the odd parity $J=3/2$ ground and $J=1/2$ metastable states respectively. The accompanying theoretical cross sections presented by \\citet{2011PhRvA..84a3413C} were evaluated using the Breit-Pauli \\textbf{R}-matrix approach. A total of 48 $LSJ\\pi$ fine-structure levels were included in the wavefunction representation with configurations 3s$^2$3p$^4$, 3s3p$^5$, 3p$^6$ and 3s$^2$3p$^2$3d$^2$. Some important correlation effects are thus omitted from this model such as levels associated with the 3s$^2$3p$^3$3d configuration and those arising from the lower $n=4$ complex. \n\n\n\n\n\\begin{figure}\n\\includegraphics[scale=0.29, angle=-90]{figure7.eps}\n\\caption{The photoionization cross section is presented on a linear scale against photon energy in eV above 261.2eV. The solid black line represents our current DARC3 model convoluted at 140meV FWHM and the dashed black line is the contribution to the cross section from valence shell photoionization of the 3s and 3p. The grey circles are experimental values of \\citet{2012PhRvA..85d3408B} for the single ionization channel and pink circles represent the total contribution. Each 2p$^5$3s$^2$3p$^5$ threshold is represented by an asterisk.\\label{fig:s_and_d}}\n\\end{figure}\n\nIn order to compare with this data we present in Figure \\ref{fig:valence} the total photoionization cross section from the initial $^2$P$^{\\rm{o}}$ ground state of Ar {\\sc ii} statistically weighted to the $J=3/2$ and $J=1/2$ states. We present two of our calculations in the figure, the most sophisticated DARC3 model and, in order to perform a direct comparison with the Breit-Pauli theoretical results of \\citet{2011PhRvA..84a3413C}, the PBP 124 level model outlined in Table \\ref{tab:calculations}. To match experimental resolving power, we convolute our total results with a 10meV gaussian profile at full-width half-maximum (FWHM). In addition, to replicate the target thresholds, we have shifted our threshold values recorded in Table \\ref{tab:energy} to the experimental NIST values where possible, during the diagonalization of the Hamiltonian matrix. The remaining levels not contained in NIST are shifted by an average proportion to each corresponding angular and spin momentum state, which has little effect on the background and is meant only for consistency. This ensures that resonance features are properly positioned with respect to the observed thresholds, making a direct comparison with experiment more meaningful.\n\n\n\n\n\\begin{figure*}\n\\includegraphics[scale=0.6, angle=-90]{figure5.eps}\n\\caption{Photoionization cross section measured in Mb on a logarithmic scale as a function of the photon energy in eV between 250-270eV. The circles are the experimental results from \\citet{2012PhRvA..85d3408B} with error bars included. The solid black line represents our current, DARC3 model results for the statistically weighted initial ground state, convoluted at 140meV at FWHM. \\label{fig:bigL}}\n\\end{figure*}\n\nWe can clearly see in Figure \\ref{fig:valence} that the low energy region just above threshold is completely dominated by 3s$^2$3p$^5$ $\\rightarrow$ 3s$^2$3p$^4nl$ transitions occurring at discrete energies prior to the ejection of an electron. This densely populated region of Rydberg resonances up to approximately 30eV is followed by a steep decline in the photoionization cross section forming the expected Cooper minimum around 45-50eV. This minimum is well known to appear in the spectra of noble gases \\citep{1962PhRv..128..681C}. Above this minimum the cross section rises due to excitations from 3p $\\rightarrow$ 3d transitions, before monotonically decreasing towards zero with increasing photon energy. \n\nExcellent agreement is evident between the 124 level PBP and the 48 level Breit-Pauli calculation of \\citet{2011PhRvA..84a3413C}, for all photon energies up to 60eV. Note that the cross section in Figure \\ref{fig:valence} is plotted on a log scale. Evidently the larger basis expansion of the present PBP evaluation, which includes the 4s, 4p and 4d orbitals, has minimal effect on the resulting photoionization cross section. Both of these Breit-Pauli evaluations, however, underestimate the cross section above roughly 45eV and lie considerably lower than the experimental measurements from ALS. The larger DARC3 evaluation, incorporating 557 fine-structure levels, gives much better agreement with experiment at photon energies above the Cooper minimum. This is partly due to the more substantial calculation, and also a more accurate description of the wavefunctions included. Both techniques in fact are known to reproduce similar results as shown in a study by \\citet{2005JPhB...38.1667B}, showing that the average difference in effective collision strengths for Fe$^{14+}$ to be 6$\\%$ between all transitions considered. The additional levels included, and the Rydberg resonances converging onto their thresholds, have the effect of raising the cross section above 45eV.\n\nIn order to further emphasize the excellent agreement between the DARC3 and the experimental measurements, we zoom in on the photon energy region just above threshold, from 27.8-29.2eV, in Figure \\ref{fig:valence_zoom}. It is clearly evident that the disparities found between theory and experiment in this very narrow energy range are negligible and excellent conformity is achieved. This high level of agreement supports the accuracy of the DARC3 evaluation and we believe that these valence shell photoionization cross sections for the ground state of Ar {\\sc ii} accurately reproduce the experimental spectrum. \n\nIn Figure \\ref{fig:excited} we present the total photoionization cross section for the process defined in Equation \\ref{eq:excited}, photoionization from the lowest excited initial 3s3p$^6\\; ^2$S$_{1/2}$ bound state of Ar {\\sc ii} to all possible allowed final states of Ar {\\sc iii}. These evaluations were carried out using the DARC3 model and present for the first time, cross sections for photoionization from an excited Ar {\\sc ii} state. There are no other theoretical or experimental data with which we can compare in this figure. The cross section is presented as a function of the photon energy in eV which ranges from just above the ionization threshold to beyond the opening of the L-shell thresholds. The photoionization cross section tends towards zero with increasing energy, and it is only due to the inclusion of the additional 10 hole states in the DARC3 model do we witness contributions to the cross section at photon energies between 200-250 eV. \n\n4.2 L-shell photoionization\n\\subsection{L-shell photoionization}\nCalculations and experiment have been carried out at the L-shell energy region between 250-280eV by \\citet{2012PhRvA..85d3408B} at the SOLEIL facility in France as described in Section \\ref{sec:introduction}. All the results herein have been convoluted with a 140meV Gaussian profile at FWHM to match the spectral resolution of experiment. Similar to the valence shell comparisons, the initial ground state cross section is formed from a statistically weighted average of the contributions from the odd $J=3/2$ and $J=1/2$ partial waves. Due to time of flight between the ion source and interacting region, excited levels can populate the main ion beam. This leads to a possible inclusion of the initial 3s3p$^6$ $^2$S$_{1/2}$ bound state which may also contribute to the total cross section. In Figure \\ref{fig:excited} we have already shown the immediate result of the lowest excited initial bound state transitions arising from the configuration 3s3p$^6$.\n\n\n\n\n\n\\begin{figure}\n\\includegraphics[scale=0.30, angle=-90]{figure6.eps}\n\\caption{The total convoluted FWHM at 140meV photoionization cross section between 254-256eV taken from Figure \\ref{fig:bigL} highlighting the intense resonant peaks. The spectra is broken into the contributions from each dipole allowed symmetry from both initial (middle third) and metastable (bottom third) initial states according to their statistical weighting. These even $J=5/2, 3/2, 1/2$ partial symmetries are the solid turquoise, red and purple lines. The total (top third) summed contribution is presented by the solid black curve and the two dominate resonances are marked by the dashed line. \\label{fig:res_peaks}}\n\\end{figure}\n\nIn order to compare with experiment, we have presented our results against various ionization channels from \\citet{2012PhRvA..85d3408B} in Figure \\ref{fig:s_and_d}. The timescale for Auger decay is much shorter than the time of flight required by the Argon ions after interaction with a photon, and therefore, the single ionization channel from experiment depicts the characteristics of photoionizing a valence electron. We can directly compare with this process in Figure \\ref{fig:s_and_d} by omitting the contribution from the additional 10 target states annotated by asterisks. Both above and during these thresholds we expect a rise in the photoionization cross section as more channels are opened and become accessible. The total result obtained by DARC3 can be compared directly to the combination of both single and double ionization modes of experiment. We have neglected the error bars for both modes in order to visualise the results more clearly.\n\nWe now present in Figure \\ref{fig:bigL} the photoionization cross section, on a linear scale, as a function of incident photon energy in eV across the L-shell threshold range from 250-270eV. Comparisons are made between the present DARC3 cross section and the measurements performed by \\citet{2012PhRvA..85d3408B}. Clearly excellent agreement is evident between theory and experiment across the range considered, as the features and energy positions of the resonance profiles exhibit good agreement. We note that as we have employed orbitals optimized on the valence state photoionization, an energy shift of 7.5eV was required to match the experimental spectra to our current results. The theory clearly predicts this process to a high standard of accuracy and allows us to benchmark the quality of results obtained from experiment.\n\nIn an attempt to investigate the features further, we have broken down the spectrum in Figure \\ref{fig:res_peaks} from the total into each of the allowed, final, even $J$ states $J=1/2$, $J=3/2$ and $J=5/2$. Clearly visible is the intense spike at $\\approx 254.9$ eV which is dominated by transitions of the form, 2p $\\rightarrow$ nd, ns which are engulfed by the convolution. The second strong peak at $\\approx 255.65$ eV is visible mostly through the metastable initial state transition from another strong 2p $\\rightarrow$ nd, J = 3/2 resonance. In reference to Figure \\ref{fig:excited}, the cross section has already reached close towards zero in the photon energy range of interest and therefore any contribution to the total cross section from these initial excited bound states would result in a reduction to the intensity of each resonant state. \n\nThis method of deconstructing the cross section is also important to identify which initial state has been photoionized during the experiment. It is clear however that the strongest profiles are not well isolated and therefore eliminates the possibility to further conduct any analysis on the weighted contributions. We therefore retain the statistical averaging of the ground state as our best result.\n\nAll resonances in this paper were identified using the technique detailed by \\citet{1996JPhB...29.4529Q} and \\citet{1998CoPhC.114..225Q}, which involves an analytic approach complementary to the \\textbf{R}-matrix method. By exploiting multichannel quantum defect theory \\citep{1983RPPh...46..167S}, each resonant state part of a Rydberg series has constant defect, $\\mu$ for each effective $n$ quantum number defined by,\n\\begin{equation}\\label{eq:resonance}\nE_r = E_{n \\rightarrow \\infty} - \\Big[\\frac{Z-N}{n-\\mu}\\Big]^2\n\\end{equation}\nwhere $E_r$ is the resonance energy converging to the target thresholds, $E_{n \\rightarrow \\infty}$. The overlapping nature of the resonant states makes it difficult to accurately evaluate resonance widths and assign each transition taking place. It is possible to deduce that the hole resonant states arise from 2p $\\rightarrow nd, (n+1)s$ transitions for $n\\ge3$, and correspond to the strongest peaks evident in Figure \\ref{fig:res_peaks}.\n\n\n\n\n\n\n\n\n\n\n\n\n\n",
        "label": "fig:excited",
        "Descriptive_question1": "What is the range of photon energy shown in figure_4?",
        "Descriptive_question2": "What unit is used to measure the photoionization cross section in figure_4?",
        "Reasoning_question1": "Why does the photoionization cross section in figure_4 tend towards zero with increasing photon energy?",
        "Reasoning_question2": "How does the inclusion of additional hole states in the DARC3 model affect the photoionization cross section between 200-250 eV in figure_4?",
        "Descriptive_answer1": "0-280 eV",
        "Descriptive_answer2": "Mb",
        "Reasoning_answer1": "As photon energy increases, the probability of photoionization generally decreases because higher energy photons tend to pass through the ion without interacting, causing the cross section to approach zero at very high energies.",
        "Reasoning_answer2": "The inclusion of additional 10 hole states in the DARC3 model introduces more accessible channels for photoionization between 200-250 eV, resulting in noticeable contributions to the cross section in this energy range, which otherwise tends towards zero without these additional states."
    },
    {
        "paper_id": "1511.03946.json",
        "image_id": "figure_5",
        "image_path": [
            "/home/yz979/scratch/chengye/scimolmo/test-images-tables/dataset_imagestables/1511.03946/images/figure7.eps"
        ],
        "caption": "The photoionization cross section is presented on a linear scale against photon energy in eV above 261.2eV. The solid black line represents our current DARC3 model convoluted at 140meV FWHM and the dashed black line is the contribution to the cross section from valence shell photoionization of the 3s and 3p. The grey circles are experimental values of \\citet{2012PhRvA..85d3408B} for the single ionization channel and pink circles represent the total contribution. Each 2p$^5$3s$^2$3p$^5$ threshold is represented by an asterisk.\\label{fig:s_and_d}",
        "classify": "Chart",
        "section_info": "4 Results\n\\section{Results}\\label{sec:results}\nBefore embarking on the large scale DARC3 calculation we thought it prudent to investigate first the important properties and characteristics found in the photoionization cross section of Ar {\\sc ii} in its ground state. In Figure \\ref{fig:comparison}\nwe present the total photoionization cross section in Mb on a logarithmic scale as a function of photon energy in eV, from the initial ground Ar {\\sc ii} $^2$P$^{\\rm{o}}_{3/2}$ state to all allowed final states. Three calculations are presented in this figure; both the 209 level DARC1 and its contributions from the 3s$^2$3p$^4$ levels indexed as 1-5 in Table \\ref{tab:energy} and the extended 257 level DARC2 calculation. Clearly Figure \\ref{fig:comparison}\nshows the importance of including at least the first five 3s$^2$3p$^4$ levels of Ar {\\sc iii} in this photoionization calculation. The contributions from these levels dominates the total cross section up to a photon energy of approximately 50eV and all three calculations exhibit excellent agreement up to this point. It is essential, therefore, that an accurate description is achieved for the wavefunction representation of those low-lying levels. Above 50eV the additional levels associated with the more complex DARC1 and DARC2 models come into play and the cross section rises as we move to higher photon energies as more channels become accessible. Interestingly the inclusion of the additional 3s$^2$3p$^3$4d and 3s$^2$3p$^3$5s levels in the DARC2 model has little or no effect on the photoionization cross section produced by the DARC1 model up to 60eV, both datasets showing near perfect agreement. Therefore we do not retain these additional 3s$^2$3p$^3$4d and 3s$^2$3p$^3$5s configurations in our largest DARC3 calculation as can be seen from Table \\ref{tab:calculations}.\n\n\n\n\n\\begin{figure*}\n\\includegraphics[scale=0.6, angle=-90]{figure3.eps}\n\\caption{Photoionization cross section measured in Mb on a logarithmic scale as a function of the photon energy between 27.8-29.2 eV just above threshold. The solid black line is the current statistically weighted, initial ground state, DARC3 calculation against the experimental values from \\citet{2011PhRvA..84a3413C} represented by the yellow circles taken from Figure \\ref{fig:valence}. \\label{fig:valence_zoom}}\n\\end{figure*}\n\n\n\n\n\\begin{figure*}\n\\includegraphics[scale=0.58, angle=-90]{figure4.eps}\n\\caption{Total ground state photoionization cross section measured in Mb as a function of the photon energy between 0-280eV. The transition is from the initial state 3s3p$^6$ $^2$S$_{1/2}$ to all allowed final states from the DARC3 model. \\label{fig:excited}}\n\\end{figure*}\n\n\\subsection{Valence shell photoionization}\nThe only available data currently in the literature for valence shell photoionization of Ar {\\sc ii} up to photon energies of 60eV is performed by \\citet{2011PhRvA..84a3413C}. In this paper both theoretical and experimental cross sections are presented. Absolute cross sections are obtained from the merged beam technique at the Advanced Light Source (ALS) with a spectral resolution of 10meV. It was found that the primary ion beam contained a mixture of both $^2$P$_{3/2}^{\\rm o}$ and $^2$P$_{1/2}^{\\rm o}$ initial states. Hence the total cross section was presented as a statistical weighting of the odd parity $J=3/2$ ground and $J=1/2$ metastable states respectively. The accompanying theoretical cross sections presented by \\citet{2011PhRvA..84a3413C} were evaluated using the Breit-Pauli \\textbf{R}-matrix approach. A total of 48 $LSJ\\pi$ fine-structure levels were included in the wavefunction representation with configurations 3s$^2$3p$^4$, 3s3p$^5$, 3p$^6$ and 3s$^2$3p$^2$3d$^2$. Some important correlation effects are thus omitted from this model such as levels associated with the 3s$^2$3p$^3$3d configuration and those arising from the lower $n=4$ complex. \n\n\n\n\n\\begin{figure}\n\\includegraphics[scale=0.29, angle=-90]{figure7.eps}\n\\caption{The photoionization cross section is presented on a linear scale against photon energy in eV above 261.2eV. The solid black line represents our current DARC3 model convoluted at 140meV FWHM and the dashed black line is the contribution to the cross section from valence shell photoionization of the 3s and 3p. The grey circles are experimental values of \\citet{2012PhRvA..85d3408B} for the single ionization channel and pink circles represent the total contribution. Each 2p$^5$3s$^2$3p$^5$ threshold is represented by an asterisk.\\label{fig:s_and_d}}\n\\end{figure}\n\nIn order to compare with this data we present in Figure \\ref{fig:valence} the total photoionization cross section from the initial $^2$P$^{\\rm{o}}$ ground state of Ar {\\sc ii} statistically weighted to the $J=3/2$ and $J=1/2$ states. We present two of our calculations in the figure, the most sophisticated DARC3 model and, in order to perform a direct comparison with the Breit-Pauli theoretical results of \\citet{2011PhRvA..84a3413C}, the PBP 124 level model outlined in Table \\ref{tab:calculations}. To match experimental resolving power, we convolute our total results with a 10meV gaussian profile at full-width half-maximum (FWHM). In addition, to replicate the target thresholds, we have shifted our threshold values recorded in Table \\ref{tab:energy} to the experimental NIST values where possible, during the diagonalization of the Hamiltonian matrix. The remaining levels not contained in NIST are shifted by an average proportion to each corresponding angular and spin momentum state, which has little effect on the background and is meant only for consistency. This ensures that resonance features are properly positioned with respect to the observed thresholds, making a direct comparison with experiment more meaningful.\n\n\n\n\n\\begin{figure*}\n\\includegraphics[scale=0.6, angle=-90]{figure5.eps}\n\\caption{Photoionization cross section measured in Mb on a logarithmic scale as a function of the photon energy in eV between 250-270eV. The circles are the experimental results from \\citet{2012PhRvA..85d3408B} with error bars included. The solid black line represents our current, DARC3 model results for the statistically weighted initial ground state, convoluted at 140meV at FWHM. \\label{fig:bigL}}\n\\end{figure*}\n\nWe can clearly see in Figure \\ref{fig:valence} that the low energy region just above threshold is completely dominated by 3s$^2$3p$^5$ $\\rightarrow$ 3s$^2$3p$^4nl$ transitions occurring at discrete energies prior to the ejection of an electron. This densely populated region of Rydberg resonances up to approximately 30eV is followed by a steep decline in the photoionization cross section forming the expected Cooper minimum around 45-50eV. This minimum is well known to appear in the spectra of noble gases \\citep{1962PhRv..128..681C}. Above this minimum the cross section rises due to excitations from 3p $\\rightarrow$ 3d transitions, before monotonically decreasing towards zero with increasing photon energy. \n\nExcellent agreement is evident between the 124 level PBP and the 48 level Breit-Pauli calculation of \\citet{2011PhRvA..84a3413C}, for all photon energies up to 60eV. Note that the cross section in Figure \\ref{fig:valence} is plotted on a log scale. Evidently the larger basis expansion of the present PBP evaluation, which includes the 4s, 4p and 4d orbitals, has minimal effect on the resulting photoionization cross section. Both of these Breit-Pauli evaluations, however, underestimate the cross section above roughly 45eV and lie considerably lower than the experimental measurements from ALS. The larger DARC3 evaluation, incorporating 557 fine-structure levels, gives much better agreement with experiment at photon energies above the Cooper minimum. This is partly due to the more substantial calculation, and also a more accurate description of the wavefunctions included. Both techniques in fact are known to reproduce similar results as shown in a study by \\citet{2005JPhB...38.1667B}, showing that the average difference in effective collision strengths for Fe$^{14+}$ to be 6$\\%$ between all transitions considered. The additional levels included, and the Rydberg resonances converging onto their thresholds, have the effect of raising the cross section above 45eV.\n\nIn order to further emphasize the excellent agreement between the DARC3 and the experimental measurements, we zoom in on the photon energy region just above threshold, from 27.8-29.2eV, in Figure \\ref{fig:valence_zoom}. It is clearly evident that the disparities found between theory and experiment in this very narrow energy range are negligible and excellent conformity is achieved. This high level of agreement supports the accuracy of the DARC3 evaluation and we believe that these valence shell photoionization cross sections for the ground state of Ar {\\sc ii} accurately reproduce the experimental spectrum. \n\nIn Figure \\ref{fig:excited} we present the total photoionization cross section for the process defined in Equation \\ref{eq:excited}, photoionization from the lowest excited initial 3s3p$^6\\; ^2$S$_{1/2}$ bound state of Ar {\\sc ii} to all possible allowed final states of Ar {\\sc iii}. These evaluations were carried out using the DARC3 model and present for the first time, cross sections for photoionization from an excited Ar {\\sc ii} state. There are no other theoretical or experimental data with which we can compare in this figure. The cross section is presented as a function of the photon energy in eV which ranges from just above the ionization threshold to beyond the opening of the L-shell thresholds. The photoionization cross section tends towards zero with increasing energy, and it is only due to the inclusion of the additional 10 hole states in the DARC3 model do we witness contributions to the cross section at photon energies between 200-250 eV. \n\n\\subsection{L-shell photoionization}\nCalculations and experiment have been carried out at the L-shell energy region between 250-280eV by \\citet{2012PhRvA..85d3408B} at the SOLEIL facility in France as described in Section \\ref{sec:introduction}. All the results herein have been convoluted with a 140meV Gaussian profile at FWHM to match the spectral resolution of experiment. Similar to the valence shell comparisons, the initial ground state cross section is formed from a statistically weighted average of the contributions from the odd $J=3/2$ and $J=1/2$ partial waves. Due to time of flight between the ion source and interacting region, excited levels can populate the main ion beam. This leads to a possible inclusion of the initial 3s3p$^6$ $^2$S$_{1/2}$ bound state which may also contribute to the total cross section. In Figure \\ref{fig:excited} we have already shown the immediate result of the lowest excited initial bound state transitions arising from the configuration 3s3p$^6$.\n\n\n\n\n\n\\begin{figure}\n\\includegraphics[scale=0.30, angle=-90]{figure6.eps}\n\\caption{The total convoluted FWHM at 140meV photoionization cross section between 254-256eV taken from Figure \\ref{fig:bigL} highlighting the intense resonant peaks. The spectra is broken into the contributions from each dipole allowed symmetry from both initial (middle third) and metastable (bottom third) initial states according to their statistical weighting. These even $J=5/2, 3/2, 1/2$ partial symmetries are the solid turquoise, red and purple lines. The total (top third) summed contribution is presented by the solid black curve and the two dominate resonances are marked by the dashed line. \\label{fig:res_peaks}}\n\\end{figure}\n\nIn order to compare with experiment, we have presented our results against various ionization channels from \\citet{2012PhRvA..85d3408B} in Figure \\ref{fig:s_and_d}. The timescale for Auger decay is much shorter than the time of flight required by the Argon ions after interaction with a photon, and therefore, the single ionization channel from experiment depicts the characteristics of photoionizing a valence electron. We can directly compare with this process in Figure \\ref{fig:s_and_d} by omitting the contribution from the additional 10 target states annotated by asterisks. Both above and during these thresholds we expect a rise in the photoionization cross section as more channels are opened and become accessible. The total result obtained by DARC3 can be compared directly to the combination of both single and double ionization modes of experiment. We have neglected the error bars for both modes in order to visualise the results more clearly.\n\nWe now present in Figure \\ref{fig:bigL} the photoionization cross section, on a linear scale, as a function of incident photon energy in eV across the L-shell threshold range from 250-270eV. Comparisons are made between the present DARC3 cross section and the measurements performed by \\citet{2012PhRvA..85d3408B}. Clearly excellent agreement is evident between theory and experiment across the range considered, as the features and energy positions of the resonance profiles exhibit good agreement. We note that as we have employed orbitals optimized on the valence state photoionization, an energy shift of 7.5eV was required to match the experimental spectra to our current results. The theory clearly predicts this process to a high standard of accuracy and allows us to benchmark the quality of results obtained from experiment.\n\nIn an attempt to investigate the features further, we have broken down the spectrum in Figure \\ref{fig:res_peaks} from the total into each of the allowed, final, even $J$ states $J=1/2$, $J=3/2$ and $J=5/2$. Clearly visible is the intense spike at $\\approx 254.9$ eV which is dominated by transitions of the form, 2p $\\rightarrow$ nd, ns which are engulfed by the convolution. The second strong peak at $\\approx 255.65$ eV is visible mostly through the metastable initial state transition from another strong 2p $\\rightarrow$ nd, J = 3/2 resonance. In reference to Figure \\ref{fig:excited}, the cross section has already reached close towards zero in the photon energy range of interest and therefore any contribution to the total cross section from these initial excited bound states would result in a reduction to the intensity of each resonant state. \n\nThis method of deconstructing the cross section is also important to identify which initial state has been photoionized during the experiment. It is clear however that the strongest profiles are not well isolated and therefore eliminates the possibility to further conduct any analysis on the weighted contributions. We therefore retain the statistical averaging of the ground state as our best result.\n\nAll resonances in this paper were identified using the technique detailed by \\citet{1996JPhB...29.4529Q} and \\citet{1998CoPhC.114..225Q}, which involves an analytic approach complementary to the \\textbf{R}-matrix method. By exploiting multichannel quantum defect theory \\citep{1983RPPh...46..167S}, each resonant state part of a Rydberg series has constant defect, $\\mu$ for each effective $n$ quantum number defined by,\n\\begin{equation}\\label{eq:resonance}\nE_r = E_{n \\rightarrow \\infty} - \\Big[\\frac{Z-N}{n-\\mu}\\Big]^2\n\\end{equation}\nwhere $E_r$ is the resonance energy converging to the target thresholds, $E_{n \\rightarrow \\infty}$. The overlapping nature of the resonant states makes it difficult to accurately evaluate resonance widths and assign each transition taking place. It is possible to deduce that the hole resonant states arise from 2p $\\rightarrow nd, (n+1)s$ transitions for $n\\ge3$, and correspond to the strongest peaks evident in Figure \\ref{fig:res_peaks}.\n\n\n\n\n\n\n\n\n\n\n\n\n\n4.2 L-shell photoionization\n\\subsection{L-shell photoionization}\nCalculations and experiment have been carried out at the L-shell energy region between 250-280eV by \\citet{2012PhRvA..85d3408B} at the SOLEIL facility in France as described in Section \\ref{sec:introduction}. All the results herein have been convoluted with a 140meV Gaussian profile at FWHM to match the spectral resolution of experiment. Similar to the valence shell comparisons, the initial ground state cross section is formed from a statistically weighted average of the contributions from the odd $J=3/2$ and $J=1/2$ partial waves. Due to time of flight between the ion source and interacting region, excited levels can populate the main ion beam. This leads to a possible inclusion of the initial 3s3p$^6$ $^2$S$_{1/2}$ bound state which may also contribute to the total cross section. In Figure \\ref{fig:excited} we have already shown the immediate result of the lowest excited initial bound state transitions arising from the configuration 3s3p$^6$.\n\n\n\n\n\n\\begin{figure}\n\\includegraphics[scale=0.30, angle=-90]{figure6.eps}\n\\caption{The total convoluted FWHM at 140meV photoionization cross section between 254-256eV taken from Figure \\ref{fig:bigL} highlighting the intense resonant peaks. The spectra is broken into the contributions from each dipole allowed symmetry from both initial (middle third) and metastable (bottom third) initial states according to their statistical weighting. These even $J=5/2, 3/2, 1/2$ partial symmetries are the solid turquoise, red and purple lines. The total (top third) summed contribution is presented by the solid black curve and the two dominate resonances are marked by the dashed line. \\label{fig:res_peaks}}\n\\end{figure}\n\nIn order to compare with experiment, we have presented our results against various ionization channels from \\citet{2012PhRvA..85d3408B} in Figure \\ref{fig:s_and_d}. The timescale for Auger decay is much shorter than the time of flight required by the Argon ions after interaction with a photon, and therefore, the single ionization channel from experiment depicts the characteristics of photoionizing a valence electron. We can directly compare with this process in Figure \\ref{fig:s_and_d} by omitting the contribution from the additional 10 target states annotated by asterisks. Both above and during these thresholds we expect a rise in the photoionization cross section as more channels are opened and become accessible. The total result obtained by DARC3 can be compared directly to the combination of both single and double ionization modes of experiment. We have neglected the error bars for both modes in order to visualise the results more clearly.\n\nWe now present in Figure \\ref{fig:bigL} the photoionization cross section, on a linear scale, as a function of incident photon energy in eV across the L-shell threshold range from 250-270eV. Comparisons are made between the present DARC3 cross section and the measurements performed by \\citet{2012PhRvA..85d3408B}. Clearly excellent agreement is evident between theory and experiment across the range considered, as the features and energy positions of the resonance profiles exhibit good agreement. We note that as we have employed orbitals optimized on the valence state photoionization, an energy shift of 7.5eV was required to match the experimental spectra to our current results. The theory clearly predicts this process to a high standard of accuracy and allows us to benchmark the quality of results obtained from experiment.\n\nIn an attempt to investigate the features further, we have broken down the spectrum in Figure \\ref{fig:res_peaks} from the total into each of the allowed, final, even $J$ states $J=1/2$, $J=3/2$ and $J=5/2$. Clearly visible is the intense spike at $\\approx 254.9$ eV which is dominated by transitions of the form, 2p $\\rightarrow$ nd, ns which are engulfed by the convolution. The second strong peak at $\\approx 255.65$ eV is visible mostly through the metastable initial state transition from another strong 2p $\\rightarrow$ nd, J = 3/2 resonance. In reference to Figure \\ref{fig:excited}, the cross section has already reached close towards zero in the photon energy range of interest and therefore any contribution to the total cross section from these initial excited bound states would result in a reduction to the intensity of each resonant state. \n\nThis method of deconstructing the cross section is also important to identify which initial state has been photoionized during the experiment. It is clear however that the strongest profiles are not well isolated and therefore eliminates the possibility to further conduct any analysis on the weighted contributions. We therefore retain the statistical averaging of the ground state as our best result.\n\nAll resonances in this paper were identified using the technique detailed by \\citet{1996JPhB...29.4529Q} and \\citet{1998CoPhC.114..225Q}, which involves an analytic approach complementary to the \\textbf{R}-matrix method. By exploiting multichannel quantum defect theory \\citep{1983RPPh...46..167S}, each resonant state part of a Rydberg series has constant defect, $\\mu$ for each effective $n$ quantum number defined by,\n\\begin{equation}\\label{eq:resonance}\nE_r = E_{n \\rightarrow \\infty} - \\Big[\\frac{Z-N}{n-\\mu}\\Big]^2\n\\end{equation}\nwhere $E_r$ is the resonance energy converging to the target thresholds, $E_{n \\rightarrow \\infty}$. The overlapping nature of the resonant states makes it difficult to accurately evaluate resonance widths and assign each transition taking place. It is possible to deduce that the hole resonant states arise from 2p $\\rightarrow nd, (n+1)s$ transitions for $n\\ge3$, and correspond to the strongest peaks evident in Figure \\ref{fig:res_peaks}.\n\n\n\n\n\n\n\n\n\n\n\n\n\n",
        "label": "fig:s_and_d",
        "Descriptive_question1": "What color represents the experimental values for the single ionization channel in figure_5?",
        "Descriptive_question2": "What symbol is used to represent each 2p^5 3s^2 3p^5 threshold in figure_5?",
        "Reasoning_question1": "How does the contribution from valence shell photoionization of the 3s and 3p compare to the total photoionization cross section in figure_5 across the displayed photon energy range?",
        "Reasoning_question2": "What might explain the difference between the experimental total contribution and the DARC3 model results in figure_5 at certain photon energies?",
        "Descriptive_answer1": "grey circles",
        "Descriptive_answer2": "asterisk",
        "Reasoning_answer1": "The dashed black line representing the valence shell photoionization cross section of 3s and 3p lies below the solid black line representing the total DARC3 model cross section across the photon energy range shown. This indicates that the valence shell contribution alone accounts for only a portion of the total photoionization cross section, with additional contributions arising from other channels or transitions included in the total DARC3 model. Hence, the valence shell photoionization is a significant but not exclusive contributor to the total cross section.",
        "Reasoning_answer2": "The difference between the experimental total contribution (pink circles) and the DARC3 model results (solid black line) at certain photon energies may arise from several factors. Experimental measurements can include contributions from additional ionization processes such as double ionization or higher order effects not fully accounted for in the theoretical model. Also, there might be limitations or approximations in the DARC3 calculation, such as orbital optimization primarily focused on valence states, which required an energy shift for matching. Another reason could be experimental conditions including beam impurities or excited state populations affecting total cross section measurements. These factors could result in discrepancies between the total experimental cross section and the theoretical model."
    },
    {
        "paper_id": "1511.03946.json",
        "image_id": "figure_6",
        "image_path": [
            "/home/yz979/scratch/chengye/scimolmo/test-images-tables/dataset_imagestables/1511.03946/images/figure5.eps"
        ],
        "caption": "Photoionization cross section measured in Mb on a logarithmic scale as a function of the photon energy in eV between 250-270eV. The circles are the experimental results from \\citet{2012PhRvA..85d3408B} with error bars included. The solid black line represents our current, DARC3 model results for the statistically weighted initial ground state, convoluted at 140meV at FWHM. \\label{fig:bigL}",
        "classify": "Chart",
        "section_info": "4 Results\n\\section{Results}\\label{sec:results}\nBefore embarking on the large scale DARC3 calculation we thought it prudent to investigate first the important properties and characteristics found in the photoionization cross section of Ar {\\sc ii} in its ground state. In Figure \\ref{fig:comparison}\nwe present the total photoionization cross section in Mb on a logarithmic scale as a function of photon energy in eV, from the initial ground Ar {\\sc ii} $^2$P$^{\\rm{o}}_{3/2}$ state to all allowed final states. Three calculations are presented in this figure; both the 209 level DARC1 and its contributions from the 3s$^2$3p$^4$ levels indexed as 1-5 in Table \\ref{tab:energy} and the extended 257 level DARC2 calculation. Clearly Figure \\ref{fig:comparison}\nshows the importance of including at least the first five 3s$^2$3p$^4$ levels of Ar {\\sc iii} in this photoionization calculation. The contributions from these levels dominates the total cross section up to a photon energy of approximately 50eV and all three calculations exhibit excellent agreement up to this point. It is essential, therefore, that an accurate description is achieved for the wavefunction representation of those low-lying levels. Above 50eV the additional levels associated with the more complex DARC1 and DARC2 models come into play and the cross section rises as we move to higher photon energies as more channels become accessible. Interestingly the inclusion of the additional 3s$^2$3p$^3$4d and 3s$^2$3p$^3$5s levels in the DARC2 model has little or no effect on the photoionization cross section produced by the DARC1 model up to 60eV, both datasets showing near perfect agreement. Therefore we do not retain these additional 3s$^2$3p$^3$4d and 3s$^2$3p$^3$5s configurations in our largest DARC3 calculation as can be seen from Table \\ref{tab:calculations}.\n\n\n\n\n\\begin{figure*}\n\\includegraphics[scale=0.6, angle=-90]{figure3.eps}\n\\caption{Photoionization cross section measured in Mb on a logarithmic scale as a function of the photon energy between 27.8-29.2 eV just above threshold. The solid black line is the current statistically weighted, initial ground state, DARC3 calculation against the experimental values from \\citet{2011PhRvA..84a3413C} represented by the yellow circles taken from Figure \\ref{fig:valence}. \\label{fig:valence_zoom}}\n\\end{figure*}\n\n\n\n\n\\begin{figure*}\n\\includegraphics[scale=0.58, angle=-90]{figure4.eps}\n\\caption{Total ground state photoionization cross section measured in Mb as a function of the photon energy between 0-280eV. The transition is from the initial state 3s3p$^6$ $^2$S$_{1/2}$ to all allowed final states from the DARC3 model. \\label{fig:excited}}\n\\end{figure*}\n\n\\subsection{Valence shell photoionization}\nThe only available data currently in the literature for valence shell photoionization of Ar {\\sc ii} up to photon energies of 60eV is performed by \\citet{2011PhRvA..84a3413C}. In this paper both theoretical and experimental cross sections are presented. Absolute cross sections are obtained from the merged beam technique at the Advanced Light Source (ALS) with a spectral resolution of 10meV. It was found that the primary ion beam contained a mixture of both $^2$P$_{3/2}^{\\rm o}$ and $^2$P$_{1/2}^{\\rm o}$ initial states. Hence the total cross section was presented as a statistical weighting of the odd parity $J=3/2$ ground and $J=1/2$ metastable states respectively. The accompanying theoretical cross sections presented by \\citet{2011PhRvA..84a3413C} were evaluated using the Breit-Pauli \\textbf{R}-matrix approach. A total of 48 $LSJ\\pi$ fine-structure levels were included in the wavefunction representation with configurations 3s$^2$3p$^4$, 3s3p$^5$, 3p$^6$ and 3s$^2$3p$^2$3d$^2$. Some important correlation effects are thus omitted from this model such as levels associated with the 3s$^2$3p$^3$3d configuration and those arising from the lower $n=4$ complex. \n\n\n\n\n\\begin{figure}\n\\includegraphics[scale=0.29, angle=-90]{figure7.eps}\n\\caption{The photoionization cross section is presented on a linear scale against photon energy in eV above 261.2eV. The solid black line represents our current DARC3 model convoluted at 140meV FWHM and the dashed black line is the contribution to the cross section from valence shell photoionization of the 3s and 3p. The grey circles are experimental values of \\citet{2012PhRvA..85d3408B} for the single ionization channel and pink circles represent the total contribution. Each 2p$^5$3s$^2$3p$^5$ threshold is represented by an asterisk.\\label{fig:s_and_d}}\n\\end{figure}\n\nIn order to compare with this data we present in Figure \\ref{fig:valence} the total photoionization cross section from the initial $^2$P$^{\\rm{o}}$ ground state of Ar {\\sc ii} statistically weighted to the $J=3/2$ and $J=1/2$ states. We present two of our calculations in the figure, the most sophisticated DARC3 model and, in order to perform a direct comparison with the Breit-Pauli theoretical results of \\citet{2011PhRvA..84a3413C}, the PBP 124 level model outlined in Table \\ref{tab:calculations}. To match experimental resolving power, we convolute our total results with a 10meV gaussian profile at full-width half-maximum (FWHM). In addition, to replicate the target thresholds, we have shifted our threshold values recorded in Table \\ref{tab:energy} to the experimental NIST values where possible, during the diagonalization of the Hamiltonian matrix. The remaining levels not contained in NIST are shifted by an average proportion to each corresponding angular and spin momentum state, which has little effect on the background and is meant only for consistency. This ensures that resonance features are properly positioned with respect to the observed thresholds, making a direct comparison with experiment more meaningful.\n\n\n\n\n\\begin{figure*}\n\\includegraphics[scale=0.6, angle=-90]{figure5.eps}\n\\caption{Photoionization cross section measured in Mb on a logarithmic scale as a function of the photon energy in eV between 250-270eV. The circles are the experimental results from \\citet{2012PhRvA..85d3408B} with error bars included. The solid black line represents our current, DARC3 model results for the statistically weighted initial ground state, convoluted at 140meV at FWHM. \\label{fig:bigL}}\n\\end{figure*}\n\nWe can clearly see in Figure \\ref{fig:valence} that the low energy region just above threshold is completely dominated by 3s$^2$3p$^5$ $\\rightarrow$ 3s$^2$3p$^4nl$ transitions occurring at discrete energies prior to the ejection of an electron. This densely populated region of Rydberg resonances up to approximately 30eV is followed by a steep decline in the photoionization cross section forming the expected Cooper minimum around 45-50eV. This minimum is well known to appear in the spectra of noble gases \\citep{1962PhRv..128..681C}. Above this minimum the cross section rises due to excitations from 3p $\\rightarrow$ 3d transitions, before monotonically decreasing towards zero with increasing photon energy. \n\nExcellent agreement is evident between the 124 level PBP and the 48 level Breit-Pauli calculation of \\citet{2011PhRvA..84a3413C}, for all photon energies up to 60eV. Note that the cross section in Figure \\ref{fig:valence} is plotted on a log scale. Evidently the larger basis expansion of the present PBP evaluation, which includes the 4s, 4p and 4d orbitals, has minimal effect on the resulting photoionization cross section. Both of these Breit-Pauli evaluations, however, underestimate the cross section above roughly 45eV and lie considerably lower than the experimental measurements from ALS. The larger DARC3 evaluation, incorporating 557 fine-structure levels, gives much better agreement with experiment at photon energies above the Cooper minimum. This is partly due to the more substantial calculation, and also a more accurate description of the wavefunctions included. Both techniques in fact are known to reproduce similar results as shown in a study by \\citet{2005JPhB...38.1667B}, showing that the average difference in effective collision strengths for Fe$^{14+}$ to be 6$\\%$ between all transitions considered. The additional levels included, and the Rydberg resonances converging onto their thresholds, have the effect of raising the cross section above 45eV.\n\nIn order to further emphasize the excellent agreement between the DARC3 and the experimental measurements, we zoom in on the photon energy region just above threshold, from 27.8-29.2eV, in Figure \\ref{fig:valence_zoom}. It is clearly evident that the disparities found between theory and experiment in this very narrow energy range are negligible and excellent conformity is achieved. This high level of agreement supports the accuracy of the DARC3 evaluation and we believe that these valence shell photoionization cross sections for the ground state of Ar {\\sc ii} accurately reproduce the experimental spectrum. \n\nIn Figure \\ref{fig:excited} we present the total photoionization cross section for the process defined in Equation \\ref{eq:excited}, photoionization from the lowest excited initial 3s3p$^6\\; ^2$S$_{1/2}$ bound state of Ar {\\sc ii} to all possible allowed final states of Ar {\\sc iii}. These evaluations were carried out using the DARC3 model and present for the first time, cross sections for photoionization from an excited Ar {\\sc ii} state. There are no other theoretical or experimental data with which we can compare in this figure. The cross section is presented as a function of the photon energy in eV which ranges from just above the ionization threshold to beyond the opening of the L-shell thresholds. The photoionization cross section tends towards zero with increasing energy, and it is only due to the inclusion of the additional 10 hole states in the DARC3 model do we witness contributions to the cross section at photon energies between 200-250 eV. \n\n\\subsection{L-shell photoionization}\nCalculations and experiment have been carried out at the L-shell energy region between 250-280eV by \\citet{2012PhRvA..85d3408B} at the SOLEIL facility in France as described in Section \\ref{sec:introduction}. All the results herein have been convoluted with a 140meV Gaussian profile at FWHM to match the spectral resolution of experiment. Similar to the valence shell comparisons, the initial ground state cross section is formed from a statistically weighted average of the contributions from the odd $J=3/2$ and $J=1/2$ partial waves. Due to time of flight between the ion source and interacting region, excited levels can populate the main ion beam. This leads to a possible inclusion of the initial 3s3p$^6$ $^2$S$_{1/2}$ bound state which may also contribute to the total cross section. In Figure \\ref{fig:excited} we have already shown the immediate result of the lowest excited initial bound state transitions arising from the configuration 3s3p$^6$.\n\n\n\n\n\n\\begin{figure}\n\\includegraphics[scale=0.30, angle=-90]{figure6.eps}\n\\caption{The total convoluted FWHM at 140meV photoionization cross section between 254-256eV taken from Figure \\ref{fig:bigL} highlighting the intense resonant peaks. The spectra is broken into the contributions from each dipole allowed symmetry from both initial (middle third) and metastable (bottom third) initial states according to their statistical weighting. These even $J=5/2, 3/2, 1/2$ partial symmetries are the solid turquoise, red and purple lines. The total (top third) summed contribution is presented by the solid black curve and the two dominate resonances are marked by the dashed line. \\label{fig:res_peaks}}\n\\end{figure}\n\nIn order to compare with experiment, we have presented our results against various ionization channels from \\citet{2012PhRvA..85d3408B} in Figure \\ref{fig:s_and_d}. The timescale for Auger decay is much shorter than the time of flight required by the Argon ions after interaction with a photon, and therefore, the single ionization channel from experiment depicts the characteristics of photoionizing a valence electron. We can directly compare with this process in Figure \\ref{fig:s_and_d} by omitting the contribution from the additional 10 target states annotated by asterisks. Both above and during these thresholds we expect a rise in the photoionization cross section as more channels are opened and become accessible. The total result obtained by DARC3 can be compared directly to the combination of both single and double ionization modes of experiment. We have neglected the error bars for both modes in order to visualise the results more clearly.\n\nWe now present in Figure \\ref{fig:bigL} the photoionization cross section, on a linear scale, as a function of incident photon energy in eV across the L-shell threshold range from 250-270eV. Comparisons are made between the present DARC3 cross section and the measurements performed by \\citet{2012PhRvA..85d3408B}. Clearly excellent agreement is evident between theory and experiment across the range considered, as the features and energy positions of the resonance profiles exhibit good agreement. We note that as we have employed orbitals optimized on the valence state photoionization, an energy shift of 7.5eV was required to match the experimental spectra to our current results. The theory clearly predicts this process to a high standard of accuracy and allows us to benchmark the quality of results obtained from experiment.\n\nIn an attempt to investigate the features further, we have broken down the spectrum in Figure \\ref{fig:res_peaks} from the total into each of the allowed, final, even $J$ states $J=1/2$, $J=3/2$ and $J=5/2$. Clearly visible is the intense spike at $\\approx 254.9$ eV which is dominated by transitions of the form, 2p $\\rightarrow$ nd, ns which are engulfed by the convolution. The second strong peak at $\\approx 255.65$ eV is visible mostly through the metastable initial state transition from another strong 2p $\\rightarrow$ nd, J = 3/2 resonance. In reference to Figure \\ref{fig:excited}, the cross section has already reached close towards zero in the photon energy range of interest and therefore any contribution to the total cross section from these initial excited bound states would result in a reduction to the intensity of each resonant state. \n\nThis method of deconstructing the cross section is also important to identify which initial state has been photoionized during the experiment. It is clear however that the strongest profiles are not well isolated and therefore eliminates the possibility to further conduct any analysis on the weighted contributions. We therefore retain the statistical averaging of the ground state as our best result.\n\nAll resonances in this paper were identified using the technique detailed by \\citet{1996JPhB...29.4529Q} and \\citet{1998CoPhC.114..225Q}, which involves an analytic approach complementary to the \\textbf{R}-matrix method. By exploiting multichannel quantum defect theory \\citep{1983RPPh...46..167S}, each resonant state part of a Rydberg series has constant defect, $\\mu$ for each effective $n$ quantum number defined by,\n\\begin{equation}\\label{eq:resonance}\nE_r = E_{n \\rightarrow \\infty} - \\Big[\\frac{Z-N}{n-\\mu}\\Big]^2\n\\end{equation}\nwhere $E_r$ is the resonance energy converging to the target thresholds, $E_{n \\rightarrow \\infty}$. The overlapping nature of the resonant states makes it difficult to accurately evaluate resonance widths and assign each transition taking place. It is possible to deduce that the hole resonant states arise from 2p $\\rightarrow nd, (n+1)s$ transitions for $n\\ge3$, and correspond to the strongest peaks evident in Figure \\ref{fig:res_peaks}.\n\n\n\n\n\n\n\n\n\n\n\n\n\n4.2 L-shell photoionization\n\\subsection{L-shell photoionization}\nCalculations and experiment have been carried out at the L-shell energy region between 250-280eV by \\citet{2012PhRvA..85d3408B} at the SOLEIL facility in France as described in Section \\ref{sec:introduction}. All the results herein have been convoluted with a 140meV Gaussian profile at FWHM to match the spectral resolution of experiment. Similar to the valence shell comparisons, the initial ground state cross section is formed from a statistically weighted average of the contributions from the odd $J=3/2$ and $J=1/2$ partial waves. Due to time of flight between the ion source and interacting region, excited levels can populate the main ion beam. This leads to a possible inclusion of the initial 3s3p$^6$ $^2$S$_{1/2}$ bound state which may also contribute to the total cross section. In Figure \\ref{fig:excited} we have already shown the immediate result of the lowest excited initial bound state transitions arising from the configuration 3s3p$^6$.\n\n\n\n\n\n\\begin{figure}\n\\includegraphics[scale=0.30, angle=-90]{figure6.eps}\n\\caption{The total convoluted FWHM at 140meV photoionization cross section between 254-256eV taken from Figure \\ref{fig:bigL} highlighting the intense resonant peaks. The spectra is broken into the contributions from each dipole allowed symmetry from both initial (middle third) and metastable (bottom third) initial states according to their statistical weighting. These even $J=5/2, 3/2, 1/2$ partial symmetries are the solid turquoise, red and purple lines. The total (top third) summed contribution is presented by the solid black curve and the two dominate resonances are marked by the dashed line. \\label{fig:res_peaks}}\n\\end{figure}\n\nIn order to compare with experiment, we have presented our results against various ionization channels from \\citet{2012PhRvA..85d3408B} in Figure \\ref{fig:s_and_d}. The timescale for Auger decay is much shorter than the time of flight required by the Argon ions after interaction with a photon, and therefore, the single ionization channel from experiment depicts the characteristics of photoionizing a valence electron. We can directly compare with this process in Figure \\ref{fig:s_and_d} by omitting the contribution from the additional 10 target states annotated by asterisks. Both above and during these thresholds we expect a rise in the photoionization cross section as more channels are opened and become accessible. The total result obtained by DARC3 can be compared directly to the combination of both single and double ionization modes of experiment. We have neglected the error bars for both modes in order to visualise the results more clearly.\n\nWe now present in Figure \\ref{fig:bigL} the photoionization cross section, on a linear scale, as a function of incident photon energy in eV across the L-shell threshold range from 250-270eV. Comparisons are made between the present DARC3 cross section and the measurements performed by \\citet{2012PhRvA..85d3408B}. Clearly excellent agreement is evident between theory and experiment across the range considered, as the features and energy positions of the resonance profiles exhibit good agreement. We note that as we have employed orbitals optimized on the valence state photoionization, an energy shift of 7.5eV was required to match the experimental spectra to our current results. The theory clearly predicts this process to a high standard of accuracy and allows us to benchmark the quality of results obtained from experiment.\n\nIn an attempt to investigate the features further, we have broken down the spectrum in Figure \\ref{fig:res_peaks} from the total into each of the allowed, final, even $J$ states $J=1/2$, $J=3/2$ and $J=5/2$. Clearly visible is the intense spike at $\\approx 254.9$ eV which is dominated by transitions of the form, 2p $\\rightarrow$ nd, ns which are engulfed by the convolution. The second strong peak at $\\approx 255.65$ eV is visible mostly through the metastable initial state transition from another strong 2p $\\rightarrow$ nd, J = 3/2 resonance. In reference to Figure \\ref{fig:excited}, the cross section has already reached close towards zero in the photon energy range of interest and therefore any contribution to the total cross section from these initial excited bound states would result in a reduction to the intensity of each resonant state. \n\nThis method of deconstructing the cross section is also important to identify which initial state has been photoionized during the experiment. It is clear however that the strongest profiles are not well isolated and therefore eliminates the possibility to further conduct any analysis on the weighted contributions. We therefore retain the statistical averaging of the ground state as our best result.\n\nAll resonances in this paper were identified using the technique detailed by \\citet{1996JPhB...29.4529Q} and \\citet{1998CoPhC.114..225Q}, which involves an analytic approach complementary to the \\textbf{R}-matrix method. By exploiting multichannel quantum defect theory \\citep{1983RPPh...46..167S}, each resonant state part of a Rydberg series has constant defect, $\\mu$ for each effective $n$ quantum number defined by,\n\\begin{equation}\\label{eq:resonance}\nE_r = E_{n \\rightarrow \\infty} - \\Big[\\frac{Z-N}{n-\\mu}\\Big]^2\n\\end{equation}\nwhere $E_r$ is the resonance energy converging to the target thresholds, $E_{n \\rightarrow \\infty}$. The overlapping nature of the resonant states makes it difficult to accurately evaluate resonance widths and assign each transition taking place. It is possible to deduce that the hole resonant states arise from 2p $\\rightarrow nd, (n+1)s$ transitions for $n\\ge3$, and correspond to the strongest peaks evident in Figure \\ref{fig:res_peaks}.\n\n\n\n\n\n\n\n\n\n\n\n\n\n",
        "label": "fig:bigL",
        "Descriptive_question1": "What is the photon energy range shown in figure_6?",
        "Descriptive_question2": "What type of line represents the DARC3 model results in figure_6?",
        "Reasoning_question1": "How does the agreement between the experimental results and the DARC3 model in figure_6 reflect the accuracy of the theoretical predictions for photoionization cross sections?",
        "Reasoning_question2": "What might be the reason for the need to apply a 7.5eV energy shift to match the experimental spectra with the DARC3 model results in figure_6?",
        "Descriptive_answer1": "250-270eV",
        "Descriptive_answer2": "solid black line",
        "Reasoning_answer1": "The solid black line representing the DARC3 model aligns closely with the experimental data points marked by circles over the photon energy range from 250 to 270eV. The resonance features, including their positions and intensities, match well between theory and experiment, indicating the model's high accuracy in capturing the physical processes involved in the photoionization cross section in this energy range.",
        "Reasoning_answer2": "The energy shift of 7.5eV applied to the theoretical DARC3 spectra is likely due to the use of orbitals optimized for valence state photoionization rather than L-shell photoionization in the calculations. This orbital optimization can cause slight discrepancies in calculated energy levels compared to experimental values. Applying the shift aligns the theoretical resonance energies with experimental thresholds, improving comparison accuracy and reflecting limitations in the orbital optimizations within the theoretical model."
    },
    {
        "paper_id": "1511.03946.json",
        "image_id": "figure_7",
        "image_path": [
            "/home/yz979/scratch/chengye/scimolmo/test-images-tables/dataset_imagestables/1511.03946/images/figure6.eps"
        ],
        "caption": "The total convoluted FWHM at 140meV photoionization cross section between 254-256eV taken from Figure \\ref{fig:bigL} highlighting the intense resonant peaks. The spectra is broken into the contributions from each dipole allowed symmetry from both initial (middle third) and metastable (bottom third) initial states according to their statistical weighting. These even $J=5/2, 3/2, 1/2$ partial symmetries are the solid turquoise, red and purple lines. The total (top third) summed contribution is presented by the solid black curve and the two dominate resonances are marked by the dashed line. \\label{fig:res_peaks}",
        "classify": "Chart",
        "section_info": "4 Results\n\\section{Results}\\label{sec:results}\nBefore embarking on the large scale DARC3 calculation we thought it prudent to investigate first the important properties and characteristics found in the photoionization cross section of Ar {\\sc ii} in its ground state. In Figure \\ref{fig:comparison}\nwe present the total photoionization cross section in Mb on a logarithmic scale as a function of photon energy in eV, from the initial ground Ar {\\sc ii} $^2$P$^{\\rm{o}}_{3/2}$ state to all allowed final states. Three calculations are presented in this figure; both the 209 level DARC1 and its contributions from the 3s$^2$3p$^4$ levels indexed as 1-5 in Table \\ref{tab:energy} and the extended 257 level DARC2 calculation. Clearly Figure \\ref{fig:comparison}\nshows the importance of including at least the first five 3s$^2$3p$^4$ levels of Ar {\\sc iii} in this photoionization calculation. The contributions from these levels dominates the total cross section up to a photon energy of approximately 50eV and all three calculations exhibit excellent agreement up to this point. It is essential, therefore, that an accurate description is achieved for the wavefunction representation of those low-lying levels. Above 50eV the additional levels associated with the more complex DARC1 and DARC2 models come into play and the cross section rises as we move to higher photon energies as more channels become accessible. Interestingly the inclusion of the additional 3s$^2$3p$^3$4d and 3s$^2$3p$^3$5s levels in the DARC2 model has little or no effect on the photoionization cross section produced by the DARC1 model up to 60eV, both datasets showing near perfect agreement. Therefore we do not retain these additional 3s$^2$3p$^3$4d and 3s$^2$3p$^3$5s configurations in our largest DARC3 calculation as can be seen from Table \\ref{tab:calculations}.\n\n\n\n\n\\begin{figure*}\n\\includegraphics[scale=0.6, angle=-90]{figure3.eps}\n\\caption{Photoionization cross section measured in Mb on a logarithmic scale as a function of the photon energy between 27.8-29.2 eV just above threshold. The solid black line is the current statistically weighted, initial ground state, DARC3 calculation against the experimental values from \\citet{2011PhRvA..84a3413C} represented by the yellow circles taken from Figure \\ref{fig:valence}. \\label{fig:valence_zoom}}\n\\end{figure*}\n\n\n\n\n\\begin{figure*}\n\\includegraphics[scale=0.58, angle=-90]{figure4.eps}\n\\caption{Total ground state photoionization cross section measured in Mb as a function of the photon energy between 0-280eV. The transition is from the initial state 3s3p$^6$ $^2$S$_{1/2}$ to all allowed final states from the DARC3 model. \\label{fig:excited}}\n\\end{figure*}\n\n\\subsection{Valence shell photoionization}\nThe only available data currently in the literature for valence shell photoionization of Ar {\\sc ii} up to photon energies of 60eV is performed by \\citet{2011PhRvA..84a3413C}. In this paper both theoretical and experimental cross sections are presented. Absolute cross sections are obtained from the merged beam technique at the Advanced Light Source (ALS) with a spectral resolution of 10meV. It was found that the primary ion beam contained a mixture of both $^2$P$_{3/2}^{\\rm o}$ and $^2$P$_{1/2}^{\\rm o}$ initial states. Hence the total cross section was presented as a statistical weighting of the odd parity $J=3/2$ ground and $J=1/2$ metastable states respectively. The accompanying theoretical cross sections presented by \\citet{2011PhRvA..84a3413C} were evaluated using the Breit-Pauli \\textbf{R}-matrix approach. A total of 48 $LSJ\\pi$ fine-structure levels were included in the wavefunction representation with configurations 3s$^2$3p$^4$, 3s3p$^5$, 3p$^6$ and 3s$^2$3p$^2$3d$^2$. Some important correlation effects are thus omitted from this model such as levels associated with the 3s$^2$3p$^3$3d configuration and those arising from the lower $n=4$ complex. \n\n\n\n\n\\begin{figure}\n\\includegraphics[scale=0.29, angle=-90]{figure7.eps}\n\\caption{The photoionization cross section is presented on a linear scale against photon energy in eV above 261.2eV. The solid black line represents our current DARC3 model convoluted at 140meV FWHM and the dashed black line is the contribution to the cross section from valence shell photoionization of the 3s and 3p. The grey circles are experimental values of \\citet{2012PhRvA..85d3408B} for the single ionization channel and pink circles represent the total contribution. Each 2p$^5$3s$^2$3p$^5$ threshold is represented by an asterisk.\\label{fig:s_and_d}}\n\\end{figure}\n\nIn order to compare with this data we present in Figure \\ref{fig:valence} the total photoionization cross section from the initial $^2$P$^{\\rm{o}}$ ground state of Ar {\\sc ii} statistically weighted to the $J=3/2$ and $J=1/2$ states. We present two of our calculations in the figure, the most sophisticated DARC3 model and, in order to perform a direct comparison with the Breit-Pauli theoretical results of \\citet{2011PhRvA..84a3413C}, the PBP 124 level model outlined in Table \\ref{tab:calculations}. To match experimental resolving power, we convolute our total results with a 10meV gaussian profile at full-width half-maximum (FWHM). In addition, to replicate the target thresholds, we have shifted our threshold values recorded in Table \\ref{tab:energy} to the experimental NIST values where possible, during the diagonalization of the Hamiltonian matrix. The remaining levels not contained in NIST are shifted by an average proportion to each corresponding angular and spin momentum state, which has little effect on the background and is meant only for consistency. This ensures that resonance features are properly positioned with respect to the observed thresholds, making a direct comparison with experiment more meaningful.\n\n\n\n\n\\begin{figure*}\n\\includegraphics[scale=0.6, angle=-90]{figure5.eps}\n\\caption{Photoionization cross section measured in Mb on a logarithmic scale as a function of the photon energy in eV between 250-270eV. The circles are the experimental results from \\citet{2012PhRvA..85d3408B} with error bars included. The solid black line represents our current, DARC3 model results for the statistically weighted initial ground state, convoluted at 140meV at FWHM. \\label{fig:bigL}}\n\\end{figure*}\n\nWe can clearly see in Figure \\ref{fig:valence} that the low energy region just above threshold is completely dominated by 3s$^2$3p$^5$ $\\rightarrow$ 3s$^2$3p$^4nl$ transitions occurring at discrete energies prior to the ejection of an electron. This densely populated region of Rydberg resonances up to approximately 30eV is followed by a steep decline in the photoionization cross section forming the expected Cooper minimum around 45-50eV. This minimum is well known to appear in the spectra of noble gases \\citep{1962PhRv..128..681C}. Above this minimum the cross section rises due to excitations from 3p $\\rightarrow$ 3d transitions, before monotonically decreasing towards zero with increasing photon energy. \n\nExcellent agreement is evident between the 124 level PBP and the 48 level Breit-Pauli calculation of \\citet{2011PhRvA..84a3413C}, for all photon energies up to 60eV. Note that the cross section in Figure \\ref{fig:valence} is plotted on a log scale. Evidently the larger basis expansion of the present PBP evaluation, which includes the 4s, 4p and 4d orbitals, has minimal effect on the resulting photoionization cross section. Both of these Breit-Pauli evaluations, however, underestimate the cross section above roughly 45eV and lie considerably lower than the experimental measurements from ALS. The larger DARC3 evaluation, incorporating 557 fine-structure levels, gives much better agreement with experiment at photon energies above the Cooper minimum. This is partly due to the more substantial calculation, and also a more accurate description of the wavefunctions included. Both techniques in fact are known to reproduce similar results as shown in a study by \\citet{2005JPhB...38.1667B}, showing that the average difference in effective collision strengths for Fe$^{14+}$ to be 6$\\%$ between all transitions considered. The additional levels included, and the Rydberg resonances converging onto their thresholds, have the effect of raising the cross section above 45eV.\n\nIn order to further emphasize the excellent agreement between the DARC3 and the experimental measurements, we zoom in on the photon energy region just above threshold, from 27.8-29.2eV, in Figure \\ref{fig:valence_zoom}. It is clearly evident that the disparities found between theory and experiment in this very narrow energy range are negligible and excellent conformity is achieved. This high level of agreement supports the accuracy of the DARC3 evaluation and we believe that these valence shell photoionization cross sections for the ground state of Ar {\\sc ii} accurately reproduce the experimental spectrum. \n\nIn Figure \\ref{fig:excited} we present the total photoionization cross section for the process defined in Equation \\ref{eq:excited}, photoionization from the lowest excited initial 3s3p$^6\\; ^2$S$_{1/2}$ bound state of Ar {\\sc ii} to all possible allowed final states of Ar {\\sc iii}. These evaluations were carried out using the DARC3 model and present for the first time, cross sections for photoionization from an excited Ar {\\sc ii} state. There are no other theoretical or experimental data with which we can compare in this figure. The cross section is presented as a function of the photon energy in eV which ranges from just above the ionization threshold to beyond the opening of the L-shell thresholds. The photoionization cross section tends towards zero with increasing energy, and it is only due to the inclusion of the additional 10 hole states in the DARC3 model do we witness contributions to the cross section at photon energies between 200-250 eV. \n\n\\subsection{L-shell photoionization}\nCalculations and experiment have been carried out at the L-shell energy region between 250-280eV by \\citet{2012PhRvA..85d3408B} at the SOLEIL facility in France as described in Section \\ref{sec:introduction}. All the results herein have been convoluted with a 140meV Gaussian profile at FWHM to match the spectral resolution of experiment. Similar to the valence shell comparisons, the initial ground state cross section is formed from a statistically weighted average of the contributions from the odd $J=3/2$ and $J=1/2$ partial waves. Due to time of flight between the ion source and interacting region, excited levels can populate the main ion beam. This leads to a possible inclusion of the initial 3s3p$^6$ $^2$S$_{1/2}$ bound state which may also contribute to the total cross section. In Figure \\ref{fig:excited} we have already shown the immediate result of the lowest excited initial bound state transitions arising from the configuration 3s3p$^6$.\n\n\n\n\n\n\\begin{figure}\n\\includegraphics[scale=0.30, angle=-90]{figure6.eps}\n\\caption{The total convoluted FWHM at 140meV photoionization cross section between 254-256eV taken from Figure \\ref{fig:bigL} highlighting the intense resonant peaks. The spectra is broken into the contributions from each dipole allowed symmetry from both initial (middle third) and metastable (bottom third) initial states according to their statistical weighting. These even $J=5/2, 3/2, 1/2$ partial symmetries are the solid turquoise, red and purple lines. The total (top third) summed contribution is presented by the solid black curve and the two dominate resonances are marked by the dashed line. \\label{fig:res_peaks}}\n\\end{figure}\n\nIn order to compare with experiment, we have presented our results against various ionization channels from \\citet{2012PhRvA..85d3408B} in Figure \\ref{fig:s_and_d}. The timescale for Auger decay is much shorter than the time of flight required by the Argon ions after interaction with a photon, and therefore, the single ionization channel from experiment depicts the characteristics of photoionizing a valence electron. We can directly compare with this process in Figure \\ref{fig:s_and_d} by omitting the contribution from the additional 10 target states annotated by asterisks. Both above and during these thresholds we expect a rise in the photoionization cross section as more channels are opened and become accessible. The total result obtained by DARC3 can be compared directly to the combination of both single and double ionization modes of experiment. We have neglected the error bars for both modes in order to visualise the results more clearly.\n\nWe now present in Figure \\ref{fig:bigL} the photoionization cross section, on a linear scale, as a function of incident photon energy in eV across the L-shell threshold range from 250-270eV. Comparisons are made between the present DARC3 cross section and the measurements performed by \\citet{2012PhRvA..85d3408B}. Clearly excellent agreement is evident between theory and experiment across the range considered, as the features and energy positions of the resonance profiles exhibit good agreement. We note that as we have employed orbitals optimized on the valence state photoionization, an energy shift of 7.5eV was required to match the experimental spectra to our current results. The theory clearly predicts this process to a high standard of accuracy and allows us to benchmark the quality of results obtained from experiment.\n\nIn an attempt to investigate the features further, we have broken down the spectrum in Figure \\ref{fig:res_peaks} from the total into each of the allowed, final, even $J$ states $J=1/2$, $J=3/2$ and $J=5/2$. Clearly visible is the intense spike at $\\approx 254.9$ eV which is dominated by transitions of the form, 2p $\\rightarrow$ nd, ns which are engulfed by the convolution. The second strong peak at $\\approx 255.65$ eV is visible mostly through the metastable initial state transition from another strong 2p $\\rightarrow$ nd, J = 3/2 resonance. In reference to Figure \\ref{fig:excited}, the cross section has already reached close towards zero in the photon energy range of interest and therefore any contribution to the total cross section from these initial excited bound states would result in a reduction to the intensity of each resonant state. \n\nThis method of deconstructing the cross section is also important to identify which initial state has been photoionized during the experiment. It is clear however that the strongest profiles are not well isolated and therefore eliminates the possibility to further conduct any analysis on the weighted contributions. We therefore retain the statistical averaging of the ground state as our best result.\n\nAll resonances in this paper were identified using the technique detailed by \\citet{1996JPhB...29.4529Q} and \\citet{1998CoPhC.114..225Q}, which involves an analytic approach complementary to the \\textbf{R}-matrix method. By exploiting multichannel quantum defect theory \\citep{1983RPPh...46..167S}, each resonant state part of a Rydberg series has constant defect, $\\mu$ for each effective $n$ quantum number defined by,\n\\begin{equation}\\label{eq:resonance}\nE_r = E_{n \\rightarrow \\infty} - \\Big[\\frac{Z-N}{n-\\mu}\\Big]^2\n\\end{equation}\nwhere $E_r$ is the resonance energy converging to the target thresholds, $E_{n \\rightarrow \\infty}$. The overlapping nature of the resonant states makes it difficult to accurately evaluate resonance widths and assign each transition taking place. It is possible to deduce that the hole resonant states arise from 2p $\\rightarrow nd, (n+1)s$ transitions for $n\\ge3$, and correspond to the strongest peaks evident in Figure \\ref{fig:res_peaks}.\n\n\n\n\n\n\n\n\n\n\n\n\n\n4.2 L-shell photoionization\n\\subsection{L-shell photoionization}\nCalculations and experiment have been carried out at the L-shell energy region between 250-280eV by \\citet{2012PhRvA..85d3408B} at the SOLEIL facility in France as described in Section \\ref{sec:introduction}. All the results herein have been convoluted with a 140meV Gaussian profile at FWHM to match the spectral resolution of experiment. Similar to the valence shell comparisons, the initial ground state cross section is formed from a statistically weighted average of the contributions from the odd $J=3/2$ and $J=1/2$ partial waves. Due to time of flight between the ion source and interacting region, excited levels can populate the main ion beam. This leads to a possible inclusion of the initial 3s3p$^6$ $^2$S$_{1/2}$ bound state which may also contribute to the total cross section. In Figure \\ref{fig:excited} we have already shown the immediate result of the lowest excited initial bound state transitions arising from the configuration 3s3p$^6$.\n\n\n\n\n\n\\begin{figure}\n\\includegraphics[scale=0.30, angle=-90]{figure6.eps}\n\\caption{The total convoluted FWHM at 140meV photoionization cross section between 254-256eV taken from Figure \\ref{fig:bigL} highlighting the intense resonant peaks. The spectra is broken into the contributions from each dipole allowed symmetry from both initial (middle third) and metastable (bottom third) initial states according to their statistical weighting. These even $J=5/2, 3/2, 1/2$ partial symmetries are the solid turquoise, red and purple lines. The total (top third) summed contribution is presented by the solid black curve and the two dominate resonances are marked by the dashed line. \\label{fig:res_peaks}}\n\\end{figure}\n\nIn order to compare with experiment, we have presented our results against various ionization channels from \\citet{2012PhRvA..85d3408B} in Figure \\ref{fig:s_and_d}. The timescale for Auger decay is much shorter than the time of flight required by the Argon ions after interaction with a photon, and therefore, the single ionization channel from experiment depicts the characteristics of photoionizing a valence electron. We can directly compare with this process in Figure \\ref{fig:s_and_d} by omitting the contribution from the additional 10 target states annotated by asterisks. Both above and during these thresholds we expect a rise in the photoionization cross section as more channels are opened and become accessible. The total result obtained by DARC3 can be compared directly to the combination of both single and double ionization modes of experiment. We have neglected the error bars for both modes in order to visualise the results more clearly.\n\nWe now present in Figure \\ref{fig:bigL} the photoionization cross section, on a linear scale, as a function of incident photon energy in eV across the L-shell threshold range from 250-270eV. Comparisons are made between the present DARC3 cross section and the measurements performed by \\citet{2012PhRvA..85d3408B}. Clearly excellent agreement is evident between theory and experiment across the range considered, as the features and energy positions of the resonance profiles exhibit good agreement. We note that as we have employed orbitals optimized on the valence state photoionization, an energy shift of 7.5eV was required to match the experimental spectra to our current results. The theory clearly predicts this process to a high standard of accuracy and allows us to benchmark the quality of results obtained from experiment.\n\nIn an attempt to investigate the features further, we have broken down the spectrum in Figure \\ref{fig:res_peaks} from the total into each of the allowed, final, even $J$ states $J=1/2$, $J=3/2$ and $J=5/2$. Clearly visible is the intense spike at $\\approx 254.9$ eV which is dominated by transitions of the form, 2p $\\rightarrow$ nd, ns which are engulfed by the convolution. The second strong peak at $\\approx 255.65$ eV is visible mostly through the metastable initial state transition from another strong 2p $\\rightarrow$ nd, J = 3/2 resonance. In reference to Figure \\ref{fig:excited}, the cross section has already reached close towards zero in the photon energy range of interest and therefore any contribution to the total cross section from these initial excited bound states would result in a reduction to the intensity of each resonant state. \n\nThis method of deconstructing the cross section is also important to identify which initial state has been photoionized during the experiment. It is clear however that the strongest profiles are not well isolated and therefore eliminates the possibility to further conduct any analysis on the weighted contributions. We therefore retain the statistical averaging of the ground state as our best result.\n\nAll resonances in this paper were identified using the technique detailed by \\citet{1996JPhB...29.4529Q} and \\citet{1998CoPhC.114..225Q}, which involves an analytic approach complementary to the \\textbf{R}-matrix method. By exploiting multichannel quantum defect theory \\citep{1983RPPh...46..167S}, each resonant state part of a Rydberg series has constant defect, $\\mu$ for each effective $n$ quantum number defined by,\n\\begin{equation}\\label{eq:resonance}\nE_r = E_{n \\rightarrow \\infty} - \\Big[\\frac{Z-N}{n-\\mu}\\Big]^2\n\\end{equation}\nwhere $E_r$ is the resonance energy converging to the target thresholds, $E_{n \\rightarrow \\infty}$. The overlapping nature of the resonant states makes it difficult to accurately evaluate resonance widths and assign each transition taking place. It is possible to deduce that the hole resonant states arise from 2p $\\rightarrow nd, (n+1)s$ transitions for $n\\ge3$, and correspond to the strongest peaks evident in Figure \\ref{fig:res_peaks}.\n\n\n\n\n\n\n\n\n\n\n\n\n\n",
        "label": "fig:res_peaks",
        "Descriptive_question1": "What color represents the total summed contribution in figure_7?",
        "Descriptive_question2": "What type of line marks the two dominant resonances in figure_7?",
        "Reasoning_question1": "Why might the intense spike at approximately 254.9 eV in figure_7 be dominated by 2p to nd, ns transitions?",
        "Reasoning_question2": "How does the contribution from metastable initial states compare to the initial ground states in influencing the resonance peak at approximately 255.65 eV in figure_7?",
        "Descriptive_answer1": "black",
        "Descriptive_answer2": "dashed",
        "Reasoning_answer1": "The intense spike at approximately 254.9 eV is dominated by transitions of the form 2p to nd, ns because these transitions correspond to hole resonant states involving excitations from the 2p orbital to higher energy nd and ns orbitals. These transitions produce strong resonances that manifest as intense peaks in the photoionization cross section, especially in the energy range around 254.9 eV, and this is further confirmed by the identification of these features using multichannel quantum defect theory and their prominent appearance in the convoluted spectra.",
        "Reasoning_answer2": "The resonance peak at approximately 255.65 eV is mostly visible through the metastable initial state transition involving a strong 2p to nd, J = 3/2 resonance. This suggests that the metastable initial states contribute significantly to this resonance peak, likely enhancing its intensity relative to the contribution from the initial ground state. Since the cross section from the excited initial states is close to zero in this energy range, their influence mainly appears through metastable states, indicating that the metastable contribution plays a dominant role in the observed resonance at 255.65 eV compared to the initial ground states."
    },
    {
        "paper_id": "1812.04103.json",
        "image_id": "figure_5",
        "image_path": [
            "/home/yz979/scratch/chengye/scimolmo/test-images-tables/dataset_imagestables/1812.04103/images/figure5.png"
        ],
        "caption": "Comparison of training processes and validation results between our proposed model and the\n\t\tbaseline model when training on the first 9 subjects and using the $10^{th}$ subject for validation.",
        "classify": "Chart",
        "section_info": "3 Results and Discussion\n\\section{Results and Discussion}\n\n\nWe perform experiments on the 3D multimodality isointense infant brain MR image segmentation task to evaluate our non-local U-Nets. The task is to perform automatic segmentation of MR images into cerebrospinal fluid~(CSF), gray matter~(GM) and white matter~(WM) regions. We first introduce the baseline model and the evaluation methods used in our experiments. Then the training and inference processes are described. We provide comparison results in terms of both effectiveness and efficiency, and conduct ablation studies to demonstrate that how each global aggregation block in our non-local U-Nets improves the performance. In addition, we explore the trade-off between the inference speed and accuracy based on different overlapping step sizes, and analyze the impact of patch size. The experimental code and dataset information have been made publicly available~\\footnote{\\url{https://github.com/divelab/Non-local-U-Nets}}.\n\n\\subsection{Experimental Setup}\n\n\nWe use CC-3D-FCN~\\cite{nie20183} as our baseline.\nCC-3D-FCN is a 3D fully convolutional network~(3D-FCN) with\nconvolution and concatenate~(CC) skip connections, which is designed\nfor 3D multimodality isointense infant brain image segmentation. It\nhas been shown to outperform traditional machine learning methods,\nsuch as FMRIB's automated segmentation\ntool~(FAST)~\\cite{zhang2001segmentation}, majority\nvoting~(MV), random forest~(RF)~\\cite{criminisi2013decision} and random forest with\nauto-context model~(LINKS)~\\cite{wang2015links}. Moreover, studies\nin~\\cite{nie20183} has showed the superiority of CC-3D-FCN to\nprevious deep learning models, like 2D, 3D\nCNNs~\\cite{zhang2015deep}, DeepMedic~\\cite{kamnitsas2017efficient},\nand the original 3D U-Net~\\cite{cciccek20163d}. Therefore, it is\nappropriate to use CC-3D-FCN as the baseline of our experiments.\nNote that our dataset is different from that in~\\cite{nie20183}.\n\n\nIn our experiments, we employ the Dice ratio~(DR) and propose the 3D modified\nHausdorff distance~(3D-MHD) as the evaluation metrics. These two\nmethods evaluate the accuracy only for binary segmentation tasks, so\nit is required to transform the 4-class segmentation map predicted by our model into\n4 binary segmentation maps for evaluation. That is, a 3D binary\nsegmentation map should be constructed for each class, where 1 denotes the voxel\nin the position belongs to the class and 0 means the opposite. In\nour experiments, we derive binary segmentation maps directly from\n4-class segmentation maps. The evaluation is performed on binary\nsegmentation maps for CSF, GM and WM.\n\n\nSpecifically, let $P$ and $L$ represent the predicted binary segmentation map for one class\nand the corresponding ground truth label, respectively. The DR is given by\n$DR=2|P \\cap L|/(|P|+|L|)$,\n\n\n\nwhere $|\\cdot|$ denotes the number of 1's in a segmentation map and $|P \\cap\nL|$ means the number of 1's shared by $P$ and $L$. Apparently, DR is a value\nin $[0,1]$ and a larger DR indicates a more accurate segmentation.\n\n\\begin{table*}[!ht]\n\t\\centering\n\t\\caption{Comparison of segmentation performance between our proposed model\n\t\tand the baseline model in terms of DR. The leave-one-subject-out\n\t\tcross-validation is used. Larger values indicate better performance.}\n\t\\label{table:results_baseline_dr}\n\t\\begin{tabular}{ l | c | c | c | c }\n\t\t\\toprule\n\t\t\\textbf{Model} & \\textbf{CSF} & \\textbf{GM} & \\textbf{WM} & \\textbf{Average} \\\\\n\t\t\\midrule\n\t\tBaseline\n\t\t& 0.9250$\\pm$0.0118\n\t\t& 0.9084$\\pm$0.0056\n\t\t& 0.8926$\\pm$0.0119\n\t\t& 0.9087$\\pm$0.0066 \\\\\n\t\tNon-local U-Net\n\t\t& \\textbf{0.9530$\\pm$0.0074}\n\t\t& \\textbf{0.9245$\\pm$0.0049}\n\t\t& \\textbf{0.9102$\\pm$0.0101}\n\t\t& \\textbf{0.9292$\\pm$0.0050} \\\\\n\t\t\\bottomrule\n\t\\end{tabular}\n\\end{table*}\n\n\\begin{table*}[!ht]\n\t\\centering\n\t\\caption{Comparison of segmentation performance between our proposed model\n\t\tand the baseline model in terms of 3D-MHD. The leave-one-subject-out\n\t\tcross-validation is used. Smaller values indicate better performance. Note that 3D-MHD gives different results from MHD.}\n\t\\label{table:results_baseline_mhd}\n\t\\begin{tabular}{ l | c | c | c | c }\n\t\t\\toprule\n\t\t\\textbf{Model} & \\textbf{CSF} & \\textbf{GM} & \\textbf{WM} & \\textbf{Average} \\\\\n\t\t\\midrule\n\t\tBaseline\n\t\t& 0.3417$\\pm$0.0245\n\t\t& 0.6537$\\pm$0.0483\n\t\t& 0.4817$\\pm$0.0454\n\t\t& 0.4924$\\pm$0.0345 \\\\\n\t\tNon-local U-Net\n\t\t& \\textbf{0.2554$\\pm$0.0207}\n\t\t& \\textbf{0.5950$\\pm$0.0428}\n\t\t& \\textbf{0.4454$\\pm$0.0040}\n\t\t& \\textbf{0.4319$\\pm$0.0313} \\\\\n\t\t\\bottomrule\n\t\\end{tabular}\n\\end{table*}\n\n\\begin{table*}[!t]\n\t\\centering\n\t\\caption{Comparison of segmentation performance on the 13 testing subjects of iSeg-2017 between our proposed model\n\t\tand the baseline model in terms of DR. Larger values indicate better performance.}\n\t\\label{table:iseg_results}\n\t\\begin{tabular}{ l | c | c | c }\n\t\t\\toprule\n\t\t\\textbf{Model} & \\textbf{CSF} & \\textbf{GM} & \\textbf{WM} \\\\\n\t\t\\midrule\n\t\tBaseline\n\t\t& 0.9324$\\pm$0.0067\n\t\t& 0.9146$\\pm$0.0074\n\t\t& 0.8974$\\pm$0.0123 \\\\\n\t\tNon-local U-Net\n\t\t& \\textbf{0.9557$\\pm$0.0060}\n\t\t& \\textbf{0.9219$\\pm$0.0089}\n\t\t& \\textbf{0.9044$\\pm$0.0153} \\\\\n\t\t\\bottomrule\n\t\\end{tabular}\n\\end{table*}\n\n\nThe modified Hausdorff distance~(MHD)~\\cite{dubuisson1994modified} is\ndesigned to compute the similarity between two objects. Here, an object is a\nset of points where a point is represented by a vector. Specifically, given\ntwo sets of vectors $A$ and $B$, MHD is computed by\n$MHD=\\max(d(A,B),d(B,A))$,\n\n\n\nwhere the distance between two sets is defined as\n$d(A,B)=1/|A|\\sum_{a \\in A}{d(a,B)}$,\n\n\n\nand the distance between a vector and a set is defined as\n$d(a,B)=\\min_{b \\in B}||a-b||$.\n\n\n\nPrevious studies~\\cite{wang2015links,zhang2015deep,nie20183}\napplied MHD for evaluation by treating a 3D $D \\times H \\times W$\nmap as $H \\times W$ $D$-dimensional vectors. However, there are two\nmore different ways to vectorize the 3D map, depending on the\ndirection of forming vectors, \\emph{i.e.,} $D \\times H$\n$W$-dimensional vectors and $D \\times W$ $H$-dimensional vectors.\nEach vectorization leads to different evaluation results by MHD. To\nmake it a direction-independent evaluation metric as DR, we define\n3D-MHD, which computes the averaged MHD based on the three different\nvectorizations. A smaller 3D-MHD indicates a\nhigher segmentation accuracy.\n\n\\subsection{Training and Inference Strategies}\n\n\nOur proposed non-local U-Nets apply Dropout~\\cite{srivastava2014dropout} with\na rate of 0.5 in each global aggregation block and the output block\nbefore the final $1 \\times 1 \\times 1$ convolution. A weight\ndecay~\\cite{krogh1992simple} with a rate of $2e-6$ is also employed.\nTo train the model, we use randomly cropped small patches. In this\nway, we obtain sufficient training data and the requirement on\nmemory is reduced. No extra data augmentation is needed. The\nexperimental results below suggest that patches\nwith a size of $32^3$ leads to the best\nperformance. The batch size is set to 5. The Adam\noptimizer~\\cite{kingma2014adam} with a learning rate of 0.001 is\nemployed to perform the gradient descent algorithm.\n\n\nIn the inference process, following~\\cite{nie20183}, we extract\npatches with the same size as that used in training. For example, to\ngenerate $32^3$ patches for inference, we slide a\nwindow of size $32^3$ through the original image\nwith a constant overlapping step size. The overlapping step size\nmust be smaller than or equal to the patch size, in order to\nguarantee that extracted patches cover the whole image.\nConsequently, prediction for all these patches provides segmentation\nprobability results for every voxel in the original image. For\nvoxels that receive multiple results due to overlapping, we average\nthem to produce the final prediction. The overlapping step size is\nan important hyper-parameter affecting the inference speed and the\nsegmentation accuracy. A smaller overlapping step size results in\nbetter accuracy, but increases the inference time as more patches\nare generated. We explore the trade-off in our experiments.\n\n\\subsection{Comparison with the Baseline}\\label{sec:baseline}\n\n\nWe compare our non-local U-Nets with the baseline on our dataset.\nFollowing~\\cite{nie20183}, the patch size is set to $32^3$\nand the overlapping step size for inference is set to $8$. To remove the\nbias of different subjects, the leave-one-subject-out cross-validation is\nused for evaluating segmentation performance. That is, for 10 subjects in our\ndataset, we train and evaluate models 10 times correspondingly. Each time one\nof the 10 subjects is left out for validation and the other 9 subjects are\nused for training. The mean and standard deviation of segmentation\nperformance of the 10 runs are reported.\n\n\nTables~\\ref{table:results_baseline_dr}\nand~\\ref{table:results_baseline_mhd} provide the experimental\nresults. In terms of both evaluation metrics, our non-local U-Nets achieve\nsignificant improvements over the baseline model. Due to the small\nvariances of the results, we focus on one of the 10 runs for\nvisualization and ablation studies, where the models are trained on the\nfirst 9 subjects and evaluated on the $10^{th}$ subject. A\nvisualization of the segmentation results in this run is given by\nFig.~\\ref{fig:results_visual}. By comparing the areas in red\ncircles, we can see that our model is capable of catching more\ndetails than the baseline model. We also visualize the training\nprocesses to illustrate the superiority of our model.\nFig.~\\ref{fig:results_training} shows the training and validation\ncurves in this run of our model and the baseline model,\nrespectively. Clearly, our model converges faster to a lower\ntraining loss. In addition, according to the better validation\nresults, our model does not suffer from over-fitting.\n\n\nTo further show the efficiency of our proposed model, we compare the\nnumber of parameters as reported in Table~\\ref{table:num_params}.\nOur model reduces $28\\%$ parameters compared to CC-3D-FCN and\nachieves better performance. A comparison of inference time is also\nprovided in Table~\\ref{table:infer_time}. The settings of our device\nare - GPU: Nvidia Titan Xp 12GB; CPU: Intel Xeon E5-2620v4 2.10GHz;\nOS: Ubuntu 16.04.3 LTS.\n\n\nSince our data has been used as the training data in the iSeg-2017\nchallenge, we also compare the\nresults evaluated on the 13 testing subjects in\nTable~\\ref{table:iseg_results}. According to the leader board, our model\nachieves one of the top performances. Results in terms of DR are reported since\nit is the only shared evaluation metric.\n\n\\begin{table}[!t]\n\t\\centering\n\t\\caption{Comparison of the number of parameters between our proposed model\n\t\tand the baseline model.}\n\t\\label{table:num_params}\n\t\\begin{tabular}{ l | c }\n\t\t\\toprule\n\t\t\\textbf{Model} & Number of Parameters \\\\\n\t\t\\midrule\n\t\tBaseline  & 2,534,276 \\\\\n\t\tNon-local U-Net & \\textbf{1,821,124} \\\\\n\t\t\\bottomrule\n\n\n\n\n\n\n\t\\end{tabular}\n\\end{table}\n\n\\begin{table}[!t]\n\t\\centering\n\t\\caption{Comparison of inference time between our proposed model and the baseline model. The leave-one-subject-out cross-validation is used. The patch size is set to $32^3$ and the overlapping step size for inference is set to $8$.}\n\t\\label{table:infer_time}\n\t\\begin{tabular}{ l | c }\n\t\t\\toprule\n\t\t\\textbf{Model} & Inference Time (min) \\\\\n\t\t\\midrule\n\t\tBaseline  & 3.85$\\pm$0.15 \\\\\n\t\tNon-local U-Net  & \\textbf{3.06$\\pm$0.12} \\\\\n\t\t\\bottomrule\n\t\\end{tabular}\n\\end{table}\n\n\\begin{figure}[!t]\n\t\\centering\n\t\\includegraphics[width=0.95\\columnwidth]{figure4.png}\n\t\\caption{Visualization of the segmentation results on the $10^{th}$ subject\n\t\tby our proposed model and the baseline model. Both models are trained on the\n\t\tfirst 9 subjects. The first column shows the original segmentation maps. The\n\t\tsecond, third and fourth columns show the binary segmentation maps for CSF,\n\t\tGM and WM, respectively.}\n\t\\label{fig:results_visual}\n\\end{figure}\n\n\\begin{figure}[!t]\n\t\\centering\n\t\\includegraphics[width=0.75\\columnwidth]{figure5.png}\n\t\\caption{Comparison of training processes and validation results between our proposed model and the\n\t\tbaseline model when training on the first 9 subjects and using the $10^{th}$ subject for validation.}\n\t\\label{fig:results_training}\n\\end{figure}\n\n\\begin{table}[!t]\n\t\\centering\n\t\\caption{Ablation study by comparing segmentation performance between\n\t\tdifferent models in terms of DR. All models are trained on the first 9\n\t\tsubjects and evaluated on the $10^{th}$ subject. Larger values indicate\n\t\tbetter performance. Details of models are provided in the text.}\n\t\\label{table:results_ablation_dr}\n\t\\resizebox{.95\\columnwidth}{!}{\n\t\\begin{tabular}{ l | c | c | c | c }\n\t\t\\toprule\n\t\t\\textbf{Model} & \\textbf{CSF} & \\textbf{GM} & \\textbf{WM} & \\textbf{Average} \\\\\n\t\t\\midrule\n\n\t\tModel1       & \\textbf{0.9585} & 0.9099 & 0.8625 & 0.9103 \\\\\n\t\tModel2       & 0.9568 & 0.9172 & 0.8728 & 0.9156 \\\\\n\t\tModel3       & 0.9576 & 0.9198 & 0.8749 & 0.9174 \\\\\n\t\tModel4       & 0.9578 & 0.9210 & 0.8769 & 0.9186 \\\\\n\t\tModel5       & 0.9554 & 0.9225 & 0.8804 & 0.9194 \\\\\n\t\tNon-local U-Net    & 0.9572 & \\textbf{0.9278} & \\textbf{0.8867} & \\textbf{0.9239} \\\\\n\t\t\\bottomrule\n\t\\end{tabular}}\n\\end{table}\n\n\\begin{table}[!t]\n\t\\centering\n\t\\caption{Ablation study by comparing segmentation performance between\n\t\tdifferent models in terms of 3D-MHD. All models are trained on the first 9\n\t\tsubjects and evaluated on the $10^{th}$ subject. Smaller values indicate\n\t\tbetter performance. Note that 3D-MHD gives different results from MHD. Details of models are provided in\n\t\tthe text.}\n\t\\label{table:results_ablation_mhd}\n\t\\resizebox{.95\\columnwidth}{!}{\n\t\\begin{tabular}{ l | c | c | c | c }\n\t\t\\toprule\n\t\t\\textbf{Model} & \\textbf{CSF} & \\textbf{GM} & \\textbf{WM} & \\textbf{Average} \\\\\n\t\t\\midrule\n\n\t\tModel1       & \\textbf{0.2363} & 0.6277 & 0.4705 & 0.4448 \\\\\n\t\tModel2       & 0.2404 & 0.6052 & 0.4480 & 0.4312 \\\\\n\t\tModel3       & 0.2392 & 0.5993 & 0.4429 & 0.4271 \\\\\n\t\tModel4       & 0.2397 & 0.5926 & 0.4336 & 0.4220 \\\\\n\t\tModel5       & 0.2444 & 0.5901 & 0.4288 & 0.4211 \\\\\n\t\tNon-local U-Net    & 0.2477 & \\textbf{0.5692} & \\textbf{0.4062} & \\textbf{0.4077} \\\\\n\t\t\\bottomrule\n\t\\end{tabular}}\n\\end{table}\n\n\\subsection{Ablation Studies of Different Modules}\\label{sec:ablation}\n\nWe perform ablation studies to show the effectiveness of each part of our\nnon-local U-Nets. Specifically, we compare the following models:\n\n\\textbf{Model1} is a 3D U-Net without short-range residual connections.\nDown-sampling and up-sampling are implemented by convolutions and\ndeconvolutions with a stride of 2, respectively. The bottom block is simply a\nconvolutional layer. Note that the baseline model, CC-3D-FCN, has showed improved performance over 3D U-Net~\\cite{nie20183}. However, the original 3D U-Net was not designed for this task~\\cite{cciccek20163d}. In our experiments, we appropriately set the hyperparameters of 3D U-Net and achieve better performance.\n\n\\textbf{Model2} is Model1 with short-range residual connections, \\emph{i.e.},\nthe blocks in Fig.~\\ref{fig:residual}(a) and (b) are applied. The bottom\nblock and up-sampling blocks are the same as those in Model1.\n\n\\textbf{Model3} replaces the first up-sampling block in Model2 with the block\nin Fig.~\\ref{fig:residual}(d).\n\n\\textbf{Model4} replaces both up-sampling blocks in Model2 with the block in\nFig.~\\ref{fig:residual}(d).\n\n\\textbf{Model5} replaces the bottom block in Model2 with the block in\nFig.~\\ref{fig:residual}(c).\n\nAll models are trained on the first 9 subjects. We report the segmentation\nperformance on the $10^{th}$ subject in Table~\\ref{table:results_ablation_dr}\nand Table~\\ref{table:results_ablation_mhd}. The results demonstrate how different\nglobal aggregation blocks in our non-local U-Nets improve the performance.\n\n\\subsection{Impact of the Overlapping Step Size}\\label{sec:overlap}\n\nAs discussed above, a small overlapping step size\nusually results in better segmentation, due to the ensemble effect.\nHowever, with a small overlapping step size, the model has to perform\ninference for more validation patches and thus decreases the inference speed. We\nexplore the trade-off in our non-local U-Nets by setting the overlapping step sizes to 4,\n8, 16, 32, respectively. Again, we train our model on the first 9 subjects and\nperform evaluation on the $10^{th}$ subject. The patch size is set to $32^3$.\nAccording to the overlapping step sizes, 11880, 1920,\n387, 80 patches need to be processed during inference, as shown in\nFig.~\\ref{fig:results_overlap_time}. In addition, Fig.~\\ref{fig:results_overlap_dr}\nplots the changes of segmentation performance in terms of DR. Obviously, 8 and\n16 are good choices that achieve accurate and fast segmentation results.\n\n\\begin{figure}[!t]\n\t\\centering\n\t\\includegraphics[width=0.7\\columnwidth]{figure6.png}\n\t\\caption{Changes of segmentation performance in terms of DR, with respect to\n\t\tdifferent overlapping step sizes during inference. The model is trained on the\n\t\tfirst 9 subjects and evaluated on the $10^{th}$ subject.}\n\t\\label{fig:results_overlap_dr}\n\\end{figure}\n\n\n\\begin{figure}[!t]\n\t\\centering\n\t\\includegraphics[width=0.75\\columnwidth]{figure7.png}\n\t\\caption{Changes of the number of validation patches for the $10^{th}$\n\t\tsubject, with respect to different overlapping step sizes during inference.}\n\t\\label{fig:results_overlap_time}\n\\end{figure}\n\n\\subsection{Impact of the Patch Size}\\label{sec:patch}\n\nThe patch size affects the total number of distinct training samples.\nMeanwhile, it controls the range of available global information when\nperforming segmentation for a patch. To choose the appropriate patch\nsize for the non-local U-Nets, we perform a grid search by training on the first 9\nsubjects and evaluating on the $10^{th}$ subject with the overlapping step\nsize of 8. Experiments are conducted with five different patch sizes:\n$16^3$, $24^3$, $32^3$, $40^3$, $48^3$. The results are provided in\nFig.~\\ref{fig:results_patch_dr}, where $32^3$ obtains the best\nperformance and is selected as the default setting of our model.\n\n\n\\begin{figure}[!t]\n\t\\centering\n\t\\includegraphics[width=0.75\\columnwidth]{figure8.png}\n\t\\caption{Changes of segmentation performance in terms of DR, with respect to\n\t\tdifferent patch sizes. The model is trained on the first 9 subjects and\n\t\tevaluated on the $10^{th}$ subject.}\n\t\\label{fig:results_patch_dr}\n\\end{figure}\n\n3.3 Comparison with the Baseline\n\\subsection{Comparison with the Baseline}\\label{sec:baseline}\n\n\nWe compare our non-local U-Nets with the baseline on our dataset.\nFollowing~\\cite{nie20183}, the patch size is set to $32^3$\nand the overlapping step size for inference is set to $8$. To remove the\nbias of different subjects, the leave-one-subject-out cross-validation is\nused for evaluating segmentation performance. That is, for 10 subjects in our\ndataset, we train and evaluate models 10 times correspondingly. Each time one\nof the 10 subjects is left out for validation and the other 9 subjects are\nused for training. The mean and standard deviation of segmentation\nperformance of the 10 runs are reported.\n\n\nTables~\\ref{table:results_baseline_dr}\nand~\\ref{table:results_baseline_mhd} provide the experimental\nresults. In terms of both evaluation metrics, our non-local U-Nets achieve\nsignificant improvements over the baseline model. Due to the small\nvariances of the results, we focus on one of the 10 runs for\nvisualization and ablation studies, where the models are trained on the\nfirst 9 subjects and evaluated on the $10^{th}$ subject. A\nvisualization of the segmentation results in this run is given by\nFig.~\\ref{fig:results_visual}. By comparing the areas in red\ncircles, we can see that our model is capable of catching more\ndetails than the baseline model. We also visualize the training\nprocesses to illustrate the superiority of our model.\nFig.~\\ref{fig:results_training} shows the training and validation\ncurves in this run of our model and the baseline model,\nrespectively. Clearly, our model converges faster to a lower\ntraining loss. In addition, according to the better validation\nresults, our model does not suffer from over-fitting.\n\n\nTo further show the efficiency of our proposed model, we compare the\nnumber of parameters as reported in Table~\\ref{table:num_params}.\nOur model reduces $28\\%$ parameters compared to CC-3D-FCN and\nachieves better performance. A comparison of inference time is also\nprovided in Table~\\ref{table:infer_time}. The settings of our device\nare - GPU: Nvidia Titan Xp 12GB; CPU: Intel Xeon E5-2620v4 2.10GHz;\nOS: Ubuntu 16.04.3 LTS.\n\n\nSince our data has been used as the training data in the iSeg-2017\nchallenge, we also compare the\nresults evaluated on the 13 testing subjects in\nTable~\\ref{table:iseg_results}. According to the leader board, our model\nachieves one of the top performances. Results in terms of DR are reported since\nit is the only shared evaluation metric.\n\n\\begin{table}[!t]\n\t\\centering\n\t\\caption{Comparison of the number of parameters between our proposed model\n\t\tand the baseline model.}\n\t\\label{table:num_params}\n\t\\begin{tabular}{ l | c }\n\t\t\\toprule\n\t\t\\textbf{Model} & Number of Parameters \\\\\n\t\t\\midrule\n\t\tBaseline  & 2,534,276 \\\\\n\t\tNon-local U-Net & \\textbf{1,821,124} \\\\\n\t\t\\bottomrule\n\n\n\n\n\n\n\t\\end{tabular}\n\\end{table}\n\n\\begin{table}[!t]\n\t\\centering\n\t\\caption{Comparison of inference time between our proposed model and the baseline model. The leave-one-subject-out cross-validation is used. The patch size is set to $32^3$ and the overlapping step size for inference is set to $8$.}\n\t\\label{table:infer_time}\n\t\\begin{tabular}{ l | c }\n\t\t\\toprule\n\t\t\\textbf{Model} & Inference Time (min) \\\\\n\t\t\\midrule\n\t\tBaseline  & 3.85$\\pm$0.15 \\\\\n\t\tNon-local U-Net  & \\textbf{3.06$\\pm$0.12} \\\\\n\t\t\\bottomrule\n\t\\end{tabular}\n\\end{table}\n\n\\begin{figure}[!t]\n\t\\centering\n\t\\includegraphics[width=0.95\\columnwidth]{figure4.png}\n\t\\caption{Visualization of the segmentation results on the $10^{th}$ subject\n\t\tby our proposed model and the baseline model. Both models are trained on the\n\t\tfirst 9 subjects. The first column shows the original segmentation maps. The\n\t\tsecond, third and fourth columns show the binary segmentation maps for CSF,\n\t\tGM and WM, respectively.}\n\t\\label{fig:results_visual}\n\\end{figure}\n\n\\begin{figure}[!t]\n\t\\centering\n\t\\includegraphics[width=0.75\\columnwidth]{figure5.png}\n\t\\caption{Comparison of training processes and validation results between our proposed model and the\n\t\tbaseline model when training on the first 9 subjects and using the $10^{th}$ subject for validation.}\n\t\\label{fig:results_training}\n\\end{figure}\n\n\\begin{table}[!t]\n\t\\centering\n\t\\caption{Ablation study by comparing segmentation performance between\n\t\tdifferent models in terms of DR. All models are trained on the first 9\n\t\tsubjects and evaluated on the $10^{th}$ subject. Larger values indicate\n\t\tbetter performance. Details of models are provided in the text.}\n\t\\label{table:results_ablation_dr}\n\t\\resizebox{.95\\columnwidth}{!}{\n\t\\begin{tabular}{ l | c | c | c | c }\n\t\t\\toprule\n\t\t\\textbf{Model} & \\textbf{CSF} & \\textbf{GM} & \\textbf{WM} & \\textbf{Average} \\\\\n\t\t\\midrule\n\n\t\tModel1       & \\textbf{0.9585} & 0.9099 & 0.8625 & 0.9103 \\\\\n\t\tModel2       & 0.9568 & 0.9172 & 0.8728 & 0.9156 \\\\\n\t\tModel3       & 0.9576 & 0.9198 & 0.8749 & 0.9174 \\\\\n\t\tModel4       & 0.9578 & 0.9210 & 0.8769 & 0.9186 \\\\\n\t\tModel5       & 0.9554 & 0.9225 & 0.8804 & 0.9194 \\\\\n\t\tNon-local U-Net    & 0.9572 & \\textbf{0.9278} & \\textbf{0.8867} & \\textbf{0.9239} \\\\\n\t\t\\bottomrule\n\t\\end{tabular}}\n\\end{table}\n\n\\begin{table}[!t]\n\t\\centering\n\t\\caption{Ablation study by comparing segmentation performance between\n\t\tdifferent models in terms of 3D-MHD. All models are trained on the first 9\n\t\tsubjects and evaluated on the $10^{th}$ subject. Smaller values indicate\n\t\tbetter performance. Note that 3D-MHD gives different results from MHD. Details of models are provided in\n\t\tthe text.}\n\t\\label{table:results_ablation_mhd}\n\t\\resizebox{.95\\columnwidth}{!}{\n\t\\begin{tabular}{ l | c | c | c | c }\n\t\t\\toprule\n\t\t\\textbf{Model} & \\textbf{CSF} & \\textbf{GM} & \\textbf{WM} & \\textbf{Average} \\\\\n\t\t\\midrule\n\n\t\tModel1       & \\textbf{0.2363} & 0.6277 & 0.4705 & 0.4448 \\\\\n\t\tModel2       & 0.2404 & 0.6052 & 0.4480 & 0.4312 \\\\\n\t\tModel3       & 0.2392 & 0.5993 & 0.4429 & 0.4271 \\\\\n\t\tModel4       & 0.2397 & 0.5926 & 0.4336 & 0.4220 \\\\\n\t\tModel5       & 0.2444 & 0.5901 & 0.4288 & 0.4211 \\\\\n\t\tNon-local U-Net    & 0.2477 & \\textbf{0.5692} & \\textbf{0.4062} & \\textbf{0.4077} \\\\\n\t\t\\bottomrule\n\t\\end{tabular}}\n\\end{table}\n\n",
        "label": "fig:results_training",
        "Descriptive_question1": "Which model shows a lower training loss in figure_5?",
        "Descriptive_question2": "What is compared in figure_5 between the proposed model and the baseline model?",
        "Reasoning_question1": "Based on figure_5, why might the proposed model be considered more effective than the baseline model in terms of training performance?",
        "Reasoning_question2": "What can be inferred from figure_5 about the potential risk of overfitting for the proposed model compared to the baseline model?",
        "Descriptive_answer1": "Proposed model",
        "Descriptive_answer2": "Training and validation results",
        "Reasoning_answer1": "Figure 5 shows that the proposed model achieves a lower training loss faster compared to the baseline model, indicating that it learns more efficiently and effectively during training. This suggests better optimization and possibly enhanced model architecture, leading to improved training performance.",
        "Reasoning_answer2": "From figure 5, the validation results of the proposed model remain better and more stable compared to the baseline model over training iterations, indicating that the proposed model does not overfit to the training data as severely as the baseline model. Hence, it has a lower potential risk of overfitting while maintaining good generalization."
    },
    {
        "paper_id": "1812.04103.json",
        "image_id": "figure_6",
        "image_path": [
            "/home/yz979/scratch/chengye/scimolmo/test-images-tables/dataset_imagestables/1812.04103/images/figure6.png"
        ],
        "caption": "Changes of segmentation performance in terms of DR, with respect to\n\t\tdifferent overlapping step sizes during inference. The model is trained on the\n\t\tfirst 9 subjects and evaluated on the $10^{th}$ subject.",
        "classify": "Chart",
        "section_info": "3 Results and Discussion\n\\section{Results and Discussion}\n\n\nWe perform experiments on the 3D multimodality isointense infant brain MR image segmentation task to evaluate our non-local U-Nets. The task is to perform automatic segmentation of MR images into cerebrospinal fluid~(CSF), gray matter~(GM) and white matter~(WM) regions. We first introduce the baseline model and the evaluation methods used in our experiments. Then the training and inference processes are described. We provide comparison results in terms of both effectiveness and efficiency, and conduct ablation studies to demonstrate that how each global aggregation block in our non-local U-Nets improves the performance. In addition, we explore the trade-off between the inference speed and accuracy based on different overlapping step sizes, and analyze the impact of patch size. The experimental code and dataset information have been made publicly available~\\footnote{\\url{https://github.com/divelab/Non-local-U-Nets}}.\n\n\\subsection{Experimental Setup}\n\n\nWe use CC-3D-FCN~\\cite{nie20183} as our baseline.\nCC-3D-FCN is a 3D fully convolutional network~(3D-FCN) with\nconvolution and concatenate~(CC) skip connections, which is designed\nfor 3D multimodality isointense infant brain image segmentation. It\nhas been shown to outperform traditional machine learning methods,\nsuch as FMRIB's automated segmentation\ntool~(FAST)~\\cite{zhang2001segmentation}, majority\nvoting~(MV), random forest~(RF)~\\cite{criminisi2013decision} and random forest with\nauto-context model~(LINKS)~\\cite{wang2015links}. Moreover, studies\nin~\\cite{nie20183} has showed the superiority of CC-3D-FCN to\nprevious deep learning models, like 2D, 3D\nCNNs~\\cite{zhang2015deep}, DeepMedic~\\cite{kamnitsas2017efficient},\nand the original 3D U-Net~\\cite{cciccek20163d}. Therefore, it is\nappropriate to use CC-3D-FCN as the baseline of our experiments.\nNote that our dataset is different from that in~\\cite{nie20183}.\n\n\nIn our experiments, we employ the Dice ratio~(DR) and propose the 3D modified\nHausdorff distance~(3D-MHD) as the evaluation metrics. These two\nmethods evaluate the accuracy only for binary segmentation tasks, so\nit is required to transform the 4-class segmentation map predicted by our model into\n4 binary segmentation maps for evaluation. That is, a 3D binary\nsegmentation map should be constructed for each class, where 1 denotes the voxel\nin the position belongs to the class and 0 means the opposite. In\nour experiments, we derive binary segmentation maps directly from\n4-class segmentation maps. The evaluation is performed on binary\nsegmentation maps for CSF, GM and WM.\n\n\nSpecifically, let $P$ and $L$ represent the predicted binary segmentation map for one class\nand the corresponding ground truth label, respectively. The DR is given by\n$DR=2|P \\cap L|/(|P|+|L|)$,\n\n\n\nwhere $|\\cdot|$ denotes the number of 1's in a segmentation map and $|P \\cap\nL|$ means the number of 1's shared by $P$ and $L$. Apparently, DR is a value\nin $[0,1]$ and a larger DR indicates a more accurate segmentation.\n\n\\begin{table*}[!ht]\n\t\\centering\n\t\\caption{Comparison of segmentation performance between our proposed model\n\t\tand the baseline model in terms of DR. The leave-one-subject-out\n\t\tcross-validation is used. Larger values indicate better performance.}\n\t\\label{table:results_baseline_dr}\n\t\\begin{tabular}{ l | c | c | c | c }\n\t\t\\toprule\n\t\t\\textbf{Model} & \\textbf{CSF} & \\textbf{GM} & \\textbf{WM} & \\textbf{Average} \\\\\n\t\t\\midrule\n\t\tBaseline\n\t\t& 0.9250$\\pm$0.0118\n\t\t& 0.9084$\\pm$0.0056\n\t\t& 0.8926$\\pm$0.0119\n\t\t& 0.9087$\\pm$0.0066 \\\\\n\t\tNon-local U-Net\n\t\t& \\textbf{0.9530$\\pm$0.0074}\n\t\t& \\textbf{0.9245$\\pm$0.0049}\n\t\t& \\textbf{0.9102$\\pm$0.0101}\n\t\t& \\textbf{0.9292$\\pm$0.0050} \\\\\n\t\t\\bottomrule\n\t\\end{tabular}\n\\end{table*}\n\n\\begin{table*}[!ht]\n\t\\centering\n\t\\caption{Comparison of segmentation performance between our proposed model\n\t\tand the baseline model in terms of 3D-MHD. The leave-one-subject-out\n\t\tcross-validation is used. Smaller values indicate better performance. Note that 3D-MHD gives different results from MHD.}\n\t\\label{table:results_baseline_mhd}\n\t\\begin{tabular}{ l | c | c | c | c }\n\t\t\\toprule\n\t\t\\textbf{Model} & \\textbf{CSF} & \\textbf{GM} & \\textbf{WM} & \\textbf{Average} \\\\\n\t\t\\midrule\n\t\tBaseline\n\t\t& 0.3417$\\pm$0.0245\n\t\t& 0.6537$\\pm$0.0483\n\t\t& 0.4817$\\pm$0.0454\n\t\t& 0.4924$\\pm$0.0345 \\\\\n\t\tNon-local U-Net\n\t\t& \\textbf{0.2554$\\pm$0.0207}\n\t\t& \\textbf{0.5950$\\pm$0.0428}\n\t\t& \\textbf{0.4454$\\pm$0.0040}\n\t\t& \\textbf{0.4319$\\pm$0.0313} \\\\\n\t\t\\bottomrule\n\t\\end{tabular}\n\\end{table*}\n\n\\begin{table*}[!t]\n\t\\centering\n\t\\caption{Comparison of segmentation performance on the 13 testing subjects of iSeg-2017 between our proposed model\n\t\tand the baseline model in terms of DR. Larger values indicate better performance.}\n\t\\label{table:iseg_results}\n\t\\begin{tabular}{ l | c | c | c }\n\t\t\\toprule\n\t\t\\textbf{Model} & \\textbf{CSF} & \\textbf{GM} & \\textbf{WM} \\\\\n\t\t\\midrule\n\t\tBaseline\n\t\t& 0.9324$\\pm$0.0067\n\t\t& 0.9146$\\pm$0.0074\n\t\t& 0.8974$\\pm$0.0123 \\\\\n\t\tNon-local U-Net\n\t\t& \\textbf{0.9557$\\pm$0.0060}\n\t\t& \\textbf{0.9219$\\pm$0.0089}\n\t\t& \\textbf{0.9044$\\pm$0.0153} \\\\\n\t\t\\bottomrule\n\t\\end{tabular}\n\\end{table*}\n\n\nThe modified Hausdorff distance~(MHD)~\\cite{dubuisson1994modified} is\ndesigned to compute the similarity between two objects. Here, an object is a\nset of points where a point is represented by a vector. Specifically, given\ntwo sets of vectors $A$ and $B$, MHD is computed by\n$MHD=\\max(d(A,B),d(B,A))$,\n\n\n\nwhere the distance between two sets is defined as\n$d(A,B)=1/|A|\\sum_{a \\in A}{d(a,B)}$,\n\n\n\nand the distance between a vector and a set is defined as\n$d(a,B)=\\min_{b \\in B}||a-b||$.\n\n\n\nPrevious studies~\\cite{wang2015links,zhang2015deep,nie20183}\napplied MHD for evaluation by treating a 3D $D \\times H \\times W$\nmap as $H \\times W$ $D$-dimensional vectors. However, there are two\nmore different ways to vectorize the 3D map, depending on the\ndirection of forming vectors, \\emph{i.e.,} $D \\times H$\n$W$-dimensional vectors and $D \\times W$ $H$-dimensional vectors.\nEach vectorization leads to different evaluation results by MHD. To\nmake it a direction-independent evaluation metric as DR, we define\n3D-MHD, which computes the averaged MHD based on the three different\nvectorizations. A smaller 3D-MHD indicates a\nhigher segmentation accuracy.\n\n\\subsection{Training and Inference Strategies}\n\n\nOur proposed non-local U-Nets apply Dropout~\\cite{srivastava2014dropout} with\na rate of 0.5 in each global aggregation block and the output block\nbefore the final $1 \\times 1 \\times 1$ convolution. A weight\ndecay~\\cite{krogh1992simple} with a rate of $2e-6$ is also employed.\nTo train the model, we use randomly cropped small patches. In this\nway, we obtain sufficient training data and the requirement on\nmemory is reduced. No extra data augmentation is needed. The\nexperimental results below suggest that patches\nwith a size of $32^3$ leads to the best\nperformance. The batch size is set to 5. The Adam\noptimizer~\\cite{kingma2014adam} with a learning rate of 0.001 is\nemployed to perform the gradient descent algorithm.\n\n\nIn the inference process, following~\\cite{nie20183}, we extract\npatches with the same size as that used in training. For example, to\ngenerate $32^3$ patches for inference, we slide a\nwindow of size $32^3$ through the original image\nwith a constant overlapping step size. The overlapping step size\nmust be smaller than or equal to the patch size, in order to\nguarantee that extracted patches cover the whole image.\nConsequently, prediction for all these patches provides segmentation\nprobability results for every voxel in the original image. For\nvoxels that receive multiple results due to overlapping, we average\nthem to produce the final prediction. The overlapping step size is\nan important hyper-parameter affecting the inference speed and the\nsegmentation accuracy. A smaller overlapping step size results in\nbetter accuracy, but increases the inference time as more patches\nare generated. We explore the trade-off in our experiments.\n\n\\subsection{Comparison with the Baseline}\\label{sec:baseline}\n\n\nWe compare our non-local U-Nets with the baseline on our dataset.\nFollowing~\\cite{nie20183}, the patch size is set to $32^3$\nand the overlapping step size for inference is set to $8$. To remove the\nbias of different subjects, the leave-one-subject-out cross-validation is\nused for evaluating segmentation performance. That is, for 10 subjects in our\ndataset, we train and evaluate models 10 times correspondingly. Each time one\nof the 10 subjects is left out for validation and the other 9 subjects are\nused for training. The mean and standard deviation of segmentation\nperformance of the 10 runs are reported.\n\n\nTables~\\ref{table:results_baseline_dr}\nand~\\ref{table:results_baseline_mhd} provide the experimental\nresults. In terms of both evaluation metrics, our non-local U-Nets achieve\nsignificant improvements over the baseline model. Due to the small\nvariances of the results, we focus on one of the 10 runs for\nvisualization and ablation studies, where the models are trained on the\nfirst 9 subjects and evaluated on the $10^{th}$ subject. A\nvisualization of the segmentation results in this run is given by\nFig.~\\ref{fig:results_visual}. By comparing the areas in red\ncircles, we can see that our model is capable of catching more\ndetails than the baseline model. We also visualize the training\nprocesses to illustrate the superiority of our model.\nFig.~\\ref{fig:results_training} shows the training and validation\ncurves in this run of our model and the baseline model,\nrespectively. Clearly, our model converges faster to a lower\ntraining loss. In addition, according to the better validation\nresults, our model does not suffer from over-fitting.\n\n\nTo further show the efficiency of our proposed model, we compare the\nnumber of parameters as reported in Table~\\ref{table:num_params}.\nOur model reduces $28\\%$ parameters compared to CC-3D-FCN and\nachieves better performance. A comparison of inference time is also\nprovided in Table~\\ref{table:infer_time}. The settings of our device\nare - GPU: Nvidia Titan Xp 12GB; CPU: Intel Xeon E5-2620v4 2.10GHz;\nOS: Ubuntu 16.04.3 LTS.\n\n\nSince our data has been used as the training data in the iSeg-2017\nchallenge, we also compare the\nresults evaluated on the 13 testing subjects in\nTable~\\ref{table:iseg_results}. According to the leader board, our model\nachieves one of the top performances. Results in terms of DR are reported since\nit is the only shared evaluation metric.\n\n\\begin{table}[!t]\n\t\\centering\n\t\\caption{Comparison of the number of parameters between our proposed model\n\t\tand the baseline model.}\n\t\\label{table:num_params}\n\t\\begin{tabular}{ l | c }\n\t\t\\toprule\n\t\t\\textbf{Model} & Number of Parameters \\\\\n\t\t\\midrule\n\t\tBaseline  & 2,534,276 \\\\\n\t\tNon-local U-Net & \\textbf{1,821,124} \\\\\n\t\t\\bottomrule\n\n\n\n\n\n\n\t\\end{tabular}\n\\end{table}\n\n\\begin{table}[!t]\n\t\\centering\n\t\\caption{Comparison of inference time between our proposed model and the baseline model. The leave-one-subject-out cross-validation is used. The patch size is set to $32^3$ and the overlapping step size for inference is set to $8$.}\n\t\\label{table:infer_time}\n\t\\begin{tabular}{ l | c }\n\t\t\\toprule\n\t\t\\textbf{Model} & Inference Time (min) \\\\\n\t\t\\midrule\n\t\tBaseline  & 3.85$\\pm$0.15 \\\\\n\t\tNon-local U-Net  & \\textbf{3.06$\\pm$0.12} \\\\\n\t\t\\bottomrule\n\t\\end{tabular}\n\\end{table}\n\n\\begin{figure}[!t]\n\t\\centering\n\t\\includegraphics[width=0.95\\columnwidth]{figure4.png}\n\t\\caption{Visualization of the segmentation results on the $10^{th}$ subject\n\t\tby our proposed model and the baseline model. Both models are trained on the\n\t\tfirst 9 subjects. The first column shows the original segmentation maps. The\n\t\tsecond, third and fourth columns show the binary segmentation maps for CSF,\n\t\tGM and WM, respectively.}\n\t\\label{fig:results_visual}\n\\end{figure}\n\n\\begin{figure}[!t]\n\t\\centering\n\t\\includegraphics[width=0.75\\columnwidth]{figure5.png}\n\t\\caption{Comparison of training processes and validation results between our proposed model and the\n\t\tbaseline model when training on the first 9 subjects and using the $10^{th}$ subject for validation.}\n\t\\label{fig:results_training}\n\\end{figure}\n\n\\begin{table}[!t]\n\t\\centering\n\t\\caption{Ablation study by comparing segmentation performance between\n\t\tdifferent models in terms of DR. All models are trained on the first 9\n\t\tsubjects and evaluated on the $10^{th}$ subject. Larger values indicate\n\t\tbetter performance. Details of models are provided in the text.}\n\t\\label{table:results_ablation_dr}\n\t\\resizebox{.95\\columnwidth}{!}{\n\t\\begin{tabular}{ l | c | c | c | c }\n\t\t\\toprule\n\t\t\\textbf{Model} & \\textbf{CSF} & \\textbf{GM} & \\textbf{WM} & \\textbf{Average} \\\\\n\t\t\\midrule\n\n\t\tModel1       & \\textbf{0.9585} & 0.9099 & 0.8625 & 0.9103 \\\\\n\t\tModel2       & 0.9568 & 0.9172 & 0.8728 & 0.9156 \\\\\n\t\tModel3       & 0.9576 & 0.9198 & 0.8749 & 0.9174 \\\\\n\t\tModel4       & 0.9578 & 0.9210 & 0.8769 & 0.9186 \\\\\n\t\tModel5       & 0.9554 & 0.9225 & 0.8804 & 0.9194 \\\\\n\t\tNon-local U-Net    & 0.9572 & \\textbf{0.9278} & \\textbf{0.8867} & \\textbf{0.9239} \\\\\n\t\t\\bottomrule\n\t\\end{tabular}}\n\\end{table}\n\n\\begin{table}[!t]\n\t\\centering\n\t\\caption{Ablation study by comparing segmentation performance between\n\t\tdifferent models in terms of 3D-MHD. All models are trained on the first 9\n\t\tsubjects and evaluated on the $10^{th}$ subject. Smaller values indicate\n\t\tbetter performance. Note that 3D-MHD gives different results from MHD. Details of models are provided in\n\t\tthe text.}\n\t\\label{table:results_ablation_mhd}\n\t\\resizebox{.95\\columnwidth}{!}{\n\t\\begin{tabular}{ l | c | c | c | c }\n\t\t\\toprule\n\t\t\\textbf{Model} & \\textbf{CSF} & \\textbf{GM} & \\textbf{WM} & \\textbf{Average} \\\\\n\t\t\\midrule\n\n\t\tModel1       & \\textbf{0.2363} & 0.6277 & 0.4705 & 0.4448 \\\\\n\t\tModel2       & 0.2404 & 0.6052 & 0.4480 & 0.4312 \\\\\n\t\tModel3       & 0.2392 & 0.5993 & 0.4429 & 0.4271 \\\\\n\t\tModel4       & 0.2397 & 0.5926 & 0.4336 & 0.4220 \\\\\n\t\tModel5       & 0.2444 & 0.5901 & 0.4288 & 0.4211 \\\\\n\t\tNon-local U-Net    & 0.2477 & \\textbf{0.5692} & \\textbf{0.4062} & \\textbf{0.4077} \\\\\n\t\t\\bottomrule\n\t\\end{tabular}}\n\\end{table}\n\n\\subsection{Ablation Studies of Different Modules}\\label{sec:ablation}\n\nWe perform ablation studies to show the effectiveness of each part of our\nnon-local U-Nets. Specifically, we compare the following models:\n\n\\textbf{Model1} is a 3D U-Net without short-range residual connections.\nDown-sampling and up-sampling are implemented by convolutions and\ndeconvolutions with a stride of 2, respectively. The bottom block is simply a\nconvolutional layer. Note that the baseline model, CC-3D-FCN, has showed improved performance over 3D U-Net~\\cite{nie20183}. However, the original 3D U-Net was not designed for this task~\\cite{cciccek20163d}. In our experiments, we appropriately set the hyperparameters of 3D U-Net and achieve better performance.\n\n\\textbf{Model2} is Model1 with short-range residual connections, \\emph{i.e.},\nthe blocks in Fig.~\\ref{fig:residual}(a) and (b) are applied. The bottom\nblock and up-sampling blocks are the same as those in Model1.\n\n\\textbf{Model3} replaces the first up-sampling block in Model2 with the block\nin Fig.~\\ref{fig:residual}(d).\n\n\\textbf{Model4} replaces both up-sampling blocks in Model2 with the block in\nFig.~\\ref{fig:residual}(d).\n\n\\textbf{Model5} replaces the bottom block in Model2 with the block in\nFig.~\\ref{fig:residual}(c).\n\nAll models are trained on the first 9 subjects. We report the segmentation\nperformance on the $10^{th}$ subject in Table~\\ref{table:results_ablation_dr}\nand Table~\\ref{table:results_ablation_mhd}. The results demonstrate how different\nglobal aggregation blocks in our non-local U-Nets improve the performance.\n\n\\subsection{Impact of the Overlapping Step Size}\\label{sec:overlap}\n\nAs discussed above, a small overlapping step size\nusually results in better segmentation, due to the ensemble effect.\nHowever, with a small overlapping step size, the model has to perform\ninference for more validation patches and thus decreases the inference speed. We\nexplore the trade-off in our non-local U-Nets by setting the overlapping step sizes to 4,\n8, 16, 32, respectively. Again, we train our model on the first 9 subjects and\nperform evaluation on the $10^{th}$ subject. The patch size is set to $32^3$.\nAccording to the overlapping step sizes, 11880, 1920,\n387, 80 patches need to be processed during inference, as shown in\nFig.~\\ref{fig:results_overlap_time}. In addition, Fig.~\\ref{fig:results_overlap_dr}\nplots the changes of segmentation performance in terms of DR. Obviously, 8 and\n16 are good choices that achieve accurate and fast segmentation results.\n\n\\begin{figure}[!t]\n\t\\centering\n\t\\includegraphics[width=0.7\\columnwidth]{figure6.png}\n\t\\caption{Changes of segmentation performance in terms of DR, with respect to\n\t\tdifferent overlapping step sizes during inference. The model is trained on the\n\t\tfirst 9 subjects and evaluated on the $10^{th}$ subject.}\n\t\\label{fig:results_overlap_dr}\n\\end{figure}\n\n\n\\begin{figure}[!t]\n\t\\centering\n\t\\includegraphics[width=0.75\\columnwidth]{figure7.png}\n\t\\caption{Changes of the number of validation patches for the $10^{th}$\n\t\tsubject, with respect to different overlapping step sizes during inference.}\n\t\\label{fig:results_overlap_time}\n\\end{figure}\n\n\\subsection{Impact of the Patch Size}\\label{sec:patch}\n\nThe patch size affects the total number of distinct training samples.\nMeanwhile, it controls the range of available global information when\nperforming segmentation for a patch. To choose the appropriate patch\nsize for the non-local U-Nets, we perform a grid search by training on the first 9\nsubjects and evaluating on the $10^{th}$ subject with the overlapping step\nsize of 8. Experiments are conducted with five different patch sizes:\n$16^3$, $24^3$, $32^3$, $40^3$, $48^3$. The results are provided in\nFig.~\\ref{fig:results_patch_dr}, where $32^3$ obtains the best\nperformance and is selected as the default setting of our model.\n\n\n\\begin{figure}[!t]\n\t\\centering\n\t\\includegraphics[width=0.75\\columnwidth]{figure8.png}\n\t\\caption{Changes of segmentation performance in terms of DR, with respect to\n\t\tdifferent patch sizes. The model is trained on the first 9 subjects and\n\t\tevaluated on the $10^{th}$ subject.}\n\t\\label{fig:results_patch_dr}\n\\end{figure}\n\n3.5 Impact of the Overlapping Step Size\n\\subsection{Impact of the Overlapping Step Size}\\label{sec:overlap}\n\nAs discussed above, a small overlapping step size\nusually results in better segmentation, due to the ensemble effect.\nHowever, with a small overlapping step size, the model has to perform\ninference for more validation patches and thus decreases the inference speed. We\nexplore the trade-off in our non-local U-Nets by setting the overlapping step sizes to 4,\n8, 16, 32, respectively. Again, we train our model on the first 9 subjects and\nperform evaluation on the $10^{th}$ subject. The patch size is set to $32^3$.\nAccording to the overlapping step sizes, 11880, 1920,\n387, 80 patches need to be processed during inference, as shown in\nFig.~\\ref{fig:results_overlap_time}. In addition, Fig.~\\ref{fig:results_overlap_dr}\nplots the changes of segmentation performance in terms of DR. Obviously, 8 and\n16 are good choices that achieve accurate and fast segmentation results.\n\n\\begin{figure}[!t]\n\t\\centering\n\t\\includegraphics[width=0.7\\columnwidth]{figure6.png}\n\t\\caption{Changes of segmentation performance in terms of DR, with respect to\n\t\tdifferent overlapping step sizes during inference. The model is trained on the\n\t\tfirst 9 subjects and evaluated on the $10^{th}$ subject.}\n\t\\label{fig:results_overlap_dr}\n\\end{figure}\n\n\n\\begin{figure}[!t]\n\t\\centering\n\t\\includegraphics[width=0.75\\columnwidth]{figure7.png}\n\t\\caption{Changes of the number of validation patches for the $10^{th}$\n\t\tsubject, with respect to different overlapping step sizes during inference.}\n\t\\label{fig:results_overlap_time}\n\\end{figure}\n\n",
        "label": "fig:results_overlap_dr",
        "Descriptive_question1": "What is the smallest overlapping step size shown in figure_6?",
        "Descriptive_question2": "What metric is used to evaluate segmentation performance in figure_6?",
        "Reasoning_question1": "How does the segmentation performance in terms of DR change as the overlapping step size increases in figure_6, and what might be the reason for this trend?",
        "Reasoning_question2": "Why might an overlapping step size of 8 or 16 be considered a good balance for segmentation performance and inference speed based on the data in figure_6?",
        "Descriptive_answer1": "4",
        "Descriptive_answer2": "DR",
        "Reasoning_answer1": "As the overlapping step size increases, the segmentation performance in terms of Dice Ratio (DR) decreases. This trend likely occurs because smaller overlapping step sizes result in more overlapping patches during inference, facilitating an ensemble effect where multiple predictions for the same voxel are averaged, leading to improved accuracy. Conversely, larger overlapping step sizes reduce overlap and ensemble benefits, thus lowering performance.",
        "Reasoning_answer2": "An overlapping step size of 8 or 16 is considered a good balance because, according to figure_6, these step sizes achieve relatively high DR values, indicating accurate segmentation, while requiring fewer validation patches compared to the smallest step size of 4. This reduces computation time and speeds up inference, providing an efficient trade-off between segmentation performance and inference speed."
    },
    {
        "paper_id": "1812.04103.json",
        "image_id": "figure_7",
        "image_path": [
            "/home/yz979/scratch/chengye/scimolmo/test-images-tables/dataset_imagestables/1812.04103/images/figure7.png"
        ],
        "caption": "Changes of the number of validation patches for the $10^{th}$\n\t\tsubject, with respect to different overlapping step sizes during inference.",
        "classify": "Chart",
        "section_info": "3 Results and Discussion\n\\section{Results and Discussion}\n\n\nWe perform experiments on the 3D multimodality isointense infant brain MR image segmentation task to evaluate our non-local U-Nets. The task is to perform automatic segmentation of MR images into cerebrospinal fluid~(CSF), gray matter~(GM) and white matter~(WM) regions. We first introduce the baseline model and the evaluation methods used in our experiments. Then the training and inference processes are described. We provide comparison results in terms of both effectiveness and efficiency, and conduct ablation studies to demonstrate that how each global aggregation block in our non-local U-Nets improves the performance. In addition, we explore the trade-off between the inference speed and accuracy based on different overlapping step sizes, and analyze the impact of patch size. The experimental code and dataset information have been made publicly available~\\footnote{\\url{https://github.com/divelab/Non-local-U-Nets}}.\n\n\\subsection{Experimental Setup}\n\n\nWe use CC-3D-FCN~\\cite{nie20183} as our baseline.\nCC-3D-FCN is a 3D fully convolutional network~(3D-FCN) with\nconvolution and concatenate~(CC) skip connections, which is designed\nfor 3D multimodality isointense infant brain image segmentation. It\nhas been shown to outperform traditional machine learning methods,\nsuch as FMRIB's automated segmentation\ntool~(FAST)~\\cite{zhang2001segmentation}, majority\nvoting~(MV), random forest~(RF)~\\cite{criminisi2013decision} and random forest with\nauto-context model~(LINKS)~\\cite{wang2015links}. Moreover, studies\nin~\\cite{nie20183} has showed the superiority of CC-3D-FCN to\nprevious deep learning models, like 2D, 3D\nCNNs~\\cite{zhang2015deep}, DeepMedic~\\cite{kamnitsas2017efficient},\nand the original 3D U-Net~\\cite{cciccek20163d}. Therefore, it is\nappropriate to use CC-3D-FCN as the baseline of our experiments.\nNote that our dataset is different from that in~\\cite{nie20183}.\n\n\nIn our experiments, we employ the Dice ratio~(DR) and propose the 3D modified\nHausdorff distance~(3D-MHD) as the evaluation metrics. These two\nmethods evaluate the accuracy only for binary segmentation tasks, so\nit is required to transform the 4-class segmentation map predicted by our model into\n4 binary segmentation maps for evaluation. That is, a 3D binary\nsegmentation map should be constructed for each class, where 1 denotes the voxel\nin the position belongs to the class and 0 means the opposite. In\nour experiments, we derive binary segmentation maps directly from\n4-class segmentation maps. The evaluation is performed on binary\nsegmentation maps for CSF, GM and WM.\n\n\nSpecifically, let $P$ and $L$ represent the predicted binary segmentation map for one class\nand the corresponding ground truth label, respectively. The DR is given by\n$DR=2|P \\cap L|/(|P|+|L|)$,\n\n\n\nwhere $|\\cdot|$ denotes the number of 1's in a segmentation map and $|P \\cap\nL|$ means the number of 1's shared by $P$ and $L$. Apparently, DR is a value\nin $[0,1]$ and a larger DR indicates a more accurate segmentation.\n\n\\begin{table*}[!ht]\n\t\\centering\n\t\\caption{Comparison of segmentation performance between our proposed model\n\t\tand the baseline model in terms of DR. The leave-one-subject-out\n\t\tcross-validation is used. Larger values indicate better performance.}\n\t\\label{table:results_baseline_dr}\n\t\\begin{tabular}{ l | c | c | c | c }\n\t\t\\toprule\n\t\t\\textbf{Model} & \\textbf{CSF} & \\textbf{GM} & \\textbf{WM} & \\textbf{Average} \\\\\n\t\t\\midrule\n\t\tBaseline\n\t\t& 0.9250$\\pm$0.0118\n\t\t& 0.9084$\\pm$0.0056\n\t\t& 0.8926$\\pm$0.0119\n\t\t& 0.9087$\\pm$0.0066 \\\\\n\t\tNon-local U-Net\n\t\t& \\textbf{0.9530$\\pm$0.0074}\n\t\t& \\textbf{0.9245$\\pm$0.0049}\n\t\t& \\textbf{0.9102$\\pm$0.0101}\n\t\t& \\textbf{0.9292$\\pm$0.0050} \\\\\n\t\t\\bottomrule\n\t\\end{tabular}\n\\end{table*}\n\n\\begin{table*}[!ht]\n\t\\centering\n\t\\caption{Comparison of segmentation performance between our proposed model\n\t\tand the baseline model in terms of 3D-MHD. The leave-one-subject-out\n\t\tcross-validation is used. Smaller values indicate better performance. Note that 3D-MHD gives different results from MHD.}\n\t\\label{table:results_baseline_mhd}\n\t\\begin{tabular}{ l | c | c | c | c }\n\t\t\\toprule\n\t\t\\textbf{Model} & \\textbf{CSF} & \\textbf{GM} & \\textbf{WM} & \\textbf{Average} \\\\\n\t\t\\midrule\n\t\tBaseline\n\t\t& 0.3417$\\pm$0.0245\n\t\t& 0.6537$\\pm$0.0483\n\t\t& 0.4817$\\pm$0.0454\n\t\t& 0.4924$\\pm$0.0345 \\\\\n\t\tNon-local U-Net\n\t\t& \\textbf{0.2554$\\pm$0.0207}\n\t\t& \\textbf{0.5950$\\pm$0.0428}\n\t\t& \\textbf{0.4454$\\pm$0.0040}\n\t\t& \\textbf{0.4319$\\pm$0.0313} \\\\\n\t\t\\bottomrule\n\t\\end{tabular}\n\\end{table*}\n\n\\begin{table*}[!t]\n\t\\centering\n\t\\caption{Comparison of segmentation performance on the 13 testing subjects of iSeg-2017 between our proposed model\n\t\tand the baseline model in terms of DR. Larger values indicate better performance.}\n\t\\label{table:iseg_results}\n\t\\begin{tabular}{ l | c | c | c }\n\t\t\\toprule\n\t\t\\textbf{Model} & \\textbf{CSF} & \\textbf{GM} & \\textbf{WM} \\\\\n\t\t\\midrule\n\t\tBaseline\n\t\t& 0.9324$\\pm$0.0067\n\t\t& 0.9146$\\pm$0.0074\n\t\t& 0.8974$\\pm$0.0123 \\\\\n\t\tNon-local U-Net\n\t\t& \\textbf{0.9557$\\pm$0.0060}\n\t\t& \\textbf{0.9219$\\pm$0.0089}\n\t\t& \\textbf{0.9044$\\pm$0.0153} \\\\\n\t\t\\bottomrule\n\t\\end{tabular}\n\\end{table*}\n\n\nThe modified Hausdorff distance~(MHD)~\\cite{dubuisson1994modified} is\ndesigned to compute the similarity between two objects. Here, an object is a\nset of points where a point is represented by a vector. Specifically, given\ntwo sets of vectors $A$ and $B$, MHD is computed by\n$MHD=\\max(d(A,B),d(B,A))$,\n\n\n\nwhere the distance between two sets is defined as\n$d(A,B)=1/|A|\\sum_{a \\in A}{d(a,B)}$,\n\n\n\nand the distance between a vector and a set is defined as\n$d(a,B)=\\min_{b \\in B}||a-b||$.\n\n\n\nPrevious studies~\\cite{wang2015links,zhang2015deep,nie20183}\napplied MHD for evaluation by treating a 3D $D \\times H \\times W$\nmap as $H \\times W$ $D$-dimensional vectors. However, there are two\nmore different ways to vectorize the 3D map, depending on the\ndirection of forming vectors, \\emph{i.e.,} $D \\times H$\n$W$-dimensional vectors and $D \\times W$ $H$-dimensional vectors.\nEach vectorization leads to different evaluation results by MHD. To\nmake it a direction-independent evaluation metric as DR, we define\n3D-MHD, which computes the averaged MHD based on the three different\nvectorizations. A smaller 3D-MHD indicates a\nhigher segmentation accuracy.\n\n\\subsection{Training and Inference Strategies}\n\n\nOur proposed non-local U-Nets apply Dropout~\\cite{srivastava2014dropout} with\na rate of 0.5 in each global aggregation block and the output block\nbefore the final $1 \\times 1 \\times 1$ convolution. A weight\ndecay~\\cite{krogh1992simple} with a rate of $2e-6$ is also employed.\nTo train the model, we use randomly cropped small patches. In this\nway, we obtain sufficient training data and the requirement on\nmemory is reduced. No extra data augmentation is needed. The\nexperimental results below suggest that patches\nwith a size of $32^3$ leads to the best\nperformance. The batch size is set to 5. The Adam\noptimizer~\\cite{kingma2014adam} with a learning rate of 0.001 is\nemployed to perform the gradient descent algorithm.\n\n\nIn the inference process, following~\\cite{nie20183}, we extract\npatches with the same size as that used in training. For example, to\ngenerate $32^3$ patches for inference, we slide a\nwindow of size $32^3$ through the original image\nwith a constant overlapping step size. The overlapping step size\nmust be smaller than or equal to the patch size, in order to\nguarantee that extracted patches cover the whole image.\nConsequently, prediction for all these patches provides segmentation\nprobability results for every voxel in the original image. For\nvoxels that receive multiple results due to overlapping, we average\nthem to produce the final prediction. The overlapping step size is\nan important hyper-parameter affecting the inference speed and the\nsegmentation accuracy. A smaller overlapping step size results in\nbetter accuracy, but increases the inference time as more patches\nare generated. We explore the trade-off in our experiments.\n\n\\subsection{Comparison with the Baseline}\\label{sec:baseline}\n\n\nWe compare our non-local U-Nets with the baseline on our dataset.\nFollowing~\\cite{nie20183}, the patch size is set to $32^3$\nand the overlapping step size for inference is set to $8$. To remove the\nbias of different subjects, the leave-one-subject-out cross-validation is\nused for evaluating segmentation performance. That is, for 10 subjects in our\ndataset, we train and evaluate models 10 times correspondingly. Each time one\nof the 10 subjects is left out for validation and the other 9 subjects are\nused for training. The mean and standard deviation of segmentation\nperformance of the 10 runs are reported.\n\n\nTables~\\ref{table:results_baseline_dr}\nand~\\ref{table:results_baseline_mhd} provide the experimental\nresults. In terms of both evaluation metrics, our non-local U-Nets achieve\nsignificant improvements over the baseline model. Due to the small\nvariances of the results, we focus on one of the 10 runs for\nvisualization and ablation studies, where the models are trained on the\nfirst 9 subjects and evaluated on the $10^{th}$ subject. A\nvisualization of the segmentation results in this run is given by\nFig.~\\ref{fig:results_visual}. By comparing the areas in red\ncircles, we can see that our model is capable of catching more\ndetails than the baseline model. We also visualize the training\nprocesses to illustrate the superiority of our model.\nFig.~\\ref{fig:results_training} shows the training and validation\ncurves in this run of our model and the baseline model,\nrespectively. Clearly, our model converges faster to a lower\ntraining loss. In addition, according to the better validation\nresults, our model does not suffer from over-fitting.\n\n\nTo further show the efficiency of our proposed model, we compare the\nnumber of parameters as reported in Table~\\ref{table:num_params}.\nOur model reduces $28\\%$ parameters compared to CC-3D-FCN and\nachieves better performance. A comparison of inference time is also\nprovided in Table~\\ref{table:infer_time}. The settings of our device\nare - GPU: Nvidia Titan Xp 12GB; CPU: Intel Xeon E5-2620v4 2.10GHz;\nOS: Ubuntu 16.04.3 LTS.\n\n\nSince our data has been used as the training data in the iSeg-2017\nchallenge, we also compare the\nresults evaluated on the 13 testing subjects in\nTable~\\ref{table:iseg_results}. According to the leader board, our model\nachieves one of the top performances. Results in terms of DR are reported since\nit is the only shared evaluation metric.\n\n\\begin{table}[!t]\n\t\\centering\n\t\\caption{Comparison of the number of parameters between our proposed model\n\t\tand the baseline model.}\n\t\\label{table:num_params}\n\t\\begin{tabular}{ l | c }\n\t\t\\toprule\n\t\t\\textbf{Model} & Number of Parameters \\\\\n\t\t\\midrule\n\t\tBaseline  & 2,534,276 \\\\\n\t\tNon-local U-Net & \\textbf{1,821,124} \\\\\n\t\t\\bottomrule\n\n\n\n\n\n\n\t\\end{tabular}\n\\end{table}\n\n\\begin{table}[!t]\n\t\\centering\n\t\\caption{Comparison of inference time between our proposed model and the baseline model. The leave-one-subject-out cross-validation is used. The patch size is set to $32^3$ and the overlapping step size for inference is set to $8$.}\n\t\\label{table:infer_time}\n\t\\begin{tabular}{ l | c }\n\t\t\\toprule\n\t\t\\textbf{Model} & Inference Time (min) \\\\\n\t\t\\midrule\n\t\tBaseline  & 3.85$\\pm$0.15 \\\\\n\t\tNon-local U-Net  & \\textbf{3.06$\\pm$0.12} \\\\\n\t\t\\bottomrule\n\t\\end{tabular}\n\\end{table}\n\n\\begin{figure}[!t]\n\t\\centering\n\t\\includegraphics[width=0.95\\columnwidth]{figure4.png}\n\t\\caption{Visualization of the segmentation results on the $10^{th}$ subject\n\t\tby our proposed model and the baseline model. Both models are trained on the\n\t\tfirst 9 subjects. The first column shows the original segmentation maps. The\n\t\tsecond, third and fourth columns show the binary segmentation maps for CSF,\n\t\tGM and WM, respectively.}\n\t\\label{fig:results_visual}\n\\end{figure}\n\n\\begin{figure}[!t]\n\t\\centering\n\t\\includegraphics[width=0.75\\columnwidth]{figure5.png}\n\t\\caption{Comparison of training processes and validation results between our proposed model and the\n\t\tbaseline model when training on the first 9 subjects and using the $10^{th}$ subject for validation.}\n\t\\label{fig:results_training}\n\\end{figure}\n\n\\begin{table}[!t]\n\t\\centering\n\t\\caption{Ablation study by comparing segmentation performance between\n\t\tdifferent models in terms of DR. All models are trained on the first 9\n\t\tsubjects and evaluated on the $10^{th}$ subject. Larger values indicate\n\t\tbetter performance. Details of models are provided in the text.}\n\t\\label{table:results_ablation_dr}\n\t\\resizebox{.95\\columnwidth}{!}{\n\t\\begin{tabular}{ l | c | c | c | c }\n\t\t\\toprule\n\t\t\\textbf{Model} & \\textbf{CSF} & \\textbf{GM} & \\textbf{WM} & \\textbf{Average} \\\\\n\t\t\\midrule\n\n\t\tModel1       & \\textbf{0.9585} & 0.9099 & 0.8625 & 0.9103 \\\\\n\t\tModel2       & 0.9568 & 0.9172 & 0.8728 & 0.9156 \\\\\n\t\tModel3       & 0.9576 & 0.9198 & 0.8749 & 0.9174 \\\\\n\t\tModel4       & 0.9578 & 0.9210 & 0.8769 & 0.9186 \\\\\n\t\tModel5       & 0.9554 & 0.9225 & 0.8804 & 0.9194 \\\\\n\t\tNon-local U-Net    & 0.9572 & \\textbf{0.9278} & \\textbf{0.8867} & \\textbf{0.9239} \\\\\n\t\t\\bottomrule\n\t\\end{tabular}}\n\\end{table}\n\n\\begin{table}[!t]\n\t\\centering\n\t\\caption{Ablation study by comparing segmentation performance between\n\t\tdifferent models in terms of 3D-MHD. All models are trained on the first 9\n\t\tsubjects and evaluated on the $10^{th}$ subject. Smaller values indicate\n\t\tbetter performance. Note that 3D-MHD gives different results from MHD. Details of models are provided in\n\t\tthe text.}\n\t\\label{table:results_ablation_mhd}\n\t\\resizebox{.95\\columnwidth}{!}{\n\t\\begin{tabular}{ l | c | c | c | c }\n\t\t\\toprule\n\t\t\\textbf{Model} & \\textbf{CSF} & \\textbf{GM} & \\textbf{WM} & \\textbf{Average} \\\\\n\t\t\\midrule\n\n\t\tModel1       & \\textbf{0.2363} & 0.6277 & 0.4705 & 0.4448 \\\\\n\t\tModel2       & 0.2404 & 0.6052 & 0.4480 & 0.4312 \\\\\n\t\tModel3       & 0.2392 & 0.5993 & 0.4429 & 0.4271 \\\\\n\t\tModel4       & 0.2397 & 0.5926 & 0.4336 & 0.4220 \\\\\n\t\tModel5       & 0.2444 & 0.5901 & 0.4288 & 0.4211 \\\\\n\t\tNon-local U-Net    & 0.2477 & \\textbf{0.5692} & \\textbf{0.4062} & \\textbf{0.4077} \\\\\n\t\t\\bottomrule\n\t\\end{tabular}}\n\\end{table}\n\n\\subsection{Ablation Studies of Different Modules}\\label{sec:ablation}\n\nWe perform ablation studies to show the effectiveness of each part of our\nnon-local U-Nets. Specifically, we compare the following models:\n\n\\textbf{Model1} is a 3D U-Net without short-range residual connections.\nDown-sampling and up-sampling are implemented by convolutions and\ndeconvolutions with a stride of 2, respectively. The bottom block is simply a\nconvolutional layer. Note that the baseline model, CC-3D-FCN, has showed improved performance over 3D U-Net~\\cite{nie20183}. However, the original 3D U-Net was not designed for this task~\\cite{cciccek20163d}. In our experiments, we appropriately set the hyperparameters of 3D U-Net and achieve better performance.\n\n\\textbf{Model2} is Model1 with short-range residual connections, \\emph{i.e.},\nthe blocks in Fig.~\\ref{fig:residual}(a) and (b) are applied. The bottom\nblock and up-sampling blocks are the same as those in Model1.\n\n\\textbf{Model3} replaces the first up-sampling block in Model2 with the block\nin Fig.~\\ref{fig:residual}(d).\n\n\\textbf{Model4} replaces both up-sampling blocks in Model2 with the block in\nFig.~\\ref{fig:residual}(d).\n\n\\textbf{Model5} replaces the bottom block in Model2 with the block in\nFig.~\\ref{fig:residual}(c).\n\nAll models are trained on the first 9 subjects. We report the segmentation\nperformance on the $10^{th}$ subject in Table~\\ref{table:results_ablation_dr}\nand Table~\\ref{table:results_ablation_mhd}. The results demonstrate how different\nglobal aggregation blocks in our non-local U-Nets improve the performance.\n\n\\subsection{Impact of the Overlapping Step Size}\\label{sec:overlap}\n\nAs discussed above, a small overlapping step size\nusually results in better segmentation, due to the ensemble effect.\nHowever, with a small overlapping step size, the model has to perform\ninference for more validation patches and thus decreases the inference speed. We\nexplore the trade-off in our non-local U-Nets by setting the overlapping step sizes to 4,\n8, 16, 32, respectively. Again, we train our model on the first 9 subjects and\nperform evaluation on the $10^{th}$ subject. The patch size is set to $32^3$.\nAccording to the overlapping step sizes, 11880, 1920,\n387, 80 patches need to be processed during inference, as shown in\nFig.~\\ref{fig:results_overlap_time}. In addition, Fig.~\\ref{fig:results_overlap_dr}\nplots the changes of segmentation performance in terms of DR. Obviously, 8 and\n16 are good choices that achieve accurate and fast segmentation results.\n\n\\begin{figure}[!t]\n\t\\centering\n\t\\includegraphics[width=0.7\\columnwidth]{figure6.png}\n\t\\caption{Changes of segmentation performance in terms of DR, with respect to\n\t\tdifferent overlapping step sizes during inference. The model is trained on the\n\t\tfirst 9 subjects and evaluated on the $10^{th}$ subject.}\n\t\\label{fig:results_overlap_dr}\n\\end{figure}\n\n\n\\begin{figure}[!t]\n\t\\centering\n\t\\includegraphics[width=0.75\\columnwidth]{figure7.png}\n\t\\caption{Changes of the number of validation patches for the $10^{th}$\n\t\tsubject, with respect to different overlapping step sizes during inference.}\n\t\\label{fig:results_overlap_time}\n\\end{figure}\n\n\\subsection{Impact of the Patch Size}\\label{sec:patch}\n\nThe patch size affects the total number of distinct training samples.\nMeanwhile, it controls the range of available global information when\nperforming segmentation for a patch. To choose the appropriate patch\nsize for the non-local U-Nets, we perform a grid search by training on the first 9\nsubjects and evaluating on the $10^{th}$ subject with the overlapping step\nsize of 8. Experiments are conducted with five different patch sizes:\n$16^3$, $24^3$, $32^3$, $40^3$, $48^3$. The results are provided in\nFig.~\\ref{fig:results_patch_dr}, where $32^3$ obtains the best\nperformance and is selected as the default setting of our model.\n\n\n\\begin{figure}[!t]\n\t\\centering\n\t\\includegraphics[width=0.75\\columnwidth]{figure8.png}\n\t\\caption{Changes of segmentation performance in terms of DR, with respect to\n\t\tdifferent patch sizes. The model is trained on the first 9 subjects and\n\t\tevaluated on the $10^{th}$ subject.}\n\t\\label{fig:results_patch_dr}\n\\end{figure}\n\n3.5 Impact of the Overlapping Step Size\n\\subsection{Impact of the Overlapping Step Size}\\label{sec:overlap}\n\nAs discussed above, a small overlapping step size\nusually results in better segmentation, due to the ensemble effect.\nHowever, with a small overlapping step size, the model has to perform\ninference for more validation patches and thus decreases the inference speed. We\nexplore the trade-off in our non-local U-Nets by setting the overlapping step sizes to 4,\n8, 16, 32, respectively. Again, we train our model on the first 9 subjects and\nperform evaluation on the $10^{th}$ subject. The patch size is set to $32^3$.\nAccording to the overlapping step sizes, 11880, 1920,\n387, 80 patches need to be processed during inference, as shown in\nFig.~\\ref{fig:results_overlap_time}. In addition, Fig.~\\ref{fig:results_overlap_dr}\nplots the changes of segmentation performance in terms of DR. Obviously, 8 and\n16 are good choices that achieve accurate and fast segmentation results.\n\n\\begin{figure}[!t]\n\t\\centering\n\t\\includegraphics[width=0.7\\columnwidth]{figure6.png}\n\t\\caption{Changes of segmentation performance in terms of DR, with respect to\n\t\tdifferent overlapping step sizes during inference. The model is trained on the\n\t\tfirst 9 subjects and evaluated on the $10^{th}$ subject.}\n\t\\label{fig:results_overlap_dr}\n\\end{figure}\n\n\n\\begin{figure}[!t]\n\t\\centering\n\t\\includegraphics[width=0.75\\columnwidth]{figure7.png}\n\t\\caption{Changes of the number of validation patches for the $10^{th}$\n\t\tsubject, with respect to different overlapping step sizes during inference.}\n\t\\label{fig:results_overlap_time}\n\\end{figure}\n\n",
        "label": "fig:results_overlap_time",
        "Descriptive_question1": "What is the number of validation patches processed during inference when the overlapping step size is 4 in figure_7?",
        "Descriptive_question2": "Which overlapping step size results in processing 80 validation patches during inference in figure_7?",
        "Reasoning_question1": "How does the number of validation patches change as the overlapping step size increases in figure_7, and what might be the reason behind this trend?",
        "Reasoning_question2": "Why might a smaller overlapping step size lead to a higher number of validation patches during inference as observed in figure_7, and how could this impact inference speed?",
        "Descriptive_answer1": "11880",
        "Descriptive_answer2": "32",
        "Reasoning_answer1": "As the overlapping step size increases, the number of validation patches decreases sharply. This trend occurs because a larger step size means that patches overlap less and fewer patches are needed to cover the entire image volume. Conversely, a smaller step size creates more overlap, requiring more patches for complete coverage.",
        "Reasoning_answer2": "A smaller overlapping step size means the sliding window moves in smaller increments, causing many patches to overlap. This increases the total number of validation patches to process during inference. Consequently, the inference speed slows down because the model must predict more patches, increasing computational workload and time."
    },
    {
        "paper_id": "1812.04103.json",
        "image_id": "figure_8",
        "image_path": [
            "/home/yz979/scratch/chengye/scimolmo/test-images-tables/dataset_imagestables/1812.04103/images/figure8.png"
        ],
        "caption": "Changes of segmentation performance in terms of DR, with respect to\n\t\tdifferent patch sizes. The model is trained on the first 9 subjects and\n\t\tevaluated on the $10^{th}$ subject.",
        "classify": "Chart",
        "section_info": "3 Results and Discussion\n\\section{Results and Discussion}\n\n\nWe perform experiments on the 3D multimodality isointense infant brain MR image segmentation task to evaluate our non-local U-Nets. The task is to perform automatic segmentation of MR images into cerebrospinal fluid~(CSF), gray matter~(GM) and white matter~(WM) regions. We first introduce the baseline model and the evaluation methods used in our experiments. Then the training and inference processes are described. We provide comparison results in terms of both effectiveness and efficiency, and conduct ablation studies to demonstrate that how each global aggregation block in our non-local U-Nets improves the performance. In addition, we explore the trade-off between the inference speed and accuracy based on different overlapping step sizes, and analyze the impact of patch size. The experimental code and dataset information have been made publicly available~\\footnote{\\url{https://github.com/divelab/Non-local-U-Nets}}.\n\n\\subsection{Experimental Setup}\n\n\nWe use CC-3D-FCN~\\cite{nie20183} as our baseline.\nCC-3D-FCN is a 3D fully convolutional network~(3D-FCN) with\nconvolution and concatenate~(CC) skip connections, which is designed\nfor 3D multimodality isointense infant brain image segmentation. It\nhas been shown to outperform traditional machine learning methods,\nsuch as FMRIB's automated segmentation\ntool~(FAST)~\\cite{zhang2001segmentation}, majority\nvoting~(MV), random forest~(RF)~\\cite{criminisi2013decision} and random forest with\nauto-context model~(LINKS)~\\cite{wang2015links}. Moreover, studies\nin~\\cite{nie20183} has showed the superiority of CC-3D-FCN to\nprevious deep learning models, like 2D, 3D\nCNNs~\\cite{zhang2015deep}, DeepMedic~\\cite{kamnitsas2017efficient},\nand the original 3D U-Net~\\cite{cciccek20163d}. Therefore, it is\nappropriate to use CC-3D-FCN as the baseline of our experiments.\nNote that our dataset is different from that in~\\cite{nie20183}.\n\n\nIn our experiments, we employ the Dice ratio~(DR) and propose the 3D modified\nHausdorff distance~(3D-MHD) as the evaluation metrics. These two\nmethods evaluate the accuracy only for binary segmentation tasks, so\nit is required to transform the 4-class segmentation map predicted by our model into\n4 binary segmentation maps for evaluation. That is, a 3D binary\nsegmentation map should be constructed for each class, where 1 denotes the voxel\nin the position belongs to the class and 0 means the opposite. In\nour experiments, we derive binary segmentation maps directly from\n4-class segmentation maps. The evaluation is performed on binary\nsegmentation maps for CSF, GM and WM.\n\n\nSpecifically, let $P$ and $L$ represent the predicted binary segmentation map for one class\nand the corresponding ground truth label, respectively. The DR is given by\n$DR=2|P \\cap L|/(|P|+|L|)$,\n\n\n\nwhere $|\\cdot|$ denotes the number of 1's in a segmentation map and $|P \\cap\nL|$ means the number of 1's shared by $P$ and $L$. Apparently, DR is a value\nin $[0,1]$ and a larger DR indicates a more accurate segmentation.\n\n\\begin{table*}[!ht]\n\t\\centering\n\t\\caption{Comparison of segmentation performance between our proposed model\n\t\tand the baseline model in terms of DR. The leave-one-subject-out\n\t\tcross-validation is used. Larger values indicate better performance.}\n\t\\label{table:results_baseline_dr}\n\t\\begin{tabular}{ l | c | c | c | c }\n\t\t\\toprule\n\t\t\\textbf{Model} & \\textbf{CSF} & \\textbf{GM} & \\textbf{WM} & \\textbf{Average} \\\\\n\t\t\\midrule\n\t\tBaseline\n\t\t& 0.9250$\\pm$0.0118\n\t\t& 0.9084$\\pm$0.0056\n\t\t& 0.8926$\\pm$0.0119\n\t\t& 0.9087$\\pm$0.0066 \\\\\n\t\tNon-local U-Net\n\t\t& \\textbf{0.9530$\\pm$0.0074}\n\t\t& \\textbf{0.9245$\\pm$0.0049}\n\t\t& \\textbf{0.9102$\\pm$0.0101}\n\t\t& \\textbf{0.9292$\\pm$0.0050} \\\\\n\t\t\\bottomrule\n\t\\end{tabular}\n\\end{table*}\n\n\\begin{table*}[!ht]\n\t\\centering\n\t\\caption{Comparison of segmentation performance between our proposed model\n\t\tand the baseline model in terms of 3D-MHD. The leave-one-subject-out\n\t\tcross-validation is used. Smaller values indicate better performance. Note that 3D-MHD gives different results from MHD.}\n\t\\label{table:results_baseline_mhd}\n\t\\begin{tabular}{ l | c | c | c | c }\n\t\t\\toprule\n\t\t\\textbf{Model} & \\textbf{CSF} & \\textbf{GM} & \\textbf{WM} & \\textbf{Average} \\\\\n\t\t\\midrule\n\t\tBaseline\n\t\t& 0.3417$\\pm$0.0245\n\t\t& 0.6537$\\pm$0.0483\n\t\t& 0.4817$\\pm$0.0454\n\t\t& 0.4924$\\pm$0.0345 \\\\\n\t\tNon-local U-Net\n\t\t& \\textbf{0.2554$\\pm$0.0207}\n\t\t& \\textbf{0.5950$\\pm$0.0428}\n\t\t& \\textbf{0.4454$\\pm$0.0040}\n\t\t& \\textbf{0.4319$\\pm$0.0313} \\\\\n\t\t\\bottomrule\n\t\\end{tabular}\n\\end{table*}\n\n\\begin{table*}[!t]\n\t\\centering\n\t\\caption{Comparison of segmentation performance on the 13 testing subjects of iSeg-2017 between our proposed model\n\t\tand the baseline model in terms of DR. Larger values indicate better performance.}\n\t\\label{table:iseg_results}\n\t\\begin{tabular}{ l | c | c | c }\n\t\t\\toprule\n\t\t\\textbf{Model} & \\textbf{CSF} & \\textbf{GM} & \\textbf{WM} \\\\\n\t\t\\midrule\n\t\tBaseline\n\t\t& 0.9324$\\pm$0.0067\n\t\t& 0.9146$\\pm$0.0074\n\t\t& 0.8974$\\pm$0.0123 \\\\\n\t\tNon-local U-Net\n\t\t& \\textbf{0.9557$\\pm$0.0060}\n\t\t& \\textbf{0.9219$\\pm$0.0089}\n\t\t& \\textbf{0.9044$\\pm$0.0153} \\\\\n\t\t\\bottomrule\n\t\\end{tabular}\n\\end{table*}\n\n\nThe modified Hausdorff distance~(MHD)~\\cite{dubuisson1994modified} is\ndesigned to compute the similarity between two objects. Here, an object is a\nset of points where a point is represented by a vector. Specifically, given\ntwo sets of vectors $A$ and $B$, MHD is computed by\n$MHD=\\max(d(A,B),d(B,A))$,\n\n\n\nwhere the distance between two sets is defined as\n$d(A,B)=1/|A|\\sum_{a \\in A}{d(a,B)}$,\n\n\n\nand the distance between a vector and a set is defined as\n$d(a,B)=\\min_{b \\in B}||a-b||$.\n\n\n\nPrevious studies~\\cite{wang2015links,zhang2015deep,nie20183}\napplied MHD for evaluation by treating a 3D $D \\times H \\times W$\nmap as $H \\times W$ $D$-dimensional vectors. However, there are two\nmore different ways to vectorize the 3D map, depending on the\ndirection of forming vectors, \\emph{i.e.,} $D \\times H$\n$W$-dimensional vectors and $D \\times W$ $H$-dimensional vectors.\nEach vectorization leads to different evaluation results by MHD. To\nmake it a direction-independent evaluation metric as DR, we define\n3D-MHD, which computes the averaged MHD based on the three different\nvectorizations. A smaller 3D-MHD indicates a\nhigher segmentation accuracy.\n\n\\subsection{Training and Inference Strategies}\n\n\nOur proposed non-local U-Nets apply Dropout~\\cite{srivastava2014dropout} with\na rate of 0.5 in each global aggregation block and the output block\nbefore the final $1 \\times 1 \\times 1$ convolution. A weight\ndecay~\\cite{krogh1992simple} with a rate of $2e-6$ is also employed.\nTo train the model, we use randomly cropped small patches. In this\nway, we obtain sufficient training data and the requirement on\nmemory is reduced. No extra data augmentation is needed. The\nexperimental results below suggest that patches\nwith a size of $32^3$ leads to the best\nperformance. The batch size is set to 5. The Adam\noptimizer~\\cite{kingma2014adam} with a learning rate of 0.001 is\nemployed to perform the gradient descent algorithm.\n\n\nIn the inference process, following~\\cite{nie20183}, we extract\npatches with the same size as that used in training. For example, to\ngenerate $32^3$ patches for inference, we slide a\nwindow of size $32^3$ through the original image\nwith a constant overlapping step size. The overlapping step size\nmust be smaller than or equal to the patch size, in order to\nguarantee that extracted patches cover the whole image.\nConsequently, prediction for all these patches provides segmentation\nprobability results for every voxel in the original image. For\nvoxels that receive multiple results due to overlapping, we average\nthem to produce the final prediction. The overlapping step size is\nan important hyper-parameter affecting the inference speed and the\nsegmentation accuracy. A smaller overlapping step size results in\nbetter accuracy, but increases the inference time as more patches\nare generated. We explore the trade-off in our experiments.\n\n\\subsection{Comparison with the Baseline}\\label{sec:baseline}\n\n\nWe compare our non-local U-Nets with the baseline on our dataset.\nFollowing~\\cite{nie20183}, the patch size is set to $32^3$\nand the overlapping step size for inference is set to $8$. To remove the\nbias of different subjects, the leave-one-subject-out cross-validation is\nused for evaluating segmentation performance. That is, for 10 subjects in our\ndataset, we train and evaluate models 10 times correspondingly. Each time one\nof the 10 subjects is left out for validation and the other 9 subjects are\nused for training. The mean and standard deviation of segmentation\nperformance of the 10 runs are reported.\n\n\nTables~\\ref{table:results_baseline_dr}\nand~\\ref{table:results_baseline_mhd} provide the experimental\nresults. In terms of both evaluation metrics, our non-local U-Nets achieve\nsignificant improvements over the baseline model. Due to the small\nvariances of the results, we focus on one of the 10 runs for\nvisualization and ablation studies, where the models are trained on the\nfirst 9 subjects and evaluated on the $10^{th}$ subject. A\nvisualization of the segmentation results in this run is given by\nFig.~\\ref{fig:results_visual}. By comparing the areas in red\ncircles, we can see that our model is capable of catching more\ndetails than the baseline model. We also visualize the training\nprocesses to illustrate the superiority of our model.\nFig.~\\ref{fig:results_training} shows the training and validation\ncurves in this run of our model and the baseline model,\nrespectively. Clearly, our model converges faster to a lower\ntraining loss. In addition, according to the better validation\nresults, our model does not suffer from over-fitting.\n\n\nTo further show the efficiency of our proposed model, we compare the\nnumber of parameters as reported in Table~\\ref{table:num_params}.\nOur model reduces $28\\%$ parameters compared to CC-3D-FCN and\nachieves better performance. A comparison of inference time is also\nprovided in Table~\\ref{table:infer_time}. The settings of our device\nare - GPU: Nvidia Titan Xp 12GB; CPU: Intel Xeon E5-2620v4 2.10GHz;\nOS: Ubuntu 16.04.3 LTS.\n\n\nSince our data has been used as the training data in the iSeg-2017\nchallenge, we also compare the\nresults evaluated on the 13 testing subjects in\nTable~\\ref{table:iseg_results}. According to the leader board, our model\nachieves one of the top performances. Results in terms of DR are reported since\nit is the only shared evaluation metric.\n\n\\begin{table}[!t]\n\t\\centering\n\t\\caption{Comparison of the number of parameters between our proposed model\n\t\tand the baseline model.}\n\t\\label{table:num_params}\n\t\\begin{tabular}{ l | c }\n\t\t\\toprule\n\t\t\\textbf{Model} & Number of Parameters \\\\\n\t\t\\midrule\n\t\tBaseline  & 2,534,276 \\\\\n\t\tNon-local U-Net & \\textbf{1,821,124} \\\\\n\t\t\\bottomrule\n\n\n\n\n\n\n\t\\end{tabular}\n\\end{table}\n\n\\begin{table}[!t]\n\t\\centering\n\t\\caption{Comparison of inference time between our proposed model and the baseline model. The leave-one-subject-out cross-validation is used. The patch size is set to $32^3$ and the overlapping step size for inference is set to $8$.}\n\t\\label{table:infer_time}\n\t\\begin{tabular}{ l | c }\n\t\t\\toprule\n\t\t\\textbf{Model} & Inference Time (min) \\\\\n\t\t\\midrule\n\t\tBaseline  & 3.85$\\pm$0.15 \\\\\n\t\tNon-local U-Net  & \\textbf{3.06$\\pm$0.12} \\\\\n\t\t\\bottomrule\n\t\\end{tabular}\n\\end{table}\n\n\\begin{figure}[!t]\n\t\\centering\n\t\\includegraphics[width=0.95\\columnwidth]{figure4.png}\n\t\\caption{Visualization of the segmentation results on the $10^{th}$ subject\n\t\tby our proposed model and the baseline model. Both models are trained on the\n\t\tfirst 9 subjects. The first column shows the original segmentation maps. The\n\t\tsecond, third and fourth columns show the binary segmentation maps for CSF,\n\t\tGM and WM, respectively.}\n\t\\label{fig:results_visual}\n\\end{figure}\n\n\\begin{figure}[!t]\n\t\\centering\n\t\\includegraphics[width=0.75\\columnwidth]{figure5.png}\n\t\\caption{Comparison of training processes and validation results between our proposed model and the\n\t\tbaseline model when training on the first 9 subjects and using the $10^{th}$ subject for validation.}\n\t\\label{fig:results_training}\n\\end{figure}\n\n\\begin{table}[!t]\n\t\\centering\n\t\\caption{Ablation study by comparing segmentation performance between\n\t\tdifferent models in terms of DR. All models are trained on the first 9\n\t\tsubjects and evaluated on the $10^{th}$ subject. Larger values indicate\n\t\tbetter performance. Details of models are provided in the text.}\n\t\\label{table:results_ablation_dr}\n\t\\resizebox{.95\\columnwidth}{!}{\n\t\\begin{tabular}{ l | c | c | c | c }\n\t\t\\toprule\n\t\t\\textbf{Model} & \\textbf{CSF} & \\textbf{GM} & \\textbf{WM} & \\textbf{Average} \\\\\n\t\t\\midrule\n\n\t\tModel1       & \\textbf{0.9585} & 0.9099 & 0.8625 & 0.9103 \\\\\n\t\tModel2       & 0.9568 & 0.9172 & 0.8728 & 0.9156 \\\\\n\t\tModel3       & 0.9576 & 0.9198 & 0.8749 & 0.9174 \\\\\n\t\tModel4       & 0.9578 & 0.9210 & 0.8769 & 0.9186 \\\\\n\t\tModel5       & 0.9554 & 0.9225 & 0.8804 & 0.9194 \\\\\n\t\tNon-local U-Net    & 0.9572 & \\textbf{0.9278} & \\textbf{0.8867} & \\textbf{0.9239} \\\\\n\t\t\\bottomrule\n\t\\end{tabular}}\n\\end{table}\n\n\\begin{table}[!t]\n\t\\centering\n\t\\caption{Ablation study by comparing segmentation performance between\n\t\tdifferent models in terms of 3D-MHD. All models are trained on the first 9\n\t\tsubjects and evaluated on the $10^{th}$ subject. Smaller values indicate\n\t\tbetter performance. Note that 3D-MHD gives different results from MHD. Details of models are provided in\n\t\tthe text.}\n\t\\label{table:results_ablation_mhd}\n\t\\resizebox{.95\\columnwidth}{!}{\n\t\\begin{tabular}{ l | c | c | c | c }\n\t\t\\toprule\n\t\t\\textbf{Model} & \\textbf{CSF} & \\textbf{GM} & \\textbf{WM} & \\textbf{Average} \\\\\n\t\t\\midrule\n\n\t\tModel1       & \\textbf{0.2363} & 0.6277 & 0.4705 & 0.4448 \\\\\n\t\tModel2       & 0.2404 & 0.6052 & 0.4480 & 0.4312 \\\\\n\t\tModel3       & 0.2392 & 0.5993 & 0.4429 & 0.4271 \\\\\n\t\tModel4       & 0.2397 & 0.5926 & 0.4336 & 0.4220 \\\\\n\t\tModel5       & 0.2444 & 0.5901 & 0.4288 & 0.4211 \\\\\n\t\tNon-local U-Net    & 0.2477 & \\textbf{0.5692} & \\textbf{0.4062} & \\textbf{0.4077} \\\\\n\t\t\\bottomrule\n\t\\end{tabular}}\n\\end{table}\n\n\\subsection{Ablation Studies of Different Modules}\\label{sec:ablation}\n\nWe perform ablation studies to show the effectiveness of each part of our\nnon-local U-Nets. Specifically, we compare the following models:\n\n\\textbf{Model1} is a 3D U-Net without short-range residual connections.\nDown-sampling and up-sampling are implemented by convolutions and\ndeconvolutions with a stride of 2, respectively. The bottom block is simply a\nconvolutional layer. Note that the baseline model, CC-3D-FCN, has showed improved performance over 3D U-Net~\\cite{nie20183}. However, the original 3D U-Net was not designed for this task~\\cite{cciccek20163d}. In our experiments, we appropriately set the hyperparameters of 3D U-Net and achieve better performance.\n\n\\textbf{Model2} is Model1 with short-range residual connections, \\emph{i.e.},\nthe blocks in Fig.~\\ref{fig:residual}(a) and (b) are applied. The bottom\nblock and up-sampling blocks are the same as those in Model1.\n\n\\textbf{Model3} replaces the first up-sampling block in Model2 with the block\nin Fig.~\\ref{fig:residual}(d).\n\n\\textbf{Model4} replaces both up-sampling blocks in Model2 with the block in\nFig.~\\ref{fig:residual}(d).\n\n\\textbf{Model5} replaces the bottom block in Model2 with the block in\nFig.~\\ref{fig:residual}(c).\n\nAll models are trained on the first 9 subjects. We report the segmentation\nperformance on the $10^{th}$ subject in Table~\\ref{table:results_ablation_dr}\nand Table~\\ref{table:results_ablation_mhd}. The results demonstrate how different\nglobal aggregation blocks in our non-local U-Nets improve the performance.\n\n\\subsection{Impact of the Overlapping Step Size}\\label{sec:overlap}\n\nAs discussed above, a small overlapping step size\nusually results in better segmentation, due to the ensemble effect.\nHowever, with a small overlapping step size, the model has to perform\ninference for more validation patches and thus decreases the inference speed. We\nexplore the trade-off in our non-local U-Nets by setting the overlapping step sizes to 4,\n8, 16, 32, respectively. Again, we train our model on the first 9 subjects and\nperform evaluation on the $10^{th}$ subject. The patch size is set to $32^3$.\nAccording to the overlapping step sizes, 11880, 1920,\n387, 80 patches need to be processed during inference, as shown in\nFig.~\\ref{fig:results_overlap_time}. In addition, Fig.~\\ref{fig:results_overlap_dr}\nplots the changes of segmentation performance in terms of DR. Obviously, 8 and\n16 are good choices that achieve accurate and fast segmentation results.\n\n\\begin{figure}[!t]\n\t\\centering\n\t\\includegraphics[width=0.7\\columnwidth]{figure6.png}\n\t\\caption{Changes of segmentation performance in terms of DR, with respect to\n\t\tdifferent overlapping step sizes during inference. The model is trained on the\n\t\tfirst 9 subjects and evaluated on the $10^{th}$ subject.}\n\t\\label{fig:results_overlap_dr}\n\\end{figure}\n\n\n\\begin{figure}[!t]\n\t\\centering\n\t\\includegraphics[width=0.75\\columnwidth]{figure7.png}\n\t\\caption{Changes of the number of validation patches for the $10^{th}$\n\t\tsubject, with respect to different overlapping step sizes during inference.}\n\t\\label{fig:results_overlap_time}\n\\end{figure}\n\n\\subsection{Impact of the Patch Size}\\label{sec:patch}\n\nThe patch size affects the total number of distinct training samples.\nMeanwhile, it controls the range of available global information when\nperforming segmentation for a patch. To choose the appropriate patch\nsize for the non-local U-Nets, we perform a grid search by training on the first 9\nsubjects and evaluating on the $10^{th}$ subject with the overlapping step\nsize of 8. Experiments are conducted with five different patch sizes:\n$16^3$, $24^3$, $32^3$, $40^3$, $48^3$. The results are provided in\nFig.~\\ref{fig:results_patch_dr}, where $32^3$ obtains the best\nperformance and is selected as the default setting of our model.\n\n\n\\begin{figure}[!t]\n\t\\centering\n\t\\includegraphics[width=0.75\\columnwidth]{figure8.png}\n\t\\caption{Changes of segmentation performance in terms of DR, with respect to\n\t\tdifferent patch sizes. The model is trained on the first 9 subjects and\n\t\tevaluated on the $10^{th}$ subject.}\n\t\\label{fig:results_patch_dr}\n\\end{figure}\n\n3.6 Impact of the Patch Size\n\\subsection{Impact of the Patch Size}\\label{sec:patch}\n\nThe patch size affects the total number of distinct training samples.\nMeanwhile, it controls the range of available global information when\nperforming segmentation for a patch. To choose the appropriate patch\nsize for the non-local U-Nets, we perform a grid search by training on the first 9\nsubjects and evaluating on the $10^{th}$ subject with the overlapping step\nsize of 8. Experiments are conducted with five different patch sizes:\n$16^3$, $24^3$, $32^3$, $40^3$, $48^3$. The results are provided in\nFig.~\\ref{fig:results_patch_dr}, where $32^3$ obtains the best\nperformance and is selected as the default setting of our model.\n\n\n\\begin{figure}[!t]\n\t\\centering\n\t\\includegraphics[width=0.75\\columnwidth]{figure8.png}\n\t\\caption{Changes of segmentation performance in terms of DR, with respect to\n\t\tdifferent patch sizes. The model is trained on the first 9 subjects and\n\t\tevaluated on the $10^{th}$ subject.}\n\t\\label{fig:results_patch_dr}\n\\end{figure}\n\n",
        "label": "fig:results_patch_dr",
        "Descriptive_question1": "What patch size achieves the best segmentation performance in terms of DR in figure_8?",
        "Descriptive_question2": "How many patch sizes are compared in figure_8 for segmentation performance evaluation?",
        "Reasoning_question1": "Why might the patch size of 32^3 have resulted in the best segmentation performance in terms of DR in figure_8?",
        "Reasoning_question2": "How does varying patch sizes in figure_8 impact the balance between available global information and the number of distinct training samples?",
        "Descriptive_answer1": "32^3",
        "Descriptive_answer2": "5",
        "Reasoning_answer1": "The patch size of 32^3 likely results in the best segmentation performance because it provides an optimal balance between capturing sufficient global context within the patch and maintaining a reasonable number of distinct training samples. Smaller patch sizes may not encompass enough volumetric information for accurate segmentation, while larger patches reduce the number of distinct samples and may lead to overfitting, thereby compromising performance.",
        "Reasoning_answer2": "Varying the patch size affects the trade-off between the availability of global information and the number of distinct training samples during training. Larger patches provide more global spatial context which can improve segmentation accuracy, but result in fewer training samples and higher computational cost. Conversely, smaller patches increase the number of training samples and diversity but capture less global information, potentially reducing segmentation accuracy. Thus, choosing an intermediate patch size like 32^3 balances these factors, resulting in better overall performance."
    },
    {
        "paper_id": "1812.04424.json",
        "image_id": "figure_7",
        "image_path": [
            "/home/yz979/scratch/chengye/scimolmo/test-images-tables/dataset_imagestables/1812.04424/images/plot_dj_int_neg.eps"
        ],
        "caption": "Holographic entanglement negativity $\\mathcal{E}$ vs time $t$ is plotted for two disjoint intervals of lengths $\\ell_1=50$, $\\ell_2=200$ with the seperation length $l_s=30$.",
        "classify": "Chart",
        "section_info": "5 Holographic entanglement negativity\n\\section{Holographic entanglement negativity}\\label{henaq}\n\nIn this section we present our holographic construction for the time evolution of the entanglement negativity of two disjoint and adjacent intervals in a $CFT_{1+1}$ following a global quench as described in \\cite{Coser:2014gsa}. Motivated by the holographic construction for the time evolution of the entanglement entropy in \\cite{Hartman:2013qma} reviewed above, we begin by computing the holographic entanglement negativity for the mixed states under consideration from the bulk dual eternal black hole geometry. The corresponding holographic entanglement negativity following a global quench may then be obtained by considering half of the result for the bulk dual eternal black hole geometry. \n\nIt interesting to note that the four point twist correlator in eq. \\eqref{4pttc} admits a factorization with the non universal function ${\\cal F}(\\{\\eta_{j,k}\\})\\to 1$ for the cross ratios $\\eta_{i,j}\\to 1$ and $\\eta_{i,j}\\to 0$, as follows\n\\begin{equation}\\label{factorization_four_pt}\n\\begin{aligned}\n&\\langle\\mathcal{T}_{n}(w_1)\\overline{\\mathcal{T}}_{n}(w_2)\\overline{\\mathcal{T}}_{n}(w_3)\\mathcal{T}_{n}(w_4)\\rangle_{\\rm strip}=\\\\\n&\\frac{\\langle\\mathcal{T}_{n}(w_1)\\overline{\\mathcal{T}}_{n}(w_4)\\rangle_{\\rm strip}~\\langle\\mathcal{T}_{n}(w_2)\\overline{\\mathcal{T}}_{n}(w_3)\\rangle_{\\rm strip}~\\Big(\\langle\\, \\mathcal{T}^2_n (w_1) \n\\bar{\\mathcal{T}}^2_n(w_3) \\,\\rangle_{\\rm strip}~\\langle\\, \\mathcal{T}^2_n (w_2) \n\\bar{\\mathcal{T}}^2_n(w_4) \\,\\rangle_{\\rm strip}\\Big)^\\frac{1}{2}}{\\langle\\mathcal{T}_{n}(w_1)\\overline{\\mathcal{T}}_{n}(w_3)\\rangle_{\\rm strip}~\\langle\\mathcal{T}_{n}(w_2)\\overline{\\mathcal{T}}_{n}(w_4)\\rangle_{\\rm strip}~\\Big(\\langle\\, \\mathcal{T}^2_n (w_1) \n\\bar{\\mathcal{T}}^2_n(w_4) \\,\\rangle_{\\rm strip}~\\langle\\, \\mathcal{T}^2_n (w_2) \n\\bar{\\mathcal{T}}^2_n(w_3) \\,\\rangle_{\\rm strip} \\Big)^\\frac{1}{2}},\n\\end{aligned}\n\\end{equation}\nwhere we have employed eq. \\eqref{renyi corr strip} and the following two point twist correlator \n\\begin{equation}\n\\label{twist squared uhp N=1}\n\\langle  \\mathcal{T}^2_n (w_1) \n\\bar{\\mathcal{T}}^2_n(w_2) \\rangle_{\\rm strip} \n=\\left( \\frac{\\pi}{2\\tau_0} \\right)^{2\\Delta^{(2)}_n}\n\\frac{c^{(2)}_{n}}{|(z_1 - \\bar{z}_1)(z_2 - \\bar{z}_2) \\,\\eta_{1,2}|^{\\Delta^{(2)}_n}}{\\cal F}(\\eta_{1,2})\\,.\n\\end{equation}\nThe entanglement negativity for two disjoint intervals may then be obtained by utilizing eqs. \\eqref{en replica} and \\eqref{factorization_four_pt} which leads to following expression\n\\begin{equation}\n\\begin{aligned}\n\\label{eq_holo_ent_neg_dj_int_holo_ent_enpy}\n{\\cal E} = &\\frac{ 3 }{ 4 }\n\\left ( S_{ A_1 \\cup A_s } + S_{ A_s \\cup A_2 }\n- S_{ A_1 \\cup A_2 \\cup A_s } - S_{ A_s } \\right ),\n\\end{aligned}\n\\end{equation}\nwhere $S_{\\gamma}~(\\gamma \\in  A_1 \\cup A_s,~A_s \\cup A_2,~A_1 \\cup A_2 \\cup A_s,~ A_s)$ is the entanglement entropy of an interval $\\gamma$  as given in eq. \\eqref{SA one interval}.\n\nNote that a similar factorization also holds for the three point twist correlator defined in eq. \\eqref{en 3pt fn} which is given as\n\n\\begin{equation}\\label{factorization_three_pt}\n\\begin{aligned}\n\\langle \\mathcal{T}_n(w_1) \\bar{\\mathcal{T}}^2_n(w_2) \\mathcal{T}_n(w_3) \\rangle_{\\rm strip}\\,=\n&\\Bigg(\\frac{\\langle\\, \\mathcal{T}^2_n (w_1) \n\\bar{\\mathcal{T}}^2_n(w_2) \\,\\rangle_{\\rm strip}~\\langle\\, \\mathcal{T}^2_n (w_2) \n\\bar{\\mathcal{T}}^2_n(w_3) \\,\\rangle_{\\rm strip} \\langle\\mathcal{T}_{n}(w_1)\\overline{\\mathcal{T}}_{n}(w_3)\\rangle_{\\rm strip}^2 }{\\langle\\, \\mathcal{T}^2_n (w_1) \n\\bar{\\mathcal{T}}^2_n(w_3) \\,\\rangle_{\\rm strip} }\\Bigg)^{1/2}.\n\\end{aligned}\n\\end{equation}\nThe entanglement negativity for two adjacent intervals may then be obtained by utilizing eqs. \\eqref{en replica} and \\eqref{factorization_three_pt} as follows\n\\begin{equation}\\label{heecon}\n\\mathcal{E} =  \\frac{3}{4}(S_{A_1}+S_{A_2}-S_{A_1\\cup A_2}).\n\\end{equation}\n\nIn what follows we utilize eqs. \\eqref{eq_holo_ent_neg_dj_int_holo_ent_enpy} and \\eqref{heecon} to establish the holographic entanglement negativity and its time evolution for two disjoint and adjacent intervals in a $CFT_{1+1}$ after a global quench employing the prescription for the holographic entanglement entropy described in \\cite{Hartman:2013qma}.\n\n\n\n\\subsection{Two disjoint intervals}\\label{hendjagq}\n\nWe first consider the case of two disjoint intervals $A_1$ and $A_2$ of lengths $\\ell_1$ and $\\ell_2$ with an interval $A_s$ of length $\\ell_s$ separating them. As earlier for the dual eternal black hole geometry it is required to consider the intervals $A_1$, $A_2$ and $A_s$ in both the Rindler wedges $I$ and $III$ of the Minkowski diamond in Fig. \\ref{coord_disjoint_intervals}(a). The end points of the intervals $A_1$, $A_2$ and $A_s$ in the Rindler wedges $I$ and $III$ projected on the $x_1$-$z$ plane of the Poincar\\'e coordinates are denoted as ($a,b$), ($c,d$), ($b,c$) and ($a',b'$), ($c',d'$), ($b',c'$) respectively which is depicted in Fig. \\ref{coord_disjoint_intervals}(b).\n\n\\begin{figure}[ht]\n\\centering\n\\includegraphics[width=\\linewidth]{coord_disjoint_intervals.eps}\n\\caption{(a) Two disjoint intervals $A_1$ (blue) and $A_2$ (red) on the Rindler wedges $I$ and $III$ respectively of the Minkowski diamond. (b) These two intervals are projected on the $x_1$-$z$ plane of the Poincar\\'e coordinates.}\\label{coord_disjoint_intervals}\n\\end{figure}\n\n\nIt is now possible to describe the time evolution of the holographic entanglement negativity for the mixed state configuration of the two disjoint intervals from the bulk dual eternal black hole geometry by utilizing  eqs. \\eqref{eq_holo_ent_neg_dj_int_holo_ent_enpy}, \\eqref{earlytimes} and \\eqref{latetimes}. It is observed that distinct sets of geodesics contribute to the holographic entanglement negativity for various values of time $t$ relative to the lengths of the intervals $\\ell_1$, $\\ell_2$ and $\\ell_s$ . In what follows we consider these significant limits and describe the relevant geodesic structures leading to the holographic entanglement negativity for each of these scenarios.\n\n\n\\subsubsection{$t<\\ell_s/2<\\ell_1/2<\\ell_2/2$}\n\nIn the limit of early times all the relevant geodesics pass through the interior region of the BTZ black string connecting one Rindler wedge $I$ to the other $III$,  depicted by dashed curves in the Fig. \\ref{neggeodjfin}(a). The corresponding holographic entanglement negativity is obtained by substituting the holographic entanglement entropy for the early time limit given in  eq. \\eqref{earlytimes} in the eq. \\eqref{eq_holo_ent_neg_dj_int_holo_ent_enpy}, which vanishes due to mutual cancellations as follows\n\\begin{equation}\\label{djlima}\n\\begin{aligned}\n\\mathcal{E} = &\\frac{3}{16G^{(3)}_N}\\left(\\underbrace{\\mathcal{L}_{d'd}+\\mathcal{L}_{b'b}}_{S_{ A_1 \\cup A_s }}+\\underbrace{\\mathcal{L}_{c'c}+\\mathcal{L}_{a'a}}_{S_{ A_s \\cup A_2 }}   -\\underbrace{\\mathcal{L}_{d'd}-\\mathcal{L}_{a'a}}_{S_{ A_1 \\cup A_2 \\cup A_s }}-  \\underbrace{\\mathcal{L}_{c'c}-\\mathcal{L}_{b'b}}_{S_{ A_s }}\\right),\\\\\n=&0.\n\\end{aligned}\n\\end{equation}\n\n\\subsubsection{$\\ell_1/2<\\ell_2/2<t<(\\ell_1+\\ell_s)/2$}\nIn this limit the geodesics (solid curves) that joins $c'$ and $b'$ and $c$ and $b$ are confined to the exterior region, and rest of the geodesics (dashed curves) pass through the interior region of the BTZ black string as depicted in Fig. \\ref{neggeodjfin}(b). The holographic entanglement negativity in this case may then be obtained by utilizing both the eqs. \\eqref{earlytimes} and \\eqref{latetimes} in the eq. \\eqref{eq_holo_ent_neg_dj_int_holo_ent_enpy}, which leads to the following expression\n\\begin{equation}\\label{djlimb}\n\\begin{aligned}\n\\mathcal{E} = &\\frac{3}{16G^{(3)}_N}\\left(\\mathcal{L}_{d'd}+\\mathcal{L}_{b'b}+\\mathcal{L}_{c'c}+\\mathcal{L}_{a'a}-\\mathcal{L}_{d'd}-\\mathcal{L}_{a'a}-\\mathcal{L}_{c'b'}-\\mathcal{L}_{cb}\\right),\\\\\n=&\\frac{3}{16G^{(3)}_N}\\left(\\mathcal{L}_{b'b}+\\mathcal{L}_{c'c}-\\mathcal{L}_{c'b'}-\\mathcal{L}_{cb}\\right),\\\\\n=&\\frac{\\pi c}{\\beta}\\left(t-\\frac{l_s}{2}\\right).\n\\end{aligned}\n\\end{equation}\n\n\\begin{figure}[h]\n\\centering\n\\includegraphics[width=\\linewidth]{neggeodjfin.eps}\n\\caption{Geodesic configurations for differnt limits of the time relative to the lengths of the intervals. The geodesics depicted by dashed lines pass through the interior region of the BTZ black string connecting the two Rindler wedges $I$ and $III$. The geodesics depicted by solid lines are confined to the exterior region of the BTZ black string outside the horizon. }\\label{neggeodjfin}\n\\end{figure}\n\n\n\n\\subsubsection{$\\ell_2/2<(\\ell_1+\\ell_s)/2<t<(\\ell_2+\\ell_s)/2$}\n\nIn this case the geodesics (solid curves) between the points $d'$ to $b'$, $d$ to $b$, $c'$ to $b'$ and $c$ to $b$ are confined in the exterior region  while the other geodesics (dashed curves) between the points $d'$ to $d$ and $c'$ to $c$ pass through the interior region of the BTZ black string as depicted in Fig. \\ref{neggeodjfin}(c). The holographic entanglement negativity in this limit is obtained by utilizing eqs. \\eqref{earlytimes} and \\eqref{latetimes} in the eq. \\eqref{eq_holo_ent_neg_dj_int_holo_ent_enpy} leading to the following expression\n\\begin{equation}\\label{djlimc}\n\\begin{aligned}\n\\mathcal{E} = &\\frac{3}{16G^{(3)}_N}\\left(\\mathcal{L}_{d'b'}+\\mathcal{L}_{db}+\\mathcal{L}_{c'c}+\\mathcal{L}_{a'a}-\\mathcal{L}_{d'd}-\\mathcal{L}_{a'a}-\\mathcal{L}_{c'b'}-\\mathcal{L}_{cb}\\right),\\\\\n=&\\frac{3}{16G^{(3)}_N}\\left(\\mathcal{L}_{d'b'}+\\mathcal{L}_{db}+\\mathcal{L}_{c'c}-\\mathcal{L}_{d'd}-\\mathcal{L}_{c'b'}-\\mathcal{L}_{cb}\\right),\\\\\n=&\\frac{\\pi c\\ell_1}{2\\beta}.\n\\end{aligned}\n\\end{equation}\n\n\\subsubsection{$(\\ell_1+\\ell_s)/2<(\\ell_2+\\ell_s)/2<t<(\\ell_1+\\ell_2+\\ell_s)/2$}\n\nIn this limit the geodesics (dashed curves) between the points $d'$ to $d$ and $a'$ to $a$ pass through the interior region whereas rest of the geodesics (solid curves) are confined to the exterior region of the BTZ black string. The holographic entanglement negativity in this case may be obtained by utilizing eqs. \\eqref{earlytimes} and \\eqref{latetimes} in the eq. \\eqref{eq_holo_ent_neg_dj_int_holo_ent_enpy}, as follows\n\\begin{equation}\\label{djlimd}\n\\begin{aligned}\n\\mathcal{E} = &\\frac{3}{16G^{(3)}_N}\\left(\\mathcal{L}_{d'b'}+\\mathcal{L}_{db}+\\mathcal{L}_{c'a'}+\\mathcal{L}_{ca}-\\mathcal{L}_{d'd}-\\mathcal{L}_{a'a}-\\mathcal{L}_{c'b'}-\\mathcal{L}_{cb}\\right),\\\\\n=&\\frac{\\pi c}{\\beta}\\left(\\frac{\\ell_1+\\ell_2+\\ell_s}{2}-t \\right)\\,.\n\\end{aligned}\n\\end{equation}\n\n\\subsubsection{$(\\ell_1+\\ell_s)/2<(\\ell_2+\\ell_s)/2<(\\ell_1+\\ell_2+\\ell_s)/2<t$}\n\n\nIn the late time limit all the geodesics (solid curves) are confined to the exterior region  of the BTZ black string as shown in Fig. \\ref{neggeodjfin}(e). Consequently, the holographic entanglement negativity is obtained by utilizing  eqs. \\eqref{earlytimes} and \\eqref{latetimes} in the eq. \\eqref{eq_holo_ent_neg_dj_int_holo_ent_enpy} which vanishes due to mutual cancellations as follows\n\\begin{equation}\\label{djlime}\n\\begin{aligned}\n\\mathcal{E} = &\\frac{3}{16G^{(3)}_N}\\left(\\mathcal{L}_{d'b'}+\\mathcal{L}_{db}+\\mathcal{L}_{c'a'}+\\mathcal{L}_{ca}-\\mathcal{L}_{d'a'}-\\mathcal{L}_{da}-\\mathcal{L}_{c'b'}-\\mathcal{L}_{cb}\\right),\\\\\n=&0.\n\\end{aligned}\n\\end{equation}\n\nIt is now possible to present a holographic description for the time evolution of the entanglement negativity for mixed state configurations of two disjoint intervals in a $CFT_{1+1}$ following a global quench. As described earlier the corresponding bulk dual geometry is the single sided black hole obtained by slicing the eternal black hole geometry in half with an ETW brane. So the time evolution of the entanglement negativity for the mixed state in question following a global quench may be obtained by considering half of the corresponding results for the eternal black hole geometry computed above.\nQuite significantly, the time evolution of the holographic entanglement negativity obtained from our construction in this case matches exactly with the corresponding replica technique results for a $CFT_{1+1}$ as described in section (\\ref{ent_neg_glo_quench_dj_int}). Once again this is a robust consistency check for our holographic construction.\n\nIn  Fig. \\ref{neg_vs_t_dj_int} we have plotted the time evolution of the holographic entanglement negativity for the mixed state in question in a $CFT_{1+1}$ following a global quench.  It is observed from Fig. \\ref{neg_vs_t_dj_int} that the holographic entanglement negativity for early times is zero and grows linearly to attain a maximum value at an intermediate time after which remains constant upto certain value of time. Beyond this value of time the holographic entanglement negativity decreases linearly and vanishes for late times. Note that this behavior for the entanglement negativity of the mixed state in question consistently matches with that described for a $CFT_{1+1}$s as described in \\cite{Coser:2014gsa}.\n\n\\begin{figure}[H]\n\\centering\n\\includegraphics[width=.5\\linewidth]{plot_dj_int_neg.eps}\n\\caption{Holographic entanglement negativity $\\mathcal{E}$ vs time $t$ is plotted for two disjoint intervals of lengths $\\ell_1=50$, $\\ell_2=200$ with the seperation length $l_s=30$.}\\label{neg_vs_t_dj_int}\n\\end{figure}\n\n\n\n\n\n\n\\subsection{Two adjacent intervals}\\label{henadjagq}\n\n\nWe now consider the time evolution of the entanglement negativity for the case of two adjacent intervals $A_1$ and $A_2$ of lengths $\\ell_1$ and $\\ell_2$ respectively. Proceeding similarly as in the case for the two disjoint intervals described above, we consider the two adjacent intervals $A_1$ and $A_2$ in both the Rindler wedges $I$ and $III$ of the Minkowski diamond as shown in Fig. \\ref{coord_adj_intervals}(a). The end points of the intervals $A_1$ and $A_2$ in the Rindler wedges projected on the $x_1 -z$ plane of the Poincaré coordinates are denoted as ($a, b$), ($b, c$) and ($a', b'$), ($b', c'$)  respectively which is depicted in Fig. \\ref{coord_adj_intervals}(b).\n\n\\begin{figure}[h]\n\\centering\n\\includegraphics[width=\\linewidth]{coord_adj_intervals.eps}\n\\caption{(a) Two adjacent intervals $A_1$ (blue) and $A_2$ (red) in the Rindler wedges $I$ and $III$ respectively of the Minkowski diamond. (b) These two intervals are projected on the $x_1$-$z$ plane of the Poincar\\'e coordinates.}\\label{coord_adj_intervals}\n\\end{figure}\n\n\nFollowing the analysis described earlier in the last subsection for the mixed state of disjoint intervals it is also possible to obtain the time evolution of the entanglement negativity for the case of two adjacent intervals under consideration here. Once again for this purpose it is required to consider the dual bulk eternal black hole geometry and utilize eqs. \\eqref{heecon}, \\eqref{earlytimes} and \\eqref{latetimes} to obtain the corresponding holographic entanglement negativity. In this case also distinct sets of geodesics contribute to the holographic entanglement negativity for various values of the time $t$ relative to the lengths of the intervals $\\ell_1$ and $\\ell_2$. As earlier we consider these interesting limits and elucidate the corresponding geodesic structures leading to the holographic entanglement negativity for each of these cases.\n\n\n\\subsubsection{$t<\\ell_1/2<\\ell_2/2<(\\ell_1+\\ell_2)/2$}\nIn the limit of early times all of the geodesics pass through the interior region of the BTZ black string connecting the one Rindler wedge $I$ to the other $III$,  depicted by dashed curves in the Fig. \\ref{neggeoadjfin}(a). The corresponding holographic entanglement negativity is obtained by employing eq. \\eqref{earlytimes} in the eq. \\eqref{heecon}, which is given as follows\n\\begin{equation}\\label{adjlima}\n\\begin{aligned}\n\\mathcal{E} = &\\frac{3}{16G^3_N}(\\mathcal{L}_{c'c}+\\mathcal{L}_{b'b}+\\mathcal{L}_{b'b}+\\mathcal{L}_{a'a}-\\mathcal{L}_{c'c}-\\mathcal{L}_{a'a}),\\\\\n=&2\\mathcal{L}_{b'b}=\\frac{\\pi ct}{\\beta}.\n\\end{aligned}\n\\end{equation}\n\n\n\n\n\\subsubsection{$\\ell_1/2<t<\\ell_2/2<(\\ell_1+\\ell_2)/2$}\n\nIn this limit the geodesics (solid curves) between the points $c'$ to $b'$ and $c$ to $b$ are confined in the exterior region, rest of the geodesics pass through the interior region of the BTZ black string connecting the Rindler wedge $I$ to the other $III$ as shown in Fig. \\ref{neggeoadjfin}(b). The holographic entanglement negativity in this limits is given as\n\\begin{equation}\\label{adjlimb}\n\\begin{aligned}\n\\mathcal{E} = &\\frac{3}{16G^3_N}\\left(\\mathcal{L}_{c'b'}+\\mathcal{L}_{cb}+\\mathcal{L}_{b'b}+\\mathcal{L}_{a'a}-\\mathcal{L}_{c'c}-\\mathcal{L}_{a'a}\\right),\\\\\n=&\\left(\\mathcal{L}_{c'b'}+\\mathcal{L}_{cb}+\\mathcal{L}_{b'b}-\\mathcal{L}_{c'c}\\right)=\\frac{\\pi c\\ell_1}{2\\beta}.\n\\end{aligned}\n\\end{equation}\n\n\n\\begin{figure}[]\n\\centering\n\\includegraphics[width=\\linewidth]{neggeoadjfin.eps}\n\\caption{Geodesic configurations for differnt limits of the time with the lenghts of the intervals. The geodesics shown by dashed line pass through the interior region of the BTZ black hole connecting two rindler wedges. The geodesics shown by solid line confine at the exterior region of the BTZ black hole and they do not pass through the interior region.}\\label{neggeoadjfin}\n\\end{figure}\n\n\n\n\\subsubsection{$\\ell_1/2<\\ell_2/2<t<(\\ell_1+\\ell_2)/2$}\n\nIn this case only the geodesics (dashed curves) between the points $c'$ to $c$ and $c$ to $b$ pass through the interior region and rest of the geodesics remain in the exterior region of the BTZ black string as depicted in Fig. \\ref{neggeoadjfin}(c). The holographic entanglement negativity in this case is given as follows\n\\begin{equation}\\label{adjlimbc}\n\\begin{aligned}\n\\mathcal{E} = &\\frac{3}{16G^3_N}\\left(\\mathcal{L}_{c'b'}+\\mathcal{L}_{cb}+\\mathcal{L}_{b'a'}+\\mathcal{L}_{ba}-\\mathcal{L}_{c'c}-\\mathcal{L}_{a'a}\\right),\\\\\n=&\\frac{\\pi c}{\\beta}\\left(\\frac{\\ell_1+\\ell_2}{2}-t \\right).\n\\end{aligned}\n\\end{equation}\n\n\n\n\\subsubsection{$\\ell_1/2<\\ell_2/2<(\\ell_1+\\ell_2)/2<t$}\n\nIn this limit all the geodesics (solid curves)  are confined in the exterior region of the BTZ black string as depicted in Fig. \\ref{neggeoadjfin}(d). The resulting holographic entanglement negativity vanishes due to mutual cancellations as follows \n\\begin{equation}\\label{adjlimd}\n\\begin{aligned}\n\\mathcal{E} = &\\frac{3}{16G^3_N}\\left(\\mathcal{L}_{c'b'}+\\mathcal{L}_{cb}+\\mathcal{L}_{b'a'}+\\mathcal{L}_{ba}-\\mathcal{L}_{c'a'}-\\mathcal{L}_{ca}\\right),\\\\\n=&0\n\\end{aligned}\n\\end{equation}\n\nAs mentioned earlier the time evolution of the holographic entanglement negativity for two adjacent intervals after a global quench in a $CFT_{1+1}$ dual to a single sided black hole is obtained by taking half of the corresponding results for the eternal black hole case. Interestingly, it is observed that the time evolution of the holographic entanglement negativity in question following a global quench match exactly with the corresponding $CFT_{1+1}$ results computed through the replica technique in eq. (\\ref{ent_neg_glo_quench_adj_int}).\n\n\\begin{figure}[H]\n\\centering\n\\includegraphics[width=.5\\linewidth]{plotajdintneg.eps}\n\\caption{Holographic entanglement negativity $\\mathcal{E}$ vs time $t$ is plotted for two adjacent intervals of lengths $\\ell_1=50$, $\\ell_2=200$.}\\label{plotajdintneg}\n\\end{figure}\n\n\n\nThe time evolution of the holographic entanglement negativity is illustrated in Fig. \\ref{plotajdintneg}. The holographic entanglement negativity grows linearly for the initial part of its evolution and reaches to its maximum value at an intermediate time beyond which it is a constant upto a certain range of time. Beyond this range the holographic entanglement negativity decreases linearly and becomes zero for late times. Note that this behavior of the holographic entanglement negativity is consistent with that described in \\cite{Coser:2014gsa}. \n\n\n\n\n\n\n5.1 Two disjoint intervals\n\\subsection{Two disjoint intervals}\\label{hendjagq}\n\nWe first consider the case of two disjoint intervals $A_1$ and $A_2$ of lengths $\\ell_1$ and $\\ell_2$ with an interval $A_s$ of length $\\ell_s$ separating them. As earlier for the dual eternal black hole geometry it is required to consider the intervals $A_1$, $A_2$ and $A_s$ in both the Rindler wedges $I$ and $III$ of the Minkowski diamond in Fig. \\ref{coord_disjoint_intervals}(a). The end points of the intervals $A_1$, $A_2$ and $A_s$ in the Rindler wedges $I$ and $III$ projected on the $x_1$-$z$ plane of the Poincar\\'e coordinates are denoted as ($a,b$), ($c,d$), ($b,c$) and ($a',b'$), ($c',d'$), ($b',c'$) respectively which is depicted in Fig. \\ref{coord_disjoint_intervals}(b).\n\n\\begin{figure}[ht]\n\\centering\n\\includegraphics[width=\\linewidth]{coord_disjoint_intervals.eps}\n\\caption{(a) Two disjoint intervals $A_1$ (blue) and $A_2$ (red) on the Rindler wedges $I$ and $III$ respectively of the Minkowski diamond. (b) These two intervals are projected on the $x_1$-$z$ plane of the Poincar\\'e coordinates.}\\label{coord_disjoint_intervals}\n\\end{figure}\n\n\nIt is now possible to describe the time evolution of the holographic entanglement negativity for the mixed state configuration of the two disjoint intervals from the bulk dual eternal black hole geometry by utilizing  eqs. \\eqref{eq_holo_ent_neg_dj_int_holo_ent_enpy}, \\eqref{earlytimes} and \\eqref{latetimes}. It is observed that distinct sets of geodesics contribute to the holographic entanglement negativity for various values of time $t$ relative to the lengths of the intervals $\\ell_1$, $\\ell_2$ and $\\ell_s$ . In what follows we consider these significant limits and describe the relevant geodesic structures leading to the holographic entanglement negativity for each of these scenarios.\n\n\n\\subsubsection{$t<\\ell_s/2<\\ell_1/2<\\ell_2/2$}\n\nIn the limit of early times all the relevant geodesics pass through the interior region of the BTZ black string connecting one Rindler wedge $I$ to the other $III$,  depicted by dashed curves in the Fig. \\ref{neggeodjfin}(a). The corresponding holographic entanglement negativity is obtained by substituting the holographic entanglement entropy for the early time limit given in  eq. \\eqref{earlytimes} in the eq. \\eqref{eq_holo_ent_neg_dj_int_holo_ent_enpy}, which vanishes due to mutual cancellations as follows\n\\begin{equation}\\label{djlima}\n\\begin{aligned}\n\\mathcal{E} = &\\frac{3}{16G^{(3)}_N}\\left(\\underbrace{\\mathcal{L}_{d'd}+\\mathcal{L}_{b'b}}_{S_{ A_1 \\cup A_s }}+\\underbrace{\\mathcal{L}_{c'c}+\\mathcal{L}_{a'a}}_{S_{ A_s \\cup A_2 }}   -\\underbrace{\\mathcal{L}_{d'd}-\\mathcal{L}_{a'a}}_{S_{ A_1 \\cup A_2 \\cup A_s }}-  \\underbrace{\\mathcal{L}_{c'c}-\\mathcal{L}_{b'b}}_{S_{ A_s }}\\right),\\\\\n=&0.\n\\end{aligned}\n\\end{equation}\n\n\\subsubsection{$\\ell_1/2<\\ell_2/2<t<(\\ell_1+\\ell_s)/2$}\nIn this limit the geodesics (solid curves) that joins $c'$ and $b'$ and $c$ and $b$ are confined to the exterior region, and rest of the geodesics (dashed curves) pass through the interior region of the BTZ black string as depicted in Fig. \\ref{neggeodjfin}(b). The holographic entanglement negativity in this case may then be obtained by utilizing both the eqs. \\eqref{earlytimes} and \\eqref{latetimes} in the eq. \\eqref{eq_holo_ent_neg_dj_int_holo_ent_enpy}, which leads to the following expression\n\\begin{equation}\\label{djlimb}\n\\begin{aligned}\n\\mathcal{E} = &\\frac{3}{16G^{(3)}_N}\\left(\\mathcal{L}_{d'd}+\\mathcal{L}_{b'b}+\\mathcal{L}_{c'c}+\\mathcal{L}_{a'a}-\\mathcal{L}_{d'd}-\\mathcal{L}_{a'a}-\\mathcal{L}_{c'b'}-\\mathcal{L}_{cb}\\right),\\\\\n=&\\frac{3}{16G^{(3)}_N}\\left(\\mathcal{L}_{b'b}+\\mathcal{L}_{c'c}-\\mathcal{L}_{c'b'}-\\mathcal{L}_{cb}\\right),\\\\\n=&\\frac{\\pi c}{\\beta}\\left(t-\\frac{l_s}{2}\\right).\n\\end{aligned}\n\\end{equation}\n\n\\begin{figure}[h]\n\\centering\n\\includegraphics[width=\\linewidth]{neggeodjfin.eps}\n\\caption{Geodesic configurations for differnt limits of the time relative to the lengths of the intervals. The geodesics depicted by dashed lines pass through the interior region of the BTZ black string connecting the two Rindler wedges $I$ and $III$. The geodesics depicted by solid lines are confined to the exterior region of the BTZ black string outside the horizon. }\\label{neggeodjfin}\n\\end{figure}\n\n\n\n\\subsubsection{$\\ell_2/2<(\\ell_1+\\ell_s)/2<t<(\\ell_2+\\ell_s)/2$}\n\nIn this case the geodesics (solid curves) between the points $d'$ to $b'$, $d$ to $b$, $c'$ to $b'$ and $c$ to $b$ are confined in the exterior region  while the other geodesics (dashed curves) between the points $d'$ to $d$ and $c'$ to $c$ pass through the interior region of the BTZ black string as depicted in Fig. \\ref{neggeodjfin}(c). The holographic entanglement negativity in this limit is obtained by utilizing eqs. \\eqref{earlytimes} and \\eqref{latetimes} in the eq. \\eqref{eq_holo_ent_neg_dj_int_holo_ent_enpy} leading to the following expression\n\\begin{equation}\\label{djlimc}\n\\begin{aligned}\n\\mathcal{E} = &\\frac{3}{16G^{(3)}_N}\\left(\\mathcal{L}_{d'b'}+\\mathcal{L}_{db}+\\mathcal{L}_{c'c}+\\mathcal{L}_{a'a}-\\mathcal{L}_{d'd}-\\mathcal{L}_{a'a}-\\mathcal{L}_{c'b'}-\\mathcal{L}_{cb}\\right),\\\\\n=&\\frac{3}{16G^{(3)}_N}\\left(\\mathcal{L}_{d'b'}+\\mathcal{L}_{db}+\\mathcal{L}_{c'c}-\\mathcal{L}_{d'd}-\\mathcal{L}_{c'b'}-\\mathcal{L}_{cb}\\right),\\\\\n=&\\frac{\\pi c\\ell_1}{2\\beta}.\n\\end{aligned}\n\\end{equation}\n\n\\subsubsection{$(\\ell_1+\\ell_s)/2<(\\ell_2+\\ell_s)/2<t<(\\ell_1+\\ell_2+\\ell_s)/2$}\n\nIn this limit the geodesics (dashed curves) between the points $d'$ to $d$ and $a'$ to $a$ pass through the interior region whereas rest of the geodesics (solid curves) are confined to the exterior region of the BTZ black string. The holographic entanglement negativity in this case may be obtained by utilizing eqs. \\eqref{earlytimes} and \\eqref{latetimes} in the eq. \\eqref{eq_holo_ent_neg_dj_int_holo_ent_enpy}, as follows\n\\begin{equation}\\label{djlimd}\n\\begin{aligned}\n\\mathcal{E} = &\\frac{3}{16G^{(3)}_N}\\left(\\mathcal{L}_{d'b'}+\\mathcal{L}_{db}+\\mathcal{L}_{c'a'}+\\mathcal{L}_{ca}-\\mathcal{L}_{d'd}-\\mathcal{L}_{a'a}-\\mathcal{L}_{c'b'}-\\mathcal{L}_{cb}\\right),\\\\\n=&\\frac{\\pi c}{\\beta}\\left(\\frac{\\ell_1+\\ell_2+\\ell_s}{2}-t \\right)\\,.\n\\end{aligned}\n\\end{equation}\n\n\\subsubsection{$(\\ell_1+\\ell_s)/2<(\\ell_2+\\ell_s)/2<(\\ell_1+\\ell_2+\\ell_s)/2<t$}\n\n\nIn the late time limit all the geodesics (solid curves) are confined to the exterior region  of the BTZ black string as shown in Fig. \\ref{neggeodjfin}(e). Consequently, the holographic entanglement negativity is obtained by utilizing  eqs. \\eqref{earlytimes} and \\eqref{latetimes} in the eq. \\eqref{eq_holo_ent_neg_dj_int_holo_ent_enpy} which vanishes due to mutual cancellations as follows\n\\begin{equation}\\label{djlime}\n\\begin{aligned}\n\\mathcal{E} = &\\frac{3}{16G^{(3)}_N}\\left(\\mathcal{L}_{d'b'}+\\mathcal{L}_{db}+\\mathcal{L}_{c'a'}+\\mathcal{L}_{ca}-\\mathcal{L}_{d'a'}-\\mathcal{L}_{da}-\\mathcal{L}_{c'b'}-\\mathcal{L}_{cb}\\right),\\\\\n=&0.\n\\end{aligned}\n\\end{equation}\n\nIt is now possible to present a holographic description for the time evolution of the entanglement negativity for mixed state configurations of two disjoint intervals in a $CFT_{1+1}$ following a global quench. As described earlier the corresponding bulk dual geometry is the single sided black hole obtained by slicing the eternal black hole geometry in half with an ETW brane. So the time evolution of the entanglement negativity for the mixed state in question following a global quench may be obtained by considering half of the corresponding results for the eternal black hole geometry computed above.\nQuite significantly, the time evolution of the holographic entanglement negativity obtained from our construction in this case matches exactly with the corresponding replica technique results for a $CFT_{1+1}$ as described in section (\\ref{ent_neg_glo_quench_dj_int}). Once again this is a robust consistency check for our holographic construction.\n\nIn  Fig. \\ref{neg_vs_t_dj_int} we have plotted the time evolution of the holographic entanglement negativity for the mixed state in question in a $CFT_{1+1}$ following a global quench.  It is observed from Fig. \\ref{neg_vs_t_dj_int} that the holographic entanglement negativity for early times is zero and grows linearly to attain a maximum value at an intermediate time after which remains constant upto certain value of time. Beyond this value of time the holographic entanglement negativity decreases linearly and vanishes for late times. Note that this behavior for the entanglement negativity of the mixed state in question consistently matches with that described for a $CFT_{1+1}$s as described in \\cite{Coser:2014gsa}.\n\n\\begin{figure}[H]\n\\centering\n\\includegraphics[width=.5\\linewidth]{plot_dj_int_neg.eps}\n\\caption{Holographic entanglement negativity $\\mathcal{E}$ vs time $t$ is plotted for two disjoint intervals of lengths $\\ell_1=50$, $\\ell_2=200$ with the seperation length $l_s=30$.}\\label{neg_vs_t_dj_int}\n\\end{figure}\n\n\n\n\n\n\n5.1.5 $(\\ell_1+\\ell_s)/2<(\\ell_2+\\ell_s)/2<(\\ell_1+\\ell_2+\\ell_s)/2<t$\n\\subsubsection{$(\\ell_1+\\ell_s)/2<(\\ell_2+\\ell_s)/2<(\\ell_1+\\ell_2+\\ell_s)/2<t$}\n\n\nIn the late time limit all the geodesics (solid curves) are confined to the exterior region  of the BTZ black string as shown in Fig. \\ref{neggeodjfin}(e). Consequently, the holographic entanglement negativity is obtained by utilizing  eqs. \\eqref{earlytimes} and \\eqref{latetimes} in the eq. \\eqref{eq_holo_ent_neg_dj_int_holo_ent_enpy} which vanishes due to mutual cancellations as follows\n\\begin{equation}\\label{djlime}\n\\begin{aligned}\n\\mathcal{E} = &\\frac{3}{16G^{(3)}_N}\\left(\\mathcal{L}_{d'b'}+\\mathcal{L}_{db}+\\mathcal{L}_{c'a'}+\\mathcal{L}_{ca}-\\mathcal{L}_{d'a'}-\\mathcal{L}_{da}-\\mathcal{L}_{c'b'}-\\mathcal{L}_{cb}\\right),\\\\\n=&0.\n\\end{aligned}\n\\end{equation}\n\nIt is now possible to present a holographic description for the time evolution of the entanglement negativity for mixed state configurations of two disjoint intervals in a $CFT_{1+1}$ following a global quench. As described earlier the corresponding bulk dual geometry is the single sided black hole obtained by slicing the eternal black hole geometry in half with an ETW brane. So the time evolution of the entanglement negativity for the mixed state in question following a global quench may be obtained by considering half of the corresponding results for the eternal black hole geometry computed above.\nQuite significantly, the time evolution of the holographic entanglement negativity obtained from our construction in this case matches exactly with the corresponding replica technique results for a $CFT_{1+1}$ as described in section (\\ref{ent_neg_glo_quench_dj_int}). Once again this is a robust consistency check for our holographic construction.\n\nIn  Fig. \\ref{neg_vs_t_dj_int} we have plotted the time evolution of the holographic entanglement negativity for the mixed state in question in a $CFT_{1+1}$ following a global quench.  It is observed from Fig. \\ref{neg_vs_t_dj_int} that the holographic entanglement negativity for early times is zero and grows linearly to attain a maximum value at an intermediate time after which remains constant upto certain value of time. Beyond this value of time the holographic entanglement negativity decreases linearly and vanishes for late times. Note that this behavior for the entanglement negativity of the mixed state in question consistently matches with that described for a $CFT_{1+1}$s as described in \\cite{Coser:2014gsa}.\n\n\\begin{figure}[H]\n\\centering\n\\includegraphics[width=.5\\linewidth]{plot_dj_int_neg.eps}\n\\caption{Holographic entanglement negativity $\\mathcal{E}$ vs time $t$ is plotted for two disjoint intervals of lengths $\\ell_1=50$, $\\ell_2=200$ with the seperation length $l_s=30$.}\\label{neg_vs_t_dj_int}\n\\end{figure}\n\n\n\n\n\n\n",
        "label": "neg_vs_t_dj_int",
        "Descriptive_question1": "What is the length of the first disjoint interval in figure_7?",
        "Descriptive_question2": "What is the separation length between the two intervals in figure_7?",
        "Reasoning_question1": "What trend can be observed in the holographic entanglement negativity over time in figure_7, and what might this suggest about the entanglement dynamics following a global quench?",
        "Reasoning_question2": "How does the maximum value of holographic entanglement negativity in figure_7 relate to the lengths of the intervals, and what physical implications might this have?",
        "Descriptive_answer1": "50",
        "Descriptive_answer2": "30",
        "Reasoning_answer1": "From figure_7, the holographic entanglement negativity $\\mathcal{E}$ starts at zero at early times, then increases linearly with time until it reaches a maximum plateau. After remaining constant for some interval, it decreases linearly again and finally returns to zero at late times. This pattern suggests that following a global quench, entanglement between the two disjoint intervals builds up over time, stabilizes for a period, and then dissipates, reflecting the dynamics of entanglement spreading and thermalization in the system.",
        "Reasoning_answer2": "The maximum value of the holographic entanglement negativity corresponds to the plateau in figure_7 and is influenced by the lengths of the intervals as well as the separation length. Specifically, the time intervals for the phases in negativity relate to half the lengths $\\ell_1$, $\\ell_2$, and the separation $l_s$. Physically, the maximum negativity indicates the peak entanglement shared between the intervals, which is constrained by their sizes and separation, reflecting how spatial separation and interval length influence entanglement spatial structure post-quench."
    },
    {
        "paper_id": "1812.04424.json",
        "image_id": "figure_10",
        "image_path": [
            "/home/yz979/scratch/chengye/scimolmo/test-images-tables/dataset_imagestables/1812.04424/images/plotajdintneg.eps"
        ],
        "caption": "Holographic entanglement negativity $\\mathcal{E}$ vs time $t$ is plotted for two adjacent intervals of lengths $\\ell_1=50$, $\\ell_2=200$.",
        "classify": "Chart",
        "section_info": "5 Holographic entanglement negativity\n\\section{Holographic entanglement negativity}\\label{henaq}\n\nIn this section we present our holographic construction for the time evolution of the entanglement negativity of two disjoint and adjacent intervals in a $CFT_{1+1}$ following a global quench as described in \\cite{Coser:2014gsa}. Motivated by the holographic construction for the time evolution of the entanglement entropy in \\cite{Hartman:2013qma} reviewed above, we begin by computing the holographic entanglement negativity for the mixed states under consideration from the bulk dual eternal black hole geometry. The corresponding holographic entanglement negativity following a global quench may then be obtained by considering half of the result for the bulk dual eternal black hole geometry. \n\nIt interesting to note that the four point twist correlator in eq. \\eqref{4pttc} admits a factorization with the non universal function ${\\cal F}(\\{\\eta_{j,k}\\})\\to 1$ for the cross ratios $\\eta_{i,j}\\to 1$ and $\\eta_{i,j}\\to 0$, as follows\n\\begin{equation}\\label{factorization_four_pt}\n\\begin{aligned}\n&\\langle\\mathcal{T}_{n}(w_1)\\overline{\\mathcal{T}}_{n}(w_2)\\overline{\\mathcal{T}}_{n}(w_3)\\mathcal{T}_{n}(w_4)\\rangle_{\\rm strip}=\\\\\n&\\frac{\\langle\\mathcal{T}_{n}(w_1)\\overline{\\mathcal{T}}_{n}(w_4)\\rangle_{\\rm strip}~\\langle\\mathcal{T}_{n}(w_2)\\overline{\\mathcal{T}}_{n}(w_3)\\rangle_{\\rm strip}~\\Big(\\langle\\, \\mathcal{T}^2_n (w_1) \n\\bar{\\mathcal{T}}^2_n(w_3) \\,\\rangle_{\\rm strip}~\\langle\\, \\mathcal{T}^2_n (w_2) \n\\bar{\\mathcal{T}}^2_n(w_4) \\,\\rangle_{\\rm strip}\\Big)^\\frac{1}{2}}{\\langle\\mathcal{T}_{n}(w_1)\\overline{\\mathcal{T}}_{n}(w_3)\\rangle_{\\rm strip}~\\langle\\mathcal{T}_{n}(w_2)\\overline{\\mathcal{T}}_{n}(w_4)\\rangle_{\\rm strip}~\\Big(\\langle\\, \\mathcal{T}^2_n (w_1) \n\\bar{\\mathcal{T}}^2_n(w_4) \\,\\rangle_{\\rm strip}~\\langle\\, \\mathcal{T}^2_n (w_2) \n\\bar{\\mathcal{T}}^2_n(w_3) \\,\\rangle_{\\rm strip} \\Big)^\\frac{1}{2}},\n\\end{aligned}\n\\end{equation}\nwhere we have employed eq. \\eqref{renyi corr strip} and the following two point twist correlator \n\\begin{equation}\n\\label{twist squared uhp N=1}\n\\langle  \\mathcal{T}^2_n (w_1) \n\\bar{\\mathcal{T}}^2_n(w_2) \\rangle_{\\rm strip} \n=\\left( \\frac{\\pi}{2\\tau_0} \\right)^{2\\Delta^{(2)}_n}\n\\frac{c^{(2)}_{n}}{|(z_1 - \\bar{z}_1)(z_2 - \\bar{z}_2) \\,\\eta_{1,2}|^{\\Delta^{(2)}_n}}{\\cal F}(\\eta_{1,2})\\,.\n\\end{equation}\nThe entanglement negativity for two disjoint intervals may then be obtained by utilizing eqs. \\eqref{en replica} and \\eqref{factorization_four_pt} which leads to following expression\n\\begin{equation}\n\\begin{aligned}\n\\label{eq_holo_ent_neg_dj_int_holo_ent_enpy}\n{\\cal E} = &\\frac{ 3 }{ 4 }\n\\left ( S_{ A_1 \\cup A_s } + S_{ A_s \\cup A_2 }\n- S_{ A_1 \\cup A_2 \\cup A_s } - S_{ A_s } \\right ),\n\\end{aligned}\n\\end{equation}\nwhere $S_{\\gamma}~(\\gamma \\in  A_1 \\cup A_s,~A_s \\cup A_2,~A_1 \\cup A_2 \\cup A_s,~ A_s)$ is the entanglement entropy of an interval $\\gamma$  as given in eq. \\eqref{SA one interval}.\n\nNote that a similar factorization also holds for the three point twist correlator defined in eq. \\eqref{en 3pt fn} which is given as\n\n\\begin{equation}\\label{factorization_three_pt}\n\\begin{aligned}\n\\langle \\mathcal{T}_n(w_1) \\bar{\\mathcal{T}}^2_n(w_2) \\mathcal{T}_n(w_3) \\rangle_{\\rm strip}\\,=\n&\\Bigg(\\frac{\\langle\\, \\mathcal{T}^2_n (w_1) \n\\bar{\\mathcal{T}}^2_n(w_2) \\,\\rangle_{\\rm strip}~\\langle\\, \\mathcal{T}^2_n (w_2) \n\\bar{\\mathcal{T}}^2_n(w_3) \\,\\rangle_{\\rm strip} \\langle\\mathcal{T}_{n}(w_1)\\overline{\\mathcal{T}}_{n}(w_3)\\rangle_{\\rm strip}^2 }{\\langle\\, \\mathcal{T}^2_n (w_1) \n\\bar{\\mathcal{T}}^2_n(w_3) \\,\\rangle_{\\rm strip} }\\Bigg)^{1/2}.\n\\end{aligned}\n\\end{equation}\nThe entanglement negativity for two adjacent intervals may then be obtained by utilizing eqs. \\eqref{en replica} and \\eqref{factorization_three_pt} as follows\n\\begin{equation}\\label{heecon}\n\\mathcal{E} =  \\frac{3}{4}(S_{A_1}+S_{A_2}-S_{A_1\\cup A_2}).\n\\end{equation}\n\nIn what follows we utilize eqs. \\eqref{eq_holo_ent_neg_dj_int_holo_ent_enpy} and \\eqref{heecon} to establish the holographic entanglement negativity and its time evolution for two disjoint and adjacent intervals in a $CFT_{1+1}$ after a global quench employing the prescription for the holographic entanglement entropy described in \\cite{Hartman:2013qma}.\n\n\n\n\\subsection{Two disjoint intervals}\\label{hendjagq}\n\nWe first consider the case of two disjoint intervals $A_1$ and $A_2$ of lengths $\\ell_1$ and $\\ell_2$ with an interval $A_s$ of length $\\ell_s$ separating them. As earlier for the dual eternal black hole geometry it is required to consider the intervals $A_1$, $A_2$ and $A_s$ in both the Rindler wedges $I$ and $III$ of the Minkowski diamond in Fig. \\ref{coord_disjoint_intervals}(a). The end points of the intervals $A_1$, $A_2$ and $A_s$ in the Rindler wedges $I$ and $III$ projected on the $x_1$-$z$ plane of the Poincar\\'e coordinates are denoted as ($a,b$), ($c,d$), ($b,c$) and ($a',b'$), ($c',d'$), ($b',c'$) respectively which is depicted in Fig. \\ref{coord_disjoint_intervals}(b).\n\n\\begin{figure}[ht]\n\\centering\n\\includegraphics[width=\\linewidth]{coord_disjoint_intervals.eps}\n\\caption{(a) Two disjoint intervals $A_1$ (blue) and $A_2$ (red) on the Rindler wedges $I$ and $III$ respectively of the Minkowski diamond. (b) These two intervals are projected on the $x_1$-$z$ plane of the Poincar\\'e coordinates.}\\label{coord_disjoint_intervals}\n\\end{figure}\n\n\nIt is now possible to describe the time evolution of the holographic entanglement negativity for the mixed state configuration of the two disjoint intervals from the bulk dual eternal black hole geometry by utilizing  eqs. \\eqref{eq_holo_ent_neg_dj_int_holo_ent_enpy}, \\eqref{earlytimes} and \\eqref{latetimes}. It is observed that distinct sets of geodesics contribute to the holographic entanglement negativity for various values of time $t$ relative to the lengths of the intervals $\\ell_1$, $\\ell_2$ and $\\ell_s$ . In what follows we consider these significant limits and describe the relevant geodesic structures leading to the holographic entanglement negativity for each of these scenarios.\n\n\n\\subsubsection{$t<\\ell_s/2<\\ell_1/2<\\ell_2/2$}\n\nIn the limit of early times all the relevant geodesics pass through the interior region of the BTZ black string connecting one Rindler wedge $I$ to the other $III$,  depicted by dashed curves in the Fig. \\ref{neggeodjfin}(a). The corresponding holographic entanglement negativity is obtained by substituting the holographic entanglement entropy for the early time limit given in  eq. \\eqref{earlytimes} in the eq. \\eqref{eq_holo_ent_neg_dj_int_holo_ent_enpy}, which vanishes due to mutual cancellations as follows\n\\begin{equation}\\label{djlima}\n\\begin{aligned}\n\\mathcal{E} = &\\frac{3}{16G^{(3)}_N}\\left(\\underbrace{\\mathcal{L}_{d'd}+\\mathcal{L}_{b'b}}_{S_{ A_1 \\cup A_s }}+\\underbrace{\\mathcal{L}_{c'c}+\\mathcal{L}_{a'a}}_{S_{ A_s \\cup A_2 }}   -\\underbrace{\\mathcal{L}_{d'd}-\\mathcal{L}_{a'a}}_{S_{ A_1 \\cup A_2 \\cup A_s }}-  \\underbrace{\\mathcal{L}_{c'c}-\\mathcal{L}_{b'b}}_{S_{ A_s }}\\right),\\\\\n=&0.\n\\end{aligned}\n\\end{equation}\n\n\\subsubsection{$\\ell_1/2<\\ell_2/2<t<(\\ell_1+\\ell_s)/2$}\nIn this limit the geodesics (solid curves) that joins $c'$ and $b'$ and $c$ and $b$ are confined to the exterior region, and rest of the geodesics (dashed curves) pass through the interior region of the BTZ black string as depicted in Fig. \\ref{neggeodjfin}(b). The holographic entanglement negativity in this case may then be obtained by utilizing both the eqs. \\eqref{earlytimes} and \\eqref{latetimes} in the eq. \\eqref{eq_holo_ent_neg_dj_int_holo_ent_enpy}, which leads to the following expression\n\\begin{equation}\\label{djlimb}\n\\begin{aligned}\n\\mathcal{E} = &\\frac{3}{16G^{(3)}_N}\\left(\\mathcal{L}_{d'd}+\\mathcal{L}_{b'b}+\\mathcal{L}_{c'c}+\\mathcal{L}_{a'a}-\\mathcal{L}_{d'd}-\\mathcal{L}_{a'a}-\\mathcal{L}_{c'b'}-\\mathcal{L}_{cb}\\right),\\\\\n=&\\frac{3}{16G^{(3)}_N}\\left(\\mathcal{L}_{b'b}+\\mathcal{L}_{c'c}-\\mathcal{L}_{c'b'}-\\mathcal{L}_{cb}\\right),\\\\\n=&\\frac{\\pi c}{\\beta}\\left(t-\\frac{l_s}{2}\\right).\n\\end{aligned}\n\\end{equation}\n\n\\begin{figure}[h]\n\\centering\n\\includegraphics[width=\\linewidth]{neggeodjfin.eps}\n\\caption{Geodesic configurations for differnt limits of the time relative to the lengths of the intervals. The geodesics depicted by dashed lines pass through the interior region of the BTZ black string connecting the two Rindler wedges $I$ and $III$. The geodesics depicted by solid lines are confined to the exterior region of the BTZ black string outside the horizon. }\\label{neggeodjfin}\n\\end{figure}\n\n\n\n\\subsubsection{$\\ell_2/2<(\\ell_1+\\ell_s)/2<t<(\\ell_2+\\ell_s)/2$}\n\nIn this case the geodesics (solid curves) between the points $d'$ to $b'$, $d$ to $b$, $c'$ to $b'$ and $c$ to $b$ are confined in the exterior region  while the other geodesics (dashed curves) between the points $d'$ to $d$ and $c'$ to $c$ pass through the interior region of the BTZ black string as depicted in Fig. \\ref{neggeodjfin}(c). The holographic entanglement negativity in this limit is obtained by utilizing eqs. \\eqref{earlytimes} and \\eqref{latetimes} in the eq. \\eqref{eq_holo_ent_neg_dj_int_holo_ent_enpy} leading to the following expression\n\\begin{equation}\\label{djlimc}\n\\begin{aligned}\n\\mathcal{E} = &\\frac{3}{16G^{(3)}_N}\\left(\\mathcal{L}_{d'b'}+\\mathcal{L}_{db}+\\mathcal{L}_{c'c}+\\mathcal{L}_{a'a}-\\mathcal{L}_{d'd}-\\mathcal{L}_{a'a}-\\mathcal{L}_{c'b'}-\\mathcal{L}_{cb}\\right),\\\\\n=&\\frac{3}{16G^{(3)}_N}\\left(\\mathcal{L}_{d'b'}+\\mathcal{L}_{db}+\\mathcal{L}_{c'c}-\\mathcal{L}_{d'd}-\\mathcal{L}_{c'b'}-\\mathcal{L}_{cb}\\right),\\\\\n=&\\frac{\\pi c\\ell_1}{2\\beta}.\n\\end{aligned}\n\\end{equation}\n\n\\subsubsection{$(\\ell_1+\\ell_s)/2<(\\ell_2+\\ell_s)/2<t<(\\ell_1+\\ell_2+\\ell_s)/2$}\n\nIn this limit the geodesics (dashed curves) between the points $d'$ to $d$ and $a'$ to $a$ pass through the interior region whereas rest of the geodesics (solid curves) are confined to the exterior region of the BTZ black string. The holographic entanglement negativity in this case may be obtained by utilizing eqs. \\eqref{earlytimes} and \\eqref{latetimes} in the eq. \\eqref{eq_holo_ent_neg_dj_int_holo_ent_enpy}, as follows\n\\begin{equation}\\label{djlimd}\n\\begin{aligned}\n\\mathcal{E} = &\\frac{3}{16G^{(3)}_N}\\left(\\mathcal{L}_{d'b'}+\\mathcal{L}_{db}+\\mathcal{L}_{c'a'}+\\mathcal{L}_{ca}-\\mathcal{L}_{d'd}-\\mathcal{L}_{a'a}-\\mathcal{L}_{c'b'}-\\mathcal{L}_{cb}\\right),\\\\\n=&\\frac{\\pi c}{\\beta}\\left(\\frac{\\ell_1+\\ell_2+\\ell_s}{2}-t \\right)\\,.\n\\end{aligned}\n\\end{equation}\n\n\\subsubsection{$(\\ell_1+\\ell_s)/2<(\\ell_2+\\ell_s)/2<(\\ell_1+\\ell_2+\\ell_s)/2<t$}\n\n\nIn the late time limit all the geodesics (solid curves) are confined to the exterior region  of the BTZ black string as shown in Fig. \\ref{neggeodjfin}(e). Consequently, the holographic entanglement negativity is obtained by utilizing  eqs. \\eqref{earlytimes} and \\eqref{latetimes} in the eq. \\eqref{eq_holo_ent_neg_dj_int_holo_ent_enpy} which vanishes due to mutual cancellations as follows\n\\begin{equation}\\label{djlime}\n\\begin{aligned}\n\\mathcal{E} = &\\frac{3}{16G^{(3)}_N}\\left(\\mathcal{L}_{d'b'}+\\mathcal{L}_{db}+\\mathcal{L}_{c'a'}+\\mathcal{L}_{ca}-\\mathcal{L}_{d'a'}-\\mathcal{L}_{da}-\\mathcal{L}_{c'b'}-\\mathcal{L}_{cb}\\right),\\\\\n=&0.\n\\end{aligned}\n\\end{equation}\n\nIt is now possible to present a holographic description for the time evolution of the entanglement negativity for mixed state configurations of two disjoint intervals in a $CFT_{1+1}$ following a global quench. As described earlier the corresponding bulk dual geometry is the single sided black hole obtained by slicing the eternal black hole geometry in half with an ETW brane. So the time evolution of the entanglement negativity for the mixed state in question following a global quench may be obtained by considering half of the corresponding results for the eternal black hole geometry computed above.\nQuite significantly, the time evolution of the holographic entanglement negativity obtained from our construction in this case matches exactly with the corresponding replica technique results for a $CFT_{1+1}$ as described in section (\\ref{ent_neg_glo_quench_dj_int}). Once again this is a robust consistency check for our holographic construction.\n\nIn  Fig. \\ref{neg_vs_t_dj_int} we have plotted the time evolution of the holographic entanglement negativity for the mixed state in question in a $CFT_{1+1}$ following a global quench.  It is observed from Fig. \\ref{neg_vs_t_dj_int} that the holographic entanglement negativity for early times is zero and grows linearly to attain a maximum value at an intermediate time after which remains constant upto certain value of time. Beyond this value of time the holographic entanglement negativity decreases linearly and vanishes for late times. Note that this behavior for the entanglement negativity of the mixed state in question consistently matches with that described for a $CFT_{1+1}$s as described in \\cite{Coser:2014gsa}.\n\n\\begin{figure}[H]\n\\centering\n\\includegraphics[width=.5\\linewidth]{plot_dj_int_neg.eps}\n\\caption{Holographic entanglement negativity $\\mathcal{E}$ vs time $t$ is plotted for two disjoint intervals of lengths $\\ell_1=50$, $\\ell_2=200$ with the seperation length $l_s=30$.}\\label{neg_vs_t_dj_int}\n\\end{figure}\n\n\n\n\n\n\n\\subsection{Two adjacent intervals}\\label{henadjagq}\n\n\nWe now consider the time evolution of the entanglement negativity for the case of two adjacent intervals $A_1$ and $A_2$ of lengths $\\ell_1$ and $\\ell_2$ respectively. Proceeding similarly as in the case for the two disjoint intervals described above, we consider the two adjacent intervals $A_1$ and $A_2$ in both the Rindler wedges $I$ and $III$ of the Minkowski diamond as shown in Fig. \\ref{coord_adj_intervals}(a). The end points of the intervals $A_1$ and $A_2$ in the Rindler wedges projected on the $x_1 -z$ plane of the Poincaré coordinates are denoted as ($a, b$), ($b, c$) and ($a', b'$), ($b', c'$)  respectively which is depicted in Fig. \\ref{coord_adj_intervals}(b).\n\n\\begin{figure}[h]\n\\centering\n\\includegraphics[width=\\linewidth]{coord_adj_intervals.eps}\n\\caption{(a) Two adjacent intervals $A_1$ (blue) and $A_2$ (red) in the Rindler wedges $I$ and $III$ respectively of the Minkowski diamond. (b) These two intervals are projected on the $x_1$-$z$ plane of the Poincar\\'e coordinates.}\\label{coord_adj_intervals}\n\\end{figure}\n\n\nFollowing the analysis described earlier in the last subsection for the mixed state of disjoint intervals it is also possible to obtain the time evolution of the entanglement negativity for the case of two adjacent intervals under consideration here. Once again for this purpose it is required to consider the dual bulk eternal black hole geometry and utilize eqs. \\eqref{heecon}, \\eqref{earlytimes} and \\eqref{latetimes} to obtain the corresponding holographic entanglement negativity. In this case also distinct sets of geodesics contribute to the holographic entanglement negativity for various values of the time $t$ relative to the lengths of the intervals $\\ell_1$ and $\\ell_2$. As earlier we consider these interesting limits and elucidate the corresponding geodesic structures leading to the holographic entanglement negativity for each of these cases.\n\n\n\\subsubsection{$t<\\ell_1/2<\\ell_2/2<(\\ell_1+\\ell_2)/2$}\nIn the limit of early times all of the geodesics pass through the interior region of the BTZ black string connecting the one Rindler wedge $I$ to the other $III$,  depicted by dashed curves in the Fig. \\ref{neggeoadjfin}(a). The corresponding holographic entanglement negativity is obtained by employing eq. \\eqref{earlytimes} in the eq. \\eqref{heecon}, which is given as follows\n\\begin{equation}\\label{adjlima}\n\\begin{aligned}\n\\mathcal{E} = &\\frac{3}{16G^3_N}(\\mathcal{L}_{c'c}+\\mathcal{L}_{b'b}+\\mathcal{L}_{b'b}+\\mathcal{L}_{a'a}-\\mathcal{L}_{c'c}-\\mathcal{L}_{a'a}),\\\\\n=&2\\mathcal{L}_{b'b}=\\frac{\\pi ct}{\\beta}.\n\\end{aligned}\n\\end{equation}\n\n\n\n\n\\subsubsection{$\\ell_1/2<t<\\ell_2/2<(\\ell_1+\\ell_2)/2$}\n\nIn this limit the geodesics (solid curves) between the points $c'$ to $b'$ and $c$ to $b$ are confined in the exterior region, rest of the geodesics pass through the interior region of the BTZ black string connecting the Rindler wedge $I$ to the other $III$ as shown in Fig. \\ref{neggeoadjfin}(b). The holographic entanglement negativity in this limits is given as\n\\begin{equation}\\label{adjlimb}\n\\begin{aligned}\n\\mathcal{E} = &\\frac{3}{16G^3_N}\\left(\\mathcal{L}_{c'b'}+\\mathcal{L}_{cb}+\\mathcal{L}_{b'b}+\\mathcal{L}_{a'a}-\\mathcal{L}_{c'c}-\\mathcal{L}_{a'a}\\right),\\\\\n=&\\left(\\mathcal{L}_{c'b'}+\\mathcal{L}_{cb}+\\mathcal{L}_{b'b}-\\mathcal{L}_{c'c}\\right)=\\frac{\\pi c\\ell_1}{2\\beta}.\n\\end{aligned}\n\\end{equation}\n\n\n\\begin{figure}[]\n\\centering\n\\includegraphics[width=\\linewidth]{neggeoadjfin.eps}\n\\caption{Geodesic configurations for differnt limits of the time with the lenghts of the intervals. The geodesics shown by dashed line pass through the interior region of the BTZ black hole connecting two rindler wedges. The geodesics shown by solid line confine at the exterior region of the BTZ black hole and they do not pass through the interior region.}\\label{neggeoadjfin}\n\\end{figure}\n\n\n\n\\subsubsection{$\\ell_1/2<\\ell_2/2<t<(\\ell_1+\\ell_2)/2$}\n\nIn this case only the geodesics (dashed curves) between the points $c'$ to $c$ and $c$ to $b$ pass through the interior region and rest of the geodesics remain in the exterior region of the BTZ black string as depicted in Fig. \\ref{neggeoadjfin}(c). The holographic entanglement negativity in this case is given as follows\n\\begin{equation}\\label{adjlimbc}\n\\begin{aligned}\n\\mathcal{E} = &\\frac{3}{16G^3_N}\\left(\\mathcal{L}_{c'b'}+\\mathcal{L}_{cb}+\\mathcal{L}_{b'a'}+\\mathcal{L}_{ba}-\\mathcal{L}_{c'c}-\\mathcal{L}_{a'a}\\right),\\\\\n=&\\frac{\\pi c}{\\beta}\\left(\\frac{\\ell_1+\\ell_2}{2}-t \\right).\n\\end{aligned}\n\\end{equation}\n\n\n\n\\subsubsection{$\\ell_1/2<\\ell_2/2<(\\ell_1+\\ell_2)/2<t$}\n\nIn this limit all the geodesics (solid curves)  are confined in the exterior region of the BTZ black string as depicted in Fig. \\ref{neggeoadjfin}(d). The resulting holographic entanglement negativity vanishes due to mutual cancellations as follows \n\\begin{equation}\\label{adjlimd}\n\\begin{aligned}\n\\mathcal{E} = &\\frac{3}{16G^3_N}\\left(\\mathcal{L}_{c'b'}+\\mathcal{L}_{cb}+\\mathcal{L}_{b'a'}+\\mathcal{L}_{ba}-\\mathcal{L}_{c'a'}-\\mathcal{L}_{ca}\\right),\\\\\n=&0\n\\end{aligned}\n\\end{equation}\n\nAs mentioned earlier the time evolution of the holographic entanglement negativity for two adjacent intervals after a global quench in a $CFT_{1+1}$ dual to a single sided black hole is obtained by taking half of the corresponding results for the eternal black hole case. Interestingly, it is observed that the time evolution of the holographic entanglement negativity in question following a global quench match exactly with the corresponding $CFT_{1+1}$ results computed through the replica technique in eq. (\\ref{ent_neg_glo_quench_adj_int}).\n\n\\begin{figure}[H]\n\\centering\n\\includegraphics[width=.5\\linewidth]{plotajdintneg.eps}\n\\caption{Holographic entanglement negativity $\\mathcal{E}$ vs time $t$ is plotted for two adjacent intervals of lengths $\\ell_1=50$, $\\ell_2=200$.}\\label{plotajdintneg}\n\\end{figure}\n\n\n\nThe time evolution of the holographic entanglement negativity is illustrated in Fig. \\ref{plotajdintneg}. The holographic entanglement negativity grows linearly for the initial part of its evolution and reaches to its maximum value at an intermediate time beyond which it is a constant upto a certain range of time. Beyond this range the holographic entanglement negativity decreases linearly and becomes zero for late times. Note that this behavior of the holographic entanglement negativity is consistent with that described in \\cite{Coser:2014gsa}. \n\n\n\n\n\n\n5.2 Two adjacent intervals\n\\subsection{Two adjacent intervals}\\label{henadjagq}\n\n\nWe now consider the time evolution of the entanglement negativity for the case of two adjacent intervals $A_1$ and $A_2$ of lengths $\\ell_1$ and $\\ell_2$ respectively. Proceeding similarly as in the case for the two disjoint intervals described above, we consider the two adjacent intervals $A_1$ and $A_2$ in both the Rindler wedges $I$ and $III$ of the Minkowski diamond as shown in Fig. \\ref{coord_adj_intervals}(a). The end points of the intervals $A_1$ and $A_2$ in the Rindler wedges projected on the $x_1 -z$ plane of the Poincaré coordinates are denoted as ($a, b$), ($b, c$) and ($a', b'$), ($b', c'$)  respectively which is depicted in Fig. \\ref{coord_adj_intervals}(b).\n\n\\begin{figure}[h]\n\\centering\n\\includegraphics[width=\\linewidth]{coord_adj_intervals.eps}\n\\caption{(a) Two adjacent intervals $A_1$ (blue) and $A_2$ (red) in the Rindler wedges $I$ and $III$ respectively of the Minkowski diamond. (b) These two intervals are projected on the $x_1$-$z$ plane of the Poincar\\'e coordinates.}\\label{coord_adj_intervals}\n\\end{figure}\n\n\nFollowing the analysis described earlier in the last subsection for the mixed state of disjoint intervals it is also possible to obtain the time evolution of the entanglement negativity for the case of two adjacent intervals under consideration here. Once again for this purpose it is required to consider the dual bulk eternal black hole geometry and utilize eqs. \\eqref{heecon}, \\eqref{earlytimes} and \\eqref{latetimes} to obtain the corresponding holographic entanglement negativity. In this case also distinct sets of geodesics contribute to the holographic entanglement negativity for various values of the time $t$ relative to the lengths of the intervals $\\ell_1$ and $\\ell_2$. As earlier we consider these interesting limits and elucidate the corresponding geodesic structures leading to the holographic entanglement negativity for each of these cases.\n\n\n\\subsubsection{$t<\\ell_1/2<\\ell_2/2<(\\ell_1+\\ell_2)/2$}\nIn the limit of early times all of the geodesics pass through the interior region of the BTZ black string connecting the one Rindler wedge $I$ to the other $III$,  depicted by dashed curves in the Fig. \\ref{neggeoadjfin}(a). The corresponding holographic entanglement negativity is obtained by employing eq. \\eqref{earlytimes} in the eq. \\eqref{heecon}, which is given as follows\n\\begin{equation}\\label{adjlima}\n\\begin{aligned}\n\\mathcal{E} = &\\frac{3}{16G^3_N}(\\mathcal{L}_{c'c}+\\mathcal{L}_{b'b}+\\mathcal{L}_{b'b}+\\mathcal{L}_{a'a}-\\mathcal{L}_{c'c}-\\mathcal{L}_{a'a}),\\\\\n=&2\\mathcal{L}_{b'b}=\\frac{\\pi ct}{\\beta}.\n\\end{aligned}\n\\end{equation}\n\n\n\n\n\\subsubsection{$\\ell_1/2<t<\\ell_2/2<(\\ell_1+\\ell_2)/2$}\n\nIn this limit the geodesics (solid curves) between the points $c'$ to $b'$ and $c$ to $b$ are confined in the exterior region, rest of the geodesics pass through the interior region of the BTZ black string connecting the Rindler wedge $I$ to the other $III$ as shown in Fig. \\ref{neggeoadjfin}(b). The holographic entanglement negativity in this limits is given as\n\\begin{equation}\\label{adjlimb}\n\\begin{aligned}\n\\mathcal{E} = &\\frac{3}{16G^3_N}\\left(\\mathcal{L}_{c'b'}+\\mathcal{L}_{cb}+\\mathcal{L}_{b'b}+\\mathcal{L}_{a'a}-\\mathcal{L}_{c'c}-\\mathcal{L}_{a'a}\\right),\\\\\n=&\\left(\\mathcal{L}_{c'b'}+\\mathcal{L}_{cb}+\\mathcal{L}_{b'b}-\\mathcal{L}_{c'c}\\right)=\\frac{\\pi c\\ell_1}{2\\beta}.\n\\end{aligned}\n\\end{equation}\n\n\n\\begin{figure}[]\n\\centering\n\\includegraphics[width=\\linewidth]{neggeoadjfin.eps}\n\\caption{Geodesic configurations for differnt limits of the time with the lenghts of the intervals. The geodesics shown by dashed line pass through the interior region of the BTZ black hole connecting two rindler wedges. The geodesics shown by solid line confine at the exterior region of the BTZ black hole and they do not pass through the interior region.}\\label{neggeoadjfin}\n\\end{figure}\n\n\n\n\\subsubsection{$\\ell_1/2<\\ell_2/2<t<(\\ell_1+\\ell_2)/2$}\n\nIn this case only the geodesics (dashed curves) between the points $c'$ to $c$ and $c$ to $b$ pass through the interior region and rest of the geodesics remain in the exterior region of the BTZ black string as depicted in Fig. \\ref{neggeoadjfin}(c). The holographic entanglement negativity in this case is given as follows\n\\begin{equation}\\label{adjlimbc}\n\\begin{aligned}\n\\mathcal{E} = &\\frac{3}{16G^3_N}\\left(\\mathcal{L}_{c'b'}+\\mathcal{L}_{cb}+\\mathcal{L}_{b'a'}+\\mathcal{L}_{ba}-\\mathcal{L}_{c'c}-\\mathcal{L}_{a'a}\\right),\\\\\n=&\\frac{\\pi c}{\\beta}\\left(\\frac{\\ell_1+\\ell_2}{2}-t \\right).\n\\end{aligned}\n\\end{equation}\n\n\n\n\\subsubsection{$\\ell_1/2<\\ell_2/2<(\\ell_1+\\ell_2)/2<t$}\n\nIn this limit all the geodesics (solid curves)  are confined in the exterior region of the BTZ black string as depicted in Fig. \\ref{neggeoadjfin}(d). The resulting holographic entanglement negativity vanishes due to mutual cancellations as follows \n\\begin{equation}\\label{adjlimd}\n\\begin{aligned}\n\\mathcal{E} = &\\frac{3}{16G^3_N}\\left(\\mathcal{L}_{c'b'}+\\mathcal{L}_{cb}+\\mathcal{L}_{b'a'}+\\mathcal{L}_{ba}-\\mathcal{L}_{c'a'}-\\mathcal{L}_{ca}\\right),\\\\\n=&0\n\\end{aligned}\n\\end{equation}\n\nAs mentioned earlier the time evolution of the holographic entanglement negativity for two adjacent intervals after a global quench in a $CFT_{1+1}$ dual to a single sided black hole is obtained by taking half of the corresponding results for the eternal black hole case. Interestingly, it is observed that the time evolution of the holographic entanglement negativity in question following a global quench match exactly with the corresponding $CFT_{1+1}$ results computed through the replica technique in eq. (\\ref{ent_neg_glo_quench_adj_int}).\n\n\\begin{figure}[H]\n\\centering\n\\includegraphics[width=.5\\linewidth]{plotajdintneg.eps}\n\\caption{Holographic entanglement negativity $\\mathcal{E}$ vs time $t$ is plotted for two adjacent intervals of lengths $\\ell_1=50$, $\\ell_2=200$.}\\label{plotajdintneg}\n\\end{figure}\n\n\n\nThe time evolution of the holographic entanglement negativity is illustrated in Fig. \\ref{plotajdintneg}. The holographic entanglement negativity grows linearly for the initial part of its evolution and reaches to its maximum value at an intermediate time beyond which it is a constant upto a certain range of time. Beyond this range the holographic entanglement negativity decreases linearly and becomes zero for late times. Note that this behavior of the holographic entanglement negativity is consistent with that described in \\cite{Coser:2014gsa}. \n\n\n\n\n\n\n5.2.4 $\\ell_1/2<\\ell_2/2<(\\ell_1+\\ell_2)/2<t$\n\\subsubsection{$\\ell_1/2<\\ell_2/2<(\\ell_1+\\ell_2)/2<t$}\n\nIn this limit all the geodesics (solid curves)  are confined in the exterior region of the BTZ black string as depicted in Fig. \\ref{neggeoadjfin}(d). The resulting holographic entanglement negativity vanishes due to mutual cancellations as follows \n\\begin{equation}\\label{adjlimd}\n\\begin{aligned}\n\\mathcal{E} = &\\frac{3}{16G^3_N}\\left(\\mathcal{L}_{c'b'}+\\mathcal{L}_{cb}+\\mathcal{L}_{b'a'}+\\mathcal{L}_{ba}-\\mathcal{L}_{c'a'}-\\mathcal{L}_{ca}\\right),\\\\\n=&0\n\\end{aligned}\n\\end{equation}\n\nAs mentioned earlier the time evolution of the holographic entanglement negativity for two adjacent intervals after a global quench in a $CFT_{1+1}$ dual to a single sided black hole is obtained by taking half of the corresponding results for the eternal black hole case. Interestingly, it is observed that the time evolution of the holographic entanglement negativity in question following a global quench match exactly with the corresponding $CFT_{1+1}$ results computed through the replica technique in eq. (\\ref{ent_neg_glo_quench_adj_int}).\n\n\\begin{figure}[H]\n\\centering\n\\includegraphics[width=.5\\linewidth]{plotajdintneg.eps}\n\\caption{Holographic entanglement negativity $\\mathcal{E}$ vs time $t$ is plotted for two adjacent intervals of lengths $\\ell_1=50$, $\\ell_2=200$.}\\label{plotajdintneg}\n\\end{figure}\n\n\n\nThe time evolution of the holographic entanglement negativity is illustrated in Fig. \\ref{plotajdintneg}. The holographic entanglement negativity grows linearly for the initial part of its evolution and reaches to its maximum value at an intermediate time beyond which it is a constant upto a certain range of time. Beyond this range the holographic entanglement negativity decreases linearly and becomes zero for late times. Note that this behavior of the holographic entanglement negativity is consistent with that described in \\cite{Coser:2014gsa}. \n\n\n\n\n\n\n",
        "label": "plotajdintneg",
        "Descriptive_question1": "What is the length of the first interval (ℓ₁) in figure_10?",
        "Descriptive_question2": "What is the length of the second interval (ℓ₂) in figure_10?",
        "Reasoning_question1": "How does the holographic entanglement negativity (ℰ) change over time in figure_10 for the two adjacent intervals?",
        "Reasoning_question2": "What can be inferred about the relationship between the interval lengths and the behavior of holographic entanglement negativity (ℰ) at intermediate times in figure_10?",
        "Descriptive_answer1": "50",
        "Descriptive_answer2": "200",
        "Reasoning_answer1": "The holographic entanglement negativity ℰ initially starts at zero, then increases linearly with time until it reaches a maximum value around an intermediate time. After maintaining this maximum constant value for a certain period, ℰ then decreases linearly and eventually drops back to zero at late times. This indicates that entanglement grows and saturates before declining as time progresses after the global quench.",
        "Reasoning_answer2": "At intermediate times, the holographic entanglement negativity ℰ reaches a plateau whose height depends on the lengths of the intervals ℓ₁ and ℓ₂. The maximum value and duration over which ℰ remains constant are influenced by these lengths, as ℰ grows until time scales related to half the interval lengths and then stays steady before decreasing. This suggests that longer intervals sustain higher and longer-lasting entanglement negativity before decay sets in."
    },
    {
        "paper_id": "2107.07598.json",
        "image_id": "figure_2",
        "image_path": [
            "/home/yz979/scratch/chengye/scimolmo/test-images-tables/dataset_imagestables/2107.07598/images/Figure2.png"
        ],
        "caption": "1-1 plots of the kNN (subfigures (a) - (d)) and DNN (subfigures (e) - (h)) emulators' performance obtained for system 1 when the kNN training base contains 10,000 samples and the DNN is trained using the same 10,000 samples. Here ``true\" means the original PHREEQC-simulated data and ``predicted\" denotes the emulated (that is, kNN-simulated and DNN-simulated) data. Hence, the $x$-axis and $y$-axis present the original and emulated 10,000 independent test data points, respectively. The RMSE and $Q_2$ coefficient denote the root-mean-square-error and coefficient of determination in testing mode, respectively, between the original and emulated 10,000 test data points.",
        "classify": "Chart",
        "section_info": "3 Results\n\\section{Results}\n\\label{results}\n\n\\subsection{Ca-Si Problem}\nFor this first cement system, the emulation problem consists of predicting at each time step of the RT simulation the (output) Ca, Si, H and O aqueous concentrations (mol/kg of water or mol/kgw) from the (input) total amounts of Ca and Si (mol). \n\n\\subsubsection{Training the emulators}\n\\label{train_res1}\nHere the kNN and DNN emulators are firstly trained using a set of 400,000 test examples for both. This training set is obtained by randomly sampling the two-dimensional input space by latin hypercube sampling (LHS) between $\\left[0,0\\right]$ and $\\rm{\\left[Ca^{tot}_{max},Si^{tot}_{max}\\right]}$, and running PHREEQC for each input sample, $\\rm{\\textbf{x}_i = \\left[Ca^{tot}_i, Si^{tot}_i\\right]}$ to get the corresponding output vectors, $\\rm{\\textbf{y}_i = \\left[Ca^{conc}_i,Si^{conc}_i,H^{conc}_i,O^{conc}_i\\right]}$. The upper bounds, $\\rm{Ca^{tot}_{max}}$ and $\\rm{Si^{tot}_{max}}$ are defined based on a cheap full RT simulation with advective-dispersive transport using a small 1D domain of 51 nodes. It is worth noting that the total amounts of $\\textbf{x}_i$, corresponding to the PHREEQC-simulated concentrations, $\\textbf{y}_i$, have to be corrected for the different amount of water between the training set and the transport simulations. Doing so, it turns out that about 20 \\% of the post-corrected $x_i$ values exceed their pre-defined upper bounds and these excessively large values need to be filtered out. Creating the 400,000 training examples thus required about 500,000 PHREEQC runs. As stated earlier, for this problem single-threaded PHREEQC performs about 670 geochemical calculations per second on our used Intel\\textsuperscript{\\textregistered} i7 CPU.\n\nWith respect to kNN, the tuning parameters are the number of neighbors, $k$, the type of distance measure, and the interpolation technique. We simply used the default settings: $k = 5$, euclidean distance and inverse-distance interpolation. Regarding training of the DNN, the 400,000 sample were split between the training set itself (90 \\% of the data) and a validation set (10 \\% of the data). The latter serves to monitor the evolution of the selected mean squared error loss function on samples that are not used for training, thereby detecting potential overfitting. If the validation loss stops decreasing before the fixed number of epochs has been completed, then  training is stopped. Importantly, the emulation is achieved in log-space for both the input, $X$ and output $Y$, domains. This because total amounts and concentrations of the involved components typically cover many orders of magnitudes (up to 10 orders or more). Using a DNN also requires some form of data normalization or standardization. Here both the $\\rm{log\\left(\\textbf{x}_i\\right)}$ and $\\rm{log\\left(\\textbf{y}_i\\right)}$ vectors are standardized around 0 with standard deviation of 1.\n\nFigure \\ref{fig1} illustrates the trained emulators' performance for geochemical predictions using an independent test set that comprises 10,000 test examples. Both kNN (Figures \\ref{fig1}a - d) and DNN (Figures \\ref{fig1}e - h) appear to be rather accurate . DNN also shows a slight degradation for the larger concentration values  (Figures \\ref{fig1}e - h). The latter is likely due to the combination of a small proportion of large concentration values in the training set with the log-transformation that implicitly pushes the DNN to try harder to fit the smaller concentrations during training. Regarding speedup and as written earlier, for this setup the single-threaded kNN method is 300 times faster than single-threaded PHREEQC for predicting the 10,000 concentration vectors all at once. The computational savings allowed by the DNN emulator when ran on our GPU is higher with a speedup as large as 4000 for predicting the same 10,000 concentration vectors all at once. \n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{0cm}\\includegraphics[width=35pc]{Figure1.png}\n\t\\caption{1-1 plots of the kNN (subfigures (a) - (d)) and DNN (subfigures (e) - (h)) emulators' performance obtained for system 1 when the kNN training base contains 400,000 samples and the DNN is trained using the same 400,000 samples. Here ``true\" means the original PHREEQC-simulated data and ``predicted\" denotes the emulated (that is, kNN-simulated and DNN-simulated) data. Hence, the $x$-axis and $y$-axis present the original and emulated 10,000 independent test data points, respectively. The RMSE and $Q_2$ coefficient denote the root-mean-square-error and coefficient of determination in testing mode, respectively, between the original and emulated 10,000 test data points.}\n\t\\label{fig1}\n\\end{figure}\n\nTo test the sensitivity of the emulators' performance to the training set size, training was also performed using reduced training sets comprising 100,000, 10,000, 1000 and 100 samples, respectively. It seen that the DNN performance achieved  when using 10,000 training samples is virtually the same as that obtained when using 400,000 training samples (Figure \\ref{fig2} and Table \\ref{table2}). It is only for training sets smaller than 1000 samples that the DNN performance starts to degrade significantly (Table \\ref{table2}). In contrast, the behavior of kNN appears to markedly decrease as the training set gets smaller (Figure \\ref{fig2} and Table \\ref{table1}).\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{0cm}\\includegraphics[width=35pc]{Figure2.png}\n\t\\caption{1-1 plots of the kNN (subfigures (a) - (d)) and DNN (subfigures (e) - (h)) emulators' performance obtained for system 1 when the kNN training base contains 10,000 samples and the DNN is trained using the same 10,000 samples. Here ``true\" means the original PHREEQC-simulated data and ``predicted\" denotes the emulated (that is, kNN-simulated and DNN-simulated) data. Hence, the $x$-axis and $y$-axis present the original and emulated 10,000 independent test data points, respectively. The RMSE and $Q_2$ coefficient denote the root-mean-square-error and coefficient of determination in testing mode, respectively, between the original and emulated 10,000 test data points.}\n\t\\label{fig2}\n\\end{figure}\n\n\\begin{table}[h!]\n\t\\caption{Performance of the DNN and kNN emulators for cement system 1 and different training set sizes. For brevity, only the results for Ca$^{conc}$ and Si$^{conc}$ are shown. The units are mol per kg of water (mol/kgw). ML refers to the type of emulator, TR signifies the size of the training set, and the RMSE and $Q_2$ coefficient denote the root-mean-square-error and coefficient of determination in testing mode, respectively, between the original and emulated 10,000 test data points.}\n\t\\begin{center}\n\t\t\\begin{tabular}{cccccccc}\n\t\t\t\\hline\n\t\t\t& & & & & \\\\\n\t\t\tML & TR & RMSE - Ca$^{conc}$ & $Q_2$ - Ca$^{conc}$ & RMSE - Si$^{conc}$ & $Q_2$ - Si$^{conc}$\\\\\n\t\t\tDNN & 4 $\\times$ 10$^{5}$ & 4.50 $\\times$ 10$^{-5}$ & 0.9999 & 1.21 $\\times$ 10$^{-6}$ & 0.9999\\\\\n\t\t\tDNN & 1 $\\times$ 10$^{5}$ & 5.06 $\\times$ 10$^{-5}$ & 0.9999 & 1.30 $\\times$ 10$^{-6}$ & 0.9999\\\\\n\t\t\tDNN & 1 $\\times$ 10$^{4}$ & 4.57 $\\times$ 10$^{-5}$ & 0.9999 & 1.48 $\\times$ 10$^{-6}$ & 0.9999\\\\\n\t\t\tDNN & 1 $\\times$ 10$^{3}$ & 8.02 $\\times$ 10$^{-5}$ & 0.9998 & 3.31 $\\times$ 10$^{-6}$ & 0.9994\\\\\n\t\t\tDNN & 1 $\\times$ 10$^{2}$ & 47.1 $\\times$ 10$^{-5}$ & 0.9935 & 9.02 $\\times$ 10$^{-6}$ & 0.9959\\\\\n\t\t\tkNN & 4 $\\times$ 10$^{5}$ & 2.52 $\\times$ 10$^{-5}$ & 1.0000 & 1.59 $\\times$ 10$^{-6}$ & 0.9999\\\\\n\t\t\tkNN & 1 $\\times$ 10$^{5}$ & 6.27 $\\times$ 10$^{-5}$ & 0.9999 & 4.20 $\\times$ 10$^{-6}$ & 0.9991\\\\\n\t\t\tkNN & 1 $\\times$ 10$^{4}$ & 14.1 $\\times$ 10$^{-5}$ & 0.9994 & 13.5 $\\times$ 10$^{-6}$ & 0.9908\\\\\n\t\t\tkNN & 1 $\\times$ 10$^{3}$ & 43.9 $\\times$ 10$^{-5}$ & 0.9944 & 74.6 $\\times$ 10$^{-6}$ & 0.7180\\\\\n\t\t\tkNN & 1 $\\times$ 10$^{2}$  & 96.4 $\\times$ 10$^{-5}$ & 0.9729 & 55.0 $\\times$ 10$^{-6}$ & 0.8468\\\\\n\t\t\\end{tabular}\n\t\\end{center}\n\t\\label{table2}\n\\end{table}\n\n\\FloatBarrier\n\n\\subsubsection{Reactive transport simulation}\n\\label{prob12Dres}\n\nThis section focuses on reactive transport simulations with HPx$_{\\rm py}$ within cement system 1, under both advective-dispersive and diffusive transport conditions. As written above, the domain sizes are both 61 $\\times$ 61 and 121 $\\times$ 121 for the advection-dispersion case and, because of computational constraints, solely 61 $\\times$ 61  for the diffusion case. In addition, the simulation time period is 2 years for the advection-dispersion case and 1 year for the diffusion case. Figures \\ref{fig3} and \\ref{fig4} present times series of original and emulated Ca, Si, H and O concentrations at 5 locations within the 2D domain for advective-dispersive transport conditions, for both our kNN-based (HPx$_{\\rm py}$-kNN) and DNN-based (HPx$_{\\rm py}$-DNN) reactive transport codes. It is seen that HPx$_{\\rm py}$-kNN and HPx$_{\\rm py}$-DNN both induce a quite good simulation accuracy. Also, the results for the diffusive transport case are of similarly good quality (not shown). Figures \\ref{fig5} - \\ref{fig6} provide more insights into the HPx$_{\\rm py}$-kNN and HPx$_{\\rm py}$-DNN performances by displaying 2D Ca, Si, H and O concentration profiles at a given time. For each experiment and chemical component, this time is selected as to be well representative of the simulated dynamics. It is observed that the original and emulated images are visually almost indistinguishable for the advection-dispersion case (Figure \\ref{fig5}). For the diffusion case, the emulators also perform quite well for Ca, H and O (Figures\\ref{fig6}a - c, g - i and j - l), while some slight to moderate discrepancies appear at the concentration front for Si (Figures \\ref{fig6}d - f). Notwithstanding, the Si concentration remains globally well predicted. Furthermore, Figures \\ref{fig7} - \\ref{fig8} present the original and emulated 2D solid amount profiles corresponding to Figures \\ref{fig5} - \\ref{fig6}. The original solid amount profiles are overall well approximated by HPx$_{\\rm py}$-kNN and HPx$_{\\rm py}$-DNN for the advection-dispersion case (Figure \\ref{fig7}), even though some mismatch appears at the border of the fully depleted zone for the H component. As of the diffusion case (Figure \\ref{fig8}), the same kind of mismatch is observed for the emulated solid amounts of H by HPx$_{\\rm py}$-kNN while the emulated profiles by HPx$_{\\rm py}$-DNN show some more discrepancies. Though we decided to present raw emulation results, we would like to stress that some if not all of the observed artifacts could likely be smoothed out by using some post-filtering such as median filtering.\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure3.png}\n\t\\caption{Time series of original (RTM, solid green lines) and HPx$_{\\rm py}$-kNN emulated (TM+kNN, dashed orange lines) concentrations (mol/kg) of Ca, H, O and Si at selected observations points for cement system 1 and advective-dispersive transport. Obs. 1 - 5 denote the selected observation points, with the following $\\left[x,y\\right]$ locations (in cm). Obs. 1:  $\\left[0.5, 2.5\\right]$, Obs. 2: $\\left[1, 2\\right]$, Obs. 3: $\\left[2, 2\\right]$, Obs. 4: $\\left[1, 1\\right]$, Obs. 5: $\\left[2, 1\\right]$. The results for the 121 $\\times$ 121 grid size are rather similar.}\n\t\\label{fig3}\n\\end{figure}\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure4.png}\n\t\\caption{Time series of original (RTM, solid green lines) and HPx$_{\\rm py}$-DNN emulated (TM+DNN, dashed orange lines) concentrations (mol/kg) of Ca, H, O and Si at selected observations points for cement system 1 and advective-dispersive transport. Obs. 1 - 5 denote the selected observation points, with the following $\\left[x,y\\right]$ locations (in cm). Obs. 1:  $\\left[0.5, 2.5\\right]$, Obs. 2: $\\left[1, 2\\right]$, Obs. 3: $\\left[2, 2\\right]$, Obs. 4: $\\left[1, 1\\right]$, Obs. 5: $\\left[2, 1\\right]$. The results for the 121 $\\times$ 121 grid size are rather similar.}\n\t\\label{fig4}\n\\end{figure}\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure5.png}\n\t\\caption{2D concentration profiles obtained for cement system 1 at the end of the 2-year simulation performed for the advection-dispersion case. RTM means the original HP$_{\\rm{4C}}$ model, TM+kNN denotes the Hydrus transport model coupled with our kNN geochemical emulator (HPx$_{\\rm py}$-kNN) and TM+DNN signifies the Hydrus transport model coupled with our DNN geochemical emulator (HPx$_{\\rm py}$-DNN). The first to fourth row present profiles for $Ca^{\\rm{conc}}$, $Si^{\\rm{conc}}$, $H^{\\rm{conc}}$, and $O^{\\rm{conc}}$, respectively. The considered grid size is 61 $\\times$ 61. The results for the 121 $\\times$ 121 grid are rather similar.}\n\t\\label{fig5}\n\\end{figure}\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure6.png}\n\t\\caption{2D concentration profiles obtained for cement system 1 at the final time step of the 1-year simulation performed for the diffusion case. RTM means the original HP$_{\\rm{4C}}$ model, TM+kNN denotes the Hydrus transport model coupled with our kNN geochemical emulator (HPx$_{\\rm py}$-kNN) and TM+DNN signifies the Hydrus transport model coupled with our DNN geochemical emulator (HPx$_{\\rm py}$-DNN). The first to fourth row present profiles for $Ca^{\\rm{conc}}$, $Si^{\\rm{conc}}$, $H^{\\rm{conc}}$, and $O^{\\rm{conc}}$, respectively. The considered grid size is 61 $\\times$ 61.}\n\t\\label{fig6}\n\\end{figure}\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure7.png}\n\t\\caption{2D solid amount profiles obtained for cement system 1 at the end of the 2-year simulation performed for the advection-dispersion case. RTM means the original HP$_{\\rm{4C}}$ model, TM+kNN denotes the Hydrus transport model coupled with our kNN geochemical emulator (HPx$_{\\rm py}$-kNN) and TM+DNN signifies the Hydrus transport model coupled with our DNN geochemical emulator (HPx$_{\\rm py}$-DNN). The first to fourth row present profiles for $Ca^{\\rm{solid}}$, $Si^{\\rm{solid}}$, $H^{\\rm{solid}}$, and $O^{\\rm{solid}}$, respectively. The considered grid size is 61 $\\times$ 61. The results for the 121 $\\times$ 121 grid are rather similar.}\n\t\\label{fig7}\n\\end{figure}\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure8.png}\n\t\\caption{2D solid amount profiles obtained for cement system 1 at the final time step of the 1-year simulation performed for the diffusion case. RTM means the original HP$_{\\rm{4C}}$ model, TM+kNN denotes the Hydrus transport model coupled with our kNN geochemical emulator (HPx$_{\\rm py}$-kNN) and TM+DNN signifies the Hydrus transport model coupled with our DNN geochemical emulator (HPx$_{\\rm py}$-DNN). The first to fourth row present profiles for $Ca^{\\rm{solid}}$, $Si^{\\rm{solid}}$, $H^{\\rm{solid}}$, and $O^{\\rm{solid}}$, respectively. The considered grid size is 61 $\\times$ 61.}\n\t\\label{fig8}\n\\end{figure}\n\nThe speedups associated with the considered problem are detailed in Table \\ref{table3}. It is noted that the GPU-based DNN emulator allows for a speedup that is close to optimal. Indeed, The DNN speedups overall represent 85 \\% to 95 \\% of the maximum possible speedups (that is, speedups that would be obtained if the geochemcical calculations would come at no cost at all). The speedups associated with single-threaded kNN remain substantial but only amount to 57 \\% - 65 \\% of the corresponding maximum speedups. As detailed in section \\ref{train_res1}, the used kNN and DNN implementations are found to be respectively 300 and 4000 faster than single-threaded PHREEQC when predicting 10,000 points all one once for this geochemical system. Based on these numbers one could have expected the achieved speedups to represent say 90 \\% (kNN) or 99 \\% (DNN) of the maximum possible ones. A large part of the gaps between achieved and maximum possible speedups is thus likely caused by the time required for communicating and exchanging data between the main C/C++ code and the Python-based emulators.\n\n\\begin{table}[h!]\n\t\\caption{Speedups offered by the KNN and DNN emulators in HPx$_{\\rm py}$ for the reactive transport simulations considered for cement system 1. The HPx$_{\\rm{4C}}$ calculations involve the parallelization of PHREEQC over our 4 CPUs. The HPx$_{\\rm{1C}}$ calculations are performed on a single CPU. The kNN predictions are performed on a single CPU using the scikit-learn implementation while the DNN predictions make use of our GPU. ML signifies the used machine learning method for emulation, TC denotes transport conditions (ADV: advection-dispersion, DIF: diffusion) and GS is the grid size. The maximum possible speedups associated with HPx$_{\\rm{4C}}$ and HPx$_{\\rm{1C}}$,  Max SP HPx$_{\\rm{4C}}$ and  Max SP HPx$_{\\rm{1C}}$, correspond to an hypothetical situation where the geochemical calculations incur zero computational cost.}\n\n\t\\begin{center}\n\t\t\\begin{tabular}{cccccccc}\n\t\t\t\\hline\n\t\t\t& & & & & & & \\\\\n\t\t\tML & TC & GS & HPx$_{\\rm{4C}}$ time (s) & SP HPx$_{\\rm{4C}}$ & Max SP HPx$_{\\rm{4C}}$ & SP HPx$_{\\rm{1C}}$ & Max SP HPx$_{\\rm{1C}}$ \\\\\n\t\t\tDNN & ADV & 61 $\\times$ 61 & 3189 & 6.8 & 7.7 & 24.5 & 28.5 \\\\\n\t\t\tkNN & ADV & 61 $\\times$ 61 & 3189 & 5.0 & 7.7 & 18.0 & 28.5 \\\\\n\t\t\tDNN & ADV & 121 $\\times$ 121 & 23,337 & 5.0 & 5.2 & 17.0 & 18.0 \\\\\n\t\t\tkNN & ADV & 121 $\\times$ 121 & 23,337 & 3.4 & 5.2 & 11.6 & 18.0 \\\\\n\t\t\tDNN & DIF & 61 $\\times$ 61 & 25,448 & 4.2 & 4.9 & 16.2 & 19.1 \\\\\n\t\t\tkNN & DIF & 61 $\\times$ 61 & 25,448 & 2.8 & 4.9 & 10.8 & 19.1 \\\\\n\t\t\t\\hline\n\t\t\\end{tabular}\n\t\\end{center}\n\n\t\\label{table3}\n\\end{table}\n\n\\FloatBarrier\n\n\\subsection{Al-C-Ca-S-Si Problem}\n\nFor this second cement system, the emulation problem consists of predicting at each time step of the RT simulation the (output) Al, C, Ca, S, Si, H and O aqueous concentrations (mol/kgw) from the (input) total amounts of Al, C, Ca, S and Si (mol). Here we focus on advective-dispersive transport only while similarly as for cement system 1, the considered domain sizes are 61 $\\times$ 61 and 121 $\\times$ 121. Moreover, the simulation time period is set to 6 years. \n\nAs mentioned earlier, for this higher-dimensional problem it is observed that the used scikit-learn kNN implementation becomes prohibitively slow compared to HPx$_{\\rm{4C}}$. We found that to get a good emulation accuracy the kNN training base needs to contain 1,000,000 samples (or more). This training base's size together with a 5-dimensional search space leads to an HPx$_{\\rm py}$-kNN reactive transport simulation time that is comparable to that of HPx$_{\\rm{4C}}$. Therefore, for this second cement system we built a custom kNN regressor around another kNN implementation contained in the FAISS package \\citep{faiss2017}. Our used FAISS variant allows for GPU computing and is much faster than scikit-learn for this cement system, but is slightly less accurate due to the use of an approximate rather than exact nearest neighbor search \\citep[see][for details]{faiss2017}.\n\n\\subsubsection{Training the DNN}\n\\label{train_res2}\n\nBuilding a good training set to perform a kNN search and learn the weights and biases of our DNN turned out to be a complicated task in this case. This because to make useful kNN predictions and/or learn an useful DNN, the training set must be sufficiently representative of the geochemical conditions encountered during the reactive transport simulation one wish to perform with HPx$_{\\rm py}$-kNN and the trained HPx$_{\\rm py}$-DNN. In contrast to cement system 1, creating the training set by sampling the $X$-space with a controlled randomness between predefined lower and upper bounds did not prove successful. We tried that strategy by drawing as much as 4,000,000 5-dimensional $\\textbf{x}$ vectors from the $X$-space using a Sobol low-discrepancy sequence \\citep[][]{Sobol1967, Joe-Kuo2003}. Such low-discrepancy sampling scheme covers the 5-dimensional hypercube more uniformly than LHS. Despite a good performance on the test set (not shown), the resulting DNN accuracy in reactive transport mode was never deemed satisfying. In other words, no satisfying ``global\" or ``universal\" DNN emulator could be devised for this cement system. This is probably caused by the fact for this problem, the input (5 total amounts) and output (7 aqueous concentrations) spaces are quite nonlinearly related and both cover 6 to 10 orders of magnitudes depending on the considered element. Therefore, we resorted to the alternative training strategy detailed below. The latter basically tries to grasp the complex correlations and higher-order dependencies that exist between the elements of $\\textbf{x}$ (total amounts, input space) for a given reactive transport simulation setup, in order to produce a training set that honors these between-input relationships.\n\n\\begin{itemize}\n\t\\item Perform a ``cheap\" full reactive transport simulation under the transport conditions and geochemistry of interest and collect the resulting $\\textbf{x}$ -$\\textbf{y}$  pairs of examples (for the considered grid nodes and time steps). Computational demand controls what domain size and simulation time period can be used for this cheap calculation. We used a modest 16 $\\times$ 16 domain and a simulation time period of 10 years. The associated HPx$_{\\rm{4C}}$ runtimes is 180 s.\n\t\n\t\\item Fit a kernel density estimator (KDE) with a Gaussian kernel to the collected $\\textbf{x}$ vectors (encapsulated in the $\\textbf{X}$ array) and generate a fixed number of new input vectors, $\\textbf{x}_{KDE}$. Then run PHREEQC for the $\\textbf{X}_{KDE}$ set to get the corresponding output set, $\\textbf{Y}_{KDE}$. Now apply the correction for porosity described in section \\ref{train_res1} to the $\\textbf{X}_{KDE}$ set and form the training set by merging the ensemble of $\\textbf{x}$-$\\textbf{y}$ pairs with that of the $\\textbf{x}_{KDE}$-$\\textbf{y}_{KDE}$ pairs. The number of produced unique examples by the considered cheap HPx$_{\\rm{4C}}$ simulations varied between 10,000 and 50,000. The KDE-based enrichment of this dataset was deemed necessary to provide more input variability, thereby avoiding overfitting of the trained DNN and improving the kNN accuracy, while still honoring the complex between-input relationships. The number of KDE-generated samples was set as to obtain a total training set size of 1,000,000 examples. A key component of the approach is the bandwith parameter of the Gaussian KDE kernel which controls how much the KDE-generated samples depart from the original ensemble. After limited trial and error, we fixed the kernel bandwith to 0.0025 for the considered case studies.\n\t\n\\end{itemize}\n\nThe scatter plots in Figure \\ref{fig9} illustrate our training set creation procedure. The orange dots depict the pairwise relationships observed between the 5 elements of $\\textbf{x}$ in the cheap simulation. The red and cyan dots in Figure \\ref{fig9} represent the KDE-generated samples with the selected bandwidth, before and after applying the correction for porosity, respectively. Training of the DNN is achieved using the ensemble of original and KDE-corrected input points. We refer to this kind of dataset as RT-based, since it is based on an albeit cheap, full RT simulation. Furthermore, we refer to the obtained DNN and kNN emulators as ``local\" emulators, since as opposed to the emulators constructed for cement system 1, the current emulators are only valid for the input conditions encapsulated in the RT-based training set.\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1.5cm}\\includegraphics[width=50pc]{Figure9.png}\n\t\\caption{Scatter plots of the complex relationships between the five considered inputs for (1) the computationally cheap RT simulation of cement system 2 (orange dots) performed with the original HPx code, (2) the corresponding sampled points by kernel density estimation (KDE) using the selected bandwith (turquoise dots) and, (3) the same sampled KDE points after porosity correction (red dots).}\n\t\\label{fig9}\n\\end{figure}\n\nTraining performance of the DNN emulator is presented in Figure \\ref{fig10}. Similarly as for cement system 1, about 90\\% of the available data was used for the actual training of the DNN while the remaining 10\\% were used a validation set to contol overfitting. Lastly, performance is evaluated for both the trained DNN and kNN emulators using an independent test set of 10,000 examples. Overall, the accuracy of our ``local \" DNN emulator for this RT-inspired dataset is rather large with $Q_2$ values always greater or equal than 0.998. Training performance of the corresponding local kNN emulator is equally good (not shown). Regarding speedup, for this problem single-threaded PHREEQC achieves about 210 geochemical calculations per second on our used Intel\\textsuperscript{\\textregistered} i7 CPU while the GPU-based DNN and kNN emulators are both about 3000 times faster when predicting the 10,000 test points at once .\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure10.png}\n\t\\caption{1-1 plots of local DNN emulation performance obtained for system 2 when the local DNN is trained using 1,000,000 samples. The $x$-axis and $y$-axis present the original and emulated 10,000 independent test data points, respectively. The RMSE and $Q_2$ coefficient denote the root-mean-square-error and coefficient of determination in testing mode, respectively, between the original and emulated 10,000 test data points.}\n\t\\label{fig10}\n\\end{figure}\n\n\\subsubsection{Reactive transport simulation}\n\\label{prob2res}\nOur ``local\" DNN performs rather well when applied to the 61 $\\times$ 61 grid size and a time period of 6 years (Figures \\ref{fig11} - \\ref{fig12}). This for all components but C, for which some localized deviations appear between original and emulated 2D concentration profiles towards the end of the simulation period (Figure \\ref{fig12}). When applied to the 121 $\\times$ 121 grid, HPx$_{\\rm py}$-DNN produces additional discrepancies for O, Si and Al towards the end of the simulation period (Figures \\ref{fig13} - \\ref{fig14}). Yet most of the observed artifacts could probably be smoothed out by using post-filtering. The associated speedups are listed in Table \\ref{table4}. These speedups are larger to those obtained for cement system 1, with values between 8 and 9 when evaluated against HPx$_{\\rm{4C}}$. These speedups represent about 85 \\% to 90 \\% of the maximum possible speedup (Table \\ref{table3}). Overall, these findings indicate that for the considered problem, our RT-based training of a local DNN only works is the training set is sufficiently representative of the particular geochemical conditions encountered in the computationally demanding simulations, which is arguably not easy to achieve. This limitation is further discussed in section \\ref{discussion}.\n\nWe note a more uniform behavior for HPx$_{\\rm py}$-kNN across grid sizes than for HPx$_{\\rm py}$-DNN. Here the results for the 121  $\\times$ 121 grid are only slightly less accurate than those associated with the 61 $ \\times$ 61 grid (see Figures \\ref{fig15} - \\ref{fig16} where for brevity we only show concentration profiles for the C Al and S components). Furthermore, whenever observed the discrepancies between original and emulated profiles are more regularly scattered than for HPx$_{\\rm py}$-DNN. Note also that herein too, post-filtering could likely smooth out a large part of these deviations. In addition, owing to the use of a GPU to achieve the kNN calculations, the speedup provided by HPx$_{\\rm py}$-kNN are as large as those provided by HPx$_{\\rm py}$-DNN (Table \\ref{table4}).\n\nWith respect to the emulated solid amounts, the HPx$_{\\rm py}$-DNN results look visually good for the the 61 $\\times$ 61 grid. This is shown in Figure \\ref{fig17} for the C, Al and S chemical components, while emulation of the H, O Ca and Si chemical components is globally of similar quality (not shown). Nevertheless, significant deviations appear towards the end of the simulation period for every chemical component (see Figure \\ref{fig18} for the C, Al and S chemical components, emulation of the H, O Ca and Si chemical components shows the same level of mismatch). The HPx$_{\\rm py}$-kNN predictions are also fairly accurate for the 61 $\\times$ 61 (see Figure \\ref{fig19} for the C, Al and S chemical components, emulation of the H, O Ca and Si chemical components exhibits a globally similar quality) while some discrepancies show up at the end of the simulation (Figure \\ref{fig20}). However, the mismatch is less pronounced than for HPx$_{\\rm py}$-DNN. Overall, HPx$_{\\rm py}$-kNN appears to be somewhat more robust than HPx$_{\\rm py}$-DNN for this cement system, while providing the same (large) speedup.\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure11.png}\n\t\\caption{2D concentration profiles obtained for cement system 2 after 2 and 6 (final time step) years. RTM means the original HPx$_{\\rm{4C}}$ model and TM+DNN denotes the Hydrus transport model coupled with our DNN geochemical emulator (HPx$_{\\rm py}$-DNN). The first to fourth row present profiles for $Ca^{\\rm{conc}}$, $Si^{\\rm{conc}}$, $H^{\\rm{conc}}$, and $O^{\\rm{conc}}$, respectively. The considered grid size is 61 $\\times$ 61.}\n\t\\label{fig11}\n\\end{figure}\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure12.png}\n\t\\caption{2D concentration profiles obtained for cement system 2 after 2 and 6 (final time step) years. RTM means the original HPx$_{\\rm{4C}}$ model and TM+DNN denotes the Hydrus transport model coupled with our DNN geochemical emulator (HPx$_{\\rm py}$-DNN). The first to third row present profiles for $C^{\\rm{conc}}$, $Al^{\\rm{conc}}$, and $S^{\\rm{conc}}$, respectively. The considered grid size is 61 $\\times$ 61.}\n\t\\label{fig12}\n\\end{figure}\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure13.png}\n\t\\caption{2D concentration profiles obtained for cement system 2 after 2 and 6 (final time step) years. RTM means the original HPx$_{\\rm{4C}}$ model and TM+DNN denotes the Hydrus transport model coupled with our DNN geochemical emulator (HPx$_{\\rm py}$-DNN). The first to fourth row present profiles for $Ca^{\\rm{conc}}$, $Si^{\\rm{conc}}$, $H^{\\rm{conc}}$, and $O^{\\rm{conc}}$, respectively. The considered grid size is 121 $\\times$ 121.}\n\t\\label{fig13}\n\\end{figure}\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure14.png}\n\t\\caption{2D concentration profiles obtained for cement system 2 after 2 and 6 (final time step) years. RTM means the original HPx$_{\\rm{4C}}$ model and TM+DNN denotes the Hydrus transport model coupled with our DNN geochemical emulator (HPx$_{\\rm py}$-DNN). The first to third row present profiles for $C^{\\rm{conc}}$, $Al^{\\rm{conc}}$, and $S^{\\rm{conc}}$, respectively. The considered grid size is 121 $\\times$ 121.}\n\t\\label{fig14}\n\\end{figure}\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure15.png}\n\t\\caption{2D concentration profiles obtained for cement system 2 after 2 and 6 (final time step) years. RTM means the original HPx$_{\\rm{4C}}$ model and TM+kNN denotes the Hydrus transport model coupled with our kNN geochemical emulator (HPx$_{\\rm py}$-kNN).  The first to third row present profiles for $C^{\\rm{conc}}$, $Al^{\\rm{conc}}$, and $S^{\\rm{conc}}$, respectively. The considered grid size is 61 $\\times$ 61.}\n\t\\label{fig15}\n\\end{figure}\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure16.png}\n\t\\caption{2D concentration profiles obtained for cement system 2 after 2 and 6 (final time step) years. RTM means the original HPx$_{\\rm{4C}}$ model and TM+kNN denotes the Hydrus transport model coupled with our kNN geochemical emulator (HPx$_{\\rm py}$-kNN). The first to third row present profiles for $C^{\\rm{conc}}$, $Al^{\\rm{conc}}$, and $S^{\\rm{conc}}$, respectively. The considered grid size is 121 $\\times$ 121.}\n\t\\label{fig16}\n\\end{figure}\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure17.png}\n\t\\caption{2D solid amount profiles of the C, Al and S chemical components of cement system 2 after 2 and 6 (final time step) years. RTM means the original HPx$_{\\rm{4C}}$ model and TM+DNN denotes the Hydrus transport model coupled with our DNN geochemical emulator (HPx$_{\\rm py}$-DNN). The first to third row present profiles for $C^{\\rm{solid}}$, $Al^{\\rm{solid}}$, and $S^{\\rm{solid}}$, respectively The considered grid size is 61 $\\times$ 61.}\n\t\\label{fig17}\n\\end{figure}\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure18.png}\n\t\\caption{2D solid amount profiles of the C, Al and S chemical components of cement system 2 after 2 and 6 (final time step) years. RTM means the original HPx$_{\\rm{4C}}$ model and TM+DNN denotes the Hydrus transport model coupled with our DNN geochemical emulator (HPx$_{\\rm py}$-DNN). The first to third row present profiles for $C^{\\rm{solid}}$, $Al^{\\rm{solid}}$, and $S^{\\rm{solid}}$, respectively The considered grid size is 121 $\\times$ 121.}\n\t\\label{fig18}\n\\end{figure}\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure19.png}\n\t\\caption{2D solid amount profiles of the C, Al and S chemical components of cement system 2 after 2 and 6 (final time step) years. RTM means the original HPx$_{\\rm{4C}}$ model and TM+kNN denotes the Hydrus transport model coupled with our kNN geochemical emulator (HPx$_{\\rm py}$-kNN). The first to third row present profiles for $C^{\\rm{solid}}$, $Al^{\\rm{solid}}$, and $S^{\\rm{solid}}$, respectively The considered grid size is 61 $\\times$ 61.}\n\t\\label{fig19}\n\\end{figure}\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure20.png}\n\t\\caption{2D solid amount profiles of the C, Al and S chemical components of cement system 2 after 2 and 6 (final time step) years. RTM means the original HPx$_{\\rm{4C}}$ model and TM+kNN denotes the Hydrus transport model coupled with our kNN geochemical emulator (HPx$_{\\rm py}$-kNN). The first to third row present profiles for $C^{\\rm{solid}}$, $Al^{\\rm{solid}}$, and $S^{\\rm{solid}}$, respectively The considered grid size is 121 $\\times$ 121.}\n\t\\label{fig20}\n\\end{figure}\n\n\\begin{table}[h!]\n\t\\caption{Speedups offered by the KNN and DNN emulators in HPx$_{\\rm py}$ for the reactive transport simulations considered for cement system 2. The HPx$_{\\rm{4C}}$ calculations involve the parallelization of PHREEQC over our 4 CPUs. The HPx$_{\\rm{1C}}$ calculations are performed on a single CPU. Both the kNN and DNN predictions make use of a GPU. ML signifies the used machine learning method for emulation, BC denotes the type of flow boundary conditions and GS is the grid size. The maximum possible speedups associated with HPx$_{\\rm{4C}}$ and HP$_{\\rm{1C}}$,  Max SP HPx$_{\\rm{4C}}$ and  Max SP HPx$_{\\rm{1C}}$, correspond to an hypothetical situation where the geochemical calculations incur zero computational cost.}\n\n\t\\begin{center}\n\t\t\\begin{tabular}{cccccccc}\n\t\t\t\\hline\n\t\t\t& & & & & & & \\\\\n\t\t\tML & BC & GS & HPx$_{\\rm{4C}}$ time (s) & SP HPx$_{\\rm{4C}}$ & Max SP HPx$_{\\rm{4C}}$ & SP HPx$_{\\rm{1C}}$ & Max SP HPx$_{\\rm{1C}}$ \\\\\n\t\t\tDNN & ADV & 61 $\\times$ 61 & 21,415 & 8.2 & 9.0 & 30.3 & 33.1\\\\\n\t\t\tkNN & ADV & 61 $\\times$ 61 & 21,415 & 7.9 & 9.0 & 28.9 & 33.1\\\\\n\t\t\tDNN & ADV & 121 $\\times$ 121 & 199,841 & 8.2 & 9.5 & 29.9 & 32.8\\\\\n\t\t\tkNN & ADV & 121 $\\times$ 121 & 199,841 & 8.5 & 9.5 & 31.3 & 32.8\\\\\n\t\t\t\\hline\n\t\t\\end{tabular}\n\t\\end{center}\n\n\t\\label{table4}\n\\end{table}\n\n\\FloatBarrier\n\n3.1 Ca-Si Problem\n\\subsection{Ca-Si Problem}\nFor this first cement system, the emulation problem consists of predicting at each time step of the RT simulation the (output) Ca, Si, H and O aqueous concentrations (mol/kg of water or mol/kgw) from the (input) total amounts of Ca and Si (mol). \n\n\\subsubsection{Training the emulators}\n\\label{train_res1}\nHere the kNN and DNN emulators are firstly trained using a set of 400,000 test examples for both. This training set is obtained by randomly sampling the two-dimensional input space by latin hypercube sampling (LHS) between $\\left[0,0\\right]$ and $\\rm{\\left[Ca^{tot}_{max},Si^{tot}_{max}\\right]}$, and running PHREEQC for each input sample, $\\rm{\\textbf{x}_i = \\left[Ca^{tot}_i, Si^{tot}_i\\right]}$ to get the corresponding output vectors, $\\rm{\\textbf{y}_i = \\left[Ca^{conc}_i,Si^{conc}_i,H^{conc}_i,O^{conc}_i\\right]}$. The upper bounds, $\\rm{Ca^{tot}_{max}}$ and $\\rm{Si^{tot}_{max}}$ are defined based on a cheap full RT simulation with advective-dispersive transport using a small 1D domain of 51 nodes. It is worth noting that the total amounts of $\\textbf{x}_i$, corresponding to the PHREEQC-simulated concentrations, $\\textbf{y}_i$, have to be corrected for the different amount of water between the training set and the transport simulations. Doing so, it turns out that about 20 \\% of the post-corrected $x_i$ values exceed their pre-defined upper bounds and these excessively large values need to be filtered out. Creating the 400,000 training examples thus required about 500,000 PHREEQC runs. As stated earlier, for this problem single-threaded PHREEQC performs about 670 geochemical calculations per second on our used Intel\\textsuperscript{\\textregistered} i7 CPU.\n\nWith respect to kNN, the tuning parameters are the number of neighbors, $k$, the type of distance measure, and the interpolation technique. We simply used the default settings: $k = 5$, euclidean distance and inverse-distance interpolation. Regarding training of the DNN, the 400,000 sample were split between the training set itself (90 \\% of the data) and a validation set (10 \\% of the data). The latter serves to monitor the evolution of the selected mean squared error loss function on samples that are not used for training, thereby detecting potential overfitting. If the validation loss stops decreasing before the fixed number of epochs has been completed, then  training is stopped. Importantly, the emulation is achieved in log-space for both the input, $X$ and output $Y$, domains. This because total amounts and concentrations of the involved components typically cover many orders of magnitudes (up to 10 orders or more). Using a DNN also requires some form of data normalization or standardization. Here both the $\\rm{log\\left(\\textbf{x}_i\\right)}$ and $\\rm{log\\left(\\textbf{y}_i\\right)}$ vectors are standardized around 0 with standard deviation of 1.\n\nFigure \\ref{fig1} illustrates the trained emulators' performance for geochemical predictions using an independent test set that comprises 10,000 test examples. Both kNN (Figures \\ref{fig1}a - d) and DNN (Figures \\ref{fig1}e - h) appear to be rather accurate . DNN also shows a slight degradation for the larger concentration values  (Figures \\ref{fig1}e - h). The latter is likely due to the combination of a small proportion of large concentration values in the training set with the log-transformation that implicitly pushes the DNN to try harder to fit the smaller concentrations during training. Regarding speedup and as written earlier, for this setup the single-threaded kNN method is 300 times faster than single-threaded PHREEQC for predicting the 10,000 concentration vectors all at once. The computational savings allowed by the DNN emulator when ran on our GPU is higher with a speedup as large as 4000 for predicting the same 10,000 concentration vectors all at once. \n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{0cm}\\includegraphics[width=35pc]{Figure1.png}\n\t\\caption{1-1 plots of the kNN (subfigures (a) - (d)) and DNN (subfigures (e) - (h)) emulators' performance obtained for system 1 when the kNN training base contains 400,000 samples and the DNN is trained using the same 400,000 samples. Here ``true\" means the original PHREEQC-simulated data and ``predicted\" denotes the emulated (that is, kNN-simulated and DNN-simulated) data. Hence, the $x$-axis and $y$-axis present the original and emulated 10,000 independent test data points, respectively. The RMSE and $Q_2$ coefficient denote the root-mean-square-error and coefficient of determination in testing mode, respectively, between the original and emulated 10,000 test data points.}\n\t\\label{fig1}\n\\end{figure}\n\nTo test the sensitivity of the emulators' performance to the training set size, training was also performed using reduced training sets comprising 100,000, 10,000, 1000 and 100 samples, respectively. It seen that the DNN performance achieved  when using 10,000 training samples is virtually the same as that obtained when using 400,000 training samples (Figure \\ref{fig2} and Table \\ref{table2}). It is only for training sets smaller than 1000 samples that the DNN performance starts to degrade significantly (Table \\ref{table2}). In contrast, the behavior of kNN appears to markedly decrease as the training set gets smaller (Figure \\ref{fig2} and Table \\ref{table1}).\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{0cm}\\includegraphics[width=35pc]{Figure2.png}\n\t\\caption{1-1 plots of the kNN (subfigures (a) - (d)) and DNN (subfigures (e) - (h)) emulators' performance obtained for system 1 when the kNN training base contains 10,000 samples and the DNN is trained using the same 10,000 samples. Here ``true\" means the original PHREEQC-simulated data and ``predicted\" denotes the emulated (that is, kNN-simulated and DNN-simulated) data. Hence, the $x$-axis and $y$-axis present the original and emulated 10,000 independent test data points, respectively. The RMSE and $Q_2$ coefficient denote the root-mean-square-error and coefficient of determination in testing mode, respectively, between the original and emulated 10,000 test data points.}\n\t\\label{fig2}\n\\end{figure}\n\n\\begin{table}[h!]\n\t\\caption{Performance of the DNN and kNN emulators for cement system 1 and different training set sizes. For brevity, only the results for Ca$^{conc}$ and Si$^{conc}$ are shown. The units are mol per kg of water (mol/kgw). ML refers to the type of emulator, TR signifies the size of the training set, and the RMSE and $Q_2$ coefficient denote the root-mean-square-error and coefficient of determination in testing mode, respectively, between the original and emulated 10,000 test data points.}\n\t\\begin{center}\n\t\t\\begin{tabular}{cccccccc}\n\t\t\t\\hline\n\t\t\t& & & & & \\\\\n\t\t\tML & TR & RMSE - Ca$^{conc}$ & $Q_2$ - Ca$^{conc}$ & RMSE - Si$^{conc}$ & $Q_2$ - Si$^{conc}$\\\\\n\t\t\tDNN & 4 $\\times$ 10$^{5}$ & 4.50 $\\times$ 10$^{-5}$ & 0.9999 & 1.21 $\\times$ 10$^{-6}$ & 0.9999\\\\\n\t\t\tDNN & 1 $\\times$ 10$^{5}$ & 5.06 $\\times$ 10$^{-5}$ & 0.9999 & 1.30 $\\times$ 10$^{-6}$ & 0.9999\\\\\n\t\t\tDNN & 1 $\\times$ 10$^{4}$ & 4.57 $\\times$ 10$^{-5}$ & 0.9999 & 1.48 $\\times$ 10$^{-6}$ & 0.9999\\\\\n\t\t\tDNN & 1 $\\times$ 10$^{3}$ & 8.02 $\\times$ 10$^{-5}$ & 0.9998 & 3.31 $\\times$ 10$^{-6}$ & 0.9994\\\\\n\t\t\tDNN & 1 $\\times$ 10$^{2}$ & 47.1 $\\times$ 10$^{-5}$ & 0.9935 & 9.02 $\\times$ 10$^{-6}$ & 0.9959\\\\\n\t\t\tkNN & 4 $\\times$ 10$^{5}$ & 2.52 $\\times$ 10$^{-5}$ & 1.0000 & 1.59 $\\times$ 10$^{-6}$ & 0.9999\\\\\n\t\t\tkNN & 1 $\\times$ 10$^{5}$ & 6.27 $\\times$ 10$^{-5}$ & 0.9999 & 4.20 $\\times$ 10$^{-6}$ & 0.9991\\\\\n\t\t\tkNN & 1 $\\times$ 10$^{4}$ & 14.1 $\\times$ 10$^{-5}$ & 0.9994 & 13.5 $\\times$ 10$^{-6}$ & 0.9908\\\\\n\t\t\tkNN & 1 $\\times$ 10$^{3}$ & 43.9 $\\times$ 10$^{-5}$ & 0.9944 & 74.6 $\\times$ 10$^{-6}$ & 0.7180\\\\\n\t\t\tkNN & 1 $\\times$ 10$^{2}$  & 96.4 $\\times$ 10$^{-5}$ & 0.9729 & 55.0 $\\times$ 10$^{-6}$ & 0.8468\\\\\n\t\t\\end{tabular}\n\t\\end{center}\n\t\\label{table2}\n\\end{table}\n\n\\FloatBarrier\n\n\\subsubsection{Reactive transport simulation}\n\\label{prob12Dres}\n\nThis section focuses on reactive transport simulations with HPx$_{\\rm py}$ within cement system 1, under both advective-dispersive and diffusive transport conditions. As written above, the domain sizes are both 61 $\\times$ 61 and 121 $\\times$ 121 for the advection-dispersion case and, because of computational constraints, solely 61 $\\times$ 61  for the diffusion case. In addition, the simulation time period is 2 years for the advection-dispersion case and 1 year for the diffusion case. Figures \\ref{fig3} and \\ref{fig4} present times series of original and emulated Ca, Si, H and O concentrations at 5 locations within the 2D domain for advective-dispersive transport conditions, for both our kNN-based (HPx$_{\\rm py}$-kNN) and DNN-based (HPx$_{\\rm py}$-DNN) reactive transport codes. It is seen that HPx$_{\\rm py}$-kNN and HPx$_{\\rm py}$-DNN both induce a quite good simulation accuracy. Also, the results for the diffusive transport case are of similarly good quality (not shown). Figures \\ref{fig5} - \\ref{fig6} provide more insights into the HPx$_{\\rm py}$-kNN and HPx$_{\\rm py}$-DNN performances by displaying 2D Ca, Si, H and O concentration profiles at a given time. For each experiment and chemical component, this time is selected as to be well representative of the simulated dynamics. It is observed that the original and emulated images are visually almost indistinguishable for the advection-dispersion case (Figure \\ref{fig5}). For the diffusion case, the emulators also perform quite well for Ca, H and O (Figures\\ref{fig6}a - c, g - i and j - l), while some slight to moderate discrepancies appear at the concentration front for Si (Figures \\ref{fig6}d - f). Notwithstanding, the Si concentration remains globally well predicted. Furthermore, Figures \\ref{fig7} - \\ref{fig8} present the original and emulated 2D solid amount profiles corresponding to Figures \\ref{fig5} - \\ref{fig6}. The original solid amount profiles are overall well approximated by HPx$_{\\rm py}$-kNN and HPx$_{\\rm py}$-DNN for the advection-dispersion case (Figure \\ref{fig7}), even though some mismatch appears at the border of the fully depleted zone for the H component. As of the diffusion case (Figure \\ref{fig8}), the same kind of mismatch is observed for the emulated solid amounts of H by HPx$_{\\rm py}$-kNN while the emulated profiles by HPx$_{\\rm py}$-DNN show some more discrepancies. Though we decided to present raw emulation results, we would like to stress that some if not all of the observed artifacts could likely be smoothed out by using some post-filtering such as median filtering.\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure3.png}\n\t\\caption{Time series of original (RTM, solid green lines) and HPx$_{\\rm py}$-kNN emulated (TM+kNN, dashed orange lines) concentrations (mol/kg) of Ca, H, O and Si at selected observations points for cement system 1 and advective-dispersive transport. Obs. 1 - 5 denote the selected observation points, with the following $\\left[x,y\\right]$ locations (in cm). Obs. 1:  $\\left[0.5, 2.5\\right]$, Obs. 2: $\\left[1, 2\\right]$, Obs. 3: $\\left[2, 2\\right]$, Obs. 4: $\\left[1, 1\\right]$, Obs. 5: $\\left[2, 1\\right]$. The results for the 121 $\\times$ 121 grid size are rather similar.}\n\t\\label{fig3}\n\\end{figure}\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure4.png}\n\t\\caption{Time series of original (RTM, solid green lines) and HPx$_{\\rm py}$-DNN emulated (TM+DNN, dashed orange lines) concentrations (mol/kg) of Ca, H, O and Si at selected observations points for cement system 1 and advective-dispersive transport. Obs. 1 - 5 denote the selected observation points, with the following $\\left[x,y\\right]$ locations (in cm). Obs. 1:  $\\left[0.5, 2.5\\right]$, Obs. 2: $\\left[1, 2\\right]$, Obs. 3: $\\left[2, 2\\right]$, Obs. 4: $\\left[1, 1\\right]$, Obs. 5: $\\left[2, 1\\right]$. The results for the 121 $\\times$ 121 grid size are rather similar.}\n\t\\label{fig4}\n\\end{figure}\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure5.png}\n\t\\caption{2D concentration profiles obtained for cement system 1 at the end of the 2-year simulation performed for the advection-dispersion case. RTM means the original HP$_{\\rm{4C}}$ model, TM+kNN denotes the Hydrus transport model coupled with our kNN geochemical emulator (HPx$_{\\rm py}$-kNN) and TM+DNN signifies the Hydrus transport model coupled with our DNN geochemical emulator (HPx$_{\\rm py}$-DNN). The first to fourth row present profiles for $Ca^{\\rm{conc}}$, $Si^{\\rm{conc}}$, $H^{\\rm{conc}}$, and $O^{\\rm{conc}}$, respectively. The considered grid size is 61 $\\times$ 61. The results for the 121 $\\times$ 121 grid are rather similar.}\n\t\\label{fig5}\n\\end{figure}\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure6.png}\n\t\\caption{2D concentration profiles obtained for cement system 1 at the final time step of the 1-year simulation performed for the diffusion case. RTM means the original HP$_{\\rm{4C}}$ model, TM+kNN denotes the Hydrus transport model coupled with our kNN geochemical emulator (HPx$_{\\rm py}$-kNN) and TM+DNN signifies the Hydrus transport model coupled with our DNN geochemical emulator (HPx$_{\\rm py}$-DNN). The first to fourth row present profiles for $Ca^{\\rm{conc}}$, $Si^{\\rm{conc}}$, $H^{\\rm{conc}}$, and $O^{\\rm{conc}}$, respectively. The considered grid size is 61 $\\times$ 61.}\n\t\\label{fig6}\n\\end{figure}\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure7.png}\n\t\\caption{2D solid amount profiles obtained for cement system 1 at the end of the 2-year simulation performed for the advection-dispersion case. RTM means the original HP$_{\\rm{4C}}$ model, TM+kNN denotes the Hydrus transport model coupled with our kNN geochemical emulator (HPx$_{\\rm py}$-kNN) and TM+DNN signifies the Hydrus transport model coupled with our DNN geochemical emulator (HPx$_{\\rm py}$-DNN). The first to fourth row present profiles for $Ca^{\\rm{solid}}$, $Si^{\\rm{solid}}$, $H^{\\rm{solid}}$, and $O^{\\rm{solid}}$, respectively. The considered grid size is 61 $\\times$ 61. The results for the 121 $\\times$ 121 grid are rather similar.}\n\t\\label{fig7}\n\\end{figure}\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure8.png}\n\t\\caption{2D solid amount profiles obtained for cement system 1 at the final time step of the 1-year simulation performed for the diffusion case. RTM means the original HP$_{\\rm{4C}}$ model, TM+kNN denotes the Hydrus transport model coupled with our kNN geochemical emulator (HPx$_{\\rm py}$-kNN) and TM+DNN signifies the Hydrus transport model coupled with our DNN geochemical emulator (HPx$_{\\rm py}$-DNN). The first to fourth row present profiles for $Ca^{\\rm{solid}}$, $Si^{\\rm{solid}}$, $H^{\\rm{solid}}$, and $O^{\\rm{solid}}$, respectively. The considered grid size is 61 $\\times$ 61.}\n\t\\label{fig8}\n\\end{figure}\n\nThe speedups associated with the considered problem are detailed in Table \\ref{table3}. It is noted that the GPU-based DNN emulator allows for a speedup that is close to optimal. Indeed, The DNN speedups overall represent 85 \\% to 95 \\% of the maximum possible speedups (that is, speedups that would be obtained if the geochemcical calculations would come at no cost at all). The speedups associated with single-threaded kNN remain substantial but only amount to 57 \\% - 65 \\% of the corresponding maximum speedups. As detailed in section \\ref{train_res1}, the used kNN and DNN implementations are found to be respectively 300 and 4000 faster than single-threaded PHREEQC when predicting 10,000 points all one once for this geochemical system. Based on these numbers one could have expected the achieved speedups to represent say 90 \\% (kNN) or 99 \\% (DNN) of the maximum possible ones. A large part of the gaps between achieved and maximum possible speedups is thus likely caused by the time required for communicating and exchanging data between the main C/C++ code and the Python-based emulators.\n\n\\begin{table}[h!]\n\t\\caption{Speedups offered by the KNN and DNN emulators in HPx$_{\\rm py}$ for the reactive transport simulations considered for cement system 1. The HPx$_{\\rm{4C}}$ calculations involve the parallelization of PHREEQC over our 4 CPUs. The HPx$_{\\rm{1C}}$ calculations are performed on a single CPU. The kNN predictions are performed on a single CPU using the scikit-learn implementation while the DNN predictions make use of our GPU. ML signifies the used machine learning method for emulation, TC denotes transport conditions (ADV: advection-dispersion, DIF: diffusion) and GS is the grid size. The maximum possible speedups associated with HPx$_{\\rm{4C}}$ and HPx$_{\\rm{1C}}$,  Max SP HPx$_{\\rm{4C}}$ and  Max SP HPx$_{\\rm{1C}}$, correspond to an hypothetical situation where the geochemical calculations incur zero computational cost.}\n\n\t\\begin{center}\n\t\t\\begin{tabular}{cccccccc}\n\t\t\t\\hline\n\t\t\t& & & & & & & \\\\\n\t\t\tML & TC & GS & HPx$_{\\rm{4C}}$ time (s) & SP HPx$_{\\rm{4C}}$ & Max SP HPx$_{\\rm{4C}}$ & SP HPx$_{\\rm{1C}}$ & Max SP HPx$_{\\rm{1C}}$ \\\\\n\t\t\tDNN & ADV & 61 $\\times$ 61 & 3189 & 6.8 & 7.7 & 24.5 & 28.5 \\\\\n\t\t\tkNN & ADV & 61 $\\times$ 61 & 3189 & 5.0 & 7.7 & 18.0 & 28.5 \\\\\n\t\t\tDNN & ADV & 121 $\\times$ 121 & 23,337 & 5.0 & 5.2 & 17.0 & 18.0 \\\\\n\t\t\tkNN & ADV & 121 $\\times$ 121 & 23,337 & 3.4 & 5.2 & 11.6 & 18.0 \\\\\n\t\t\tDNN & DIF & 61 $\\times$ 61 & 25,448 & 4.2 & 4.9 & 16.2 & 19.1 \\\\\n\t\t\tkNN & DIF & 61 $\\times$ 61 & 25,448 & 2.8 & 4.9 & 10.8 & 19.1 \\\\\n\t\t\t\\hline\n\t\t\\end{tabular}\n\t\\end{center}\n\n\t\\label{table3}\n\\end{table}\n\n\\FloatBarrier\n\n3.1.1 Training the emulators\n\\subsubsection{Training the emulators}\n\\label{train_res1}\nHere the kNN and DNN emulators are firstly trained using a set of 400,000 test examples for both. This training set is obtained by randomly sampling the two-dimensional input space by latin hypercube sampling (LHS) between $\\left[0,0\\right]$ and $\\rm{\\left[Ca^{tot}_{max},Si^{tot}_{max}\\right]}$, and running PHREEQC for each input sample, $\\rm{\\textbf{x}_i = \\left[Ca^{tot}_i, Si^{tot}_i\\right]}$ to get the corresponding output vectors, $\\rm{\\textbf{y}_i = \\left[Ca^{conc}_i,Si^{conc}_i,H^{conc}_i,O^{conc}_i\\right]}$. The upper bounds, $\\rm{Ca^{tot}_{max}}$ and $\\rm{Si^{tot}_{max}}$ are defined based on a cheap full RT simulation with advective-dispersive transport using a small 1D domain of 51 nodes. It is worth noting that the total amounts of $\\textbf{x}_i$, corresponding to the PHREEQC-simulated concentrations, $\\textbf{y}_i$, have to be corrected for the different amount of water between the training set and the transport simulations. Doing so, it turns out that about 20 \\% of the post-corrected $x_i$ values exceed their pre-defined upper bounds and these excessively large values need to be filtered out. Creating the 400,000 training examples thus required about 500,000 PHREEQC runs. As stated earlier, for this problem single-threaded PHREEQC performs about 670 geochemical calculations per second on our used Intel\\textsuperscript{\\textregistered} i7 CPU.\n\nWith respect to kNN, the tuning parameters are the number of neighbors, $k$, the type of distance measure, and the interpolation technique. We simply used the default settings: $k = 5$, euclidean distance and inverse-distance interpolation. Regarding training of the DNN, the 400,000 sample were split between the training set itself (90 \\% of the data) and a validation set (10 \\% of the data). The latter serves to monitor the evolution of the selected mean squared error loss function on samples that are not used for training, thereby detecting potential overfitting. If the validation loss stops decreasing before the fixed number of epochs has been completed, then  training is stopped. Importantly, the emulation is achieved in log-space for both the input, $X$ and output $Y$, domains. This because total amounts and concentrations of the involved components typically cover many orders of magnitudes (up to 10 orders or more). Using a DNN also requires some form of data normalization or standardization. Here both the $\\rm{log\\left(\\textbf{x}_i\\right)}$ and $\\rm{log\\left(\\textbf{y}_i\\right)}$ vectors are standardized around 0 with standard deviation of 1.\n\nFigure \\ref{fig1} illustrates the trained emulators' performance for geochemical predictions using an independent test set that comprises 10,000 test examples. Both kNN (Figures \\ref{fig1}a - d) and DNN (Figures \\ref{fig1}e - h) appear to be rather accurate . DNN also shows a slight degradation for the larger concentration values  (Figures \\ref{fig1}e - h). The latter is likely due to the combination of a small proportion of large concentration values in the training set with the log-transformation that implicitly pushes the DNN to try harder to fit the smaller concentrations during training. Regarding speedup and as written earlier, for this setup the single-threaded kNN method is 300 times faster than single-threaded PHREEQC for predicting the 10,000 concentration vectors all at once. The computational savings allowed by the DNN emulator when ran on our GPU is higher with a speedup as large as 4000 for predicting the same 10,000 concentration vectors all at once. \n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{0cm}\\includegraphics[width=35pc]{Figure1.png}\n\t\\caption{1-1 plots of the kNN (subfigures (a) - (d)) and DNN (subfigures (e) - (h)) emulators' performance obtained for system 1 when the kNN training base contains 400,000 samples and the DNN is trained using the same 400,000 samples. Here ``true\" means the original PHREEQC-simulated data and ``predicted\" denotes the emulated (that is, kNN-simulated and DNN-simulated) data. Hence, the $x$-axis and $y$-axis present the original and emulated 10,000 independent test data points, respectively. The RMSE and $Q_2$ coefficient denote the root-mean-square-error and coefficient of determination in testing mode, respectively, between the original and emulated 10,000 test data points.}\n\t\\label{fig1}\n\\end{figure}\n\nTo test the sensitivity of the emulators' performance to the training set size, training was also performed using reduced training sets comprising 100,000, 10,000, 1000 and 100 samples, respectively. It seen that the DNN performance achieved  when using 10,000 training samples is virtually the same as that obtained when using 400,000 training samples (Figure \\ref{fig2} and Table \\ref{table2}). It is only for training sets smaller than 1000 samples that the DNN performance starts to degrade significantly (Table \\ref{table2}). In contrast, the behavior of kNN appears to markedly decrease as the training set gets smaller (Figure \\ref{fig2} and Table \\ref{table1}).\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{0cm}\\includegraphics[width=35pc]{Figure2.png}\n\t\\caption{1-1 plots of the kNN (subfigures (a) - (d)) and DNN (subfigures (e) - (h)) emulators' performance obtained for system 1 when the kNN training base contains 10,000 samples and the DNN is trained using the same 10,000 samples. Here ``true\" means the original PHREEQC-simulated data and ``predicted\" denotes the emulated (that is, kNN-simulated and DNN-simulated) data. Hence, the $x$-axis and $y$-axis present the original and emulated 10,000 independent test data points, respectively. The RMSE and $Q_2$ coefficient denote the root-mean-square-error and coefficient of determination in testing mode, respectively, between the original and emulated 10,000 test data points.}\n\t\\label{fig2}\n\\end{figure}\n\n\\begin{table}[h!]\n\t\\caption{Performance of the DNN and kNN emulators for cement system 1 and different training set sizes. For brevity, only the results for Ca$^{conc}$ and Si$^{conc}$ are shown. The units are mol per kg of water (mol/kgw). ML refers to the type of emulator, TR signifies the size of the training set, and the RMSE and $Q_2$ coefficient denote the root-mean-square-error and coefficient of determination in testing mode, respectively, between the original and emulated 10,000 test data points.}\n\t\\begin{center}\n\t\t\\begin{tabular}{cccccccc}\n\t\t\t\\hline\n\t\t\t& & & & & \\\\\n\t\t\tML & TR & RMSE - Ca$^{conc}$ & $Q_2$ - Ca$^{conc}$ & RMSE - Si$^{conc}$ & $Q_2$ - Si$^{conc}$\\\\\n\t\t\tDNN & 4 $\\times$ 10$^{5}$ & 4.50 $\\times$ 10$^{-5}$ & 0.9999 & 1.21 $\\times$ 10$^{-6}$ & 0.9999\\\\\n\t\t\tDNN & 1 $\\times$ 10$^{5}$ & 5.06 $\\times$ 10$^{-5}$ & 0.9999 & 1.30 $\\times$ 10$^{-6}$ & 0.9999\\\\\n\t\t\tDNN & 1 $\\times$ 10$^{4}$ & 4.57 $\\times$ 10$^{-5}$ & 0.9999 & 1.48 $\\times$ 10$^{-6}$ & 0.9999\\\\\n\t\t\tDNN & 1 $\\times$ 10$^{3}$ & 8.02 $\\times$ 10$^{-5}$ & 0.9998 & 3.31 $\\times$ 10$^{-6}$ & 0.9994\\\\\n\t\t\tDNN & 1 $\\times$ 10$^{2}$ & 47.1 $\\times$ 10$^{-5}$ & 0.9935 & 9.02 $\\times$ 10$^{-6}$ & 0.9959\\\\\n\t\t\tkNN & 4 $\\times$ 10$^{5}$ & 2.52 $\\times$ 10$^{-5}$ & 1.0000 & 1.59 $\\times$ 10$^{-6}$ & 0.9999\\\\\n\t\t\tkNN & 1 $\\times$ 10$^{5}$ & 6.27 $\\times$ 10$^{-5}$ & 0.9999 & 4.20 $\\times$ 10$^{-6}$ & 0.9991\\\\\n\t\t\tkNN & 1 $\\times$ 10$^{4}$ & 14.1 $\\times$ 10$^{-5}$ & 0.9994 & 13.5 $\\times$ 10$^{-6}$ & 0.9908\\\\\n\t\t\tkNN & 1 $\\times$ 10$^{3}$ & 43.9 $\\times$ 10$^{-5}$ & 0.9944 & 74.6 $\\times$ 10$^{-6}$ & 0.7180\\\\\n\t\t\tkNN & 1 $\\times$ 10$^{2}$  & 96.4 $\\times$ 10$^{-5}$ & 0.9729 & 55.0 $\\times$ 10$^{-6}$ & 0.8468\\\\\n\t\t\\end{tabular}\n\t\\end{center}\n\t\\label{table2}\n\\end{table}\n\n\\FloatBarrier\n\n",
        "label": "fig2",
        "Descriptive_question1": "What is the training set size used for both kNN and DNN emulators in figure_2?",
        "Descriptive_question2": "What does the term 'true' refer to in the context of figure_2?",
        "Reasoning_question1": "How does the performance of the kNN emulator compare to the DNN emulator in figure_2 when using a reduced training set of 10,000 samples?",
        "Reasoning_question2": "What might explain the observed differences in performance between kNN and DNN emulators for smaller training sets in figure_2?",
        "Descriptive_answer1": "10,000 samples",
        "Descriptive_answer2": "original PHREEQC-simulated data",
        "Reasoning_answer1": "In figure_2, when using a reduced training set of 10,000 samples, the DNN emulator shows better emulation accuracy compared to the kNN emulator. This is indicated by the closer alignment of the predicted values to the true values for DNN, whereas kNN shows more significant deviations from the true data. The DNN maintains lower RMSE and higher Q_2 coefficients, demonstrating superior predictive performance at this training set size.",
        "Reasoning_answer2": "The differences in performance between the kNN and DNN emulators for smaller training sets in figure_2 can be attributed to their inherent learning mechanisms. DNNs, as parametric models, can generalize better from limited data by capturing nonlinear relationships through learned weights and biases. Conversely, kNN relies heavily on the density and representativeness of training samples because it predicts based on nearest neighbors. With fewer samples, kNN has less local information to interpolate accurately, leading to decreased performance. Additionally, DNN's use of log-space transformation and standardization likely aids its robustness with smaller datasets."
    },
    {
        "paper_id": "2107.07598.json",
        "image_id": "figure_3",
        "image_path": [
            "/home/yz979/scratch/chengye/scimolmo/test-images-tables/dataset_imagestables/2107.07598/images/Figure3.png"
        ],
        "caption": "Time series of original (RTM, solid green lines) and HPx$_{\\rm py}$-kNN emulated (TM+kNN, dashed orange lines) concentrations (mol/kg) of Ca, H, O and Si at selected observations points for cement system 1 and advective-dispersive transport. Obs. 1 - 5 denote the selected observation points, with the following $\\left[x,y\\right]$ locations (in cm). Obs. 1:  $\\left[0.5, 2.5\\right]$, Obs. 2: $\\left[1, 2\\right]$, Obs. 3: $\\left[2, 2\\right]$, Obs. 4: $\\left[1, 1\\right]$, Obs. 5: $\\left[2, 1\\right]$. The results for the 121 $\\times$ 121 grid size are rather similar.",
        "classify": "Chart",
        "section_info": "3 Results\n\\section{Results}\n\\label{results}\n\n\\subsection{Ca-Si Problem}\nFor this first cement system, the emulation problem consists of predicting at each time step of the RT simulation the (output) Ca, Si, H and O aqueous concentrations (mol/kg of water or mol/kgw) from the (input) total amounts of Ca and Si (mol). \n\n\\subsubsection{Training the emulators}\n\\label{train_res1}\nHere the kNN and DNN emulators are firstly trained using a set of 400,000 test examples for both. This training set is obtained by randomly sampling the two-dimensional input space by latin hypercube sampling (LHS) between $\\left[0,0\\right]$ and $\\rm{\\left[Ca^{tot}_{max},Si^{tot}_{max}\\right]}$, and running PHREEQC for each input sample, $\\rm{\\textbf{x}_i = \\left[Ca^{tot}_i, Si^{tot}_i\\right]}$ to get the corresponding output vectors, $\\rm{\\textbf{y}_i = \\left[Ca^{conc}_i,Si^{conc}_i,H^{conc}_i,O^{conc}_i\\right]}$. The upper bounds, $\\rm{Ca^{tot}_{max}}$ and $\\rm{Si^{tot}_{max}}$ are defined based on a cheap full RT simulation with advective-dispersive transport using a small 1D domain of 51 nodes. It is worth noting that the total amounts of $\\textbf{x}_i$, corresponding to the PHREEQC-simulated concentrations, $\\textbf{y}_i$, have to be corrected for the different amount of water between the training set and the transport simulations. Doing so, it turns out that about 20 \\% of the post-corrected $x_i$ values exceed their pre-defined upper bounds and these excessively large values need to be filtered out. Creating the 400,000 training examples thus required about 500,000 PHREEQC runs. As stated earlier, for this problem single-threaded PHREEQC performs about 670 geochemical calculations per second on our used Intel\\textsuperscript{\\textregistered} i7 CPU.\n\nWith respect to kNN, the tuning parameters are the number of neighbors, $k$, the type of distance measure, and the interpolation technique. We simply used the default settings: $k = 5$, euclidean distance and inverse-distance interpolation. Regarding training of the DNN, the 400,000 sample were split between the training set itself (90 \\% of the data) and a validation set (10 \\% of the data). The latter serves to monitor the evolution of the selected mean squared error loss function on samples that are not used for training, thereby detecting potential overfitting. If the validation loss stops decreasing before the fixed number of epochs has been completed, then  training is stopped. Importantly, the emulation is achieved in log-space for both the input, $X$ and output $Y$, domains. This because total amounts and concentrations of the involved components typically cover many orders of magnitudes (up to 10 orders or more). Using a DNN also requires some form of data normalization or standardization. Here both the $\\rm{log\\left(\\textbf{x}_i\\right)}$ and $\\rm{log\\left(\\textbf{y}_i\\right)}$ vectors are standardized around 0 with standard deviation of 1.\n\nFigure \\ref{fig1} illustrates the trained emulators' performance for geochemical predictions using an independent test set that comprises 10,000 test examples. Both kNN (Figures \\ref{fig1}a - d) and DNN (Figures \\ref{fig1}e - h) appear to be rather accurate . DNN also shows a slight degradation for the larger concentration values  (Figures \\ref{fig1}e - h). The latter is likely due to the combination of a small proportion of large concentration values in the training set with the log-transformation that implicitly pushes the DNN to try harder to fit the smaller concentrations during training. Regarding speedup and as written earlier, for this setup the single-threaded kNN method is 300 times faster than single-threaded PHREEQC for predicting the 10,000 concentration vectors all at once. The computational savings allowed by the DNN emulator when ran on our GPU is higher with a speedup as large as 4000 for predicting the same 10,000 concentration vectors all at once. \n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{0cm}\\includegraphics[width=35pc]{Figure1.png}\n\t\\caption{1-1 plots of the kNN (subfigures (a) - (d)) and DNN (subfigures (e) - (h)) emulators' performance obtained for system 1 when the kNN training base contains 400,000 samples and the DNN is trained using the same 400,000 samples. Here ``true\" means the original PHREEQC-simulated data and ``predicted\" denotes the emulated (that is, kNN-simulated and DNN-simulated) data. Hence, the $x$-axis and $y$-axis present the original and emulated 10,000 independent test data points, respectively. The RMSE and $Q_2$ coefficient denote the root-mean-square-error and coefficient of determination in testing mode, respectively, between the original and emulated 10,000 test data points.}\n\t\\label{fig1}\n\\end{figure}\n\nTo test the sensitivity of the emulators' performance to the training set size, training was also performed using reduced training sets comprising 100,000, 10,000, 1000 and 100 samples, respectively. It seen that the DNN performance achieved  when using 10,000 training samples is virtually the same as that obtained when using 400,000 training samples (Figure \\ref{fig2} and Table \\ref{table2}). It is only for training sets smaller than 1000 samples that the DNN performance starts to degrade significantly (Table \\ref{table2}). In contrast, the behavior of kNN appears to markedly decrease as the training set gets smaller (Figure \\ref{fig2} and Table \\ref{table1}).\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{0cm}\\includegraphics[width=35pc]{Figure2.png}\n\t\\caption{1-1 plots of the kNN (subfigures (a) - (d)) and DNN (subfigures (e) - (h)) emulators' performance obtained for system 1 when the kNN training base contains 10,000 samples and the DNN is trained using the same 10,000 samples. Here ``true\" means the original PHREEQC-simulated data and ``predicted\" denotes the emulated (that is, kNN-simulated and DNN-simulated) data. Hence, the $x$-axis and $y$-axis present the original and emulated 10,000 independent test data points, respectively. The RMSE and $Q_2$ coefficient denote the root-mean-square-error and coefficient of determination in testing mode, respectively, between the original and emulated 10,000 test data points.}\n\t\\label{fig2}\n\\end{figure}\n\n\\begin{table}[h!]\n\t\\caption{Performance of the DNN and kNN emulators for cement system 1 and different training set sizes. For brevity, only the results for Ca$^{conc}$ and Si$^{conc}$ are shown. The units are mol per kg of water (mol/kgw). ML refers to the type of emulator, TR signifies the size of the training set, and the RMSE and $Q_2$ coefficient denote the root-mean-square-error and coefficient of determination in testing mode, respectively, between the original and emulated 10,000 test data points.}\n\t\\begin{center}\n\t\t\\begin{tabular}{cccccccc}\n\t\t\t\\hline\n\t\t\t& & & & & \\\\\n\t\t\tML & TR & RMSE - Ca$^{conc}$ & $Q_2$ - Ca$^{conc}$ & RMSE - Si$^{conc}$ & $Q_2$ - Si$^{conc}$\\\\\n\t\t\tDNN & 4 $\\times$ 10$^{5}$ & 4.50 $\\times$ 10$^{-5}$ & 0.9999 & 1.21 $\\times$ 10$^{-6}$ & 0.9999\\\\\n\t\t\tDNN & 1 $\\times$ 10$^{5}$ & 5.06 $\\times$ 10$^{-5}$ & 0.9999 & 1.30 $\\times$ 10$^{-6}$ & 0.9999\\\\\n\t\t\tDNN & 1 $\\times$ 10$^{4}$ & 4.57 $\\times$ 10$^{-5}$ & 0.9999 & 1.48 $\\times$ 10$^{-6}$ & 0.9999\\\\\n\t\t\tDNN & 1 $\\times$ 10$^{3}$ & 8.02 $\\times$ 10$^{-5}$ & 0.9998 & 3.31 $\\times$ 10$^{-6}$ & 0.9994\\\\\n\t\t\tDNN & 1 $\\times$ 10$^{2}$ & 47.1 $\\times$ 10$^{-5}$ & 0.9935 & 9.02 $\\times$ 10$^{-6}$ & 0.9959\\\\\n\t\t\tkNN & 4 $\\times$ 10$^{5}$ & 2.52 $\\times$ 10$^{-5}$ & 1.0000 & 1.59 $\\times$ 10$^{-6}$ & 0.9999\\\\\n\t\t\tkNN & 1 $\\times$ 10$^{5}$ & 6.27 $\\times$ 10$^{-5}$ & 0.9999 & 4.20 $\\times$ 10$^{-6}$ & 0.9991\\\\\n\t\t\tkNN & 1 $\\times$ 10$^{4}$ & 14.1 $\\times$ 10$^{-5}$ & 0.9994 & 13.5 $\\times$ 10$^{-6}$ & 0.9908\\\\\n\t\t\tkNN & 1 $\\times$ 10$^{3}$ & 43.9 $\\times$ 10$^{-5}$ & 0.9944 & 74.6 $\\times$ 10$^{-6}$ & 0.7180\\\\\n\t\t\tkNN & 1 $\\times$ 10$^{2}$  & 96.4 $\\times$ 10$^{-5}$ & 0.9729 & 55.0 $\\times$ 10$^{-6}$ & 0.8468\\\\\n\t\t\\end{tabular}\n\t\\end{center}\n\t\\label{table2}\n\\end{table}\n\n\\FloatBarrier\n\n\\subsubsection{Reactive transport simulation}\n\\label{prob12Dres}\n\nThis section focuses on reactive transport simulations with HPx$_{\\rm py}$ within cement system 1, under both advective-dispersive and diffusive transport conditions. As written above, the domain sizes are both 61 $\\times$ 61 and 121 $\\times$ 121 for the advection-dispersion case and, because of computational constraints, solely 61 $\\times$ 61  for the diffusion case. In addition, the simulation time period is 2 years for the advection-dispersion case and 1 year for the diffusion case. Figures \\ref{fig3} and \\ref{fig4} present times series of original and emulated Ca, Si, H and O concentrations at 5 locations within the 2D domain for advective-dispersive transport conditions, for both our kNN-based (HPx$_{\\rm py}$-kNN) and DNN-based (HPx$_{\\rm py}$-DNN) reactive transport codes. It is seen that HPx$_{\\rm py}$-kNN and HPx$_{\\rm py}$-DNN both induce a quite good simulation accuracy. Also, the results for the diffusive transport case are of similarly good quality (not shown). Figures \\ref{fig5} - \\ref{fig6} provide more insights into the HPx$_{\\rm py}$-kNN and HPx$_{\\rm py}$-DNN performances by displaying 2D Ca, Si, H and O concentration profiles at a given time. For each experiment and chemical component, this time is selected as to be well representative of the simulated dynamics. It is observed that the original and emulated images are visually almost indistinguishable for the advection-dispersion case (Figure \\ref{fig5}). For the diffusion case, the emulators also perform quite well for Ca, H and O (Figures\\ref{fig6}a - c, g - i and j - l), while some slight to moderate discrepancies appear at the concentration front for Si (Figures \\ref{fig6}d - f). Notwithstanding, the Si concentration remains globally well predicted. Furthermore, Figures \\ref{fig7} - \\ref{fig8} present the original and emulated 2D solid amount profiles corresponding to Figures \\ref{fig5} - \\ref{fig6}. The original solid amount profiles are overall well approximated by HPx$_{\\rm py}$-kNN and HPx$_{\\rm py}$-DNN for the advection-dispersion case (Figure \\ref{fig7}), even though some mismatch appears at the border of the fully depleted zone for the H component. As of the diffusion case (Figure \\ref{fig8}), the same kind of mismatch is observed for the emulated solid amounts of H by HPx$_{\\rm py}$-kNN while the emulated profiles by HPx$_{\\rm py}$-DNN show some more discrepancies. Though we decided to present raw emulation results, we would like to stress that some if not all of the observed artifacts could likely be smoothed out by using some post-filtering such as median filtering.\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure3.png}\n\t\\caption{Time series of original (RTM, solid green lines) and HPx$_{\\rm py}$-kNN emulated (TM+kNN, dashed orange lines) concentrations (mol/kg) of Ca, H, O and Si at selected observations points for cement system 1 and advective-dispersive transport. Obs. 1 - 5 denote the selected observation points, with the following $\\left[x,y\\right]$ locations (in cm). Obs. 1:  $\\left[0.5, 2.5\\right]$, Obs. 2: $\\left[1, 2\\right]$, Obs. 3: $\\left[2, 2\\right]$, Obs. 4: $\\left[1, 1\\right]$, Obs. 5: $\\left[2, 1\\right]$. The results for the 121 $\\times$ 121 grid size are rather similar.}\n\t\\label{fig3}\n\\end{figure}\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure4.png}\n\t\\caption{Time series of original (RTM, solid green lines) and HPx$_{\\rm py}$-DNN emulated (TM+DNN, dashed orange lines) concentrations (mol/kg) of Ca, H, O and Si at selected observations points for cement system 1 and advective-dispersive transport. Obs. 1 - 5 denote the selected observation points, with the following $\\left[x,y\\right]$ locations (in cm). Obs. 1:  $\\left[0.5, 2.5\\right]$, Obs. 2: $\\left[1, 2\\right]$, Obs. 3: $\\left[2, 2\\right]$, Obs. 4: $\\left[1, 1\\right]$, Obs. 5: $\\left[2, 1\\right]$. The results for the 121 $\\times$ 121 grid size are rather similar.}\n\t\\label{fig4}\n\\end{figure}\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure5.png}\n\t\\caption{2D concentration profiles obtained for cement system 1 at the end of the 2-year simulation performed for the advection-dispersion case. RTM means the original HP$_{\\rm{4C}}$ model, TM+kNN denotes the Hydrus transport model coupled with our kNN geochemical emulator (HPx$_{\\rm py}$-kNN) and TM+DNN signifies the Hydrus transport model coupled with our DNN geochemical emulator (HPx$_{\\rm py}$-DNN). The first to fourth row present profiles for $Ca^{\\rm{conc}}$, $Si^{\\rm{conc}}$, $H^{\\rm{conc}}$, and $O^{\\rm{conc}}$, respectively. The considered grid size is 61 $\\times$ 61. The results for the 121 $\\times$ 121 grid are rather similar.}\n\t\\label{fig5}\n\\end{figure}\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure6.png}\n\t\\caption{2D concentration profiles obtained for cement system 1 at the final time step of the 1-year simulation performed for the diffusion case. RTM means the original HP$_{\\rm{4C}}$ model, TM+kNN denotes the Hydrus transport model coupled with our kNN geochemical emulator (HPx$_{\\rm py}$-kNN) and TM+DNN signifies the Hydrus transport model coupled with our DNN geochemical emulator (HPx$_{\\rm py}$-DNN). The first to fourth row present profiles for $Ca^{\\rm{conc}}$, $Si^{\\rm{conc}}$, $H^{\\rm{conc}}$, and $O^{\\rm{conc}}$, respectively. The considered grid size is 61 $\\times$ 61.}\n\t\\label{fig6}\n\\end{figure}\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure7.png}\n\t\\caption{2D solid amount profiles obtained for cement system 1 at the end of the 2-year simulation performed for the advection-dispersion case. RTM means the original HP$_{\\rm{4C}}$ model, TM+kNN denotes the Hydrus transport model coupled with our kNN geochemical emulator (HPx$_{\\rm py}$-kNN) and TM+DNN signifies the Hydrus transport model coupled with our DNN geochemical emulator (HPx$_{\\rm py}$-DNN). The first to fourth row present profiles for $Ca^{\\rm{solid}}$, $Si^{\\rm{solid}}$, $H^{\\rm{solid}}$, and $O^{\\rm{solid}}$, respectively. The considered grid size is 61 $\\times$ 61. The results for the 121 $\\times$ 121 grid are rather similar.}\n\t\\label{fig7}\n\\end{figure}\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure8.png}\n\t\\caption{2D solid amount profiles obtained for cement system 1 at the final time step of the 1-year simulation performed for the diffusion case. RTM means the original HP$_{\\rm{4C}}$ model, TM+kNN denotes the Hydrus transport model coupled with our kNN geochemical emulator (HPx$_{\\rm py}$-kNN) and TM+DNN signifies the Hydrus transport model coupled with our DNN geochemical emulator (HPx$_{\\rm py}$-DNN). The first to fourth row present profiles for $Ca^{\\rm{solid}}$, $Si^{\\rm{solid}}$, $H^{\\rm{solid}}$, and $O^{\\rm{solid}}$, respectively. The considered grid size is 61 $\\times$ 61.}\n\t\\label{fig8}\n\\end{figure}\n\nThe speedups associated with the considered problem are detailed in Table \\ref{table3}. It is noted that the GPU-based DNN emulator allows for a speedup that is close to optimal. Indeed, The DNN speedups overall represent 85 \\% to 95 \\% of the maximum possible speedups (that is, speedups that would be obtained if the geochemcical calculations would come at no cost at all). The speedups associated with single-threaded kNN remain substantial but only amount to 57 \\% - 65 \\% of the corresponding maximum speedups. As detailed in section \\ref{train_res1}, the used kNN and DNN implementations are found to be respectively 300 and 4000 faster than single-threaded PHREEQC when predicting 10,000 points all one once for this geochemical system. Based on these numbers one could have expected the achieved speedups to represent say 90 \\% (kNN) or 99 \\% (DNN) of the maximum possible ones. A large part of the gaps between achieved and maximum possible speedups is thus likely caused by the time required for communicating and exchanging data between the main C/C++ code and the Python-based emulators.\n\n\\begin{table}[h!]\n\t\\caption{Speedups offered by the KNN and DNN emulators in HPx$_{\\rm py}$ for the reactive transport simulations considered for cement system 1. The HPx$_{\\rm{4C}}$ calculations involve the parallelization of PHREEQC over our 4 CPUs. The HPx$_{\\rm{1C}}$ calculations are performed on a single CPU. The kNN predictions are performed on a single CPU using the scikit-learn implementation while the DNN predictions make use of our GPU. ML signifies the used machine learning method for emulation, TC denotes transport conditions (ADV: advection-dispersion, DIF: diffusion) and GS is the grid size. The maximum possible speedups associated with HPx$_{\\rm{4C}}$ and HPx$_{\\rm{1C}}$,  Max SP HPx$_{\\rm{4C}}$ and  Max SP HPx$_{\\rm{1C}}$, correspond to an hypothetical situation where the geochemical calculations incur zero computational cost.}\n\n\t\\begin{center}\n\t\t\\begin{tabular}{cccccccc}\n\t\t\t\\hline\n\t\t\t& & & & & & & \\\\\n\t\t\tML & TC & GS & HPx$_{\\rm{4C}}$ time (s) & SP HPx$_{\\rm{4C}}$ & Max SP HPx$_{\\rm{4C}}$ & SP HPx$_{\\rm{1C}}$ & Max SP HPx$_{\\rm{1C}}$ \\\\\n\t\t\tDNN & ADV & 61 $\\times$ 61 & 3189 & 6.8 & 7.7 & 24.5 & 28.5 \\\\\n\t\t\tkNN & ADV & 61 $\\times$ 61 & 3189 & 5.0 & 7.7 & 18.0 & 28.5 \\\\\n\t\t\tDNN & ADV & 121 $\\times$ 121 & 23,337 & 5.0 & 5.2 & 17.0 & 18.0 \\\\\n\t\t\tkNN & ADV & 121 $\\times$ 121 & 23,337 & 3.4 & 5.2 & 11.6 & 18.0 \\\\\n\t\t\tDNN & DIF & 61 $\\times$ 61 & 25,448 & 4.2 & 4.9 & 16.2 & 19.1 \\\\\n\t\t\tkNN & DIF & 61 $\\times$ 61 & 25,448 & 2.8 & 4.9 & 10.8 & 19.1 \\\\\n\t\t\t\\hline\n\t\t\\end{tabular}\n\t\\end{center}\n\n\t\\label{table3}\n\\end{table}\n\n\\FloatBarrier\n\n\\subsection{Al-C-Ca-S-Si Problem}\n\nFor this second cement system, the emulation problem consists of predicting at each time step of the RT simulation the (output) Al, C, Ca, S, Si, H and O aqueous concentrations (mol/kgw) from the (input) total amounts of Al, C, Ca, S and Si (mol). Here we focus on advective-dispersive transport only while similarly as for cement system 1, the considered domain sizes are 61 $\\times$ 61 and 121 $\\times$ 121. Moreover, the simulation time period is set to 6 years. \n\nAs mentioned earlier, for this higher-dimensional problem it is observed that the used scikit-learn kNN implementation becomes prohibitively slow compared to HPx$_{\\rm{4C}}$. We found that to get a good emulation accuracy the kNN training base needs to contain 1,000,000 samples (or more). This training base's size together with a 5-dimensional search space leads to an HPx$_{\\rm py}$-kNN reactive transport simulation time that is comparable to that of HPx$_{\\rm{4C}}$. Therefore, for this second cement system we built a custom kNN regressor around another kNN implementation contained in the FAISS package \\citep{faiss2017}. Our used FAISS variant allows for GPU computing and is much faster than scikit-learn for this cement system, but is slightly less accurate due to the use of an approximate rather than exact nearest neighbor search \\citep[see][for details]{faiss2017}.\n\n\\subsubsection{Training the DNN}\n\\label{train_res2}\n\nBuilding a good training set to perform a kNN search and learn the weights and biases of our DNN turned out to be a complicated task in this case. This because to make useful kNN predictions and/or learn an useful DNN, the training set must be sufficiently representative of the geochemical conditions encountered during the reactive transport simulation one wish to perform with HPx$_{\\rm py}$-kNN and the trained HPx$_{\\rm py}$-DNN. In contrast to cement system 1, creating the training set by sampling the $X$-space with a controlled randomness between predefined lower and upper bounds did not prove successful. We tried that strategy by drawing as much as 4,000,000 5-dimensional $\\textbf{x}$ vectors from the $X$-space using a Sobol low-discrepancy sequence \\citep[][]{Sobol1967, Joe-Kuo2003}. Such low-discrepancy sampling scheme covers the 5-dimensional hypercube more uniformly than LHS. Despite a good performance on the test set (not shown), the resulting DNN accuracy in reactive transport mode was never deemed satisfying. In other words, no satisfying ``global\" or ``universal\" DNN emulator could be devised for this cement system. This is probably caused by the fact for this problem, the input (5 total amounts) and output (7 aqueous concentrations) spaces are quite nonlinearly related and both cover 6 to 10 orders of magnitudes depending on the considered element. Therefore, we resorted to the alternative training strategy detailed below. The latter basically tries to grasp the complex correlations and higher-order dependencies that exist between the elements of $\\textbf{x}$ (total amounts, input space) for a given reactive transport simulation setup, in order to produce a training set that honors these between-input relationships.\n\n\\begin{itemize}\n\t\\item Perform a ``cheap\" full reactive transport simulation under the transport conditions and geochemistry of interest and collect the resulting $\\textbf{x}$ -$\\textbf{y}$  pairs of examples (for the considered grid nodes and time steps). Computational demand controls what domain size and simulation time period can be used for this cheap calculation. We used a modest 16 $\\times$ 16 domain and a simulation time period of 10 years. The associated HPx$_{\\rm{4C}}$ runtimes is 180 s.\n\t\n\t\\item Fit a kernel density estimator (KDE) with a Gaussian kernel to the collected $\\textbf{x}$ vectors (encapsulated in the $\\textbf{X}$ array) and generate a fixed number of new input vectors, $\\textbf{x}_{KDE}$. Then run PHREEQC for the $\\textbf{X}_{KDE}$ set to get the corresponding output set, $\\textbf{Y}_{KDE}$. Now apply the correction for porosity described in section \\ref{train_res1} to the $\\textbf{X}_{KDE}$ set and form the training set by merging the ensemble of $\\textbf{x}$-$\\textbf{y}$ pairs with that of the $\\textbf{x}_{KDE}$-$\\textbf{y}_{KDE}$ pairs. The number of produced unique examples by the considered cheap HPx$_{\\rm{4C}}$ simulations varied between 10,000 and 50,000. The KDE-based enrichment of this dataset was deemed necessary to provide more input variability, thereby avoiding overfitting of the trained DNN and improving the kNN accuracy, while still honoring the complex between-input relationships. The number of KDE-generated samples was set as to obtain a total training set size of 1,000,000 examples. A key component of the approach is the bandwith parameter of the Gaussian KDE kernel which controls how much the KDE-generated samples depart from the original ensemble. After limited trial and error, we fixed the kernel bandwith to 0.0025 for the considered case studies.\n\t\n\\end{itemize}\n\nThe scatter plots in Figure \\ref{fig9} illustrate our training set creation procedure. The orange dots depict the pairwise relationships observed between the 5 elements of $\\textbf{x}$ in the cheap simulation. The red and cyan dots in Figure \\ref{fig9} represent the KDE-generated samples with the selected bandwidth, before and after applying the correction for porosity, respectively. Training of the DNN is achieved using the ensemble of original and KDE-corrected input points. We refer to this kind of dataset as RT-based, since it is based on an albeit cheap, full RT simulation. Furthermore, we refer to the obtained DNN and kNN emulators as ``local\" emulators, since as opposed to the emulators constructed for cement system 1, the current emulators are only valid for the input conditions encapsulated in the RT-based training set.\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1.5cm}\\includegraphics[width=50pc]{Figure9.png}\n\t\\caption{Scatter plots of the complex relationships between the five considered inputs for (1) the computationally cheap RT simulation of cement system 2 (orange dots) performed with the original HPx code, (2) the corresponding sampled points by kernel density estimation (KDE) using the selected bandwith (turquoise dots) and, (3) the same sampled KDE points after porosity correction (red dots).}\n\t\\label{fig9}\n\\end{figure}\n\nTraining performance of the DNN emulator is presented in Figure \\ref{fig10}. Similarly as for cement system 1, about 90\\% of the available data was used for the actual training of the DNN while the remaining 10\\% were used a validation set to contol overfitting. Lastly, performance is evaluated for both the trained DNN and kNN emulators using an independent test set of 10,000 examples. Overall, the accuracy of our ``local \" DNN emulator for this RT-inspired dataset is rather large with $Q_2$ values always greater or equal than 0.998. Training performance of the corresponding local kNN emulator is equally good (not shown). Regarding speedup, for this problem single-threaded PHREEQC achieves about 210 geochemical calculations per second on our used Intel\\textsuperscript{\\textregistered} i7 CPU while the GPU-based DNN and kNN emulators are both about 3000 times faster when predicting the 10,000 test points at once .\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure10.png}\n\t\\caption{1-1 plots of local DNN emulation performance obtained for system 2 when the local DNN is trained using 1,000,000 samples. The $x$-axis and $y$-axis present the original and emulated 10,000 independent test data points, respectively. The RMSE and $Q_2$ coefficient denote the root-mean-square-error and coefficient of determination in testing mode, respectively, between the original and emulated 10,000 test data points.}\n\t\\label{fig10}\n\\end{figure}\n\n\\subsubsection{Reactive transport simulation}\n\\label{prob2res}\nOur ``local\" DNN performs rather well when applied to the 61 $\\times$ 61 grid size and a time period of 6 years (Figures \\ref{fig11} - \\ref{fig12}). This for all components but C, for which some localized deviations appear between original and emulated 2D concentration profiles towards the end of the simulation period (Figure \\ref{fig12}). When applied to the 121 $\\times$ 121 grid, HPx$_{\\rm py}$-DNN produces additional discrepancies for O, Si and Al towards the end of the simulation period (Figures \\ref{fig13} - \\ref{fig14}). Yet most of the observed artifacts could probably be smoothed out by using post-filtering. The associated speedups are listed in Table \\ref{table4}. These speedups are larger to those obtained for cement system 1, with values between 8 and 9 when evaluated against HPx$_{\\rm{4C}}$. These speedups represent about 85 \\% to 90 \\% of the maximum possible speedup (Table \\ref{table3}). Overall, these findings indicate that for the considered problem, our RT-based training of a local DNN only works is the training set is sufficiently representative of the particular geochemical conditions encountered in the computationally demanding simulations, which is arguably not easy to achieve. This limitation is further discussed in section \\ref{discussion}.\n\nWe note a more uniform behavior for HPx$_{\\rm py}$-kNN across grid sizes than for HPx$_{\\rm py}$-DNN. Here the results for the 121  $\\times$ 121 grid are only slightly less accurate than those associated with the 61 $ \\times$ 61 grid (see Figures \\ref{fig15} - \\ref{fig16} where for brevity we only show concentration profiles for the C Al and S components). Furthermore, whenever observed the discrepancies between original and emulated profiles are more regularly scattered than for HPx$_{\\rm py}$-DNN. Note also that herein too, post-filtering could likely smooth out a large part of these deviations. In addition, owing to the use of a GPU to achieve the kNN calculations, the speedup provided by HPx$_{\\rm py}$-kNN are as large as those provided by HPx$_{\\rm py}$-DNN (Table \\ref{table4}).\n\nWith respect to the emulated solid amounts, the HPx$_{\\rm py}$-DNN results look visually good for the the 61 $\\times$ 61 grid. This is shown in Figure \\ref{fig17} for the C, Al and S chemical components, while emulation of the H, O Ca and Si chemical components is globally of similar quality (not shown). Nevertheless, significant deviations appear towards the end of the simulation period for every chemical component (see Figure \\ref{fig18} for the C, Al and S chemical components, emulation of the H, O Ca and Si chemical components shows the same level of mismatch). The HPx$_{\\rm py}$-kNN predictions are also fairly accurate for the 61 $\\times$ 61 (see Figure \\ref{fig19} for the C, Al and S chemical components, emulation of the H, O Ca and Si chemical components exhibits a globally similar quality) while some discrepancies show up at the end of the simulation (Figure \\ref{fig20}). However, the mismatch is less pronounced than for HPx$_{\\rm py}$-DNN. Overall, HPx$_{\\rm py}$-kNN appears to be somewhat more robust than HPx$_{\\rm py}$-DNN for this cement system, while providing the same (large) speedup.\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure11.png}\n\t\\caption{2D concentration profiles obtained for cement system 2 after 2 and 6 (final time step) years. RTM means the original HPx$_{\\rm{4C}}$ model and TM+DNN denotes the Hydrus transport model coupled with our DNN geochemical emulator (HPx$_{\\rm py}$-DNN). The first to fourth row present profiles for $Ca^{\\rm{conc}}$, $Si^{\\rm{conc}}$, $H^{\\rm{conc}}$, and $O^{\\rm{conc}}$, respectively. The considered grid size is 61 $\\times$ 61.}\n\t\\label{fig11}\n\\end{figure}\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure12.png}\n\t\\caption{2D concentration profiles obtained for cement system 2 after 2 and 6 (final time step) years. RTM means the original HPx$_{\\rm{4C}}$ model and TM+DNN denotes the Hydrus transport model coupled with our DNN geochemical emulator (HPx$_{\\rm py}$-DNN). The first to third row present profiles for $C^{\\rm{conc}}$, $Al^{\\rm{conc}}$, and $S^{\\rm{conc}}$, respectively. The considered grid size is 61 $\\times$ 61.}\n\t\\label{fig12}\n\\end{figure}\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure13.png}\n\t\\caption{2D concentration profiles obtained for cement system 2 after 2 and 6 (final time step) years. RTM means the original HPx$_{\\rm{4C}}$ model and TM+DNN denotes the Hydrus transport model coupled with our DNN geochemical emulator (HPx$_{\\rm py}$-DNN). The first to fourth row present profiles for $Ca^{\\rm{conc}}$, $Si^{\\rm{conc}}$, $H^{\\rm{conc}}$, and $O^{\\rm{conc}}$, respectively. The considered grid size is 121 $\\times$ 121.}\n\t\\label{fig13}\n\\end{figure}\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure14.png}\n\t\\caption{2D concentration profiles obtained for cement system 2 after 2 and 6 (final time step) years. RTM means the original HPx$_{\\rm{4C}}$ model and TM+DNN denotes the Hydrus transport model coupled with our DNN geochemical emulator (HPx$_{\\rm py}$-DNN). The first to third row present profiles for $C^{\\rm{conc}}$, $Al^{\\rm{conc}}$, and $S^{\\rm{conc}}$, respectively. The considered grid size is 121 $\\times$ 121.}\n\t\\label{fig14}\n\\end{figure}\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure15.png}\n\t\\caption{2D concentration profiles obtained for cement system 2 after 2 and 6 (final time step) years. RTM means the original HPx$_{\\rm{4C}}$ model and TM+kNN denotes the Hydrus transport model coupled with our kNN geochemical emulator (HPx$_{\\rm py}$-kNN).  The first to third row present profiles for $C^{\\rm{conc}}$, $Al^{\\rm{conc}}$, and $S^{\\rm{conc}}$, respectively. The considered grid size is 61 $\\times$ 61.}\n\t\\label{fig15}\n\\end{figure}\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure16.png}\n\t\\caption{2D concentration profiles obtained for cement system 2 after 2 and 6 (final time step) years. RTM means the original HPx$_{\\rm{4C}}$ model and TM+kNN denotes the Hydrus transport model coupled with our kNN geochemical emulator (HPx$_{\\rm py}$-kNN). The first to third row present profiles for $C^{\\rm{conc}}$, $Al^{\\rm{conc}}$, and $S^{\\rm{conc}}$, respectively. The considered grid size is 121 $\\times$ 121.}\n\t\\label{fig16}\n\\end{figure}\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure17.png}\n\t\\caption{2D solid amount profiles of the C, Al and S chemical components of cement system 2 after 2 and 6 (final time step) years. RTM means the original HPx$_{\\rm{4C}}$ model and TM+DNN denotes the Hydrus transport model coupled with our DNN geochemical emulator (HPx$_{\\rm py}$-DNN). The first to third row present profiles for $C^{\\rm{solid}}$, $Al^{\\rm{solid}}$, and $S^{\\rm{solid}}$, respectively The considered grid size is 61 $\\times$ 61.}\n\t\\label{fig17}\n\\end{figure}\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure18.png}\n\t\\caption{2D solid amount profiles of the C, Al and S chemical components of cement system 2 after 2 and 6 (final time step) years. RTM means the original HPx$_{\\rm{4C}}$ model and TM+DNN denotes the Hydrus transport model coupled with our DNN geochemical emulator (HPx$_{\\rm py}$-DNN). The first to third row present profiles for $C^{\\rm{solid}}$, $Al^{\\rm{solid}}$, and $S^{\\rm{solid}}$, respectively The considered grid size is 121 $\\times$ 121.}\n\t\\label{fig18}\n\\end{figure}\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure19.png}\n\t\\caption{2D solid amount profiles of the C, Al and S chemical components of cement system 2 after 2 and 6 (final time step) years. RTM means the original HPx$_{\\rm{4C}}$ model and TM+kNN denotes the Hydrus transport model coupled with our kNN geochemical emulator (HPx$_{\\rm py}$-kNN). The first to third row present profiles for $C^{\\rm{solid}}$, $Al^{\\rm{solid}}$, and $S^{\\rm{solid}}$, respectively The considered grid size is 61 $\\times$ 61.}\n\t\\label{fig19}\n\\end{figure}\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure20.png}\n\t\\caption{2D solid amount profiles of the C, Al and S chemical components of cement system 2 after 2 and 6 (final time step) years. RTM means the original HPx$_{\\rm{4C}}$ model and TM+kNN denotes the Hydrus transport model coupled with our kNN geochemical emulator (HPx$_{\\rm py}$-kNN). The first to third row present profiles for $C^{\\rm{solid}}$, $Al^{\\rm{solid}}$, and $S^{\\rm{solid}}$, respectively The considered grid size is 121 $\\times$ 121.}\n\t\\label{fig20}\n\\end{figure}\n\n\\begin{table}[h!]\n\t\\caption{Speedups offered by the KNN and DNN emulators in HPx$_{\\rm py}$ for the reactive transport simulations considered for cement system 2. The HPx$_{\\rm{4C}}$ calculations involve the parallelization of PHREEQC over our 4 CPUs. The HPx$_{\\rm{1C}}$ calculations are performed on a single CPU. Both the kNN and DNN predictions make use of a GPU. ML signifies the used machine learning method for emulation, BC denotes the type of flow boundary conditions and GS is the grid size. The maximum possible speedups associated with HPx$_{\\rm{4C}}$ and HP$_{\\rm{1C}}$,  Max SP HPx$_{\\rm{4C}}$ and  Max SP HPx$_{\\rm{1C}}$, correspond to an hypothetical situation where the geochemical calculations incur zero computational cost.}\n\n\t\\begin{center}\n\t\t\\begin{tabular}{cccccccc}\n\t\t\t\\hline\n\t\t\t& & & & & & & \\\\\n\t\t\tML & BC & GS & HPx$_{\\rm{4C}}$ time (s) & SP HPx$_{\\rm{4C}}$ & Max SP HPx$_{\\rm{4C}}$ & SP HPx$_{\\rm{1C}}$ & Max SP HPx$_{\\rm{1C}}$ \\\\\n\t\t\tDNN & ADV & 61 $\\times$ 61 & 21,415 & 8.2 & 9.0 & 30.3 & 33.1\\\\\n\t\t\tkNN & ADV & 61 $\\times$ 61 & 21,415 & 7.9 & 9.0 & 28.9 & 33.1\\\\\n\t\t\tDNN & ADV & 121 $\\times$ 121 & 199,841 & 8.2 & 9.5 & 29.9 & 32.8\\\\\n\t\t\tkNN & ADV & 121 $\\times$ 121 & 199,841 & 8.5 & 9.5 & 31.3 & 32.8\\\\\n\t\t\t\\hline\n\t\t\\end{tabular}\n\t\\end{center}\n\n\t\\label{table4}\n\\end{table}\n\n\\FloatBarrier\n\n3.1 Ca-Si Problem\n\\subsection{Ca-Si Problem}\nFor this first cement system, the emulation problem consists of predicting at each time step of the RT simulation the (output) Ca, Si, H and O aqueous concentrations (mol/kg of water or mol/kgw) from the (input) total amounts of Ca and Si (mol). \n\n\\subsubsection{Training the emulators}\n\\label{train_res1}\nHere the kNN and DNN emulators are firstly trained using a set of 400,000 test examples for both. This training set is obtained by randomly sampling the two-dimensional input space by latin hypercube sampling (LHS) between $\\left[0,0\\right]$ and $\\rm{\\left[Ca^{tot}_{max},Si^{tot}_{max}\\right]}$, and running PHREEQC for each input sample, $\\rm{\\textbf{x}_i = \\left[Ca^{tot}_i, Si^{tot}_i\\right]}$ to get the corresponding output vectors, $\\rm{\\textbf{y}_i = \\left[Ca^{conc}_i,Si^{conc}_i,H^{conc}_i,O^{conc}_i\\right]}$. The upper bounds, $\\rm{Ca^{tot}_{max}}$ and $\\rm{Si^{tot}_{max}}$ are defined based on a cheap full RT simulation with advective-dispersive transport using a small 1D domain of 51 nodes. It is worth noting that the total amounts of $\\textbf{x}_i$, corresponding to the PHREEQC-simulated concentrations, $\\textbf{y}_i$, have to be corrected for the different amount of water between the training set and the transport simulations. Doing so, it turns out that about 20 \\% of the post-corrected $x_i$ values exceed their pre-defined upper bounds and these excessively large values need to be filtered out. Creating the 400,000 training examples thus required about 500,000 PHREEQC runs. As stated earlier, for this problem single-threaded PHREEQC performs about 670 geochemical calculations per second on our used Intel\\textsuperscript{\\textregistered} i7 CPU.\n\nWith respect to kNN, the tuning parameters are the number of neighbors, $k$, the type of distance measure, and the interpolation technique. We simply used the default settings: $k = 5$, euclidean distance and inverse-distance interpolation. Regarding training of the DNN, the 400,000 sample were split between the training set itself (90 \\% of the data) and a validation set (10 \\% of the data). The latter serves to monitor the evolution of the selected mean squared error loss function on samples that are not used for training, thereby detecting potential overfitting. If the validation loss stops decreasing before the fixed number of epochs has been completed, then  training is stopped. Importantly, the emulation is achieved in log-space for both the input, $X$ and output $Y$, domains. This because total amounts and concentrations of the involved components typically cover many orders of magnitudes (up to 10 orders or more). Using a DNN also requires some form of data normalization or standardization. Here both the $\\rm{log\\left(\\textbf{x}_i\\right)}$ and $\\rm{log\\left(\\textbf{y}_i\\right)}$ vectors are standardized around 0 with standard deviation of 1.\n\nFigure \\ref{fig1} illustrates the trained emulators' performance for geochemical predictions using an independent test set that comprises 10,000 test examples. Both kNN (Figures \\ref{fig1}a - d) and DNN (Figures \\ref{fig1}e - h) appear to be rather accurate . DNN also shows a slight degradation for the larger concentration values  (Figures \\ref{fig1}e - h). The latter is likely due to the combination of a small proportion of large concentration values in the training set with the log-transformation that implicitly pushes the DNN to try harder to fit the smaller concentrations during training. Regarding speedup and as written earlier, for this setup the single-threaded kNN method is 300 times faster than single-threaded PHREEQC for predicting the 10,000 concentration vectors all at once. The computational savings allowed by the DNN emulator when ran on our GPU is higher with a speedup as large as 4000 for predicting the same 10,000 concentration vectors all at once. \n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{0cm}\\includegraphics[width=35pc]{Figure1.png}\n\t\\caption{1-1 plots of the kNN (subfigures (a) - (d)) and DNN (subfigures (e) - (h)) emulators' performance obtained for system 1 when the kNN training base contains 400,000 samples and the DNN is trained using the same 400,000 samples. Here ``true\" means the original PHREEQC-simulated data and ``predicted\" denotes the emulated (that is, kNN-simulated and DNN-simulated) data. Hence, the $x$-axis and $y$-axis present the original and emulated 10,000 independent test data points, respectively. The RMSE and $Q_2$ coefficient denote the root-mean-square-error and coefficient of determination in testing mode, respectively, between the original and emulated 10,000 test data points.}\n\t\\label{fig1}\n\\end{figure}\n\nTo test the sensitivity of the emulators' performance to the training set size, training was also performed using reduced training sets comprising 100,000, 10,000, 1000 and 100 samples, respectively. It seen that the DNN performance achieved  when using 10,000 training samples is virtually the same as that obtained when using 400,000 training samples (Figure \\ref{fig2} and Table \\ref{table2}). It is only for training sets smaller than 1000 samples that the DNN performance starts to degrade significantly (Table \\ref{table2}). In contrast, the behavior of kNN appears to markedly decrease as the training set gets smaller (Figure \\ref{fig2} and Table \\ref{table1}).\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{0cm}\\includegraphics[width=35pc]{Figure2.png}\n\t\\caption{1-1 plots of the kNN (subfigures (a) - (d)) and DNN (subfigures (e) - (h)) emulators' performance obtained for system 1 when the kNN training base contains 10,000 samples and the DNN is trained using the same 10,000 samples. Here ``true\" means the original PHREEQC-simulated data and ``predicted\" denotes the emulated (that is, kNN-simulated and DNN-simulated) data. Hence, the $x$-axis and $y$-axis present the original and emulated 10,000 independent test data points, respectively. The RMSE and $Q_2$ coefficient denote the root-mean-square-error and coefficient of determination in testing mode, respectively, between the original and emulated 10,000 test data points.}\n\t\\label{fig2}\n\\end{figure}\n\n\\begin{table}[h!]\n\t\\caption{Performance of the DNN and kNN emulators for cement system 1 and different training set sizes. For brevity, only the results for Ca$^{conc}$ and Si$^{conc}$ are shown. The units are mol per kg of water (mol/kgw). ML refers to the type of emulator, TR signifies the size of the training set, and the RMSE and $Q_2$ coefficient denote the root-mean-square-error and coefficient of determination in testing mode, respectively, between the original and emulated 10,000 test data points.}\n\t\\begin{center}\n\t\t\\begin{tabular}{cccccccc}\n\t\t\t\\hline\n\t\t\t& & & & & \\\\\n\t\t\tML & TR & RMSE - Ca$^{conc}$ & $Q_2$ - Ca$^{conc}$ & RMSE - Si$^{conc}$ & $Q_2$ - Si$^{conc}$\\\\\n\t\t\tDNN & 4 $\\times$ 10$^{5}$ & 4.50 $\\times$ 10$^{-5}$ & 0.9999 & 1.21 $\\times$ 10$^{-6}$ & 0.9999\\\\\n\t\t\tDNN & 1 $\\times$ 10$^{5}$ & 5.06 $\\times$ 10$^{-5}$ & 0.9999 & 1.30 $\\times$ 10$^{-6}$ & 0.9999\\\\\n\t\t\tDNN & 1 $\\times$ 10$^{4}$ & 4.57 $\\times$ 10$^{-5}$ & 0.9999 & 1.48 $\\times$ 10$^{-6}$ & 0.9999\\\\\n\t\t\tDNN & 1 $\\times$ 10$^{3}$ & 8.02 $\\times$ 10$^{-5}$ & 0.9998 & 3.31 $\\times$ 10$^{-6}$ & 0.9994\\\\\n\t\t\tDNN & 1 $\\times$ 10$^{2}$ & 47.1 $\\times$ 10$^{-5}$ & 0.9935 & 9.02 $\\times$ 10$^{-6}$ & 0.9959\\\\\n\t\t\tkNN & 4 $\\times$ 10$^{5}$ & 2.52 $\\times$ 10$^{-5}$ & 1.0000 & 1.59 $\\times$ 10$^{-6}$ & 0.9999\\\\\n\t\t\tkNN & 1 $\\times$ 10$^{5}$ & 6.27 $\\times$ 10$^{-5}$ & 0.9999 & 4.20 $\\times$ 10$^{-6}$ & 0.9991\\\\\n\t\t\tkNN & 1 $\\times$ 10$^{4}$ & 14.1 $\\times$ 10$^{-5}$ & 0.9994 & 13.5 $\\times$ 10$^{-6}$ & 0.9908\\\\\n\t\t\tkNN & 1 $\\times$ 10$^{3}$ & 43.9 $\\times$ 10$^{-5}$ & 0.9944 & 74.6 $\\times$ 10$^{-6}$ & 0.7180\\\\\n\t\t\tkNN & 1 $\\times$ 10$^{2}$  & 96.4 $\\times$ 10$^{-5}$ & 0.9729 & 55.0 $\\times$ 10$^{-6}$ & 0.8468\\\\\n\t\t\\end{tabular}\n\t\\end{center}\n\t\\label{table2}\n\\end{table}\n\n\\FloatBarrier\n\n\\subsubsection{Reactive transport simulation}\n\\label{prob12Dres}\n\nThis section focuses on reactive transport simulations with HPx$_{\\rm py}$ within cement system 1, under both advective-dispersive and diffusive transport conditions. As written above, the domain sizes are both 61 $\\times$ 61 and 121 $\\times$ 121 for the advection-dispersion case and, because of computational constraints, solely 61 $\\times$ 61  for the diffusion case. In addition, the simulation time period is 2 years for the advection-dispersion case and 1 year for the diffusion case. Figures \\ref{fig3} and \\ref{fig4} present times series of original and emulated Ca, Si, H and O concentrations at 5 locations within the 2D domain for advective-dispersive transport conditions, for both our kNN-based (HPx$_{\\rm py}$-kNN) and DNN-based (HPx$_{\\rm py}$-DNN) reactive transport codes. It is seen that HPx$_{\\rm py}$-kNN and HPx$_{\\rm py}$-DNN both induce a quite good simulation accuracy. Also, the results for the diffusive transport case are of similarly good quality (not shown). Figures \\ref{fig5} - \\ref{fig6} provide more insights into the HPx$_{\\rm py}$-kNN and HPx$_{\\rm py}$-DNN performances by displaying 2D Ca, Si, H and O concentration profiles at a given time. For each experiment and chemical component, this time is selected as to be well representative of the simulated dynamics. It is observed that the original and emulated images are visually almost indistinguishable for the advection-dispersion case (Figure \\ref{fig5}). For the diffusion case, the emulators also perform quite well for Ca, H and O (Figures\\ref{fig6}a - c, g - i and j - l), while some slight to moderate discrepancies appear at the concentration front for Si (Figures \\ref{fig6}d - f). Notwithstanding, the Si concentration remains globally well predicted. Furthermore, Figures \\ref{fig7} - \\ref{fig8} present the original and emulated 2D solid amount profiles corresponding to Figures \\ref{fig5} - \\ref{fig6}. The original solid amount profiles are overall well approximated by HPx$_{\\rm py}$-kNN and HPx$_{\\rm py}$-DNN for the advection-dispersion case (Figure \\ref{fig7}), even though some mismatch appears at the border of the fully depleted zone for the H component. As of the diffusion case (Figure \\ref{fig8}), the same kind of mismatch is observed for the emulated solid amounts of H by HPx$_{\\rm py}$-kNN while the emulated profiles by HPx$_{\\rm py}$-DNN show some more discrepancies. Though we decided to present raw emulation results, we would like to stress that some if not all of the observed artifacts could likely be smoothed out by using some post-filtering such as median filtering.\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure3.png}\n\t\\caption{Time series of original (RTM, solid green lines) and HPx$_{\\rm py}$-kNN emulated (TM+kNN, dashed orange lines) concentrations (mol/kg) of Ca, H, O and Si at selected observations points for cement system 1 and advective-dispersive transport. Obs. 1 - 5 denote the selected observation points, with the following $\\left[x,y\\right]$ locations (in cm). Obs. 1:  $\\left[0.5, 2.5\\right]$, Obs. 2: $\\left[1, 2\\right]$, Obs. 3: $\\left[2, 2\\right]$, Obs. 4: $\\left[1, 1\\right]$, Obs. 5: $\\left[2, 1\\right]$. The results for the 121 $\\times$ 121 grid size are rather similar.}\n\t\\label{fig3}\n\\end{figure}\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure4.png}\n\t\\caption{Time series of original (RTM, solid green lines) and HPx$_{\\rm py}$-DNN emulated (TM+DNN, dashed orange lines) concentrations (mol/kg) of Ca, H, O and Si at selected observations points for cement system 1 and advective-dispersive transport. Obs. 1 - 5 denote the selected observation points, with the following $\\left[x,y\\right]$ locations (in cm). Obs. 1:  $\\left[0.5, 2.5\\right]$, Obs. 2: $\\left[1, 2\\right]$, Obs. 3: $\\left[2, 2\\right]$, Obs. 4: $\\left[1, 1\\right]$, Obs. 5: $\\left[2, 1\\right]$. The results for the 121 $\\times$ 121 grid size are rather similar.}\n\t\\label{fig4}\n\\end{figure}\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure5.png}\n\t\\caption{2D concentration profiles obtained for cement system 1 at the end of the 2-year simulation performed for the advection-dispersion case. RTM means the original HP$_{\\rm{4C}}$ model, TM+kNN denotes the Hydrus transport model coupled with our kNN geochemical emulator (HPx$_{\\rm py}$-kNN) and TM+DNN signifies the Hydrus transport model coupled with our DNN geochemical emulator (HPx$_{\\rm py}$-DNN). The first to fourth row present profiles for $Ca^{\\rm{conc}}$, $Si^{\\rm{conc}}$, $H^{\\rm{conc}}$, and $O^{\\rm{conc}}$, respectively. The considered grid size is 61 $\\times$ 61. The results for the 121 $\\times$ 121 grid are rather similar.}\n\t\\label{fig5}\n\\end{figure}\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure6.png}\n\t\\caption{2D concentration profiles obtained for cement system 1 at the final time step of the 1-year simulation performed for the diffusion case. RTM means the original HP$_{\\rm{4C}}$ model, TM+kNN denotes the Hydrus transport model coupled with our kNN geochemical emulator (HPx$_{\\rm py}$-kNN) and TM+DNN signifies the Hydrus transport model coupled with our DNN geochemical emulator (HPx$_{\\rm py}$-DNN). The first to fourth row present profiles for $Ca^{\\rm{conc}}$, $Si^{\\rm{conc}}$, $H^{\\rm{conc}}$, and $O^{\\rm{conc}}$, respectively. The considered grid size is 61 $\\times$ 61.}\n\t\\label{fig6}\n\\end{figure}\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure7.png}\n\t\\caption{2D solid amount profiles obtained for cement system 1 at the end of the 2-year simulation performed for the advection-dispersion case. RTM means the original HP$_{\\rm{4C}}$ model, TM+kNN denotes the Hydrus transport model coupled with our kNN geochemical emulator (HPx$_{\\rm py}$-kNN) and TM+DNN signifies the Hydrus transport model coupled with our DNN geochemical emulator (HPx$_{\\rm py}$-DNN). The first to fourth row present profiles for $Ca^{\\rm{solid}}$, $Si^{\\rm{solid}}$, $H^{\\rm{solid}}$, and $O^{\\rm{solid}}$, respectively. The considered grid size is 61 $\\times$ 61. The results for the 121 $\\times$ 121 grid are rather similar.}\n\t\\label{fig7}\n\\end{figure}\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure8.png}\n\t\\caption{2D solid amount profiles obtained for cement system 1 at the final time step of the 1-year simulation performed for the diffusion case. RTM means the original HP$_{\\rm{4C}}$ model, TM+kNN denotes the Hydrus transport model coupled with our kNN geochemical emulator (HPx$_{\\rm py}$-kNN) and TM+DNN signifies the Hydrus transport model coupled with our DNN geochemical emulator (HPx$_{\\rm py}$-DNN). The first to fourth row present profiles for $Ca^{\\rm{solid}}$, $Si^{\\rm{solid}}$, $H^{\\rm{solid}}$, and $O^{\\rm{solid}}$, respectively. The considered grid size is 61 $\\times$ 61.}\n\t\\label{fig8}\n\\end{figure}\n\nThe speedups associated with the considered problem are detailed in Table \\ref{table3}. It is noted that the GPU-based DNN emulator allows for a speedup that is close to optimal. Indeed, The DNN speedups overall represent 85 \\% to 95 \\% of the maximum possible speedups (that is, speedups that would be obtained if the geochemcical calculations would come at no cost at all). The speedups associated with single-threaded kNN remain substantial but only amount to 57 \\% - 65 \\% of the corresponding maximum speedups. As detailed in section \\ref{train_res1}, the used kNN and DNN implementations are found to be respectively 300 and 4000 faster than single-threaded PHREEQC when predicting 10,000 points all one once for this geochemical system. Based on these numbers one could have expected the achieved speedups to represent say 90 \\% (kNN) or 99 \\% (DNN) of the maximum possible ones. A large part of the gaps between achieved and maximum possible speedups is thus likely caused by the time required for communicating and exchanging data between the main C/C++ code and the Python-based emulators.\n\n\\begin{table}[h!]\n\t\\caption{Speedups offered by the KNN and DNN emulators in HPx$_{\\rm py}$ for the reactive transport simulations considered for cement system 1. The HPx$_{\\rm{4C}}$ calculations involve the parallelization of PHREEQC over our 4 CPUs. The HPx$_{\\rm{1C}}$ calculations are performed on a single CPU. The kNN predictions are performed on a single CPU using the scikit-learn implementation while the DNN predictions make use of our GPU. ML signifies the used machine learning method for emulation, TC denotes transport conditions (ADV: advection-dispersion, DIF: diffusion) and GS is the grid size. The maximum possible speedups associated with HPx$_{\\rm{4C}}$ and HPx$_{\\rm{1C}}$,  Max SP HPx$_{\\rm{4C}}$ and  Max SP HPx$_{\\rm{1C}}$, correspond to an hypothetical situation where the geochemical calculations incur zero computational cost.}\n\n\t\\begin{center}\n\t\t\\begin{tabular}{cccccccc}\n\t\t\t\\hline\n\t\t\t& & & & & & & \\\\\n\t\t\tML & TC & GS & HPx$_{\\rm{4C}}$ time (s) & SP HPx$_{\\rm{4C}}$ & Max SP HPx$_{\\rm{4C}}$ & SP HPx$_{\\rm{1C}}$ & Max SP HPx$_{\\rm{1C}}$ \\\\\n\t\t\tDNN & ADV & 61 $\\times$ 61 & 3189 & 6.8 & 7.7 & 24.5 & 28.5 \\\\\n\t\t\tkNN & ADV & 61 $\\times$ 61 & 3189 & 5.0 & 7.7 & 18.0 & 28.5 \\\\\n\t\t\tDNN & ADV & 121 $\\times$ 121 & 23,337 & 5.0 & 5.2 & 17.0 & 18.0 \\\\\n\t\t\tkNN & ADV & 121 $\\times$ 121 & 23,337 & 3.4 & 5.2 & 11.6 & 18.0 \\\\\n\t\t\tDNN & DIF & 61 $\\times$ 61 & 25,448 & 4.2 & 4.9 & 16.2 & 19.1 \\\\\n\t\t\tkNN & DIF & 61 $\\times$ 61 & 25,448 & 2.8 & 4.9 & 10.8 & 19.1 \\\\\n\t\t\t\\hline\n\t\t\\end{tabular}\n\t\\end{center}\n\n\t\\label{table3}\n\\end{table}\n\n\\FloatBarrier\n\n3.1.2 Reactive transport simulation\n\\subsubsection{Reactive transport simulation}\n\\label{prob12Dres}\n\nThis section focuses on reactive transport simulations with HPx$_{\\rm py}$ within cement system 1, under both advective-dispersive and diffusive transport conditions. As written above, the domain sizes are both 61 $\\times$ 61 and 121 $\\times$ 121 for the advection-dispersion case and, because of computational constraints, solely 61 $\\times$ 61  for the diffusion case. In addition, the simulation time period is 2 years for the advection-dispersion case and 1 year for the diffusion case. Figures \\ref{fig3} and \\ref{fig4} present times series of original and emulated Ca, Si, H and O concentrations at 5 locations within the 2D domain for advective-dispersive transport conditions, for both our kNN-based (HPx$_{\\rm py}$-kNN) and DNN-based (HPx$_{\\rm py}$-DNN) reactive transport codes. It is seen that HPx$_{\\rm py}$-kNN and HPx$_{\\rm py}$-DNN both induce a quite good simulation accuracy. Also, the results for the diffusive transport case are of similarly good quality (not shown). Figures \\ref{fig5} - \\ref{fig6} provide more insights into the HPx$_{\\rm py}$-kNN and HPx$_{\\rm py}$-DNN performances by displaying 2D Ca, Si, H and O concentration profiles at a given time. For each experiment and chemical component, this time is selected as to be well representative of the simulated dynamics. It is observed that the original and emulated images are visually almost indistinguishable for the advection-dispersion case (Figure \\ref{fig5}). For the diffusion case, the emulators also perform quite well for Ca, H and O (Figures\\ref{fig6}a - c, g - i and j - l), while some slight to moderate discrepancies appear at the concentration front for Si (Figures \\ref{fig6}d - f). Notwithstanding, the Si concentration remains globally well predicted. Furthermore, Figures \\ref{fig7} - \\ref{fig8} present the original and emulated 2D solid amount profiles corresponding to Figures \\ref{fig5} - \\ref{fig6}. The original solid amount profiles are overall well approximated by HPx$_{\\rm py}$-kNN and HPx$_{\\rm py}$-DNN for the advection-dispersion case (Figure \\ref{fig7}), even though some mismatch appears at the border of the fully depleted zone for the H component. As of the diffusion case (Figure \\ref{fig8}), the same kind of mismatch is observed for the emulated solid amounts of H by HPx$_{\\rm py}$-kNN while the emulated profiles by HPx$_{\\rm py}$-DNN show some more discrepancies. Though we decided to present raw emulation results, we would like to stress that some if not all of the observed artifacts could likely be smoothed out by using some post-filtering such as median filtering.\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure3.png}\n\t\\caption{Time series of original (RTM, solid green lines) and HPx$_{\\rm py}$-kNN emulated (TM+kNN, dashed orange lines) concentrations (mol/kg) of Ca, H, O and Si at selected observations points for cement system 1 and advective-dispersive transport. Obs. 1 - 5 denote the selected observation points, with the following $\\left[x,y\\right]$ locations (in cm). Obs. 1:  $\\left[0.5, 2.5\\right]$, Obs. 2: $\\left[1, 2\\right]$, Obs. 3: $\\left[2, 2\\right]$, Obs. 4: $\\left[1, 1\\right]$, Obs. 5: $\\left[2, 1\\right]$. The results for the 121 $\\times$ 121 grid size are rather similar.}\n\t\\label{fig3}\n\\end{figure}\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure4.png}\n\t\\caption{Time series of original (RTM, solid green lines) and HPx$_{\\rm py}$-DNN emulated (TM+DNN, dashed orange lines) concentrations (mol/kg) of Ca, H, O and Si at selected observations points for cement system 1 and advective-dispersive transport. Obs. 1 - 5 denote the selected observation points, with the following $\\left[x,y\\right]$ locations (in cm). Obs. 1:  $\\left[0.5, 2.5\\right]$, Obs. 2: $\\left[1, 2\\right]$, Obs. 3: $\\left[2, 2\\right]$, Obs. 4: $\\left[1, 1\\right]$, Obs. 5: $\\left[2, 1\\right]$. The results for the 121 $\\times$ 121 grid size are rather similar.}\n\t\\label{fig4}\n\\end{figure}\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure5.png}\n\t\\caption{2D concentration profiles obtained for cement system 1 at the end of the 2-year simulation performed for the advection-dispersion case. RTM means the original HP$_{\\rm{4C}}$ model, TM+kNN denotes the Hydrus transport model coupled with our kNN geochemical emulator (HPx$_{\\rm py}$-kNN) and TM+DNN signifies the Hydrus transport model coupled with our DNN geochemical emulator (HPx$_{\\rm py}$-DNN). The first to fourth row present profiles for $Ca^{\\rm{conc}}$, $Si^{\\rm{conc}}$, $H^{\\rm{conc}}$, and $O^{\\rm{conc}}$, respectively. The considered grid size is 61 $\\times$ 61. The results for the 121 $\\times$ 121 grid are rather similar.}\n\t\\label{fig5}\n\\end{figure}\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure6.png}\n\t\\caption{2D concentration profiles obtained for cement system 1 at the final time step of the 1-year simulation performed for the diffusion case. RTM means the original HP$_{\\rm{4C}}$ model, TM+kNN denotes the Hydrus transport model coupled with our kNN geochemical emulator (HPx$_{\\rm py}$-kNN) and TM+DNN signifies the Hydrus transport model coupled with our DNN geochemical emulator (HPx$_{\\rm py}$-DNN). The first to fourth row present profiles for $Ca^{\\rm{conc}}$, $Si^{\\rm{conc}}$, $H^{\\rm{conc}}$, and $O^{\\rm{conc}}$, respectively. The considered grid size is 61 $\\times$ 61.}\n\t\\label{fig6}\n\\end{figure}\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure7.png}\n\t\\caption{2D solid amount profiles obtained for cement system 1 at the end of the 2-year simulation performed for the advection-dispersion case. RTM means the original HP$_{\\rm{4C}}$ model, TM+kNN denotes the Hydrus transport model coupled with our kNN geochemical emulator (HPx$_{\\rm py}$-kNN) and TM+DNN signifies the Hydrus transport model coupled with our DNN geochemical emulator (HPx$_{\\rm py}$-DNN). The first to fourth row present profiles for $Ca^{\\rm{solid}}$, $Si^{\\rm{solid}}$, $H^{\\rm{solid}}$, and $O^{\\rm{solid}}$, respectively. The considered grid size is 61 $\\times$ 61. The results for the 121 $\\times$ 121 grid are rather similar.}\n\t\\label{fig7}\n\\end{figure}\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure8.png}\n\t\\caption{2D solid amount profiles obtained for cement system 1 at the final time step of the 1-year simulation performed for the diffusion case. RTM means the original HP$_{\\rm{4C}}$ model, TM+kNN denotes the Hydrus transport model coupled with our kNN geochemical emulator (HPx$_{\\rm py}$-kNN) and TM+DNN signifies the Hydrus transport model coupled with our DNN geochemical emulator (HPx$_{\\rm py}$-DNN). The first to fourth row present profiles for $Ca^{\\rm{solid}}$, $Si^{\\rm{solid}}$, $H^{\\rm{solid}}$, and $O^{\\rm{solid}}$, respectively. The considered grid size is 61 $\\times$ 61.}\n\t\\label{fig8}\n\\end{figure}\n\nThe speedups associated with the considered problem are detailed in Table \\ref{table3}. It is noted that the GPU-based DNN emulator allows for a speedup that is close to optimal. Indeed, The DNN speedups overall represent 85 \\% to 95 \\% of the maximum possible speedups (that is, speedups that would be obtained if the geochemcical calculations would come at no cost at all). The speedups associated with single-threaded kNN remain substantial but only amount to 57 \\% - 65 \\% of the corresponding maximum speedups. As detailed in section \\ref{train_res1}, the used kNN and DNN implementations are found to be respectively 300 and 4000 faster than single-threaded PHREEQC when predicting 10,000 points all one once for this geochemical system. Based on these numbers one could have expected the achieved speedups to represent say 90 \\% (kNN) or 99 \\% (DNN) of the maximum possible ones. A large part of the gaps between achieved and maximum possible speedups is thus likely caused by the time required for communicating and exchanging data between the main C/C++ code and the Python-based emulators.\n\n\\begin{table}[h!]\n\t\\caption{Speedups offered by the KNN and DNN emulators in HPx$_{\\rm py}$ for the reactive transport simulations considered for cement system 1. The HPx$_{\\rm{4C}}$ calculations involve the parallelization of PHREEQC over our 4 CPUs. The HPx$_{\\rm{1C}}$ calculations are performed on a single CPU. The kNN predictions are performed on a single CPU using the scikit-learn implementation while the DNN predictions make use of our GPU. ML signifies the used machine learning method for emulation, TC denotes transport conditions (ADV: advection-dispersion, DIF: diffusion) and GS is the grid size. The maximum possible speedups associated with HPx$_{\\rm{4C}}$ and HPx$_{\\rm{1C}}$,  Max SP HPx$_{\\rm{4C}}$ and  Max SP HPx$_{\\rm{1C}}$, correspond to an hypothetical situation where the geochemical calculations incur zero computational cost.}\n\n\t\\begin{center}\n\t\t\\begin{tabular}{cccccccc}\n\t\t\t\\hline\n\t\t\t& & & & & & & \\\\\n\t\t\tML & TC & GS & HPx$_{\\rm{4C}}$ time (s) & SP HPx$_{\\rm{4C}}$ & Max SP HPx$_{\\rm{4C}}$ & SP HPx$_{\\rm{1C}}$ & Max SP HPx$_{\\rm{1C}}$ \\\\\n\t\t\tDNN & ADV & 61 $\\times$ 61 & 3189 & 6.8 & 7.7 & 24.5 & 28.5 \\\\\n\t\t\tkNN & ADV & 61 $\\times$ 61 & 3189 & 5.0 & 7.7 & 18.0 & 28.5 \\\\\n\t\t\tDNN & ADV & 121 $\\times$ 121 & 23,337 & 5.0 & 5.2 & 17.0 & 18.0 \\\\\n\t\t\tkNN & ADV & 121 $\\times$ 121 & 23,337 & 3.4 & 5.2 & 11.6 & 18.0 \\\\\n\t\t\tDNN & DIF & 61 $\\times$ 61 & 25,448 & 4.2 & 4.9 & 16.2 & 19.1 \\\\\n\t\t\tkNN & DIF & 61 $\\times$ 61 & 25,448 & 2.8 & 4.9 & 10.8 & 19.1 \\\\\n\t\t\t\\hline\n\t\t\\end{tabular}\n\t\\end{center}\n\n\t\\label{table3}\n\\end{table}\n\n\\FloatBarrier\n\n",
        "label": "fig3",
        "Descriptive_question1": "What color represents the original RTM concentrations in figure_3?",
        "Descriptive_question2": "What type of line is used for the HPx_py-kNN emulated concentrations in figure_3?",
        "Reasoning_question1": "How does the accuracy of the HPx_py-kNN emulator compare to the original RTM concentrations across different observation points in figure_3, and what might this suggest about the emulator's performance?",
        "Reasoning_question2": "What trends can be observed in the concentration changes of Ca, H, O, and Si over time at Obs. 1 in figure_3, and what might be the underlying chemical or physical processes driving these changes?",
        "Descriptive_answer1": "solid green",
        "Descriptive_answer2": "dashed orange",
        "Reasoning_answer1": "In figure_3, the HPx_py-kNN emulated concentrations, represented by dashed orange lines, closely follow the original RTM concentrations shown by solid green lines at all five observation points (Obs. 1 to 5). The close overlap and similar trends between the emulated and original lines suggest that the HPx_py-kNN emulator accurately replicates the reactive transport model's results spatially across different locations. This indicates a robust emulator performance capable of capturing the temporal evolution and spatial variation of Ca, H, O, and Si concentrations within the tested domain under advective-dispersive transport conditions.",
        "Reasoning_answer2": "At Obs. 1, figure_3 shows the concentrations of Ca, H, O, and Si vary over time, each following distinct temporal patterns. Ca concentration likely increases or varies due to dissolution/precipitation or transport processes, while Si, H, and O concentrations also change reflecting aqueous phase chemistry and interactions. The trends may indicate progressive chemical reactions such as cement hydration or mineral dissolution coupled with advective-dispersive transport moving reactants and products through the domain. The observed concentration dynamics result from the interplay of geochemical reactions and transport processes that evolve the aqueous concentrations spatially and temporally."
    },
    {
        "paper_id": "2107.07598.json",
        "image_id": "figure_4",
        "image_path": [
            "/home/yz979/scratch/chengye/scimolmo/test-images-tables/dataset_imagestables/2107.07598/images/Figure4.png"
        ],
        "caption": "Time series of original (RTM, solid green lines) and HPx$_{\\rm py}$-DNN emulated (TM+DNN, dashed orange lines) concentrations (mol/kg) of Ca, H, O and Si at selected observations points for cement system 1 and advective-dispersive transport. Obs. 1 - 5 denote the selected observation points, with the following $\\left[x,y\\right]$ locations (in cm). Obs. 1:  $\\left[0.5, 2.5\\right]$, Obs. 2: $\\left[1, 2\\right]$, Obs. 3: $\\left[2, 2\\right]$, Obs. 4: $\\left[1, 1\\right]$, Obs. 5: $\\left[2, 1\\right]$. The results for the 121 $\\times$ 121 grid size are rather similar.",
        "classify": "Chart",
        "section_info": "3 Results\n\\section{Results}\n\\label{results}\n\n\\subsection{Ca-Si Problem}\nFor this first cement system, the emulation problem consists of predicting at each time step of the RT simulation the (output) Ca, Si, H and O aqueous concentrations (mol/kg of water or mol/kgw) from the (input) total amounts of Ca and Si (mol). \n\n\\subsubsection{Training the emulators}\n\\label{train_res1}\nHere the kNN and DNN emulators are firstly trained using a set of 400,000 test examples for both. This training set is obtained by randomly sampling the two-dimensional input space by latin hypercube sampling (LHS) between $\\left[0,0\\right]$ and $\\rm{\\left[Ca^{tot}_{max},Si^{tot}_{max}\\right]}$, and running PHREEQC for each input sample, $\\rm{\\textbf{x}_i = \\left[Ca^{tot}_i, Si^{tot}_i\\right]}$ to get the corresponding output vectors, $\\rm{\\textbf{y}_i = \\left[Ca^{conc}_i,Si^{conc}_i,H^{conc}_i,O^{conc}_i\\right]}$. The upper bounds, $\\rm{Ca^{tot}_{max}}$ and $\\rm{Si^{tot}_{max}}$ are defined based on a cheap full RT simulation with advective-dispersive transport using a small 1D domain of 51 nodes. It is worth noting that the total amounts of $\\textbf{x}_i$, corresponding to the PHREEQC-simulated concentrations, $\\textbf{y}_i$, have to be corrected for the different amount of water between the training set and the transport simulations. Doing so, it turns out that about 20 \\% of the post-corrected $x_i$ values exceed their pre-defined upper bounds and these excessively large values need to be filtered out. Creating the 400,000 training examples thus required about 500,000 PHREEQC runs. As stated earlier, for this problem single-threaded PHREEQC performs about 670 geochemical calculations per second on our used Intel\\textsuperscript{\\textregistered} i7 CPU.\n\nWith respect to kNN, the tuning parameters are the number of neighbors, $k$, the type of distance measure, and the interpolation technique. We simply used the default settings: $k = 5$, euclidean distance and inverse-distance interpolation. Regarding training of the DNN, the 400,000 sample were split between the training set itself (90 \\% of the data) and a validation set (10 \\% of the data). The latter serves to monitor the evolution of the selected mean squared error loss function on samples that are not used for training, thereby detecting potential overfitting. If the validation loss stops decreasing before the fixed number of epochs has been completed, then  training is stopped. Importantly, the emulation is achieved in log-space for both the input, $X$ and output $Y$, domains. This because total amounts and concentrations of the involved components typically cover many orders of magnitudes (up to 10 orders or more). Using a DNN also requires some form of data normalization or standardization. Here both the $\\rm{log\\left(\\textbf{x}_i\\right)}$ and $\\rm{log\\left(\\textbf{y}_i\\right)}$ vectors are standardized around 0 with standard deviation of 1.\n\nFigure \\ref{fig1} illustrates the trained emulators' performance for geochemical predictions using an independent test set that comprises 10,000 test examples. Both kNN (Figures \\ref{fig1}a - d) and DNN (Figures \\ref{fig1}e - h) appear to be rather accurate . DNN also shows a slight degradation for the larger concentration values  (Figures \\ref{fig1}e - h). The latter is likely due to the combination of a small proportion of large concentration values in the training set with the log-transformation that implicitly pushes the DNN to try harder to fit the smaller concentrations during training. Regarding speedup and as written earlier, for this setup the single-threaded kNN method is 300 times faster than single-threaded PHREEQC for predicting the 10,000 concentration vectors all at once. The computational savings allowed by the DNN emulator when ran on our GPU is higher with a speedup as large as 4000 for predicting the same 10,000 concentration vectors all at once. \n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{0cm}\\includegraphics[width=35pc]{Figure1.png}\n\t\\caption{1-1 plots of the kNN (subfigures (a) - (d)) and DNN (subfigures (e) - (h)) emulators' performance obtained for system 1 when the kNN training base contains 400,000 samples and the DNN is trained using the same 400,000 samples. Here ``true\" means the original PHREEQC-simulated data and ``predicted\" denotes the emulated (that is, kNN-simulated and DNN-simulated) data. Hence, the $x$-axis and $y$-axis present the original and emulated 10,000 independent test data points, respectively. The RMSE and $Q_2$ coefficient denote the root-mean-square-error and coefficient of determination in testing mode, respectively, between the original and emulated 10,000 test data points.}\n\t\\label{fig1}\n\\end{figure}\n\nTo test the sensitivity of the emulators' performance to the training set size, training was also performed using reduced training sets comprising 100,000, 10,000, 1000 and 100 samples, respectively. It seen that the DNN performance achieved  when using 10,000 training samples is virtually the same as that obtained when using 400,000 training samples (Figure \\ref{fig2} and Table \\ref{table2}). It is only for training sets smaller than 1000 samples that the DNN performance starts to degrade significantly (Table \\ref{table2}). In contrast, the behavior of kNN appears to markedly decrease as the training set gets smaller (Figure \\ref{fig2} and Table \\ref{table1}).\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{0cm}\\includegraphics[width=35pc]{Figure2.png}\n\t\\caption{1-1 plots of the kNN (subfigures (a) - (d)) and DNN (subfigures (e) - (h)) emulators' performance obtained for system 1 when the kNN training base contains 10,000 samples and the DNN is trained using the same 10,000 samples. Here ``true\" means the original PHREEQC-simulated data and ``predicted\" denotes the emulated (that is, kNN-simulated and DNN-simulated) data. Hence, the $x$-axis and $y$-axis present the original and emulated 10,000 independent test data points, respectively. The RMSE and $Q_2$ coefficient denote the root-mean-square-error and coefficient of determination in testing mode, respectively, between the original and emulated 10,000 test data points.}\n\t\\label{fig2}\n\\end{figure}\n\n\\begin{table}[h!]\n\t\\caption{Performance of the DNN and kNN emulators for cement system 1 and different training set sizes. For brevity, only the results for Ca$^{conc}$ and Si$^{conc}$ are shown. The units are mol per kg of water (mol/kgw). ML refers to the type of emulator, TR signifies the size of the training set, and the RMSE and $Q_2$ coefficient denote the root-mean-square-error and coefficient of determination in testing mode, respectively, between the original and emulated 10,000 test data points.}\n\t\\begin{center}\n\t\t\\begin{tabular}{cccccccc}\n\t\t\t\\hline\n\t\t\t& & & & & \\\\\n\t\t\tML & TR & RMSE - Ca$^{conc}$ & $Q_2$ - Ca$^{conc}$ & RMSE - Si$^{conc}$ & $Q_2$ - Si$^{conc}$\\\\\n\t\t\tDNN & 4 $\\times$ 10$^{5}$ & 4.50 $\\times$ 10$^{-5}$ & 0.9999 & 1.21 $\\times$ 10$^{-6}$ & 0.9999\\\\\n\t\t\tDNN & 1 $\\times$ 10$^{5}$ & 5.06 $\\times$ 10$^{-5}$ & 0.9999 & 1.30 $\\times$ 10$^{-6}$ & 0.9999\\\\\n\t\t\tDNN & 1 $\\times$ 10$^{4}$ & 4.57 $\\times$ 10$^{-5}$ & 0.9999 & 1.48 $\\times$ 10$^{-6}$ & 0.9999\\\\\n\t\t\tDNN & 1 $\\times$ 10$^{3}$ & 8.02 $\\times$ 10$^{-5}$ & 0.9998 & 3.31 $\\times$ 10$^{-6}$ & 0.9994\\\\\n\t\t\tDNN & 1 $\\times$ 10$^{2}$ & 47.1 $\\times$ 10$^{-5}$ & 0.9935 & 9.02 $\\times$ 10$^{-6}$ & 0.9959\\\\\n\t\t\tkNN & 4 $\\times$ 10$^{5}$ & 2.52 $\\times$ 10$^{-5}$ & 1.0000 & 1.59 $\\times$ 10$^{-6}$ & 0.9999\\\\\n\t\t\tkNN & 1 $\\times$ 10$^{5}$ & 6.27 $\\times$ 10$^{-5}$ & 0.9999 & 4.20 $\\times$ 10$^{-6}$ & 0.9991\\\\\n\t\t\tkNN & 1 $\\times$ 10$^{4}$ & 14.1 $\\times$ 10$^{-5}$ & 0.9994 & 13.5 $\\times$ 10$^{-6}$ & 0.9908\\\\\n\t\t\tkNN & 1 $\\times$ 10$^{3}$ & 43.9 $\\times$ 10$^{-5}$ & 0.9944 & 74.6 $\\times$ 10$^{-6}$ & 0.7180\\\\\n\t\t\tkNN & 1 $\\times$ 10$^{2}$  & 96.4 $\\times$ 10$^{-5}$ & 0.9729 & 55.0 $\\times$ 10$^{-6}$ & 0.8468\\\\\n\t\t\\end{tabular}\n\t\\end{center}\n\t\\label{table2}\n\\end{table}\n\n\\FloatBarrier\n\n\\subsubsection{Reactive transport simulation}\n\\label{prob12Dres}\n\nThis section focuses on reactive transport simulations with HPx$_{\\rm py}$ within cement system 1, under both advective-dispersive and diffusive transport conditions. As written above, the domain sizes are both 61 $\\times$ 61 and 121 $\\times$ 121 for the advection-dispersion case and, because of computational constraints, solely 61 $\\times$ 61  for the diffusion case. In addition, the simulation time period is 2 years for the advection-dispersion case and 1 year for the diffusion case. Figures \\ref{fig3} and \\ref{fig4} present times series of original and emulated Ca, Si, H and O concentrations at 5 locations within the 2D domain for advective-dispersive transport conditions, for both our kNN-based (HPx$_{\\rm py}$-kNN) and DNN-based (HPx$_{\\rm py}$-DNN) reactive transport codes. It is seen that HPx$_{\\rm py}$-kNN and HPx$_{\\rm py}$-DNN both induce a quite good simulation accuracy. Also, the results for the diffusive transport case are of similarly good quality (not shown). Figures \\ref{fig5} - \\ref{fig6} provide more insights into the HPx$_{\\rm py}$-kNN and HPx$_{\\rm py}$-DNN performances by displaying 2D Ca, Si, H and O concentration profiles at a given time. For each experiment and chemical component, this time is selected as to be well representative of the simulated dynamics. It is observed that the original and emulated images are visually almost indistinguishable for the advection-dispersion case (Figure \\ref{fig5}). For the diffusion case, the emulators also perform quite well for Ca, H and O (Figures\\ref{fig6}a - c, g - i and j - l), while some slight to moderate discrepancies appear at the concentration front for Si (Figures \\ref{fig6}d - f). Notwithstanding, the Si concentration remains globally well predicted. Furthermore, Figures \\ref{fig7} - \\ref{fig8} present the original and emulated 2D solid amount profiles corresponding to Figures \\ref{fig5} - \\ref{fig6}. The original solid amount profiles are overall well approximated by HPx$_{\\rm py}$-kNN and HPx$_{\\rm py}$-DNN for the advection-dispersion case (Figure \\ref{fig7}), even though some mismatch appears at the border of the fully depleted zone for the H component. As of the diffusion case (Figure \\ref{fig8}), the same kind of mismatch is observed for the emulated solid amounts of H by HPx$_{\\rm py}$-kNN while the emulated profiles by HPx$_{\\rm py}$-DNN show some more discrepancies. Though we decided to present raw emulation results, we would like to stress that some if not all of the observed artifacts could likely be smoothed out by using some post-filtering such as median filtering.\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure3.png}\n\t\\caption{Time series of original (RTM, solid green lines) and HPx$_{\\rm py}$-kNN emulated (TM+kNN, dashed orange lines) concentrations (mol/kg) of Ca, H, O and Si at selected observations points for cement system 1 and advective-dispersive transport. Obs. 1 - 5 denote the selected observation points, with the following $\\left[x,y\\right]$ locations (in cm). Obs. 1:  $\\left[0.5, 2.5\\right]$, Obs. 2: $\\left[1, 2\\right]$, Obs. 3: $\\left[2, 2\\right]$, Obs. 4: $\\left[1, 1\\right]$, Obs. 5: $\\left[2, 1\\right]$. The results for the 121 $\\times$ 121 grid size are rather similar.}\n\t\\label{fig3}\n\\end{figure}\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure4.png}\n\t\\caption{Time series of original (RTM, solid green lines) and HPx$_{\\rm py}$-DNN emulated (TM+DNN, dashed orange lines) concentrations (mol/kg) of Ca, H, O and Si at selected observations points for cement system 1 and advective-dispersive transport. Obs. 1 - 5 denote the selected observation points, with the following $\\left[x,y\\right]$ locations (in cm). Obs. 1:  $\\left[0.5, 2.5\\right]$, Obs. 2: $\\left[1, 2\\right]$, Obs. 3: $\\left[2, 2\\right]$, Obs. 4: $\\left[1, 1\\right]$, Obs. 5: $\\left[2, 1\\right]$. The results for the 121 $\\times$ 121 grid size are rather similar.}\n\t\\label{fig4}\n\\end{figure}\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure5.png}\n\t\\caption{2D concentration profiles obtained for cement system 1 at the end of the 2-year simulation performed for the advection-dispersion case. RTM means the original HP$_{\\rm{4C}}$ model, TM+kNN denotes the Hydrus transport model coupled with our kNN geochemical emulator (HPx$_{\\rm py}$-kNN) and TM+DNN signifies the Hydrus transport model coupled with our DNN geochemical emulator (HPx$_{\\rm py}$-DNN). The first to fourth row present profiles for $Ca^{\\rm{conc}}$, $Si^{\\rm{conc}}$, $H^{\\rm{conc}}$, and $O^{\\rm{conc}}$, respectively. The considered grid size is 61 $\\times$ 61. The results for the 121 $\\times$ 121 grid are rather similar.}\n\t\\label{fig5}\n\\end{figure}\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure6.png}\n\t\\caption{2D concentration profiles obtained for cement system 1 at the final time step of the 1-year simulation performed for the diffusion case. RTM means the original HP$_{\\rm{4C}}$ model, TM+kNN denotes the Hydrus transport model coupled with our kNN geochemical emulator (HPx$_{\\rm py}$-kNN) and TM+DNN signifies the Hydrus transport model coupled with our DNN geochemical emulator (HPx$_{\\rm py}$-DNN). The first to fourth row present profiles for $Ca^{\\rm{conc}}$, $Si^{\\rm{conc}}$, $H^{\\rm{conc}}$, and $O^{\\rm{conc}}$, respectively. The considered grid size is 61 $\\times$ 61.}\n\t\\label{fig6}\n\\end{figure}\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure7.png}\n\t\\caption{2D solid amount profiles obtained for cement system 1 at the end of the 2-year simulation performed for the advection-dispersion case. RTM means the original HP$_{\\rm{4C}}$ model, TM+kNN denotes the Hydrus transport model coupled with our kNN geochemical emulator (HPx$_{\\rm py}$-kNN) and TM+DNN signifies the Hydrus transport model coupled with our DNN geochemical emulator (HPx$_{\\rm py}$-DNN). The first to fourth row present profiles for $Ca^{\\rm{solid}}$, $Si^{\\rm{solid}}$, $H^{\\rm{solid}}$, and $O^{\\rm{solid}}$, respectively. The considered grid size is 61 $\\times$ 61. The results for the 121 $\\times$ 121 grid are rather similar.}\n\t\\label{fig7}\n\\end{figure}\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure8.png}\n\t\\caption{2D solid amount profiles obtained for cement system 1 at the final time step of the 1-year simulation performed for the diffusion case. RTM means the original HP$_{\\rm{4C}}$ model, TM+kNN denotes the Hydrus transport model coupled with our kNN geochemical emulator (HPx$_{\\rm py}$-kNN) and TM+DNN signifies the Hydrus transport model coupled with our DNN geochemical emulator (HPx$_{\\rm py}$-DNN). The first to fourth row present profiles for $Ca^{\\rm{solid}}$, $Si^{\\rm{solid}}$, $H^{\\rm{solid}}$, and $O^{\\rm{solid}}$, respectively. The considered grid size is 61 $\\times$ 61.}\n\t\\label{fig8}\n\\end{figure}\n\nThe speedups associated with the considered problem are detailed in Table \\ref{table3}. It is noted that the GPU-based DNN emulator allows for a speedup that is close to optimal. Indeed, The DNN speedups overall represent 85 \\% to 95 \\% of the maximum possible speedups (that is, speedups that would be obtained if the geochemcical calculations would come at no cost at all). The speedups associated with single-threaded kNN remain substantial but only amount to 57 \\% - 65 \\% of the corresponding maximum speedups. As detailed in section \\ref{train_res1}, the used kNN and DNN implementations are found to be respectively 300 and 4000 faster than single-threaded PHREEQC when predicting 10,000 points all one once for this geochemical system. Based on these numbers one could have expected the achieved speedups to represent say 90 \\% (kNN) or 99 \\% (DNN) of the maximum possible ones. A large part of the gaps between achieved and maximum possible speedups is thus likely caused by the time required for communicating and exchanging data between the main C/C++ code and the Python-based emulators.\n\n\\begin{table}[h!]\n\t\\caption{Speedups offered by the KNN and DNN emulators in HPx$_{\\rm py}$ for the reactive transport simulations considered for cement system 1. The HPx$_{\\rm{4C}}$ calculations involve the parallelization of PHREEQC over our 4 CPUs. The HPx$_{\\rm{1C}}$ calculations are performed on a single CPU. The kNN predictions are performed on a single CPU using the scikit-learn implementation while the DNN predictions make use of our GPU. ML signifies the used machine learning method for emulation, TC denotes transport conditions (ADV: advection-dispersion, DIF: diffusion) and GS is the grid size. The maximum possible speedups associated with HPx$_{\\rm{4C}}$ and HPx$_{\\rm{1C}}$,  Max SP HPx$_{\\rm{4C}}$ and  Max SP HPx$_{\\rm{1C}}$, correspond to an hypothetical situation where the geochemical calculations incur zero computational cost.}\n\n\t\\begin{center}\n\t\t\\begin{tabular}{cccccccc}\n\t\t\t\\hline\n\t\t\t& & & & & & & \\\\\n\t\t\tML & TC & GS & HPx$_{\\rm{4C}}$ time (s) & SP HPx$_{\\rm{4C}}$ & Max SP HPx$_{\\rm{4C}}$ & SP HPx$_{\\rm{1C}}$ & Max SP HPx$_{\\rm{1C}}$ \\\\\n\t\t\tDNN & ADV & 61 $\\times$ 61 & 3189 & 6.8 & 7.7 & 24.5 & 28.5 \\\\\n\t\t\tkNN & ADV & 61 $\\times$ 61 & 3189 & 5.0 & 7.7 & 18.0 & 28.5 \\\\\n\t\t\tDNN & ADV & 121 $\\times$ 121 & 23,337 & 5.0 & 5.2 & 17.0 & 18.0 \\\\\n\t\t\tkNN & ADV & 121 $\\times$ 121 & 23,337 & 3.4 & 5.2 & 11.6 & 18.0 \\\\\n\t\t\tDNN & DIF & 61 $\\times$ 61 & 25,448 & 4.2 & 4.9 & 16.2 & 19.1 \\\\\n\t\t\tkNN & DIF & 61 $\\times$ 61 & 25,448 & 2.8 & 4.9 & 10.8 & 19.1 \\\\\n\t\t\t\\hline\n\t\t\\end{tabular}\n\t\\end{center}\n\n\t\\label{table3}\n\\end{table}\n\n\\FloatBarrier\n\n\\subsection{Al-C-Ca-S-Si Problem}\n\nFor this second cement system, the emulation problem consists of predicting at each time step of the RT simulation the (output) Al, C, Ca, S, Si, H and O aqueous concentrations (mol/kgw) from the (input) total amounts of Al, C, Ca, S and Si (mol). Here we focus on advective-dispersive transport only while similarly as for cement system 1, the considered domain sizes are 61 $\\times$ 61 and 121 $\\times$ 121. Moreover, the simulation time period is set to 6 years. \n\nAs mentioned earlier, for this higher-dimensional problem it is observed that the used scikit-learn kNN implementation becomes prohibitively slow compared to HPx$_{\\rm{4C}}$. We found that to get a good emulation accuracy the kNN training base needs to contain 1,000,000 samples (or more). This training base's size together with a 5-dimensional search space leads to an HPx$_{\\rm py}$-kNN reactive transport simulation time that is comparable to that of HPx$_{\\rm{4C}}$. Therefore, for this second cement system we built a custom kNN regressor around another kNN implementation contained in the FAISS package \\citep{faiss2017}. Our used FAISS variant allows for GPU computing and is much faster than scikit-learn for this cement system, but is slightly less accurate due to the use of an approximate rather than exact nearest neighbor search \\citep[see][for details]{faiss2017}.\n\n\\subsubsection{Training the DNN}\n\\label{train_res2}\n\nBuilding a good training set to perform a kNN search and learn the weights and biases of our DNN turned out to be a complicated task in this case. This because to make useful kNN predictions and/or learn an useful DNN, the training set must be sufficiently representative of the geochemical conditions encountered during the reactive transport simulation one wish to perform with HPx$_{\\rm py}$-kNN and the trained HPx$_{\\rm py}$-DNN. In contrast to cement system 1, creating the training set by sampling the $X$-space with a controlled randomness between predefined lower and upper bounds did not prove successful. We tried that strategy by drawing as much as 4,000,000 5-dimensional $\\textbf{x}$ vectors from the $X$-space using a Sobol low-discrepancy sequence \\citep[][]{Sobol1967, Joe-Kuo2003}. Such low-discrepancy sampling scheme covers the 5-dimensional hypercube more uniformly than LHS. Despite a good performance on the test set (not shown), the resulting DNN accuracy in reactive transport mode was never deemed satisfying. In other words, no satisfying ``global\" or ``universal\" DNN emulator could be devised for this cement system. This is probably caused by the fact for this problem, the input (5 total amounts) and output (7 aqueous concentrations) spaces are quite nonlinearly related and both cover 6 to 10 orders of magnitudes depending on the considered element. Therefore, we resorted to the alternative training strategy detailed below. The latter basically tries to grasp the complex correlations and higher-order dependencies that exist between the elements of $\\textbf{x}$ (total amounts, input space) for a given reactive transport simulation setup, in order to produce a training set that honors these between-input relationships.\n\n\\begin{itemize}\n\t\\item Perform a ``cheap\" full reactive transport simulation under the transport conditions and geochemistry of interest and collect the resulting $\\textbf{x}$ -$\\textbf{y}$  pairs of examples (for the considered grid nodes and time steps). Computational demand controls what domain size and simulation time period can be used for this cheap calculation. We used a modest 16 $\\times$ 16 domain and a simulation time period of 10 years. The associated HPx$_{\\rm{4C}}$ runtimes is 180 s.\n\t\n\t\\item Fit a kernel density estimator (KDE) with a Gaussian kernel to the collected $\\textbf{x}$ vectors (encapsulated in the $\\textbf{X}$ array) and generate a fixed number of new input vectors, $\\textbf{x}_{KDE}$. Then run PHREEQC for the $\\textbf{X}_{KDE}$ set to get the corresponding output set, $\\textbf{Y}_{KDE}$. Now apply the correction for porosity described in section \\ref{train_res1} to the $\\textbf{X}_{KDE}$ set and form the training set by merging the ensemble of $\\textbf{x}$-$\\textbf{y}$ pairs with that of the $\\textbf{x}_{KDE}$-$\\textbf{y}_{KDE}$ pairs. The number of produced unique examples by the considered cheap HPx$_{\\rm{4C}}$ simulations varied between 10,000 and 50,000. The KDE-based enrichment of this dataset was deemed necessary to provide more input variability, thereby avoiding overfitting of the trained DNN and improving the kNN accuracy, while still honoring the complex between-input relationships. The number of KDE-generated samples was set as to obtain a total training set size of 1,000,000 examples. A key component of the approach is the bandwith parameter of the Gaussian KDE kernel which controls how much the KDE-generated samples depart from the original ensemble. After limited trial and error, we fixed the kernel bandwith to 0.0025 for the considered case studies.\n\t\n\\end{itemize}\n\nThe scatter plots in Figure \\ref{fig9} illustrate our training set creation procedure. The orange dots depict the pairwise relationships observed between the 5 elements of $\\textbf{x}$ in the cheap simulation. The red and cyan dots in Figure \\ref{fig9} represent the KDE-generated samples with the selected bandwidth, before and after applying the correction for porosity, respectively. Training of the DNN is achieved using the ensemble of original and KDE-corrected input points. We refer to this kind of dataset as RT-based, since it is based on an albeit cheap, full RT simulation. Furthermore, we refer to the obtained DNN and kNN emulators as ``local\" emulators, since as opposed to the emulators constructed for cement system 1, the current emulators are only valid for the input conditions encapsulated in the RT-based training set.\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1.5cm}\\includegraphics[width=50pc]{Figure9.png}\n\t\\caption{Scatter plots of the complex relationships between the five considered inputs for (1) the computationally cheap RT simulation of cement system 2 (orange dots) performed with the original HPx code, (2) the corresponding sampled points by kernel density estimation (KDE) using the selected bandwith (turquoise dots) and, (3) the same sampled KDE points after porosity correction (red dots).}\n\t\\label{fig9}\n\\end{figure}\n\nTraining performance of the DNN emulator is presented in Figure \\ref{fig10}. Similarly as for cement system 1, about 90\\% of the available data was used for the actual training of the DNN while the remaining 10\\% were used a validation set to contol overfitting. Lastly, performance is evaluated for both the trained DNN and kNN emulators using an independent test set of 10,000 examples. Overall, the accuracy of our ``local \" DNN emulator for this RT-inspired dataset is rather large with $Q_2$ values always greater or equal than 0.998. Training performance of the corresponding local kNN emulator is equally good (not shown). Regarding speedup, for this problem single-threaded PHREEQC achieves about 210 geochemical calculations per second on our used Intel\\textsuperscript{\\textregistered} i7 CPU while the GPU-based DNN and kNN emulators are both about 3000 times faster when predicting the 10,000 test points at once .\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure10.png}\n\t\\caption{1-1 plots of local DNN emulation performance obtained for system 2 when the local DNN is trained using 1,000,000 samples. The $x$-axis and $y$-axis present the original and emulated 10,000 independent test data points, respectively. The RMSE and $Q_2$ coefficient denote the root-mean-square-error and coefficient of determination in testing mode, respectively, between the original and emulated 10,000 test data points.}\n\t\\label{fig10}\n\\end{figure}\n\n\\subsubsection{Reactive transport simulation}\n\\label{prob2res}\nOur ``local\" DNN performs rather well when applied to the 61 $\\times$ 61 grid size and a time period of 6 years (Figures \\ref{fig11} - \\ref{fig12}). This for all components but C, for which some localized deviations appear between original and emulated 2D concentration profiles towards the end of the simulation period (Figure \\ref{fig12}). When applied to the 121 $\\times$ 121 grid, HPx$_{\\rm py}$-DNN produces additional discrepancies for O, Si and Al towards the end of the simulation period (Figures \\ref{fig13} - \\ref{fig14}). Yet most of the observed artifacts could probably be smoothed out by using post-filtering. The associated speedups are listed in Table \\ref{table4}. These speedups are larger to those obtained for cement system 1, with values between 8 and 9 when evaluated against HPx$_{\\rm{4C}}$. These speedups represent about 85 \\% to 90 \\% of the maximum possible speedup (Table \\ref{table3}). Overall, these findings indicate that for the considered problem, our RT-based training of a local DNN only works is the training set is sufficiently representative of the particular geochemical conditions encountered in the computationally demanding simulations, which is arguably not easy to achieve. This limitation is further discussed in section \\ref{discussion}.\n\nWe note a more uniform behavior for HPx$_{\\rm py}$-kNN across grid sizes than for HPx$_{\\rm py}$-DNN. Here the results for the 121  $\\times$ 121 grid are only slightly less accurate than those associated with the 61 $ \\times$ 61 grid (see Figures \\ref{fig15} - \\ref{fig16} where for brevity we only show concentration profiles for the C Al and S components). Furthermore, whenever observed the discrepancies between original and emulated profiles are more regularly scattered than for HPx$_{\\rm py}$-DNN. Note also that herein too, post-filtering could likely smooth out a large part of these deviations. In addition, owing to the use of a GPU to achieve the kNN calculations, the speedup provided by HPx$_{\\rm py}$-kNN are as large as those provided by HPx$_{\\rm py}$-DNN (Table \\ref{table4}).\n\nWith respect to the emulated solid amounts, the HPx$_{\\rm py}$-DNN results look visually good for the the 61 $\\times$ 61 grid. This is shown in Figure \\ref{fig17} for the C, Al and S chemical components, while emulation of the H, O Ca and Si chemical components is globally of similar quality (not shown). Nevertheless, significant deviations appear towards the end of the simulation period for every chemical component (see Figure \\ref{fig18} for the C, Al and S chemical components, emulation of the H, O Ca and Si chemical components shows the same level of mismatch). The HPx$_{\\rm py}$-kNN predictions are also fairly accurate for the 61 $\\times$ 61 (see Figure \\ref{fig19} for the C, Al and S chemical components, emulation of the H, O Ca and Si chemical components exhibits a globally similar quality) while some discrepancies show up at the end of the simulation (Figure \\ref{fig20}). However, the mismatch is less pronounced than for HPx$_{\\rm py}$-DNN. Overall, HPx$_{\\rm py}$-kNN appears to be somewhat more robust than HPx$_{\\rm py}$-DNN for this cement system, while providing the same (large) speedup.\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure11.png}\n\t\\caption{2D concentration profiles obtained for cement system 2 after 2 and 6 (final time step) years. RTM means the original HPx$_{\\rm{4C}}$ model and TM+DNN denotes the Hydrus transport model coupled with our DNN geochemical emulator (HPx$_{\\rm py}$-DNN). The first to fourth row present profiles for $Ca^{\\rm{conc}}$, $Si^{\\rm{conc}}$, $H^{\\rm{conc}}$, and $O^{\\rm{conc}}$, respectively. The considered grid size is 61 $\\times$ 61.}\n\t\\label{fig11}\n\\end{figure}\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure12.png}\n\t\\caption{2D concentration profiles obtained for cement system 2 after 2 and 6 (final time step) years. RTM means the original HPx$_{\\rm{4C}}$ model and TM+DNN denotes the Hydrus transport model coupled with our DNN geochemical emulator (HPx$_{\\rm py}$-DNN). The first to third row present profiles for $C^{\\rm{conc}}$, $Al^{\\rm{conc}}$, and $S^{\\rm{conc}}$, respectively. The considered grid size is 61 $\\times$ 61.}\n\t\\label{fig12}\n\\end{figure}\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure13.png}\n\t\\caption{2D concentration profiles obtained for cement system 2 after 2 and 6 (final time step) years. RTM means the original HPx$_{\\rm{4C}}$ model and TM+DNN denotes the Hydrus transport model coupled with our DNN geochemical emulator (HPx$_{\\rm py}$-DNN). The first to fourth row present profiles for $Ca^{\\rm{conc}}$, $Si^{\\rm{conc}}$, $H^{\\rm{conc}}$, and $O^{\\rm{conc}}$, respectively. The considered grid size is 121 $\\times$ 121.}\n\t\\label{fig13}\n\\end{figure}\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure14.png}\n\t\\caption{2D concentration profiles obtained for cement system 2 after 2 and 6 (final time step) years. RTM means the original HPx$_{\\rm{4C}}$ model and TM+DNN denotes the Hydrus transport model coupled with our DNN geochemical emulator (HPx$_{\\rm py}$-DNN). The first to third row present profiles for $C^{\\rm{conc}}$, $Al^{\\rm{conc}}$, and $S^{\\rm{conc}}$, respectively. The considered grid size is 121 $\\times$ 121.}\n\t\\label{fig14}\n\\end{figure}\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure15.png}\n\t\\caption{2D concentration profiles obtained for cement system 2 after 2 and 6 (final time step) years. RTM means the original HPx$_{\\rm{4C}}$ model and TM+kNN denotes the Hydrus transport model coupled with our kNN geochemical emulator (HPx$_{\\rm py}$-kNN).  The first to third row present profiles for $C^{\\rm{conc}}$, $Al^{\\rm{conc}}$, and $S^{\\rm{conc}}$, respectively. The considered grid size is 61 $\\times$ 61.}\n\t\\label{fig15}\n\\end{figure}\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure16.png}\n\t\\caption{2D concentration profiles obtained for cement system 2 after 2 and 6 (final time step) years. RTM means the original HPx$_{\\rm{4C}}$ model and TM+kNN denotes the Hydrus transport model coupled with our kNN geochemical emulator (HPx$_{\\rm py}$-kNN). The first to third row present profiles for $C^{\\rm{conc}}$, $Al^{\\rm{conc}}$, and $S^{\\rm{conc}}$, respectively. The considered grid size is 121 $\\times$ 121.}\n\t\\label{fig16}\n\\end{figure}\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure17.png}\n\t\\caption{2D solid amount profiles of the C, Al and S chemical components of cement system 2 after 2 and 6 (final time step) years. RTM means the original HPx$_{\\rm{4C}}$ model and TM+DNN denotes the Hydrus transport model coupled with our DNN geochemical emulator (HPx$_{\\rm py}$-DNN). The first to third row present profiles for $C^{\\rm{solid}}$, $Al^{\\rm{solid}}$, and $S^{\\rm{solid}}$, respectively The considered grid size is 61 $\\times$ 61.}\n\t\\label{fig17}\n\\end{figure}\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure18.png}\n\t\\caption{2D solid amount profiles of the C, Al and S chemical components of cement system 2 after 2 and 6 (final time step) years. RTM means the original HPx$_{\\rm{4C}}$ model and TM+DNN denotes the Hydrus transport model coupled with our DNN geochemical emulator (HPx$_{\\rm py}$-DNN). The first to third row present profiles for $C^{\\rm{solid}}$, $Al^{\\rm{solid}}$, and $S^{\\rm{solid}}$, respectively The considered grid size is 121 $\\times$ 121.}\n\t\\label{fig18}\n\\end{figure}\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure19.png}\n\t\\caption{2D solid amount profiles of the C, Al and S chemical components of cement system 2 after 2 and 6 (final time step) years. RTM means the original HPx$_{\\rm{4C}}$ model and TM+kNN denotes the Hydrus transport model coupled with our kNN geochemical emulator (HPx$_{\\rm py}$-kNN). The first to third row present profiles for $C^{\\rm{solid}}$, $Al^{\\rm{solid}}$, and $S^{\\rm{solid}}$, respectively The considered grid size is 61 $\\times$ 61.}\n\t\\label{fig19}\n\\end{figure}\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure20.png}\n\t\\caption{2D solid amount profiles of the C, Al and S chemical components of cement system 2 after 2 and 6 (final time step) years. RTM means the original HPx$_{\\rm{4C}}$ model and TM+kNN denotes the Hydrus transport model coupled with our kNN geochemical emulator (HPx$_{\\rm py}$-kNN). The first to third row present profiles for $C^{\\rm{solid}}$, $Al^{\\rm{solid}}$, and $S^{\\rm{solid}}$, respectively The considered grid size is 121 $\\times$ 121.}\n\t\\label{fig20}\n\\end{figure}\n\n\\begin{table}[h!]\n\t\\caption{Speedups offered by the KNN and DNN emulators in HPx$_{\\rm py}$ for the reactive transport simulations considered for cement system 2. The HPx$_{\\rm{4C}}$ calculations involve the parallelization of PHREEQC over our 4 CPUs. The HPx$_{\\rm{1C}}$ calculations are performed on a single CPU. Both the kNN and DNN predictions make use of a GPU. ML signifies the used machine learning method for emulation, BC denotes the type of flow boundary conditions and GS is the grid size. The maximum possible speedups associated with HPx$_{\\rm{4C}}$ and HP$_{\\rm{1C}}$,  Max SP HPx$_{\\rm{4C}}$ and  Max SP HPx$_{\\rm{1C}}$, correspond to an hypothetical situation where the geochemical calculations incur zero computational cost.}\n\n\t\\begin{center}\n\t\t\\begin{tabular}{cccccccc}\n\t\t\t\\hline\n\t\t\t& & & & & & & \\\\\n\t\t\tML & BC & GS & HPx$_{\\rm{4C}}$ time (s) & SP HPx$_{\\rm{4C}}$ & Max SP HPx$_{\\rm{4C}}$ & SP HPx$_{\\rm{1C}}$ & Max SP HPx$_{\\rm{1C}}$ \\\\\n\t\t\tDNN & ADV & 61 $\\times$ 61 & 21,415 & 8.2 & 9.0 & 30.3 & 33.1\\\\\n\t\t\tkNN & ADV & 61 $\\times$ 61 & 21,415 & 7.9 & 9.0 & 28.9 & 33.1\\\\\n\t\t\tDNN & ADV & 121 $\\times$ 121 & 199,841 & 8.2 & 9.5 & 29.9 & 32.8\\\\\n\t\t\tkNN & ADV & 121 $\\times$ 121 & 199,841 & 8.5 & 9.5 & 31.3 & 32.8\\\\\n\t\t\t\\hline\n\t\t\\end{tabular}\n\t\\end{center}\n\n\t\\label{table4}\n\\end{table}\n\n\\FloatBarrier\n\n3.1 Ca-Si Problem\n\\subsection{Ca-Si Problem}\nFor this first cement system, the emulation problem consists of predicting at each time step of the RT simulation the (output) Ca, Si, H and O aqueous concentrations (mol/kg of water or mol/kgw) from the (input) total amounts of Ca and Si (mol). \n\n\\subsubsection{Training the emulators}\n\\label{train_res1}\nHere the kNN and DNN emulators are firstly trained using a set of 400,000 test examples for both. This training set is obtained by randomly sampling the two-dimensional input space by latin hypercube sampling (LHS) between $\\left[0,0\\right]$ and $\\rm{\\left[Ca^{tot}_{max},Si^{tot}_{max}\\right]}$, and running PHREEQC for each input sample, $\\rm{\\textbf{x}_i = \\left[Ca^{tot}_i, Si^{tot}_i\\right]}$ to get the corresponding output vectors, $\\rm{\\textbf{y}_i = \\left[Ca^{conc}_i,Si^{conc}_i,H^{conc}_i,O^{conc}_i\\right]}$. The upper bounds, $\\rm{Ca^{tot}_{max}}$ and $\\rm{Si^{tot}_{max}}$ are defined based on a cheap full RT simulation with advective-dispersive transport using a small 1D domain of 51 nodes. It is worth noting that the total amounts of $\\textbf{x}_i$, corresponding to the PHREEQC-simulated concentrations, $\\textbf{y}_i$, have to be corrected for the different amount of water between the training set and the transport simulations. Doing so, it turns out that about 20 \\% of the post-corrected $x_i$ values exceed their pre-defined upper bounds and these excessively large values need to be filtered out. Creating the 400,000 training examples thus required about 500,000 PHREEQC runs. As stated earlier, for this problem single-threaded PHREEQC performs about 670 geochemical calculations per second on our used Intel\\textsuperscript{\\textregistered} i7 CPU.\n\nWith respect to kNN, the tuning parameters are the number of neighbors, $k$, the type of distance measure, and the interpolation technique. We simply used the default settings: $k = 5$, euclidean distance and inverse-distance interpolation. Regarding training of the DNN, the 400,000 sample were split between the training set itself (90 \\% of the data) and a validation set (10 \\% of the data). The latter serves to monitor the evolution of the selected mean squared error loss function on samples that are not used for training, thereby detecting potential overfitting. If the validation loss stops decreasing before the fixed number of epochs has been completed, then  training is stopped. Importantly, the emulation is achieved in log-space for both the input, $X$ and output $Y$, domains. This because total amounts and concentrations of the involved components typically cover many orders of magnitudes (up to 10 orders or more). Using a DNN also requires some form of data normalization or standardization. Here both the $\\rm{log\\left(\\textbf{x}_i\\right)}$ and $\\rm{log\\left(\\textbf{y}_i\\right)}$ vectors are standardized around 0 with standard deviation of 1.\n\nFigure \\ref{fig1} illustrates the trained emulators' performance for geochemical predictions using an independent test set that comprises 10,000 test examples. Both kNN (Figures \\ref{fig1}a - d) and DNN (Figures \\ref{fig1}e - h) appear to be rather accurate . DNN also shows a slight degradation for the larger concentration values  (Figures \\ref{fig1}e - h). The latter is likely due to the combination of a small proportion of large concentration values in the training set with the log-transformation that implicitly pushes the DNN to try harder to fit the smaller concentrations during training. Regarding speedup and as written earlier, for this setup the single-threaded kNN method is 300 times faster than single-threaded PHREEQC for predicting the 10,000 concentration vectors all at once. The computational savings allowed by the DNN emulator when ran on our GPU is higher with a speedup as large as 4000 for predicting the same 10,000 concentration vectors all at once. \n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{0cm}\\includegraphics[width=35pc]{Figure1.png}\n\t\\caption{1-1 plots of the kNN (subfigures (a) - (d)) and DNN (subfigures (e) - (h)) emulators' performance obtained for system 1 when the kNN training base contains 400,000 samples and the DNN is trained using the same 400,000 samples. Here ``true\" means the original PHREEQC-simulated data and ``predicted\" denotes the emulated (that is, kNN-simulated and DNN-simulated) data. Hence, the $x$-axis and $y$-axis present the original and emulated 10,000 independent test data points, respectively. The RMSE and $Q_2$ coefficient denote the root-mean-square-error and coefficient of determination in testing mode, respectively, between the original and emulated 10,000 test data points.}\n\t\\label{fig1}\n\\end{figure}\n\nTo test the sensitivity of the emulators' performance to the training set size, training was also performed using reduced training sets comprising 100,000, 10,000, 1000 and 100 samples, respectively. It seen that the DNN performance achieved  when using 10,000 training samples is virtually the same as that obtained when using 400,000 training samples (Figure \\ref{fig2} and Table \\ref{table2}). It is only for training sets smaller than 1000 samples that the DNN performance starts to degrade significantly (Table \\ref{table2}). In contrast, the behavior of kNN appears to markedly decrease as the training set gets smaller (Figure \\ref{fig2} and Table \\ref{table1}).\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{0cm}\\includegraphics[width=35pc]{Figure2.png}\n\t\\caption{1-1 plots of the kNN (subfigures (a) - (d)) and DNN (subfigures (e) - (h)) emulators' performance obtained for system 1 when the kNN training base contains 10,000 samples and the DNN is trained using the same 10,000 samples. Here ``true\" means the original PHREEQC-simulated data and ``predicted\" denotes the emulated (that is, kNN-simulated and DNN-simulated) data. Hence, the $x$-axis and $y$-axis present the original and emulated 10,000 independent test data points, respectively. The RMSE and $Q_2$ coefficient denote the root-mean-square-error and coefficient of determination in testing mode, respectively, between the original and emulated 10,000 test data points.}\n\t\\label{fig2}\n\\end{figure}\n\n\\begin{table}[h!]\n\t\\caption{Performance of the DNN and kNN emulators for cement system 1 and different training set sizes. For brevity, only the results for Ca$^{conc}$ and Si$^{conc}$ are shown. The units are mol per kg of water (mol/kgw). ML refers to the type of emulator, TR signifies the size of the training set, and the RMSE and $Q_2$ coefficient denote the root-mean-square-error and coefficient of determination in testing mode, respectively, between the original and emulated 10,000 test data points.}\n\t\\begin{center}\n\t\t\\begin{tabular}{cccccccc}\n\t\t\t\\hline\n\t\t\t& & & & & \\\\\n\t\t\tML & TR & RMSE - Ca$^{conc}$ & $Q_2$ - Ca$^{conc}$ & RMSE - Si$^{conc}$ & $Q_2$ - Si$^{conc}$\\\\\n\t\t\tDNN & 4 $\\times$ 10$^{5}$ & 4.50 $\\times$ 10$^{-5}$ & 0.9999 & 1.21 $\\times$ 10$^{-6}$ & 0.9999\\\\\n\t\t\tDNN & 1 $\\times$ 10$^{5}$ & 5.06 $\\times$ 10$^{-5}$ & 0.9999 & 1.30 $\\times$ 10$^{-6}$ & 0.9999\\\\\n\t\t\tDNN & 1 $\\times$ 10$^{4}$ & 4.57 $\\times$ 10$^{-5}$ & 0.9999 & 1.48 $\\times$ 10$^{-6}$ & 0.9999\\\\\n\t\t\tDNN & 1 $\\times$ 10$^{3}$ & 8.02 $\\times$ 10$^{-5}$ & 0.9998 & 3.31 $\\times$ 10$^{-6}$ & 0.9994\\\\\n\t\t\tDNN & 1 $\\times$ 10$^{2}$ & 47.1 $\\times$ 10$^{-5}$ & 0.9935 & 9.02 $\\times$ 10$^{-6}$ & 0.9959\\\\\n\t\t\tkNN & 4 $\\times$ 10$^{5}$ & 2.52 $\\times$ 10$^{-5}$ & 1.0000 & 1.59 $\\times$ 10$^{-6}$ & 0.9999\\\\\n\t\t\tkNN & 1 $\\times$ 10$^{5}$ & 6.27 $\\times$ 10$^{-5}$ & 0.9999 & 4.20 $\\times$ 10$^{-6}$ & 0.9991\\\\\n\t\t\tkNN & 1 $\\times$ 10$^{4}$ & 14.1 $\\times$ 10$^{-5}$ & 0.9994 & 13.5 $\\times$ 10$^{-6}$ & 0.9908\\\\\n\t\t\tkNN & 1 $\\times$ 10$^{3}$ & 43.9 $\\times$ 10$^{-5}$ & 0.9944 & 74.6 $\\times$ 10$^{-6}$ & 0.7180\\\\\n\t\t\tkNN & 1 $\\times$ 10$^{2}$  & 96.4 $\\times$ 10$^{-5}$ & 0.9729 & 55.0 $\\times$ 10$^{-6}$ & 0.8468\\\\\n\t\t\\end{tabular}\n\t\\end{center}\n\t\\label{table2}\n\\end{table}\n\n\\FloatBarrier\n\n\\subsubsection{Reactive transport simulation}\n\\label{prob12Dres}\n\nThis section focuses on reactive transport simulations with HPx$_{\\rm py}$ within cement system 1, under both advective-dispersive and diffusive transport conditions. As written above, the domain sizes are both 61 $\\times$ 61 and 121 $\\times$ 121 for the advection-dispersion case and, because of computational constraints, solely 61 $\\times$ 61  for the diffusion case. In addition, the simulation time period is 2 years for the advection-dispersion case and 1 year for the diffusion case. Figures \\ref{fig3} and \\ref{fig4} present times series of original and emulated Ca, Si, H and O concentrations at 5 locations within the 2D domain for advective-dispersive transport conditions, for both our kNN-based (HPx$_{\\rm py}$-kNN) and DNN-based (HPx$_{\\rm py}$-DNN) reactive transport codes. It is seen that HPx$_{\\rm py}$-kNN and HPx$_{\\rm py}$-DNN both induce a quite good simulation accuracy. Also, the results for the diffusive transport case are of similarly good quality (not shown). Figures \\ref{fig5} - \\ref{fig6} provide more insights into the HPx$_{\\rm py}$-kNN and HPx$_{\\rm py}$-DNN performances by displaying 2D Ca, Si, H and O concentration profiles at a given time. For each experiment and chemical component, this time is selected as to be well representative of the simulated dynamics. It is observed that the original and emulated images are visually almost indistinguishable for the advection-dispersion case (Figure \\ref{fig5}). For the diffusion case, the emulators also perform quite well for Ca, H and O (Figures\\ref{fig6}a - c, g - i and j - l), while some slight to moderate discrepancies appear at the concentration front for Si (Figures \\ref{fig6}d - f). Notwithstanding, the Si concentration remains globally well predicted. Furthermore, Figures \\ref{fig7} - \\ref{fig8} present the original and emulated 2D solid amount profiles corresponding to Figures \\ref{fig5} - \\ref{fig6}. The original solid amount profiles are overall well approximated by HPx$_{\\rm py}$-kNN and HPx$_{\\rm py}$-DNN for the advection-dispersion case (Figure \\ref{fig7}), even though some mismatch appears at the border of the fully depleted zone for the H component. As of the diffusion case (Figure \\ref{fig8}), the same kind of mismatch is observed for the emulated solid amounts of H by HPx$_{\\rm py}$-kNN while the emulated profiles by HPx$_{\\rm py}$-DNN show some more discrepancies. Though we decided to present raw emulation results, we would like to stress that some if not all of the observed artifacts could likely be smoothed out by using some post-filtering such as median filtering.\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure3.png}\n\t\\caption{Time series of original (RTM, solid green lines) and HPx$_{\\rm py}$-kNN emulated (TM+kNN, dashed orange lines) concentrations (mol/kg) of Ca, H, O and Si at selected observations points for cement system 1 and advective-dispersive transport. Obs. 1 - 5 denote the selected observation points, with the following $\\left[x,y\\right]$ locations (in cm). Obs. 1:  $\\left[0.5, 2.5\\right]$, Obs. 2: $\\left[1, 2\\right]$, Obs. 3: $\\left[2, 2\\right]$, Obs. 4: $\\left[1, 1\\right]$, Obs. 5: $\\left[2, 1\\right]$. The results for the 121 $\\times$ 121 grid size are rather similar.}\n\t\\label{fig3}\n\\end{figure}\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure4.png}\n\t\\caption{Time series of original (RTM, solid green lines) and HPx$_{\\rm py}$-DNN emulated (TM+DNN, dashed orange lines) concentrations (mol/kg) of Ca, H, O and Si at selected observations points for cement system 1 and advective-dispersive transport. Obs. 1 - 5 denote the selected observation points, with the following $\\left[x,y\\right]$ locations (in cm). Obs. 1:  $\\left[0.5, 2.5\\right]$, Obs. 2: $\\left[1, 2\\right]$, Obs. 3: $\\left[2, 2\\right]$, Obs. 4: $\\left[1, 1\\right]$, Obs. 5: $\\left[2, 1\\right]$. The results for the 121 $\\times$ 121 grid size are rather similar.}\n\t\\label{fig4}\n\\end{figure}\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure5.png}\n\t\\caption{2D concentration profiles obtained for cement system 1 at the end of the 2-year simulation performed for the advection-dispersion case. RTM means the original HP$_{\\rm{4C}}$ model, TM+kNN denotes the Hydrus transport model coupled with our kNN geochemical emulator (HPx$_{\\rm py}$-kNN) and TM+DNN signifies the Hydrus transport model coupled with our DNN geochemical emulator (HPx$_{\\rm py}$-DNN). The first to fourth row present profiles for $Ca^{\\rm{conc}}$, $Si^{\\rm{conc}}$, $H^{\\rm{conc}}$, and $O^{\\rm{conc}}$, respectively. The considered grid size is 61 $\\times$ 61. The results for the 121 $\\times$ 121 grid are rather similar.}\n\t\\label{fig5}\n\\end{figure}\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure6.png}\n\t\\caption{2D concentration profiles obtained for cement system 1 at the final time step of the 1-year simulation performed for the diffusion case. RTM means the original HP$_{\\rm{4C}}$ model, TM+kNN denotes the Hydrus transport model coupled with our kNN geochemical emulator (HPx$_{\\rm py}$-kNN) and TM+DNN signifies the Hydrus transport model coupled with our DNN geochemical emulator (HPx$_{\\rm py}$-DNN). The first to fourth row present profiles for $Ca^{\\rm{conc}}$, $Si^{\\rm{conc}}$, $H^{\\rm{conc}}$, and $O^{\\rm{conc}}$, respectively. The considered grid size is 61 $\\times$ 61.}\n\t\\label{fig6}\n\\end{figure}\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure7.png}\n\t\\caption{2D solid amount profiles obtained for cement system 1 at the end of the 2-year simulation performed for the advection-dispersion case. RTM means the original HP$_{\\rm{4C}}$ model, TM+kNN denotes the Hydrus transport model coupled with our kNN geochemical emulator (HPx$_{\\rm py}$-kNN) and TM+DNN signifies the Hydrus transport model coupled with our DNN geochemical emulator (HPx$_{\\rm py}$-DNN). The first to fourth row present profiles for $Ca^{\\rm{solid}}$, $Si^{\\rm{solid}}$, $H^{\\rm{solid}}$, and $O^{\\rm{solid}}$, respectively. The considered grid size is 61 $\\times$ 61. The results for the 121 $\\times$ 121 grid are rather similar.}\n\t\\label{fig7}\n\\end{figure}\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure8.png}\n\t\\caption{2D solid amount profiles obtained for cement system 1 at the final time step of the 1-year simulation performed for the diffusion case. RTM means the original HP$_{\\rm{4C}}$ model, TM+kNN denotes the Hydrus transport model coupled with our kNN geochemical emulator (HPx$_{\\rm py}$-kNN) and TM+DNN signifies the Hydrus transport model coupled with our DNN geochemical emulator (HPx$_{\\rm py}$-DNN). The first to fourth row present profiles for $Ca^{\\rm{solid}}$, $Si^{\\rm{solid}}$, $H^{\\rm{solid}}$, and $O^{\\rm{solid}}$, respectively. The considered grid size is 61 $\\times$ 61.}\n\t\\label{fig8}\n\\end{figure}\n\nThe speedups associated with the considered problem are detailed in Table \\ref{table3}. It is noted that the GPU-based DNN emulator allows for a speedup that is close to optimal. Indeed, The DNN speedups overall represent 85 \\% to 95 \\% of the maximum possible speedups (that is, speedups that would be obtained if the geochemcical calculations would come at no cost at all). The speedups associated with single-threaded kNN remain substantial but only amount to 57 \\% - 65 \\% of the corresponding maximum speedups. As detailed in section \\ref{train_res1}, the used kNN and DNN implementations are found to be respectively 300 and 4000 faster than single-threaded PHREEQC when predicting 10,000 points all one once for this geochemical system. Based on these numbers one could have expected the achieved speedups to represent say 90 \\% (kNN) or 99 \\% (DNN) of the maximum possible ones. A large part of the gaps between achieved and maximum possible speedups is thus likely caused by the time required for communicating and exchanging data between the main C/C++ code and the Python-based emulators.\n\n\\begin{table}[h!]\n\t\\caption{Speedups offered by the KNN and DNN emulators in HPx$_{\\rm py}$ for the reactive transport simulations considered for cement system 1. The HPx$_{\\rm{4C}}$ calculations involve the parallelization of PHREEQC over our 4 CPUs. The HPx$_{\\rm{1C}}$ calculations are performed on a single CPU. The kNN predictions are performed on a single CPU using the scikit-learn implementation while the DNN predictions make use of our GPU. ML signifies the used machine learning method for emulation, TC denotes transport conditions (ADV: advection-dispersion, DIF: diffusion) and GS is the grid size. The maximum possible speedups associated with HPx$_{\\rm{4C}}$ and HPx$_{\\rm{1C}}$,  Max SP HPx$_{\\rm{4C}}$ and  Max SP HPx$_{\\rm{1C}}$, correspond to an hypothetical situation where the geochemical calculations incur zero computational cost.}\n\n\t\\begin{center}\n\t\t\\begin{tabular}{cccccccc}\n\t\t\t\\hline\n\t\t\t& & & & & & & \\\\\n\t\t\tML & TC & GS & HPx$_{\\rm{4C}}$ time (s) & SP HPx$_{\\rm{4C}}$ & Max SP HPx$_{\\rm{4C}}$ & SP HPx$_{\\rm{1C}}$ & Max SP HPx$_{\\rm{1C}}$ \\\\\n\t\t\tDNN & ADV & 61 $\\times$ 61 & 3189 & 6.8 & 7.7 & 24.5 & 28.5 \\\\\n\t\t\tkNN & ADV & 61 $\\times$ 61 & 3189 & 5.0 & 7.7 & 18.0 & 28.5 \\\\\n\t\t\tDNN & ADV & 121 $\\times$ 121 & 23,337 & 5.0 & 5.2 & 17.0 & 18.0 \\\\\n\t\t\tkNN & ADV & 121 $\\times$ 121 & 23,337 & 3.4 & 5.2 & 11.6 & 18.0 \\\\\n\t\t\tDNN & DIF & 61 $\\times$ 61 & 25,448 & 4.2 & 4.9 & 16.2 & 19.1 \\\\\n\t\t\tkNN & DIF & 61 $\\times$ 61 & 25,448 & 2.8 & 4.9 & 10.8 & 19.1 \\\\\n\t\t\t\\hline\n\t\t\\end{tabular}\n\t\\end{center}\n\n\t\\label{table3}\n\\end{table}\n\n\\FloatBarrier\n\n3.1.2 Reactive transport simulation\n\\subsubsection{Reactive transport simulation}\n\\label{prob12Dres}\n\nThis section focuses on reactive transport simulations with HPx$_{\\rm py}$ within cement system 1, under both advective-dispersive and diffusive transport conditions. As written above, the domain sizes are both 61 $\\times$ 61 and 121 $\\times$ 121 for the advection-dispersion case and, because of computational constraints, solely 61 $\\times$ 61  for the diffusion case. In addition, the simulation time period is 2 years for the advection-dispersion case and 1 year for the diffusion case. Figures \\ref{fig3} and \\ref{fig4} present times series of original and emulated Ca, Si, H and O concentrations at 5 locations within the 2D domain for advective-dispersive transport conditions, for both our kNN-based (HPx$_{\\rm py}$-kNN) and DNN-based (HPx$_{\\rm py}$-DNN) reactive transport codes. It is seen that HPx$_{\\rm py}$-kNN and HPx$_{\\rm py}$-DNN both induce a quite good simulation accuracy. Also, the results for the diffusive transport case are of similarly good quality (not shown). Figures \\ref{fig5} - \\ref{fig6} provide more insights into the HPx$_{\\rm py}$-kNN and HPx$_{\\rm py}$-DNN performances by displaying 2D Ca, Si, H and O concentration profiles at a given time. For each experiment and chemical component, this time is selected as to be well representative of the simulated dynamics. It is observed that the original and emulated images are visually almost indistinguishable for the advection-dispersion case (Figure \\ref{fig5}). For the diffusion case, the emulators also perform quite well for Ca, H and O (Figures\\ref{fig6}a - c, g - i and j - l), while some slight to moderate discrepancies appear at the concentration front for Si (Figures \\ref{fig6}d - f). Notwithstanding, the Si concentration remains globally well predicted. Furthermore, Figures \\ref{fig7} - \\ref{fig8} present the original and emulated 2D solid amount profiles corresponding to Figures \\ref{fig5} - \\ref{fig6}. The original solid amount profiles are overall well approximated by HPx$_{\\rm py}$-kNN and HPx$_{\\rm py}$-DNN for the advection-dispersion case (Figure \\ref{fig7}), even though some mismatch appears at the border of the fully depleted zone for the H component. As of the diffusion case (Figure \\ref{fig8}), the same kind of mismatch is observed for the emulated solid amounts of H by HPx$_{\\rm py}$-kNN while the emulated profiles by HPx$_{\\rm py}$-DNN show some more discrepancies. Though we decided to present raw emulation results, we would like to stress that some if not all of the observed artifacts could likely be smoothed out by using some post-filtering such as median filtering.\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure3.png}\n\t\\caption{Time series of original (RTM, solid green lines) and HPx$_{\\rm py}$-kNN emulated (TM+kNN, dashed orange lines) concentrations (mol/kg) of Ca, H, O and Si at selected observations points for cement system 1 and advective-dispersive transport. Obs. 1 - 5 denote the selected observation points, with the following $\\left[x,y\\right]$ locations (in cm). Obs. 1:  $\\left[0.5, 2.5\\right]$, Obs. 2: $\\left[1, 2\\right]$, Obs. 3: $\\left[2, 2\\right]$, Obs. 4: $\\left[1, 1\\right]$, Obs. 5: $\\left[2, 1\\right]$. The results for the 121 $\\times$ 121 grid size are rather similar.}\n\t\\label{fig3}\n\\end{figure}\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure4.png}\n\t\\caption{Time series of original (RTM, solid green lines) and HPx$_{\\rm py}$-DNN emulated (TM+DNN, dashed orange lines) concentrations (mol/kg) of Ca, H, O and Si at selected observations points for cement system 1 and advective-dispersive transport. Obs. 1 - 5 denote the selected observation points, with the following $\\left[x,y\\right]$ locations (in cm). Obs. 1:  $\\left[0.5, 2.5\\right]$, Obs. 2: $\\left[1, 2\\right]$, Obs. 3: $\\left[2, 2\\right]$, Obs. 4: $\\left[1, 1\\right]$, Obs. 5: $\\left[2, 1\\right]$. The results for the 121 $\\times$ 121 grid size are rather similar.}\n\t\\label{fig4}\n\\end{figure}\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure5.png}\n\t\\caption{2D concentration profiles obtained for cement system 1 at the end of the 2-year simulation performed for the advection-dispersion case. RTM means the original HP$_{\\rm{4C}}$ model, TM+kNN denotes the Hydrus transport model coupled with our kNN geochemical emulator (HPx$_{\\rm py}$-kNN) and TM+DNN signifies the Hydrus transport model coupled with our DNN geochemical emulator (HPx$_{\\rm py}$-DNN). The first to fourth row present profiles for $Ca^{\\rm{conc}}$, $Si^{\\rm{conc}}$, $H^{\\rm{conc}}$, and $O^{\\rm{conc}}$, respectively. The considered grid size is 61 $\\times$ 61. The results for the 121 $\\times$ 121 grid are rather similar.}\n\t\\label{fig5}\n\\end{figure}\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure6.png}\n\t\\caption{2D concentration profiles obtained for cement system 1 at the final time step of the 1-year simulation performed for the diffusion case. RTM means the original HP$_{\\rm{4C}}$ model, TM+kNN denotes the Hydrus transport model coupled with our kNN geochemical emulator (HPx$_{\\rm py}$-kNN) and TM+DNN signifies the Hydrus transport model coupled with our DNN geochemical emulator (HPx$_{\\rm py}$-DNN). The first to fourth row present profiles for $Ca^{\\rm{conc}}$, $Si^{\\rm{conc}}$, $H^{\\rm{conc}}$, and $O^{\\rm{conc}}$, respectively. The considered grid size is 61 $\\times$ 61.}\n\t\\label{fig6}\n\\end{figure}\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure7.png}\n\t\\caption{2D solid amount profiles obtained for cement system 1 at the end of the 2-year simulation performed for the advection-dispersion case. RTM means the original HP$_{\\rm{4C}}$ model, TM+kNN denotes the Hydrus transport model coupled with our kNN geochemical emulator (HPx$_{\\rm py}$-kNN) and TM+DNN signifies the Hydrus transport model coupled with our DNN geochemical emulator (HPx$_{\\rm py}$-DNN). The first to fourth row present profiles for $Ca^{\\rm{solid}}$, $Si^{\\rm{solid}}$, $H^{\\rm{solid}}$, and $O^{\\rm{solid}}$, respectively. The considered grid size is 61 $\\times$ 61. The results for the 121 $\\times$ 121 grid are rather similar.}\n\t\\label{fig7}\n\\end{figure}\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure8.png}\n\t\\caption{2D solid amount profiles obtained for cement system 1 at the final time step of the 1-year simulation performed for the diffusion case. RTM means the original HP$_{\\rm{4C}}$ model, TM+kNN denotes the Hydrus transport model coupled with our kNN geochemical emulator (HPx$_{\\rm py}$-kNN) and TM+DNN signifies the Hydrus transport model coupled with our DNN geochemical emulator (HPx$_{\\rm py}$-DNN). The first to fourth row present profiles for $Ca^{\\rm{solid}}$, $Si^{\\rm{solid}}$, $H^{\\rm{solid}}$, and $O^{\\rm{solid}}$, respectively. The considered grid size is 61 $\\times$ 61.}\n\t\\label{fig8}\n\\end{figure}\n\nThe speedups associated with the considered problem are detailed in Table \\ref{table3}. It is noted that the GPU-based DNN emulator allows for a speedup that is close to optimal. Indeed, The DNN speedups overall represent 85 \\% to 95 \\% of the maximum possible speedups (that is, speedups that would be obtained if the geochemcical calculations would come at no cost at all). The speedups associated with single-threaded kNN remain substantial but only amount to 57 \\% - 65 \\% of the corresponding maximum speedups. As detailed in section \\ref{train_res1}, the used kNN and DNN implementations are found to be respectively 300 and 4000 faster than single-threaded PHREEQC when predicting 10,000 points all one once for this geochemical system. Based on these numbers one could have expected the achieved speedups to represent say 90 \\% (kNN) or 99 \\% (DNN) of the maximum possible ones. A large part of the gaps between achieved and maximum possible speedups is thus likely caused by the time required for communicating and exchanging data between the main C/C++ code and the Python-based emulators.\n\n\\begin{table}[h!]\n\t\\caption{Speedups offered by the KNN and DNN emulators in HPx$_{\\rm py}$ for the reactive transport simulations considered for cement system 1. The HPx$_{\\rm{4C}}$ calculations involve the parallelization of PHREEQC over our 4 CPUs. The HPx$_{\\rm{1C}}$ calculations are performed on a single CPU. The kNN predictions are performed on a single CPU using the scikit-learn implementation while the DNN predictions make use of our GPU. ML signifies the used machine learning method for emulation, TC denotes transport conditions (ADV: advection-dispersion, DIF: diffusion) and GS is the grid size. The maximum possible speedups associated with HPx$_{\\rm{4C}}$ and HPx$_{\\rm{1C}}$,  Max SP HPx$_{\\rm{4C}}$ and  Max SP HPx$_{\\rm{1C}}$, correspond to an hypothetical situation where the geochemical calculations incur zero computational cost.}\n\n\t\\begin{center}\n\t\t\\begin{tabular}{cccccccc}\n\t\t\t\\hline\n\t\t\t& & & & & & & \\\\\n\t\t\tML & TC & GS & HPx$_{\\rm{4C}}$ time (s) & SP HPx$_{\\rm{4C}}$ & Max SP HPx$_{\\rm{4C}}$ & SP HPx$_{\\rm{1C}}$ & Max SP HPx$_{\\rm{1C}}$ \\\\\n\t\t\tDNN & ADV & 61 $\\times$ 61 & 3189 & 6.8 & 7.7 & 24.5 & 28.5 \\\\\n\t\t\tkNN & ADV & 61 $\\times$ 61 & 3189 & 5.0 & 7.7 & 18.0 & 28.5 \\\\\n\t\t\tDNN & ADV & 121 $\\times$ 121 & 23,337 & 5.0 & 5.2 & 17.0 & 18.0 \\\\\n\t\t\tkNN & ADV & 121 $\\times$ 121 & 23,337 & 3.4 & 5.2 & 11.6 & 18.0 \\\\\n\t\t\tDNN & DIF & 61 $\\times$ 61 & 25,448 & 4.2 & 4.9 & 16.2 & 19.1 \\\\\n\t\t\tkNN & DIF & 61 $\\times$ 61 & 25,448 & 2.8 & 4.9 & 10.8 & 19.1 \\\\\n\t\t\t\\hline\n\t\t\\end{tabular}\n\t\\end{center}\n\n\t\\label{table3}\n\\end{table}\n\n\\FloatBarrier\n\n",
        "label": "fig4",
        "Descriptive_question1": "What color represents the original RTM concentrations in figure_4?",
        "Descriptive_question2": "Which observation point in figure_4 is located at [0.5, 2.5] cm?",
        "Reasoning_question1": "How do the HPx_py-DNN emulated concentrations compare to the original RTM concentrations across different observation points in figure_4, and what might explain any discrepancies observed?",
        "Reasoning_question2": "Based on the time series data in figure_4, what trends can be inferred about the concentration changes of Ca, H, O, and Si over time at various observation points, and how do these trends differ between the original and emulated data?",
        "Descriptive_answer1": "solid green",
        "Descriptive_answer2": "Obs. 1",
        "Reasoning_answer1": "The HPx_py-DNN emulated concentrations, represented by dashed orange lines, closely follow the original RTM concentrations shown as solid green lines across all observation points in figure_4. This indicates that the DNN emulator effectively captures the dynamic changes and magnitudes of Ca, H, O, and Si concentrations in the cement system under advective-dispersive transport. Any minor discrepancies observed may be attributed to the inherent differences in emulation versus full reactive transport modeling, potential smoothing effects in machine learning predictions, or localized complexities in geochemical interactions that challenge the emulator's accuracy at certain points or times.",
        "Reasoning_answer2": "The time series data in figure_4 show that concentrations of Ca, H, O, and Si vary over time at the different observation points in the 2D domain. Generally, concentrations either increase or decrease gradually, reflecting chemical transport and reactions. The emulated data (dashed orange lines) track these temporal trends very similarly to the original RTM data (solid green lines), indicating good emulation fidelity. However, subtle differences in the amplitude or timing of peaks and troughs may occur, reflecting slight deviations in the emulator's precision, especially during rapid concentration changes or at boundary points with complex transport behavior."
    },
    {
        "paper_id": "2107.07598.json",
        "image_id": "figure_5",
        "image_path": [
            "/home/yz979/scratch/chengye/scimolmo/test-images-tables/dataset_imagestables/2107.07598/images/Figure5.png"
        ],
        "caption": "2D concentration profiles obtained for cement system 1 at the end of the 2-year simulation performed for the advection-dispersion case. RTM means the original HP$_{\\rm{4C}}$ model, TM+kNN denotes the Hydrus transport model coupled with our kNN geochemical emulator (HPx$_{\\rm py}$-kNN) and TM+DNN signifies the Hydrus transport model coupled with our DNN geochemical emulator (HPx$_{\\rm py}$-DNN). The first to fourth row present profiles for $Ca^{\\rm{conc}}$, $Si^{\\rm{conc}}$, $H^{\\rm{conc}}$, and $O^{\\rm{conc}}$, respectively. The considered grid size is 61 $\\times$ 61. The results for the 121 $\\times$ 121 grid are rather similar.",
        "classify": "Chart",
        "section_info": "3 Results\n\\section{Results}\n\\label{results}\n\n\\subsection{Ca-Si Problem}\nFor this first cement system, the emulation problem consists of predicting at each time step of the RT simulation the (output) Ca, Si, H and O aqueous concentrations (mol/kg of water or mol/kgw) from the (input) total amounts of Ca and Si (mol). \n\n\\subsubsection{Training the emulators}\n\\label{train_res1}\nHere the kNN and DNN emulators are firstly trained using a set of 400,000 test examples for both. This training set is obtained by randomly sampling the two-dimensional input space by latin hypercube sampling (LHS) between $\\left[0,0\\right]$ and $\\rm{\\left[Ca^{tot}_{max},Si^{tot}_{max}\\right]}$, and running PHREEQC for each input sample, $\\rm{\\textbf{x}_i = \\left[Ca^{tot}_i, Si^{tot}_i\\right]}$ to get the corresponding output vectors, $\\rm{\\textbf{y}_i = \\left[Ca^{conc}_i,Si^{conc}_i,H^{conc}_i,O^{conc}_i\\right]}$. The upper bounds, $\\rm{Ca^{tot}_{max}}$ and $\\rm{Si^{tot}_{max}}$ are defined based on a cheap full RT simulation with advective-dispersive transport using a small 1D domain of 51 nodes. It is worth noting that the total amounts of $\\textbf{x}_i$, corresponding to the PHREEQC-simulated concentrations, $\\textbf{y}_i$, have to be corrected for the different amount of water between the training set and the transport simulations. Doing so, it turns out that about 20 \\% of the post-corrected $x_i$ values exceed their pre-defined upper bounds and these excessively large values need to be filtered out. Creating the 400,000 training examples thus required about 500,000 PHREEQC runs. As stated earlier, for this problem single-threaded PHREEQC performs about 670 geochemical calculations per second on our used Intel\\textsuperscript{\\textregistered} i7 CPU.\n\nWith respect to kNN, the tuning parameters are the number of neighbors, $k$, the type of distance measure, and the interpolation technique. We simply used the default settings: $k = 5$, euclidean distance and inverse-distance interpolation. Regarding training of the DNN, the 400,000 sample were split between the training set itself (90 \\% of the data) and a validation set (10 \\% of the data). The latter serves to monitor the evolution of the selected mean squared error loss function on samples that are not used for training, thereby detecting potential overfitting. If the validation loss stops decreasing before the fixed number of epochs has been completed, then  training is stopped. Importantly, the emulation is achieved in log-space for both the input, $X$ and output $Y$, domains. This because total amounts and concentrations of the involved components typically cover many orders of magnitudes (up to 10 orders or more). Using a DNN also requires some form of data normalization or standardization. Here both the $\\rm{log\\left(\\textbf{x}_i\\right)}$ and $\\rm{log\\left(\\textbf{y}_i\\right)}$ vectors are standardized around 0 with standard deviation of 1.\n\nFigure \\ref{fig1} illustrates the trained emulators' performance for geochemical predictions using an independent test set that comprises 10,000 test examples. Both kNN (Figures \\ref{fig1}a - d) and DNN (Figures \\ref{fig1}e - h) appear to be rather accurate . DNN also shows a slight degradation for the larger concentration values  (Figures \\ref{fig1}e - h). The latter is likely due to the combination of a small proportion of large concentration values in the training set with the log-transformation that implicitly pushes the DNN to try harder to fit the smaller concentrations during training. Regarding speedup and as written earlier, for this setup the single-threaded kNN method is 300 times faster than single-threaded PHREEQC for predicting the 10,000 concentration vectors all at once. The computational savings allowed by the DNN emulator when ran on our GPU is higher with a speedup as large as 4000 for predicting the same 10,000 concentration vectors all at once. \n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{0cm}\\includegraphics[width=35pc]{Figure1.png}\n\t\\caption{1-1 plots of the kNN (subfigures (a) - (d)) and DNN (subfigures (e) - (h)) emulators' performance obtained for system 1 when the kNN training base contains 400,000 samples and the DNN is trained using the same 400,000 samples. Here ``true\" means the original PHREEQC-simulated data and ``predicted\" denotes the emulated (that is, kNN-simulated and DNN-simulated) data. Hence, the $x$-axis and $y$-axis present the original and emulated 10,000 independent test data points, respectively. The RMSE and $Q_2$ coefficient denote the root-mean-square-error and coefficient of determination in testing mode, respectively, between the original and emulated 10,000 test data points.}\n\t\\label{fig1}\n\\end{figure}\n\nTo test the sensitivity of the emulators' performance to the training set size, training was also performed using reduced training sets comprising 100,000, 10,000, 1000 and 100 samples, respectively. It seen that the DNN performance achieved  when using 10,000 training samples is virtually the same as that obtained when using 400,000 training samples (Figure \\ref{fig2} and Table \\ref{table2}). It is only for training sets smaller than 1000 samples that the DNN performance starts to degrade significantly (Table \\ref{table2}). In contrast, the behavior of kNN appears to markedly decrease as the training set gets smaller (Figure \\ref{fig2} and Table \\ref{table1}).\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{0cm}\\includegraphics[width=35pc]{Figure2.png}\n\t\\caption{1-1 plots of the kNN (subfigures (a) - (d)) and DNN (subfigures (e) - (h)) emulators' performance obtained for system 1 when the kNN training base contains 10,000 samples and the DNN is trained using the same 10,000 samples. Here ``true\" means the original PHREEQC-simulated data and ``predicted\" denotes the emulated (that is, kNN-simulated and DNN-simulated) data. Hence, the $x$-axis and $y$-axis present the original and emulated 10,000 independent test data points, respectively. The RMSE and $Q_2$ coefficient denote the root-mean-square-error and coefficient of determination in testing mode, respectively, between the original and emulated 10,000 test data points.}\n\t\\label{fig2}\n\\end{figure}\n\n\\begin{table}[h!]\n\t\\caption{Performance of the DNN and kNN emulators for cement system 1 and different training set sizes. For brevity, only the results for Ca$^{conc}$ and Si$^{conc}$ are shown. The units are mol per kg of water (mol/kgw). ML refers to the type of emulator, TR signifies the size of the training set, and the RMSE and $Q_2$ coefficient denote the root-mean-square-error and coefficient of determination in testing mode, respectively, between the original and emulated 10,000 test data points.}\n\t\\begin{center}\n\t\t\\begin{tabular}{cccccccc}\n\t\t\t\\hline\n\t\t\t& & & & & \\\\\n\t\t\tML & TR & RMSE - Ca$^{conc}$ & $Q_2$ - Ca$^{conc}$ & RMSE - Si$^{conc}$ & $Q_2$ - Si$^{conc}$\\\\\n\t\t\tDNN & 4 $\\times$ 10$^{5}$ & 4.50 $\\times$ 10$^{-5}$ & 0.9999 & 1.21 $\\times$ 10$^{-6}$ & 0.9999\\\\\n\t\t\tDNN & 1 $\\times$ 10$^{5}$ & 5.06 $\\times$ 10$^{-5}$ & 0.9999 & 1.30 $\\times$ 10$^{-6}$ & 0.9999\\\\\n\t\t\tDNN & 1 $\\times$ 10$^{4}$ & 4.57 $\\times$ 10$^{-5}$ & 0.9999 & 1.48 $\\times$ 10$^{-6}$ & 0.9999\\\\\n\t\t\tDNN & 1 $\\times$ 10$^{3}$ & 8.02 $\\times$ 10$^{-5}$ & 0.9998 & 3.31 $\\times$ 10$^{-6}$ & 0.9994\\\\\n\t\t\tDNN & 1 $\\times$ 10$^{2}$ & 47.1 $\\times$ 10$^{-5}$ & 0.9935 & 9.02 $\\times$ 10$^{-6}$ & 0.9959\\\\\n\t\t\tkNN & 4 $\\times$ 10$^{5}$ & 2.52 $\\times$ 10$^{-5}$ & 1.0000 & 1.59 $\\times$ 10$^{-6}$ & 0.9999\\\\\n\t\t\tkNN & 1 $\\times$ 10$^{5}$ & 6.27 $\\times$ 10$^{-5}$ & 0.9999 & 4.20 $\\times$ 10$^{-6}$ & 0.9991\\\\\n\t\t\tkNN & 1 $\\times$ 10$^{4}$ & 14.1 $\\times$ 10$^{-5}$ & 0.9994 & 13.5 $\\times$ 10$^{-6}$ & 0.9908\\\\\n\t\t\tkNN & 1 $\\times$ 10$^{3}$ & 43.9 $\\times$ 10$^{-5}$ & 0.9944 & 74.6 $\\times$ 10$^{-6}$ & 0.7180\\\\\n\t\t\tkNN & 1 $\\times$ 10$^{2}$  & 96.4 $\\times$ 10$^{-5}$ & 0.9729 & 55.0 $\\times$ 10$^{-6}$ & 0.8468\\\\\n\t\t\\end{tabular}\n\t\\end{center}\n\t\\label{table2}\n\\end{table}\n\n\\FloatBarrier\n\n\\subsubsection{Reactive transport simulation}\n\\label{prob12Dres}\n\nThis section focuses on reactive transport simulations with HPx$_{\\rm py}$ within cement system 1, under both advective-dispersive and diffusive transport conditions. As written above, the domain sizes are both 61 $\\times$ 61 and 121 $\\times$ 121 for the advection-dispersion case and, because of computational constraints, solely 61 $\\times$ 61  for the diffusion case. In addition, the simulation time period is 2 years for the advection-dispersion case and 1 year for the diffusion case. Figures \\ref{fig3} and \\ref{fig4} present times series of original and emulated Ca, Si, H and O concentrations at 5 locations within the 2D domain for advective-dispersive transport conditions, for both our kNN-based (HPx$_{\\rm py}$-kNN) and DNN-based (HPx$_{\\rm py}$-DNN) reactive transport codes. It is seen that HPx$_{\\rm py}$-kNN and HPx$_{\\rm py}$-DNN both induce a quite good simulation accuracy. Also, the results for the diffusive transport case are of similarly good quality (not shown). Figures \\ref{fig5} - \\ref{fig6} provide more insights into the HPx$_{\\rm py}$-kNN and HPx$_{\\rm py}$-DNN performances by displaying 2D Ca, Si, H and O concentration profiles at a given time. For each experiment and chemical component, this time is selected as to be well representative of the simulated dynamics. It is observed that the original and emulated images are visually almost indistinguishable for the advection-dispersion case (Figure \\ref{fig5}). For the diffusion case, the emulators also perform quite well for Ca, H and O (Figures\\ref{fig6}a - c, g - i and j - l), while some slight to moderate discrepancies appear at the concentration front for Si (Figures \\ref{fig6}d - f). Notwithstanding, the Si concentration remains globally well predicted. Furthermore, Figures \\ref{fig7} - \\ref{fig8} present the original and emulated 2D solid amount profiles corresponding to Figures \\ref{fig5} - \\ref{fig6}. The original solid amount profiles are overall well approximated by HPx$_{\\rm py}$-kNN and HPx$_{\\rm py}$-DNN for the advection-dispersion case (Figure \\ref{fig7}), even though some mismatch appears at the border of the fully depleted zone for the H component. As of the diffusion case (Figure \\ref{fig8}), the same kind of mismatch is observed for the emulated solid amounts of H by HPx$_{\\rm py}$-kNN while the emulated profiles by HPx$_{\\rm py}$-DNN show some more discrepancies. Though we decided to present raw emulation results, we would like to stress that some if not all of the observed artifacts could likely be smoothed out by using some post-filtering such as median filtering.\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure3.png}\n\t\\caption{Time series of original (RTM, solid green lines) and HPx$_{\\rm py}$-kNN emulated (TM+kNN, dashed orange lines) concentrations (mol/kg) of Ca, H, O and Si at selected observations points for cement system 1 and advective-dispersive transport. Obs. 1 - 5 denote the selected observation points, with the following $\\left[x,y\\right]$ locations (in cm). Obs. 1:  $\\left[0.5, 2.5\\right]$, Obs. 2: $\\left[1, 2\\right]$, Obs. 3: $\\left[2, 2\\right]$, Obs. 4: $\\left[1, 1\\right]$, Obs. 5: $\\left[2, 1\\right]$. The results for the 121 $\\times$ 121 grid size are rather similar.}\n\t\\label{fig3}\n\\end{figure}\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure4.png}\n\t\\caption{Time series of original (RTM, solid green lines) and HPx$_{\\rm py}$-DNN emulated (TM+DNN, dashed orange lines) concentrations (mol/kg) of Ca, H, O and Si at selected observations points for cement system 1 and advective-dispersive transport. Obs. 1 - 5 denote the selected observation points, with the following $\\left[x,y\\right]$ locations (in cm). Obs. 1:  $\\left[0.5, 2.5\\right]$, Obs. 2: $\\left[1, 2\\right]$, Obs. 3: $\\left[2, 2\\right]$, Obs. 4: $\\left[1, 1\\right]$, Obs. 5: $\\left[2, 1\\right]$. The results for the 121 $\\times$ 121 grid size are rather similar.}\n\t\\label{fig4}\n\\end{figure}\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure5.png}\n\t\\caption{2D concentration profiles obtained for cement system 1 at the end of the 2-year simulation performed for the advection-dispersion case. RTM means the original HP$_{\\rm{4C}}$ model, TM+kNN denotes the Hydrus transport model coupled with our kNN geochemical emulator (HPx$_{\\rm py}$-kNN) and TM+DNN signifies the Hydrus transport model coupled with our DNN geochemical emulator (HPx$_{\\rm py}$-DNN). The first to fourth row present profiles for $Ca^{\\rm{conc}}$, $Si^{\\rm{conc}}$, $H^{\\rm{conc}}$, and $O^{\\rm{conc}}$, respectively. The considered grid size is 61 $\\times$ 61. The results for the 121 $\\times$ 121 grid are rather similar.}\n\t\\label{fig5}\n\\end{figure}\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure6.png}\n\t\\caption{2D concentration profiles obtained for cement system 1 at the final time step of the 1-year simulation performed for the diffusion case. RTM means the original HP$_{\\rm{4C}}$ model, TM+kNN denotes the Hydrus transport model coupled with our kNN geochemical emulator (HPx$_{\\rm py}$-kNN) and TM+DNN signifies the Hydrus transport model coupled with our DNN geochemical emulator (HPx$_{\\rm py}$-DNN). The first to fourth row present profiles for $Ca^{\\rm{conc}}$, $Si^{\\rm{conc}}$, $H^{\\rm{conc}}$, and $O^{\\rm{conc}}$, respectively. The considered grid size is 61 $\\times$ 61.}\n\t\\label{fig6}\n\\end{figure}\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure7.png}\n\t\\caption{2D solid amount profiles obtained for cement system 1 at the end of the 2-year simulation performed for the advection-dispersion case. RTM means the original HP$_{\\rm{4C}}$ model, TM+kNN denotes the Hydrus transport model coupled with our kNN geochemical emulator (HPx$_{\\rm py}$-kNN) and TM+DNN signifies the Hydrus transport model coupled with our DNN geochemical emulator (HPx$_{\\rm py}$-DNN). The first to fourth row present profiles for $Ca^{\\rm{solid}}$, $Si^{\\rm{solid}}$, $H^{\\rm{solid}}$, and $O^{\\rm{solid}}$, respectively. The considered grid size is 61 $\\times$ 61. The results for the 121 $\\times$ 121 grid are rather similar.}\n\t\\label{fig7}\n\\end{figure}\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure8.png}\n\t\\caption{2D solid amount profiles obtained for cement system 1 at the final time step of the 1-year simulation performed for the diffusion case. RTM means the original HP$_{\\rm{4C}}$ model, TM+kNN denotes the Hydrus transport model coupled with our kNN geochemical emulator (HPx$_{\\rm py}$-kNN) and TM+DNN signifies the Hydrus transport model coupled with our DNN geochemical emulator (HPx$_{\\rm py}$-DNN). The first to fourth row present profiles for $Ca^{\\rm{solid}}$, $Si^{\\rm{solid}}$, $H^{\\rm{solid}}$, and $O^{\\rm{solid}}$, respectively. The considered grid size is 61 $\\times$ 61.}\n\t\\label{fig8}\n\\end{figure}\n\nThe speedups associated with the considered problem are detailed in Table \\ref{table3}. It is noted that the GPU-based DNN emulator allows for a speedup that is close to optimal. Indeed, The DNN speedups overall represent 85 \\% to 95 \\% of the maximum possible speedups (that is, speedups that would be obtained if the geochemcical calculations would come at no cost at all). The speedups associated with single-threaded kNN remain substantial but only amount to 57 \\% - 65 \\% of the corresponding maximum speedups. As detailed in section \\ref{train_res1}, the used kNN and DNN implementations are found to be respectively 300 and 4000 faster than single-threaded PHREEQC when predicting 10,000 points all one once for this geochemical system. Based on these numbers one could have expected the achieved speedups to represent say 90 \\% (kNN) or 99 \\% (DNN) of the maximum possible ones. A large part of the gaps between achieved and maximum possible speedups is thus likely caused by the time required for communicating and exchanging data between the main C/C++ code and the Python-based emulators.\n\n\\begin{table}[h!]\n\t\\caption{Speedups offered by the KNN and DNN emulators in HPx$_{\\rm py}$ for the reactive transport simulations considered for cement system 1. The HPx$_{\\rm{4C}}$ calculations involve the parallelization of PHREEQC over our 4 CPUs. The HPx$_{\\rm{1C}}$ calculations are performed on a single CPU. The kNN predictions are performed on a single CPU using the scikit-learn implementation while the DNN predictions make use of our GPU. ML signifies the used machine learning method for emulation, TC denotes transport conditions (ADV: advection-dispersion, DIF: diffusion) and GS is the grid size. The maximum possible speedups associated with HPx$_{\\rm{4C}}$ and HPx$_{\\rm{1C}}$,  Max SP HPx$_{\\rm{4C}}$ and  Max SP HPx$_{\\rm{1C}}$, correspond to an hypothetical situation where the geochemical calculations incur zero computational cost.}\n\n\t\\begin{center}\n\t\t\\begin{tabular}{cccccccc}\n\t\t\t\\hline\n\t\t\t& & & & & & & \\\\\n\t\t\tML & TC & GS & HPx$_{\\rm{4C}}$ time (s) & SP HPx$_{\\rm{4C}}$ & Max SP HPx$_{\\rm{4C}}$ & SP HPx$_{\\rm{1C}}$ & Max SP HPx$_{\\rm{1C}}$ \\\\\n\t\t\tDNN & ADV & 61 $\\times$ 61 & 3189 & 6.8 & 7.7 & 24.5 & 28.5 \\\\\n\t\t\tkNN & ADV & 61 $\\times$ 61 & 3189 & 5.0 & 7.7 & 18.0 & 28.5 \\\\\n\t\t\tDNN & ADV & 121 $\\times$ 121 & 23,337 & 5.0 & 5.2 & 17.0 & 18.0 \\\\\n\t\t\tkNN & ADV & 121 $\\times$ 121 & 23,337 & 3.4 & 5.2 & 11.6 & 18.0 \\\\\n\t\t\tDNN & DIF & 61 $\\times$ 61 & 25,448 & 4.2 & 4.9 & 16.2 & 19.1 \\\\\n\t\t\tkNN & DIF & 61 $\\times$ 61 & 25,448 & 2.8 & 4.9 & 10.8 & 19.1 \\\\\n\t\t\t\\hline\n\t\t\\end{tabular}\n\t\\end{center}\n\n\t\\label{table3}\n\\end{table}\n\n\\FloatBarrier\n\n\\subsection{Al-C-Ca-S-Si Problem}\n\nFor this second cement system, the emulation problem consists of predicting at each time step of the RT simulation the (output) Al, C, Ca, S, Si, H and O aqueous concentrations (mol/kgw) from the (input) total amounts of Al, C, Ca, S and Si (mol). Here we focus on advective-dispersive transport only while similarly as for cement system 1, the considered domain sizes are 61 $\\times$ 61 and 121 $\\times$ 121. Moreover, the simulation time period is set to 6 years. \n\nAs mentioned earlier, for this higher-dimensional problem it is observed that the used scikit-learn kNN implementation becomes prohibitively slow compared to HPx$_{\\rm{4C}}$. We found that to get a good emulation accuracy the kNN training base needs to contain 1,000,000 samples (or more). This training base's size together with a 5-dimensional search space leads to an HPx$_{\\rm py}$-kNN reactive transport simulation time that is comparable to that of HPx$_{\\rm{4C}}$. Therefore, for this second cement system we built a custom kNN regressor around another kNN implementation contained in the FAISS package \\citep{faiss2017}. Our used FAISS variant allows for GPU computing and is much faster than scikit-learn for this cement system, but is slightly less accurate due to the use of an approximate rather than exact nearest neighbor search \\citep[see][for details]{faiss2017}.\n\n\\subsubsection{Training the DNN}\n\\label{train_res2}\n\nBuilding a good training set to perform a kNN search and learn the weights and biases of our DNN turned out to be a complicated task in this case. This because to make useful kNN predictions and/or learn an useful DNN, the training set must be sufficiently representative of the geochemical conditions encountered during the reactive transport simulation one wish to perform with HPx$_{\\rm py}$-kNN and the trained HPx$_{\\rm py}$-DNN. In contrast to cement system 1, creating the training set by sampling the $X$-space with a controlled randomness between predefined lower and upper bounds did not prove successful. We tried that strategy by drawing as much as 4,000,000 5-dimensional $\\textbf{x}$ vectors from the $X$-space using a Sobol low-discrepancy sequence \\citep[][]{Sobol1967, Joe-Kuo2003}. Such low-discrepancy sampling scheme covers the 5-dimensional hypercube more uniformly than LHS. Despite a good performance on the test set (not shown), the resulting DNN accuracy in reactive transport mode was never deemed satisfying. In other words, no satisfying ``global\" or ``universal\" DNN emulator could be devised for this cement system. This is probably caused by the fact for this problem, the input (5 total amounts) and output (7 aqueous concentrations) spaces are quite nonlinearly related and both cover 6 to 10 orders of magnitudes depending on the considered element. Therefore, we resorted to the alternative training strategy detailed below. The latter basically tries to grasp the complex correlations and higher-order dependencies that exist between the elements of $\\textbf{x}$ (total amounts, input space) for a given reactive transport simulation setup, in order to produce a training set that honors these between-input relationships.\n\n\\begin{itemize}\n\t\\item Perform a ``cheap\" full reactive transport simulation under the transport conditions and geochemistry of interest and collect the resulting $\\textbf{x}$ -$\\textbf{y}$  pairs of examples (for the considered grid nodes and time steps). Computational demand controls what domain size and simulation time period can be used for this cheap calculation. We used a modest 16 $\\times$ 16 domain and a simulation time period of 10 years. The associated HPx$_{\\rm{4C}}$ runtimes is 180 s.\n\t\n\t\\item Fit a kernel density estimator (KDE) with a Gaussian kernel to the collected $\\textbf{x}$ vectors (encapsulated in the $\\textbf{X}$ array) and generate a fixed number of new input vectors, $\\textbf{x}_{KDE}$. Then run PHREEQC for the $\\textbf{X}_{KDE}$ set to get the corresponding output set, $\\textbf{Y}_{KDE}$. Now apply the correction for porosity described in section \\ref{train_res1} to the $\\textbf{X}_{KDE}$ set and form the training set by merging the ensemble of $\\textbf{x}$-$\\textbf{y}$ pairs with that of the $\\textbf{x}_{KDE}$-$\\textbf{y}_{KDE}$ pairs. The number of produced unique examples by the considered cheap HPx$_{\\rm{4C}}$ simulations varied between 10,000 and 50,000. The KDE-based enrichment of this dataset was deemed necessary to provide more input variability, thereby avoiding overfitting of the trained DNN and improving the kNN accuracy, while still honoring the complex between-input relationships. The number of KDE-generated samples was set as to obtain a total training set size of 1,000,000 examples. A key component of the approach is the bandwith parameter of the Gaussian KDE kernel which controls how much the KDE-generated samples depart from the original ensemble. After limited trial and error, we fixed the kernel bandwith to 0.0025 for the considered case studies.\n\t\n\\end{itemize}\n\nThe scatter plots in Figure \\ref{fig9} illustrate our training set creation procedure. The orange dots depict the pairwise relationships observed between the 5 elements of $\\textbf{x}$ in the cheap simulation. The red and cyan dots in Figure \\ref{fig9} represent the KDE-generated samples with the selected bandwidth, before and after applying the correction for porosity, respectively. Training of the DNN is achieved using the ensemble of original and KDE-corrected input points. We refer to this kind of dataset as RT-based, since it is based on an albeit cheap, full RT simulation. Furthermore, we refer to the obtained DNN and kNN emulators as ``local\" emulators, since as opposed to the emulators constructed for cement system 1, the current emulators are only valid for the input conditions encapsulated in the RT-based training set.\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1.5cm}\\includegraphics[width=50pc]{Figure9.png}\n\t\\caption{Scatter plots of the complex relationships between the five considered inputs for (1) the computationally cheap RT simulation of cement system 2 (orange dots) performed with the original HPx code, (2) the corresponding sampled points by kernel density estimation (KDE) using the selected bandwith (turquoise dots) and, (3) the same sampled KDE points after porosity correction (red dots).}\n\t\\label{fig9}\n\\end{figure}\n\nTraining performance of the DNN emulator is presented in Figure \\ref{fig10}. Similarly as for cement system 1, about 90\\% of the available data was used for the actual training of the DNN while the remaining 10\\% were used a validation set to contol overfitting. Lastly, performance is evaluated for both the trained DNN and kNN emulators using an independent test set of 10,000 examples. Overall, the accuracy of our ``local \" DNN emulator for this RT-inspired dataset is rather large with $Q_2$ values always greater or equal than 0.998. Training performance of the corresponding local kNN emulator is equally good (not shown). Regarding speedup, for this problem single-threaded PHREEQC achieves about 210 geochemical calculations per second on our used Intel\\textsuperscript{\\textregistered} i7 CPU while the GPU-based DNN and kNN emulators are both about 3000 times faster when predicting the 10,000 test points at once .\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure10.png}\n\t\\caption{1-1 plots of local DNN emulation performance obtained for system 2 when the local DNN is trained using 1,000,000 samples. The $x$-axis and $y$-axis present the original and emulated 10,000 independent test data points, respectively. The RMSE and $Q_2$ coefficient denote the root-mean-square-error and coefficient of determination in testing mode, respectively, between the original and emulated 10,000 test data points.}\n\t\\label{fig10}\n\\end{figure}\n\n\\subsubsection{Reactive transport simulation}\n\\label{prob2res}\nOur ``local\" DNN performs rather well when applied to the 61 $\\times$ 61 grid size and a time period of 6 years (Figures \\ref{fig11} - \\ref{fig12}). This for all components but C, for which some localized deviations appear between original and emulated 2D concentration profiles towards the end of the simulation period (Figure \\ref{fig12}). When applied to the 121 $\\times$ 121 grid, HPx$_{\\rm py}$-DNN produces additional discrepancies for O, Si and Al towards the end of the simulation period (Figures \\ref{fig13} - \\ref{fig14}). Yet most of the observed artifacts could probably be smoothed out by using post-filtering. The associated speedups are listed in Table \\ref{table4}. These speedups are larger to those obtained for cement system 1, with values between 8 and 9 when evaluated against HPx$_{\\rm{4C}}$. These speedups represent about 85 \\% to 90 \\% of the maximum possible speedup (Table \\ref{table3}). Overall, these findings indicate that for the considered problem, our RT-based training of a local DNN only works is the training set is sufficiently representative of the particular geochemical conditions encountered in the computationally demanding simulations, which is arguably not easy to achieve. This limitation is further discussed in section \\ref{discussion}.\n\nWe note a more uniform behavior for HPx$_{\\rm py}$-kNN across grid sizes than for HPx$_{\\rm py}$-DNN. Here the results for the 121  $\\times$ 121 grid are only slightly less accurate than those associated with the 61 $ \\times$ 61 grid (see Figures \\ref{fig15} - \\ref{fig16} where for brevity we only show concentration profiles for the C Al and S components). Furthermore, whenever observed the discrepancies between original and emulated profiles are more regularly scattered than for HPx$_{\\rm py}$-DNN. Note also that herein too, post-filtering could likely smooth out a large part of these deviations. In addition, owing to the use of a GPU to achieve the kNN calculations, the speedup provided by HPx$_{\\rm py}$-kNN are as large as those provided by HPx$_{\\rm py}$-DNN (Table \\ref{table4}).\n\nWith respect to the emulated solid amounts, the HPx$_{\\rm py}$-DNN results look visually good for the the 61 $\\times$ 61 grid. This is shown in Figure \\ref{fig17} for the C, Al and S chemical components, while emulation of the H, O Ca and Si chemical components is globally of similar quality (not shown). Nevertheless, significant deviations appear towards the end of the simulation period for every chemical component (see Figure \\ref{fig18} for the C, Al and S chemical components, emulation of the H, O Ca and Si chemical components shows the same level of mismatch). The HPx$_{\\rm py}$-kNN predictions are also fairly accurate for the 61 $\\times$ 61 (see Figure \\ref{fig19} for the C, Al and S chemical components, emulation of the H, O Ca and Si chemical components exhibits a globally similar quality) while some discrepancies show up at the end of the simulation (Figure \\ref{fig20}). However, the mismatch is less pronounced than for HPx$_{\\rm py}$-DNN. Overall, HPx$_{\\rm py}$-kNN appears to be somewhat more robust than HPx$_{\\rm py}$-DNN for this cement system, while providing the same (large) speedup.\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure11.png}\n\t\\caption{2D concentration profiles obtained for cement system 2 after 2 and 6 (final time step) years. RTM means the original HPx$_{\\rm{4C}}$ model and TM+DNN denotes the Hydrus transport model coupled with our DNN geochemical emulator (HPx$_{\\rm py}$-DNN). The first to fourth row present profiles for $Ca^{\\rm{conc}}$, $Si^{\\rm{conc}}$, $H^{\\rm{conc}}$, and $O^{\\rm{conc}}$, respectively. The considered grid size is 61 $\\times$ 61.}\n\t\\label{fig11}\n\\end{figure}\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure12.png}\n\t\\caption{2D concentration profiles obtained for cement system 2 after 2 and 6 (final time step) years. RTM means the original HPx$_{\\rm{4C}}$ model and TM+DNN denotes the Hydrus transport model coupled with our DNN geochemical emulator (HPx$_{\\rm py}$-DNN). The first to third row present profiles for $C^{\\rm{conc}}$, $Al^{\\rm{conc}}$, and $S^{\\rm{conc}}$, respectively. The considered grid size is 61 $\\times$ 61.}\n\t\\label{fig12}\n\\end{figure}\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure13.png}\n\t\\caption{2D concentration profiles obtained for cement system 2 after 2 and 6 (final time step) years. RTM means the original HPx$_{\\rm{4C}}$ model and TM+DNN denotes the Hydrus transport model coupled with our DNN geochemical emulator (HPx$_{\\rm py}$-DNN). The first to fourth row present profiles for $Ca^{\\rm{conc}}$, $Si^{\\rm{conc}}$, $H^{\\rm{conc}}$, and $O^{\\rm{conc}}$, respectively. The considered grid size is 121 $\\times$ 121.}\n\t\\label{fig13}\n\\end{figure}\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure14.png}\n\t\\caption{2D concentration profiles obtained for cement system 2 after 2 and 6 (final time step) years. RTM means the original HPx$_{\\rm{4C}}$ model and TM+DNN denotes the Hydrus transport model coupled with our DNN geochemical emulator (HPx$_{\\rm py}$-DNN). The first to third row present profiles for $C^{\\rm{conc}}$, $Al^{\\rm{conc}}$, and $S^{\\rm{conc}}$, respectively. The considered grid size is 121 $\\times$ 121.}\n\t\\label{fig14}\n\\end{figure}\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure15.png}\n\t\\caption{2D concentration profiles obtained for cement system 2 after 2 and 6 (final time step) years. RTM means the original HPx$_{\\rm{4C}}$ model and TM+kNN denotes the Hydrus transport model coupled with our kNN geochemical emulator (HPx$_{\\rm py}$-kNN).  The first to third row present profiles for $C^{\\rm{conc}}$, $Al^{\\rm{conc}}$, and $S^{\\rm{conc}}$, respectively. The considered grid size is 61 $\\times$ 61.}\n\t\\label{fig15}\n\\end{figure}\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure16.png}\n\t\\caption{2D concentration profiles obtained for cement system 2 after 2 and 6 (final time step) years. RTM means the original HPx$_{\\rm{4C}}$ model and TM+kNN denotes the Hydrus transport model coupled with our kNN geochemical emulator (HPx$_{\\rm py}$-kNN). The first to third row present profiles for $C^{\\rm{conc}}$, $Al^{\\rm{conc}}$, and $S^{\\rm{conc}}$, respectively. The considered grid size is 121 $\\times$ 121.}\n\t\\label{fig16}\n\\end{figure}\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure17.png}\n\t\\caption{2D solid amount profiles of the C, Al and S chemical components of cement system 2 after 2 and 6 (final time step) years. RTM means the original HPx$_{\\rm{4C}}$ model and TM+DNN denotes the Hydrus transport model coupled with our DNN geochemical emulator (HPx$_{\\rm py}$-DNN). The first to third row present profiles for $C^{\\rm{solid}}$, $Al^{\\rm{solid}}$, and $S^{\\rm{solid}}$, respectively The considered grid size is 61 $\\times$ 61.}\n\t\\label{fig17}\n\\end{figure}\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure18.png}\n\t\\caption{2D solid amount profiles of the C, Al and S chemical components of cement system 2 after 2 and 6 (final time step) years. RTM means the original HPx$_{\\rm{4C}}$ model and TM+DNN denotes the Hydrus transport model coupled with our DNN geochemical emulator (HPx$_{\\rm py}$-DNN). The first to third row present profiles for $C^{\\rm{solid}}$, $Al^{\\rm{solid}}$, and $S^{\\rm{solid}}$, respectively The considered grid size is 121 $\\times$ 121.}\n\t\\label{fig18}\n\\end{figure}\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure19.png}\n\t\\caption{2D solid amount profiles of the C, Al and S chemical components of cement system 2 after 2 and 6 (final time step) years. RTM means the original HPx$_{\\rm{4C}}$ model and TM+kNN denotes the Hydrus transport model coupled with our kNN geochemical emulator (HPx$_{\\rm py}$-kNN). The first to third row present profiles for $C^{\\rm{solid}}$, $Al^{\\rm{solid}}$, and $S^{\\rm{solid}}$, respectively The considered grid size is 61 $\\times$ 61.}\n\t\\label{fig19}\n\\end{figure}\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure20.png}\n\t\\caption{2D solid amount profiles of the C, Al and S chemical components of cement system 2 after 2 and 6 (final time step) years. RTM means the original HPx$_{\\rm{4C}}$ model and TM+kNN denotes the Hydrus transport model coupled with our kNN geochemical emulator (HPx$_{\\rm py}$-kNN). The first to third row present profiles for $C^{\\rm{solid}}$, $Al^{\\rm{solid}}$, and $S^{\\rm{solid}}$, respectively The considered grid size is 121 $\\times$ 121.}\n\t\\label{fig20}\n\\end{figure}\n\n\\begin{table}[h!]\n\t\\caption{Speedups offered by the KNN and DNN emulators in HPx$_{\\rm py}$ for the reactive transport simulations considered for cement system 2. The HPx$_{\\rm{4C}}$ calculations involve the parallelization of PHREEQC over our 4 CPUs. The HPx$_{\\rm{1C}}$ calculations are performed on a single CPU. Both the kNN and DNN predictions make use of a GPU. ML signifies the used machine learning method for emulation, BC denotes the type of flow boundary conditions and GS is the grid size. The maximum possible speedups associated with HPx$_{\\rm{4C}}$ and HP$_{\\rm{1C}}$,  Max SP HPx$_{\\rm{4C}}$ and  Max SP HPx$_{\\rm{1C}}$, correspond to an hypothetical situation where the geochemical calculations incur zero computational cost.}\n\n\t\\begin{center}\n\t\t\\begin{tabular}{cccccccc}\n\t\t\t\\hline\n\t\t\t& & & & & & & \\\\\n\t\t\tML & BC & GS & HPx$_{\\rm{4C}}$ time (s) & SP HPx$_{\\rm{4C}}$ & Max SP HPx$_{\\rm{4C}}$ & SP HPx$_{\\rm{1C}}$ & Max SP HPx$_{\\rm{1C}}$ \\\\\n\t\t\tDNN & ADV & 61 $\\times$ 61 & 21,415 & 8.2 & 9.0 & 30.3 & 33.1\\\\\n\t\t\tkNN & ADV & 61 $\\times$ 61 & 21,415 & 7.9 & 9.0 & 28.9 & 33.1\\\\\n\t\t\tDNN & ADV & 121 $\\times$ 121 & 199,841 & 8.2 & 9.5 & 29.9 & 32.8\\\\\n\t\t\tkNN & ADV & 121 $\\times$ 121 & 199,841 & 8.5 & 9.5 & 31.3 & 32.8\\\\\n\t\t\t\\hline\n\t\t\\end{tabular}\n\t\\end{center}\n\n\t\\label{table4}\n\\end{table}\n\n\\FloatBarrier\n\n3.1 Ca-Si Problem\n\\subsection{Ca-Si Problem}\nFor this first cement system, the emulation problem consists of predicting at each time step of the RT simulation the (output) Ca, Si, H and O aqueous concentrations (mol/kg of water or mol/kgw) from the (input) total amounts of Ca and Si (mol). \n\n\\subsubsection{Training the emulators}\n\\label{train_res1}\nHere the kNN and DNN emulators are firstly trained using a set of 400,000 test examples for both. This training set is obtained by randomly sampling the two-dimensional input space by latin hypercube sampling (LHS) between $\\left[0,0\\right]$ and $\\rm{\\left[Ca^{tot}_{max},Si^{tot}_{max}\\right]}$, and running PHREEQC for each input sample, $\\rm{\\textbf{x}_i = \\left[Ca^{tot}_i, Si^{tot}_i\\right]}$ to get the corresponding output vectors, $\\rm{\\textbf{y}_i = \\left[Ca^{conc}_i,Si^{conc}_i,H^{conc}_i,O^{conc}_i\\right]}$. The upper bounds, $\\rm{Ca^{tot}_{max}}$ and $\\rm{Si^{tot}_{max}}$ are defined based on a cheap full RT simulation with advective-dispersive transport using a small 1D domain of 51 nodes. It is worth noting that the total amounts of $\\textbf{x}_i$, corresponding to the PHREEQC-simulated concentrations, $\\textbf{y}_i$, have to be corrected for the different amount of water between the training set and the transport simulations. Doing so, it turns out that about 20 \\% of the post-corrected $x_i$ values exceed their pre-defined upper bounds and these excessively large values need to be filtered out. Creating the 400,000 training examples thus required about 500,000 PHREEQC runs. As stated earlier, for this problem single-threaded PHREEQC performs about 670 geochemical calculations per second on our used Intel\\textsuperscript{\\textregistered} i7 CPU.\n\nWith respect to kNN, the tuning parameters are the number of neighbors, $k$, the type of distance measure, and the interpolation technique. We simply used the default settings: $k = 5$, euclidean distance and inverse-distance interpolation. Regarding training of the DNN, the 400,000 sample were split between the training set itself (90 \\% of the data) and a validation set (10 \\% of the data). The latter serves to monitor the evolution of the selected mean squared error loss function on samples that are not used for training, thereby detecting potential overfitting. If the validation loss stops decreasing before the fixed number of epochs has been completed, then  training is stopped. Importantly, the emulation is achieved in log-space for both the input, $X$ and output $Y$, domains. This because total amounts and concentrations of the involved components typically cover many orders of magnitudes (up to 10 orders or more). Using a DNN also requires some form of data normalization or standardization. Here both the $\\rm{log\\left(\\textbf{x}_i\\right)}$ and $\\rm{log\\left(\\textbf{y}_i\\right)}$ vectors are standardized around 0 with standard deviation of 1.\n\nFigure \\ref{fig1} illustrates the trained emulators' performance for geochemical predictions using an independent test set that comprises 10,000 test examples. Both kNN (Figures \\ref{fig1}a - d) and DNN (Figures \\ref{fig1}e - h) appear to be rather accurate . DNN also shows a slight degradation for the larger concentration values  (Figures \\ref{fig1}e - h). The latter is likely due to the combination of a small proportion of large concentration values in the training set with the log-transformation that implicitly pushes the DNN to try harder to fit the smaller concentrations during training. Regarding speedup and as written earlier, for this setup the single-threaded kNN method is 300 times faster than single-threaded PHREEQC for predicting the 10,000 concentration vectors all at once. The computational savings allowed by the DNN emulator when ran on our GPU is higher with a speedup as large as 4000 for predicting the same 10,000 concentration vectors all at once. \n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{0cm}\\includegraphics[width=35pc]{Figure1.png}\n\t\\caption{1-1 plots of the kNN (subfigures (a) - (d)) and DNN (subfigures (e) - (h)) emulators' performance obtained for system 1 when the kNN training base contains 400,000 samples and the DNN is trained using the same 400,000 samples. Here ``true\" means the original PHREEQC-simulated data and ``predicted\" denotes the emulated (that is, kNN-simulated and DNN-simulated) data. Hence, the $x$-axis and $y$-axis present the original and emulated 10,000 independent test data points, respectively. The RMSE and $Q_2$ coefficient denote the root-mean-square-error and coefficient of determination in testing mode, respectively, between the original and emulated 10,000 test data points.}\n\t\\label{fig1}\n\\end{figure}\n\nTo test the sensitivity of the emulators' performance to the training set size, training was also performed using reduced training sets comprising 100,000, 10,000, 1000 and 100 samples, respectively. It seen that the DNN performance achieved  when using 10,000 training samples is virtually the same as that obtained when using 400,000 training samples (Figure \\ref{fig2} and Table \\ref{table2}). It is only for training sets smaller than 1000 samples that the DNN performance starts to degrade significantly (Table \\ref{table2}). In contrast, the behavior of kNN appears to markedly decrease as the training set gets smaller (Figure \\ref{fig2} and Table \\ref{table1}).\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{0cm}\\includegraphics[width=35pc]{Figure2.png}\n\t\\caption{1-1 plots of the kNN (subfigures (a) - (d)) and DNN (subfigures (e) - (h)) emulators' performance obtained for system 1 when the kNN training base contains 10,000 samples and the DNN is trained using the same 10,000 samples. Here ``true\" means the original PHREEQC-simulated data and ``predicted\" denotes the emulated (that is, kNN-simulated and DNN-simulated) data. Hence, the $x$-axis and $y$-axis present the original and emulated 10,000 independent test data points, respectively. The RMSE and $Q_2$ coefficient denote the root-mean-square-error and coefficient of determination in testing mode, respectively, between the original and emulated 10,000 test data points.}\n\t\\label{fig2}\n\\end{figure}\n\n\\begin{table}[h!]\n\t\\caption{Performance of the DNN and kNN emulators for cement system 1 and different training set sizes. For brevity, only the results for Ca$^{conc}$ and Si$^{conc}$ are shown. The units are mol per kg of water (mol/kgw). ML refers to the type of emulator, TR signifies the size of the training set, and the RMSE and $Q_2$ coefficient denote the root-mean-square-error and coefficient of determination in testing mode, respectively, between the original and emulated 10,000 test data points.}\n\t\\begin{center}\n\t\t\\begin{tabular}{cccccccc}\n\t\t\t\\hline\n\t\t\t& & & & & \\\\\n\t\t\tML & TR & RMSE - Ca$^{conc}$ & $Q_2$ - Ca$^{conc}$ & RMSE - Si$^{conc}$ & $Q_2$ - Si$^{conc}$\\\\\n\t\t\tDNN & 4 $\\times$ 10$^{5}$ & 4.50 $\\times$ 10$^{-5}$ & 0.9999 & 1.21 $\\times$ 10$^{-6}$ & 0.9999\\\\\n\t\t\tDNN & 1 $\\times$ 10$^{5}$ & 5.06 $\\times$ 10$^{-5}$ & 0.9999 & 1.30 $\\times$ 10$^{-6}$ & 0.9999\\\\\n\t\t\tDNN & 1 $\\times$ 10$^{4}$ & 4.57 $\\times$ 10$^{-5}$ & 0.9999 & 1.48 $\\times$ 10$^{-6}$ & 0.9999\\\\\n\t\t\tDNN & 1 $\\times$ 10$^{3}$ & 8.02 $\\times$ 10$^{-5}$ & 0.9998 & 3.31 $\\times$ 10$^{-6}$ & 0.9994\\\\\n\t\t\tDNN & 1 $\\times$ 10$^{2}$ & 47.1 $\\times$ 10$^{-5}$ & 0.9935 & 9.02 $\\times$ 10$^{-6}$ & 0.9959\\\\\n\t\t\tkNN & 4 $\\times$ 10$^{5}$ & 2.52 $\\times$ 10$^{-5}$ & 1.0000 & 1.59 $\\times$ 10$^{-6}$ & 0.9999\\\\\n\t\t\tkNN & 1 $\\times$ 10$^{5}$ & 6.27 $\\times$ 10$^{-5}$ & 0.9999 & 4.20 $\\times$ 10$^{-6}$ & 0.9991\\\\\n\t\t\tkNN & 1 $\\times$ 10$^{4}$ & 14.1 $\\times$ 10$^{-5}$ & 0.9994 & 13.5 $\\times$ 10$^{-6}$ & 0.9908\\\\\n\t\t\tkNN & 1 $\\times$ 10$^{3}$ & 43.9 $\\times$ 10$^{-5}$ & 0.9944 & 74.6 $\\times$ 10$^{-6}$ & 0.7180\\\\\n\t\t\tkNN & 1 $\\times$ 10$^{2}$  & 96.4 $\\times$ 10$^{-5}$ & 0.9729 & 55.0 $\\times$ 10$^{-6}$ & 0.8468\\\\\n\t\t\\end{tabular}\n\t\\end{center}\n\t\\label{table2}\n\\end{table}\n\n\\FloatBarrier\n\n\\subsubsection{Reactive transport simulation}\n\\label{prob12Dres}\n\nThis section focuses on reactive transport simulations with HPx$_{\\rm py}$ within cement system 1, under both advective-dispersive and diffusive transport conditions. As written above, the domain sizes are both 61 $\\times$ 61 and 121 $\\times$ 121 for the advection-dispersion case and, because of computational constraints, solely 61 $\\times$ 61  for the diffusion case. In addition, the simulation time period is 2 years for the advection-dispersion case and 1 year for the diffusion case. Figures \\ref{fig3} and \\ref{fig4} present times series of original and emulated Ca, Si, H and O concentrations at 5 locations within the 2D domain for advective-dispersive transport conditions, for both our kNN-based (HPx$_{\\rm py}$-kNN) and DNN-based (HPx$_{\\rm py}$-DNN) reactive transport codes. It is seen that HPx$_{\\rm py}$-kNN and HPx$_{\\rm py}$-DNN both induce a quite good simulation accuracy. Also, the results for the diffusive transport case are of similarly good quality (not shown). Figures \\ref{fig5} - \\ref{fig6} provide more insights into the HPx$_{\\rm py}$-kNN and HPx$_{\\rm py}$-DNN performances by displaying 2D Ca, Si, H and O concentration profiles at a given time. For each experiment and chemical component, this time is selected as to be well representative of the simulated dynamics. It is observed that the original and emulated images are visually almost indistinguishable for the advection-dispersion case (Figure \\ref{fig5}). For the diffusion case, the emulators also perform quite well for Ca, H and O (Figures\\ref{fig6}a - c, g - i and j - l), while some slight to moderate discrepancies appear at the concentration front for Si (Figures \\ref{fig6}d - f). Notwithstanding, the Si concentration remains globally well predicted. Furthermore, Figures \\ref{fig7} - \\ref{fig8} present the original and emulated 2D solid amount profiles corresponding to Figures \\ref{fig5} - \\ref{fig6}. The original solid amount profiles are overall well approximated by HPx$_{\\rm py}$-kNN and HPx$_{\\rm py}$-DNN for the advection-dispersion case (Figure \\ref{fig7}), even though some mismatch appears at the border of the fully depleted zone for the H component. As of the diffusion case (Figure \\ref{fig8}), the same kind of mismatch is observed for the emulated solid amounts of H by HPx$_{\\rm py}$-kNN while the emulated profiles by HPx$_{\\rm py}$-DNN show some more discrepancies. Though we decided to present raw emulation results, we would like to stress that some if not all of the observed artifacts could likely be smoothed out by using some post-filtering such as median filtering.\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure3.png}\n\t\\caption{Time series of original (RTM, solid green lines) and HPx$_{\\rm py}$-kNN emulated (TM+kNN, dashed orange lines) concentrations (mol/kg) of Ca, H, O and Si at selected observations points for cement system 1 and advective-dispersive transport. Obs. 1 - 5 denote the selected observation points, with the following $\\left[x,y\\right]$ locations (in cm). Obs. 1:  $\\left[0.5, 2.5\\right]$, Obs. 2: $\\left[1, 2\\right]$, Obs. 3: $\\left[2, 2\\right]$, Obs. 4: $\\left[1, 1\\right]$, Obs. 5: $\\left[2, 1\\right]$. The results for the 121 $\\times$ 121 grid size are rather similar.}\n\t\\label{fig3}\n\\end{figure}\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure4.png}\n\t\\caption{Time series of original (RTM, solid green lines) and HPx$_{\\rm py}$-DNN emulated (TM+DNN, dashed orange lines) concentrations (mol/kg) of Ca, H, O and Si at selected observations points for cement system 1 and advective-dispersive transport. Obs. 1 - 5 denote the selected observation points, with the following $\\left[x,y\\right]$ locations (in cm). Obs. 1:  $\\left[0.5, 2.5\\right]$, Obs. 2: $\\left[1, 2\\right]$, Obs. 3: $\\left[2, 2\\right]$, Obs. 4: $\\left[1, 1\\right]$, Obs. 5: $\\left[2, 1\\right]$. The results for the 121 $\\times$ 121 grid size are rather similar.}\n\t\\label{fig4}\n\\end{figure}\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure5.png}\n\t\\caption{2D concentration profiles obtained for cement system 1 at the end of the 2-year simulation performed for the advection-dispersion case. RTM means the original HP$_{\\rm{4C}}$ model, TM+kNN denotes the Hydrus transport model coupled with our kNN geochemical emulator (HPx$_{\\rm py}$-kNN) and TM+DNN signifies the Hydrus transport model coupled with our DNN geochemical emulator (HPx$_{\\rm py}$-DNN). The first to fourth row present profiles for $Ca^{\\rm{conc}}$, $Si^{\\rm{conc}}$, $H^{\\rm{conc}}$, and $O^{\\rm{conc}}$, respectively. The considered grid size is 61 $\\times$ 61. The results for the 121 $\\times$ 121 grid are rather similar.}\n\t\\label{fig5}\n\\end{figure}\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure6.png}\n\t\\caption{2D concentration profiles obtained for cement system 1 at the final time step of the 1-year simulation performed for the diffusion case. RTM means the original HP$_{\\rm{4C}}$ model, TM+kNN denotes the Hydrus transport model coupled with our kNN geochemical emulator (HPx$_{\\rm py}$-kNN) and TM+DNN signifies the Hydrus transport model coupled with our DNN geochemical emulator (HPx$_{\\rm py}$-DNN). The first to fourth row present profiles for $Ca^{\\rm{conc}}$, $Si^{\\rm{conc}}$, $H^{\\rm{conc}}$, and $O^{\\rm{conc}}$, respectively. The considered grid size is 61 $\\times$ 61.}\n\t\\label{fig6}\n\\end{figure}\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure7.png}\n\t\\caption{2D solid amount profiles obtained for cement system 1 at the end of the 2-year simulation performed for the advection-dispersion case. RTM means the original HP$_{\\rm{4C}}$ model, TM+kNN denotes the Hydrus transport model coupled with our kNN geochemical emulator (HPx$_{\\rm py}$-kNN) and TM+DNN signifies the Hydrus transport model coupled with our DNN geochemical emulator (HPx$_{\\rm py}$-DNN). The first to fourth row present profiles for $Ca^{\\rm{solid}}$, $Si^{\\rm{solid}}$, $H^{\\rm{solid}}$, and $O^{\\rm{solid}}$, respectively. The considered grid size is 61 $\\times$ 61. The results for the 121 $\\times$ 121 grid are rather similar.}\n\t\\label{fig7}\n\\end{figure}\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure8.png}\n\t\\caption{2D solid amount profiles obtained for cement system 1 at the final time step of the 1-year simulation performed for the diffusion case. RTM means the original HP$_{\\rm{4C}}$ model, TM+kNN denotes the Hydrus transport model coupled with our kNN geochemical emulator (HPx$_{\\rm py}$-kNN) and TM+DNN signifies the Hydrus transport model coupled with our DNN geochemical emulator (HPx$_{\\rm py}$-DNN). The first to fourth row present profiles for $Ca^{\\rm{solid}}$, $Si^{\\rm{solid}}$, $H^{\\rm{solid}}$, and $O^{\\rm{solid}}$, respectively. The considered grid size is 61 $\\times$ 61.}\n\t\\label{fig8}\n\\end{figure}\n\nThe speedups associated with the considered problem are detailed in Table \\ref{table3}. It is noted that the GPU-based DNN emulator allows for a speedup that is close to optimal. Indeed, The DNN speedups overall represent 85 \\% to 95 \\% of the maximum possible speedups (that is, speedups that would be obtained if the geochemcical calculations would come at no cost at all). The speedups associated with single-threaded kNN remain substantial but only amount to 57 \\% - 65 \\% of the corresponding maximum speedups. As detailed in section \\ref{train_res1}, the used kNN and DNN implementations are found to be respectively 300 and 4000 faster than single-threaded PHREEQC when predicting 10,000 points all one once for this geochemical system. Based on these numbers one could have expected the achieved speedups to represent say 90 \\% (kNN) or 99 \\% (DNN) of the maximum possible ones. A large part of the gaps between achieved and maximum possible speedups is thus likely caused by the time required for communicating and exchanging data between the main C/C++ code and the Python-based emulators.\n\n\\begin{table}[h!]\n\t\\caption{Speedups offered by the KNN and DNN emulators in HPx$_{\\rm py}$ for the reactive transport simulations considered for cement system 1. The HPx$_{\\rm{4C}}$ calculations involve the parallelization of PHREEQC over our 4 CPUs. The HPx$_{\\rm{1C}}$ calculations are performed on a single CPU. The kNN predictions are performed on a single CPU using the scikit-learn implementation while the DNN predictions make use of our GPU. ML signifies the used machine learning method for emulation, TC denotes transport conditions (ADV: advection-dispersion, DIF: diffusion) and GS is the grid size. The maximum possible speedups associated with HPx$_{\\rm{4C}}$ and HPx$_{\\rm{1C}}$,  Max SP HPx$_{\\rm{4C}}$ and  Max SP HPx$_{\\rm{1C}}$, correspond to an hypothetical situation where the geochemical calculations incur zero computational cost.}\n\n\t\\begin{center}\n\t\t\\begin{tabular}{cccccccc}\n\t\t\t\\hline\n\t\t\t& & & & & & & \\\\\n\t\t\tML & TC & GS & HPx$_{\\rm{4C}}$ time (s) & SP HPx$_{\\rm{4C}}$ & Max SP HPx$_{\\rm{4C}}$ & SP HPx$_{\\rm{1C}}$ & Max SP HPx$_{\\rm{1C}}$ \\\\\n\t\t\tDNN & ADV & 61 $\\times$ 61 & 3189 & 6.8 & 7.7 & 24.5 & 28.5 \\\\\n\t\t\tkNN & ADV & 61 $\\times$ 61 & 3189 & 5.0 & 7.7 & 18.0 & 28.5 \\\\\n\t\t\tDNN & ADV & 121 $\\times$ 121 & 23,337 & 5.0 & 5.2 & 17.0 & 18.0 \\\\\n\t\t\tkNN & ADV & 121 $\\times$ 121 & 23,337 & 3.4 & 5.2 & 11.6 & 18.0 \\\\\n\t\t\tDNN & DIF & 61 $\\times$ 61 & 25,448 & 4.2 & 4.9 & 16.2 & 19.1 \\\\\n\t\t\tkNN & DIF & 61 $\\times$ 61 & 25,448 & 2.8 & 4.9 & 10.8 & 19.1 \\\\\n\t\t\t\\hline\n\t\t\\end{tabular}\n\t\\end{center}\n\n\t\\label{table3}\n\\end{table}\n\n\\FloatBarrier\n\n3.1.2 Reactive transport simulation\n\\subsubsection{Reactive transport simulation}\n\\label{prob12Dres}\n\nThis section focuses on reactive transport simulations with HPx$_{\\rm py}$ within cement system 1, under both advective-dispersive and diffusive transport conditions. As written above, the domain sizes are both 61 $\\times$ 61 and 121 $\\times$ 121 for the advection-dispersion case and, because of computational constraints, solely 61 $\\times$ 61  for the diffusion case. In addition, the simulation time period is 2 years for the advection-dispersion case and 1 year for the diffusion case. Figures \\ref{fig3} and \\ref{fig4} present times series of original and emulated Ca, Si, H and O concentrations at 5 locations within the 2D domain for advective-dispersive transport conditions, for both our kNN-based (HPx$_{\\rm py}$-kNN) and DNN-based (HPx$_{\\rm py}$-DNN) reactive transport codes. It is seen that HPx$_{\\rm py}$-kNN and HPx$_{\\rm py}$-DNN both induce a quite good simulation accuracy. Also, the results for the diffusive transport case are of similarly good quality (not shown). Figures \\ref{fig5} - \\ref{fig6} provide more insights into the HPx$_{\\rm py}$-kNN and HPx$_{\\rm py}$-DNN performances by displaying 2D Ca, Si, H and O concentration profiles at a given time. For each experiment and chemical component, this time is selected as to be well representative of the simulated dynamics. It is observed that the original and emulated images are visually almost indistinguishable for the advection-dispersion case (Figure \\ref{fig5}). For the diffusion case, the emulators also perform quite well for Ca, H and O (Figures\\ref{fig6}a - c, g - i and j - l), while some slight to moderate discrepancies appear at the concentration front for Si (Figures \\ref{fig6}d - f). Notwithstanding, the Si concentration remains globally well predicted. Furthermore, Figures \\ref{fig7} - \\ref{fig8} present the original and emulated 2D solid amount profiles corresponding to Figures \\ref{fig5} - \\ref{fig6}. The original solid amount profiles are overall well approximated by HPx$_{\\rm py}$-kNN and HPx$_{\\rm py}$-DNN for the advection-dispersion case (Figure \\ref{fig7}), even though some mismatch appears at the border of the fully depleted zone for the H component. As of the diffusion case (Figure \\ref{fig8}), the same kind of mismatch is observed for the emulated solid amounts of H by HPx$_{\\rm py}$-kNN while the emulated profiles by HPx$_{\\rm py}$-DNN show some more discrepancies. Though we decided to present raw emulation results, we would like to stress that some if not all of the observed artifacts could likely be smoothed out by using some post-filtering such as median filtering.\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure3.png}\n\t\\caption{Time series of original (RTM, solid green lines) and HPx$_{\\rm py}$-kNN emulated (TM+kNN, dashed orange lines) concentrations (mol/kg) of Ca, H, O and Si at selected observations points for cement system 1 and advective-dispersive transport. Obs. 1 - 5 denote the selected observation points, with the following $\\left[x,y\\right]$ locations (in cm). Obs. 1:  $\\left[0.5, 2.5\\right]$, Obs. 2: $\\left[1, 2\\right]$, Obs. 3: $\\left[2, 2\\right]$, Obs. 4: $\\left[1, 1\\right]$, Obs. 5: $\\left[2, 1\\right]$. The results for the 121 $\\times$ 121 grid size are rather similar.}\n\t\\label{fig3}\n\\end{figure}\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure4.png}\n\t\\caption{Time series of original (RTM, solid green lines) and HPx$_{\\rm py}$-DNN emulated (TM+DNN, dashed orange lines) concentrations (mol/kg) of Ca, H, O and Si at selected observations points for cement system 1 and advective-dispersive transport. Obs. 1 - 5 denote the selected observation points, with the following $\\left[x,y\\right]$ locations (in cm). Obs. 1:  $\\left[0.5, 2.5\\right]$, Obs. 2: $\\left[1, 2\\right]$, Obs. 3: $\\left[2, 2\\right]$, Obs. 4: $\\left[1, 1\\right]$, Obs. 5: $\\left[2, 1\\right]$. The results for the 121 $\\times$ 121 grid size are rather similar.}\n\t\\label{fig4}\n\\end{figure}\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure5.png}\n\t\\caption{2D concentration profiles obtained for cement system 1 at the end of the 2-year simulation performed for the advection-dispersion case. RTM means the original HP$_{\\rm{4C}}$ model, TM+kNN denotes the Hydrus transport model coupled with our kNN geochemical emulator (HPx$_{\\rm py}$-kNN) and TM+DNN signifies the Hydrus transport model coupled with our DNN geochemical emulator (HPx$_{\\rm py}$-DNN). The first to fourth row present profiles for $Ca^{\\rm{conc}}$, $Si^{\\rm{conc}}$, $H^{\\rm{conc}}$, and $O^{\\rm{conc}}$, respectively. The considered grid size is 61 $\\times$ 61. The results for the 121 $\\times$ 121 grid are rather similar.}\n\t\\label{fig5}\n\\end{figure}\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure6.png}\n\t\\caption{2D concentration profiles obtained for cement system 1 at the final time step of the 1-year simulation performed for the diffusion case. RTM means the original HP$_{\\rm{4C}}$ model, TM+kNN denotes the Hydrus transport model coupled with our kNN geochemical emulator (HPx$_{\\rm py}$-kNN) and TM+DNN signifies the Hydrus transport model coupled with our DNN geochemical emulator (HPx$_{\\rm py}$-DNN). The first to fourth row present profiles for $Ca^{\\rm{conc}}$, $Si^{\\rm{conc}}$, $H^{\\rm{conc}}$, and $O^{\\rm{conc}}$, respectively. The considered grid size is 61 $\\times$ 61.}\n\t\\label{fig6}\n\\end{figure}\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure7.png}\n\t\\caption{2D solid amount profiles obtained for cement system 1 at the end of the 2-year simulation performed for the advection-dispersion case. RTM means the original HP$_{\\rm{4C}}$ model, TM+kNN denotes the Hydrus transport model coupled with our kNN geochemical emulator (HPx$_{\\rm py}$-kNN) and TM+DNN signifies the Hydrus transport model coupled with our DNN geochemical emulator (HPx$_{\\rm py}$-DNN). The first to fourth row present profiles for $Ca^{\\rm{solid}}$, $Si^{\\rm{solid}}$, $H^{\\rm{solid}}$, and $O^{\\rm{solid}}$, respectively. The considered grid size is 61 $\\times$ 61. The results for the 121 $\\times$ 121 grid are rather similar.}\n\t\\label{fig7}\n\\end{figure}\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure8.png}\n\t\\caption{2D solid amount profiles obtained for cement system 1 at the final time step of the 1-year simulation performed for the diffusion case. RTM means the original HP$_{\\rm{4C}}$ model, TM+kNN denotes the Hydrus transport model coupled with our kNN geochemical emulator (HPx$_{\\rm py}$-kNN) and TM+DNN signifies the Hydrus transport model coupled with our DNN geochemical emulator (HPx$_{\\rm py}$-DNN). The first to fourth row present profiles for $Ca^{\\rm{solid}}$, $Si^{\\rm{solid}}$, $H^{\\rm{solid}}$, and $O^{\\rm{solid}}$, respectively. The considered grid size is 61 $\\times$ 61.}\n\t\\label{fig8}\n\\end{figure}\n\nThe speedups associated with the considered problem are detailed in Table \\ref{table3}. It is noted that the GPU-based DNN emulator allows for a speedup that is close to optimal. Indeed, The DNN speedups overall represent 85 \\% to 95 \\% of the maximum possible speedups (that is, speedups that would be obtained if the geochemcical calculations would come at no cost at all). The speedups associated with single-threaded kNN remain substantial but only amount to 57 \\% - 65 \\% of the corresponding maximum speedups. As detailed in section \\ref{train_res1}, the used kNN and DNN implementations are found to be respectively 300 and 4000 faster than single-threaded PHREEQC when predicting 10,000 points all one once for this geochemical system. Based on these numbers one could have expected the achieved speedups to represent say 90 \\% (kNN) or 99 \\% (DNN) of the maximum possible ones. A large part of the gaps between achieved and maximum possible speedups is thus likely caused by the time required for communicating and exchanging data between the main C/C++ code and the Python-based emulators.\n\n\\begin{table}[h!]\n\t\\caption{Speedups offered by the KNN and DNN emulators in HPx$_{\\rm py}$ for the reactive transport simulations considered for cement system 1. The HPx$_{\\rm{4C}}$ calculations involve the parallelization of PHREEQC over our 4 CPUs. The HPx$_{\\rm{1C}}$ calculations are performed on a single CPU. The kNN predictions are performed on a single CPU using the scikit-learn implementation while the DNN predictions make use of our GPU. ML signifies the used machine learning method for emulation, TC denotes transport conditions (ADV: advection-dispersion, DIF: diffusion) and GS is the grid size. The maximum possible speedups associated with HPx$_{\\rm{4C}}$ and HPx$_{\\rm{1C}}$,  Max SP HPx$_{\\rm{4C}}$ and  Max SP HPx$_{\\rm{1C}}$, correspond to an hypothetical situation where the geochemical calculations incur zero computational cost.}\n\n\t\\begin{center}\n\t\t\\begin{tabular}{cccccccc}\n\t\t\t\\hline\n\t\t\t& & & & & & & \\\\\n\t\t\tML & TC & GS & HPx$_{\\rm{4C}}$ time (s) & SP HPx$_{\\rm{4C}}$ & Max SP HPx$_{\\rm{4C}}$ & SP HPx$_{\\rm{1C}}$ & Max SP HPx$_{\\rm{1C}}$ \\\\\n\t\t\tDNN & ADV & 61 $\\times$ 61 & 3189 & 6.8 & 7.7 & 24.5 & 28.5 \\\\\n\t\t\tkNN & ADV & 61 $\\times$ 61 & 3189 & 5.0 & 7.7 & 18.0 & 28.5 \\\\\n\t\t\tDNN & ADV & 121 $\\times$ 121 & 23,337 & 5.0 & 5.2 & 17.0 & 18.0 \\\\\n\t\t\tkNN & ADV & 121 $\\times$ 121 & 23,337 & 3.4 & 5.2 & 11.6 & 18.0 \\\\\n\t\t\tDNN & DIF & 61 $\\times$ 61 & 25,448 & 4.2 & 4.9 & 16.2 & 19.1 \\\\\n\t\t\tkNN & DIF & 61 $\\times$ 61 & 25,448 & 2.8 & 4.9 & 10.8 & 19.1 \\\\\n\t\t\t\\hline\n\t\t\\end{tabular}\n\t\\end{center}\n\n\t\\label{table3}\n\\end{table}\n\n\\FloatBarrier\n\n",
        "label": "fig5",
        "Descriptive_question1": "What is the grid size used for the 2D concentration profiles shown in figure_5?",
        "Descriptive_question2": "Which chemical component is presented in the second row of figure_5?",
        "Reasoning_question1": "How do the concentration profiles of Ca and Si in figure_5 compare between the original RTM model and the TM+kNN emulator, and what might this suggest about the accuracy of the kNN emulator for advection-dispersion conditions?",
        "Reasoning_question2": "What trends or differences can be observed in the 2D concentration profiles of H and O across the different models in figure_5, and what could be the potential reasons for any discrepancies seen in the emulated results?",
        "Descriptive_answer1": "61 \\times 61",
        "Descriptive_answer2": "Si^{\\rm{conc}}",
        "Reasoning_answer1": "In figure_5, the concentration profiles for Ca and Si show that the images for the original RTM and the TM+kNN emulator are visually almost indistinguishable under the advection-dispersion case. This close visual similarity suggests that the kNN emulator is quite accurate in reproducing the spatial distribution of Ca and Si concentrations as predicted by the original HP_4C model under these transport conditions.",
        "Reasoning_answer2": "The 2D concentration profiles for H and O across the RTM, TM+kNN, and TM+DNN models in figure_5 appear to be very similar, with the emulated results closely matching the original RTM profiles. Minor mismatches, such as slight deviations at the border of the depleted zone for H in the solid amount profiles, could be due to limitations in the emulators' approximations or numerical artifacts. These discrepancies might result from the interpolation methods or computational communication overheads between the main code and Python-based emulators, but overall the emulators produce highly accurate predictions."
    },
    {
        "paper_id": "2107.07598.json",
        "image_id": "figure_6",
        "image_path": [
            "/home/yz979/scratch/chengye/scimolmo/test-images-tables/dataset_imagestables/2107.07598/images/Figure6.png"
        ],
        "caption": "2D concentration profiles obtained for cement system 1 at the final time step of the 1-year simulation performed for the diffusion case. RTM means the original HP$_{\\rm{4C}}$ model, TM+kNN denotes the Hydrus transport model coupled with our kNN geochemical emulator (HPx$_{\\rm py}$-kNN) and TM+DNN signifies the Hydrus transport model coupled with our DNN geochemical emulator (HPx$_{\\rm py}$-DNN). The first to fourth row present profiles for $Ca^{\\rm{conc}}$, $Si^{\\rm{conc}}$, $H^{\\rm{conc}}$, and $O^{\\rm{conc}}$, respectively. The considered grid size is 61 $\\times$ 61.",
        "classify": "Chart",
        "section_info": "3 Results\n\\section{Results}\n\\label{results}\n\n\\subsection{Ca-Si Problem}\nFor this first cement system, the emulation problem consists of predicting at each time step of the RT simulation the (output) Ca, Si, H and O aqueous concentrations (mol/kg of water or mol/kgw) from the (input) total amounts of Ca and Si (mol). \n\n\\subsubsection{Training the emulators}\n\\label{train_res1}\nHere the kNN and DNN emulators are firstly trained using a set of 400,000 test examples for both. This training set is obtained by randomly sampling the two-dimensional input space by latin hypercube sampling (LHS) between $\\left[0,0\\right]$ and $\\rm{\\left[Ca^{tot}_{max},Si^{tot}_{max}\\right]}$, and running PHREEQC for each input sample, $\\rm{\\textbf{x}_i = \\left[Ca^{tot}_i, Si^{tot}_i\\right]}$ to get the corresponding output vectors, $\\rm{\\textbf{y}_i = \\left[Ca^{conc}_i,Si^{conc}_i,H^{conc}_i,O^{conc}_i\\right]}$. The upper bounds, $\\rm{Ca^{tot}_{max}}$ and $\\rm{Si^{tot}_{max}}$ are defined based on a cheap full RT simulation with advective-dispersive transport using a small 1D domain of 51 nodes. It is worth noting that the total amounts of $\\textbf{x}_i$, corresponding to the PHREEQC-simulated concentrations, $\\textbf{y}_i$, have to be corrected for the different amount of water between the training set and the transport simulations. Doing so, it turns out that about 20 \\% of the post-corrected $x_i$ values exceed their pre-defined upper bounds and these excessively large values need to be filtered out. Creating the 400,000 training examples thus required about 500,000 PHREEQC runs. As stated earlier, for this problem single-threaded PHREEQC performs about 670 geochemical calculations per second on our used Intel\\textsuperscript{\\textregistered} i7 CPU.\n\nWith respect to kNN, the tuning parameters are the number of neighbors, $k$, the type of distance measure, and the interpolation technique. We simply used the default settings: $k = 5$, euclidean distance and inverse-distance interpolation. Regarding training of the DNN, the 400,000 sample were split between the training set itself (90 \\% of the data) and a validation set (10 \\% of the data). The latter serves to monitor the evolution of the selected mean squared error loss function on samples that are not used for training, thereby detecting potential overfitting. If the validation loss stops decreasing before the fixed number of epochs has been completed, then  training is stopped. Importantly, the emulation is achieved in log-space for both the input, $X$ and output $Y$, domains. This because total amounts and concentrations of the involved components typically cover many orders of magnitudes (up to 10 orders or more). Using a DNN also requires some form of data normalization or standardization. Here both the $\\rm{log\\left(\\textbf{x}_i\\right)}$ and $\\rm{log\\left(\\textbf{y}_i\\right)}$ vectors are standardized around 0 with standard deviation of 1.\n\nFigure \\ref{fig1} illustrates the trained emulators' performance for geochemical predictions using an independent test set that comprises 10,000 test examples. Both kNN (Figures \\ref{fig1}a - d) and DNN (Figures \\ref{fig1}e - h) appear to be rather accurate . DNN also shows a slight degradation for the larger concentration values  (Figures \\ref{fig1}e - h). The latter is likely due to the combination of a small proportion of large concentration values in the training set with the log-transformation that implicitly pushes the DNN to try harder to fit the smaller concentrations during training. Regarding speedup and as written earlier, for this setup the single-threaded kNN method is 300 times faster than single-threaded PHREEQC for predicting the 10,000 concentration vectors all at once. The computational savings allowed by the DNN emulator when ran on our GPU is higher with a speedup as large as 4000 for predicting the same 10,000 concentration vectors all at once. \n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{0cm}\\includegraphics[width=35pc]{Figure1.png}\n\t\\caption{1-1 plots of the kNN (subfigures (a) - (d)) and DNN (subfigures (e) - (h)) emulators' performance obtained for system 1 when the kNN training base contains 400,000 samples and the DNN is trained using the same 400,000 samples. Here ``true\" means the original PHREEQC-simulated data and ``predicted\" denotes the emulated (that is, kNN-simulated and DNN-simulated) data. Hence, the $x$-axis and $y$-axis present the original and emulated 10,000 independent test data points, respectively. The RMSE and $Q_2$ coefficient denote the root-mean-square-error and coefficient of determination in testing mode, respectively, between the original and emulated 10,000 test data points.}\n\t\\label{fig1}\n\\end{figure}\n\nTo test the sensitivity of the emulators' performance to the training set size, training was also performed using reduced training sets comprising 100,000, 10,000, 1000 and 100 samples, respectively. It seen that the DNN performance achieved  when using 10,000 training samples is virtually the same as that obtained when using 400,000 training samples (Figure \\ref{fig2} and Table \\ref{table2}). It is only for training sets smaller than 1000 samples that the DNN performance starts to degrade significantly (Table \\ref{table2}). In contrast, the behavior of kNN appears to markedly decrease as the training set gets smaller (Figure \\ref{fig2} and Table \\ref{table1}).\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{0cm}\\includegraphics[width=35pc]{Figure2.png}\n\t\\caption{1-1 plots of the kNN (subfigures (a) - (d)) and DNN (subfigures (e) - (h)) emulators' performance obtained for system 1 when the kNN training base contains 10,000 samples and the DNN is trained using the same 10,000 samples. Here ``true\" means the original PHREEQC-simulated data and ``predicted\" denotes the emulated (that is, kNN-simulated and DNN-simulated) data. Hence, the $x$-axis and $y$-axis present the original and emulated 10,000 independent test data points, respectively. The RMSE and $Q_2$ coefficient denote the root-mean-square-error and coefficient of determination in testing mode, respectively, between the original and emulated 10,000 test data points.}\n\t\\label{fig2}\n\\end{figure}\n\n\\begin{table}[h!]\n\t\\caption{Performance of the DNN and kNN emulators for cement system 1 and different training set sizes. For brevity, only the results for Ca$^{conc}$ and Si$^{conc}$ are shown. The units are mol per kg of water (mol/kgw). ML refers to the type of emulator, TR signifies the size of the training set, and the RMSE and $Q_2$ coefficient denote the root-mean-square-error and coefficient of determination in testing mode, respectively, between the original and emulated 10,000 test data points.}\n\t\\begin{center}\n\t\t\\begin{tabular}{cccccccc}\n\t\t\t\\hline\n\t\t\t& & & & & \\\\\n\t\t\tML & TR & RMSE - Ca$^{conc}$ & $Q_2$ - Ca$^{conc}$ & RMSE - Si$^{conc}$ & $Q_2$ - Si$^{conc}$\\\\\n\t\t\tDNN & 4 $\\times$ 10$^{5}$ & 4.50 $\\times$ 10$^{-5}$ & 0.9999 & 1.21 $\\times$ 10$^{-6}$ & 0.9999\\\\\n\t\t\tDNN & 1 $\\times$ 10$^{5}$ & 5.06 $\\times$ 10$^{-5}$ & 0.9999 & 1.30 $\\times$ 10$^{-6}$ & 0.9999\\\\\n\t\t\tDNN & 1 $\\times$ 10$^{4}$ & 4.57 $\\times$ 10$^{-5}$ & 0.9999 & 1.48 $\\times$ 10$^{-6}$ & 0.9999\\\\\n\t\t\tDNN & 1 $\\times$ 10$^{3}$ & 8.02 $\\times$ 10$^{-5}$ & 0.9998 & 3.31 $\\times$ 10$^{-6}$ & 0.9994\\\\\n\t\t\tDNN & 1 $\\times$ 10$^{2}$ & 47.1 $\\times$ 10$^{-5}$ & 0.9935 & 9.02 $\\times$ 10$^{-6}$ & 0.9959\\\\\n\t\t\tkNN & 4 $\\times$ 10$^{5}$ & 2.52 $\\times$ 10$^{-5}$ & 1.0000 & 1.59 $\\times$ 10$^{-6}$ & 0.9999\\\\\n\t\t\tkNN & 1 $\\times$ 10$^{5}$ & 6.27 $\\times$ 10$^{-5}$ & 0.9999 & 4.20 $\\times$ 10$^{-6}$ & 0.9991\\\\\n\t\t\tkNN & 1 $\\times$ 10$^{4}$ & 14.1 $\\times$ 10$^{-5}$ & 0.9994 & 13.5 $\\times$ 10$^{-6}$ & 0.9908\\\\\n\t\t\tkNN & 1 $\\times$ 10$^{3}$ & 43.9 $\\times$ 10$^{-5}$ & 0.9944 & 74.6 $\\times$ 10$^{-6}$ & 0.7180\\\\\n\t\t\tkNN & 1 $\\times$ 10$^{2}$  & 96.4 $\\times$ 10$^{-5}$ & 0.9729 & 55.0 $\\times$ 10$^{-6}$ & 0.8468\\\\\n\t\t\\end{tabular}\n\t\\end{center}\n\t\\label{table2}\n\\end{table}\n\n\\FloatBarrier\n\n\\subsubsection{Reactive transport simulation}\n\\label{prob12Dres}\n\nThis section focuses on reactive transport simulations with HPx$_{\\rm py}$ within cement system 1, under both advective-dispersive and diffusive transport conditions. As written above, the domain sizes are both 61 $\\times$ 61 and 121 $\\times$ 121 for the advection-dispersion case and, because of computational constraints, solely 61 $\\times$ 61  for the diffusion case. In addition, the simulation time period is 2 years for the advection-dispersion case and 1 year for the diffusion case. Figures \\ref{fig3} and \\ref{fig4} present times series of original and emulated Ca, Si, H and O concentrations at 5 locations within the 2D domain for advective-dispersive transport conditions, for both our kNN-based (HPx$_{\\rm py}$-kNN) and DNN-based (HPx$_{\\rm py}$-DNN) reactive transport codes. It is seen that HPx$_{\\rm py}$-kNN and HPx$_{\\rm py}$-DNN both induce a quite good simulation accuracy. Also, the results for the diffusive transport case are of similarly good quality (not shown). Figures \\ref{fig5} - \\ref{fig6} provide more insights into the HPx$_{\\rm py}$-kNN and HPx$_{\\rm py}$-DNN performances by displaying 2D Ca, Si, H and O concentration profiles at a given time. For each experiment and chemical component, this time is selected as to be well representative of the simulated dynamics. It is observed that the original and emulated images are visually almost indistinguishable for the advection-dispersion case (Figure \\ref{fig5}). For the diffusion case, the emulators also perform quite well for Ca, H and O (Figures\\ref{fig6}a - c, g - i and j - l), while some slight to moderate discrepancies appear at the concentration front for Si (Figures \\ref{fig6}d - f). Notwithstanding, the Si concentration remains globally well predicted. Furthermore, Figures \\ref{fig7} - \\ref{fig8} present the original and emulated 2D solid amount profiles corresponding to Figures \\ref{fig5} - \\ref{fig6}. The original solid amount profiles are overall well approximated by HPx$_{\\rm py}$-kNN and HPx$_{\\rm py}$-DNN for the advection-dispersion case (Figure \\ref{fig7}), even though some mismatch appears at the border of the fully depleted zone for the H component. As of the diffusion case (Figure \\ref{fig8}), the same kind of mismatch is observed for the emulated solid amounts of H by HPx$_{\\rm py}$-kNN while the emulated profiles by HPx$_{\\rm py}$-DNN show some more discrepancies. Though we decided to present raw emulation results, we would like to stress that some if not all of the observed artifacts could likely be smoothed out by using some post-filtering such as median filtering.\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure3.png}\n\t\\caption{Time series of original (RTM, solid green lines) and HPx$_{\\rm py}$-kNN emulated (TM+kNN, dashed orange lines) concentrations (mol/kg) of Ca, H, O and Si at selected observations points for cement system 1 and advective-dispersive transport. Obs. 1 - 5 denote the selected observation points, with the following $\\left[x,y\\right]$ locations (in cm). Obs. 1:  $\\left[0.5, 2.5\\right]$, Obs. 2: $\\left[1, 2\\right]$, Obs. 3: $\\left[2, 2\\right]$, Obs. 4: $\\left[1, 1\\right]$, Obs. 5: $\\left[2, 1\\right]$. The results for the 121 $\\times$ 121 grid size are rather similar.}\n\t\\label{fig3}\n\\end{figure}\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure4.png}\n\t\\caption{Time series of original (RTM, solid green lines) and HPx$_{\\rm py}$-DNN emulated (TM+DNN, dashed orange lines) concentrations (mol/kg) of Ca, H, O and Si at selected observations points for cement system 1 and advective-dispersive transport. Obs. 1 - 5 denote the selected observation points, with the following $\\left[x,y\\right]$ locations (in cm). Obs. 1:  $\\left[0.5, 2.5\\right]$, Obs. 2: $\\left[1, 2\\right]$, Obs. 3: $\\left[2, 2\\right]$, Obs. 4: $\\left[1, 1\\right]$, Obs. 5: $\\left[2, 1\\right]$. The results for the 121 $\\times$ 121 grid size are rather similar.}\n\t\\label{fig4}\n\\end{figure}\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure5.png}\n\t\\caption{2D concentration profiles obtained for cement system 1 at the end of the 2-year simulation performed for the advection-dispersion case. RTM means the original HP$_{\\rm{4C}}$ model, TM+kNN denotes the Hydrus transport model coupled with our kNN geochemical emulator (HPx$_{\\rm py}$-kNN) and TM+DNN signifies the Hydrus transport model coupled with our DNN geochemical emulator (HPx$_{\\rm py}$-DNN). The first to fourth row present profiles for $Ca^{\\rm{conc}}$, $Si^{\\rm{conc}}$, $H^{\\rm{conc}}$, and $O^{\\rm{conc}}$, respectively. The considered grid size is 61 $\\times$ 61. The results for the 121 $\\times$ 121 grid are rather similar.}\n\t\\label{fig5}\n\\end{figure}\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure6.png}\n\t\\caption{2D concentration profiles obtained for cement system 1 at the final time step of the 1-year simulation performed for the diffusion case. RTM means the original HP$_{\\rm{4C}}$ model, TM+kNN denotes the Hydrus transport model coupled with our kNN geochemical emulator (HPx$_{\\rm py}$-kNN) and TM+DNN signifies the Hydrus transport model coupled with our DNN geochemical emulator (HPx$_{\\rm py}$-DNN). The first to fourth row present profiles for $Ca^{\\rm{conc}}$, $Si^{\\rm{conc}}$, $H^{\\rm{conc}}$, and $O^{\\rm{conc}}$, respectively. The considered grid size is 61 $\\times$ 61.}\n\t\\label{fig6}\n\\end{figure}\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure7.png}\n\t\\caption{2D solid amount profiles obtained for cement system 1 at the end of the 2-year simulation performed for the advection-dispersion case. RTM means the original HP$_{\\rm{4C}}$ model, TM+kNN denotes the Hydrus transport model coupled with our kNN geochemical emulator (HPx$_{\\rm py}$-kNN) and TM+DNN signifies the Hydrus transport model coupled with our DNN geochemical emulator (HPx$_{\\rm py}$-DNN). The first to fourth row present profiles for $Ca^{\\rm{solid}}$, $Si^{\\rm{solid}}$, $H^{\\rm{solid}}$, and $O^{\\rm{solid}}$, respectively. The considered grid size is 61 $\\times$ 61. The results for the 121 $\\times$ 121 grid are rather similar.}\n\t\\label{fig7}\n\\end{figure}\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure8.png}\n\t\\caption{2D solid amount profiles obtained for cement system 1 at the final time step of the 1-year simulation performed for the diffusion case. RTM means the original HP$_{\\rm{4C}}$ model, TM+kNN denotes the Hydrus transport model coupled with our kNN geochemical emulator (HPx$_{\\rm py}$-kNN) and TM+DNN signifies the Hydrus transport model coupled with our DNN geochemical emulator (HPx$_{\\rm py}$-DNN). The first to fourth row present profiles for $Ca^{\\rm{solid}}$, $Si^{\\rm{solid}}$, $H^{\\rm{solid}}$, and $O^{\\rm{solid}}$, respectively. The considered grid size is 61 $\\times$ 61.}\n\t\\label{fig8}\n\\end{figure}\n\nThe speedups associated with the considered problem are detailed in Table \\ref{table3}. It is noted that the GPU-based DNN emulator allows for a speedup that is close to optimal. Indeed, The DNN speedups overall represent 85 \\% to 95 \\% of the maximum possible speedups (that is, speedups that would be obtained if the geochemcical calculations would come at no cost at all). The speedups associated with single-threaded kNN remain substantial but only amount to 57 \\% - 65 \\% of the corresponding maximum speedups. As detailed in section \\ref{train_res1}, the used kNN and DNN implementations are found to be respectively 300 and 4000 faster than single-threaded PHREEQC when predicting 10,000 points all one once for this geochemical system. Based on these numbers one could have expected the achieved speedups to represent say 90 \\% (kNN) or 99 \\% (DNN) of the maximum possible ones. A large part of the gaps between achieved and maximum possible speedups is thus likely caused by the time required for communicating and exchanging data between the main C/C++ code and the Python-based emulators.\n\n\\begin{table}[h!]\n\t\\caption{Speedups offered by the KNN and DNN emulators in HPx$_{\\rm py}$ for the reactive transport simulations considered for cement system 1. The HPx$_{\\rm{4C}}$ calculations involve the parallelization of PHREEQC over our 4 CPUs. The HPx$_{\\rm{1C}}$ calculations are performed on a single CPU. The kNN predictions are performed on a single CPU using the scikit-learn implementation while the DNN predictions make use of our GPU. ML signifies the used machine learning method for emulation, TC denotes transport conditions (ADV: advection-dispersion, DIF: diffusion) and GS is the grid size. The maximum possible speedups associated with HPx$_{\\rm{4C}}$ and HPx$_{\\rm{1C}}$,  Max SP HPx$_{\\rm{4C}}$ and  Max SP HPx$_{\\rm{1C}}$, correspond to an hypothetical situation where the geochemical calculations incur zero computational cost.}\n\n\t\\begin{center}\n\t\t\\begin{tabular}{cccccccc}\n\t\t\t\\hline\n\t\t\t& & & & & & & \\\\\n\t\t\tML & TC & GS & HPx$_{\\rm{4C}}$ time (s) & SP HPx$_{\\rm{4C}}$ & Max SP HPx$_{\\rm{4C}}$ & SP HPx$_{\\rm{1C}}$ & Max SP HPx$_{\\rm{1C}}$ \\\\\n\t\t\tDNN & ADV & 61 $\\times$ 61 & 3189 & 6.8 & 7.7 & 24.5 & 28.5 \\\\\n\t\t\tkNN & ADV & 61 $\\times$ 61 & 3189 & 5.0 & 7.7 & 18.0 & 28.5 \\\\\n\t\t\tDNN & ADV & 121 $\\times$ 121 & 23,337 & 5.0 & 5.2 & 17.0 & 18.0 \\\\\n\t\t\tkNN & ADV & 121 $\\times$ 121 & 23,337 & 3.4 & 5.2 & 11.6 & 18.0 \\\\\n\t\t\tDNN & DIF & 61 $\\times$ 61 & 25,448 & 4.2 & 4.9 & 16.2 & 19.1 \\\\\n\t\t\tkNN & DIF & 61 $\\times$ 61 & 25,448 & 2.8 & 4.9 & 10.8 & 19.1 \\\\\n\t\t\t\\hline\n\t\t\\end{tabular}\n\t\\end{center}\n\n\t\\label{table3}\n\\end{table}\n\n\\FloatBarrier\n\n\\subsection{Al-C-Ca-S-Si Problem}\n\nFor this second cement system, the emulation problem consists of predicting at each time step of the RT simulation the (output) Al, C, Ca, S, Si, H and O aqueous concentrations (mol/kgw) from the (input) total amounts of Al, C, Ca, S and Si (mol). Here we focus on advective-dispersive transport only while similarly as for cement system 1, the considered domain sizes are 61 $\\times$ 61 and 121 $\\times$ 121. Moreover, the simulation time period is set to 6 years. \n\nAs mentioned earlier, for this higher-dimensional problem it is observed that the used scikit-learn kNN implementation becomes prohibitively slow compared to HPx$_{\\rm{4C}}$. We found that to get a good emulation accuracy the kNN training base needs to contain 1,000,000 samples (or more). This training base's size together with a 5-dimensional search space leads to an HPx$_{\\rm py}$-kNN reactive transport simulation time that is comparable to that of HPx$_{\\rm{4C}}$. Therefore, for this second cement system we built a custom kNN regressor around another kNN implementation contained in the FAISS package \\citep{faiss2017}. Our used FAISS variant allows for GPU computing and is much faster than scikit-learn for this cement system, but is slightly less accurate due to the use of an approximate rather than exact nearest neighbor search \\citep[see][for details]{faiss2017}.\n\n\\subsubsection{Training the DNN}\n\\label{train_res2}\n\nBuilding a good training set to perform a kNN search and learn the weights and biases of our DNN turned out to be a complicated task in this case. This because to make useful kNN predictions and/or learn an useful DNN, the training set must be sufficiently representative of the geochemical conditions encountered during the reactive transport simulation one wish to perform with HPx$_{\\rm py}$-kNN and the trained HPx$_{\\rm py}$-DNN. In contrast to cement system 1, creating the training set by sampling the $X$-space with a controlled randomness between predefined lower and upper bounds did not prove successful. We tried that strategy by drawing as much as 4,000,000 5-dimensional $\\textbf{x}$ vectors from the $X$-space using a Sobol low-discrepancy sequence \\citep[][]{Sobol1967, Joe-Kuo2003}. Such low-discrepancy sampling scheme covers the 5-dimensional hypercube more uniformly than LHS. Despite a good performance on the test set (not shown), the resulting DNN accuracy in reactive transport mode was never deemed satisfying. In other words, no satisfying ``global\" or ``universal\" DNN emulator could be devised for this cement system. This is probably caused by the fact for this problem, the input (5 total amounts) and output (7 aqueous concentrations) spaces are quite nonlinearly related and both cover 6 to 10 orders of magnitudes depending on the considered element. Therefore, we resorted to the alternative training strategy detailed below. The latter basically tries to grasp the complex correlations and higher-order dependencies that exist between the elements of $\\textbf{x}$ (total amounts, input space) for a given reactive transport simulation setup, in order to produce a training set that honors these between-input relationships.\n\n\\begin{itemize}\n\t\\item Perform a ``cheap\" full reactive transport simulation under the transport conditions and geochemistry of interest and collect the resulting $\\textbf{x}$ -$\\textbf{y}$  pairs of examples (for the considered grid nodes and time steps). Computational demand controls what domain size and simulation time period can be used for this cheap calculation. We used a modest 16 $\\times$ 16 domain and a simulation time period of 10 years. The associated HPx$_{\\rm{4C}}$ runtimes is 180 s.\n\t\n\t\\item Fit a kernel density estimator (KDE) with a Gaussian kernel to the collected $\\textbf{x}$ vectors (encapsulated in the $\\textbf{X}$ array) and generate a fixed number of new input vectors, $\\textbf{x}_{KDE}$. Then run PHREEQC for the $\\textbf{X}_{KDE}$ set to get the corresponding output set, $\\textbf{Y}_{KDE}$. Now apply the correction for porosity described in section \\ref{train_res1} to the $\\textbf{X}_{KDE}$ set and form the training set by merging the ensemble of $\\textbf{x}$-$\\textbf{y}$ pairs with that of the $\\textbf{x}_{KDE}$-$\\textbf{y}_{KDE}$ pairs. The number of produced unique examples by the considered cheap HPx$_{\\rm{4C}}$ simulations varied between 10,000 and 50,000. The KDE-based enrichment of this dataset was deemed necessary to provide more input variability, thereby avoiding overfitting of the trained DNN and improving the kNN accuracy, while still honoring the complex between-input relationships. The number of KDE-generated samples was set as to obtain a total training set size of 1,000,000 examples. A key component of the approach is the bandwith parameter of the Gaussian KDE kernel which controls how much the KDE-generated samples depart from the original ensemble. After limited trial and error, we fixed the kernel bandwith to 0.0025 for the considered case studies.\n\t\n\\end{itemize}\n\nThe scatter plots in Figure \\ref{fig9} illustrate our training set creation procedure. The orange dots depict the pairwise relationships observed between the 5 elements of $\\textbf{x}$ in the cheap simulation. The red and cyan dots in Figure \\ref{fig9} represent the KDE-generated samples with the selected bandwidth, before and after applying the correction for porosity, respectively. Training of the DNN is achieved using the ensemble of original and KDE-corrected input points. We refer to this kind of dataset as RT-based, since it is based on an albeit cheap, full RT simulation. Furthermore, we refer to the obtained DNN and kNN emulators as ``local\" emulators, since as opposed to the emulators constructed for cement system 1, the current emulators are only valid for the input conditions encapsulated in the RT-based training set.\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1.5cm}\\includegraphics[width=50pc]{Figure9.png}\n\t\\caption{Scatter plots of the complex relationships between the five considered inputs for (1) the computationally cheap RT simulation of cement system 2 (orange dots) performed with the original HPx code, (2) the corresponding sampled points by kernel density estimation (KDE) using the selected bandwith (turquoise dots) and, (3) the same sampled KDE points after porosity correction (red dots).}\n\t\\label{fig9}\n\\end{figure}\n\nTraining performance of the DNN emulator is presented in Figure \\ref{fig10}. Similarly as for cement system 1, about 90\\% of the available data was used for the actual training of the DNN while the remaining 10\\% were used a validation set to contol overfitting. Lastly, performance is evaluated for both the trained DNN and kNN emulators using an independent test set of 10,000 examples. Overall, the accuracy of our ``local \" DNN emulator for this RT-inspired dataset is rather large with $Q_2$ values always greater or equal than 0.998. Training performance of the corresponding local kNN emulator is equally good (not shown). Regarding speedup, for this problem single-threaded PHREEQC achieves about 210 geochemical calculations per second on our used Intel\\textsuperscript{\\textregistered} i7 CPU while the GPU-based DNN and kNN emulators are both about 3000 times faster when predicting the 10,000 test points at once .\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure10.png}\n\t\\caption{1-1 plots of local DNN emulation performance obtained for system 2 when the local DNN is trained using 1,000,000 samples. The $x$-axis and $y$-axis present the original and emulated 10,000 independent test data points, respectively. The RMSE and $Q_2$ coefficient denote the root-mean-square-error and coefficient of determination in testing mode, respectively, between the original and emulated 10,000 test data points.}\n\t\\label{fig10}\n\\end{figure}\n\n\\subsubsection{Reactive transport simulation}\n\\label{prob2res}\nOur ``local\" DNN performs rather well when applied to the 61 $\\times$ 61 grid size and a time period of 6 years (Figures \\ref{fig11} - \\ref{fig12}). This for all components but C, for which some localized deviations appear between original and emulated 2D concentration profiles towards the end of the simulation period (Figure \\ref{fig12}). When applied to the 121 $\\times$ 121 grid, HPx$_{\\rm py}$-DNN produces additional discrepancies for O, Si and Al towards the end of the simulation period (Figures \\ref{fig13} - \\ref{fig14}). Yet most of the observed artifacts could probably be smoothed out by using post-filtering. The associated speedups are listed in Table \\ref{table4}. These speedups are larger to those obtained for cement system 1, with values between 8 and 9 when evaluated against HPx$_{\\rm{4C}}$. These speedups represent about 85 \\% to 90 \\% of the maximum possible speedup (Table \\ref{table3}). Overall, these findings indicate that for the considered problem, our RT-based training of a local DNN only works is the training set is sufficiently representative of the particular geochemical conditions encountered in the computationally demanding simulations, which is arguably not easy to achieve. This limitation is further discussed in section \\ref{discussion}.\n\nWe note a more uniform behavior for HPx$_{\\rm py}$-kNN across grid sizes than for HPx$_{\\rm py}$-DNN. Here the results for the 121  $\\times$ 121 grid are only slightly less accurate than those associated with the 61 $ \\times$ 61 grid (see Figures \\ref{fig15} - \\ref{fig16} where for brevity we only show concentration profiles for the C Al and S components). Furthermore, whenever observed the discrepancies between original and emulated profiles are more regularly scattered than for HPx$_{\\rm py}$-DNN. Note also that herein too, post-filtering could likely smooth out a large part of these deviations. In addition, owing to the use of a GPU to achieve the kNN calculations, the speedup provided by HPx$_{\\rm py}$-kNN are as large as those provided by HPx$_{\\rm py}$-DNN (Table \\ref{table4}).\n\nWith respect to the emulated solid amounts, the HPx$_{\\rm py}$-DNN results look visually good for the the 61 $\\times$ 61 grid. This is shown in Figure \\ref{fig17} for the C, Al and S chemical components, while emulation of the H, O Ca and Si chemical components is globally of similar quality (not shown). Nevertheless, significant deviations appear towards the end of the simulation period for every chemical component (see Figure \\ref{fig18} for the C, Al and S chemical components, emulation of the H, O Ca and Si chemical components shows the same level of mismatch). The HPx$_{\\rm py}$-kNN predictions are also fairly accurate for the 61 $\\times$ 61 (see Figure \\ref{fig19} for the C, Al and S chemical components, emulation of the H, O Ca and Si chemical components exhibits a globally similar quality) while some discrepancies show up at the end of the simulation (Figure \\ref{fig20}). However, the mismatch is less pronounced than for HPx$_{\\rm py}$-DNN. Overall, HPx$_{\\rm py}$-kNN appears to be somewhat more robust than HPx$_{\\rm py}$-DNN for this cement system, while providing the same (large) speedup.\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure11.png}\n\t\\caption{2D concentration profiles obtained for cement system 2 after 2 and 6 (final time step) years. RTM means the original HPx$_{\\rm{4C}}$ model and TM+DNN denotes the Hydrus transport model coupled with our DNN geochemical emulator (HPx$_{\\rm py}$-DNN). The first to fourth row present profiles for $Ca^{\\rm{conc}}$, $Si^{\\rm{conc}}$, $H^{\\rm{conc}}$, and $O^{\\rm{conc}}$, respectively. The considered grid size is 61 $\\times$ 61.}\n\t\\label{fig11}\n\\end{figure}\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure12.png}\n\t\\caption{2D concentration profiles obtained for cement system 2 after 2 and 6 (final time step) years. RTM means the original HPx$_{\\rm{4C}}$ model and TM+DNN denotes the Hydrus transport model coupled with our DNN geochemical emulator (HPx$_{\\rm py}$-DNN). The first to third row present profiles for $C^{\\rm{conc}}$, $Al^{\\rm{conc}}$, and $S^{\\rm{conc}}$, respectively. The considered grid size is 61 $\\times$ 61.}\n\t\\label{fig12}\n\\end{figure}\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure13.png}\n\t\\caption{2D concentration profiles obtained for cement system 2 after 2 and 6 (final time step) years. RTM means the original HPx$_{\\rm{4C}}$ model and TM+DNN denotes the Hydrus transport model coupled with our DNN geochemical emulator (HPx$_{\\rm py}$-DNN). The first to fourth row present profiles for $Ca^{\\rm{conc}}$, $Si^{\\rm{conc}}$, $H^{\\rm{conc}}$, and $O^{\\rm{conc}}$, respectively. The considered grid size is 121 $\\times$ 121.}\n\t\\label{fig13}\n\\end{figure}\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure14.png}\n\t\\caption{2D concentration profiles obtained for cement system 2 after 2 and 6 (final time step) years. RTM means the original HPx$_{\\rm{4C}}$ model and TM+DNN denotes the Hydrus transport model coupled with our DNN geochemical emulator (HPx$_{\\rm py}$-DNN). The first to third row present profiles for $C^{\\rm{conc}}$, $Al^{\\rm{conc}}$, and $S^{\\rm{conc}}$, respectively. The considered grid size is 121 $\\times$ 121.}\n\t\\label{fig14}\n\\end{figure}\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure15.png}\n\t\\caption{2D concentration profiles obtained for cement system 2 after 2 and 6 (final time step) years. RTM means the original HPx$_{\\rm{4C}}$ model and TM+kNN denotes the Hydrus transport model coupled with our kNN geochemical emulator (HPx$_{\\rm py}$-kNN).  The first to third row present profiles for $C^{\\rm{conc}}$, $Al^{\\rm{conc}}$, and $S^{\\rm{conc}}$, respectively. The considered grid size is 61 $\\times$ 61.}\n\t\\label{fig15}\n\\end{figure}\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure16.png}\n\t\\caption{2D concentration profiles obtained for cement system 2 after 2 and 6 (final time step) years. RTM means the original HPx$_{\\rm{4C}}$ model and TM+kNN denotes the Hydrus transport model coupled with our kNN geochemical emulator (HPx$_{\\rm py}$-kNN). The first to third row present profiles for $C^{\\rm{conc}}$, $Al^{\\rm{conc}}$, and $S^{\\rm{conc}}$, respectively. The considered grid size is 121 $\\times$ 121.}\n\t\\label{fig16}\n\\end{figure}\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure17.png}\n\t\\caption{2D solid amount profiles of the C, Al and S chemical components of cement system 2 after 2 and 6 (final time step) years. RTM means the original HPx$_{\\rm{4C}}$ model and TM+DNN denotes the Hydrus transport model coupled with our DNN geochemical emulator (HPx$_{\\rm py}$-DNN). The first to third row present profiles for $C^{\\rm{solid}}$, $Al^{\\rm{solid}}$, and $S^{\\rm{solid}}$, respectively The considered grid size is 61 $\\times$ 61.}\n\t\\label{fig17}\n\\end{figure}\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure18.png}\n\t\\caption{2D solid amount profiles of the C, Al and S chemical components of cement system 2 after 2 and 6 (final time step) years. RTM means the original HPx$_{\\rm{4C}}$ model and TM+DNN denotes the Hydrus transport model coupled with our DNN geochemical emulator (HPx$_{\\rm py}$-DNN). The first to third row present profiles for $C^{\\rm{solid}}$, $Al^{\\rm{solid}}$, and $S^{\\rm{solid}}$, respectively The considered grid size is 121 $\\times$ 121.}\n\t\\label{fig18}\n\\end{figure}\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure19.png}\n\t\\caption{2D solid amount profiles of the C, Al and S chemical components of cement system 2 after 2 and 6 (final time step) years. RTM means the original HPx$_{\\rm{4C}}$ model and TM+kNN denotes the Hydrus transport model coupled with our kNN geochemical emulator (HPx$_{\\rm py}$-kNN). The first to third row present profiles for $C^{\\rm{solid}}$, $Al^{\\rm{solid}}$, and $S^{\\rm{solid}}$, respectively The considered grid size is 61 $\\times$ 61.}\n\t\\label{fig19}\n\\end{figure}\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure20.png}\n\t\\caption{2D solid amount profiles of the C, Al and S chemical components of cement system 2 after 2 and 6 (final time step) years. RTM means the original HPx$_{\\rm{4C}}$ model and TM+kNN denotes the Hydrus transport model coupled with our kNN geochemical emulator (HPx$_{\\rm py}$-kNN). The first to third row present profiles for $C^{\\rm{solid}}$, $Al^{\\rm{solid}}$, and $S^{\\rm{solid}}$, respectively The considered grid size is 121 $\\times$ 121.}\n\t\\label{fig20}\n\\end{figure}\n\n\\begin{table}[h!]\n\t\\caption{Speedups offered by the KNN and DNN emulators in HPx$_{\\rm py}$ for the reactive transport simulations considered for cement system 2. The HPx$_{\\rm{4C}}$ calculations involve the parallelization of PHREEQC over our 4 CPUs. The HPx$_{\\rm{1C}}$ calculations are performed on a single CPU. Both the kNN and DNN predictions make use of a GPU. ML signifies the used machine learning method for emulation, BC denotes the type of flow boundary conditions and GS is the grid size. The maximum possible speedups associated with HPx$_{\\rm{4C}}$ and HP$_{\\rm{1C}}$,  Max SP HPx$_{\\rm{4C}}$ and  Max SP HPx$_{\\rm{1C}}$, correspond to an hypothetical situation where the geochemical calculations incur zero computational cost.}\n\n\t\\begin{center}\n\t\t\\begin{tabular}{cccccccc}\n\t\t\t\\hline\n\t\t\t& & & & & & & \\\\\n\t\t\tML & BC & GS & HPx$_{\\rm{4C}}$ time (s) & SP HPx$_{\\rm{4C}}$ & Max SP HPx$_{\\rm{4C}}$ & SP HPx$_{\\rm{1C}}$ & Max SP HPx$_{\\rm{1C}}$ \\\\\n\t\t\tDNN & ADV & 61 $\\times$ 61 & 21,415 & 8.2 & 9.0 & 30.3 & 33.1\\\\\n\t\t\tkNN & ADV & 61 $\\times$ 61 & 21,415 & 7.9 & 9.0 & 28.9 & 33.1\\\\\n\t\t\tDNN & ADV & 121 $\\times$ 121 & 199,841 & 8.2 & 9.5 & 29.9 & 32.8\\\\\n\t\t\tkNN & ADV & 121 $\\times$ 121 & 199,841 & 8.5 & 9.5 & 31.3 & 32.8\\\\\n\t\t\t\\hline\n\t\t\\end{tabular}\n\t\\end{center}\n\n\t\\label{table4}\n\\end{table}\n\n\\FloatBarrier\n\n3.1 Ca-Si Problem\n\\subsection{Ca-Si Problem}\nFor this first cement system, the emulation problem consists of predicting at each time step of the RT simulation the (output) Ca, Si, H and O aqueous concentrations (mol/kg of water or mol/kgw) from the (input) total amounts of Ca and Si (mol). \n\n\\subsubsection{Training the emulators}\n\\label{train_res1}\nHere the kNN and DNN emulators are firstly trained using a set of 400,000 test examples for both. This training set is obtained by randomly sampling the two-dimensional input space by latin hypercube sampling (LHS) between $\\left[0,0\\right]$ and $\\rm{\\left[Ca^{tot}_{max},Si^{tot}_{max}\\right]}$, and running PHREEQC for each input sample, $\\rm{\\textbf{x}_i = \\left[Ca^{tot}_i, Si^{tot}_i\\right]}$ to get the corresponding output vectors, $\\rm{\\textbf{y}_i = \\left[Ca^{conc}_i,Si^{conc}_i,H^{conc}_i,O^{conc}_i\\right]}$. The upper bounds, $\\rm{Ca^{tot}_{max}}$ and $\\rm{Si^{tot}_{max}}$ are defined based on a cheap full RT simulation with advective-dispersive transport using a small 1D domain of 51 nodes. It is worth noting that the total amounts of $\\textbf{x}_i$, corresponding to the PHREEQC-simulated concentrations, $\\textbf{y}_i$, have to be corrected for the different amount of water between the training set and the transport simulations. Doing so, it turns out that about 20 \\% of the post-corrected $x_i$ values exceed their pre-defined upper bounds and these excessively large values need to be filtered out. Creating the 400,000 training examples thus required about 500,000 PHREEQC runs. As stated earlier, for this problem single-threaded PHREEQC performs about 670 geochemical calculations per second on our used Intel\\textsuperscript{\\textregistered} i7 CPU.\n\nWith respect to kNN, the tuning parameters are the number of neighbors, $k$, the type of distance measure, and the interpolation technique. We simply used the default settings: $k = 5$, euclidean distance and inverse-distance interpolation. Regarding training of the DNN, the 400,000 sample were split between the training set itself (90 \\% of the data) and a validation set (10 \\% of the data). The latter serves to monitor the evolution of the selected mean squared error loss function on samples that are not used for training, thereby detecting potential overfitting. If the validation loss stops decreasing before the fixed number of epochs has been completed, then  training is stopped. Importantly, the emulation is achieved in log-space for both the input, $X$ and output $Y$, domains. This because total amounts and concentrations of the involved components typically cover many orders of magnitudes (up to 10 orders or more). Using a DNN also requires some form of data normalization or standardization. Here both the $\\rm{log\\left(\\textbf{x}_i\\right)}$ and $\\rm{log\\left(\\textbf{y}_i\\right)}$ vectors are standardized around 0 with standard deviation of 1.\n\nFigure \\ref{fig1} illustrates the trained emulators' performance for geochemical predictions using an independent test set that comprises 10,000 test examples. Both kNN (Figures \\ref{fig1}a - d) and DNN (Figures \\ref{fig1}e - h) appear to be rather accurate . DNN also shows a slight degradation for the larger concentration values  (Figures \\ref{fig1}e - h). The latter is likely due to the combination of a small proportion of large concentration values in the training set with the log-transformation that implicitly pushes the DNN to try harder to fit the smaller concentrations during training. Regarding speedup and as written earlier, for this setup the single-threaded kNN method is 300 times faster than single-threaded PHREEQC for predicting the 10,000 concentration vectors all at once. The computational savings allowed by the DNN emulator when ran on our GPU is higher with a speedup as large as 4000 for predicting the same 10,000 concentration vectors all at once. \n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{0cm}\\includegraphics[width=35pc]{Figure1.png}\n\t\\caption{1-1 plots of the kNN (subfigures (a) - (d)) and DNN (subfigures (e) - (h)) emulators' performance obtained for system 1 when the kNN training base contains 400,000 samples and the DNN is trained using the same 400,000 samples. Here ``true\" means the original PHREEQC-simulated data and ``predicted\" denotes the emulated (that is, kNN-simulated and DNN-simulated) data. Hence, the $x$-axis and $y$-axis present the original and emulated 10,000 independent test data points, respectively. The RMSE and $Q_2$ coefficient denote the root-mean-square-error and coefficient of determination in testing mode, respectively, between the original and emulated 10,000 test data points.}\n\t\\label{fig1}\n\\end{figure}\n\nTo test the sensitivity of the emulators' performance to the training set size, training was also performed using reduced training sets comprising 100,000, 10,000, 1000 and 100 samples, respectively. It seen that the DNN performance achieved  when using 10,000 training samples is virtually the same as that obtained when using 400,000 training samples (Figure \\ref{fig2} and Table \\ref{table2}). It is only for training sets smaller than 1000 samples that the DNN performance starts to degrade significantly (Table \\ref{table2}). In contrast, the behavior of kNN appears to markedly decrease as the training set gets smaller (Figure \\ref{fig2} and Table \\ref{table1}).\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{0cm}\\includegraphics[width=35pc]{Figure2.png}\n\t\\caption{1-1 plots of the kNN (subfigures (a) - (d)) and DNN (subfigures (e) - (h)) emulators' performance obtained for system 1 when the kNN training base contains 10,000 samples and the DNN is trained using the same 10,000 samples. Here ``true\" means the original PHREEQC-simulated data and ``predicted\" denotes the emulated (that is, kNN-simulated and DNN-simulated) data. Hence, the $x$-axis and $y$-axis present the original and emulated 10,000 independent test data points, respectively. The RMSE and $Q_2$ coefficient denote the root-mean-square-error and coefficient of determination in testing mode, respectively, between the original and emulated 10,000 test data points.}\n\t\\label{fig2}\n\\end{figure}\n\n\\begin{table}[h!]\n\t\\caption{Performance of the DNN and kNN emulators for cement system 1 and different training set sizes. For brevity, only the results for Ca$^{conc}$ and Si$^{conc}$ are shown. The units are mol per kg of water (mol/kgw). ML refers to the type of emulator, TR signifies the size of the training set, and the RMSE and $Q_2$ coefficient denote the root-mean-square-error and coefficient of determination in testing mode, respectively, between the original and emulated 10,000 test data points.}\n\t\\begin{center}\n\t\t\\begin{tabular}{cccccccc}\n\t\t\t\\hline\n\t\t\t& & & & & \\\\\n\t\t\tML & TR & RMSE - Ca$^{conc}$ & $Q_2$ - Ca$^{conc}$ & RMSE - Si$^{conc}$ & $Q_2$ - Si$^{conc}$\\\\\n\t\t\tDNN & 4 $\\times$ 10$^{5}$ & 4.50 $\\times$ 10$^{-5}$ & 0.9999 & 1.21 $\\times$ 10$^{-6}$ & 0.9999\\\\\n\t\t\tDNN & 1 $\\times$ 10$^{5}$ & 5.06 $\\times$ 10$^{-5}$ & 0.9999 & 1.30 $\\times$ 10$^{-6}$ & 0.9999\\\\\n\t\t\tDNN & 1 $\\times$ 10$^{4}$ & 4.57 $\\times$ 10$^{-5}$ & 0.9999 & 1.48 $\\times$ 10$^{-6}$ & 0.9999\\\\\n\t\t\tDNN & 1 $\\times$ 10$^{3}$ & 8.02 $\\times$ 10$^{-5}$ & 0.9998 & 3.31 $\\times$ 10$^{-6}$ & 0.9994\\\\\n\t\t\tDNN & 1 $\\times$ 10$^{2}$ & 47.1 $\\times$ 10$^{-5}$ & 0.9935 & 9.02 $\\times$ 10$^{-6}$ & 0.9959\\\\\n\t\t\tkNN & 4 $\\times$ 10$^{5}$ & 2.52 $\\times$ 10$^{-5}$ & 1.0000 & 1.59 $\\times$ 10$^{-6}$ & 0.9999\\\\\n\t\t\tkNN & 1 $\\times$ 10$^{5}$ & 6.27 $\\times$ 10$^{-5}$ & 0.9999 & 4.20 $\\times$ 10$^{-6}$ & 0.9991\\\\\n\t\t\tkNN & 1 $\\times$ 10$^{4}$ & 14.1 $\\times$ 10$^{-5}$ & 0.9994 & 13.5 $\\times$ 10$^{-6}$ & 0.9908\\\\\n\t\t\tkNN & 1 $\\times$ 10$^{3}$ & 43.9 $\\times$ 10$^{-5}$ & 0.9944 & 74.6 $\\times$ 10$^{-6}$ & 0.7180\\\\\n\t\t\tkNN & 1 $\\times$ 10$^{2}$  & 96.4 $\\times$ 10$^{-5}$ & 0.9729 & 55.0 $\\times$ 10$^{-6}$ & 0.8468\\\\\n\t\t\\end{tabular}\n\t\\end{center}\n\t\\label{table2}\n\\end{table}\n\n\\FloatBarrier\n\n\\subsubsection{Reactive transport simulation}\n\\label{prob12Dres}\n\nThis section focuses on reactive transport simulations with HPx$_{\\rm py}$ within cement system 1, under both advective-dispersive and diffusive transport conditions. As written above, the domain sizes are both 61 $\\times$ 61 and 121 $\\times$ 121 for the advection-dispersion case and, because of computational constraints, solely 61 $\\times$ 61  for the diffusion case. In addition, the simulation time period is 2 years for the advection-dispersion case and 1 year for the diffusion case. Figures \\ref{fig3} and \\ref{fig4} present times series of original and emulated Ca, Si, H and O concentrations at 5 locations within the 2D domain for advective-dispersive transport conditions, for both our kNN-based (HPx$_{\\rm py}$-kNN) and DNN-based (HPx$_{\\rm py}$-DNN) reactive transport codes. It is seen that HPx$_{\\rm py}$-kNN and HPx$_{\\rm py}$-DNN both induce a quite good simulation accuracy. Also, the results for the diffusive transport case are of similarly good quality (not shown). Figures \\ref{fig5} - \\ref{fig6} provide more insights into the HPx$_{\\rm py}$-kNN and HPx$_{\\rm py}$-DNN performances by displaying 2D Ca, Si, H and O concentration profiles at a given time. For each experiment and chemical component, this time is selected as to be well representative of the simulated dynamics. It is observed that the original and emulated images are visually almost indistinguishable for the advection-dispersion case (Figure \\ref{fig5}). For the diffusion case, the emulators also perform quite well for Ca, H and O (Figures\\ref{fig6}a - c, g - i and j - l), while some slight to moderate discrepancies appear at the concentration front for Si (Figures \\ref{fig6}d - f). Notwithstanding, the Si concentration remains globally well predicted. Furthermore, Figures \\ref{fig7} - \\ref{fig8} present the original and emulated 2D solid amount profiles corresponding to Figures \\ref{fig5} - \\ref{fig6}. The original solid amount profiles are overall well approximated by HPx$_{\\rm py}$-kNN and HPx$_{\\rm py}$-DNN for the advection-dispersion case (Figure \\ref{fig7}), even though some mismatch appears at the border of the fully depleted zone for the H component. As of the diffusion case (Figure \\ref{fig8}), the same kind of mismatch is observed for the emulated solid amounts of H by HPx$_{\\rm py}$-kNN while the emulated profiles by HPx$_{\\rm py}$-DNN show some more discrepancies. Though we decided to present raw emulation results, we would like to stress that some if not all of the observed artifacts could likely be smoothed out by using some post-filtering such as median filtering.\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure3.png}\n\t\\caption{Time series of original (RTM, solid green lines) and HPx$_{\\rm py}$-kNN emulated (TM+kNN, dashed orange lines) concentrations (mol/kg) of Ca, H, O and Si at selected observations points for cement system 1 and advective-dispersive transport. Obs. 1 - 5 denote the selected observation points, with the following $\\left[x,y\\right]$ locations (in cm). Obs. 1:  $\\left[0.5, 2.5\\right]$, Obs. 2: $\\left[1, 2\\right]$, Obs. 3: $\\left[2, 2\\right]$, Obs. 4: $\\left[1, 1\\right]$, Obs. 5: $\\left[2, 1\\right]$. The results for the 121 $\\times$ 121 grid size are rather similar.}\n\t\\label{fig3}\n\\end{figure}\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure4.png}\n\t\\caption{Time series of original (RTM, solid green lines) and HPx$_{\\rm py}$-DNN emulated (TM+DNN, dashed orange lines) concentrations (mol/kg) of Ca, H, O and Si at selected observations points for cement system 1 and advective-dispersive transport. Obs. 1 - 5 denote the selected observation points, with the following $\\left[x,y\\right]$ locations (in cm). Obs. 1:  $\\left[0.5, 2.5\\right]$, Obs. 2: $\\left[1, 2\\right]$, Obs. 3: $\\left[2, 2\\right]$, Obs. 4: $\\left[1, 1\\right]$, Obs. 5: $\\left[2, 1\\right]$. The results for the 121 $\\times$ 121 grid size are rather similar.}\n\t\\label{fig4}\n\\end{figure}\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure5.png}\n\t\\caption{2D concentration profiles obtained for cement system 1 at the end of the 2-year simulation performed for the advection-dispersion case. RTM means the original HP$_{\\rm{4C}}$ model, TM+kNN denotes the Hydrus transport model coupled with our kNN geochemical emulator (HPx$_{\\rm py}$-kNN) and TM+DNN signifies the Hydrus transport model coupled with our DNN geochemical emulator (HPx$_{\\rm py}$-DNN). The first to fourth row present profiles for $Ca^{\\rm{conc}}$, $Si^{\\rm{conc}}$, $H^{\\rm{conc}}$, and $O^{\\rm{conc}}$, respectively. The considered grid size is 61 $\\times$ 61. The results for the 121 $\\times$ 121 grid are rather similar.}\n\t\\label{fig5}\n\\end{figure}\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure6.png}\n\t\\caption{2D concentration profiles obtained for cement system 1 at the final time step of the 1-year simulation performed for the diffusion case. RTM means the original HP$_{\\rm{4C}}$ model, TM+kNN denotes the Hydrus transport model coupled with our kNN geochemical emulator (HPx$_{\\rm py}$-kNN) and TM+DNN signifies the Hydrus transport model coupled with our DNN geochemical emulator (HPx$_{\\rm py}$-DNN). The first to fourth row present profiles for $Ca^{\\rm{conc}}$, $Si^{\\rm{conc}}$, $H^{\\rm{conc}}$, and $O^{\\rm{conc}}$, respectively. The considered grid size is 61 $\\times$ 61.}\n\t\\label{fig6}\n\\end{figure}\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure7.png}\n\t\\caption{2D solid amount profiles obtained for cement system 1 at the end of the 2-year simulation performed for the advection-dispersion case. RTM means the original HP$_{\\rm{4C}}$ model, TM+kNN denotes the Hydrus transport model coupled with our kNN geochemical emulator (HPx$_{\\rm py}$-kNN) and TM+DNN signifies the Hydrus transport model coupled with our DNN geochemical emulator (HPx$_{\\rm py}$-DNN). The first to fourth row present profiles for $Ca^{\\rm{solid}}$, $Si^{\\rm{solid}}$, $H^{\\rm{solid}}$, and $O^{\\rm{solid}}$, respectively. The considered grid size is 61 $\\times$ 61. The results for the 121 $\\times$ 121 grid are rather similar.}\n\t\\label{fig7}\n\\end{figure}\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure8.png}\n\t\\caption{2D solid amount profiles obtained for cement system 1 at the final time step of the 1-year simulation performed for the diffusion case. RTM means the original HP$_{\\rm{4C}}$ model, TM+kNN denotes the Hydrus transport model coupled with our kNN geochemical emulator (HPx$_{\\rm py}$-kNN) and TM+DNN signifies the Hydrus transport model coupled with our DNN geochemical emulator (HPx$_{\\rm py}$-DNN). The first to fourth row present profiles for $Ca^{\\rm{solid}}$, $Si^{\\rm{solid}}$, $H^{\\rm{solid}}$, and $O^{\\rm{solid}}$, respectively. The considered grid size is 61 $\\times$ 61.}\n\t\\label{fig8}\n\\end{figure}\n\nThe speedups associated with the considered problem are detailed in Table \\ref{table3}. It is noted that the GPU-based DNN emulator allows for a speedup that is close to optimal. Indeed, The DNN speedups overall represent 85 \\% to 95 \\% of the maximum possible speedups (that is, speedups that would be obtained if the geochemcical calculations would come at no cost at all). The speedups associated with single-threaded kNN remain substantial but only amount to 57 \\% - 65 \\% of the corresponding maximum speedups. As detailed in section \\ref{train_res1}, the used kNN and DNN implementations are found to be respectively 300 and 4000 faster than single-threaded PHREEQC when predicting 10,000 points all one once for this geochemical system. Based on these numbers one could have expected the achieved speedups to represent say 90 \\% (kNN) or 99 \\% (DNN) of the maximum possible ones. A large part of the gaps between achieved and maximum possible speedups is thus likely caused by the time required for communicating and exchanging data between the main C/C++ code and the Python-based emulators.\n\n\\begin{table}[h!]\n\t\\caption{Speedups offered by the KNN and DNN emulators in HPx$_{\\rm py}$ for the reactive transport simulations considered for cement system 1. The HPx$_{\\rm{4C}}$ calculations involve the parallelization of PHREEQC over our 4 CPUs. The HPx$_{\\rm{1C}}$ calculations are performed on a single CPU. The kNN predictions are performed on a single CPU using the scikit-learn implementation while the DNN predictions make use of our GPU. ML signifies the used machine learning method for emulation, TC denotes transport conditions (ADV: advection-dispersion, DIF: diffusion) and GS is the grid size. The maximum possible speedups associated with HPx$_{\\rm{4C}}$ and HPx$_{\\rm{1C}}$,  Max SP HPx$_{\\rm{4C}}$ and  Max SP HPx$_{\\rm{1C}}$, correspond to an hypothetical situation where the geochemical calculations incur zero computational cost.}\n\n\t\\begin{center}\n\t\t\\begin{tabular}{cccccccc}\n\t\t\t\\hline\n\t\t\t& & & & & & & \\\\\n\t\t\tML & TC & GS & HPx$_{\\rm{4C}}$ time (s) & SP HPx$_{\\rm{4C}}$ & Max SP HPx$_{\\rm{4C}}$ & SP HPx$_{\\rm{1C}}$ & Max SP HPx$_{\\rm{1C}}$ \\\\\n\t\t\tDNN & ADV & 61 $\\times$ 61 & 3189 & 6.8 & 7.7 & 24.5 & 28.5 \\\\\n\t\t\tkNN & ADV & 61 $\\times$ 61 & 3189 & 5.0 & 7.7 & 18.0 & 28.5 \\\\\n\t\t\tDNN & ADV & 121 $\\times$ 121 & 23,337 & 5.0 & 5.2 & 17.0 & 18.0 \\\\\n\t\t\tkNN & ADV & 121 $\\times$ 121 & 23,337 & 3.4 & 5.2 & 11.6 & 18.0 \\\\\n\t\t\tDNN & DIF & 61 $\\times$ 61 & 25,448 & 4.2 & 4.9 & 16.2 & 19.1 \\\\\n\t\t\tkNN & DIF & 61 $\\times$ 61 & 25,448 & 2.8 & 4.9 & 10.8 & 19.1 \\\\\n\t\t\t\\hline\n\t\t\\end{tabular}\n\t\\end{center}\n\n\t\\label{table3}\n\\end{table}\n\n\\FloatBarrier\n\n3.1.2 Reactive transport simulation\n\\subsubsection{Reactive transport simulation}\n\\label{prob12Dres}\n\nThis section focuses on reactive transport simulations with HPx$_{\\rm py}$ within cement system 1, under both advective-dispersive and diffusive transport conditions. As written above, the domain sizes are both 61 $\\times$ 61 and 121 $\\times$ 121 for the advection-dispersion case and, because of computational constraints, solely 61 $\\times$ 61  for the diffusion case. In addition, the simulation time period is 2 years for the advection-dispersion case and 1 year for the diffusion case. Figures \\ref{fig3} and \\ref{fig4} present times series of original and emulated Ca, Si, H and O concentrations at 5 locations within the 2D domain for advective-dispersive transport conditions, for both our kNN-based (HPx$_{\\rm py}$-kNN) and DNN-based (HPx$_{\\rm py}$-DNN) reactive transport codes. It is seen that HPx$_{\\rm py}$-kNN and HPx$_{\\rm py}$-DNN both induce a quite good simulation accuracy. Also, the results for the diffusive transport case are of similarly good quality (not shown). Figures \\ref{fig5} - \\ref{fig6} provide more insights into the HPx$_{\\rm py}$-kNN and HPx$_{\\rm py}$-DNN performances by displaying 2D Ca, Si, H and O concentration profiles at a given time. For each experiment and chemical component, this time is selected as to be well representative of the simulated dynamics. It is observed that the original and emulated images are visually almost indistinguishable for the advection-dispersion case (Figure \\ref{fig5}). For the diffusion case, the emulators also perform quite well for Ca, H and O (Figures\\ref{fig6}a - c, g - i and j - l), while some slight to moderate discrepancies appear at the concentration front for Si (Figures \\ref{fig6}d - f). Notwithstanding, the Si concentration remains globally well predicted. Furthermore, Figures \\ref{fig7} - \\ref{fig8} present the original and emulated 2D solid amount profiles corresponding to Figures \\ref{fig5} - \\ref{fig6}. The original solid amount profiles are overall well approximated by HPx$_{\\rm py}$-kNN and HPx$_{\\rm py}$-DNN for the advection-dispersion case (Figure \\ref{fig7}), even though some mismatch appears at the border of the fully depleted zone for the H component. As of the diffusion case (Figure \\ref{fig8}), the same kind of mismatch is observed for the emulated solid amounts of H by HPx$_{\\rm py}$-kNN while the emulated profiles by HPx$_{\\rm py}$-DNN show some more discrepancies. Though we decided to present raw emulation results, we would like to stress that some if not all of the observed artifacts could likely be smoothed out by using some post-filtering such as median filtering.\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure3.png}\n\t\\caption{Time series of original (RTM, solid green lines) and HPx$_{\\rm py}$-kNN emulated (TM+kNN, dashed orange lines) concentrations (mol/kg) of Ca, H, O and Si at selected observations points for cement system 1 and advective-dispersive transport. Obs. 1 - 5 denote the selected observation points, with the following $\\left[x,y\\right]$ locations (in cm). Obs. 1:  $\\left[0.5, 2.5\\right]$, Obs. 2: $\\left[1, 2\\right]$, Obs. 3: $\\left[2, 2\\right]$, Obs. 4: $\\left[1, 1\\right]$, Obs. 5: $\\left[2, 1\\right]$. The results for the 121 $\\times$ 121 grid size are rather similar.}\n\t\\label{fig3}\n\\end{figure}\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure4.png}\n\t\\caption{Time series of original (RTM, solid green lines) and HPx$_{\\rm py}$-DNN emulated (TM+DNN, dashed orange lines) concentrations (mol/kg) of Ca, H, O and Si at selected observations points for cement system 1 and advective-dispersive transport. Obs. 1 - 5 denote the selected observation points, with the following $\\left[x,y\\right]$ locations (in cm). Obs. 1:  $\\left[0.5, 2.5\\right]$, Obs. 2: $\\left[1, 2\\right]$, Obs. 3: $\\left[2, 2\\right]$, Obs. 4: $\\left[1, 1\\right]$, Obs. 5: $\\left[2, 1\\right]$. The results for the 121 $\\times$ 121 grid size are rather similar.}\n\t\\label{fig4}\n\\end{figure}\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure5.png}\n\t\\caption{2D concentration profiles obtained for cement system 1 at the end of the 2-year simulation performed for the advection-dispersion case. RTM means the original HP$_{\\rm{4C}}$ model, TM+kNN denotes the Hydrus transport model coupled with our kNN geochemical emulator (HPx$_{\\rm py}$-kNN) and TM+DNN signifies the Hydrus transport model coupled with our DNN geochemical emulator (HPx$_{\\rm py}$-DNN). The first to fourth row present profiles for $Ca^{\\rm{conc}}$, $Si^{\\rm{conc}}$, $H^{\\rm{conc}}$, and $O^{\\rm{conc}}$, respectively. The considered grid size is 61 $\\times$ 61. The results for the 121 $\\times$ 121 grid are rather similar.}\n\t\\label{fig5}\n\\end{figure}\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure6.png}\n\t\\caption{2D concentration profiles obtained for cement system 1 at the final time step of the 1-year simulation performed for the diffusion case. RTM means the original HP$_{\\rm{4C}}$ model, TM+kNN denotes the Hydrus transport model coupled with our kNN geochemical emulator (HPx$_{\\rm py}$-kNN) and TM+DNN signifies the Hydrus transport model coupled with our DNN geochemical emulator (HPx$_{\\rm py}$-DNN). The first to fourth row present profiles for $Ca^{\\rm{conc}}$, $Si^{\\rm{conc}}$, $H^{\\rm{conc}}$, and $O^{\\rm{conc}}$, respectively. The considered grid size is 61 $\\times$ 61.}\n\t\\label{fig6}\n\\end{figure}\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure7.png}\n\t\\caption{2D solid amount profiles obtained for cement system 1 at the end of the 2-year simulation performed for the advection-dispersion case. RTM means the original HP$_{\\rm{4C}}$ model, TM+kNN denotes the Hydrus transport model coupled with our kNN geochemical emulator (HPx$_{\\rm py}$-kNN) and TM+DNN signifies the Hydrus transport model coupled with our DNN geochemical emulator (HPx$_{\\rm py}$-DNN). The first to fourth row present profiles for $Ca^{\\rm{solid}}$, $Si^{\\rm{solid}}$, $H^{\\rm{solid}}$, and $O^{\\rm{solid}}$, respectively. The considered grid size is 61 $\\times$ 61. The results for the 121 $\\times$ 121 grid are rather similar.}\n\t\\label{fig7}\n\\end{figure}\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure8.png}\n\t\\caption{2D solid amount profiles obtained for cement system 1 at the final time step of the 1-year simulation performed for the diffusion case. RTM means the original HP$_{\\rm{4C}}$ model, TM+kNN denotes the Hydrus transport model coupled with our kNN geochemical emulator (HPx$_{\\rm py}$-kNN) and TM+DNN signifies the Hydrus transport model coupled with our DNN geochemical emulator (HPx$_{\\rm py}$-DNN). The first to fourth row present profiles for $Ca^{\\rm{solid}}$, $Si^{\\rm{solid}}$, $H^{\\rm{solid}}$, and $O^{\\rm{solid}}$, respectively. The considered grid size is 61 $\\times$ 61.}\n\t\\label{fig8}\n\\end{figure}\n\nThe speedups associated with the considered problem are detailed in Table \\ref{table3}. It is noted that the GPU-based DNN emulator allows for a speedup that is close to optimal. Indeed, The DNN speedups overall represent 85 \\% to 95 \\% of the maximum possible speedups (that is, speedups that would be obtained if the geochemcical calculations would come at no cost at all). The speedups associated with single-threaded kNN remain substantial but only amount to 57 \\% - 65 \\% of the corresponding maximum speedups. As detailed in section \\ref{train_res1}, the used kNN and DNN implementations are found to be respectively 300 and 4000 faster than single-threaded PHREEQC when predicting 10,000 points all one once for this geochemical system. Based on these numbers one could have expected the achieved speedups to represent say 90 \\% (kNN) or 99 \\% (DNN) of the maximum possible ones. A large part of the gaps between achieved and maximum possible speedups is thus likely caused by the time required for communicating and exchanging data between the main C/C++ code and the Python-based emulators.\n\n\\begin{table}[h!]\n\t\\caption{Speedups offered by the KNN and DNN emulators in HPx$_{\\rm py}$ for the reactive transport simulations considered for cement system 1. The HPx$_{\\rm{4C}}$ calculations involve the parallelization of PHREEQC over our 4 CPUs. The HPx$_{\\rm{1C}}$ calculations are performed on a single CPU. The kNN predictions are performed on a single CPU using the scikit-learn implementation while the DNN predictions make use of our GPU. ML signifies the used machine learning method for emulation, TC denotes transport conditions (ADV: advection-dispersion, DIF: diffusion) and GS is the grid size. The maximum possible speedups associated with HPx$_{\\rm{4C}}$ and HPx$_{\\rm{1C}}$,  Max SP HPx$_{\\rm{4C}}$ and  Max SP HPx$_{\\rm{1C}}$, correspond to an hypothetical situation where the geochemical calculations incur zero computational cost.}\n\n\t\\begin{center}\n\t\t\\begin{tabular}{cccccccc}\n\t\t\t\\hline\n\t\t\t& & & & & & & \\\\\n\t\t\tML & TC & GS & HPx$_{\\rm{4C}}$ time (s) & SP HPx$_{\\rm{4C}}$ & Max SP HPx$_{\\rm{4C}}$ & SP HPx$_{\\rm{1C}}$ & Max SP HPx$_{\\rm{1C}}$ \\\\\n\t\t\tDNN & ADV & 61 $\\times$ 61 & 3189 & 6.8 & 7.7 & 24.5 & 28.5 \\\\\n\t\t\tkNN & ADV & 61 $\\times$ 61 & 3189 & 5.0 & 7.7 & 18.0 & 28.5 \\\\\n\t\t\tDNN & ADV & 121 $\\times$ 121 & 23,337 & 5.0 & 5.2 & 17.0 & 18.0 \\\\\n\t\t\tkNN & ADV & 121 $\\times$ 121 & 23,337 & 3.4 & 5.2 & 11.6 & 18.0 \\\\\n\t\t\tDNN & DIF & 61 $\\times$ 61 & 25,448 & 4.2 & 4.9 & 16.2 & 19.1 \\\\\n\t\t\tkNN & DIF & 61 $\\times$ 61 & 25,448 & 2.8 & 4.9 & 10.8 & 19.1 \\\\\n\t\t\t\\hline\n\t\t\\end{tabular}\n\t\\end{center}\n\n\t\\label{table3}\n\\end{table}\n\n\\FloatBarrier\n\n",
        "label": "fig6",
        "Descriptive_question1": "Which chemical component is represented in the first row of the concentration profiles in figure_6?",
        "Descriptive_question2": "What is the grid size used for the simulation shown in figure_6?",
        "Reasoning_question1": "How do the concentration profiles of Si in figure_6 compare between the original RTM model and the emulated models (TM+kNN and TM+DNN), and what might explain any observed discrepancies?",
        "Reasoning_question2": "Based on the concentration profiles in figure_6, which chemical component shows the most significant variation across the different models (RTM, TM+kNN, TM+DNN), and what could be the potential reasons for this variation?",
        "Descriptive_answer1": "Ca^{conc}",
        "Descriptive_answer2": "61 \\times 61",
        "Reasoning_answer1": "In figure_6, the Si concentration profiles (second row) show that the original HP$_{4C}$ model (RTM) and the emulated models (TM+kNN and TM+DNN) are generally similar, but there are some slight to moderate discrepancies at the concentration front for Si. This discrepancy may arise due to limitations of the emulators in accurately capturing the sharp concentration gradients or nonlinearities in the Si distribution, especially under diffusive transport conditions. Additionally, the emulators might have difficulty interpolating accurately in regions with rapidly changing concentrations, leading to observable deviations from the original model.",
        "Reasoning_answer2": "Among the chemical components in figure_6, Si shows the most significant variation across RTM, TM+kNN, and TM+DNN models, particularly at the concentration front where discrepancies are noted. Potential reasons for this variation include the complex geochemical behavior of Si under diffusion, challenges in the emulators' ability to capture the nonlinearity and spatial heterogeneity of Si concentrations, and possible limitations in the training data representing these conditions fully. As a result, Si concentrations are less accurately reproduced compared to Ca, H, and O, which exhibit better overall agreement."
    },
    {
        "paper_id": "2107.07598.json",
        "image_id": "figure_7",
        "image_path": [
            "/home/yz979/scratch/chengye/scimolmo/test-images-tables/dataset_imagestables/2107.07598/images/Figure7.png"
        ],
        "caption": "2D solid amount profiles obtained for cement system 1 at the end of the 2-year simulation performed for the advection-dispersion case. RTM means the original HP$_{\\rm{4C}}$ model, TM+kNN denotes the Hydrus transport model coupled with our kNN geochemical emulator (HPx$_{\\rm py}$-kNN) and TM+DNN signifies the Hydrus transport model coupled with our DNN geochemical emulator (HPx$_{\\rm py}$-DNN). The first to fourth row present profiles for $Ca^{\\rm{solid}}$, $Si^{\\rm{solid}}$, $H^{\\rm{solid}}$, and $O^{\\rm{solid}}$, respectively. The considered grid size is 61 $\\times$ 61. The results for the 121 $\\times$ 121 grid are rather similar.",
        "classify": "Chart",
        "section_info": "3 Results\n\\section{Results}\n\\label{results}\n\n\\subsection{Ca-Si Problem}\nFor this first cement system, the emulation problem consists of predicting at each time step of the RT simulation the (output) Ca, Si, H and O aqueous concentrations (mol/kg of water or mol/kgw) from the (input) total amounts of Ca and Si (mol). \n\n\\subsubsection{Training the emulators}\n\\label{train_res1}\nHere the kNN and DNN emulators are firstly trained using a set of 400,000 test examples for both. This training set is obtained by randomly sampling the two-dimensional input space by latin hypercube sampling (LHS) between $\\left[0,0\\right]$ and $\\rm{\\left[Ca^{tot}_{max},Si^{tot}_{max}\\right]}$, and running PHREEQC for each input sample, $\\rm{\\textbf{x}_i = \\left[Ca^{tot}_i, Si^{tot}_i\\right]}$ to get the corresponding output vectors, $\\rm{\\textbf{y}_i = \\left[Ca^{conc}_i,Si^{conc}_i,H^{conc}_i,O^{conc}_i\\right]}$. The upper bounds, $\\rm{Ca^{tot}_{max}}$ and $\\rm{Si^{tot}_{max}}$ are defined based on a cheap full RT simulation with advective-dispersive transport using a small 1D domain of 51 nodes. It is worth noting that the total amounts of $\\textbf{x}_i$, corresponding to the PHREEQC-simulated concentrations, $\\textbf{y}_i$, have to be corrected for the different amount of water between the training set and the transport simulations. Doing so, it turns out that about 20 \\% of the post-corrected $x_i$ values exceed their pre-defined upper bounds and these excessively large values need to be filtered out. Creating the 400,000 training examples thus required about 500,000 PHREEQC runs. As stated earlier, for this problem single-threaded PHREEQC performs about 670 geochemical calculations per second on our used Intel\\textsuperscript{\\textregistered} i7 CPU.\n\nWith respect to kNN, the tuning parameters are the number of neighbors, $k$, the type of distance measure, and the interpolation technique. We simply used the default settings: $k = 5$, euclidean distance and inverse-distance interpolation. Regarding training of the DNN, the 400,000 sample were split between the training set itself (90 \\% of the data) and a validation set (10 \\% of the data). The latter serves to monitor the evolution of the selected mean squared error loss function on samples that are not used for training, thereby detecting potential overfitting. If the validation loss stops decreasing before the fixed number of epochs has been completed, then  training is stopped. Importantly, the emulation is achieved in log-space for both the input, $X$ and output $Y$, domains. This because total amounts and concentrations of the involved components typically cover many orders of magnitudes (up to 10 orders or more). Using a DNN also requires some form of data normalization or standardization. Here both the $\\rm{log\\left(\\textbf{x}_i\\right)}$ and $\\rm{log\\left(\\textbf{y}_i\\right)}$ vectors are standardized around 0 with standard deviation of 1.\n\nFigure \\ref{fig1} illustrates the trained emulators' performance for geochemical predictions using an independent test set that comprises 10,000 test examples. Both kNN (Figures \\ref{fig1}a - d) and DNN (Figures \\ref{fig1}e - h) appear to be rather accurate . DNN also shows a slight degradation for the larger concentration values  (Figures \\ref{fig1}e - h). The latter is likely due to the combination of a small proportion of large concentration values in the training set with the log-transformation that implicitly pushes the DNN to try harder to fit the smaller concentrations during training. Regarding speedup and as written earlier, for this setup the single-threaded kNN method is 300 times faster than single-threaded PHREEQC for predicting the 10,000 concentration vectors all at once. The computational savings allowed by the DNN emulator when ran on our GPU is higher with a speedup as large as 4000 for predicting the same 10,000 concentration vectors all at once. \n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{0cm}\\includegraphics[width=35pc]{Figure1.png}\n\t\\caption{1-1 plots of the kNN (subfigures (a) - (d)) and DNN (subfigures (e) - (h)) emulators' performance obtained for system 1 when the kNN training base contains 400,000 samples and the DNN is trained using the same 400,000 samples. Here ``true\" means the original PHREEQC-simulated data and ``predicted\" denotes the emulated (that is, kNN-simulated and DNN-simulated) data. Hence, the $x$-axis and $y$-axis present the original and emulated 10,000 independent test data points, respectively. The RMSE and $Q_2$ coefficient denote the root-mean-square-error and coefficient of determination in testing mode, respectively, between the original and emulated 10,000 test data points.}\n\t\\label{fig1}\n\\end{figure}\n\nTo test the sensitivity of the emulators' performance to the training set size, training was also performed using reduced training sets comprising 100,000, 10,000, 1000 and 100 samples, respectively. It seen that the DNN performance achieved  when using 10,000 training samples is virtually the same as that obtained when using 400,000 training samples (Figure \\ref{fig2} and Table \\ref{table2}). It is only for training sets smaller than 1000 samples that the DNN performance starts to degrade significantly (Table \\ref{table2}). In contrast, the behavior of kNN appears to markedly decrease as the training set gets smaller (Figure \\ref{fig2} and Table \\ref{table1}).\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{0cm}\\includegraphics[width=35pc]{Figure2.png}\n\t\\caption{1-1 plots of the kNN (subfigures (a) - (d)) and DNN (subfigures (e) - (h)) emulators' performance obtained for system 1 when the kNN training base contains 10,000 samples and the DNN is trained using the same 10,000 samples. Here ``true\" means the original PHREEQC-simulated data and ``predicted\" denotes the emulated (that is, kNN-simulated and DNN-simulated) data. Hence, the $x$-axis and $y$-axis present the original and emulated 10,000 independent test data points, respectively. The RMSE and $Q_2$ coefficient denote the root-mean-square-error and coefficient of determination in testing mode, respectively, between the original and emulated 10,000 test data points.}\n\t\\label{fig2}\n\\end{figure}\n\n\\begin{table}[h!]\n\t\\caption{Performance of the DNN and kNN emulators for cement system 1 and different training set sizes. For brevity, only the results for Ca$^{conc}$ and Si$^{conc}$ are shown. The units are mol per kg of water (mol/kgw). ML refers to the type of emulator, TR signifies the size of the training set, and the RMSE and $Q_2$ coefficient denote the root-mean-square-error and coefficient of determination in testing mode, respectively, between the original and emulated 10,000 test data points.}\n\t\\begin{center}\n\t\t\\begin{tabular}{cccccccc}\n\t\t\t\\hline\n\t\t\t& & & & & \\\\\n\t\t\tML & TR & RMSE - Ca$^{conc}$ & $Q_2$ - Ca$^{conc}$ & RMSE - Si$^{conc}$ & $Q_2$ - Si$^{conc}$\\\\\n\t\t\tDNN & 4 $\\times$ 10$^{5}$ & 4.50 $\\times$ 10$^{-5}$ & 0.9999 & 1.21 $\\times$ 10$^{-6}$ & 0.9999\\\\\n\t\t\tDNN & 1 $\\times$ 10$^{5}$ & 5.06 $\\times$ 10$^{-5}$ & 0.9999 & 1.30 $\\times$ 10$^{-6}$ & 0.9999\\\\\n\t\t\tDNN & 1 $\\times$ 10$^{4}$ & 4.57 $\\times$ 10$^{-5}$ & 0.9999 & 1.48 $\\times$ 10$^{-6}$ & 0.9999\\\\\n\t\t\tDNN & 1 $\\times$ 10$^{3}$ & 8.02 $\\times$ 10$^{-5}$ & 0.9998 & 3.31 $\\times$ 10$^{-6}$ & 0.9994\\\\\n\t\t\tDNN & 1 $\\times$ 10$^{2}$ & 47.1 $\\times$ 10$^{-5}$ & 0.9935 & 9.02 $\\times$ 10$^{-6}$ & 0.9959\\\\\n\t\t\tkNN & 4 $\\times$ 10$^{5}$ & 2.52 $\\times$ 10$^{-5}$ & 1.0000 & 1.59 $\\times$ 10$^{-6}$ & 0.9999\\\\\n\t\t\tkNN & 1 $\\times$ 10$^{5}$ & 6.27 $\\times$ 10$^{-5}$ & 0.9999 & 4.20 $\\times$ 10$^{-6}$ & 0.9991\\\\\n\t\t\tkNN & 1 $\\times$ 10$^{4}$ & 14.1 $\\times$ 10$^{-5}$ & 0.9994 & 13.5 $\\times$ 10$^{-6}$ & 0.9908\\\\\n\t\t\tkNN & 1 $\\times$ 10$^{3}$ & 43.9 $\\times$ 10$^{-5}$ & 0.9944 & 74.6 $\\times$ 10$^{-6}$ & 0.7180\\\\\n\t\t\tkNN & 1 $\\times$ 10$^{2}$  & 96.4 $\\times$ 10$^{-5}$ & 0.9729 & 55.0 $\\times$ 10$^{-6}$ & 0.8468\\\\\n\t\t\\end{tabular}\n\t\\end{center}\n\t\\label{table2}\n\\end{table}\n\n\\FloatBarrier\n\n\\subsubsection{Reactive transport simulation}\n\\label{prob12Dres}\n\nThis section focuses on reactive transport simulations with HPx$_{\\rm py}$ within cement system 1, under both advective-dispersive and diffusive transport conditions. As written above, the domain sizes are both 61 $\\times$ 61 and 121 $\\times$ 121 for the advection-dispersion case and, because of computational constraints, solely 61 $\\times$ 61  for the diffusion case. In addition, the simulation time period is 2 years for the advection-dispersion case and 1 year for the diffusion case. Figures \\ref{fig3} and \\ref{fig4} present times series of original and emulated Ca, Si, H and O concentrations at 5 locations within the 2D domain for advective-dispersive transport conditions, for both our kNN-based (HPx$_{\\rm py}$-kNN) and DNN-based (HPx$_{\\rm py}$-DNN) reactive transport codes. It is seen that HPx$_{\\rm py}$-kNN and HPx$_{\\rm py}$-DNN both induce a quite good simulation accuracy. Also, the results for the diffusive transport case are of similarly good quality (not shown). Figures \\ref{fig5} - \\ref{fig6} provide more insights into the HPx$_{\\rm py}$-kNN and HPx$_{\\rm py}$-DNN performances by displaying 2D Ca, Si, H and O concentration profiles at a given time. For each experiment and chemical component, this time is selected as to be well representative of the simulated dynamics. It is observed that the original and emulated images are visually almost indistinguishable for the advection-dispersion case (Figure \\ref{fig5}). For the diffusion case, the emulators also perform quite well for Ca, H and O (Figures\\ref{fig6}a - c, g - i and j - l), while some slight to moderate discrepancies appear at the concentration front for Si (Figures \\ref{fig6}d - f). Notwithstanding, the Si concentration remains globally well predicted. Furthermore, Figures \\ref{fig7} - \\ref{fig8} present the original and emulated 2D solid amount profiles corresponding to Figures \\ref{fig5} - \\ref{fig6}. The original solid amount profiles are overall well approximated by HPx$_{\\rm py}$-kNN and HPx$_{\\rm py}$-DNN for the advection-dispersion case (Figure \\ref{fig7}), even though some mismatch appears at the border of the fully depleted zone for the H component. As of the diffusion case (Figure \\ref{fig8}), the same kind of mismatch is observed for the emulated solid amounts of H by HPx$_{\\rm py}$-kNN while the emulated profiles by HPx$_{\\rm py}$-DNN show some more discrepancies. Though we decided to present raw emulation results, we would like to stress that some if not all of the observed artifacts could likely be smoothed out by using some post-filtering such as median filtering.\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure3.png}\n\t\\caption{Time series of original (RTM, solid green lines) and HPx$_{\\rm py}$-kNN emulated (TM+kNN, dashed orange lines) concentrations (mol/kg) of Ca, H, O and Si at selected observations points for cement system 1 and advective-dispersive transport. Obs. 1 - 5 denote the selected observation points, with the following $\\left[x,y\\right]$ locations (in cm). Obs. 1:  $\\left[0.5, 2.5\\right]$, Obs. 2: $\\left[1, 2\\right]$, Obs. 3: $\\left[2, 2\\right]$, Obs. 4: $\\left[1, 1\\right]$, Obs. 5: $\\left[2, 1\\right]$. The results for the 121 $\\times$ 121 grid size are rather similar.}\n\t\\label{fig3}\n\\end{figure}\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure4.png}\n\t\\caption{Time series of original (RTM, solid green lines) and HPx$_{\\rm py}$-DNN emulated (TM+DNN, dashed orange lines) concentrations (mol/kg) of Ca, H, O and Si at selected observations points for cement system 1 and advective-dispersive transport. Obs. 1 - 5 denote the selected observation points, with the following $\\left[x,y\\right]$ locations (in cm). Obs. 1:  $\\left[0.5, 2.5\\right]$, Obs. 2: $\\left[1, 2\\right]$, Obs. 3: $\\left[2, 2\\right]$, Obs. 4: $\\left[1, 1\\right]$, Obs. 5: $\\left[2, 1\\right]$. The results for the 121 $\\times$ 121 grid size are rather similar.}\n\t\\label{fig4}\n\\end{figure}\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure5.png}\n\t\\caption{2D concentration profiles obtained for cement system 1 at the end of the 2-year simulation performed for the advection-dispersion case. RTM means the original HP$_{\\rm{4C}}$ model, TM+kNN denotes the Hydrus transport model coupled with our kNN geochemical emulator (HPx$_{\\rm py}$-kNN) and TM+DNN signifies the Hydrus transport model coupled with our DNN geochemical emulator (HPx$_{\\rm py}$-DNN). The first to fourth row present profiles for $Ca^{\\rm{conc}}$, $Si^{\\rm{conc}}$, $H^{\\rm{conc}}$, and $O^{\\rm{conc}}$, respectively. The considered grid size is 61 $\\times$ 61. The results for the 121 $\\times$ 121 grid are rather similar.}\n\t\\label{fig5}\n\\end{figure}\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure6.png}\n\t\\caption{2D concentration profiles obtained for cement system 1 at the final time step of the 1-year simulation performed for the diffusion case. RTM means the original HP$_{\\rm{4C}}$ model, TM+kNN denotes the Hydrus transport model coupled with our kNN geochemical emulator (HPx$_{\\rm py}$-kNN) and TM+DNN signifies the Hydrus transport model coupled with our DNN geochemical emulator (HPx$_{\\rm py}$-DNN). The first to fourth row present profiles for $Ca^{\\rm{conc}}$, $Si^{\\rm{conc}}$, $H^{\\rm{conc}}$, and $O^{\\rm{conc}}$, respectively. The considered grid size is 61 $\\times$ 61.}\n\t\\label{fig6}\n\\end{figure}\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure7.png}\n\t\\caption{2D solid amount profiles obtained for cement system 1 at the end of the 2-year simulation performed for the advection-dispersion case. RTM means the original HP$_{\\rm{4C}}$ model, TM+kNN denotes the Hydrus transport model coupled with our kNN geochemical emulator (HPx$_{\\rm py}$-kNN) and TM+DNN signifies the Hydrus transport model coupled with our DNN geochemical emulator (HPx$_{\\rm py}$-DNN). The first to fourth row present profiles for $Ca^{\\rm{solid}}$, $Si^{\\rm{solid}}$, $H^{\\rm{solid}}$, and $O^{\\rm{solid}}$, respectively. The considered grid size is 61 $\\times$ 61. The results for the 121 $\\times$ 121 grid are rather similar.}\n\t\\label{fig7}\n\\end{figure}\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure8.png}\n\t\\caption{2D solid amount profiles obtained for cement system 1 at the final time step of the 1-year simulation performed for the diffusion case. RTM means the original HP$_{\\rm{4C}}$ model, TM+kNN denotes the Hydrus transport model coupled with our kNN geochemical emulator (HPx$_{\\rm py}$-kNN) and TM+DNN signifies the Hydrus transport model coupled with our DNN geochemical emulator (HPx$_{\\rm py}$-DNN). The first to fourth row present profiles for $Ca^{\\rm{solid}}$, $Si^{\\rm{solid}}$, $H^{\\rm{solid}}$, and $O^{\\rm{solid}}$, respectively. The considered grid size is 61 $\\times$ 61.}\n\t\\label{fig8}\n\\end{figure}\n\nThe speedups associated with the considered problem are detailed in Table \\ref{table3}. It is noted that the GPU-based DNN emulator allows for a speedup that is close to optimal. Indeed, The DNN speedups overall represent 85 \\% to 95 \\% of the maximum possible speedups (that is, speedups that would be obtained if the geochemcical calculations would come at no cost at all). The speedups associated with single-threaded kNN remain substantial but only amount to 57 \\% - 65 \\% of the corresponding maximum speedups. As detailed in section \\ref{train_res1}, the used kNN and DNN implementations are found to be respectively 300 and 4000 faster than single-threaded PHREEQC when predicting 10,000 points all one once for this geochemical system. Based on these numbers one could have expected the achieved speedups to represent say 90 \\% (kNN) or 99 \\% (DNN) of the maximum possible ones. A large part of the gaps between achieved and maximum possible speedups is thus likely caused by the time required for communicating and exchanging data between the main C/C++ code and the Python-based emulators.\n\n\\begin{table}[h!]\n\t\\caption{Speedups offered by the KNN and DNN emulators in HPx$_{\\rm py}$ for the reactive transport simulations considered for cement system 1. The HPx$_{\\rm{4C}}$ calculations involve the parallelization of PHREEQC over our 4 CPUs. The HPx$_{\\rm{1C}}$ calculations are performed on a single CPU. The kNN predictions are performed on a single CPU using the scikit-learn implementation while the DNN predictions make use of our GPU. ML signifies the used machine learning method for emulation, TC denotes transport conditions (ADV: advection-dispersion, DIF: diffusion) and GS is the grid size. The maximum possible speedups associated with HPx$_{\\rm{4C}}$ and HPx$_{\\rm{1C}}$,  Max SP HPx$_{\\rm{4C}}$ and  Max SP HPx$_{\\rm{1C}}$, correspond to an hypothetical situation where the geochemical calculations incur zero computational cost.}\n\n\t\\begin{center}\n\t\t\\begin{tabular}{cccccccc}\n\t\t\t\\hline\n\t\t\t& & & & & & & \\\\\n\t\t\tML & TC & GS & HPx$_{\\rm{4C}}$ time (s) & SP HPx$_{\\rm{4C}}$ & Max SP HPx$_{\\rm{4C}}$ & SP HPx$_{\\rm{1C}}$ & Max SP HPx$_{\\rm{1C}}$ \\\\\n\t\t\tDNN & ADV & 61 $\\times$ 61 & 3189 & 6.8 & 7.7 & 24.5 & 28.5 \\\\\n\t\t\tkNN & ADV & 61 $\\times$ 61 & 3189 & 5.0 & 7.7 & 18.0 & 28.5 \\\\\n\t\t\tDNN & ADV & 121 $\\times$ 121 & 23,337 & 5.0 & 5.2 & 17.0 & 18.0 \\\\\n\t\t\tkNN & ADV & 121 $\\times$ 121 & 23,337 & 3.4 & 5.2 & 11.6 & 18.0 \\\\\n\t\t\tDNN & DIF & 61 $\\times$ 61 & 25,448 & 4.2 & 4.9 & 16.2 & 19.1 \\\\\n\t\t\tkNN & DIF & 61 $\\times$ 61 & 25,448 & 2.8 & 4.9 & 10.8 & 19.1 \\\\\n\t\t\t\\hline\n\t\t\\end{tabular}\n\t\\end{center}\n\n\t\\label{table3}\n\\end{table}\n\n\\FloatBarrier\n\n\\subsection{Al-C-Ca-S-Si Problem}\n\nFor this second cement system, the emulation problem consists of predicting at each time step of the RT simulation the (output) Al, C, Ca, S, Si, H and O aqueous concentrations (mol/kgw) from the (input) total amounts of Al, C, Ca, S and Si (mol). Here we focus on advective-dispersive transport only while similarly as for cement system 1, the considered domain sizes are 61 $\\times$ 61 and 121 $\\times$ 121. Moreover, the simulation time period is set to 6 years. \n\nAs mentioned earlier, for this higher-dimensional problem it is observed that the used scikit-learn kNN implementation becomes prohibitively slow compared to HPx$_{\\rm{4C}}$. We found that to get a good emulation accuracy the kNN training base needs to contain 1,000,000 samples (or more). This training base's size together with a 5-dimensional search space leads to an HPx$_{\\rm py}$-kNN reactive transport simulation time that is comparable to that of HPx$_{\\rm{4C}}$. Therefore, for this second cement system we built a custom kNN regressor around another kNN implementation contained in the FAISS package \\citep{faiss2017}. Our used FAISS variant allows for GPU computing and is much faster than scikit-learn for this cement system, but is slightly less accurate due to the use of an approximate rather than exact nearest neighbor search \\citep[see][for details]{faiss2017}.\n\n\\subsubsection{Training the DNN}\n\\label{train_res2}\n\nBuilding a good training set to perform a kNN search and learn the weights and biases of our DNN turned out to be a complicated task in this case. This because to make useful kNN predictions and/or learn an useful DNN, the training set must be sufficiently representative of the geochemical conditions encountered during the reactive transport simulation one wish to perform with HPx$_{\\rm py}$-kNN and the trained HPx$_{\\rm py}$-DNN. In contrast to cement system 1, creating the training set by sampling the $X$-space with a controlled randomness between predefined lower and upper bounds did not prove successful. We tried that strategy by drawing as much as 4,000,000 5-dimensional $\\textbf{x}$ vectors from the $X$-space using a Sobol low-discrepancy sequence \\citep[][]{Sobol1967, Joe-Kuo2003}. Such low-discrepancy sampling scheme covers the 5-dimensional hypercube more uniformly than LHS. Despite a good performance on the test set (not shown), the resulting DNN accuracy in reactive transport mode was never deemed satisfying. In other words, no satisfying ``global\" or ``universal\" DNN emulator could be devised for this cement system. This is probably caused by the fact for this problem, the input (5 total amounts) and output (7 aqueous concentrations) spaces are quite nonlinearly related and both cover 6 to 10 orders of magnitudes depending on the considered element. Therefore, we resorted to the alternative training strategy detailed below. The latter basically tries to grasp the complex correlations and higher-order dependencies that exist between the elements of $\\textbf{x}$ (total amounts, input space) for a given reactive transport simulation setup, in order to produce a training set that honors these between-input relationships.\n\n\\begin{itemize}\n\t\\item Perform a ``cheap\" full reactive transport simulation under the transport conditions and geochemistry of interest and collect the resulting $\\textbf{x}$ -$\\textbf{y}$  pairs of examples (for the considered grid nodes and time steps). Computational demand controls what domain size and simulation time period can be used for this cheap calculation. We used a modest 16 $\\times$ 16 domain and a simulation time period of 10 years. The associated HPx$_{\\rm{4C}}$ runtimes is 180 s.\n\t\n\t\\item Fit a kernel density estimator (KDE) with a Gaussian kernel to the collected $\\textbf{x}$ vectors (encapsulated in the $\\textbf{X}$ array) and generate a fixed number of new input vectors, $\\textbf{x}_{KDE}$. Then run PHREEQC for the $\\textbf{X}_{KDE}$ set to get the corresponding output set, $\\textbf{Y}_{KDE}$. Now apply the correction for porosity described in section \\ref{train_res1} to the $\\textbf{X}_{KDE}$ set and form the training set by merging the ensemble of $\\textbf{x}$-$\\textbf{y}$ pairs with that of the $\\textbf{x}_{KDE}$-$\\textbf{y}_{KDE}$ pairs. The number of produced unique examples by the considered cheap HPx$_{\\rm{4C}}$ simulations varied between 10,000 and 50,000. The KDE-based enrichment of this dataset was deemed necessary to provide more input variability, thereby avoiding overfitting of the trained DNN and improving the kNN accuracy, while still honoring the complex between-input relationships. The number of KDE-generated samples was set as to obtain a total training set size of 1,000,000 examples. A key component of the approach is the bandwith parameter of the Gaussian KDE kernel which controls how much the KDE-generated samples depart from the original ensemble. After limited trial and error, we fixed the kernel bandwith to 0.0025 for the considered case studies.\n\t\n\\end{itemize}\n\nThe scatter plots in Figure \\ref{fig9} illustrate our training set creation procedure. The orange dots depict the pairwise relationships observed between the 5 elements of $\\textbf{x}$ in the cheap simulation. The red and cyan dots in Figure \\ref{fig9} represent the KDE-generated samples with the selected bandwidth, before and after applying the correction for porosity, respectively. Training of the DNN is achieved using the ensemble of original and KDE-corrected input points. We refer to this kind of dataset as RT-based, since it is based on an albeit cheap, full RT simulation. Furthermore, we refer to the obtained DNN and kNN emulators as ``local\" emulators, since as opposed to the emulators constructed for cement system 1, the current emulators are only valid for the input conditions encapsulated in the RT-based training set.\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1.5cm}\\includegraphics[width=50pc]{Figure9.png}\n\t\\caption{Scatter plots of the complex relationships between the five considered inputs for (1) the computationally cheap RT simulation of cement system 2 (orange dots) performed with the original HPx code, (2) the corresponding sampled points by kernel density estimation (KDE) using the selected bandwith (turquoise dots) and, (3) the same sampled KDE points after porosity correction (red dots).}\n\t\\label{fig9}\n\\end{figure}\n\nTraining performance of the DNN emulator is presented in Figure \\ref{fig10}. Similarly as for cement system 1, about 90\\% of the available data was used for the actual training of the DNN while the remaining 10\\% were used a validation set to contol overfitting. Lastly, performance is evaluated for both the trained DNN and kNN emulators using an independent test set of 10,000 examples. Overall, the accuracy of our ``local \" DNN emulator for this RT-inspired dataset is rather large with $Q_2$ values always greater or equal than 0.998. Training performance of the corresponding local kNN emulator is equally good (not shown). Regarding speedup, for this problem single-threaded PHREEQC achieves about 210 geochemical calculations per second on our used Intel\\textsuperscript{\\textregistered} i7 CPU while the GPU-based DNN and kNN emulators are both about 3000 times faster when predicting the 10,000 test points at once .\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure10.png}\n\t\\caption{1-1 plots of local DNN emulation performance obtained for system 2 when the local DNN is trained using 1,000,000 samples. The $x$-axis and $y$-axis present the original and emulated 10,000 independent test data points, respectively. The RMSE and $Q_2$ coefficient denote the root-mean-square-error and coefficient of determination in testing mode, respectively, between the original and emulated 10,000 test data points.}\n\t\\label{fig10}\n\\end{figure}\n\n\\subsubsection{Reactive transport simulation}\n\\label{prob2res}\nOur ``local\" DNN performs rather well when applied to the 61 $\\times$ 61 grid size and a time period of 6 years (Figures \\ref{fig11} - \\ref{fig12}). This for all components but C, for which some localized deviations appear between original and emulated 2D concentration profiles towards the end of the simulation period (Figure \\ref{fig12}). When applied to the 121 $\\times$ 121 grid, HPx$_{\\rm py}$-DNN produces additional discrepancies for O, Si and Al towards the end of the simulation period (Figures \\ref{fig13} - \\ref{fig14}). Yet most of the observed artifacts could probably be smoothed out by using post-filtering. The associated speedups are listed in Table \\ref{table4}. These speedups are larger to those obtained for cement system 1, with values between 8 and 9 when evaluated against HPx$_{\\rm{4C}}$. These speedups represent about 85 \\% to 90 \\% of the maximum possible speedup (Table \\ref{table3}). Overall, these findings indicate that for the considered problem, our RT-based training of a local DNN only works is the training set is sufficiently representative of the particular geochemical conditions encountered in the computationally demanding simulations, which is arguably not easy to achieve. This limitation is further discussed in section \\ref{discussion}.\n\nWe note a more uniform behavior for HPx$_{\\rm py}$-kNN across grid sizes than for HPx$_{\\rm py}$-DNN. Here the results for the 121  $\\times$ 121 grid are only slightly less accurate than those associated with the 61 $ \\times$ 61 grid (see Figures \\ref{fig15} - \\ref{fig16} where for brevity we only show concentration profiles for the C Al and S components). Furthermore, whenever observed the discrepancies between original and emulated profiles are more regularly scattered than for HPx$_{\\rm py}$-DNN. Note also that herein too, post-filtering could likely smooth out a large part of these deviations. In addition, owing to the use of a GPU to achieve the kNN calculations, the speedup provided by HPx$_{\\rm py}$-kNN are as large as those provided by HPx$_{\\rm py}$-DNN (Table \\ref{table4}).\n\nWith respect to the emulated solid amounts, the HPx$_{\\rm py}$-DNN results look visually good for the the 61 $\\times$ 61 grid. This is shown in Figure \\ref{fig17} for the C, Al and S chemical components, while emulation of the H, O Ca and Si chemical components is globally of similar quality (not shown). Nevertheless, significant deviations appear towards the end of the simulation period for every chemical component (see Figure \\ref{fig18} for the C, Al and S chemical components, emulation of the H, O Ca and Si chemical components shows the same level of mismatch). The HPx$_{\\rm py}$-kNN predictions are also fairly accurate for the 61 $\\times$ 61 (see Figure \\ref{fig19} for the C, Al and S chemical components, emulation of the H, O Ca and Si chemical components exhibits a globally similar quality) while some discrepancies show up at the end of the simulation (Figure \\ref{fig20}). However, the mismatch is less pronounced than for HPx$_{\\rm py}$-DNN. Overall, HPx$_{\\rm py}$-kNN appears to be somewhat more robust than HPx$_{\\rm py}$-DNN for this cement system, while providing the same (large) speedup.\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure11.png}\n\t\\caption{2D concentration profiles obtained for cement system 2 after 2 and 6 (final time step) years. RTM means the original HPx$_{\\rm{4C}}$ model and TM+DNN denotes the Hydrus transport model coupled with our DNN geochemical emulator (HPx$_{\\rm py}$-DNN). The first to fourth row present profiles for $Ca^{\\rm{conc}}$, $Si^{\\rm{conc}}$, $H^{\\rm{conc}}$, and $O^{\\rm{conc}}$, respectively. The considered grid size is 61 $\\times$ 61.}\n\t\\label{fig11}\n\\end{figure}\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure12.png}\n\t\\caption{2D concentration profiles obtained for cement system 2 after 2 and 6 (final time step) years. RTM means the original HPx$_{\\rm{4C}}$ model and TM+DNN denotes the Hydrus transport model coupled with our DNN geochemical emulator (HPx$_{\\rm py}$-DNN). The first to third row present profiles for $C^{\\rm{conc}}$, $Al^{\\rm{conc}}$, and $S^{\\rm{conc}}$, respectively. The considered grid size is 61 $\\times$ 61.}\n\t\\label{fig12}\n\\end{figure}\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure13.png}\n\t\\caption{2D concentration profiles obtained for cement system 2 after 2 and 6 (final time step) years. RTM means the original HPx$_{\\rm{4C}}$ model and TM+DNN denotes the Hydrus transport model coupled with our DNN geochemical emulator (HPx$_{\\rm py}$-DNN). The first to fourth row present profiles for $Ca^{\\rm{conc}}$, $Si^{\\rm{conc}}$, $H^{\\rm{conc}}$, and $O^{\\rm{conc}}$, respectively. The considered grid size is 121 $\\times$ 121.}\n\t\\label{fig13}\n\\end{figure}\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure14.png}\n\t\\caption{2D concentration profiles obtained for cement system 2 after 2 and 6 (final time step) years. RTM means the original HPx$_{\\rm{4C}}$ model and TM+DNN denotes the Hydrus transport model coupled with our DNN geochemical emulator (HPx$_{\\rm py}$-DNN). The first to third row present profiles for $C^{\\rm{conc}}$, $Al^{\\rm{conc}}$, and $S^{\\rm{conc}}$, respectively. The considered grid size is 121 $\\times$ 121.}\n\t\\label{fig14}\n\\end{figure}\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure15.png}\n\t\\caption{2D concentration profiles obtained for cement system 2 after 2 and 6 (final time step) years. RTM means the original HPx$_{\\rm{4C}}$ model and TM+kNN denotes the Hydrus transport model coupled with our kNN geochemical emulator (HPx$_{\\rm py}$-kNN).  The first to third row present profiles for $C^{\\rm{conc}}$, $Al^{\\rm{conc}}$, and $S^{\\rm{conc}}$, respectively. The considered grid size is 61 $\\times$ 61.}\n\t\\label{fig15}\n\\end{figure}\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure16.png}\n\t\\caption{2D concentration profiles obtained for cement system 2 after 2 and 6 (final time step) years. RTM means the original HPx$_{\\rm{4C}}$ model and TM+kNN denotes the Hydrus transport model coupled with our kNN geochemical emulator (HPx$_{\\rm py}$-kNN). The first to third row present profiles for $C^{\\rm{conc}}$, $Al^{\\rm{conc}}$, and $S^{\\rm{conc}}$, respectively. The considered grid size is 121 $\\times$ 121.}\n\t\\label{fig16}\n\\end{figure}\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure17.png}\n\t\\caption{2D solid amount profiles of the C, Al and S chemical components of cement system 2 after 2 and 6 (final time step) years. RTM means the original HPx$_{\\rm{4C}}$ model and TM+DNN denotes the Hydrus transport model coupled with our DNN geochemical emulator (HPx$_{\\rm py}$-DNN). The first to third row present profiles for $C^{\\rm{solid}}$, $Al^{\\rm{solid}}$, and $S^{\\rm{solid}}$, respectively The considered grid size is 61 $\\times$ 61.}\n\t\\label{fig17}\n\\end{figure}\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure18.png}\n\t\\caption{2D solid amount profiles of the C, Al and S chemical components of cement system 2 after 2 and 6 (final time step) years. RTM means the original HPx$_{\\rm{4C}}$ model and TM+DNN denotes the Hydrus transport model coupled with our DNN geochemical emulator (HPx$_{\\rm py}$-DNN). The first to third row present profiles for $C^{\\rm{solid}}$, $Al^{\\rm{solid}}$, and $S^{\\rm{solid}}$, respectively The considered grid size is 121 $\\times$ 121.}\n\t\\label{fig18}\n\\end{figure}\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure19.png}\n\t\\caption{2D solid amount profiles of the C, Al and S chemical components of cement system 2 after 2 and 6 (final time step) years. RTM means the original HPx$_{\\rm{4C}}$ model and TM+kNN denotes the Hydrus transport model coupled with our kNN geochemical emulator (HPx$_{\\rm py}$-kNN). The first to third row present profiles for $C^{\\rm{solid}}$, $Al^{\\rm{solid}}$, and $S^{\\rm{solid}}$, respectively The considered grid size is 61 $\\times$ 61.}\n\t\\label{fig19}\n\\end{figure}\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure20.png}\n\t\\caption{2D solid amount profiles of the C, Al and S chemical components of cement system 2 after 2 and 6 (final time step) years. RTM means the original HPx$_{\\rm{4C}}$ model and TM+kNN denotes the Hydrus transport model coupled with our kNN geochemical emulator (HPx$_{\\rm py}$-kNN). The first to third row present profiles for $C^{\\rm{solid}}$, $Al^{\\rm{solid}}$, and $S^{\\rm{solid}}$, respectively The considered grid size is 121 $\\times$ 121.}\n\t\\label{fig20}\n\\end{figure}\n\n\\begin{table}[h!]\n\t\\caption{Speedups offered by the KNN and DNN emulators in HPx$_{\\rm py}$ for the reactive transport simulations considered for cement system 2. The HPx$_{\\rm{4C}}$ calculations involve the parallelization of PHREEQC over our 4 CPUs. The HPx$_{\\rm{1C}}$ calculations are performed on a single CPU. Both the kNN and DNN predictions make use of a GPU. ML signifies the used machine learning method for emulation, BC denotes the type of flow boundary conditions and GS is the grid size. The maximum possible speedups associated with HPx$_{\\rm{4C}}$ and HP$_{\\rm{1C}}$,  Max SP HPx$_{\\rm{4C}}$ and  Max SP HPx$_{\\rm{1C}}$, correspond to an hypothetical situation where the geochemical calculations incur zero computational cost.}\n\n\t\\begin{center}\n\t\t\\begin{tabular}{cccccccc}\n\t\t\t\\hline\n\t\t\t& & & & & & & \\\\\n\t\t\tML & BC & GS & HPx$_{\\rm{4C}}$ time (s) & SP HPx$_{\\rm{4C}}$ & Max SP HPx$_{\\rm{4C}}$ & SP HPx$_{\\rm{1C}}$ & Max SP HPx$_{\\rm{1C}}$ \\\\\n\t\t\tDNN & ADV & 61 $\\times$ 61 & 21,415 & 8.2 & 9.0 & 30.3 & 33.1\\\\\n\t\t\tkNN & ADV & 61 $\\times$ 61 & 21,415 & 7.9 & 9.0 & 28.9 & 33.1\\\\\n\t\t\tDNN & ADV & 121 $\\times$ 121 & 199,841 & 8.2 & 9.5 & 29.9 & 32.8\\\\\n\t\t\tkNN & ADV & 121 $\\times$ 121 & 199,841 & 8.5 & 9.5 & 31.3 & 32.8\\\\\n\t\t\t\\hline\n\t\t\\end{tabular}\n\t\\end{center}\n\n\t\\label{table4}\n\\end{table}\n\n\\FloatBarrier\n\n3.1 Ca-Si Problem\n\\subsection{Ca-Si Problem}\nFor this first cement system, the emulation problem consists of predicting at each time step of the RT simulation the (output) Ca, Si, H and O aqueous concentrations (mol/kg of water or mol/kgw) from the (input) total amounts of Ca and Si (mol). \n\n\\subsubsection{Training the emulators}\n\\label{train_res1}\nHere the kNN and DNN emulators are firstly trained using a set of 400,000 test examples for both. This training set is obtained by randomly sampling the two-dimensional input space by latin hypercube sampling (LHS) between $\\left[0,0\\right]$ and $\\rm{\\left[Ca^{tot}_{max},Si^{tot}_{max}\\right]}$, and running PHREEQC for each input sample, $\\rm{\\textbf{x}_i = \\left[Ca^{tot}_i, Si^{tot}_i\\right]}$ to get the corresponding output vectors, $\\rm{\\textbf{y}_i = \\left[Ca^{conc}_i,Si^{conc}_i,H^{conc}_i,O^{conc}_i\\right]}$. The upper bounds, $\\rm{Ca^{tot}_{max}}$ and $\\rm{Si^{tot}_{max}}$ are defined based on a cheap full RT simulation with advective-dispersive transport using a small 1D domain of 51 nodes. It is worth noting that the total amounts of $\\textbf{x}_i$, corresponding to the PHREEQC-simulated concentrations, $\\textbf{y}_i$, have to be corrected for the different amount of water between the training set and the transport simulations. Doing so, it turns out that about 20 \\% of the post-corrected $x_i$ values exceed their pre-defined upper bounds and these excessively large values need to be filtered out. Creating the 400,000 training examples thus required about 500,000 PHREEQC runs. As stated earlier, for this problem single-threaded PHREEQC performs about 670 geochemical calculations per second on our used Intel\\textsuperscript{\\textregistered} i7 CPU.\n\nWith respect to kNN, the tuning parameters are the number of neighbors, $k$, the type of distance measure, and the interpolation technique. We simply used the default settings: $k = 5$, euclidean distance and inverse-distance interpolation. Regarding training of the DNN, the 400,000 sample were split between the training set itself (90 \\% of the data) and a validation set (10 \\% of the data). The latter serves to monitor the evolution of the selected mean squared error loss function on samples that are not used for training, thereby detecting potential overfitting. If the validation loss stops decreasing before the fixed number of epochs has been completed, then  training is stopped. Importantly, the emulation is achieved in log-space for both the input, $X$ and output $Y$, domains. This because total amounts and concentrations of the involved components typically cover many orders of magnitudes (up to 10 orders or more). Using a DNN also requires some form of data normalization or standardization. Here both the $\\rm{log\\left(\\textbf{x}_i\\right)}$ and $\\rm{log\\left(\\textbf{y}_i\\right)}$ vectors are standardized around 0 with standard deviation of 1.\n\nFigure \\ref{fig1} illustrates the trained emulators' performance for geochemical predictions using an independent test set that comprises 10,000 test examples. Both kNN (Figures \\ref{fig1}a - d) and DNN (Figures \\ref{fig1}e - h) appear to be rather accurate . DNN also shows a slight degradation for the larger concentration values  (Figures \\ref{fig1}e - h). The latter is likely due to the combination of a small proportion of large concentration values in the training set with the log-transformation that implicitly pushes the DNN to try harder to fit the smaller concentrations during training. Regarding speedup and as written earlier, for this setup the single-threaded kNN method is 300 times faster than single-threaded PHREEQC for predicting the 10,000 concentration vectors all at once. The computational savings allowed by the DNN emulator when ran on our GPU is higher with a speedup as large as 4000 for predicting the same 10,000 concentration vectors all at once. \n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{0cm}\\includegraphics[width=35pc]{Figure1.png}\n\t\\caption{1-1 plots of the kNN (subfigures (a) - (d)) and DNN (subfigures (e) - (h)) emulators' performance obtained for system 1 when the kNN training base contains 400,000 samples and the DNN is trained using the same 400,000 samples. Here ``true\" means the original PHREEQC-simulated data and ``predicted\" denotes the emulated (that is, kNN-simulated and DNN-simulated) data. Hence, the $x$-axis and $y$-axis present the original and emulated 10,000 independent test data points, respectively. The RMSE and $Q_2$ coefficient denote the root-mean-square-error and coefficient of determination in testing mode, respectively, between the original and emulated 10,000 test data points.}\n\t\\label{fig1}\n\\end{figure}\n\nTo test the sensitivity of the emulators' performance to the training set size, training was also performed using reduced training sets comprising 100,000, 10,000, 1000 and 100 samples, respectively. It seen that the DNN performance achieved  when using 10,000 training samples is virtually the same as that obtained when using 400,000 training samples (Figure \\ref{fig2} and Table \\ref{table2}). It is only for training sets smaller than 1000 samples that the DNN performance starts to degrade significantly (Table \\ref{table2}). In contrast, the behavior of kNN appears to markedly decrease as the training set gets smaller (Figure \\ref{fig2} and Table \\ref{table1}).\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{0cm}\\includegraphics[width=35pc]{Figure2.png}\n\t\\caption{1-1 plots of the kNN (subfigures (a) - (d)) and DNN (subfigures (e) - (h)) emulators' performance obtained for system 1 when the kNN training base contains 10,000 samples and the DNN is trained using the same 10,000 samples. Here ``true\" means the original PHREEQC-simulated data and ``predicted\" denotes the emulated (that is, kNN-simulated and DNN-simulated) data. Hence, the $x$-axis and $y$-axis present the original and emulated 10,000 independent test data points, respectively. The RMSE and $Q_2$ coefficient denote the root-mean-square-error and coefficient of determination in testing mode, respectively, between the original and emulated 10,000 test data points.}\n\t\\label{fig2}\n\\end{figure}\n\n\\begin{table}[h!]\n\t\\caption{Performance of the DNN and kNN emulators for cement system 1 and different training set sizes. For brevity, only the results for Ca$^{conc}$ and Si$^{conc}$ are shown. The units are mol per kg of water (mol/kgw). ML refers to the type of emulator, TR signifies the size of the training set, and the RMSE and $Q_2$ coefficient denote the root-mean-square-error and coefficient of determination in testing mode, respectively, between the original and emulated 10,000 test data points.}\n\t\\begin{center}\n\t\t\\begin{tabular}{cccccccc}\n\t\t\t\\hline\n\t\t\t& & & & & \\\\\n\t\t\tML & TR & RMSE - Ca$^{conc}$ & $Q_2$ - Ca$^{conc}$ & RMSE - Si$^{conc}$ & $Q_2$ - Si$^{conc}$\\\\\n\t\t\tDNN & 4 $\\times$ 10$^{5}$ & 4.50 $\\times$ 10$^{-5}$ & 0.9999 & 1.21 $\\times$ 10$^{-6}$ & 0.9999\\\\\n\t\t\tDNN & 1 $\\times$ 10$^{5}$ & 5.06 $\\times$ 10$^{-5}$ & 0.9999 & 1.30 $\\times$ 10$^{-6}$ & 0.9999\\\\\n\t\t\tDNN & 1 $\\times$ 10$^{4}$ & 4.57 $\\times$ 10$^{-5}$ & 0.9999 & 1.48 $\\times$ 10$^{-6}$ & 0.9999\\\\\n\t\t\tDNN & 1 $\\times$ 10$^{3}$ & 8.02 $\\times$ 10$^{-5}$ & 0.9998 & 3.31 $\\times$ 10$^{-6}$ & 0.9994\\\\\n\t\t\tDNN & 1 $\\times$ 10$^{2}$ & 47.1 $\\times$ 10$^{-5}$ & 0.9935 & 9.02 $\\times$ 10$^{-6}$ & 0.9959\\\\\n\t\t\tkNN & 4 $\\times$ 10$^{5}$ & 2.52 $\\times$ 10$^{-5}$ & 1.0000 & 1.59 $\\times$ 10$^{-6}$ & 0.9999\\\\\n\t\t\tkNN & 1 $\\times$ 10$^{5}$ & 6.27 $\\times$ 10$^{-5}$ & 0.9999 & 4.20 $\\times$ 10$^{-6}$ & 0.9991\\\\\n\t\t\tkNN & 1 $\\times$ 10$^{4}$ & 14.1 $\\times$ 10$^{-5}$ & 0.9994 & 13.5 $\\times$ 10$^{-6}$ & 0.9908\\\\\n\t\t\tkNN & 1 $\\times$ 10$^{3}$ & 43.9 $\\times$ 10$^{-5}$ & 0.9944 & 74.6 $\\times$ 10$^{-6}$ & 0.7180\\\\\n\t\t\tkNN & 1 $\\times$ 10$^{2}$  & 96.4 $\\times$ 10$^{-5}$ & 0.9729 & 55.0 $\\times$ 10$^{-6}$ & 0.8468\\\\\n\t\t\\end{tabular}\n\t\\end{center}\n\t\\label{table2}\n\\end{table}\n\n\\FloatBarrier\n\n\\subsubsection{Reactive transport simulation}\n\\label{prob12Dres}\n\nThis section focuses on reactive transport simulations with HPx$_{\\rm py}$ within cement system 1, under both advective-dispersive and diffusive transport conditions. As written above, the domain sizes are both 61 $\\times$ 61 and 121 $\\times$ 121 for the advection-dispersion case and, because of computational constraints, solely 61 $\\times$ 61  for the diffusion case. In addition, the simulation time period is 2 years for the advection-dispersion case and 1 year for the diffusion case. Figures \\ref{fig3} and \\ref{fig4} present times series of original and emulated Ca, Si, H and O concentrations at 5 locations within the 2D domain for advective-dispersive transport conditions, for both our kNN-based (HPx$_{\\rm py}$-kNN) and DNN-based (HPx$_{\\rm py}$-DNN) reactive transport codes. It is seen that HPx$_{\\rm py}$-kNN and HPx$_{\\rm py}$-DNN both induce a quite good simulation accuracy. Also, the results for the diffusive transport case are of similarly good quality (not shown). Figures \\ref{fig5} - \\ref{fig6} provide more insights into the HPx$_{\\rm py}$-kNN and HPx$_{\\rm py}$-DNN performances by displaying 2D Ca, Si, H and O concentration profiles at a given time. For each experiment and chemical component, this time is selected as to be well representative of the simulated dynamics. It is observed that the original and emulated images are visually almost indistinguishable for the advection-dispersion case (Figure \\ref{fig5}). For the diffusion case, the emulators also perform quite well for Ca, H and O (Figures\\ref{fig6}a - c, g - i and j - l), while some slight to moderate discrepancies appear at the concentration front for Si (Figures \\ref{fig6}d - f). Notwithstanding, the Si concentration remains globally well predicted. Furthermore, Figures \\ref{fig7} - \\ref{fig8} present the original and emulated 2D solid amount profiles corresponding to Figures \\ref{fig5} - \\ref{fig6}. The original solid amount profiles are overall well approximated by HPx$_{\\rm py}$-kNN and HPx$_{\\rm py}$-DNN for the advection-dispersion case (Figure \\ref{fig7}), even though some mismatch appears at the border of the fully depleted zone for the H component. As of the diffusion case (Figure \\ref{fig8}), the same kind of mismatch is observed for the emulated solid amounts of H by HPx$_{\\rm py}$-kNN while the emulated profiles by HPx$_{\\rm py}$-DNN show some more discrepancies. Though we decided to present raw emulation results, we would like to stress that some if not all of the observed artifacts could likely be smoothed out by using some post-filtering such as median filtering.\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure3.png}\n\t\\caption{Time series of original (RTM, solid green lines) and HPx$_{\\rm py}$-kNN emulated (TM+kNN, dashed orange lines) concentrations (mol/kg) of Ca, H, O and Si at selected observations points for cement system 1 and advective-dispersive transport. Obs. 1 - 5 denote the selected observation points, with the following $\\left[x,y\\right]$ locations (in cm). Obs. 1:  $\\left[0.5, 2.5\\right]$, Obs. 2: $\\left[1, 2\\right]$, Obs. 3: $\\left[2, 2\\right]$, Obs. 4: $\\left[1, 1\\right]$, Obs. 5: $\\left[2, 1\\right]$. The results for the 121 $\\times$ 121 grid size are rather similar.}\n\t\\label{fig3}\n\\end{figure}\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure4.png}\n\t\\caption{Time series of original (RTM, solid green lines) and HPx$_{\\rm py}$-DNN emulated (TM+DNN, dashed orange lines) concentrations (mol/kg) of Ca, H, O and Si at selected observations points for cement system 1 and advective-dispersive transport. Obs. 1 - 5 denote the selected observation points, with the following $\\left[x,y\\right]$ locations (in cm). Obs. 1:  $\\left[0.5, 2.5\\right]$, Obs. 2: $\\left[1, 2\\right]$, Obs. 3: $\\left[2, 2\\right]$, Obs. 4: $\\left[1, 1\\right]$, Obs. 5: $\\left[2, 1\\right]$. The results for the 121 $\\times$ 121 grid size are rather similar.}\n\t\\label{fig4}\n\\end{figure}\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure5.png}\n\t\\caption{2D concentration profiles obtained for cement system 1 at the end of the 2-year simulation performed for the advection-dispersion case. RTM means the original HP$_{\\rm{4C}}$ model, TM+kNN denotes the Hydrus transport model coupled with our kNN geochemical emulator (HPx$_{\\rm py}$-kNN) and TM+DNN signifies the Hydrus transport model coupled with our DNN geochemical emulator (HPx$_{\\rm py}$-DNN). The first to fourth row present profiles for $Ca^{\\rm{conc}}$, $Si^{\\rm{conc}}$, $H^{\\rm{conc}}$, and $O^{\\rm{conc}}$, respectively. The considered grid size is 61 $\\times$ 61. The results for the 121 $\\times$ 121 grid are rather similar.}\n\t\\label{fig5}\n\\end{figure}\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure6.png}\n\t\\caption{2D concentration profiles obtained for cement system 1 at the final time step of the 1-year simulation performed for the diffusion case. RTM means the original HP$_{\\rm{4C}}$ model, TM+kNN denotes the Hydrus transport model coupled with our kNN geochemical emulator (HPx$_{\\rm py}$-kNN) and TM+DNN signifies the Hydrus transport model coupled with our DNN geochemical emulator (HPx$_{\\rm py}$-DNN). The first to fourth row present profiles for $Ca^{\\rm{conc}}$, $Si^{\\rm{conc}}$, $H^{\\rm{conc}}$, and $O^{\\rm{conc}}$, respectively. The considered grid size is 61 $\\times$ 61.}\n\t\\label{fig6}\n\\end{figure}\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure7.png}\n\t\\caption{2D solid amount profiles obtained for cement system 1 at the end of the 2-year simulation performed for the advection-dispersion case. RTM means the original HP$_{\\rm{4C}}$ model, TM+kNN denotes the Hydrus transport model coupled with our kNN geochemical emulator (HPx$_{\\rm py}$-kNN) and TM+DNN signifies the Hydrus transport model coupled with our DNN geochemical emulator (HPx$_{\\rm py}$-DNN). The first to fourth row present profiles for $Ca^{\\rm{solid}}$, $Si^{\\rm{solid}}$, $H^{\\rm{solid}}$, and $O^{\\rm{solid}}$, respectively. The considered grid size is 61 $\\times$ 61. The results for the 121 $\\times$ 121 grid are rather similar.}\n\t\\label{fig7}\n\\end{figure}\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure8.png}\n\t\\caption{2D solid amount profiles obtained for cement system 1 at the final time step of the 1-year simulation performed for the diffusion case. RTM means the original HP$_{\\rm{4C}}$ model, TM+kNN denotes the Hydrus transport model coupled with our kNN geochemical emulator (HPx$_{\\rm py}$-kNN) and TM+DNN signifies the Hydrus transport model coupled with our DNN geochemical emulator (HPx$_{\\rm py}$-DNN). The first to fourth row present profiles for $Ca^{\\rm{solid}}$, $Si^{\\rm{solid}}$, $H^{\\rm{solid}}$, and $O^{\\rm{solid}}$, respectively. The considered grid size is 61 $\\times$ 61.}\n\t\\label{fig8}\n\\end{figure}\n\nThe speedups associated with the considered problem are detailed in Table \\ref{table3}. It is noted that the GPU-based DNN emulator allows for a speedup that is close to optimal. Indeed, The DNN speedups overall represent 85 \\% to 95 \\% of the maximum possible speedups (that is, speedups that would be obtained if the geochemcical calculations would come at no cost at all). The speedups associated with single-threaded kNN remain substantial but only amount to 57 \\% - 65 \\% of the corresponding maximum speedups. As detailed in section \\ref{train_res1}, the used kNN and DNN implementations are found to be respectively 300 and 4000 faster than single-threaded PHREEQC when predicting 10,000 points all one once for this geochemical system. Based on these numbers one could have expected the achieved speedups to represent say 90 \\% (kNN) or 99 \\% (DNN) of the maximum possible ones. A large part of the gaps between achieved and maximum possible speedups is thus likely caused by the time required for communicating and exchanging data between the main C/C++ code and the Python-based emulators.\n\n\\begin{table}[h!]\n\t\\caption{Speedups offered by the KNN and DNN emulators in HPx$_{\\rm py}$ for the reactive transport simulations considered for cement system 1. The HPx$_{\\rm{4C}}$ calculations involve the parallelization of PHREEQC over our 4 CPUs. The HPx$_{\\rm{1C}}$ calculations are performed on a single CPU. The kNN predictions are performed on a single CPU using the scikit-learn implementation while the DNN predictions make use of our GPU. ML signifies the used machine learning method for emulation, TC denotes transport conditions (ADV: advection-dispersion, DIF: diffusion) and GS is the grid size. The maximum possible speedups associated with HPx$_{\\rm{4C}}$ and HPx$_{\\rm{1C}}$,  Max SP HPx$_{\\rm{4C}}$ and  Max SP HPx$_{\\rm{1C}}$, correspond to an hypothetical situation where the geochemical calculations incur zero computational cost.}\n\n\t\\begin{center}\n\t\t\\begin{tabular}{cccccccc}\n\t\t\t\\hline\n\t\t\t& & & & & & & \\\\\n\t\t\tML & TC & GS & HPx$_{\\rm{4C}}$ time (s) & SP HPx$_{\\rm{4C}}$ & Max SP HPx$_{\\rm{4C}}$ & SP HPx$_{\\rm{1C}}$ & Max SP HPx$_{\\rm{1C}}$ \\\\\n\t\t\tDNN & ADV & 61 $\\times$ 61 & 3189 & 6.8 & 7.7 & 24.5 & 28.5 \\\\\n\t\t\tkNN & ADV & 61 $\\times$ 61 & 3189 & 5.0 & 7.7 & 18.0 & 28.5 \\\\\n\t\t\tDNN & ADV & 121 $\\times$ 121 & 23,337 & 5.0 & 5.2 & 17.0 & 18.0 \\\\\n\t\t\tkNN & ADV & 121 $\\times$ 121 & 23,337 & 3.4 & 5.2 & 11.6 & 18.0 \\\\\n\t\t\tDNN & DIF & 61 $\\times$ 61 & 25,448 & 4.2 & 4.9 & 16.2 & 19.1 \\\\\n\t\t\tkNN & DIF & 61 $\\times$ 61 & 25,448 & 2.8 & 4.9 & 10.8 & 19.1 \\\\\n\t\t\t\\hline\n\t\t\\end{tabular}\n\t\\end{center}\n\n\t\\label{table3}\n\\end{table}\n\n\\FloatBarrier\n\n3.1.2 Reactive transport simulation\n\\subsubsection{Reactive transport simulation}\n\\label{prob12Dres}\n\nThis section focuses on reactive transport simulations with HPx$_{\\rm py}$ within cement system 1, under both advective-dispersive and diffusive transport conditions. As written above, the domain sizes are both 61 $\\times$ 61 and 121 $\\times$ 121 for the advection-dispersion case and, because of computational constraints, solely 61 $\\times$ 61  for the diffusion case. In addition, the simulation time period is 2 years for the advection-dispersion case and 1 year for the diffusion case. Figures \\ref{fig3} and \\ref{fig4} present times series of original and emulated Ca, Si, H and O concentrations at 5 locations within the 2D domain for advective-dispersive transport conditions, for both our kNN-based (HPx$_{\\rm py}$-kNN) and DNN-based (HPx$_{\\rm py}$-DNN) reactive transport codes. It is seen that HPx$_{\\rm py}$-kNN and HPx$_{\\rm py}$-DNN both induce a quite good simulation accuracy. Also, the results for the diffusive transport case are of similarly good quality (not shown). Figures \\ref{fig5} - \\ref{fig6} provide more insights into the HPx$_{\\rm py}$-kNN and HPx$_{\\rm py}$-DNN performances by displaying 2D Ca, Si, H and O concentration profiles at a given time. For each experiment and chemical component, this time is selected as to be well representative of the simulated dynamics. It is observed that the original and emulated images are visually almost indistinguishable for the advection-dispersion case (Figure \\ref{fig5}). For the diffusion case, the emulators also perform quite well for Ca, H and O (Figures\\ref{fig6}a - c, g - i and j - l), while some slight to moderate discrepancies appear at the concentration front for Si (Figures \\ref{fig6}d - f). Notwithstanding, the Si concentration remains globally well predicted. Furthermore, Figures \\ref{fig7} - \\ref{fig8} present the original and emulated 2D solid amount profiles corresponding to Figures \\ref{fig5} - \\ref{fig6}. The original solid amount profiles are overall well approximated by HPx$_{\\rm py}$-kNN and HPx$_{\\rm py}$-DNN for the advection-dispersion case (Figure \\ref{fig7}), even though some mismatch appears at the border of the fully depleted zone for the H component. As of the diffusion case (Figure \\ref{fig8}), the same kind of mismatch is observed for the emulated solid amounts of H by HPx$_{\\rm py}$-kNN while the emulated profiles by HPx$_{\\rm py}$-DNN show some more discrepancies. Though we decided to present raw emulation results, we would like to stress that some if not all of the observed artifacts could likely be smoothed out by using some post-filtering such as median filtering.\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure3.png}\n\t\\caption{Time series of original (RTM, solid green lines) and HPx$_{\\rm py}$-kNN emulated (TM+kNN, dashed orange lines) concentrations (mol/kg) of Ca, H, O and Si at selected observations points for cement system 1 and advective-dispersive transport. Obs. 1 - 5 denote the selected observation points, with the following $\\left[x,y\\right]$ locations (in cm). Obs. 1:  $\\left[0.5, 2.5\\right]$, Obs. 2: $\\left[1, 2\\right]$, Obs. 3: $\\left[2, 2\\right]$, Obs. 4: $\\left[1, 1\\right]$, Obs. 5: $\\left[2, 1\\right]$. The results for the 121 $\\times$ 121 grid size are rather similar.}\n\t\\label{fig3}\n\\end{figure}\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure4.png}\n\t\\caption{Time series of original (RTM, solid green lines) and HPx$_{\\rm py}$-DNN emulated (TM+DNN, dashed orange lines) concentrations (mol/kg) of Ca, H, O and Si at selected observations points for cement system 1 and advective-dispersive transport. Obs. 1 - 5 denote the selected observation points, with the following $\\left[x,y\\right]$ locations (in cm). Obs. 1:  $\\left[0.5, 2.5\\right]$, Obs. 2: $\\left[1, 2\\right]$, Obs. 3: $\\left[2, 2\\right]$, Obs. 4: $\\left[1, 1\\right]$, Obs. 5: $\\left[2, 1\\right]$. The results for the 121 $\\times$ 121 grid size are rather similar.}\n\t\\label{fig4}\n\\end{figure}\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure5.png}\n\t\\caption{2D concentration profiles obtained for cement system 1 at the end of the 2-year simulation performed for the advection-dispersion case. RTM means the original HP$_{\\rm{4C}}$ model, TM+kNN denotes the Hydrus transport model coupled with our kNN geochemical emulator (HPx$_{\\rm py}$-kNN) and TM+DNN signifies the Hydrus transport model coupled with our DNN geochemical emulator (HPx$_{\\rm py}$-DNN). The first to fourth row present profiles for $Ca^{\\rm{conc}}$, $Si^{\\rm{conc}}$, $H^{\\rm{conc}}$, and $O^{\\rm{conc}}$, respectively. The considered grid size is 61 $\\times$ 61. The results for the 121 $\\times$ 121 grid are rather similar.}\n\t\\label{fig5}\n\\end{figure}\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure6.png}\n\t\\caption{2D concentration profiles obtained for cement system 1 at the final time step of the 1-year simulation performed for the diffusion case. RTM means the original HP$_{\\rm{4C}}$ model, TM+kNN denotes the Hydrus transport model coupled with our kNN geochemical emulator (HPx$_{\\rm py}$-kNN) and TM+DNN signifies the Hydrus transport model coupled with our DNN geochemical emulator (HPx$_{\\rm py}$-DNN). The first to fourth row present profiles for $Ca^{\\rm{conc}}$, $Si^{\\rm{conc}}$, $H^{\\rm{conc}}$, and $O^{\\rm{conc}}$, respectively. The considered grid size is 61 $\\times$ 61.}\n\t\\label{fig6}\n\\end{figure}\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure7.png}\n\t\\caption{2D solid amount profiles obtained for cement system 1 at the end of the 2-year simulation performed for the advection-dispersion case. RTM means the original HP$_{\\rm{4C}}$ model, TM+kNN denotes the Hydrus transport model coupled with our kNN geochemical emulator (HPx$_{\\rm py}$-kNN) and TM+DNN signifies the Hydrus transport model coupled with our DNN geochemical emulator (HPx$_{\\rm py}$-DNN). The first to fourth row present profiles for $Ca^{\\rm{solid}}$, $Si^{\\rm{solid}}$, $H^{\\rm{solid}}$, and $O^{\\rm{solid}}$, respectively. The considered grid size is 61 $\\times$ 61. The results for the 121 $\\times$ 121 grid are rather similar.}\n\t\\label{fig7}\n\\end{figure}\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure8.png}\n\t\\caption{2D solid amount profiles obtained for cement system 1 at the final time step of the 1-year simulation performed for the diffusion case. RTM means the original HP$_{\\rm{4C}}$ model, TM+kNN denotes the Hydrus transport model coupled with our kNN geochemical emulator (HPx$_{\\rm py}$-kNN) and TM+DNN signifies the Hydrus transport model coupled with our DNN geochemical emulator (HPx$_{\\rm py}$-DNN). The first to fourth row present profiles for $Ca^{\\rm{solid}}$, $Si^{\\rm{solid}}$, $H^{\\rm{solid}}$, and $O^{\\rm{solid}}$, respectively. The considered grid size is 61 $\\times$ 61.}\n\t\\label{fig8}\n\\end{figure}\n\nThe speedups associated with the considered problem are detailed in Table \\ref{table3}. It is noted that the GPU-based DNN emulator allows for a speedup that is close to optimal. Indeed, The DNN speedups overall represent 85 \\% to 95 \\% of the maximum possible speedups (that is, speedups that would be obtained if the geochemcical calculations would come at no cost at all). The speedups associated with single-threaded kNN remain substantial but only amount to 57 \\% - 65 \\% of the corresponding maximum speedups. As detailed in section \\ref{train_res1}, the used kNN and DNN implementations are found to be respectively 300 and 4000 faster than single-threaded PHREEQC when predicting 10,000 points all one once for this geochemical system. Based on these numbers one could have expected the achieved speedups to represent say 90 \\% (kNN) or 99 \\% (DNN) of the maximum possible ones. A large part of the gaps between achieved and maximum possible speedups is thus likely caused by the time required for communicating and exchanging data between the main C/C++ code and the Python-based emulators.\n\n\\begin{table}[h!]\n\t\\caption{Speedups offered by the KNN and DNN emulators in HPx$_{\\rm py}$ for the reactive transport simulations considered for cement system 1. The HPx$_{\\rm{4C}}$ calculations involve the parallelization of PHREEQC over our 4 CPUs. The HPx$_{\\rm{1C}}$ calculations are performed on a single CPU. The kNN predictions are performed on a single CPU using the scikit-learn implementation while the DNN predictions make use of our GPU. ML signifies the used machine learning method for emulation, TC denotes transport conditions (ADV: advection-dispersion, DIF: diffusion) and GS is the grid size. The maximum possible speedups associated with HPx$_{\\rm{4C}}$ and HPx$_{\\rm{1C}}$,  Max SP HPx$_{\\rm{4C}}$ and  Max SP HPx$_{\\rm{1C}}$, correspond to an hypothetical situation where the geochemical calculations incur zero computational cost.}\n\n\t\\begin{center}\n\t\t\\begin{tabular}{cccccccc}\n\t\t\t\\hline\n\t\t\t& & & & & & & \\\\\n\t\t\tML & TC & GS & HPx$_{\\rm{4C}}$ time (s) & SP HPx$_{\\rm{4C}}$ & Max SP HPx$_{\\rm{4C}}$ & SP HPx$_{\\rm{1C}}$ & Max SP HPx$_{\\rm{1C}}$ \\\\\n\t\t\tDNN & ADV & 61 $\\times$ 61 & 3189 & 6.8 & 7.7 & 24.5 & 28.5 \\\\\n\t\t\tkNN & ADV & 61 $\\times$ 61 & 3189 & 5.0 & 7.7 & 18.0 & 28.5 \\\\\n\t\t\tDNN & ADV & 121 $\\times$ 121 & 23,337 & 5.0 & 5.2 & 17.0 & 18.0 \\\\\n\t\t\tkNN & ADV & 121 $\\times$ 121 & 23,337 & 3.4 & 5.2 & 11.6 & 18.0 \\\\\n\t\t\tDNN & DIF & 61 $\\times$ 61 & 25,448 & 4.2 & 4.9 & 16.2 & 19.1 \\\\\n\t\t\tkNN & DIF & 61 $\\times$ 61 & 25,448 & 2.8 & 4.9 & 10.8 & 19.1 \\\\\n\t\t\t\\hline\n\t\t\\end{tabular}\n\t\\end{center}\n\n\t\\label{table3}\n\\end{table}\n\n\\FloatBarrier\n\n",
        "label": "fig7",
        "Descriptive_question1": "Which chemical component is represented in the first row of figure_7?",
        "Descriptive_question2": "What is the grid size used for the simulation profiles shown in figure_7?",
        "Reasoning_question1": "How does the accuracy of the TM+kNN model compare to the TM+DNN model for the H solid amount profiles in figure_7, particularly at the border of the fully depleted zone?",
        "Reasoning_question2": "What might be the potential reasons for any discrepancies observed between the original RTM profiles and the emulated profiles (TM+kNN and TM+DNN) for different chemical components in figure_7?",
        "Descriptive_answer1": "Ca^{solid}",
        "Descriptive_answer2": "61 \\times 61",
        "Reasoning_answer1": "In figure_7, it is observed that both TM+kNN and TM+DNN models overall approximate the original RTM solid amount profiles quite well for the various chemical components. However, for the H solid amount profile in the first to fourth rows, a mismatch appears at the border of the fully depleted zone. This mismatch is more pronounced for the TM+DNN results compared to TM+kNN, indicating that TM+kNN achieves slightly better accuracy in this region for the H solid amounts.",
        "Reasoning_answer2": "The discrepancies observed between the original RTM profiles and the emulated profiles (TM+kNN and TM+DNN) can be attributed to several factors. Firstly, the emulators approximate complex geochemical processes using machine learning models, which inherently introduce approximation errors. Specifically, the mismatch at border zones could arise from the difficulty in accurately capturing sharp gradients or depletion fronts. Secondly, data communication overhead between the main simulation code and the Python-based emulators can introduce artifacts or reduce effective accuracy. Finally, limitations in training data coverage or the interpolation/extrapolation capability of the kNN and DNN emulators, especially in regions with sparse data or highly nonlinear behavior, can cause these observed differences. Some of these artifacts might be mitigated by applying post-filtering techniques such as median filtering."
    },
    {
        "paper_id": "2107.07598.json",
        "image_id": "figure_8",
        "image_path": [
            "/home/yz979/scratch/chengye/scimolmo/test-images-tables/dataset_imagestables/2107.07598/images/Figure8.png"
        ],
        "caption": "2D solid amount profiles obtained for cement system 1 at the final time step of the 1-year simulation performed for the diffusion case. RTM means the original HP$_{\\rm{4C}}$ model, TM+kNN denotes the Hydrus transport model coupled with our kNN geochemical emulator (HPx$_{\\rm py}$-kNN) and TM+DNN signifies the Hydrus transport model coupled with our DNN geochemical emulator (HPx$_{\\rm py}$-DNN). The first to fourth row present profiles for $Ca^{\\rm{solid}}$, $Si^{\\rm{solid}}$, $H^{\\rm{solid}}$, and $O^{\\rm{solid}}$, respectively. The considered grid size is 61 $\\times$ 61.",
        "classify": "Chart",
        "section_info": "3 Results\n\\section{Results}\n\\label{results}\n\n\\subsection{Ca-Si Problem}\nFor this first cement system, the emulation problem consists of predicting at each time step of the RT simulation the (output) Ca, Si, H and O aqueous concentrations (mol/kg of water or mol/kgw) from the (input) total amounts of Ca and Si (mol). \n\n\\subsubsection{Training the emulators}\n\\label{train_res1}\nHere the kNN and DNN emulators are firstly trained using a set of 400,000 test examples for both. This training set is obtained by randomly sampling the two-dimensional input space by latin hypercube sampling (LHS) between $\\left[0,0\\right]$ and $\\rm{\\left[Ca^{tot}_{max},Si^{tot}_{max}\\right]}$, and running PHREEQC for each input sample, $\\rm{\\textbf{x}_i = \\left[Ca^{tot}_i, Si^{tot}_i\\right]}$ to get the corresponding output vectors, $\\rm{\\textbf{y}_i = \\left[Ca^{conc}_i,Si^{conc}_i,H^{conc}_i,O^{conc}_i\\right]}$. The upper bounds, $\\rm{Ca^{tot}_{max}}$ and $\\rm{Si^{tot}_{max}}$ are defined based on a cheap full RT simulation with advective-dispersive transport using a small 1D domain of 51 nodes. It is worth noting that the total amounts of $\\textbf{x}_i$, corresponding to the PHREEQC-simulated concentrations, $\\textbf{y}_i$, have to be corrected for the different amount of water between the training set and the transport simulations. Doing so, it turns out that about 20 \\% of the post-corrected $x_i$ values exceed their pre-defined upper bounds and these excessively large values need to be filtered out. Creating the 400,000 training examples thus required about 500,000 PHREEQC runs. As stated earlier, for this problem single-threaded PHREEQC performs about 670 geochemical calculations per second on our used Intel\\textsuperscript{\\textregistered} i7 CPU.\n\nWith respect to kNN, the tuning parameters are the number of neighbors, $k$, the type of distance measure, and the interpolation technique. We simply used the default settings: $k = 5$, euclidean distance and inverse-distance interpolation. Regarding training of the DNN, the 400,000 sample were split between the training set itself (90 \\% of the data) and a validation set (10 \\% of the data). The latter serves to monitor the evolution of the selected mean squared error loss function on samples that are not used for training, thereby detecting potential overfitting. If the validation loss stops decreasing before the fixed number of epochs has been completed, then  training is stopped. Importantly, the emulation is achieved in log-space for both the input, $X$ and output $Y$, domains. This because total amounts and concentrations of the involved components typically cover many orders of magnitudes (up to 10 orders or more). Using a DNN also requires some form of data normalization or standardization. Here both the $\\rm{log\\left(\\textbf{x}_i\\right)}$ and $\\rm{log\\left(\\textbf{y}_i\\right)}$ vectors are standardized around 0 with standard deviation of 1.\n\nFigure \\ref{fig1} illustrates the trained emulators' performance for geochemical predictions using an independent test set that comprises 10,000 test examples. Both kNN (Figures \\ref{fig1}a - d) and DNN (Figures \\ref{fig1}e - h) appear to be rather accurate . DNN also shows a slight degradation for the larger concentration values  (Figures \\ref{fig1}e - h). The latter is likely due to the combination of a small proportion of large concentration values in the training set with the log-transformation that implicitly pushes the DNN to try harder to fit the smaller concentrations during training. Regarding speedup and as written earlier, for this setup the single-threaded kNN method is 300 times faster than single-threaded PHREEQC for predicting the 10,000 concentration vectors all at once. The computational savings allowed by the DNN emulator when ran on our GPU is higher with a speedup as large as 4000 for predicting the same 10,000 concentration vectors all at once. \n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{0cm}\\includegraphics[width=35pc]{Figure1.png}\n\t\\caption{1-1 plots of the kNN (subfigures (a) - (d)) and DNN (subfigures (e) - (h)) emulators' performance obtained for system 1 when the kNN training base contains 400,000 samples and the DNN is trained using the same 400,000 samples. Here ``true\" means the original PHREEQC-simulated data and ``predicted\" denotes the emulated (that is, kNN-simulated and DNN-simulated) data. Hence, the $x$-axis and $y$-axis present the original and emulated 10,000 independent test data points, respectively. The RMSE and $Q_2$ coefficient denote the root-mean-square-error and coefficient of determination in testing mode, respectively, between the original and emulated 10,000 test data points.}\n\t\\label{fig1}\n\\end{figure}\n\nTo test the sensitivity of the emulators' performance to the training set size, training was also performed using reduced training sets comprising 100,000, 10,000, 1000 and 100 samples, respectively. It seen that the DNN performance achieved  when using 10,000 training samples is virtually the same as that obtained when using 400,000 training samples (Figure \\ref{fig2} and Table \\ref{table2}). It is only for training sets smaller than 1000 samples that the DNN performance starts to degrade significantly (Table \\ref{table2}). In contrast, the behavior of kNN appears to markedly decrease as the training set gets smaller (Figure \\ref{fig2} and Table \\ref{table1}).\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{0cm}\\includegraphics[width=35pc]{Figure2.png}\n\t\\caption{1-1 plots of the kNN (subfigures (a) - (d)) and DNN (subfigures (e) - (h)) emulators' performance obtained for system 1 when the kNN training base contains 10,000 samples and the DNN is trained using the same 10,000 samples. Here ``true\" means the original PHREEQC-simulated data and ``predicted\" denotes the emulated (that is, kNN-simulated and DNN-simulated) data. Hence, the $x$-axis and $y$-axis present the original and emulated 10,000 independent test data points, respectively. The RMSE and $Q_2$ coefficient denote the root-mean-square-error and coefficient of determination in testing mode, respectively, between the original and emulated 10,000 test data points.}\n\t\\label{fig2}\n\\end{figure}\n\n\\begin{table}[h!]\n\t\\caption{Performance of the DNN and kNN emulators for cement system 1 and different training set sizes. For brevity, only the results for Ca$^{conc}$ and Si$^{conc}$ are shown. The units are mol per kg of water (mol/kgw). ML refers to the type of emulator, TR signifies the size of the training set, and the RMSE and $Q_2$ coefficient denote the root-mean-square-error and coefficient of determination in testing mode, respectively, between the original and emulated 10,000 test data points.}\n\t\\begin{center}\n\t\t\\begin{tabular}{cccccccc}\n\t\t\t\\hline\n\t\t\t& & & & & \\\\\n\t\t\tML & TR & RMSE - Ca$^{conc}$ & $Q_2$ - Ca$^{conc}$ & RMSE - Si$^{conc}$ & $Q_2$ - Si$^{conc}$\\\\\n\t\t\tDNN & 4 $\\times$ 10$^{5}$ & 4.50 $\\times$ 10$^{-5}$ & 0.9999 & 1.21 $\\times$ 10$^{-6}$ & 0.9999\\\\\n\t\t\tDNN & 1 $\\times$ 10$^{5}$ & 5.06 $\\times$ 10$^{-5}$ & 0.9999 & 1.30 $\\times$ 10$^{-6}$ & 0.9999\\\\\n\t\t\tDNN & 1 $\\times$ 10$^{4}$ & 4.57 $\\times$ 10$^{-5}$ & 0.9999 & 1.48 $\\times$ 10$^{-6}$ & 0.9999\\\\\n\t\t\tDNN & 1 $\\times$ 10$^{3}$ & 8.02 $\\times$ 10$^{-5}$ & 0.9998 & 3.31 $\\times$ 10$^{-6}$ & 0.9994\\\\\n\t\t\tDNN & 1 $\\times$ 10$^{2}$ & 47.1 $\\times$ 10$^{-5}$ & 0.9935 & 9.02 $\\times$ 10$^{-6}$ & 0.9959\\\\\n\t\t\tkNN & 4 $\\times$ 10$^{5}$ & 2.52 $\\times$ 10$^{-5}$ & 1.0000 & 1.59 $\\times$ 10$^{-6}$ & 0.9999\\\\\n\t\t\tkNN & 1 $\\times$ 10$^{5}$ & 6.27 $\\times$ 10$^{-5}$ & 0.9999 & 4.20 $\\times$ 10$^{-6}$ & 0.9991\\\\\n\t\t\tkNN & 1 $\\times$ 10$^{4}$ & 14.1 $\\times$ 10$^{-5}$ & 0.9994 & 13.5 $\\times$ 10$^{-6}$ & 0.9908\\\\\n\t\t\tkNN & 1 $\\times$ 10$^{3}$ & 43.9 $\\times$ 10$^{-5}$ & 0.9944 & 74.6 $\\times$ 10$^{-6}$ & 0.7180\\\\\n\t\t\tkNN & 1 $\\times$ 10$^{2}$  & 96.4 $\\times$ 10$^{-5}$ & 0.9729 & 55.0 $\\times$ 10$^{-6}$ & 0.8468\\\\\n\t\t\\end{tabular}\n\t\\end{center}\n\t\\label{table2}\n\\end{table}\n\n\\FloatBarrier\n\n\\subsubsection{Reactive transport simulation}\n\\label{prob12Dres}\n\nThis section focuses on reactive transport simulations with HPx$_{\\rm py}$ within cement system 1, under both advective-dispersive and diffusive transport conditions. As written above, the domain sizes are both 61 $\\times$ 61 and 121 $\\times$ 121 for the advection-dispersion case and, because of computational constraints, solely 61 $\\times$ 61  for the diffusion case. In addition, the simulation time period is 2 years for the advection-dispersion case and 1 year for the diffusion case. Figures \\ref{fig3} and \\ref{fig4} present times series of original and emulated Ca, Si, H and O concentrations at 5 locations within the 2D domain for advective-dispersive transport conditions, for both our kNN-based (HPx$_{\\rm py}$-kNN) and DNN-based (HPx$_{\\rm py}$-DNN) reactive transport codes. It is seen that HPx$_{\\rm py}$-kNN and HPx$_{\\rm py}$-DNN both induce a quite good simulation accuracy. Also, the results for the diffusive transport case are of similarly good quality (not shown). Figures \\ref{fig5} - \\ref{fig6} provide more insights into the HPx$_{\\rm py}$-kNN and HPx$_{\\rm py}$-DNN performances by displaying 2D Ca, Si, H and O concentration profiles at a given time. For each experiment and chemical component, this time is selected as to be well representative of the simulated dynamics. It is observed that the original and emulated images are visually almost indistinguishable for the advection-dispersion case (Figure \\ref{fig5}). For the diffusion case, the emulators also perform quite well for Ca, H and O (Figures\\ref{fig6}a - c, g - i and j - l), while some slight to moderate discrepancies appear at the concentration front for Si (Figures \\ref{fig6}d - f). Notwithstanding, the Si concentration remains globally well predicted. Furthermore, Figures \\ref{fig7} - \\ref{fig8} present the original and emulated 2D solid amount profiles corresponding to Figures \\ref{fig5} - \\ref{fig6}. The original solid amount profiles are overall well approximated by HPx$_{\\rm py}$-kNN and HPx$_{\\rm py}$-DNN for the advection-dispersion case (Figure \\ref{fig7}), even though some mismatch appears at the border of the fully depleted zone for the H component. As of the diffusion case (Figure \\ref{fig8}), the same kind of mismatch is observed for the emulated solid amounts of H by HPx$_{\\rm py}$-kNN while the emulated profiles by HPx$_{\\rm py}$-DNN show some more discrepancies. Though we decided to present raw emulation results, we would like to stress that some if not all of the observed artifacts could likely be smoothed out by using some post-filtering such as median filtering.\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure3.png}\n\t\\caption{Time series of original (RTM, solid green lines) and HPx$_{\\rm py}$-kNN emulated (TM+kNN, dashed orange lines) concentrations (mol/kg) of Ca, H, O and Si at selected observations points for cement system 1 and advective-dispersive transport. Obs. 1 - 5 denote the selected observation points, with the following $\\left[x,y\\right]$ locations (in cm). Obs. 1:  $\\left[0.5, 2.5\\right]$, Obs. 2: $\\left[1, 2\\right]$, Obs. 3: $\\left[2, 2\\right]$, Obs. 4: $\\left[1, 1\\right]$, Obs. 5: $\\left[2, 1\\right]$. The results for the 121 $\\times$ 121 grid size are rather similar.}\n\t\\label{fig3}\n\\end{figure}\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure4.png}\n\t\\caption{Time series of original (RTM, solid green lines) and HPx$_{\\rm py}$-DNN emulated (TM+DNN, dashed orange lines) concentrations (mol/kg) of Ca, H, O and Si at selected observations points for cement system 1 and advective-dispersive transport. Obs. 1 - 5 denote the selected observation points, with the following $\\left[x,y\\right]$ locations (in cm). Obs. 1:  $\\left[0.5, 2.5\\right]$, Obs. 2: $\\left[1, 2\\right]$, Obs. 3: $\\left[2, 2\\right]$, Obs. 4: $\\left[1, 1\\right]$, Obs. 5: $\\left[2, 1\\right]$. The results for the 121 $\\times$ 121 grid size are rather similar.}\n\t\\label{fig4}\n\\end{figure}\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure5.png}\n\t\\caption{2D concentration profiles obtained for cement system 1 at the end of the 2-year simulation performed for the advection-dispersion case. RTM means the original HP$_{\\rm{4C}}$ model, TM+kNN denotes the Hydrus transport model coupled with our kNN geochemical emulator (HPx$_{\\rm py}$-kNN) and TM+DNN signifies the Hydrus transport model coupled with our DNN geochemical emulator (HPx$_{\\rm py}$-DNN). The first to fourth row present profiles for $Ca^{\\rm{conc}}$, $Si^{\\rm{conc}}$, $H^{\\rm{conc}}$, and $O^{\\rm{conc}}$, respectively. The considered grid size is 61 $\\times$ 61. The results for the 121 $\\times$ 121 grid are rather similar.}\n\t\\label{fig5}\n\\end{figure}\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure6.png}\n\t\\caption{2D concentration profiles obtained for cement system 1 at the final time step of the 1-year simulation performed for the diffusion case. RTM means the original HP$_{\\rm{4C}}$ model, TM+kNN denotes the Hydrus transport model coupled with our kNN geochemical emulator (HPx$_{\\rm py}$-kNN) and TM+DNN signifies the Hydrus transport model coupled with our DNN geochemical emulator (HPx$_{\\rm py}$-DNN). The first to fourth row present profiles for $Ca^{\\rm{conc}}$, $Si^{\\rm{conc}}$, $H^{\\rm{conc}}$, and $O^{\\rm{conc}}$, respectively. The considered grid size is 61 $\\times$ 61.}\n\t\\label{fig6}\n\\end{figure}\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure7.png}\n\t\\caption{2D solid amount profiles obtained for cement system 1 at the end of the 2-year simulation performed for the advection-dispersion case. RTM means the original HP$_{\\rm{4C}}$ model, TM+kNN denotes the Hydrus transport model coupled with our kNN geochemical emulator (HPx$_{\\rm py}$-kNN) and TM+DNN signifies the Hydrus transport model coupled with our DNN geochemical emulator (HPx$_{\\rm py}$-DNN). The first to fourth row present profiles for $Ca^{\\rm{solid}}$, $Si^{\\rm{solid}}$, $H^{\\rm{solid}}$, and $O^{\\rm{solid}}$, respectively. The considered grid size is 61 $\\times$ 61. The results for the 121 $\\times$ 121 grid are rather similar.}\n\t\\label{fig7}\n\\end{figure}\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure8.png}\n\t\\caption{2D solid amount profiles obtained for cement system 1 at the final time step of the 1-year simulation performed for the diffusion case. RTM means the original HP$_{\\rm{4C}}$ model, TM+kNN denotes the Hydrus transport model coupled with our kNN geochemical emulator (HPx$_{\\rm py}$-kNN) and TM+DNN signifies the Hydrus transport model coupled with our DNN geochemical emulator (HPx$_{\\rm py}$-DNN). The first to fourth row present profiles for $Ca^{\\rm{solid}}$, $Si^{\\rm{solid}}$, $H^{\\rm{solid}}$, and $O^{\\rm{solid}}$, respectively. The considered grid size is 61 $\\times$ 61.}\n\t\\label{fig8}\n\\end{figure}\n\nThe speedups associated with the considered problem are detailed in Table \\ref{table3}. It is noted that the GPU-based DNN emulator allows for a speedup that is close to optimal. Indeed, The DNN speedups overall represent 85 \\% to 95 \\% of the maximum possible speedups (that is, speedups that would be obtained if the geochemcical calculations would come at no cost at all). The speedups associated with single-threaded kNN remain substantial but only amount to 57 \\% - 65 \\% of the corresponding maximum speedups. As detailed in section \\ref{train_res1}, the used kNN and DNN implementations are found to be respectively 300 and 4000 faster than single-threaded PHREEQC when predicting 10,000 points all one once for this geochemical system. Based on these numbers one could have expected the achieved speedups to represent say 90 \\% (kNN) or 99 \\% (DNN) of the maximum possible ones. A large part of the gaps between achieved and maximum possible speedups is thus likely caused by the time required for communicating and exchanging data between the main C/C++ code and the Python-based emulators.\n\n\\begin{table}[h!]\n\t\\caption{Speedups offered by the KNN and DNN emulators in HPx$_{\\rm py}$ for the reactive transport simulations considered for cement system 1. The HPx$_{\\rm{4C}}$ calculations involve the parallelization of PHREEQC over our 4 CPUs. The HPx$_{\\rm{1C}}$ calculations are performed on a single CPU. The kNN predictions are performed on a single CPU using the scikit-learn implementation while the DNN predictions make use of our GPU. ML signifies the used machine learning method for emulation, TC denotes transport conditions (ADV: advection-dispersion, DIF: diffusion) and GS is the grid size. The maximum possible speedups associated with HPx$_{\\rm{4C}}$ and HPx$_{\\rm{1C}}$,  Max SP HPx$_{\\rm{4C}}$ and  Max SP HPx$_{\\rm{1C}}$, correspond to an hypothetical situation where the geochemical calculations incur zero computational cost.}\n\n\t\\begin{center}\n\t\t\\begin{tabular}{cccccccc}\n\t\t\t\\hline\n\t\t\t& & & & & & & \\\\\n\t\t\tML & TC & GS & HPx$_{\\rm{4C}}$ time (s) & SP HPx$_{\\rm{4C}}$ & Max SP HPx$_{\\rm{4C}}$ & SP HPx$_{\\rm{1C}}$ & Max SP HPx$_{\\rm{1C}}$ \\\\\n\t\t\tDNN & ADV & 61 $\\times$ 61 & 3189 & 6.8 & 7.7 & 24.5 & 28.5 \\\\\n\t\t\tkNN & ADV & 61 $\\times$ 61 & 3189 & 5.0 & 7.7 & 18.0 & 28.5 \\\\\n\t\t\tDNN & ADV & 121 $\\times$ 121 & 23,337 & 5.0 & 5.2 & 17.0 & 18.0 \\\\\n\t\t\tkNN & ADV & 121 $\\times$ 121 & 23,337 & 3.4 & 5.2 & 11.6 & 18.0 \\\\\n\t\t\tDNN & DIF & 61 $\\times$ 61 & 25,448 & 4.2 & 4.9 & 16.2 & 19.1 \\\\\n\t\t\tkNN & DIF & 61 $\\times$ 61 & 25,448 & 2.8 & 4.9 & 10.8 & 19.1 \\\\\n\t\t\t\\hline\n\t\t\\end{tabular}\n\t\\end{center}\n\n\t\\label{table3}\n\\end{table}\n\n\\FloatBarrier\n\n\\subsection{Al-C-Ca-S-Si Problem}\n\nFor this second cement system, the emulation problem consists of predicting at each time step of the RT simulation the (output) Al, C, Ca, S, Si, H and O aqueous concentrations (mol/kgw) from the (input) total amounts of Al, C, Ca, S and Si (mol). Here we focus on advective-dispersive transport only while similarly as for cement system 1, the considered domain sizes are 61 $\\times$ 61 and 121 $\\times$ 121. Moreover, the simulation time period is set to 6 years. \n\nAs mentioned earlier, for this higher-dimensional problem it is observed that the used scikit-learn kNN implementation becomes prohibitively slow compared to HPx$_{\\rm{4C}}$. We found that to get a good emulation accuracy the kNN training base needs to contain 1,000,000 samples (or more). This training base's size together with a 5-dimensional search space leads to an HPx$_{\\rm py}$-kNN reactive transport simulation time that is comparable to that of HPx$_{\\rm{4C}}$. Therefore, for this second cement system we built a custom kNN regressor around another kNN implementation contained in the FAISS package \\citep{faiss2017}. Our used FAISS variant allows for GPU computing and is much faster than scikit-learn for this cement system, but is slightly less accurate due to the use of an approximate rather than exact nearest neighbor search \\citep[see][for details]{faiss2017}.\n\n\\subsubsection{Training the DNN}\n\\label{train_res2}\n\nBuilding a good training set to perform a kNN search and learn the weights and biases of our DNN turned out to be a complicated task in this case. This because to make useful kNN predictions and/or learn an useful DNN, the training set must be sufficiently representative of the geochemical conditions encountered during the reactive transport simulation one wish to perform with HPx$_{\\rm py}$-kNN and the trained HPx$_{\\rm py}$-DNN. In contrast to cement system 1, creating the training set by sampling the $X$-space with a controlled randomness between predefined lower and upper bounds did not prove successful. We tried that strategy by drawing as much as 4,000,000 5-dimensional $\\textbf{x}$ vectors from the $X$-space using a Sobol low-discrepancy sequence \\citep[][]{Sobol1967, Joe-Kuo2003}. Such low-discrepancy sampling scheme covers the 5-dimensional hypercube more uniformly than LHS. Despite a good performance on the test set (not shown), the resulting DNN accuracy in reactive transport mode was never deemed satisfying. In other words, no satisfying ``global\" or ``universal\" DNN emulator could be devised for this cement system. This is probably caused by the fact for this problem, the input (5 total amounts) and output (7 aqueous concentrations) spaces are quite nonlinearly related and both cover 6 to 10 orders of magnitudes depending on the considered element. Therefore, we resorted to the alternative training strategy detailed below. The latter basically tries to grasp the complex correlations and higher-order dependencies that exist between the elements of $\\textbf{x}$ (total amounts, input space) for a given reactive transport simulation setup, in order to produce a training set that honors these between-input relationships.\n\n\\begin{itemize}\n\t\\item Perform a ``cheap\" full reactive transport simulation under the transport conditions and geochemistry of interest and collect the resulting $\\textbf{x}$ -$\\textbf{y}$  pairs of examples (for the considered grid nodes and time steps). Computational demand controls what domain size and simulation time period can be used for this cheap calculation. We used a modest 16 $\\times$ 16 domain and a simulation time period of 10 years. The associated HPx$_{\\rm{4C}}$ runtimes is 180 s.\n\t\n\t\\item Fit a kernel density estimator (KDE) with a Gaussian kernel to the collected $\\textbf{x}$ vectors (encapsulated in the $\\textbf{X}$ array) and generate a fixed number of new input vectors, $\\textbf{x}_{KDE}$. Then run PHREEQC for the $\\textbf{X}_{KDE}$ set to get the corresponding output set, $\\textbf{Y}_{KDE}$. Now apply the correction for porosity described in section \\ref{train_res1} to the $\\textbf{X}_{KDE}$ set and form the training set by merging the ensemble of $\\textbf{x}$-$\\textbf{y}$ pairs with that of the $\\textbf{x}_{KDE}$-$\\textbf{y}_{KDE}$ pairs. The number of produced unique examples by the considered cheap HPx$_{\\rm{4C}}$ simulations varied between 10,000 and 50,000. The KDE-based enrichment of this dataset was deemed necessary to provide more input variability, thereby avoiding overfitting of the trained DNN and improving the kNN accuracy, while still honoring the complex between-input relationships. The number of KDE-generated samples was set as to obtain a total training set size of 1,000,000 examples. A key component of the approach is the bandwith parameter of the Gaussian KDE kernel which controls how much the KDE-generated samples depart from the original ensemble. After limited trial and error, we fixed the kernel bandwith to 0.0025 for the considered case studies.\n\t\n\\end{itemize}\n\nThe scatter plots in Figure \\ref{fig9} illustrate our training set creation procedure. The orange dots depict the pairwise relationships observed between the 5 elements of $\\textbf{x}$ in the cheap simulation. The red and cyan dots in Figure \\ref{fig9} represent the KDE-generated samples with the selected bandwidth, before and after applying the correction for porosity, respectively. Training of the DNN is achieved using the ensemble of original and KDE-corrected input points. We refer to this kind of dataset as RT-based, since it is based on an albeit cheap, full RT simulation. Furthermore, we refer to the obtained DNN and kNN emulators as ``local\" emulators, since as opposed to the emulators constructed for cement system 1, the current emulators are only valid for the input conditions encapsulated in the RT-based training set.\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1.5cm}\\includegraphics[width=50pc]{Figure9.png}\n\t\\caption{Scatter plots of the complex relationships between the five considered inputs for (1) the computationally cheap RT simulation of cement system 2 (orange dots) performed with the original HPx code, (2) the corresponding sampled points by kernel density estimation (KDE) using the selected bandwith (turquoise dots) and, (3) the same sampled KDE points after porosity correction (red dots).}\n\t\\label{fig9}\n\\end{figure}\n\nTraining performance of the DNN emulator is presented in Figure \\ref{fig10}. Similarly as for cement system 1, about 90\\% of the available data was used for the actual training of the DNN while the remaining 10\\% were used a validation set to contol overfitting. Lastly, performance is evaluated for both the trained DNN and kNN emulators using an independent test set of 10,000 examples. Overall, the accuracy of our ``local \" DNN emulator for this RT-inspired dataset is rather large with $Q_2$ values always greater or equal than 0.998. Training performance of the corresponding local kNN emulator is equally good (not shown). Regarding speedup, for this problem single-threaded PHREEQC achieves about 210 geochemical calculations per second on our used Intel\\textsuperscript{\\textregistered} i7 CPU while the GPU-based DNN and kNN emulators are both about 3000 times faster when predicting the 10,000 test points at once .\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure10.png}\n\t\\caption{1-1 plots of local DNN emulation performance obtained for system 2 when the local DNN is trained using 1,000,000 samples. The $x$-axis and $y$-axis present the original and emulated 10,000 independent test data points, respectively. The RMSE and $Q_2$ coefficient denote the root-mean-square-error and coefficient of determination in testing mode, respectively, between the original and emulated 10,000 test data points.}\n\t\\label{fig10}\n\\end{figure}\n\n\\subsubsection{Reactive transport simulation}\n\\label{prob2res}\nOur ``local\" DNN performs rather well when applied to the 61 $\\times$ 61 grid size and a time period of 6 years (Figures \\ref{fig11} - \\ref{fig12}). This for all components but C, for which some localized deviations appear between original and emulated 2D concentration profiles towards the end of the simulation period (Figure \\ref{fig12}). When applied to the 121 $\\times$ 121 grid, HPx$_{\\rm py}$-DNN produces additional discrepancies for O, Si and Al towards the end of the simulation period (Figures \\ref{fig13} - \\ref{fig14}). Yet most of the observed artifacts could probably be smoothed out by using post-filtering. The associated speedups are listed in Table \\ref{table4}. These speedups are larger to those obtained for cement system 1, with values between 8 and 9 when evaluated against HPx$_{\\rm{4C}}$. These speedups represent about 85 \\% to 90 \\% of the maximum possible speedup (Table \\ref{table3}). Overall, these findings indicate that for the considered problem, our RT-based training of a local DNN only works is the training set is sufficiently representative of the particular geochemical conditions encountered in the computationally demanding simulations, which is arguably not easy to achieve. This limitation is further discussed in section \\ref{discussion}.\n\nWe note a more uniform behavior for HPx$_{\\rm py}$-kNN across grid sizes than for HPx$_{\\rm py}$-DNN. Here the results for the 121  $\\times$ 121 grid are only slightly less accurate than those associated with the 61 $ \\times$ 61 grid (see Figures \\ref{fig15} - \\ref{fig16} where for brevity we only show concentration profiles for the C Al and S components). Furthermore, whenever observed the discrepancies between original and emulated profiles are more regularly scattered than for HPx$_{\\rm py}$-DNN. Note also that herein too, post-filtering could likely smooth out a large part of these deviations. In addition, owing to the use of a GPU to achieve the kNN calculations, the speedup provided by HPx$_{\\rm py}$-kNN are as large as those provided by HPx$_{\\rm py}$-DNN (Table \\ref{table4}).\n\nWith respect to the emulated solid amounts, the HPx$_{\\rm py}$-DNN results look visually good for the the 61 $\\times$ 61 grid. This is shown in Figure \\ref{fig17} for the C, Al and S chemical components, while emulation of the H, O Ca and Si chemical components is globally of similar quality (not shown). Nevertheless, significant deviations appear towards the end of the simulation period for every chemical component (see Figure \\ref{fig18} for the C, Al and S chemical components, emulation of the H, O Ca and Si chemical components shows the same level of mismatch). The HPx$_{\\rm py}$-kNN predictions are also fairly accurate for the 61 $\\times$ 61 (see Figure \\ref{fig19} for the C, Al and S chemical components, emulation of the H, O Ca and Si chemical components exhibits a globally similar quality) while some discrepancies show up at the end of the simulation (Figure \\ref{fig20}). However, the mismatch is less pronounced than for HPx$_{\\rm py}$-DNN. Overall, HPx$_{\\rm py}$-kNN appears to be somewhat more robust than HPx$_{\\rm py}$-DNN for this cement system, while providing the same (large) speedup.\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure11.png}\n\t\\caption{2D concentration profiles obtained for cement system 2 after 2 and 6 (final time step) years. RTM means the original HPx$_{\\rm{4C}}$ model and TM+DNN denotes the Hydrus transport model coupled with our DNN geochemical emulator (HPx$_{\\rm py}$-DNN). The first to fourth row present profiles for $Ca^{\\rm{conc}}$, $Si^{\\rm{conc}}$, $H^{\\rm{conc}}$, and $O^{\\rm{conc}}$, respectively. The considered grid size is 61 $\\times$ 61.}\n\t\\label{fig11}\n\\end{figure}\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure12.png}\n\t\\caption{2D concentration profiles obtained for cement system 2 after 2 and 6 (final time step) years. RTM means the original HPx$_{\\rm{4C}}$ model and TM+DNN denotes the Hydrus transport model coupled with our DNN geochemical emulator (HPx$_{\\rm py}$-DNN). The first to third row present profiles for $C^{\\rm{conc}}$, $Al^{\\rm{conc}}$, and $S^{\\rm{conc}}$, respectively. The considered grid size is 61 $\\times$ 61.}\n\t\\label{fig12}\n\\end{figure}\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure13.png}\n\t\\caption{2D concentration profiles obtained for cement system 2 after 2 and 6 (final time step) years. RTM means the original HPx$_{\\rm{4C}}$ model and TM+DNN denotes the Hydrus transport model coupled with our DNN geochemical emulator (HPx$_{\\rm py}$-DNN). The first to fourth row present profiles for $Ca^{\\rm{conc}}$, $Si^{\\rm{conc}}$, $H^{\\rm{conc}}$, and $O^{\\rm{conc}}$, respectively. The considered grid size is 121 $\\times$ 121.}\n\t\\label{fig13}\n\\end{figure}\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure14.png}\n\t\\caption{2D concentration profiles obtained for cement system 2 after 2 and 6 (final time step) years. RTM means the original HPx$_{\\rm{4C}}$ model and TM+DNN denotes the Hydrus transport model coupled with our DNN geochemical emulator (HPx$_{\\rm py}$-DNN). The first to third row present profiles for $C^{\\rm{conc}}$, $Al^{\\rm{conc}}$, and $S^{\\rm{conc}}$, respectively. The considered grid size is 121 $\\times$ 121.}\n\t\\label{fig14}\n\\end{figure}\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure15.png}\n\t\\caption{2D concentration profiles obtained for cement system 2 after 2 and 6 (final time step) years. RTM means the original HPx$_{\\rm{4C}}$ model and TM+kNN denotes the Hydrus transport model coupled with our kNN geochemical emulator (HPx$_{\\rm py}$-kNN).  The first to third row present profiles for $C^{\\rm{conc}}$, $Al^{\\rm{conc}}$, and $S^{\\rm{conc}}$, respectively. The considered grid size is 61 $\\times$ 61.}\n\t\\label{fig15}\n\\end{figure}\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure16.png}\n\t\\caption{2D concentration profiles obtained for cement system 2 after 2 and 6 (final time step) years. RTM means the original HPx$_{\\rm{4C}}$ model and TM+kNN denotes the Hydrus transport model coupled with our kNN geochemical emulator (HPx$_{\\rm py}$-kNN). The first to third row present profiles for $C^{\\rm{conc}}$, $Al^{\\rm{conc}}$, and $S^{\\rm{conc}}$, respectively. The considered grid size is 121 $\\times$ 121.}\n\t\\label{fig16}\n\\end{figure}\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure17.png}\n\t\\caption{2D solid amount profiles of the C, Al and S chemical components of cement system 2 after 2 and 6 (final time step) years. RTM means the original HPx$_{\\rm{4C}}$ model and TM+DNN denotes the Hydrus transport model coupled with our DNN geochemical emulator (HPx$_{\\rm py}$-DNN). The first to third row present profiles for $C^{\\rm{solid}}$, $Al^{\\rm{solid}}$, and $S^{\\rm{solid}}$, respectively The considered grid size is 61 $\\times$ 61.}\n\t\\label{fig17}\n\\end{figure}\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure18.png}\n\t\\caption{2D solid amount profiles of the C, Al and S chemical components of cement system 2 after 2 and 6 (final time step) years. RTM means the original HPx$_{\\rm{4C}}$ model and TM+DNN denotes the Hydrus transport model coupled with our DNN geochemical emulator (HPx$_{\\rm py}$-DNN). The first to third row present profiles for $C^{\\rm{solid}}$, $Al^{\\rm{solid}}$, and $S^{\\rm{solid}}$, respectively The considered grid size is 121 $\\times$ 121.}\n\t\\label{fig18}\n\\end{figure}\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure19.png}\n\t\\caption{2D solid amount profiles of the C, Al and S chemical components of cement system 2 after 2 and 6 (final time step) years. RTM means the original HPx$_{\\rm{4C}}$ model and TM+kNN denotes the Hydrus transport model coupled with our kNN geochemical emulator (HPx$_{\\rm py}$-kNN). The first to third row present profiles for $C^{\\rm{solid}}$, $Al^{\\rm{solid}}$, and $S^{\\rm{solid}}$, respectively The considered grid size is 61 $\\times$ 61.}\n\t\\label{fig19}\n\\end{figure}\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure20.png}\n\t\\caption{2D solid amount profiles of the C, Al and S chemical components of cement system 2 after 2 and 6 (final time step) years. RTM means the original HPx$_{\\rm{4C}}$ model and TM+kNN denotes the Hydrus transport model coupled with our kNN geochemical emulator (HPx$_{\\rm py}$-kNN). The first to third row present profiles for $C^{\\rm{solid}}$, $Al^{\\rm{solid}}$, and $S^{\\rm{solid}}$, respectively The considered grid size is 121 $\\times$ 121.}\n\t\\label{fig20}\n\\end{figure}\n\n\\begin{table}[h!]\n\t\\caption{Speedups offered by the KNN and DNN emulators in HPx$_{\\rm py}$ for the reactive transport simulations considered for cement system 2. The HPx$_{\\rm{4C}}$ calculations involve the parallelization of PHREEQC over our 4 CPUs. The HPx$_{\\rm{1C}}$ calculations are performed on a single CPU. Both the kNN and DNN predictions make use of a GPU. ML signifies the used machine learning method for emulation, BC denotes the type of flow boundary conditions and GS is the grid size. The maximum possible speedups associated with HPx$_{\\rm{4C}}$ and HP$_{\\rm{1C}}$,  Max SP HPx$_{\\rm{4C}}$ and  Max SP HPx$_{\\rm{1C}}$, correspond to an hypothetical situation where the geochemical calculations incur zero computational cost.}\n\n\t\\begin{center}\n\t\t\\begin{tabular}{cccccccc}\n\t\t\t\\hline\n\t\t\t& & & & & & & \\\\\n\t\t\tML & BC & GS & HPx$_{\\rm{4C}}$ time (s) & SP HPx$_{\\rm{4C}}$ & Max SP HPx$_{\\rm{4C}}$ & SP HPx$_{\\rm{1C}}$ & Max SP HPx$_{\\rm{1C}}$ \\\\\n\t\t\tDNN & ADV & 61 $\\times$ 61 & 21,415 & 8.2 & 9.0 & 30.3 & 33.1\\\\\n\t\t\tkNN & ADV & 61 $\\times$ 61 & 21,415 & 7.9 & 9.0 & 28.9 & 33.1\\\\\n\t\t\tDNN & ADV & 121 $\\times$ 121 & 199,841 & 8.2 & 9.5 & 29.9 & 32.8\\\\\n\t\t\tkNN & ADV & 121 $\\times$ 121 & 199,841 & 8.5 & 9.5 & 31.3 & 32.8\\\\\n\t\t\t\\hline\n\t\t\\end{tabular}\n\t\\end{center}\n\n\t\\label{table4}\n\\end{table}\n\n\\FloatBarrier\n\n3.1 Ca-Si Problem\n\\subsection{Ca-Si Problem}\nFor this first cement system, the emulation problem consists of predicting at each time step of the RT simulation the (output) Ca, Si, H and O aqueous concentrations (mol/kg of water or mol/kgw) from the (input) total amounts of Ca and Si (mol). \n\n\\subsubsection{Training the emulators}\n\\label{train_res1}\nHere the kNN and DNN emulators are firstly trained using a set of 400,000 test examples for both. This training set is obtained by randomly sampling the two-dimensional input space by latin hypercube sampling (LHS) between $\\left[0,0\\right]$ and $\\rm{\\left[Ca^{tot}_{max},Si^{tot}_{max}\\right]}$, and running PHREEQC for each input sample, $\\rm{\\textbf{x}_i = \\left[Ca^{tot}_i, Si^{tot}_i\\right]}$ to get the corresponding output vectors, $\\rm{\\textbf{y}_i = \\left[Ca^{conc}_i,Si^{conc}_i,H^{conc}_i,O^{conc}_i\\right]}$. The upper bounds, $\\rm{Ca^{tot}_{max}}$ and $\\rm{Si^{tot}_{max}}$ are defined based on a cheap full RT simulation with advective-dispersive transport using a small 1D domain of 51 nodes. It is worth noting that the total amounts of $\\textbf{x}_i$, corresponding to the PHREEQC-simulated concentrations, $\\textbf{y}_i$, have to be corrected for the different amount of water between the training set and the transport simulations. Doing so, it turns out that about 20 \\% of the post-corrected $x_i$ values exceed their pre-defined upper bounds and these excessively large values need to be filtered out. Creating the 400,000 training examples thus required about 500,000 PHREEQC runs. As stated earlier, for this problem single-threaded PHREEQC performs about 670 geochemical calculations per second on our used Intel\\textsuperscript{\\textregistered} i7 CPU.\n\nWith respect to kNN, the tuning parameters are the number of neighbors, $k$, the type of distance measure, and the interpolation technique. We simply used the default settings: $k = 5$, euclidean distance and inverse-distance interpolation. Regarding training of the DNN, the 400,000 sample were split between the training set itself (90 \\% of the data) and a validation set (10 \\% of the data). The latter serves to monitor the evolution of the selected mean squared error loss function on samples that are not used for training, thereby detecting potential overfitting. If the validation loss stops decreasing before the fixed number of epochs has been completed, then  training is stopped. Importantly, the emulation is achieved in log-space for both the input, $X$ and output $Y$, domains. This because total amounts and concentrations of the involved components typically cover many orders of magnitudes (up to 10 orders or more). Using a DNN also requires some form of data normalization or standardization. Here both the $\\rm{log\\left(\\textbf{x}_i\\right)}$ and $\\rm{log\\left(\\textbf{y}_i\\right)}$ vectors are standardized around 0 with standard deviation of 1.\n\nFigure \\ref{fig1} illustrates the trained emulators' performance for geochemical predictions using an independent test set that comprises 10,000 test examples. Both kNN (Figures \\ref{fig1}a - d) and DNN (Figures \\ref{fig1}e - h) appear to be rather accurate . DNN also shows a slight degradation for the larger concentration values  (Figures \\ref{fig1}e - h). The latter is likely due to the combination of a small proportion of large concentration values in the training set with the log-transformation that implicitly pushes the DNN to try harder to fit the smaller concentrations during training. Regarding speedup and as written earlier, for this setup the single-threaded kNN method is 300 times faster than single-threaded PHREEQC for predicting the 10,000 concentration vectors all at once. The computational savings allowed by the DNN emulator when ran on our GPU is higher with a speedup as large as 4000 for predicting the same 10,000 concentration vectors all at once. \n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{0cm}\\includegraphics[width=35pc]{Figure1.png}\n\t\\caption{1-1 plots of the kNN (subfigures (a) - (d)) and DNN (subfigures (e) - (h)) emulators' performance obtained for system 1 when the kNN training base contains 400,000 samples and the DNN is trained using the same 400,000 samples. Here ``true\" means the original PHREEQC-simulated data and ``predicted\" denotes the emulated (that is, kNN-simulated and DNN-simulated) data. Hence, the $x$-axis and $y$-axis present the original and emulated 10,000 independent test data points, respectively. The RMSE and $Q_2$ coefficient denote the root-mean-square-error and coefficient of determination in testing mode, respectively, between the original and emulated 10,000 test data points.}\n\t\\label{fig1}\n\\end{figure}\n\nTo test the sensitivity of the emulators' performance to the training set size, training was also performed using reduced training sets comprising 100,000, 10,000, 1000 and 100 samples, respectively. It seen that the DNN performance achieved  when using 10,000 training samples is virtually the same as that obtained when using 400,000 training samples (Figure \\ref{fig2} and Table \\ref{table2}). It is only for training sets smaller than 1000 samples that the DNN performance starts to degrade significantly (Table \\ref{table2}). In contrast, the behavior of kNN appears to markedly decrease as the training set gets smaller (Figure \\ref{fig2} and Table \\ref{table1}).\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{0cm}\\includegraphics[width=35pc]{Figure2.png}\n\t\\caption{1-1 plots of the kNN (subfigures (a) - (d)) and DNN (subfigures (e) - (h)) emulators' performance obtained for system 1 when the kNN training base contains 10,000 samples and the DNN is trained using the same 10,000 samples. Here ``true\" means the original PHREEQC-simulated data and ``predicted\" denotes the emulated (that is, kNN-simulated and DNN-simulated) data. Hence, the $x$-axis and $y$-axis present the original and emulated 10,000 independent test data points, respectively. The RMSE and $Q_2$ coefficient denote the root-mean-square-error and coefficient of determination in testing mode, respectively, between the original and emulated 10,000 test data points.}\n\t\\label{fig2}\n\\end{figure}\n\n\\begin{table}[h!]\n\t\\caption{Performance of the DNN and kNN emulators for cement system 1 and different training set sizes. For brevity, only the results for Ca$^{conc}$ and Si$^{conc}$ are shown. The units are mol per kg of water (mol/kgw). ML refers to the type of emulator, TR signifies the size of the training set, and the RMSE and $Q_2$ coefficient denote the root-mean-square-error and coefficient of determination in testing mode, respectively, between the original and emulated 10,000 test data points.}\n\t\\begin{center}\n\t\t\\begin{tabular}{cccccccc}\n\t\t\t\\hline\n\t\t\t& & & & & \\\\\n\t\t\tML & TR & RMSE - Ca$^{conc}$ & $Q_2$ - Ca$^{conc}$ & RMSE - Si$^{conc}$ & $Q_2$ - Si$^{conc}$\\\\\n\t\t\tDNN & 4 $\\times$ 10$^{5}$ & 4.50 $\\times$ 10$^{-5}$ & 0.9999 & 1.21 $\\times$ 10$^{-6}$ & 0.9999\\\\\n\t\t\tDNN & 1 $\\times$ 10$^{5}$ & 5.06 $\\times$ 10$^{-5}$ & 0.9999 & 1.30 $\\times$ 10$^{-6}$ & 0.9999\\\\\n\t\t\tDNN & 1 $\\times$ 10$^{4}$ & 4.57 $\\times$ 10$^{-5}$ & 0.9999 & 1.48 $\\times$ 10$^{-6}$ & 0.9999\\\\\n\t\t\tDNN & 1 $\\times$ 10$^{3}$ & 8.02 $\\times$ 10$^{-5}$ & 0.9998 & 3.31 $\\times$ 10$^{-6}$ & 0.9994\\\\\n\t\t\tDNN & 1 $\\times$ 10$^{2}$ & 47.1 $\\times$ 10$^{-5}$ & 0.9935 & 9.02 $\\times$ 10$^{-6}$ & 0.9959\\\\\n\t\t\tkNN & 4 $\\times$ 10$^{5}$ & 2.52 $\\times$ 10$^{-5}$ & 1.0000 & 1.59 $\\times$ 10$^{-6}$ & 0.9999\\\\\n\t\t\tkNN & 1 $\\times$ 10$^{5}$ & 6.27 $\\times$ 10$^{-5}$ & 0.9999 & 4.20 $\\times$ 10$^{-6}$ & 0.9991\\\\\n\t\t\tkNN & 1 $\\times$ 10$^{4}$ & 14.1 $\\times$ 10$^{-5}$ & 0.9994 & 13.5 $\\times$ 10$^{-6}$ & 0.9908\\\\\n\t\t\tkNN & 1 $\\times$ 10$^{3}$ & 43.9 $\\times$ 10$^{-5}$ & 0.9944 & 74.6 $\\times$ 10$^{-6}$ & 0.7180\\\\\n\t\t\tkNN & 1 $\\times$ 10$^{2}$  & 96.4 $\\times$ 10$^{-5}$ & 0.9729 & 55.0 $\\times$ 10$^{-6}$ & 0.8468\\\\\n\t\t\\end{tabular}\n\t\\end{center}\n\t\\label{table2}\n\\end{table}\n\n\\FloatBarrier\n\n\\subsubsection{Reactive transport simulation}\n\\label{prob12Dres}\n\nThis section focuses on reactive transport simulations with HPx$_{\\rm py}$ within cement system 1, under both advective-dispersive and diffusive transport conditions. As written above, the domain sizes are both 61 $\\times$ 61 and 121 $\\times$ 121 for the advection-dispersion case and, because of computational constraints, solely 61 $\\times$ 61  for the diffusion case. In addition, the simulation time period is 2 years for the advection-dispersion case and 1 year for the diffusion case. Figures \\ref{fig3} and \\ref{fig4} present times series of original and emulated Ca, Si, H and O concentrations at 5 locations within the 2D domain for advective-dispersive transport conditions, for both our kNN-based (HPx$_{\\rm py}$-kNN) and DNN-based (HPx$_{\\rm py}$-DNN) reactive transport codes. It is seen that HPx$_{\\rm py}$-kNN and HPx$_{\\rm py}$-DNN both induce a quite good simulation accuracy. Also, the results for the diffusive transport case are of similarly good quality (not shown). Figures \\ref{fig5} - \\ref{fig6} provide more insights into the HPx$_{\\rm py}$-kNN and HPx$_{\\rm py}$-DNN performances by displaying 2D Ca, Si, H and O concentration profiles at a given time. For each experiment and chemical component, this time is selected as to be well representative of the simulated dynamics. It is observed that the original and emulated images are visually almost indistinguishable for the advection-dispersion case (Figure \\ref{fig5}). For the diffusion case, the emulators also perform quite well for Ca, H and O (Figures\\ref{fig6}a - c, g - i and j - l), while some slight to moderate discrepancies appear at the concentration front for Si (Figures \\ref{fig6}d - f). Notwithstanding, the Si concentration remains globally well predicted. Furthermore, Figures \\ref{fig7} - \\ref{fig8} present the original and emulated 2D solid amount profiles corresponding to Figures \\ref{fig5} - \\ref{fig6}. The original solid amount profiles are overall well approximated by HPx$_{\\rm py}$-kNN and HPx$_{\\rm py}$-DNN for the advection-dispersion case (Figure \\ref{fig7}), even though some mismatch appears at the border of the fully depleted zone for the H component. As of the diffusion case (Figure \\ref{fig8}), the same kind of mismatch is observed for the emulated solid amounts of H by HPx$_{\\rm py}$-kNN while the emulated profiles by HPx$_{\\rm py}$-DNN show some more discrepancies. Though we decided to present raw emulation results, we would like to stress that some if not all of the observed artifacts could likely be smoothed out by using some post-filtering such as median filtering.\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure3.png}\n\t\\caption{Time series of original (RTM, solid green lines) and HPx$_{\\rm py}$-kNN emulated (TM+kNN, dashed orange lines) concentrations (mol/kg) of Ca, H, O and Si at selected observations points for cement system 1 and advective-dispersive transport. Obs. 1 - 5 denote the selected observation points, with the following $\\left[x,y\\right]$ locations (in cm). Obs. 1:  $\\left[0.5, 2.5\\right]$, Obs. 2: $\\left[1, 2\\right]$, Obs. 3: $\\left[2, 2\\right]$, Obs. 4: $\\left[1, 1\\right]$, Obs. 5: $\\left[2, 1\\right]$. The results for the 121 $\\times$ 121 grid size are rather similar.}\n\t\\label{fig3}\n\\end{figure}\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure4.png}\n\t\\caption{Time series of original (RTM, solid green lines) and HPx$_{\\rm py}$-DNN emulated (TM+DNN, dashed orange lines) concentrations (mol/kg) of Ca, H, O and Si at selected observations points for cement system 1 and advective-dispersive transport. Obs. 1 - 5 denote the selected observation points, with the following $\\left[x,y\\right]$ locations (in cm). Obs. 1:  $\\left[0.5, 2.5\\right]$, Obs. 2: $\\left[1, 2\\right]$, Obs. 3: $\\left[2, 2\\right]$, Obs. 4: $\\left[1, 1\\right]$, Obs. 5: $\\left[2, 1\\right]$. The results for the 121 $\\times$ 121 grid size are rather similar.}\n\t\\label{fig4}\n\\end{figure}\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure5.png}\n\t\\caption{2D concentration profiles obtained for cement system 1 at the end of the 2-year simulation performed for the advection-dispersion case. RTM means the original HP$_{\\rm{4C}}$ model, TM+kNN denotes the Hydrus transport model coupled with our kNN geochemical emulator (HPx$_{\\rm py}$-kNN) and TM+DNN signifies the Hydrus transport model coupled with our DNN geochemical emulator (HPx$_{\\rm py}$-DNN). The first to fourth row present profiles for $Ca^{\\rm{conc}}$, $Si^{\\rm{conc}}$, $H^{\\rm{conc}}$, and $O^{\\rm{conc}}$, respectively. The considered grid size is 61 $\\times$ 61. The results for the 121 $\\times$ 121 grid are rather similar.}\n\t\\label{fig5}\n\\end{figure}\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure6.png}\n\t\\caption{2D concentration profiles obtained for cement system 1 at the final time step of the 1-year simulation performed for the diffusion case. RTM means the original HP$_{\\rm{4C}}$ model, TM+kNN denotes the Hydrus transport model coupled with our kNN geochemical emulator (HPx$_{\\rm py}$-kNN) and TM+DNN signifies the Hydrus transport model coupled with our DNN geochemical emulator (HPx$_{\\rm py}$-DNN). The first to fourth row present profiles for $Ca^{\\rm{conc}}$, $Si^{\\rm{conc}}$, $H^{\\rm{conc}}$, and $O^{\\rm{conc}}$, respectively. The considered grid size is 61 $\\times$ 61.}\n\t\\label{fig6}\n\\end{figure}\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure7.png}\n\t\\caption{2D solid amount profiles obtained for cement system 1 at the end of the 2-year simulation performed for the advection-dispersion case. RTM means the original HP$_{\\rm{4C}}$ model, TM+kNN denotes the Hydrus transport model coupled with our kNN geochemical emulator (HPx$_{\\rm py}$-kNN) and TM+DNN signifies the Hydrus transport model coupled with our DNN geochemical emulator (HPx$_{\\rm py}$-DNN). The first to fourth row present profiles for $Ca^{\\rm{solid}}$, $Si^{\\rm{solid}}$, $H^{\\rm{solid}}$, and $O^{\\rm{solid}}$, respectively. The considered grid size is 61 $\\times$ 61. The results for the 121 $\\times$ 121 grid are rather similar.}\n\t\\label{fig7}\n\\end{figure}\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure8.png}\n\t\\caption{2D solid amount profiles obtained for cement system 1 at the final time step of the 1-year simulation performed for the diffusion case. RTM means the original HP$_{\\rm{4C}}$ model, TM+kNN denotes the Hydrus transport model coupled with our kNN geochemical emulator (HPx$_{\\rm py}$-kNN) and TM+DNN signifies the Hydrus transport model coupled with our DNN geochemical emulator (HPx$_{\\rm py}$-DNN). The first to fourth row present profiles for $Ca^{\\rm{solid}}$, $Si^{\\rm{solid}}$, $H^{\\rm{solid}}$, and $O^{\\rm{solid}}$, respectively. The considered grid size is 61 $\\times$ 61.}\n\t\\label{fig8}\n\\end{figure}\n\nThe speedups associated with the considered problem are detailed in Table \\ref{table3}. It is noted that the GPU-based DNN emulator allows for a speedup that is close to optimal. Indeed, The DNN speedups overall represent 85 \\% to 95 \\% of the maximum possible speedups (that is, speedups that would be obtained if the geochemcical calculations would come at no cost at all). The speedups associated with single-threaded kNN remain substantial but only amount to 57 \\% - 65 \\% of the corresponding maximum speedups. As detailed in section \\ref{train_res1}, the used kNN and DNN implementations are found to be respectively 300 and 4000 faster than single-threaded PHREEQC when predicting 10,000 points all one once for this geochemical system. Based on these numbers one could have expected the achieved speedups to represent say 90 \\% (kNN) or 99 \\% (DNN) of the maximum possible ones. A large part of the gaps between achieved and maximum possible speedups is thus likely caused by the time required for communicating and exchanging data between the main C/C++ code and the Python-based emulators.\n\n\\begin{table}[h!]\n\t\\caption{Speedups offered by the KNN and DNN emulators in HPx$_{\\rm py}$ for the reactive transport simulations considered for cement system 1. The HPx$_{\\rm{4C}}$ calculations involve the parallelization of PHREEQC over our 4 CPUs. The HPx$_{\\rm{1C}}$ calculations are performed on a single CPU. The kNN predictions are performed on a single CPU using the scikit-learn implementation while the DNN predictions make use of our GPU. ML signifies the used machine learning method for emulation, TC denotes transport conditions (ADV: advection-dispersion, DIF: diffusion) and GS is the grid size. The maximum possible speedups associated with HPx$_{\\rm{4C}}$ and HPx$_{\\rm{1C}}$,  Max SP HPx$_{\\rm{4C}}$ and  Max SP HPx$_{\\rm{1C}}$, correspond to an hypothetical situation where the geochemical calculations incur zero computational cost.}\n\n\t\\begin{center}\n\t\t\\begin{tabular}{cccccccc}\n\t\t\t\\hline\n\t\t\t& & & & & & & \\\\\n\t\t\tML & TC & GS & HPx$_{\\rm{4C}}$ time (s) & SP HPx$_{\\rm{4C}}$ & Max SP HPx$_{\\rm{4C}}$ & SP HPx$_{\\rm{1C}}$ & Max SP HPx$_{\\rm{1C}}$ \\\\\n\t\t\tDNN & ADV & 61 $\\times$ 61 & 3189 & 6.8 & 7.7 & 24.5 & 28.5 \\\\\n\t\t\tkNN & ADV & 61 $\\times$ 61 & 3189 & 5.0 & 7.7 & 18.0 & 28.5 \\\\\n\t\t\tDNN & ADV & 121 $\\times$ 121 & 23,337 & 5.0 & 5.2 & 17.0 & 18.0 \\\\\n\t\t\tkNN & ADV & 121 $\\times$ 121 & 23,337 & 3.4 & 5.2 & 11.6 & 18.0 \\\\\n\t\t\tDNN & DIF & 61 $\\times$ 61 & 25,448 & 4.2 & 4.9 & 16.2 & 19.1 \\\\\n\t\t\tkNN & DIF & 61 $\\times$ 61 & 25,448 & 2.8 & 4.9 & 10.8 & 19.1 \\\\\n\t\t\t\\hline\n\t\t\\end{tabular}\n\t\\end{center}\n\n\t\\label{table3}\n\\end{table}\n\n\\FloatBarrier\n\n3.1.2 Reactive transport simulation\n\\subsubsection{Reactive transport simulation}\n\\label{prob12Dres}\n\nThis section focuses on reactive transport simulations with HPx$_{\\rm py}$ within cement system 1, under both advective-dispersive and diffusive transport conditions. As written above, the domain sizes are both 61 $\\times$ 61 and 121 $\\times$ 121 for the advection-dispersion case and, because of computational constraints, solely 61 $\\times$ 61  for the diffusion case. In addition, the simulation time period is 2 years for the advection-dispersion case and 1 year for the diffusion case. Figures \\ref{fig3} and \\ref{fig4} present times series of original and emulated Ca, Si, H and O concentrations at 5 locations within the 2D domain for advective-dispersive transport conditions, for both our kNN-based (HPx$_{\\rm py}$-kNN) and DNN-based (HPx$_{\\rm py}$-DNN) reactive transport codes. It is seen that HPx$_{\\rm py}$-kNN and HPx$_{\\rm py}$-DNN both induce a quite good simulation accuracy. Also, the results for the diffusive transport case are of similarly good quality (not shown). Figures \\ref{fig5} - \\ref{fig6} provide more insights into the HPx$_{\\rm py}$-kNN and HPx$_{\\rm py}$-DNN performances by displaying 2D Ca, Si, H and O concentration profiles at a given time. For each experiment and chemical component, this time is selected as to be well representative of the simulated dynamics. It is observed that the original and emulated images are visually almost indistinguishable for the advection-dispersion case (Figure \\ref{fig5}). For the diffusion case, the emulators also perform quite well for Ca, H and O (Figures\\ref{fig6}a - c, g - i and j - l), while some slight to moderate discrepancies appear at the concentration front for Si (Figures \\ref{fig6}d - f). Notwithstanding, the Si concentration remains globally well predicted. Furthermore, Figures \\ref{fig7} - \\ref{fig8} present the original and emulated 2D solid amount profiles corresponding to Figures \\ref{fig5} - \\ref{fig6}. The original solid amount profiles are overall well approximated by HPx$_{\\rm py}$-kNN and HPx$_{\\rm py}$-DNN for the advection-dispersion case (Figure \\ref{fig7}), even though some mismatch appears at the border of the fully depleted zone for the H component. As of the diffusion case (Figure \\ref{fig8}), the same kind of mismatch is observed for the emulated solid amounts of H by HPx$_{\\rm py}$-kNN while the emulated profiles by HPx$_{\\rm py}$-DNN show some more discrepancies. Though we decided to present raw emulation results, we would like to stress that some if not all of the observed artifacts could likely be smoothed out by using some post-filtering such as median filtering.\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure3.png}\n\t\\caption{Time series of original (RTM, solid green lines) and HPx$_{\\rm py}$-kNN emulated (TM+kNN, dashed orange lines) concentrations (mol/kg) of Ca, H, O and Si at selected observations points for cement system 1 and advective-dispersive transport. Obs. 1 - 5 denote the selected observation points, with the following $\\left[x,y\\right]$ locations (in cm). Obs. 1:  $\\left[0.5, 2.5\\right]$, Obs. 2: $\\left[1, 2\\right]$, Obs. 3: $\\left[2, 2\\right]$, Obs. 4: $\\left[1, 1\\right]$, Obs. 5: $\\left[2, 1\\right]$. The results for the 121 $\\times$ 121 grid size are rather similar.}\n\t\\label{fig3}\n\\end{figure}\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure4.png}\n\t\\caption{Time series of original (RTM, solid green lines) and HPx$_{\\rm py}$-DNN emulated (TM+DNN, dashed orange lines) concentrations (mol/kg) of Ca, H, O and Si at selected observations points for cement system 1 and advective-dispersive transport. Obs. 1 - 5 denote the selected observation points, with the following $\\left[x,y\\right]$ locations (in cm). Obs. 1:  $\\left[0.5, 2.5\\right]$, Obs. 2: $\\left[1, 2\\right]$, Obs. 3: $\\left[2, 2\\right]$, Obs. 4: $\\left[1, 1\\right]$, Obs. 5: $\\left[2, 1\\right]$. The results for the 121 $\\times$ 121 grid size are rather similar.}\n\t\\label{fig4}\n\\end{figure}\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure5.png}\n\t\\caption{2D concentration profiles obtained for cement system 1 at the end of the 2-year simulation performed for the advection-dispersion case. RTM means the original HP$_{\\rm{4C}}$ model, TM+kNN denotes the Hydrus transport model coupled with our kNN geochemical emulator (HPx$_{\\rm py}$-kNN) and TM+DNN signifies the Hydrus transport model coupled with our DNN geochemical emulator (HPx$_{\\rm py}$-DNN). The first to fourth row present profiles for $Ca^{\\rm{conc}}$, $Si^{\\rm{conc}}$, $H^{\\rm{conc}}$, and $O^{\\rm{conc}}$, respectively. The considered grid size is 61 $\\times$ 61. The results for the 121 $\\times$ 121 grid are rather similar.}\n\t\\label{fig5}\n\\end{figure}\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure6.png}\n\t\\caption{2D concentration profiles obtained for cement system 1 at the final time step of the 1-year simulation performed for the diffusion case. RTM means the original HP$_{\\rm{4C}}$ model, TM+kNN denotes the Hydrus transport model coupled with our kNN geochemical emulator (HPx$_{\\rm py}$-kNN) and TM+DNN signifies the Hydrus transport model coupled with our DNN geochemical emulator (HPx$_{\\rm py}$-DNN). The first to fourth row present profiles for $Ca^{\\rm{conc}}$, $Si^{\\rm{conc}}$, $H^{\\rm{conc}}$, and $O^{\\rm{conc}}$, respectively. The considered grid size is 61 $\\times$ 61.}\n\t\\label{fig6}\n\\end{figure}\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure7.png}\n\t\\caption{2D solid amount profiles obtained for cement system 1 at the end of the 2-year simulation performed for the advection-dispersion case. RTM means the original HP$_{\\rm{4C}}$ model, TM+kNN denotes the Hydrus transport model coupled with our kNN geochemical emulator (HPx$_{\\rm py}$-kNN) and TM+DNN signifies the Hydrus transport model coupled with our DNN geochemical emulator (HPx$_{\\rm py}$-DNN). The first to fourth row present profiles for $Ca^{\\rm{solid}}$, $Si^{\\rm{solid}}$, $H^{\\rm{solid}}$, and $O^{\\rm{solid}}$, respectively. The considered grid size is 61 $\\times$ 61. The results for the 121 $\\times$ 121 grid are rather similar.}\n\t\\label{fig7}\n\\end{figure}\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure8.png}\n\t\\caption{2D solid amount profiles obtained for cement system 1 at the final time step of the 1-year simulation performed for the diffusion case. RTM means the original HP$_{\\rm{4C}}$ model, TM+kNN denotes the Hydrus transport model coupled with our kNN geochemical emulator (HPx$_{\\rm py}$-kNN) and TM+DNN signifies the Hydrus transport model coupled with our DNN geochemical emulator (HPx$_{\\rm py}$-DNN). The first to fourth row present profiles for $Ca^{\\rm{solid}}$, $Si^{\\rm{solid}}$, $H^{\\rm{solid}}$, and $O^{\\rm{solid}}$, respectively. The considered grid size is 61 $\\times$ 61.}\n\t\\label{fig8}\n\\end{figure}\n\nThe speedups associated with the considered problem are detailed in Table \\ref{table3}. It is noted that the GPU-based DNN emulator allows for a speedup that is close to optimal. Indeed, The DNN speedups overall represent 85 \\% to 95 \\% of the maximum possible speedups (that is, speedups that would be obtained if the geochemcical calculations would come at no cost at all). The speedups associated with single-threaded kNN remain substantial but only amount to 57 \\% - 65 \\% of the corresponding maximum speedups. As detailed in section \\ref{train_res1}, the used kNN and DNN implementations are found to be respectively 300 and 4000 faster than single-threaded PHREEQC when predicting 10,000 points all one once for this geochemical system. Based on these numbers one could have expected the achieved speedups to represent say 90 \\% (kNN) or 99 \\% (DNN) of the maximum possible ones. A large part of the gaps between achieved and maximum possible speedups is thus likely caused by the time required for communicating and exchanging data between the main C/C++ code and the Python-based emulators.\n\n\\begin{table}[h!]\n\t\\caption{Speedups offered by the KNN and DNN emulators in HPx$_{\\rm py}$ for the reactive transport simulations considered for cement system 1. The HPx$_{\\rm{4C}}$ calculations involve the parallelization of PHREEQC over our 4 CPUs. The HPx$_{\\rm{1C}}$ calculations are performed on a single CPU. The kNN predictions are performed on a single CPU using the scikit-learn implementation while the DNN predictions make use of our GPU. ML signifies the used machine learning method for emulation, TC denotes transport conditions (ADV: advection-dispersion, DIF: diffusion) and GS is the grid size. The maximum possible speedups associated with HPx$_{\\rm{4C}}$ and HPx$_{\\rm{1C}}$,  Max SP HPx$_{\\rm{4C}}$ and  Max SP HPx$_{\\rm{1C}}$, correspond to an hypothetical situation where the geochemical calculations incur zero computational cost.}\n\n\t\\begin{center}\n\t\t\\begin{tabular}{cccccccc}\n\t\t\t\\hline\n\t\t\t& & & & & & & \\\\\n\t\t\tML & TC & GS & HPx$_{\\rm{4C}}$ time (s) & SP HPx$_{\\rm{4C}}$ & Max SP HPx$_{\\rm{4C}}$ & SP HPx$_{\\rm{1C}}$ & Max SP HPx$_{\\rm{1C}}$ \\\\\n\t\t\tDNN & ADV & 61 $\\times$ 61 & 3189 & 6.8 & 7.7 & 24.5 & 28.5 \\\\\n\t\t\tkNN & ADV & 61 $\\times$ 61 & 3189 & 5.0 & 7.7 & 18.0 & 28.5 \\\\\n\t\t\tDNN & ADV & 121 $\\times$ 121 & 23,337 & 5.0 & 5.2 & 17.0 & 18.0 \\\\\n\t\t\tkNN & ADV & 121 $\\times$ 121 & 23,337 & 3.4 & 5.2 & 11.6 & 18.0 \\\\\n\t\t\tDNN & DIF & 61 $\\times$ 61 & 25,448 & 4.2 & 4.9 & 16.2 & 19.1 \\\\\n\t\t\tkNN & DIF & 61 $\\times$ 61 & 25,448 & 2.8 & 4.9 & 10.8 & 19.1 \\\\\n\t\t\t\\hline\n\t\t\\end{tabular}\n\t\\end{center}\n\n\t\\label{table3}\n\\end{table}\n\n\\FloatBarrier\n\n",
        "label": "fig8",
        "Descriptive_question1": "Which chemical component is represented in the first row of figure_8?",
        "Descriptive_question2": "What is the grid size used in the simulation shown in figure_8?",
        "Reasoning_question1": "How does the performance of the DNN emulator compare to the kNN emulator in predicting solid amount profiles for H in figure_8, and what might explain any observed discrepancies?",
        "Reasoning_question2": "What trends can be inferred about the spatial distribution of Ca solid amounts across the different models (RTM, TM+kNN, TM+DNN) in figure_8, and how might these reflect the underlying transport conditions?",
        "Descriptive_answer1": "Ca^{solid}",
        "Descriptive_answer2": "61 \\times 61",
        "Reasoning_answer1": "In figure_8, which shows 2D solid amount profiles for the diffusion case, the emulated solid amounts of H by the HPx_py-kNN model exhibit some mismatch, while the HPx_py-DNN emulated profiles show even more discrepancies. This suggests that the DNN emulator performs slightly worse than the kNN emulator for the H solid amounts in this scenario. The observed discrepancies might be explained by the challenges in accurately capturing sharp concentration fronts or boundary effects within the diffusion transport case. Additionally, the raw emulation results have not been post-processed with smoothing filters, which could otherwise reduce artifacts arising from model approximations or numerical noise.",
        "Reasoning_answer2": "Examining the spatial distribution of Ca solid amounts across RTM (original HP_4C model), TM+kNN, and TM+DNN in figure_8, the profiles are overall similar, indicating that both emulator models approximate the original model well for Ca solid amounts under diffusion transport. There are no major visual discrepancies, suggesting that the diffusion conditions produce smooth concentration gradients well captured by all models. The consistent depletion and accumulation zones likely reflect steady diffusion-driven transport processes without advection, which tend to produce smoother spatial profiles and are easier for emulators to replicate accurately compared to advective cases."
    },
    {
        "paper_id": "2107.07598.json",
        "image_id": "figure_9",
        "image_path": [
            "/home/yz979/scratch/chengye/scimolmo/test-images-tables/dataset_imagestables/2107.07598/images/Figure9.png"
        ],
        "caption": "Scatter plots of the complex relationships between the five considered inputs for (1) the computationally cheap RT simulation of cement system 2 (orange dots) performed with the original HPx code, (2) the corresponding sampled points by kernel density estimation (KDE) using the selected bandwith (turquoise dots) and, (3) the same sampled KDE points after porosity correction (red dots).",
        "classify": "Chart",
        "section_info": "3 Results\n\\section{Results}\n\\label{results}\n\n\\subsection{Ca-Si Problem}\nFor this first cement system, the emulation problem consists of predicting at each time step of the RT simulation the (output) Ca, Si, H and O aqueous concentrations (mol/kg of water or mol/kgw) from the (input) total amounts of Ca and Si (mol). \n\n\\subsubsection{Training the emulators}\n\\label{train_res1}\nHere the kNN and DNN emulators are firstly trained using a set of 400,000 test examples for both. This training set is obtained by randomly sampling the two-dimensional input space by latin hypercube sampling (LHS) between $\\left[0,0\\right]$ and $\\rm{\\left[Ca^{tot}_{max},Si^{tot}_{max}\\right]}$, and running PHREEQC for each input sample, $\\rm{\\textbf{x}_i = \\left[Ca^{tot}_i, Si^{tot}_i\\right]}$ to get the corresponding output vectors, $\\rm{\\textbf{y}_i = \\left[Ca^{conc}_i,Si^{conc}_i,H^{conc}_i,O^{conc}_i\\right]}$. The upper bounds, $\\rm{Ca^{tot}_{max}}$ and $\\rm{Si^{tot}_{max}}$ are defined based on a cheap full RT simulation with advective-dispersive transport using a small 1D domain of 51 nodes. It is worth noting that the total amounts of $\\textbf{x}_i$, corresponding to the PHREEQC-simulated concentrations, $\\textbf{y}_i$, have to be corrected for the different amount of water between the training set and the transport simulations. Doing so, it turns out that about 20 \\% of the post-corrected $x_i$ values exceed their pre-defined upper bounds and these excessively large values need to be filtered out. Creating the 400,000 training examples thus required about 500,000 PHREEQC runs. As stated earlier, for this problem single-threaded PHREEQC performs about 670 geochemical calculations per second on our used Intel\\textsuperscript{\\textregistered} i7 CPU.\n\nWith respect to kNN, the tuning parameters are the number of neighbors, $k$, the type of distance measure, and the interpolation technique. We simply used the default settings: $k = 5$, euclidean distance and inverse-distance interpolation. Regarding training of the DNN, the 400,000 sample were split between the training set itself (90 \\% of the data) and a validation set (10 \\% of the data). The latter serves to monitor the evolution of the selected mean squared error loss function on samples that are not used for training, thereby detecting potential overfitting. If the validation loss stops decreasing before the fixed number of epochs has been completed, then  training is stopped. Importantly, the emulation is achieved in log-space for both the input, $X$ and output $Y$, domains. This because total amounts and concentrations of the involved components typically cover many orders of magnitudes (up to 10 orders or more). Using a DNN also requires some form of data normalization or standardization. Here both the $\\rm{log\\left(\\textbf{x}_i\\right)}$ and $\\rm{log\\left(\\textbf{y}_i\\right)}$ vectors are standardized around 0 with standard deviation of 1.\n\nFigure \\ref{fig1} illustrates the trained emulators' performance for geochemical predictions using an independent test set that comprises 10,000 test examples. Both kNN (Figures \\ref{fig1}a - d) and DNN (Figures \\ref{fig1}e - h) appear to be rather accurate . DNN also shows a slight degradation for the larger concentration values  (Figures \\ref{fig1}e - h). The latter is likely due to the combination of a small proportion of large concentration values in the training set with the log-transformation that implicitly pushes the DNN to try harder to fit the smaller concentrations during training. Regarding speedup and as written earlier, for this setup the single-threaded kNN method is 300 times faster than single-threaded PHREEQC for predicting the 10,000 concentration vectors all at once. The computational savings allowed by the DNN emulator when ran on our GPU is higher with a speedup as large as 4000 for predicting the same 10,000 concentration vectors all at once. \n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{0cm}\\includegraphics[width=35pc]{Figure1.png}\n\t\\caption{1-1 plots of the kNN (subfigures (a) - (d)) and DNN (subfigures (e) - (h)) emulators' performance obtained for system 1 when the kNN training base contains 400,000 samples and the DNN is trained using the same 400,000 samples. Here ``true\" means the original PHREEQC-simulated data and ``predicted\" denotes the emulated (that is, kNN-simulated and DNN-simulated) data. Hence, the $x$-axis and $y$-axis present the original and emulated 10,000 independent test data points, respectively. The RMSE and $Q_2$ coefficient denote the root-mean-square-error and coefficient of determination in testing mode, respectively, between the original and emulated 10,000 test data points.}\n\t\\label{fig1}\n\\end{figure}\n\nTo test the sensitivity of the emulators' performance to the training set size, training was also performed using reduced training sets comprising 100,000, 10,000, 1000 and 100 samples, respectively. It seen that the DNN performance achieved  when using 10,000 training samples is virtually the same as that obtained when using 400,000 training samples (Figure \\ref{fig2} and Table \\ref{table2}). It is only for training sets smaller than 1000 samples that the DNN performance starts to degrade significantly (Table \\ref{table2}). In contrast, the behavior of kNN appears to markedly decrease as the training set gets smaller (Figure \\ref{fig2} and Table \\ref{table1}).\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{0cm}\\includegraphics[width=35pc]{Figure2.png}\n\t\\caption{1-1 plots of the kNN (subfigures (a) - (d)) and DNN (subfigures (e) - (h)) emulators' performance obtained for system 1 when the kNN training base contains 10,000 samples and the DNN is trained using the same 10,000 samples. Here ``true\" means the original PHREEQC-simulated data and ``predicted\" denotes the emulated (that is, kNN-simulated and DNN-simulated) data. Hence, the $x$-axis and $y$-axis present the original and emulated 10,000 independent test data points, respectively. The RMSE and $Q_2$ coefficient denote the root-mean-square-error and coefficient of determination in testing mode, respectively, between the original and emulated 10,000 test data points.}\n\t\\label{fig2}\n\\end{figure}\n\n\\begin{table}[h!]\n\t\\caption{Performance of the DNN and kNN emulators for cement system 1 and different training set sizes. For brevity, only the results for Ca$^{conc}$ and Si$^{conc}$ are shown. The units are mol per kg of water (mol/kgw). ML refers to the type of emulator, TR signifies the size of the training set, and the RMSE and $Q_2$ coefficient denote the root-mean-square-error and coefficient of determination in testing mode, respectively, between the original and emulated 10,000 test data points.}\n\t\\begin{center}\n\t\t\\begin{tabular}{cccccccc}\n\t\t\t\\hline\n\t\t\t& & & & & \\\\\n\t\t\tML & TR & RMSE - Ca$^{conc}$ & $Q_2$ - Ca$^{conc}$ & RMSE - Si$^{conc}$ & $Q_2$ - Si$^{conc}$\\\\\n\t\t\tDNN & 4 $\\times$ 10$^{5}$ & 4.50 $\\times$ 10$^{-5}$ & 0.9999 & 1.21 $\\times$ 10$^{-6}$ & 0.9999\\\\\n\t\t\tDNN & 1 $\\times$ 10$^{5}$ & 5.06 $\\times$ 10$^{-5}$ & 0.9999 & 1.30 $\\times$ 10$^{-6}$ & 0.9999\\\\\n\t\t\tDNN & 1 $\\times$ 10$^{4}$ & 4.57 $\\times$ 10$^{-5}$ & 0.9999 & 1.48 $\\times$ 10$^{-6}$ & 0.9999\\\\\n\t\t\tDNN & 1 $\\times$ 10$^{3}$ & 8.02 $\\times$ 10$^{-5}$ & 0.9998 & 3.31 $\\times$ 10$^{-6}$ & 0.9994\\\\\n\t\t\tDNN & 1 $\\times$ 10$^{2}$ & 47.1 $\\times$ 10$^{-5}$ & 0.9935 & 9.02 $\\times$ 10$^{-6}$ & 0.9959\\\\\n\t\t\tkNN & 4 $\\times$ 10$^{5}$ & 2.52 $\\times$ 10$^{-5}$ & 1.0000 & 1.59 $\\times$ 10$^{-6}$ & 0.9999\\\\\n\t\t\tkNN & 1 $\\times$ 10$^{5}$ & 6.27 $\\times$ 10$^{-5}$ & 0.9999 & 4.20 $\\times$ 10$^{-6}$ & 0.9991\\\\\n\t\t\tkNN & 1 $\\times$ 10$^{4}$ & 14.1 $\\times$ 10$^{-5}$ & 0.9994 & 13.5 $\\times$ 10$^{-6}$ & 0.9908\\\\\n\t\t\tkNN & 1 $\\times$ 10$^{3}$ & 43.9 $\\times$ 10$^{-5}$ & 0.9944 & 74.6 $\\times$ 10$^{-6}$ & 0.7180\\\\\n\t\t\tkNN & 1 $\\times$ 10$^{2}$  & 96.4 $\\times$ 10$^{-5}$ & 0.9729 & 55.0 $\\times$ 10$^{-6}$ & 0.8468\\\\\n\t\t\\end{tabular}\n\t\\end{center}\n\t\\label{table2}\n\\end{table}\n\n\\FloatBarrier\n\n\\subsubsection{Reactive transport simulation}\n\\label{prob12Dres}\n\nThis section focuses on reactive transport simulations with HPx$_{\\rm py}$ within cement system 1, under both advective-dispersive and diffusive transport conditions. As written above, the domain sizes are both 61 $\\times$ 61 and 121 $\\times$ 121 for the advection-dispersion case and, because of computational constraints, solely 61 $\\times$ 61  for the diffusion case. In addition, the simulation time period is 2 years for the advection-dispersion case and 1 year for the diffusion case. Figures \\ref{fig3} and \\ref{fig4} present times series of original and emulated Ca, Si, H and O concentrations at 5 locations within the 2D domain for advective-dispersive transport conditions, for both our kNN-based (HPx$_{\\rm py}$-kNN) and DNN-based (HPx$_{\\rm py}$-DNN) reactive transport codes. It is seen that HPx$_{\\rm py}$-kNN and HPx$_{\\rm py}$-DNN both induce a quite good simulation accuracy. Also, the results for the diffusive transport case are of similarly good quality (not shown). Figures \\ref{fig5} - \\ref{fig6} provide more insights into the HPx$_{\\rm py}$-kNN and HPx$_{\\rm py}$-DNN performances by displaying 2D Ca, Si, H and O concentration profiles at a given time. For each experiment and chemical component, this time is selected as to be well representative of the simulated dynamics. It is observed that the original and emulated images are visually almost indistinguishable for the advection-dispersion case (Figure \\ref{fig5}). For the diffusion case, the emulators also perform quite well for Ca, H and O (Figures\\ref{fig6}a - c, g - i and j - l), while some slight to moderate discrepancies appear at the concentration front for Si (Figures \\ref{fig6}d - f). Notwithstanding, the Si concentration remains globally well predicted. Furthermore, Figures \\ref{fig7} - \\ref{fig8} present the original and emulated 2D solid amount profiles corresponding to Figures \\ref{fig5} - \\ref{fig6}. The original solid amount profiles are overall well approximated by HPx$_{\\rm py}$-kNN and HPx$_{\\rm py}$-DNN for the advection-dispersion case (Figure \\ref{fig7}), even though some mismatch appears at the border of the fully depleted zone for the H component. As of the diffusion case (Figure \\ref{fig8}), the same kind of mismatch is observed for the emulated solid amounts of H by HPx$_{\\rm py}$-kNN while the emulated profiles by HPx$_{\\rm py}$-DNN show some more discrepancies. Though we decided to present raw emulation results, we would like to stress that some if not all of the observed artifacts could likely be smoothed out by using some post-filtering such as median filtering.\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure3.png}\n\t\\caption{Time series of original (RTM, solid green lines) and HPx$_{\\rm py}$-kNN emulated (TM+kNN, dashed orange lines) concentrations (mol/kg) of Ca, H, O and Si at selected observations points for cement system 1 and advective-dispersive transport. Obs. 1 - 5 denote the selected observation points, with the following $\\left[x,y\\right]$ locations (in cm). Obs. 1:  $\\left[0.5, 2.5\\right]$, Obs. 2: $\\left[1, 2\\right]$, Obs. 3: $\\left[2, 2\\right]$, Obs. 4: $\\left[1, 1\\right]$, Obs. 5: $\\left[2, 1\\right]$. The results for the 121 $\\times$ 121 grid size are rather similar.}\n\t\\label{fig3}\n\\end{figure}\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure4.png}\n\t\\caption{Time series of original (RTM, solid green lines) and HPx$_{\\rm py}$-DNN emulated (TM+DNN, dashed orange lines) concentrations (mol/kg) of Ca, H, O and Si at selected observations points for cement system 1 and advective-dispersive transport. Obs. 1 - 5 denote the selected observation points, with the following $\\left[x,y\\right]$ locations (in cm). Obs. 1:  $\\left[0.5, 2.5\\right]$, Obs. 2: $\\left[1, 2\\right]$, Obs. 3: $\\left[2, 2\\right]$, Obs. 4: $\\left[1, 1\\right]$, Obs. 5: $\\left[2, 1\\right]$. The results for the 121 $\\times$ 121 grid size are rather similar.}\n\t\\label{fig4}\n\\end{figure}\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure5.png}\n\t\\caption{2D concentration profiles obtained for cement system 1 at the end of the 2-year simulation performed for the advection-dispersion case. RTM means the original HP$_{\\rm{4C}}$ model, TM+kNN denotes the Hydrus transport model coupled with our kNN geochemical emulator (HPx$_{\\rm py}$-kNN) and TM+DNN signifies the Hydrus transport model coupled with our DNN geochemical emulator (HPx$_{\\rm py}$-DNN). The first to fourth row present profiles for $Ca^{\\rm{conc}}$, $Si^{\\rm{conc}}$, $H^{\\rm{conc}}$, and $O^{\\rm{conc}}$, respectively. The considered grid size is 61 $\\times$ 61. The results for the 121 $\\times$ 121 grid are rather similar.}\n\t\\label{fig5}\n\\end{figure}\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure6.png}\n\t\\caption{2D concentration profiles obtained for cement system 1 at the final time step of the 1-year simulation performed for the diffusion case. RTM means the original HP$_{\\rm{4C}}$ model, TM+kNN denotes the Hydrus transport model coupled with our kNN geochemical emulator (HPx$_{\\rm py}$-kNN) and TM+DNN signifies the Hydrus transport model coupled with our DNN geochemical emulator (HPx$_{\\rm py}$-DNN). The first to fourth row present profiles for $Ca^{\\rm{conc}}$, $Si^{\\rm{conc}}$, $H^{\\rm{conc}}$, and $O^{\\rm{conc}}$, respectively. The considered grid size is 61 $\\times$ 61.}\n\t\\label{fig6}\n\\end{figure}\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure7.png}\n\t\\caption{2D solid amount profiles obtained for cement system 1 at the end of the 2-year simulation performed for the advection-dispersion case. RTM means the original HP$_{\\rm{4C}}$ model, TM+kNN denotes the Hydrus transport model coupled with our kNN geochemical emulator (HPx$_{\\rm py}$-kNN) and TM+DNN signifies the Hydrus transport model coupled with our DNN geochemical emulator (HPx$_{\\rm py}$-DNN). The first to fourth row present profiles for $Ca^{\\rm{solid}}$, $Si^{\\rm{solid}}$, $H^{\\rm{solid}}$, and $O^{\\rm{solid}}$, respectively. The considered grid size is 61 $\\times$ 61. The results for the 121 $\\times$ 121 grid are rather similar.}\n\t\\label{fig7}\n\\end{figure}\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure8.png}\n\t\\caption{2D solid amount profiles obtained for cement system 1 at the final time step of the 1-year simulation performed for the diffusion case. RTM means the original HP$_{\\rm{4C}}$ model, TM+kNN denotes the Hydrus transport model coupled with our kNN geochemical emulator (HPx$_{\\rm py}$-kNN) and TM+DNN signifies the Hydrus transport model coupled with our DNN geochemical emulator (HPx$_{\\rm py}$-DNN). The first to fourth row present profiles for $Ca^{\\rm{solid}}$, $Si^{\\rm{solid}}$, $H^{\\rm{solid}}$, and $O^{\\rm{solid}}$, respectively. The considered grid size is 61 $\\times$ 61.}\n\t\\label{fig8}\n\\end{figure}\n\nThe speedups associated with the considered problem are detailed in Table \\ref{table3}. It is noted that the GPU-based DNN emulator allows for a speedup that is close to optimal. Indeed, The DNN speedups overall represent 85 \\% to 95 \\% of the maximum possible speedups (that is, speedups that would be obtained if the geochemcical calculations would come at no cost at all). The speedups associated with single-threaded kNN remain substantial but only amount to 57 \\% - 65 \\% of the corresponding maximum speedups. As detailed in section \\ref{train_res1}, the used kNN and DNN implementations are found to be respectively 300 and 4000 faster than single-threaded PHREEQC when predicting 10,000 points all one once for this geochemical system. Based on these numbers one could have expected the achieved speedups to represent say 90 \\% (kNN) or 99 \\% (DNN) of the maximum possible ones. A large part of the gaps between achieved and maximum possible speedups is thus likely caused by the time required for communicating and exchanging data between the main C/C++ code and the Python-based emulators.\n\n\\begin{table}[h!]\n\t\\caption{Speedups offered by the KNN and DNN emulators in HPx$_{\\rm py}$ for the reactive transport simulations considered for cement system 1. The HPx$_{\\rm{4C}}$ calculations involve the parallelization of PHREEQC over our 4 CPUs. The HPx$_{\\rm{1C}}$ calculations are performed on a single CPU. The kNN predictions are performed on a single CPU using the scikit-learn implementation while the DNN predictions make use of our GPU. ML signifies the used machine learning method for emulation, TC denotes transport conditions (ADV: advection-dispersion, DIF: diffusion) and GS is the grid size. The maximum possible speedups associated with HPx$_{\\rm{4C}}$ and HPx$_{\\rm{1C}}$,  Max SP HPx$_{\\rm{4C}}$ and  Max SP HPx$_{\\rm{1C}}$, correspond to an hypothetical situation where the geochemical calculations incur zero computational cost.}\n\n\t\\begin{center}\n\t\t\\begin{tabular}{cccccccc}\n\t\t\t\\hline\n\t\t\t& & & & & & & \\\\\n\t\t\tML & TC & GS & HPx$_{\\rm{4C}}$ time (s) & SP HPx$_{\\rm{4C}}$ & Max SP HPx$_{\\rm{4C}}$ & SP HPx$_{\\rm{1C}}$ & Max SP HPx$_{\\rm{1C}}$ \\\\\n\t\t\tDNN & ADV & 61 $\\times$ 61 & 3189 & 6.8 & 7.7 & 24.5 & 28.5 \\\\\n\t\t\tkNN & ADV & 61 $\\times$ 61 & 3189 & 5.0 & 7.7 & 18.0 & 28.5 \\\\\n\t\t\tDNN & ADV & 121 $\\times$ 121 & 23,337 & 5.0 & 5.2 & 17.0 & 18.0 \\\\\n\t\t\tkNN & ADV & 121 $\\times$ 121 & 23,337 & 3.4 & 5.2 & 11.6 & 18.0 \\\\\n\t\t\tDNN & DIF & 61 $\\times$ 61 & 25,448 & 4.2 & 4.9 & 16.2 & 19.1 \\\\\n\t\t\tkNN & DIF & 61 $\\times$ 61 & 25,448 & 2.8 & 4.9 & 10.8 & 19.1 \\\\\n\t\t\t\\hline\n\t\t\\end{tabular}\n\t\\end{center}\n\n\t\\label{table3}\n\\end{table}\n\n\\FloatBarrier\n\n\\subsection{Al-C-Ca-S-Si Problem}\n\nFor this second cement system, the emulation problem consists of predicting at each time step of the RT simulation the (output) Al, C, Ca, S, Si, H and O aqueous concentrations (mol/kgw) from the (input) total amounts of Al, C, Ca, S and Si (mol). Here we focus on advective-dispersive transport only while similarly as for cement system 1, the considered domain sizes are 61 $\\times$ 61 and 121 $\\times$ 121. Moreover, the simulation time period is set to 6 years. \n\nAs mentioned earlier, for this higher-dimensional problem it is observed that the used scikit-learn kNN implementation becomes prohibitively slow compared to HPx$_{\\rm{4C}}$. We found that to get a good emulation accuracy the kNN training base needs to contain 1,000,000 samples (or more). This training base's size together with a 5-dimensional search space leads to an HPx$_{\\rm py}$-kNN reactive transport simulation time that is comparable to that of HPx$_{\\rm{4C}}$. Therefore, for this second cement system we built a custom kNN regressor around another kNN implementation contained in the FAISS package \\citep{faiss2017}. Our used FAISS variant allows for GPU computing and is much faster than scikit-learn for this cement system, but is slightly less accurate due to the use of an approximate rather than exact nearest neighbor search \\citep[see][for details]{faiss2017}.\n\n\\subsubsection{Training the DNN}\n\\label{train_res2}\n\nBuilding a good training set to perform a kNN search and learn the weights and biases of our DNN turned out to be a complicated task in this case. This because to make useful kNN predictions and/or learn an useful DNN, the training set must be sufficiently representative of the geochemical conditions encountered during the reactive transport simulation one wish to perform with HPx$_{\\rm py}$-kNN and the trained HPx$_{\\rm py}$-DNN. In contrast to cement system 1, creating the training set by sampling the $X$-space with a controlled randomness between predefined lower and upper bounds did not prove successful. We tried that strategy by drawing as much as 4,000,000 5-dimensional $\\textbf{x}$ vectors from the $X$-space using a Sobol low-discrepancy sequence \\citep[][]{Sobol1967, Joe-Kuo2003}. Such low-discrepancy sampling scheme covers the 5-dimensional hypercube more uniformly than LHS. Despite a good performance on the test set (not shown), the resulting DNN accuracy in reactive transport mode was never deemed satisfying. In other words, no satisfying ``global\" or ``universal\" DNN emulator could be devised for this cement system. This is probably caused by the fact for this problem, the input (5 total amounts) and output (7 aqueous concentrations) spaces are quite nonlinearly related and both cover 6 to 10 orders of magnitudes depending on the considered element. Therefore, we resorted to the alternative training strategy detailed below. The latter basically tries to grasp the complex correlations and higher-order dependencies that exist between the elements of $\\textbf{x}$ (total amounts, input space) for a given reactive transport simulation setup, in order to produce a training set that honors these between-input relationships.\n\n\\begin{itemize}\n\t\\item Perform a ``cheap\" full reactive transport simulation under the transport conditions and geochemistry of interest and collect the resulting $\\textbf{x}$ -$\\textbf{y}$  pairs of examples (for the considered grid nodes and time steps). Computational demand controls what domain size and simulation time period can be used for this cheap calculation. We used a modest 16 $\\times$ 16 domain and a simulation time period of 10 years. The associated HPx$_{\\rm{4C}}$ runtimes is 180 s.\n\t\n\t\\item Fit a kernel density estimator (KDE) with a Gaussian kernel to the collected $\\textbf{x}$ vectors (encapsulated in the $\\textbf{X}$ array) and generate a fixed number of new input vectors, $\\textbf{x}_{KDE}$. Then run PHREEQC for the $\\textbf{X}_{KDE}$ set to get the corresponding output set, $\\textbf{Y}_{KDE}$. Now apply the correction for porosity described in section \\ref{train_res1} to the $\\textbf{X}_{KDE}$ set and form the training set by merging the ensemble of $\\textbf{x}$-$\\textbf{y}$ pairs with that of the $\\textbf{x}_{KDE}$-$\\textbf{y}_{KDE}$ pairs. The number of produced unique examples by the considered cheap HPx$_{\\rm{4C}}$ simulations varied between 10,000 and 50,000. The KDE-based enrichment of this dataset was deemed necessary to provide more input variability, thereby avoiding overfitting of the trained DNN and improving the kNN accuracy, while still honoring the complex between-input relationships. The number of KDE-generated samples was set as to obtain a total training set size of 1,000,000 examples. A key component of the approach is the bandwith parameter of the Gaussian KDE kernel which controls how much the KDE-generated samples depart from the original ensemble. After limited trial and error, we fixed the kernel bandwith to 0.0025 for the considered case studies.\n\t\n\\end{itemize}\n\nThe scatter plots in Figure \\ref{fig9} illustrate our training set creation procedure. The orange dots depict the pairwise relationships observed between the 5 elements of $\\textbf{x}$ in the cheap simulation. The red and cyan dots in Figure \\ref{fig9} represent the KDE-generated samples with the selected bandwidth, before and after applying the correction for porosity, respectively. Training of the DNN is achieved using the ensemble of original and KDE-corrected input points. We refer to this kind of dataset as RT-based, since it is based on an albeit cheap, full RT simulation. Furthermore, we refer to the obtained DNN and kNN emulators as ``local\" emulators, since as opposed to the emulators constructed for cement system 1, the current emulators are only valid for the input conditions encapsulated in the RT-based training set.\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1.5cm}\\includegraphics[width=50pc]{Figure9.png}\n\t\\caption{Scatter plots of the complex relationships between the five considered inputs for (1) the computationally cheap RT simulation of cement system 2 (orange dots) performed with the original HPx code, (2) the corresponding sampled points by kernel density estimation (KDE) using the selected bandwith (turquoise dots) and, (3) the same sampled KDE points after porosity correction (red dots).}\n\t\\label{fig9}\n\\end{figure}\n\nTraining performance of the DNN emulator is presented in Figure \\ref{fig10}. Similarly as for cement system 1, about 90\\% of the available data was used for the actual training of the DNN while the remaining 10\\% were used a validation set to contol overfitting. Lastly, performance is evaluated for both the trained DNN and kNN emulators using an independent test set of 10,000 examples. Overall, the accuracy of our ``local \" DNN emulator for this RT-inspired dataset is rather large with $Q_2$ values always greater or equal than 0.998. Training performance of the corresponding local kNN emulator is equally good (not shown). Regarding speedup, for this problem single-threaded PHREEQC achieves about 210 geochemical calculations per second on our used Intel\\textsuperscript{\\textregistered} i7 CPU while the GPU-based DNN and kNN emulators are both about 3000 times faster when predicting the 10,000 test points at once .\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure10.png}\n\t\\caption{1-1 plots of local DNN emulation performance obtained for system 2 when the local DNN is trained using 1,000,000 samples. The $x$-axis and $y$-axis present the original and emulated 10,000 independent test data points, respectively. The RMSE and $Q_2$ coefficient denote the root-mean-square-error and coefficient of determination in testing mode, respectively, between the original and emulated 10,000 test data points.}\n\t\\label{fig10}\n\\end{figure}\n\n\\subsubsection{Reactive transport simulation}\n\\label{prob2res}\nOur ``local\" DNN performs rather well when applied to the 61 $\\times$ 61 grid size and a time period of 6 years (Figures \\ref{fig11} - \\ref{fig12}). This for all components but C, for which some localized deviations appear between original and emulated 2D concentration profiles towards the end of the simulation period (Figure \\ref{fig12}). When applied to the 121 $\\times$ 121 grid, HPx$_{\\rm py}$-DNN produces additional discrepancies for O, Si and Al towards the end of the simulation period (Figures \\ref{fig13} - \\ref{fig14}). Yet most of the observed artifacts could probably be smoothed out by using post-filtering. The associated speedups are listed in Table \\ref{table4}. These speedups are larger to those obtained for cement system 1, with values between 8 and 9 when evaluated against HPx$_{\\rm{4C}}$. These speedups represent about 85 \\% to 90 \\% of the maximum possible speedup (Table \\ref{table3}). Overall, these findings indicate that for the considered problem, our RT-based training of a local DNN only works is the training set is sufficiently representative of the particular geochemical conditions encountered in the computationally demanding simulations, which is arguably not easy to achieve. This limitation is further discussed in section \\ref{discussion}.\n\nWe note a more uniform behavior for HPx$_{\\rm py}$-kNN across grid sizes than for HPx$_{\\rm py}$-DNN. Here the results for the 121  $\\times$ 121 grid are only slightly less accurate than those associated with the 61 $ \\times$ 61 grid (see Figures \\ref{fig15} - \\ref{fig16} where for brevity we only show concentration profiles for the C Al and S components). Furthermore, whenever observed the discrepancies between original and emulated profiles are more regularly scattered than for HPx$_{\\rm py}$-DNN. Note also that herein too, post-filtering could likely smooth out a large part of these deviations. In addition, owing to the use of a GPU to achieve the kNN calculations, the speedup provided by HPx$_{\\rm py}$-kNN are as large as those provided by HPx$_{\\rm py}$-DNN (Table \\ref{table4}).\n\nWith respect to the emulated solid amounts, the HPx$_{\\rm py}$-DNN results look visually good for the the 61 $\\times$ 61 grid. This is shown in Figure \\ref{fig17} for the C, Al and S chemical components, while emulation of the H, O Ca and Si chemical components is globally of similar quality (not shown). Nevertheless, significant deviations appear towards the end of the simulation period for every chemical component (see Figure \\ref{fig18} for the C, Al and S chemical components, emulation of the H, O Ca and Si chemical components shows the same level of mismatch). The HPx$_{\\rm py}$-kNN predictions are also fairly accurate for the 61 $\\times$ 61 (see Figure \\ref{fig19} for the C, Al and S chemical components, emulation of the H, O Ca and Si chemical components exhibits a globally similar quality) while some discrepancies show up at the end of the simulation (Figure \\ref{fig20}). However, the mismatch is less pronounced than for HPx$_{\\rm py}$-DNN. Overall, HPx$_{\\rm py}$-kNN appears to be somewhat more robust than HPx$_{\\rm py}$-DNN for this cement system, while providing the same (large) speedup.\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure11.png}\n\t\\caption{2D concentration profiles obtained for cement system 2 after 2 and 6 (final time step) years. RTM means the original HPx$_{\\rm{4C}}$ model and TM+DNN denotes the Hydrus transport model coupled with our DNN geochemical emulator (HPx$_{\\rm py}$-DNN). The first to fourth row present profiles for $Ca^{\\rm{conc}}$, $Si^{\\rm{conc}}$, $H^{\\rm{conc}}$, and $O^{\\rm{conc}}$, respectively. The considered grid size is 61 $\\times$ 61.}\n\t\\label{fig11}\n\\end{figure}\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure12.png}\n\t\\caption{2D concentration profiles obtained for cement system 2 after 2 and 6 (final time step) years. RTM means the original HPx$_{\\rm{4C}}$ model and TM+DNN denotes the Hydrus transport model coupled with our DNN geochemical emulator (HPx$_{\\rm py}$-DNN). The first to third row present profiles for $C^{\\rm{conc}}$, $Al^{\\rm{conc}}$, and $S^{\\rm{conc}}$, respectively. The considered grid size is 61 $\\times$ 61.}\n\t\\label{fig12}\n\\end{figure}\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure13.png}\n\t\\caption{2D concentration profiles obtained for cement system 2 after 2 and 6 (final time step) years. RTM means the original HPx$_{\\rm{4C}}$ model and TM+DNN denotes the Hydrus transport model coupled with our DNN geochemical emulator (HPx$_{\\rm py}$-DNN). The first to fourth row present profiles for $Ca^{\\rm{conc}}$, $Si^{\\rm{conc}}$, $H^{\\rm{conc}}$, and $O^{\\rm{conc}}$, respectively. The considered grid size is 121 $\\times$ 121.}\n\t\\label{fig13}\n\\end{figure}\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure14.png}\n\t\\caption{2D concentration profiles obtained for cement system 2 after 2 and 6 (final time step) years. RTM means the original HPx$_{\\rm{4C}}$ model and TM+DNN denotes the Hydrus transport model coupled with our DNN geochemical emulator (HPx$_{\\rm py}$-DNN). The first to third row present profiles for $C^{\\rm{conc}}$, $Al^{\\rm{conc}}$, and $S^{\\rm{conc}}$, respectively. The considered grid size is 121 $\\times$ 121.}\n\t\\label{fig14}\n\\end{figure}\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure15.png}\n\t\\caption{2D concentration profiles obtained for cement system 2 after 2 and 6 (final time step) years. RTM means the original HPx$_{\\rm{4C}}$ model and TM+kNN denotes the Hydrus transport model coupled with our kNN geochemical emulator (HPx$_{\\rm py}$-kNN).  The first to third row present profiles for $C^{\\rm{conc}}$, $Al^{\\rm{conc}}$, and $S^{\\rm{conc}}$, respectively. The considered grid size is 61 $\\times$ 61.}\n\t\\label{fig15}\n\\end{figure}\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure16.png}\n\t\\caption{2D concentration profiles obtained for cement system 2 after 2 and 6 (final time step) years. RTM means the original HPx$_{\\rm{4C}}$ model and TM+kNN denotes the Hydrus transport model coupled with our kNN geochemical emulator (HPx$_{\\rm py}$-kNN). The first to third row present profiles for $C^{\\rm{conc}}$, $Al^{\\rm{conc}}$, and $S^{\\rm{conc}}$, respectively. The considered grid size is 121 $\\times$ 121.}\n\t\\label{fig16}\n\\end{figure}\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure17.png}\n\t\\caption{2D solid amount profiles of the C, Al and S chemical components of cement system 2 after 2 and 6 (final time step) years. RTM means the original HPx$_{\\rm{4C}}$ model and TM+DNN denotes the Hydrus transport model coupled with our DNN geochemical emulator (HPx$_{\\rm py}$-DNN). The first to third row present profiles for $C^{\\rm{solid}}$, $Al^{\\rm{solid}}$, and $S^{\\rm{solid}}$, respectively The considered grid size is 61 $\\times$ 61.}\n\t\\label{fig17}\n\\end{figure}\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure18.png}\n\t\\caption{2D solid amount profiles of the C, Al and S chemical components of cement system 2 after 2 and 6 (final time step) years. RTM means the original HPx$_{\\rm{4C}}$ model and TM+DNN denotes the Hydrus transport model coupled with our DNN geochemical emulator (HPx$_{\\rm py}$-DNN). The first to third row present profiles for $C^{\\rm{solid}}$, $Al^{\\rm{solid}}$, and $S^{\\rm{solid}}$, respectively The considered grid size is 121 $\\times$ 121.}\n\t\\label{fig18}\n\\end{figure}\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure19.png}\n\t\\caption{2D solid amount profiles of the C, Al and S chemical components of cement system 2 after 2 and 6 (final time step) years. RTM means the original HPx$_{\\rm{4C}}$ model and TM+kNN denotes the Hydrus transport model coupled with our kNN geochemical emulator (HPx$_{\\rm py}$-kNN). The first to third row present profiles for $C^{\\rm{solid}}$, $Al^{\\rm{solid}}$, and $S^{\\rm{solid}}$, respectively The considered grid size is 61 $\\times$ 61.}\n\t\\label{fig19}\n\\end{figure}\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure20.png}\n\t\\caption{2D solid amount profiles of the C, Al and S chemical components of cement system 2 after 2 and 6 (final time step) years. RTM means the original HPx$_{\\rm{4C}}$ model and TM+kNN denotes the Hydrus transport model coupled with our kNN geochemical emulator (HPx$_{\\rm py}$-kNN). The first to third row present profiles for $C^{\\rm{solid}}$, $Al^{\\rm{solid}}$, and $S^{\\rm{solid}}$, respectively The considered grid size is 121 $\\times$ 121.}\n\t\\label{fig20}\n\\end{figure}\n\n\\begin{table}[h!]\n\t\\caption{Speedups offered by the KNN and DNN emulators in HPx$_{\\rm py}$ for the reactive transport simulations considered for cement system 2. The HPx$_{\\rm{4C}}$ calculations involve the parallelization of PHREEQC over our 4 CPUs. The HPx$_{\\rm{1C}}$ calculations are performed on a single CPU. Both the kNN and DNN predictions make use of a GPU. ML signifies the used machine learning method for emulation, BC denotes the type of flow boundary conditions and GS is the grid size. The maximum possible speedups associated with HPx$_{\\rm{4C}}$ and HP$_{\\rm{1C}}$,  Max SP HPx$_{\\rm{4C}}$ and  Max SP HPx$_{\\rm{1C}}$, correspond to an hypothetical situation where the geochemical calculations incur zero computational cost.}\n\n\t\\begin{center}\n\t\t\\begin{tabular}{cccccccc}\n\t\t\t\\hline\n\t\t\t& & & & & & & \\\\\n\t\t\tML & BC & GS & HPx$_{\\rm{4C}}$ time (s) & SP HPx$_{\\rm{4C}}$ & Max SP HPx$_{\\rm{4C}}$ & SP HPx$_{\\rm{1C}}$ & Max SP HPx$_{\\rm{1C}}$ \\\\\n\t\t\tDNN & ADV & 61 $\\times$ 61 & 21,415 & 8.2 & 9.0 & 30.3 & 33.1\\\\\n\t\t\tkNN & ADV & 61 $\\times$ 61 & 21,415 & 7.9 & 9.0 & 28.9 & 33.1\\\\\n\t\t\tDNN & ADV & 121 $\\times$ 121 & 199,841 & 8.2 & 9.5 & 29.9 & 32.8\\\\\n\t\t\tkNN & ADV & 121 $\\times$ 121 & 199,841 & 8.5 & 9.5 & 31.3 & 32.8\\\\\n\t\t\t\\hline\n\t\t\\end{tabular}\n\t\\end{center}\n\n\t\\label{table4}\n\\end{table}\n\n\\FloatBarrier\n\n3.2 Al-C-Ca-S-Si Problem\n\\subsection{Al-C-Ca-S-Si Problem}\n\nFor this second cement system, the emulation problem consists of predicting at each time step of the RT simulation the (output) Al, C, Ca, S, Si, H and O aqueous concentrations (mol/kgw) from the (input) total amounts of Al, C, Ca, S and Si (mol). Here we focus on advective-dispersive transport only while similarly as for cement system 1, the considered domain sizes are 61 $\\times$ 61 and 121 $\\times$ 121. Moreover, the simulation time period is set to 6 years. \n\nAs mentioned earlier, for this higher-dimensional problem it is observed that the used scikit-learn kNN implementation becomes prohibitively slow compared to HPx$_{\\rm{4C}}$. We found that to get a good emulation accuracy the kNN training base needs to contain 1,000,000 samples (or more). This training base's size together with a 5-dimensional search space leads to an HPx$_{\\rm py}$-kNN reactive transport simulation time that is comparable to that of HPx$_{\\rm{4C}}$. Therefore, for this second cement system we built a custom kNN regressor around another kNN implementation contained in the FAISS package \\citep{faiss2017}. Our used FAISS variant allows for GPU computing and is much faster than scikit-learn for this cement system, but is slightly less accurate due to the use of an approximate rather than exact nearest neighbor search \\citep[see][for details]{faiss2017}.\n\n\\subsubsection{Training the DNN}\n\\label{train_res2}\n\nBuilding a good training set to perform a kNN search and learn the weights and biases of our DNN turned out to be a complicated task in this case. This because to make useful kNN predictions and/or learn an useful DNN, the training set must be sufficiently representative of the geochemical conditions encountered during the reactive transport simulation one wish to perform with HPx$_{\\rm py}$-kNN and the trained HPx$_{\\rm py}$-DNN. In contrast to cement system 1, creating the training set by sampling the $X$-space with a controlled randomness between predefined lower and upper bounds did not prove successful. We tried that strategy by drawing as much as 4,000,000 5-dimensional $\\textbf{x}$ vectors from the $X$-space using a Sobol low-discrepancy sequence \\citep[][]{Sobol1967, Joe-Kuo2003}. Such low-discrepancy sampling scheme covers the 5-dimensional hypercube more uniformly than LHS. Despite a good performance on the test set (not shown), the resulting DNN accuracy in reactive transport mode was never deemed satisfying. In other words, no satisfying ``global\" or ``universal\" DNN emulator could be devised for this cement system. This is probably caused by the fact for this problem, the input (5 total amounts) and output (7 aqueous concentrations) spaces are quite nonlinearly related and both cover 6 to 10 orders of magnitudes depending on the considered element. Therefore, we resorted to the alternative training strategy detailed below. The latter basically tries to grasp the complex correlations and higher-order dependencies that exist between the elements of $\\textbf{x}$ (total amounts, input space) for a given reactive transport simulation setup, in order to produce a training set that honors these between-input relationships.\n\n\\begin{itemize}\n\t\\item Perform a ``cheap\" full reactive transport simulation under the transport conditions and geochemistry of interest and collect the resulting $\\textbf{x}$ -$\\textbf{y}$  pairs of examples (for the considered grid nodes and time steps). Computational demand controls what domain size and simulation time period can be used for this cheap calculation. We used a modest 16 $\\times$ 16 domain and a simulation time period of 10 years. The associated HPx$_{\\rm{4C}}$ runtimes is 180 s.\n\t\n\t\\item Fit a kernel density estimator (KDE) with a Gaussian kernel to the collected $\\textbf{x}$ vectors (encapsulated in the $\\textbf{X}$ array) and generate a fixed number of new input vectors, $\\textbf{x}_{KDE}$. Then run PHREEQC for the $\\textbf{X}_{KDE}$ set to get the corresponding output set, $\\textbf{Y}_{KDE}$. Now apply the correction for porosity described in section \\ref{train_res1} to the $\\textbf{X}_{KDE}$ set and form the training set by merging the ensemble of $\\textbf{x}$-$\\textbf{y}$ pairs with that of the $\\textbf{x}_{KDE}$-$\\textbf{y}_{KDE}$ pairs. The number of produced unique examples by the considered cheap HPx$_{\\rm{4C}}$ simulations varied between 10,000 and 50,000. The KDE-based enrichment of this dataset was deemed necessary to provide more input variability, thereby avoiding overfitting of the trained DNN and improving the kNN accuracy, while still honoring the complex between-input relationships. The number of KDE-generated samples was set as to obtain a total training set size of 1,000,000 examples. A key component of the approach is the bandwith parameter of the Gaussian KDE kernel which controls how much the KDE-generated samples depart from the original ensemble. After limited trial and error, we fixed the kernel bandwith to 0.0025 for the considered case studies.\n\t\n\\end{itemize}\n\nThe scatter plots in Figure \\ref{fig9} illustrate our training set creation procedure. The orange dots depict the pairwise relationships observed between the 5 elements of $\\textbf{x}$ in the cheap simulation. The red and cyan dots in Figure \\ref{fig9} represent the KDE-generated samples with the selected bandwidth, before and after applying the correction for porosity, respectively. Training of the DNN is achieved using the ensemble of original and KDE-corrected input points. We refer to this kind of dataset as RT-based, since it is based on an albeit cheap, full RT simulation. Furthermore, we refer to the obtained DNN and kNN emulators as ``local\" emulators, since as opposed to the emulators constructed for cement system 1, the current emulators are only valid for the input conditions encapsulated in the RT-based training set.\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1.5cm}\\includegraphics[width=50pc]{Figure9.png}\n\t\\caption{Scatter plots of the complex relationships between the five considered inputs for (1) the computationally cheap RT simulation of cement system 2 (orange dots) performed with the original HPx code, (2) the corresponding sampled points by kernel density estimation (KDE) using the selected bandwith (turquoise dots) and, (3) the same sampled KDE points after porosity correction (red dots).}\n\t\\label{fig9}\n\\end{figure}\n\nTraining performance of the DNN emulator is presented in Figure \\ref{fig10}. Similarly as for cement system 1, about 90\\% of the available data was used for the actual training of the DNN while the remaining 10\\% were used a validation set to contol overfitting. Lastly, performance is evaluated for both the trained DNN and kNN emulators using an independent test set of 10,000 examples. Overall, the accuracy of our ``local \" DNN emulator for this RT-inspired dataset is rather large with $Q_2$ values always greater or equal than 0.998. Training performance of the corresponding local kNN emulator is equally good (not shown). Regarding speedup, for this problem single-threaded PHREEQC achieves about 210 geochemical calculations per second on our used Intel\\textsuperscript{\\textregistered} i7 CPU while the GPU-based DNN and kNN emulators are both about 3000 times faster when predicting the 10,000 test points at once .\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure10.png}\n\t\\caption{1-1 plots of local DNN emulation performance obtained for system 2 when the local DNN is trained using 1,000,000 samples. The $x$-axis and $y$-axis present the original and emulated 10,000 independent test data points, respectively. The RMSE and $Q_2$ coefficient denote the root-mean-square-error and coefficient of determination in testing mode, respectively, between the original and emulated 10,000 test data points.}\n\t\\label{fig10}\n\\end{figure}\n\n\\subsubsection{Reactive transport simulation}\n\\label{prob2res}\nOur ``local\" DNN performs rather well when applied to the 61 $\\times$ 61 grid size and a time period of 6 years (Figures \\ref{fig11} - \\ref{fig12}). This for all components but C, for which some localized deviations appear between original and emulated 2D concentration profiles towards the end of the simulation period (Figure \\ref{fig12}). When applied to the 121 $\\times$ 121 grid, HPx$_{\\rm py}$-DNN produces additional discrepancies for O, Si and Al towards the end of the simulation period (Figures \\ref{fig13} - \\ref{fig14}). Yet most of the observed artifacts could probably be smoothed out by using post-filtering. The associated speedups are listed in Table \\ref{table4}. These speedups are larger to those obtained for cement system 1, with values between 8 and 9 when evaluated against HPx$_{\\rm{4C}}$. These speedups represent about 85 \\% to 90 \\% of the maximum possible speedup (Table \\ref{table3}). Overall, these findings indicate that for the considered problem, our RT-based training of a local DNN only works is the training set is sufficiently representative of the particular geochemical conditions encountered in the computationally demanding simulations, which is arguably not easy to achieve. This limitation is further discussed in section \\ref{discussion}.\n\nWe note a more uniform behavior for HPx$_{\\rm py}$-kNN across grid sizes than for HPx$_{\\rm py}$-DNN. Here the results for the 121  $\\times$ 121 grid are only slightly less accurate than those associated with the 61 $ \\times$ 61 grid (see Figures \\ref{fig15} - \\ref{fig16} where for brevity we only show concentration profiles for the C Al and S components). Furthermore, whenever observed the discrepancies between original and emulated profiles are more regularly scattered than for HPx$_{\\rm py}$-DNN. Note also that herein too, post-filtering could likely smooth out a large part of these deviations. In addition, owing to the use of a GPU to achieve the kNN calculations, the speedup provided by HPx$_{\\rm py}$-kNN are as large as those provided by HPx$_{\\rm py}$-DNN (Table \\ref{table4}).\n\nWith respect to the emulated solid amounts, the HPx$_{\\rm py}$-DNN results look visually good for the the 61 $\\times$ 61 grid. This is shown in Figure \\ref{fig17} for the C, Al and S chemical components, while emulation of the H, O Ca and Si chemical components is globally of similar quality (not shown). Nevertheless, significant deviations appear towards the end of the simulation period for every chemical component (see Figure \\ref{fig18} for the C, Al and S chemical components, emulation of the H, O Ca and Si chemical components shows the same level of mismatch). The HPx$_{\\rm py}$-kNN predictions are also fairly accurate for the 61 $\\times$ 61 (see Figure \\ref{fig19} for the C, Al and S chemical components, emulation of the H, O Ca and Si chemical components exhibits a globally similar quality) while some discrepancies show up at the end of the simulation (Figure \\ref{fig20}). However, the mismatch is less pronounced than for HPx$_{\\rm py}$-DNN. Overall, HPx$_{\\rm py}$-kNN appears to be somewhat more robust than HPx$_{\\rm py}$-DNN for this cement system, while providing the same (large) speedup.\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure11.png}\n\t\\caption{2D concentration profiles obtained for cement system 2 after 2 and 6 (final time step) years. RTM means the original HPx$_{\\rm{4C}}$ model and TM+DNN denotes the Hydrus transport model coupled with our DNN geochemical emulator (HPx$_{\\rm py}$-DNN). The first to fourth row present profiles for $Ca^{\\rm{conc}}$, $Si^{\\rm{conc}}$, $H^{\\rm{conc}}$, and $O^{\\rm{conc}}$, respectively. The considered grid size is 61 $\\times$ 61.}\n\t\\label{fig11}\n\\end{figure}\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure12.png}\n\t\\caption{2D concentration profiles obtained for cement system 2 after 2 and 6 (final time step) years. RTM means the original HPx$_{\\rm{4C}}$ model and TM+DNN denotes the Hydrus transport model coupled with our DNN geochemical emulator (HPx$_{\\rm py}$-DNN). The first to third row present profiles for $C^{\\rm{conc}}$, $Al^{\\rm{conc}}$, and $S^{\\rm{conc}}$, respectively. The considered grid size is 61 $\\times$ 61.}\n\t\\label{fig12}\n\\end{figure}\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure13.png}\n\t\\caption{2D concentration profiles obtained for cement system 2 after 2 and 6 (final time step) years. RTM means the original HPx$_{\\rm{4C}}$ model and TM+DNN denotes the Hydrus transport model coupled with our DNN geochemical emulator (HPx$_{\\rm py}$-DNN). The first to fourth row present profiles for $Ca^{\\rm{conc}}$, $Si^{\\rm{conc}}$, $H^{\\rm{conc}}$, and $O^{\\rm{conc}}$, respectively. The considered grid size is 121 $\\times$ 121.}\n\t\\label{fig13}\n\\end{figure}\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure14.png}\n\t\\caption{2D concentration profiles obtained for cement system 2 after 2 and 6 (final time step) years. RTM means the original HPx$_{\\rm{4C}}$ model and TM+DNN denotes the Hydrus transport model coupled with our DNN geochemical emulator (HPx$_{\\rm py}$-DNN). The first to third row present profiles for $C^{\\rm{conc}}$, $Al^{\\rm{conc}}$, and $S^{\\rm{conc}}$, respectively. The considered grid size is 121 $\\times$ 121.}\n\t\\label{fig14}\n\\end{figure}\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure15.png}\n\t\\caption{2D concentration profiles obtained for cement system 2 after 2 and 6 (final time step) years. RTM means the original HPx$_{\\rm{4C}}$ model and TM+kNN denotes the Hydrus transport model coupled with our kNN geochemical emulator (HPx$_{\\rm py}$-kNN).  The first to third row present profiles for $C^{\\rm{conc}}$, $Al^{\\rm{conc}}$, and $S^{\\rm{conc}}$, respectively. The considered grid size is 61 $\\times$ 61.}\n\t\\label{fig15}\n\\end{figure}\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure16.png}\n\t\\caption{2D concentration profiles obtained for cement system 2 after 2 and 6 (final time step) years. RTM means the original HPx$_{\\rm{4C}}$ model and TM+kNN denotes the Hydrus transport model coupled with our kNN geochemical emulator (HPx$_{\\rm py}$-kNN). The first to third row present profiles for $C^{\\rm{conc}}$, $Al^{\\rm{conc}}$, and $S^{\\rm{conc}}$, respectively. The considered grid size is 121 $\\times$ 121.}\n\t\\label{fig16}\n\\end{figure}\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure17.png}\n\t\\caption{2D solid amount profiles of the C, Al and S chemical components of cement system 2 after 2 and 6 (final time step) years. RTM means the original HPx$_{\\rm{4C}}$ model and TM+DNN denotes the Hydrus transport model coupled with our DNN geochemical emulator (HPx$_{\\rm py}$-DNN). The first to third row present profiles for $C^{\\rm{solid}}$, $Al^{\\rm{solid}}$, and $S^{\\rm{solid}}$, respectively The considered grid size is 61 $\\times$ 61.}\n\t\\label{fig17}\n\\end{figure}\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure18.png}\n\t\\caption{2D solid amount profiles of the C, Al and S chemical components of cement system 2 after 2 and 6 (final time step) years. RTM means the original HPx$_{\\rm{4C}}$ model and TM+DNN denotes the Hydrus transport model coupled with our DNN geochemical emulator (HPx$_{\\rm py}$-DNN). The first to third row present profiles for $C^{\\rm{solid}}$, $Al^{\\rm{solid}}$, and $S^{\\rm{solid}}$, respectively The considered grid size is 121 $\\times$ 121.}\n\t\\label{fig18}\n\\end{figure}\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure19.png}\n\t\\caption{2D solid amount profiles of the C, Al and S chemical components of cement system 2 after 2 and 6 (final time step) years. RTM means the original HPx$_{\\rm{4C}}$ model and TM+kNN denotes the Hydrus transport model coupled with our kNN geochemical emulator (HPx$_{\\rm py}$-kNN). The first to third row present profiles for $C^{\\rm{solid}}$, $Al^{\\rm{solid}}$, and $S^{\\rm{solid}}$, respectively The considered grid size is 61 $\\times$ 61.}\n\t\\label{fig19}\n\\end{figure}\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure20.png}\n\t\\caption{2D solid amount profiles of the C, Al and S chemical components of cement system 2 after 2 and 6 (final time step) years. RTM means the original HPx$_{\\rm{4C}}$ model and TM+kNN denotes the Hydrus transport model coupled with our kNN geochemical emulator (HPx$_{\\rm py}$-kNN). The first to third row present profiles for $C^{\\rm{solid}}$, $Al^{\\rm{solid}}$, and $S^{\\rm{solid}}$, respectively The considered grid size is 121 $\\times$ 121.}\n\t\\label{fig20}\n\\end{figure}\n\n\\begin{table}[h!]\n\t\\caption{Speedups offered by the KNN and DNN emulators in HPx$_{\\rm py}$ for the reactive transport simulations considered for cement system 2. The HPx$_{\\rm{4C}}$ calculations involve the parallelization of PHREEQC over our 4 CPUs. The HPx$_{\\rm{1C}}$ calculations are performed on a single CPU. Both the kNN and DNN predictions make use of a GPU. ML signifies the used machine learning method for emulation, BC denotes the type of flow boundary conditions and GS is the grid size. The maximum possible speedups associated with HPx$_{\\rm{4C}}$ and HP$_{\\rm{1C}}$,  Max SP HPx$_{\\rm{4C}}$ and  Max SP HPx$_{\\rm{1C}}$, correspond to an hypothetical situation where the geochemical calculations incur zero computational cost.}\n\n\t\\begin{center}\n\t\t\\begin{tabular}{cccccccc}\n\t\t\t\\hline\n\t\t\t& & & & & & & \\\\\n\t\t\tML & BC & GS & HPx$_{\\rm{4C}}$ time (s) & SP HPx$_{\\rm{4C}}$ & Max SP HPx$_{\\rm{4C}}$ & SP HPx$_{\\rm{1C}}$ & Max SP HPx$_{\\rm{1C}}$ \\\\\n\t\t\tDNN & ADV & 61 $\\times$ 61 & 21,415 & 8.2 & 9.0 & 30.3 & 33.1\\\\\n\t\t\tkNN & ADV & 61 $\\times$ 61 & 21,415 & 7.9 & 9.0 & 28.9 & 33.1\\\\\n\t\t\tDNN & ADV & 121 $\\times$ 121 & 199,841 & 8.2 & 9.5 & 29.9 & 32.8\\\\\n\t\t\tkNN & ADV & 121 $\\times$ 121 & 199,841 & 8.5 & 9.5 & 31.3 & 32.8\\\\\n\t\t\t\\hline\n\t\t\\end{tabular}\n\t\\end{center}\n\n\t\\label{table4}\n\\end{table}\n\n\\FloatBarrier\n\n3.2.1 Training the DNN\n\\subsubsection{Training the DNN}\n\\label{train_res2}\n\nBuilding a good training set to perform a kNN search and learn the weights and biases of our DNN turned out to be a complicated task in this case. This because to make useful kNN predictions and/or learn an useful DNN, the training set must be sufficiently representative of the geochemical conditions encountered during the reactive transport simulation one wish to perform with HPx$_{\\rm py}$-kNN and the trained HPx$_{\\rm py}$-DNN. In contrast to cement system 1, creating the training set by sampling the $X$-space with a controlled randomness between predefined lower and upper bounds did not prove successful. We tried that strategy by drawing as much as 4,000,000 5-dimensional $\\textbf{x}$ vectors from the $X$-space using a Sobol low-discrepancy sequence \\citep[][]{Sobol1967, Joe-Kuo2003}. Such low-discrepancy sampling scheme covers the 5-dimensional hypercube more uniformly than LHS. Despite a good performance on the test set (not shown), the resulting DNN accuracy in reactive transport mode was never deemed satisfying. In other words, no satisfying ``global\" or ``universal\" DNN emulator could be devised for this cement system. This is probably caused by the fact for this problem, the input (5 total amounts) and output (7 aqueous concentrations) spaces are quite nonlinearly related and both cover 6 to 10 orders of magnitudes depending on the considered element. Therefore, we resorted to the alternative training strategy detailed below. The latter basically tries to grasp the complex correlations and higher-order dependencies that exist between the elements of $\\textbf{x}$ (total amounts, input space) for a given reactive transport simulation setup, in order to produce a training set that honors these between-input relationships.\n\n\\begin{itemize}\n\t\\item Perform a ``cheap\" full reactive transport simulation under the transport conditions and geochemistry of interest and collect the resulting $\\textbf{x}$ -$\\textbf{y}$  pairs of examples (for the considered grid nodes and time steps). Computational demand controls what domain size and simulation time period can be used for this cheap calculation. We used a modest 16 $\\times$ 16 domain and a simulation time period of 10 years. The associated HPx$_{\\rm{4C}}$ runtimes is 180 s.\n\t\n\t\\item Fit a kernel density estimator (KDE) with a Gaussian kernel to the collected $\\textbf{x}$ vectors (encapsulated in the $\\textbf{X}$ array) and generate a fixed number of new input vectors, $\\textbf{x}_{KDE}$. Then run PHREEQC for the $\\textbf{X}_{KDE}$ set to get the corresponding output set, $\\textbf{Y}_{KDE}$. Now apply the correction for porosity described in section \\ref{train_res1} to the $\\textbf{X}_{KDE}$ set and form the training set by merging the ensemble of $\\textbf{x}$-$\\textbf{y}$ pairs with that of the $\\textbf{x}_{KDE}$-$\\textbf{y}_{KDE}$ pairs. The number of produced unique examples by the considered cheap HPx$_{\\rm{4C}}$ simulations varied between 10,000 and 50,000. The KDE-based enrichment of this dataset was deemed necessary to provide more input variability, thereby avoiding overfitting of the trained DNN and improving the kNN accuracy, while still honoring the complex between-input relationships. The number of KDE-generated samples was set as to obtain a total training set size of 1,000,000 examples. A key component of the approach is the bandwith parameter of the Gaussian KDE kernel which controls how much the KDE-generated samples depart from the original ensemble. After limited trial and error, we fixed the kernel bandwith to 0.0025 for the considered case studies.\n\t\n\\end{itemize}\n\nThe scatter plots in Figure \\ref{fig9} illustrate our training set creation procedure. The orange dots depict the pairwise relationships observed between the 5 elements of $\\textbf{x}$ in the cheap simulation. The red and cyan dots in Figure \\ref{fig9} represent the KDE-generated samples with the selected bandwidth, before and after applying the correction for porosity, respectively. Training of the DNN is achieved using the ensemble of original and KDE-corrected input points. We refer to this kind of dataset as RT-based, since it is based on an albeit cheap, full RT simulation. Furthermore, we refer to the obtained DNN and kNN emulators as ``local\" emulators, since as opposed to the emulators constructed for cement system 1, the current emulators are only valid for the input conditions encapsulated in the RT-based training set.\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1.5cm}\\includegraphics[width=50pc]{Figure9.png}\n\t\\caption{Scatter plots of the complex relationships between the five considered inputs for (1) the computationally cheap RT simulation of cement system 2 (orange dots) performed with the original HPx code, (2) the corresponding sampled points by kernel density estimation (KDE) using the selected bandwith (turquoise dots) and, (3) the same sampled KDE points after porosity correction (red dots).}\n\t\\label{fig9}\n\\end{figure}\n\nTraining performance of the DNN emulator is presented in Figure \\ref{fig10}. Similarly as for cement system 1, about 90\\% of the available data was used for the actual training of the DNN while the remaining 10\\% were used a validation set to contol overfitting. Lastly, performance is evaluated for both the trained DNN and kNN emulators using an independent test set of 10,000 examples. Overall, the accuracy of our ``local \" DNN emulator for this RT-inspired dataset is rather large with $Q_2$ values always greater or equal than 0.998. Training performance of the corresponding local kNN emulator is equally good (not shown). Regarding speedup, for this problem single-threaded PHREEQC achieves about 210 geochemical calculations per second on our used Intel\\textsuperscript{\\textregistered} i7 CPU while the GPU-based DNN and kNN emulators are both about 3000 times faster when predicting the 10,000 test points at once .\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure10.png}\n\t\\caption{1-1 plots of local DNN emulation performance obtained for system 2 when the local DNN is trained using 1,000,000 samples. The $x$-axis and $y$-axis present the original and emulated 10,000 independent test data points, respectively. The RMSE and $Q_2$ coefficient denote the root-mean-square-error and coefficient of determination in testing mode, respectively, between the original and emulated 10,000 test data points.}\n\t\\label{fig10}\n\\end{figure}\n\n",
        "label": "fig9",
        "Descriptive_question1": "What color represents the computationally cheap RT simulation data points in figure_9?",
        "Descriptive_question2": "What color represents the KDE sampled points after porosity correction in figure_9?",
        "Reasoning_question1": "How does the distribution of KDE sampled points before and after porosity correction differ in figure_9, and what might this indicate about the effect of porosity correction on the data?",
        "Reasoning_question2": "Based on the scatter plots in figure_9, what can be inferred about the relationship between the five input variables for cement system 2, and how might this complexity impact the training of emulators?",
        "Descriptive_answer1": "orange dots",
        "Descriptive_answer2": "red dots",
        "Reasoning_answer1": "In figure_9, the KDE sampled points before porosity correction are shown as turquoise dots, while the KDE sampled points after porosity correction are shown as red dots. The red dots (after correction) appear to slightly shift or adjust positions relative to the turquoise dots, indicating that porosity correction modifies the KDE generated samples to better reflect the corrected or true input conditions. This suggests that porosity correction refines the distribution of synthetic data points to more accurately represent the underlying geochemical states, ensuring the training data better matches real simulation scenarios.",
        "Reasoning_answer2": "The scatter plots in figure_9 display complex, non-linear relationships among the five input variables for cement system 2, as evident by the non-trivial distribution and clustering patterns of the orange dots (original RT simulation data) and the KDE sampled points. This complexity reflects strong correlations and dependencies among inputs rather than simple independent or linear relationships. Such complexity makes it challenging to construct a universally valid emulator by naive sampling methods, necessitating targeted training datasets, like the RT-based KDE-enriched sets, to capture these intricate dependencies and thus improve emulator accuracy and reliability."
    },
    {
        "paper_id": "2107.07598.json",
        "image_id": "figure_10",
        "image_path": [
            "/home/yz979/scratch/chengye/scimolmo/test-images-tables/dataset_imagestables/2107.07598/images/Figure10.png"
        ],
        "caption": "1-1 plots of local DNN emulation performance obtained for system 2 when the local DNN is trained using 1,000,000 samples. The $x$-axis and $y$-axis present the original and emulated 10,000 independent test data points, respectively. The RMSE and $Q_2$ coefficient denote the root-mean-square-error and coefficient of determination in testing mode, respectively, between the original and emulated 10,000 test data points.",
        "classify": "Chart",
        "section_info": "3 Results\n\\section{Results}\n\\label{results}\n\n\\subsection{Ca-Si Problem}\nFor this first cement system, the emulation problem consists of predicting at each time step of the RT simulation the (output) Ca, Si, H and O aqueous concentrations (mol/kg of water or mol/kgw) from the (input) total amounts of Ca and Si (mol). \n\n\\subsubsection{Training the emulators}\n\\label{train_res1}\nHere the kNN and DNN emulators are firstly trained using a set of 400,000 test examples for both. This training set is obtained by randomly sampling the two-dimensional input space by latin hypercube sampling (LHS) between $\\left[0,0\\right]$ and $\\rm{\\left[Ca^{tot}_{max},Si^{tot}_{max}\\right]}$, and running PHREEQC for each input sample, $\\rm{\\textbf{x}_i = \\left[Ca^{tot}_i, Si^{tot}_i\\right]}$ to get the corresponding output vectors, $\\rm{\\textbf{y}_i = \\left[Ca^{conc}_i,Si^{conc}_i,H^{conc}_i,O^{conc}_i\\right]}$. The upper bounds, $\\rm{Ca^{tot}_{max}}$ and $\\rm{Si^{tot}_{max}}$ are defined based on a cheap full RT simulation with advective-dispersive transport using a small 1D domain of 51 nodes. It is worth noting that the total amounts of $\\textbf{x}_i$, corresponding to the PHREEQC-simulated concentrations, $\\textbf{y}_i$, have to be corrected for the different amount of water between the training set and the transport simulations. Doing so, it turns out that about 20 \\% of the post-corrected $x_i$ values exceed their pre-defined upper bounds and these excessively large values need to be filtered out. Creating the 400,000 training examples thus required about 500,000 PHREEQC runs. As stated earlier, for this problem single-threaded PHREEQC performs about 670 geochemical calculations per second on our used Intel\\textsuperscript{\\textregistered} i7 CPU.\n\nWith respect to kNN, the tuning parameters are the number of neighbors, $k$, the type of distance measure, and the interpolation technique. We simply used the default settings: $k = 5$, euclidean distance and inverse-distance interpolation. Regarding training of the DNN, the 400,000 sample were split between the training set itself (90 \\% of the data) and a validation set (10 \\% of the data). The latter serves to monitor the evolution of the selected mean squared error loss function on samples that are not used for training, thereby detecting potential overfitting. If the validation loss stops decreasing before the fixed number of epochs has been completed, then  training is stopped. Importantly, the emulation is achieved in log-space for both the input, $X$ and output $Y$, domains. This because total amounts and concentrations of the involved components typically cover many orders of magnitudes (up to 10 orders or more). Using a DNN also requires some form of data normalization or standardization. Here both the $\\rm{log\\left(\\textbf{x}_i\\right)}$ and $\\rm{log\\left(\\textbf{y}_i\\right)}$ vectors are standardized around 0 with standard deviation of 1.\n\nFigure \\ref{fig1} illustrates the trained emulators' performance for geochemical predictions using an independent test set that comprises 10,000 test examples. Both kNN (Figures \\ref{fig1}a - d) and DNN (Figures \\ref{fig1}e - h) appear to be rather accurate . DNN also shows a slight degradation for the larger concentration values  (Figures \\ref{fig1}e - h). The latter is likely due to the combination of a small proportion of large concentration values in the training set with the log-transformation that implicitly pushes the DNN to try harder to fit the smaller concentrations during training. Regarding speedup and as written earlier, for this setup the single-threaded kNN method is 300 times faster than single-threaded PHREEQC for predicting the 10,000 concentration vectors all at once. The computational savings allowed by the DNN emulator when ran on our GPU is higher with a speedup as large as 4000 for predicting the same 10,000 concentration vectors all at once. \n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{0cm}\\includegraphics[width=35pc]{Figure1.png}\n\t\\caption{1-1 plots of the kNN (subfigures (a) - (d)) and DNN (subfigures (e) - (h)) emulators' performance obtained for system 1 when the kNN training base contains 400,000 samples and the DNN is trained using the same 400,000 samples. Here ``true\" means the original PHREEQC-simulated data and ``predicted\" denotes the emulated (that is, kNN-simulated and DNN-simulated) data. Hence, the $x$-axis and $y$-axis present the original and emulated 10,000 independent test data points, respectively. The RMSE and $Q_2$ coefficient denote the root-mean-square-error and coefficient of determination in testing mode, respectively, between the original and emulated 10,000 test data points.}\n\t\\label{fig1}\n\\end{figure}\n\nTo test the sensitivity of the emulators' performance to the training set size, training was also performed using reduced training sets comprising 100,000, 10,000, 1000 and 100 samples, respectively. It seen that the DNN performance achieved  when using 10,000 training samples is virtually the same as that obtained when using 400,000 training samples (Figure \\ref{fig2} and Table \\ref{table2}). It is only for training sets smaller than 1000 samples that the DNN performance starts to degrade significantly (Table \\ref{table2}). In contrast, the behavior of kNN appears to markedly decrease as the training set gets smaller (Figure \\ref{fig2} and Table \\ref{table1}).\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{0cm}\\includegraphics[width=35pc]{Figure2.png}\n\t\\caption{1-1 plots of the kNN (subfigures (a) - (d)) and DNN (subfigures (e) - (h)) emulators' performance obtained for system 1 when the kNN training base contains 10,000 samples and the DNN is trained using the same 10,000 samples. Here ``true\" means the original PHREEQC-simulated data and ``predicted\" denotes the emulated (that is, kNN-simulated and DNN-simulated) data. Hence, the $x$-axis and $y$-axis present the original and emulated 10,000 independent test data points, respectively. The RMSE and $Q_2$ coefficient denote the root-mean-square-error and coefficient of determination in testing mode, respectively, between the original and emulated 10,000 test data points.}\n\t\\label{fig2}\n\\end{figure}\n\n\\begin{table}[h!]\n\t\\caption{Performance of the DNN and kNN emulators for cement system 1 and different training set sizes. For brevity, only the results for Ca$^{conc}$ and Si$^{conc}$ are shown. The units are mol per kg of water (mol/kgw). ML refers to the type of emulator, TR signifies the size of the training set, and the RMSE and $Q_2$ coefficient denote the root-mean-square-error and coefficient of determination in testing mode, respectively, between the original and emulated 10,000 test data points.}\n\t\\begin{center}\n\t\t\\begin{tabular}{cccccccc}\n\t\t\t\\hline\n\t\t\t& & & & & \\\\\n\t\t\tML & TR & RMSE - Ca$^{conc}$ & $Q_2$ - Ca$^{conc}$ & RMSE - Si$^{conc}$ & $Q_2$ - Si$^{conc}$\\\\\n\t\t\tDNN & 4 $\\times$ 10$^{5}$ & 4.50 $\\times$ 10$^{-5}$ & 0.9999 & 1.21 $\\times$ 10$^{-6}$ & 0.9999\\\\\n\t\t\tDNN & 1 $\\times$ 10$^{5}$ & 5.06 $\\times$ 10$^{-5}$ & 0.9999 & 1.30 $\\times$ 10$^{-6}$ & 0.9999\\\\\n\t\t\tDNN & 1 $\\times$ 10$^{4}$ & 4.57 $\\times$ 10$^{-5}$ & 0.9999 & 1.48 $\\times$ 10$^{-6}$ & 0.9999\\\\\n\t\t\tDNN & 1 $\\times$ 10$^{3}$ & 8.02 $\\times$ 10$^{-5}$ & 0.9998 & 3.31 $\\times$ 10$^{-6}$ & 0.9994\\\\\n\t\t\tDNN & 1 $\\times$ 10$^{2}$ & 47.1 $\\times$ 10$^{-5}$ & 0.9935 & 9.02 $\\times$ 10$^{-6}$ & 0.9959\\\\\n\t\t\tkNN & 4 $\\times$ 10$^{5}$ & 2.52 $\\times$ 10$^{-5}$ & 1.0000 & 1.59 $\\times$ 10$^{-6}$ & 0.9999\\\\\n\t\t\tkNN & 1 $\\times$ 10$^{5}$ & 6.27 $\\times$ 10$^{-5}$ & 0.9999 & 4.20 $\\times$ 10$^{-6}$ & 0.9991\\\\\n\t\t\tkNN & 1 $\\times$ 10$^{4}$ & 14.1 $\\times$ 10$^{-5}$ & 0.9994 & 13.5 $\\times$ 10$^{-6}$ & 0.9908\\\\\n\t\t\tkNN & 1 $\\times$ 10$^{3}$ & 43.9 $\\times$ 10$^{-5}$ & 0.9944 & 74.6 $\\times$ 10$^{-6}$ & 0.7180\\\\\n\t\t\tkNN & 1 $\\times$ 10$^{2}$  & 96.4 $\\times$ 10$^{-5}$ & 0.9729 & 55.0 $\\times$ 10$^{-6}$ & 0.8468\\\\\n\t\t\\end{tabular}\n\t\\end{center}\n\t\\label{table2}\n\\end{table}\n\n\\FloatBarrier\n\n\\subsubsection{Reactive transport simulation}\n\\label{prob12Dres}\n\nThis section focuses on reactive transport simulations with HPx$_{\\rm py}$ within cement system 1, under both advective-dispersive and diffusive transport conditions. As written above, the domain sizes are both 61 $\\times$ 61 and 121 $\\times$ 121 for the advection-dispersion case and, because of computational constraints, solely 61 $\\times$ 61  for the diffusion case. In addition, the simulation time period is 2 years for the advection-dispersion case and 1 year for the diffusion case. Figures \\ref{fig3} and \\ref{fig4} present times series of original and emulated Ca, Si, H and O concentrations at 5 locations within the 2D domain for advective-dispersive transport conditions, for both our kNN-based (HPx$_{\\rm py}$-kNN) and DNN-based (HPx$_{\\rm py}$-DNN) reactive transport codes. It is seen that HPx$_{\\rm py}$-kNN and HPx$_{\\rm py}$-DNN both induce a quite good simulation accuracy. Also, the results for the diffusive transport case are of similarly good quality (not shown). Figures \\ref{fig5} - \\ref{fig6} provide more insights into the HPx$_{\\rm py}$-kNN and HPx$_{\\rm py}$-DNN performances by displaying 2D Ca, Si, H and O concentration profiles at a given time. For each experiment and chemical component, this time is selected as to be well representative of the simulated dynamics. It is observed that the original and emulated images are visually almost indistinguishable for the advection-dispersion case (Figure \\ref{fig5}). For the diffusion case, the emulators also perform quite well for Ca, H and O (Figures\\ref{fig6}a - c, g - i and j - l), while some slight to moderate discrepancies appear at the concentration front for Si (Figures \\ref{fig6}d - f). Notwithstanding, the Si concentration remains globally well predicted. Furthermore, Figures \\ref{fig7} - \\ref{fig8} present the original and emulated 2D solid amount profiles corresponding to Figures \\ref{fig5} - \\ref{fig6}. The original solid amount profiles are overall well approximated by HPx$_{\\rm py}$-kNN and HPx$_{\\rm py}$-DNN for the advection-dispersion case (Figure \\ref{fig7}), even though some mismatch appears at the border of the fully depleted zone for the H component. As of the diffusion case (Figure \\ref{fig8}), the same kind of mismatch is observed for the emulated solid amounts of H by HPx$_{\\rm py}$-kNN while the emulated profiles by HPx$_{\\rm py}$-DNN show some more discrepancies. Though we decided to present raw emulation results, we would like to stress that some if not all of the observed artifacts could likely be smoothed out by using some post-filtering such as median filtering.\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure3.png}\n\t\\caption{Time series of original (RTM, solid green lines) and HPx$_{\\rm py}$-kNN emulated (TM+kNN, dashed orange lines) concentrations (mol/kg) of Ca, H, O and Si at selected observations points for cement system 1 and advective-dispersive transport. Obs. 1 - 5 denote the selected observation points, with the following $\\left[x,y\\right]$ locations (in cm). Obs. 1:  $\\left[0.5, 2.5\\right]$, Obs. 2: $\\left[1, 2\\right]$, Obs. 3: $\\left[2, 2\\right]$, Obs. 4: $\\left[1, 1\\right]$, Obs. 5: $\\left[2, 1\\right]$. The results for the 121 $\\times$ 121 grid size are rather similar.}\n\t\\label{fig3}\n\\end{figure}\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure4.png}\n\t\\caption{Time series of original (RTM, solid green lines) and HPx$_{\\rm py}$-DNN emulated (TM+DNN, dashed orange lines) concentrations (mol/kg) of Ca, H, O and Si at selected observations points for cement system 1 and advective-dispersive transport. Obs. 1 - 5 denote the selected observation points, with the following $\\left[x,y\\right]$ locations (in cm). Obs. 1:  $\\left[0.5, 2.5\\right]$, Obs. 2: $\\left[1, 2\\right]$, Obs. 3: $\\left[2, 2\\right]$, Obs. 4: $\\left[1, 1\\right]$, Obs. 5: $\\left[2, 1\\right]$. The results for the 121 $\\times$ 121 grid size are rather similar.}\n\t\\label{fig4}\n\\end{figure}\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure5.png}\n\t\\caption{2D concentration profiles obtained for cement system 1 at the end of the 2-year simulation performed for the advection-dispersion case. RTM means the original HP$_{\\rm{4C}}$ model, TM+kNN denotes the Hydrus transport model coupled with our kNN geochemical emulator (HPx$_{\\rm py}$-kNN) and TM+DNN signifies the Hydrus transport model coupled with our DNN geochemical emulator (HPx$_{\\rm py}$-DNN). The first to fourth row present profiles for $Ca^{\\rm{conc}}$, $Si^{\\rm{conc}}$, $H^{\\rm{conc}}$, and $O^{\\rm{conc}}$, respectively. The considered grid size is 61 $\\times$ 61. The results for the 121 $\\times$ 121 grid are rather similar.}\n\t\\label{fig5}\n\\end{figure}\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure6.png}\n\t\\caption{2D concentration profiles obtained for cement system 1 at the final time step of the 1-year simulation performed for the diffusion case. RTM means the original HP$_{\\rm{4C}}$ model, TM+kNN denotes the Hydrus transport model coupled with our kNN geochemical emulator (HPx$_{\\rm py}$-kNN) and TM+DNN signifies the Hydrus transport model coupled with our DNN geochemical emulator (HPx$_{\\rm py}$-DNN). The first to fourth row present profiles for $Ca^{\\rm{conc}}$, $Si^{\\rm{conc}}$, $H^{\\rm{conc}}$, and $O^{\\rm{conc}}$, respectively. The considered grid size is 61 $\\times$ 61.}\n\t\\label{fig6}\n\\end{figure}\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure7.png}\n\t\\caption{2D solid amount profiles obtained for cement system 1 at the end of the 2-year simulation performed for the advection-dispersion case. RTM means the original HP$_{\\rm{4C}}$ model, TM+kNN denotes the Hydrus transport model coupled with our kNN geochemical emulator (HPx$_{\\rm py}$-kNN) and TM+DNN signifies the Hydrus transport model coupled with our DNN geochemical emulator (HPx$_{\\rm py}$-DNN). The first to fourth row present profiles for $Ca^{\\rm{solid}}$, $Si^{\\rm{solid}}$, $H^{\\rm{solid}}$, and $O^{\\rm{solid}}$, respectively. The considered grid size is 61 $\\times$ 61. The results for the 121 $\\times$ 121 grid are rather similar.}\n\t\\label{fig7}\n\\end{figure}\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure8.png}\n\t\\caption{2D solid amount profiles obtained for cement system 1 at the final time step of the 1-year simulation performed for the diffusion case. RTM means the original HP$_{\\rm{4C}}$ model, TM+kNN denotes the Hydrus transport model coupled with our kNN geochemical emulator (HPx$_{\\rm py}$-kNN) and TM+DNN signifies the Hydrus transport model coupled with our DNN geochemical emulator (HPx$_{\\rm py}$-DNN). The first to fourth row present profiles for $Ca^{\\rm{solid}}$, $Si^{\\rm{solid}}$, $H^{\\rm{solid}}$, and $O^{\\rm{solid}}$, respectively. The considered grid size is 61 $\\times$ 61.}\n\t\\label{fig8}\n\\end{figure}\n\nThe speedups associated with the considered problem are detailed in Table \\ref{table3}. It is noted that the GPU-based DNN emulator allows for a speedup that is close to optimal. Indeed, The DNN speedups overall represent 85 \\% to 95 \\% of the maximum possible speedups (that is, speedups that would be obtained if the geochemcical calculations would come at no cost at all). The speedups associated with single-threaded kNN remain substantial but only amount to 57 \\% - 65 \\% of the corresponding maximum speedups. As detailed in section \\ref{train_res1}, the used kNN and DNN implementations are found to be respectively 300 and 4000 faster than single-threaded PHREEQC when predicting 10,000 points all one once for this geochemical system. Based on these numbers one could have expected the achieved speedups to represent say 90 \\% (kNN) or 99 \\% (DNN) of the maximum possible ones. A large part of the gaps between achieved and maximum possible speedups is thus likely caused by the time required for communicating and exchanging data between the main C/C++ code and the Python-based emulators.\n\n\\begin{table}[h!]\n\t\\caption{Speedups offered by the KNN and DNN emulators in HPx$_{\\rm py}$ for the reactive transport simulations considered for cement system 1. The HPx$_{\\rm{4C}}$ calculations involve the parallelization of PHREEQC over our 4 CPUs. The HPx$_{\\rm{1C}}$ calculations are performed on a single CPU. The kNN predictions are performed on a single CPU using the scikit-learn implementation while the DNN predictions make use of our GPU. ML signifies the used machine learning method for emulation, TC denotes transport conditions (ADV: advection-dispersion, DIF: diffusion) and GS is the grid size. The maximum possible speedups associated with HPx$_{\\rm{4C}}$ and HPx$_{\\rm{1C}}$,  Max SP HPx$_{\\rm{4C}}$ and  Max SP HPx$_{\\rm{1C}}$, correspond to an hypothetical situation where the geochemical calculations incur zero computational cost.}\n\n\t\\begin{center}\n\t\t\\begin{tabular}{cccccccc}\n\t\t\t\\hline\n\t\t\t& & & & & & & \\\\\n\t\t\tML & TC & GS & HPx$_{\\rm{4C}}$ time (s) & SP HPx$_{\\rm{4C}}$ & Max SP HPx$_{\\rm{4C}}$ & SP HPx$_{\\rm{1C}}$ & Max SP HPx$_{\\rm{1C}}$ \\\\\n\t\t\tDNN & ADV & 61 $\\times$ 61 & 3189 & 6.8 & 7.7 & 24.5 & 28.5 \\\\\n\t\t\tkNN & ADV & 61 $\\times$ 61 & 3189 & 5.0 & 7.7 & 18.0 & 28.5 \\\\\n\t\t\tDNN & ADV & 121 $\\times$ 121 & 23,337 & 5.0 & 5.2 & 17.0 & 18.0 \\\\\n\t\t\tkNN & ADV & 121 $\\times$ 121 & 23,337 & 3.4 & 5.2 & 11.6 & 18.0 \\\\\n\t\t\tDNN & DIF & 61 $\\times$ 61 & 25,448 & 4.2 & 4.9 & 16.2 & 19.1 \\\\\n\t\t\tkNN & DIF & 61 $\\times$ 61 & 25,448 & 2.8 & 4.9 & 10.8 & 19.1 \\\\\n\t\t\t\\hline\n\t\t\\end{tabular}\n\t\\end{center}\n\n\t\\label{table3}\n\\end{table}\n\n\\FloatBarrier\n\n\\subsection{Al-C-Ca-S-Si Problem}\n\nFor this second cement system, the emulation problem consists of predicting at each time step of the RT simulation the (output) Al, C, Ca, S, Si, H and O aqueous concentrations (mol/kgw) from the (input) total amounts of Al, C, Ca, S and Si (mol). Here we focus on advective-dispersive transport only while similarly as for cement system 1, the considered domain sizes are 61 $\\times$ 61 and 121 $\\times$ 121. Moreover, the simulation time period is set to 6 years. \n\nAs mentioned earlier, for this higher-dimensional problem it is observed that the used scikit-learn kNN implementation becomes prohibitively slow compared to HPx$_{\\rm{4C}}$. We found that to get a good emulation accuracy the kNN training base needs to contain 1,000,000 samples (or more). This training base's size together with a 5-dimensional search space leads to an HPx$_{\\rm py}$-kNN reactive transport simulation time that is comparable to that of HPx$_{\\rm{4C}}$. Therefore, for this second cement system we built a custom kNN regressor around another kNN implementation contained in the FAISS package \\citep{faiss2017}. Our used FAISS variant allows for GPU computing and is much faster than scikit-learn for this cement system, but is slightly less accurate due to the use of an approximate rather than exact nearest neighbor search \\citep[see][for details]{faiss2017}.\n\n\\subsubsection{Training the DNN}\n\\label{train_res2}\n\nBuilding a good training set to perform a kNN search and learn the weights and biases of our DNN turned out to be a complicated task in this case. This because to make useful kNN predictions and/or learn an useful DNN, the training set must be sufficiently representative of the geochemical conditions encountered during the reactive transport simulation one wish to perform with HPx$_{\\rm py}$-kNN and the trained HPx$_{\\rm py}$-DNN. In contrast to cement system 1, creating the training set by sampling the $X$-space with a controlled randomness between predefined lower and upper bounds did not prove successful. We tried that strategy by drawing as much as 4,000,000 5-dimensional $\\textbf{x}$ vectors from the $X$-space using a Sobol low-discrepancy sequence \\citep[][]{Sobol1967, Joe-Kuo2003}. Such low-discrepancy sampling scheme covers the 5-dimensional hypercube more uniformly than LHS. Despite a good performance on the test set (not shown), the resulting DNN accuracy in reactive transport mode was never deemed satisfying. In other words, no satisfying ``global\" or ``universal\" DNN emulator could be devised for this cement system. This is probably caused by the fact for this problem, the input (5 total amounts) and output (7 aqueous concentrations) spaces are quite nonlinearly related and both cover 6 to 10 orders of magnitudes depending on the considered element. Therefore, we resorted to the alternative training strategy detailed below. The latter basically tries to grasp the complex correlations and higher-order dependencies that exist between the elements of $\\textbf{x}$ (total amounts, input space) for a given reactive transport simulation setup, in order to produce a training set that honors these between-input relationships.\n\n\\begin{itemize}\n\t\\item Perform a ``cheap\" full reactive transport simulation under the transport conditions and geochemistry of interest and collect the resulting $\\textbf{x}$ -$\\textbf{y}$  pairs of examples (for the considered grid nodes and time steps). Computational demand controls what domain size and simulation time period can be used for this cheap calculation. We used a modest 16 $\\times$ 16 domain and a simulation time period of 10 years. The associated HPx$_{\\rm{4C}}$ runtimes is 180 s.\n\t\n\t\\item Fit a kernel density estimator (KDE) with a Gaussian kernel to the collected $\\textbf{x}$ vectors (encapsulated in the $\\textbf{X}$ array) and generate a fixed number of new input vectors, $\\textbf{x}_{KDE}$. Then run PHREEQC for the $\\textbf{X}_{KDE}$ set to get the corresponding output set, $\\textbf{Y}_{KDE}$. Now apply the correction for porosity described in section \\ref{train_res1} to the $\\textbf{X}_{KDE}$ set and form the training set by merging the ensemble of $\\textbf{x}$-$\\textbf{y}$ pairs with that of the $\\textbf{x}_{KDE}$-$\\textbf{y}_{KDE}$ pairs. The number of produced unique examples by the considered cheap HPx$_{\\rm{4C}}$ simulations varied between 10,000 and 50,000. The KDE-based enrichment of this dataset was deemed necessary to provide more input variability, thereby avoiding overfitting of the trained DNN and improving the kNN accuracy, while still honoring the complex between-input relationships. The number of KDE-generated samples was set as to obtain a total training set size of 1,000,000 examples. A key component of the approach is the bandwith parameter of the Gaussian KDE kernel which controls how much the KDE-generated samples depart from the original ensemble. After limited trial and error, we fixed the kernel bandwith to 0.0025 for the considered case studies.\n\t\n\\end{itemize}\n\nThe scatter plots in Figure \\ref{fig9} illustrate our training set creation procedure. The orange dots depict the pairwise relationships observed between the 5 elements of $\\textbf{x}$ in the cheap simulation. The red and cyan dots in Figure \\ref{fig9} represent the KDE-generated samples with the selected bandwidth, before and after applying the correction for porosity, respectively. Training of the DNN is achieved using the ensemble of original and KDE-corrected input points. We refer to this kind of dataset as RT-based, since it is based on an albeit cheap, full RT simulation. Furthermore, we refer to the obtained DNN and kNN emulators as ``local\" emulators, since as opposed to the emulators constructed for cement system 1, the current emulators are only valid for the input conditions encapsulated in the RT-based training set.\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1.5cm}\\includegraphics[width=50pc]{Figure9.png}\n\t\\caption{Scatter plots of the complex relationships between the five considered inputs for (1) the computationally cheap RT simulation of cement system 2 (orange dots) performed with the original HPx code, (2) the corresponding sampled points by kernel density estimation (KDE) using the selected bandwith (turquoise dots) and, (3) the same sampled KDE points after porosity correction (red dots).}\n\t\\label{fig9}\n\\end{figure}\n\nTraining performance of the DNN emulator is presented in Figure \\ref{fig10}. Similarly as for cement system 1, about 90\\% of the available data was used for the actual training of the DNN while the remaining 10\\% were used a validation set to contol overfitting. Lastly, performance is evaluated for both the trained DNN and kNN emulators using an independent test set of 10,000 examples. Overall, the accuracy of our ``local \" DNN emulator for this RT-inspired dataset is rather large with $Q_2$ values always greater or equal than 0.998. Training performance of the corresponding local kNN emulator is equally good (not shown). Regarding speedup, for this problem single-threaded PHREEQC achieves about 210 geochemical calculations per second on our used Intel\\textsuperscript{\\textregistered} i7 CPU while the GPU-based DNN and kNN emulators are both about 3000 times faster when predicting the 10,000 test points at once .\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure10.png}\n\t\\caption{1-1 plots of local DNN emulation performance obtained for system 2 when the local DNN is trained using 1,000,000 samples. The $x$-axis and $y$-axis present the original and emulated 10,000 independent test data points, respectively. The RMSE and $Q_2$ coefficient denote the root-mean-square-error and coefficient of determination in testing mode, respectively, between the original and emulated 10,000 test data points.}\n\t\\label{fig10}\n\\end{figure}\n\n\\subsubsection{Reactive transport simulation}\n\\label{prob2res}\nOur ``local\" DNN performs rather well when applied to the 61 $\\times$ 61 grid size and a time period of 6 years (Figures \\ref{fig11} - \\ref{fig12}). This for all components but C, for which some localized deviations appear between original and emulated 2D concentration profiles towards the end of the simulation period (Figure \\ref{fig12}). When applied to the 121 $\\times$ 121 grid, HPx$_{\\rm py}$-DNN produces additional discrepancies for O, Si and Al towards the end of the simulation period (Figures \\ref{fig13} - \\ref{fig14}). Yet most of the observed artifacts could probably be smoothed out by using post-filtering. The associated speedups are listed in Table \\ref{table4}. These speedups are larger to those obtained for cement system 1, with values between 8 and 9 when evaluated against HPx$_{\\rm{4C}}$. These speedups represent about 85 \\% to 90 \\% of the maximum possible speedup (Table \\ref{table3}). Overall, these findings indicate that for the considered problem, our RT-based training of a local DNN only works is the training set is sufficiently representative of the particular geochemical conditions encountered in the computationally demanding simulations, which is arguably not easy to achieve. This limitation is further discussed in section \\ref{discussion}.\n\nWe note a more uniform behavior for HPx$_{\\rm py}$-kNN across grid sizes than for HPx$_{\\rm py}$-DNN. Here the results for the 121  $\\times$ 121 grid are only slightly less accurate than those associated with the 61 $ \\times$ 61 grid (see Figures \\ref{fig15} - \\ref{fig16} where for brevity we only show concentration profiles for the C Al and S components). Furthermore, whenever observed the discrepancies between original and emulated profiles are more regularly scattered than for HPx$_{\\rm py}$-DNN. Note also that herein too, post-filtering could likely smooth out a large part of these deviations. In addition, owing to the use of a GPU to achieve the kNN calculations, the speedup provided by HPx$_{\\rm py}$-kNN are as large as those provided by HPx$_{\\rm py}$-DNN (Table \\ref{table4}).\n\nWith respect to the emulated solid amounts, the HPx$_{\\rm py}$-DNN results look visually good for the the 61 $\\times$ 61 grid. This is shown in Figure \\ref{fig17} for the C, Al and S chemical components, while emulation of the H, O Ca and Si chemical components is globally of similar quality (not shown). Nevertheless, significant deviations appear towards the end of the simulation period for every chemical component (see Figure \\ref{fig18} for the C, Al and S chemical components, emulation of the H, O Ca and Si chemical components shows the same level of mismatch). The HPx$_{\\rm py}$-kNN predictions are also fairly accurate for the 61 $\\times$ 61 (see Figure \\ref{fig19} for the C, Al and S chemical components, emulation of the H, O Ca and Si chemical components exhibits a globally similar quality) while some discrepancies show up at the end of the simulation (Figure \\ref{fig20}). However, the mismatch is less pronounced than for HPx$_{\\rm py}$-DNN. Overall, HPx$_{\\rm py}$-kNN appears to be somewhat more robust than HPx$_{\\rm py}$-DNN for this cement system, while providing the same (large) speedup.\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure11.png}\n\t\\caption{2D concentration profiles obtained for cement system 2 after 2 and 6 (final time step) years. RTM means the original HPx$_{\\rm{4C}}$ model and TM+DNN denotes the Hydrus transport model coupled with our DNN geochemical emulator (HPx$_{\\rm py}$-DNN). The first to fourth row present profiles for $Ca^{\\rm{conc}}$, $Si^{\\rm{conc}}$, $H^{\\rm{conc}}$, and $O^{\\rm{conc}}$, respectively. The considered grid size is 61 $\\times$ 61.}\n\t\\label{fig11}\n\\end{figure}\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure12.png}\n\t\\caption{2D concentration profiles obtained for cement system 2 after 2 and 6 (final time step) years. RTM means the original HPx$_{\\rm{4C}}$ model and TM+DNN denotes the Hydrus transport model coupled with our DNN geochemical emulator (HPx$_{\\rm py}$-DNN). The first to third row present profiles for $C^{\\rm{conc}}$, $Al^{\\rm{conc}}$, and $S^{\\rm{conc}}$, respectively. The considered grid size is 61 $\\times$ 61.}\n\t\\label{fig12}\n\\end{figure}\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure13.png}\n\t\\caption{2D concentration profiles obtained for cement system 2 after 2 and 6 (final time step) years. RTM means the original HPx$_{\\rm{4C}}$ model and TM+DNN denotes the Hydrus transport model coupled with our DNN geochemical emulator (HPx$_{\\rm py}$-DNN). The first to fourth row present profiles for $Ca^{\\rm{conc}}$, $Si^{\\rm{conc}}$, $H^{\\rm{conc}}$, and $O^{\\rm{conc}}$, respectively. The considered grid size is 121 $\\times$ 121.}\n\t\\label{fig13}\n\\end{figure}\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure14.png}\n\t\\caption{2D concentration profiles obtained for cement system 2 after 2 and 6 (final time step) years. RTM means the original HPx$_{\\rm{4C}}$ model and TM+DNN denotes the Hydrus transport model coupled with our DNN geochemical emulator (HPx$_{\\rm py}$-DNN). The first to third row present profiles for $C^{\\rm{conc}}$, $Al^{\\rm{conc}}$, and $S^{\\rm{conc}}$, respectively. The considered grid size is 121 $\\times$ 121.}\n\t\\label{fig14}\n\\end{figure}\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure15.png}\n\t\\caption{2D concentration profiles obtained for cement system 2 after 2 and 6 (final time step) years. RTM means the original HPx$_{\\rm{4C}}$ model and TM+kNN denotes the Hydrus transport model coupled with our kNN geochemical emulator (HPx$_{\\rm py}$-kNN).  The first to third row present profiles for $C^{\\rm{conc}}$, $Al^{\\rm{conc}}$, and $S^{\\rm{conc}}$, respectively. The considered grid size is 61 $\\times$ 61.}\n\t\\label{fig15}\n\\end{figure}\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure16.png}\n\t\\caption{2D concentration profiles obtained for cement system 2 after 2 and 6 (final time step) years. RTM means the original HPx$_{\\rm{4C}}$ model and TM+kNN denotes the Hydrus transport model coupled with our kNN geochemical emulator (HPx$_{\\rm py}$-kNN). The first to third row present profiles for $C^{\\rm{conc}}$, $Al^{\\rm{conc}}$, and $S^{\\rm{conc}}$, respectively. The considered grid size is 121 $\\times$ 121.}\n\t\\label{fig16}\n\\end{figure}\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure17.png}\n\t\\caption{2D solid amount profiles of the C, Al and S chemical components of cement system 2 after 2 and 6 (final time step) years. RTM means the original HPx$_{\\rm{4C}}$ model and TM+DNN denotes the Hydrus transport model coupled with our DNN geochemical emulator (HPx$_{\\rm py}$-DNN). The first to third row present profiles for $C^{\\rm{solid}}$, $Al^{\\rm{solid}}$, and $S^{\\rm{solid}}$, respectively The considered grid size is 61 $\\times$ 61.}\n\t\\label{fig17}\n\\end{figure}\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure18.png}\n\t\\caption{2D solid amount profiles of the C, Al and S chemical components of cement system 2 after 2 and 6 (final time step) years. RTM means the original HPx$_{\\rm{4C}}$ model and TM+DNN denotes the Hydrus transport model coupled with our DNN geochemical emulator (HPx$_{\\rm py}$-DNN). The first to third row present profiles for $C^{\\rm{solid}}$, $Al^{\\rm{solid}}$, and $S^{\\rm{solid}}$, respectively The considered grid size is 121 $\\times$ 121.}\n\t\\label{fig18}\n\\end{figure}\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure19.png}\n\t\\caption{2D solid amount profiles of the C, Al and S chemical components of cement system 2 after 2 and 6 (final time step) years. RTM means the original HPx$_{\\rm{4C}}$ model and TM+kNN denotes the Hydrus transport model coupled with our kNN geochemical emulator (HPx$_{\\rm py}$-kNN). The first to third row present profiles for $C^{\\rm{solid}}$, $Al^{\\rm{solid}}$, and $S^{\\rm{solid}}$, respectively The considered grid size is 61 $\\times$ 61.}\n\t\\label{fig19}\n\\end{figure}\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure20.png}\n\t\\caption{2D solid amount profiles of the C, Al and S chemical components of cement system 2 after 2 and 6 (final time step) years. RTM means the original HPx$_{\\rm{4C}}$ model and TM+kNN denotes the Hydrus transport model coupled with our kNN geochemical emulator (HPx$_{\\rm py}$-kNN). The first to third row present profiles for $C^{\\rm{solid}}$, $Al^{\\rm{solid}}$, and $S^{\\rm{solid}}$, respectively The considered grid size is 121 $\\times$ 121.}\n\t\\label{fig20}\n\\end{figure}\n\n\\begin{table}[h!]\n\t\\caption{Speedups offered by the KNN and DNN emulators in HPx$_{\\rm py}$ for the reactive transport simulations considered for cement system 2. The HPx$_{\\rm{4C}}$ calculations involve the parallelization of PHREEQC over our 4 CPUs. The HPx$_{\\rm{1C}}$ calculations are performed on a single CPU. Both the kNN and DNN predictions make use of a GPU. ML signifies the used machine learning method for emulation, BC denotes the type of flow boundary conditions and GS is the grid size. The maximum possible speedups associated with HPx$_{\\rm{4C}}$ and HP$_{\\rm{1C}}$,  Max SP HPx$_{\\rm{4C}}$ and  Max SP HPx$_{\\rm{1C}}$, correspond to an hypothetical situation where the geochemical calculations incur zero computational cost.}\n\n\t\\begin{center}\n\t\t\\begin{tabular}{cccccccc}\n\t\t\t\\hline\n\t\t\t& & & & & & & \\\\\n\t\t\tML & BC & GS & HPx$_{\\rm{4C}}$ time (s) & SP HPx$_{\\rm{4C}}$ & Max SP HPx$_{\\rm{4C}}$ & SP HPx$_{\\rm{1C}}$ & Max SP HPx$_{\\rm{1C}}$ \\\\\n\t\t\tDNN & ADV & 61 $\\times$ 61 & 21,415 & 8.2 & 9.0 & 30.3 & 33.1\\\\\n\t\t\tkNN & ADV & 61 $\\times$ 61 & 21,415 & 7.9 & 9.0 & 28.9 & 33.1\\\\\n\t\t\tDNN & ADV & 121 $\\times$ 121 & 199,841 & 8.2 & 9.5 & 29.9 & 32.8\\\\\n\t\t\tkNN & ADV & 121 $\\times$ 121 & 199,841 & 8.5 & 9.5 & 31.3 & 32.8\\\\\n\t\t\t\\hline\n\t\t\\end{tabular}\n\t\\end{center}\n\n\t\\label{table4}\n\\end{table}\n\n\\FloatBarrier\n\n3.2 Al-C-Ca-S-Si Problem\n\\subsection{Al-C-Ca-S-Si Problem}\n\nFor this second cement system, the emulation problem consists of predicting at each time step of the RT simulation the (output) Al, C, Ca, S, Si, H and O aqueous concentrations (mol/kgw) from the (input) total amounts of Al, C, Ca, S and Si (mol). Here we focus on advective-dispersive transport only while similarly as for cement system 1, the considered domain sizes are 61 $\\times$ 61 and 121 $\\times$ 121. Moreover, the simulation time period is set to 6 years. \n\nAs mentioned earlier, for this higher-dimensional problem it is observed that the used scikit-learn kNN implementation becomes prohibitively slow compared to HPx$_{\\rm{4C}}$. We found that to get a good emulation accuracy the kNN training base needs to contain 1,000,000 samples (or more). This training base's size together with a 5-dimensional search space leads to an HPx$_{\\rm py}$-kNN reactive transport simulation time that is comparable to that of HPx$_{\\rm{4C}}$. Therefore, for this second cement system we built a custom kNN regressor around another kNN implementation contained in the FAISS package \\citep{faiss2017}. Our used FAISS variant allows for GPU computing and is much faster than scikit-learn for this cement system, but is slightly less accurate due to the use of an approximate rather than exact nearest neighbor search \\citep[see][for details]{faiss2017}.\n\n\\subsubsection{Training the DNN}\n\\label{train_res2}\n\nBuilding a good training set to perform a kNN search and learn the weights and biases of our DNN turned out to be a complicated task in this case. This because to make useful kNN predictions and/or learn an useful DNN, the training set must be sufficiently representative of the geochemical conditions encountered during the reactive transport simulation one wish to perform with HPx$_{\\rm py}$-kNN and the trained HPx$_{\\rm py}$-DNN. In contrast to cement system 1, creating the training set by sampling the $X$-space with a controlled randomness between predefined lower and upper bounds did not prove successful. We tried that strategy by drawing as much as 4,000,000 5-dimensional $\\textbf{x}$ vectors from the $X$-space using a Sobol low-discrepancy sequence \\citep[][]{Sobol1967, Joe-Kuo2003}. Such low-discrepancy sampling scheme covers the 5-dimensional hypercube more uniformly than LHS. Despite a good performance on the test set (not shown), the resulting DNN accuracy in reactive transport mode was never deemed satisfying. In other words, no satisfying ``global\" or ``universal\" DNN emulator could be devised for this cement system. This is probably caused by the fact for this problem, the input (5 total amounts) and output (7 aqueous concentrations) spaces are quite nonlinearly related and both cover 6 to 10 orders of magnitudes depending on the considered element. Therefore, we resorted to the alternative training strategy detailed below. The latter basically tries to grasp the complex correlations and higher-order dependencies that exist between the elements of $\\textbf{x}$ (total amounts, input space) for a given reactive transport simulation setup, in order to produce a training set that honors these between-input relationships.\n\n\\begin{itemize}\n\t\\item Perform a ``cheap\" full reactive transport simulation under the transport conditions and geochemistry of interest and collect the resulting $\\textbf{x}$ -$\\textbf{y}$  pairs of examples (for the considered grid nodes and time steps). Computational demand controls what domain size and simulation time period can be used for this cheap calculation. We used a modest 16 $\\times$ 16 domain and a simulation time period of 10 years. The associated HPx$_{\\rm{4C}}$ runtimes is 180 s.\n\t\n\t\\item Fit a kernel density estimator (KDE) with a Gaussian kernel to the collected $\\textbf{x}$ vectors (encapsulated in the $\\textbf{X}$ array) and generate a fixed number of new input vectors, $\\textbf{x}_{KDE}$. Then run PHREEQC for the $\\textbf{X}_{KDE}$ set to get the corresponding output set, $\\textbf{Y}_{KDE}$. Now apply the correction for porosity described in section \\ref{train_res1} to the $\\textbf{X}_{KDE}$ set and form the training set by merging the ensemble of $\\textbf{x}$-$\\textbf{y}$ pairs with that of the $\\textbf{x}_{KDE}$-$\\textbf{y}_{KDE}$ pairs. The number of produced unique examples by the considered cheap HPx$_{\\rm{4C}}$ simulations varied between 10,000 and 50,000. The KDE-based enrichment of this dataset was deemed necessary to provide more input variability, thereby avoiding overfitting of the trained DNN and improving the kNN accuracy, while still honoring the complex between-input relationships. The number of KDE-generated samples was set as to obtain a total training set size of 1,000,000 examples. A key component of the approach is the bandwith parameter of the Gaussian KDE kernel which controls how much the KDE-generated samples depart from the original ensemble. After limited trial and error, we fixed the kernel bandwith to 0.0025 for the considered case studies.\n\t\n\\end{itemize}\n\nThe scatter plots in Figure \\ref{fig9} illustrate our training set creation procedure. The orange dots depict the pairwise relationships observed between the 5 elements of $\\textbf{x}$ in the cheap simulation. The red and cyan dots in Figure \\ref{fig9} represent the KDE-generated samples with the selected bandwidth, before and after applying the correction for porosity, respectively. Training of the DNN is achieved using the ensemble of original and KDE-corrected input points. We refer to this kind of dataset as RT-based, since it is based on an albeit cheap, full RT simulation. Furthermore, we refer to the obtained DNN and kNN emulators as ``local\" emulators, since as opposed to the emulators constructed for cement system 1, the current emulators are only valid for the input conditions encapsulated in the RT-based training set.\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1.5cm}\\includegraphics[width=50pc]{Figure9.png}\n\t\\caption{Scatter plots of the complex relationships between the five considered inputs for (1) the computationally cheap RT simulation of cement system 2 (orange dots) performed with the original HPx code, (2) the corresponding sampled points by kernel density estimation (KDE) using the selected bandwith (turquoise dots) and, (3) the same sampled KDE points after porosity correction (red dots).}\n\t\\label{fig9}\n\\end{figure}\n\nTraining performance of the DNN emulator is presented in Figure \\ref{fig10}. Similarly as for cement system 1, about 90\\% of the available data was used for the actual training of the DNN while the remaining 10\\% were used a validation set to contol overfitting. Lastly, performance is evaluated for both the trained DNN and kNN emulators using an independent test set of 10,000 examples. Overall, the accuracy of our ``local \" DNN emulator for this RT-inspired dataset is rather large with $Q_2$ values always greater or equal than 0.998. Training performance of the corresponding local kNN emulator is equally good (not shown). Regarding speedup, for this problem single-threaded PHREEQC achieves about 210 geochemical calculations per second on our used Intel\\textsuperscript{\\textregistered} i7 CPU while the GPU-based DNN and kNN emulators are both about 3000 times faster when predicting the 10,000 test points at once .\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure10.png}\n\t\\caption{1-1 plots of local DNN emulation performance obtained for system 2 when the local DNN is trained using 1,000,000 samples. The $x$-axis and $y$-axis present the original and emulated 10,000 independent test data points, respectively. The RMSE and $Q_2$ coefficient denote the root-mean-square-error and coefficient of determination in testing mode, respectively, between the original and emulated 10,000 test data points.}\n\t\\label{fig10}\n\\end{figure}\n\n\\subsubsection{Reactive transport simulation}\n\\label{prob2res}\nOur ``local\" DNN performs rather well when applied to the 61 $\\times$ 61 grid size and a time period of 6 years (Figures \\ref{fig11} - \\ref{fig12}). This for all components but C, for which some localized deviations appear between original and emulated 2D concentration profiles towards the end of the simulation period (Figure \\ref{fig12}). When applied to the 121 $\\times$ 121 grid, HPx$_{\\rm py}$-DNN produces additional discrepancies for O, Si and Al towards the end of the simulation period (Figures \\ref{fig13} - \\ref{fig14}). Yet most of the observed artifacts could probably be smoothed out by using post-filtering. The associated speedups are listed in Table \\ref{table4}. These speedups are larger to those obtained for cement system 1, with values between 8 and 9 when evaluated against HPx$_{\\rm{4C}}$. These speedups represent about 85 \\% to 90 \\% of the maximum possible speedup (Table \\ref{table3}). Overall, these findings indicate that for the considered problem, our RT-based training of a local DNN only works is the training set is sufficiently representative of the particular geochemical conditions encountered in the computationally demanding simulations, which is arguably not easy to achieve. This limitation is further discussed in section \\ref{discussion}.\n\nWe note a more uniform behavior for HPx$_{\\rm py}$-kNN across grid sizes than for HPx$_{\\rm py}$-DNN. Here the results for the 121  $\\times$ 121 grid are only slightly less accurate than those associated with the 61 $ \\times$ 61 grid (see Figures \\ref{fig15} - \\ref{fig16} where for brevity we only show concentration profiles for the C Al and S components). Furthermore, whenever observed the discrepancies between original and emulated profiles are more regularly scattered than for HPx$_{\\rm py}$-DNN. Note also that herein too, post-filtering could likely smooth out a large part of these deviations. In addition, owing to the use of a GPU to achieve the kNN calculations, the speedup provided by HPx$_{\\rm py}$-kNN are as large as those provided by HPx$_{\\rm py}$-DNN (Table \\ref{table4}).\n\nWith respect to the emulated solid amounts, the HPx$_{\\rm py}$-DNN results look visually good for the the 61 $\\times$ 61 grid. This is shown in Figure \\ref{fig17} for the C, Al and S chemical components, while emulation of the H, O Ca and Si chemical components is globally of similar quality (not shown). Nevertheless, significant deviations appear towards the end of the simulation period for every chemical component (see Figure \\ref{fig18} for the C, Al and S chemical components, emulation of the H, O Ca and Si chemical components shows the same level of mismatch). The HPx$_{\\rm py}$-kNN predictions are also fairly accurate for the 61 $\\times$ 61 (see Figure \\ref{fig19} for the C, Al and S chemical components, emulation of the H, O Ca and Si chemical components exhibits a globally similar quality) while some discrepancies show up at the end of the simulation (Figure \\ref{fig20}). However, the mismatch is less pronounced than for HPx$_{\\rm py}$-DNN. Overall, HPx$_{\\rm py}$-kNN appears to be somewhat more robust than HPx$_{\\rm py}$-DNN for this cement system, while providing the same (large) speedup.\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure11.png}\n\t\\caption{2D concentration profiles obtained for cement system 2 after 2 and 6 (final time step) years. RTM means the original HPx$_{\\rm{4C}}$ model and TM+DNN denotes the Hydrus transport model coupled with our DNN geochemical emulator (HPx$_{\\rm py}$-DNN). The first to fourth row present profiles for $Ca^{\\rm{conc}}$, $Si^{\\rm{conc}}$, $H^{\\rm{conc}}$, and $O^{\\rm{conc}}$, respectively. The considered grid size is 61 $\\times$ 61.}\n\t\\label{fig11}\n\\end{figure}\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure12.png}\n\t\\caption{2D concentration profiles obtained for cement system 2 after 2 and 6 (final time step) years. RTM means the original HPx$_{\\rm{4C}}$ model and TM+DNN denotes the Hydrus transport model coupled with our DNN geochemical emulator (HPx$_{\\rm py}$-DNN). The first to third row present profiles for $C^{\\rm{conc}}$, $Al^{\\rm{conc}}$, and $S^{\\rm{conc}}$, respectively. The considered grid size is 61 $\\times$ 61.}\n\t\\label{fig12}\n\\end{figure}\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure13.png}\n\t\\caption{2D concentration profiles obtained for cement system 2 after 2 and 6 (final time step) years. RTM means the original HPx$_{\\rm{4C}}$ model and TM+DNN denotes the Hydrus transport model coupled with our DNN geochemical emulator (HPx$_{\\rm py}$-DNN). The first to fourth row present profiles for $Ca^{\\rm{conc}}$, $Si^{\\rm{conc}}$, $H^{\\rm{conc}}$, and $O^{\\rm{conc}}$, respectively. The considered grid size is 121 $\\times$ 121.}\n\t\\label{fig13}\n\\end{figure}\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure14.png}\n\t\\caption{2D concentration profiles obtained for cement system 2 after 2 and 6 (final time step) years. RTM means the original HPx$_{\\rm{4C}}$ model and TM+DNN denotes the Hydrus transport model coupled with our DNN geochemical emulator (HPx$_{\\rm py}$-DNN). The first to third row present profiles for $C^{\\rm{conc}}$, $Al^{\\rm{conc}}$, and $S^{\\rm{conc}}$, respectively. The considered grid size is 121 $\\times$ 121.}\n\t\\label{fig14}\n\\end{figure}\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure15.png}\n\t\\caption{2D concentration profiles obtained for cement system 2 after 2 and 6 (final time step) years. RTM means the original HPx$_{\\rm{4C}}$ model and TM+kNN denotes the Hydrus transport model coupled with our kNN geochemical emulator (HPx$_{\\rm py}$-kNN).  The first to third row present profiles for $C^{\\rm{conc}}$, $Al^{\\rm{conc}}$, and $S^{\\rm{conc}}$, respectively. The considered grid size is 61 $\\times$ 61.}\n\t\\label{fig15}\n\\end{figure}\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure16.png}\n\t\\caption{2D concentration profiles obtained for cement system 2 after 2 and 6 (final time step) years. RTM means the original HPx$_{\\rm{4C}}$ model and TM+kNN denotes the Hydrus transport model coupled with our kNN geochemical emulator (HPx$_{\\rm py}$-kNN). The first to third row present profiles for $C^{\\rm{conc}}$, $Al^{\\rm{conc}}$, and $S^{\\rm{conc}}$, respectively. The considered grid size is 121 $\\times$ 121.}\n\t\\label{fig16}\n\\end{figure}\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure17.png}\n\t\\caption{2D solid amount profiles of the C, Al and S chemical components of cement system 2 after 2 and 6 (final time step) years. RTM means the original HPx$_{\\rm{4C}}$ model and TM+DNN denotes the Hydrus transport model coupled with our DNN geochemical emulator (HPx$_{\\rm py}$-DNN). The first to third row present profiles for $C^{\\rm{solid}}$, $Al^{\\rm{solid}}$, and $S^{\\rm{solid}}$, respectively The considered grid size is 61 $\\times$ 61.}\n\t\\label{fig17}\n\\end{figure}\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure18.png}\n\t\\caption{2D solid amount profiles of the C, Al and S chemical components of cement system 2 after 2 and 6 (final time step) years. RTM means the original HPx$_{\\rm{4C}}$ model and TM+DNN denotes the Hydrus transport model coupled with our DNN geochemical emulator (HPx$_{\\rm py}$-DNN). The first to third row present profiles for $C^{\\rm{solid}}$, $Al^{\\rm{solid}}$, and $S^{\\rm{solid}}$, respectively The considered grid size is 121 $\\times$ 121.}\n\t\\label{fig18}\n\\end{figure}\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure19.png}\n\t\\caption{2D solid amount profiles of the C, Al and S chemical components of cement system 2 after 2 and 6 (final time step) years. RTM means the original HPx$_{\\rm{4C}}$ model and TM+kNN denotes the Hydrus transport model coupled with our kNN geochemical emulator (HPx$_{\\rm py}$-kNN). The first to third row present profiles for $C^{\\rm{solid}}$, $Al^{\\rm{solid}}$, and $S^{\\rm{solid}}$, respectively The considered grid size is 61 $\\times$ 61.}\n\t\\label{fig19}\n\\end{figure}\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure20.png}\n\t\\caption{2D solid amount profiles of the C, Al and S chemical components of cement system 2 after 2 and 6 (final time step) years. RTM means the original HPx$_{\\rm{4C}}$ model and TM+kNN denotes the Hydrus transport model coupled with our kNN geochemical emulator (HPx$_{\\rm py}$-kNN). The first to third row present profiles for $C^{\\rm{solid}}$, $Al^{\\rm{solid}}$, and $S^{\\rm{solid}}$, respectively The considered grid size is 121 $\\times$ 121.}\n\t\\label{fig20}\n\\end{figure}\n\n\\begin{table}[h!]\n\t\\caption{Speedups offered by the KNN and DNN emulators in HPx$_{\\rm py}$ for the reactive transport simulations considered for cement system 2. The HPx$_{\\rm{4C}}$ calculations involve the parallelization of PHREEQC over our 4 CPUs. The HPx$_{\\rm{1C}}$ calculations are performed on a single CPU. Both the kNN and DNN predictions make use of a GPU. ML signifies the used machine learning method for emulation, BC denotes the type of flow boundary conditions and GS is the grid size. The maximum possible speedups associated with HPx$_{\\rm{4C}}$ and HP$_{\\rm{1C}}$,  Max SP HPx$_{\\rm{4C}}$ and  Max SP HPx$_{\\rm{1C}}$, correspond to an hypothetical situation where the geochemical calculations incur zero computational cost.}\n\n\t\\begin{center}\n\t\t\\begin{tabular}{cccccccc}\n\t\t\t\\hline\n\t\t\t& & & & & & & \\\\\n\t\t\tML & BC & GS & HPx$_{\\rm{4C}}$ time (s) & SP HPx$_{\\rm{4C}}$ & Max SP HPx$_{\\rm{4C}}$ & SP HPx$_{\\rm{1C}}$ & Max SP HPx$_{\\rm{1C}}$ \\\\\n\t\t\tDNN & ADV & 61 $\\times$ 61 & 21,415 & 8.2 & 9.0 & 30.3 & 33.1\\\\\n\t\t\tkNN & ADV & 61 $\\times$ 61 & 21,415 & 7.9 & 9.0 & 28.9 & 33.1\\\\\n\t\t\tDNN & ADV & 121 $\\times$ 121 & 199,841 & 8.2 & 9.5 & 29.9 & 32.8\\\\\n\t\t\tkNN & ADV & 121 $\\times$ 121 & 199,841 & 8.5 & 9.5 & 31.3 & 32.8\\\\\n\t\t\t\\hline\n\t\t\\end{tabular}\n\t\\end{center}\n\n\t\\label{table4}\n\\end{table}\n\n\\FloatBarrier\n\n3.2.1 Training the DNN\n\\subsubsection{Training the DNN}\n\\label{train_res2}\n\nBuilding a good training set to perform a kNN search and learn the weights and biases of our DNN turned out to be a complicated task in this case. This because to make useful kNN predictions and/or learn an useful DNN, the training set must be sufficiently representative of the geochemical conditions encountered during the reactive transport simulation one wish to perform with HPx$_{\\rm py}$-kNN and the trained HPx$_{\\rm py}$-DNN. In contrast to cement system 1, creating the training set by sampling the $X$-space with a controlled randomness between predefined lower and upper bounds did not prove successful. We tried that strategy by drawing as much as 4,000,000 5-dimensional $\\textbf{x}$ vectors from the $X$-space using a Sobol low-discrepancy sequence \\citep[][]{Sobol1967, Joe-Kuo2003}. Such low-discrepancy sampling scheme covers the 5-dimensional hypercube more uniformly than LHS. Despite a good performance on the test set (not shown), the resulting DNN accuracy in reactive transport mode was never deemed satisfying. In other words, no satisfying ``global\" or ``universal\" DNN emulator could be devised for this cement system. This is probably caused by the fact for this problem, the input (5 total amounts) and output (7 aqueous concentrations) spaces are quite nonlinearly related and both cover 6 to 10 orders of magnitudes depending on the considered element. Therefore, we resorted to the alternative training strategy detailed below. The latter basically tries to grasp the complex correlations and higher-order dependencies that exist between the elements of $\\textbf{x}$ (total amounts, input space) for a given reactive transport simulation setup, in order to produce a training set that honors these between-input relationships.\n\n\\begin{itemize}\n\t\\item Perform a ``cheap\" full reactive transport simulation under the transport conditions and geochemistry of interest and collect the resulting $\\textbf{x}$ -$\\textbf{y}$  pairs of examples (for the considered grid nodes and time steps). Computational demand controls what domain size and simulation time period can be used for this cheap calculation. We used a modest 16 $\\times$ 16 domain and a simulation time period of 10 years. The associated HPx$_{\\rm{4C}}$ runtimes is 180 s.\n\t\n\t\\item Fit a kernel density estimator (KDE) with a Gaussian kernel to the collected $\\textbf{x}$ vectors (encapsulated in the $\\textbf{X}$ array) and generate a fixed number of new input vectors, $\\textbf{x}_{KDE}$. Then run PHREEQC for the $\\textbf{X}_{KDE}$ set to get the corresponding output set, $\\textbf{Y}_{KDE}$. Now apply the correction for porosity described in section \\ref{train_res1} to the $\\textbf{X}_{KDE}$ set and form the training set by merging the ensemble of $\\textbf{x}$-$\\textbf{y}$ pairs with that of the $\\textbf{x}_{KDE}$-$\\textbf{y}_{KDE}$ pairs. The number of produced unique examples by the considered cheap HPx$_{\\rm{4C}}$ simulations varied between 10,000 and 50,000. The KDE-based enrichment of this dataset was deemed necessary to provide more input variability, thereby avoiding overfitting of the trained DNN and improving the kNN accuracy, while still honoring the complex between-input relationships. The number of KDE-generated samples was set as to obtain a total training set size of 1,000,000 examples. A key component of the approach is the bandwith parameter of the Gaussian KDE kernel which controls how much the KDE-generated samples depart from the original ensemble. After limited trial and error, we fixed the kernel bandwith to 0.0025 for the considered case studies.\n\t\n\\end{itemize}\n\nThe scatter plots in Figure \\ref{fig9} illustrate our training set creation procedure. The orange dots depict the pairwise relationships observed between the 5 elements of $\\textbf{x}$ in the cheap simulation. The red and cyan dots in Figure \\ref{fig9} represent the KDE-generated samples with the selected bandwidth, before and after applying the correction for porosity, respectively. Training of the DNN is achieved using the ensemble of original and KDE-corrected input points. We refer to this kind of dataset as RT-based, since it is based on an albeit cheap, full RT simulation. Furthermore, we refer to the obtained DNN and kNN emulators as ``local\" emulators, since as opposed to the emulators constructed for cement system 1, the current emulators are only valid for the input conditions encapsulated in the RT-based training set.\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1.5cm}\\includegraphics[width=50pc]{Figure9.png}\n\t\\caption{Scatter plots of the complex relationships between the five considered inputs for (1) the computationally cheap RT simulation of cement system 2 (orange dots) performed with the original HPx code, (2) the corresponding sampled points by kernel density estimation (KDE) using the selected bandwith (turquoise dots) and, (3) the same sampled KDE points after porosity correction (red dots).}\n\t\\label{fig9}\n\\end{figure}\n\nTraining performance of the DNN emulator is presented in Figure \\ref{fig10}. Similarly as for cement system 1, about 90\\% of the available data was used for the actual training of the DNN while the remaining 10\\% were used a validation set to contol overfitting. Lastly, performance is evaluated for both the trained DNN and kNN emulators using an independent test set of 10,000 examples. Overall, the accuracy of our ``local \" DNN emulator for this RT-inspired dataset is rather large with $Q_2$ values always greater or equal than 0.998. Training performance of the corresponding local kNN emulator is equally good (not shown). Regarding speedup, for this problem single-threaded PHREEQC achieves about 210 geochemical calculations per second on our used Intel\\textsuperscript{\\textregistered} i7 CPU while the GPU-based DNN and kNN emulators are both about 3000 times faster when predicting the 10,000 test points at once .\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure10.png}\n\t\\caption{1-1 plots of local DNN emulation performance obtained for system 2 when the local DNN is trained using 1,000,000 samples. The $x$-axis and $y$-axis present the original and emulated 10,000 independent test data points, respectively. The RMSE and $Q_2$ coefficient denote the root-mean-square-error and coefficient of determination in testing mode, respectively, between the original and emulated 10,000 test data points.}\n\t\\label{fig10}\n\\end{figure}\n\n",
        "label": "fig10",
        "Descriptive_question1": "What does the x-axis represent in figure_10?",
        "Descriptive_question2": "What is the sample size used for training the local DNN in figure_10?",
        "Reasoning_question1": "How does the RMSE value in figure_10 reflect the accuracy of the local DNN emulation for system 2 compared to the original data?",
        "Reasoning_question2": "What can be inferred about the performance consistency of the local DNN in figure_10 across the range of test data points?",
        "Descriptive_answer1": "original test data points",
        "Descriptive_answer2": "1,000,000 samples",
        "Reasoning_answer1": "The RMSE values shown in figure_10 are very low, with values such as 5.42e-05 for Al and 6.80e-05 for Ca, indicating a very close match between the emulated and original data. Low RMSE signifies small average prediction errors, thus reflecting high accuracy of the local DNN emulator for system 2 compared to the original data.",
        "Reasoning_answer2": "The high and consistently large Q2 coefficients (all above 0.998) across all measured elements suggest that the local DNN maintains a strong predictive performance uniformly over the entire range of test data points. The 1-1 scatter plots closely follow the diagonal line, indicating consistent accuracy without systematic bias or degradation over the data range."
    },
    {
        "paper_id": "2107.07598.json",
        "image_id": "figure_11",
        "image_path": [
            "/home/yz979/scratch/chengye/scimolmo/test-images-tables/dataset_imagestables/2107.07598/images/Figure11.png"
        ],
        "caption": "2D concentration profiles obtained for cement system 2 after 2 and 6 (final time step) years. RTM means the original HPx$_{\\rm{4C}}$ model and TM+DNN denotes the Hydrus transport model coupled with our DNN geochemical emulator (HPx$_{\\rm py}$-DNN). The first to fourth row present profiles for $Ca^{\\rm{conc}}$, $Si^{\\rm{conc}}$, $H^{\\rm{conc}}$, and $O^{\\rm{conc}}$, respectively. The considered grid size is 61 $\\times$ 61.",
        "classify": "Chart",
        "section_info": "3 Results\n\\section{Results}\n\\label{results}\n\n\\subsection{Ca-Si Problem}\nFor this first cement system, the emulation problem consists of predicting at each time step of the RT simulation the (output) Ca, Si, H and O aqueous concentrations (mol/kg of water or mol/kgw) from the (input) total amounts of Ca and Si (mol). \n\n\\subsubsection{Training the emulators}\n\\label{train_res1}\nHere the kNN and DNN emulators are firstly trained using a set of 400,000 test examples for both. This training set is obtained by randomly sampling the two-dimensional input space by latin hypercube sampling (LHS) between $\\left[0,0\\right]$ and $\\rm{\\left[Ca^{tot}_{max},Si^{tot}_{max}\\right]}$, and running PHREEQC for each input sample, $\\rm{\\textbf{x}_i = \\left[Ca^{tot}_i, Si^{tot}_i\\right]}$ to get the corresponding output vectors, $\\rm{\\textbf{y}_i = \\left[Ca^{conc}_i,Si^{conc}_i,H^{conc}_i,O^{conc}_i\\right]}$. The upper bounds, $\\rm{Ca^{tot}_{max}}$ and $\\rm{Si^{tot}_{max}}$ are defined based on a cheap full RT simulation with advective-dispersive transport using a small 1D domain of 51 nodes. It is worth noting that the total amounts of $\\textbf{x}_i$, corresponding to the PHREEQC-simulated concentrations, $\\textbf{y}_i$, have to be corrected for the different amount of water between the training set and the transport simulations. Doing so, it turns out that about 20 \\% of the post-corrected $x_i$ values exceed their pre-defined upper bounds and these excessively large values need to be filtered out. Creating the 400,000 training examples thus required about 500,000 PHREEQC runs. As stated earlier, for this problem single-threaded PHREEQC performs about 670 geochemical calculations per second on our used Intel\\textsuperscript{\\textregistered} i7 CPU.\n\nWith respect to kNN, the tuning parameters are the number of neighbors, $k$, the type of distance measure, and the interpolation technique. We simply used the default settings: $k = 5$, euclidean distance and inverse-distance interpolation. Regarding training of the DNN, the 400,000 sample were split between the training set itself (90 \\% of the data) and a validation set (10 \\% of the data). The latter serves to monitor the evolution of the selected mean squared error loss function on samples that are not used for training, thereby detecting potential overfitting. If the validation loss stops decreasing before the fixed number of epochs has been completed, then  training is stopped. Importantly, the emulation is achieved in log-space for both the input, $X$ and output $Y$, domains. This because total amounts and concentrations of the involved components typically cover many orders of magnitudes (up to 10 orders or more). Using a DNN also requires some form of data normalization or standardization. Here both the $\\rm{log\\left(\\textbf{x}_i\\right)}$ and $\\rm{log\\left(\\textbf{y}_i\\right)}$ vectors are standardized around 0 with standard deviation of 1.\n\nFigure \\ref{fig1} illustrates the trained emulators' performance for geochemical predictions using an independent test set that comprises 10,000 test examples. Both kNN (Figures \\ref{fig1}a - d) and DNN (Figures \\ref{fig1}e - h) appear to be rather accurate . DNN also shows a slight degradation for the larger concentration values  (Figures \\ref{fig1}e - h). The latter is likely due to the combination of a small proportion of large concentration values in the training set with the log-transformation that implicitly pushes the DNN to try harder to fit the smaller concentrations during training. Regarding speedup and as written earlier, for this setup the single-threaded kNN method is 300 times faster than single-threaded PHREEQC for predicting the 10,000 concentration vectors all at once. The computational savings allowed by the DNN emulator when ran on our GPU is higher with a speedup as large as 4000 for predicting the same 10,000 concentration vectors all at once. \n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{0cm}\\includegraphics[width=35pc]{Figure1.png}\n\t\\caption{1-1 plots of the kNN (subfigures (a) - (d)) and DNN (subfigures (e) - (h)) emulators' performance obtained for system 1 when the kNN training base contains 400,000 samples and the DNN is trained using the same 400,000 samples. Here ``true\" means the original PHREEQC-simulated data and ``predicted\" denotes the emulated (that is, kNN-simulated and DNN-simulated) data. Hence, the $x$-axis and $y$-axis present the original and emulated 10,000 independent test data points, respectively. The RMSE and $Q_2$ coefficient denote the root-mean-square-error and coefficient of determination in testing mode, respectively, between the original and emulated 10,000 test data points.}\n\t\\label{fig1}\n\\end{figure}\n\nTo test the sensitivity of the emulators' performance to the training set size, training was also performed using reduced training sets comprising 100,000, 10,000, 1000 and 100 samples, respectively. It seen that the DNN performance achieved  when using 10,000 training samples is virtually the same as that obtained when using 400,000 training samples (Figure \\ref{fig2} and Table \\ref{table2}). It is only for training sets smaller than 1000 samples that the DNN performance starts to degrade significantly (Table \\ref{table2}). In contrast, the behavior of kNN appears to markedly decrease as the training set gets smaller (Figure \\ref{fig2} and Table \\ref{table1}).\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{0cm}\\includegraphics[width=35pc]{Figure2.png}\n\t\\caption{1-1 plots of the kNN (subfigures (a) - (d)) and DNN (subfigures (e) - (h)) emulators' performance obtained for system 1 when the kNN training base contains 10,000 samples and the DNN is trained using the same 10,000 samples. Here ``true\" means the original PHREEQC-simulated data and ``predicted\" denotes the emulated (that is, kNN-simulated and DNN-simulated) data. Hence, the $x$-axis and $y$-axis present the original and emulated 10,000 independent test data points, respectively. The RMSE and $Q_2$ coefficient denote the root-mean-square-error and coefficient of determination in testing mode, respectively, between the original and emulated 10,000 test data points.}\n\t\\label{fig2}\n\\end{figure}\n\n\\begin{table}[h!]\n\t\\caption{Performance of the DNN and kNN emulators for cement system 1 and different training set sizes. For brevity, only the results for Ca$^{conc}$ and Si$^{conc}$ are shown. The units are mol per kg of water (mol/kgw). ML refers to the type of emulator, TR signifies the size of the training set, and the RMSE and $Q_2$ coefficient denote the root-mean-square-error and coefficient of determination in testing mode, respectively, between the original and emulated 10,000 test data points.}\n\t\\begin{center}\n\t\t\\begin{tabular}{cccccccc}\n\t\t\t\\hline\n\t\t\t& & & & & \\\\\n\t\t\tML & TR & RMSE - Ca$^{conc}$ & $Q_2$ - Ca$^{conc}$ & RMSE - Si$^{conc}$ & $Q_2$ - Si$^{conc}$\\\\\n\t\t\tDNN & 4 $\\times$ 10$^{5}$ & 4.50 $\\times$ 10$^{-5}$ & 0.9999 & 1.21 $\\times$ 10$^{-6}$ & 0.9999\\\\\n\t\t\tDNN & 1 $\\times$ 10$^{5}$ & 5.06 $\\times$ 10$^{-5}$ & 0.9999 & 1.30 $\\times$ 10$^{-6}$ & 0.9999\\\\\n\t\t\tDNN & 1 $\\times$ 10$^{4}$ & 4.57 $\\times$ 10$^{-5}$ & 0.9999 & 1.48 $\\times$ 10$^{-6}$ & 0.9999\\\\\n\t\t\tDNN & 1 $\\times$ 10$^{3}$ & 8.02 $\\times$ 10$^{-5}$ & 0.9998 & 3.31 $\\times$ 10$^{-6}$ & 0.9994\\\\\n\t\t\tDNN & 1 $\\times$ 10$^{2}$ & 47.1 $\\times$ 10$^{-5}$ & 0.9935 & 9.02 $\\times$ 10$^{-6}$ & 0.9959\\\\\n\t\t\tkNN & 4 $\\times$ 10$^{5}$ & 2.52 $\\times$ 10$^{-5}$ & 1.0000 & 1.59 $\\times$ 10$^{-6}$ & 0.9999\\\\\n\t\t\tkNN & 1 $\\times$ 10$^{5}$ & 6.27 $\\times$ 10$^{-5}$ & 0.9999 & 4.20 $\\times$ 10$^{-6}$ & 0.9991\\\\\n\t\t\tkNN & 1 $\\times$ 10$^{4}$ & 14.1 $\\times$ 10$^{-5}$ & 0.9994 & 13.5 $\\times$ 10$^{-6}$ & 0.9908\\\\\n\t\t\tkNN & 1 $\\times$ 10$^{3}$ & 43.9 $\\times$ 10$^{-5}$ & 0.9944 & 74.6 $\\times$ 10$^{-6}$ & 0.7180\\\\\n\t\t\tkNN & 1 $\\times$ 10$^{2}$  & 96.4 $\\times$ 10$^{-5}$ & 0.9729 & 55.0 $\\times$ 10$^{-6}$ & 0.8468\\\\\n\t\t\\end{tabular}\n\t\\end{center}\n\t\\label{table2}\n\\end{table}\n\n\\FloatBarrier\n\n\\subsubsection{Reactive transport simulation}\n\\label{prob12Dres}\n\nThis section focuses on reactive transport simulations with HPx$_{\\rm py}$ within cement system 1, under both advective-dispersive and diffusive transport conditions. As written above, the domain sizes are both 61 $\\times$ 61 and 121 $\\times$ 121 for the advection-dispersion case and, because of computational constraints, solely 61 $\\times$ 61  for the diffusion case. In addition, the simulation time period is 2 years for the advection-dispersion case and 1 year for the diffusion case. Figures \\ref{fig3} and \\ref{fig4} present times series of original and emulated Ca, Si, H and O concentrations at 5 locations within the 2D domain for advective-dispersive transport conditions, for both our kNN-based (HPx$_{\\rm py}$-kNN) and DNN-based (HPx$_{\\rm py}$-DNN) reactive transport codes. It is seen that HPx$_{\\rm py}$-kNN and HPx$_{\\rm py}$-DNN both induce a quite good simulation accuracy. Also, the results for the diffusive transport case are of similarly good quality (not shown). Figures \\ref{fig5} - \\ref{fig6} provide more insights into the HPx$_{\\rm py}$-kNN and HPx$_{\\rm py}$-DNN performances by displaying 2D Ca, Si, H and O concentration profiles at a given time. For each experiment and chemical component, this time is selected as to be well representative of the simulated dynamics. It is observed that the original and emulated images are visually almost indistinguishable for the advection-dispersion case (Figure \\ref{fig5}). For the diffusion case, the emulators also perform quite well for Ca, H and O (Figures\\ref{fig6}a - c, g - i and j - l), while some slight to moderate discrepancies appear at the concentration front for Si (Figures \\ref{fig6}d - f). Notwithstanding, the Si concentration remains globally well predicted. Furthermore, Figures \\ref{fig7} - \\ref{fig8} present the original and emulated 2D solid amount profiles corresponding to Figures \\ref{fig5} - \\ref{fig6}. The original solid amount profiles are overall well approximated by HPx$_{\\rm py}$-kNN and HPx$_{\\rm py}$-DNN for the advection-dispersion case (Figure \\ref{fig7}), even though some mismatch appears at the border of the fully depleted zone for the H component. As of the diffusion case (Figure \\ref{fig8}), the same kind of mismatch is observed for the emulated solid amounts of H by HPx$_{\\rm py}$-kNN while the emulated profiles by HPx$_{\\rm py}$-DNN show some more discrepancies. Though we decided to present raw emulation results, we would like to stress that some if not all of the observed artifacts could likely be smoothed out by using some post-filtering such as median filtering.\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure3.png}\n\t\\caption{Time series of original (RTM, solid green lines) and HPx$_{\\rm py}$-kNN emulated (TM+kNN, dashed orange lines) concentrations (mol/kg) of Ca, H, O and Si at selected observations points for cement system 1 and advective-dispersive transport. Obs. 1 - 5 denote the selected observation points, with the following $\\left[x,y\\right]$ locations (in cm). Obs. 1:  $\\left[0.5, 2.5\\right]$, Obs. 2: $\\left[1, 2\\right]$, Obs. 3: $\\left[2, 2\\right]$, Obs. 4: $\\left[1, 1\\right]$, Obs. 5: $\\left[2, 1\\right]$. The results for the 121 $\\times$ 121 grid size are rather similar.}\n\t\\label{fig3}\n\\end{figure}\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure4.png}\n\t\\caption{Time series of original (RTM, solid green lines) and HPx$_{\\rm py}$-DNN emulated (TM+DNN, dashed orange lines) concentrations (mol/kg) of Ca, H, O and Si at selected observations points for cement system 1 and advective-dispersive transport. Obs. 1 - 5 denote the selected observation points, with the following $\\left[x,y\\right]$ locations (in cm). Obs. 1:  $\\left[0.5, 2.5\\right]$, Obs. 2: $\\left[1, 2\\right]$, Obs. 3: $\\left[2, 2\\right]$, Obs. 4: $\\left[1, 1\\right]$, Obs. 5: $\\left[2, 1\\right]$. The results for the 121 $\\times$ 121 grid size are rather similar.}\n\t\\label{fig4}\n\\end{figure}\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure5.png}\n\t\\caption{2D concentration profiles obtained for cement system 1 at the end of the 2-year simulation performed for the advection-dispersion case. RTM means the original HP$_{\\rm{4C}}$ model, TM+kNN denotes the Hydrus transport model coupled with our kNN geochemical emulator (HPx$_{\\rm py}$-kNN) and TM+DNN signifies the Hydrus transport model coupled with our DNN geochemical emulator (HPx$_{\\rm py}$-DNN). The first to fourth row present profiles for $Ca^{\\rm{conc}}$, $Si^{\\rm{conc}}$, $H^{\\rm{conc}}$, and $O^{\\rm{conc}}$, respectively. The considered grid size is 61 $\\times$ 61. The results for the 121 $\\times$ 121 grid are rather similar.}\n\t\\label{fig5}\n\\end{figure}\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure6.png}\n\t\\caption{2D concentration profiles obtained for cement system 1 at the final time step of the 1-year simulation performed for the diffusion case. RTM means the original HP$_{\\rm{4C}}$ model, TM+kNN denotes the Hydrus transport model coupled with our kNN geochemical emulator (HPx$_{\\rm py}$-kNN) and TM+DNN signifies the Hydrus transport model coupled with our DNN geochemical emulator (HPx$_{\\rm py}$-DNN). The first to fourth row present profiles for $Ca^{\\rm{conc}}$, $Si^{\\rm{conc}}$, $H^{\\rm{conc}}$, and $O^{\\rm{conc}}$, respectively. The considered grid size is 61 $\\times$ 61.}\n\t\\label{fig6}\n\\end{figure}\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure7.png}\n\t\\caption{2D solid amount profiles obtained for cement system 1 at the end of the 2-year simulation performed for the advection-dispersion case. RTM means the original HP$_{\\rm{4C}}$ model, TM+kNN denotes the Hydrus transport model coupled with our kNN geochemical emulator (HPx$_{\\rm py}$-kNN) and TM+DNN signifies the Hydrus transport model coupled with our DNN geochemical emulator (HPx$_{\\rm py}$-DNN). The first to fourth row present profiles for $Ca^{\\rm{solid}}$, $Si^{\\rm{solid}}$, $H^{\\rm{solid}}$, and $O^{\\rm{solid}}$, respectively. The considered grid size is 61 $\\times$ 61. The results for the 121 $\\times$ 121 grid are rather similar.}\n\t\\label{fig7}\n\\end{figure}\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure8.png}\n\t\\caption{2D solid amount profiles obtained for cement system 1 at the final time step of the 1-year simulation performed for the diffusion case. RTM means the original HP$_{\\rm{4C}}$ model, TM+kNN denotes the Hydrus transport model coupled with our kNN geochemical emulator (HPx$_{\\rm py}$-kNN) and TM+DNN signifies the Hydrus transport model coupled with our DNN geochemical emulator (HPx$_{\\rm py}$-DNN). The first to fourth row present profiles for $Ca^{\\rm{solid}}$, $Si^{\\rm{solid}}$, $H^{\\rm{solid}}$, and $O^{\\rm{solid}}$, respectively. The considered grid size is 61 $\\times$ 61.}\n\t\\label{fig8}\n\\end{figure}\n\nThe speedups associated with the considered problem are detailed in Table \\ref{table3}. It is noted that the GPU-based DNN emulator allows for a speedup that is close to optimal. Indeed, The DNN speedups overall represent 85 \\% to 95 \\% of the maximum possible speedups (that is, speedups that would be obtained if the geochemcical calculations would come at no cost at all). The speedups associated with single-threaded kNN remain substantial but only amount to 57 \\% - 65 \\% of the corresponding maximum speedups. As detailed in section \\ref{train_res1}, the used kNN and DNN implementations are found to be respectively 300 and 4000 faster than single-threaded PHREEQC when predicting 10,000 points all one once for this geochemical system. Based on these numbers one could have expected the achieved speedups to represent say 90 \\% (kNN) or 99 \\% (DNN) of the maximum possible ones. A large part of the gaps between achieved and maximum possible speedups is thus likely caused by the time required for communicating and exchanging data between the main C/C++ code and the Python-based emulators.\n\n\\begin{table}[h!]\n\t\\caption{Speedups offered by the KNN and DNN emulators in HPx$_{\\rm py}$ for the reactive transport simulations considered for cement system 1. The HPx$_{\\rm{4C}}$ calculations involve the parallelization of PHREEQC over our 4 CPUs. The HPx$_{\\rm{1C}}$ calculations are performed on a single CPU. The kNN predictions are performed on a single CPU using the scikit-learn implementation while the DNN predictions make use of our GPU. ML signifies the used machine learning method for emulation, TC denotes transport conditions (ADV: advection-dispersion, DIF: diffusion) and GS is the grid size. The maximum possible speedups associated with HPx$_{\\rm{4C}}$ and HPx$_{\\rm{1C}}$,  Max SP HPx$_{\\rm{4C}}$ and  Max SP HPx$_{\\rm{1C}}$, correspond to an hypothetical situation where the geochemical calculations incur zero computational cost.}\n\n\t\\begin{center}\n\t\t\\begin{tabular}{cccccccc}\n\t\t\t\\hline\n\t\t\t& & & & & & & \\\\\n\t\t\tML & TC & GS & HPx$_{\\rm{4C}}$ time (s) & SP HPx$_{\\rm{4C}}$ & Max SP HPx$_{\\rm{4C}}$ & SP HPx$_{\\rm{1C}}$ & Max SP HPx$_{\\rm{1C}}$ \\\\\n\t\t\tDNN & ADV & 61 $\\times$ 61 & 3189 & 6.8 & 7.7 & 24.5 & 28.5 \\\\\n\t\t\tkNN & ADV & 61 $\\times$ 61 & 3189 & 5.0 & 7.7 & 18.0 & 28.5 \\\\\n\t\t\tDNN & ADV & 121 $\\times$ 121 & 23,337 & 5.0 & 5.2 & 17.0 & 18.0 \\\\\n\t\t\tkNN & ADV & 121 $\\times$ 121 & 23,337 & 3.4 & 5.2 & 11.6 & 18.0 \\\\\n\t\t\tDNN & DIF & 61 $\\times$ 61 & 25,448 & 4.2 & 4.9 & 16.2 & 19.1 \\\\\n\t\t\tkNN & DIF & 61 $\\times$ 61 & 25,448 & 2.8 & 4.9 & 10.8 & 19.1 \\\\\n\t\t\t\\hline\n\t\t\\end{tabular}\n\t\\end{center}\n\n\t\\label{table3}\n\\end{table}\n\n\\FloatBarrier\n\n\\subsection{Al-C-Ca-S-Si Problem}\n\nFor this second cement system, the emulation problem consists of predicting at each time step of the RT simulation the (output) Al, C, Ca, S, Si, H and O aqueous concentrations (mol/kgw) from the (input) total amounts of Al, C, Ca, S and Si (mol). Here we focus on advective-dispersive transport only while similarly as for cement system 1, the considered domain sizes are 61 $\\times$ 61 and 121 $\\times$ 121. Moreover, the simulation time period is set to 6 years. \n\nAs mentioned earlier, for this higher-dimensional problem it is observed that the used scikit-learn kNN implementation becomes prohibitively slow compared to HPx$_{\\rm{4C}}$. We found that to get a good emulation accuracy the kNN training base needs to contain 1,000,000 samples (or more). This training base's size together with a 5-dimensional search space leads to an HPx$_{\\rm py}$-kNN reactive transport simulation time that is comparable to that of HPx$_{\\rm{4C}}$. Therefore, for this second cement system we built a custom kNN regressor around another kNN implementation contained in the FAISS package \\citep{faiss2017}. Our used FAISS variant allows for GPU computing and is much faster than scikit-learn for this cement system, but is slightly less accurate due to the use of an approximate rather than exact nearest neighbor search \\citep[see][for details]{faiss2017}.\n\n\\subsubsection{Training the DNN}\n\\label{train_res2}\n\nBuilding a good training set to perform a kNN search and learn the weights and biases of our DNN turned out to be a complicated task in this case. This because to make useful kNN predictions and/or learn an useful DNN, the training set must be sufficiently representative of the geochemical conditions encountered during the reactive transport simulation one wish to perform with HPx$_{\\rm py}$-kNN and the trained HPx$_{\\rm py}$-DNN. In contrast to cement system 1, creating the training set by sampling the $X$-space with a controlled randomness between predefined lower and upper bounds did not prove successful. We tried that strategy by drawing as much as 4,000,000 5-dimensional $\\textbf{x}$ vectors from the $X$-space using a Sobol low-discrepancy sequence \\citep[][]{Sobol1967, Joe-Kuo2003}. Such low-discrepancy sampling scheme covers the 5-dimensional hypercube more uniformly than LHS. Despite a good performance on the test set (not shown), the resulting DNN accuracy in reactive transport mode was never deemed satisfying. In other words, no satisfying ``global\" or ``universal\" DNN emulator could be devised for this cement system. This is probably caused by the fact for this problem, the input (5 total amounts) and output (7 aqueous concentrations) spaces are quite nonlinearly related and both cover 6 to 10 orders of magnitudes depending on the considered element. Therefore, we resorted to the alternative training strategy detailed below. The latter basically tries to grasp the complex correlations and higher-order dependencies that exist between the elements of $\\textbf{x}$ (total amounts, input space) for a given reactive transport simulation setup, in order to produce a training set that honors these between-input relationships.\n\n\\begin{itemize}\n\t\\item Perform a ``cheap\" full reactive transport simulation under the transport conditions and geochemistry of interest and collect the resulting $\\textbf{x}$ -$\\textbf{y}$  pairs of examples (for the considered grid nodes and time steps). Computational demand controls what domain size and simulation time period can be used for this cheap calculation. We used a modest 16 $\\times$ 16 domain and a simulation time period of 10 years. The associated HPx$_{\\rm{4C}}$ runtimes is 180 s.\n\t\n\t\\item Fit a kernel density estimator (KDE) with a Gaussian kernel to the collected $\\textbf{x}$ vectors (encapsulated in the $\\textbf{X}$ array) and generate a fixed number of new input vectors, $\\textbf{x}_{KDE}$. Then run PHREEQC for the $\\textbf{X}_{KDE}$ set to get the corresponding output set, $\\textbf{Y}_{KDE}$. Now apply the correction for porosity described in section \\ref{train_res1} to the $\\textbf{X}_{KDE}$ set and form the training set by merging the ensemble of $\\textbf{x}$-$\\textbf{y}$ pairs with that of the $\\textbf{x}_{KDE}$-$\\textbf{y}_{KDE}$ pairs. The number of produced unique examples by the considered cheap HPx$_{\\rm{4C}}$ simulations varied between 10,000 and 50,000. The KDE-based enrichment of this dataset was deemed necessary to provide more input variability, thereby avoiding overfitting of the trained DNN and improving the kNN accuracy, while still honoring the complex between-input relationships. The number of KDE-generated samples was set as to obtain a total training set size of 1,000,000 examples. A key component of the approach is the bandwith parameter of the Gaussian KDE kernel which controls how much the KDE-generated samples depart from the original ensemble. After limited trial and error, we fixed the kernel bandwith to 0.0025 for the considered case studies.\n\t\n\\end{itemize}\n\nThe scatter plots in Figure \\ref{fig9} illustrate our training set creation procedure. The orange dots depict the pairwise relationships observed between the 5 elements of $\\textbf{x}$ in the cheap simulation. The red and cyan dots in Figure \\ref{fig9} represent the KDE-generated samples with the selected bandwidth, before and after applying the correction for porosity, respectively. Training of the DNN is achieved using the ensemble of original and KDE-corrected input points. We refer to this kind of dataset as RT-based, since it is based on an albeit cheap, full RT simulation. Furthermore, we refer to the obtained DNN and kNN emulators as ``local\" emulators, since as opposed to the emulators constructed for cement system 1, the current emulators are only valid for the input conditions encapsulated in the RT-based training set.\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1.5cm}\\includegraphics[width=50pc]{Figure9.png}\n\t\\caption{Scatter plots of the complex relationships between the five considered inputs for (1) the computationally cheap RT simulation of cement system 2 (orange dots) performed with the original HPx code, (2) the corresponding sampled points by kernel density estimation (KDE) using the selected bandwith (turquoise dots) and, (3) the same sampled KDE points after porosity correction (red dots).}\n\t\\label{fig9}\n\\end{figure}\n\nTraining performance of the DNN emulator is presented in Figure \\ref{fig10}. Similarly as for cement system 1, about 90\\% of the available data was used for the actual training of the DNN while the remaining 10\\% were used a validation set to contol overfitting. Lastly, performance is evaluated for both the trained DNN and kNN emulators using an independent test set of 10,000 examples. Overall, the accuracy of our ``local \" DNN emulator for this RT-inspired dataset is rather large with $Q_2$ values always greater or equal than 0.998. Training performance of the corresponding local kNN emulator is equally good (not shown). Regarding speedup, for this problem single-threaded PHREEQC achieves about 210 geochemical calculations per second on our used Intel\\textsuperscript{\\textregistered} i7 CPU while the GPU-based DNN and kNN emulators are both about 3000 times faster when predicting the 10,000 test points at once .\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure10.png}\n\t\\caption{1-1 plots of local DNN emulation performance obtained for system 2 when the local DNN is trained using 1,000,000 samples. The $x$-axis and $y$-axis present the original and emulated 10,000 independent test data points, respectively. The RMSE and $Q_2$ coefficient denote the root-mean-square-error and coefficient of determination in testing mode, respectively, between the original and emulated 10,000 test data points.}\n\t\\label{fig10}\n\\end{figure}\n\n\\subsubsection{Reactive transport simulation}\n\\label{prob2res}\nOur ``local\" DNN performs rather well when applied to the 61 $\\times$ 61 grid size and a time period of 6 years (Figures \\ref{fig11} - \\ref{fig12}). This for all components but C, for which some localized deviations appear between original and emulated 2D concentration profiles towards the end of the simulation period (Figure \\ref{fig12}). When applied to the 121 $\\times$ 121 grid, HPx$_{\\rm py}$-DNN produces additional discrepancies for O, Si and Al towards the end of the simulation period (Figures \\ref{fig13} - \\ref{fig14}). Yet most of the observed artifacts could probably be smoothed out by using post-filtering. The associated speedups are listed in Table \\ref{table4}. These speedups are larger to those obtained for cement system 1, with values between 8 and 9 when evaluated against HPx$_{\\rm{4C}}$. These speedups represent about 85 \\% to 90 \\% of the maximum possible speedup (Table \\ref{table3}). Overall, these findings indicate that for the considered problem, our RT-based training of a local DNN only works is the training set is sufficiently representative of the particular geochemical conditions encountered in the computationally demanding simulations, which is arguably not easy to achieve. This limitation is further discussed in section \\ref{discussion}.\n\nWe note a more uniform behavior for HPx$_{\\rm py}$-kNN across grid sizes than for HPx$_{\\rm py}$-DNN. Here the results for the 121  $\\times$ 121 grid are only slightly less accurate than those associated with the 61 $ \\times$ 61 grid (see Figures \\ref{fig15} - \\ref{fig16} where for brevity we only show concentration profiles for the C Al and S components). Furthermore, whenever observed the discrepancies between original and emulated profiles are more regularly scattered than for HPx$_{\\rm py}$-DNN. Note also that herein too, post-filtering could likely smooth out a large part of these deviations. In addition, owing to the use of a GPU to achieve the kNN calculations, the speedup provided by HPx$_{\\rm py}$-kNN are as large as those provided by HPx$_{\\rm py}$-DNN (Table \\ref{table4}).\n\nWith respect to the emulated solid amounts, the HPx$_{\\rm py}$-DNN results look visually good for the the 61 $\\times$ 61 grid. This is shown in Figure \\ref{fig17} for the C, Al and S chemical components, while emulation of the H, O Ca and Si chemical components is globally of similar quality (not shown). Nevertheless, significant deviations appear towards the end of the simulation period for every chemical component (see Figure \\ref{fig18} for the C, Al and S chemical components, emulation of the H, O Ca and Si chemical components shows the same level of mismatch). The HPx$_{\\rm py}$-kNN predictions are also fairly accurate for the 61 $\\times$ 61 (see Figure \\ref{fig19} for the C, Al and S chemical components, emulation of the H, O Ca and Si chemical components exhibits a globally similar quality) while some discrepancies show up at the end of the simulation (Figure \\ref{fig20}). However, the mismatch is less pronounced than for HPx$_{\\rm py}$-DNN. Overall, HPx$_{\\rm py}$-kNN appears to be somewhat more robust than HPx$_{\\rm py}$-DNN for this cement system, while providing the same (large) speedup.\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure11.png}\n\t\\caption{2D concentration profiles obtained for cement system 2 after 2 and 6 (final time step) years. RTM means the original HPx$_{\\rm{4C}}$ model and TM+DNN denotes the Hydrus transport model coupled with our DNN geochemical emulator (HPx$_{\\rm py}$-DNN). The first to fourth row present profiles for $Ca^{\\rm{conc}}$, $Si^{\\rm{conc}}$, $H^{\\rm{conc}}$, and $O^{\\rm{conc}}$, respectively. The considered grid size is 61 $\\times$ 61.}\n\t\\label{fig11}\n\\end{figure}\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure12.png}\n\t\\caption{2D concentration profiles obtained for cement system 2 after 2 and 6 (final time step) years. RTM means the original HPx$_{\\rm{4C}}$ model and TM+DNN denotes the Hydrus transport model coupled with our DNN geochemical emulator (HPx$_{\\rm py}$-DNN). The first to third row present profiles for $C^{\\rm{conc}}$, $Al^{\\rm{conc}}$, and $S^{\\rm{conc}}$, respectively. The considered grid size is 61 $\\times$ 61.}\n\t\\label{fig12}\n\\end{figure}\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure13.png}\n\t\\caption{2D concentration profiles obtained for cement system 2 after 2 and 6 (final time step) years. RTM means the original HPx$_{\\rm{4C}}$ model and TM+DNN denotes the Hydrus transport model coupled with our DNN geochemical emulator (HPx$_{\\rm py}$-DNN). The first to fourth row present profiles for $Ca^{\\rm{conc}}$, $Si^{\\rm{conc}}$, $H^{\\rm{conc}}$, and $O^{\\rm{conc}}$, respectively. The considered grid size is 121 $\\times$ 121.}\n\t\\label{fig13}\n\\end{figure}\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure14.png}\n\t\\caption{2D concentration profiles obtained for cement system 2 after 2 and 6 (final time step) years. RTM means the original HPx$_{\\rm{4C}}$ model and TM+DNN denotes the Hydrus transport model coupled with our DNN geochemical emulator (HPx$_{\\rm py}$-DNN). The first to third row present profiles for $C^{\\rm{conc}}$, $Al^{\\rm{conc}}$, and $S^{\\rm{conc}}$, respectively. The considered grid size is 121 $\\times$ 121.}\n\t\\label{fig14}\n\\end{figure}\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure15.png}\n\t\\caption{2D concentration profiles obtained for cement system 2 after 2 and 6 (final time step) years. RTM means the original HPx$_{\\rm{4C}}$ model and TM+kNN denotes the Hydrus transport model coupled with our kNN geochemical emulator (HPx$_{\\rm py}$-kNN).  The first to third row present profiles for $C^{\\rm{conc}}$, $Al^{\\rm{conc}}$, and $S^{\\rm{conc}}$, respectively. The considered grid size is 61 $\\times$ 61.}\n\t\\label{fig15}\n\\end{figure}\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure16.png}\n\t\\caption{2D concentration profiles obtained for cement system 2 after 2 and 6 (final time step) years. RTM means the original HPx$_{\\rm{4C}}$ model and TM+kNN denotes the Hydrus transport model coupled with our kNN geochemical emulator (HPx$_{\\rm py}$-kNN). The first to third row present profiles for $C^{\\rm{conc}}$, $Al^{\\rm{conc}}$, and $S^{\\rm{conc}}$, respectively. The considered grid size is 121 $\\times$ 121.}\n\t\\label{fig16}\n\\end{figure}\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure17.png}\n\t\\caption{2D solid amount profiles of the C, Al and S chemical components of cement system 2 after 2 and 6 (final time step) years. RTM means the original HPx$_{\\rm{4C}}$ model and TM+DNN denotes the Hydrus transport model coupled with our DNN geochemical emulator (HPx$_{\\rm py}$-DNN). The first to third row present profiles for $C^{\\rm{solid}}$, $Al^{\\rm{solid}}$, and $S^{\\rm{solid}}$, respectively The considered grid size is 61 $\\times$ 61.}\n\t\\label{fig17}\n\\end{figure}\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure18.png}\n\t\\caption{2D solid amount profiles of the C, Al and S chemical components of cement system 2 after 2 and 6 (final time step) years. RTM means the original HPx$_{\\rm{4C}}$ model and TM+DNN denotes the Hydrus transport model coupled with our DNN geochemical emulator (HPx$_{\\rm py}$-DNN). The first to third row present profiles for $C^{\\rm{solid}}$, $Al^{\\rm{solid}}$, and $S^{\\rm{solid}}$, respectively The considered grid size is 121 $\\times$ 121.}\n\t\\label{fig18}\n\\end{figure}\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure19.png}\n\t\\caption{2D solid amount profiles of the C, Al and S chemical components of cement system 2 after 2 and 6 (final time step) years. RTM means the original HPx$_{\\rm{4C}}$ model and TM+kNN denotes the Hydrus transport model coupled with our kNN geochemical emulator (HPx$_{\\rm py}$-kNN). The first to third row present profiles for $C^{\\rm{solid}}$, $Al^{\\rm{solid}}$, and $S^{\\rm{solid}}$, respectively The considered grid size is 61 $\\times$ 61.}\n\t\\label{fig19}\n\\end{figure}\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure20.png}\n\t\\caption{2D solid amount profiles of the C, Al and S chemical components of cement system 2 after 2 and 6 (final time step) years. RTM means the original HPx$_{\\rm{4C}}$ model and TM+kNN denotes the Hydrus transport model coupled with our kNN geochemical emulator (HPx$_{\\rm py}$-kNN). The first to third row present profiles for $C^{\\rm{solid}}$, $Al^{\\rm{solid}}$, and $S^{\\rm{solid}}$, respectively The considered grid size is 121 $\\times$ 121.}\n\t\\label{fig20}\n\\end{figure}\n\n\\begin{table}[h!]\n\t\\caption{Speedups offered by the KNN and DNN emulators in HPx$_{\\rm py}$ for the reactive transport simulations considered for cement system 2. The HPx$_{\\rm{4C}}$ calculations involve the parallelization of PHREEQC over our 4 CPUs. The HPx$_{\\rm{1C}}$ calculations are performed on a single CPU. Both the kNN and DNN predictions make use of a GPU. ML signifies the used machine learning method for emulation, BC denotes the type of flow boundary conditions and GS is the grid size. The maximum possible speedups associated with HPx$_{\\rm{4C}}$ and HP$_{\\rm{1C}}$,  Max SP HPx$_{\\rm{4C}}$ and  Max SP HPx$_{\\rm{1C}}$, correspond to an hypothetical situation where the geochemical calculations incur zero computational cost.}\n\n\t\\begin{center}\n\t\t\\begin{tabular}{cccccccc}\n\t\t\t\\hline\n\t\t\t& & & & & & & \\\\\n\t\t\tML & BC & GS & HPx$_{\\rm{4C}}$ time (s) & SP HPx$_{\\rm{4C}}$ & Max SP HPx$_{\\rm{4C}}$ & SP HPx$_{\\rm{1C}}$ & Max SP HPx$_{\\rm{1C}}$ \\\\\n\t\t\tDNN & ADV & 61 $\\times$ 61 & 21,415 & 8.2 & 9.0 & 30.3 & 33.1\\\\\n\t\t\tkNN & ADV & 61 $\\times$ 61 & 21,415 & 7.9 & 9.0 & 28.9 & 33.1\\\\\n\t\t\tDNN & ADV & 121 $\\times$ 121 & 199,841 & 8.2 & 9.5 & 29.9 & 32.8\\\\\n\t\t\tkNN & ADV & 121 $\\times$ 121 & 199,841 & 8.5 & 9.5 & 31.3 & 32.8\\\\\n\t\t\t\\hline\n\t\t\\end{tabular}\n\t\\end{center}\n\n\t\\label{table4}\n\\end{table}\n\n\\FloatBarrier\n\n3.2 Al-C-Ca-S-Si Problem\n\\subsection{Al-C-Ca-S-Si Problem}\n\nFor this second cement system, the emulation problem consists of predicting at each time step of the RT simulation the (output) Al, C, Ca, S, Si, H and O aqueous concentrations (mol/kgw) from the (input) total amounts of Al, C, Ca, S and Si (mol). Here we focus on advective-dispersive transport only while similarly as for cement system 1, the considered domain sizes are 61 $\\times$ 61 and 121 $\\times$ 121. Moreover, the simulation time period is set to 6 years. \n\nAs mentioned earlier, for this higher-dimensional problem it is observed that the used scikit-learn kNN implementation becomes prohibitively slow compared to HPx$_{\\rm{4C}}$. We found that to get a good emulation accuracy the kNN training base needs to contain 1,000,000 samples (or more). This training base's size together with a 5-dimensional search space leads to an HPx$_{\\rm py}$-kNN reactive transport simulation time that is comparable to that of HPx$_{\\rm{4C}}$. Therefore, for this second cement system we built a custom kNN regressor around another kNN implementation contained in the FAISS package \\citep{faiss2017}. Our used FAISS variant allows for GPU computing and is much faster than scikit-learn for this cement system, but is slightly less accurate due to the use of an approximate rather than exact nearest neighbor search \\citep[see][for details]{faiss2017}.\n\n\\subsubsection{Training the DNN}\n\\label{train_res2}\n\nBuilding a good training set to perform a kNN search and learn the weights and biases of our DNN turned out to be a complicated task in this case. This because to make useful kNN predictions and/or learn an useful DNN, the training set must be sufficiently representative of the geochemical conditions encountered during the reactive transport simulation one wish to perform with HPx$_{\\rm py}$-kNN and the trained HPx$_{\\rm py}$-DNN. In contrast to cement system 1, creating the training set by sampling the $X$-space with a controlled randomness between predefined lower and upper bounds did not prove successful. We tried that strategy by drawing as much as 4,000,000 5-dimensional $\\textbf{x}$ vectors from the $X$-space using a Sobol low-discrepancy sequence \\citep[][]{Sobol1967, Joe-Kuo2003}. Such low-discrepancy sampling scheme covers the 5-dimensional hypercube more uniformly than LHS. Despite a good performance on the test set (not shown), the resulting DNN accuracy in reactive transport mode was never deemed satisfying. In other words, no satisfying ``global\" or ``universal\" DNN emulator could be devised for this cement system. This is probably caused by the fact for this problem, the input (5 total amounts) and output (7 aqueous concentrations) spaces are quite nonlinearly related and both cover 6 to 10 orders of magnitudes depending on the considered element. Therefore, we resorted to the alternative training strategy detailed below. The latter basically tries to grasp the complex correlations and higher-order dependencies that exist between the elements of $\\textbf{x}$ (total amounts, input space) for a given reactive transport simulation setup, in order to produce a training set that honors these between-input relationships.\n\n\\begin{itemize}\n\t\\item Perform a ``cheap\" full reactive transport simulation under the transport conditions and geochemistry of interest and collect the resulting $\\textbf{x}$ -$\\textbf{y}$  pairs of examples (for the considered grid nodes and time steps). Computational demand controls what domain size and simulation time period can be used for this cheap calculation. We used a modest 16 $\\times$ 16 domain and a simulation time period of 10 years. The associated HPx$_{\\rm{4C}}$ runtimes is 180 s.\n\t\n\t\\item Fit a kernel density estimator (KDE) with a Gaussian kernel to the collected $\\textbf{x}$ vectors (encapsulated in the $\\textbf{X}$ array) and generate a fixed number of new input vectors, $\\textbf{x}_{KDE}$. Then run PHREEQC for the $\\textbf{X}_{KDE}$ set to get the corresponding output set, $\\textbf{Y}_{KDE}$. Now apply the correction for porosity described in section \\ref{train_res1} to the $\\textbf{X}_{KDE}$ set and form the training set by merging the ensemble of $\\textbf{x}$-$\\textbf{y}$ pairs with that of the $\\textbf{x}_{KDE}$-$\\textbf{y}_{KDE}$ pairs. The number of produced unique examples by the considered cheap HPx$_{\\rm{4C}}$ simulations varied between 10,000 and 50,000. The KDE-based enrichment of this dataset was deemed necessary to provide more input variability, thereby avoiding overfitting of the trained DNN and improving the kNN accuracy, while still honoring the complex between-input relationships. The number of KDE-generated samples was set as to obtain a total training set size of 1,000,000 examples. A key component of the approach is the bandwith parameter of the Gaussian KDE kernel which controls how much the KDE-generated samples depart from the original ensemble. After limited trial and error, we fixed the kernel bandwith to 0.0025 for the considered case studies.\n\t\n\\end{itemize}\n\nThe scatter plots in Figure \\ref{fig9} illustrate our training set creation procedure. The orange dots depict the pairwise relationships observed between the 5 elements of $\\textbf{x}$ in the cheap simulation. The red and cyan dots in Figure \\ref{fig9} represent the KDE-generated samples with the selected bandwidth, before and after applying the correction for porosity, respectively. Training of the DNN is achieved using the ensemble of original and KDE-corrected input points. We refer to this kind of dataset as RT-based, since it is based on an albeit cheap, full RT simulation. Furthermore, we refer to the obtained DNN and kNN emulators as ``local\" emulators, since as opposed to the emulators constructed for cement system 1, the current emulators are only valid for the input conditions encapsulated in the RT-based training set.\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1.5cm}\\includegraphics[width=50pc]{Figure9.png}\n\t\\caption{Scatter plots of the complex relationships between the five considered inputs for (1) the computationally cheap RT simulation of cement system 2 (orange dots) performed with the original HPx code, (2) the corresponding sampled points by kernel density estimation (KDE) using the selected bandwith (turquoise dots) and, (3) the same sampled KDE points after porosity correction (red dots).}\n\t\\label{fig9}\n\\end{figure}\n\nTraining performance of the DNN emulator is presented in Figure \\ref{fig10}. Similarly as for cement system 1, about 90\\% of the available data was used for the actual training of the DNN while the remaining 10\\% were used a validation set to contol overfitting. Lastly, performance is evaluated for both the trained DNN and kNN emulators using an independent test set of 10,000 examples. Overall, the accuracy of our ``local \" DNN emulator for this RT-inspired dataset is rather large with $Q_2$ values always greater or equal than 0.998. Training performance of the corresponding local kNN emulator is equally good (not shown). Regarding speedup, for this problem single-threaded PHREEQC achieves about 210 geochemical calculations per second on our used Intel\\textsuperscript{\\textregistered} i7 CPU while the GPU-based DNN and kNN emulators are both about 3000 times faster when predicting the 10,000 test points at once .\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure10.png}\n\t\\caption{1-1 plots of local DNN emulation performance obtained for system 2 when the local DNN is trained using 1,000,000 samples. The $x$-axis and $y$-axis present the original and emulated 10,000 independent test data points, respectively. The RMSE and $Q_2$ coefficient denote the root-mean-square-error and coefficient of determination in testing mode, respectively, between the original and emulated 10,000 test data points.}\n\t\\label{fig10}\n\\end{figure}\n\n\\subsubsection{Reactive transport simulation}\n\\label{prob2res}\nOur ``local\" DNN performs rather well when applied to the 61 $\\times$ 61 grid size and a time period of 6 years (Figures \\ref{fig11} - \\ref{fig12}). This for all components but C, for which some localized deviations appear between original and emulated 2D concentration profiles towards the end of the simulation period (Figure \\ref{fig12}). When applied to the 121 $\\times$ 121 grid, HPx$_{\\rm py}$-DNN produces additional discrepancies for O, Si and Al towards the end of the simulation period (Figures \\ref{fig13} - \\ref{fig14}). Yet most of the observed artifacts could probably be smoothed out by using post-filtering. The associated speedups are listed in Table \\ref{table4}. These speedups are larger to those obtained for cement system 1, with values between 8 and 9 when evaluated against HPx$_{\\rm{4C}}$. These speedups represent about 85 \\% to 90 \\% of the maximum possible speedup (Table \\ref{table3}). Overall, these findings indicate that for the considered problem, our RT-based training of a local DNN only works is the training set is sufficiently representative of the particular geochemical conditions encountered in the computationally demanding simulations, which is arguably not easy to achieve. This limitation is further discussed in section \\ref{discussion}.\n\nWe note a more uniform behavior for HPx$_{\\rm py}$-kNN across grid sizes than for HPx$_{\\rm py}$-DNN. Here the results for the 121  $\\times$ 121 grid are only slightly less accurate than those associated with the 61 $ \\times$ 61 grid (see Figures \\ref{fig15} - \\ref{fig16} where for brevity we only show concentration profiles for the C Al and S components). Furthermore, whenever observed the discrepancies between original and emulated profiles are more regularly scattered than for HPx$_{\\rm py}$-DNN. Note also that herein too, post-filtering could likely smooth out a large part of these deviations. In addition, owing to the use of a GPU to achieve the kNN calculations, the speedup provided by HPx$_{\\rm py}$-kNN are as large as those provided by HPx$_{\\rm py}$-DNN (Table \\ref{table4}).\n\nWith respect to the emulated solid amounts, the HPx$_{\\rm py}$-DNN results look visually good for the the 61 $\\times$ 61 grid. This is shown in Figure \\ref{fig17} for the C, Al and S chemical components, while emulation of the H, O Ca and Si chemical components is globally of similar quality (not shown). Nevertheless, significant deviations appear towards the end of the simulation period for every chemical component (see Figure \\ref{fig18} for the C, Al and S chemical components, emulation of the H, O Ca and Si chemical components shows the same level of mismatch). The HPx$_{\\rm py}$-kNN predictions are also fairly accurate for the 61 $\\times$ 61 (see Figure \\ref{fig19} for the C, Al and S chemical components, emulation of the H, O Ca and Si chemical components exhibits a globally similar quality) while some discrepancies show up at the end of the simulation (Figure \\ref{fig20}). However, the mismatch is less pronounced than for HPx$_{\\rm py}$-DNN. Overall, HPx$_{\\rm py}$-kNN appears to be somewhat more robust than HPx$_{\\rm py}$-DNN for this cement system, while providing the same (large) speedup.\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure11.png}\n\t\\caption{2D concentration profiles obtained for cement system 2 after 2 and 6 (final time step) years. RTM means the original HPx$_{\\rm{4C}}$ model and TM+DNN denotes the Hydrus transport model coupled with our DNN geochemical emulator (HPx$_{\\rm py}$-DNN). The first to fourth row present profiles for $Ca^{\\rm{conc}}$, $Si^{\\rm{conc}}$, $H^{\\rm{conc}}$, and $O^{\\rm{conc}}$, respectively. The considered grid size is 61 $\\times$ 61.}\n\t\\label{fig11}\n\\end{figure}\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure12.png}\n\t\\caption{2D concentration profiles obtained for cement system 2 after 2 and 6 (final time step) years. RTM means the original HPx$_{\\rm{4C}}$ model and TM+DNN denotes the Hydrus transport model coupled with our DNN geochemical emulator (HPx$_{\\rm py}$-DNN). The first to third row present profiles for $C^{\\rm{conc}}$, $Al^{\\rm{conc}}$, and $S^{\\rm{conc}}$, respectively. The considered grid size is 61 $\\times$ 61.}\n\t\\label{fig12}\n\\end{figure}\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure13.png}\n\t\\caption{2D concentration profiles obtained for cement system 2 after 2 and 6 (final time step) years. RTM means the original HPx$_{\\rm{4C}}$ model and TM+DNN denotes the Hydrus transport model coupled with our DNN geochemical emulator (HPx$_{\\rm py}$-DNN). The first to fourth row present profiles for $Ca^{\\rm{conc}}$, $Si^{\\rm{conc}}$, $H^{\\rm{conc}}$, and $O^{\\rm{conc}}$, respectively. The considered grid size is 121 $\\times$ 121.}\n\t\\label{fig13}\n\\end{figure}\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure14.png}\n\t\\caption{2D concentration profiles obtained for cement system 2 after 2 and 6 (final time step) years. RTM means the original HPx$_{\\rm{4C}}$ model and TM+DNN denotes the Hydrus transport model coupled with our DNN geochemical emulator (HPx$_{\\rm py}$-DNN). The first to third row present profiles for $C^{\\rm{conc}}$, $Al^{\\rm{conc}}$, and $S^{\\rm{conc}}$, respectively. The considered grid size is 121 $\\times$ 121.}\n\t\\label{fig14}\n\\end{figure}\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure15.png}\n\t\\caption{2D concentration profiles obtained for cement system 2 after 2 and 6 (final time step) years. RTM means the original HPx$_{\\rm{4C}}$ model and TM+kNN denotes the Hydrus transport model coupled with our kNN geochemical emulator (HPx$_{\\rm py}$-kNN).  The first to third row present profiles for $C^{\\rm{conc}}$, $Al^{\\rm{conc}}$, and $S^{\\rm{conc}}$, respectively. The considered grid size is 61 $\\times$ 61.}\n\t\\label{fig15}\n\\end{figure}\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure16.png}\n\t\\caption{2D concentration profiles obtained for cement system 2 after 2 and 6 (final time step) years. RTM means the original HPx$_{\\rm{4C}}$ model and TM+kNN denotes the Hydrus transport model coupled with our kNN geochemical emulator (HPx$_{\\rm py}$-kNN). The first to third row present profiles for $C^{\\rm{conc}}$, $Al^{\\rm{conc}}$, and $S^{\\rm{conc}}$, respectively. The considered grid size is 121 $\\times$ 121.}\n\t\\label{fig16}\n\\end{figure}\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure17.png}\n\t\\caption{2D solid amount profiles of the C, Al and S chemical components of cement system 2 after 2 and 6 (final time step) years. RTM means the original HPx$_{\\rm{4C}}$ model and TM+DNN denotes the Hydrus transport model coupled with our DNN geochemical emulator (HPx$_{\\rm py}$-DNN). The first to third row present profiles for $C^{\\rm{solid}}$, $Al^{\\rm{solid}}$, and $S^{\\rm{solid}}$, respectively The considered grid size is 61 $\\times$ 61.}\n\t\\label{fig17}\n\\end{figure}\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure18.png}\n\t\\caption{2D solid amount profiles of the C, Al and S chemical components of cement system 2 after 2 and 6 (final time step) years. RTM means the original HPx$_{\\rm{4C}}$ model and TM+DNN denotes the Hydrus transport model coupled with our DNN geochemical emulator (HPx$_{\\rm py}$-DNN). The first to third row present profiles for $C^{\\rm{solid}}$, $Al^{\\rm{solid}}$, and $S^{\\rm{solid}}$, respectively The considered grid size is 121 $\\times$ 121.}\n\t\\label{fig18}\n\\end{figure}\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure19.png}\n\t\\caption{2D solid amount profiles of the C, Al and S chemical components of cement system 2 after 2 and 6 (final time step) years. RTM means the original HPx$_{\\rm{4C}}$ model and TM+kNN denotes the Hydrus transport model coupled with our kNN geochemical emulator (HPx$_{\\rm py}$-kNN). The first to third row present profiles for $C^{\\rm{solid}}$, $Al^{\\rm{solid}}$, and $S^{\\rm{solid}}$, respectively The considered grid size is 61 $\\times$ 61.}\n\t\\label{fig19}\n\\end{figure}\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure20.png}\n\t\\caption{2D solid amount profiles of the C, Al and S chemical components of cement system 2 after 2 and 6 (final time step) years. RTM means the original HPx$_{\\rm{4C}}$ model and TM+kNN denotes the Hydrus transport model coupled with our kNN geochemical emulator (HPx$_{\\rm py}$-kNN). The first to third row present profiles for $C^{\\rm{solid}}$, $Al^{\\rm{solid}}$, and $S^{\\rm{solid}}$, respectively The considered grid size is 121 $\\times$ 121.}\n\t\\label{fig20}\n\\end{figure}\n\n\\begin{table}[h!]\n\t\\caption{Speedups offered by the KNN and DNN emulators in HPx$_{\\rm py}$ for the reactive transport simulations considered for cement system 2. The HPx$_{\\rm{4C}}$ calculations involve the parallelization of PHREEQC over our 4 CPUs. The HPx$_{\\rm{1C}}$ calculations are performed on a single CPU. Both the kNN and DNN predictions make use of a GPU. ML signifies the used machine learning method for emulation, BC denotes the type of flow boundary conditions and GS is the grid size. The maximum possible speedups associated with HPx$_{\\rm{4C}}$ and HP$_{\\rm{1C}}$,  Max SP HPx$_{\\rm{4C}}$ and  Max SP HPx$_{\\rm{1C}}$, correspond to an hypothetical situation where the geochemical calculations incur zero computational cost.}\n\n\t\\begin{center}\n\t\t\\begin{tabular}{cccccccc}\n\t\t\t\\hline\n\t\t\t& & & & & & & \\\\\n\t\t\tML & BC & GS & HPx$_{\\rm{4C}}$ time (s) & SP HPx$_{\\rm{4C}}$ & Max SP HPx$_{\\rm{4C}}$ & SP HPx$_{\\rm{1C}}$ & Max SP HPx$_{\\rm{1C}}$ \\\\\n\t\t\tDNN & ADV & 61 $\\times$ 61 & 21,415 & 8.2 & 9.0 & 30.3 & 33.1\\\\\n\t\t\tkNN & ADV & 61 $\\times$ 61 & 21,415 & 7.9 & 9.0 & 28.9 & 33.1\\\\\n\t\t\tDNN & ADV & 121 $\\times$ 121 & 199,841 & 8.2 & 9.5 & 29.9 & 32.8\\\\\n\t\t\tkNN & ADV & 121 $\\times$ 121 & 199,841 & 8.5 & 9.5 & 31.3 & 32.8\\\\\n\t\t\t\\hline\n\t\t\\end{tabular}\n\t\\end{center}\n\n\t\\label{table4}\n\\end{table}\n\n\\FloatBarrier\n\n3.2.2 Reactive transport simulation\n\\subsubsection{Reactive transport simulation}\n\\label{prob2res}\nOur ``local\" DNN performs rather well when applied to the 61 $\\times$ 61 grid size and a time period of 6 years (Figures \\ref{fig11} - \\ref{fig12}). This for all components but C, for which some localized deviations appear between original and emulated 2D concentration profiles towards the end of the simulation period (Figure \\ref{fig12}). When applied to the 121 $\\times$ 121 grid, HPx$_{\\rm py}$-DNN produces additional discrepancies for O, Si and Al towards the end of the simulation period (Figures \\ref{fig13} - \\ref{fig14}). Yet most of the observed artifacts could probably be smoothed out by using post-filtering. The associated speedups are listed in Table \\ref{table4}. These speedups are larger to those obtained for cement system 1, with values between 8 and 9 when evaluated against HPx$_{\\rm{4C}}$. These speedups represent about 85 \\% to 90 \\% of the maximum possible speedup (Table \\ref{table3}). Overall, these findings indicate that for the considered problem, our RT-based training of a local DNN only works is the training set is sufficiently representative of the particular geochemical conditions encountered in the computationally demanding simulations, which is arguably not easy to achieve. This limitation is further discussed in section \\ref{discussion}.\n\nWe note a more uniform behavior for HPx$_{\\rm py}$-kNN across grid sizes than for HPx$_{\\rm py}$-DNN. Here the results for the 121  $\\times$ 121 grid are only slightly less accurate than those associated with the 61 $ \\times$ 61 grid (see Figures \\ref{fig15} - \\ref{fig16} where for brevity we only show concentration profiles for the C Al and S components). Furthermore, whenever observed the discrepancies between original and emulated profiles are more regularly scattered than for HPx$_{\\rm py}$-DNN. Note also that herein too, post-filtering could likely smooth out a large part of these deviations. In addition, owing to the use of a GPU to achieve the kNN calculations, the speedup provided by HPx$_{\\rm py}$-kNN are as large as those provided by HPx$_{\\rm py}$-DNN (Table \\ref{table4}).\n\nWith respect to the emulated solid amounts, the HPx$_{\\rm py}$-DNN results look visually good for the the 61 $\\times$ 61 grid. This is shown in Figure \\ref{fig17} for the C, Al and S chemical components, while emulation of the H, O Ca and Si chemical components is globally of similar quality (not shown). Nevertheless, significant deviations appear towards the end of the simulation period for every chemical component (see Figure \\ref{fig18} for the C, Al and S chemical components, emulation of the H, O Ca and Si chemical components shows the same level of mismatch). The HPx$_{\\rm py}$-kNN predictions are also fairly accurate for the 61 $\\times$ 61 (see Figure \\ref{fig19} for the C, Al and S chemical components, emulation of the H, O Ca and Si chemical components exhibits a globally similar quality) while some discrepancies show up at the end of the simulation (Figure \\ref{fig20}). However, the mismatch is less pronounced than for HPx$_{\\rm py}$-DNN. Overall, HPx$_{\\rm py}$-kNN appears to be somewhat more robust than HPx$_{\\rm py}$-DNN for this cement system, while providing the same (large) speedup.\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure11.png}\n\t\\caption{2D concentration profiles obtained for cement system 2 after 2 and 6 (final time step) years. RTM means the original HPx$_{\\rm{4C}}$ model and TM+DNN denotes the Hydrus transport model coupled with our DNN geochemical emulator (HPx$_{\\rm py}$-DNN). The first to fourth row present profiles for $Ca^{\\rm{conc}}$, $Si^{\\rm{conc}}$, $H^{\\rm{conc}}$, and $O^{\\rm{conc}}$, respectively. The considered grid size is 61 $\\times$ 61.}\n\t\\label{fig11}\n\\end{figure}\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure12.png}\n\t\\caption{2D concentration profiles obtained for cement system 2 after 2 and 6 (final time step) years. RTM means the original HPx$_{\\rm{4C}}$ model and TM+DNN denotes the Hydrus transport model coupled with our DNN geochemical emulator (HPx$_{\\rm py}$-DNN). The first to third row present profiles for $C^{\\rm{conc}}$, $Al^{\\rm{conc}}$, and $S^{\\rm{conc}}$, respectively. The considered grid size is 61 $\\times$ 61.}\n\t\\label{fig12}\n\\end{figure}\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure13.png}\n\t\\caption{2D concentration profiles obtained for cement system 2 after 2 and 6 (final time step) years. RTM means the original HPx$_{\\rm{4C}}$ model and TM+DNN denotes the Hydrus transport model coupled with our DNN geochemical emulator (HPx$_{\\rm py}$-DNN). The first to fourth row present profiles for $Ca^{\\rm{conc}}$, $Si^{\\rm{conc}}$, $H^{\\rm{conc}}$, and $O^{\\rm{conc}}$, respectively. The considered grid size is 121 $\\times$ 121.}\n\t\\label{fig13}\n\\end{figure}\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure14.png}\n\t\\caption{2D concentration profiles obtained for cement system 2 after 2 and 6 (final time step) years. RTM means the original HPx$_{\\rm{4C}}$ model and TM+DNN denotes the Hydrus transport model coupled with our DNN geochemical emulator (HPx$_{\\rm py}$-DNN). The first to third row present profiles for $C^{\\rm{conc}}$, $Al^{\\rm{conc}}$, and $S^{\\rm{conc}}$, respectively. The considered grid size is 121 $\\times$ 121.}\n\t\\label{fig14}\n\\end{figure}\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure15.png}\n\t\\caption{2D concentration profiles obtained for cement system 2 after 2 and 6 (final time step) years. RTM means the original HPx$_{\\rm{4C}}$ model and TM+kNN denotes the Hydrus transport model coupled with our kNN geochemical emulator (HPx$_{\\rm py}$-kNN).  The first to third row present profiles for $C^{\\rm{conc}}$, $Al^{\\rm{conc}}$, and $S^{\\rm{conc}}$, respectively. The considered grid size is 61 $\\times$ 61.}\n\t\\label{fig15}\n\\end{figure}\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure16.png}\n\t\\caption{2D concentration profiles obtained for cement system 2 after 2 and 6 (final time step) years. RTM means the original HPx$_{\\rm{4C}}$ model and TM+kNN denotes the Hydrus transport model coupled with our kNN geochemical emulator (HPx$_{\\rm py}$-kNN). The first to third row present profiles for $C^{\\rm{conc}}$, $Al^{\\rm{conc}}$, and $S^{\\rm{conc}}$, respectively. The considered grid size is 121 $\\times$ 121.}\n\t\\label{fig16}\n\\end{figure}\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure17.png}\n\t\\caption{2D solid amount profiles of the C, Al and S chemical components of cement system 2 after 2 and 6 (final time step) years. RTM means the original HPx$_{\\rm{4C}}$ model and TM+DNN denotes the Hydrus transport model coupled with our DNN geochemical emulator (HPx$_{\\rm py}$-DNN). The first to third row present profiles for $C^{\\rm{solid}}$, $Al^{\\rm{solid}}$, and $S^{\\rm{solid}}$, respectively The considered grid size is 61 $\\times$ 61.}\n\t\\label{fig17}\n\\end{figure}\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure18.png}\n\t\\caption{2D solid amount profiles of the C, Al and S chemical components of cement system 2 after 2 and 6 (final time step) years. RTM means the original HPx$_{\\rm{4C}}$ model and TM+DNN denotes the Hydrus transport model coupled with our DNN geochemical emulator (HPx$_{\\rm py}$-DNN). The first to third row present profiles for $C^{\\rm{solid}}$, $Al^{\\rm{solid}}$, and $S^{\\rm{solid}}$, respectively The considered grid size is 121 $\\times$ 121.}\n\t\\label{fig18}\n\\end{figure}\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure19.png}\n\t\\caption{2D solid amount profiles of the C, Al and S chemical components of cement system 2 after 2 and 6 (final time step) years. RTM means the original HPx$_{\\rm{4C}}$ model and TM+kNN denotes the Hydrus transport model coupled with our kNN geochemical emulator (HPx$_{\\rm py}$-kNN). The first to third row present profiles for $C^{\\rm{solid}}$, $Al^{\\rm{solid}}$, and $S^{\\rm{solid}}$, respectively The considered grid size is 61 $\\times$ 61.}\n\t\\label{fig19}\n\\end{figure}\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure20.png}\n\t\\caption{2D solid amount profiles of the C, Al and S chemical components of cement system 2 after 2 and 6 (final time step) years. RTM means the original HPx$_{\\rm{4C}}$ model and TM+kNN denotes the Hydrus transport model coupled with our kNN geochemical emulator (HPx$_{\\rm py}$-kNN). The first to third row present profiles for $C^{\\rm{solid}}$, $Al^{\\rm{solid}}$, and $S^{\\rm{solid}}$, respectively The considered grid size is 121 $\\times$ 121.}\n\t\\label{fig20}\n\\end{figure}\n\n\\begin{table}[h!]\n\t\\caption{Speedups offered by the KNN and DNN emulators in HPx$_{\\rm py}$ for the reactive transport simulations considered for cement system 2. The HPx$_{\\rm{4C}}$ calculations involve the parallelization of PHREEQC over our 4 CPUs. The HPx$_{\\rm{1C}}$ calculations are performed on a single CPU. Both the kNN and DNN predictions make use of a GPU. ML signifies the used machine learning method for emulation, BC denotes the type of flow boundary conditions and GS is the grid size. The maximum possible speedups associated with HPx$_{\\rm{4C}}$ and HP$_{\\rm{1C}}$,  Max SP HPx$_{\\rm{4C}}$ and  Max SP HPx$_{\\rm{1C}}$, correspond to an hypothetical situation where the geochemical calculations incur zero computational cost.}\n\n\t\\begin{center}\n\t\t\\begin{tabular}{cccccccc}\n\t\t\t\\hline\n\t\t\t& & & & & & & \\\\\n\t\t\tML & BC & GS & HPx$_{\\rm{4C}}$ time (s) & SP HPx$_{\\rm{4C}}$ & Max SP HPx$_{\\rm{4C}}$ & SP HPx$_{\\rm{1C}}$ & Max SP HPx$_{\\rm{1C}}$ \\\\\n\t\t\tDNN & ADV & 61 $\\times$ 61 & 21,415 & 8.2 & 9.0 & 30.3 & 33.1\\\\\n\t\t\tkNN & ADV & 61 $\\times$ 61 & 21,415 & 7.9 & 9.0 & 28.9 & 33.1\\\\\n\t\t\tDNN & ADV & 121 $\\times$ 121 & 199,841 & 8.2 & 9.5 & 29.9 & 32.8\\\\\n\t\t\tkNN & ADV & 121 $\\times$ 121 & 199,841 & 8.5 & 9.5 & 31.3 & 32.8\\\\\n\t\t\t\\hline\n\t\t\\end{tabular}\n\t\\end{center}\n\n\t\\label{table4}\n\\end{table}\n\n\\FloatBarrier\n\n",
        "label": "fig11",
        "Descriptive_question1": "What is the grid size used in figure_11 for the 2D concentration profiles of cement system 2?",
        "Descriptive_question2": "Which chemical component is presented in the first row of figure_11?",
        "Reasoning_question1": "How do the concentration profiles of Ca and Si in figure_11 compare between the RTM and TM+DNN models after 6 years, and what might this suggest about the accuracy of the DNN geochemical emulator?",
        "Reasoning_question2": "What trends can be observed in the concentration profiles of H and O in figure_11 from 2 years to 6 years, and what could be the potential reasons for these changes over time?",
        "Descriptive_answer1": "61 × 61",
        "Descriptive_answer2": "Ca^{conc}",
        "Reasoning_answer1": "The concentration profiles of Ca and Si in figure_11 show that the TM+DNN model closely matches the RTM model after 6 years, with similar spatial distribution and concentration levels. This close agreement suggests that the DNN geochemical emulator is accurately capturing the reactive transport processes and can reliably emulate the geochemical behavior for these components in cement system 2 over the simulation period.",
        "Reasoning_answer2": "From 2 years to 6 years in figure_11, the concentration profiles of H and O show progression in their spatial distribution, likely indicating chemical transformations and transport effects over time. The observed changes could be due to ongoing geochemical reactions and advective-dispersive transport affecting the aqueous concentrations of these elements. This temporal evolution reflects the dynamic nature of the system as it evolves toward equilibrium or steady-state conditions."
    },
    {
        "paper_id": "2107.07598.json",
        "image_id": "figure_12",
        "image_path": [
            "/home/yz979/scratch/chengye/scimolmo/test-images-tables/dataset_imagestables/2107.07598/images/Figure12.png"
        ],
        "caption": "2D concentration profiles obtained for cement system 2 after 2 and 6 (final time step) years. RTM means the original HPx$_{\\rm{4C}}$ model and TM+DNN denotes the Hydrus transport model coupled with our DNN geochemical emulator (HPx$_{\\rm py}$-DNN). The first to third row present profiles for $C^{\\rm{conc}}$, $Al^{\\rm{conc}}$, and $S^{\\rm{conc}}$, respectively. The considered grid size is 61 $\\times$ 61.",
        "classify": "Chart",
        "section_info": "3 Results\n\\section{Results}\n\\label{results}\n\n\\subsection{Ca-Si Problem}\nFor this first cement system, the emulation problem consists of predicting at each time step of the RT simulation the (output) Ca, Si, H and O aqueous concentrations (mol/kg of water or mol/kgw) from the (input) total amounts of Ca and Si (mol). \n\n\\subsubsection{Training the emulators}\n\\label{train_res1}\nHere the kNN and DNN emulators are firstly trained using a set of 400,000 test examples for both. This training set is obtained by randomly sampling the two-dimensional input space by latin hypercube sampling (LHS) between $\\left[0,0\\right]$ and $\\rm{\\left[Ca^{tot}_{max},Si^{tot}_{max}\\right]}$, and running PHREEQC for each input sample, $\\rm{\\textbf{x}_i = \\left[Ca^{tot}_i, Si^{tot}_i\\right]}$ to get the corresponding output vectors, $\\rm{\\textbf{y}_i = \\left[Ca^{conc}_i,Si^{conc}_i,H^{conc}_i,O^{conc}_i\\right]}$. The upper bounds, $\\rm{Ca^{tot}_{max}}$ and $\\rm{Si^{tot}_{max}}$ are defined based on a cheap full RT simulation with advective-dispersive transport using a small 1D domain of 51 nodes. It is worth noting that the total amounts of $\\textbf{x}_i$, corresponding to the PHREEQC-simulated concentrations, $\\textbf{y}_i$, have to be corrected for the different amount of water between the training set and the transport simulations. Doing so, it turns out that about 20 \\% of the post-corrected $x_i$ values exceed their pre-defined upper bounds and these excessively large values need to be filtered out. Creating the 400,000 training examples thus required about 500,000 PHREEQC runs. As stated earlier, for this problem single-threaded PHREEQC performs about 670 geochemical calculations per second on our used Intel\\textsuperscript{\\textregistered} i7 CPU.\n\nWith respect to kNN, the tuning parameters are the number of neighbors, $k$, the type of distance measure, and the interpolation technique. We simply used the default settings: $k = 5$, euclidean distance and inverse-distance interpolation. Regarding training of the DNN, the 400,000 sample were split between the training set itself (90 \\% of the data) and a validation set (10 \\% of the data). The latter serves to monitor the evolution of the selected mean squared error loss function on samples that are not used for training, thereby detecting potential overfitting. If the validation loss stops decreasing before the fixed number of epochs has been completed, then  training is stopped. Importantly, the emulation is achieved in log-space for both the input, $X$ and output $Y$, domains. This because total amounts and concentrations of the involved components typically cover many orders of magnitudes (up to 10 orders or more). Using a DNN also requires some form of data normalization or standardization. Here both the $\\rm{log\\left(\\textbf{x}_i\\right)}$ and $\\rm{log\\left(\\textbf{y}_i\\right)}$ vectors are standardized around 0 with standard deviation of 1.\n\nFigure \\ref{fig1} illustrates the trained emulators' performance for geochemical predictions using an independent test set that comprises 10,000 test examples. Both kNN (Figures \\ref{fig1}a - d) and DNN (Figures \\ref{fig1}e - h) appear to be rather accurate . DNN also shows a slight degradation for the larger concentration values  (Figures \\ref{fig1}e - h). The latter is likely due to the combination of a small proportion of large concentration values in the training set with the log-transformation that implicitly pushes the DNN to try harder to fit the smaller concentrations during training. Regarding speedup and as written earlier, for this setup the single-threaded kNN method is 300 times faster than single-threaded PHREEQC for predicting the 10,000 concentration vectors all at once. The computational savings allowed by the DNN emulator when ran on our GPU is higher with a speedup as large as 4000 for predicting the same 10,000 concentration vectors all at once. \n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{0cm}\\includegraphics[width=35pc]{Figure1.png}\n\t\\caption{1-1 plots of the kNN (subfigures (a) - (d)) and DNN (subfigures (e) - (h)) emulators' performance obtained for system 1 when the kNN training base contains 400,000 samples and the DNN is trained using the same 400,000 samples. Here ``true\" means the original PHREEQC-simulated data and ``predicted\" denotes the emulated (that is, kNN-simulated and DNN-simulated) data. Hence, the $x$-axis and $y$-axis present the original and emulated 10,000 independent test data points, respectively. The RMSE and $Q_2$ coefficient denote the root-mean-square-error and coefficient of determination in testing mode, respectively, between the original and emulated 10,000 test data points.}\n\t\\label{fig1}\n\\end{figure}\n\nTo test the sensitivity of the emulators' performance to the training set size, training was also performed using reduced training sets comprising 100,000, 10,000, 1000 and 100 samples, respectively. It seen that the DNN performance achieved  when using 10,000 training samples is virtually the same as that obtained when using 400,000 training samples (Figure \\ref{fig2} and Table \\ref{table2}). It is only for training sets smaller than 1000 samples that the DNN performance starts to degrade significantly (Table \\ref{table2}). In contrast, the behavior of kNN appears to markedly decrease as the training set gets smaller (Figure \\ref{fig2} and Table \\ref{table1}).\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{0cm}\\includegraphics[width=35pc]{Figure2.png}\n\t\\caption{1-1 plots of the kNN (subfigures (a) - (d)) and DNN (subfigures (e) - (h)) emulators' performance obtained for system 1 when the kNN training base contains 10,000 samples and the DNN is trained using the same 10,000 samples. Here ``true\" means the original PHREEQC-simulated data and ``predicted\" denotes the emulated (that is, kNN-simulated and DNN-simulated) data. Hence, the $x$-axis and $y$-axis present the original and emulated 10,000 independent test data points, respectively. The RMSE and $Q_2$ coefficient denote the root-mean-square-error and coefficient of determination in testing mode, respectively, between the original and emulated 10,000 test data points.}\n\t\\label{fig2}\n\\end{figure}\n\n\\begin{table}[h!]\n\t\\caption{Performance of the DNN and kNN emulators for cement system 1 and different training set sizes. For brevity, only the results for Ca$^{conc}$ and Si$^{conc}$ are shown. The units are mol per kg of water (mol/kgw). ML refers to the type of emulator, TR signifies the size of the training set, and the RMSE and $Q_2$ coefficient denote the root-mean-square-error and coefficient of determination in testing mode, respectively, between the original and emulated 10,000 test data points.}\n\t\\begin{center}\n\t\t\\begin{tabular}{cccccccc}\n\t\t\t\\hline\n\t\t\t& & & & & \\\\\n\t\t\tML & TR & RMSE - Ca$^{conc}$ & $Q_2$ - Ca$^{conc}$ & RMSE - Si$^{conc}$ & $Q_2$ - Si$^{conc}$\\\\\n\t\t\tDNN & 4 $\\times$ 10$^{5}$ & 4.50 $\\times$ 10$^{-5}$ & 0.9999 & 1.21 $\\times$ 10$^{-6}$ & 0.9999\\\\\n\t\t\tDNN & 1 $\\times$ 10$^{5}$ & 5.06 $\\times$ 10$^{-5}$ & 0.9999 & 1.30 $\\times$ 10$^{-6}$ & 0.9999\\\\\n\t\t\tDNN & 1 $\\times$ 10$^{4}$ & 4.57 $\\times$ 10$^{-5}$ & 0.9999 & 1.48 $\\times$ 10$^{-6}$ & 0.9999\\\\\n\t\t\tDNN & 1 $\\times$ 10$^{3}$ & 8.02 $\\times$ 10$^{-5}$ & 0.9998 & 3.31 $\\times$ 10$^{-6}$ & 0.9994\\\\\n\t\t\tDNN & 1 $\\times$ 10$^{2}$ & 47.1 $\\times$ 10$^{-5}$ & 0.9935 & 9.02 $\\times$ 10$^{-6}$ & 0.9959\\\\\n\t\t\tkNN & 4 $\\times$ 10$^{5}$ & 2.52 $\\times$ 10$^{-5}$ & 1.0000 & 1.59 $\\times$ 10$^{-6}$ & 0.9999\\\\\n\t\t\tkNN & 1 $\\times$ 10$^{5}$ & 6.27 $\\times$ 10$^{-5}$ & 0.9999 & 4.20 $\\times$ 10$^{-6}$ & 0.9991\\\\\n\t\t\tkNN & 1 $\\times$ 10$^{4}$ & 14.1 $\\times$ 10$^{-5}$ & 0.9994 & 13.5 $\\times$ 10$^{-6}$ & 0.9908\\\\\n\t\t\tkNN & 1 $\\times$ 10$^{3}$ & 43.9 $\\times$ 10$^{-5}$ & 0.9944 & 74.6 $\\times$ 10$^{-6}$ & 0.7180\\\\\n\t\t\tkNN & 1 $\\times$ 10$^{2}$  & 96.4 $\\times$ 10$^{-5}$ & 0.9729 & 55.0 $\\times$ 10$^{-6}$ & 0.8468\\\\\n\t\t\\end{tabular}\n\t\\end{center}\n\t\\label{table2}\n\\end{table}\n\n\\FloatBarrier\n\n\\subsubsection{Reactive transport simulation}\n\\label{prob12Dres}\n\nThis section focuses on reactive transport simulations with HPx$_{\\rm py}$ within cement system 1, under both advective-dispersive and diffusive transport conditions. As written above, the domain sizes are both 61 $\\times$ 61 and 121 $\\times$ 121 for the advection-dispersion case and, because of computational constraints, solely 61 $\\times$ 61  for the diffusion case. In addition, the simulation time period is 2 years for the advection-dispersion case and 1 year for the diffusion case. Figures \\ref{fig3} and \\ref{fig4} present times series of original and emulated Ca, Si, H and O concentrations at 5 locations within the 2D domain for advective-dispersive transport conditions, for both our kNN-based (HPx$_{\\rm py}$-kNN) and DNN-based (HPx$_{\\rm py}$-DNN) reactive transport codes. It is seen that HPx$_{\\rm py}$-kNN and HPx$_{\\rm py}$-DNN both induce a quite good simulation accuracy. Also, the results for the diffusive transport case are of similarly good quality (not shown). Figures \\ref{fig5} - \\ref{fig6} provide more insights into the HPx$_{\\rm py}$-kNN and HPx$_{\\rm py}$-DNN performances by displaying 2D Ca, Si, H and O concentration profiles at a given time. For each experiment and chemical component, this time is selected as to be well representative of the simulated dynamics. It is observed that the original and emulated images are visually almost indistinguishable for the advection-dispersion case (Figure \\ref{fig5}). For the diffusion case, the emulators also perform quite well for Ca, H and O (Figures\\ref{fig6}a - c, g - i and j - l), while some slight to moderate discrepancies appear at the concentration front for Si (Figures \\ref{fig6}d - f). Notwithstanding, the Si concentration remains globally well predicted. Furthermore, Figures \\ref{fig7} - \\ref{fig8} present the original and emulated 2D solid amount profiles corresponding to Figures \\ref{fig5} - \\ref{fig6}. The original solid amount profiles are overall well approximated by HPx$_{\\rm py}$-kNN and HPx$_{\\rm py}$-DNN for the advection-dispersion case (Figure \\ref{fig7}), even though some mismatch appears at the border of the fully depleted zone for the H component. As of the diffusion case (Figure \\ref{fig8}), the same kind of mismatch is observed for the emulated solid amounts of H by HPx$_{\\rm py}$-kNN while the emulated profiles by HPx$_{\\rm py}$-DNN show some more discrepancies. Though we decided to present raw emulation results, we would like to stress that some if not all of the observed artifacts could likely be smoothed out by using some post-filtering such as median filtering.\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure3.png}\n\t\\caption{Time series of original (RTM, solid green lines) and HPx$_{\\rm py}$-kNN emulated (TM+kNN, dashed orange lines) concentrations (mol/kg) of Ca, H, O and Si at selected observations points for cement system 1 and advective-dispersive transport. Obs. 1 - 5 denote the selected observation points, with the following $\\left[x,y\\right]$ locations (in cm). Obs. 1:  $\\left[0.5, 2.5\\right]$, Obs. 2: $\\left[1, 2\\right]$, Obs. 3: $\\left[2, 2\\right]$, Obs. 4: $\\left[1, 1\\right]$, Obs. 5: $\\left[2, 1\\right]$. The results for the 121 $\\times$ 121 grid size are rather similar.}\n\t\\label{fig3}\n\\end{figure}\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure4.png}\n\t\\caption{Time series of original (RTM, solid green lines) and HPx$_{\\rm py}$-DNN emulated (TM+DNN, dashed orange lines) concentrations (mol/kg) of Ca, H, O and Si at selected observations points for cement system 1 and advective-dispersive transport. Obs. 1 - 5 denote the selected observation points, with the following $\\left[x,y\\right]$ locations (in cm). Obs. 1:  $\\left[0.5, 2.5\\right]$, Obs. 2: $\\left[1, 2\\right]$, Obs. 3: $\\left[2, 2\\right]$, Obs. 4: $\\left[1, 1\\right]$, Obs. 5: $\\left[2, 1\\right]$. The results for the 121 $\\times$ 121 grid size are rather similar.}\n\t\\label{fig4}\n\\end{figure}\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure5.png}\n\t\\caption{2D concentration profiles obtained for cement system 1 at the end of the 2-year simulation performed for the advection-dispersion case. RTM means the original HP$_{\\rm{4C}}$ model, TM+kNN denotes the Hydrus transport model coupled with our kNN geochemical emulator (HPx$_{\\rm py}$-kNN) and TM+DNN signifies the Hydrus transport model coupled with our DNN geochemical emulator (HPx$_{\\rm py}$-DNN). The first to fourth row present profiles for $Ca^{\\rm{conc}}$, $Si^{\\rm{conc}}$, $H^{\\rm{conc}}$, and $O^{\\rm{conc}}$, respectively. The considered grid size is 61 $\\times$ 61. The results for the 121 $\\times$ 121 grid are rather similar.}\n\t\\label{fig5}\n\\end{figure}\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure6.png}\n\t\\caption{2D concentration profiles obtained for cement system 1 at the final time step of the 1-year simulation performed for the diffusion case. RTM means the original HP$_{\\rm{4C}}$ model, TM+kNN denotes the Hydrus transport model coupled with our kNN geochemical emulator (HPx$_{\\rm py}$-kNN) and TM+DNN signifies the Hydrus transport model coupled with our DNN geochemical emulator (HPx$_{\\rm py}$-DNN). The first to fourth row present profiles for $Ca^{\\rm{conc}}$, $Si^{\\rm{conc}}$, $H^{\\rm{conc}}$, and $O^{\\rm{conc}}$, respectively. The considered grid size is 61 $\\times$ 61.}\n\t\\label{fig6}\n\\end{figure}\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure7.png}\n\t\\caption{2D solid amount profiles obtained for cement system 1 at the end of the 2-year simulation performed for the advection-dispersion case. RTM means the original HP$_{\\rm{4C}}$ model, TM+kNN denotes the Hydrus transport model coupled with our kNN geochemical emulator (HPx$_{\\rm py}$-kNN) and TM+DNN signifies the Hydrus transport model coupled with our DNN geochemical emulator (HPx$_{\\rm py}$-DNN). The first to fourth row present profiles for $Ca^{\\rm{solid}}$, $Si^{\\rm{solid}}$, $H^{\\rm{solid}}$, and $O^{\\rm{solid}}$, respectively. The considered grid size is 61 $\\times$ 61. The results for the 121 $\\times$ 121 grid are rather similar.}\n\t\\label{fig7}\n\\end{figure}\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure8.png}\n\t\\caption{2D solid amount profiles obtained for cement system 1 at the final time step of the 1-year simulation performed for the diffusion case. RTM means the original HP$_{\\rm{4C}}$ model, TM+kNN denotes the Hydrus transport model coupled with our kNN geochemical emulator (HPx$_{\\rm py}$-kNN) and TM+DNN signifies the Hydrus transport model coupled with our DNN geochemical emulator (HPx$_{\\rm py}$-DNN). The first to fourth row present profiles for $Ca^{\\rm{solid}}$, $Si^{\\rm{solid}}$, $H^{\\rm{solid}}$, and $O^{\\rm{solid}}$, respectively. The considered grid size is 61 $\\times$ 61.}\n\t\\label{fig8}\n\\end{figure}\n\nThe speedups associated with the considered problem are detailed in Table \\ref{table3}. It is noted that the GPU-based DNN emulator allows for a speedup that is close to optimal. Indeed, The DNN speedups overall represent 85 \\% to 95 \\% of the maximum possible speedups (that is, speedups that would be obtained if the geochemcical calculations would come at no cost at all). The speedups associated with single-threaded kNN remain substantial but only amount to 57 \\% - 65 \\% of the corresponding maximum speedups. As detailed in section \\ref{train_res1}, the used kNN and DNN implementations are found to be respectively 300 and 4000 faster than single-threaded PHREEQC when predicting 10,000 points all one once for this geochemical system. Based on these numbers one could have expected the achieved speedups to represent say 90 \\% (kNN) or 99 \\% (DNN) of the maximum possible ones. A large part of the gaps between achieved and maximum possible speedups is thus likely caused by the time required for communicating and exchanging data between the main C/C++ code and the Python-based emulators.\n\n\\begin{table}[h!]\n\t\\caption{Speedups offered by the KNN and DNN emulators in HPx$_{\\rm py}$ for the reactive transport simulations considered for cement system 1. The HPx$_{\\rm{4C}}$ calculations involve the parallelization of PHREEQC over our 4 CPUs. The HPx$_{\\rm{1C}}$ calculations are performed on a single CPU. The kNN predictions are performed on a single CPU using the scikit-learn implementation while the DNN predictions make use of our GPU. ML signifies the used machine learning method for emulation, TC denotes transport conditions (ADV: advection-dispersion, DIF: diffusion) and GS is the grid size. The maximum possible speedups associated with HPx$_{\\rm{4C}}$ and HPx$_{\\rm{1C}}$,  Max SP HPx$_{\\rm{4C}}$ and  Max SP HPx$_{\\rm{1C}}$, correspond to an hypothetical situation where the geochemical calculations incur zero computational cost.}\n\n\t\\begin{center}\n\t\t\\begin{tabular}{cccccccc}\n\t\t\t\\hline\n\t\t\t& & & & & & & \\\\\n\t\t\tML & TC & GS & HPx$_{\\rm{4C}}$ time (s) & SP HPx$_{\\rm{4C}}$ & Max SP HPx$_{\\rm{4C}}$ & SP HPx$_{\\rm{1C}}$ & Max SP HPx$_{\\rm{1C}}$ \\\\\n\t\t\tDNN & ADV & 61 $\\times$ 61 & 3189 & 6.8 & 7.7 & 24.5 & 28.5 \\\\\n\t\t\tkNN & ADV & 61 $\\times$ 61 & 3189 & 5.0 & 7.7 & 18.0 & 28.5 \\\\\n\t\t\tDNN & ADV & 121 $\\times$ 121 & 23,337 & 5.0 & 5.2 & 17.0 & 18.0 \\\\\n\t\t\tkNN & ADV & 121 $\\times$ 121 & 23,337 & 3.4 & 5.2 & 11.6 & 18.0 \\\\\n\t\t\tDNN & DIF & 61 $\\times$ 61 & 25,448 & 4.2 & 4.9 & 16.2 & 19.1 \\\\\n\t\t\tkNN & DIF & 61 $\\times$ 61 & 25,448 & 2.8 & 4.9 & 10.8 & 19.1 \\\\\n\t\t\t\\hline\n\t\t\\end{tabular}\n\t\\end{center}\n\n\t\\label{table3}\n\\end{table}\n\n\\FloatBarrier\n\n\\subsection{Al-C-Ca-S-Si Problem}\n\nFor this second cement system, the emulation problem consists of predicting at each time step of the RT simulation the (output) Al, C, Ca, S, Si, H and O aqueous concentrations (mol/kgw) from the (input) total amounts of Al, C, Ca, S and Si (mol). Here we focus on advective-dispersive transport only while similarly as for cement system 1, the considered domain sizes are 61 $\\times$ 61 and 121 $\\times$ 121. Moreover, the simulation time period is set to 6 years. \n\nAs mentioned earlier, for this higher-dimensional problem it is observed that the used scikit-learn kNN implementation becomes prohibitively slow compared to HPx$_{\\rm{4C}}$. We found that to get a good emulation accuracy the kNN training base needs to contain 1,000,000 samples (or more). This training base's size together with a 5-dimensional search space leads to an HPx$_{\\rm py}$-kNN reactive transport simulation time that is comparable to that of HPx$_{\\rm{4C}}$. Therefore, for this second cement system we built a custom kNN regressor around another kNN implementation contained in the FAISS package \\citep{faiss2017}. Our used FAISS variant allows for GPU computing and is much faster than scikit-learn for this cement system, but is slightly less accurate due to the use of an approximate rather than exact nearest neighbor search \\citep[see][for details]{faiss2017}.\n\n\\subsubsection{Training the DNN}\n\\label{train_res2}\n\nBuilding a good training set to perform a kNN search and learn the weights and biases of our DNN turned out to be a complicated task in this case. This because to make useful kNN predictions and/or learn an useful DNN, the training set must be sufficiently representative of the geochemical conditions encountered during the reactive transport simulation one wish to perform with HPx$_{\\rm py}$-kNN and the trained HPx$_{\\rm py}$-DNN. In contrast to cement system 1, creating the training set by sampling the $X$-space with a controlled randomness between predefined lower and upper bounds did not prove successful. We tried that strategy by drawing as much as 4,000,000 5-dimensional $\\textbf{x}$ vectors from the $X$-space using a Sobol low-discrepancy sequence \\citep[][]{Sobol1967, Joe-Kuo2003}. Such low-discrepancy sampling scheme covers the 5-dimensional hypercube more uniformly than LHS. Despite a good performance on the test set (not shown), the resulting DNN accuracy in reactive transport mode was never deemed satisfying. In other words, no satisfying ``global\" or ``universal\" DNN emulator could be devised for this cement system. This is probably caused by the fact for this problem, the input (5 total amounts) and output (7 aqueous concentrations) spaces are quite nonlinearly related and both cover 6 to 10 orders of magnitudes depending on the considered element. Therefore, we resorted to the alternative training strategy detailed below. The latter basically tries to grasp the complex correlations and higher-order dependencies that exist between the elements of $\\textbf{x}$ (total amounts, input space) for a given reactive transport simulation setup, in order to produce a training set that honors these between-input relationships.\n\n\\begin{itemize}\n\t\\item Perform a ``cheap\" full reactive transport simulation under the transport conditions and geochemistry of interest and collect the resulting $\\textbf{x}$ -$\\textbf{y}$  pairs of examples (for the considered grid nodes and time steps). Computational demand controls what domain size and simulation time period can be used for this cheap calculation. We used a modest 16 $\\times$ 16 domain and a simulation time period of 10 years. The associated HPx$_{\\rm{4C}}$ runtimes is 180 s.\n\t\n\t\\item Fit a kernel density estimator (KDE) with a Gaussian kernel to the collected $\\textbf{x}$ vectors (encapsulated in the $\\textbf{X}$ array) and generate a fixed number of new input vectors, $\\textbf{x}_{KDE}$. Then run PHREEQC for the $\\textbf{X}_{KDE}$ set to get the corresponding output set, $\\textbf{Y}_{KDE}$. Now apply the correction for porosity described in section \\ref{train_res1} to the $\\textbf{X}_{KDE}$ set and form the training set by merging the ensemble of $\\textbf{x}$-$\\textbf{y}$ pairs with that of the $\\textbf{x}_{KDE}$-$\\textbf{y}_{KDE}$ pairs. The number of produced unique examples by the considered cheap HPx$_{\\rm{4C}}$ simulations varied between 10,000 and 50,000. The KDE-based enrichment of this dataset was deemed necessary to provide more input variability, thereby avoiding overfitting of the trained DNN and improving the kNN accuracy, while still honoring the complex between-input relationships. The number of KDE-generated samples was set as to obtain a total training set size of 1,000,000 examples. A key component of the approach is the bandwith parameter of the Gaussian KDE kernel which controls how much the KDE-generated samples depart from the original ensemble. After limited trial and error, we fixed the kernel bandwith to 0.0025 for the considered case studies.\n\t\n\\end{itemize}\n\nThe scatter plots in Figure \\ref{fig9} illustrate our training set creation procedure. The orange dots depict the pairwise relationships observed between the 5 elements of $\\textbf{x}$ in the cheap simulation. The red and cyan dots in Figure \\ref{fig9} represent the KDE-generated samples with the selected bandwidth, before and after applying the correction for porosity, respectively. Training of the DNN is achieved using the ensemble of original and KDE-corrected input points. We refer to this kind of dataset as RT-based, since it is based on an albeit cheap, full RT simulation. Furthermore, we refer to the obtained DNN and kNN emulators as ``local\" emulators, since as opposed to the emulators constructed for cement system 1, the current emulators are only valid for the input conditions encapsulated in the RT-based training set.\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1.5cm}\\includegraphics[width=50pc]{Figure9.png}\n\t\\caption{Scatter plots of the complex relationships between the five considered inputs for (1) the computationally cheap RT simulation of cement system 2 (orange dots) performed with the original HPx code, (2) the corresponding sampled points by kernel density estimation (KDE) using the selected bandwith (turquoise dots) and, (3) the same sampled KDE points after porosity correction (red dots).}\n\t\\label{fig9}\n\\end{figure}\n\nTraining performance of the DNN emulator is presented in Figure \\ref{fig10}. Similarly as for cement system 1, about 90\\% of the available data was used for the actual training of the DNN while the remaining 10\\% were used a validation set to contol overfitting. Lastly, performance is evaluated for both the trained DNN and kNN emulators using an independent test set of 10,000 examples. Overall, the accuracy of our ``local \" DNN emulator for this RT-inspired dataset is rather large with $Q_2$ values always greater or equal than 0.998. Training performance of the corresponding local kNN emulator is equally good (not shown). Regarding speedup, for this problem single-threaded PHREEQC achieves about 210 geochemical calculations per second on our used Intel\\textsuperscript{\\textregistered} i7 CPU while the GPU-based DNN and kNN emulators are both about 3000 times faster when predicting the 10,000 test points at once .\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure10.png}\n\t\\caption{1-1 plots of local DNN emulation performance obtained for system 2 when the local DNN is trained using 1,000,000 samples. The $x$-axis and $y$-axis present the original and emulated 10,000 independent test data points, respectively. The RMSE and $Q_2$ coefficient denote the root-mean-square-error and coefficient of determination in testing mode, respectively, between the original and emulated 10,000 test data points.}\n\t\\label{fig10}\n\\end{figure}\n\n\\subsubsection{Reactive transport simulation}\n\\label{prob2res}\nOur ``local\" DNN performs rather well when applied to the 61 $\\times$ 61 grid size and a time period of 6 years (Figures \\ref{fig11} - \\ref{fig12}). This for all components but C, for which some localized deviations appear between original and emulated 2D concentration profiles towards the end of the simulation period (Figure \\ref{fig12}). When applied to the 121 $\\times$ 121 grid, HPx$_{\\rm py}$-DNN produces additional discrepancies for O, Si and Al towards the end of the simulation period (Figures \\ref{fig13} - \\ref{fig14}). Yet most of the observed artifacts could probably be smoothed out by using post-filtering. The associated speedups are listed in Table \\ref{table4}. These speedups are larger to those obtained for cement system 1, with values between 8 and 9 when evaluated against HPx$_{\\rm{4C}}$. These speedups represent about 85 \\% to 90 \\% of the maximum possible speedup (Table \\ref{table3}). Overall, these findings indicate that for the considered problem, our RT-based training of a local DNN only works is the training set is sufficiently representative of the particular geochemical conditions encountered in the computationally demanding simulations, which is arguably not easy to achieve. This limitation is further discussed in section \\ref{discussion}.\n\nWe note a more uniform behavior for HPx$_{\\rm py}$-kNN across grid sizes than for HPx$_{\\rm py}$-DNN. Here the results for the 121  $\\times$ 121 grid are only slightly less accurate than those associated with the 61 $ \\times$ 61 grid (see Figures \\ref{fig15} - \\ref{fig16} where for brevity we only show concentration profiles for the C Al and S components). Furthermore, whenever observed the discrepancies between original and emulated profiles are more regularly scattered than for HPx$_{\\rm py}$-DNN. Note also that herein too, post-filtering could likely smooth out a large part of these deviations. In addition, owing to the use of a GPU to achieve the kNN calculations, the speedup provided by HPx$_{\\rm py}$-kNN are as large as those provided by HPx$_{\\rm py}$-DNN (Table \\ref{table4}).\n\nWith respect to the emulated solid amounts, the HPx$_{\\rm py}$-DNN results look visually good for the the 61 $\\times$ 61 grid. This is shown in Figure \\ref{fig17} for the C, Al and S chemical components, while emulation of the H, O Ca and Si chemical components is globally of similar quality (not shown). Nevertheless, significant deviations appear towards the end of the simulation period for every chemical component (see Figure \\ref{fig18} for the C, Al and S chemical components, emulation of the H, O Ca and Si chemical components shows the same level of mismatch). The HPx$_{\\rm py}$-kNN predictions are also fairly accurate for the 61 $\\times$ 61 (see Figure \\ref{fig19} for the C, Al and S chemical components, emulation of the H, O Ca and Si chemical components exhibits a globally similar quality) while some discrepancies show up at the end of the simulation (Figure \\ref{fig20}). However, the mismatch is less pronounced than for HPx$_{\\rm py}$-DNN. Overall, HPx$_{\\rm py}$-kNN appears to be somewhat more robust than HPx$_{\\rm py}$-DNN for this cement system, while providing the same (large) speedup.\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure11.png}\n\t\\caption{2D concentration profiles obtained for cement system 2 after 2 and 6 (final time step) years. RTM means the original HPx$_{\\rm{4C}}$ model and TM+DNN denotes the Hydrus transport model coupled with our DNN geochemical emulator (HPx$_{\\rm py}$-DNN). The first to fourth row present profiles for $Ca^{\\rm{conc}}$, $Si^{\\rm{conc}}$, $H^{\\rm{conc}}$, and $O^{\\rm{conc}}$, respectively. The considered grid size is 61 $\\times$ 61.}\n\t\\label{fig11}\n\\end{figure}\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure12.png}\n\t\\caption{2D concentration profiles obtained for cement system 2 after 2 and 6 (final time step) years. RTM means the original HPx$_{\\rm{4C}}$ model and TM+DNN denotes the Hydrus transport model coupled with our DNN geochemical emulator (HPx$_{\\rm py}$-DNN). The first to third row present profiles for $C^{\\rm{conc}}$, $Al^{\\rm{conc}}$, and $S^{\\rm{conc}}$, respectively. The considered grid size is 61 $\\times$ 61.}\n\t\\label{fig12}\n\\end{figure}\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure13.png}\n\t\\caption{2D concentration profiles obtained for cement system 2 after 2 and 6 (final time step) years. RTM means the original HPx$_{\\rm{4C}}$ model and TM+DNN denotes the Hydrus transport model coupled with our DNN geochemical emulator (HPx$_{\\rm py}$-DNN). The first to fourth row present profiles for $Ca^{\\rm{conc}}$, $Si^{\\rm{conc}}$, $H^{\\rm{conc}}$, and $O^{\\rm{conc}}$, respectively. The considered grid size is 121 $\\times$ 121.}\n\t\\label{fig13}\n\\end{figure}\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure14.png}\n\t\\caption{2D concentration profiles obtained for cement system 2 after 2 and 6 (final time step) years. RTM means the original HPx$_{\\rm{4C}}$ model and TM+DNN denotes the Hydrus transport model coupled with our DNN geochemical emulator (HPx$_{\\rm py}$-DNN). The first to third row present profiles for $C^{\\rm{conc}}$, $Al^{\\rm{conc}}$, and $S^{\\rm{conc}}$, respectively. The considered grid size is 121 $\\times$ 121.}\n\t\\label{fig14}\n\\end{figure}\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure15.png}\n\t\\caption{2D concentration profiles obtained for cement system 2 after 2 and 6 (final time step) years. RTM means the original HPx$_{\\rm{4C}}$ model and TM+kNN denotes the Hydrus transport model coupled with our kNN geochemical emulator (HPx$_{\\rm py}$-kNN).  The first to third row present profiles for $C^{\\rm{conc}}$, $Al^{\\rm{conc}}$, and $S^{\\rm{conc}}$, respectively. The considered grid size is 61 $\\times$ 61.}\n\t\\label{fig15}\n\\end{figure}\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure16.png}\n\t\\caption{2D concentration profiles obtained for cement system 2 after 2 and 6 (final time step) years. RTM means the original HPx$_{\\rm{4C}}$ model and TM+kNN denotes the Hydrus transport model coupled with our kNN geochemical emulator (HPx$_{\\rm py}$-kNN). The first to third row present profiles for $C^{\\rm{conc}}$, $Al^{\\rm{conc}}$, and $S^{\\rm{conc}}$, respectively. The considered grid size is 121 $\\times$ 121.}\n\t\\label{fig16}\n\\end{figure}\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure17.png}\n\t\\caption{2D solid amount profiles of the C, Al and S chemical components of cement system 2 after 2 and 6 (final time step) years. RTM means the original HPx$_{\\rm{4C}}$ model and TM+DNN denotes the Hydrus transport model coupled with our DNN geochemical emulator (HPx$_{\\rm py}$-DNN). The first to third row present profiles for $C^{\\rm{solid}}$, $Al^{\\rm{solid}}$, and $S^{\\rm{solid}}$, respectively The considered grid size is 61 $\\times$ 61.}\n\t\\label{fig17}\n\\end{figure}\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure18.png}\n\t\\caption{2D solid amount profiles of the C, Al and S chemical components of cement system 2 after 2 and 6 (final time step) years. RTM means the original HPx$_{\\rm{4C}}$ model and TM+DNN denotes the Hydrus transport model coupled with our DNN geochemical emulator (HPx$_{\\rm py}$-DNN). The first to third row present profiles for $C^{\\rm{solid}}$, $Al^{\\rm{solid}}$, and $S^{\\rm{solid}}$, respectively The considered grid size is 121 $\\times$ 121.}\n\t\\label{fig18}\n\\end{figure}\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure19.png}\n\t\\caption{2D solid amount profiles of the C, Al and S chemical components of cement system 2 after 2 and 6 (final time step) years. RTM means the original HPx$_{\\rm{4C}}$ model and TM+kNN denotes the Hydrus transport model coupled with our kNN geochemical emulator (HPx$_{\\rm py}$-kNN). The first to third row present profiles for $C^{\\rm{solid}}$, $Al^{\\rm{solid}}$, and $S^{\\rm{solid}}$, respectively The considered grid size is 61 $\\times$ 61.}\n\t\\label{fig19}\n\\end{figure}\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure20.png}\n\t\\caption{2D solid amount profiles of the C, Al and S chemical components of cement system 2 after 2 and 6 (final time step) years. RTM means the original HPx$_{\\rm{4C}}$ model and TM+kNN denotes the Hydrus transport model coupled with our kNN geochemical emulator (HPx$_{\\rm py}$-kNN). The first to third row present profiles for $C^{\\rm{solid}}$, $Al^{\\rm{solid}}$, and $S^{\\rm{solid}}$, respectively The considered grid size is 121 $\\times$ 121.}\n\t\\label{fig20}\n\\end{figure}\n\n\\begin{table}[h!]\n\t\\caption{Speedups offered by the KNN and DNN emulators in HPx$_{\\rm py}$ for the reactive transport simulations considered for cement system 2. The HPx$_{\\rm{4C}}$ calculations involve the parallelization of PHREEQC over our 4 CPUs. The HPx$_{\\rm{1C}}$ calculations are performed on a single CPU. Both the kNN and DNN predictions make use of a GPU. ML signifies the used machine learning method for emulation, BC denotes the type of flow boundary conditions and GS is the grid size. The maximum possible speedups associated with HPx$_{\\rm{4C}}$ and HP$_{\\rm{1C}}$,  Max SP HPx$_{\\rm{4C}}$ and  Max SP HPx$_{\\rm{1C}}$, correspond to an hypothetical situation where the geochemical calculations incur zero computational cost.}\n\n\t\\begin{center}\n\t\t\\begin{tabular}{cccccccc}\n\t\t\t\\hline\n\t\t\t& & & & & & & \\\\\n\t\t\tML & BC & GS & HPx$_{\\rm{4C}}$ time (s) & SP HPx$_{\\rm{4C}}$ & Max SP HPx$_{\\rm{4C}}$ & SP HPx$_{\\rm{1C}}$ & Max SP HPx$_{\\rm{1C}}$ \\\\\n\t\t\tDNN & ADV & 61 $\\times$ 61 & 21,415 & 8.2 & 9.0 & 30.3 & 33.1\\\\\n\t\t\tkNN & ADV & 61 $\\times$ 61 & 21,415 & 7.9 & 9.0 & 28.9 & 33.1\\\\\n\t\t\tDNN & ADV & 121 $\\times$ 121 & 199,841 & 8.2 & 9.5 & 29.9 & 32.8\\\\\n\t\t\tkNN & ADV & 121 $\\times$ 121 & 199,841 & 8.5 & 9.5 & 31.3 & 32.8\\\\\n\t\t\t\\hline\n\t\t\\end{tabular}\n\t\\end{center}\n\n\t\\label{table4}\n\\end{table}\n\n\\FloatBarrier\n\n3.2 Al-C-Ca-S-Si Problem\n\\subsection{Al-C-Ca-S-Si Problem}\n\nFor this second cement system, the emulation problem consists of predicting at each time step of the RT simulation the (output) Al, C, Ca, S, Si, H and O aqueous concentrations (mol/kgw) from the (input) total amounts of Al, C, Ca, S and Si (mol). Here we focus on advective-dispersive transport only while similarly as for cement system 1, the considered domain sizes are 61 $\\times$ 61 and 121 $\\times$ 121. Moreover, the simulation time period is set to 6 years. \n\nAs mentioned earlier, for this higher-dimensional problem it is observed that the used scikit-learn kNN implementation becomes prohibitively slow compared to HPx$_{\\rm{4C}}$. We found that to get a good emulation accuracy the kNN training base needs to contain 1,000,000 samples (or more). This training base's size together with a 5-dimensional search space leads to an HPx$_{\\rm py}$-kNN reactive transport simulation time that is comparable to that of HPx$_{\\rm{4C}}$. Therefore, for this second cement system we built a custom kNN regressor around another kNN implementation contained in the FAISS package \\citep{faiss2017}. Our used FAISS variant allows for GPU computing and is much faster than scikit-learn for this cement system, but is slightly less accurate due to the use of an approximate rather than exact nearest neighbor search \\citep[see][for details]{faiss2017}.\n\n\\subsubsection{Training the DNN}\n\\label{train_res2}\n\nBuilding a good training set to perform a kNN search and learn the weights and biases of our DNN turned out to be a complicated task in this case. This because to make useful kNN predictions and/or learn an useful DNN, the training set must be sufficiently representative of the geochemical conditions encountered during the reactive transport simulation one wish to perform with HPx$_{\\rm py}$-kNN and the trained HPx$_{\\rm py}$-DNN. In contrast to cement system 1, creating the training set by sampling the $X$-space with a controlled randomness between predefined lower and upper bounds did not prove successful. We tried that strategy by drawing as much as 4,000,000 5-dimensional $\\textbf{x}$ vectors from the $X$-space using a Sobol low-discrepancy sequence \\citep[][]{Sobol1967, Joe-Kuo2003}. Such low-discrepancy sampling scheme covers the 5-dimensional hypercube more uniformly than LHS. Despite a good performance on the test set (not shown), the resulting DNN accuracy in reactive transport mode was never deemed satisfying. In other words, no satisfying ``global\" or ``universal\" DNN emulator could be devised for this cement system. This is probably caused by the fact for this problem, the input (5 total amounts) and output (7 aqueous concentrations) spaces are quite nonlinearly related and both cover 6 to 10 orders of magnitudes depending on the considered element. Therefore, we resorted to the alternative training strategy detailed below. The latter basically tries to grasp the complex correlations and higher-order dependencies that exist between the elements of $\\textbf{x}$ (total amounts, input space) for a given reactive transport simulation setup, in order to produce a training set that honors these between-input relationships.\n\n\\begin{itemize}\n\t\\item Perform a ``cheap\" full reactive transport simulation under the transport conditions and geochemistry of interest and collect the resulting $\\textbf{x}$ -$\\textbf{y}$  pairs of examples (for the considered grid nodes and time steps). Computational demand controls what domain size and simulation time period can be used for this cheap calculation. We used a modest 16 $\\times$ 16 domain and a simulation time period of 10 years. The associated HPx$_{\\rm{4C}}$ runtimes is 180 s.\n\t\n\t\\item Fit a kernel density estimator (KDE) with a Gaussian kernel to the collected $\\textbf{x}$ vectors (encapsulated in the $\\textbf{X}$ array) and generate a fixed number of new input vectors, $\\textbf{x}_{KDE}$. Then run PHREEQC for the $\\textbf{X}_{KDE}$ set to get the corresponding output set, $\\textbf{Y}_{KDE}$. Now apply the correction for porosity described in section \\ref{train_res1} to the $\\textbf{X}_{KDE}$ set and form the training set by merging the ensemble of $\\textbf{x}$-$\\textbf{y}$ pairs with that of the $\\textbf{x}_{KDE}$-$\\textbf{y}_{KDE}$ pairs. The number of produced unique examples by the considered cheap HPx$_{\\rm{4C}}$ simulations varied between 10,000 and 50,000. The KDE-based enrichment of this dataset was deemed necessary to provide more input variability, thereby avoiding overfitting of the trained DNN and improving the kNN accuracy, while still honoring the complex between-input relationships. The number of KDE-generated samples was set as to obtain a total training set size of 1,000,000 examples. A key component of the approach is the bandwith parameter of the Gaussian KDE kernel which controls how much the KDE-generated samples depart from the original ensemble. After limited trial and error, we fixed the kernel bandwith to 0.0025 for the considered case studies.\n\t\n\\end{itemize}\n\nThe scatter plots in Figure \\ref{fig9} illustrate our training set creation procedure. The orange dots depict the pairwise relationships observed between the 5 elements of $\\textbf{x}$ in the cheap simulation. The red and cyan dots in Figure \\ref{fig9} represent the KDE-generated samples with the selected bandwidth, before and after applying the correction for porosity, respectively. Training of the DNN is achieved using the ensemble of original and KDE-corrected input points. We refer to this kind of dataset as RT-based, since it is based on an albeit cheap, full RT simulation. Furthermore, we refer to the obtained DNN and kNN emulators as ``local\" emulators, since as opposed to the emulators constructed for cement system 1, the current emulators are only valid for the input conditions encapsulated in the RT-based training set.\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1.5cm}\\includegraphics[width=50pc]{Figure9.png}\n\t\\caption{Scatter plots of the complex relationships between the five considered inputs for (1) the computationally cheap RT simulation of cement system 2 (orange dots) performed with the original HPx code, (2) the corresponding sampled points by kernel density estimation (KDE) using the selected bandwith (turquoise dots) and, (3) the same sampled KDE points after porosity correction (red dots).}\n\t\\label{fig9}\n\\end{figure}\n\nTraining performance of the DNN emulator is presented in Figure \\ref{fig10}. Similarly as for cement system 1, about 90\\% of the available data was used for the actual training of the DNN while the remaining 10\\% were used a validation set to contol overfitting. Lastly, performance is evaluated for both the trained DNN and kNN emulators using an independent test set of 10,000 examples. Overall, the accuracy of our ``local \" DNN emulator for this RT-inspired dataset is rather large with $Q_2$ values always greater or equal than 0.998. Training performance of the corresponding local kNN emulator is equally good (not shown). Regarding speedup, for this problem single-threaded PHREEQC achieves about 210 geochemical calculations per second on our used Intel\\textsuperscript{\\textregistered} i7 CPU while the GPU-based DNN and kNN emulators are both about 3000 times faster when predicting the 10,000 test points at once .\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure10.png}\n\t\\caption{1-1 plots of local DNN emulation performance obtained for system 2 when the local DNN is trained using 1,000,000 samples. The $x$-axis and $y$-axis present the original and emulated 10,000 independent test data points, respectively. The RMSE and $Q_2$ coefficient denote the root-mean-square-error and coefficient of determination in testing mode, respectively, between the original and emulated 10,000 test data points.}\n\t\\label{fig10}\n\\end{figure}\n\n\\subsubsection{Reactive transport simulation}\n\\label{prob2res}\nOur ``local\" DNN performs rather well when applied to the 61 $\\times$ 61 grid size and a time period of 6 years (Figures \\ref{fig11} - \\ref{fig12}). This for all components but C, for which some localized deviations appear between original and emulated 2D concentration profiles towards the end of the simulation period (Figure \\ref{fig12}). When applied to the 121 $\\times$ 121 grid, HPx$_{\\rm py}$-DNN produces additional discrepancies for O, Si and Al towards the end of the simulation period (Figures \\ref{fig13} - \\ref{fig14}). Yet most of the observed artifacts could probably be smoothed out by using post-filtering. The associated speedups are listed in Table \\ref{table4}. These speedups are larger to those obtained for cement system 1, with values between 8 and 9 when evaluated against HPx$_{\\rm{4C}}$. These speedups represent about 85 \\% to 90 \\% of the maximum possible speedup (Table \\ref{table3}). Overall, these findings indicate that for the considered problem, our RT-based training of a local DNN only works is the training set is sufficiently representative of the particular geochemical conditions encountered in the computationally demanding simulations, which is arguably not easy to achieve. This limitation is further discussed in section \\ref{discussion}.\n\nWe note a more uniform behavior for HPx$_{\\rm py}$-kNN across grid sizes than for HPx$_{\\rm py}$-DNN. Here the results for the 121  $\\times$ 121 grid are only slightly less accurate than those associated with the 61 $ \\times$ 61 grid (see Figures \\ref{fig15} - \\ref{fig16} where for brevity we only show concentration profiles for the C Al and S components). Furthermore, whenever observed the discrepancies between original and emulated profiles are more regularly scattered than for HPx$_{\\rm py}$-DNN. Note also that herein too, post-filtering could likely smooth out a large part of these deviations. In addition, owing to the use of a GPU to achieve the kNN calculations, the speedup provided by HPx$_{\\rm py}$-kNN are as large as those provided by HPx$_{\\rm py}$-DNN (Table \\ref{table4}).\n\nWith respect to the emulated solid amounts, the HPx$_{\\rm py}$-DNN results look visually good for the the 61 $\\times$ 61 grid. This is shown in Figure \\ref{fig17} for the C, Al and S chemical components, while emulation of the H, O Ca and Si chemical components is globally of similar quality (not shown). Nevertheless, significant deviations appear towards the end of the simulation period for every chemical component (see Figure \\ref{fig18} for the C, Al and S chemical components, emulation of the H, O Ca and Si chemical components shows the same level of mismatch). The HPx$_{\\rm py}$-kNN predictions are also fairly accurate for the 61 $\\times$ 61 (see Figure \\ref{fig19} for the C, Al and S chemical components, emulation of the H, O Ca and Si chemical components exhibits a globally similar quality) while some discrepancies show up at the end of the simulation (Figure \\ref{fig20}). However, the mismatch is less pronounced than for HPx$_{\\rm py}$-DNN. Overall, HPx$_{\\rm py}$-kNN appears to be somewhat more robust than HPx$_{\\rm py}$-DNN for this cement system, while providing the same (large) speedup.\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure11.png}\n\t\\caption{2D concentration profiles obtained for cement system 2 after 2 and 6 (final time step) years. RTM means the original HPx$_{\\rm{4C}}$ model and TM+DNN denotes the Hydrus transport model coupled with our DNN geochemical emulator (HPx$_{\\rm py}$-DNN). The first to fourth row present profiles for $Ca^{\\rm{conc}}$, $Si^{\\rm{conc}}$, $H^{\\rm{conc}}$, and $O^{\\rm{conc}}$, respectively. The considered grid size is 61 $\\times$ 61.}\n\t\\label{fig11}\n\\end{figure}\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure12.png}\n\t\\caption{2D concentration profiles obtained for cement system 2 after 2 and 6 (final time step) years. RTM means the original HPx$_{\\rm{4C}}$ model and TM+DNN denotes the Hydrus transport model coupled with our DNN geochemical emulator (HPx$_{\\rm py}$-DNN). The first to third row present profiles for $C^{\\rm{conc}}$, $Al^{\\rm{conc}}$, and $S^{\\rm{conc}}$, respectively. The considered grid size is 61 $\\times$ 61.}\n\t\\label{fig12}\n\\end{figure}\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure13.png}\n\t\\caption{2D concentration profiles obtained for cement system 2 after 2 and 6 (final time step) years. RTM means the original HPx$_{\\rm{4C}}$ model and TM+DNN denotes the Hydrus transport model coupled with our DNN geochemical emulator (HPx$_{\\rm py}$-DNN). The first to fourth row present profiles for $Ca^{\\rm{conc}}$, $Si^{\\rm{conc}}$, $H^{\\rm{conc}}$, and $O^{\\rm{conc}}$, respectively. The considered grid size is 121 $\\times$ 121.}\n\t\\label{fig13}\n\\end{figure}\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure14.png}\n\t\\caption{2D concentration profiles obtained for cement system 2 after 2 and 6 (final time step) years. RTM means the original HPx$_{\\rm{4C}}$ model and TM+DNN denotes the Hydrus transport model coupled with our DNN geochemical emulator (HPx$_{\\rm py}$-DNN). The first to third row present profiles for $C^{\\rm{conc}}$, $Al^{\\rm{conc}}$, and $S^{\\rm{conc}}$, respectively. The considered grid size is 121 $\\times$ 121.}\n\t\\label{fig14}\n\\end{figure}\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure15.png}\n\t\\caption{2D concentration profiles obtained for cement system 2 after 2 and 6 (final time step) years. RTM means the original HPx$_{\\rm{4C}}$ model and TM+kNN denotes the Hydrus transport model coupled with our kNN geochemical emulator (HPx$_{\\rm py}$-kNN).  The first to third row present profiles for $C^{\\rm{conc}}$, $Al^{\\rm{conc}}$, and $S^{\\rm{conc}}$, respectively. The considered grid size is 61 $\\times$ 61.}\n\t\\label{fig15}\n\\end{figure}\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure16.png}\n\t\\caption{2D concentration profiles obtained for cement system 2 after 2 and 6 (final time step) years. RTM means the original HPx$_{\\rm{4C}}$ model and TM+kNN denotes the Hydrus transport model coupled with our kNN geochemical emulator (HPx$_{\\rm py}$-kNN). The first to third row present profiles for $C^{\\rm{conc}}$, $Al^{\\rm{conc}}$, and $S^{\\rm{conc}}$, respectively. The considered grid size is 121 $\\times$ 121.}\n\t\\label{fig16}\n\\end{figure}\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure17.png}\n\t\\caption{2D solid amount profiles of the C, Al and S chemical components of cement system 2 after 2 and 6 (final time step) years. RTM means the original HPx$_{\\rm{4C}}$ model and TM+DNN denotes the Hydrus transport model coupled with our DNN geochemical emulator (HPx$_{\\rm py}$-DNN). The first to third row present profiles for $C^{\\rm{solid}}$, $Al^{\\rm{solid}}$, and $S^{\\rm{solid}}$, respectively The considered grid size is 61 $\\times$ 61.}\n\t\\label{fig17}\n\\end{figure}\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure18.png}\n\t\\caption{2D solid amount profiles of the C, Al and S chemical components of cement system 2 after 2 and 6 (final time step) years. RTM means the original HPx$_{\\rm{4C}}$ model and TM+DNN denotes the Hydrus transport model coupled with our DNN geochemical emulator (HPx$_{\\rm py}$-DNN). The first to third row present profiles for $C^{\\rm{solid}}$, $Al^{\\rm{solid}}$, and $S^{\\rm{solid}}$, respectively The considered grid size is 121 $\\times$ 121.}\n\t\\label{fig18}\n\\end{figure}\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure19.png}\n\t\\caption{2D solid amount profiles of the C, Al and S chemical components of cement system 2 after 2 and 6 (final time step) years. RTM means the original HPx$_{\\rm{4C}}$ model and TM+kNN denotes the Hydrus transport model coupled with our kNN geochemical emulator (HPx$_{\\rm py}$-kNN). The first to third row present profiles for $C^{\\rm{solid}}$, $Al^{\\rm{solid}}$, and $S^{\\rm{solid}}$, respectively The considered grid size is 61 $\\times$ 61.}\n\t\\label{fig19}\n\\end{figure}\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure20.png}\n\t\\caption{2D solid amount profiles of the C, Al and S chemical components of cement system 2 after 2 and 6 (final time step) years. RTM means the original HPx$_{\\rm{4C}}$ model and TM+kNN denotes the Hydrus transport model coupled with our kNN geochemical emulator (HPx$_{\\rm py}$-kNN). The first to third row present profiles for $C^{\\rm{solid}}$, $Al^{\\rm{solid}}$, and $S^{\\rm{solid}}$, respectively The considered grid size is 121 $\\times$ 121.}\n\t\\label{fig20}\n\\end{figure}\n\n\\begin{table}[h!]\n\t\\caption{Speedups offered by the KNN and DNN emulators in HPx$_{\\rm py}$ for the reactive transport simulations considered for cement system 2. The HPx$_{\\rm{4C}}$ calculations involve the parallelization of PHREEQC over our 4 CPUs. The HPx$_{\\rm{1C}}$ calculations are performed on a single CPU. Both the kNN and DNN predictions make use of a GPU. ML signifies the used machine learning method for emulation, BC denotes the type of flow boundary conditions and GS is the grid size. The maximum possible speedups associated with HPx$_{\\rm{4C}}$ and HP$_{\\rm{1C}}$,  Max SP HPx$_{\\rm{4C}}$ and  Max SP HPx$_{\\rm{1C}}$, correspond to an hypothetical situation where the geochemical calculations incur zero computational cost.}\n\n\t\\begin{center}\n\t\t\\begin{tabular}{cccccccc}\n\t\t\t\\hline\n\t\t\t& & & & & & & \\\\\n\t\t\tML & BC & GS & HPx$_{\\rm{4C}}$ time (s) & SP HPx$_{\\rm{4C}}$ & Max SP HPx$_{\\rm{4C}}$ & SP HPx$_{\\rm{1C}}$ & Max SP HPx$_{\\rm{1C}}$ \\\\\n\t\t\tDNN & ADV & 61 $\\times$ 61 & 21,415 & 8.2 & 9.0 & 30.3 & 33.1\\\\\n\t\t\tkNN & ADV & 61 $\\times$ 61 & 21,415 & 7.9 & 9.0 & 28.9 & 33.1\\\\\n\t\t\tDNN & ADV & 121 $\\times$ 121 & 199,841 & 8.2 & 9.5 & 29.9 & 32.8\\\\\n\t\t\tkNN & ADV & 121 $\\times$ 121 & 199,841 & 8.5 & 9.5 & 31.3 & 32.8\\\\\n\t\t\t\\hline\n\t\t\\end{tabular}\n\t\\end{center}\n\n\t\\label{table4}\n\\end{table}\n\n\\FloatBarrier\n\n3.2.2 Reactive transport simulation\n\\subsubsection{Reactive transport simulation}\n\\label{prob2res}\nOur ``local\" DNN performs rather well when applied to the 61 $\\times$ 61 grid size and a time period of 6 years (Figures \\ref{fig11} - \\ref{fig12}). This for all components but C, for which some localized deviations appear between original and emulated 2D concentration profiles towards the end of the simulation period (Figure \\ref{fig12}). When applied to the 121 $\\times$ 121 grid, HPx$_{\\rm py}$-DNN produces additional discrepancies for O, Si and Al towards the end of the simulation period (Figures \\ref{fig13} - \\ref{fig14}). Yet most of the observed artifacts could probably be smoothed out by using post-filtering. The associated speedups are listed in Table \\ref{table4}. These speedups are larger to those obtained for cement system 1, with values between 8 and 9 when evaluated against HPx$_{\\rm{4C}}$. These speedups represent about 85 \\% to 90 \\% of the maximum possible speedup (Table \\ref{table3}). Overall, these findings indicate that for the considered problem, our RT-based training of a local DNN only works is the training set is sufficiently representative of the particular geochemical conditions encountered in the computationally demanding simulations, which is arguably not easy to achieve. This limitation is further discussed in section \\ref{discussion}.\n\nWe note a more uniform behavior for HPx$_{\\rm py}$-kNN across grid sizes than for HPx$_{\\rm py}$-DNN. Here the results for the 121  $\\times$ 121 grid are only slightly less accurate than those associated with the 61 $ \\times$ 61 grid (see Figures \\ref{fig15} - \\ref{fig16} where for brevity we only show concentration profiles for the C Al and S components). Furthermore, whenever observed the discrepancies between original and emulated profiles are more regularly scattered than for HPx$_{\\rm py}$-DNN. Note also that herein too, post-filtering could likely smooth out a large part of these deviations. In addition, owing to the use of a GPU to achieve the kNN calculations, the speedup provided by HPx$_{\\rm py}$-kNN are as large as those provided by HPx$_{\\rm py}$-DNN (Table \\ref{table4}).\n\nWith respect to the emulated solid amounts, the HPx$_{\\rm py}$-DNN results look visually good for the the 61 $\\times$ 61 grid. This is shown in Figure \\ref{fig17} for the C, Al and S chemical components, while emulation of the H, O Ca and Si chemical components is globally of similar quality (not shown). Nevertheless, significant deviations appear towards the end of the simulation period for every chemical component (see Figure \\ref{fig18} for the C, Al and S chemical components, emulation of the H, O Ca and Si chemical components shows the same level of mismatch). The HPx$_{\\rm py}$-kNN predictions are also fairly accurate for the 61 $\\times$ 61 (see Figure \\ref{fig19} for the C, Al and S chemical components, emulation of the H, O Ca and Si chemical components exhibits a globally similar quality) while some discrepancies show up at the end of the simulation (Figure \\ref{fig20}). However, the mismatch is less pronounced than for HPx$_{\\rm py}$-DNN. Overall, HPx$_{\\rm py}$-kNN appears to be somewhat more robust than HPx$_{\\rm py}$-DNN for this cement system, while providing the same (large) speedup.\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure11.png}\n\t\\caption{2D concentration profiles obtained for cement system 2 after 2 and 6 (final time step) years. RTM means the original HPx$_{\\rm{4C}}$ model and TM+DNN denotes the Hydrus transport model coupled with our DNN geochemical emulator (HPx$_{\\rm py}$-DNN). The first to fourth row present profiles for $Ca^{\\rm{conc}}$, $Si^{\\rm{conc}}$, $H^{\\rm{conc}}$, and $O^{\\rm{conc}}$, respectively. The considered grid size is 61 $\\times$ 61.}\n\t\\label{fig11}\n\\end{figure}\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure12.png}\n\t\\caption{2D concentration profiles obtained for cement system 2 after 2 and 6 (final time step) years. RTM means the original HPx$_{\\rm{4C}}$ model and TM+DNN denotes the Hydrus transport model coupled with our DNN geochemical emulator (HPx$_{\\rm py}$-DNN). The first to third row present profiles for $C^{\\rm{conc}}$, $Al^{\\rm{conc}}$, and $S^{\\rm{conc}}$, respectively. The considered grid size is 61 $\\times$ 61.}\n\t\\label{fig12}\n\\end{figure}\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure13.png}\n\t\\caption{2D concentration profiles obtained for cement system 2 after 2 and 6 (final time step) years. RTM means the original HPx$_{\\rm{4C}}$ model and TM+DNN denotes the Hydrus transport model coupled with our DNN geochemical emulator (HPx$_{\\rm py}$-DNN). The first to fourth row present profiles for $Ca^{\\rm{conc}}$, $Si^{\\rm{conc}}$, $H^{\\rm{conc}}$, and $O^{\\rm{conc}}$, respectively. The considered grid size is 121 $\\times$ 121.}\n\t\\label{fig13}\n\\end{figure}\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure14.png}\n\t\\caption{2D concentration profiles obtained for cement system 2 after 2 and 6 (final time step) years. RTM means the original HPx$_{\\rm{4C}}$ model and TM+DNN denotes the Hydrus transport model coupled with our DNN geochemical emulator (HPx$_{\\rm py}$-DNN). The first to third row present profiles for $C^{\\rm{conc}}$, $Al^{\\rm{conc}}$, and $S^{\\rm{conc}}$, respectively. The considered grid size is 121 $\\times$ 121.}\n\t\\label{fig14}\n\\end{figure}\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure15.png}\n\t\\caption{2D concentration profiles obtained for cement system 2 after 2 and 6 (final time step) years. RTM means the original HPx$_{\\rm{4C}}$ model and TM+kNN denotes the Hydrus transport model coupled with our kNN geochemical emulator (HPx$_{\\rm py}$-kNN).  The first to third row present profiles for $C^{\\rm{conc}}$, $Al^{\\rm{conc}}$, and $S^{\\rm{conc}}$, respectively. The considered grid size is 61 $\\times$ 61.}\n\t\\label{fig15}\n\\end{figure}\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure16.png}\n\t\\caption{2D concentration profiles obtained for cement system 2 after 2 and 6 (final time step) years. RTM means the original HPx$_{\\rm{4C}}$ model and TM+kNN denotes the Hydrus transport model coupled with our kNN geochemical emulator (HPx$_{\\rm py}$-kNN). The first to third row present profiles for $C^{\\rm{conc}}$, $Al^{\\rm{conc}}$, and $S^{\\rm{conc}}$, respectively. The considered grid size is 121 $\\times$ 121.}\n\t\\label{fig16}\n\\end{figure}\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure17.png}\n\t\\caption{2D solid amount profiles of the C, Al and S chemical components of cement system 2 after 2 and 6 (final time step) years. RTM means the original HPx$_{\\rm{4C}}$ model and TM+DNN denotes the Hydrus transport model coupled with our DNN geochemical emulator (HPx$_{\\rm py}$-DNN). The first to third row present profiles for $C^{\\rm{solid}}$, $Al^{\\rm{solid}}$, and $S^{\\rm{solid}}$, respectively The considered grid size is 61 $\\times$ 61.}\n\t\\label{fig17}\n\\end{figure}\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure18.png}\n\t\\caption{2D solid amount profiles of the C, Al and S chemical components of cement system 2 after 2 and 6 (final time step) years. RTM means the original HPx$_{\\rm{4C}}$ model and TM+DNN denotes the Hydrus transport model coupled with our DNN geochemical emulator (HPx$_{\\rm py}$-DNN). The first to third row present profiles for $C^{\\rm{solid}}$, $Al^{\\rm{solid}}$, and $S^{\\rm{solid}}$, respectively The considered grid size is 121 $\\times$ 121.}\n\t\\label{fig18}\n\\end{figure}\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure19.png}\n\t\\caption{2D solid amount profiles of the C, Al and S chemical components of cement system 2 after 2 and 6 (final time step) years. RTM means the original HPx$_{\\rm{4C}}$ model and TM+kNN denotes the Hydrus transport model coupled with our kNN geochemical emulator (HPx$_{\\rm py}$-kNN). The first to third row present profiles for $C^{\\rm{solid}}$, $Al^{\\rm{solid}}$, and $S^{\\rm{solid}}$, respectively The considered grid size is 61 $\\times$ 61.}\n\t\\label{fig19}\n\\end{figure}\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure20.png}\n\t\\caption{2D solid amount profiles of the C, Al and S chemical components of cement system 2 after 2 and 6 (final time step) years. RTM means the original HPx$_{\\rm{4C}}$ model and TM+kNN denotes the Hydrus transport model coupled with our kNN geochemical emulator (HPx$_{\\rm py}$-kNN). The first to third row present profiles for $C^{\\rm{solid}}$, $Al^{\\rm{solid}}$, and $S^{\\rm{solid}}$, respectively The considered grid size is 121 $\\times$ 121.}\n\t\\label{fig20}\n\\end{figure}\n\n\\begin{table}[h!]\n\t\\caption{Speedups offered by the KNN and DNN emulators in HPx$_{\\rm py}$ for the reactive transport simulations considered for cement system 2. The HPx$_{\\rm{4C}}$ calculations involve the parallelization of PHREEQC over our 4 CPUs. The HPx$_{\\rm{1C}}$ calculations are performed on a single CPU. Both the kNN and DNN predictions make use of a GPU. ML signifies the used machine learning method for emulation, BC denotes the type of flow boundary conditions and GS is the grid size. The maximum possible speedups associated with HPx$_{\\rm{4C}}$ and HP$_{\\rm{1C}}$,  Max SP HPx$_{\\rm{4C}}$ and  Max SP HPx$_{\\rm{1C}}$, correspond to an hypothetical situation where the geochemical calculations incur zero computational cost.}\n\n\t\\begin{center}\n\t\t\\begin{tabular}{cccccccc}\n\t\t\t\\hline\n\t\t\t& & & & & & & \\\\\n\t\t\tML & BC & GS & HPx$_{\\rm{4C}}$ time (s) & SP HPx$_{\\rm{4C}}$ & Max SP HPx$_{\\rm{4C}}$ & SP HPx$_{\\rm{1C}}$ & Max SP HPx$_{\\rm{1C}}$ \\\\\n\t\t\tDNN & ADV & 61 $\\times$ 61 & 21,415 & 8.2 & 9.0 & 30.3 & 33.1\\\\\n\t\t\tkNN & ADV & 61 $\\times$ 61 & 21,415 & 7.9 & 9.0 & 28.9 & 33.1\\\\\n\t\t\tDNN & ADV & 121 $\\times$ 121 & 199,841 & 8.2 & 9.5 & 29.9 & 32.8\\\\\n\t\t\tkNN & ADV & 121 $\\times$ 121 & 199,841 & 8.5 & 9.5 & 31.3 & 32.8\\\\\n\t\t\t\\hline\n\t\t\\end{tabular}\n\t\\end{center}\n\n\t\\label{table4}\n\\end{table}\n\n\\FloatBarrier\n\n",
        "label": "fig12",
        "Descriptive_question1": "Which chemical component is represented in the first row of figure_12?",
        "Descriptive_question2": "What is the grid size used in the simulation shown in figure_12?",
        "Reasoning_question1": "How does the concentration profile of Al change between 2 and 6 years in figure_12, and what might this indicate about the chemical behavior over time?",
        "Reasoning_question2": "Comparing the RTM and TM+DNN models in figure_12, what can be inferred about the accuracy of the DNN geochemical emulator for predicting S concentration profiles after 6 years?",
        "Descriptive_answer1": "Ca^{\\rm{conc}}",
        "Descriptive_answer2": "61 \\times 61",
        "Reasoning_answer1": "In figure_12, the Al concentration profiles (second row) appear to change between 2 and 6 years, showing variations in concentration magnitude and spatial distribution. This suggests that Al undergoes dynamic chemical processes over time, potentially involving dissolution or precipitation reactions affecting its concentration within the system.",
        "Reasoning_answer2": "Comparing the RTM (original HPx$_{\\rm{4C}}$ model) and TM+DNN (Hydrus transport model with DNN emulator) concentration profiles of S (third row) after 6 years, the profiles are visually quite similar, indicating that the DNN geochemical emulator accurately predicts the S concentration distribution. Any small discrepancies may be due to localized deviations, but overall the DNN performs well in replicating the original model's results."
    },
    {
        "paper_id": "2107.07598.json",
        "image_id": "figure_13",
        "image_path": [
            "/home/yz979/scratch/chengye/scimolmo/test-images-tables/dataset_imagestables/2107.07598/images/Figure13.png"
        ],
        "caption": "2D concentration profiles obtained for cement system 2 after 2 and 6 (final time step) years. RTM means the original HPx$_{\\rm{4C}}$ model and TM+DNN denotes the Hydrus transport model coupled with our DNN geochemical emulator (HPx$_{\\rm py}$-DNN). The first to fourth row present profiles for $Ca^{\\rm{conc}}$, $Si^{\\rm{conc}}$, $H^{\\rm{conc}}$, and $O^{\\rm{conc}}$, respectively. The considered grid size is 121 $\\times$ 121.",
        "classify": "Chart",
        "section_info": "3 Results\n\\section{Results}\n\\label{results}\n\n\\subsection{Ca-Si Problem}\nFor this first cement system, the emulation problem consists of predicting at each time step of the RT simulation the (output) Ca, Si, H and O aqueous concentrations (mol/kg of water or mol/kgw) from the (input) total amounts of Ca and Si (mol). \n\n\\subsubsection{Training the emulators}\n\\label{train_res1}\nHere the kNN and DNN emulators are firstly trained using a set of 400,000 test examples for both. This training set is obtained by randomly sampling the two-dimensional input space by latin hypercube sampling (LHS) between $\\left[0,0\\right]$ and $\\rm{\\left[Ca^{tot}_{max},Si^{tot}_{max}\\right]}$, and running PHREEQC for each input sample, $\\rm{\\textbf{x}_i = \\left[Ca^{tot}_i, Si^{tot}_i\\right]}$ to get the corresponding output vectors, $\\rm{\\textbf{y}_i = \\left[Ca^{conc}_i,Si^{conc}_i,H^{conc}_i,O^{conc}_i\\right]}$. The upper bounds, $\\rm{Ca^{tot}_{max}}$ and $\\rm{Si^{tot}_{max}}$ are defined based on a cheap full RT simulation with advective-dispersive transport using a small 1D domain of 51 nodes. It is worth noting that the total amounts of $\\textbf{x}_i$, corresponding to the PHREEQC-simulated concentrations, $\\textbf{y}_i$, have to be corrected for the different amount of water between the training set and the transport simulations. Doing so, it turns out that about 20 \\% of the post-corrected $x_i$ values exceed their pre-defined upper bounds and these excessively large values need to be filtered out. Creating the 400,000 training examples thus required about 500,000 PHREEQC runs. As stated earlier, for this problem single-threaded PHREEQC performs about 670 geochemical calculations per second on our used Intel\\textsuperscript{\\textregistered} i7 CPU.\n\nWith respect to kNN, the tuning parameters are the number of neighbors, $k$, the type of distance measure, and the interpolation technique. We simply used the default settings: $k = 5$, euclidean distance and inverse-distance interpolation. Regarding training of the DNN, the 400,000 sample were split between the training set itself (90 \\% of the data) and a validation set (10 \\% of the data). The latter serves to monitor the evolution of the selected mean squared error loss function on samples that are not used for training, thereby detecting potential overfitting. If the validation loss stops decreasing before the fixed number of epochs has been completed, then  training is stopped. Importantly, the emulation is achieved in log-space for both the input, $X$ and output $Y$, domains. This because total amounts and concentrations of the involved components typically cover many orders of magnitudes (up to 10 orders or more). Using a DNN also requires some form of data normalization or standardization. Here both the $\\rm{log\\left(\\textbf{x}_i\\right)}$ and $\\rm{log\\left(\\textbf{y}_i\\right)}$ vectors are standardized around 0 with standard deviation of 1.\n\nFigure \\ref{fig1} illustrates the trained emulators' performance for geochemical predictions using an independent test set that comprises 10,000 test examples. Both kNN (Figures \\ref{fig1}a - d) and DNN (Figures \\ref{fig1}e - h) appear to be rather accurate . DNN also shows a slight degradation for the larger concentration values  (Figures \\ref{fig1}e - h). The latter is likely due to the combination of a small proportion of large concentration values in the training set with the log-transformation that implicitly pushes the DNN to try harder to fit the smaller concentrations during training. Regarding speedup and as written earlier, for this setup the single-threaded kNN method is 300 times faster than single-threaded PHREEQC for predicting the 10,000 concentration vectors all at once. The computational savings allowed by the DNN emulator when ran on our GPU is higher with a speedup as large as 4000 for predicting the same 10,000 concentration vectors all at once. \n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{0cm}\\includegraphics[width=35pc]{Figure1.png}\n\t\\caption{1-1 plots of the kNN (subfigures (a) - (d)) and DNN (subfigures (e) - (h)) emulators' performance obtained for system 1 when the kNN training base contains 400,000 samples and the DNN is trained using the same 400,000 samples. Here ``true\" means the original PHREEQC-simulated data and ``predicted\" denotes the emulated (that is, kNN-simulated and DNN-simulated) data. Hence, the $x$-axis and $y$-axis present the original and emulated 10,000 independent test data points, respectively. The RMSE and $Q_2$ coefficient denote the root-mean-square-error and coefficient of determination in testing mode, respectively, between the original and emulated 10,000 test data points.}\n\t\\label{fig1}\n\\end{figure}\n\nTo test the sensitivity of the emulators' performance to the training set size, training was also performed using reduced training sets comprising 100,000, 10,000, 1000 and 100 samples, respectively. It seen that the DNN performance achieved  when using 10,000 training samples is virtually the same as that obtained when using 400,000 training samples (Figure \\ref{fig2} and Table \\ref{table2}). It is only for training sets smaller than 1000 samples that the DNN performance starts to degrade significantly (Table \\ref{table2}). In contrast, the behavior of kNN appears to markedly decrease as the training set gets smaller (Figure \\ref{fig2} and Table \\ref{table1}).\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{0cm}\\includegraphics[width=35pc]{Figure2.png}\n\t\\caption{1-1 plots of the kNN (subfigures (a) - (d)) and DNN (subfigures (e) - (h)) emulators' performance obtained for system 1 when the kNN training base contains 10,000 samples and the DNN is trained using the same 10,000 samples. Here ``true\" means the original PHREEQC-simulated data and ``predicted\" denotes the emulated (that is, kNN-simulated and DNN-simulated) data. Hence, the $x$-axis and $y$-axis present the original and emulated 10,000 independent test data points, respectively. The RMSE and $Q_2$ coefficient denote the root-mean-square-error and coefficient of determination in testing mode, respectively, between the original and emulated 10,000 test data points.}\n\t\\label{fig2}\n\\end{figure}\n\n\\begin{table}[h!]\n\t\\caption{Performance of the DNN and kNN emulators for cement system 1 and different training set sizes. For brevity, only the results for Ca$^{conc}$ and Si$^{conc}$ are shown. The units are mol per kg of water (mol/kgw). ML refers to the type of emulator, TR signifies the size of the training set, and the RMSE and $Q_2$ coefficient denote the root-mean-square-error and coefficient of determination in testing mode, respectively, between the original and emulated 10,000 test data points.}\n\t\\begin{center}\n\t\t\\begin{tabular}{cccccccc}\n\t\t\t\\hline\n\t\t\t& & & & & \\\\\n\t\t\tML & TR & RMSE - Ca$^{conc}$ & $Q_2$ - Ca$^{conc}$ & RMSE - Si$^{conc}$ & $Q_2$ - Si$^{conc}$\\\\\n\t\t\tDNN & 4 $\\times$ 10$^{5}$ & 4.50 $\\times$ 10$^{-5}$ & 0.9999 & 1.21 $\\times$ 10$^{-6}$ & 0.9999\\\\\n\t\t\tDNN & 1 $\\times$ 10$^{5}$ & 5.06 $\\times$ 10$^{-5}$ & 0.9999 & 1.30 $\\times$ 10$^{-6}$ & 0.9999\\\\\n\t\t\tDNN & 1 $\\times$ 10$^{4}$ & 4.57 $\\times$ 10$^{-5}$ & 0.9999 & 1.48 $\\times$ 10$^{-6}$ & 0.9999\\\\\n\t\t\tDNN & 1 $\\times$ 10$^{3}$ & 8.02 $\\times$ 10$^{-5}$ & 0.9998 & 3.31 $\\times$ 10$^{-6}$ & 0.9994\\\\\n\t\t\tDNN & 1 $\\times$ 10$^{2}$ & 47.1 $\\times$ 10$^{-5}$ & 0.9935 & 9.02 $\\times$ 10$^{-6}$ & 0.9959\\\\\n\t\t\tkNN & 4 $\\times$ 10$^{5}$ & 2.52 $\\times$ 10$^{-5}$ & 1.0000 & 1.59 $\\times$ 10$^{-6}$ & 0.9999\\\\\n\t\t\tkNN & 1 $\\times$ 10$^{5}$ & 6.27 $\\times$ 10$^{-5}$ & 0.9999 & 4.20 $\\times$ 10$^{-6}$ & 0.9991\\\\\n\t\t\tkNN & 1 $\\times$ 10$^{4}$ & 14.1 $\\times$ 10$^{-5}$ & 0.9994 & 13.5 $\\times$ 10$^{-6}$ & 0.9908\\\\\n\t\t\tkNN & 1 $\\times$ 10$^{3}$ & 43.9 $\\times$ 10$^{-5}$ & 0.9944 & 74.6 $\\times$ 10$^{-6}$ & 0.7180\\\\\n\t\t\tkNN & 1 $\\times$ 10$^{2}$  & 96.4 $\\times$ 10$^{-5}$ & 0.9729 & 55.0 $\\times$ 10$^{-6}$ & 0.8468\\\\\n\t\t\\end{tabular}\n\t\\end{center}\n\t\\label{table2}\n\\end{table}\n\n\\FloatBarrier\n\n\\subsubsection{Reactive transport simulation}\n\\label{prob12Dres}\n\nThis section focuses on reactive transport simulations with HPx$_{\\rm py}$ within cement system 1, under both advective-dispersive and diffusive transport conditions. As written above, the domain sizes are both 61 $\\times$ 61 and 121 $\\times$ 121 for the advection-dispersion case and, because of computational constraints, solely 61 $\\times$ 61  for the diffusion case. In addition, the simulation time period is 2 years for the advection-dispersion case and 1 year for the diffusion case. Figures \\ref{fig3} and \\ref{fig4} present times series of original and emulated Ca, Si, H and O concentrations at 5 locations within the 2D domain for advective-dispersive transport conditions, for both our kNN-based (HPx$_{\\rm py}$-kNN) and DNN-based (HPx$_{\\rm py}$-DNN) reactive transport codes. It is seen that HPx$_{\\rm py}$-kNN and HPx$_{\\rm py}$-DNN both induce a quite good simulation accuracy. Also, the results for the diffusive transport case are of similarly good quality (not shown). Figures \\ref{fig5} - \\ref{fig6} provide more insights into the HPx$_{\\rm py}$-kNN and HPx$_{\\rm py}$-DNN performances by displaying 2D Ca, Si, H and O concentration profiles at a given time. For each experiment and chemical component, this time is selected as to be well representative of the simulated dynamics. It is observed that the original and emulated images are visually almost indistinguishable for the advection-dispersion case (Figure \\ref{fig5}). For the diffusion case, the emulators also perform quite well for Ca, H and O (Figures\\ref{fig6}a - c, g - i and j - l), while some slight to moderate discrepancies appear at the concentration front for Si (Figures \\ref{fig6}d - f). Notwithstanding, the Si concentration remains globally well predicted. Furthermore, Figures \\ref{fig7} - \\ref{fig8} present the original and emulated 2D solid amount profiles corresponding to Figures \\ref{fig5} - \\ref{fig6}. The original solid amount profiles are overall well approximated by HPx$_{\\rm py}$-kNN and HPx$_{\\rm py}$-DNN for the advection-dispersion case (Figure \\ref{fig7}), even though some mismatch appears at the border of the fully depleted zone for the H component. As of the diffusion case (Figure \\ref{fig8}), the same kind of mismatch is observed for the emulated solid amounts of H by HPx$_{\\rm py}$-kNN while the emulated profiles by HPx$_{\\rm py}$-DNN show some more discrepancies. Though we decided to present raw emulation results, we would like to stress that some if not all of the observed artifacts could likely be smoothed out by using some post-filtering such as median filtering.\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure3.png}\n\t\\caption{Time series of original (RTM, solid green lines) and HPx$_{\\rm py}$-kNN emulated (TM+kNN, dashed orange lines) concentrations (mol/kg) of Ca, H, O and Si at selected observations points for cement system 1 and advective-dispersive transport. Obs. 1 - 5 denote the selected observation points, with the following $\\left[x,y\\right]$ locations (in cm). Obs. 1:  $\\left[0.5, 2.5\\right]$, Obs. 2: $\\left[1, 2\\right]$, Obs. 3: $\\left[2, 2\\right]$, Obs. 4: $\\left[1, 1\\right]$, Obs. 5: $\\left[2, 1\\right]$. The results for the 121 $\\times$ 121 grid size are rather similar.}\n\t\\label{fig3}\n\\end{figure}\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure4.png}\n\t\\caption{Time series of original (RTM, solid green lines) and HPx$_{\\rm py}$-DNN emulated (TM+DNN, dashed orange lines) concentrations (mol/kg) of Ca, H, O and Si at selected observations points for cement system 1 and advective-dispersive transport. Obs. 1 - 5 denote the selected observation points, with the following $\\left[x,y\\right]$ locations (in cm). Obs. 1:  $\\left[0.5, 2.5\\right]$, Obs. 2: $\\left[1, 2\\right]$, Obs. 3: $\\left[2, 2\\right]$, Obs. 4: $\\left[1, 1\\right]$, Obs. 5: $\\left[2, 1\\right]$. The results for the 121 $\\times$ 121 grid size are rather similar.}\n\t\\label{fig4}\n\\end{figure}\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure5.png}\n\t\\caption{2D concentration profiles obtained for cement system 1 at the end of the 2-year simulation performed for the advection-dispersion case. RTM means the original HP$_{\\rm{4C}}$ model, TM+kNN denotes the Hydrus transport model coupled with our kNN geochemical emulator (HPx$_{\\rm py}$-kNN) and TM+DNN signifies the Hydrus transport model coupled with our DNN geochemical emulator (HPx$_{\\rm py}$-DNN). The first to fourth row present profiles for $Ca^{\\rm{conc}}$, $Si^{\\rm{conc}}$, $H^{\\rm{conc}}$, and $O^{\\rm{conc}}$, respectively. The considered grid size is 61 $\\times$ 61. The results for the 121 $\\times$ 121 grid are rather similar.}\n\t\\label{fig5}\n\\end{figure}\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure6.png}\n\t\\caption{2D concentration profiles obtained for cement system 1 at the final time step of the 1-year simulation performed for the diffusion case. RTM means the original HP$_{\\rm{4C}}$ model, TM+kNN denotes the Hydrus transport model coupled with our kNN geochemical emulator (HPx$_{\\rm py}$-kNN) and TM+DNN signifies the Hydrus transport model coupled with our DNN geochemical emulator (HPx$_{\\rm py}$-DNN). The first to fourth row present profiles for $Ca^{\\rm{conc}}$, $Si^{\\rm{conc}}$, $H^{\\rm{conc}}$, and $O^{\\rm{conc}}$, respectively. The considered grid size is 61 $\\times$ 61.}\n\t\\label{fig6}\n\\end{figure}\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure7.png}\n\t\\caption{2D solid amount profiles obtained for cement system 1 at the end of the 2-year simulation performed for the advection-dispersion case. RTM means the original HP$_{\\rm{4C}}$ model, TM+kNN denotes the Hydrus transport model coupled with our kNN geochemical emulator (HPx$_{\\rm py}$-kNN) and TM+DNN signifies the Hydrus transport model coupled with our DNN geochemical emulator (HPx$_{\\rm py}$-DNN). The first to fourth row present profiles for $Ca^{\\rm{solid}}$, $Si^{\\rm{solid}}$, $H^{\\rm{solid}}$, and $O^{\\rm{solid}}$, respectively. The considered grid size is 61 $\\times$ 61. The results for the 121 $\\times$ 121 grid are rather similar.}\n\t\\label{fig7}\n\\end{figure}\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure8.png}\n\t\\caption{2D solid amount profiles obtained for cement system 1 at the final time step of the 1-year simulation performed for the diffusion case. RTM means the original HP$_{\\rm{4C}}$ model, TM+kNN denotes the Hydrus transport model coupled with our kNN geochemical emulator (HPx$_{\\rm py}$-kNN) and TM+DNN signifies the Hydrus transport model coupled with our DNN geochemical emulator (HPx$_{\\rm py}$-DNN). The first to fourth row present profiles for $Ca^{\\rm{solid}}$, $Si^{\\rm{solid}}$, $H^{\\rm{solid}}$, and $O^{\\rm{solid}}$, respectively. The considered grid size is 61 $\\times$ 61.}\n\t\\label{fig8}\n\\end{figure}\n\nThe speedups associated with the considered problem are detailed in Table \\ref{table3}. It is noted that the GPU-based DNN emulator allows for a speedup that is close to optimal. Indeed, The DNN speedups overall represent 85 \\% to 95 \\% of the maximum possible speedups (that is, speedups that would be obtained if the geochemcical calculations would come at no cost at all). The speedups associated with single-threaded kNN remain substantial but only amount to 57 \\% - 65 \\% of the corresponding maximum speedups. As detailed in section \\ref{train_res1}, the used kNN and DNN implementations are found to be respectively 300 and 4000 faster than single-threaded PHREEQC when predicting 10,000 points all one once for this geochemical system. Based on these numbers one could have expected the achieved speedups to represent say 90 \\% (kNN) or 99 \\% (DNN) of the maximum possible ones. A large part of the gaps between achieved and maximum possible speedups is thus likely caused by the time required for communicating and exchanging data between the main C/C++ code and the Python-based emulators.\n\n\\begin{table}[h!]\n\t\\caption{Speedups offered by the KNN and DNN emulators in HPx$_{\\rm py}$ for the reactive transport simulations considered for cement system 1. The HPx$_{\\rm{4C}}$ calculations involve the parallelization of PHREEQC over our 4 CPUs. The HPx$_{\\rm{1C}}$ calculations are performed on a single CPU. The kNN predictions are performed on a single CPU using the scikit-learn implementation while the DNN predictions make use of our GPU. ML signifies the used machine learning method for emulation, TC denotes transport conditions (ADV: advection-dispersion, DIF: diffusion) and GS is the grid size. The maximum possible speedups associated with HPx$_{\\rm{4C}}$ and HPx$_{\\rm{1C}}$,  Max SP HPx$_{\\rm{4C}}$ and  Max SP HPx$_{\\rm{1C}}$, correspond to an hypothetical situation where the geochemical calculations incur zero computational cost.}\n\n\t\\begin{center}\n\t\t\\begin{tabular}{cccccccc}\n\t\t\t\\hline\n\t\t\t& & & & & & & \\\\\n\t\t\tML & TC & GS & HPx$_{\\rm{4C}}$ time (s) & SP HPx$_{\\rm{4C}}$ & Max SP HPx$_{\\rm{4C}}$ & SP HPx$_{\\rm{1C}}$ & Max SP HPx$_{\\rm{1C}}$ \\\\\n\t\t\tDNN & ADV & 61 $\\times$ 61 & 3189 & 6.8 & 7.7 & 24.5 & 28.5 \\\\\n\t\t\tkNN & ADV & 61 $\\times$ 61 & 3189 & 5.0 & 7.7 & 18.0 & 28.5 \\\\\n\t\t\tDNN & ADV & 121 $\\times$ 121 & 23,337 & 5.0 & 5.2 & 17.0 & 18.0 \\\\\n\t\t\tkNN & ADV & 121 $\\times$ 121 & 23,337 & 3.4 & 5.2 & 11.6 & 18.0 \\\\\n\t\t\tDNN & DIF & 61 $\\times$ 61 & 25,448 & 4.2 & 4.9 & 16.2 & 19.1 \\\\\n\t\t\tkNN & DIF & 61 $\\times$ 61 & 25,448 & 2.8 & 4.9 & 10.8 & 19.1 \\\\\n\t\t\t\\hline\n\t\t\\end{tabular}\n\t\\end{center}\n\n\t\\label{table3}\n\\end{table}\n\n\\FloatBarrier\n\n\\subsection{Al-C-Ca-S-Si Problem}\n\nFor this second cement system, the emulation problem consists of predicting at each time step of the RT simulation the (output) Al, C, Ca, S, Si, H and O aqueous concentrations (mol/kgw) from the (input) total amounts of Al, C, Ca, S and Si (mol). Here we focus on advective-dispersive transport only while similarly as for cement system 1, the considered domain sizes are 61 $\\times$ 61 and 121 $\\times$ 121. Moreover, the simulation time period is set to 6 years. \n\nAs mentioned earlier, for this higher-dimensional problem it is observed that the used scikit-learn kNN implementation becomes prohibitively slow compared to HPx$_{\\rm{4C}}$. We found that to get a good emulation accuracy the kNN training base needs to contain 1,000,000 samples (or more). This training base's size together with a 5-dimensional search space leads to an HPx$_{\\rm py}$-kNN reactive transport simulation time that is comparable to that of HPx$_{\\rm{4C}}$. Therefore, for this second cement system we built a custom kNN regressor around another kNN implementation contained in the FAISS package \\citep{faiss2017}. Our used FAISS variant allows for GPU computing and is much faster than scikit-learn for this cement system, but is slightly less accurate due to the use of an approximate rather than exact nearest neighbor search \\citep[see][for details]{faiss2017}.\n\n\\subsubsection{Training the DNN}\n\\label{train_res2}\n\nBuilding a good training set to perform a kNN search and learn the weights and biases of our DNN turned out to be a complicated task in this case. This because to make useful kNN predictions and/or learn an useful DNN, the training set must be sufficiently representative of the geochemical conditions encountered during the reactive transport simulation one wish to perform with HPx$_{\\rm py}$-kNN and the trained HPx$_{\\rm py}$-DNN. In contrast to cement system 1, creating the training set by sampling the $X$-space with a controlled randomness between predefined lower and upper bounds did not prove successful. We tried that strategy by drawing as much as 4,000,000 5-dimensional $\\textbf{x}$ vectors from the $X$-space using a Sobol low-discrepancy sequence \\citep[][]{Sobol1967, Joe-Kuo2003}. Such low-discrepancy sampling scheme covers the 5-dimensional hypercube more uniformly than LHS. Despite a good performance on the test set (not shown), the resulting DNN accuracy in reactive transport mode was never deemed satisfying. In other words, no satisfying ``global\" or ``universal\" DNN emulator could be devised for this cement system. This is probably caused by the fact for this problem, the input (5 total amounts) and output (7 aqueous concentrations) spaces are quite nonlinearly related and both cover 6 to 10 orders of magnitudes depending on the considered element. Therefore, we resorted to the alternative training strategy detailed below. The latter basically tries to grasp the complex correlations and higher-order dependencies that exist between the elements of $\\textbf{x}$ (total amounts, input space) for a given reactive transport simulation setup, in order to produce a training set that honors these between-input relationships.\n\n\\begin{itemize}\n\t\\item Perform a ``cheap\" full reactive transport simulation under the transport conditions and geochemistry of interest and collect the resulting $\\textbf{x}$ -$\\textbf{y}$  pairs of examples (for the considered grid nodes and time steps). Computational demand controls what domain size and simulation time period can be used for this cheap calculation. We used a modest 16 $\\times$ 16 domain and a simulation time period of 10 years. The associated HPx$_{\\rm{4C}}$ runtimes is 180 s.\n\t\n\t\\item Fit a kernel density estimator (KDE) with a Gaussian kernel to the collected $\\textbf{x}$ vectors (encapsulated in the $\\textbf{X}$ array) and generate a fixed number of new input vectors, $\\textbf{x}_{KDE}$. Then run PHREEQC for the $\\textbf{X}_{KDE}$ set to get the corresponding output set, $\\textbf{Y}_{KDE}$. Now apply the correction for porosity described in section \\ref{train_res1} to the $\\textbf{X}_{KDE}$ set and form the training set by merging the ensemble of $\\textbf{x}$-$\\textbf{y}$ pairs with that of the $\\textbf{x}_{KDE}$-$\\textbf{y}_{KDE}$ pairs. The number of produced unique examples by the considered cheap HPx$_{\\rm{4C}}$ simulations varied between 10,000 and 50,000. The KDE-based enrichment of this dataset was deemed necessary to provide more input variability, thereby avoiding overfitting of the trained DNN and improving the kNN accuracy, while still honoring the complex between-input relationships. The number of KDE-generated samples was set as to obtain a total training set size of 1,000,000 examples. A key component of the approach is the bandwith parameter of the Gaussian KDE kernel which controls how much the KDE-generated samples depart from the original ensemble. After limited trial and error, we fixed the kernel bandwith to 0.0025 for the considered case studies.\n\t\n\\end{itemize}\n\nThe scatter plots in Figure \\ref{fig9} illustrate our training set creation procedure. The orange dots depict the pairwise relationships observed between the 5 elements of $\\textbf{x}$ in the cheap simulation. The red and cyan dots in Figure \\ref{fig9} represent the KDE-generated samples with the selected bandwidth, before and after applying the correction for porosity, respectively. Training of the DNN is achieved using the ensemble of original and KDE-corrected input points. We refer to this kind of dataset as RT-based, since it is based on an albeit cheap, full RT simulation. Furthermore, we refer to the obtained DNN and kNN emulators as ``local\" emulators, since as opposed to the emulators constructed for cement system 1, the current emulators are only valid for the input conditions encapsulated in the RT-based training set.\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1.5cm}\\includegraphics[width=50pc]{Figure9.png}\n\t\\caption{Scatter plots of the complex relationships between the five considered inputs for (1) the computationally cheap RT simulation of cement system 2 (orange dots) performed with the original HPx code, (2) the corresponding sampled points by kernel density estimation (KDE) using the selected bandwith (turquoise dots) and, (3) the same sampled KDE points after porosity correction (red dots).}\n\t\\label{fig9}\n\\end{figure}\n\nTraining performance of the DNN emulator is presented in Figure \\ref{fig10}. Similarly as for cement system 1, about 90\\% of the available data was used for the actual training of the DNN while the remaining 10\\% were used a validation set to contol overfitting. Lastly, performance is evaluated for both the trained DNN and kNN emulators using an independent test set of 10,000 examples. Overall, the accuracy of our ``local \" DNN emulator for this RT-inspired dataset is rather large with $Q_2$ values always greater or equal than 0.998. Training performance of the corresponding local kNN emulator is equally good (not shown). Regarding speedup, for this problem single-threaded PHREEQC achieves about 210 geochemical calculations per second on our used Intel\\textsuperscript{\\textregistered} i7 CPU while the GPU-based DNN and kNN emulators are both about 3000 times faster when predicting the 10,000 test points at once .\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure10.png}\n\t\\caption{1-1 plots of local DNN emulation performance obtained for system 2 when the local DNN is trained using 1,000,000 samples. The $x$-axis and $y$-axis present the original and emulated 10,000 independent test data points, respectively. The RMSE and $Q_2$ coefficient denote the root-mean-square-error and coefficient of determination in testing mode, respectively, between the original and emulated 10,000 test data points.}\n\t\\label{fig10}\n\\end{figure}\n\n\\subsubsection{Reactive transport simulation}\n\\label{prob2res}\nOur ``local\" DNN performs rather well when applied to the 61 $\\times$ 61 grid size and a time period of 6 years (Figures \\ref{fig11} - \\ref{fig12}). This for all components but C, for which some localized deviations appear between original and emulated 2D concentration profiles towards the end of the simulation period (Figure \\ref{fig12}). When applied to the 121 $\\times$ 121 grid, HPx$_{\\rm py}$-DNN produces additional discrepancies for O, Si and Al towards the end of the simulation period (Figures \\ref{fig13} - \\ref{fig14}). Yet most of the observed artifacts could probably be smoothed out by using post-filtering. The associated speedups are listed in Table \\ref{table4}. These speedups are larger to those obtained for cement system 1, with values between 8 and 9 when evaluated against HPx$_{\\rm{4C}}$. These speedups represent about 85 \\% to 90 \\% of the maximum possible speedup (Table \\ref{table3}). Overall, these findings indicate that for the considered problem, our RT-based training of a local DNN only works is the training set is sufficiently representative of the particular geochemical conditions encountered in the computationally demanding simulations, which is arguably not easy to achieve. This limitation is further discussed in section \\ref{discussion}.\n\nWe note a more uniform behavior for HPx$_{\\rm py}$-kNN across grid sizes than for HPx$_{\\rm py}$-DNN. Here the results for the 121  $\\times$ 121 grid are only slightly less accurate than those associated with the 61 $ \\times$ 61 grid (see Figures \\ref{fig15} - \\ref{fig16} where for brevity we only show concentration profiles for the C Al and S components). Furthermore, whenever observed the discrepancies between original and emulated profiles are more regularly scattered than for HPx$_{\\rm py}$-DNN. Note also that herein too, post-filtering could likely smooth out a large part of these deviations. In addition, owing to the use of a GPU to achieve the kNN calculations, the speedup provided by HPx$_{\\rm py}$-kNN are as large as those provided by HPx$_{\\rm py}$-DNN (Table \\ref{table4}).\n\nWith respect to the emulated solid amounts, the HPx$_{\\rm py}$-DNN results look visually good for the the 61 $\\times$ 61 grid. This is shown in Figure \\ref{fig17} for the C, Al and S chemical components, while emulation of the H, O Ca and Si chemical components is globally of similar quality (not shown). Nevertheless, significant deviations appear towards the end of the simulation period for every chemical component (see Figure \\ref{fig18} for the C, Al and S chemical components, emulation of the H, O Ca and Si chemical components shows the same level of mismatch). The HPx$_{\\rm py}$-kNN predictions are also fairly accurate for the 61 $\\times$ 61 (see Figure \\ref{fig19} for the C, Al and S chemical components, emulation of the H, O Ca and Si chemical components exhibits a globally similar quality) while some discrepancies show up at the end of the simulation (Figure \\ref{fig20}). However, the mismatch is less pronounced than for HPx$_{\\rm py}$-DNN. Overall, HPx$_{\\rm py}$-kNN appears to be somewhat more robust than HPx$_{\\rm py}$-DNN for this cement system, while providing the same (large) speedup.\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure11.png}\n\t\\caption{2D concentration profiles obtained for cement system 2 after 2 and 6 (final time step) years. RTM means the original HPx$_{\\rm{4C}}$ model and TM+DNN denotes the Hydrus transport model coupled with our DNN geochemical emulator (HPx$_{\\rm py}$-DNN). The first to fourth row present profiles for $Ca^{\\rm{conc}}$, $Si^{\\rm{conc}}$, $H^{\\rm{conc}}$, and $O^{\\rm{conc}}$, respectively. The considered grid size is 61 $\\times$ 61.}\n\t\\label{fig11}\n\\end{figure}\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure12.png}\n\t\\caption{2D concentration profiles obtained for cement system 2 after 2 and 6 (final time step) years. RTM means the original HPx$_{\\rm{4C}}$ model and TM+DNN denotes the Hydrus transport model coupled with our DNN geochemical emulator (HPx$_{\\rm py}$-DNN). The first to third row present profiles for $C^{\\rm{conc}}$, $Al^{\\rm{conc}}$, and $S^{\\rm{conc}}$, respectively. The considered grid size is 61 $\\times$ 61.}\n\t\\label{fig12}\n\\end{figure}\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure13.png}\n\t\\caption{2D concentration profiles obtained for cement system 2 after 2 and 6 (final time step) years. RTM means the original HPx$_{\\rm{4C}}$ model and TM+DNN denotes the Hydrus transport model coupled with our DNN geochemical emulator (HPx$_{\\rm py}$-DNN). The first to fourth row present profiles for $Ca^{\\rm{conc}}$, $Si^{\\rm{conc}}$, $H^{\\rm{conc}}$, and $O^{\\rm{conc}}$, respectively. The considered grid size is 121 $\\times$ 121.}\n\t\\label{fig13}\n\\end{figure}\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure14.png}\n\t\\caption{2D concentration profiles obtained for cement system 2 after 2 and 6 (final time step) years. RTM means the original HPx$_{\\rm{4C}}$ model and TM+DNN denotes the Hydrus transport model coupled with our DNN geochemical emulator (HPx$_{\\rm py}$-DNN). The first to third row present profiles for $C^{\\rm{conc}}$, $Al^{\\rm{conc}}$, and $S^{\\rm{conc}}$, respectively. The considered grid size is 121 $\\times$ 121.}\n\t\\label{fig14}\n\\end{figure}\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure15.png}\n\t\\caption{2D concentration profiles obtained for cement system 2 after 2 and 6 (final time step) years. RTM means the original HPx$_{\\rm{4C}}$ model and TM+kNN denotes the Hydrus transport model coupled with our kNN geochemical emulator (HPx$_{\\rm py}$-kNN).  The first to third row present profiles for $C^{\\rm{conc}}$, $Al^{\\rm{conc}}$, and $S^{\\rm{conc}}$, respectively. The considered grid size is 61 $\\times$ 61.}\n\t\\label{fig15}\n\\end{figure}\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure16.png}\n\t\\caption{2D concentration profiles obtained for cement system 2 after 2 and 6 (final time step) years. RTM means the original HPx$_{\\rm{4C}}$ model and TM+kNN denotes the Hydrus transport model coupled with our kNN geochemical emulator (HPx$_{\\rm py}$-kNN). The first to third row present profiles for $C^{\\rm{conc}}$, $Al^{\\rm{conc}}$, and $S^{\\rm{conc}}$, respectively. The considered grid size is 121 $\\times$ 121.}\n\t\\label{fig16}\n\\end{figure}\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure17.png}\n\t\\caption{2D solid amount profiles of the C, Al and S chemical components of cement system 2 after 2 and 6 (final time step) years. RTM means the original HPx$_{\\rm{4C}}$ model and TM+DNN denotes the Hydrus transport model coupled with our DNN geochemical emulator (HPx$_{\\rm py}$-DNN). The first to third row present profiles for $C^{\\rm{solid}}$, $Al^{\\rm{solid}}$, and $S^{\\rm{solid}}$, respectively The considered grid size is 61 $\\times$ 61.}\n\t\\label{fig17}\n\\end{figure}\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure18.png}\n\t\\caption{2D solid amount profiles of the C, Al and S chemical components of cement system 2 after 2 and 6 (final time step) years. RTM means the original HPx$_{\\rm{4C}}$ model and TM+DNN denotes the Hydrus transport model coupled with our DNN geochemical emulator (HPx$_{\\rm py}$-DNN). The first to third row present profiles for $C^{\\rm{solid}}$, $Al^{\\rm{solid}}$, and $S^{\\rm{solid}}$, respectively The considered grid size is 121 $\\times$ 121.}\n\t\\label{fig18}\n\\end{figure}\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure19.png}\n\t\\caption{2D solid amount profiles of the C, Al and S chemical components of cement system 2 after 2 and 6 (final time step) years. RTM means the original HPx$_{\\rm{4C}}$ model and TM+kNN denotes the Hydrus transport model coupled with our kNN geochemical emulator (HPx$_{\\rm py}$-kNN). The first to third row present profiles for $C^{\\rm{solid}}$, $Al^{\\rm{solid}}$, and $S^{\\rm{solid}}$, respectively The considered grid size is 61 $\\times$ 61.}\n\t\\label{fig19}\n\\end{figure}\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure20.png}\n\t\\caption{2D solid amount profiles of the C, Al and S chemical components of cement system 2 after 2 and 6 (final time step) years. RTM means the original HPx$_{\\rm{4C}}$ model and TM+kNN denotes the Hydrus transport model coupled with our kNN geochemical emulator (HPx$_{\\rm py}$-kNN). The first to third row present profiles for $C^{\\rm{solid}}$, $Al^{\\rm{solid}}$, and $S^{\\rm{solid}}$, respectively The considered grid size is 121 $\\times$ 121.}\n\t\\label{fig20}\n\\end{figure}\n\n\\begin{table}[h!]\n\t\\caption{Speedups offered by the KNN and DNN emulators in HPx$_{\\rm py}$ for the reactive transport simulations considered for cement system 2. The HPx$_{\\rm{4C}}$ calculations involve the parallelization of PHREEQC over our 4 CPUs. The HPx$_{\\rm{1C}}$ calculations are performed on a single CPU. Both the kNN and DNN predictions make use of a GPU. ML signifies the used machine learning method for emulation, BC denotes the type of flow boundary conditions and GS is the grid size. The maximum possible speedups associated with HPx$_{\\rm{4C}}$ and HP$_{\\rm{1C}}$,  Max SP HPx$_{\\rm{4C}}$ and  Max SP HPx$_{\\rm{1C}}$, correspond to an hypothetical situation where the geochemical calculations incur zero computational cost.}\n\n\t\\begin{center}\n\t\t\\begin{tabular}{cccccccc}\n\t\t\t\\hline\n\t\t\t& & & & & & & \\\\\n\t\t\tML & BC & GS & HPx$_{\\rm{4C}}$ time (s) & SP HPx$_{\\rm{4C}}$ & Max SP HPx$_{\\rm{4C}}$ & SP HPx$_{\\rm{1C}}$ & Max SP HPx$_{\\rm{1C}}$ \\\\\n\t\t\tDNN & ADV & 61 $\\times$ 61 & 21,415 & 8.2 & 9.0 & 30.3 & 33.1\\\\\n\t\t\tkNN & ADV & 61 $\\times$ 61 & 21,415 & 7.9 & 9.0 & 28.9 & 33.1\\\\\n\t\t\tDNN & ADV & 121 $\\times$ 121 & 199,841 & 8.2 & 9.5 & 29.9 & 32.8\\\\\n\t\t\tkNN & ADV & 121 $\\times$ 121 & 199,841 & 8.5 & 9.5 & 31.3 & 32.8\\\\\n\t\t\t\\hline\n\t\t\\end{tabular}\n\t\\end{center}\n\n\t\\label{table4}\n\\end{table}\n\n\\FloatBarrier\n\n3.2 Al-C-Ca-S-Si Problem\n\\subsection{Al-C-Ca-S-Si Problem}\n\nFor this second cement system, the emulation problem consists of predicting at each time step of the RT simulation the (output) Al, C, Ca, S, Si, H and O aqueous concentrations (mol/kgw) from the (input) total amounts of Al, C, Ca, S and Si (mol). Here we focus on advective-dispersive transport only while similarly as for cement system 1, the considered domain sizes are 61 $\\times$ 61 and 121 $\\times$ 121. Moreover, the simulation time period is set to 6 years. \n\nAs mentioned earlier, for this higher-dimensional problem it is observed that the used scikit-learn kNN implementation becomes prohibitively slow compared to HPx$_{\\rm{4C}}$. We found that to get a good emulation accuracy the kNN training base needs to contain 1,000,000 samples (or more). This training base's size together with a 5-dimensional search space leads to an HPx$_{\\rm py}$-kNN reactive transport simulation time that is comparable to that of HPx$_{\\rm{4C}}$. Therefore, for this second cement system we built a custom kNN regressor around another kNN implementation contained in the FAISS package \\citep{faiss2017}. Our used FAISS variant allows for GPU computing and is much faster than scikit-learn for this cement system, but is slightly less accurate due to the use of an approximate rather than exact nearest neighbor search \\citep[see][for details]{faiss2017}.\n\n\\subsubsection{Training the DNN}\n\\label{train_res2}\n\nBuilding a good training set to perform a kNN search and learn the weights and biases of our DNN turned out to be a complicated task in this case. This because to make useful kNN predictions and/or learn an useful DNN, the training set must be sufficiently representative of the geochemical conditions encountered during the reactive transport simulation one wish to perform with HPx$_{\\rm py}$-kNN and the trained HPx$_{\\rm py}$-DNN. In contrast to cement system 1, creating the training set by sampling the $X$-space with a controlled randomness between predefined lower and upper bounds did not prove successful. We tried that strategy by drawing as much as 4,000,000 5-dimensional $\\textbf{x}$ vectors from the $X$-space using a Sobol low-discrepancy sequence \\citep[][]{Sobol1967, Joe-Kuo2003}. Such low-discrepancy sampling scheme covers the 5-dimensional hypercube more uniformly than LHS. Despite a good performance on the test set (not shown), the resulting DNN accuracy in reactive transport mode was never deemed satisfying. In other words, no satisfying ``global\" or ``universal\" DNN emulator could be devised for this cement system. This is probably caused by the fact for this problem, the input (5 total amounts) and output (7 aqueous concentrations) spaces are quite nonlinearly related and both cover 6 to 10 orders of magnitudes depending on the considered element. Therefore, we resorted to the alternative training strategy detailed below. The latter basically tries to grasp the complex correlations and higher-order dependencies that exist between the elements of $\\textbf{x}$ (total amounts, input space) for a given reactive transport simulation setup, in order to produce a training set that honors these between-input relationships.\n\n\\begin{itemize}\n\t\\item Perform a ``cheap\" full reactive transport simulation under the transport conditions and geochemistry of interest and collect the resulting $\\textbf{x}$ -$\\textbf{y}$  pairs of examples (for the considered grid nodes and time steps). Computational demand controls what domain size and simulation time period can be used for this cheap calculation. We used a modest 16 $\\times$ 16 domain and a simulation time period of 10 years. The associated HPx$_{\\rm{4C}}$ runtimes is 180 s.\n\t\n\t\\item Fit a kernel density estimator (KDE) with a Gaussian kernel to the collected $\\textbf{x}$ vectors (encapsulated in the $\\textbf{X}$ array) and generate a fixed number of new input vectors, $\\textbf{x}_{KDE}$. Then run PHREEQC for the $\\textbf{X}_{KDE}$ set to get the corresponding output set, $\\textbf{Y}_{KDE}$. Now apply the correction for porosity described in section \\ref{train_res1} to the $\\textbf{X}_{KDE}$ set and form the training set by merging the ensemble of $\\textbf{x}$-$\\textbf{y}$ pairs with that of the $\\textbf{x}_{KDE}$-$\\textbf{y}_{KDE}$ pairs. The number of produced unique examples by the considered cheap HPx$_{\\rm{4C}}$ simulations varied between 10,000 and 50,000. The KDE-based enrichment of this dataset was deemed necessary to provide more input variability, thereby avoiding overfitting of the trained DNN and improving the kNN accuracy, while still honoring the complex between-input relationships. The number of KDE-generated samples was set as to obtain a total training set size of 1,000,000 examples. A key component of the approach is the bandwith parameter of the Gaussian KDE kernel which controls how much the KDE-generated samples depart from the original ensemble. After limited trial and error, we fixed the kernel bandwith to 0.0025 for the considered case studies.\n\t\n\\end{itemize}\n\nThe scatter plots in Figure \\ref{fig9} illustrate our training set creation procedure. The orange dots depict the pairwise relationships observed between the 5 elements of $\\textbf{x}$ in the cheap simulation. The red and cyan dots in Figure \\ref{fig9} represent the KDE-generated samples with the selected bandwidth, before and after applying the correction for porosity, respectively. Training of the DNN is achieved using the ensemble of original and KDE-corrected input points. We refer to this kind of dataset as RT-based, since it is based on an albeit cheap, full RT simulation. Furthermore, we refer to the obtained DNN and kNN emulators as ``local\" emulators, since as opposed to the emulators constructed for cement system 1, the current emulators are only valid for the input conditions encapsulated in the RT-based training set.\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1.5cm}\\includegraphics[width=50pc]{Figure9.png}\n\t\\caption{Scatter plots of the complex relationships between the five considered inputs for (1) the computationally cheap RT simulation of cement system 2 (orange dots) performed with the original HPx code, (2) the corresponding sampled points by kernel density estimation (KDE) using the selected bandwith (turquoise dots) and, (3) the same sampled KDE points after porosity correction (red dots).}\n\t\\label{fig9}\n\\end{figure}\n\nTraining performance of the DNN emulator is presented in Figure \\ref{fig10}. Similarly as for cement system 1, about 90\\% of the available data was used for the actual training of the DNN while the remaining 10\\% were used a validation set to contol overfitting. Lastly, performance is evaluated for both the trained DNN and kNN emulators using an independent test set of 10,000 examples. Overall, the accuracy of our ``local \" DNN emulator for this RT-inspired dataset is rather large with $Q_2$ values always greater or equal than 0.998. Training performance of the corresponding local kNN emulator is equally good (not shown). Regarding speedup, for this problem single-threaded PHREEQC achieves about 210 geochemical calculations per second on our used Intel\\textsuperscript{\\textregistered} i7 CPU while the GPU-based DNN and kNN emulators are both about 3000 times faster when predicting the 10,000 test points at once .\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure10.png}\n\t\\caption{1-1 plots of local DNN emulation performance obtained for system 2 when the local DNN is trained using 1,000,000 samples. The $x$-axis and $y$-axis present the original and emulated 10,000 independent test data points, respectively. The RMSE and $Q_2$ coefficient denote the root-mean-square-error and coefficient of determination in testing mode, respectively, between the original and emulated 10,000 test data points.}\n\t\\label{fig10}\n\\end{figure}\n\n\\subsubsection{Reactive transport simulation}\n\\label{prob2res}\nOur ``local\" DNN performs rather well when applied to the 61 $\\times$ 61 grid size and a time period of 6 years (Figures \\ref{fig11} - \\ref{fig12}). This for all components but C, for which some localized deviations appear between original and emulated 2D concentration profiles towards the end of the simulation period (Figure \\ref{fig12}). When applied to the 121 $\\times$ 121 grid, HPx$_{\\rm py}$-DNN produces additional discrepancies for O, Si and Al towards the end of the simulation period (Figures \\ref{fig13} - \\ref{fig14}). Yet most of the observed artifacts could probably be smoothed out by using post-filtering. The associated speedups are listed in Table \\ref{table4}. These speedups are larger to those obtained for cement system 1, with values between 8 and 9 when evaluated against HPx$_{\\rm{4C}}$. These speedups represent about 85 \\% to 90 \\% of the maximum possible speedup (Table \\ref{table3}). Overall, these findings indicate that for the considered problem, our RT-based training of a local DNN only works is the training set is sufficiently representative of the particular geochemical conditions encountered in the computationally demanding simulations, which is arguably not easy to achieve. This limitation is further discussed in section \\ref{discussion}.\n\nWe note a more uniform behavior for HPx$_{\\rm py}$-kNN across grid sizes than for HPx$_{\\rm py}$-DNN. Here the results for the 121  $\\times$ 121 grid are only slightly less accurate than those associated with the 61 $ \\times$ 61 grid (see Figures \\ref{fig15} - \\ref{fig16} where for brevity we only show concentration profiles for the C Al and S components). Furthermore, whenever observed the discrepancies between original and emulated profiles are more regularly scattered than for HPx$_{\\rm py}$-DNN. Note also that herein too, post-filtering could likely smooth out a large part of these deviations. In addition, owing to the use of a GPU to achieve the kNN calculations, the speedup provided by HPx$_{\\rm py}$-kNN are as large as those provided by HPx$_{\\rm py}$-DNN (Table \\ref{table4}).\n\nWith respect to the emulated solid amounts, the HPx$_{\\rm py}$-DNN results look visually good for the the 61 $\\times$ 61 grid. This is shown in Figure \\ref{fig17} for the C, Al and S chemical components, while emulation of the H, O Ca and Si chemical components is globally of similar quality (not shown). Nevertheless, significant deviations appear towards the end of the simulation period for every chemical component (see Figure \\ref{fig18} for the C, Al and S chemical components, emulation of the H, O Ca and Si chemical components shows the same level of mismatch). The HPx$_{\\rm py}$-kNN predictions are also fairly accurate for the 61 $\\times$ 61 (see Figure \\ref{fig19} for the C, Al and S chemical components, emulation of the H, O Ca and Si chemical components exhibits a globally similar quality) while some discrepancies show up at the end of the simulation (Figure \\ref{fig20}). However, the mismatch is less pronounced than for HPx$_{\\rm py}$-DNN. Overall, HPx$_{\\rm py}$-kNN appears to be somewhat more robust than HPx$_{\\rm py}$-DNN for this cement system, while providing the same (large) speedup.\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure11.png}\n\t\\caption{2D concentration profiles obtained for cement system 2 after 2 and 6 (final time step) years. RTM means the original HPx$_{\\rm{4C}}$ model and TM+DNN denotes the Hydrus transport model coupled with our DNN geochemical emulator (HPx$_{\\rm py}$-DNN). The first to fourth row present profiles for $Ca^{\\rm{conc}}$, $Si^{\\rm{conc}}$, $H^{\\rm{conc}}$, and $O^{\\rm{conc}}$, respectively. The considered grid size is 61 $\\times$ 61.}\n\t\\label{fig11}\n\\end{figure}\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure12.png}\n\t\\caption{2D concentration profiles obtained for cement system 2 after 2 and 6 (final time step) years. RTM means the original HPx$_{\\rm{4C}}$ model and TM+DNN denotes the Hydrus transport model coupled with our DNN geochemical emulator (HPx$_{\\rm py}$-DNN). The first to third row present profiles for $C^{\\rm{conc}}$, $Al^{\\rm{conc}}$, and $S^{\\rm{conc}}$, respectively. The considered grid size is 61 $\\times$ 61.}\n\t\\label{fig12}\n\\end{figure}\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure13.png}\n\t\\caption{2D concentration profiles obtained for cement system 2 after 2 and 6 (final time step) years. RTM means the original HPx$_{\\rm{4C}}$ model and TM+DNN denotes the Hydrus transport model coupled with our DNN geochemical emulator (HPx$_{\\rm py}$-DNN). The first to fourth row present profiles for $Ca^{\\rm{conc}}$, $Si^{\\rm{conc}}$, $H^{\\rm{conc}}$, and $O^{\\rm{conc}}$, respectively. The considered grid size is 121 $\\times$ 121.}\n\t\\label{fig13}\n\\end{figure}\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure14.png}\n\t\\caption{2D concentration profiles obtained for cement system 2 after 2 and 6 (final time step) years. RTM means the original HPx$_{\\rm{4C}}$ model and TM+DNN denotes the Hydrus transport model coupled with our DNN geochemical emulator (HPx$_{\\rm py}$-DNN). The first to third row present profiles for $C^{\\rm{conc}}$, $Al^{\\rm{conc}}$, and $S^{\\rm{conc}}$, respectively. The considered grid size is 121 $\\times$ 121.}\n\t\\label{fig14}\n\\end{figure}\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure15.png}\n\t\\caption{2D concentration profiles obtained for cement system 2 after 2 and 6 (final time step) years. RTM means the original HPx$_{\\rm{4C}}$ model and TM+kNN denotes the Hydrus transport model coupled with our kNN geochemical emulator (HPx$_{\\rm py}$-kNN).  The first to third row present profiles for $C^{\\rm{conc}}$, $Al^{\\rm{conc}}$, and $S^{\\rm{conc}}$, respectively. The considered grid size is 61 $\\times$ 61.}\n\t\\label{fig15}\n\\end{figure}\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure16.png}\n\t\\caption{2D concentration profiles obtained for cement system 2 after 2 and 6 (final time step) years. RTM means the original HPx$_{\\rm{4C}}$ model and TM+kNN denotes the Hydrus transport model coupled with our kNN geochemical emulator (HPx$_{\\rm py}$-kNN). The first to third row present profiles for $C^{\\rm{conc}}$, $Al^{\\rm{conc}}$, and $S^{\\rm{conc}}$, respectively. The considered grid size is 121 $\\times$ 121.}\n\t\\label{fig16}\n\\end{figure}\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure17.png}\n\t\\caption{2D solid amount profiles of the C, Al and S chemical components of cement system 2 after 2 and 6 (final time step) years. RTM means the original HPx$_{\\rm{4C}}$ model and TM+DNN denotes the Hydrus transport model coupled with our DNN geochemical emulator (HPx$_{\\rm py}$-DNN). The first to third row present profiles for $C^{\\rm{solid}}$, $Al^{\\rm{solid}}$, and $S^{\\rm{solid}}$, respectively The considered grid size is 61 $\\times$ 61.}\n\t\\label{fig17}\n\\end{figure}\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure18.png}\n\t\\caption{2D solid amount profiles of the C, Al and S chemical components of cement system 2 after 2 and 6 (final time step) years. RTM means the original HPx$_{\\rm{4C}}$ model and TM+DNN denotes the Hydrus transport model coupled with our DNN geochemical emulator (HPx$_{\\rm py}$-DNN). The first to third row present profiles for $C^{\\rm{solid}}$, $Al^{\\rm{solid}}$, and $S^{\\rm{solid}}$, respectively The considered grid size is 121 $\\times$ 121.}\n\t\\label{fig18}\n\\end{figure}\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure19.png}\n\t\\caption{2D solid amount profiles of the C, Al and S chemical components of cement system 2 after 2 and 6 (final time step) years. RTM means the original HPx$_{\\rm{4C}}$ model and TM+kNN denotes the Hydrus transport model coupled with our kNN geochemical emulator (HPx$_{\\rm py}$-kNN). The first to third row present profiles for $C^{\\rm{solid}}$, $Al^{\\rm{solid}}$, and $S^{\\rm{solid}}$, respectively The considered grid size is 61 $\\times$ 61.}\n\t\\label{fig19}\n\\end{figure}\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure20.png}\n\t\\caption{2D solid amount profiles of the C, Al and S chemical components of cement system 2 after 2 and 6 (final time step) years. RTM means the original HPx$_{\\rm{4C}}$ model and TM+kNN denotes the Hydrus transport model coupled with our kNN geochemical emulator (HPx$_{\\rm py}$-kNN). The first to third row present profiles for $C^{\\rm{solid}}$, $Al^{\\rm{solid}}$, and $S^{\\rm{solid}}$, respectively The considered grid size is 121 $\\times$ 121.}\n\t\\label{fig20}\n\\end{figure}\n\n\\begin{table}[h!]\n\t\\caption{Speedups offered by the KNN and DNN emulators in HPx$_{\\rm py}$ for the reactive transport simulations considered for cement system 2. The HPx$_{\\rm{4C}}$ calculations involve the parallelization of PHREEQC over our 4 CPUs. The HPx$_{\\rm{1C}}$ calculations are performed on a single CPU. Both the kNN and DNN predictions make use of a GPU. ML signifies the used machine learning method for emulation, BC denotes the type of flow boundary conditions and GS is the grid size. The maximum possible speedups associated with HPx$_{\\rm{4C}}$ and HP$_{\\rm{1C}}$,  Max SP HPx$_{\\rm{4C}}$ and  Max SP HPx$_{\\rm{1C}}$, correspond to an hypothetical situation where the geochemical calculations incur zero computational cost.}\n\n\t\\begin{center}\n\t\t\\begin{tabular}{cccccccc}\n\t\t\t\\hline\n\t\t\t& & & & & & & \\\\\n\t\t\tML & BC & GS & HPx$_{\\rm{4C}}$ time (s) & SP HPx$_{\\rm{4C}}$ & Max SP HPx$_{\\rm{4C}}$ & SP HPx$_{\\rm{1C}}$ & Max SP HPx$_{\\rm{1C}}$ \\\\\n\t\t\tDNN & ADV & 61 $\\times$ 61 & 21,415 & 8.2 & 9.0 & 30.3 & 33.1\\\\\n\t\t\tkNN & ADV & 61 $\\times$ 61 & 21,415 & 7.9 & 9.0 & 28.9 & 33.1\\\\\n\t\t\tDNN & ADV & 121 $\\times$ 121 & 199,841 & 8.2 & 9.5 & 29.9 & 32.8\\\\\n\t\t\tkNN & ADV & 121 $\\times$ 121 & 199,841 & 8.5 & 9.5 & 31.3 & 32.8\\\\\n\t\t\t\\hline\n\t\t\\end{tabular}\n\t\\end{center}\n\n\t\\label{table4}\n\\end{table}\n\n\\FloatBarrier\n\n3.2.2 Reactive transport simulation\n\\subsubsection{Reactive transport simulation}\n\\label{prob2res}\nOur ``local\" DNN performs rather well when applied to the 61 $\\times$ 61 grid size and a time period of 6 years (Figures \\ref{fig11} - \\ref{fig12}). This for all components but C, for which some localized deviations appear between original and emulated 2D concentration profiles towards the end of the simulation period (Figure \\ref{fig12}). When applied to the 121 $\\times$ 121 grid, HPx$_{\\rm py}$-DNN produces additional discrepancies for O, Si and Al towards the end of the simulation period (Figures \\ref{fig13} - \\ref{fig14}). Yet most of the observed artifacts could probably be smoothed out by using post-filtering. The associated speedups are listed in Table \\ref{table4}. These speedups are larger to those obtained for cement system 1, with values between 8 and 9 when evaluated against HPx$_{\\rm{4C}}$. These speedups represent about 85 \\% to 90 \\% of the maximum possible speedup (Table \\ref{table3}). Overall, these findings indicate that for the considered problem, our RT-based training of a local DNN only works is the training set is sufficiently representative of the particular geochemical conditions encountered in the computationally demanding simulations, which is arguably not easy to achieve. This limitation is further discussed in section \\ref{discussion}.\n\nWe note a more uniform behavior for HPx$_{\\rm py}$-kNN across grid sizes than for HPx$_{\\rm py}$-DNN. Here the results for the 121  $\\times$ 121 grid are only slightly less accurate than those associated with the 61 $ \\times$ 61 grid (see Figures \\ref{fig15} - \\ref{fig16} where for brevity we only show concentration profiles for the C Al and S components). Furthermore, whenever observed the discrepancies between original and emulated profiles are more regularly scattered than for HPx$_{\\rm py}$-DNN. Note also that herein too, post-filtering could likely smooth out a large part of these deviations. In addition, owing to the use of a GPU to achieve the kNN calculations, the speedup provided by HPx$_{\\rm py}$-kNN are as large as those provided by HPx$_{\\rm py}$-DNN (Table \\ref{table4}).\n\nWith respect to the emulated solid amounts, the HPx$_{\\rm py}$-DNN results look visually good for the the 61 $\\times$ 61 grid. This is shown in Figure \\ref{fig17} for the C, Al and S chemical components, while emulation of the H, O Ca and Si chemical components is globally of similar quality (not shown). Nevertheless, significant deviations appear towards the end of the simulation period for every chemical component (see Figure \\ref{fig18} for the C, Al and S chemical components, emulation of the H, O Ca and Si chemical components shows the same level of mismatch). The HPx$_{\\rm py}$-kNN predictions are also fairly accurate for the 61 $\\times$ 61 (see Figure \\ref{fig19} for the C, Al and S chemical components, emulation of the H, O Ca and Si chemical components exhibits a globally similar quality) while some discrepancies show up at the end of the simulation (Figure \\ref{fig20}). However, the mismatch is less pronounced than for HPx$_{\\rm py}$-DNN. Overall, HPx$_{\\rm py}$-kNN appears to be somewhat more robust than HPx$_{\\rm py}$-DNN for this cement system, while providing the same (large) speedup.\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure11.png}\n\t\\caption{2D concentration profiles obtained for cement system 2 after 2 and 6 (final time step) years. RTM means the original HPx$_{\\rm{4C}}$ model and TM+DNN denotes the Hydrus transport model coupled with our DNN geochemical emulator (HPx$_{\\rm py}$-DNN). The first to fourth row present profiles for $Ca^{\\rm{conc}}$, $Si^{\\rm{conc}}$, $H^{\\rm{conc}}$, and $O^{\\rm{conc}}$, respectively. The considered grid size is 61 $\\times$ 61.}\n\t\\label{fig11}\n\\end{figure}\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure12.png}\n\t\\caption{2D concentration profiles obtained for cement system 2 after 2 and 6 (final time step) years. RTM means the original HPx$_{\\rm{4C}}$ model and TM+DNN denotes the Hydrus transport model coupled with our DNN geochemical emulator (HPx$_{\\rm py}$-DNN). The first to third row present profiles for $C^{\\rm{conc}}$, $Al^{\\rm{conc}}$, and $S^{\\rm{conc}}$, respectively. The considered grid size is 61 $\\times$ 61.}\n\t\\label{fig12}\n\\end{figure}\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure13.png}\n\t\\caption{2D concentration profiles obtained for cement system 2 after 2 and 6 (final time step) years. RTM means the original HPx$_{\\rm{4C}}$ model and TM+DNN denotes the Hydrus transport model coupled with our DNN geochemical emulator (HPx$_{\\rm py}$-DNN). The first to fourth row present profiles for $Ca^{\\rm{conc}}$, $Si^{\\rm{conc}}$, $H^{\\rm{conc}}$, and $O^{\\rm{conc}}$, respectively. The considered grid size is 121 $\\times$ 121.}\n\t\\label{fig13}\n\\end{figure}\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure14.png}\n\t\\caption{2D concentration profiles obtained for cement system 2 after 2 and 6 (final time step) years. RTM means the original HPx$_{\\rm{4C}}$ model and TM+DNN denotes the Hydrus transport model coupled with our DNN geochemical emulator (HPx$_{\\rm py}$-DNN). The first to third row present profiles for $C^{\\rm{conc}}$, $Al^{\\rm{conc}}$, and $S^{\\rm{conc}}$, respectively. The considered grid size is 121 $\\times$ 121.}\n\t\\label{fig14}\n\\end{figure}\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure15.png}\n\t\\caption{2D concentration profiles obtained for cement system 2 after 2 and 6 (final time step) years. RTM means the original HPx$_{\\rm{4C}}$ model and TM+kNN denotes the Hydrus transport model coupled with our kNN geochemical emulator (HPx$_{\\rm py}$-kNN).  The first to third row present profiles for $C^{\\rm{conc}}$, $Al^{\\rm{conc}}$, and $S^{\\rm{conc}}$, respectively. The considered grid size is 61 $\\times$ 61.}\n\t\\label{fig15}\n\\end{figure}\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure16.png}\n\t\\caption{2D concentration profiles obtained for cement system 2 after 2 and 6 (final time step) years. RTM means the original HPx$_{\\rm{4C}}$ model and TM+kNN denotes the Hydrus transport model coupled with our kNN geochemical emulator (HPx$_{\\rm py}$-kNN). The first to third row present profiles for $C^{\\rm{conc}}$, $Al^{\\rm{conc}}$, and $S^{\\rm{conc}}$, respectively. The considered grid size is 121 $\\times$ 121.}\n\t\\label{fig16}\n\\end{figure}\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure17.png}\n\t\\caption{2D solid amount profiles of the C, Al and S chemical components of cement system 2 after 2 and 6 (final time step) years. RTM means the original HPx$_{\\rm{4C}}$ model and TM+DNN denotes the Hydrus transport model coupled with our DNN geochemical emulator (HPx$_{\\rm py}$-DNN). The first to third row present profiles for $C^{\\rm{solid}}$, $Al^{\\rm{solid}}$, and $S^{\\rm{solid}}$, respectively The considered grid size is 61 $\\times$ 61.}\n\t\\label{fig17}\n\\end{figure}\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure18.png}\n\t\\caption{2D solid amount profiles of the C, Al and S chemical components of cement system 2 after 2 and 6 (final time step) years. RTM means the original HPx$_{\\rm{4C}}$ model and TM+DNN denotes the Hydrus transport model coupled with our DNN geochemical emulator (HPx$_{\\rm py}$-DNN). The first to third row present profiles for $C^{\\rm{solid}}$, $Al^{\\rm{solid}}$, and $S^{\\rm{solid}}$, respectively The considered grid size is 121 $\\times$ 121.}\n\t\\label{fig18}\n\\end{figure}\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure19.png}\n\t\\caption{2D solid amount profiles of the C, Al and S chemical components of cement system 2 after 2 and 6 (final time step) years. RTM means the original HPx$_{\\rm{4C}}$ model and TM+kNN denotes the Hydrus transport model coupled with our kNN geochemical emulator (HPx$_{\\rm py}$-kNN). The first to third row present profiles for $C^{\\rm{solid}}$, $Al^{\\rm{solid}}$, and $S^{\\rm{solid}}$, respectively The considered grid size is 61 $\\times$ 61.}\n\t\\label{fig19}\n\\end{figure}\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure20.png}\n\t\\caption{2D solid amount profiles of the C, Al and S chemical components of cement system 2 after 2 and 6 (final time step) years. RTM means the original HPx$_{\\rm{4C}}$ model and TM+kNN denotes the Hydrus transport model coupled with our kNN geochemical emulator (HPx$_{\\rm py}$-kNN). The first to third row present profiles for $C^{\\rm{solid}}$, $Al^{\\rm{solid}}$, and $S^{\\rm{solid}}$, respectively The considered grid size is 121 $\\times$ 121.}\n\t\\label{fig20}\n\\end{figure}\n\n\\begin{table}[h!]\n\t\\caption{Speedups offered by the KNN and DNN emulators in HPx$_{\\rm py}$ for the reactive transport simulations considered for cement system 2. The HPx$_{\\rm{4C}}$ calculations involve the parallelization of PHREEQC over our 4 CPUs. The HPx$_{\\rm{1C}}$ calculations are performed on a single CPU. Both the kNN and DNN predictions make use of a GPU. ML signifies the used machine learning method for emulation, BC denotes the type of flow boundary conditions and GS is the grid size. The maximum possible speedups associated with HPx$_{\\rm{4C}}$ and HP$_{\\rm{1C}}$,  Max SP HPx$_{\\rm{4C}}$ and  Max SP HPx$_{\\rm{1C}}$, correspond to an hypothetical situation where the geochemical calculations incur zero computational cost.}\n\n\t\\begin{center}\n\t\t\\begin{tabular}{cccccccc}\n\t\t\t\\hline\n\t\t\t& & & & & & & \\\\\n\t\t\tML & BC & GS & HPx$_{\\rm{4C}}$ time (s) & SP HPx$_{\\rm{4C}}$ & Max SP HPx$_{\\rm{4C}}$ & SP HPx$_{\\rm{1C}}$ & Max SP HPx$_{\\rm{1C}}$ \\\\\n\t\t\tDNN & ADV & 61 $\\times$ 61 & 21,415 & 8.2 & 9.0 & 30.3 & 33.1\\\\\n\t\t\tkNN & ADV & 61 $\\times$ 61 & 21,415 & 7.9 & 9.0 & 28.9 & 33.1\\\\\n\t\t\tDNN & ADV & 121 $\\times$ 121 & 199,841 & 8.2 & 9.5 & 29.9 & 32.8\\\\\n\t\t\tkNN & ADV & 121 $\\times$ 121 & 199,841 & 8.5 & 9.5 & 31.3 & 32.8\\\\\n\t\t\t\\hline\n\t\t\\end{tabular}\n\t\\end{center}\n\n\t\\label{table4}\n\\end{table}\n\n\\FloatBarrier\n\n",
        "label": "fig13",
        "Descriptive_question1": "What is the grid size used in figure_13 for the 2D concentration profiles of cement system 2?",
        "Descriptive_question2": "Which chemical component is represented in the first row of figure_13?",
        "Reasoning_question1": "How do the concentration profiles of Ca, Si, H, and O in figure_13 compare between the RTM and TM+DNN models after 6 years, and what might this suggest about the accuracy of the DNN emulator for larger grid sizes?",
        "Reasoning_question2": "What trends can be observed in the concentration changes from 2 years to 6 years for the different chemical components in figure_13, and what could be the possible geochemical processes driving these changes?",
        "Descriptive_answer1": "121 × 121",
        "Descriptive_answer2": "Ca^{conc}",
        "Reasoning_answer1": "In figure_13, the concentration profiles of Ca, Si, H, and O at the 6-year mark show visual discrepancies between the RTM (original HPx_{4C} model) and TM+DNN (Hydrus transport model coupled with DNN emulator) results. While the general spatial patterns remain similar, there are noticeable differences particularly toward the boundaries and some areas within the domain, indicating localized mismatches. This suggests that while the DNN emulator can broadly capture the concentration profiles for a larger grid size (121 × 121), its accuracy slightly diminishes compared to smaller grids, possibly due to increased numerical complexities or less representative training data coverage in the larger domain space.",
        "Reasoning_answer2": "From 2 years to 6 years in figure_13, the concentration profiles for Ca, Si, H, and O exhibit noticeable temporal evolution. Generally, concentrations of reactive species such as Ca and Si decrease or redistribute due to ongoing advective-dispersive transport and geochemical reactions. H and O concentrations also change, reflecting water-rock interactions and saturation state changes. These patterns imply ongoing dissolution and precipitation reactions affecting element availability, as well as transport-driven dilution or concentration gradients forming over time, consistent with expected geochemical processes in cement systems under transport conditions."
    },
    {
        "paper_id": "2107.07598.json",
        "image_id": "figure_14",
        "image_path": [
            "/home/yz979/scratch/chengye/scimolmo/test-images-tables/dataset_imagestables/2107.07598/images/Figure14.png"
        ],
        "caption": "2D concentration profiles obtained for cement system 2 after 2 and 6 (final time step) years. RTM means the original HPx$_{\\rm{4C}}$ model and TM+DNN denotes the Hydrus transport model coupled with our DNN geochemical emulator (HPx$_{\\rm py}$-DNN). The first to third row present profiles for $C^{\\rm{conc}}$, $Al^{\\rm{conc}}$, and $S^{\\rm{conc}}$, respectively. The considered grid size is 121 $\\times$ 121.",
        "classify": "Chart",
        "section_info": "3 Results\n\\section{Results}\n\\label{results}\n\n\\subsection{Ca-Si Problem}\nFor this first cement system, the emulation problem consists of predicting at each time step of the RT simulation the (output) Ca, Si, H and O aqueous concentrations (mol/kg of water or mol/kgw) from the (input) total amounts of Ca and Si (mol). \n\n\\subsubsection{Training the emulators}\n\\label{train_res1}\nHere the kNN and DNN emulators are firstly trained using a set of 400,000 test examples for both. This training set is obtained by randomly sampling the two-dimensional input space by latin hypercube sampling (LHS) between $\\left[0,0\\right]$ and $\\rm{\\left[Ca^{tot}_{max},Si^{tot}_{max}\\right]}$, and running PHREEQC for each input sample, $\\rm{\\textbf{x}_i = \\left[Ca^{tot}_i, Si^{tot}_i\\right]}$ to get the corresponding output vectors, $\\rm{\\textbf{y}_i = \\left[Ca^{conc}_i,Si^{conc}_i,H^{conc}_i,O^{conc}_i\\right]}$. The upper bounds, $\\rm{Ca^{tot}_{max}}$ and $\\rm{Si^{tot}_{max}}$ are defined based on a cheap full RT simulation with advective-dispersive transport using a small 1D domain of 51 nodes. It is worth noting that the total amounts of $\\textbf{x}_i$, corresponding to the PHREEQC-simulated concentrations, $\\textbf{y}_i$, have to be corrected for the different amount of water between the training set and the transport simulations. Doing so, it turns out that about 20 \\% of the post-corrected $x_i$ values exceed their pre-defined upper bounds and these excessively large values need to be filtered out. Creating the 400,000 training examples thus required about 500,000 PHREEQC runs. As stated earlier, for this problem single-threaded PHREEQC performs about 670 geochemical calculations per second on our used Intel\\textsuperscript{\\textregistered} i7 CPU.\n\nWith respect to kNN, the tuning parameters are the number of neighbors, $k$, the type of distance measure, and the interpolation technique. We simply used the default settings: $k = 5$, euclidean distance and inverse-distance interpolation. Regarding training of the DNN, the 400,000 sample were split between the training set itself (90 \\% of the data) and a validation set (10 \\% of the data). The latter serves to monitor the evolution of the selected mean squared error loss function on samples that are not used for training, thereby detecting potential overfitting. If the validation loss stops decreasing before the fixed number of epochs has been completed, then  training is stopped. Importantly, the emulation is achieved in log-space for both the input, $X$ and output $Y$, domains. This because total amounts and concentrations of the involved components typically cover many orders of magnitudes (up to 10 orders or more). Using a DNN also requires some form of data normalization or standardization. Here both the $\\rm{log\\left(\\textbf{x}_i\\right)}$ and $\\rm{log\\left(\\textbf{y}_i\\right)}$ vectors are standardized around 0 with standard deviation of 1.\n\nFigure \\ref{fig1} illustrates the trained emulators' performance for geochemical predictions using an independent test set that comprises 10,000 test examples. Both kNN (Figures \\ref{fig1}a - d) and DNN (Figures \\ref{fig1}e - h) appear to be rather accurate . DNN also shows a slight degradation for the larger concentration values  (Figures \\ref{fig1}e - h). The latter is likely due to the combination of a small proportion of large concentration values in the training set with the log-transformation that implicitly pushes the DNN to try harder to fit the smaller concentrations during training. Regarding speedup and as written earlier, for this setup the single-threaded kNN method is 300 times faster than single-threaded PHREEQC for predicting the 10,000 concentration vectors all at once. The computational savings allowed by the DNN emulator when ran on our GPU is higher with a speedup as large as 4000 for predicting the same 10,000 concentration vectors all at once. \n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{0cm}\\includegraphics[width=35pc]{Figure1.png}\n\t\\caption{1-1 plots of the kNN (subfigures (a) - (d)) and DNN (subfigures (e) - (h)) emulators' performance obtained for system 1 when the kNN training base contains 400,000 samples and the DNN is trained using the same 400,000 samples. Here ``true\" means the original PHREEQC-simulated data and ``predicted\" denotes the emulated (that is, kNN-simulated and DNN-simulated) data. Hence, the $x$-axis and $y$-axis present the original and emulated 10,000 independent test data points, respectively. The RMSE and $Q_2$ coefficient denote the root-mean-square-error and coefficient of determination in testing mode, respectively, between the original and emulated 10,000 test data points.}\n\t\\label{fig1}\n\\end{figure}\n\nTo test the sensitivity of the emulators' performance to the training set size, training was also performed using reduced training sets comprising 100,000, 10,000, 1000 and 100 samples, respectively. It seen that the DNN performance achieved  when using 10,000 training samples is virtually the same as that obtained when using 400,000 training samples (Figure \\ref{fig2} and Table \\ref{table2}). It is only for training sets smaller than 1000 samples that the DNN performance starts to degrade significantly (Table \\ref{table2}). In contrast, the behavior of kNN appears to markedly decrease as the training set gets smaller (Figure \\ref{fig2} and Table \\ref{table1}).\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{0cm}\\includegraphics[width=35pc]{Figure2.png}\n\t\\caption{1-1 plots of the kNN (subfigures (a) - (d)) and DNN (subfigures (e) - (h)) emulators' performance obtained for system 1 when the kNN training base contains 10,000 samples and the DNN is trained using the same 10,000 samples. Here ``true\" means the original PHREEQC-simulated data and ``predicted\" denotes the emulated (that is, kNN-simulated and DNN-simulated) data. Hence, the $x$-axis and $y$-axis present the original and emulated 10,000 independent test data points, respectively. The RMSE and $Q_2$ coefficient denote the root-mean-square-error and coefficient of determination in testing mode, respectively, between the original and emulated 10,000 test data points.}\n\t\\label{fig2}\n\\end{figure}\n\n\\begin{table}[h!]\n\t\\caption{Performance of the DNN and kNN emulators for cement system 1 and different training set sizes. For brevity, only the results for Ca$^{conc}$ and Si$^{conc}$ are shown. The units are mol per kg of water (mol/kgw). ML refers to the type of emulator, TR signifies the size of the training set, and the RMSE and $Q_2$ coefficient denote the root-mean-square-error and coefficient of determination in testing mode, respectively, between the original and emulated 10,000 test data points.}\n\t\\begin{center}\n\t\t\\begin{tabular}{cccccccc}\n\t\t\t\\hline\n\t\t\t& & & & & \\\\\n\t\t\tML & TR & RMSE - Ca$^{conc}$ & $Q_2$ - Ca$^{conc}$ & RMSE - Si$^{conc}$ & $Q_2$ - Si$^{conc}$\\\\\n\t\t\tDNN & 4 $\\times$ 10$^{5}$ & 4.50 $\\times$ 10$^{-5}$ & 0.9999 & 1.21 $\\times$ 10$^{-6}$ & 0.9999\\\\\n\t\t\tDNN & 1 $\\times$ 10$^{5}$ & 5.06 $\\times$ 10$^{-5}$ & 0.9999 & 1.30 $\\times$ 10$^{-6}$ & 0.9999\\\\\n\t\t\tDNN & 1 $\\times$ 10$^{4}$ & 4.57 $\\times$ 10$^{-5}$ & 0.9999 & 1.48 $\\times$ 10$^{-6}$ & 0.9999\\\\\n\t\t\tDNN & 1 $\\times$ 10$^{3}$ & 8.02 $\\times$ 10$^{-5}$ & 0.9998 & 3.31 $\\times$ 10$^{-6}$ & 0.9994\\\\\n\t\t\tDNN & 1 $\\times$ 10$^{2}$ & 47.1 $\\times$ 10$^{-5}$ & 0.9935 & 9.02 $\\times$ 10$^{-6}$ & 0.9959\\\\\n\t\t\tkNN & 4 $\\times$ 10$^{5}$ & 2.52 $\\times$ 10$^{-5}$ & 1.0000 & 1.59 $\\times$ 10$^{-6}$ & 0.9999\\\\\n\t\t\tkNN & 1 $\\times$ 10$^{5}$ & 6.27 $\\times$ 10$^{-5}$ & 0.9999 & 4.20 $\\times$ 10$^{-6}$ & 0.9991\\\\\n\t\t\tkNN & 1 $\\times$ 10$^{4}$ & 14.1 $\\times$ 10$^{-5}$ & 0.9994 & 13.5 $\\times$ 10$^{-6}$ & 0.9908\\\\\n\t\t\tkNN & 1 $\\times$ 10$^{3}$ & 43.9 $\\times$ 10$^{-5}$ & 0.9944 & 74.6 $\\times$ 10$^{-6}$ & 0.7180\\\\\n\t\t\tkNN & 1 $\\times$ 10$^{2}$  & 96.4 $\\times$ 10$^{-5}$ & 0.9729 & 55.0 $\\times$ 10$^{-6}$ & 0.8468\\\\\n\t\t\\end{tabular}\n\t\\end{center}\n\t\\label{table2}\n\\end{table}\n\n\\FloatBarrier\n\n\\subsubsection{Reactive transport simulation}\n\\label{prob12Dres}\n\nThis section focuses on reactive transport simulations with HPx$_{\\rm py}$ within cement system 1, under both advective-dispersive and diffusive transport conditions. As written above, the domain sizes are both 61 $\\times$ 61 and 121 $\\times$ 121 for the advection-dispersion case and, because of computational constraints, solely 61 $\\times$ 61  for the diffusion case. In addition, the simulation time period is 2 years for the advection-dispersion case and 1 year for the diffusion case. Figures \\ref{fig3} and \\ref{fig4} present times series of original and emulated Ca, Si, H and O concentrations at 5 locations within the 2D domain for advective-dispersive transport conditions, for both our kNN-based (HPx$_{\\rm py}$-kNN) and DNN-based (HPx$_{\\rm py}$-DNN) reactive transport codes. It is seen that HPx$_{\\rm py}$-kNN and HPx$_{\\rm py}$-DNN both induce a quite good simulation accuracy. Also, the results for the diffusive transport case are of similarly good quality (not shown). Figures \\ref{fig5} - \\ref{fig6} provide more insights into the HPx$_{\\rm py}$-kNN and HPx$_{\\rm py}$-DNN performances by displaying 2D Ca, Si, H and O concentration profiles at a given time. For each experiment and chemical component, this time is selected as to be well representative of the simulated dynamics. It is observed that the original and emulated images are visually almost indistinguishable for the advection-dispersion case (Figure \\ref{fig5}). For the diffusion case, the emulators also perform quite well for Ca, H and O (Figures\\ref{fig6}a - c, g - i and j - l), while some slight to moderate discrepancies appear at the concentration front for Si (Figures \\ref{fig6}d - f). Notwithstanding, the Si concentration remains globally well predicted. Furthermore, Figures \\ref{fig7} - \\ref{fig8} present the original and emulated 2D solid amount profiles corresponding to Figures \\ref{fig5} - \\ref{fig6}. The original solid amount profiles are overall well approximated by HPx$_{\\rm py}$-kNN and HPx$_{\\rm py}$-DNN for the advection-dispersion case (Figure \\ref{fig7}), even though some mismatch appears at the border of the fully depleted zone for the H component. As of the diffusion case (Figure \\ref{fig8}), the same kind of mismatch is observed for the emulated solid amounts of H by HPx$_{\\rm py}$-kNN while the emulated profiles by HPx$_{\\rm py}$-DNN show some more discrepancies. Though we decided to present raw emulation results, we would like to stress that some if not all of the observed artifacts could likely be smoothed out by using some post-filtering such as median filtering.\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure3.png}\n\t\\caption{Time series of original (RTM, solid green lines) and HPx$_{\\rm py}$-kNN emulated (TM+kNN, dashed orange lines) concentrations (mol/kg) of Ca, H, O and Si at selected observations points for cement system 1 and advective-dispersive transport. Obs. 1 - 5 denote the selected observation points, with the following $\\left[x,y\\right]$ locations (in cm). Obs. 1:  $\\left[0.5, 2.5\\right]$, Obs. 2: $\\left[1, 2\\right]$, Obs. 3: $\\left[2, 2\\right]$, Obs. 4: $\\left[1, 1\\right]$, Obs. 5: $\\left[2, 1\\right]$. The results for the 121 $\\times$ 121 grid size are rather similar.}\n\t\\label{fig3}\n\\end{figure}\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure4.png}\n\t\\caption{Time series of original (RTM, solid green lines) and HPx$_{\\rm py}$-DNN emulated (TM+DNN, dashed orange lines) concentrations (mol/kg) of Ca, H, O and Si at selected observations points for cement system 1 and advective-dispersive transport. Obs. 1 - 5 denote the selected observation points, with the following $\\left[x,y\\right]$ locations (in cm). Obs. 1:  $\\left[0.5, 2.5\\right]$, Obs. 2: $\\left[1, 2\\right]$, Obs. 3: $\\left[2, 2\\right]$, Obs. 4: $\\left[1, 1\\right]$, Obs. 5: $\\left[2, 1\\right]$. The results for the 121 $\\times$ 121 grid size are rather similar.}\n\t\\label{fig4}\n\\end{figure}\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure5.png}\n\t\\caption{2D concentration profiles obtained for cement system 1 at the end of the 2-year simulation performed for the advection-dispersion case. RTM means the original HP$_{\\rm{4C}}$ model, TM+kNN denotes the Hydrus transport model coupled with our kNN geochemical emulator (HPx$_{\\rm py}$-kNN) and TM+DNN signifies the Hydrus transport model coupled with our DNN geochemical emulator (HPx$_{\\rm py}$-DNN). The first to fourth row present profiles for $Ca^{\\rm{conc}}$, $Si^{\\rm{conc}}$, $H^{\\rm{conc}}$, and $O^{\\rm{conc}}$, respectively. The considered grid size is 61 $\\times$ 61. The results for the 121 $\\times$ 121 grid are rather similar.}\n\t\\label{fig5}\n\\end{figure}\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure6.png}\n\t\\caption{2D concentration profiles obtained for cement system 1 at the final time step of the 1-year simulation performed for the diffusion case. RTM means the original HP$_{\\rm{4C}}$ model, TM+kNN denotes the Hydrus transport model coupled with our kNN geochemical emulator (HPx$_{\\rm py}$-kNN) and TM+DNN signifies the Hydrus transport model coupled with our DNN geochemical emulator (HPx$_{\\rm py}$-DNN). The first to fourth row present profiles for $Ca^{\\rm{conc}}$, $Si^{\\rm{conc}}$, $H^{\\rm{conc}}$, and $O^{\\rm{conc}}$, respectively. The considered grid size is 61 $\\times$ 61.}\n\t\\label{fig6}\n\\end{figure}\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure7.png}\n\t\\caption{2D solid amount profiles obtained for cement system 1 at the end of the 2-year simulation performed for the advection-dispersion case. RTM means the original HP$_{\\rm{4C}}$ model, TM+kNN denotes the Hydrus transport model coupled with our kNN geochemical emulator (HPx$_{\\rm py}$-kNN) and TM+DNN signifies the Hydrus transport model coupled with our DNN geochemical emulator (HPx$_{\\rm py}$-DNN). The first to fourth row present profiles for $Ca^{\\rm{solid}}$, $Si^{\\rm{solid}}$, $H^{\\rm{solid}}$, and $O^{\\rm{solid}}$, respectively. The considered grid size is 61 $\\times$ 61. The results for the 121 $\\times$ 121 grid are rather similar.}\n\t\\label{fig7}\n\\end{figure}\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure8.png}\n\t\\caption{2D solid amount profiles obtained for cement system 1 at the final time step of the 1-year simulation performed for the diffusion case. RTM means the original HP$_{\\rm{4C}}$ model, TM+kNN denotes the Hydrus transport model coupled with our kNN geochemical emulator (HPx$_{\\rm py}$-kNN) and TM+DNN signifies the Hydrus transport model coupled with our DNN geochemical emulator (HPx$_{\\rm py}$-DNN). The first to fourth row present profiles for $Ca^{\\rm{solid}}$, $Si^{\\rm{solid}}$, $H^{\\rm{solid}}$, and $O^{\\rm{solid}}$, respectively. The considered grid size is 61 $\\times$ 61.}\n\t\\label{fig8}\n\\end{figure}\n\nThe speedups associated with the considered problem are detailed in Table \\ref{table3}. It is noted that the GPU-based DNN emulator allows for a speedup that is close to optimal. Indeed, The DNN speedups overall represent 85 \\% to 95 \\% of the maximum possible speedups (that is, speedups that would be obtained if the geochemcical calculations would come at no cost at all). The speedups associated with single-threaded kNN remain substantial but only amount to 57 \\% - 65 \\% of the corresponding maximum speedups. As detailed in section \\ref{train_res1}, the used kNN and DNN implementations are found to be respectively 300 and 4000 faster than single-threaded PHREEQC when predicting 10,000 points all one once for this geochemical system. Based on these numbers one could have expected the achieved speedups to represent say 90 \\% (kNN) or 99 \\% (DNN) of the maximum possible ones. A large part of the gaps between achieved and maximum possible speedups is thus likely caused by the time required for communicating and exchanging data between the main C/C++ code and the Python-based emulators.\n\n\\begin{table}[h!]\n\t\\caption{Speedups offered by the KNN and DNN emulators in HPx$_{\\rm py}$ for the reactive transport simulations considered for cement system 1. The HPx$_{\\rm{4C}}$ calculations involve the parallelization of PHREEQC over our 4 CPUs. The HPx$_{\\rm{1C}}$ calculations are performed on a single CPU. The kNN predictions are performed on a single CPU using the scikit-learn implementation while the DNN predictions make use of our GPU. ML signifies the used machine learning method for emulation, TC denotes transport conditions (ADV: advection-dispersion, DIF: diffusion) and GS is the grid size. The maximum possible speedups associated with HPx$_{\\rm{4C}}$ and HPx$_{\\rm{1C}}$,  Max SP HPx$_{\\rm{4C}}$ and  Max SP HPx$_{\\rm{1C}}$, correspond to an hypothetical situation where the geochemical calculations incur zero computational cost.}\n\n\t\\begin{center}\n\t\t\\begin{tabular}{cccccccc}\n\t\t\t\\hline\n\t\t\t& & & & & & & \\\\\n\t\t\tML & TC & GS & HPx$_{\\rm{4C}}$ time (s) & SP HPx$_{\\rm{4C}}$ & Max SP HPx$_{\\rm{4C}}$ & SP HPx$_{\\rm{1C}}$ & Max SP HPx$_{\\rm{1C}}$ \\\\\n\t\t\tDNN & ADV & 61 $\\times$ 61 & 3189 & 6.8 & 7.7 & 24.5 & 28.5 \\\\\n\t\t\tkNN & ADV & 61 $\\times$ 61 & 3189 & 5.0 & 7.7 & 18.0 & 28.5 \\\\\n\t\t\tDNN & ADV & 121 $\\times$ 121 & 23,337 & 5.0 & 5.2 & 17.0 & 18.0 \\\\\n\t\t\tkNN & ADV & 121 $\\times$ 121 & 23,337 & 3.4 & 5.2 & 11.6 & 18.0 \\\\\n\t\t\tDNN & DIF & 61 $\\times$ 61 & 25,448 & 4.2 & 4.9 & 16.2 & 19.1 \\\\\n\t\t\tkNN & DIF & 61 $\\times$ 61 & 25,448 & 2.8 & 4.9 & 10.8 & 19.1 \\\\\n\t\t\t\\hline\n\t\t\\end{tabular}\n\t\\end{center}\n\n\t\\label{table3}\n\\end{table}\n\n\\FloatBarrier\n\n\\subsection{Al-C-Ca-S-Si Problem}\n\nFor this second cement system, the emulation problem consists of predicting at each time step of the RT simulation the (output) Al, C, Ca, S, Si, H and O aqueous concentrations (mol/kgw) from the (input) total amounts of Al, C, Ca, S and Si (mol). Here we focus on advective-dispersive transport only while similarly as for cement system 1, the considered domain sizes are 61 $\\times$ 61 and 121 $\\times$ 121. Moreover, the simulation time period is set to 6 years. \n\nAs mentioned earlier, for this higher-dimensional problem it is observed that the used scikit-learn kNN implementation becomes prohibitively slow compared to HPx$_{\\rm{4C}}$. We found that to get a good emulation accuracy the kNN training base needs to contain 1,000,000 samples (or more). This training base's size together with a 5-dimensional search space leads to an HPx$_{\\rm py}$-kNN reactive transport simulation time that is comparable to that of HPx$_{\\rm{4C}}$. Therefore, for this second cement system we built a custom kNN regressor around another kNN implementation contained in the FAISS package \\citep{faiss2017}. Our used FAISS variant allows for GPU computing and is much faster than scikit-learn for this cement system, but is slightly less accurate due to the use of an approximate rather than exact nearest neighbor search \\citep[see][for details]{faiss2017}.\n\n\\subsubsection{Training the DNN}\n\\label{train_res2}\n\nBuilding a good training set to perform a kNN search and learn the weights and biases of our DNN turned out to be a complicated task in this case. This because to make useful kNN predictions and/or learn an useful DNN, the training set must be sufficiently representative of the geochemical conditions encountered during the reactive transport simulation one wish to perform with HPx$_{\\rm py}$-kNN and the trained HPx$_{\\rm py}$-DNN. In contrast to cement system 1, creating the training set by sampling the $X$-space with a controlled randomness between predefined lower and upper bounds did not prove successful. We tried that strategy by drawing as much as 4,000,000 5-dimensional $\\textbf{x}$ vectors from the $X$-space using a Sobol low-discrepancy sequence \\citep[][]{Sobol1967, Joe-Kuo2003}. Such low-discrepancy sampling scheme covers the 5-dimensional hypercube more uniformly than LHS. Despite a good performance on the test set (not shown), the resulting DNN accuracy in reactive transport mode was never deemed satisfying. In other words, no satisfying ``global\" or ``universal\" DNN emulator could be devised for this cement system. This is probably caused by the fact for this problem, the input (5 total amounts) and output (7 aqueous concentrations) spaces are quite nonlinearly related and both cover 6 to 10 orders of magnitudes depending on the considered element. Therefore, we resorted to the alternative training strategy detailed below. The latter basically tries to grasp the complex correlations and higher-order dependencies that exist between the elements of $\\textbf{x}$ (total amounts, input space) for a given reactive transport simulation setup, in order to produce a training set that honors these between-input relationships.\n\n\\begin{itemize}\n\t\\item Perform a ``cheap\" full reactive transport simulation under the transport conditions and geochemistry of interest and collect the resulting $\\textbf{x}$ -$\\textbf{y}$  pairs of examples (for the considered grid nodes and time steps). Computational demand controls what domain size and simulation time period can be used for this cheap calculation. We used a modest 16 $\\times$ 16 domain and a simulation time period of 10 years. The associated HPx$_{\\rm{4C}}$ runtimes is 180 s.\n\t\n\t\\item Fit a kernel density estimator (KDE) with a Gaussian kernel to the collected $\\textbf{x}$ vectors (encapsulated in the $\\textbf{X}$ array) and generate a fixed number of new input vectors, $\\textbf{x}_{KDE}$. Then run PHREEQC for the $\\textbf{X}_{KDE}$ set to get the corresponding output set, $\\textbf{Y}_{KDE}$. Now apply the correction for porosity described in section \\ref{train_res1} to the $\\textbf{X}_{KDE}$ set and form the training set by merging the ensemble of $\\textbf{x}$-$\\textbf{y}$ pairs with that of the $\\textbf{x}_{KDE}$-$\\textbf{y}_{KDE}$ pairs. The number of produced unique examples by the considered cheap HPx$_{\\rm{4C}}$ simulations varied between 10,000 and 50,000. The KDE-based enrichment of this dataset was deemed necessary to provide more input variability, thereby avoiding overfitting of the trained DNN and improving the kNN accuracy, while still honoring the complex between-input relationships. The number of KDE-generated samples was set as to obtain a total training set size of 1,000,000 examples. A key component of the approach is the bandwith parameter of the Gaussian KDE kernel which controls how much the KDE-generated samples depart from the original ensemble. After limited trial and error, we fixed the kernel bandwith to 0.0025 for the considered case studies.\n\t\n\\end{itemize}\n\nThe scatter plots in Figure \\ref{fig9} illustrate our training set creation procedure. The orange dots depict the pairwise relationships observed between the 5 elements of $\\textbf{x}$ in the cheap simulation. The red and cyan dots in Figure \\ref{fig9} represent the KDE-generated samples with the selected bandwidth, before and after applying the correction for porosity, respectively. Training of the DNN is achieved using the ensemble of original and KDE-corrected input points. We refer to this kind of dataset as RT-based, since it is based on an albeit cheap, full RT simulation. Furthermore, we refer to the obtained DNN and kNN emulators as ``local\" emulators, since as opposed to the emulators constructed for cement system 1, the current emulators are only valid for the input conditions encapsulated in the RT-based training set.\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1.5cm}\\includegraphics[width=50pc]{Figure9.png}\n\t\\caption{Scatter plots of the complex relationships between the five considered inputs for (1) the computationally cheap RT simulation of cement system 2 (orange dots) performed with the original HPx code, (2) the corresponding sampled points by kernel density estimation (KDE) using the selected bandwith (turquoise dots) and, (3) the same sampled KDE points after porosity correction (red dots).}\n\t\\label{fig9}\n\\end{figure}\n\nTraining performance of the DNN emulator is presented in Figure \\ref{fig10}. Similarly as for cement system 1, about 90\\% of the available data was used for the actual training of the DNN while the remaining 10\\% were used a validation set to contol overfitting. Lastly, performance is evaluated for both the trained DNN and kNN emulators using an independent test set of 10,000 examples. Overall, the accuracy of our ``local \" DNN emulator for this RT-inspired dataset is rather large with $Q_2$ values always greater or equal than 0.998. Training performance of the corresponding local kNN emulator is equally good (not shown). Regarding speedup, for this problem single-threaded PHREEQC achieves about 210 geochemical calculations per second on our used Intel\\textsuperscript{\\textregistered} i7 CPU while the GPU-based DNN and kNN emulators are both about 3000 times faster when predicting the 10,000 test points at once .\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure10.png}\n\t\\caption{1-1 plots of local DNN emulation performance obtained for system 2 when the local DNN is trained using 1,000,000 samples. The $x$-axis and $y$-axis present the original and emulated 10,000 independent test data points, respectively. The RMSE and $Q_2$ coefficient denote the root-mean-square-error and coefficient of determination in testing mode, respectively, between the original and emulated 10,000 test data points.}\n\t\\label{fig10}\n\\end{figure}\n\n\\subsubsection{Reactive transport simulation}\n\\label{prob2res}\nOur ``local\" DNN performs rather well when applied to the 61 $\\times$ 61 grid size and a time period of 6 years (Figures \\ref{fig11} - \\ref{fig12}). This for all components but C, for which some localized deviations appear between original and emulated 2D concentration profiles towards the end of the simulation period (Figure \\ref{fig12}). When applied to the 121 $\\times$ 121 grid, HPx$_{\\rm py}$-DNN produces additional discrepancies for O, Si and Al towards the end of the simulation period (Figures \\ref{fig13} - \\ref{fig14}). Yet most of the observed artifacts could probably be smoothed out by using post-filtering. The associated speedups are listed in Table \\ref{table4}. These speedups are larger to those obtained for cement system 1, with values between 8 and 9 when evaluated against HPx$_{\\rm{4C}}$. These speedups represent about 85 \\% to 90 \\% of the maximum possible speedup (Table \\ref{table3}). Overall, these findings indicate that for the considered problem, our RT-based training of a local DNN only works is the training set is sufficiently representative of the particular geochemical conditions encountered in the computationally demanding simulations, which is arguably not easy to achieve. This limitation is further discussed in section \\ref{discussion}.\n\nWe note a more uniform behavior for HPx$_{\\rm py}$-kNN across grid sizes than for HPx$_{\\rm py}$-DNN. Here the results for the 121  $\\times$ 121 grid are only slightly less accurate than those associated with the 61 $ \\times$ 61 grid (see Figures \\ref{fig15} - \\ref{fig16} where for brevity we only show concentration profiles for the C Al and S components). Furthermore, whenever observed the discrepancies between original and emulated profiles are more regularly scattered than for HPx$_{\\rm py}$-DNN. Note also that herein too, post-filtering could likely smooth out a large part of these deviations. In addition, owing to the use of a GPU to achieve the kNN calculations, the speedup provided by HPx$_{\\rm py}$-kNN are as large as those provided by HPx$_{\\rm py}$-DNN (Table \\ref{table4}).\n\nWith respect to the emulated solid amounts, the HPx$_{\\rm py}$-DNN results look visually good for the the 61 $\\times$ 61 grid. This is shown in Figure \\ref{fig17} for the C, Al and S chemical components, while emulation of the H, O Ca and Si chemical components is globally of similar quality (not shown). Nevertheless, significant deviations appear towards the end of the simulation period for every chemical component (see Figure \\ref{fig18} for the C, Al and S chemical components, emulation of the H, O Ca and Si chemical components shows the same level of mismatch). The HPx$_{\\rm py}$-kNN predictions are also fairly accurate for the 61 $\\times$ 61 (see Figure \\ref{fig19} for the C, Al and S chemical components, emulation of the H, O Ca and Si chemical components exhibits a globally similar quality) while some discrepancies show up at the end of the simulation (Figure \\ref{fig20}). However, the mismatch is less pronounced than for HPx$_{\\rm py}$-DNN. Overall, HPx$_{\\rm py}$-kNN appears to be somewhat more robust than HPx$_{\\rm py}$-DNN for this cement system, while providing the same (large) speedup.\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure11.png}\n\t\\caption{2D concentration profiles obtained for cement system 2 after 2 and 6 (final time step) years. RTM means the original HPx$_{\\rm{4C}}$ model and TM+DNN denotes the Hydrus transport model coupled with our DNN geochemical emulator (HPx$_{\\rm py}$-DNN). The first to fourth row present profiles for $Ca^{\\rm{conc}}$, $Si^{\\rm{conc}}$, $H^{\\rm{conc}}$, and $O^{\\rm{conc}}$, respectively. The considered grid size is 61 $\\times$ 61.}\n\t\\label{fig11}\n\\end{figure}\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure12.png}\n\t\\caption{2D concentration profiles obtained for cement system 2 after 2 and 6 (final time step) years. RTM means the original HPx$_{\\rm{4C}}$ model and TM+DNN denotes the Hydrus transport model coupled with our DNN geochemical emulator (HPx$_{\\rm py}$-DNN). The first to third row present profiles for $C^{\\rm{conc}}$, $Al^{\\rm{conc}}$, and $S^{\\rm{conc}}$, respectively. The considered grid size is 61 $\\times$ 61.}\n\t\\label{fig12}\n\\end{figure}\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure13.png}\n\t\\caption{2D concentration profiles obtained for cement system 2 after 2 and 6 (final time step) years. RTM means the original HPx$_{\\rm{4C}}$ model and TM+DNN denotes the Hydrus transport model coupled with our DNN geochemical emulator (HPx$_{\\rm py}$-DNN). The first to fourth row present profiles for $Ca^{\\rm{conc}}$, $Si^{\\rm{conc}}$, $H^{\\rm{conc}}$, and $O^{\\rm{conc}}$, respectively. The considered grid size is 121 $\\times$ 121.}\n\t\\label{fig13}\n\\end{figure}\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure14.png}\n\t\\caption{2D concentration profiles obtained for cement system 2 after 2 and 6 (final time step) years. RTM means the original HPx$_{\\rm{4C}}$ model and TM+DNN denotes the Hydrus transport model coupled with our DNN geochemical emulator (HPx$_{\\rm py}$-DNN). The first to third row present profiles for $C^{\\rm{conc}}$, $Al^{\\rm{conc}}$, and $S^{\\rm{conc}}$, respectively. The considered grid size is 121 $\\times$ 121.}\n\t\\label{fig14}\n\\end{figure}\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure15.png}\n\t\\caption{2D concentration profiles obtained for cement system 2 after 2 and 6 (final time step) years. RTM means the original HPx$_{\\rm{4C}}$ model and TM+kNN denotes the Hydrus transport model coupled with our kNN geochemical emulator (HPx$_{\\rm py}$-kNN).  The first to third row present profiles for $C^{\\rm{conc}}$, $Al^{\\rm{conc}}$, and $S^{\\rm{conc}}$, respectively. The considered grid size is 61 $\\times$ 61.}\n\t\\label{fig15}\n\\end{figure}\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure16.png}\n\t\\caption{2D concentration profiles obtained for cement system 2 after 2 and 6 (final time step) years. RTM means the original HPx$_{\\rm{4C}}$ model and TM+kNN denotes the Hydrus transport model coupled with our kNN geochemical emulator (HPx$_{\\rm py}$-kNN). The first to third row present profiles for $C^{\\rm{conc}}$, $Al^{\\rm{conc}}$, and $S^{\\rm{conc}}$, respectively. The considered grid size is 121 $\\times$ 121.}\n\t\\label{fig16}\n\\end{figure}\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure17.png}\n\t\\caption{2D solid amount profiles of the C, Al and S chemical components of cement system 2 after 2 and 6 (final time step) years. RTM means the original HPx$_{\\rm{4C}}$ model and TM+DNN denotes the Hydrus transport model coupled with our DNN geochemical emulator (HPx$_{\\rm py}$-DNN). The first to third row present profiles for $C^{\\rm{solid}}$, $Al^{\\rm{solid}}$, and $S^{\\rm{solid}}$, respectively The considered grid size is 61 $\\times$ 61.}\n\t\\label{fig17}\n\\end{figure}\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure18.png}\n\t\\caption{2D solid amount profiles of the C, Al and S chemical components of cement system 2 after 2 and 6 (final time step) years. RTM means the original HPx$_{\\rm{4C}}$ model and TM+DNN denotes the Hydrus transport model coupled with our DNN geochemical emulator (HPx$_{\\rm py}$-DNN). The first to third row present profiles for $C^{\\rm{solid}}$, $Al^{\\rm{solid}}$, and $S^{\\rm{solid}}$, respectively The considered grid size is 121 $\\times$ 121.}\n\t\\label{fig18}\n\\end{figure}\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure19.png}\n\t\\caption{2D solid amount profiles of the C, Al and S chemical components of cement system 2 after 2 and 6 (final time step) years. RTM means the original HPx$_{\\rm{4C}}$ model and TM+kNN denotes the Hydrus transport model coupled with our kNN geochemical emulator (HPx$_{\\rm py}$-kNN). The first to third row present profiles for $C^{\\rm{solid}}$, $Al^{\\rm{solid}}$, and $S^{\\rm{solid}}$, respectively The considered grid size is 61 $\\times$ 61.}\n\t\\label{fig19}\n\\end{figure}\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure20.png}\n\t\\caption{2D solid amount profiles of the C, Al and S chemical components of cement system 2 after 2 and 6 (final time step) years. RTM means the original HPx$_{\\rm{4C}}$ model and TM+kNN denotes the Hydrus transport model coupled with our kNN geochemical emulator (HPx$_{\\rm py}$-kNN). The first to third row present profiles for $C^{\\rm{solid}}$, $Al^{\\rm{solid}}$, and $S^{\\rm{solid}}$, respectively The considered grid size is 121 $\\times$ 121.}\n\t\\label{fig20}\n\\end{figure}\n\n\\begin{table}[h!]\n\t\\caption{Speedups offered by the KNN and DNN emulators in HPx$_{\\rm py}$ for the reactive transport simulations considered for cement system 2. The HPx$_{\\rm{4C}}$ calculations involve the parallelization of PHREEQC over our 4 CPUs. The HPx$_{\\rm{1C}}$ calculations are performed on a single CPU. Both the kNN and DNN predictions make use of a GPU. ML signifies the used machine learning method for emulation, BC denotes the type of flow boundary conditions and GS is the grid size. The maximum possible speedups associated with HPx$_{\\rm{4C}}$ and HP$_{\\rm{1C}}$,  Max SP HPx$_{\\rm{4C}}$ and  Max SP HPx$_{\\rm{1C}}$, correspond to an hypothetical situation where the geochemical calculations incur zero computational cost.}\n\n\t\\begin{center}\n\t\t\\begin{tabular}{cccccccc}\n\t\t\t\\hline\n\t\t\t& & & & & & & \\\\\n\t\t\tML & BC & GS & HPx$_{\\rm{4C}}$ time (s) & SP HPx$_{\\rm{4C}}$ & Max SP HPx$_{\\rm{4C}}$ & SP HPx$_{\\rm{1C}}$ & Max SP HPx$_{\\rm{1C}}$ \\\\\n\t\t\tDNN & ADV & 61 $\\times$ 61 & 21,415 & 8.2 & 9.0 & 30.3 & 33.1\\\\\n\t\t\tkNN & ADV & 61 $\\times$ 61 & 21,415 & 7.9 & 9.0 & 28.9 & 33.1\\\\\n\t\t\tDNN & ADV & 121 $\\times$ 121 & 199,841 & 8.2 & 9.5 & 29.9 & 32.8\\\\\n\t\t\tkNN & ADV & 121 $\\times$ 121 & 199,841 & 8.5 & 9.5 & 31.3 & 32.8\\\\\n\t\t\t\\hline\n\t\t\\end{tabular}\n\t\\end{center}\n\n\t\\label{table4}\n\\end{table}\n\n\\FloatBarrier\n\n3.2 Al-C-Ca-S-Si Problem\n\\subsection{Al-C-Ca-S-Si Problem}\n\nFor this second cement system, the emulation problem consists of predicting at each time step of the RT simulation the (output) Al, C, Ca, S, Si, H and O aqueous concentrations (mol/kgw) from the (input) total amounts of Al, C, Ca, S and Si (mol). Here we focus on advective-dispersive transport only while similarly as for cement system 1, the considered domain sizes are 61 $\\times$ 61 and 121 $\\times$ 121. Moreover, the simulation time period is set to 6 years. \n\nAs mentioned earlier, for this higher-dimensional problem it is observed that the used scikit-learn kNN implementation becomes prohibitively slow compared to HPx$_{\\rm{4C}}$. We found that to get a good emulation accuracy the kNN training base needs to contain 1,000,000 samples (or more). This training base's size together with a 5-dimensional search space leads to an HPx$_{\\rm py}$-kNN reactive transport simulation time that is comparable to that of HPx$_{\\rm{4C}}$. Therefore, for this second cement system we built a custom kNN regressor around another kNN implementation contained in the FAISS package \\citep{faiss2017}. Our used FAISS variant allows for GPU computing and is much faster than scikit-learn for this cement system, but is slightly less accurate due to the use of an approximate rather than exact nearest neighbor search \\citep[see][for details]{faiss2017}.\n\n\\subsubsection{Training the DNN}\n\\label{train_res2}\n\nBuilding a good training set to perform a kNN search and learn the weights and biases of our DNN turned out to be a complicated task in this case. This because to make useful kNN predictions and/or learn an useful DNN, the training set must be sufficiently representative of the geochemical conditions encountered during the reactive transport simulation one wish to perform with HPx$_{\\rm py}$-kNN and the trained HPx$_{\\rm py}$-DNN. In contrast to cement system 1, creating the training set by sampling the $X$-space with a controlled randomness between predefined lower and upper bounds did not prove successful. We tried that strategy by drawing as much as 4,000,000 5-dimensional $\\textbf{x}$ vectors from the $X$-space using a Sobol low-discrepancy sequence \\citep[][]{Sobol1967, Joe-Kuo2003}. Such low-discrepancy sampling scheme covers the 5-dimensional hypercube more uniformly than LHS. Despite a good performance on the test set (not shown), the resulting DNN accuracy in reactive transport mode was never deemed satisfying. In other words, no satisfying ``global\" or ``universal\" DNN emulator could be devised for this cement system. This is probably caused by the fact for this problem, the input (5 total amounts) and output (7 aqueous concentrations) spaces are quite nonlinearly related and both cover 6 to 10 orders of magnitudes depending on the considered element. Therefore, we resorted to the alternative training strategy detailed below. The latter basically tries to grasp the complex correlations and higher-order dependencies that exist between the elements of $\\textbf{x}$ (total amounts, input space) for a given reactive transport simulation setup, in order to produce a training set that honors these between-input relationships.\n\n\\begin{itemize}\n\t\\item Perform a ``cheap\" full reactive transport simulation under the transport conditions and geochemistry of interest and collect the resulting $\\textbf{x}$ -$\\textbf{y}$  pairs of examples (for the considered grid nodes and time steps). Computational demand controls what domain size and simulation time period can be used for this cheap calculation. We used a modest 16 $\\times$ 16 domain and a simulation time period of 10 years. The associated HPx$_{\\rm{4C}}$ runtimes is 180 s.\n\t\n\t\\item Fit a kernel density estimator (KDE) with a Gaussian kernel to the collected $\\textbf{x}$ vectors (encapsulated in the $\\textbf{X}$ array) and generate a fixed number of new input vectors, $\\textbf{x}_{KDE}$. Then run PHREEQC for the $\\textbf{X}_{KDE}$ set to get the corresponding output set, $\\textbf{Y}_{KDE}$. Now apply the correction for porosity described in section \\ref{train_res1} to the $\\textbf{X}_{KDE}$ set and form the training set by merging the ensemble of $\\textbf{x}$-$\\textbf{y}$ pairs with that of the $\\textbf{x}_{KDE}$-$\\textbf{y}_{KDE}$ pairs. The number of produced unique examples by the considered cheap HPx$_{\\rm{4C}}$ simulations varied between 10,000 and 50,000. The KDE-based enrichment of this dataset was deemed necessary to provide more input variability, thereby avoiding overfitting of the trained DNN and improving the kNN accuracy, while still honoring the complex between-input relationships. The number of KDE-generated samples was set as to obtain a total training set size of 1,000,000 examples. A key component of the approach is the bandwith parameter of the Gaussian KDE kernel which controls how much the KDE-generated samples depart from the original ensemble. After limited trial and error, we fixed the kernel bandwith to 0.0025 for the considered case studies.\n\t\n\\end{itemize}\n\nThe scatter plots in Figure \\ref{fig9} illustrate our training set creation procedure. The orange dots depict the pairwise relationships observed between the 5 elements of $\\textbf{x}$ in the cheap simulation. The red and cyan dots in Figure \\ref{fig9} represent the KDE-generated samples with the selected bandwidth, before and after applying the correction for porosity, respectively. Training of the DNN is achieved using the ensemble of original and KDE-corrected input points. We refer to this kind of dataset as RT-based, since it is based on an albeit cheap, full RT simulation. Furthermore, we refer to the obtained DNN and kNN emulators as ``local\" emulators, since as opposed to the emulators constructed for cement system 1, the current emulators are only valid for the input conditions encapsulated in the RT-based training set.\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1.5cm}\\includegraphics[width=50pc]{Figure9.png}\n\t\\caption{Scatter plots of the complex relationships between the five considered inputs for (1) the computationally cheap RT simulation of cement system 2 (orange dots) performed with the original HPx code, (2) the corresponding sampled points by kernel density estimation (KDE) using the selected bandwith (turquoise dots) and, (3) the same sampled KDE points after porosity correction (red dots).}\n\t\\label{fig9}\n\\end{figure}\n\nTraining performance of the DNN emulator is presented in Figure \\ref{fig10}. Similarly as for cement system 1, about 90\\% of the available data was used for the actual training of the DNN while the remaining 10\\% were used a validation set to contol overfitting. Lastly, performance is evaluated for both the trained DNN and kNN emulators using an independent test set of 10,000 examples. Overall, the accuracy of our ``local \" DNN emulator for this RT-inspired dataset is rather large with $Q_2$ values always greater or equal than 0.998. Training performance of the corresponding local kNN emulator is equally good (not shown). Regarding speedup, for this problem single-threaded PHREEQC achieves about 210 geochemical calculations per second on our used Intel\\textsuperscript{\\textregistered} i7 CPU while the GPU-based DNN and kNN emulators are both about 3000 times faster when predicting the 10,000 test points at once .\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure10.png}\n\t\\caption{1-1 plots of local DNN emulation performance obtained for system 2 when the local DNN is trained using 1,000,000 samples. The $x$-axis and $y$-axis present the original and emulated 10,000 independent test data points, respectively. The RMSE and $Q_2$ coefficient denote the root-mean-square-error and coefficient of determination in testing mode, respectively, between the original and emulated 10,000 test data points.}\n\t\\label{fig10}\n\\end{figure}\n\n\\subsubsection{Reactive transport simulation}\n\\label{prob2res}\nOur ``local\" DNN performs rather well when applied to the 61 $\\times$ 61 grid size and a time period of 6 years (Figures \\ref{fig11} - \\ref{fig12}). This for all components but C, for which some localized deviations appear between original and emulated 2D concentration profiles towards the end of the simulation period (Figure \\ref{fig12}). When applied to the 121 $\\times$ 121 grid, HPx$_{\\rm py}$-DNN produces additional discrepancies for O, Si and Al towards the end of the simulation period (Figures \\ref{fig13} - \\ref{fig14}). Yet most of the observed artifacts could probably be smoothed out by using post-filtering. The associated speedups are listed in Table \\ref{table4}. These speedups are larger to those obtained for cement system 1, with values between 8 and 9 when evaluated against HPx$_{\\rm{4C}}$. These speedups represent about 85 \\% to 90 \\% of the maximum possible speedup (Table \\ref{table3}). Overall, these findings indicate that for the considered problem, our RT-based training of a local DNN only works is the training set is sufficiently representative of the particular geochemical conditions encountered in the computationally demanding simulations, which is arguably not easy to achieve. This limitation is further discussed in section \\ref{discussion}.\n\nWe note a more uniform behavior for HPx$_{\\rm py}$-kNN across grid sizes than for HPx$_{\\rm py}$-DNN. Here the results for the 121  $\\times$ 121 grid are only slightly less accurate than those associated with the 61 $ \\times$ 61 grid (see Figures \\ref{fig15} - \\ref{fig16} where for brevity we only show concentration profiles for the C Al and S components). Furthermore, whenever observed the discrepancies between original and emulated profiles are more regularly scattered than for HPx$_{\\rm py}$-DNN. Note also that herein too, post-filtering could likely smooth out a large part of these deviations. In addition, owing to the use of a GPU to achieve the kNN calculations, the speedup provided by HPx$_{\\rm py}$-kNN are as large as those provided by HPx$_{\\rm py}$-DNN (Table \\ref{table4}).\n\nWith respect to the emulated solid amounts, the HPx$_{\\rm py}$-DNN results look visually good for the the 61 $\\times$ 61 grid. This is shown in Figure \\ref{fig17} for the C, Al and S chemical components, while emulation of the H, O Ca and Si chemical components is globally of similar quality (not shown). Nevertheless, significant deviations appear towards the end of the simulation period for every chemical component (see Figure \\ref{fig18} for the C, Al and S chemical components, emulation of the H, O Ca and Si chemical components shows the same level of mismatch). The HPx$_{\\rm py}$-kNN predictions are also fairly accurate for the 61 $\\times$ 61 (see Figure \\ref{fig19} for the C, Al and S chemical components, emulation of the H, O Ca and Si chemical components exhibits a globally similar quality) while some discrepancies show up at the end of the simulation (Figure \\ref{fig20}). However, the mismatch is less pronounced than for HPx$_{\\rm py}$-DNN. Overall, HPx$_{\\rm py}$-kNN appears to be somewhat more robust than HPx$_{\\rm py}$-DNN for this cement system, while providing the same (large) speedup.\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure11.png}\n\t\\caption{2D concentration profiles obtained for cement system 2 after 2 and 6 (final time step) years. RTM means the original HPx$_{\\rm{4C}}$ model and TM+DNN denotes the Hydrus transport model coupled with our DNN geochemical emulator (HPx$_{\\rm py}$-DNN). The first to fourth row present profiles for $Ca^{\\rm{conc}}$, $Si^{\\rm{conc}}$, $H^{\\rm{conc}}$, and $O^{\\rm{conc}}$, respectively. The considered grid size is 61 $\\times$ 61.}\n\t\\label{fig11}\n\\end{figure}\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure12.png}\n\t\\caption{2D concentration profiles obtained for cement system 2 after 2 and 6 (final time step) years. RTM means the original HPx$_{\\rm{4C}}$ model and TM+DNN denotes the Hydrus transport model coupled with our DNN geochemical emulator (HPx$_{\\rm py}$-DNN). The first to third row present profiles for $C^{\\rm{conc}}$, $Al^{\\rm{conc}}$, and $S^{\\rm{conc}}$, respectively. The considered grid size is 61 $\\times$ 61.}\n\t\\label{fig12}\n\\end{figure}\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure13.png}\n\t\\caption{2D concentration profiles obtained for cement system 2 after 2 and 6 (final time step) years. RTM means the original HPx$_{\\rm{4C}}$ model and TM+DNN denotes the Hydrus transport model coupled with our DNN geochemical emulator (HPx$_{\\rm py}$-DNN). The first to fourth row present profiles for $Ca^{\\rm{conc}}$, $Si^{\\rm{conc}}$, $H^{\\rm{conc}}$, and $O^{\\rm{conc}}$, respectively. The considered grid size is 121 $\\times$ 121.}\n\t\\label{fig13}\n\\end{figure}\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure14.png}\n\t\\caption{2D concentration profiles obtained for cement system 2 after 2 and 6 (final time step) years. RTM means the original HPx$_{\\rm{4C}}$ model and TM+DNN denotes the Hydrus transport model coupled with our DNN geochemical emulator (HPx$_{\\rm py}$-DNN). The first to third row present profiles for $C^{\\rm{conc}}$, $Al^{\\rm{conc}}$, and $S^{\\rm{conc}}$, respectively. The considered grid size is 121 $\\times$ 121.}\n\t\\label{fig14}\n\\end{figure}\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure15.png}\n\t\\caption{2D concentration profiles obtained for cement system 2 after 2 and 6 (final time step) years. RTM means the original HPx$_{\\rm{4C}}$ model and TM+kNN denotes the Hydrus transport model coupled with our kNN geochemical emulator (HPx$_{\\rm py}$-kNN).  The first to third row present profiles for $C^{\\rm{conc}}$, $Al^{\\rm{conc}}$, and $S^{\\rm{conc}}$, respectively. The considered grid size is 61 $\\times$ 61.}\n\t\\label{fig15}\n\\end{figure}\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure16.png}\n\t\\caption{2D concentration profiles obtained for cement system 2 after 2 and 6 (final time step) years. RTM means the original HPx$_{\\rm{4C}}$ model and TM+kNN denotes the Hydrus transport model coupled with our kNN geochemical emulator (HPx$_{\\rm py}$-kNN). The first to third row present profiles for $C^{\\rm{conc}}$, $Al^{\\rm{conc}}$, and $S^{\\rm{conc}}$, respectively. The considered grid size is 121 $\\times$ 121.}\n\t\\label{fig16}\n\\end{figure}\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure17.png}\n\t\\caption{2D solid amount profiles of the C, Al and S chemical components of cement system 2 after 2 and 6 (final time step) years. RTM means the original HPx$_{\\rm{4C}}$ model and TM+DNN denotes the Hydrus transport model coupled with our DNN geochemical emulator (HPx$_{\\rm py}$-DNN). The first to third row present profiles for $C^{\\rm{solid}}$, $Al^{\\rm{solid}}$, and $S^{\\rm{solid}}$, respectively The considered grid size is 61 $\\times$ 61.}\n\t\\label{fig17}\n\\end{figure}\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure18.png}\n\t\\caption{2D solid amount profiles of the C, Al and S chemical components of cement system 2 after 2 and 6 (final time step) years. RTM means the original HPx$_{\\rm{4C}}$ model and TM+DNN denotes the Hydrus transport model coupled with our DNN geochemical emulator (HPx$_{\\rm py}$-DNN). The first to third row present profiles for $C^{\\rm{solid}}$, $Al^{\\rm{solid}}$, and $S^{\\rm{solid}}$, respectively The considered grid size is 121 $\\times$ 121.}\n\t\\label{fig18}\n\\end{figure}\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure19.png}\n\t\\caption{2D solid amount profiles of the C, Al and S chemical components of cement system 2 after 2 and 6 (final time step) years. RTM means the original HPx$_{\\rm{4C}}$ model and TM+kNN denotes the Hydrus transport model coupled with our kNN geochemical emulator (HPx$_{\\rm py}$-kNN). The first to third row present profiles for $C^{\\rm{solid}}$, $Al^{\\rm{solid}}$, and $S^{\\rm{solid}}$, respectively The considered grid size is 61 $\\times$ 61.}\n\t\\label{fig19}\n\\end{figure}\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure20.png}\n\t\\caption{2D solid amount profiles of the C, Al and S chemical components of cement system 2 after 2 and 6 (final time step) years. RTM means the original HPx$_{\\rm{4C}}$ model and TM+kNN denotes the Hydrus transport model coupled with our kNN geochemical emulator (HPx$_{\\rm py}$-kNN). The first to third row present profiles for $C^{\\rm{solid}}$, $Al^{\\rm{solid}}$, and $S^{\\rm{solid}}$, respectively The considered grid size is 121 $\\times$ 121.}\n\t\\label{fig20}\n\\end{figure}\n\n\\begin{table}[h!]\n\t\\caption{Speedups offered by the KNN and DNN emulators in HPx$_{\\rm py}$ for the reactive transport simulations considered for cement system 2. The HPx$_{\\rm{4C}}$ calculations involve the parallelization of PHREEQC over our 4 CPUs. The HPx$_{\\rm{1C}}$ calculations are performed on a single CPU. Both the kNN and DNN predictions make use of a GPU. ML signifies the used machine learning method for emulation, BC denotes the type of flow boundary conditions and GS is the grid size. The maximum possible speedups associated with HPx$_{\\rm{4C}}$ and HP$_{\\rm{1C}}$,  Max SP HPx$_{\\rm{4C}}$ and  Max SP HPx$_{\\rm{1C}}$, correspond to an hypothetical situation where the geochemical calculations incur zero computational cost.}\n\n\t\\begin{center}\n\t\t\\begin{tabular}{cccccccc}\n\t\t\t\\hline\n\t\t\t& & & & & & & \\\\\n\t\t\tML & BC & GS & HPx$_{\\rm{4C}}$ time (s) & SP HPx$_{\\rm{4C}}$ & Max SP HPx$_{\\rm{4C}}$ & SP HPx$_{\\rm{1C}}$ & Max SP HPx$_{\\rm{1C}}$ \\\\\n\t\t\tDNN & ADV & 61 $\\times$ 61 & 21,415 & 8.2 & 9.0 & 30.3 & 33.1\\\\\n\t\t\tkNN & ADV & 61 $\\times$ 61 & 21,415 & 7.9 & 9.0 & 28.9 & 33.1\\\\\n\t\t\tDNN & ADV & 121 $\\times$ 121 & 199,841 & 8.2 & 9.5 & 29.9 & 32.8\\\\\n\t\t\tkNN & ADV & 121 $\\times$ 121 & 199,841 & 8.5 & 9.5 & 31.3 & 32.8\\\\\n\t\t\t\\hline\n\t\t\\end{tabular}\n\t\\end{center}\n\n\t\\label{table4}\n\\end{table}\n\n\\FloatBarrier\n\n3.2.2 Reactive transport simulation\n\\subsubsection{Reactive transport simulation}\n\\label{prob2res}\nOur ``local\" DNN performs rather well when applied to the 61 $\\times$ 61 grid size and a time period of 6 years (Figures \\ref{fig11} - \\ref{fig12}). This for all components but C, for which some localized deviations appear between original and emulated 2D concentration profiles towards the end of the simulation period (Figure \\ref{fig12}). When applied to the 121 $\\times$ 121 grid, HPx$_{\\rm py}$-DNN produces additional discrepancies for O, Si and Al towards the end of the simulation period (Figures \\ref{fig13} - \\ref{fig14}). Yet most of the observed artifacts could probably be smoothed out by using post-filtering. The associated speedups are listed in Table \\ref{table4}. These speedups are larger to those obtained for cement system 1, with values between 8 and 9 when evaluated against HPx$_{\\rm{4C}}$. These speedups represent about 85 \\% to 90 \\% of the maximum possible speedup (Table \\ref{table3}). Overall, these findings indicate that for the considered problem, our RT-based training of a local DNN only works is the training set is sufficiently representative of the particular geochemical conditions encountered in the computationally demanding simulations, which is arguably not easy to achieve. This limitation is further discussed in section \\ref{discussion}.\n\nWe note a more uniform behavior for HPx$_{\\rm py}$-kNN across grid sizes than for HPx$_{\\rm py}$-DNN. Here the results for the 121  $\\times$ 121 grid are only slightly less accurate than those associated with the 61 $ \\times$ 61 grid (see Figures \\ref{fig15} - \\ref{fig16} where for brevity we only show concentration profiles for the C Al and S components). Furthermore, whenever observed the discrepancies between original and emulated profiles are more regularly scattered than for HPx$_{\\rm py}$-DNN. Note also that herein too, post-filtering could likely smooth out a large part of these deviations. In addition, owing to the use of a GPU to achieve the kNN calculations, the speedup provided by HPx$_{\\rm py}$-kNN are as large as those provided by HPx$_{\\rm py}$-DNN (Table \\ref{table4}).\n\nWith respect to the emulated solid amounts, the HPx$_{\\rm py}$-DNN results look visually good for the the 61 $\\times$ 61 grid. This is shown in Figure \\ref{fig17} for the C, Al and S chemical components, while emulation of the H, O Ca and Si chemical components is globally of similar quality (not shown). Nevertheless, significant deviations appear towards the end of the simulation period for every chemical component (see Figure \\ref{fig18} for the C, Al and S chemical components, emulation of the H, O Ca and Si chemical components shows the same level of mismatch). The HPx$_{\\rm py}$-kNN predictions are also fairly accurate for the 61 $\\times$ 61 (see Figure \\ref{fig19} for the C, Al and S chemical components, emulation of the H, O Ca and Si chemical components exhibits a globally similar quality) while some discrepancies show up at the end of the simulation (Figure \\ref{fig20}). However, the mismatch is less pronounced than for HPx$_{\\rm py}$-DNN. Overall, HPx$_{\\rm py}$-kNN appears to be somewhat more robust than HPx$_{\\rm py}$-DNN for this cement system, while providing the same (large) speedup.\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure11.png}\n\t\\caption{2D concentration profiles obtained for cement system 2 after 2 and 6 (final time step) years. RTM means the original HPx$_{\\rm{4C}}$ model and TM+DNN denotes the Hydrus transport model coupled with our DNN geochemical emulator (HPx$_{\\rm py}$-DNN). The first to fourth row present profiles for $Ca^{\\rm{conc}}$, $Si^{\\rm{conc}}$, $H^{\\rm{conc}}$, and $O^{\\rm{conc}}$, respectively. The considered grid size is 61 $\\times$ 61.}\n\t\\label{fig11}\n\\end{figure}\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure12.png}\n\t\\caption{2D concentration profiles obtained for cement system 2 after 2 and 6 (final time step) years. RTM means the original HPx$_{\\rm{4C}}$ model and TM+DNN denotes the Hydrus transport model coupled with our DNN geochemical emulator (HPx$_{\\rm py}$-DNN). The first to third row present profiles for $C^{\\rm{conc}}$, $Al^{\\rm{conc}}$, and $S^{\\rm{conc}}$, respectively. The considered grid size is 61 $\\times$ 61.}\n\t\\label{fig12}\n\\end{figure}\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure13.png}\n\t\\caption{2D concentration profiles obtained for cement system 2 after 2 and 6 (final time step) years. RTM means the original HPx$_{\\rm{4C}}$ model and TM+DNN denotes the Hydrus transport model coupled with our DNN geochemical emulator (HPx$_{\\rm py}$-DNN). The first to fourth row present profiles for $Ca^{\\rm{conc}}$, $Si^{\\rm{conc}}$, $H^{\\rm{conc}}$, and $O^{\\rm{conc}}$, respectively. The considered grid size is 121 $\\times$ 121.}\n\t\\label{fig13}\n\\end{figure}\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure14.png}\n\t\\caption{2D concentration profiles obtained for cement system 2 after 2 and 6 (final time step) years. RTM means the original HPx$_{\\rm{4C}}$ model and TM+DNN denotes the Hydrus transport model coupled with our DNN geochemical emulator (HPx$_{\\rm py}$-DNN). The first to third row present profiles for $C^{\\rm{conc}}$, $Al^{\\rm{conc}}$, and $S^{\\rm{conc}}$, respectively. The considered grid size is 121 $\\times$ 121.}\n\t\\label{fig14}\n\\end{figure}\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure15.png}\n\t\\caption{2D concentration profiles obtained for cement system 2 after 2 and 6 (final time step) years. RTM means the original HPx$_{\\rm{4C}}$ model and TM+kNN denotes the Hydrus transport model coupled with our kNN geochemical emulator (HPx$_{\\rm py}$-kNN).  The first to third row present profiles for $C^{\\rm{conc}}$, $Al^{\\rm{conc}}$, and $S^{\\rm{conc}}$, respectively. The considered grid size is 61 $\\times$ 61.}\n\t\\label{fig15}\n\\end{figure}\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure16.png}\n\t\\caption{2D concentration profiles obtained for cement system 2 after 2 and 6 (final time step) years. RTM means the original HPx$_{\\rm{4C}}$ model and TM+kNN denotes the Hydrus transport model coupled with our kNN geochemical emulator (HPx$_{\\rm py}$-kNN). The first to third row present profiles for $C^{\\rm{conc}}$, $Al^{\\rm{conc}}$, and $S^{\\rm{conc}}$, respectively. The considered grid size is 121 $\\times$ 121.}\n\t\\label{fig16}\n\\end{figure}\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure17.png}\n\t\\caption{2D solid amount profiles of the C, Al and S chemical components of cement system 2 after 2 and 6 (final time step) years. RTM means the original HPx$_{\\rm{4C}}$ model and TM+DNN denotes the Hydrus transport model coupled with our DNN geochemical emulator (HPx$_{\\rm py}$-DNN). The first to third row present profiles for $C^{\\rm{solid}}$, $Al^{\\rm{solid}}$, and $S^{\\rm{solid}}$, respectively The considered grid size is 61 $\\times$ 61.}\n\t\\label{fig17}\n\\end{figure}\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure18.png}\n\t\\caption{2D solid amount profiles of the C, Al and S chemical components of cement system 2 after 2 and 6 (final time step) years. RTM means the original HPx$_{\\rm{4C}}$ model and TM+DNN denotes the Hydrus transport model coupled with our DNN geochemical emulator (HPx$_{\\rm py}$-DNN). The first to third row present profiles for $C^{\\rm{solid}}$, $Al^{\\rm{solid}}$, and $S^{\\rm{solid}}$, respectively The considered grid size is 121 $\\times$ 121.}\n\t\\label{fig18}\n\\end{figure}\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure19.png}\n\t\\caption{2D solid amount profiles of the C, Al and S chemical components of cement system 2 after 2 and 6 (final time step) years. RTM means the original HPx$_{\\rm{4C}}$ model and TM+kNN denotes the Hydrus transport model coupled with our kNN geochemical emulator (HPx$_{\\rm py}$-kNN). The first to third row present profiles for $C^{\\rm{solid}}$, $Al^{\\rm{solid}}$, and $S^{\\rm{solid}}$, respectively The considered grid size is 61 $\\times$ 61.}\n\t\\label{fig19}\n\\end{figure}\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure20.png}\n\t\\caption{2D solid amount profiles of the C, Al and S chemical components of cement system 2 after 2 and 6 (final time step) years. RTM means the original HPx$_{\\rm{4C}}$ model and TM+kNN denotes the Hydrus transport model coupled with our kNN geochemical emulator (HPx$_{\\rm py}$-kNN). The first to third row present profiles for $C^{\\rm{solid}}$, $Al^{\\rm{solid}}$, and $S^{\\rm{solid}}$, respectively The considered grid size is 121 $\\times$ 121.}\n\t\\label{fig20}\n\\end{figure}\n\n\\begin{table}[h!]\n\t\\caption{Speedups offered by the KNN and DNN emulators in HPx$_{\\rm py}$ for the reactive transport simulations considered for cement system 2. The HPx$_{\\rm{4C}}$ calculations involve the parallelization of PHREEQC over our 4 CPUs. The HPx$_{\\rm{1C}}$ calculations are performed on a single CPU. Both the kNN and DNN predictions make use of a GPU. ML signifies the used machine learning method for emulation, BC denotes the type of flow boundary conditions and GS is the grid size. The maximum possible speedups associated with HPx$_{\\rm{4C}}$ and HP$_{\\rm{1C}}$,  Max SP HPx$_{\\rm{4C}}$ and  Max SP HPx$_{\\rm{1C}}$, correspond to an hypothetical situation where the geochemical calculations incur zero computational cost.}\n\n\t\\begin{center}\n\t\t\\begin{tabular}{cccccccc}\n\t\t\t\\hline\n\t\t\t& & & & & & & \\\\\n\t\t\tML & BC & GS & HPx$_{\\rm{4C}}$ time (s) & SP HPx$_{\\rm{4C}}$ & Max SP HPx$_{\\rm{4C}}$ & SP HPx$_{\\rm{1C}}$ & Max SP HPx$_{\\rm{1C}}$ \\\\\n\t\t\tDNN & ADV & 61 $\\times$ 61 & 21,415 & 8.2 & 9.0 & 30.3 & 33.1\\\\\n\t\t\tkNN & ADV & 61 $\\times$ 61 & 21,415 & 7.9 & 9.0 & 28.9 & 33.1\\\\\n\t\t\tDNN & ADV & 121 $\\times$ 121 & 199,841 & 8.2 & 9.5 & 29.9 & 32.8\\\\\n\t\t\tkNN & ADV & 121 $\\times$ 121 & 199,841 & 8.5 & 9.5 & 31.3 & 32.8\\\\\n\t\t\t\\hline\n\t\t\\end{tabular}\n\t\\end{center}\n\n\t\\label{table4}\n\\end{table}\n\n\\FloatBarrier\n\n",
        "label": "fig14",
        "Descriptive_question1": "Which chemical component is represented in the first row of figure_14?",
        "Descriptive_question2": "What is the grid size used in figure_14?",
        "Reasoning_question1": "How do the concentration profiles of C, Al, and S in figure_14 compare between the RTM and TM+DNN models after 6 years?",
        "Reasoning_question2": "What might be the possible reasons for any observed discrepancies in the concentration profiles between the RTM and TM+DNN models in figure_14 at the final time step?",
        "Descriptive_answer1": "C^{conc}",
        "Descriptive_answer2": "121 \\times 121",
        "Reasoning_answer1": "The concentration profiles of C, Al, and S in figure_14 at the final time step (6 years) show that the TM+DNN model approximates the RTM profiles reasonably well for most regions. However, there are some localized discrepancies especially towards the boundary or edges of the concentration fronts, indicating that while the general trend and spatial patterns are captured by TM+DNN, some fine-scale deviations exist in certain areas.",
        "Reasoning_answer2": "The observed discrepancies between RTM and TM+DNN concentration profiles in figure_14 at the final time step likely stem from the fact that the DNN emulator is 'local' and trained on a reactive transport (RT)-based dataset that may not fully capture all conditions present at larger grid sizes or longer simulation times. Additionally, the nonlinear and multi-dimensional nature of the chemical interactions, coupled with approximations in the KDE-based training data generation and potential numerical artifacts, could cause slight divergence from the fully resolved RTM results. Post-filtering might also mitigate some of these discrepancies."
    },
    {
        "paper_id": "2107.07598.json",
        "image_id": "figure_15",
        "image_path": [
            "/home/yz979/scratch/chengye/scimolmo/test-images-tables/dataset_imagestables/2107.07598/images/Figure15.png"
        ],
        "caption": "2D concentration profiles obtained for cement system 2 after 2 and 6 (final time step) years. RTM means the original HPx$_{\\rm{4C}}$ model and TM+kNN denotes the Hydrus transport model coupled with our kNN geochemical emulator (HPx$_{\\rm py}$-kNN).  The first to third row present profiles for $C^{\\rm{conc}}$, $Al^{\\rm{conc}}$, and $S^{\\rm{conc}}$, respectively. The considered grid size is 61 $\\times$ 61.",
        "classify": "Chart",
        "section_info": "3 Results\n\\section{Results}\n\\label{results}\n\n\\subsection{Ca-Si Problem}\nFor this first cement system, the emulation problem consists of predicting at each time step of the RT simulation the (output) Ca, Si, H and O aqueous concentrations (mol/kg of water or mol/kgw) from the (input) total amounts of Ca and Si (mol). \n\n\\subsubsection{Training the emulators}\n\\label{train_res1}\nHere the kNN and DNN emulators are firstly trained using a set of 400,000 test examples for both. This training set is obtained by randomly sampling the two-dimensional input space by latin hypercube sampling (LHS) between $\\left[0,0\\right]$ and $\\rm{\\left[Ca^{tot}_{max},Si^{tot}_{max}\\right]}$, and running PHREEQC for each input sample, $\\rm{\\textbf{x}_i = \\left[Ca^{tot}_i, Si^{tot}_i\\right]}$ to get the corresponding output vectors, $\\rm{\\textbf{y}_i = \\left[Ca^{conc}_i,Si^{conc}_i,H^{conc}_i,O^{conc}_i\\right]}$. The upper bounds, $\\rm{Ca^{tot}_{max}}$ and $\\rm{Si^{tot}_{max}}$ are defined based on a cheap full RT simulation with advective-dispersive transport using a small 1D domain of 51 nodes. It is worth noting that the total amounts of $\\textbf{x}_i$, corresponding to the PHREEQC-simulated concentrations, $\\textbf{y}_i$, have to be corrected for the different amount of water between the training set and the transport simulations. Doing so, it turns out that about 20 \\% of the post-corrected $x_i$ values exceed their pre-defined upper bounds and these excessively large values need to be filtered out. Creating the 400,000 training examples thus required about 500,000 PHREEQC runs. As stated earlier, for this problem single-threaded PHREEQC performs about 670 geochemical calculations per second on our used Intel\\textsuperscript{\\textregistered} i7 CPU.\n\nWith respect to kNN, the tuning parameters are the number of neighbors, $k$, the type of distance measure, and the interpolation technique. We simply used the default settings: $k = 5$, euclidean distance and inverse-distance interpolation. Regarding training of the DNN, the 400,000 sample were split between the training set itself (90 \\% of the data) and a validation set (10 \\% of the data). The latter serves to monitor the evolution of the selected mean squared error loss function on samples that are not used for training, thereby detecting potential overfitting. If the validation loss stops decreasing before the fixed number of epochs has been completed, then  training is stopped. Importantly, the emulation is achieved in log-space for both the input, $X$ and output $Y$, domains. This because total amounts and concentrations of the involved components typically cover many orders of magnitudes (up to 10 orders or more). Using a DNN also requires some form of data normalization or standardization. Here both the $\\rm{log\\left(\\textbf{x}_i\\right)}$ and $\\rm{log\\left(\\textbf{y}_i\\right)}$ vectors are standardized around 0 with standard deviation of 1.\n\nFigure \\ref{fig1} illustrates the trained emulators' performance for geochemical predictions using an independent test set that comprises 10,000 test examples. Both kNN (Figures \\ref{fig1}a - d) and DNN (Figures \\ref{fig1}e - h) appear to be rather accurate . DNN also shows a slight degradation for the larger concentration values  (Figures \\ref{fig1}e - h). The latter is likely due to the combination of a small proportion of large concentration values in the training set with the log-transformation that implicitly pushes the DNN to try harder to fit the smaller concentrations during training. Regarding speedup and as written earlier, for this setup the single-threaded kNN method is 300 times faster than single-threaded PHREEQC for predicting the 10,000 concentration vectors all at once. The computational savings allowed by the DNN emulator when ran on our GPU is higher with a speedup as large as 4000 for predicting the same 10,000 concentration vectors all at once. \n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{0cm}\\includegraphics[width=35pc]{Figure1.png}\n\t\\caption{1-1 plots of the kNN (subfigures (a) - (d)) and DNN (subfigures (e) - (h)) emulators' performance obtained for system 1 when the kNN training base contains 400,000 samples and the DNN is trained using the same 400,000 samples. Here ``true\" means the original PHREEQC-simulated data and ``predicted\" denotes the emulated (that is, kNN-simulated and DNN-simulated) data. Hence, the $x$-axis and $y$-axis present the original and emulated 10,000 independent test data points, respectively. The RMSE and $Q_2$ coefficient denote the root-mean-square-error and coefficient of determination in testing mode, respectively, between the original and emulated 10,000 test data points.}\n\t\\label{fig1}\n\\end{figure}\n\nTo test the sensitivity of the emulators' performance to the training set size, training was also performed using reduced training sets comprising 100,000, 10,000, 1000 and 100 samples, respectively. It seen that the DNN performance achieved  when using 10,000 training samples is virtually the same as that obtained when using 400,000 training samples (Figure \\ref{fig2} and Table \\ref{table2}). It is only for training sets smaller than 1000 samples that the DNN performance starts to degrade significantly (Table \\ref{table2}). In contrast, the behavior of kNN appears to markedly decrease as the training set gets smaller (Figure \\ref{fig2} and Table \\ref{table1}).\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{0cm}\\includegraphics[width=35pc]{Figure2.png}\n\t\\caption{1-1 plots of the kNN (subfigures (a) - (d)) and DNN (subfigures (e) - (h)) emulators' performance obtained for system 1 when the kNN training base contains 10,000 samples and the DNN is trained using the same 10,000 samples. Here ``true\" means the original PHREEQC-simulated data and ``predicted\" denotes the emulated (that is, kNN-simulated and DNN-simulated) data. Hence, the $x$-axis and $y$-axis present the original and emulated 10,000 independent test data points, respectively. The RMSE and $Q_2$ coefficient denote the root-mean-square-error and coefficient of determination in testing mode, respectively, between the original and emulated 10,000 test data points.}\n\t\\label{fig2}\n\\end{figure}\n\n\\begin{table}[h!]\n\t\\caption{Performance of the DNN and kNN emulators for cement system 1 and different training set sizes. For brevity, only the results for Ca$^{conc}$ and Si$^{conc}$ are shown. The units are mol per kg of water (mol/kgw). ML refers to the type of emulator, TR signifies the size of the training set, and the RMSE and $Q_2$ coefficient denote the root-mean-square-error and coefficient of determination in testing mode, respectively, between the original and emulated 10,000 test data points.}\n\t\\begin{center}\n\t\t\\begin{tabular}{cccccccc}\n\t\t\t\\hline\n\t\t\t& & & & & \\\\\n\t\t\tML & TR & RMSE - Ca$^{conc}$ & $Q_2$ - Ca$^{conc}$ & RMSE - Si$^{conc}$ & $Q_2$ - Si$^{conc}$\\\\\n\t\t\tDNN & 4 $\\times$ 10$^{5}$ & 4.50 $\\times$ 10$^{-5}$ & 0.9999 & 1.21 $\\times$ 10$^{-6}$ & 0.9999\\\\\n\t\t\tDNN & 1 $\\times$ 10$^{5}$ & 5.06 $\\times$ 10$^{-5}$ & 0.9999 & 1.30 $\\times$ 10$^{-6}$ & 0.9999\\\\\n\t\t\tDNN & 1 $\\times$ 10$^{4}$ & 4.57 $\\times$ 10$^{-5}$ & 0.9999 & 1.48 $\\times$ 10$^{-6}$ & 0.9999\\\\\n\t\t\tDNN & 1 $\\times$ 10$^{3}$ & 8.02 $\\times$ 10$^{-5}$ & 0.9998 & 3.31 $\\times$ 10$^{-6}$ & 0.9994\\\\\n\t\t\tDNN & 1 $\\times$ 10$^{2}$ & 47.1 $\\times$ 10$^{-5}$ & 0.9935 & 9.02 $\\times$ 10$^{-6}$ & 0.9959\\\\\n\t\t\tkNN & 4 $\\times$ 10$^{5}$ & 2.52 $\\times$ 10$^{-5}$ & 1.0000 & 1.59 $\\times$ 10$^{-6}$ & 0.9999\\\\\n\t\t\tkNN & 1 $\\times$ 10$^{5}$ & 6.27 $\\times$ 10$^{-5}$ & 0.9999 & 4.20 $\\times$ 10$^{-6}$ & 0.9991\\\\\n\t\t\tkNN & 1 $\\times$ 10$^{4}$ & 14.1 $\\times$ 10$^{-5}$ & 0.9994 & 13.5 $\\times$ 10$^{-6}$ & 0.9908\\\\\n\t\t\tkNN & 1 $\\times$ 10$^{3}$ & 43.9 $\\times$ 10$^{-5}$ & 0.9944 & 74.6 $\\times$ 10$^{-6}$ & 0.7180\\\\\n\t\t\tkNN & 1 $\\times$ 10$^{2}$  & 96.4 $\\times$ 10$^{-5}$ & 0.9729 & 55.0 $\\times$ 10$^{-6}$ & 0.8468\\\\\n\t\t\\end{tabular}\n\t\\end{center}\n\t\\label{table2}\n\\end{table}\n\n\\FloatBarrier\n\n\\subsubsection{Reactive transport simulation}\n\\label{prob12Dres}\n\nThis section focuses on reactive transport simulations with HPx$_{\\rm py}$ within cement system 1, under both advective-dispersive and diffusive transport conditions. As written above, the domain sizes are both 61 $\\times$ 61 and 121 $\\times$ 121 for the advection-dispersion case and, because of computational constraints, solely 61 $\\times$ 61  for the diffusion case. In addition, the simulation time period is 2 years for the advection-dispersion case and 1 year for the diffusion case. Figures \\ref{fig3} and \\ref{fig4} present times series of original and emulated Ca, Si, H and O concentrations at 5 locations within the 2D domain for advective-dispersive transport conditions, for both our kNN-based (HPx$_{\\rm py}$-kNN) and DNN-based (HPx$_{\\rm py}$-DNN) reactive transport codes. It is seen that HPx$_{\\rm py}$-kNN and HPx$_{\\rm py}$-DNN both induce a quite good simulation accuracy. Also, the results for the diffusive transport case are of similarly good quality (not shown). Figures \\ref{fig5} - \\ref{fig6} provide more insights into the HPx$_{\\rm py}$-kNN and HPx$_{\\rm py}$-DNN performances by displaying 2D Ca, Si, H and O concentration profiles at a given time. For each experiment and chemical component, this time is selected as to be well representative of the simulated dynamics. It is observed that the original and emulated images are visually almost indistinguishable for the advection-dispersion case (Figure \\ref{fig5}). For the diffusion case, the emulators also perform quite well for Ca, H and O (Figures\\ref{fig6}a - c, g - i and j - l), while some slight to moderate discrepancies appear at the concentration front for Si (Figures \\ref{fig6}d - f). Notwithstanding, the Si concentration remains globally well predicted. Furthermore, Figures \\ref{fig7} - \\ref{fig8} present the original and emulated 2D solid amount profiles corresponding to Figures \\ref{fig5} - \\ref{fig6}. The original solid amount profiles are overall well approximated by HPx$_{\\rm py}$-kNN and HPx$_{\\rm py}$-DNN for the advection-dispersion case (Figure \\ref{fig7}), even though some mismatch appears at the border of the fully depleted zone for the H component. As of the diffusion case (Figure \\ref{fig8}), the same kind of mismatch is observed for the emulated solid amounts of H by HPx$_{\\rm py}$-kNN while the emulated profiles by HPx$_{\\rm py}$-DNN show some more discrepancies. Though we decided to present raw emulation results, we would like to stress that some if not all of the observed artifacts could likely be smoothed out by using some post-filtering such as median filtering.\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure3.png}\n\t\\caption{Time series of original (RTM, solid green lines) and HPx$_{\\rm py}$-kNN emulated (TM+kNN, dashed orange lines) concentrations (mol/kg) of Ca, H, O and Si at selected observations points for cement system 1 and advective-dispersive transport. Obs. 1 - 5 denote the selected observation points, with the following $\\left[x,y\\right]$ locations (in cm). Obs. 1:  $\\left[0.5, 2.5\\right]$, Obs. 2: $\\left[1, 2\\right]$, Obs. 3: $\\left[2, 2\\right]$, Obs. 4: $\\left[1, 1\\right]$, Obs. 5: $\\left[2, 1\\right]$. The results for the 121 $\\times$ 121 grid size are rather similar.}\n\t\\label{fig3}\n\\end{figure}\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure4.png}\n\t\\caption{Time series of original (RTM, solid green lines) and HPx$_{\\rm py}$-DNN emulated (TM+DNN, dashed orange lines) concentrations (mol/kg) of Ca, H, O and Si at selected observations points for cement system 1 and advective-dispersive transport. Obs. 1 - 5 denote the selected observation points, with the following $\\left[x,y\\right]$ locations (in cm). Obs. 1:  $\\left[0.5, 2.5\\right]$, Obs. 2: $\\left[1, 2\\right]$, Obs. 3: $\\left[2, 2\\right]$, Obs. 4: $\\left[1, 1\\right]$, Obs. 5: $\\left[2, 1\\right]$. The results for the 121 $\\times$ 121 grid size are rather similar.}\n\t\\label{fig4}\n\\end{figure}\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure5.png}\n\t\\caption{2D concentration profiles obtained for cement system 1 at the end of the 2-year simulation performed for the advection-dispersion case. RTM means the original HP$_{\\rm{4C}}$ model, TM+kNN denotes the Hydrus transport model coupled with our kNN geochemical emulator (HPx$_{\\rm py}$-kNN) and TM+DNN signifies the Hydrus transport model coupled with our DNN geochemical emulator (HPx$_{\\rm py}$-DNN). The first to fourth row present profiles for $Ca^{\\rm{conc}}$, $Si^{\\rm{conc}}$, $H^{\\rm{conc}}$, and $O^{\\rm{conc}}$, respectively. The considered grid size is 61 $\\times$ 61. The results for the 121 $\\times$ 121 grid are rather similar.}\n\t\\label{fig5}\n\\end{figure}\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure6.png}\n\t\\caption{2D concentration profiles obtained for cement system 1 at the final time step of the 1-year simulation performed for the diffusion case. RTM means the original HP$_{\\rm{4C}}$ model, TM+kNN denotes the Hydrus transport model coupled with our kNN geochemical emulator (HPx$_{\\rm py}$-kNN) and TM+DNN signifies the Hydrus transport model coupled with our DNN geochemical emulator (HPx$_{\\rm py}$-DNN). The first to fourth row present profiles for $Ca^{\\rm{conc}}$, $Si^{\\rm{conc}}$, $H^{\\rm{conc}}$, and $O^{\\rm{conc}}$, respectively. The considered grid size is 61 $\\times$ 61.}\n\t\\label{fig6}\n\\end{figure}\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure7.png}\n\t\\caption{2D solid amount profiles obtained for cement system 1 at the end of the 2-year simulation performed for the advection-dispersion case. RTM means the original HP$_{\\rm{4C}}$ model, TM+kNN denotes the Hydrus transport model coupled with our kNN geochemical emulator (HPx$_{\\rm py}$-kNN) and TM+DNN signifies the Hydrus transport model coupled with our DNN geochemical emulator (HPx$_{\\rm py}$-DNN). The first to fourth row present profiles for $Ca^{\\rm{solid}}$, $Si^{\\rm{solid}}$, $H^{\\rm{solid}}$, and $O^{\\rm{solid}}$, respectively. The considered grid size is 61 $\\times$ 61. The results for the 121 $\\times$ 121 grid are rather similar.}\n\t\\label{fig7}\n\\end{figure}\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure8.png}\n\t\\caption{2D solid amount profiles obtained for cement system 1 at the final time step of the 1-year simulation performed for the diffusion case. RTM means the original HP$_{\\rm{4C}}$ model, TM+kNN denotes the Hydrus transport model coupled with our kNN geochemical emulator (HPx$_{\\rm py}$-kNN) and TM+DNN signifies the Hydrus transport model coupled with our DNN geochemical emulator (HPx$_{\\rm py}$-DNN). The first to fourth row present profiles for $Ca^{\\rm{solid}}$, $Si^{\\rm{solid}}$, $H^{\\rm{solid}}$, and $O^{\\rm{solid}}$, respectively. The considered grid size is 61 $\\times$ 61.}\n\t\\label{fig8}\n\\end{figure}\n\nThe speedups associated with the considered problem are detailed in Table \\ref{table3}. It is noted that the GPU-based DNN emulator allows for a speedup that is close to optimal. Indeed, The DNN speedups overall represent 85 \\% to 95 \\% of the maximum possible speedups (that is, speedups that would be obtained if the geochemcical calculations would come at no cost at all). The speedups associated with single-threaded kNN remain substantial but only amount to 57 \\% - 65 \\% of the corresponding maximum speedups. As detailed in section \\ref{train_res1}, the used kNN and DNN implementations are found to be respectively 300 and 4000 faster than single-threaded PHREEQC when predicting 10,000 points all one once for this geochemical system. Based on these numbers one could have expected the achieved speedups to represent say 90 \\% (kNN) or 99 \\% (DNN) of the maximum possible ones. A large part of the gaps between achieved and maximum possible speedups is thus likely caused by the time required for communicating and exchanging data between the main C/C++ code and the Python-based emulators.\n\n\\begin{table}[h!]\n\t\\caption{Speedups offered by the KNN and DNN emulators in HPx$_{\\rm py}$ for the reactive transport simulations considered for cement system 1. The HPx$_{\\rm{4C}}$ calculations involve the parallelization of PHREEQC over our 4 CPUs. The HPx$_{\\rm{1C}}$ calculations are performed on a single CPU. The kNN predictions are performed on a single CPU using the scikit-learn implementation while the DNN predictions make use of our GPU. ML signifies the used machine learning method for emulation, TC denotes transport conditions (ADV: advection-dispersion, DIF: diffusion) and GS is the grid size. The maximum possible speedups associated with HPx$_{\\rm{4C}}$ and HPx$_{\\rm{1C}}$,  Max SP HPx$_{\\rm{4C}}$ and  Max SP HPx$_{\\rm{1C}}$, correspond to an hypothetical situation where the geochemical calculations incur zero computational cost.}\n\n\t\\begin{center}\n\t\t\\begin{tabular}{cccccccc}\n\t\t\t\\hline\n\t\t\t& & & & & & & \\\\\n\t\t\tML & TC & GS & HPx$_{\\rm{4C}}$ time (s) & SP HPx$_{\\rm{4C}}$ & Max SP HPx$_{\\rm{4C}}$ & SP HPx$_{\\rm{1C}}$ & Max SP HPx$_{\\rm{1C}}$ \\\\\n\t\t\tDNN & ADV & 61 $\\times$ 61 & 3189 & 6.8 & 7.7 & 24.5 & 28.5 \\\\\n\t\t\tkNN & ADV & 61 $\\times$ 61 & 3189 & 5.0 & 7.7 & 18.0 & 28.5 \\\\\n\t\t\tDNN & ADV & 121 $\\times$ 121 & 23,337 & 5.0 & 5.2 & 17.0 & 18.0 \\\\\n\t\t\tkNN & ADV & 121 $\\times$ 121 & 23,337 & 3.4 & 5.2 & 11.6 & 18.0 \\\\\n\t\t\tDNN & DIF & 61 $\\times$ 61 & 25,448 & 4.2 & 4.9 & 16.2 & 19.1 \\\\\n\t\t\tkNN & DIF & 61 $\\times$ 61 & 25,448 & 2.8 & 4.9 & 10.8 & 19.1 \\\\\n\t\t\t\\hline\n\t\t\\end{tabular}\n\t\\end{center}\n\n\t\\label{table3}\n\\end{table}\n\n\\FloatBarrier\n\n\\subsection{Al-C-Ca-S-Si Problem}\n\nFor this second cement system, the emulation problem consists of predicting at each time step of the RT simulation the (output) Al, C, Ca, S, Si, H and O aqueous concentrations (mol/kgw) from the (input) total amounts of Al, C, Ca, S and Si (mol). Here we focus on advective-dispersive transport only while similarly as for cement system 1, the considered domain sizes are 61 $\\times$ 61 and 121 $\\times$ 121. Moreover, the simulation time period is set to 6 years. \n\nAs mentioned earlier, for this higher-dimensional problem it is observed that the used scikit-learn kNN implementation becomes prohibitively slow compared to HPx$_{\\rm{4C}}$. We found that to get a good emulation accuracy the kNN training base needs to contain 1,000,000 samples (or more). This training base's size together with a 5-dimensional search space leads to an HPx$_{\\rm py}$-kNN reactive transport simulation time that is comparable to that of HPx$_{\\rm{4C}}$. Therefore, for this second cement system we built a custom kNN regressor around another kNN implementation contained in the FAISS package \\citep{faiss2017}. Our used FAISS variant allows for GPU computing and is much faster than scikit-learn for this cement system, but is slightly less accurate due to the use of an approximate rather than exact nearest neighbor search \\citep[see][for details]{faiss2017}.\n\n\\subsubsection{Training the DNN}\n\\label{train_res2}\n\nBuilding a good training set to perform a kNN search and learn the weights and biases of our DNN turned out to be a complicated task in this case. This because to make useful kNN predictions and/or learn an useful DNN, the training set must be sufficiently representative of the geochemical conditions encountered during the reactive transport simulation one wish to perform with HPx$_{\\rm py}$-kNN and the trained HPx$_{\\rm py}$-DNN. In contrast to cement system 1, creating the training set by sampling the $X$-space with a controlled randomness between predefined lower and upper bounds did not prove successful. We tried that strategy by drawing as much as 4,000,000 5-dimensional $\\textbf{x}$ vectors from the $X$-space using a Sobol low-discrepancy sequence \\citep[][]{Sobol1967, Joe-Kuo2003}. Such low-discrepancy sampling scheme covers the 5-dimensional hypercube more uniformly than LHS. Despite a good performance on the test set (not shown), the resulting DNN accuracy in reactive transport mode was never deemed satisfying. In other words, no satisfying ``global\" or ``universal\" DNN emulator could be devised for this cement system. This is probably caused by the fact for this problem, the input (5 total amounts) and output (7 aqueous concentrations) spaces are quite nonlinearly related and both cover 6 to 10 orders of magnitudes depending on the considered element. Therefore, we resorted to the alternative training strategy detailed below. The latter basically tries to grasp the complex correlations and higher-order dependencies that exist between the elements of $\\textbf{x}$ (total amounts, input space) for a given reactive transport simulation setup, in order to produce a training set that honors these between-input relationships.\n\n\\begin{itemize}\n\t\\item Perform a ``cheap\" full reactive transport simulation under the transport conditions and geochemistry of interest and collect the resulting $\\textbf{x}$ -$\\textbf{y}$  pairs of examples (for the considered grid nodes and time steps). Computational demand controls what domain size and simulation time period can be used for this cheap calculation. We used a modest 16 $\\times$ 16 domain and a simulation time period of 10 years. The associated HPx$_{\\rm{4C}}$ runtimes is 180 s.\n\t\n\t\\item Fit a kernel density estimator (KDE) with a Gaussian kernel to the collected $\\textbf{x}$ vectors (encapsulated in the $\\textbf{X}$ array) and generate a fixed number of new input vectors, $\\textbf{x}_{KDE}$. Then run PHREEQC for the $\\textbf{X}_{KDE}$ set to get the corresponding output set, $\\textbf{Y}_{KDE}$. Now apply the correction for porosity described in section \\ref{train_res1} to the $\\textbf{X}_{KDE}$ set and form the training set by merging the ensemble of $\\textbf{x}$-$\\textbf{y}$ pairs with that of the $\\textbf{x}_{KDE}$-$\\textbf{y}_{KDE}$ pairs. The number of produced unique examples by the considered cheap HPx$_{\\rm{4C}}$ simulations varied between 10,000 and 50,000. The KDE-based enrichment of this dataset was deemed necessary to provide more input variability, thereby avoiding overfitting of the trained DNN and improving the kNN accuracy, while still honoring the complex between-input relationships. The number of KDE-generated samples was set as to obtain a total training set size of 1,000,000 examples. A key component of the approach is the bandwith parameter of the Gaussian KDE kernel which controls how much the KDE-generated samples depart from the original ensemble. After limited trial and error, we fixed the kernel bandwith to 0.0025 for the considered case studies.\n\t\n\\end{itemize}\n\nThe scatter plots in Figure \\ref{fig9} illustrate our training set creation procedure. The orange dots depict the pairwise relationships observed between the 5 elements of $\\textbf{x}$ in the cheap simulation. The red and cyan dots in Figure \\ref{fig9} represent the KDE-generated samples with the selected bandwidth, before and after applying the correction for porosity, respectively. Training of the DNN is achieved using the ensemble of original and KDE-corrected input points. We refer to this kind of dataset as RT-based, since it is based on an albeit cheap, full RT simulation. Furthermore, we refer to the obtained DNN and kNN emulators as ``local\" emulators, since as opposed to the emulators constructed for cement system 1, the current emulators are only valid for the input conditions encapsulated in the RT-based training set.\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1.5cm}\\includegraphics[width=50pc]{Figure9.png}\n\t\\caption{Scatter plots of the complex relationships between the five considered inputs for (1) the computationally cheap RT simulation of cement system 2 (orange dots) performed with the original HPx code, (2) the corresponding sampled points by kernel density estimation (KDE) using the selected bandwith (turquoise dots) and, (3) the same sampled KDE points after porosity correction (red dots).}\n\t\\label{fig9}\n\\end{figure}\n\nTraining performance of the DNN emulator is presented in Figure \\ref{fig10}. Similarly as for cement system 1, about 90\\% of the available data was used for the actual training of the DNN while the remaining 10\\% were used a validation set to contol overfitting. Lastly, performance is evaluated for both the trained DNN and kNN emulators using an independent test set of 10,000 examples. Overall, the accuracy of our ``local \" DNN emulator for this RT-inspired dataset is rather large with $Q_2$ values always greater or equal than 0.998. Training performance of the corresponding local kNN emulator is equally good (not shown). Regarding speedup, for this problem single-threaded PHREEQC achieves about 210 geochemical calculations per second on our used Intel\\textsuperscript{\\textregistered} i7 CPU while the GPU-based DNN and kNN emulators are both about 3000 times faster when predicting the 10,000 test points at once .\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure10.png}\n\t\\caption{1-1 plots of local DNN emulation performance obtained for system 2 when the local DNN is trained using 1,000,000 samples. The $x$-axis and $y$-axis present the original and emulated 10,000 independent test data points, respectively. The RMSE and $Q_2$ coefficient denote the root-mean-square-error and coefficient of determination in testing mode, respectively, between the original and emulated 10,000 test data points.}\n\t\\label{fig10}\n\\end{figure}\n\n\\subsubsection{Reactive transport simulation}\n\\label{prob2res}\nOur ``local\" DNN performs rather well when applied to the 61 $\\times$ 61 grid size and a time period of 6 years (Figures \\ref{fig11} - \\ref{fig12}). This for all components but C, for which some localized deviations appear between original and emulated 2D concentration profiles towards the end of the simulation period (Figure \\ref{fig12}). When applied to the 121 $\\times$ 121 grid, HPx$_{\\rm py}$-DNN produces additional discrepancies for O, Si and Al towards the end of the simulation period (Figures \\ref{fig13} - \\ref{fig14}). Yet most of the observed artifacts could probably be smoothed out by using post-filtering. The associated speedups are listed in Table \\ref{table4}. These speedups are larger to those obtained for cement system 1, with values between 8 and 9 when evaluated against HPx$_{\\rm{4C}}$. These speedups represent about 85 \\% to 90 \\% of the maximum possible speedup (Table \\ref{table3}). Overall, these findings indicate that for the considered problem, our RT-based training of a local DNN only works is the training set is sufficiently representative of the particular geochemical conditions encountered in the computationally demanding simulations, which is arguably not easy to achieve. This limitation is further discussed in section \\ref{discussion}.\n\nWe note a more uniform behavior for HPx$_{\\rm py}$-kNN across grid sizes than for HPx$_{\\rm py}$-DNN. Here the results for the 121  $\\times$ 121 grid are only slightly less accurate than those associated with the 61 $ \\times$ 61 grid (see Figures \\ref{fig15} - \\ref{fig16} where for brevity we only show concentration profiles for the C Al and S components). Furthermore, whenever observed the discrepancies between original and emulated profiles are more regularly scattered than for HPx$_{\\rm py}$-DNN. Note also that herein too, post-filtering could likely smooth out a large part of these deviations. In addition, owing to the use of a GPU to achieve the kNN calculations, the speedup provided by HPx$_{\\rm py}$-kNN are as large as those provided by HPx$_{\\rm py}$-DNN (Table \\ref{table4}).\n\nWith respect to the emulated solid amounts, the HPx$_{\\rm py}$-DNN results look visually good for the the 61 $\\times$ 61 grid. This is shown in Figure \\ref{fig17} for the C, Al and S chemical components, while emulation of the H, O Ca and Si chemical components is globally of similar quality (not shown). Nevertheless, significant deviations appear towards the end of the simulation period for every chemical component (see Figure \\ref{fig18} for the C, Al and S chemical components, emulation of the H, O Ca and Si chemical components shows the same level of mismatch). The HPx$_{\\rm py}$-kNN predictions are also fairly accurate for the 61 $\\times$ 61 (see Figure \\ref{fig19} for the C, Al and S chemical components, emulation of the H, O Ca and Si chemical components exhibits a globally similar quality) while some discrepancies show up at the end of the simulation (Figure \\ref{fig20}). However, the mismatch is less pronounced than for HPx$_{\\rm py}$-DNN. Overall, HPx$_{\\rm py}$-kNN appears to be somewhat more robust than HPx$_{\\rm py}$-DNN for this cement system, while providing the same (large) speedup.\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure11.png}\n\t\\caption{2D concentration profiles obtained for cement system 2 after 2 and 6 (final time step) years. RTM means the original HPx$_{\\rm{4C}}$ model and TM+DNN denotes the Hydrus transport model coupled with our DNN geochemical emulator (HPx$_{\\rm py}$-DNN). The first to fourth row present profiles for $Ca^{\\rm{conc}}$, $Si^{\\rm{conc}}$, $H^{\\rm{conc}}$, and $O^{\\rm{conc}}$, respectively. The considered grid size is 61 $\\times$ 61.}\n\t\\label{fig11}\n\\end{figure}\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure12.png}\n\t\\caption{2D concentration profiles obtained for cement system 2 after 2 and 6 (final time step) years. RTM means the original HPx$_{\\rm{4C}}$ model and TM+DNN denotes the Hydrus transport model coupled with our DNN geochemical emulator (HPx$_{\\rm py}$-DNN). The first to third row present profiles for $C^{\\rm{conc}}$, $Al^{\\rm{conc}}$, and $S^{\\rm{conc}}$, respectively. The considered grid size is 61 $\\times$ 61.}\n\t\\label{fig12}\n\\end{figure}\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure13.png}\n\t\\caption{2D concentration profiles obtained for cement system 2 after 2 and 6 (final time step) years. RTM means the original HPx$_{\\rm{4C}}$ model and TM+DNN denotes the Hydrus transport model coupled with our DNN geochemical emulator (HPx$_{\\rm py}$-DNN). The first to fourth row present profiles for $Ca^{\\rm{conc}}$, $Si^{\\rm{conc}}$, $H^{\\rm{conc}}$, and $O^{\\rm{conc}}$, respectively. The considered grid size is 121 $\\times$ 121.}\n\t\\label{fig13}\n\\end{figure}\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure14.png}\n\t\\caption{2D concentration profiles obtained for cement system 2 after 2 and 6 (final time step) years. RTM means the original HPx$_{\\rm{4C}}$ model and TM+DNN denotes the Hydrus transport model coupled with our DNN geochemical emulator (HPx$_{\\rm py}$-DNN). The first to third row present profiles for $C^{\\rm{conc}}$, $Al^{\\rm{conc}}$, and $S^{\\rm{conc}}$, respectively. The considered grid size is 121 $\\times$ 121.}\n\t\\label{fig14}\n\\end{figure}\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure15.png}\n\t\\caption{2D concentration profiles obtained for cement system 2 after 2 and 6 (final time step) years. RTM means the original HPx$_{\\rm{4C}}$ model and TM+kNN denotes the Hydrus transport model coupled with our kNN geochemical emulator (HPx$_{\\rm py}$-kNN).  The first to third row present profiles for $C^{\\rm{conc}}$, $Al^{\\rm{conc}}$, and $S^{\\rm{conc}}$, respectively. The considered grid size is 61 $\\times$ 61.}\n\t\\label{fig15}\n\\end{figure}\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure16.png}\n\t\\caption{2D concentration profiles obtained for cement system 2 after 2 and 6 (final time step) years. RTM means the original HPx$_{\\rm{4C}}$ model and TM+kNN denotes the Hydrus transport model coupled with our kNN geochemical emulator (HPx$_{\\rm py}$-kNN). The first to third row present profiles for $C^{\\rm{conc}}$, $Al^{\\rm{conc}}$, and $S^{\\rm{conc}}$, respectively. The considered grid size is 121 $\\times$ 121.}\n\t\\label{fig16}\n\\end{figure}\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure17.png}\n\t\\caption{2D solid amount profiles of the C, Al and S chemical components of cement system 2 after 2 and 6 (final time step) years. RTM means the original HPx$_{\\rm{4C}}$ model and TM+DNN denotes the Hydrus transport model coupled with our DNN geochemical emulator (HPx$_{\\rm py}$-DNN). The first to third row present profiles for $C^{\\rm{solid}}$, $Al^{\\rm{solid}}$, and $S^{\\rm{solid}}$, respectively The considered grid size is 61 $\\times$ 61.}\n\t\\label{fig17}\n\\end{figure}\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure18.png}\n\t\\caption{2D solid amount profiles of the C, Al and S chemical components of cement system 2 after 2 and 6 (final time step) years. RTM means the original HPx$_{\\rm{4C}}$ model and TM+DNN denotes the Hydrus transport model coupled with our DNN geochemical emulator (HPx$_{\\rm py}$-DNN). The first to third row present profiles for $C^{\\rm{solid}}$, $Al^{\\rm{solid}}$, and $S^{\\rm{solid}}$, respectively The considered grid size is 121 $\\times$ 121.}\n\t\\label{fig18}\n\\end{figure}\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure19.png}\n\t\\caption{2D solid amount profiles of the C, Al and S chemical components of cement system 2 after 2 and 6 (final time step) years. RTM means the original HPx$_{\\rm{4C}}$ model and TM+kNN denotes the Hydrus transport model coupled with our kNN geochemical emulator (HPx$_{\\rm py}$-kNN). The first to third row present profiles for $C^{\\rm{solid}}$, $Al^{\\rm{solid}}$, and $S^{\\rm{solid}}$, respectively The considered grid size is 61 $\\times$ 61.}\n\t\\label{fig19}\n\\end{figure}\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure20.png}\n\t\\caption{2D solid amount profiles of the C, Al and S chemical components of cement system 2 after 2 and 6 (final time step) years. RTM means the original HPx$_{\\rm{4C}}$ model and TM+kNN denotes the Hydrus transport model coupled with our kNN geochemical emulator (HPx$_{\\rm py}$-kNN). The first to third row present profiles for $C^{\\rm{solid}}$, $Al^{\\rm{solid}}$, and $S^{\\rm{solid}}$, respectively The considered grid size is 121 $\\times$ 121.}\n\t\\label{fig20}\n\\end{figure}\n\n\\begin{table}[h!]\n\t\\caption{Speedups offered by the KNN and DNN emulators in HPx$_{\\rm py}$ for the reactive transport simulations considered for cement system 2. The HPx$_{\\rm{4C}}$ calculations involve the parallelization of PHREEQC over our 4 CPUs. The HPx$_{\\rm{1C}}$ calculations are performed on a single CPU. Both the kNN and DNN predictions make use of a GPU. ML signifies the used machine learning method for emulation, BC denotes the type of flow boundary conditions and GS is the grid size. The maximum possible speedups associated with HPx$_{\\rm{4C}}$ and HP$_{\\rm{1C}}$,  Max SP HPx$_{\\rm{4C}}$ and  Max SP HPx$_{\\rm{1C}}$, correspond to an hypothetical situation where the geochemical calculations incur zero computational cost.}\n\n\t\\begin{center}\n\t\t\\begin{tabular}{cccccccc}\n\t\t\t\\hline\n\t\t\t& & & & & & & \\\\\n\t\t\tML & BC & GS & HPx$_{\\rm{4C}}$ time (s) & SP HPx$_{\\rm{4C}}$ & Max SP HPx$_{\\rm{4C}}$ & SP HPx$_{\\rm{1C}}$ & Max SP HPx$_{\\rm{1C}}$ \\\\\n\t\t\tDNN & ADV & 61 $\\times$ 61 & 21,415 & 8.2 & 9.0 & 30.3 & 33.1\\\\\n\t\t\tkNN & ADV & 61 $\\times$ 61 & 21,415 & 7.9 & 9.0 & 28.9 & 33.1\\\\\n\t\t\tDNN & ADV & 121 $\\times$ 121 & 199,841 & 8.2 & 9.5 & 29.9 & 32.8\\\\\n\t\t\tkNN & ADV & 121 $\\times$ 121 & 199,841 & 8.5 & 9.5 & 31.3 & 32.8\\\\\n\t\t\t\\hline\n\t\t\\end{tabular}\n\t\\end{center}\n\n\t\\label{table4}\n\\end{table}\n\n\\FloatBarrier\n\n3.2 Al-C-Ca-S-Si Problem\n\\subsection{Al-C-Ca-S-Si Problem}\n\nFor this second cement system, the emulation problem consists of predicting at each time step of the RT simulation the (output) Al, C, Ca, S, Si, H and O aqueous concentrations (mol/kgw) from the (input) total amounts of Al, C, Ca, S and Si (mol). Here we focus on advective-dispersive transport only while similarly as for cement system 1, the considered domain sizes are 61 $\\times$ 61 and 121 $\\times$ 121. Moreover, the simulation time period is set to 6 years. \n\nAs mentioned earlier, for this higher-dimensional problem it is observed that the used scikit-learn kNN implementation becomes prohibitively slow compared to HPx$_{\\rm{4C}}$. We found that to get a good emulation accuracy the kNN training base needs to contain 1,000,000 samples (or more). This training base's size together with a 5-dimensional search space leads to an HPx$_{\\rm py}$-kNN reactive transport simulation time that is comparable to that of HPx$_{\\rm{4C}}$. Therefore, for this second cement system we built a custom kNN regressor around another kNN implementation contained in the FAISS package \\citep{faiss2017}. Our used FAISS variant allows for GPU computing and is much faster than scikit-learn for this cement system, but is slightly less accurate due to the use of an approximate rather than exact nearest neighbor search \\citep[see][for details]{faiss2017}.\n\n\\subsubsection{Training the DNN}\n\\label{train_res2}\n\nBuilding a good training set to perform a kNN search and learn the weights and biases of our DNN turned out to be a complicated task in this case. This because to make useful kNN predictions and/or learn an useful DNN, the training set must be sufficiently representative of the geochemical conditions encountered during the reactive transport simulation one wish to perform with HPx$_{\\rm py}$-kNN and the trained HPx$_{\\rm py}$-DNN. In contrast to cement system 1, creating the training set by sampling the $X$-space with a controlled randomness between predefined lower and upper bounds did not prove successful. We tried that strategy by drawing as much as 4,000,000 5-dimensional $\\textbf{x}$ vectors from the $X$-space using a Sobol low-discrepancy sequence \\citep[][]{Sobol1967, Joe-Kuo2003}. Such low-discrepancy sampling scheme covers the 5-dimensional hypercube more uniformly than LHS. Despite a good performance on the test set (not shown), the resulting DNN accuracy in reactive transport mode was never deemed satisfying. In other words, no satisfying ``global\" or ``universal\" DNN emulator could be devised for this cement system. This is probably caused by the fact for this problem, the input (5 total amounts) and output (7 aqueous concentrations) spaces are quite nonlinearly related and both cover 6 to 10 orders of magnitudes depending on the considered element. Therefore, we resorted to the alternative training strategy detailed below. The latter basically tries to grasp the complex correlations and higher-order dependencies that exist between the elements of $\\textbf{x}$ (total amounts, input space) for a given reactive transport simulation setup, in order to produce a training set that honors these between-input relationships.\n\n\\begin{itemize}\n\t\\item Perform a ``cheap\" full reactive transport simulation under the transport conditions and geochemistry of interest and collect the resulting $\\textbf{x}$ -$\\textbf{y}$  pairs of examples (for the considered grid nodes and time steps). Computational demand controls what domain size and simulation time period can be used for this cheap calculation. We used a modest 16 $\\times$ 16 domain and a simulation time period of 10 years. The associated HPx$_{\\rm{4C}}$ runtimes is 180 s.\n\t\n\t\\item Fit a kernel density estimator (KDE) with a Gaussian kernel to the collected $\\textbf{x}$ vectors (encapsulated in the $\\textbf{X}$ array) and generate a fixed number of new input vectors, $\\textbf{x}_{KDE}$. Then run PHREEQC for the $\\textbf{X}_{KDE}$ set to get the corresponding output set, $\\textbf{Y}_{KDE}$. Now apply the correction for porosity described in section \\ref{train_res1} to the $\\textbf{X}_{KDE}$ set and form the training set by merging the ensemble of $\\textbf{x}$-$\\textbf{y}$ pairs with that of the $\\textbf{x}_{KDE}$-$\\textbf{y}_{KDE}$ pairs. The number of produced unique examples by the considered cheap HPx$_{\\rm{4C}}$ simulations varied between 10,000 and 50,000. The KDE-based enrichment of this dataset was deemed necessary to provide more input variability, thereby avoiding overfitting of the trained DNN and improving the kNN accuracy, while still honoring the complex between-input relationships. The number of KDE-generated samples was set as to obtain a total training set size of 1,000,000 examples. A key component of the approach is the bandwith parameter of the Gaussian KDE kernel which controls how much the KDE-generated samples depart from the original ensemble. After limited trial and error, we fixed the kernel bandwith to 0.0025 for the considered case studies.\n\t\n\\end{itemize}\n\nThe scatter plots in Figure \\ref{fig9} illustrate our training set creation procedure. The orange dots depict the pairwise relationships observed between the 5 elements of $\\textbf{x}$ in the cheap simulation. The red and cyan dots in Figure \\ref{fig9} represent the KDE-generated samples with the selected bandwidth, before and after applying the correction for porosity, respectively. Training of the DNN is achieved using the ensemble of original and KDE-corrected input points. We refer to this kind of dataset as RT-based, since it is based on an albeit cheap, full RT simulation. Furthermore, we refer to the obtained DNN and kNN emulators as ``local\" emulators, since as opposed to the emulators constructed for cement system 1, the current emulators are only valid for the input conditions encapsulated in the RT-based training set.\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1.5cm}\\includegraphics[width=50pc]{Figure9.png}\n\t\\caption{Scatter plots of the complex relationships between the five considered inputs for (1) the computationally cheap RT simulation of cement system 2 (orange dots) performed with the original HPx code, (2) the corresponding sampled points by kernel density estimation (KDE) using the selected bandwith (turquoise dots) and, (3) the same sampled KDE points after porosity correction (red dots).}\n\t\\label{fig9}\n\\end{figure}\n\nTraining performance of the DNN emulator is presented in Figure \\ref{fig10}. Similarly as for cement system 1, about 90\\% of the available data was used for the actual training of the DNN while the remaining 10\\% were used a validation set to contol overfitting. Lastly, performance is evaluated for both the trained DNN and kNN emulators using an independent test set of 10,000 examples. Overall, the accuracy of our ``local \" DNN emulator for this RT-inspired dataset is rather large with $Q_2$ values always greater or equal than 0.998. Training performance of the corresponding local kNN emulator is equally good (not shown). Regarding speedup, for this problem single-threaded PHREEQC achieves about 210 geochemical calculations per second on our used Intel\\textsuperscript{\\textregistered} i7 CPU while the GPU-based DNN and kNN emulators are both about 3000 times faster when predicting the 10,000 test points at once .\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure10.png}\n\t\\caption{1-1 plots of local DNN emulation performance obtained for system 2 when the local DNN is trained using 1,000,000 samples. The $x$-axis and $y$-axis present the original and emulated 10,000 independent test data points, respectively. The RMSE and $Q_2$ coefficient denote the root-mean-square-error and coefficient of determination in testing mode, respectively, between the original and emulated 10,000 test data points.}\n\t\\label{fig10}\n\\end{figure}\n\n\\subsubsection{Reactive transport simulation}\n\\label{prob2res}\nOur ``local\" DNN performs rather well when applied to the 61 $\\times$ 61 grid size and a time period of 6 years (Figures \\ref{fig11} - \\ref{fig12}). This for all components but C, for which some localized deviations appear between original and emulated 2D concentration profiles towards the end of the simulation period (Figure \\ref{fig12}). When applied to the 121 $\\times$ 121 grid, HPx$_{\\rm py}$-DNN produces additional discrepancies for O, Si and Al towards the end of the simulation period (Figures \\ref{fig13} - \\ref{fig14}). Yet most of the observed artifacts could probably be smoothed out by using post-filtering. The associated speedups are listed in Table \\ref{table4}. These speedups are larger to those obtained for cement system 1, with values between 8 and 9 when evaluated against HPx$_{\\rm{4C}}$. These speedups represent about 85 \\% to 90 \\% of the maximum possible speedup (Table \\ref{table3}). Overall, these findings indicate that for the considered problem, our RT-based training of a local DNN only works is the training set is sufficiently representative of the particular geochemical conditions encountered in the computationally demanding simulations, which is arguably not easy to achieve. This limitation is further discussed in section \\ref{discussion}.\n\nWe note a more uniform behavior for HPx$_{\\rm py}$-kNN across grid sizes than for HPx$_{\\rm py}$-DNN. Here the results for the 121  $\\times$ 121 grid are only slightly less accurate than those associated with the 61 $ \\times$ 61 grid (see Figures \\ref{fig15} - \\ref{fig16} where for brevity we only show concentration profiles for the C Al and S components). Furthermore, whenever observed the discrepancies between original and emulated profiles are more regularly scattered than for HPx$_{\\rm py}$-DNN. Note also that herein too, post-filtering could likely smooth out a large part of these deviations. In addition, owing to the use of a GPU to achieve the kNN calculations, the speedup provided by HPx$_{\\rm py}$-kNN are as large as those provided by HPx$_{\\rm py}$-DNN (Table \\ref{table4}).\n\nWith respect to the emulated solid amounts, the HPx$_{\\rm py}$-DNN results look visually good for the the 61 $\\times$ 61 grid. This is shown in Figure \\ref{fig17} for the C, Al and S chemical components, while emulation of the H, O Ca and Si chemical components is globally of similar quality (not shown). Nevertheless, significant deviations appear towards the end of the simulation period for every chemical component (see Figure \\ref{fig18} for the C, Al and S chemical components, emulation of the H, O Ca and Si chemical components shows the same level of mismatch). The HPx$_{\\rm py}$-kNN predictions are also fairly accurate for the 61 $\\times$ 61 (see Figure \\ref{fig19} for the C, Al and S chemical components, emulation of the H, O Ca and Si chemical components exhibits a globally similar quality) while some discrepancies show up at the end of the simulation (Figure \\ref{fig20}). However, the mismatch is less pronounced than for HPx$_{\\rm py}$-DNN. Overall, HPx$_{\\rm py}$-kNN appears to be somewhat more robust than HPx$_{\\rm py}$-DNN for this cement system, while providing the same (large) speedup.\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure11.png}\n\t\\caption{2D concentration profiles obtained for cement system 2 after 2 and 6 (final time step) years. RTM means the original HPx$_{\\rm{4C}}$ model and TM+DNN denotes the Hydrus transport model coupled with our DNN geochemical emulator (HPx$_{\\rm py}$-DNN). The first to fourth row present profiles for $Ca^{\\rm{conc}}$, $Si^{\\rm{conc}}$, $H^{\\rm{conc}}$, and $O^{\\rm{conc}}$, respectively. The considered grid size is 61 $\\times$ 61.}\n\t\\label{fig11}\n\\end{figure}\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure12.png}\n\t\\caption{2D concentration profiles obtained for cement system 2 after 2 and 6 (final time step) years. RTM means the original HPx$_{\\rm{4C}}$ model and TM+DNN denotes the Hydrus transport model coupled with our DNN geochemical emulator (HPx$_{\\rm py}$-DNN). The first to third row present profiles for $C^{\\rm{conc}}$, $Al^{\\rm{conc}}$, and $S^{\\rm{conc}}$, respectively. The considered grid size is 61 $\\times$ 61.}\n\t\\label{fig12}\n\\end{figure}\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure13.png}\n\t\\caption{2D concentration profiles obtained for cement system 2 after 2 and 6 (final time step) years. RTM means the original HPx$_{\\rm{4C}}$ model and TM+DNN denotes the Hydrus transport model coupled with our DNN geochemical emulator (HPx$_{\\rm py}$-DNN). The first to fourth row present profiles for $Ca^{\\rm{conc}}$, $Si^{\\rm{conc}}$, $H^{\\rm{conc}}$, and $O^{\\rm{conc}}$, respectively. The considered grid size is 121 $\\times$ 121.}\n\t\\label{fig13}\n\\end{figure}\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure14.png}\n\t\\caption{2D concentration profiles obtained for cement system 2 after 2 and 6 (final time step) years. RTM means the original HPx$_{\\rm{4C}}$ model and TM+DNN denotes the Hydrus transport model coupled with our DNN geochemical emulator (HPx$_{\\rm py}$-DNN). The first to third row present profiles for $C^{\\rm{conc}}$, $Al^{\\rm{conc}}$, and $S^{\\rm{conc}}$, respectively. The considered grid size is 121 $\\times$ 121.}\n\t\\label{fig14}\n\\end{figure}\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure15.png}\n\t\\caption{2D concentration profiles obtained for cement system 2 after 2 and 6 (final time step) years. RTM means the original HPx$_{\\rm{4C}}$ model and TM+kNN denotes the Hydrus transport model coupled with our kNN geochemical emulator (HPx$_{\\rm py}$-kNN).  The first to third row present profiles for $C^{\\rm{conc}}$, $Al^{\\rm{conc}}$, and $S^{\\rm{conc}}$, respectively. The considered grid size is 61 $\\times$ 61.}\n\t\\label{fig15}\n\\end{figure}\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure16.png}\n\t\\caption{2D concentration profiles obtained for cement system 2 after 2 and 6 (final time step) years. RTM means the original HPx$_{\\rm{4C}}$ model and TM+kNN denotes the Hydrus transport model coupled with our kNN geochemical emulator (HPx$_{\\rm py}$-kNN). The first to third row present profiles for $C^{\\rm{conc}}$, $Al^{\\rm{conc}}$, and $S^{\\rm{conc}}$, respectively. The considered grid size is 121 $\\times$ 121.}\n\t\\label{fig16}\n\\end{figure}\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure17.png}\n\t\\caption{2D solid amount profiles of the C, Al and S chemical components of cement system 2 after 2 and 6 (final time step) years. RTM means the original HPx$_{\\rm{4C}}$ model and TM+DNN denotes the Hydrus transport model coupled with our DNN geochemical emulator (HPx$_{\\rm py}$-DNN). The first to third row present profiles for $C^{\\rm{solid}}$, $Al^{\\rm{solid}}$, and $S^{\\rm{solid}}$, respectively The considered grid size is 61 $\\times$ 61.}\n\t\\label{fig17}\n\\end{figure}\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure18.png}\n\t\\caption{2D solid amount profiles of the C, Al and S chemical components of cement system 2 after 2 and 6 (final time step) years. RTM means the original HPx$_{\\rm{4C}}$ model and TM+DNN denotes the Hydrus transport model coupled with our DNN geochemical emulator (HPx$_{\\rm py}$-DNN). The first to third row present profiles for $C^{\\rm{solid}}$, $Al^{\\rm{solid}}$, and $S^{\\rm{solid}}$, respectively The considered grid size is 121 $\\times$ 121.}\n\t\\label{fig18}\n\\end{figure}\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure19.png}\n\t\\caption{2D solid amount profiles of the C, Al and S chemical components of cement system 2 after 2 and 6 (final time step) years. RTM means the original HPx$_{\\rm{4C}}$ model and TM+kNN denotes the Hydrus transport model coupled with our kNN geochemical emulator (HPx$_{\\rm py}$-kNN). The first to third row present profiles for $C^{\\rm{solid}}$, $Al^{\\rm{solid}}$, and $S^{\\rm{solid}}$, respectively The considered grid size is 61 $\\times$ 61.}\n\t\\label{fig19}\n\\end{figure}\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure20.png}\n\t\\caption{2D solid amount profiles of the C, Al and S chemical components of cement system 2 after 2 and 6 (final time step) years. RTM means the original HPx$_{\\rm{4C}}$ model and TM+kNN denotes the Hydrus transport model coupled with our kNN geochemical emulator (HPx$_{\\rm py}$-kNN). The first to third row present profiles for $C^{\\rm{solid}}$, $Al^{\\rm{solid}}$, and $S^{\\rm{solid}}$, respectively The considered grid size is 121 $\\times$ 121.}\n\t\\label{fig20}\n\\end{figure}\n\n\\begin{table}[h!]\n\t\\caption{Speedups offered by the KNN and DNN emulators in HPx$_{\\rm py}$ for the reactive transport simulations considered for cement system 2. The HPx$_{\\rm{4C}}$ calculations involve the parallelization of PHREEQC over our 4 CPUs. The HPx$_{\\rm{1C}}$ calculations are performed on a single CPU. Both the kNN and DNN predictions make use of a GPU. ML signifies the used machine learning method for emulation, BC denotes the type of flow boundary conditions and GS is the grid size. The maximum possible speedups associated with HPx$_{\\rm{4C}}$ and HP$_{\\rm{1C}}$,  Max SP HPx$_{\\rm{4C}}$ and  Max SP HPx$_{\\rm{1C}}$, correspond to an hypothetical situation where the geochemical calculations incur zero computational cost.}\n\n\t\\begin{center}\n\t\t\\begin{tabular}{cccccccc}\n\t\t\t\\hline\n\t\t\t& & & & & & & \\\\\n\t\t\tML & BC & GS & HPx$_{\\rm{4C}}$ time (s) & SP HPx$_{\\rm{4C}}$ & Max SP HPx$_{\\rm{4C}}$ & SP HPx$_{\\rm{1C}}$ & Max SP HPx$_{\\rm{1C}}$ \\\\\n\t\t\tDNN & ADV & 61 $\\times$ 61 & 21,415 & 8.2 & 9.0 & 30.3 & 33.1\\\\\n\t\t\tkNN & ADV & 61 $\\times$ 61 & 21,415 & 7.9 & 9.0 & 28.9 & 33.1\\\\\n\t\t\tDNN & ADV & 121 $\\times$ 121 & 199,841 & 8.2 & 9.5 & 29.9 & 32.8\\\\\n\t\t\tkNN & ADV & 121 $\\times$ 121 & 199,841 & 8.5 & 9.5 & 31.3 & 32.8\\\\\n\t\t\t\\hline\n\t\t\\end{tabular}\n\t\\end{center}\n\n\t\\label{table4}\n\\end{table}\n\n\\FloatBarrier\n\n3.2.2 Reactive transport simulation\n\\subsubsection{Reactive transport simulation}\n\\label{prob2res}\nOur ``local\" DNN performs rather well when applied to the 61 $\\times$ 61 grid size and a time period of 6 years (Figures \\ref{fig11} - \\ref{fig12}). This for all components but C, for which some localized deviations appear between original and emulated 2D concentration profiles towards the end of the simulation period (Figure \\ref{fig12}). When applied to the 121 $\\times$ 121 grid, HPx$_{\\rm py}$-DNN produces additional discrepancies for O, Si and Al towards the end of the simulation period (Figures \\ref{fig13} - \\ref{fig14}). Yet most of the observed artifacts could probably be smoothed out by using post-filtering. The associated speedups are listed in Table \\ref{table4}. These speedups are larger to those obtained for cement system 1, with values between 8 and 9 when evaluated against HPx$_{\\rm{4C}}$. These speedups represent about 85 \\% to 90 \\% of the maximum possible speedup (Table \\ref{table3}). Overall, these findings indicate that for the considered problem, our RT-based training of a local DNN only works is the training set is sufficiently representative of the particular geochemical conditions encountered in the computationally demanding simulations, which is arguably not easy to achieve. This limitation is further discussed in section \\ref{discussion}.\n\nWe note a more uniform behavior for HPx$_{\\rm py}$-kNN across grid sizes than for HPx$_{\\rm py}$-DNN. Here the results for the 121  $\\times$ 121 grid are only slightly less accurate than those associated with the 61 $ \\times$ 61 grid (see Figures \\ref{fig15} - \\ref{fig16} where for brevity we only show concentration profiles for the C Al and S components). Furthermore, whenever observed the discrepancies between original and emulated profiles are more regularly scattered than for HPx$_{\\rm py}$-DNN. Note also that herein too, post-filtering could likely smooth out a large part of these deviations. In addition, owing to the use of a GPU to achieve the kNN calculations, the speedup provided by HPx$_{\\rm py}$-kNN are as large as those provided by HPx$_{\\rm py}$-DNN (Table \\ref{table4}).\n\nWith respect to the emulated solid amounts, the HPx$_{\\rm py}$-DNN results look visually good for the the 61 $\\times$ 61 grid. This is shown in Figure \\ref{fig17} for the C, Al and S chemical components, while emulation of the H, O Ca and Si chemical components is globally of similar quality (not shown). Nevertheless, significant deviations appear towards the end of the simulation period for every chemical component (see Figure \\ref{fig18} for the C, Al and S chemical components, emulation of the H, O Ca and Si chemical components shows the same level of mismatch). The HPx$_{\\rm py}$-kNN predictions are also fairly accurate for the 61 $\\times$ 61 (see Figure \\ref{fig19} for the C, Al and S chemical components, emulation of the H, O Ca and Si chemical components exhibits a globally similar quality) while some discrepancies show up at the end of the simulation (Figure \\ref{fig20}). However, the mismatch is less pronounced than for HPx$_{\\rm py}$-DNN. Overall, HPx$_{\\rm py}$-kNN appears to be somewhat more robust than HPx$_{\\rm py}$-DNN for this cement system, while providing the same (large) speedup.\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure11.png}\n\t\\caption{2D concentration profiles obtained for cement system 2 after 2 and 6 (final time step) years. RTM means the original HPx$_{\\rm{4C}}$ model and TM+DNN denotes the Hydrus transport model coupled with our DNN geochemical emulator (HPx$_{\\rm py}$-DNN). The first to fourth row present profiles for $Ca^{\\rm{conc}}$, $Si^{\\rm{conc}}$, $H^{\\rm{conc}}$, and $O^{\\rm{conc}}$, respectively. The considered grid size is 61 $\\times$ 61.}\n\t\\label{fig11}\n\\end{figure}\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure12.png}\n\t\\caption{2D concentration profiles obtained for cement system 2 after 2 and 6 (final time step) years. RTM means the original HPx$_{\\rm{4C}}$ model and TM+DNN denotes the Hydrus transport model coupled with our DNN geochemical emulator (HPx$_{\\rm py}$-DNN). The first to third row present profiles for $C^{\\rm{conc}}$, $Al^{\\rm{conc}}$, and $S^{\\rm{conc}}$, respectively. The considered grid size is 61 $\\times$ 61.}\n\t\\label{fig12}\n\\end{figure}\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure13.png}\n\t\\caption{2D concentration profiles obtained for cement system 2 after 2 and 6 (final time step) years. RTM means the original HPx$_{\\rm{4C}}$ model and TM+DNN denotes the Hydrus transport model coupled with our DNN geochemical emulator (HPx$_{\\rm py}$-DNN). The first to fourth row present profiles for $Ca^{\\rm{conc}}$, $Si^{\\rm{conc}}$, $H^{\\rm{conc}}$, and $O^{\\rm{conc}}$, respectively. The considered grid size is 121 $\\times$ 121.}\n\t\\label{fig13}\n\\end{figure}\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure14.png}\n\t\\caption{2D concentration profiles obtained for cement system 2 after 2 and 6 (final time step) years. RTM means the original HPx$_{\\rm{4C}}$ model and TM+DNN denotes the Hydrus transport model coupled with our DNN geochemical emulator (HPx$_{\\rm py}$-DNN). The first to third row present profiles for $C^{\\rm{conc}}$, $Al^{\\rm{conc}}$, and $S^{\\rm{conc}}$, respectively. The considered grid size is 121 $\\times$ 121.}\n\t\\label{fig14}\n\\end{figure}\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure15.png}\n\t\\caption{2D concentration profiles obtained for cement system 2 after 2 and 6 (final time step) years. RTM means the original HPx$_{\\rm{4C}}$ model and TM+kNN denotes the Hydrus transport model coupled with our kNN geochemical emulator (HPx$_{\\rm py}$-kNN).  The first to third row present profiles for $C^{\\rm{conc}}$, $Al^{\\rm{conc}}$, and $S^{\\rm{conc}}$, respectively. The considered grid size is 61 $\\times$ 61.}\n\t\\label{fig15}\n\\end{figure}\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure16.png}\n\t\\caption{2D concentration profiles obtained for cement system 2 after 2 and 6 (final time step) years. RTM means the original HPx$_{\\rm{4C}}$ model and TM+kNN denotes the Hydrus transport model coupled with our kNN geochemical emulator (HPx$_{\\rm py}$-kNN). The first to third row present profiles for $C^{\\rm{conc}}$, $Al^{\\rm{conc}}$, and $S^{\\rm{conc}}$, respectively. The considered grid size is 121 $\\times$ 121.}\n\t\\label{fig16}\n\\end{figure}\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure17.png}\n\t\\caption{2D solid amount profiles of the C, Al and S chemical components of cement system 2 after 2 and 6 (final time step) years. RTM means the original HPx$_{\\rm{4C}}$ model and TM+DNN denotes the Hydrus transport model coupled with our DNN geochemical emulator (HPx$_{\\rm py}$-DNN). The first to third row present profiles for $C^{\\rm{solid}}$, $Al^{\\rm{solid}}$, and $S^{\\rm{solid}}$, respectively The considered grid size is 61 $\\times$ 61.}\n\t\\label{fig17}\n\\end{figure}\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure18.png}\n\t\\caption{2D solid amount profiles of the C, Al and S chemical components of cement system 2 after 2 and 6 (final time step) years. RTM means the original HPx$_{\\rm{4C}}$ model and TM+DNN denotes the Hydrus transport model coupled with our DNN geochemical emulator (HPx$_{\\rm py}$-DNN). The first to third row present profiles for $C^{\\rm{solid}}$, $Al^{\\rm{solid}}$, and $S^{\\rm{solid}}$, respectively The considered grid size is 121 $\\times$ 121.}\n\t\\label{fig18}\n\\end{figure}\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure19.png}\n\t\\caption{2D solid amount profiles of the C, Al and S chemical components of cement system 2 after 2 and 6 (final time step) years. RTM means the original HPx$_{\\rm{4C}}$ model and TM+kNN denotes the Hydrus transport model coupled with our kNN geochemical emulator (HPx$_{\\rm py}$-kNN). The first to third row present profiles for $C^{\\rm{solid}}$, $Al^{\\rm{solid}}$, and $S^{\\rm{solid}}$, respectively The considered grid size is 61 $\\times$ 61.}\n\t\\label{fig19}\n\\end{figure}\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure20.png}\n\t\\caption{2D solid amount profiles of the C, Al and S chemical components of cement system 2 after 2 and 6 (final time step) years. RTM means the original HPx$_{\\rm{4C}}$ model and TM+kNN denotes the Hydrus transport model coupled with our kNN geochemical emulator (HPx$_{\\rm py}$-kNN). The first to third row present profiles for $C^{\\rm{solid}}$, $Al^{\\rm{solid}}$, and $S^{\\rm{solid}}$, respectively The considered grid size is 121 $\\times$ 121.}\n\t\\label{fig20}\n\\end{figure}\n\n\\begin{table}[h!]\n\t\\caption{Speedups offered by the KNN and DNN emulators in HPx$_{\\rm py}$ for the reactive transport simulations considered for cement system 2. The HPx$_{\\rm{4C}}$ calculations involve the parallelization of PHREEQC over our 4 CPUs. The HPx$_{\\rm{1C}}$ calculations are performed on a single CPU. Both the kNN and DNN predictions make use of a GPU. ML signifies the used machine learning method for emulation, BC denotes the type of flow boundary conditions and GS is the grid size. The maximum possible speedups associated with HPx$_{\\rm{4C}}$ and HP$_{\\rm{1C}}$,  Max SP HPx$_{\\rm{4C}}$ and  Max SP HPx$_{\\rm{1C}}$, correspond to an hypothetical situation where the geochemical calculations incur zero computational cost.}\n\n\t\\begin{center}\n\t\t\\begin{tabular}{cccccccc}\n\t\t\t\\hline\n\t\t\t& & & & & & & \\\\\n\t\t\tML & BC & GS & HPx$_{\\rm{4C}}$ time (s) & SP HPx$_{\\rm{4C}}$ & Max SP HPx$_{\\rm{4C}}$ & SP HPx$_{\\rm{1C}}$ & Max SP HPx$_{\\rm{1C}}$ \\\\\n\t\t\tDNN & ADV & 61 $\\times$ 61 & 21,415 & 8.2 & 9.0 & 30.3 & 33.1\\\\\n\t\t\tkNN & ADV & 61 $\\times$ 61 & 21,415 & 7.9 & 9.0 & 28.9 & 33.1\\\\\n\t\t\tDNN & ADV & 121 $\\times$ 121 & 199,841 & 8.2 & 9.5 & 29.9 & 32.8\\\\\n\t\t\tkNN & ADV & 121 $\\times$ 121 & 199,841 & 8.5 & 9.5 & 31.3 & 32.8\\\\\n\t\t\t\\hline\n\t\t\\end{tabular}\n\t\\end{center}\n\n\t\\label{table4}\n\\end{table}\n\n\\FloatBarrier\n\n",
        "label": "fig15",
        "Descriptive_question1": "What chemical component is represented in the first row of figure_15?",
        "Descriptive_question2": "What is the grid size used in figure_15?",
        "Reasoning_question1": "How do the concentration profiles of C, Al, and S in figure_15 compare between the RTM and TM+kNN models after 6 years?",
        "Reasoning_question2": "What might be the reasons for any observed differences between the original RTM and the emulated TM+kNN concentration profiles in figure_15 over the simulation period?",
        "Descriptive_answer1": "C^{\\rm{conc}}",
        "Descriptive_answer2": "61 \\times 61",
        "Reasoning_answer1": "By examining figure_15, the concentration profiles for C, Al, and S after 6 years in both the RTM (original HPx_{4C} model) and TM+kNN (Hydrus transport model with kNN geochemical emulator) are visually compared. The profiles for these components show similar spatial distribution patterns and magnitudes between RTM and TM+kNN, indicating that the kNN model effectively emulates the original model for these elements. Minor discrepancies might be present but overall, the concentration fields appear closely aligned after 6 years.",
        "Reasoning_answer2": "The observed differences between the original RTM and the emulated TM+kNN concentration profiles in figure_15 over the simulation period can be attributed to several factors. First, the kNN emulator relies on an approximate nearest neighbor search (via the FAISS package) rather than exact matches, potentially introducing small prediction errors. Second, the kNN model is trained on a local dataset generated through kernel density estimation based on a limited and possibly imperfectly representative set of reactive transport simulations, which may lead to slight inaccuracies when extrapolating to the entire domain or later times. Third, data communication overhead and interpolation during the coupling of the transport model with the kNN emulator could introduce minor mismatches. Additionally, inherent nonlinearities and high dimensionality in the geochemical system challenge the emulator's precision. Despite these factors, the kNN model still captures the main trends effectively."
    },
    {
        "paper_id": "2107.07598.json",
        "image_id": "figure_16",
        "image_path": [
            "/home/yz979/scratch/chengye/scimolmo/test-images-tables/dataset_imagestables/2107.07598/images/Figure16.png"
        ],
        "caption": "2D concentration profiles obtained for cement system 2 after 2 and 6 (final time step) years. RTM means the original HPx$_{\\rm{4C}}$ model and TM+kNN denotes the Hydrus transport model coupled with our kNN geochemical emulator (HPx$_{\\rm py}$-kNN). The first to third row present profiles for $C^{\\rm{conc}}$, $Al^{\\rm{conc}}$, and $S^{\\rm{conc}}$, respectively. The considered grid size is 121 $\\times$ 121.",
        "classify": "Chart",
        "section_info": "3 Results\n\\section{Results}\n\\label{results}\n\n\\subsection{Ca-Si Problem}\nFor this first cement system, the emulation problem consists of predicting at each time step of the RT simulation the (output) Ca, Si, H and O aqueous concentrations (mol/kg of water or mol/kgw) from the (input) total amounts of Ca and Si (mol). \n\n\\subsubsection{Training the emulators}\n\\label{train_res1}\nHere the kNN and DNN emulators are firstly trained using a set of 400,000 test examples for both. This training set is obtained by randomly sampling the two-dimensional input space by latin hypercube sampling (LHS) between $\\left[0,0\\right]$ and $\\rm{\\left[Ca^{tot}_{max},Si^{tot}_{max}\\right]}$, and running PHREEQC for each input sample, $\\rm{\\textbf{x}_i = \\left[Ca^{tot}_i, Si^{tot}_i\\right]}$ to get the corresponding output vectors, $\\rm{\\textbf{y}_i = \\left[Ca^{conc}_i,Si^{conc}_i,H^{conc}_i,O^{conc}_i\\right]}$. The upper bounds, $\\rm{Ca^{tot}_{max}}$ and $\\rm{Si^{tot}_{max}}$ are defined based on a cheap full RT simulation with advective-dispersive transport using a small 1D domain of 51 nodes. It is worth noting that the total amounts of $\\textbf{x}_i$, corresponding to the PHREEQC-simulated concentrations, $\\textbf{y}_i$, have to be corrected for the different amount of water between the training set and the transport simulations. Doing so, it turns out that about 20 \\% of the post-corrected $x_i$ values exceed their pre-defined upper bounds and these excessively large values need to be filtered out. Creating the 400,000 training examples thus required about 500,000 PHREEQC runs. As stated earlier, for this problem single-threaded PHREEQC performs about 670 geochemical calculations per second on our used Intel\\textsuperscript{\\textregistered} i7 CPU.\n\nWith respect to kNN, the tuning parameters are the number of neighbors, $k$, the type of distance measure, and the interpolation technique. We simply used the default settings: $k = 5$, euclidean distance and inverse-distance interpolation. Regarding training of the DNN, the 400,000 sample were split between the training set itself (90 \\% of the data) and a validation set (10 \\% of the data). The latter serves to monitor the evolution of the selected mean squared error loss function on samples that are not used for training, thereby detecting potential overfitting. If the validation loss stops decreasing before the fixed number of epochs has been completed, then  training is stopped. Importantly, the emulation is achieved in log-space for both the input, $X$ and output $Y$, domains. This because total amounts and concentrations of the involved components typically cover many orders of magnitudes (up to 10 orders or more). Using a DNN also requires some form of data normalization or standardization. Here both the $\\rm{log\\left(\\textbf{x}_i\\right)}$ and $\\rm{log\\left(\\textbf{y}_i\\right)}$ vectors are standardized around 0 with standard deviation of 1.\n\nFigure \\ref{fig1} illustrates the trained emulators' performance for geochemical predictions using an independent test set that comprises 10,000 test examples. Both kNN (Figures \\ref{fig1}a - d) and DNN (Figures \\ref{fig1}e - h) appear to be rather accurate . DNN also shows a slight degradation for the larger concentration values  (Figures \\ref{fig1}e - h). The latter is likely due to the combination of a small proportion of large concentration values in the training set with the log-transformation that implicitly pushes the DNN to try harder to fit the smaller concentrations during training. Regarding speedup and as written earlier, for this setup the single-threaded kNN method is 300 times faster than single-threaded PHREEQC for predicting the 10,000 concentration vectors all at once. The computational savings allowed by the DNN emulator when ran on our GPU is higher with a speedup as large as 4000 for predicting the same 10,000 concentration vectors all at once. \n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{0cm}\\includegraphics[width=35pc]{Figure1.png}\n\t\\caption{1-1 plots of the kNN (subfigures (a) - (d)) and DNN (subfigures (e) - (h)) emulators' performance obtained for system 1 when the kNN training base contains 400,000 samples and the DNN is trained using the same 400,000 samples. Here ``true\" means the original PHREEQC-simulated data and ``predicted\" denotes the emulated (that is, kNN-simulated and DNN-simulated) data. Hence, the $x$-axis and $y$-axis present the original and emulated 10,000 independent test data points, respectively. The RMSE and $Q_2$ coefficient denote the root-mean-square-error and coefficient of determination in testing mode, respectively, between the original and emulated 10,000 test data points.}\n\t\\label{fig1}\n\\end{figure}\n\nTo test the sensitivity of the emulators' performance to the training set size, training was also performed using reduced training sets comprising 100,000, 10,000, 1000 and 100 samples, respectively. It seen that the DNN performance achieved  when using 10,000 training samples is virtually the same as that obtained when using 400,000 training samples (Figure \\ref{fig2} and Table \\ref{table2}). It is only for training sets smaller than 1000 samples that the DNN performance starts to degrade significantly (Table \\ref{table2}). In contrast, the behavior of kNN appears to markedly decrease as the training set gets smaller (Figure \\ref{fig2} and Table \\ref{table1}).\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{0cm}\\includegraphics[width=35pc]{Figure2.png}\n\t\\caption{1-1 plots of the kNN (subfigures (a) - (d)) and DNN (subfigures (e) - (h)) emulators' performance obtained for system 1 when the kNN training base contains 10,000 samples and the DNN is trained using the same 10,000 samples. Here ``true\" means the original PHREEQC-simulated data and ``predicted\" denotes the emulated (that is, kNN-simulated and DNN-simulated) data. Hence, the $x$-axis and $y$-axis present the original and emulated 10,000 independent test data points, respectively. The RMSE and $Q_2$ coefficient denote the root-mean-square-error and coefficient of determination in testing mode, respectively, between the original and emulated 10,000 test data points.}\n\t\\label{fig2}\n\\end{figure}\n\n\\begin{table}[h!]\n\t\\caption{Performance of the DNN and kNN emulators for cement system 1 and different training set sizes. For brevity, only the results for Ca$^{conc}$ and Si$^{conc}$ are shown. The units are mol per kg of water (mol/kgw). ML refers to the type of emulator, TR signifies the size of the training set, and the RMSE and $Q_2$ coefficient denote the root-mean-square-error and coefficient of determination in testing mode, respectively, between the original and emulated 10,000 test data points.}\n\t\\begin{center}\n\t\t\\begin{tabular}{cccccccc}\n\t\t\t\\hline\n\t\t\t& & & & & \\\\\n\t\t\tML & TR & RMSE - Ca$^{conc}$ & $Q_2$ - Ca$^{conc}$ & RMSE - Si$^{conc}$ & $Q_2$ - Si$^{conc}$\\\\\n\t\t\tDNN & 4 $\\times$ 10$^{5}$ & 4.50 $\\times$ 10$^{-5}$ & 0.9999 & 1.21 $\\times$ 10$^{-6}$ & 0.9999\\\\\n\t\t\tDNN & 1 $\\times$ 10$^{5}$ & 5.06 $\\times$ 10$^{-5}$ & 0.9999 & 1.30 $\\times$ 10$^{-6}$ & 0.9999\\\\\n\t\t\tDNN & 1 $\\times$ 10$^{4}$ & 4.57 $\\times$ 10$^{-5}$ & 0.9999 & 1.48 $\\times$ 10$^{-6}$ & 0.9999\\\\\n\t\t\tDNN & 1 $\\times$ 10$^{3}$ & 8.02 $\\times$ 10$^{-5}$ & 0.9998 & 3.31 $\\times$ 10$^{-6}$ & 0.9994\\\\\n\t\t\tDNN & 1 $\\times$ 10$^{2}$ & 47.1 $\\times$ 10$^{-5}$ & 0.9935 & 9.02 $\\times$ 10$^{-6}$ & 0.9959\\\\\n\t\t\tkNN & 4 $\\times$ 10$^{5}$ & 2.52 $\\times$ 10$^{-5}$ & 1.0000 & 1.59 $\\times$ 10$^{-6}$ & 0.9999\\\\\n\t\t\tkNN & 1 $\\times$ 10$^{5}$ & 6.27 $\\times$ 10$^{-5}$ & 0.9999 & 4.20 $\\times$ 10$^{-6}$ & 0.9991\\\\\n\t\t\tkNN & 1 $\\times$ 10$^{4}$ & 14.1 $\\times$ 10$^{-5}$ & 0.9994 & 13.5 $\\times$ 10$^{-6}$ & 0.9908\\\\\n\t\t\tkNN & 1 $\\times$ 10$^{3}$ & 43.9 $\\times$ 10$^{-5}$ & 0.9944 & 74.6 $\\times$ 10$^{-6}$ & 0.7180\\\\\n\t\t\tkNN & 1 $\\times$ 10$^{2}$  & 96.4 $\\times$ 10$^{-5}$ & 0.9729 & 55.0 $\\times$ 10$^{-6}$ & 0.8468\\\\\n\t\t\\end{tabular}\n\t\\end{center}\n\t\\label{table2}\n\\end{table}\n\n\\FloatBarrier\n\n\\subsubsection{Reactive transport simulation}\n\\label{prob12Dres}\n\nThis section focuses on reactive transport simulations with HPx$_{\\rm py}$ within cement system 1, under both advective-dispersive and diffusive transport conditions. As written above, the domain sizes are both 61 $\\times$ 61 and 121 $\\times$ 121 for the advection-dispersion case and, because of computational constraints, solely 61 $\\times$ 61  for the diffusion case. In addition, the simulation time period is 2 years for the advection-dispersion case and 1 year for the diffusion case. Figures \\ref{fig3} and \\ref{fig4} present times series of original and emulated Ca, Si, H and O concentrations at 5 locations within the 2D domain for advective-dispersive transport conditions, for both our kNN-based (HPx$_{\\rm py}$-kNN) and DNN-based (HPx$_{\\rm py}$-DNN) reactive transport codes. It is seen that HPx$_{\\rm py}$-kNN and HPx$_{\\rm py}$-DNN both induce a quite good simulation accuracy. Also, the results for the diffusive transport case are of similarly good quality (not shown). Figures \\ref{fig5} - \\ref{fig6} provide more insights into the HPx$_{\\rm py}$-kNN and HPx$_{\\rm py}$-DNN performances by displaying 2D Ca, Si, H and O concentration profiles at a given time. For each experiment and chemical component, this time is selected as to be well representative of the simulated dynamics. It is observed that the original and emulated images are visually almost indistinguishable for the advection-dispersion case (Figure \\ref{fig5}). For the diffusion case, the emulators also perform quite well for Ca, H and O (Figures\\ref{fig6}a - c, g - i and j - l), while some slight to moderate discrepancies appear at the concentration front for Si (Figures \\ref{fig6}d - f). Notwithstanding, the Si concentration remains globally well predicted. Furthermore, Figures \\ref{fig7} - \\ref{fig8} present the original and emulated 2D solid amount profiles corresponding to Figures \\ref{fig5} - \\ref{fig6}. The original solid amount profiles are overall well approximated by HPx$_{\\rm py}$-kNN and HPx$_{\\rm py}$-DNN for the advection-dispersion case (Figure \\ref{fig7}), even though some mismatch appears at the border of the fully depleted zone for the H component. As of the diffusion case (Figure \\ref{fig8}), the same kind of mismatch is observed for the emulated solid amounts of H by HPx$_{\\rm py}$-kNN while the emulated profiles by HPx$_{\\rm py}$-DNN show some more discrepancies. Though we decided to present raw emulation results, we would like to stress that some if not all of the observed artifacts could likely be smoothed out by using some post-filtering such as median filtering.\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure3.png}\n\t\\caption{Time series of original (RTM, solid green lines) and HPx$_{\\rm py}$-kNN emulated (TM+kNN, dashed orange lines) concentrations (mol/kg) of Ca, H, O and Si at selected observations points for cement system 1 and advective-dispersive transport. Obs. 1 - 5 denote the selected observation points, with the following $\\left[x,y\\right]$ locations (in cm). Obs. 1:  $\\left[0.5, 2.5\\right]$, Obs. 2: $\\left[1, 2\\right]$, Obs. 3: $\\left[2, 2\\right]$, Obs. 4: $\\left[1, 1\\right]$, Obs. 5: $\\left[2, 1\\right]$. The results for the 121 $\\times$ 121 grid size are rather similar.}\n\t\\label{fig3}\n\\end{figure}\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure4.png}\n\t\\caption{Time series of original (RTM, solid green lines) and HPx$_{\\rm py}$-DNN emulated (TM+DNN, dashed orange lines) concentrations (mol/kg) of Ca, H, O and Si at selected observations points for cement system 1 and advective-dispersive transport. Obs. 1 - 5 denote the selected observation points, with the following $\\left[x,y\\right]$ locations (in cm). Obs. 1:  $\\left[0.5, 2.5\\right]$, Obs. 2: $\\left[1, 2\\right]$, Obs. 3: $\\left[2, 2\\right]$, Obs. 4: $\\left[1, 1\\right]$, Obs. 5: $\\left[2, 1\\right]$. The results for the 121 $\\times$ 121 grid size are rather similar.}\n\t\\label{fig4}\n\\end{figure}\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure5.png}\n\t\\caption{2D concentration profiles obtained for cement system 1 at the end of the 2-year simulation performed for the advection-dispersion case. RTM means the original HP$_{\\rm{4C}}$ model, TM+kNN denotes the Hydrus transport model coupled with our kNN geochemical emulator (HPx$_{\\rm py}$-kNN) and TM+DNN signifies the Hydrus transport model coupled with our DNN geochemical emulator (HPx$_{\\rm py}$-DNN). The first to fourth row present profiles for $Ca^{\\rm{conc}}$, $Si^{\\rm{conc}}$, $H^{\\rm{conc}}$, and $O^{\\rm{conc}}$, respectively. The considered grid size is 61 $\\times$ 61. The results for the 121 $\\times$ 121 grid are rather similar.}\n\t\\label{fig5}\n\\end{figure}\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure6.png}\n\t\\caption{2D concentration profiles obtained for cement system 1 at the final time step of the 1-year simulation performed for the diffusion case. RTM means the original HP$_{\\rm{4C}}$ model, TM+kNN denotes the Hydrus transport model coupled with our kNN geochemical emulator (HPx$_{\\rm py}$-kNN) and TM+DNN signifies the Hydrus transport model coupled with our DNN geochemical emulator (HPx$_{\\rm py}$-DNN). The first to fourth row present profiles for $Ca^{\\rm{conc}}$, $Si^{\\rm{conc}}$, $H^{\\rm{conc}}$, and $O^{\\rm{conc}}$, respectively. The considered grid size is 61 $\\times$ 61.}\n\t\\label{fig6}\n\\end{figure}\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure7.png}\n\t\\caption{2D solid amount profiles obtained for cement system 1 at the end of the 2-year simulation performed for the advection-dispersion case. RTM means the original HP$_{\\rm{4C}}$ model, TM+kNN denotes the Hydrus transport model coupled with our kNN geochemical emulator (HPx$_{\\rm py}$-kNN) and TM+DNN signifies the Hydrus transport model coupled with our DNN geochemical emulator (HPx$_{\\rm py}$-DNN). The first to fourth row present profiles for $Ca^{\\rm{solid}}$, $Si^{\\rm{solid}}$, $H^{\\rm{solid}}$, and $O^{\\rm{solid}}$, respectively. The considered grid size is 61 $\\times$ 61. The results for the 121 $\\times$ 121 grid are rather similar.}\n\t\\label{fig7}\n\\end{figure}\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure8.png}\n\t\\caption{2D solid amount profiles obtained for cement system 1 at the final time step of the 1-year simulation performed for the diffusion case. RTM means the original HP$_{\\rm{4C}}$ model, TM+kNN denotes the Hydrus transport model coupled with our kNN geochemical emulator (HPx$_{\\rm py}$-kNN) and TM+DNN signifies the Hydrus transport model coupled with our DNN geochemical emulator (HPx$_{\\rm py}$-DNN). The first to fourth row present profiles for $Ca^{\\rm{solid}}$, $Si^{\\rm{solid}}$, $H^{\\rm{solid}}$, and $O^{\\rm{solid}}$, respectively. The considered grid size is 61 $\\times$ 61.}\n\t\\label{fig8}\n\\end{figure}\n\nThe speedups associated with the considered problem are detailed in Table \\ref{table3}. It is noted that the GPU-based DNN emulator allows for a speedup that is close to optimal. Indeed, The DNN speedups overall represent 85 \\% to 95 \\% of the maximum possible speedups (that is, speedups that would be obtained if the geochemcical calculations would come at no cost at all). The speedups associated with single-threaded kNN remain substantial but only amount to 57 \\% - 65 \\% of the corresponding maximum speedups. As detailed in section \\ref{train_res1}, the used kNN and DNN implementations are found to be respectively 300 and 4000 faster than single-threaded PHREEQC when predicting 10,000 points all one once for this geochemical system. Based on these numbers one could have expected the achieved speedups to represent say 90 \\% (kNN) or 99 \\% (DNN) of the maximum possible ones. A large part of the gaps between achieved and maximum possible speedups is thus likely caused by the time required for communicating and exchanging data between the main C/C++ code and the Python-based emulators.\n\n\\begin{table}[h!]\n\t\\caption{Speedups offered by the KNN and DNN emulators in HPx$_{\\rm py}$ for the reactive transport simulations considered for cement system 1. The HPx$_{\\rm{4C}}$ calculations involve the parallelization of PHREEQC over our 4 CPUs. The HPx$_{\\rm{1C}}$ calculations are performed on a single CPU. The kNN predictions are performed on a single CPU using the scikit-learn implementation while the DNN predictions make use of our GPU. ML signifies the used machine learning method for emulation, TC denotes transport conditions (ADV: advection-dispersion, DIF: diffusion) and GS is the grid size. The maximum possible speedups associated with HPx$_{\\rm{4C}}$ and HPx$_{\\rm{1C}}$,  Max SP HPx$_{\\rm{4C}}$ and  Max SP HPx$_{\\rm{1C}}$, correspond to an hypothetical situation where the geochemical calculations incur zero computational cost.}\n\n\t\\begin{center}\n\t\t\\begin{tabular}{cccccccc}\n\t\t\t\\hline\n\t\t\t& & & & & & & \\\\\n\t\t\tML & TC & GS & HPx$_{\\rm{4C}}$ time (s) & SP HPx$_{\\rm{4C}}$ & Max SP HPx$_{\\rm{4C}}$ & SP HPx$_{\\rm{1C}}$ & Max SP HPx$_{\\rm{1C}}$ \\\\\n\t\t\tDNN & ADV & 61 $\\times$ 61 & 3189 & 6.8 & 7.7 & 24.5 & 28.5 \\\\\n\t\t\tkNN & ADV & 61 $\\times$ 61 & 3189 & 5.0 & 7.7 & 18.0 & 28.5 \\\\\n\t\t\tDNN & ADV & 121 $\\times$ 121 & 23,337 & 5.0 & 5.2 & 17.0 & 18.0 \\\\\n\t\t\tkNN & ADV & 121 $\\times$ 121 & 23,337 & 3.4 & 5.2 & 11.6 & 18.0 \\\\\n\t\t\tDNN & DIF & 61 $\\times$ 61 & 25,448 & 4.2 & 4.9 & 16.2 & 19.1 \\\\\n\t\t\tkNN & DIF & 61 $\\times$ 61 & 25,448 & 2.8 & 4.9 & 10.8 & 19.1 \\\\\n\t\t\t\\hline\n\t\t\\end{tabular}\n\t\\end{center}\n\n\t\\label{table3}\n\\end{table}\n\n\\FloatBarrier\n\n\\subsection{Al-C-Ca-S-Si Problem}\n\nFor this second cement system, the emulation problem consists of predicting at each time step of the RT simulation the (output) Al, C, Ca, S, Si, H and O aqueous concentrations (mol/kgw) from the (input) total amounts of Al, C, Ca, S and Si (mol). Here we focus on advective-dispersive transport only while similarly as for cement system 1, the considered domain sizes are 61 $\\times$ 61 and 121 $\\times$ 121. Moreover, the simulation time period is set to 6 years. \n\nAs mentioned earlier, for this higher-dimensional problem it is observed that the used scikit-learn kNN implementation becomes prohibitively slow compared to HPx$_{\\rm{4C}}$. We found that to get a good emulation accuracy the kNN training base needs to contain 1,000,000 samples (or more). This training base's size together with a 5-dimensional search space leads to an HPx$_{\\rm py}$-kNN reactive transport simulation time that is comparable to that of HPx$_{\\rm{4C}}$. Therefore, for this second cement system we built a custom kNN regressor around another kNN implementation contained in the FAISS package \\citep{faiss2017}. Our used FAISS variant allows for GPU computing and is much faster than scikit-learn for this cement system, but is slightly less accurate due to the use of an approximate rather than exact nearest neighbor search \\citep[see][for details]{faiss2017}.\n\n\\subsubsection{Training the DNN}\n\\label{train_res2}\n\nBuilding a good training set to perform a kNN search and learn the weights and biases of our DNN turned out to be a complicated task in this case. This because to make useful kNN predictions and/or learn an useful DNN, the training set must be sufficiently representative of the geochemical conditions encountered during the reactive transport simulation one wish to perform with HPx$_{\\rm py}$-kNN and the trained HPx$_{\\rm py}$-DNN. In contrast to cement system 1, creating the training set by sampling the $X$-space with a controlled randomness between predefined lower and upper bounds did not prove successful. We tried that strategy by drawing as much as 4,000,000 5-dimensional $\\textbf{x}$ vectors from the $X$-space using a Sobol low-discrepancy sequence \\citep[][]{Sobol1967, Joe-Kuo2003}. Such low-discrepancy sampling scheme covers the 5-dimensional hypercube more uniformly than LHS. Despite a good performance on the test set (not shown), the resulting DNN accuracy in reactive transport mode was never deemed satisfying. In other words, no satisfying ``global\" or ``universal\" DNN emulator could be devised for this cement system. This is probably caused by the fact for this problem, the input (5 total amounts) and output (7 aqueous concentrations) spaces are quite nonlinearly related and both cover 6 to 10 orders of magnitudes depending on the considered element. Therefore, we resorted to the alternative training strategy detailed below. The latter basically tries to grasp the complex correlations and higher-order dependencies that exist between the elements of $\\textbf{x}$ (total amounts, input space) for a given reactive transport simulation setup, in order to produce a training set that honors these between-input relationships.\n\n\\begin{itemize}\n\t\\item Perform a ``cheap\" full reactive transport simulation under the transport conditions and geochemistry of interest and collect the resulting $\\textbf{x}$ -$\\textbf{y}$  pairs of examples (for the considered grid nodes and time steps). Computational demand controls what domain size and simulation time period can be used for this cheap calculation. We used a modest 16 $\\times$ 16 domain and a simulation time period of 10 years. The associated HPx$_{\\rm{4C}}$ runtimes is 180 s.\n\t\n\t\\item Fit a kernel density estimator (KDE) with a Gaussian kernel to the collected $\\textbf{x}$ vectors (encapsulated in the $\\textbf{X}$ array) and generate a fixed number of new input vectors, $\\textbf{x}_{KDE}$. Then run PHREEQC for the $\\textbf{X}_{KDE}$ set to get the corresponding output set, $\\textbf{Y}_{KDE}$. Now apply the correction for porosity described in section \\ref{train_res1} to the $\\textbf{X}_{KDE}$ set and form the training set by merging the ensemble of $\\textbf{x}$-$\\textbf{y}$ pairs with that of the $\\textbf{x}_{KDE}$-$\\textbf{y}_{KDE}$ pairs. The number of produced unique examples by the considered cheap HPx$_{\\rm{4C}}$ simulations varied between 10,000 and 50,000. The KDE-based enrichment of this dataset was deemed necessary to provide more input variability, thereby avoiding overfitting of the trained DNN and improving the kNN accuracy, while still honoring the complex between-input relationships. The number of KDE-generated samples was set as to obtain a total training set size of 1,000,000 examples. A key component of the approach is the bandwith parameter of the Gaussian KDE kernel which controls how much the KDE-generated samples depart from the original ensemble. After limited trial and error, we fixed the kernel bandwith to 0.0025 for the considered case studies.\n\t\n\\end{itemize}\n\nThe scatter plots in Figure \\ref{fig9} illustrate our training set creation procedure. The orange dots depict the pairwise relationships observed between the 5 elements of $\\textbf{x}$ in the cheap simulation. The red and cyan dots in Figure \\ref{fig9} represent the KDE-generated samples with the selected bandwidth, before and after applying the correction for porosity, respectively. Training of the DNN is achieved using the ensemble of original and KDE-corrected input points. We refer to this kind of dataset as RT-based, since it is based on an albeit cheap, full RT simulation. Furthermore, we refer to the obtained DNN and kNN emulators as ``local\" emulators, since as opposed to the emulators constructed for cement system 1, the current emulators are only valid for the input conditions encapsulated in the RT-based training set.\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1.5cm}\\includegraphics[width=50pc]{Figure9.png}\n\t\\caption{Scatter plots of the complex relationships between the five considered inputs for (1) the computationally cheap RT simulation of cement system 2 (orange dots) performed with the original HPx code, (2) the corresponding sampled points by kernel density estimation (KDE) using the selected bandwith (turquoise dots) and, (3) the same sampled KDE points after porosity correction (red dots).}\n\t\\label{fig9}\n\\end{figure}\n\nTraining performance of the DNN emulator is presented in Figure \\ref{fig10}. Similarly as for cement system 1, about 90\\% of the available data was used for the actual training of the DNN while the remaining 10\\% were used a validation set to contol overfitting. Lastly, performance is evaluated for both the trained DNN and kNN emulators using an independent test set of 10,000 examples. Overall, the accuracy of our ``local \" DNN emulator for this RT-inspired dataset is rather large with $Q_2$ values always greater or equal than 0.998. Training performance of the corresponding local kNN emulator is equally good (not shown). Regarding speedup, for this problem single-threaded PHREEQC achieves about 210 geochemical calculations per second on our used Intel\\textsuperscript{\\textregistered} i7 CPU while the GPU-based DNN and kNN emulators are both about 3000 times faster when predicting the 10,000 test points at once .\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure10.png}\n\t\\caption{1-1 plots of local DNN emulation performance obtained for system 2 when the local DNN is trained using 1,000,000 samples. The $x$-axis and $y$-axis present the original and emulated 10,000 independent test data points, respectively. The RMSE and $Q_2$ coefficient denote the root-mean-square-error and coefficient of determination in testing mode, respectively, between the original and emulated 10,000 test data points.}\n\t\\label{fig10}\n\\end{figure}\n\n\\subsubsection{Reactive transport simulation}\n\\label{prob2res}\nOur ``local\" DNN performs rather well when applied to the 61 $\\times$ 61 grid size and a time period of 6 years (Figures \\ref{fig11} - \\ref{fig12}). This for all components but C, for which some localized deviations appear between original and emulated 2D concentration profiles towards the end of the simulation period (Figure \\ref{fig12}). When applied to the 121 $\\times$ 121 grid, HPx$_{\\rm py}$-DNN produces additional discrepancies for O, Si and Al towards the end of the simulation period (Figures \\ref{fig13} - \\ref{fig14}). Yet most of the observed artifacts could probably be smoothed out by using post-filtering. The associated speedups are listed in Table \\ref{table4}. These speedups are larger to those obtained for cement system 1, with values between 8 and 9 when evaluated against HPx$_{\\rm{4C}}$. These speedups represent about 85 \\% to 90 \\% of the maximum possible speedup (Table \\ref{table3}). Overall, these findings indicate that for the considered problem, our RT-based training of a local DNN only works is the training set is sufficiently representative of the particular geochemical conditions encountered in the computationally demanding simulations, which is arguably not easy to achieve. This limitation is further discussed in section \\ref{discussion}.\n\nWe note a more uniform behavior for HPx$_{\\rm py}$-kNN across grid sizes than for HPx$_{\\rm py}$-DNN. Here the results for the 121  $\\times$ 121 grid are only slightly less accurate than those associated with the 61 $ \\times$ 61 grid (see Figures \\ref{fig15} - \\ref{fig16} where for brevity we only show concentration profiles for the C Al and S components). Furthermore, whenever observed the discrepancies between original and emulated profiles are more regularly scattered than for HPx$_{\\rm py}$-DNN. Note also that herein too, post-filtering could likely smooth out a large part of these deviations. In addition, owing to the use of a GPU to achieve the kNN calculations, the speedup provided by HPx$_{\\rm py}$-kNN are as large as those provided by HPx$_{\\rm py}$-DNN (Table \\ref{table4}).\n\nWith respect to the emulated solid amounts, the HPx$_{\\rm py}$-DNN results look visually good for the the 61 $\\times$ 61 grid. This is shown in Figure \\ref{fig17} for the C, Al and S chemical components, while emulation of the H, O Ca and Si chemical components is globally of similar quality (not shown). Nevertheless, significant deviations appear towards the end of the simulation period for every chemical component (see Figure \\ref{fig18} for the C, Al and S chemical components, emulation of the H, O Ca and Si chemical components shows the same level of mismatch). The HPx$_{\\rm py}$-kNN predictions are also fairly accurate for the 61 $\\times$ 61 (see Figure \\ref{fig19} for the C, Al and S chemical components, emulation of the H, O Ca and Si chemical components exhibits a globally similar quality) while some discrepancies show up at the end of the simulation (Figure \\ref{fig20}). However, the mismatch is less pronounced than for HPx$_{\\rm py}$-DNN. Overall, HPx$_{\\rm py}$-kNN appears to be somewhat more robust than HPx$_{\\rm py}$-DNN for this cement system, while providing the same (large) speedup.\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure11.png}\n\t\\caption{2D concentration profiles obtained for cement system 2 after 2 and 6 (final time step) years. RTM means the original HPx$_{\\rm{4C}}$ model and TM+DNN denotes the Hydrus transport model coupled with our DNN geochemical emulator (HPx$_{\\rm py}$-DNN). The first to fourth row present profiles for $Ca^{\\rm{conc}}$, $Si^{\\rm{conc}}$, $H^{\\rm{conc}}$, and $O^{\\rm{conc}}$, respectively. The considered grid size is 61 $\\times$ 61.}\n\t\\label{fig11}\n\\end{figure}\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure12.png}\n\t\\caption{2D concentration profiles obtained for cement system 2 after 2 and 6 (final time step) years. RTM means the original HPx$_{\\rm{4C}}$ model and TM+DNN denotes the Hydrus transport model coupled with our DNN geochemical emulator (HPx$_{\\rm py}$-DNN). The first to third row present profiles for $C^{\\rm{conc}}$, $Al^{\\rm{conc}}$, and $S^{\\rm{conc}}$, respectively. The considered grid size is 61 $\\times$ 61.}\n\t\\label{fig12}\n\\end{figure}\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure13.png}\n\t\\caption{2D concentration profiles obtained for cement system 2 after 2 and 6 (final time step) years. RTM means the original HPx$_{\\rm{4C}}$ model and TM+DNN denotes the Hydrus transport model coupled with our DNN geochemical emulator (HPx$_{\\rm py}$-DNN). The first to fourth row present profiles for $Ca^{\\rm{conc}}$, $Si^{\\rm{conc}}$, $H^{\\rm{conc}}$, and $O^{\\rm{conc}}$, respectively. The considered grid size is 121 $\\times$ 121.}\n\t\\label{fig13}\n\\end{figure}\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure14.png}\n\t\\caption{2D concentration profiles obtained for cement system 2 after 2 and 6 (final time step) years. RTM means the original HPx$_{\\rm{4C}}$ model and TM+DNN denotes the Hydrus transport model coupled with our DNN geochemical emulator (HPx$_{\\rm py}$-DNN). The first to third row present profiles for $C^{\\rm{conc}}$, $Al^{\\rm{conc}}$, and $S^{\\rm{conc}}$, respectively. The considered grid size is 121 $\\times$ 121.}\n\t\\label{fig14}\n\\end{figure}\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure15.png}\n\t\\caption{2D concentration profiles obtained for cement system 2 after 2 and 6 (final time step) years. RTM means the original HPx$_{\\rm{4C}}$ model and TM+kNN denotes the Hydrus transport model coupled with our kNN geochemical emulator (HPx$_{\\rm py}$-kNN).  The first to third row present profiles for $C^{\\rm{conc}}$, $Al^{\\rm{conc}}$, and $S^{\\rm{conc}}$, respectively. The considered grid size is 61 $\\times$ 61.}\n\t\\label{fig15}\n\\end{figure}\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure16.png}\n\t\\caption{2D concentration profiles obtained for cement system 2 after 2 and 6 (final time step) years. RTM means the original HPx$_{\\rm{4C}}$ model and TM+kNN denotes the Hydrus transport model coupled with our kNN geochemical emulator (HPx$_{\\rm py}$-kNN). The first to third row present profiles for $C^{\\rm{conc}}$, $Al^{\\rm{conc}}$, and $S^{\\rm{conc}}$, respectively. The considered grid size is 121 $\\times$ 121.}\n\t\\label{fig16}\n\\end{figure}\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure17.png}\n\t\\caption{2D solid amount profiles of the C, Al and S chemical components of cement system 2 after 2 and 6 (final time step) years. RTM means the original HPx$_{\\rm{4C}}$ model and TM+DNN denotes the Hydrus transport model coupled with our DNN geochemical emulator (HPx$_{\\rm py}$-DNN). The first to third row present profiles for $C^{\\rm{solid}}$, $Al^{\\rm{solid}}$, and $S^{\\rm{solid}}$, respectively The considered grid size is 61 $\\times$ 61.}\n\t\\label{fig17}\n\\end{figure}\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure18.png}\n\t\\caption{2D solid amount profiles of the C, Al and S chemical components of cement system 2 after 2 and 6 (final time step) years. RTM means the original HPx$_{\\rm{4C}}$ model and TM+DNN denotes the Hydrus transport model coupled with our DNN geochemical emulator (HPx$_{\\rm py}$-DNN). The first to third row present profiles for $C^{\\rm{solid}}$, $Al^{\\rm{solid}}$, and $S^{\\rm{solid}}$, respectively The considered grid size is 121 $\\times$ 121.}\n\t\\label{fig18}\n\\end{figure}\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure19.png}\n\t\\caption{2D solid amount profiles of the C, Al and S chemical components of cement system 2 after 2 and 6 (final time step) years. RTM means the original HPx$_{\\rm{4C}}$ model and TM+kNN denotes the Hydrus transport model coupled with our kNN geochemical emulator (HPx$_{\\rm py}$-kNN). The first to third row present profiles for $C^{\\rm{solid}}$, $Al^{\\rm{solid}}$, and $S^{\\rm{solid}}$, respectively The considered grid size is 61 $\\times$ 61.}\n\t\\label{fig19}\n\\end{figure}\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure20.png}\n\t\\caption{2D solid amount profiles of the C, Al and S chemical components of cement system 2 after 2 and 6 (final time step) years. RTM means the original HPx$_{\\rm{4C}}$ model and TM+kNN denotes the Hydrus transport model coupled with our kNN geochemical emulator (HPx$_{\\rm py}$-kNN). The first to third row present profiles for $C^{\\rm{solid}}$, $Al^{\\rm{solid}}$, and $S^{\\rm{solid}}$, respectively The considered grid size is 121 $\\times$ 121.}\n\t\\label{fig20}\n\\end{figure}\n\n\\begin{table}[h!]\n\t\\caption{Speedups offered by the KNN and DNN emulators in HPx$_{\\rm py}$ for the reactive transport simulations considered for cement system 2. The HPx$_{\\rm{4C}}$ calculations involve the parallelization of PHREEQC over our 4 CPUs. The HPx$_{\\rm{1C}}$ calculations are performed on a single CPU. Both the kNN and DNN predictions make use of a GPU. ML signifies the used machine learning method for emulation, BC denotes the type of flow boundary conditions and GS is the grid size. The maximum possible speedups associated with HPx$_{\\rm{4C}}$ and HP$_{\\rm{1C}}$,  Max SP HPx$_{\\rm{4C}}$ and  Max SP HPx$_{\\rm{1C}}$, correspond to an hypothetical situation where the geochemical calculations incur zero computational cost.}\n\n\t\\begin{center}\n\t\t\\begin{tabular}{cccccccc}\n\t\t\t\\hline\n\t\t\t& & & & & & & \\\\\n\t\t\tML & BC & GS & HPx$_{\\rm{4C}}$ time (s) & SP HPx$_{\\rm{4C}}$ & Max SP HPx$_{\\rm{4C}}$ & SP HPx$_{\\rm{1C}}$ & Max SP HPx$_{\\rm{1C}}$ \\\\\n\t\t\tDNN & ADV & 61 $\\times$ 61 & 21,415 & 8.2 & 9.0 & 30.3 & 33.1\\\\\n\t\t\tkNN & ADV & 61 $\\times$ 61 & 21,415 & 7.9 & 9.0 & 28.9 & 33.1\\\\\n\t\t\tDNN & ADV & 121 $\\times$ 121 & 199,841 & 8.2 & 9.5 & 29.9 & 32.8\\\\\n\t\t\tkNN & ADV & 121 $\\times$ 121 & 199,841 & 8.5 & 9.5 & 31.3 & 32.8\\\\\n\t\t\t\\hline\n\t\t\\end{tabular}\n\t\\end{center}\n\n\t\\label{table4}\n\\end{table}\n\n\\FloatBarrier\n\n3.2 Al-C-Ca-S-Si Problem\n\\subsection{Al-C-Ca-S-Si Problem}\n\nFor this second cement system, the emulation problem consists of predicting at each time step of the RT simulation the (output) Al, C, Ca, S, Si, H and O aqueous concentrations (mol/kgw) from the (input) total amounts of Al, C, Ca, S and Si (mol). Here we focus on advective-dispersive transport only while similarly as for cement system 1, the considered domain sizes are 61 $\\times$ 61 and 121 $\\times$ 121. Moreover, the simulation time period is set to 6 years. \n\nAs mentioned earlier, for this higher-dimensional problem it is observed that the used scikit-learn kNN implementation becomes prohibitively slow compared to HPx$_{\\rm{4C}}$. We found that to get a good emulation accuracy the kNN training base needs to contain 1,000,000 samples (or more). This training base's size together with a 5-dimensional search space leads to an HPx$_{\\rm py}$-kNN reactive transport simulation time that is comparable to that of HPx$_{\\rm{4C}}$. Therefore, for this second cement system we built a custom kNN regressor around another kNN implementation contained in the FAISS package \\citep{faiss2017}. Our used FAISS variant allows for GPU computing and is much faster than scikit-learn for this cement system, but is slightly less accurate due to the use of an approximate rather than exact nearest neighbor search \\citep[see][for details]{faiss2017}.\n\n\\subsubsection{Training the DNN}\n\\label{train_res2}\n\nBuilding a good training set to perform a kNN search and learn the weights and biases of our DNN turned out to be a complicated task in this case. This because to make useful kNN predictions and/or learn an useful DNN, the training set must be sufficiently representative of the geochemical conditions encountered during the reactive transport simulation one wish to perform with HPx$_{\\rm py}$-kNN and the trained HPx$_{\\rm py}$-DNN. In contrast to cement system 1, creating the training set by sampling the $X$-space with a controlled randomness between predefined lower and upper bounds did not prove successful. We tried that strategy by drawing as much as 4,000,000 5-dimensional $\\textbf{x}$ vectors from the $X$-space using a Sobol low-discrepancy sequence \\citep[][]{Sobol1967, Joe-Kuo2003}. Such low-discrepancy sampling scheme covers the 5-dimensional hypercube more uniformly than LHS. Despite a good performance on the test set (not shown), the resulting DNN accuracy in reactive transport mode was never deemed satisfying. In other words, no satisfying ``global\" or ``universal\" DNN emulator could be devised for this cement system. This is probably caused by the fact for this problem, the input (5 total amounts) and output (7 aqueous concentrations) spaces are quite nonlinearly related and both cover 6 to 10 orders of magnitudes depending on the considered element. Therefore, we resorted to the alternative training strategy detailed below. The latter basically tries to grasp the complex correlations and higher-order dependencies that exist between the elements of $\\textbf{x}$ (total amounts, input space) for a given reactive transport simulation setup, in order to produce a training set that honors these between-input relationships.\n\n\\begin{itemize}\n\t\\item Perform a ``cheap\" full reactive transport simulation under the transport conditions and geochemistry of interest and collect the resulting $\\textbf{x}$ -$\\textbf{y}$  pairs of examples (for the considered grid nodes and time steps). Computational demand controls what domain size and simulation time period can be used for this cheap calculation. We used a modest 16 $\\times$ 16 domain and a simulation time period of 10 years. The associated HPx$_{\\rm{4C}}$ runtimes is 180 s.\n\t\n\t\\item Fit a kernel density estimator (KDE) with a Gaussian kernel to the collected $\\textbf{x}$ vectors (encapsulated in the $\\textbf{X}$ array) and generate a fixed number of new input vectors, $\\textbf{x}_{KDE}$. Then run PHREEQC for the $\\textbf{X}_{KDE}$ set to get the corresponding output set, $\\textbf{Y}_{KDE}$. Now apply the correction for porosity described in section \\ref{train_res1} to the $\\textbf{X}_{KDE}$ set and form the training set by merging the ensemble of $\\textbf{x}$-$\\textbf{y}$ pairs with that of the $\\textbf{x}_{KDE}$-$\\textbf{y}_{KDE}$ pairs. The number of produced unique examples by the considered cheap HPx$_{\\rm{4C}}$ simulations varied between 10,000 and 50,000. The KDE-based enrichment of this dataset was deemed necessary to provide more input variability, thereby avoiding overfitting of the trained DNN and improving the kNN accuracy, while still honoring the complex between-input relationships. The number of KDE-generated samples was set as to obtain a total training set size of 1,000,000 examples. A key component of the approach is the bandwith parameter of the Gaussian KDE kernel which controls how much the KDE-generated samples depart from the original ensemble. After limited trial and error, we fixed the kernel bandwith to 0.0025 for the considered case studies.\n\t\n\\end{itemize}\n\nThe scatter plots in Figure \\ref{fig9} illustrate our training set creation procedure. The orange dots depict the pairwise relationships observed between the 5 elements of $\\textbf{x}$ in the cheap simulation. The red and cyan dots in Figure \\ref{fig9} represent the KDE-generated samples with the selected bandwidth, before and after applying the correction for porosity, respectively. Training of the DNN is achieved using the ensemble of original and KDE-corrected input points. We refer to this kind of dataset as RT-based, since it is based on an albeit cheap, full RT simulation. Furthermore, we refer to the obtained DNN and kNN emulators as ``local\" emulators, since as opposed to the emulators constructed for cement system 1, the current emulators are only valid for the input conditions encapsulated in the RT-based training set.\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1.5cm}\\includegraphics[width=50pc]{Figure9.png}\n\t\\caption{Scatter plots of the complex relationships between the five considered inputs for (1) the computationally cheap RT simulation of cement system 2 (orange dots) performed with the original HPx code, (2) the corresponding sampled points by kernel density estimation (KDE) using the selected bandwith (turquoise dots) and, (3) the same sampled KDE points after porosity correction (red dots).}\n\t\\label{fig9}\n\\end{figure}\n\nTraining performance of the DNN emulator is presented in Figure \\ref{fig10}. Similarly as for cement system 1, about 90\\% of the available data was used for the actual training of the DNN while the remaining 10\\% were used a validation set to contol overfitting. Lastly, performance is evaluated for both the trained DNN and kNN emulators using an independent test set of 10,000 examples. Overall, the accuracy of our ``local \" DNN emulator for this RT-inspired dataset is rather large with $Q_2$ values always greater or equal than 0.998. Training performance of the corresponding local kNN emulator is equally good (not shown). Regarding speedup, for this problem single-threaded PHREEQC achieves about 210 geochemical calculations per second on our used Intel\\textsuperscript{\\textregistered} i7 CPU while the GPU-based DNN and kNN emulators are both about 3000 times faster when predicting the 10,000 test points at once .\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure10.png}\n\t\\caption{1-1 plots of local DNN emulation performance obtained for system 2 when the local DNN is trained using 1,000,000 samples. The $x$-axis and $y$-axis present the original and emulated 10,000 independent test data points, respectively. The RMSE and $Q_2$ coefficient denote the root-mean-square-error and coefficient of determination in testing mode, respectively, between the original and emulated 10,000 test data points.}\n\t\\label{fig10}\n\\end{figure}\n\n\\subsubsection{Reactive transport simulation}\n\\label{prob2res}\nOur ``local\" DNN performs rather well when applied to the 61 $\\times$ 61 grid size and a time period of 6 years (Figures \\ref{fig11} - \\ref{fig12}). This for all components but C, for which some localized deviations appear between original and emulated 2D concentration profiles towards the end of the simulation period (Figure \\ref{fig12}). When applied to the 121 $\\times$ 121 grid, HPx$_{\\rm py}$-DNN produces additional discrepancies for O, Si and Al towards the end of the simulation period (Figures \\ref{fig13} - \\ref{fig14}). Yet most of the observed artifacts could probably be smoothed out by using post-filtering. The associated speedups are listed in Table \\ref{table4}. These speedups are larger to those obtained for cement system 1, with values between 8 and 9 when evaluated against HPx$_{\\rm{4C}}$. These speedups represent about 85 \\% to 90 \\% of the maximum possible speedup (Table \\ref{table3}). Overall, these findings indicate that for the considered problem, our RT-based training of a local DNN only works is the training set is sufficiently representative of the particular geochemical conditions encountered in the computationally demanding simulations, which is arguably not easy to achieve. This limitation is further discussed in section \\ref{discussion}.\n\nWe note a more uniform behavior for HPx$_{\\rm py}$-kNN across grid sizes than for HPx$_{\\rm py}$-DNN. Here the results for the 121  $\\times$ 121 grid are only slightly less accurate than those associated with the 61 $ \\times$ 61 grid (see Figures \\ref{fig15} - \\ref{fig16} where for brevity we only show concentration profiles for the C Al and S components). Furthermore, whenever observed the discrepancies between original and emulated profiles are more regularly scattered than for HPx$_{\\rm py}$-DNN. Note also that herein too, post-filtering could likely smooth out a large part of these deviations. In addition, owing to the use of a GPU to achieve the kNN calculations, the speedup provided by HPx$_{\\rm py}$-kNN are as large as those provided by HPx$_{\\rm py}$-DNN (Table \\ref{table4}).\n\nWith respect to the emulated solid amounts, the HPx$_{\\rm py}$-DNN results look visually good for the the 61 $\\times$ 61 grid. This is shown in Figure \\ref{fig17} for the C, Al and S chemical components, while emulation of the H, O Ca and Si chemical components is globally of similar quality (not shown). Nevertheless, significant deviations appear towards the end of the simulation period for every chemical component (see Figure \\ref{fig18} for the C, Al and S chemical components, emulation of the H, O Ca and Si chemical components shows the same level of mismatch). The HPx$_{\\rm py}$-kNN predictions are also fairly accurate for the 61 $\\times$ 61 (see Figure \\ref{fig19} for the C, Al and S chemical components, emulation of the H, O Ca and Si chemical components exhibits a globally similar quality) while some discrepancies show up at the end of the simulation (Figure \\ref{fig20}). However, the mismatch is less pronounced than for HPx$_{\\rm py}$-DNN. Overall, HPx$_{\\rm py}$-kNN appears to be somewhat more robust than HPx$_{\\rm py}$-DNN for this cement system, while providing the same (large) speedup.\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure11.png}\n\t\\caption{2D concentration profiles obtained for cement system 2 after 2 and 6 (final time step) years. RTM means the original HPx$_{\\rm{4C}}$ model and TM+DNN denotes the Hydrus transport model coupled with our DNN geochemical emulator (HPx$_{\\rm py}$-DNN). The first to fourth row present profiles for $Ca^{\\rm{conc}}$, $Si^{\\rm{conc}}$, $H^{\\rm{conc}}$, and $O^{\\rm{conc}}$, respectively. The considered grid size is 61 $\\times$ 61.}\n\t\\label{fig11}\n\\end{figure}\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure12.png}\n\t\\caption{2D concentration profiles obtained for cement system 2 after 2 and 6 (final time step) years. RTM means the original HPx$_{\\rm{4C}}$ model and TM+DNN denotes the Hydrus transport model coupled with our DNN geochemical emulator (HPx$_{\\rm py}$-DNN). The first to third row present profiles for $C^{\\rm{conc}}$, $Al^{\\rm{conc}}$, and $S^{\\rm{conc}}$, respectively. The considered grid size is 61 $\\times$ 61.}\n\t\\label{fig12}\n\\end{figure}\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure13.png}\n\t\\caption{2D concentration profiles obtained for cement system 2 after 2 and 6 (final time step) years. RTM means the original HPx$_{\\rm{4C}}$ model and TM+DNN denotes the Hydrus transport model coupled with our DNN geochemical emulator (HPx$_{\\rm py}$-DNN). The first to fourth row present profiles for $Ca^{\\rm{conc}}$, $Si^{\\rm{conc}}$, $H^{\\rm{conc}}$, and $O^{\\rm{conc}}$, respectively. The considered grid size is 121 $\\times$ 121.}\n\t\\label{fig13}\n\\end{figure}\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure14.png}\n\t\\caption{2D concentration profiles obtained for cement system 2 after 2 and 6 (final time step) years. RTM means the original HPx$_{\\rm{4C}}$ model and TM+DNN denotes the Hydrus transport model coupled with our DNN geochemical emulator (HPx$_{\\rm py}$-DNN). The first to third row present profiles for $C^{\\rm{conc}}$, $Al^{\\rm{conc}}$, and $S^{\\rm{conc}}$, respectively. The considered grid size is 121 $\\times$ 121.}\n\t\\label{fig14}\n\\end{figure}\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure15.png}\n\t\\caption{2D concentration profiles obtained for cement system 2 after 2 and 6 (final time step) years. RTM means the original HPx$_{\\rm{4C}}$ model and TM+kNN denotes the Hydrus transport model coupled with our kNN geochemical emulator (HPx$_{\\rm py}$-kNN).  The first to third row present profiles for $C^{\\rm{conc}}$, $Al^{\\rm{conc}}$, and $S^{\\rm{conc}}$, respectively. The considered grid size is 61 $\\times$ 61.}\n\t\\label{fig15}\n\\end{figure}\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure16.png}\n\t\\caption{2D concentration profiles obtained for cement system 2 after 2 and 6 (final time step) years. RTM means the original HPx$_{\\rm{4C}}$ model and TM+kNN denotes the Hydrus transport model coupled with our kNN geochemical emulator (HPx$_{\\rm py}$-kNN). The first to third row present profiles for $C^{\\rm{conc}}$, $Al^{\\rm{conc}}$, and $S^{\\rm{conc}}$, respectively. The considered grid size is 121 $\\times$ 121.}\n\t\\label{fig16}\n\\end{figure}\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure17.png}\n\t\\caption{2D solid amount profiles of the C, Al and S chemical components of cement system 2 after 2 and 6 (final time step) years. RTM means the original HPx$_{\\rm{4C}}$ model and TM+DNN denotes the Hydrus transport model coupled with our DNN geochemical emulator (HPx$_{\\rm py}$-DNN). The first to third row present profiles for $C^{\\rm{solid}}$, $Al^{\\rm{solid}}$, and $S^{\\rm{solid}}$, respectively The considered grid size is 61 $\\times$ 61.}\n\t\\label{fig17}\n\\end{figure}\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure18.png}\n\t\\caption{2D solid amount profiles of the C, Al and S chemical components of cement system 2 after 2 and 6 (final time step) years. RTM means the original HPx$_{\\rm{4C}}$ model and TM+DNN denotes the Hydrus transport model coupled with our DNN geochemical emulator (HPx$_{\\rm py}$-DNN). The first to third row present profiles for $C^{\\rm{solid}}$, $Al^{\\rm{solid}}$, and $S^{\\rm{solid}}$, respectively The considered grid size is 121 $\\times$ 121.}\n\t\\label{fig18}\n\\end{figure}\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure19.png}\n\t\\caption{2D solid amount profiles of the C, Al and S chemical components of cement system 2 after 2 and 6 (final time step) years. RTM means the original HPx$_{\\rm{4C}}$ model and TM+kNN denotes the Hydrus transport model coupled with our kNN geochemical emulator (HPx$_{\\rm py}$-kNN). The first to third row present profiles for $C^{\\rm{solid}}$, $Al^{\\rm{solid}}$, and $S^{\\rm{solid}}$, respectively The considered grid size is 61 $\\times$ 61.}\n\t\\label{fig19}\n\\end{figure}\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure20.png}\n\t\\caption{2D solid amount profiles of the C, Al and S chemical components of cement system 2 after 2 and 6 (final time step) years. RTM means the original HPx$_{\\rm{4C}}$ model and TM+kNN denotes the Hydrus transport model coupled with our kNN geochemical emulator (HPx$_{\\rm py}$-kNN). The first to third row present profiles for $C^{\\rm{solid}}$, $Al^{\\rm{solid}}$, and $S^{\\rm{solid}}$, respectively The considered grid size is 121 $\\times$ 121.}\n\t\\label{fig20}\n\\end{figure}\n\n\\begin{table}[h!]\n\t\\caption{Speedups offered by the KNN and DNN emulators in HPx$_{\\rm py}$ for the reactive transport simulations considered for cement system 2. The HPx$_{\\rm{4C}}$ calculations involve the parallelization of PHREEQC over our 4 CPUs. The HPx$_{\\rm{1C}}$ calculations are performed on a single CPU. Both the kNN and DNN predictions make use of a GPU. ML signifies the used machine learning method for emulation, BC denotes the type of flow boundary conditions and GS is the grid size. The maximum possible speedups associated with HPx$_{\\rm{4C}}$ and HP$_{\\rm{1C}}$,  Max SP HPx$_{\\rm{4C}}$ and  Max SP HPx$_{\\rm{1C}}$, correspond to an hypothetical situation where the geochemical calculations incur zero computational cost.}\n\n\t\\begin{center}\n\t\t\\begin{tabular}{cccccccc}\n\t\t\t\\hline\n\t\t\t& & & & & & & \\\\\n\t\t\tML & BC & GS & HPx$_{\\rm{4C}}$ time (s) & SP HPx$_{\\rm{4C}}$ & Max SP HPx$_{\\rm{4C}}$ & SP HPx$_{\\rm{1C}}$ & Max SP HPx$_{\\rm{1C}}$ \\\\\n\t\t\tDNN & ADV & 61 $\\times$ 61 & 21,415 & 8.2 & 9.0 & 30.3 & 33.1\\\\\n\t\t\tkNN & ADV & 61 $\\times$ 61 & 21,415 & 7.9 & 9.0 & 28.9 & 33.1\\\\\n\t\t\tDNN & ADV & 121 $\\times$ 121 & 199,841 & 8.2 & 9.5 & 29.9 & 32.8\\\\\n\t\t\tkNN & ADV & 121 $\\times$ 121 & 199,841 & 8.5 & 9.5 & 31.3 & 32.8\\\\\n\t\t\t\\hline\n\t\t\\end{tabular}\n\t\\end{center}\n\n\t\\label{table4}\n\\end{table}\n\n\\FloatBarrier\n\n3.2.2 Reactive transport simulation\n\\subsubsection{Reactive transport simulation}\n\\label{prob2res}\nOur ``local\" DNN performs rather well when applied to the 61 $\\times$ 61 grid size and a time period of 6 years (Figures \\ref{fig11} - \\ref{fig12}). This for all components but C, for which some localized deviations appear between original and emulated 2D concentration profiles towards the end of the simulation period (Figure \\ref{fig12}). When applied to the 121 $\\times$ 121 grid, HPx$_{\\rm py}$-DNN produces additional discrepancies for O, Si and Al towards the end of the simulation period (Figures \\ref{fig13} - \\ref{fig14}). Yet most of the observed artifacts could probably be smoothed out by using post-filtering. The associated speedups are listed in Table \\ref{table4}. These speedups are larger to those obtained for cement system 1, with values between 8 and 9 when evaluated against HPx$_{\\rm{4C}}$. These speedups represent about 85 \\% to 90 \\% of the maximum possible speedup (Table \\ref{table3}). Overall, these findings indicate that for the considered problem, our RT-based training of a local DNN only works is the training set is sufficiently representative of the particular geochemical conditions encountered in the computationally demanding simulations, which is arguably not easy to achieve. This limitation is further discussed in section \\ref{discussion}.\n\nWe note a more uniform behavior for HPx$_{\\rm py}$-kNN across grid sizes than for HPx$_{\\rm py}$-DNN. Here the results for the 121  $\\times$ 121 grid are only slightly less accurate than those associated with the 61 $ \\times$ 61 grid (see Figures \\ref{fig15} - \\ref{fig16} where for brevity we only show concentration profiles for the C Al and S components). Furthermore, whenever observed the discrepancies between original and emulated profiles are more regularly scattered than for HPx$_{\\rm py}$-DNN. Note also that herein too, post-filtering could likely smooth out a large part of these deviations. In addition, owing to the use of a GPU to achieve the kNN calculations, the speedup provided by HPx$_{\\rm py}$-kNN are as large as those provided by HPx$_{\\rm py}$-DNN (Table \\ref{table4}).\n\nWith respect to the emulated solid amounts, the HPx$_{\\rm py}$-DNN results look visually good for the the 61 $\\times$ 61 grid. This is shown in Figure \\ref{fig17} for the C, Al and S chemical components, while emulation of the H, O Ca and Si chemical components is globally of similar quality (not shown). Nevertheless, significant deviations appear towards the end of the simulation period for every chemical component (see Figure \\ref{fig18} for the C, Al and S chemical components, emulation of the H, O Ca and Si chemical components shows the same level of mismatch). The HPx$_{\\rm py}$-kNN predictions are also fairly accurate for the 61 $\\times$ 61 (see Figure \\ref{fig19} for the C, Al and S chemical components, emulation of the H, O Ca and Si chemical components exhibits a globally similar quality) while some discrepancies show up at the end of the simulation (Figure \\ref{fig20}). However, the mismatch is less pronounced than for HPx$_{\\rm py}$-DNN. Overall, HPx$_{\\rm py}$-kNN appears to be somewhat more robust than HPx$_{\\rm py}$-DNN for this cement system, while providing the same (large) speedup.\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure11.png}\n\t\\caption{2D concentration profiles obtained for cement system 2 after 2 and 6 (final time step) years. RTM means the original HPx$_{\\rm{4C}}$ model and TM+DNN denotes the Hydrus transport model coupled with our DNN geochemical emulator (HPx$_{\\rm py}$-DNN). The first to fourth row present profiles for $Ca^{\\rm{conc}}$, $Si^{\\rm{conc}}$, $H^{\\rm{conc}}$, and $O^{\\rm{conc}}$, respectively. The considered grid size is 61 $\\times$ 61.}\n\t\\label{fig11}\n\\end{figure}\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure12.png}\n\t\\caption{2D concentration profiles obtained for cement system 2 after 2 and 6 (final time step) years. RTM means the original HPx$_{\\rm{4C}}$ model and TM+DNN denotes the Hydrus transport model coupled with our DNN geochemical emulator (HPx$_{\\rm py}$-DNN). The first to third row present profiles for $C^{\\rm{conc}}$, $Al^{\\rm{conc}}$, and $S^{\\rm{conc}}$, respectively. The considered grid size is 61 $\\times$ 61.}\n\t\\label{fig12}\n\\end{figure}\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure13.png}\n\t\\caption{2D concentration profiles obtained for cement system 2 after 2 and 6 (final time step) years. RTM means the original HPx$_{\\rm{4C}}$ model and TM+DNN denotes the Hydrus transport model coupled with our DNN geochemical emulator (HPx$_{\\rm py}$-DNN). The first to fourth row present profiles for $Ca^{\\rm{conc}}$, $Si^{\\rm{conc}}$, $H^{\\rm{conc}}$, and $O^{\\rm{conc}}$, respectively. The considered grid size is 121 $\\times$ 121.}\n\t\\label{fig13}\n\\end{figure}\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure14.png}\n\t\\caption{2D concentration profiles obtained for cement system 2 after 2 and 6 (final time step) years. RTM means the original HPx$_{\\rm{4C}}$ model and TM+DNN denotes the Hydrus transport model coupled with our DNN geochemical emulator (HPx$_{\\rm py}$-DNN). The first to third row present profiles for $C^{\\rm{conc}}$, $Al^{\\rm{conc}}$, and $S^{\\rm{conc}}$, respectively. The considered grid size is 121 $\\times$ 121.}\n\t\\label{fig14}\n\\end{figure}\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure15.png}\n\t\\caption{2D concentration profiles obtained for cement system 2 after 2 and 6 (final time step) years. RTM means the original HPx$_{\\rm{4C}}$ model and TM+kNN denotes the Hydrus transport model coupled with our kNN geochemical emulator (HPx$_{\\rm py}$-kNN).  The first to third row present profiles for $C^{\\rm{conc}}$, $Al^{\\rm{conc}}$, and $S^{\\rm{conc}}$, respectively. The considered grid size is 61 $\\times$ 61.}\n\t\\label{fig15}\n\\end{figure}\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure16.png}\n\t\\caption{2D concentration profiles obtained for cement system 2 after 2 and 6 (final time step) years. RTM means the original HPx$_{\\rm{4C}}$ model and TM+kNN denotes the Hydrus transport model coupled with our kNN geochemical emulator (HPx$_{\\rm py}$-kNN). The first to third row present profiles for $C^{\\rm{conc}}$, $Al^{\\rm{conc}}$, and $S^{\\rm{conc}}$, respectively. The considered grid size is 121 $\\times$ 121.}\n\t\\label{fig16}\n\\end{figure}\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure17.png}\n\t\\caption{2D solid amount profiles of the C, Al and S chemical components of cement system 2 after 2 and 6 (final time step) years. RTM means the original HPx$_{\\rm{4C}}$ model and TM+DNN denotes the Hydrus transport model coupled with our DNN geochemical emulator (HPx$_{\\rm py}$-DNN). The first to third row present profiles for $C^{\\rm{solid}}$, $Al^{\\rm{solid}}$, and $S^{\\rm{solid}}$, respectively The considered grid size is 61 $\\times$ 61.}\n\t\\label{fig17}\n\\end{figure}\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure18.png}\n\t\\caption{2D solid amount profiles of the C, Al and S chemical components of cement system 2 after 2 and 6 (final time step) years. RTM means the original HPx$_{\\rm{4C}}$ model and TM+DNN denotes the Hydrus transport model coupled with our DNN geochemical emulator (HPx$_{\\rm py}$-DNN). The first to third row present profiles for $C^{\\rm{solid}}$, $Al^{\\rm{solid}}$, and $S^{\\rm{solid}}$, respectively The considered grid size is 121 $\\times$ 121.}\n\t\\label{fig18}\n\\end{figure}\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure19.png}\n\t\\caption{2D solid amount profiles of the C, Al and S chemical components of cement system 2 after 2 and 6 (final time step) years. RTM means the original HPx$_{\\rm{4C}}$ model and TM+kNN denotes the Hydrus transport model coupled with our kNN geochemical emulator (HPx$_{\\rm py}$-kNN). The first to third row present profiles for $C^{\\rm{solid}}$, $Al^{\\rm{solid}}$, and $S^{\\rm{solid}}$, respectively The considered grid size is 61 $\\times$ 61.}\n\t\\label{fig19}\n\\end{figure}\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure20.png}\n\t\\caption{2D solid amount profiles of the C, Al and S chemical components of cement system 2 after 2 and 6 (final time step) years. RTM means the original HPx$_{\\rm{4C}}$ model and TM+kNN denotes the Hydrus transport model coupled with our kNN geochemical emulator (HPx$_{\\rm py}$-kNN). The first to third row present profiles for $C^{\\rm{solid}}$, $Al^{\\rm{solid}}$, and $S^{\\rm{solid}}$, respectively The considered grid size is 121 $\\times$ 121.}\n\t\\label{fig20}\n\\end{figure}\n\n\\begin{table}[h!]\n\t\\caption{Speedups offered by the KNN and DNN emulators in HPx$_{\\rm py}$ for the reactive transport simulations considered for cement system 2. The HPx$_{\\rm{4C}}$ calculations involve the parallelization of PHREEQC over our 4 CPUs. The HPx$_{\\rm{1C}}$ calculations are performed on a single CPU. Both the kNN and DNN predictions make use of a GPU. ML signifies the used machine learning method for emulation, BC denotes the type of flow boundary conditions and GS is the grid size. The maximum possible speedups associated with HPx$_{\\rm{4C}}$ and HP$_{\\rm{1C}}$,  Max SP HPx$_{\\rm{4C}}$ and  Max SP HPx$_{\\rm{1C}}$, correspond to an hypothetical situation where the geochemical calculations incur zero computational cost.}\n\n\t\\begin{center}\n\t\t\\begin{tabular}{cccccccc}\n\t\t\t\\hline\n\t\t\t& & & & & & & \\\\\n\t\t\tML & BC & GS & HPx$_{\\rm{4C}}$ time (s) & SP HPx$_{\\rm{4C}}$ & Max SP HPx$_{\\rm{4C}}$ & SP HPx$_{\\rm{1C}}$ & Max SP HPx$_{\\rm{1C}}$ \\\\\n\t\t\tDNN & ADV & 61 $\\times$ 61 & 21,415 & 8.2 & 9.0 & 30.3 & 33.1\\\\\n\t\t\tkNN & ADV & 61 $\\times$ 61 & 21,415 & 7.9 & 9.0 & 28.9 & 33.1\\\\\n\t\t\tDNN & ADV & 121 $\\times$ 121 & 199,841 & 8.2 & 9.5 & 29.9 & 32.8\\\\\n\t\t\tkNN & ADV & 121 $\\times$ 121 & 199,841 & 8.5 & 9.5 & 31.3 & 32.8\\\\\n\t\t\t\\hline\n\t\t\\end{tabular}\n\t\\end{center}\n\n\t\\label{table4}\n\\end{table}\n\n\\FloatBarrier\n\n",
        "label": "fig16",
        "Descriptive_question1": "What is the grid size used in the simulation shown in figure_16?",
        "Descriptive_question2": "Which chemical component is represented in the first row of figure_16?",
        "Reasoning_question1": "How do the concentration profiles of C, Al, and S in figure_16 change between 2 and 6 years, and what might this indicate about the chemical dynamics in cement system 2?",
        "Reasoning_question2": "Compare the performance of the RTM and TM+kNN models for the S concentration profile in figure_16 after 6 years. What can be inferred about the accuracy of the kNN geochemical emulator in this context?",
        "Descriptive_answer1": "121 × 121",
        "Descriptive_answer2": "C",
        "Reasoning_answer1": "Between 2 and 6 years, the concentration profiles of C, Al, and S in figure_16 exhibit noticeable changes, such as spreading and variations in concentration intensity. This indicates ongoing geochemical reactions and transport processes affecting these elements, reflecting the dynamic nature of the system over time as cement system 2 evolves chemically and spatially.",
        "Reasoning_answer2": "Comparing RTM and TM+kNN for the S concentration profile after 6 years in figure_16 shows that the TM+kNN (kNN emulator) closely matches the RTM results with only minor discrepancies. This suggests that the kNN geochemical emulator provides an accurate approximation of the original model for S concentrations, effectively capturing the spatial distribution and temporal evolution with good fidelity in the simulation context."
    },
    {
        "paper_id": "2107.07598.json",
        "image_id": "figure_17",
        "image_path": [
            "/home/yz979/scratch/chengye/scimolmo/test-images-tables/dataset_imagestables/2107.07598/images/Figure17.png"
        ],
        "caption": "2D solid amount profiles of the C, Al and S chemical components of cement system 2 after 2 and 6 (final time step) years. RTM means the original HPx$_{\\rm{4C}}$ model and TM+DNN denotes the Hydrus transport model coupled with our DNN geochemical emulator (HPx$_{\\rm py}$-DNN). The first to third row present profiles for $C^{\\rm{solid}}$, $Al^{\\rm{solid}}$, and $S^{\\rm{solid}}$, respectively The considered grid size is 61 $\\times$ 61.",
        "classify": "Chart",
        "section_info": "3 Results\n\\section{Results}\n\\label{results}\n\n\\subsection{Ca-Si Problem}\nFor this first cement system, the emulation problem consists of predicting at each time step of the RT simulation the (output) Ca, Si, H and O aqueous concentrations (mol/kg of water or mol/kgw) from the (input) total amounts of Ca and Si (mol). \n\n\\subsubsection{Training the emulators}\n\\label{train_res1}\nHere the kNN and DNN emulators are firstly trained using a set of 400,000 test examples for both. This training set is obtained by randomly sampling the two-dimensional input space by latin hypercube sampling (LHS) between $\\left[0,0\\right]$ and $\\rm{\\left[Ca^{tot}_{max},Si^{tot}_{max}\\right]}$, and running PHREEQC for each input sample, $\\rm{\\textbf{x}_i = \\left[Ca^{tot}_i, Si^{tot}_i\\right]}$ to get the corresponding output vectors, $\\rm{\\textbf{y}_i = \\left[Ca^{conc}_i,Si^{conc}_i,H^{conc}_i,O^{conc}_i\\right]}$. The upper bounds, $\\rm{Ca^{tot}_{max}}$ and $\\rm{Si^{tot}_{max}}$ are defined based on a cheap full RT simulation with advective-dispersive transport using a small 1D domain of 51 nodes. It is worth noting that the total amounts of $\\textbf{x}_i$, corresponding to the PHREEQC-simulated concentrations, $\\textbf{y}_i$, have to be corrected for the different amount of water between the training set and the transport simulations. Doing so, it turns out that about 20 \\% of the post-corrected $x_i$ values exceed their pre-defined upper bounds and these excessively large values need to be filtered out. Creating the 400,000 training examples thus required about 500,000 PHREEQC runs. As stated earlier, for this problem single-threaded PHREEQC performs about 670 geochemical calculations per second on our used Intel\\textsuperscript{\\textregistered} i7 CPU.\n\nWith respect to kNN, the tuning parameters are the number of neighbors, $k$, the type of distance measure, and the interpolation technique. We simply used the default settings: $k = 5$, euclidean distance and inverse-distance interpolation. Regarding training of the DNN, the 400,000 sample were split between the training set itself (90 \\% of the data) and a validation set (10 \\% of the data). The latter serves to monitor the evolution of the selected mean squared error loss function on samples that are not used for training, thereby detecting potential overfitting. If the validation loss stops decreasing before the fixed number of epochs has been completed, then  training is stopped. Importantly, the emulation is achieved in log-space for both the input, $X$ and output $Y$, domains. This because total amounts and concentrations of the involved components typically cover many orders of magnitudes (up to 10 orders or more). Using a DNN also requires some form of data normalization or standardization. Here both the $\\rm{log\\left(\\textbf{x}_i\\right)}$ and $\\rm{log\\left(\\textbf{y}_i\\right)}$ vectors are standardized around 0 with standard deviation of 1.\n\nFigure \\ref{fig1} illustrates the trained emulators' performance for geochemical predictions using an independent test set that comprises 10,000 test examples. Both kNN (Figures \\ref{fig1}a - d) and DNN (Figures \\ref{fig1}e - h) appear to be rather accurate . DNN also shows a slight degradation for the larger concentration values  (Figures \\ref{fig1}e - h). The latter is likely due to the combination of a small proportion of large concentration values in the training set with the log-transformation that implicitly pushes the DNN to try harder to fit the smaller concentrations during training. Regarding speedup and as written earlier, for this setup the single-threaded kNN method is 300 times faster than single-threaded PHREEQC for predicting the 10,000 concentration vectors all at once. The computational savings allowed by the DNN emulator when ran on our GPU is higher with a speedup as large as 4000 for predicting the same 10,000 concentration vectors all at once. \n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{0cm}\\includegraphics[width=35pc]{Figure1.png}\n\t\\caption{1-1 plots of the kNN (subfigures (a) - (d)) and DNN (subfigures (e) - (h)) emulators' performance obtained for system 1 when the kNN training base contains 400,000 samples and the DNN is trained using the same 400,000 samples. Here ``true\" means the original PHREEQC-simulated data and ``predicted\" denotes the emulated (that is, kNN-simulated and DNN-simulated) data. Hence, the $x$-axis and $y$-axis present the original and emulated 10,000 independent test data points, respectively. The RMSE and $Q_2$ coefficient denote the root-mean-square-error and coefficient of determination in testing mode, respectively, between the original and emulated 10,000 test data points.}\n\t\\label{fig1}\n\\end{figure}\n\nTo test the sensitivity of the emulators' performance to the training set size, training was also performed using reduced training sets comprising 100,000, 10,000, 1000 and 100 samples, respectively. It seen that the DNN performance achieved  when using 10,000 training samples is virtually the same as that obtained when using 400,000 training samples (Figure \\ref{fig2} and Table \\ref{table2}). It is only for training sets smaller than 1000 samples that the DNN performance starts to degrade significantly (Table \\ref{table2}). In contrast, the behavior of kNN appears to markedly decrease as the training set gets smaller (Figure \\ref{fig2} and Table \\ref{table1}).\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{0cm}\\includegraphics[width=35pc]{Figure2.png}\n\t\\caption{1-1 plots of the kNN (subfigures (a) - (d)) and DNN (subfigures (e) - (h)) emulators' performance obtained for system 1 when the kNN training base contains 10,000 samples and the DNN is trained using the same 10,000 samples. Here ``true\" means the original PHREEQC-simulated data and ``predicted\" denotes the emulated (that is, kNN-simulated and DNN-simulated) data. Hence, the $x$-axis and $y$-axis present the original and emulated 10,000 independent test data points, respectively. The RMSE and $Q_2$ coefficient denote the root-mean-square-error and coefficient of determination in testing mode, respectively, between the original and emulated 10,000 test data points.}\n\t\\label{fig2}\n\\end{figure}\n\n\\begin{table}[h!]\n\t\\caption{Performance of the DNN and kNN emulators for cement system 1 and different training set sizes. For brevity, only the results for Ca$^{conc}$ and Si$^{conc}$ are shown. The units are mol per kg of water (mol/kgw). ML refers to the type of emulator, TR signifies the size of the training set, and the RMSE and $Q_2$ coefficient denote the root-mean-square-error and coefficient of determination in testing mode, respectively, between the original and emulated 10,000 test data points.}\n\t\\begin{center}\n\t\t\\begin{tabular}{cccccccc}\n\t\t\t\\hline\n\t\t\t& & & & & \\\\\n\t\t\tML & TR & RMSE - Ca$^{conc}$ & $Q_2$ - Ca$^{conc}$ & RMSE - Si$^{conc}$ & $Q_2$ - Si$^{conc}$\\\\\n\t\t\tDNN & 4 $\\times$ 10$^{5}$ & 4.50 $\\times$ 10$^{-5}$ & 0.9999 & 1.21 $\\times$ 10$^{-6}$ & 0.9999\\\\\n\t\t\tDNN & 1 $\\times$ 10$^{5}$ & 5.06 $\\times$ 10$^{-5}$ & 0.9999 & 1.30 $\\times$ 10$^{-6}$ & 0.9999\\\\\n\t\t\tDNN & 1 $\\times$ 10$^{4}$ & 4.57 $\\times$ 10$^{-5}$ & 0.9999 & 1.48 $\\times$ 10$^{-6}$ & 0.9999\\\\\n\t\t\tDNN & 1 $\\times$ 10$^{3}$ & 8.02 $\\times$ 10$^{-5}$ & 0.9998 & 3.31 $\\times$ 10$^{-6}$ & 0.9994\\\\\n\t\t\tDNN & 1 $\\times$ 10$^{2}$ & 47.1 $\\times$ 10$^{-5}$ & 0.9935 & 9.02 $\\times$ 10$^{-6}$ & 0.9959\\\\\n\t\t\tkNN & 4 $\\times$ 10$^{5}$ & 2.52 $\\times$ 10$^{-5}$ & 1.0000 & 1.59 $\\times$ 10$^{-6}$ & 0.9999\\\\\n\t\t\tkNN & 1 $\\times$ 10$^{5}$ & 6.27 $\\times$ 10$^{-5}$ & 0.9999 & 4.20 $\\times$ 10$^{-6}$ & 0.9991\\\\\n\t\t\tkNN & 1 $\\times$ 10$^{4}$ & 14.1 $\\times$ 10$^{-5}$ & 0.9994 & 13.5 $\\times$ 10$^{-6}$ & 0.9908\\\\\n\t\t\tkNN & 1 $\\times$ 10$^{3}$ & 43.9 $\\times$ 10$^{-5}$ & 0.9944 & 74.6 $\\times$ 10$^{-6}$ & 0.7180\\\\\n\t\t\tkNN & 1 $\\times$ 10$^{2}$  & 96.4 $\\times$ 10$^{-5}$ & 0.9729 & 55.0 $\\times$ 10$^{-6}$ & 0.8468\\\\\n\t\t\\end{tabular}\n\t\\end{center}\n\t\\label{table2}\n\\end{table}\n\n\\FloatBarrier\n\n\\subsubsection{Reactive transport simulation}\n\\label{prob12Dres}\n\nThis section focuses on reactive transport simulations with HPx$_{\\rm py}$ within cement system 1, under both advective-dispersive and diffusive transport conditions. As written above, the domain sizes are both 61 $\\times$ 61 and 121 $\\times$ 121 for the advection-dispersion case and, because of computational constraints, solely 61 $\\times$ 61  for the diffusion case. In addition, the simulation time period is 2 years for the advection-dispersion case and 1 year for the diffusion case. Figures \\ref{fig3} and \\ref{fig4} present times series of original and emulated Ca, Si, H and O concentrations at 5 locations within the 2D domain for advective-dispersive transport conditions, for both our kNN-based (HPx$_{\\rm py}$-kNN) and DNN-based (HPx$_{\\rm py}$-DNN) reactive transport codes. It is seen that HPx$_{\\rm py}$-kNN and HPx$_{\\rm py}$-DNN both induce a quite good simulation accuracy. Also, the results for the diffusive transport case are of similarly good quality (not shown). Figures \\ref{fig5} - \\ref{fig6} provide more insights into the HPx$_{\\rm py}$-kNN and HPx$_{\\rm py}$-DNN performances by displaying 2D Ca, Si, H and O concentration profiles at a given time. For each experiment and chemical component, this time is selected as to be well representative of the simulated dynamics. It is observed that the original and emulated images are visually almost indistinguishable for the advection-dispersion case (Figure \\ref{fig5}). For the diffusion case, the emulators also perform quite well for Ca, H and O (Figures\\ref{fig6}a - c, g - i and j - l), while some slight to moderate discrepancies appear at the concentration front for Si (Figures \\ref{fig6}d - f). Notwithstanding, the Si concentration remains globally well predicted. Furthermore, Figures \\ref{fig7} - \\ref{fig8} present the original and emulated 2D solid amount profiles corresponding to Figures \\ref{fig5} - \\ref{fig6}. The original solid amount profiles are overall well approximated by HPx$_{\\rm py}$-kNN and HPx$_{\\rm py}$-DNN for the advection-dispersion case (Figure \\ref{fig7}), even though some mismatch appears at the border of the fully depleted zone for the H component. As of the diffusion case (Figure \\ref{fig8}), the same kind of mismatch is observed for the emulated solid amounts of H by HPx$_{\\rm py}$-kNN while the emulated profiles by HPx$_{\\rm py}$-DNN show some more discrepancies. Though we decided to present raw emulation results, we would like to stress that some if not all of the observed artifacts could likely be smoothed out by using some post-filtering such as median filtering.\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure3.png}\n\t\\caption{Time series of original (RTM, solid green lines) and HPx$_{\\rm py}$-kNN emulated (TM+kNN, dashed orange lines) concentrations (mol/kg) of Ca, H, O and Si at selected observations points for cement system 1 and advective-dispersive transport. Obs. 1 - 5 denote the selected observation points, with the following $\\left[x,y\\right]$ locations (in cm). Obs. 1:  $\\left[0.5, 2.5\\right]$, Obs. 2: $\\left[1, 2\\right]$, Obs. 3: $\\left[2, 2\\right]$, Obs. 4: $\\left[1, 1\\right]$, Obs. 5: $\\left[2, 1\\right]$. The results for the 121 $\\times$ 121 grid size are rather similar.}\n\t\\label{fig3}\n\\end{figure}\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure4.png}\n\t\\caption{Time series of original (RTM, solid green lines) and HPx$_{\\rm py}$-DNN emulated (TM+DNN, dashed orange lines) concentrations (mol/kg) of Ca, H, O and Si at selected observations points for cement system 1 and advective-dispersive transport. Obs. 1 - 5 denote the selected observation points, with the following $\\left[x,y\\right]$ locations (in cm). Obs. 1:  $\\left[0.5, 2.5\\right]$, Obs. 2: $\\left[1, 2\\right]$, Obs. 3: $\\left[2, 2\\right]$, Obs. 4: $\\left[1, 1\\right]$, Obs. 5: $\\left[2, 1\\right]$. The results for the 121 $\\times$ 121 grid size are rather similar.}\n\t\\label{fig4}\n\\end{figure}\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure5.png}\n\t\\caption{2D concentration profiles obtained for cement system 1 at the end of the 2-year simulation performed for the advection-dispersion case. RTM means the original HP$_{\\rm{4C}}$ model, TM+kNN denotes the Hydrus transport model coupled with our kNN geochemical emulator (HPx$_{\\rm py}$-kNN) and TM+DNN signifies the Hydrus transport model coupled with our DNN geochemical emulator (HPx$_{\\rm py}$-DNN). The first to fourth row present profiles for $Ca^{\\rm{conc}}$, $Si^{\\rm{conc}}$, $H^{\\rm{conc}}$, and $O^{\\rm{conc}}$, respectively. The considered grid size is 61 $\\times$ 61. The results for the 121 $\\times$ 121 grid are rather similar.}\n\t\\label{fig5}\n\\end{figure}\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure6.png}\n\t\\caption{2D concentration profiles obtained for cement system 1 at the final time step of the 1-year simulation performed for the diffusion case. RTM means the original HP$_{\\rm{4C}}$ model, TM+kNN denotes the Hydrus transport model coupled with our kNN geochemical emulator (HPx$_{\\rm py}$-kNN) and TM+DNN signifies the Hydrus transport model coupled with our DNN geochemical emulator (HPx$_{\\rm py}$-DNN). The first to fourth row present profiles for $Ca^{\\rm{conc}}$, $Si^{\\rm{conc}}$, $H^{\\rm{conc}}$, and $O^{\\rm{conc}}$, respectively. The considered grid size is 61 $\\times$ 61.}\n\t\\label{fig6}\n\\end{figure}\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure7.png}\n\t\\caption{2D solid amount profiles obtained for cement system 1 at the end of the 2-year simulation performed for the advection-dispersion case. RTM means the original HP$_{\\rm{4C}}$ model, TM+kNN denotes the Hydrus transport model coupled with our kNN geochemical emulator (HPx$_{\\rm py}$-kNN) and TM+DNN signifies the Hydrus transport model coupled with our DNN geochemical emulator (HPx$_{\\rm py}$-DNN). The first to fourth row present profiles for $Ca^{\\rm{solid}}$, $Si^{\\rm{solid}}$, $H^{\\rm{solid}}$, and $O^{\\rm{solid}}$, respectively. The considered grid size is 61 $\\times$ 61. The results for the 121 $\\times$ 121 grid are rather similar.}\n\t\\label{fig7}\n\\end{figure}\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure8.png}\n\t\\caption{2D solid amount profiles obtained for cement system 1 at the final time step of the 1-year simulation performed for the diffusion case. RTM means the original HP$_{\\rm{4C}}$ model, TM+kNN denotes the Hydrus transport model coupled with our kNN geochemical emulator (HPx$_{\\rm py}$-kNN) and TM+DNN signifies the Hydrus transport model coupled with our DNN geochemical emulator (HPx$_{\\rm py}$-DNN). The first to fourth row present profiles for $Ca^{\\rm{solid}}$, $Si^{\\rm{solid}}$, $H^{\\rm{solid}}$, and $O^{\\rm{solid}}$, respectively. The considered grid size is 61 $\\times$ 61.}\n\t\\label{fig8}\n\\end{figure}\n\nThe speedups associated with the considered problem are detailed in Table \\ref{table3}. It is noted that the GPU-based DNN emulator allows for a speedup that is close to optimal. Indeed, The DNN speedups overall represent 85 \\% to 95 \\% of the maximum possible speedups (that is, speedups that would be obtained if the geochemcical calculations would come at no cost at all). The speedups associated with single-threaded kNN remain substantial but only amount to 57 \\% - 65 \\% of the corresponding maximum speedups. As detailed in section \\ref{train_res1}, the used kNN and DNN implementations are found to be respectively 300 and 4000 faster than single-threaded PHREEQC when predicting 10,000 points all one once for this geochemical system. Based on these numbers one could have expected the achieved speedups to represent say 90 \\% (kNN) or 99 \\% (DNN) of the maximum possible ones. A large part of the gaps between achieved and maximum possible speedups is thus likely caused by the time required for communicating and exchanging data between the main C/C++ code and the Python-based emulators.\n\n\\begin{table}[h!]\n\t\\caption{Speedups offered by the KNN and DNN emulators in HPx$_{\\rm py}$ for the reactive transport simulations considered for cement system 1. The HPx$_{\\rm{4C}}$ calculations involve the parallelization of PHREEQC over our 4 CPUs. The HPx$_{\\rm{1C}}$ calculations are performed on a single CPU. The kNN predictions are performed on a single CPU using the scikit-learn implementation while the DNN predictions make use of our GPU. ML signifies the used machine learning method for emulation, TC denotes transport conditions (ADV: advection-dispersion, DIF: diffusion) and GS is the grid size. The maximum possible speedups associated with HPx$_{\\rm{4C}}$ and HPx$_{\\rm{1C}}$,  Max SP HPx$_{\\rm{4C}}$ and  Max SP HPx$_{\\rm{1C}}$, correspond to an hypothetical situation where the geochemical calculations incur zero computational cost.}\n\n\t\\begin{center}\n\t\t\\begin{tabular}{cccccccc}\n\t\t\t\\hline\n\t\t\t& & & & & & & \\\\\n\t\t\tML & TC & GS & HPx$_{\\rm{4C}}$ time (s) & SP HPx$_{\\rm{4C}}$ & Max SP HPx$_{\\rm{4C}}$ & SP HPx$_{\\rm{1C}}$ & Max SP HPx$_{\\rm{1C}}$ \\\\\n\t\t\tDNN & ADV & 61 $\\times$ 61 & 3189 & 6.8 & 7.7 & 24.5 & 28.5 \\\\\n\t\t\tkNN & ADV & 61 $\\times$ 61 & 3189 & 5.0 & 7.7 & 18.0 & 28.5 \\\\\n\t\t\tDNN & ADV & 121 $\\times$ 121 & 23,337 & 5.0 & 5.2 & 17.0 & 18.0 \\\\\n\t\t\tkNN & ADV & 121 $\\times$ 121 & 23,337 & 3.4 & 5.2 & 11.6 & 18.0 \\\\\n\t\t\tDNN & DIF & 61 $\\times$ 61 & 25,448 & 4.2 & 4.9 & 16.2 & 19.1 \\\\\n\t\t\tkNN & DIF & 61 $\\times$ 61 & 25,448 & 2.8 & 4.9 & 10.8 & 19.1 \\\\\n\t\t\t\\hline\n\t\t\\end{tabular}\n\t\\end{center}\n\n\t\\label{table3}\n\\end{table}\n\n\\FloatBarrier\n\n\\subsection{Al-C-Ca-S-Si Problem}\n\nFor this second cement system, the emulation problem consists of predicting at each time step of the RT simulation the (output) Al, C, Ca, S, Si, H and O aqueous concentrations (mol/kgw) from the (input) total amounts of Al, C, Ca, S and Si (mol). Here we focus on advective-dispersive transport only while similarly as for cement system 1, the considered domain sizes are 61 $\\times$ 61 and 121 $\\times$ 121. Moreover, the simulation time period is set to 6 years. \n\nAs mentioned earlier, for this higher-dimensional problem it is observed that the used scikit-learn kNN implementation becomes prohibitively slow compared to HPx$_{\\rm{4C}}$. We found that to get a good emulation accuracy the kNN training base needs to contain 1,000,000 samples (or more). This training base's size together with a 5-dimensional search space leads to an HPx$_{\\rm py}$-kNN reactive transport simulation time that is comparable to that of HPx$_{\\rm{4C}}$. Therefore, for this second cement system we built a custom kNN regressor around another kNN implementation contained in the FAISS package \\citep{faiss2017}. Our used FAISS variant allows for GPU computing and is much faster than scikit-learn for this cement system, but is slightly less accurate due to the use of an approximate rather than exact nearest neighbor search \\citep[see][for details]{faiss2017}.\n\n\\subsubsection{Training the DNN}\n\\label{train_res2}\n\nBuilding a good training set to perform a kNN search and learn the weights and biases of our DNN turned out to be a complicated task in this case. This because to make useful kNN predictions and/or learn an useful DNN, the training set must be sufficiently representative of the geochemical conditions encountered during the reactive transport simulation one wish to perform with HPx$_{\\rm py}$-kNN and the trained HPx$_{\\rm py}$-DNN. In contrast to cement system 1, creating the training set by sampling the $X$-space with a controlled randomness between predefined lower and upper bounds did not prove successful. We tried that strategy by drawing as much as 4,000,000 5-dimensional $\\textbf{x}$ vectors from the $X$-space using a Sobol low-discrepancy sequence \\citep[][]{Sobol1967, Joe-Kuo2003}. Such low-discrepancy sampling scheme covers the 5-dimensional hypercube more uniformly than LHS. Despite a good performance on the test set (not shown), the resulting DNN accuracy in reactive transport mode was never deemed satisfying. In other words, no satisfying ``global\" or ``universal\" DNN emulator could be devised for this cement system. This is probably caused by the fact for this problem, the input (5 total amounts) and output (7 aqueous concentrations) spaces are quite nonlinearly related and both cover 6 to 10 orders of magnitudes depending on the considered element. Therefore, we resorted to the alternative training strategy detailed below. The latter basically tries to grasp the complex correlations and higher-order dependencies that exist between the elements of $\\textbf{x}$ (total amounts, input space) for a given reactive transport simulation setup, in order to produce a training set that honors these between-input relationships.\n\n\\begin{itemize}\n\t\\item Perform a ``cheap\" full reactive transport simulation under the transport conditions and geochemistry of interest and collect the resulting $\\textbf{x}$ -$\\textbf{y}$  pairs of examples (for the considered grid nodes and time steps). Computational demand controls what domain size and simulation time period can be used for this cheap calculation. We used a modest 16 $\\times$ 16 domain and a simulation time period of 10 years. The associated HPx$_{\\rm{4C}}$ runtimes is 180 s.\n\t\n\t\\item Fit a kernel density estimator (KDE) with a Gaussian kernel to the collected $\\textbf{x}$ vectors (encapsulated in the $\\textbf{X}$ array) and generate a fixed number of new input vectors, $\\textbf{x}_{KDE}$. Then run PHREEQC for the $\\textbf{X}_{KDE}$ set to get the corresponding output set, $\\textbf{Y}_{KDE}$. Now apply the correction for porosity described in section \\ref{train_res1} to the $\\textbf{X}_{KDE}$ set and form the training set by merging the ensemble of $\\textbf{x}$-$\\textbf{y}$ pairs with that of the $\\textbf{x}_{KDE}$-$\\textbf{y}_{KDE}$ pairs. The number of produced unique examples by the considered cheap HPx$_{\\rm{4C}}$ simulations varied between 10,000 and 50,000. The KDE-based enrichment of this dataset was deemed necessary to provide more input variability, thereby avoiding overfitting of the trained DNN and improving the kNN accuracy, while still honoring the complex between-input relationships. The number of KDE-generated samples was set as to obtain a total training set size of 1,000,000 examples. A key component of the approach is the bandwith parameter of the Gaussian KDE kernel which controls how much the KDE-generated samples depart from the original ensemble. After limited trial and error, we fixed the kernel bandwith to 0.0025 for the considered case studies.\n\t\n\\end{itemize}\n\nThe scatter plots in Figure \\ref{fig9} illustrate our training set creation procedure. The orange dots depict the pairwise relationships observed between the 5 elements of $\\textbf{x}$ in the cheap simulation. The red and cyan dots in Figure \\ref{fig9} represent the KDE-generated samples with the selected bandwidth, before and after applying the correction for porosity, respectively. Training of the DNN is achieved using the ensemble of original and KDE-corrected input points. We refer to this kind of dataset as RT-based, since it is based on an albeit cheap, full RT simulation. Furthermore, we refer to the obtained DNN and kNN emulators as ``local\" emulators, since as opposed to the emulators constructed for cement system 1, the current emulators are only valid for the input conditions encapsulated in the RT-based training set.\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1.5cm}\\includegraphics[width=50pc]{Figure9.png}\n\t\\caption{Scatter plots of the complex relationships between the five considered inputs for (1) the computationally cheap RT simulation of cement system 2 (orange dots) performed with the original HPx code, (2) the corresponding sampled points by kernel density estimation (KDE) using the selected bandwith (turquoise dots) and, (3) the same sampled KDE points after porosity correction (red dots).}\n\t\\label{fig9}\n\\end{figure}\n\nTraining performance of the DNN emulator is presented in Figure \\ref{fig10}. Similarly as for cement system 1, about 90\\% of the available data was used for the actual training of the DNN while the remaining 10\\% were used a validation set to contol overfitting. Lastly, performance is evaluated for both the trained DNN and kNN emulators using an independent test set of 10,000 examples. Overall, the accuracy of our ``local \" DNN emulator for this RT-inspired dataset is rather large with $Q_2$ values always greater or equal than 0.998. Training performance of the corresponding local kNN emulator is equally good (not shown). Regarding speedup, for this problem single-threaded PHREEQC achieves about 210 geochemical calculations per second on our used Intel\\textsuperscript{\\textregistered} i7 CPU while the GPU-based DNN and kNN emulators are both about 3000 times faster when predicting the 10,000 test points at once .\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure10.png}\n\t\\caption{1-1 plots of local DNN emulation performance obtained for system 2 when the local DNN is trained using 1,000,000 samples. The $x$-axis and $y$-axis present the original and emulated 10,000 independent test data points, respectively. The RMSE and $Q_2$ coefficient denote the root-mean-square-error and coefficient of determination in testing mode, respectively, between the original and emulated 10,000 test data points.}\n\t\\label{fig10}\n\\end{figure}\n\n\\subsubsection{Reactive transport simulation}\n\\label{prob2res}\nOur ``local\" DNN performs rather well when applied to the 61 $\\times$ 61 grid size and a time period of 6 years (Figures \\ref{fig11} - \\ref{fig12}). This for all components but C, for which some localized deviations appear between original and emulated 2D concentration profiles towards the end of the simulation period (Figure \\ref{fig12}). When applied to the 121 $\\times$ 121 grid, HPx$_{\\rm py}$-DNN produces additional discrepancies for O, Si and Al towards the end of the simulation period (Figures \\ref{fig13} - \\ref{fig14}). Yet most of the observed artifacts could probably be smoothed out by using post-filtering. The associated speedups are listed in Table \\ref{table4}. These speedups are larger to those obtained for cement system 1, with values between 8 and 9 when evaluated against HPx$_{\\rm{4C}}$. These speedups represent about 85 \\% to 90 \\% of the maximum possible speedup (Table \\ref{table3}). Overall, these findings indicate that for the considered problem, our RT-based training of a local DNN only works is the training set is sufficiently representative of the particular geochemical conditions encountered in the computationally demanding simulations, which is arguably not easy to achieve. This limitation is further discussed in section \\ref{discussion}.\n\nWe note a more uniform behavior for HPx$_{\\rm py}$-kNN across grid sizes than for HPx$_{\\rm py}$-DNN. Here the results for the 121  $\\times$ 121 grid are only slightly less accurate than those associated with the 61 $ \\times$ 61 grid (see Figures \\ref{fig15} - \\ref{fig16} where for brevity we only show concentration profiles for the C Al and S components). Furthermore, whenever observed the discrepancies between original and emulated profiles are more regularly scattered than for HPx$_{\\rm py}$-DNN. Note also that herein too, post-filtering could likely smooth out a large part of these deviations. In addition, owing to the use of a GPU to achieve the kNN calculations, the speedup provided by HPx$_{\\rm py}$-kNN are as large as those provided by HPx$_{\\rm py}$-DNN (Table \\ref{table4}).\n\nWith respect to the emulated solid amounts, the HPx$_{\\rm py}$-DNN results look visually good for the the 61 $\\times$ 61 grid. This is shown in Figure \\ref{fig17} for the C, Al and S chemical components, while emulation of the H, O Ca and Si chemical components is globally of similar quality (not shown). Nevertheless, significant deviations appear towards the end of the simulation period for every chemical component (see Figure \\ref{fig18} for the C, Al and S chemical components, emulation of the H, O Ca and Si chemical components shows the same level of mismatch). The HPx$_{\\rm py}$-kNN predictions are also fairly accurate for the 61 $\\times$ 61 (see Figure \\ref{fig19} for the C, Al and S chemical components, emulation of the H, O Ca and Si chemical components exhibits a globally similar quality) while some discrepancies show up at the end of the simulation (Figure \\ref{fig20}). However, the mismatch is less pronounced than for HPx$_{\\rm py}$-DNN. Overall, HPx$_{\\rm py}$-kNN appears to be somewhat more robust than HPx$_{\\rm py}$-DNN for this cement system, while providing the same (large) speedup.\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure11.png}\n\t\\caption{2D concentration profiles obtained for cement system 2 after 2 and 6 (final time step) years. RTM means the original HPx$_{\\rm{4C}}$ model and TM+DNN denotes the Hydrus transport model coupled with our DNN geochemical emulator (HPx$_{\\rm py}$-DNN). The first to fourth row present profiles for $Ca^{\\rm{conc}}$, $Si^{\\rm{conc}}$, $H^{\\rm{conc}}$, and $O^{\\rm{conc}}$, respectively. The considered grid size is 61 $\\times$ 61.}\n\t\\label{fig11}\n\\end{figure}\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure12.png}\n\t\\caption{2D concentration profiles obtained for cement system 2 after 2 and 6 (final time step) years. RTM means the original HPx$_{\\rm{4C}}$ model and TM+DNN denotes the Hydrus transport model coupled with our DNN geochemical emulator (HPx$_{\\rm py}$-DNN). The first to third row present profiles for $C^{\\rm{conc}}$, $Al^{\\rm{conc}}$, and $S^{\\rm{conc}}$, respectively. The considered grid size is 61 $\\times$ 61.}\n\t\\label{fig12}\n\\end{figure}\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure13.png}\n\t\\caption{2D concentration profiles obtained for cement system 2 after 2 and 6 (final time step) years. RTM means the original HPx$_{\\rm{4C}}$ model and TM+DNN denotes the Hydrus transport model coupled with our DNN geochemical emulator (HPx$_{\\rm py}$-DNN). The first to fourth row present profiles for $Ca^{\\rm{conc}}$, $Si^{\\rm{conc}}$, $H^{\\rm{conc}}$, and $O^{\\rm{conc}}$, respectively. The considered grid size is 121 $\\times$ 121.}\n\t\\label{fig13}\n\\end{figure}\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure14.png}\n\t\\caption{2D concentration profiles obtained for cement system 2 after 2 and 6 (final time step) years. RTM means the original HPx$_{\\rm{4C}}$ model and TM+DNN denotes the Hydrus transport model coupled with our DNN geochemical emulator (HPx$_{\\rm py}$-DNN). The first to third row present profiles for $C^{\\rm{conc}}$, $Al^{\\rm{conc}}$, and $S^{\\rm{conc}}$, respectively. The considered grid size is 121 $\\times$ 121.}\n\t\\label{fig14}\n\\end{figure}\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure15.png}\n\t\\caption{2D concentration profiles obtained for cement system 2 after 2 and 6 (final time step) years. RTM means the original HPx$_{\\rm{4C}}$ model and TM+kNN denotes the Hydrus transport model coupled with our kNN geochemical emulator (HPx$_{\\rm py}$-kNN).  The first to third row present profiles for $C^{\\rm{conc}}$, $Al^{\\rm{conc}}$, and $S^{\\rm{conc}}$, respectively. The considered grid size is 61 $\\times$ 61.}\n\t\\label{fig15}\n\\end{figure}\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure16.png}\n\t\\caption{2D concentration profiles obtained for cement system 2 after 2 and 6 (final time step) years. RTM means the original HPx$_{\\rm{4C}}$ model and TM+kNN denotes the Hydrus transport model coupled with our kNN geochemical emulator (HPx$_{\\rm py}$-kNN). The first to third row present profiles for $C^{\\rm{conc}}$, $Al^{\\rm{conc}}$, and $S^{\\rm{conc}}$, respectively. The considered grid size is 121 $\\times$ 121.}\n\t\\label{fig16}\n\\end{figure}\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure17.png}\n\t\\caption{2D solid amount profiles of the C, Al and S chemical components of cement system 2 after 2 and 6 (final time step) years. RTM means the original HPx$_{\\rm{4C}}$ model and TM+DNN denotes the Hydrus transport model coupled with our DNN geochemical emulator (HPx$_{\\rm py}$-DNN). The first to third row present profiles for $C^{\\rm{solid}}$, $Al^{\\rm{solid}}$, and $S^{\\rm{solid}}$, respectively The considered grid size is 61 $\\times$ 61.}\n\t\\label{fig17}\n\\end{figure}\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure18.png}\n\t\\caption{2D solid amount profiles of the C, Al and S chemical components of cement system 2 after 2 and 6 (final time step) years. RTM means the original HPx$_{\\rm{4C}}$ model and TM+DNN denotes the Hydrus transport model coupled with our DNN geochemical emulator (HPx$_{\\rm py}$-DNN). The first to third row present profiles for $C^{\\rm{solid}}$, $Al^{\\rm{solid}}$, and $S^{\\rm{solid}}$, respectively The considered grid size is 121 $\\times$ 121.}\n\t\\label{fig18}\n\\end{figure}\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure19.png}\n\t\\caption{2D solid amount profiles of the C, Al and S chemical components of cement system 2 after 2 and 6 (final time step) years. RTM means the original HPx$_{\\rm{4C}}$ model and TM+kNN denotes the Hydrus transport model coupled with our kNN geochemical emulator (HPx$_{\\rm py}$-kNN). The first to third row present profiles for $C^{\\rm{solid}}$, $Al^{\\rm{solid}}$, and $S^{\\rm{solid}}$, respectively The considered grid size is 61 $\\times$ 61.}\n\t\\label{fig19}\n\\end{figure}\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure20.png}\n\t\\caption{2D solid amount profiles of the C, Al and S chemical components of cement system 2 after 2 and 6 (final time step) years. RTM means the original HPx$_{\\rm{4C}}$ model and TM+kNN denotes the Hydrus transport model coupled with our kNN geochemical emulator (HPx$_{\\rm py}$-kNN). The first to third row present profiles for $C^{\\rm{solid}}$, $Al^{\\rm{solid}}$, and $S^{\\rm{solid}}$, respectively The considered grid size is 121 $\\times$ 121.}\n\t\\label{fig20}\n\\end{figure}\n\n\\begin{table}[h!]\n\t\\caption{Speedups offered by the KNN and DNN emulators in HPx$_{\\rm py}$ for the reactive transport simulations considered for cement system 2. The HPx$_{\\rm{4C}}$ calculations involve the parallelization of PHREEQC over our 4 CPUs. The HPx$_{\\rm{1C}}$ calculations are performed on a single CPU. Both the kNN and DNN predictions make use of a GPU. ML signifies the used machine learning method for emulation, BC denotes the type of flow boundary conditions and GS is the grid size. The maximum possible speedups associated with HPx$_{\\rm{4C}}$ and HP$_{\\rm{1C}}$,  Max SP HPx$_{\\rm{4C}}$ and  Max SP HPx$_{\\rm{1C}}$, correspond to an hypothetical situation where the geochemical calculations incur zero computational cost.}\n\n\t\\begin{center}\n\t\t\\begin{tabular}{cccccccc}\n\t\t\t\\hline\n\t\t\t& & & & & & & \\\\\n\t\t\tML & BC & GS & HPx$_{\\rm{4C}}$ time (s) & SP HPx$_{\\rm{4C}}$ & Max SP HPx$_{\\rm{4C}}$ & SP HPx$_{\\rm{1C}}$ & Max SP HPx$_{\\rm{1C}}$ \\\\\n\t\t\tDNN & ADV & 61 $\\times$ 61 & 21,415 & 8.2 & 9.0 & 30.3 & 33.1\\\\\n\t\t\tkNN & ADV & 61 $\\times$ 61 & 21,415 & 7.9 & 9.0 & 28.9 & 33.1\\\\\n\t\t\tDNN & ADV & 121 $\\times$ 121 & 199,841 & 8.2 & 9.5 & 29.9 & 32.8\\\\\n\t\t\tkNN & ADV & 121 $\\times$ 121 & 199,841 & 8.5 & 9.5 & 31.3 & 32.8\\\\\n\t\t\t\\hline\n\t\t\\end{tabular}\n\t\\end{center}\n\n\t\\label{table4}\n\\end{table}\n\n\\FloatBarrier\n\n3.2 Al-C-Ca-S-Si Problem\n\\subsection{Al-C-Ca-S-Si Problem}\n\nFor this second cement system, the emulation problem consists of predicting at each time step of the RT simulation the (output) Al, C, Ca, S, Si, H and O aqueous concentrations (mol/kgw) from the (input) total amounts of Al, C, Ca, S and Si (mol). Here we focus on advective-dispersive transport only while similarly as for cement system 1, the considered domain sizes are 61 $\\times$ 61 and 121 $\\times$ 121. Moreover, the simulation time period is set to 6 years. \n\nAs mentioned earlier, for this higher-dimensional problem it is observed that the used scikit-learn kNN implementation becomes prohibitively slow compared to HPx$_{\\rm{4C}}$. We found that to get a good emulation accuracy the kNN training base needs to contain 1,000,000 samples (or more). This training base's size together with a 5-dimensional search space leads to an HPx$_{\\rm py}$-kNN reactive transport simulation time that is comparable to that of HPx$_{\\rm{4C}}$. Therefore, for this second cement system we built a custom kNN regressor around another kNN implementation contained in the FAISS package \\citep{faiss2017}. Our used FAISS variant allows for GPU computing and is much faster than scikit-learn for this cement system, but is slightly less accurate due to the use of an approximate rather than exact nearest neighbor search \\citep[see][for details]{faiss2017}.\n\n\\subsubsection{Training the DNN}\n\\label{train_res2}\n\nBuilding a good training set to perform a kNN search and learn the weights and biases of our DNN turned out to be a complicated task in this case. This because to make useful kNN predictions and/or learn an useful DNN, the training set must be sufficiently representative of the geochemical conditions encountered during the reactive transport simulation one wish to perform with HPx$_{\\rm py}$-kNN and the trained HPx$_{\\rm py}$-DNN. In contrast to cement system 1, creating the training set by sampling the $X$-space with a controlled randomness between predefined lower and upper bounds did not prove successful. We tried that strategy by drawing as much as 4,000,000 5-dimensional $\\textbf{x}$ vectors from the $X$-space using a Sobol low-discrepancy sequence \\citep[][]{Sobol1967, Joe-Kuo2003}. Such low-discrepancy sampling scheme covers the 5-dimensional hypercube more uniformly than LHS. Despite a good performance on the test set (not shown), the resulting DNN accuracy in reactive transport mode was never deemed satisfying. In other words, no satisfying ``global\" or ``universal\" DNN emulator could be devised for this cement system. This is probably caused by the fact for this problem, the input (5 total amounts) and output (7 aqueous concentrations) spaces are quite nonlinearly related and both cover 6 to 10 orders of magnitudes depending on the considered element. Therefore, we resorted to the alternative training strategy detailed below. The latter basically tries to grasp the complex correlations and higher-order dependencies that exist between the elements of $\\textbf{x}$ (total amounts, input space) for a given reactive transport simulation setup, in order to produce a training set that honors these between-input relationships.\n\n\\begin{itemize}\n\t\\item Perform a ``cheap\" full reactive transport simulation under the transport conditions and geochemistry of interest and collect the resulting $\\textbf{x}$ -$\\textbf{y}$  pairs of examples (for the considered grid nodes and time steps). Computational demand controls what domain size and simulation time period can be used for this cheap calculation. We used a modest 16 $\\times$ 16 domain and a simulation time period of 10 years. The associated HPx$_{\\rm{4C}}$ runtimes is 180 s.\n\t\n\t\\item Fit a kernel density estimator (KDE) with a Gaussian kernel to the collected $\\textbf{x}$ vectors (encapsulated in the $\\textbf{X}$ array) and generate a fixed number of new input vectors, $\\textbf{x}_{KDE}$. Then run PHREEQC for the $\\textbf{X}_{KDE}$ set to get the corresponding output set, $\\textbf{Y}_{KDE}$. Now apply the correction for porosity described in section \\ref{train_res1} to the $\\textbf{X}_{KDE}$ set and form the training set by merging the ensemble of $\\textbf{x}$-$\\textbf{y}$ pairs with that of the $\\textbf{x}_{KDE}$-$\\textbf{y}_{KDE}$ pairs. The number of produced unique examples by the considered cheap HPx$_{\\rm{4C}}$ simulations varied between 10,000 and 50,000. The KDE-based enrichment of this dataset was deemed necessary to provide more input variability, thereby avoiding overfitting of the trained DNN and improving the kNN accuracy, while still honoring the complex between-input relationships. The number of KDE-generated samples was set as to obtain a total training set size of 1,000,000 examples. A key component of the approach is the bandwith parameter of the Gaussian KDE kernel which controls how much the KDE-generated samples depart from the original ensemble. After limited trial and error, we fixed the kernel bandwith to 0.0025 for the considered case studies.\n\t\n\\end{itemize}\n\nThe scatter plots in Figure \\ref{fig9} illustrate our training set creation procedure. The orange dots depict the pairwise relationships observed between the 5 elements of $\\textbf{x}$ in the cheap simulation. The red and cyan dots in Figure \\ref{fig9} represent the KDE-generated samples with the selected bandwidth, before and after applying the correction for porosity, respectively. Training of the DNN is achieved using the ensemble of original and KDE-corrected input points. We refer to this kind of dataset as RT-based, since it is based on an albeit cheap, full RT simulation. Furthermore, we refer to the obtained DNN and kNN emulators as ``local\" emulators, since as opposed to the emulators constructed for cement system 1, the current emulators are only valid for the input conditions encapsulated in the RT-based training set.\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1.5cm}\\includegraphics[width=50pc]{Figure9.png}\n\t\\caption{Scatter plots of the complex relationships between the five considered inputs for (1) the computationally cheap RT simulation of cement system 2 (orange dots) performed with the original HPx code, (2) the corresponding sampled points by kernel density estimation (KDE) using the selected bandwith (turquoise dots) and, (3) the same sampled KDE points after porosity correction (red dots).}\n\t\\label{fig9}\n\\end{figure}\n\nTraining performance of the DNN emulator is presented in Figure \\ref{fig10}. Similarly as for cement system 1, about 90\\% of the available data was used for the actual training of the DNN while the remaining 10\\% were used a validation set to contol overfitting. Lastly, performance is evaluated for both the trained DNN and kNN emulators using an independent test set of 10,000 examples. Overall, the accuracy of our ``local \" DNN emulator for this RT-inspired dataset is rather large with $Q_2$ values always greater or equal than 0.998. Training performance of the corresponding local kNN emulator is equally good (not shown). Regarding speedup, for this problem single-threaded PHREEQC achieves about 210 geochemical calculations per second on our used Intel\\textsuperscript{\\textregistered} i7 CPU while the GPU-based DNN and kNN emulators are both about 3000 times faster when predicting the 10,000 test points at once .\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure10.png}\n\t\\caption{1-1 plots of local DNN emulation performance obtained for system 2 when the local DNN is trained using 1,000,000 samples. The $x$-axis and $y$-axis present the original and emulated 10,000 independent test data points, respectively. The RMSE and $Q_2$ coefficient denote the root-mean-square-error and coefficient of determination in testing mode, respectively, between the original and emulated 10,000 test data points.}\n\t\\label{fig10}\n\\end{figure}\n\n\\subsubsection{Reactive transport simulation}\n\\label{prob2res}\nOur ``local\" DNN performs rather well when applied to the 61 $\\times$ 61 grid size and a time period of 6 years (Figures \\ref{fig11} - \\ref{fig12}). This for all components but C, for which some localized deviations appear between original and emulated 2D concentration profiles towards the end of the simulation period (Figure \\ref{fig12}). When applied to the 121 $\\times$ 121 grid, HPx$_{\\rm py}$-DNN produces additional discrepancies for O, Si and Al towards the end of the simulation period (Figures \\ref{fig13} - \\ref{fig14}). Yet most of the observed artifacts could probably be smoothed out by using post-filtering. The associated speedups are listed in Table \\ref{table4}. These speedups are larger to those obtained for cement system 1, with values between 8 and 9 when evaluated against HPx$_{\\rm{4C}}$. These speedups represent about 85 \\% to 90 \\% of the maximum possible speedup (Table \\ref{table3}). Overall, these findings indicate that for the considered problem, our RT-based training of a local DNN only works is the training set is sufficiently representative of the particular geochemical conditions encountered in the computationally demanding simulations, which is arguably not easy to achieve. This limitation is further discussed in section \\ref{discussion}.\n\nWe note a more uniform behavior for HPx$_{\\rm py}$-kNN across grid sizes than for HPx$_{\\rm py}$-DNN. Here the results for the 121  $\\times$ 121 grid are only slightly less accurate than those associated with the 61 $ \\times$ 61 grid (see Figures \\ref{fig15} - \\ref{fig16} where for brevity we only show concentration profiles for the C Al and S components). Furthermore, whenever observed the discrepancies between original and emulated profiles are more regularly scattered than for HPx$_{\\rm py}$-DNN. Note also that herein too, post-filtering could likely smooth out a large part of these deviations. In addition, owing to the use of a GPU to achieve the kNN calculations, the speedup provided by HPx$_{\\rm py}$-kNN are as large as those provided by HPx$_{\\rm py}$-DNN (Table \\ref{table4}).\n\nWith respect to the emulated solid amounts, the HPx$_{\\rm py}$-DNN results look visually good for the the 61 $\\times$ 61 grid. This is shown in Figure \\ref{fig17} for the C, Al and S chemical components, while emulation of the H, O Ca and Si chemical components is globally of similar quality (not shown). Nevertheless, significant deviations appear towards the end of the simulation period for every chemical component (see Figure \\ref{fig18} for the C, Al and S chemical components, emulation of the H, O Ca and Si chemical components shows the same level of mismatch). The HPx$_{\\rm py}$-kNN predictions are also fairly accurate for the 61 $\\times$ 61 (see Figure \\ref{fig19} for the C, Al and S chemical components, emulation of the H, O Ca and Si chemical components exhibits a globally similar quality) while some discrepancies show up at the end of the simulation (Figure \\ref{fig20}). However, the mismatch is less pronounced than for HPx$_{\\rm py}$-DNN. Overall, HPx$_{\\rm py}$-kNN appears to be somewhat more robust than HPx$_{\\rm py}$-DNN for this cement system, while providing the same (large) speedup.\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure11.png}\n\t\\caption{2D concentration profiles obtained for cement system 2 after 2 and 6 (final time step) years. RTM means the original HPx$_{\\rm{4C}}$ model and TM+DNN denotes the Hydrus transport model coupled with our DNN geochemical emulator (HPx$_{\\rm py}$-DNN). The first to fourth row present profiles for $Ca^{\\rm{conc}}$, $Si^{\\rm{conc}}$, $H^{\\rm{conc}}$, and $O^{\\rm{conc}}$, respectively. The considered grid size is 61 $\\times$ 61.}\n\t\\label{fig11}\n\\end{figure}\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure12.png}\n\t\\caption{2D concentration profiles obtained for cement system 2 after 2 and 6 (final time step) years. RTM means the original HPx$_{\\rm{4C}}$ model and TM+DNN denotes the Hydrus transport model coupled with our DNN geochemical emulator (HPx$_{\\rm py}$-DNN). The first to third row present profiles for $C^{\\rm{conc}}$, $Al^{\\rm{conc}}$, and $S^{\\rm{conc}}$, respectively. The considered grid size is 61 $\\times$ 61.}\n\t\\label{fig12}\n\\end{figure}\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure13.png}\n\t\\caption{2D concentration profiles obtained for cement system 2 after 2 and 6 (final time step) years. RTM means the original HPx$_{\\rm{4C}}$ model and TM+DNN denotes the Hydrus transport model coupled with our DNN geochemical emulator (HPx$_{\\rm py}$-DNN). The first to fourth row present profiles for $Ca^{\\rm{conc}}$, $Si^{\\rm{conc}}$, $H^{\\rm{conc}}$, and $O^{\\rm{conc}}$, respectively. The considered grid size is 121 $\\times$ 121.}\n\t\\label{fig13}\n\\end{figure}\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure14.png}\n\t\\caption{2D concentration profiles obtained for cement system 2 after 2 and 6 (final time step) years. RTM means the original HPx$_{\\rm{4C}}$ model and TM+DNN denotes the Hydrus transport model coupled with our DNN geochemical emulator (HPx$_{\\rm py}$-DNN). The first to third row present profiles for $C^{\\rm{conc}}$, $Al^{\\rm{conc}}$, and $S^{\\rm{conc}}$, respectively. The considered grid size is 121 $\\times$ 121.}\n\t\\label{fig14}\n\\end{figure}\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure15.png}\n\t\\caption{2D concentration profiles obtained for cement system 2 after 2 and 6 (final time step) years. RTM means the original HPx$_{\\rm{4C}}$ model and TM+kNN denotes the Hydrus transport model coupled with our kNN geochemical emulator (HPx$_{\\rm py}$-kNN).  The first to third row present profiles for $C^{\\rm{conc}}$, $Al^{\\rm{conc}}$, and $S^{\\rm{conc}}$, respectively. The considered grid size is 61 $\\times$ 61.}\n\t\\label{fig15}\n\\end{figure}\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure16.png}\n\t\\caption{2D concentration profiles obtained for cement system 2 after 2 and 6 (final time step) years. RTM means the original HPx$_{\\rm{4C}}$ model and TM+kNN denotes the Hydrus transport model coupled with our kNN geochemical emulator (HPx$_{\\rm py}$-kNN). The first to third row present profiles for $C^{\\rm{conc}}$, $Al^{\\rm{conc}}$, and $S^{\\rm{conc}}$, respectively. The considered grid size is 121 $\\times$ 121.}\n\t\\label{fig16}\n\\end{figure}\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure17.png}\n\t\\caption{2D solid amount profiles of the C, Al and S chemical components of cement system 2 after 2 and 6 (final time step) years. RTM means the original HPx$_{\\rm{4C}}$ model and TM+DNN denotes the Hydrus transport model coupled with our DNN geochemical emulator (HPx$_{\\rm py}$-DNN). The first to third row present profiles for $C^{\\rm{solid}}$, $Al^{\\rm{solid}}$, and $S^{\\rm{solid}}$, respectively The considered grid size is 61 $\\times$ 61.}\n\t\\label{fig17}\n\\end{figure}\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure18.png}\n\t\\caption{2D solid amount profiles of the C, Al and S chemical components of cement system 2 after 2 and 6 (final time step) years. RTM means the original HPx$_{\\rm{4C}}$ model and TM+DNN denotes the Hydrus transport model coupled with our DNN geochemical emulator (HPx$_{\\rm py}$-DNN). The first to third row present profiles for $C^{\\rm{solid}}$, $Al^{\\rm{solid}}$, and $S^{\\rm{solid}}$, respectively The considered grid size is 121 $\\times$ 121.}\n\t\\label{fig18}\n\\end{figure}\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure19.png}\n\t\\caption{2D solid amount profiles of the C, Al and S chemical components of cement system 2 after 2 and 6 (final time step) years. RTM means the original HPx$_{\\rm{4C}}$ model and TM+kNN denotes the Hydrus transport model coupled with our kNN geochemical emulator (HPx$_{\\rm py}$-kNN). The first to third row present profiles for $C^{\\rm{solid}}$, $Al^{\\rm{solid}}$, and $S^{\\rm{solid}}$, respectively The considered grid size is 61 $\\times$ 61.}\n\t\\label{fig19}\n\\end{figure}\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure20.png}\n\t\\caption{2D solid amount profiles of the C, Al and S chemical components of cement system 2 after 2 and 6 (final time step) years. RTM means the original HPx$_{\\rm{4C}}$ model and TM+kNN denotes the Hydrus transport model coupled with our kNN geochemical emulator (HPx$_{\\rm py}$-kNN). The first to third row present profiles for $C^{\\rm{solid}}$, $Al^{\\rm{solid}}$, and $S^{\\rm{solid}}$, respectively The considered grid size is 121 $\\times$ 121.}\n\t\\label{fig20}\n\\end{figure}\n\n\\begin{table}[h!]\n\t\\caption{Speedups offered by the KNN and DNN emulators in HPx$_{\\rm py}$ for the reactive transport simulations considered for cement system 2. The HPx$_{\\rm{4C}}$ calculations involve the parallelization of PHREEQC over our 4 CPUs. The HPx$_{\\rm{1C}}$ calculations are performed on a single CPU. Both the kNN and DNN predictions make use of a GPU. ML signifies the used machine learning method for emulation, BC denotes the type of flow boundary conditions and GS is the grid size. The maximum possible speedups associated with HPx$_{\\rm{4C}}$ and HP$_{\\rm{1C}}$,  Max SP HPx$_{\\rm{4C}}$ and  Max SP HPx$_{\\rm{1C}}$, correspond to an hypothetical situation where the geochemical calculations incur zero computational cost.}\n\n\t\\begin{center}\n\t\t\\begin{tabular}{cccccccc}\n\t\t\t\\hline\n\t\t\t& & & & & & & \\\\\n\t\t\tML & BC & GS & HPx$_{\\rm{4C}}$ time (s) & SP HPx$_{\\rm{4C}}$ & Max SP HPx$_{\\rm{4C}}$ & SP HPx$_{\\rm{1C}}$ & Max SP HPx$_{\\rm{1C}}$ \\\\\n\t\t\tDNN & ADV & 61 $\\times$ 61 & 21,415 & 8.2 & 9.0 & 30.3 & 33.1\\\\\n\t\t\tkNN & ADV & 61 $\\times$ 61 & 21,415 & 7.9 & 9.0 & 28.9 & 33.1\\\\\n\t\t\tDNN & ADV & 121 $\\times$ 121 & 199,841 & 8.2 & 9.5 & 29.9 & 32.8\\\\\n\t\t\tkNN & ADV & 121 $\\times$ 121 & 199,841 & 8.5 & 9.5 & 31.3 & 32.8\\\\\n\t\t\t\\hline\n\t\t\\end{tabular}\n\t\\end{center}\n\n\t\\label{table4}\n\\end{table}\n\n\\FloatBarrier\n\n3.2.2 Reactive transport simulation\n\\subsubsection{Reactive transport simulation}\n\\label{prob2res}\nOur ``local\" DNN performs rather well when applied to the 61 $\\times$ 61 grid size and a time period of 6 years (Figures \\ref{fig11} - \\ref{fig12}). This for all components but C, for which some localized deviations appear between original and emulated 2D concentration profiles towards the end of the simulation period (Figure \\ref{fig12}). When applied to the 121 $\\times$ 121 grid, HPx$_{\\rm py}$-DNN produces additional discrepancies for O, Si and Al towards the end of the simulation period (Figures \\ref{fig13} - \\ref{fig14}). Yet most of the observed artifacts could probably be smoothed out by using post-filtering. The associated speedups are listed in Table \\ref{table4}. These speedups are larger to those obtained for cement system 1, with values between 8 and 9 when evaluated against HPx$_{\\rm{4C}}$. These speedups represent about 85 \\% to 90 \\% of the maximum possible speedup (Table \\ref{table3}). Overall, these findings indicate that for the considered problem, our RT-based training of a local DNN only works is the training set is sufficiently representative of the particular geochemical conditions encountered in the computationally demanding simulations, which is arguably not easy to achieve. This limitation is further discussed in section \\ref{discussion}.\n\nWe note a more uniform behavior for HPx$_{\\rm py}$-kNN across grid sizes than for HPx$_{\\rm py}$-DNN. Here the results for the 121  $\\times$ 121 grid are only slightly less accurate than those associated with the 61 $ \\times$ 61 grid (see Figures \\ref{fig15} - \\ref{fig16} where for brevity we only show concentration profiles for the C Al and S components). Furthermore, whenever observed the discrepancies between original and emulated profiles are more regularly scattered than for HPx$_{\\rm py}$-DNN. Note also that herein too, post-filtering could likely smooth out a large part of these deviations. In addition, owing to the use of a GPU to achieve the kNN calculations, the speedup provided by HPx$_{\\rm py}$-kNN are as large as those provided by HPx$_{\\rm py}$-DNN (Table \\ref{table4}).\n\nWith respect to the emulated solid amounts, the HPx$_{\\rm py}$-DNN results look visually good for the the 61 $\\times$ 61 grid. This is shown in Figure \\ref{fig17} for the C, Al and S chemical components, while emulation of the H, O Ca and Si chemical components is globally of similar quality (not shown). Nevertheless, significant deviations appear towards the end of the simulation period for every chemical component (see Figure \\ref{fig18} for the C, Al and S chemical components, emulation of the H, O Ca and Si chemical components shows the same level of mismatch). The HPx$_{\\rm py}$-kNN predictions are also fairly accurate for the 61 $\\times$ 61 (see Figure \\ref{fig19} for the C, Al and S chemical components, emulation of the H, O Ca and Si chemical components exhibits a globally similar quality) while some discrepancies show up at the end of the simulation (Figure \\ref{fig20}). However, the mismatch is less pronounced than for HPx$_{\\rm py}$-DNN. Overall, HPx$_{\\rm py}$-kNN appears to be somewhat more robust than HPx$_{\\rm py}$-DNN for this cement system, while providing the same (large) speedup.\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure11.png}\n\t\\caption{2D concentration profiles obtained for cement system 2 after 2 and 6 (final time step) years. RTM means the original HPx$_{\\rm{4C}}$ model and TM+DNN denotes the Hydrus transport model coupled with our DNN geochemical emulator (HPx$_{\\rm py}$-DNN). The first to fourth row present profiles for $Ca^{\\rm{conc}}$, $Si^{\\rm{conc}}$, $H^{\\rm{conc}}$, and $O^{\\rm{conc}}$, respectively. The considered grid size is 61 $\\times$ 61.}\n\t\\label{fig11}\n\\end{figure}\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure12.png}\n\t\\caption{2D concentration profiles obtained for cement system 2 after 2 and 6 (final time step) years. RTM means the original HPx$_{\\rm{4C}}$ model and TM+DNN denotes the Hydrus transport model coupled with our DNN geochemical emulator (HPx$_{\\rm py}$-DNN). The first to third row present profiles for $C^{\\rm{conc}}$, $Al^{\\rm{conc}}$, and $S^{\\rm{conc}}$, respectively. The considered grid size is 61 $\\times$ 61.}\n\t\\label{fig12}\n\\end{figure}\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure13.png}\n\t\\caption{2D concentration profiles obtained for cement system 2 after 2 and 6 (final time step) years. RTM means the original HPx$_{\\rm{4C}}$ model and TM+DNN denotes the Hydrus transport model coupled with our DNN geochemical emulator (HPx$_{\\rm py}$-DNN). The first to fourth row present profiles for $Ca^{\\rm{conc}}$, $Si^{\\rm{conc}}$, $H^{\\rm{conc}}$, and $O^{\\rm{conc}}$, respectively. The considered grid size is 121 $\\times$ 121.}\n\t\\label{fig13}\n\\end{figure}\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure14.png}\n\t\\caption{2D concentration profiles obtained for cement system 2 after 2 and 6 (final time step) years. RTM means the original HPx$_{\\rm{4C}}$ model and TM+DNN denotes the Hydrus transport model coupled with our DNN geochemical emulator (HPx$_{\\rm py}$-DNN). The first to third row present profiles for $C^{\\rm{conc}}$, $Al^{\\rm{conc}}$, and $S^{\\rm{conc}}$, respectively. The considered grid size is 121 $\\times$ 121.}\n\t\\label{fig14}\n\\end{figure}\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure15.png}\n\t\\caption{2D concentration profiles obtained for cement system 2 after 2 and 6 (final time step) years. RTM means the original HPx$_{\\rm{4C}}$ model and TM+kNN denotes the Hydrus transport model coupled with our kNN geochemical emulator (HPx$_{\\rm py}$-kNN).  The first to third row present profiles for $C^{\\rm{conc}}$, $Al^{\\rm{conc}}$, and $S^{\\rm{conc}}$, respectively. The considered grid size is 61 $\\times$ 61.}\n\t\\label{fig15}\n\\end{figure}\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure16.png}\n\t\\caption{2D concentration profiles obtained for cement system 2 after 2 and 6 (final time step) years. RTM means the original HPx$_{\\rm{4C}}$ model and TM+kNN denotes the Hydrus transport model coupled with our kNN geochemical emulator (HPx$_{\\rm py}$-kNN). The first to third row present profiles for $C^{\\rm{conc}}$, $Al^{\\rm{conc}}$, and $S^{\\rm{conc}}$, respectively. The considered grid size is 121 $\\times$ 121.}\n\t\\label{fig16}\n\\end{figure}\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure17.png}\n\t\\caption{2D solid amount profiles of the C, Al and S chemical components of cement system 2 after 2 and 6 (final time step) years. RTM means the original HPx$_{\\rm{4C}}$ model and TM+DNN denotes the Hydrus transport model coupled with our DNN geochemical emulator (HPx$_{\\rm py}$-DNN). The first to third row present profiles for $C^{\\rm{solid}}$, $Al^{\\rm{solid}}$, and $S^{\\rm{solid}}$, respectively The considered grid size is 61 $\\times$ 61.}\n\t\\label{fig17}\n\\end{figure}\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure18.png}\n\t\\caption{2D solid amount profiles of the C, Al and S chemical components of cement system 2 after 2 and 6 (final time step) years. RTM means the original HPx$_{\\rm{4C}}$ model and TM+DNN denotes the Hydrus transport model coupled with our DNN geochemical emulator (HPx$_{\\rm py}$-DNN). The first to third row present profiles for $C^{\\rm{solid}}$, $Al^{\\rm{solid}}$, and $S^{\\rm{solid}}$, respectively The considered grid size is 121 $\\times$ 121.}\n\t\\label{fig18}\n\\end{figure}\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure19.png}\n\t\\caption{2D solid amount profiles of the C, Al and S chemical components of cement system 2 after 2 and 6 (final time step) years. RTM means the original HPx$_{\\rm{4C}}$ model and TM+kNN denotes the Hydrus transport model coupled with our kNN geochemical emulator (HPx$_{\\rm py}$-kNN). The first to third row present profiles for $C^{\\rm{solid}}$, $Al^{\\rm{solid}}$, and $S^{\\rm{solid}}$, respectively The considered grid size is 61 $\\times$ 61.}\n\t\\label{fig19}\n\\end{figure}\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure20.png}\n\t\\caption{2D solid amount profiles of the C, Al and S chemical components of cement system 2 after 2 and 6 (final time step) years. RTM means the original HPx$_{\\rm{4C}}$ model and TM+kNN denotes the Hydrus transport model coupled with our kNN geochemical emulator (HPx$_{\\rm py}$-kNN). The first to third row present profiles for $C^{\\rm{solid}}$, $Al^{\\rm{solid}}$, and $S^{\\rm{solid}}$, respectively The considered grid size is 121 $\\times$ 121.}\n\t\\label{fig20}\n\\end{figure}\n\n\\begin{table}[h!]\n\t\\caption{Speedups offered by the KNN and DNN emulators in HPx$_{\\rm py}$ for the reactive transport simulations considered for cement system 2. The HPx$_{\\rm{4C}}$ calculations involve the parallelization of PHREEQC over our 4 CPUs. The HPx$_{\\rm{1C}}$ calculations are performed on a single CPU. Both the kNN and DNN predictions make use of a GPU. ML signifies the used machine learning method for emulation, BC denotes the type of flow boundary conditions and GS is the grid size. The maximum possible speedups associated with HPx$_{\\rm{4C}}$ and HP$_{\\rm{1C}}$,  Max SP HPx$_{\\rm{4C}}$ and  Max SP HPx$_{\\rm{1C}}$, correspond to an hypothetical situation where the geochemical calculations incur zero computational cost.}\n\n\t\\begin{center}\n\t\t\\begin{tabular}{cccccccc}\n\t\t\t\\hline\n\t\t\t& & & & & & & \\\\\n\t\t\tML & BC & GS & HPx$_{\\rm{4C}}$ time (s) & SP HPx$_{\\rm{4C}}$ & Max SP HPx$_{\\rm{4C}}$ & SP HPx$_{\\rm{1C}}$ & Max SP HPx$_{\\rm{1C}}$ \\\\\n\t\t\tDNN & ADV & 61 $\\times$ 61 & 21,415 & 8.2 & 9.0 & 30.3 & 33.1\\\\\n\t\t\tkNN & ADV & 61 $\\times$ 61 & 21,415 & 7.9 & 9.0 & 28.9 & 33.1\\\\\n\t\t\tDNN & ADV & 121 $\\times$ 121 & 199,841 & 8.2 & 9.5 & 29.9 & 32.8\\\\\n\t\t\tkNN & ADV & 121 $\\times$ 121 & 199,841 & 8.5 & 9.5 & 31.3 & 32.8\\\\\n\t\t\t\\hline\n\t\t\\end{tabular}\n\t\\end{center}\n\n\t\\label{table4}\n\\end{table}\n\n\\FloatBarrier\n\n",
        "label": "fig17",
        "Descriptive_question1": "Which chemical component is represented in the first row of figure_17?",
        "Descriptive_question2": "What is the grid size used in figure_17?",
        "Reasoning_question1": "How do the solid amount profiles of C, Al, and S in figure_17 compare between the RTM and TM+DNN models after 6 years?",
        "Reasoning_question2": "What might be the reasons for any discrepancies observed between the original RTM and emulated TM+DNN profiles for the chemical components in figure_17 at the final time step?",
        "Descriptive_answer1": "C",
        "Descriptive_answer2": "61 × 61",
        "Reasoning_answer1": "In figure_17, the solid amount profiles of C, Al, and S after 6 years show that TM+DNN profiles generally follow the same spatial patterns as the original RTM profiles, indicating good agreement. However, some differences may be seen locally, but overall the emulated TM+DNN profiles capture the distribution of solid amounts for these components fairly well compared to RTM at the final time step.",
        "Reasoning_answer2": "The observed discrepancies between the original RTM and the emulated TM+DNN solid amount profiles at the final time step likely arise from limitations in the DNN training set representativeness, especially since the cement system 2 requires a local DNN trained on reactive transport-based data. Additionally, complex nonlinear geochemical interactions, potential model overfitting or underfitting, and accumulation of approximation errors over the 6-year simulation period could contribute to localized deviations. The lack of post-filtering smoothing in the raw DNN outputs may also accentuate these differences."
    },
    {
        "paper_id": "2107.07598.json",
        "image_id": "figure_18",
        "image_path": [
            "/home/yz979/scratch/chengye/scimolmo/test-images-tables/dataset_imagestables/2107.07598/images/Figure18.png"
        ],
        "caption": "2D solid amount profiles of the C, Al and S chemical components of cement system 2 after 2 and 6 (final time step) years. RTM means the original HPx$_{\\rm{4C}}$ model and TM+DNN denotes the Hydrus transport model coupled with our DNN geochemical emulator (HPx$_{\\rm py}$-DNN). The first to third row present profiles for $C^{\\rm{solid}}$, $Al^{\\rm{solid}}$, and $S^{\\rm{solid}}$, respectively The considered grid size is 121 $\\times$ 121.",
        "classify": "Chart",
        "section_info": "3 Results\n\\section{Results}\n\\label{results}\n\n\\subsection{Ca-Si Problem}\nFor this first cement system, the emulation problem consists of predicting at each time step of the RT simulation the (output) Ca, Si, H and O aqueous concentrations (mol/kg of water or mol/kgw) from the (input) total amounts of Ca and Si (mol). \n\n\\subsubsection{Training the emulators}\n\\label{train_res1}\nHere the kNN and DNN emulators are firstly trained using a set of 400,000 test examples for both. This training set is obtained by randomly sampling the two-dimensional input space by latin hypercube sampling (LHS) between $\\left[0,0\\right]$ and $\\rm{\\left[Ca^{tot}_{max},Si^{tot}_{max}\\right]}$, and running PHREEQC for each input sample, $\\rm{\\textbf{x}_i = \\left[Ca^{tot}_i, Si^{tot}_i\\right]}$ to get the corresponding output vectors, $\\rm{\\textbf{y}_i = \\left[Ca^{conc}_i,Si^{conc}_i,H^{conc}_i,O^{conc}_i\\right]}$. The upper bounds, $\\rm{Ca^{tot}_{max}}$ and $\\rm{Si^{tot}_{max}}$ are defined based on a cheap full RT simulation with advective-dispersive transport using a small 1D domain of 51 nodes. It is worth noting that the total amounts of $\\textbf{x}_i$, corresponding to the PHREEQC-simulated concentrations, $\\textbf{y}_i$, have to be corrected for the different amount of water between the training set and the transport simulations. Doing so, it turns out that about 20 \\% of the post-corrected $x_i$ values exceed their pre-defined upper bounds and these excessively large values need to be filtered out. Creating the 400,000 training examples thus required about 500,000 PHREEQC runs. As stated earlier, for this problem single-threaded PHREEQC performs about 670 geochemical calculations per second on our used Intel\\textsuperscript{\\textregistered} i7 CPU.\n\nWith respect to kNN, the tuning parameters are the number of neighbors, $k$, the type of distance measure, and the interpolation technique. We simply used the default settings: $k = 5$, euclidean distance and inverse-distance interpolation. Regarding training of the DNN, the 400,000 sample were split between the training set itself (90 \\% of the data) and a validation set (10 \\% of the data). The latter serves to monitor the evolution of the selected mean squared error loss function on samples that are not used for training, thereby detecting potential overfitting. If the validation loss stops decreasing before the fixed number of epochs has been completed, then  training is stopped. Importantly, the emulation is achieved in log-space for both the input, $X$ and output $Y$, domains. This because total amounts and concentrations of the involved components typically cover many orders of magnitudes (up to 10 orders or more). Using a DNN also requires some form of data normalization or standardization. Here both the $\\rm{log\\left(\\textbf{x}_i\\right)}$ and $\\rm{log\\left(\\textbf{y}_i\\right)}$ vectors are standardized around 0 with standard deviation of 1.\n\nFigure \\ref{fig1} illustrates the trained emulators' performance for geochemical predictions using an independent test set that comprises 10,000 test examples. Both kNN (Figures \\ref{fig1}a - d) and DNN (Figures \\ref{fig1}e - h) appear to be rather accurate . DNN also shows a slight degradation for the larger concentration values  (Figures \\ref{fig1}e - h). The latter is likely due to the combination of a small proportion of large concentration values in the training set with the log-transformation that implicitly pushes the DNN to try harder to fit the smaller concentrations during training. Regarding speedup and as written earlier, for this setup the single-threaded kNN method is 300 times faster than single-threaded PHREEQC for predicting the 10,000 concentration vectors all at once. The computational savings allowed by the DNN emulator when ran on our GPU is higher with a speedup as large as 4000 for predicting the same 10,000 concentration vectors all at once. \n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{0cm}\\includegraphics[width=35pc]{Figure1.png}\n\t\\caption{1-1 plots of the kNN (subfigures (a) - (d)) and DNN (subfigures (e) - (h)) emulators' performance obtained for system 1 when the kNN training base contains 400,000 samples and the DNN is trained using the same 400,000 samples. Here ``true\" means the original PHREEQC-simulated data and ``predicted\" denotes the emulated (that is, kNN-simulated and DNN-simulated) data. Hence, the $x$-axis and $y$-axis present the original and emulated 10,000 independent test data points, respectively. The RMSE and $Q_2$ coefficient denote the root-mean-square-error and coefficient of determination in testing mode, respectively, between the original and emulated 10,000 test data points.}\n\t\\label{fig1}\n\\end{figure}\n\nTo test the sensitivity of the emulators' performance to the training set size, training was also performed using reduced training sets comprising 100,000, 10,000, 1000 and 100 samples, respectively. It seen that the DNN performance achieved  when using 10,000 training samples is virtually the same as that obtained when using 400,000 training samples (Figure \\ref{fig2} and Table \\ref{table2}). It is only for training sets smaller than 1000 samples that the DNN performance starts to degrade significantly (Table \\ref{table2}). In contrast, the behavior of kNN appears to markedly decrease as the training set gets smaller (Figure \\ref{fig2} and Table \\ref{table1}).\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{0cm}\\includegraphics[width=35pc]{Figure2.png}\n\t\\caption{1-1 plots of the kNN (subfigures (a) - (d)) and DNN (subfigures (e) - (h)) emulators' performance obtained for system 1 when the kNN training base contains 10,000 samples and the DNN is trained using the same 10,000 samples. Here ``true\" means the original PHREEQC-simulated data and ``predicted\" denotes the emulated (that is, kNN-simulated and DNN-simulated) data. Hence, the $x$-axis and $y$-axis present the original and emulated 10,000 independent test data points, respectively. The RMSE and $Q_2$ coefficient denote the root-mean-square-error and coefficient of determination in testing mode, respectively, between the original and emulated 10,000 test data points.}\n\t\\label{fig2}\n\\end{figure}\n\n\\begin{table}[h!]\n\t\\caption{Performance of the DNN and kNN emulators for cement system 1 and different training set sizes. For brevity, only the results for Ca$^{conc}$ and Si$^{conc}$ are shown. The units are mol per kg of water (mol/kgw). ML refers to the type of emulator, TR signifies the size of the training set, and the RMSE and $Q_2$ coefficient denote the root-mean-square-error and coefficient of determination in testing mode, respectively, between the original and emulated 10,000 test data points.}\n\t\\begin{center}\n\t\t\\begin{tabular}{cccccccc}\n\t\t\t\\hline\n\t\t\t& & & & & \\\\\n\t\t\tML & TR & RMSE - Ca$^{conc}$ & $Q_2$ - Ca$^{conc}$ & RMSE - Si$^{conc}$ & $Q_2$ - Si$^{conc}$\\\\\n\t\t\tDNN & 4 $\\times$ 10$^{5}$ & 4.50 $\\times$ 10$^{-5}$ & 0.9999 & 1.21 $\\times$ 10$^{-6}$ & 0.9999\\\\\n\t\t\tDNN & 1 $\\times$ 10$^{5}$ & 5.06 $\\times$ 10$^{-5}$ & 0.9999 & 1.30 $\\times$ 10$^{-6}$ & 0.9999\\\\\n\t\t\tDNN & 1 $\\times$ 10$^{4}$ & 4.57 $\\times$ 10$^{-5}$ & 0.9999 & 1.48 $\\times$ 10$^{-6}$ & 0.9999\\\\\n\t\t\tDNN & 1 $\\times$ 10$^{3}$ & 8.02 $\\times$ 10$^{-5}$ & 0.9998 & 3.31 $\\times$ 10$^{-6}$ & 0.9994\\\\\n\t\t\tDNN & 1 $\\times$ 10$^{2}$ & 47.1 $\\times$ 10$^{-5}$ & 0.9935 & 9.02 $\\times$ 10$^{-6}$ & 0.9959\\\\\n\t\t\tkNN & 4 $\\times$ 10$^{5}$ & 2.52 $\\times$ 10$^{-5}$ & 1.0000 & 1.59 $\\times$ 10$^{-6}$ & 0.9999\\\\\n\t\t\tkNN & 1 $\\times$ 10$^{5}$ & 6.27 $\\times$ 10$^{-5}$ & 0.9999 & 4.20 $\\times$ 10$^{-6}$ & 0.9991\\\\\n\t\t\tkNN & 1 $\\times$ 10$^{4}$ & 14.1 $\\times$ 10$^{-5}$ & 0.9994 & 13.5 $\\times$ 10$^{-6}$ & 0.9908\\\\\n\t\t\tkNN & 1 $\\times$ 10$^{3}$ & 43.9 $\\times$ 10$^{-5}$ & 0.9944 & 74.6 $\\times$ 10$^{-6}$ & 0.7180\\\\\n\t\t\tkNN & 1 $\\times$ 10$^{2}$  & 96.4 $\\times$ 10$^{-5}$ & 0.9729 & 55.0 $\\times$ 10$^{-6}$ & 0.8468\\\\\n\t\t\\end{tabular}\n\t\\end{center}\n\t\\label{table2}\n\\end{table}\n\n\\FloatBarrier\n\n\\subsubsection{Reactive transport simulation}\n\\label{prob12Dres}\n\nThis section focuses on reactive transport simulations with HPx$_{\\rm py}$ within cement system 1, under both advective-dispersive and diffusive transport conditions. As written above, the domain sizes are both 61 $\\times$ 61 and 121 $\\times$ 121 for the advection-dispersion case and, because of computational constraints, solely 61 $\\times$ 61  for the diffusion case. In addition, the simulation time period is 2 years for the advection-dispersion case and 1 year for the diffusion case. Figures \\ref{fig3} and \\ref{fig4} present times series of original and emulated Ca, Si, H and O concentrations at 5 locations within the 2D domain for advective-dispersive transport conditions, for both our kNN-based (HPx$_{\\rm py}$-kNN) and DNN-based (HPx$_{\\rm py}$-DNN) reactive transport codes. It is seen that HPx$_{\\rm py}$-kNN and HPx$_{\\rm py}$-DNN both induce a quite good simulation accuracy. Also, the results for the diffusive transport case are of similarly good quality (not shown). Figures \\ref{fig5} - \\ref{fig6} provide more insights into the HPx$_{\\rm py}$-kNN and HPx$_{\\rm py}$-DNN performances by displaying 2D Ca, Si, H and O concentration profiles at a given time. For each experiment and chemical component, this time is selected as to be well representative of the simulated dynamics. It is observed that the original and emulated images are visually almost indistinguishable for the advection-dispersion case (Figure \\ref{fig5}). For the diffusion case, the emulators also perform quite well for Ca, H and O (Figures\\ref{fig6}a - c, g - i and j - l), while some slight to moderate discrepancies appear at the concentration front for Si (Figures \\ref{fig6}d - f). Notwithstanding, the Si concentration remains globally well predicted. Furthermore, Figures \\ref{fig7} - \\ref{fig8} present the original and emulated 2D solid amount profiles corresponding to Figures \\ref{fig5} - \\ref{fig6}. The original solid amount profiles are overall well approximated by HPx$_{\\rm py}$-kNN and HPx$_{\\rm py}$-DNN for the advection-dispersion case (Figure \\ref{fig7}), even though some mismatch appears at the border of the fully depleted zone for the H component. As of the diffusion case (Figure \\ref{fig8}), the same kind of mismatch is observed for the emulated solid amounts of H by HPx$_{\\rm py}$-kNN while the emulated profiles by HPx$_{\\rm py}$-DNN show some more discrepancies. Though we decided to present raw emulation results, we would like to stress that some if not all of the observed artifacts could likely be smoothed out by using some post-filtering such as median filtering.\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure3.png}\n\t\\caption{Time series of original (RTM, solid green lines) and HPx$_{\\rm py}$-kNN emulated (TM+kNN, dashed orange lines) concentrations (mol/kg) of Ca, H, O and Si at selected observations points for cement system 1 and advective-dispersive transport. Obs. 1 - 5 denote the selected observation points, with the following $\\left[x,y\\right]$ locations (in cm). Obs. 1:  $\\left[0.5, 2.5\\right]$, Obs. 2: $\\left[1, 2\\right]$, Obs. 3: $\\left[2, 2\\right]$, Obs. 4: $\\left[1, 1\\right]$, Obs. 5: $\\left[2, 1\\right]$. The results for the 121 $\\times$ 121 grid size are rather similar.}\n\t\\label{fig3}\n\\end{figure}\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure4.png}\n\t\\caption{Time series of original (RTM, solid green lines) and HPx$_{\\rm py}$-DNN emulated (TM+DNN, dashed orange lines) concentrations (mol/kg) of Ca, H, O and Si at selected observations points for cement system 1 and advective-dispersive transport. Obs. 1 - 5 denote the selected observation points, with the following $\\left[x,y\\right]$ locations (in cm). Obs. 1:  $\\left[0.5, 2.5\\right]$, Obs. 2: $\\left[1, 2\\right]$, Obs. 3: $\\left[2, 2\\right]$, Obs. 4: $\\left[1, 1\\right]$, Obs. 5: $\\left[2, 1\\right]$. The results for the 121 $\\times$ 121 grid size are rather similar.}\n\t\\label{fig4}\n\\end{figure}\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure5.png}\n\t\\caption{2D concentration profiles obtained for cement system 1 at the end of the 2-year simulation performed for the advection-dispersion case. RTM means the original HP$_{\\rm{4C}}$ model, TM+kNN denotes the Hydrus transport model coupled with our kNN geochemical emulator (HPx$_{\\rm py}$-kNN) and TM+DNN signifies the Hydrus transport model coupled with our DNN geochemical emulator (HPx$_{\\rm py}$-DNN). The first to fourth row present profiles for $Ca^{\\rm{conc}}$, $Si^{\\rm{conc}}$, $H^{\\rm{conc}}$, and $O^{\\rm{conc}}$, respectively. The considered grid size is 61 $\\times$ 61. The results for the 121 $\\times$ 121 grid are rather similar.}\n\t\\label{fig5}\n\\end{figure}\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure6.png}\n\t\\caption{2D concentration profiles obtained for cement system 1 at the final time step of the 1-year simulation performed for the diffusion case. RTM means the original HP$_{\\rm{4C}}$ model, TM+kNN denotes the Hydrus transport model coupled with our kNN geochemical emulator (HPx$_{\\rm py}$-kNN) and TM+DNN signifies the Hydrus transport model coupled with our DNN geochemical emulator (HPx$_{\\rm py}$-DNN). The first to fourth row present profiles for $Ca^{\\rm{conc}}$, $Si^{\\rm{conc}}$, $H^{\\rm{conc}}$, and $O^{\\rm{conc}}$, respectively. The considered grid size is 61 $\\times$ 61.}\n\t\\label{fig6}\n\\end{figure}\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure7.png}\n\t\\caption{2D solid amount profiles obtained for cement system 1 at the end of the 2-year simulation performed for the advection-dispersion case. RTM means the original HP$_{\\rm{4C}}$ model, TM+kNN denotes the Hydrus transport model coupled with our kNN geochemical emulator (HPx$_{\\rm py}$-kNN) and TM+DNN signifies the Hydrus transport model coupled with our DNN geochemical emulator (HPx$_{\\rm py}$-DNN). The first to fourth row present profiles for $Ca^{\\rm{solid}}$, $Si^{\\rm{solid}}$, $H^{\\rm{solid}}$, and $O^{\\rm{solid}}$, respectively. The considered grid size is 61 $\\times$ 61. The results for the 121 $\\times$ 121 grid are rather similar.}\n\t\\label{fig7}\n\\end{figure}\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure8.png}\n\t\\caption{2D solid amount profiles obtained for cement system 1 at the final time step of the 1-year simulation performed for the diffusion case. RTM means the original HP$_{\\rm{4C}}$ model, TM+kNN denotes the Hydrus transport model coupled with our kNN geochemical emulator (HPx$_{\\rm py}$-kNN) and TM+DNN signifies the Hydrus transport model coupled with our DNN geochemical emulator (HPx$_{\\rm py}$-DNN). The first to fourth row present profiles for $Ca^{\\rm{solid}}$, $Si^{\\rm{solid}}$, $H^{\\rm{solid}}$, and $O^{\\rm{solid}}$, respectively. The considered grid size is 61 $\\times$ 61.}\n\t\\label{fig8}\n\\end{figure}\n\nThe speedups associated with the considered problem are detailed in Table \\ref{table3}. It is noted that the GPU-based DNN emulator allows for a speedup that is close to optimal. Indeed, The DNN speedups overall represent 85 \\% to 95 \\% of the maximum possible speedups (that is, speedups that would be obtained if the geochemcical calculations would come at no cost at all). The speedups associated with single-threaded kNN remain substantial but only amount to 57 \\% - 65 \\% of the corresponding maximum speedups. As detailed in section \\ref{train_res1}, the used kNN and DNN implementations are found to be respectively 300 and 4000 faster than single-threaded PHREEQC when predicting 10,000 points all one once for this geochemical system. Based on these numbers one could have expected the achieved speedups to represent say 90 \\% (kNN) or 99 \\% (DNN) of the maximum possible ones. A large part of the gaps between achieved and maximum possible speedups is thus likely caused by the time required for communicating and exchanging data between the main C/C++ code and the Python-based emulators.\n\n\\begin{table}[h!]\n\t\\caption{Speedups offered by the KNN and DNN emulators in HPx$_{\\rm py}$ for the reactive transport simulations considered for cement system 1. The HPx$_{\\rm{4C}}$ calculations involve the parallelization of PHREEQC over our 4 CPUs. The HPx$_{\\rm{1C}}$ calculations are performed on a single CPU. The kNN predictions are performed on a single CPU using the scikit-learn implementation while the DNN predictions make use of our GPU. ML signifies the used machine learning method for emulation, TC denotes transport conditions (ADV: advection-dispersion, DIF: diffusion) and GS is the grid size. The maximum possible speedups associated with HPx$_{\\rm{4C}}$ and HPx$_{\\rm{1C}}$,  Max SP HPx$_{\\rm{4C}}$ and  Max SP HPx$_{\\rm{1C}}$, correspond to an hypothetical situation where the geochemical calculations incur zero computational cost.}\n\n\t\\begin{center}\n\t\t\\begin{tabular}{cccccccc}\n\t\t\t\\hline\n\t\t\t& & & & & & & \\\\\n\t\t\tML & TC & GS & HPx$_{\\rm{4C}}$ time (s) & SP HPx$_{\\rm{4C}}$ & Max SP HPx$_{\\rm{4C}}$ & SP HPx$_{\\rm{1C}}$ & Max SP HPx$_{\\rm{1C}}$ \\\\\n\t\t\tDNN & ADV & 61 $\\times$ 61 & 3189 & 6.8 & 7.7 & 24.5 & 28.5 \\\\\n\t\t\tkNN & ADV & 61 $\\times$ 61 & 3189 & 5.0 & 7.7 & 18.0 & 28.5 \\\\\n\t\t\tDNN & ADV & 121 $\\times$ 121 & 23,337 & 5.0 & 5.2 & 17.0 & 18.0 \\\\\n\t\t\tkNN & ADV & 121 $\\times$ 121 & 23,337 & 3.4 & 5.2 & 11.6 & 18.0 \\\\\n\t\t\tDNN & DIF & 61 $\\times$ 61 & 25,448 & 4.2 & 4.9 & 16.2 & 19.1 \\\\\n\t\t\tkNN & DIF & 61 $\\times$ 61 & 25,448 & 2.8 & 4.9 & 10.8 & 19.1 \\\\\n\t\t\t\\hline\n\t\t\\end{tabular}\n\t\\end{center}\n\n\t\\label{table3}\n\\end{table}\n\n\\FloatBarrier\n\n\\subsection{Al-C-Ca-S-Si Problem}\n\nFor this second cement system, the emulation problem consists of predicting at each time step of the RT simulation the (output) Al, C, Ca, S, Si, H and O aqueous concentrations (mol/kgw) from the (input) total amounts of Al, C, Ca, S and Si (mol). Here we focus on advective-dispersive transport only while similarly as for cement system 1, the considered domain sizes are 61 $\\times$ 61 and 121 $\\times$ 121. Moreover, the simulation time period is set to 6 years. \n\nAs mentioned earlier, for this higher-dimensional problem it is observed that the used scikit-learn kNN implementation becomes prohibitively slow compared to HPx$_{\\rm{4C}}$. We found that to get a good emulation accuracy the kNN training base needs to contain 1,000,000 samples (or more). This training base's size together with a 5-dimensional search space leads to an HPx$_{\\rm py}$-kNN reactive transport simulation time that is comparable to that of HPx$_{\\rm{4C}}$. Therefore, for this second cement system we built a custom kNN regressor around another kNN implementation contained in the FAISS package \\citep{faiss2017}. Our used FAISS variant allows for GPU computing and is much faster than scikit-learn for this cement system, but is slightly less accurate due to the use of an approximate rather than exact nearest neighbor search \\citep[see][for details]{faiss2017}.\n\n\\subsubsection{Training the DNN}\n\\label{train_res2}\n\nBuilding a good training set to perform a kNN search and learn the weights and biases of our DNN turned out to be a complicated task in this case. This because to make useful kNN predictions and/or learn an useful DNN, the training set must be sufficiently representative of the geochemical conditions encountered during the reactive transport simulation one wish to perform with HPx$_{\\rm py}$-kNN and the trained HPx$_{\\rm py}$-DNN. In contrast to cement system 1, creating the training set by sampling the $X$-space with a controlled randomness between predefined lower and upper bounds did not prove successful. We tried that strategy by drawing as much as 4,000,000 5-dimensional $\\textbf{x}$ vectors from the $X$-space using a Sobol low-discrepancy sequence \\citep[][]{Sobol1967, Joe-Kuo2003}. Such low-discrepancy sampling scheme covers the 5-dimensional hypercube more uniformly than LHS. Despite a good performance on the test set (not shown), the resulting DNN accuracy in reactive transport mode was never deemed satisfying. In other words, no satisfying ``global\" or ``universal\" DNN emulator could be devised for this cement system. This is probably caused by the fact for this problem, the input (5 total amounts) and output (7 aqueous concentrations) spaces are quite nonlinearly related and both cover 6 to 10 orders of magnitudes depending on the considered element. Therefore, we resorted to the alternative training strategy detailed below. The latter basically tries to grasp the complex correlations and higher-order dependencies that exist between the elements of $\\textbf{x}$ (total amounts, input space) for a given reactive transport simulation setup, in order to produce a training set that honors these between-input relationships.\n\n\\begin{itemize}\n\t\\item Perform a ``cheap\" full reactive transport simulation under the transport conditions and geochemistry of interest and collect the resulting $\\textbf{x}$ -$\\textbf{y}$  pairs of examples (for the considered grid nodes and time steps). Computational demand controls what domain size and simulation time period can be used for this cheap calculation. We used a modest 16 $\\times$ 16 domain and a simulation time period of 10 years. The associated HPx$_{\\rm{4C}}$ runtimes is 180 s.\n\t\n\t\\item Fit a kernel density estimator (KDE) with a Gaussian kernel to the collected $\\textbf{x}$ vectors (encapsulated in the $\\textbf{X}$ array) and generate a fixed number of new input vectors, $\\textbf{x}_{KDE}$. Then run PHREEQC for the $\\textbf{X}_{KDE}$ set to get the corresponding output set, $\\textbf{Y}_{KDE}$. Now apply the correction for porosity described in section \\ref{train_res1} to the $\\textbf{X}_{KDE}$ set and form the training set by merging the ensemble of $\\textbf{x}$-$\\textbf{y}$ pairs with that of the $\\textbf{x}_{KDE}$-$\\textbf{y}_{KDE}$ pairs. The number of produced unique examples by the considered cheap HPx$_{\\rm{4C}}$ simulations varied between 10,000 and 50,000. The KDE-based enrichment of this dataset was deemed necessary to provide more input variability, thereby avoiding overfitting of the trained DNN and improving the kNN accuracy, while still honoring the complex between-input relationships. The number of KDE-generated samples was set as to obtain a total training set size of 1,000,000 examples. A key component of the approach is the bandwith parameter of the Gaussian KDE kernel which controls how much the KDE-generated samples depart from the original ensemble. After limited trial and error, we fixed the kernel bandwith to 0.0025 for the considered case studies.\n\t\n\\end{itemize}\n\nThe scatter plots in Figure \\ref{fig9} illustrate our training set creation procedure. The orange dots depict the pairwise relationships observed between the 5 elements of $\\textbf{x}$ in the cheap simulation. The red and cyan dots in Figure \\ref{fig9} represent the KDE-generated samples with the selected bandwidth, before and after applying the correction for porosity, respectively. Training of the DNN is achieved using the ensemble of original and KDE-corrected input points. We refer to this kind of dataset as RT-based, since it is based on an albeit cheap, full RT simulation. Furthermore, we refer to the obtained DNN and kNN emulators as ``local\" emulators, since as opposed to the emulators constructed for cement system 1, the current emulators are only valid for the input conditions encapsulated in the RT-based training set.\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1.5cm}\\includegraphics[width=50pc]{Figure9.png}\n\t\\caption{Scatter plots of the complex relationships between the five considered inputs for (1) the computationally cheap RT simulation of cement system 2 (orange dots) performed with the original HPx code, (2) the corresponding sampled points by kernel density estimation (KDE) using the selected bandwith (turquoise dots) and, (3) the same sampled KDE points after porosity correction (red dots).}\n\t\\label{fig9}\n\\end{figure}\n\nTraining performance of the DNN emulator is presented in Figure \\ref{fig10}. Similarly as for cement system 1, about 90\\% of the available data was used for the actual training of the DNN while the remaining 10\\% were used a validation set to contol overfitting. Lastly, performance is evaluated for both the trained DNN and kNN emulators using an independent test set of 10,000 examples. Overall, the accuracy of our ``local \" DNN emulator for this RT-inspired dataset is rather large with $Q_2$ values always greater or equal than 0.998. Training performance of the corresponding local kNN emulator is equally good (not shown). Regarding speedup, for this problem single-threaded PHREEQC achieves about 210 geochemical calculations per second on our used Intel\\textsuperscript{\\textregistered} i7 CPU while the GPU-based DNN and kNN emulators are both about 3000 times faster when predicting the 10,000 test points at once .\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure10.png}\n\t\\caption{1-1 plots of local DNN emulation performance obtained for system 2 when the local DNN is trained using 1,000,000 samples. The $x$-axis and $y$-axis present the original and emulated 10,000 independent test data points, respectively. The RMSE and $Q_2$ coefficient denote the root-mean-square-error and coefficient of determination in testing mode, respectively, between the original and emulated 10,000 test data points.}\n\t\\label{fig10}\n\\end{figure}\n\n\\subsubsection{Reactive transport simulation}\n\\label{prob2res}\nOur ``local\" DNN performs rather well when applied to the 61 $\\times$ 61 grid size and a time period of 6 years (Figures \\ref{fig11} - \\ref{fig12}). This for all components but C, for which some localized deviations appear between original and emulated 2D concentration profiles towards the end of the simulation period (Figure \\ref{fig12}). When applied to the 121 $\\times$ 121 grid, HPx$_{\\rm py}$-DNN produces additional discrepancies for O, Si and Al towards the end of the simulation period (Figures \\ref{fig13} - \\ref{fig14}). Yet most of the observed artifacts could probably be smoothed out by using post-filtering. The associated speedups are listed in Table \\ref{table4}. These speedups are larger to those obtained for cement system 1, with values between 8 and 9 when evaluated against HPx$_{\\rm{4C}}$. These speedups represent about 85 \\% to 90 \\% of the maximum possible speedup (Table \\ref{table3}). Overall, these findings indicate that for the considered problem, our RT-based training of a local DNN only works is the training set is sufficiently representative of the particular geochemical conditions encountered in the computationally demanding simulations, which is arguably not easy to achieve. This limitation is further discussed in section \\ref{discussion}.\n\nWe note a more uniform behavior for HPx$_{\\rm py}$-kNN across grid sizes than for HPx$_{\\rm py}$-DNN. Here the results for the 121  $\\times$ 121 grid are only slightly less accurate than those associated with the 61 $ \\times$ 61 grid (see Figures \\ref{fig15} - \\ref{fig16} where for brevity we only show concentration profiles for the C Al and S components). Furthermore, whenever observed the discrepancies between original and emulated profiles are more regularly scattered than for HPx$_{\\rm py}$-DNN. Note also that herein too, post-filtering could likely smooth out a large part of these deviations. In addition, owing to the use of a GPU to achieve the kNN calculations, the speedup provided by HPx$_{\\rm py}$-kNN are as large as those provided by HPx$_{\\rm py}$-DNN (Table \\ref{table4}).\n\nWith respect to the emulated solid amounts, the HPx$_{\\rm py}$-DNN results look visually good for the the 61 $\\times$ 61 grid. This is shown in Figure \\ref{fig17} for the C, Al and S chemical components, while emulation of the H, O Ca and Si chemical components is globally of similar quality (not shown). Nevertheless, significant deviations appear towards the end of the simulation period for every chemical component (see Figure \\ref{fig18} for the C, Al and S chemical components, emulation of the H, O Ca and Si chemical components shows the same level of mismatch). The HPx$_{\\rm py}$-kNN predictions are also fairly accurate for the 61 $\\times$ 61 (see Figure \\ref{fig19} for the C, Al and S chemical components, emulation of the H, O Ca and Si chemical components exhibits a globally similar quality) while some discrepancies show up at the end of the simulation (Figure \\ref{fig20}). However, the mismatch is less pronounced than for HPx$_{\\rm py}$-DNN. Overall, HPx$_{\\rm py}$-kNN appears to be somewhat more robust than HPx$_{\\rm py}$-DNN for this cement system, while providing the same (large) speedup.\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure11.png}\n\t\\caption{2D concentration profiles obtained for cement system 2 after 2 and 6 (final time step) years. RTM means the original HPx$_{\\rm{4C}}$ model and TM+DNN denotes the Hydrus transport model coupled with our DNN geochemical emulator (HPx$_{\\rm py}$-DNN). The first to fourth row present profiles for $Ca^{\\rm{conc}}$, $Si^{\\rm{conc}}$, $H^{\\rm{conc}}$, and $O^{\\rm{conc}}$, respectively. The considered grid size is 61 $\\times$ 61.}\n\t\\label{fig11}\n\\end{figure}\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure12.png}\n\t\\caption{2D concentration profiles obtained for cement system 2 after 2 and 6 (final time step) years. RTM means the original HPx$_{\\rm{4C}}$ model and TM+DNN denotes the Hydrus transport model coupled with our DNN geochemical emulator (HPx$_{\\rm py}$-DNN). The first to third row present profiles for $C^{\\rm{conc}}$, $Al^{\\rm{conc}}$, and $S^{\\rm{conc}}$, respectively. The considered grid size is 61 $\\times$ 61.}\n\t\\label{fig12}\n\\end{figure}\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure13.png}\n\t\\caption{2D concentration profiles obtained for cement system 2 after 2 and 6 (final time step) years. RTM means the original HPx$_{\\rm{4C}}$ model and TM+DNN denotes the Hydrus transport model coupled with our DNN geochemical emulator (HPx$_{\\rm py}$-DNN). The first to fourth row present profiles for $Ca^{\\rm{conc}}$, $Si^{\\rm{conc}}$, $H^{\\rm{conc}}$, and $O^{\\rm{conc}}$, respectively. The considered grid size is 121 $\\times$ 121.}\n\t\\label{fig13}\n\\end{figure}\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure14.png}\n\t\\caption{2D concentration profiles obtained for cement system 2 after 2 and 6 (final time step) years. RTM means the original HPx$_{\\rm{4C}}$ model and TM+DNN denotes the Hydrus transport model coupled with our DNN geochemical emulator (HPx$_{\\rm py}$-DNN). The first to third row present profiles for $C^{\\rm{conc}}$, $Al^{\\rm{conc}}$, and $S^{\\rm{conc}}$, respectively. The considered grid size is 121 $\\times$ 121.}\n\t\\label{fig14}\n\\end{figure}\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure15.png}\n\t\\caption{2D concentration profiles obtained for cement system 2 after 2 and 6 (final time step) years. RTM means the original HPx$_{\\rm{4C}}$ model and TM+kNN denotes the Hydrus transport model coupled with our kNN geochemical emulator (HPx$_{\\rm py}$-kNN).  The first to third row present profiles for $C^{\\rm{conc}}$, $Al^{\\rm{conc}}$, and $S^{\\rm{conc}}$, respectively. The considered grid size is 61 $\\times$ 61.}\n\t\\label{fig15}\n\\end{figure}\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure16.png}\n\t\\caption{2D concentration profiles obtained for cement system 2 after 2 and 6 (final time step) years. RTM means the original HPx$_{\\rm{4C}}$ model and TM+kNN denotes the Hydrus transport model coupled with our kNN geochemical emulator (HPx$_{\\rm py}$-kNN). The first to third row present profiles for $C^{\\rm{conc}}$, $Al^{\\rm{conc}}$, and $S^{\\rm{conc}}$, respectively. The considered grid size is 121 $\\times$ 121.}\n\t\\label{fig16}\n\\end{figure}\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure17.png}\n\t\\caption{2D solid amount profiles of the C, Al and S chemical components of cement system 2 after 2 and 6 (final time step) years. RTM means the original HPx$_{\\rm{4C}}$ model and TM+DNN denotes the Hydrus transport model coupled with our DNN geochemical emulator (HPx$_{\\rm py}$-DNN). The first to third row present profiles for $C^{\\rm{solid}}$, $Al^{\\rm{solid}}$, and $S^{\\rm{solid}}$, respectively The considered grid size is 61 $\\times$ 61.}\n\t\\label{fig17}\n\\end{figure}\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure18.png}\n\t\\caption{2D solid amount profiles of the C, Al and S chemical components of cement system 2 after 2 and 6 (final time step) years. RTM means the original HPx$_{\\rm{4C}}$ model and TM+DNN denotes the Hydrus transport model coupled with our DNN geochemical emulator (HPx$_{\\rm py}$-DNN). The first to third row present profiles for $C^{\\rm{solid}}$, $Al^{\\rm{solid}}$, and $S^{\\rm{solid}}$, respectively The considered grid size is 121 $\\times$ 121.}\n\t\\label{fig18}\n\\end{figure}\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure19.png}\n\t\\caption{2D solid amount profiles of the C, Al and S chemical components of cement system 2 after 2 and 6 (final time step) years. RTM means the original HPx$_{\\rm{4C}}$ model and TM+kNN denotes the Hydrus transport model coupled with our kNN geochemical emulator (HPx$_{\\rm py}$-kNN). The first to third row present profiles for $C^{\\rm{solid}}$, $Al^{\\rm{solid}}$, and $S^{\\rm{solid}}$, respectively The considered grid size is 61 $\\times$ 61.}\n\t\\label{fig19}\n\\end{figure}\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure20.png}\n\t\\caption{2D solid amount profiles of the C, Al and S chemical components of cement system 2 after 2 and 6 (final time step) years. RTM means the original HPx$_{\\rm{4C}}$ model and TM+kNN denotes the Hydrus transport model coupled with our kNN geochemical emulator (HPx$_{\\rm py}$-kNN). The first to third row present profiles for $C^{\\rm{solid}}$, $Al^{\\rm{solid}}$, and $S^{\\rm{solid}}$, respectively The considered grid size is 121 $\\times$ 121.}\n\t\\label{fig20}\n\\end{figure}\n\n\\begin{table}[h!]\n\t\\caption{Speedups offered by the KNN and DNN emulators in HPx$_{\\rm py}$ for the reactive transport simulations considered for cement system 2. The HPx$_{\\rm{4C}}$ calculations involve the parallelization of PHREEQC over our 4 CPUs. The HPx$_{\\rm{1C}}$ calculations are performed on a single CPU. Both the kNN and DNN predictions make use of a GPU. ML signifies the used machine learning method for emulation, BC denotes the type of flow boundary conditions and GS is the grid size. The maximum possible speedups associated with HPx$_{\\rm{4C}}$ and HP$_{\\rm{1C}}$,  Max SP HPx$_{\\rm{4C}}$ and  Max SP HPx$_{\\rm{1C}}$, correspond to an hypothetical situation where the geochemical calculations incur zero computational cost.}\n\n\t\\begin{center}\n\t\t\\begin{tabular}{cccccccc}\n\t\t\t\\hline\n\t\t\t& & & & & & & \\\\\n\t\t\tML & BC & GS & HPx$_{\\rm{4C}}$ time (s) & SP HPx$_{\\rm{4C}}$ & Max SP HPx$_{\\rm{4C}}$ & SP HPx$_{\\rm{1C}}$ & Max SP HPx$_{\\rm{1C}}$ \\\\\n\t\t\tDNN & ADV & 61 $\\times$ 61 & 21,415 & 8.2 & 9.0 & 30.3 & 33.1\\\\\n\t\t\tkNN & ADV & 61 $\\times$ 61 & 21,415 & 7.9 & 9.0 & 28.9 & 33.1\\\\\n\t\t\tDNN & ADV & 121 $\\times$ 121 & 199,841 & 8.2 & 9.5 & 29.9 & 32.8\\\\\n\t\t\tkNN & ADV & 121 $\\times$ 121 & 199,841 & 8.5 & 9.5 & 31.3 & 32.8\\\\\n\t\t\t\\hline\n\t\t\\end{tabular}\n\t\\end{center}\n\n\t\\label{table4}\n\\end{table}\n\n\\FloatBarrier\n\n3.2 Al-C-Ca-S-Si Problem\n\\subsection{Al-C-Ca-S-Si Problem}\n\nFor this second cement system, the emulation problem consists of predicting at each time step of the RT simulation the (output) Al, C, Ca, S, Si, H and O aqueous concentrations (mol/kgw) from the (input) total amounts of Al, C, Ca, S and Si (mol). Here we focus on advective-dispersive transport only while similarly as for cement system 1, the considered domain sizes are 61 $\\times$ 61 and 121 $\\times$ 121. Moreover, the simulation time period is set to 6 years. \n\nAs mentioned earlier, for this higher-dimensional problem it is observed that the used scikit-learn kNN implementation becomes prohibitively slow compared to HPx$_{\\rm{4C}}$. We found that to get a good emulation accuracy the kNN training base needs to contain 1,000,000 samples (or more). This training base's size together with a 5-dimensional search space leads to an HPx$_{\\rm py}$-kNN reactive transport simulation time that is comparable to that of HPx$_{\\rm{4C}}$. Therefore, for this second cement system we built a custom kNN regressor around another kNN implementation contained in the FAISS package \\citep{faiss2017}. Our used FAISS variant allows for GPU computing and is much faster than scikit-learn for this cement system, but is slightly less accurate due to the use of an approximate rather than exact nearest neighbor search \\citep[see][for details]{faiss2017}.\n\n\\subsubsection{Training the DNN}\n\\label{train_res2}\n\nBuilding a good training set to perform a kNN search and learn the weights and biases of our DNN turned out to be a complicated task in this case. This because to make useful kNN predictions and/or learn an useful DNN, the training set must be sufficiently representative of the geochemical conditions encountered during the reactive transport simulation one wish to perform with HPx$_{\\rm py}$-kNN and the trained HPx$_{\\rm py}$-DNN. In contrast to cement system 1, creating the training set by sampling the $X$-space with a controlled randomness between predefined lower and upper bounds did not prove successful. We tried that strategy by drawing as much as 4,000,000 5-dimensional $\\textbf{x}$ vectors from the $X$-space using a Sobol low-discrepancy sequence \\citep[][]{Sobol1967, Joe-Kuo2003}. Such low-discrepancy sampling scheme covers the 5-dimensional hypercube more uniformly than LHS. Despite a good performance on the test set (not shown), the resulting DNN accuracy in reactive transport mode was never deemed satisfying. In other words, no satisfying ``global\" or ``universal\" DNN emulator could be devised for this cement system. This is probably caused by the fact for this problem, the input (5 total amounts) and output (7 aqueous concentrations) spaces are quite nonlinearly related and both cover 6 to 10 orders of magnitudes depending on the considered element. Therefore, we resorted to the alternative training strategy detailed below. The latter basically tries to grasp the complex correlations and higher-order dependencies that exist between the elements of $\\textbf{x}$ (total amounts, input space) for a given reactive transport simulation setup, in order to produce a training set that honors these between-input relationships.\n\n\\begin{itemize}\n\t\\item Perform a ``cheap\" full reactive transport simulation under the transport conditions and geochemistry of interest and collect the resulting $\\textbf{x}$ -$\\textbf{y}$  pairs of examples (for the considered grid nodes and time steps). Computational demand controls what domain size and simulation time period can be used for this cheap calculation. We used a modest 16 $\\times$ 16 domain and a simulation time period of 10 years. The associated HPx$_{\\rm{4C}}$ runtimes is 180 s.\n\t\n\t\\item Fit a kernel density estimator (KDE) with a Gaussian kernel to the collected $\\textbf{x}$ vectors (encapsulated in the $\\textbf{X}$ array) and generate a fixed number of new input vectors, $\\textbf{x}_{KDE}$. Then run PHREEQC for the $\\textbf{X}_{KDE}$ set to get the corresponding output set, $\\textbf{Y}_{KDE}$. Now apply the correction for porosity described in section \\ref{train_res1} to the $\\textbf{X}_{KDE}$ set and form the training set by merging the ensemble of $\\textbf{x}$-$\\textbf{y}$ pairs with that of the $\\textbf{x}_{KDE}$-$\\textbf{y}_{KDE}$ pairs. The number of produced unique examples by the considered cheap HPx$_{\\rm{4C}}$ simulations varied between 10,000 and 50,000. The KDE-based enrichment of this dataset was deemed necessary to provide more input variability, thereby avoiding overfitting of the trained DNN and improving the kNN accuracy, while still honoring the complex between-input relationships. The number of KDE-generated samples was set as to obtain a total training set size of 1,000,000 examples. A key component of the approach is the bandwith parameter of the Gaussian KDE kernel which controls how much the KDE-generated samples depart from the original ensemble. After limited trial and error, we fixed the kernel bandwith to 0.0025 for the considered case studies.\n\t\n\\end{itemize}\n\nThe scatter plots in Figure \\ref{fig9} illustrate our training set creation procedure. The orange dots depict the pairwise relationships observed between the 5 elements of $\\textbf{x}$ in the cheap simulation. The red and cyan dots in Figure \\ref{fig9} represent the KDE-generated samples with the selected bandwidth, before and after applying the correction for porosity, respectively. Training of the DNN is achieved using the ensemble of original and KDE-corrected input points. We refer to this kind of dataset as RT-based, since it is based on an albeit cheap, full RT simulation. Furthermore, we refer to the obtained DNN and kNN emulators as ``local\" emulators, since as opposed to the emulators constructed for cement system 1, the current emulators are only valid for the input conditions encapsulated in the RT-based training set.\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1.5cm}\\includegraphics[width=50pc]{Figure9.png}\n\t\\caption{Scatter plots of the complex relationships between the five considered inputs for (1) the computationally cheap RT simulation of cement system 2 (orange dots) performed with the original HPx code, (2) the corresponding sampled points by kernel density estimation (KDE) using the selected bandwith (turquoise dots) and, (3) the same sampled KDE points after porosity correction (red dots).}\n\t\\label{fig9}\n\\end{figure}\n\nTraining performance of the DNN emulator is presented in Figure \\ref{fig10}. Similarly as for cement system 1, about 90\\% of the available data was used for the actual training of the DNN while the remaining 10\\% were used a validation set to contol overfitting. Lastly, performance is evaluated for both the trained DNN and kNN emulators using an independent test set of 10,000 examples. Overall, the accuracy of our ``local \" DNN emulator for this RT-inspired dataset is rather large with $Q_2$ values always greater or equal than 0.998. Training performance of the corresponding local kNN emulator is equally good (not shown). Regarding speedup, for this problem single-threaded PHREEQC achieves about 210 geochemical calculations per second on our used Intel\\textsuperscript{\\textregistered} i7 CPU while the GPU-based DNN and kNN emulators are both about 3000 times faster when predicting the 10,000 test points at once .\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure10.png}\n\t\\caption{1-1 plots of local DNN emulation performance obtained for system 2 when the local DNN is trained using 1,000,000 samples. The $x$-axis and $y$-axis present the original and emulated 10,000 independent test data points, respectively. The RMSE and $Q_2$ coefficient denote the root-mean-square-error and coefficient of determination in testing mode, respectively, between the original and emulated 10,000 test data points.}\n\t\\label{fig10}\n\\end{figure}\n\n\\subsubsection{Reactive transport simulation}\n\\label{prob2res}\nOur ``local\" DNN performs rather well when applied to the 61 $\\times$ 61 grid size and a time period of 6 years (Figures \\ref{fig11} - \\ref{fig12}). This for all components but C, for which some localized deviations appear between original and emulated 2D concentration profiles towards the end of the simulation period (Figure \\ref{fig12}). When applied to the 121 $\\times$ 121 grid, HPx$_{\\rm py}$-DNN produces additional discrepancies for O, Si and Al towards the end of the simulation period (Figures \\ref{fig13} - \\ref{fig14}). Yet most of the observed artifacts could probably be smoothed out by using post-filtering. The associated speedups are listed in Table \\ref{table4}. These speedups are larger to those obtained for cement system 1, with values between 8 and 9 when evaluated against HPx$_{\\rm{4C}}$. These speedups represent about 85 \\% to 90 \\% of the maximum possible speedup (Table \\ref{table3}). Overall, these findings indicate that for the considered problem, our RT-based training of a local DNN only works is the training set is sufficiently representative of the particular geochemical conditions encountered in the computationally demanding simulations, which is arguably not easy to achieve. This limitation is further discussed in section \\ref{discussion}.\n\nWe note a more uniform behavior for HPx$_{\\rm py}$-kNN across grid sizes than for HPx$_{\\rm py}$-DNN. Here the results for the 121  $\\times$ 121 grid are only slightly less accurate than those associated with the 61 $ \\times$ 61 grid (see Figures \\ref{fig15} - \\ref{fig16} where for brevity we only show concentration profiles for the C Al and S components). Furthermore, whenever observed the discrepancies between original and emulated profiles are more regularly scattered than for HPx$_{\\rm py}$-DNN. Note also that herein too, post-filtering could likely smooth out a large part of these deviations. In addition, owing to the use of a GPU to achieve the kNN calculations, the speedup provided by HPx$_{\\rm py}$-kNN are as large as those provided by HPx$_{\\rm py}$-DNN (Table \\ref{table4}).\n\nWith respect to the emulated solid amounts, the HPx$_{\\rm py}$-DNN results look visually good for the the 61 $\\times$ 61 grid. This is shown in Figure \\ref{fig17} for the C, Al and S chemical components, while emulation of the H, O Ca and Si chemical components is globally of similar quality (not shown). Nevertheless, significant deviations appear towards the end of the simulation period for every chemical component (see Figure \\ref{fig18} for the C, Al and S chemical components, emulation of the H, O Ca and Si chemical components shows the same level of mismatch). The HPx$_{\\rm py}$-kNN predictions are also fairly accurate for the 61 $\\times$ 61 (see Figure \\ref{fig19} for the C, Al and S chemical components, emulation of the H, O Ca and Si chemical components exhibits a globally similar quality) while some discrepancies show up at the end of the simulation (Figure \\ref{fig20}). However, the mismatch is less pronounced than for HPx$_{\\rm py}$-DNN. Overall, HPx$_{\\rm py}$-kNN appears to be somewhat more robust than HPx$_{\\rm py}$-DNN for this cement system, while providing the same (large) speedup.\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure11.png}\n\t\\caption{2D concentration profiles obtained for cement system 2 after 2 and 6 (final time step) years. RTM means the original HPx$_{\\rm{4C}}$ model and TM+DNN denotes the Hydrus transport model coupled with our DNN geochemical emulator (HPx$_{\\rm py}$-DNN). The first to fourth row present profiles for $Ca^{\\rm{conc}}$, $Si^{\\rm{conc}}$, $H^{\\rm{conc}}$, and $O^{\\rm{conc}}$, respectively. The considered grid size is 61 $\\times$ 61.}\n\t\\label{fig11}\n\\end{figure}\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure12.png}\n\t\\caption{2D concentration profiles obtained for cement system 2 after 2 and 6 (final time step) years. RTM means the original HPx$_{\\rm{4C}}$ model and TM+DNN denotes the Hydrus transport model coupled with our DNN geochemical emulator (HPx$_{\\rm py}$-DNN). The first to third row present profiles for $C^{\\rm{conc}}$, $Al^{\\rm{conc}}$, and $S^{\\rm{conc}}$, respectively. The considered grid size is 61 $\\times$ 61.}\n\t\\label{fig12}\n\\end{figure}\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure13.png}\n\t\\caption{2D concentration profiles obtained for cement system 2 after 2 and 6 (final time step) years. RTM means the original HPx$_{\\rm{4C}}$ model and TM+DNN denotes the Hydrus transport model coupled with our DNN geochemical emulator (HPx$_{\\rm py}$-DNN). The first to fourth row present profiles for $Ca^{\\rm{conc}}$, $Si^{\\rm{conc}}$, $H^{\\rm{conc}}$, and $O^{\\rm{conc}}$, respectively. The considered grid size is 121 $\\times$ 121.}\n\t\\label{fig13}\n\\end{figure}\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure14.png}\n\t\\caption{2D concentration profiles obtained for cement system 2 after 2 and 6 (final time step) years. RTM means the original HPx$_{\\rm{4C}}$ model and TM+DNN denotes the Hydrus transport model coupled with our DNN geochemical emulator (HPx$_{\\rm py}$-DNN). The first to third row present profiles for $C^{\\rm{conc}}$, $Al^{\\rm{conc}}$, and $S^{\\rm{conc}}$, respectively. The considered grid size is 121 $\\times$ 121.}\n\t\\label{fig14}\n\\end{figure}\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure15.png}\n\t\\caption{2D concentration profiles obtained for cement system 2 after 2 and 6 (final time step) years. RTM means the original HPx$_{\\rm{4C}}$ model and TM+kNN denotes the Hydrus transport model coupled with our kNN geochemical emulator (HPx$_{\\rm py}$-kNN).  The first to third row present profiles for $C^{\\rm{conc}}$, $Al^{\\rm{conc}}$, and $S^{\\rm{conc}}$, respectively. The considered grid size is 61 $\\times$ 61.}\n\t\\label{fig15}\n\\end{figure}\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure16.png}\n\t\\caption{2D concentration profiles obtained for cement system 2 after 2 and 6 (final time step) years. RTM means the original HPx$_{\\rm{4C}}$ model and TM+kNN denotes the Hydrus transport model coupled with our kNN geochemical emulator (HPx$_{\\rm py}$-kNN). The first to third row present profiles for $C^{\\rm{conc}}$, $Al^{\\rm{conc}}$, and $S^{\\rm{conc}}$, respectively. The considered grid size is 121 $\\times$ 121.}\n\t\\label{fig16}\n\\end{figure}\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure17.png}\n\t\\caption{2D solid amount profiles of the C, Al and S chemical components of cement system 2 after 2 and 6 (final time step) years. RTM means the original HPx$_{\\rm{4C}}$ model and TM+DNN denotes the Hydrus transport model coupled with our DNN geochemical emulator (HPx$_{\\rm py}$-DNN). The first to third row present profiles for $C^{\\rm{solid}}$, $Al^{\\rm{solid}}$, and $S^{\\rm{solid}}$, respectively The considered grid size is 61 $\\times$ 61.}\n\t\\label{fig17}\n\\end{figure}\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure18.png}\n\t\\caption{2D solid amount profiles of the C, Al and S chemical components of cement system 2 after 2 and 6 (final time step) years. RTM means the original HPx$_{\\rm{4C}}$ model and TM+DNN denotes the Hydrus transport model coupled with our DNN geochemical emulator (HPx$_{\\rm py}$-DNN). The first to third row present profiles for $C^{\\rm{solid}}$, $Al^{\\rm{solid}}$, and $S^{\\rm{solid}}$, respectively The considered grid size is 121 $\\times$ 121.}\n\t\\label{fig18}\n\\end{figure}\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure19.png}\n\t\\caption{2D solid amount profiles of the C, Al and S chemical components of cement system 2 after 2 and 6 (final time step) years. RTM means the original HPx$_{\\rm{4C}}$ model and TM+kNN denotes the Hydrus transport model coupled with our kNN geochemical emulator (HPx$_{\\rm py}$-kNN). The first to third row present profiles for $C^{\\rm{solid}}$, $Al^{\\rm{solid}}$, and $S^{\\rm{solid}}$, respectively The considered grid size is 61 $\\times$ 61.}\n\t\\label{fig19}\n\\end{figure}\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure20.png}\n\t\\caption{2D solid amount profiles of the C, Al and S chemical components of cement system 2 after 2 and 6 (final time step) years. RTM means the original HPx$_{\\rm{4C}}$ model and TM+kNN denotes the Hydrus transport model coupled with our kNN geochemical emulator (HPx$_{\\rm py}$-kNN). The first to third row present profiles for $C^{\\rm{solid}}$, $Al^{\\rm{solid}}$, and $S^{\\rm{solid}}$, respectively The considered grid size is 121 $\\times$ 121.}\n\t\\label{fig20}\n\\end{figure}\n\n\\begin{table}[h!]\n\t\\caption{Speedups offered by the KNN and DNN emulators in HPx$_{\\rm py}$ for the reactive transport simulations considered for cement system 2. The HPx$_{\\rm{4C}}$ calculations involve the parallelization of PHREEQC over our 4 CPUs. The HPx$_{\\rm{1C}}$ calculations are performed on a single CPU. Both the kNN and DNN predictions make use of a GPU. ML signifies the used machine learning method for emulation, BC denotes the type of flow boundary conditions and GS is the grid size. The maximum possible speedups associated with HPx$_{\\rm{4C}}$ and HP$_{\\rm{1C}}$,  Max SP HPx$_{\\rm{4C}}$ and  Max SP HPx$_{\\rm{1C}}$, correspond to an hypothetical situation where the geochemical calculations incur zero computational cost.}\n\n\t\\begin{center}\n\t\t\\begin{tabular}{cccccccc}\n\t\t\t\\hline\n\t\t\t& & & & & & & \\\\\n\t\t\tML & BC & GS & HPx$_{\\rm{4C}}$ time (s) & SP HPx$_{\\rm{4C}}$ & Max SP HPx$_{\\rm{4C}}$ & SP HPx$_{\\rm{1C}}$ & Max SP HPx$_{\\rm{1C}}$ \\\\\n\t\t\tDNN & ADV & 61 $\\times$ 61 & 21,415 & 8.2 & 9.0 & 30.3 & 33.1\\\\\n\t\t\tkNN & ADV & 61 $\\times$ 61 & 21,415 & 7.9 & 9.0 & 28.9 & 33.1\\\\\n\t\t\tDNN & ADV & 121 $\\times$ 121 & 199,841 & 8.2 & 9.5 & 29.9 & 32.8\\\\\n\t\t\tkNN & ADV & 121 $\\times$ 121 & 199,841 & 8.5 & 9.5 & 31.3 & 32.8\\\\\n\t\t\t\\hline\n\t\t\\end{tabular}\n\t\\end{center}\n\n\t\\label{table4}\n\\end{table}\n\n\\FloatBarrier\n\n3.2.2 Reactive transport simulation\n\\subsubsection{Reactive transport simulation}\n\\label{prob2res}\nOur ``local\" DNN performs rather well when applied to the 61 $\\times$ 61 grid size and a time period of 6 years (Figures \\ref{fig11} - \\ref{fig12}). This for all components but C, for which some localized deviations appear between original and emulated 2D concentration profiles towards the end of the simulation period (Figure \\ref{fig12}). When applied to the 121 $\\times$ 121 grid, HPx$_{\\rm py}$-DNN produces additional discrepancies for O, Si and Al towards the end of the simulation period (Figures \\ref{fig13} - \\ref{fig14}). Yet most of the observed artifacts could probably be smoothed out by using post-filtering. The associated speedups are listed in Table \\ref{table4}. These speedups are larger to those obtained for cement system 1, with values between 8 and 9 when evaluated against HPx$_{\\rm{4C}}$. These speedups represent about 85 \\% to 90 \\% of the maximum possible speedup (Table \\ref{table3}). Overall, these findings indicate that for the considered problem, our RT-based training of a local DNN only works is the training set is sufficiently representative of the particular geochemical conditions encountered in the computationally demanding simulations, which is arguably not easy to achieve. This limitation is further discussed in section \\ref{discussion}.\n\nWe note a more uniform behavior for HPx$_{\\rm py}$-kNN across grid sizes than for HPx$_{\\rm py}$-DNN. Here the results for the 121  $\\times$ 121 grid are only slightly less accurate than those associated with the 61 $ \\times$ 61 grid (see Figures \\ref{fig15} - \\ref{fig16} where for brevity we only show concentration profiles for the C Al and S components). Furthermore, whenever observed the discrepancies between original and emulated profiles are more regularly scattered than for HPx$_{\\rm py}$-DNN. Note also that herein too, post-filtering could likely smooth out a large part of these deviations. In addition, owing to the use of a GPU to achieve the kNN calculations, the speedup provided by HPx$_{\\rm py}$-kNN are as large as those provided by HPx$_{\\rm py}$-DNN (Table \\ref{table4}).\n\nWith respect to the emulated solid amounts, the HPx$_{\\rm py}$-DNN results look visually good for the the 61 $\\times$ 61 grid. This is shown in Figure \\ref{fig17} for the C, Al and S chemical components, while emulation of the H, O Ca and Si chemical components is globally of similar quality (not shown). Nevertheless, significant deviations appear towards the end of the simulation period for every chemical component (see Figure \\ref{fig18} for the C, Al and S chemical components, emulation of the H, O Ca and Si chemical components shows the same level of mismatch). The HPx$_{\\rm py}$-kNN predictions are also fairly accurate for the 61 $\\times$ 61 (see Figure \\ref{fig19} for the C, Al and S chemical components, emulation of the H, O Ca and Si chemical components exhibits a globally similar quality) while some discrepancies show up at the end of the simulation (Figure \\ref{fig20}). However, the mismatch is less pronounced than for HPx$_{\\rm py}$-DNN. Overall, HPx$_{\\rm py}$-kNN appears to be somewhat more robust than HPx$_{\\rm py}$-DNN for this cement system, while providing the same (large) speedup.\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure11.png}\n\t\\caption{2D concentration profiles obtained for cement system 2 after 2 and 6 (final time step) years. RTM means the original HPx$_{\\rm{4C}}$ model and TM+DNN denotes the Hydrus transport model coupled with our DNN geochemical emulator (HPx$_{\\rm py}$-DNN). The first to fourth row present profiles for $Ca^{\\rm{conc}}$, $Si^{\\rm{conc}}$, $H^{\\rm{conc}}$, and $O^{\\rm{conc}}$, respectively. The considered grid size is 61 $\\times$ 61.}\n\t\\label{fig11}\n\\end{figure}\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure12.png}\n\t\\caption{2D concentration profiles obtained for cement system 2 after 2 and 6 (final time step) years. RTM means the original HPx$_{\\rm{4C}}$ model and TM+DNN denotes the Hydrus transport model coupled with our DNN geochemical emulator (HPx$_{\\rm py}$-DNN). The first to third row present profiles for $C^{\\rm{conc}}$, $Al^{\\rm{conc}}$, and $S^{\\rm{conc}}$, respectively. The considered grid size is 61 $\\times$ 61.}\n\t\\label{fig12}\n\\end{figure}\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure13.png}\n\t\\caption{2D concentration profiles obtained for cement system 2 after 2 and 6 (final time step) years. RTM means the original HPx$_{\\rm{4C}}$ model and TM+DNN denotes the Hydrus transport model coupled with our DNN geochemical emulator (HPx$_{\\rm py}$-DNN). The first to fourth row present profiles for $Ca^{\\rm{conc}}$, $Si^{\\rm{conc}}$, $H^{\\rm{conc}}$, and $O^{\\rm{conc}}$, respectively. The considered grid size is 121 $\\times$ 121.}\n\t\\label{fig13}\n\\end{figure}\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure14.png}\n\t\\caption{2D concentration profiles obtained for cement system 2 after 2 and 6 (final time step) years. RTM means the original HPx$_{\\rm{4C}}$ model and TM+DNN denotes the Hydrus transport model coupled with our DNN geochemical emulator (HPx$_{\\rm py}$-DNN). The first to third row present profiles for $C^{\\rm{conc}}$, $Al^{\\rm{conc}}$, and $S^{\\rm{conc}}$, respectively. The considered grid size is 121 $\\times$ 121.}\n\t\\label{fig14}\n\\end{figure}\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure15.png}\n\t\\caption{2D concentration profiles obtained for cement system 2 after 2 and 6 (final time step) years. RTM means the original HPx$_{\\rm{4C}}$ model and TM+kNN denotes the Hydrus transport model coupled with our kNN geochemical emulator (HPx$_{\\rm py}$-kNN).  The first to third row present profiles for $C^{\\rm{conc}}$, $Al^{\\rm{conc}}$, and $S^{\\rm{conc}}$, respectively. The considered grid size is 61 $\\times$ 61.}\n\t\\label{fig15}\n\\end{figure}\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure16.png}\n\t\\caption{2D concentration profiles obtained for cement system 2 after 2 and 6 (final time step) years. RTM means the original HPx$_{\\rm{4C}}$ model and TM+kNN denotes the Hydrus transport model coupled with our kNN geochemical emulator (HPx$_{\\rm py}$-kNN). The first to third row present profiles for $C^{\\rm{conc}}$, $Al^{\\rm{conc}}$, and $S^{\\rm{conc}}$, respectively. The considered grid size is 121 $\\times$ 121.}\n\t\\label{fig16}\n\\end{figure}\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure17.png}\n\t\\caption{2D solid amount profiles of the C, Al and S chemical components of cement system 2 after 2 and 6 (final time step) years. RTM means the original HPx$_{\\rm{4C}}$ model and TM+DNN denotes the Hydrus transport model coupled with our DNN geochemical emulator (HPx$_{\\rm py}$-DNN). The first to third row present profiles for $C^{\\rm{solid}}$, $Al^{\\rm{solid}}$, and $S^{\\rm{solid}}$, respectively The considered grid size is 61 $\\times$ 61.}\n\t\\label{fig17}\n\\end{figure}\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure18.png}\n\t\\caption{2D solid amount profiles of the C, Al and S chemical components of cement system 2 after 2 and 6 (final time step) years. RTM means the original HPx$_{\\rm{4C}}$ model and TM+DNN denotes the Hydrus transport model coupled with our DNN geochemical emulator (HPx$_{\\rm py}$-DNN). The first to third row present profiles for $C^{\\rm{solid}}$, $Al^{\\rm{solid}}$, and $S^{\\rm{solid}}$, respectively The considered grid size is 121 $\\times$ 121.}\n\t\\label{fig18}\n\\end{figure}\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure19.png}\n\t\\caption{2D solid amount profiles of the C, Al and S chemical components of cement system 2 after 2 and 6 (final time step) years. RTM means the original HPx$_{\\rm{4C}}$ model and TM+kNN denotes the Hydrus transport model coupled with our kNN geochemical emulator (HPx$_{\\rm py}$-kNN). The first to third row present profiles for $C^{\\rm{solid}}$, $Al^{\\rm{solid}}$, and $S^{\\rm{solid}}$, respectively The considered grid size is 61 $\\times$ 61.}\n\t\\label{fig19}\n\\end{figure}\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure20.png}\n\t\\caption{2D solid amount profiles of the C, Al and S chemical components of cement system 2 after 2 and 6 (final time step) years. RTM means the original HPx$_{\\rm{4C}}$ model and TM+kNN denotes the Hydrus transport model coupled with our kNN geochemical emulator (HPx$_{\\rm py}$-kNN). The first to third row present profiles for $C^{\\rm{solid}}$, $Al^{\\rm{solid}}$, and $S^{\\rm{solid}}$, respectively The considered grid size is 121 $\\times$ 121.}\n\t\\label{fig20}\n\\end{figure}\n\n\\begin{table}[h!]\n\t\\caption{Speedups offered by the KNN and DNN emulators in HPx$_{\\rm py}$ for the reactive transport simulations considered for cement system 2. The HPx$_{\\rm{4C}}$ calculations involve the parallelization of PHREEQC over our 4 CPUs. The HPx$_{\\rm{1C}}$ calculations are performed on a single CPU. Both the kNN and DNN predictions make use of a GPU. ML signifies the used machine learning method for emulation, BC denotes the type of flow boundary conditions and GS is the grid size. The maximum possible speedups associated with HPx$_{\\rm{4C}}$ and HP$_{\\rm{1C}}$,  Max SP HPx$_{\\rm{4C}}$ and  Max SP HPx$_{\\rm{1C}}$, correspond to an hypothetical situation where the geochemical calculations incur zero computational cost.}\n\n\t\\begin{center}\n\t\t\\begin{tabular}{cccccccc}\n\t\t\t\\hline\n\t\t\t& & & & & & & \\\\\n\t\t\tML & BC & GS & HPx$_{\\rm{4C}}$ time (s) & SP HPx$_{\\rm{4C}}$ & Max SP HPx$_{\\rm{4C}}$ & SP HPx$_{\\rm{1C}}$ & Max SP HPx$_{\\rm{1C}}$ \\\\\n\t\t\tDNN & ADV & 61 $\\times$ 61 & 21,415 & 8.2 & 9.0 & 30.3 & 33.1\\\\\n\t\t\tkNN & ADV & 61 $\\times$ 61 & 21,415 & 7.9 & 9.0 & 28.9 & 33.1\\\\\n\t\t\tDNN & ADV & 121 $\\times$ 121 & 199,841 & 8.2 & 9.5 & 29.9 & 32.8\\\\\n\t\t\tkNN & ADV & 121 $\\times$ 121 & 199,841 & 8.5 & 9.5 & 31.3 & 32.8\\\\\n\t\t\t\\hline\n\t\t\\end{tabular}\n\t\\end{center}\n\n\t\\label{table4}\n\\end{table}\n\n\\FloatBarrier\n\n",
        "label": "fig18",
        "Descriptive_question1": "Which chemical component is represented in the first row of figure_18?",
        "Descriptive_question2": "What is the grid size used in figure_18?",
        "Reasoning_question1": "How do the solid amount profiles of C, Al, and S in figure_18 compare between the RTM and TM+DNN models after 6 years?",
        "Reasoning_question2": "What might be the reasons for the observed deviations in the solid amount profiles between the original RTM and the emulated TM+DNN in figure_18 at the final time step?",
        "Descriptive_answer1": "C^{\\rm{solid}}",
        "Descriptive_answer2": "121 \\times 121",
        "Reasoning_answer1": "In figure_18, after 6 years, the solid amount profiles of C, Al, and S for the TM+DNN model show noticeable deviations compared to the RTM profiles. The TM+DNN profiles appear less smooth and exhibit discrepancies, especially towards the borders and some localized areas, indicating that while the general distribution pattern of the chemical components is captured, the DNN emulator has difficulty accurately replicating the original HPx_{4C} solid amounts at the finer scale and after prolonged simulation time.",
        "Reasoning_answer2": "The observed deviations between the original RTM and the TM+DNN emulated solid amount profiles at the final time step in figure_18 likely stem from several factors: (1) The 'local' DNN emulator was trained on a RT-based dataset that might not fully encompass the complex nonlinear relationships and conditions at later times or higher resolutions, leading to reduced accuracy; (2) The high dimensionality and wide range of orders of magnitude in inputs and outputs make learning and generalization challenging, especially for longer simulations; (3) Potential accumulation of small prediction errors over time steps can amplify discrepancies; and (4) The emulation approach might not perfectly capture boundary effects or sharp gradients which develop towards the end of simulations, contributing to mismatches in solid amount profiles."
    },
    {
        "paper_id": "2107.07598.json",
        "image_id": "figure_19",
        "image_path": [
            "/home/yz979/scratch/chengye/scimolmo/test-images-tables/dataset_imagestables/2107.07598/images/Figure19.png"
        ],
        "caption": "2D solid amount profiles of the C, Al and S chemical components of cement system 2 after 2 and 6 (final time step) years. RTM means the original HPx$_{\\rm{4C}}$ model and TM+kNN denotes the Hydrus transport model coupled with our kNN geochemical emulator (HPx$_{\\rm py}$-kNN). The first to third row present profiles for $C^{\\rm{solid}}$, $Al^{\\rm{solid}}$, and $S^{\\rm{solid}}$, respectively The considered grid size is 61 $\\times$ 61.",
        "classify": "Chart",
        "section_info": "3 Results\n\\section{Results}\n\\label{results}\n\n\\subsection{Ca-Si Problem}\nFor this first cement system, the emulation problem consists of predicting at each time step of the RT simulation the (output) Ca, Si, H and O aqueous concentrations (mol/kg of water or mol/kgw) from the (input) total amounts of Ca and Si (mol). \n\n\\subsubsection{Training the emulators}\n\\label{train_res1}\nHere the kNN and DNN emulators are firstly trained using a set of 400,000 test examples for both. This training set is obtained by randomly sampling the two-dimensional input space by latin hypercube sampling (LHS) between $\\left[0,0\\right]$ and $\\rm{\\left[Ca^{tot}_{max},Si^{tot}_{max}\\right]}$, and running PHREEQC for each input sample, $\\rm{\\textbf{x}_i = \\left[Ca^{tot}_i, Si^{tot}_i\\right]}$ to get the corresponding output vectors, $\\rm{\\textbf{y}_i = \\left[Ca^{conc}_i,Si^{conc}_i,H^{conc}_i,O^{conc}_i\\right]}$. The upper bounds, $\\rm{Ca^{tot}_{max}}$ and $\\rm{Si^{tot}_{max}}$ are defined based on a cheap full RT simulation with advective-dispersive transport using a small 1D domain of 51 nodes. It is worth noting that the total amounts of $\\textbf{x}_i$, corresponding to the PHREEQC-simulated concentrations, $\\textbf{y}_i$, have to be corrected for the different amount of water between the training set and the transport simulations. Doing so, it turns out that about 20 \\% of the post-corrected $x_i$ values exceed their pre-defined upper bounds and these excessively large values need to be filtered out. Creating the 400,000 training examples thus required about 500,000 PHREEQC runs. As stated earlier, for this problem single-threaded PHREEQC performs about 670 geochemical calculations per second on our used Intel\\textsuperscript{\\textregistered} i7 CPU.\n\nWith respect to kNN, the tuning parameters are the number of neighbors, $k$, the type of distance measure, and the interpolation technique. We simply used the default settings: $k = 5$, euclidean distance and inverse-distance interpolation. Regarding training of the DNN, the 400,000 sample were split between the training set itself (90 \\% of the data) and a validation set (10 \\% of the data). The latter serves to monitor the evolution of the selected mean squared error loss function on samples that are not used for training, thereby detecting potential overfitting. If the validation loss stops decreasing before the fixed number of epochs has been completed, then  training is stopped. Importantly, the emulation is achieved in log-space for both the input, $X$ and output $Y$, domains. This because total amounts and concentrations of the involved components typically cover many orders of magnitudes (up to 10 orders or more). Using a DNN also requires some form of data normalization or standardization. Here both the $\\rm{log\\left(\\textbf{x}_i\\right)}$ and $\\rm{log\\left(\\textbf{y}_i\\right)}$ vectors are standardized around 0 with standard deviation of 1.\n\nFigure \\ref{fig1} illustrates the trained emulators' performance for geochemical predictions using an independent test set that comprises 10,000 test examples. Both kNN (Figures \\ref{fig1}a - d) and DNN (Figures \\ref{fig1}e - h) appear to be rather accurate . DNN also shows a slight degradation for the larger concentration values  (Figures \\ref{fig1}e - h). The latter is likely due to the combination of a small proportion of large concentration values in the training set with the log-transformation that implicitly pushes the DNN to try harder to fit the smaller concentrations during training. Regarding speedup and as written earlier, for this setup the single-threaded kNN method is 300 times faster than single-threaded PHREEQC for predicting the 10,000 concentration vectors all at once. The computational savings allowed by the DNN emulator when ran on our GPU is higher with a speedup as large as 4000 for predicting the same 10,000 concentration vectors all at once. \n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{0cm}\\includegraphics[width=35pc]{Figure1.png}\n\t\\caption{1-1 plots of the kNN (subfigures (a) - (d)) and DNN (subfigures (e) - (h)) emulators' performance obtained for system 1 when the kNN training base contains 400,000 samples and the DNN is trained using the same 400,000 samples. Here ``true\" means the original PHREEQC-simulated data and ``predicted\" denotes the emulated (that is, kNN-simulated and DNN-simulated) data. Hence, the $x$-axis and $y$-axis present the original and emulated 10,000 independent test data points, respectively. The RMSE and $Q_2$ coefficient denote the root-mean-square-error and coefficient of determination in testing mode, respectively, between the original and emulated 10,000 test data points.}\n\t\\label{fig1}\n\\end{figure}\n\nTo test the sensitivity of the emulators' performance to the training set size, training was also performed using reduced training sets comprising 100,000, 10,000, 1000 and 100 samples, respectively. It seen that the DNN performance achieved  when using 10,000 training samples is virtually the same as that obtained when using 400,000 training samples (Figure \\ref{fig2} and Table \\ref{table2}). It is only for training sets smaller than 1000 samples that the DNN performance starts to degrade significantly (Table \\ref{table2}). In contrast, the behavior of kNN appears to markedly decrease as the training set gets smaller (Figure \\ref{fig2} and Table \\ref{table1}).\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{0cm}\\includegraphics[width=35pc]{Figure2.png}\n\t\\caption{1-1 plots of the kNN (subfigures (a) - (d)) and DNN (subfigures (e) - (h)) emulators' performance obtained for system 1 when the kNN training base contains 10,000 samples and the DNN is trained using the same 10,000 samples. Here ``true\" means the original PHREEQC-simulated data and ``predicted\" denotes the emulated (that is, kNN-simulated and DNN-simulated) data. Hence, the $x$-axis and $y$-axis present the original and emulated 10,000 independent test data points, respectively. The RMSE and $Q_2$ coefficient denote the root-mean-square-error and coefficient of determination in testing mode, respectively, between the original and emulated 10,000 test data points.}\n\t\\label{fig2}\n\\end{figure}\n\n\\begin{table}[h!]\n\t\\caption{Performance of the DNN and kNN emulators for cement system 1 and different training set sizes. For brevity, only the results for Ca$^{conc}$ and Si$^{conc}$ are shown. The units are mol per kg of water (mol/kgw). ML refers to the type of emulator, TR signifies the size of the training set, and the RMSE and $Q_2$ coefficient denote the root-mean-square-error and coefficient of determination in testing mode, respectively, between the original and emulated 10,000 test data points.}\n\t\\begin{center}\n\t\t\\begin{tabular}{cccccccc}\n\t\t\t\\hline\n\t\t\t& & & & & \\\\\n\t\t\tML & TR & RMSE - Ca$^{conc}$ & $Q_2$ - Ca$^{conc}$ & RMSE - Si$^{conc}$ & $Q_2$ - Si$^{conc}$\\\\\n\t\t\tDNN & 4 $\\times$ 10$^{5}$ & 4.50 $\\times$ 10$^{-5}$ & 0.9999 & 1.21 $\\times$ 10$^{-6}$ & 0.9999\\\\\n\t\t\tDNN & 1 $\\times$ 10$^{5}$ & 5.06 $\\times$ 10$^{-5}$ & 0.9999 & 1.30 $\\times$ 10$^{-6}$ & 0.9999\\\\\n\t\t\tDNN & 1 $\\times$ 10$^{4}$ & 4.57 $\\times$ 10$^{-5}$ & 0.9999 & 1.48 $\\times$ 10$^{-6}$ & 0.9999\\\\\n\t\t\tDNN & 1 $\\times$ 10$^{3}$ & 8.02 $\\times$ 10$^{-5}$ & 0.9998 & 3.31 $\\times$ 10$^{-6}$ & 0.9994\\\\\n\t\t\tDNN & 1 $\\times$ 10$^{2}$ & 47.1 $\\times$ 10$^{-5}$ & 0.9935 & 9.02 $\\times$ 10$^{-6}$ & 0.9959\\\\\n\t\t\tkNN & 4 $\\times$ 10$^{5}$ & 2.52 $\\times$ 10$^{-5}$ & 1.0000 & 1.59 $\\times$ 10$^{-6}$ & 0.9999\\\\\n\t\t\tkNN & 1 $\\times$ 10$^{5}$ & 6.27 $\\times$ 10$^{-5}$ & 0.9999 & 4.20 $\\times$ 10$^{-6}$ & 0.9991\\\\\n\t\t\tkNN & 1 $\\times$ 10$^{4}$ & 14.1 $\\times$ 10$^{-5}$ & 0.9994 & 13.5 $\\times$ 10$^{-6}$ & 0.9908\\\\\n\t\t\tkNN & 1 $\\times$ 10$^{3}$ & 43.9 $\\times$ 10$^{-5}$ & 0.9944 & 74.6 $\\times$ 10$^{-6}$ & 0.7180\\\\\n\t\t\tkNN & 1 $\\times$ 10$^{2}$  & 96.4 $\\times$ 10$^{-5}$ & 0.9729 & 55.0 $\\times$ 10$^{-6}$ & 0.8468\\\\\n\t\t\\end{tabular}\n\t\\end{center}\n\t\\label{table2}\n\\end{table}\n\n\\FloatBarrier\n\n\\subsubsection{Reactive transport simulation}\n\\label{prob12Dres}\n\nThis section focuses on reactive transport simulations with HPx$_{\\rm py}$ within cement system 1, under both advective-dispersive and diffusive transport conditions. As written above, the domain sizes are both 61 $\\times$ 61 and 121 $\\times$ 121 for the advection-dispersion case and, because of computational constraints, solely 61 $\\times$ 61  for the diffusion case. In addition, the simulation time period is 2 years for the advection-dispersion case and 1 year for the diffusion case. Figures \\ref{fig3} and \\ref{fig4} present times series of original and emulated Ca, Si, H and O concentrations at 5 locations within the 2D domain for advective-dispersive transport conditions, for both our kNN-based (HPx$_{\\rm py}$-kNN) and DNN-based (HPx$_{\\rm py}$-DNN) reactive transport codes. It is seen that HPx$_{\\rm py}$-kNN and HPx$_{\\rm py}$-DNN both induce a quite good simulation accuracy. Also, the results for the diffusive transport case are of similarly good quality (not shown). Figures \\ref{fig5} - \\ref{fig6} provide more insights into the HPx$_{\\rm py}$-kNN and HPx$_{\\rm py}$-DNN performances by displaying 2D Ca, Si, H and O concentration profiles at a given time. For each experiment and chemical component, this time is selected as to be well representative of the simulated dynamics. It is observed that the original and emulated images are visually almost indistinguishable for the advection-dispersion case (Figure \\ref{fig5}). For the diffusion case, the emulators also perform quite well for Ca, H and O (Figures\\ref{fig6}a - c, g - i and j - l), while some slight to moderate discrepancies appear at the concentration front for Si (Figures \\ref{fig6}d - f). Notwithstanding, the Si concentration remains globally well predicted. Furthermore, Figures \\ref{fig7} - \\ref{fig8} present the original and emulated 2D solid amount profiles corresponding to Figures \\ref{fig5} - \\ref{fig6}. The original solid amount profiles are overall well approximated by HPx$_{\\rm py}$-kNN and HPx$_{\\rm py}$-DNN for the advection-dispersion case (Figure \\ref{fig7}), even though some mismatch appears at the border of the fully depleted zone for the H component. As of the diffusion case (Figure \\ref{fig8}), the same kind of mismatch is observed for the emulated solid amounts of H by HPx$_{\\rm py}$-kNN while the emulated profiles by HPx$_{\\rm py}$-DNN show some more discrepancies. Though we decided to present raw emulation results, we would like to stress that some if not all of the observed artifacts could likely be smoothed out by using some post-filtering such as median filtering.\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure3.png}\n\t\\caption{Time series of original (RTM, solid green lines) and HPx$_{\\rm py}$-kNN emulated (TM+kNN, dashed orange lines) concentrations (mol/kg) of Ca, H, O and Si at selected observations points for cement system 1 and advective-dispersive transport. Obs. 1 - 5 denote the selected observation points, with the following $\\left[x,y\\right]$ locations (in cm). Obs. 1:  $\\left[0.5, 2.5\\right]$, Obs. 2: $\\left[1, 2\\right]$, Obs. 3: $\\left[2, 2\\right]$, Obs. 4: $\\left[1, 1\\right]$, Obs. 5: $\\left[2, 1\\right]$. The results for the 121 $\\times$ 121 grid size are rather similar.}\n\t\\label{fig3}\n\\end{figure}\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure4.png}\n\t\\caption{Time series of original (RTM, solid green lines) and HPx$_{\\rm py}$-DNN emulated (TM+DNN, dashed orange lines) concentrations (mol/kg) of Ca, H, O and Si at selected observations points for cement system 1 and advective-dispersive transport. Obs. 1 - 5 denote the selected observation points, with the following $\\left[x,y\\right]$ locations (in cm). Obs. 1:  $\\left[0.5, 2.5\\right]$, Obs. 2: $\\left[1, 2\\right]$, Obs. 3: $\\left[2, 2\\right]$, Obs. 4: $\\left[1, 1\\right]$, Obs. 5: $\\left[2, 1\\right]$. The results for the 121 $\\times$ 121 grid size are rather similar.}\n\t\\label{fig4}\n\\end{figure}\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure5.png}\n\t\\caption{2D concentration profiles obtained for cement system 1 at the end of the 2-year simulation performed for the advection-dispersion case. RTM means the original HP$_{\\rm{4C}}$ model, TM+kNN denotes the Hydrus transport model coupled with our kNN geochemical emulator (HPx$_{\\rm py}$-kNN) and TM+DNN signifies the Hydrus transport model coupled with our DNN geochemical emulator (HPx$_{\\rm py}$-DNN). The first to fourth row present profiles for $Ca^{\\rm{conc}}$, $Si^{\\rm{conc}}$, $H^{\\rm{conc}}$, and $O^{\\rm{conc}}$, respectively. The considered grid size is 61 $\\times$ 61. The results for the 121 $\\times$ 121 grid are rather similar.}\n\t\\label{fig5}\n\\end{figure}\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure6.png}\n\t\\caption{2D concentration profiles obtained for cement system 1 at the final time step of the 1-year simulation performed for the diffusion case. RTM means the original HP$_{\\rm{4C}}$ model, TM+kNN denotes the Hydrus transport model coupled with our kNN geochemical emulator (HPx$_{\\rm py}$-kNN) and TM+DNN signifies the Hydrus transport model coupled with our DNN geochemical emulator (HPx$_{\\rm py}$-DNN). The first to fourth row present profiles for $Ca^{\\rm{conc}}$, $Si^{\\rm{conc}}$, $H^{\\rm{conc}}$, and $O^{\\rm{conc}}$, respectively. The considered grid size is 61 $\\times$ 61.}\n\t\\label{fig6}\n\\end{figure}\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure7.png}\n\t\\caption{2D solid amount profiles obtained for cement system 1 at the end of the 2-year simulation performed for the advection-dispersion case. RTM means the original HP$_{\\rm{4C}}$ model, TM+kNN denotes the Hydrus transport model coupled with our kNN geochemical emulator (HPx$_{\\rm py}$-kNN) and TM+DNN signifies the Hydrus transport model coupled with our DNN geochemical emulator (HPx$_{\\rm py}$-DNN). The first to fourth row present profiles for $Ca^{\\rm{solid}}$, $Si^{\\rm{solid}}$, $H^{\\rm{solid}}$, and $O^{\\rm{solid}}$, respectively. The considered grid size is 61 $\\times$ 61. The results for the 121 $\\times$ 121 grid are rather similar.}\n\t\\label{fig7}\n\\end{figure}\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure8.png}\n\t\\caption{2D solid amount profiles obtained for cement system 1 at the final time step of the 1-year simulation performed for the diffusion case. RTM means the original HP$_{\\rm{4C}}$ model, TM+kNN denotes the Hydrus transport model coupled with our kNN geochemical emulator (HPx$_{\\rm py}$-kNN) and TM+DNN signifies the Hydrus transport model coupled with our DNN geochemical emulator (HPx$_{\\rm py}$-DNN). The first to fourth row present profiles for $Ca^{\\rm{solid}}$, $Si^{\\rm{solid}}$, $H^{\\rm{solid}}$, and $O^{\\rm{solid}}$, respectively. The considered grid size is 61 $\\times$ 61.}\n\t\\label{fig8}\n\\end{figure}\n\nThe speedups associated with the considered problem are detailed in Table \\ref{table3}. It is noted that the GPU-based DNN emulator allows for a speedup that is close to optimal. Indeed, The DNN speedups overall represent 85 \\% to 95 \\% of the maximum possible speedups (that is, speedups that would be obtained if the geochemcical calculations would come at no cost at all). The speedups associated with single-threaded kNN remain substantial but only amount to 57 \\% - 65 \\% of the corresponding maximum speedups. As detailed in section \\ref{train_res1}, the used kNN and DNN implementations are found to be respectively 300 and 4000 faster than single-threaded PHREEQC when predicting 10,000 points all one once for this geochemical system. Based on these numbers one could have expected the achieved speedups to represent say 90 \\% (kNN) or 99 \\% (DNN) of the maximum possible ones. A large part of the gaps between achieved and maximum possible speedups is thus likely caused by the time required for communicating and exchanging data between the main C/C++ code and the Python-based emulators.\n\n\\begin{table}[h!]\n\t\\caption{Speedups offered by the KNN and DNN emulators in HPx$_{\\rm py}$ for the reactive transport simulations considered for cement system 1. The HPx$_{\\rm{4C}}$ calculations involve the parallelization of PHREEQC over our 4 CPUs. The HPx$_{\\rm{1C}}$ calculations are performed on a single CPU. The kNN predictions are performed on a single CPU using the scikit-learn implementation while the DNN predictions make use of our GPU. ML signifies the used machine learning method for emulation, TC denotes transport conditions (ADV: advection-dispersion, DIF: diffusion) and GS is the grid size. The maximum possible speedups associated with HPx$_{\\rm{4C}}$ and HPx$_{\\rm{1C}}$,  Max SP HPx$_{\\rm{4C}}$ and  Max SP HPx$_{\\rm{1C}}$, correspond to an hypothetical situation where the geochemical calculations incur zero computational cost.}\n\n\t\\begin{center}\n\t\t\\begin{tabular}{cccccccc}\n\t\t\t\\hline\n\t\t\t& & & & & & & \\\\\n\t\t\tML & TC & GS & HPx$_{\\rm{4C}}$ time (s) & SP HPx$_{\\rm{4C}}$ & Max SP HPx$_{\\rm{4C}}$ & SP HPx$_{\\rm{1C}}$ & Max SP HPx$_{\\rm{1C}}$ \\\\\n\t\t\tDNN & ADV & 61 $\\times$ 61 & 3189 & 6.8 & 7.7 & 24.5 & 28.5 \\\\\n\t\t\tkNN & ADV & 61 $\\times$ 61 & 3189 & 5.0 & 7.7 & 18.0 & 28.5 \\\\\n\t\t\tDNN & ADV & 121 $\\times$ 121 & 23,337 & 5.0 & 5.2 & 17.0 & 18.0 \\\\\n\t\t\tkNN & ADV & 121 $\\times$ 121 & 23,337 & 3.4 & 5.2 & 11.6 & 18.0 \\\\\n\t\t\tDNN & DIF & 61 $\\times$ 61 & 25,448 & 4.2 & 4.9 & 16.2 & 19.1 \\\\\n\t\t\tkNN & DIF & 61 $\\times$ 61 & 25,448 & 2.8 & 4.9 & 10.8 & 19.1 \\\\\n\t\t\t\\hline\n\t\t\\end{tabular}\n\t\\end{center}\n\n\t\\label{table3}\n\\end{table}\n\n\\FloatBarrier\n\n\\subsection{Al-C-Ca-S-Si Problem}\n\nFor this second cement system, the emulation problem consists of predicting at each time step of the RT simulation the (output) Al, C, Ca, S, Si, H and O aqueous concentrations (mol/kgw) from the (input) total amounts of Al, C, Ca, S and Si (mol). Here we focus on advective-dispersive transport only while similarly as for cement system 1, the considered domain sizes are 61 $\\times$ 61 and 121 $\\times$ 121. Moreover, the simulation time period is set to 6 years. \n\nAs mentioned earlier, for this higher-dimensional problem it is observed that the used scikit-learn kNN implementation becomes prohibitively slow compared to HPx$_{\\rm{4C}}$. We found that to get a good emulation accuracy the kNN training base needs to contain 1,000,000 samples (or more). This training base's size together with a 5-dimensional search space leads to an HPx$_{\\rm py}$-kNN reactive transport simulation time that is comparable to that of HPx$_{\\rm{4C}}$. Therefore, for this second cement system we built a custom kNN regressor around another kNN implementation contained in the FAISS package \\citep{faiss2017}. Our used FAISS variant allows for GPU computing and is much faster than scikit-learn for this cement system, but is slightly less accurate due to the use of an approximate rather than exact nearest neighbor search \\citep[see][for details]{faiss2017}.\n\n\\subsubsection{Training the DNN}\n\\label{train_res2}\n\nBuilding a good training set to perform a kNN search and learn the weights and biases of our DNN turned out to be a complicated task in this case. This because to make useful kNN predictions and/or learn an useful DNN, the training set must be sufficiently representative of the geochemical conditions encountered during the reactive transport simulation one wish to perform with HPx$_{\\rm py}$-kNN and the trained HPx$_{\\rm py}$-DNN. In contrast to cement system 1, creating the training set by sampling the $X$-space with a controlled randomness between predefined lower and upper bounds did not prove successful. We tried that strategy by drawing as much as 4,000,000 5-dimensional $\\textbf{x}$ vectors from the $X$-space using a Sobol low-discrepancy sequence \\citep[][]{Sobol1967, Joe-Kuo2003}. Such low-discrepancy sampling scheme covers the 5-dimensional hypercube more uniformly than LHS. Despite a good performance on the test set (not shown), the resulting DNN accuracy in reactive transport mode was never deemed satisfying. In other words, no satisfying ``global\" or ``universal\" DNN emulator could be devised for this cement system. This is probably caused by the fact for this problem, the input (5 total amounts) and output (7 aqueous concentrations) spaces are quite nonlinearly related and both cover 6 to 10 orders of magnitudes depending on the considered element. Therefore, we resorted to the alternative training strategy detailed below. The latter basically tries to grasp the complex correlations and higher-order dependencies that exist between the elements of $\\textbf{x}$ (total amounts, input space) for a given reactive transport simulation setup, in order to produce a training set that honors these between-input relationships.\n\n\\begin{itemize}\n\t\\item Perform a ``cheap\" full reactive transport simulation under the transport conditions and geochemistry of interest and collect the resulting $\\textbf{x}$ -$\\textbf{y}$  pairs of examples (for the considered grid nodes and time steps). Computational demand controls what domain size and simulation time period can be used for this cheap calculation. We used a modest 16 $\\times$ 16 domain and a simulation time period of 10 years. The associated HPx$_{\\rm{4C}}$ runtimes is 180 s.\n\t\n\t\\item Fit a kernel density estimator (KDE) with a Gaussian kernel to the collected $\\textbf{x}$ vectors (encapsulated in the $\\textbf{X}$ array) and generate a fixed number of new input vectors, $\\textbf{x}_{KDE}$. Then run PHREEQC for the $\\textbf{X}_{KDE}$ set to get the corresponding output set, $\\textbf{Y}_{KDE}$. Now apply the correction for porosity described in section \\ref{train_res1} to the $\\textbf{X}_{KDE}$ set and form the training set by merging the ensemble of $\\textbf{x}$-$\\textbf{y}$ pairs with that of the $\\textbf{x}_{KDE}$-$\\textbf{y}_{KDE}$ pairs. The number of produced unique examples by the considered cheap HPx$_{\\rm{4C}}$ simulations varied between 10,000 and 50,000. The KDE-based enrichment of this dataset was deemed necessary to provide more input variability, thereby avoiding overfitting of the trained DNN and improving the kNN accuracy, while still honoring the complex between-input relationships. The number of KDE-generated samples was set as to obtain a total training set size of 1,000,000 examples. A key component of the approach is the bandwith parameter of the Gaussian KDE kernel which controls how much the KDE-generated samples depart from the original ensemble. After limited trial and error, we fixed the kernel bandwith to 0.0025 for the considered case studies.\n\t\n\\end{itemize}\n\nThe scatter plots in Figure \\ref{fig9} illustrate our training set creation procedure. The orange dots depict the pairwise relationships observed between the 5 elements of $\\textbf{x}$ in the cheap simulation. The red and cyan dots in Figure \\ref{fig9} represent the KDE-generated samples with the selected bandwidth, before and after applying the correction for porosity, respectively. Training of the DNN is achieved using the ensemble of original and KDE-corrected input points. We refer to this kind of dataset as RT-based, since it is based on an albeit cheap, full RT simulation. Furthermore, we refer to the obtained DNN and kNN emulators as ``local\" emulators, since as opposed to the emulators constructed for cement system 1, the current emulators are only valid for the input conditions encapsulated in the RT-based training set.\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1.5cm}\\includegraphics[width=50pc]{Figure9.png}\n\t\\caption{Scatter plots of the complex relationships between the five considered inputs for (1) the computationally cheap RT simulation of cement system 2 (orange dots) performed with the original HPx code, (2) the corresponding sampled points by kernel density estimation (KDE) using the selected bandwith (turquoise dots) and, (3) the same sampled KDE points after porosity correction (red dots).}\n\t\\label{fig9}\n\\end{figure}\n\nTraining performance of the DNN emulator is presented in Figure \\ref{fig10}. Similarly as for cement system 1, about 90\\% of the available data was used for the actual training of the DNN while the remaining 10\\% were used a validation set to contol overfitting. Lastly, performance is evaluated for both the trained DNN and kNN emulators using an independent test set of 10,000 examples. Overall, the accuracy of our ``local \" DNN emulator for this RT-inspired dataset is rather large with $Q_2$ values always greater or equal than 0.998. Training performance of the corresponding local kNN emulator is equally good (not shown). Regarding speedup, for this problem single-threaded PHREEQC achieves about 210 geochemical calculations per second on our used Intel\\textsuperscript{\\textregistered} i7 CPU while the GPU-based DNN and kNN emulators are both about 3000 times faster when predicting the 10,000 test points at once .\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure10.png}\n\t\\caption{1-1 plots of local DNN emulation performance obtained for system 2 when the local DNN is trained using 1,000,000 samples. The $x$-axis and $y$-axis present the original and emulated 10,000 independent test data points, respectively. The RMSE and $Q_2$ coefficient denote the root-mean-square-error and coefficient of determination in testing mode, respectively, between the original and emulated 10,000 test data points.}\n\t\\label{fig10}\n\\end{figure}\n\n\\subsubsection{Reactive transport simulation}\n\\label{prob2res}\nOur ``local\" DNN performs rather well when applied to the 61 $\\times$ 61 grid size and a time period of 6 years (Figures \\ref{fig11} - \\ref{fig12}). This for all components but C, for which some localized deviations appear between original and emulated 2D concentration profiles towards the end of the simulation period (Figure \\ref{fig12}). When applied to the 121 $\\times$ 121 grid, HPx$_{\\rm py}$-DNN produces additional discrepancies for O, Si and Al towards the end of the simulation period (Figures \\ref{fig13} - \\ref{fig14}). Yet most of the observed artifacts could probably be smoothed out by using post-filtering. The associated speedups are listed in Table \\ref{table4}. These speedups are larger to those obtained for cement system 1, with values between 8 and 9 when evaluated against HPx$_{\\rm{4C}}$. These speedups represent about 85 \\% to 90 \\% of the maximum possible speedup (Table \\ref{table3}). Overall, these findings indicate that for the considered problem, our RT-based training of a local DNN only works is the training set is sufficiently representative of the particular geochemical conditions encountered in the computationally demanding simulations, which is arguably not easy to achieve. This limitation is further discussed in section \\ref{discussion}.\n\nWe note a more uniform behavior for HPx$_{\\rm py}$-kNN across grid sizes than for HPx$_{\\rm py}$-DNN. Here the results for the 121  $\\times$ 121 grid are only slightly less accurate than those associated with the 61 $ \\times$ 61 grid (see Figures \\ref{fig15} - \\ref{fig16} where for brevity we only show concentration profiles for the C Al and S components). Furthermore, whenever observed the discrepancies between original and emulated profiles are more regularly scattered than for HPx$_{\\rm py}$-DNN. Note also that herein too, post-filtering could likely smooth out a large part of these deviations. In addition, owing to the use of a GPU to achieve the kNN calculations, the speedup provided by HPx$_{\\rm py}$-kNN are as large as those provided by HPx$_{\\rm py}$-DNN (Table \\ref{table4}).\n\nWith respect to the emulated solid amounts, the HPx$_{\\rm py}$-DNN results look visually good for the the 61 $\\times$ 61 grid. This is shown in Figure \\ref{fig17} for the C, Al and S chemical components, while emulation of the H, O Ca and Si chemical components is globally of similar quality (not shown). Nevertheless, significant deviations appear towards the end of the simulation period for every chemical component (see Figure \\ref{fig18} for the C, Al and S chemical components, emulation of the H, O Ca and Si chemical components shows the same level of mismatch). The HPx$_{\\rm py}$-kNN predictions are also fairly accurate for the 61 $\\times$ 61 (see Figure \\ref{fig19} for the C, Al and S chemical components, emulation of the H, O Ca and Si chemical components exhibits a globally similar quality) while some discrepancies show up at the end of the simulation (Figure \\ref{fig20}). However, the mismatch is less pronounced than for HPx$_{\\rm py}$-DNN. Overall, HPx$_{\\rm py}$-kNN appears to be somewhat more robust than HPx$_{\\rm py}$-DNN for this cement system, while providing the same (large) speedup.\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure11.png}\n\t\\caption{2D concentration profiles obtained for cement system 2 after 2 and 6 (final time step) years. RTM means the original HPx$_{\\rm{4C}}$ model and TM+DNN denotes the Hydrus transport model coupled with our DNN geochemical emulator (HPx$_{\\rm py}$-DNN). The first to fourth row present profiles for $Ca^{\\rm{conc}}$, $Si^{\\rm{conc}}$, $H^{\\rm{conc}}$, and $O^{\\rm{conc}}$, respectively. The considered grid size is 61 $\\times$ 61.}\n\t\\label{fig11}\n\\end{figure}\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure12.png}\n\t\\caption{2D concentration profiles obtained for cement system 2 after 2 and 6 (final time step) years. RTM means the original HPx$_{\\rm{4C}}$ model and TM+DNN denotes the Hydrus transport model coupled with our DNN geochemical emulator (HPx$_{\\rm py}$-DNN). The first to third row present profiles for $C^{\\rm{conc}}$, $Al^{\\rm{conc}}$, and $S^{\\rm{conc}}$, respectively. The considered grid size is 61 $\\times$ 61.}\n\t\\label{fig12}\n\\end{figure}\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure13.png}\n\t\\caption{2D concentration profiles obtained for cement system 2 after 2 and 6 (final time step) years. RTM means the original HPx$_{\\rm{4C}}$ model and TM+DNN denotes the Hydrus transport model coupled with our DNN geochemical emulator (HPx$_{\\rm py}$-DNN). The first to fourth row present profiles for $Ca^{\\rm{conc}}$, $Si^{\\rm{conc}}$, $H^{\\rm{conc}}$, and $O^{\\rm{conc}}$, respectively. The considered grid size is 121 $\\times$ 121.}\n\t\\label{fig13}\n\\end{figure}\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure14.png}\n\t\\caption{2D concentration profiles obtained for cement system 2 after 2 and 6 (final time step) years. RTM means the original HPx$_{\\rm{4C}}$ model and TM+DNN denotes the Hydrus transport model coupled with our DNN geochemical emulator (HPx$_{\\rm py}$-DNN). The first to third row present profiles for $C^{\\rm{conc}}$, $Al^{\\rm{conc}}$, and $S^{\\rm{conc}}$, respectively. The considered grid size is 121 $\\times$ 121.}\n\t\\label{fig14}\n\\end{figure}\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure15.png}\n\t\\caption{2D concentration profiles obtained for cement system 2 after 2 and 6 (final time step) years. RTM means the original HPx$_{\\rm{4C}}$ model and TM+kNN denotes the Hydrus transport model coupled with our kNN geochemical emulator (HPx$_{\\rm py}$-kNN).  The first to third row present profiles for $C^{\\rm{conc}}$, $Al^{\\rm{conc}}$, and $S^{\\rm{conc}}$, respectively. The considered grid size is 61 $\\times$ 61.}\n\t\\label{fig15}\n\\end{figure}\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure16.png}\n\t\\caption{2D concentration profiles obtained for cement system 2 after 2 and 6 (final time step) years. RTM means the original HPx$_{\\rm{4C}}$ model and TM+kNN denotes the Hydrus transport model coupled with our kNN geochemical emulator (HPx$_{\\rm py}$-kNN). The first to third row present profiles for $C^{\\rm{conc}}$, $Al^{\\rm{conc}}$, and $S^{\\rm{conc}}$, respectively. The considered grid size is 121 $\\times$ 121.}\n\t\\label{fig16}\n\\end{figure}\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure17.png}\n\t\\caption{2D solid amount profiles of the C, Al and S chemical components of cement system 2 after 2 and 6 (final time step) years. RTM means the original HPx$_{\\rm{4C}}$ model and TM+DNN denotes the Hydrus transport model coupled with our DNN geochemical emulator (HPx$_{\\rm py}$-DNN). The first to third row present profiles for $C^{\\rm{solid}}$, $Al^{\\rm{solid}}$, and $S^{\\rm{solid}}$, respectively The considered grid size is 61 $\\times$ 61.}\n\t\\label{fig17}\n\\end{figure}\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure18.png}\n\t\\caption{2D solid amount profiles of the C, Al and S chemical components of cement system 2 after 2 and 6 (final time step) years. RTM means the original HPx$_{\\rm{4C}}$ model and TM+DNN denotes the Hydrus transport model coupled with our DNN geochemical emulator (HPx$_{\\rm py}$-DNN). The first to third row present profiles for $C^{\\rm{solid}}$, $Al^{\\rm{solid}}$, and $S^{\\rm{solid}}$, respectively The considered grid size is 121 $\\times$ 121.}\n\t\\label{fig18}\n\\end{figure}\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure19.png}\n\t\\caption{2D solid amount profiles of the C, Al and S chemical components of cement system 2 after 2 and 6 (final time step) years. RTM means the original HPx$_{\\rm{4C}}$ model and TM+kNN denotes the Hydrus transport model coupled with our kNN geochemical emulator (HPx$_{\\rm py}$-kNN). The first to third row present profiles for $C^{\\rm{solid}}$, $Al^{\\rm{solid}}$, and $S^{\\rm{solid}}$, respectively The considered grid size is 61 $\\times$ 61.}\n\t\\label{fig19}\n\\end{figure}\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure20.png}\n\t\\caption{2D solid amount profiles of the C, Al and S chemical components of cement system 2 after 2 and 6 (final time step) years. RTM means the original HPx$_{\\rm{4C}}$ model and TM+kNN denotes the Hydrus transport model coupled with our kNN geochemical emulator (HPx$_{\\rm py}$-kNN). The first to third row present profiles for $C^{\\rm{solid}}$, $Al^{\\rm{solid}}$, and $S^{\\rm{solid}}$, respectively The considered grid size is 121 $\\times$ 121.}\n\t\\label{fig20}\n\\end{figure}\n\n\\begin{table}[h!]\n\t\\caption{Speedups offered by the KNN and DNN emulators in HPx$_{\\rm py}$ for the reactive transport simulations considered for cement system 2. The HPx$_{\\rm{4C}}$ calculations involve the parallelization of PHREEQC over our 4 CPUs. The HPx$_{\\rm{1C}}$ calculations are performed on a single CPU. Both the kNN and DNN predictions make use of a GPU. ML signifies the used machine learning method for emulation, BC denotes the type of flow boundary conditions and GS is the grid size. The maximum possible speedups associated with HPx$_{\\rm{4C}}$ and HP$_{\\rm{1C}}$,  Max SP HPx$_{\\rm{4C}}$ and  Max SP HPx$_{\\rm{1C}}$, correspond to an hypothetical situation where the geochemical calculations incur zero computational cost.}\n\n\t\\begin{center}\n\t\t\\begin{tabular}{cccccccc}\n\t\t\t\\hline\n\t\t\t& & & & & & & \\\\\n\t\t\tML & BC & GS & HPx$_{\\rm{4C}}$ time (s) & SP HPx$_{\\rm{4C}}$ & Max SP HPx$_{\\rm{4C}}$ & SP HPx$_{\\rm{1C}}$ & Max SP HPx$_{\\rm{1C}}$ \\\\\n\t\t\tDNN & ADV & 61 $\\times$ 61 & 21,415 & 8.2 & 9.0 & 30.3 & 33.1\\\\\n\t\t\tkNN & ADV & 61 $\\times$ 61 & 21,415 & 7.9 & 9.0 & 28.9 & 33.1\\\\\n\t\t\tDNN & ADV & 121 $\\times$ 121 & 199,841 & 8.2 & 9.5 & 29.9 & 32.8\\\\\n\t\t\tkNN & ADV & 121 $\\times$ 121 & 199,841 & 8.5 & 9.5 & 31.3 & 32.8\\\\\n\t\t\t\\hline\n\t\t\\end{tabular}\n\t\\end{center}\n\n\t\\label{table4}\n\\end{table}\n\n\\FloatBarrier\n\n3.2 Al-C-Ca-S-Si Problem\n\\subsection{Al-C-Ca-S-Si Problem}\n\nFor this second cement system, the emulation problem consists of predicting at each time step of the RT simulation the (output) Al, C, Ca, S, Si, H and O aqueous concentrations (mol/kgw) from the (input) total amounts of Al, C, Ca, S and Si (mol). Here we focus on advective-dispersive transport only while similarly as for cement system 1, the considered domain sizes are 61 $\\times$ 61 and 121 $\\times$ 121. Moreover, the simulation time period is set to 6 years. \n\nAs mentioned earlier, for this higher-dimensional problem it is observed that the used scikit-learn kNN implementation becomes prohibitively slow compared to HPx$_{\\rm{4C}}$. We found that to get a good emulation accuracy the kNN training base needs to contain 1,000,000 samples (or more). This training base's size together with a 5-dimensional search space leads to an HPx$_{\\rm py}$-kNN reactive transport simulation time that is comparable to that of HPx$_{\\rm{4C}}$. Therefore, for this second cement system we built a custom kNN regressor around another kNN implementation contained in the FAISS package \\citep{faiss2017}. Our used FAISS variant allows for GPU computing and is much faster than scikit-learn for this cement system, but is slightly less accurate due to the use of an approximate rather than exact nearest neighbor search \\citep[see][for details]{faiss2017}.\n\n\\subsubsection{Training the DNN}\n\\label{train_res2}\n\nBuilding a good training set to perform a kNN search and learn the weights and biases of our DNN turned out to be a complicated task in this case. This because to make useful kNN predictions and/or learn an useful DNN, the training set must be sufficiently representative of the geochemical conditions encountered during the reactive transport simulation one wish to perform with HPx$_{\\rm py}$-kNN and the trained HPx$_{\\rm py}$-DNN. In contrast to cement system 1, creating the training set by sampling the $X$-space with a controlled randomness between predefined lower and upper bounds did not prove successful. We tried that strategy by drawing as much as 4,000,000 5-dimensional $\\textbf{x}$ vectors from the $X$-space using a Sobol low-discrepancy sequence \\citep[][]{Sobol1967, Joe-Kuo2003}. Such low-discrepancy sampling scheme covers the 5-dimensional hypercube more uniformly than LHS. Despite a good performance on the test set (not shown), the resulting DNN accuracy in reactive transport mode was never deemed satisfying. In other words, no satisfying ``global\" or ``universal\" DNN emulator could be devised for this cement system. This is probably caused by the fact for this problem, the input (5 total amounts) and output (7 aqueous concentrations) spaces are quite nonlinearly related and both cover 6 to 10 orders of magnitudes depending on the considered element. Therefore, we resorted to the alternative training strategy detailed below. The latter basically tries to grasp the complex correlations and higher-order dependencies that exist between the elements of $\\textbf{x}$ (total amounts, input space) for a given reactive transport simulation setup, in order to produce a training set that honors these between-input relationships.\n\n\\begin{itemize}\n\t\\item Perform a ``cheap\" full reactive transport simulation under the transport conditions and geochemistry of interest and collect the resulting $\\textbf{x}$ -$\\textbf{y}$  pairs of examples (for the considered grid nodes and time steps). Computational demand controls what domain size and simulation time period can be used for this cheap calculation. We used a modest 16 $\\times$ 16 domain and a simulation time period of 10 years. The associated HPx$_{\\rm{4C}}$ runtimes is 180 s.\n\t\n\t\\item Fit a kernel density estimator (KDE) with a Gaussian kernel to the collected $\\textbf{x}$ vectors (encapsulated in the $\\textbf{X}$ array) and generate a fixed number of new input vectors, $\\textbf{x}_{KDE}$. Then run PHREEQC for the $\\textbf{X}_{KDE}$ set to get the corresponding output set, $\\textbf{Y}_{KDE}$. Now apply the correction for porosity described in section \\ref{train_res1} to the $\\textbf{X}_{KDE}$ set and form the training set by merging the ensemble of $\\textbf{x}$-$\\textbf{y}$ pairs with that of the $\\textbf{x}_{KDE}$-$\\textbf{y}_{KDE}$ pairs. The number of produced unique examples by the considered cheap HPx$_{\\rm{4C}}$ simulations varied between 10,000 and 50,000. The KDE-based enrichment of this dataset was deemed necessary to provide more input variability, thereby avoiding overfitting of the trained DNN and improving the kNN accuracy, while still honoring the complex between-input relationships. The number of KDE-generated samples was set as to obtain a total training set size of 1,000,000 examples. A key component of the approach is the bandwith parameter of the Gaussian KDE kernel which controls how much the KDE-generated samples depart from the original ensemble. After limited trial and error, we fixed the kernel bandwith to 0.0025 for the considered case studies.\n\t\n\\end{itemize}\n\nThe scatter plots in Figure \\ref{fig9} illustrate our training set creation procedure. The orange dots depict the pairwise relationships observed between the 5 elements of $\\textbf{x}$ in the cheap simulation. The red and cyan dots in Figure \\ref{fig9} represent the KDE-generated samples with the selected bandwidth, before and after applying the correction for porosity, respectively. Training of the DNN is achieved using the ensemble of original and KDE-corrected input points. We refer to this kind of dataset as RT-based, since it is based on an albeit cheap, full RT simulation. Furthermore, we refer to the obtained DNN and kNN emulators as ``local\" emulators, since as opposed to the emulators constructed for cement system 1, the current emulators are only valid for the input conditions encapsulated in the RT-based training set.\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1.5cm}\\includegraphics[width=50pc]{Figure9.png}\n\t\\caption{Scatter plots of the complex relationships between the five considered inputs for (1) the computationally cheap RT simulation of cement system 2 (orange dots) performed with the original HPx code, (2) the corresponding sampled points by kernel density estimation (KDE) using the selected bandwith (turquoise dots) and, (3) the same sampled KDE points after porosity correction (red dots).}\n\t\\label{fig9}\n\\end{figure}\n\nTraining performance of the DNN emulator is presented in Figure \\ref{fig10}. Similarly as for cement system 1, about 90\\% of the available data was used for the actual training of the DNN while the remaining 10\\% were used a validation set to contol overfitting. Lastly, performance is evaluated for both the trained DNN and kNN emulators using an independent test set of 10,000 examples. Overall, the accuracy of our ``local \" DNN emulator for this RT-inspired dataset is rather large with $Q_2$ values always greater or equal than 0.998. Training performance of the corresponding local kNN emulator is equally good (not shown). Regarding speedup, for this problem single-threaded PHREEQC achieves about 210 geochemical calculations per second on our used Intel\\textsuperscript{\\textregistered} i7 CPU while the GPU-based DNN and kNN emulators are both about 3000 times faster when predicting the 10,000 test points at once .\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure10.png}\n\t\\caption{1-1 plots of local DNN emulation performance obtained for system 2 when the local DNN is trained using 1,000,000 samples. The $x$-axis and $y$-axis present the original and emulated 10,000 independent test data points, respectively. The RMSE and $Q_2$ coefficient denote the root-mean-square-error and coefficient of determination in testing mode, respectively, between the original and emulated 10,000 test data points.}\n\t\\label{fig10}\n\\end{figure}\n\n\\subsubsection{Reactive transport simulation}\n\\label{prob2res}\nOur ``local\" DNN performs rather well when applied to the 61 $\\times$ 61 grid size and a time period of 6 years (Figures \\ref{fig11} - \\ref{fig12}). This for all components but C, for which some localized deviations appear between original and emulated 2D concentration profiles towards the end of the simulation period (Figure \\ref{fig12}). When applied to the 121 $\\times$ 121 grid, HPx$_{\\rm py}$-DNN produces additional discrepancies for O, Si and Al towards the end of the simulation period (Figures \\ref{fig13} - \\ref{fig14}). Yet most of the observed artifacts could probably be smoothed out by using post-filtering. The associated speedups are listed in Table \\ref{table4}. These speedups are larger to those obtained for cement system 1, with values between 8 and 9 when evaluated against HPx$_{\\rm{4C}}$. These speedups represent about 85 \\% to 90 \\% of the maximum possible speedup (Table \\ref{table3}). Overall, these findings indicate that for the considered problem, our RT-based training of a local DNN only works is the training set is sufficiently representative of the particular geochemical conditions encountered in the computationally demanding simulations, which is arguably not easy to achieve. This limitation is further discussed in section \\ref{discussion}.\n\nWe note a more uniform behavior for HPx$_{\\rm py}$-kNN across grid sizes than for HPx$_{\\rm py}$-DNN. Here the results for the 121  $\\times$ 121 grid are only slightly less accurate than those associated with the 61 $ \\times$ 61 grid (see Figures \\ref{fig15} - \\ref{fig16} where for brevity we only show concentration profiles for the C Al and S components). Furthermore, whenever observed the discrepancies between original and emulated profiles are more regularly scattered than for HPx$_{\\rm py}$-DNN. Note also that herein too, post-filtering could likely smooth out a large part of these deviations. In addition, owing to the use of a GPU to achieve the kNN calculations, the speedup provided by HPx$_{\\rm py}$-kNN are as large as those provided by HPx$_{\\rm py}$-DNN (Table \\ref{table4}).\n\nWith respect to the emulated solid amounts, the HPx$_{\\rm py}$-DNN results look visually good for the the 61 $\\times$ 61 grid. This is shown in Figure \\ref{fig17} for the C, Al and S chemical components, while emulation of the H, O Ca and Si chemical components is globally of similar quality (not shown). Nevertheless, significant deviations appear towards the end of the simulation period for every chemical component (see Figure \\ref{fig18} for the C, Al and S chemical components, emulation of the H, O Ca and Si chemical components shows the same level of mismatch). The HPx$_{\\rm py}$-kNN predictions are also fairly accurate for the 61 $\\times$ 61 (see Figure \\ref{fig19} for the C, Al and S chemical components, emulation of the H, O Ca and Si chemical components exhibits a globally similar quality) while some discrepancies show up at the end of the simulation (Figure \\ref{fig20}). However, the mismatch is less pronounced than for HPx$_{\\rm py}$-DNN. Overall, HPx$_{\\rm py}$-kNN appears to be somewhat more robust than HPx$_{\\rm py}$-DNN for this cement system, while providing the same (large) speedup.\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure11.png}\n\t\\caption{2D concentration profiles obtained for cement system 2 after 2 and 6 (final time step) years. RTM means the original HPx$_{\\rm{4C}}$ model and TM+DNN denotes the Hydrus transport model coupled with our DNN geochemical emulator (HPx$_{\\rm py}$-DNN). The first to fourth row present profiles for $Ca^{\\rm{conc}}$, $Si^{\\rm{conc}}$, $H^{\\rm{conc}}$, and $O^{\\rm{conc}}$, respectively. The considered grid size is 61 $\\times$ 61.}\n\t\\label{fig11}\n\\end{figure}\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure12.png}\n\t\\caption{2D concentration profiles obtained for cement system 2 after 2 and 6 (final time step) years. RTM means the original HPx$_{\\rm{4C}}$ model and TM+DNN denotes the Hydrus transport model coupled with our DNN geochemical emulator (HPx$_{\\rm py}$-DNN). The first to third row present profiles for $C^{\\rm{conc}}$, $Al^{\\rm{conc}}$, and $S^{\\rm{conc}}$, respectively. The considered grid size is 61 $\\times$ 61.}\n\t\\label{fig12}\n\\end{figure}\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure13.png}\n\t\\caption{2D concentration profiles obtained for cement system 2 after 2 and 6 (final time step) years. RTM means the original HPx$_{\\rm{4C}}$ model and TM+DNN denotes the Hydrus transport model coupled with our DNN geochemical emulator (HPx$_{\\rm py}$-DNN). The first to fourth row present profiles for $Ca^{\\rm{conc}}$, $Si^{\\rm{conc}}$, $H^{\\rm{conc}}$, and $O^{\\rm{conc}}$, respectively. The considered grid size is 121 $\\times$ 121.}\n\t\\label{fig13}\n\\end{figure}\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure14.png}\n\t\\caption{2D concentration profiles obtained for cement system 2 after 2 and 6 (final time step) years. RTM means the original HPx$_{\\rm{4C}}$ model and TM+DNN denotes the Hydrus transport model coupled with our DNN geochemical emulator (HPx$_{\\rm py}$-DNN). The first to third row present profiles for $C^{\\rm{conc}}$, $Al^{\\rm{conc}}$, and $S^{\\rm{conc}}$, respectively. The considered grid size is 121 $\\times$ 121.}\n\t\\label{fig14}\n\\end{figure}\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure15.png}\n\t\\caption{2D concentration profiles obtained for cement system 2 after 2 and 6 (final time step) years. RTM means the original HPx$_{\\rm{4C}}$ model and TM+kNN denotes the Hydrus transport model coupled with our kNN geochemical emulator (HPx$_{\\rm py}$-kNN).  The first to third row present profiles for $C^{\\rm{conc}}$, $Al^{\\rm{conc}}$, and $S^{\\rm{conc}}$, respectively. The considered grid size is 61 $\\times$ 61.}\n\t\\label{fig15}\n\\end{figure}\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure16.png}\n\t\\caption{2D concentration profiles obtained for cement system 2 after 2 and 6 (final time step) years. RTM means the original HPx$_{\\rm{4C}}$ model and TM+kNN denotes the Hydrus transport model coupled with our kNN geochemical emulator (HPx$_{\\rm py}$-kNN). The first to third row present profiles for $C^{\\rm{conc}}$, $Al^{\\rm{conc}}$, and $S^{\\rm{conc}}$, respectively. The considered grid size is 121 $\\times$ 121.}\n\t\\label{fig16}\n\\end{figure}\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure17.png}\n\t\\caption{2D solid amount profiles of the C, Al and S chemical components of cement system 2 after 2 and 6 (final time step) years. RTM means the original HPx$_{\\rm{4C}}$ model and TM+DNN denotes the Hydrus transport model coupled with our DNN geochemical emulator (HPx$_{\\rm py}$-DNN). The first to third row present profiles for $C^{\\rm{solid}}$, $Al^{\\rm{solid}}$, and $S^{\\rm{solid}}$, respectively The considered grid size is 61 $\\times$ 61.}\n\t\\label{fig17}\n\\end{figure}\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure18.png}\n\t\\caption{2D solid amount profiles of the C, Al and S chemical components of cement system 2 after 2 and 6 (final time step) years. RTM means the original HPx$_{\\rm{4C}}$ model and TM+DNN denotes the Hydrus transport model coupled with our DNN geochemical emulator (HPx$_{\\rm py}$-DNN). The first to third row present profiles for $C^{\\rm{solid}}$, $Al^{\\rm{solid}}$, and $S^{\\rm{solid}}$, respectively The considered grid size is 121 $\\times$ 121.}\n\t\\label{fig18}\n\\end{figure}\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure19.png}\n\t\\caption{2D solid amount profiles of the C, Al and S chemical components of cement system 2 after 2 and 6 (final time step) years. RTM means the original HPx$_{\\rm{4C}}$ model and TM+kNN denotes the Hydrus transport model coupled with our kNN geochemical emulator (HPx$_{\\rm py}$-kNN). The first to third row present profiles for $C^{\\rm{solid}}$, $Al^{\\rm{solid}}$, and $S^{\\rm{solid}}$, respectively The considered grid size is 61 $\\times$ 61.}\n\t\\label{fig19}\n\\end{figure}\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure20.png}\n\t\\caption{2D solid amount profiles of the C, Al and S chemical components of cement system 2 after 2 and 6 (final time step) years. RTM means the original HPx$_{\\rm{4C}}$ model and TM+kNN denotes the Hydrus transport model coupled with our kNN geochemical emulator (HPx$_{\\rm py}$-kNN). The first to third row present profiles for $C^{\\rm{solid}}$, $Al^{\\rm{solid}}$, and $S^{\\rm{solid}}$, respectively The considered grid size is 121 $\\times$ 121.}\n\t\\label{fig20}\n\\end{figure}\n\n\\begin{table}[h!]\n\t\\caption{Speedups offered by the KNN and DNN emulators in HPx$_{\\rm py}$ for the reactive transport simulations considered for cement system 2. The HPx$_{\\rm{4C}}$ calculations involve the parallelization of PHREEQC over our 4 CPUs. The HPx$_{\\rm{1C}}$ calculations are performed on a single CPU. Both the kNN and DNN predictions make use of a GPU. ML signifies the used machine learning method for emulation, BC denotes the type of flow boundary conditions and GS is the grid size. The maximum possible speedups associated with HPx$_{\\rm{4C}}$ and HP$_{\\rm{1C}}$,  Max SP HPx$_{\\rm{4C}}$ and  Max SP HPx$_{\\rm{1C}}$, correspond to an hypothetical situation where the geochemical calculations incur zero computational cost.}\n\n\t\\begin{center}\n\t\t\\begin{tabular}{cccccccc}\n\t\t\t\\hline\n\t\t\t& & & & & & & \\\\\n\t\t\tML & BC & GS & HPx$_{\\rm{4C}}$ time (s) & SP HPx$_{\\rm{4C}}$ & Max SP HPx$_{\\rm{4C}}$ & SP HPx$_{\\rm{1C}}$ & Max SP HPx$_{\\rm{1C}}$ \\\\\n\t\t\tDNN & ADV & 61 $\\times$ 61 & 21,415 & 8.2 & 9.0 & 30.3 & 33.1\\\\\n\t\t\tkNN & ADV & 61 $\\times$ 61 & 21,415 & 7.9 & 9.0 & 28.9 & 33.1\\\\\n\t\t\tDNN & ADV & 121 $\\times$ 121 & 199,841 & 8.2 & 9.5 & 29.9 & 32.8\\\\\n\t\t\tkNN & ADV & 121 $\\times$ 121 & 199,841 & 8.5 & 9.5 & 31.3 & 32.8\\\\\n\t\t\t\\hline\n\t\t\\end{tabular}\n\t\\end{center}\n\n\t\\label{table4}\n\\end{table}\n\n\\FloatBarrier\n\n3.2.2 Reactive transport simulation\n\\subsubsection{Reactive transport simulation}\n\\label{prob2res}\nOur ``local\" DNN performs rather well when applied to the 61 $\\times$ 61 grid size and a time period of 6 years (Figures \\ref{fig11} - \\ref{fig12}). This for all components but C, for which some localized deviations appear between original and emulated 2D concentration profiles towards the end of the simulation period (Figure \\ref{fig12}). When applied to the 121 $\\times$ 121 grid, HPx$_{\\rm py}$-DNN produces additional discrepancies for O, Si and Al towards the end of the simulation period (Figures \\ref{fig13} - \\ref{fig14}). Yet most of the observed artifacts could probably be smoothed out by using post-filtering. The associated speedups are listed in Table \\ref{table4}. These speedups are larger to those obtained for cement system 1, with values between 8 and 9 when evaluated against HPx$_{\\rm{4C}}$. These speedups represent about 85 \\% to 90 \\% of the maximum possible speedup (Table \\ref{table3}). Overall, these findings indicate that for the considered problem, our RT-based training of a local DNN only works is the training set is sufficiently representative of the particular geochemical conditions encountered in the computationally demanding simulations, which is arguably not easy to achieve. This limitation is further discussed in section \\ref{discussion}.\n\nWe note a more uniform behavior for HPx$_{\\rm py}$-kNN across grid sizes than for HPx$_{\\rm py}$-DNN. Here the results for the 121  $\\times$ 121 grid are only slightly less accurate than those associated with the 61 $ \\times$ 61 grid (see Figures \\ref{fig15} - \\ref{fig16} where for brevity we only show concentration profiles for the C Al and S components). Furthermore, whenever observed the discrepancies between original and emulated profiles are more regularly scattered than for HPx$_{\\rm py}$-DNN. Note also that herein too, post-filtering could likely smooth out a large part of these deviations. In addition, owing to the use of a GPU to achieve the kNN calculations, the speedup provided by HPx$_{\\rm py}$-kNN are as large as those provided by HPx$_{\\rm py}$-DNN (Table \\ref{table4}).\n\nWith respect to the emulated solid amounts, the HPx$_{\\rm py}$-DNN results look visually good for the the 61 $\\times$ 61 grid. This is shown in Figure \\ref{fig17} for the C, Al and S chemical components, while emulation of the H, O Ca and Si chemical components is globally of similar quality (not shown). Nevertheless, significant deviations appear towards the end of the simulation period for every chemical component (see Figure \\ref{fig18} for the C, Al and S chemical components, emulation of the H, O Ca and Si chemical components shows the same level of mismatch). The HPx$_{\\rm py}$-kNN predictions are also fairly accurate for the 61 $\\times$ 61 (see Figure \\ref{fig19} for the C, Al and S chemical components, emulation of the H, O Ca and Si chemical components exhibits a globally similar quality) while some discrepancies show up at the end of the simulation (Figure \\ref{fig20}). However, the mismatch is less pronounced than for HPx$_{\\rm py}$-DNN. Overall, HPx$_{\\rm py}$-kNN appears to be somewhat more robust than HPx$_{\\rm py}$-DNN for this cement system, while providing the same (large) speedup.\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure11.png}\n\t\\caption{2D concentration profiles obtained for cement system 2 after 2 and 6 (final time step) years. RTM means the original HPx$_{\\rm{4C}}$ model and TM+DNN denotes the Hydrus transport model coupled with our DNN geochemical emulator (HPx$_{\\rm py}$-DNN). The first to fourth row present profiles for $Ca^{\\rm{conc}}$, $Si^{\\rm{conc}}$, $H^{\\rm{conc}}$, and $O^{\\rm{conc}}$, respectively. The considered grid size is 61 $\\times$ 61.}\n\t\\label{fig11}\n\\end{figure}\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure12.png}\n\t\\caption{2D concentration profiles obtained for cement system 2 after 2 and 6 (final time step) years. RTM means the original HPx$_{\\rm{4C}}$ model and TM+DNN denotes the Hydrus transport model coupled with our DNN geochemical emulator (HPx$_{\\rm py}$-DNN). The first to third row present profiles for $C^{\\rm{conc}}$, $Al^{\\rm{conc}}$, and $S^{\\rm{conc}}$, respectively. The considered grid size is 61 $\\times$ 61.}\n\t\\label{fig12}\n\\end{figure}\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure13.png}\n\t\\caption{2D concentration profiles obtained for cement system 2 after 2 and 6 (final time step) years. RTM means the original HPx$_{\\rm{4C}}$ model and TM+DNN denotes the Hydrus transport model coupled with our DNN geochemical emulator (HPx$_{\\rm py}$-DNN). The first to fourth row present profiles for $Ca^{\\rm{conc}}$, $Si^{\\rm{conc}}$, $H^{\\rm{conc}}$, and $O^{\\rm{conc}}$, respectively. The considered grid size is 121 $\\times$ 121.}\n\t\\label{fig13}\n\\end{figure}\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure14.png}\n\t\\caption{2D concentration profiles obtained for cement system 2 after 2 and 6 (final time step) years. RTM means the original HPx$_{\\rm{4C}}$ model and TM+DNN denotes the Hydrus transport model coupled with our DNN geochemical emulator (HPx$_{\\rm py}$-DNN). The first to third row present profiles for $C^{\\rm{conc}}$, $Al^{\\rm{conc}}$, and $S^{\\rm{conc}}$, respectively. The considered grid size is 121 $\\times$ 121.}\n\t\\label{fig14}\n\\end{figure}\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure15.png}\n\t\\caption{2D concentration profiles obtained for cement system 2 after 2 and 6 (final time step) years. RTM means the original HPx$_{\\rm{4C}}$ model and TM+kNN denotes the Hydrus transport model coupled with our kNN geochemical emulator (HPx$_{\\rm py}$-kNN).  The first to third row present profiles for $C^{\\rm{conc}}$, $Al^{\\rm{conc}}$, and $S^{\\rm{conc}}$, respectively. The considered grid size is 61 $\\times$ 61.}\n\t\\label{fig15}\n\\end{figure}\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure16.png}\n\t\\caption{2D concentration profiles obtained for cement system 2 after 2 and 6 (final time step) years. RTM means the original HPx$_{\\rm{4C}}$ model and TM+kNN denotes the Hydrus transport model coupled with our kNN geochemical emulator (HPx$_{\\rm py}$-kNN). The first to third row present profiles for $C^{\\rm{conc}}$, $Al^{\\rm{conc}}$, and $S^{\\rm{conc}}$, respectively. The considered grid size is 121 $\\times$ 121.}\n\t\\label{fig16}\n\\end{figure}\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure17.png}\n\t\\caption{2D solid amount profiles of the C, Al and S chemical components of cement system 2 after 2 and 6 (final time step) years. RTM means the original HPx$_{\\rm{4C}}$ model and TM+DNN denotes the Hydrus transport model coupled with our DNN geochemical emulator (HPx$_{\\rm py}$-DNN). The first to third row present profiles for $C^{\\rm{solid}}$, $Al^{\\rm{solid}}$, and $S^{\\rm{solid}}$, respectively The considered grid size is 61 $\\times$ 61.}\n\t\\label{fig17}\n\\end{figure}\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure18.png}\n\t\\caption{2D solid amount profiles of the C, Al and S chemical components of cement system 2 after 2 and 6 (final time step) years. RTM means the original HPx$_{\\rm{4C}}$ model and TM+DNN denotes the Hydrus transport model coupled with our DNN geochemical emulator (HPx$_{\\rm py}$-DNN). The first to third row present profiles for $C^{\\rm{solid}}$, $Al^{\\rm{solid}}$, and $S^{\\rm{solid}}$, respectively The considered grid size is 121 $\\times$ 121.}\n\t\\label{fig18}\n\\end{figure}\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure19.png}\n\t\\caption{2D solid amount profiles of the C, Al and S chemical components of cement system 2 after 2 and 6 (final time step) years. RTM means the original HPx$_{\\rm{4C}}$ model and TM+kNN denotes the Hydrus transport model coupled with our kNN geochemical emulator (HPx$_{\\rm py}$-kNN). The first to third row present profiles for $C^{\\rm{solid}}$, $Al^{\\rm{solid}}$, and $S^{\\rm{solid}}$, respectively The considered grid size is 61 $\\times$ 61.}\n\t\\label{fig19}\n\\end{figure}\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure20.png}\n\t\\caption{2D solid amount profiles of the C, Al and S chemical components of cement system 2 after 2 and 6 (final time step) years. RTM means the original HPx$_{\\rm{4C}}$ model and TM+kNN denotes the Hydrus transport model coupled with our kNN geochemical emulator (HPx$_{\\rm py}$-kNN). The first to third row present profiles for $C^{\\rm{solid}}$, $Al^{\\rm{solid}}$, and $S^{\\rm{solid}}$, respectively The considered grid size is 121 $\\times$ 121.}\n\t\\label{fig20}\n\\end{figure}\n\n\\begin{table}[h!]\n\t\\caption{Speedups offered by the KNN and DNN emulators in HPx$_{\\rm py}$ for the reactive transport simulations considered for cement system 2. The HPx$_{\\rm{4C}}$ calculations involve the parallelization of PHREEQC over our 4 CPUs. The HPx$_{\\rm{1C}}$ calculations are performed on a single CPU. Both the kNN and DNN predictions make use of a GPU. ML signifies the used machine learning method for emulation, BC denotes the type of flow boundary conditions and GS is the grid size. The maximum possible speedups associated with HPx$_{\\rm{4C}}$ and HP$_{\\rm{1C}}$,  Max SP HPx$_{\\rm{4C}}$ and  Max SP HPx$_{\\rm{1C}}$, correspond to an hypothetical situation where the geochemical calculations incur zero computational cost.}\n\n\t\\begin{center}\n\t\t\\begin{tabular}{cccccccc}\n\t\t\t\\hline\n\t\t\t& & & & & & & \\\\\n\t\t\tML & BC & GS & HPx$_{\\rm{4C}}$ time (s) & SP HPx$_{\\rm{4C}}$ & Max SP HPx$_{\\rm{4C}}$ & SP HPx$_{\\rm{1C}}$ & Max SP HPx$_{\\rm{1C}}$ \\\\\n\t\t\tDNN & ADV & 61 $\\times$ 61 & 21,415 & 8.2 & 9.0 & 30.3 & 33.1\\\\\n\t\t\tkNN & ADV & 61 $\\times$ 61 & 21,415 & 7.9 & 9.0 & 28.9 & 33.1\\\\\n\t\t\tDNN & ADV & 121 $\\times$ 121 & 199,841 & 8.2 & 9.5 & 29.9 & 32.8\\\\\n\t\t\tkNN & ADV & 121 $\\times$ 121 & 199,841 & 8.5 & 9.5 & 31.3 & 32.8\\\\\n\t\t\t\\hline\n\t\t\\end{tabular}\n\t\\end{center}\n\n\t\\label{table4}\n\\end{table}\n\n\\FloatBarrier\n\n",
        "label": "fig19",
        "Descriptive_question1": "Which chemical component is represented in the first row of figure_19?",
        "Descriptive_question2": "What is the grid size used in figure_19?",
        "Reasoning_question1": "How does the solid amount profile of Aluminum (Al) change between 2 and 6 years in figure_19, and what might this indicate about the cement system's behavior over time?",
        "Reasoning_question2": "Comparing the RTM and TM+kNN models in figure_19, what can be inferred about the accuracy of the kNN geochemical emulator for predicting solid amount profiles of Carbon (C) after 6 years?",
        "Descriptive_answer1": "C",
        "Descriptive_answer2": "61 \\times 61",
        "Reasoning_answer1": "Between 2 and 6 years, the solid amount profile of Aluminum (Al) in figure_19 shows a noticeable change, with decreased solid amounts in certain regions possibly indicating dissolution or reaction over time. This suggests that the cement system undergoes chemical transformations impacting Aluminum's solid phases during the 6-year period.",
        "Reasoning_answer2": "Comparing RTM and TM+kNN models in figure_19 after 6 years, the solid amount profiles of Carbon (C) appear visually similar with minor discrepancies. This implies that the kNN geochemical emulator predicts the solid Carbon amounts with good accuracy, capturing the spatial distribution and changes effectively, although slight deviations may exist."
    },
    {
        "paper_id": "2107.07598.json",
        "image_id": "figure_20",
        "image_path": [
            "/home/yz979/scratch/chengye/scimolmo/test-images-tables/dataset_imagestables/2107.07598/images/Figure20.png"
        ],
        "caption": "2D solid amount profiles of the C, Al and S chemical components of cement system 2 after 2 and 6 (final time step) years. RTM means the original HPx$_{\\rm{4C}}$ model and TM+kNN denotes the Hydrus transport model coupled with our kNN geochemical emulator (HPx$_{\\rm py}$-kNN). The first to third row present profiles for $C^{\\rm{solid}}$, $Al^{\\rm{solid}}$, and $S^{\\rm{solid}}$, respectively The considered grid size is 121 $\\times$ 121.",
        "classify": "Chart",
        "section_info": "3 Results\n\\section{Results}\n\\label{results}\n\n\\subsection{Ca-Si Problem}\nFor this first cement system, the emulation problem consists of predicting at each time step of the RT simulation the (output) Ca, Si, H and O aqueous concentrations (mol/kg of water or mol/kgw) from the (input) total amounts of Ca and Si (mol). \n\n\\subsubsection{Training the emulators}\n\\label{train_res1}\nHere the kNN and DNN emulators are firstly trained using a set of 400,000 test examples for both. This training set is obtained by randomly sampling the two-dimensional input space by latin hypercube sampling (LHS) between $\\left[0,0\\right]$ and $\\rm{\\left[Ca^{tot}_{max},Si^{tot}_{max}\\right]}$, and running PHREEQC for each input sample, $\\rm{\\textbf{x}_i = \\left[Ca^{tot}_i, Si^{tot}_i\\right]}$ to get the corresponding output vectors, $\\rm{\\textbf{y}_i = \\left[Ca^{conc}_i,Si^{conc}_i,H^{conc}_i,O^{conc}_i\\right]}$. The upper bounds, $\\rm{Ca^{tot}_{max}}$ and $\\rm{Si^{tot}_{max}}$ are defined based on a cheap full RT simulation with advective-dispersive transport using a small 1D domain of 51 nodes. It is worth noting that the total amounts of $\\textbf{x}_i$, corresponding to the PHREEQC-simulated concentrations, $\\textbf{y}_i$, have to be corrected for the different amount of water between the training set and the transport simulations. Doing so, it turns out that about 20 \\% of the post-corrected $x_i$ values exceed their pre-defined upper bounds and these excessively large values need to be filtered out. Creating the 400,000 training examples thus required about 500,000 PHREEQC runs. As stated earlier, for this problem single-threaded PHREEQC performs about 670 geochemical calculations per second on our used Intel\\textsuperscript{\\textregistered} i7 CPU.\n\nWith respect to kNN, the tuning parameters are the number of neighbors, $k$, the type of distance measure, and the interpolation technique. We simply used the default settings: $k = 5$, euclidean distance and inverse-distance interpolation. Regarding training of the DNN, the 400,000 sample were split between the training set itself (90 \\% of the data) and a validation set (10 \\% of the data). The latter serves to monitor the evolution of the selected mean squared error loss function on samples that are not used for training, thereby detecting potential overfitting. If the validation loss stops decreasing before the fixed number of epochs has been completed, then  training is stopped. Importantly, the emulation is achieved in log-space for both the input, $X$ and output $Y$, domains. This because total amounts and concentrations of the involved components typically cover many orders of magnitudes (up to 10 orders or more). Using a DNN also requires some form of data normalization or standardization. Here both the $\\rm{log\\left(\\textbf{x}_i\\right)}$ and $\\rm{log\\left(\\textbf{y}_i\\right)}$ vectors are standardized around 0 with standard deviation of 1.\n\nFigure \\ref{fig1} illustrates the trained emulators' performance for geochemical predictions using an independent test set that comprises 10,000 test examples. Both kNN (Figures \\ref{fig1}a - d) and DNN (Figures \\ref{fig1}e - h) appear to be rather accurate . DNN also shows a slight degradation for the larger concentration values  (Figures \\ref{fig1}e - h). The latter is likely due to the combination of a small proportion of large concentration values in the training set with the log-transformation that implicitly pushes the DNN to try harder to fit the smaller concentrations during training. Regarding speedup and as written earlier, for this setup the single-threaded kNN method is 300 times faster than single-threaded PHREEQC for predicting the 10,000 concentration vectors all at once. The computational savings allowed by the DNN emulator when ran on our GPU is higher with a speedup as large as 4000 for predicting the same 10,000 concentration vectors all at once. \n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{0cm}\\includegraphics[width=35pc]{Figure1.png}\n\t\\caption{1-1 plots of the kNN (subfigures (a) - (d)) and DNN (subfigures (e) - (h)) emulators' performance obtained for system 1 when the kNN training base contains 400,000 samples and the DNN is trained using the same 400,000 samples. Here ``true\" means the original PHREEQC-simulated data and ``predicted\" denotes the emulated (that is, kNN-simulated and DNN-simulated) data. Hence, the $x$-axis and $y$-axis present the original and emulated 10,000 independent test data points, respectively. The RMSE and $Q_2$ coefficient denote the root-mean-square-error and coefficient of determination in testing mode, respectively, between the original and emulated 10,000 test data points.}\n\t\\label{fig1}\n\\end{figure}\n\nTo test the sensitivity of the emulators' performance to the training set size, training was also performed using reduced training sets comprising 100,000, 10,000, 1000 and 100 samples, respectively. It seen that the DNN performance achieved  when using 10,000 training samples is virtually the same as that obtained when using 400,000 training samples (Figure \\ref{fig2} and Table \\ref{table2}). It is only for training sets smaller than 1000 samples that the DNN performance starts to degrade significantly (Table \\ref{table2}). In contrast, the behavior of kNN appears to markedly decrease as the training set gets smaller (Figure \\ref{fig2} and Table \\ref{table1}).\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{0cm}\\includegraphics[width=35pc]{Figure2.png}\n\t\\caption{1-1 plots of the kNN (subfigures (a) - (d)) and DNN (subfigures (e) - (h)) emulators' performance obtained for system 1 when the kNN training base contains 10,000 samples and the DNN is trained using the same 10,000 samples. Here ``true\" means the original PHREEQC-simulated data and ``predicted\" denotes the emulated (that is, kNN-simulated and DNN-simulated) data. Hence, the $x$-axis and $y$-axis present the original and emulated 10,000 independent test data points, respectively. The RMSE and $Q_2$ coefficient denote the root-mean-square-error and coefficient of determination in testing mode, respectively, between the original and emulated 10,000 test data points.}\n\t\\label{fig2}\n\\end{figure}\n\n\\begin{table}[h!]\n\t\\caption{Performance of the DNN and kNN emulators for cement system 1 and different training set sizes. For brevity, only the results for Ca$^{conc}$ and Si$^{conc}$ are shown. The units are mol per kg of water (mol/kgw). ML refers to the type of emulator, TR signifies the size of the training set, and the RMSE and $Q_2$ coefficient denote the root-mean-square-error and coefficient of determination in testing mode, respectively, between the original and emulated 10,000 test data points.}\n\t\\begin{center}\n\t\t\\begin{tabular}{cccccccc}\n\t\t\t\\hline\n\t\t\t& & & & & \\\\\n\t\t\tML & TR & RMSE - Ca$^{conc}$ & $Q_2$ - Ca$^{conc}$ & RMSE - Si$^{conc}$ & $Q_2$ - Si$^{conc}$\\\\\n\t\t\tDNN & 4 $\\times$ 10$^{5}$ & 4.50 $\\times$ 10$^{-5}$ & 0.9999 & 1.21 $\\times$ 10$^{-6}$ & 0.9999\\\\\n\t\t\tDNN & 1 $\\times$ 10$^{5}$ & 5.06 $\\times$ 10$^{-5}$ & 0.9999 & 1.30 $\\times$ 10$^{-6}$ & 0.9999\\\\\n\t\t\tDNN & 1 $\\times$ 10$^{4}$ & 4.57 $\\times$ 10$^{-5}$ & 0.9999 & 1.48 $\\times$ 10$^{-6}$ & 0.9999\\\\\n\t\t\tDNN & 1 $\\times$ 10$^{3}$ & 8.02 $\\times$ 10$^{-5}$ & 0.9998 & 3.31 $\\times$ 10$^{-6}$ & 0.9994\\\\\n\t\t\tDNN & 1 $\\times$ 10$^{2}$ & 47.1 $\\times$ 10$^{-5}$ & 0.9935 & 9.02 $\\times$ 10$^{-6}$ & 0.9959\\\\\n\t\t\tkNN & 4 $\\times$ 10$^{5}$ & 2.52 $\\times$ 10$^{-5}$ & 1.0000 & 1.59 $\\times$ 10$^{-6}$ & 0.9999\\\\\n\t\t\tkNN & 1 $\\times$ 10$^{5}$ & 6.27 $\\times$ 10$^{-5}$ & 0.9999 & 4.20 $\\times$ 10$^{-6}$ & 0.9991\\\\\n\t\t\tkNN & 1 $\\times$ 10$^{4}$ & 14.1 $\\times$ 10$^{-5}$ & 0.9994 & 13.5 $\\times$ 10$^{-6}$ & 0.9908\\\\\n\t\t\tkNN & 1 $\\times$ 10$^{3}$ & 43.9 $\\times$ 10$^{-5}$ & 0.9944 & 74.6 $\\times$ 10$^{-6}$ & 0.7180\\\\\n\t\t\tkNN & 1 $\\times$ 10$^{2}$  & 96.4 $\\times$ 10$^{-5}$ & 0.9729 & 55.0 $\\times$ 10$^{-6}$ & 0.8468\\\\\n\t\t\\end{tabular}\n\t\\end{center}\n\t\\label{table2}\n\\end{table}\n\n\\FloatBarrier\n\n\\subsubsection{Reactive transport simulation}\n\\label{prob12Dres}\n\nThis section focuses on reactive transport simulations with HPx$_{\\rm py}$ within cement system 1, under both advective-dispersive and diffusive transport conditions. As written above, the domain sizes are both 61 $\\times$ 61 and 121 $\\times$ 121 for the advection-dispersion case and, because of computational constraints, solely 61 $\\times$ 61  for the diffusion case. In addition, the simulation time period is 2 years for the advection-dispersion case and 1 year for the diffusion case. Figures \\ref{fig3} and \\ref{fig4} present times series of original and emulated Ca, Si, H and O concentrations at 5 locations within the 2D domain for advective-dispersive transport conditions, for both our kNN-based (HPx$_{\\rm py}$-kNN) and DNN-based (HPx$_{\\rm py}$-DNN) reactive transport codes. It is seen that HPx$_{\\rm py}$-kNN and HPx$_{\\rm py}$-DNN both induce a quite good simulation accuracy. Also, the results for the diffusive transport case are of similarly good quality (not shown). Figures \\ref{fig5} - \\ref{fig6} provide more insights into the HPx$_{\\rm py}$-kNN and HPx$_{\\rm py}$-DNN performances by displaying 2D Ca, Si, H and O concentration profiles at a given time. For each experiment and chemical component, this time is selected as to be well representative of the simulated dynamics. It is observed that the original and emulated images are visually almost indistinguishable for the advection-dispersion case (Figure \\ref{fig5}). For the diffusion case, the emulators also perform quite well for Ca, H and O (Figures\\ref{fig6}a - c, g - i and j - l), while some slight to moderate discrepancies appear at the concentration front for Si (Figures \\ref{fig6}d - f). Notwithstanding, the Si concentration remains globally well predicted. Furthermore, Figures \\ref{fig7} - \\ref{fig8} present the original and emulated 2D solid amount profiles corresponding to Figures \\ref{fig5} - \\ref{fig6}. The original solid amount profiles are overall well approximated by HPx$_{\\rm py}$-kNN and HPx$_{\\rm py}$-DNN for the advection-dispersion case (Figure \\ref{fig7}), even though some mismatch appears at the border of the fully depleted zone for the H component. As of the diffusion case (Figure \\ref{fig8}), the same kind of mismatch is observed for the emulated solid amounts of H by HPx$_{\\rm py}$-kNN while the emulated profiles by HPx$_{\\rm py}$-DNN show some more discrepancies. Though we decided to present raw emulation results, we would like to stress that some if not all of the observed artifacts could likely be smoothed out by using some post-filtering such as median filtering.\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure3.png}\n\t\\caption{Time series of original (RTM, solid green lines) and HPx$_{\\rm py}$-kNN emulated (TM+kNN, dashed orange lines) concentrations (mol/kg) of Ca, H, O and Si at selected observations points for cement system 1 and advective-dispersive transport. Obs. 1 - 5 denote the selected observation points, with the following $\\left[x,y\\right]$ locations (in cm). Obs. 1:  $\\left[0.5, 2.5\\right]$, Obs. 2: $\\left[1, 2\\right]$, Obs. 3: $\\left[2, 2\\right]$, Obs. 4: $\\left[1, 1\\right]$, Obs. 5: $\\left[2, 1\\right]$. The results for the 121 $\\times$ 121 grid size are rather similar.}\n\t\\label{fig3}\n\\end{figure}\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure4.png}\n\t\\caption{Time series of original (RTM, solid green lines) and HPx$_{\\rm py}$-DNN emulated (TM+DNN, dashed orange lines) concentrations (mol/kg) of Ca, H, O and Si at selected observations points for cement system 1 and advective-dispersive transport. Obs. 1 - 5 denote the selected observation points, with the following $\\left[x,y\\right]$ locations (in cm). Obs. 1:  $\\left[0.5, 2.5\\right]$, Obs. 2: $\\left[1, 2\\right]$, Obs. 3: $\\left[2, 2\\right]$, Obs. 4: $\\left[1, 1\\right]$, Obs. 5: $\\left[2, 1\\right]$. The results for the 121 $\\times$ 121 grid size are rather similar.}\n\t\\label{fig4}\n\\end{figure}\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure5.png}\n\t\\caption{2D concentration profiles obtained for cement system 1 at the end of the 2-year simulation performed for the advection-dispersion case. RTM means the original HP$_{\\rm{4C}}$ model, TM+kNN denotes the Hydrus transport model coupled with our kNN geochemical emulator (HPx$_{\\rm py}$-kNN) and TM+DNN signifies the Hydrus transport model coupled with our DNN geochemical emulator (HPx$_{\\rm py}$-DNN). The first to fourth row present profiles for $Ca^{\\rm{conc}}$, $Si^{\\rm{conc}}$, $H^{\\rm{conc}}$, and $O^{\\rm{conc}}$, respectively. The considered grid size is 61 $\\times$ 61. The results for the 121 $\\times$ 121 grid are rather similar.}\n\t\\label{fig5}\n\\end{figure}\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure6.png}\n\t\\caption{2D concentration profiles obtained for cement system 1 at the final time step of the 1-year simulation performed for the diffusion case. RTM means the original HP$_{\\rm{4C}}$ model, TM+kNN denotes the Hydrus transport model coupled with our kNN geochemical emulator (HPx$_{\\rm py}$-kNN) and TM+DNN signifies the Hydrus transport model coupled with our DNN geochemical emulator (HPx$_{\\rm py}$-DNN). The first to fourth row present profiles for $Ca^{\\rm{conc}}$, $Si^{\\rm{conc}}$, $H^{\\rm{conc}}$, and $O^{\\rm{conc}}$, respectively. The considered grid size is 61 $\\times$ 61.}\n\t\\label{fig6}\n\\end{figure}\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure7.png}\n\t\\caption{2D solid amount profiles obtained for cement system 1 at the end of the 2-year simulation performed for the advection-dispersion case. RTM means the original HP$_{\\rm{4C}}$ model, TM+kNN denotes the Hydrus transport model coupled with our kNN geochemical emulator (HPx$_{\\rm py}$-kNN) and TM+DNN signifies the Hydrus transport model coupled with our DNN geochemical emulator (HPx$_{\\rm py}$-DNN). The first to fourth row present profiles for $Ca^{\\rm{solid}}$, $Si^{\\rm{solid}}$, $H^{\\rm{solid}}$, and $O^{\\rm{solid}}$, respectively. The considered grid size is 61 $\\times$ 61. The results for the 121 $\\times$ 121 grid are rather similar.}\n\t\\label{fig7}\n\\end{figure}\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure8.png}\n\t\\caption{2D solid amount profiles obtained for cement system 1 at the final time step of the 1-year simulation performed for the diffusion case. RTM means the original HP$_{\\rm{4C}}$ model, TM+kNN denotes the Hydrus transport model coupled with our kNN geochemical emulator (HPx$_{\\rm py}$-kNN) and TM+DNN signifies the Hydrus transport model coupled with our DNN geochemical emulator (HPx$_{\\rm py}$-DNN). The first to fourth row present profiles for $Ca^{\\rm{solid}}$, $Si^{\\rm{solid}}$, $H^{\\rm{solid}}$, and $O^{\\rm{solid}}$, respectively. The considered grid size is 61 $\\times$ 61.}\n\t\\label{fig8}\n\\end{figure}\n\nThe speedups associated with the considered problem are detailed in Table \\ref{table3}. It is noted that the GPU-based DNN emulator allows for a speedup that is close to optimal. Indeed, The DNN speedups overall represent 85 \\% to 95 \\% of the maximum possible speedups (that is, speedups that would be obtained if the geochemcical calculations would come at no cost at all). The speedups associated with single-threaded kNN remain substantial but only amount to 57 \\% - 65 \\% of the corresponding maximum speedups. As detailed in section \\ref{train_res1}, the used kNN and DNN implementations are found to be respectively 300 and 4000 faster than single-threaded PHREEQC when predicting 10,000 points all one once for this geochemical system. Based on these numbers one could have expected the achieved speedups to represent say 90 \\% (kNN) or 99 \\% (DNN) of the maximum possible ones. A large part of the gaps between achieved and maximum possible speedups is thus likely caused by the time required for communicating and exchanging data between the main C/C++ code and the Python-based emulators.\n\n\\begin{table}[h!]\n\t\\caption{Speedups offered by the KNN and DNN emulators in HPx$_{\\rm py}$ for the reactive transport simulations considered for cement system 1. The HPx$_{\\rm{4C}}$ calculations involve the parallelization of PHREEQC over our 4 CPUs. The HPx$_{\\rm{1C}}$ calculations are performed on a single CPU. The kNN predictions are performed on a single CPU using the scikit-learn implementation while the DNN predictions make use of our GPU. ML signifies the used machine learning method for emulation, TC denotes transport conditions (ADV: advection-dispersion, DIF: diffusion) and GS is the grid size. The maximum possible speedups associated with HPx$_{\\rm{4C}}$ and HPx$_{\\rm{1C}}$,  Max SP HPx$_{\\rm{4C}}$ and  Max SP HPx$_{\\rm{1C}}$, correspond to an hypothetical situation where the geochemical calculations incur zero computational cost.}\n\n\t\\begin{center}\n\t\t\\begin{tabular}{cccccccc}\n\t\t\t\\hline\n\t\t\t& & & & & & & \\\\\n\t\t\tML & TC & GS & HPx$_{\\rm{4C}}$ time (s) & SP HPx$_{\\rm{4C}}$ & Max SP HPx$_{\\rm{4C}}$ & SP HPx$_{\\rm{1C}}$ & Max SP HPx$_{\\rm{1C}}$ \\\\\n\t\t\tDNN & ADV & 61 $\\times$ 61 & 3189 & 6.8 & 7.7 & 24.5 & 28.5 \\\\\n\t\t\tkNN & ADV & 61 $\\times$ 61 & 3189 & 5.0 & 7.7 & 18.0 & 28.5 \\\\\n\t\t\tDNN & ADV & 121 $\\times$ 121 & 23,337 & 5.0 & 5.2 & 17.0 & 18.0 \\\\\n\t\t\tkNN & ADV & 121 $\\times$ 121 & 23,337 & 3.4 & 5.2 & 11.6 & 18.0 \\\\\n\t\t\tDNN & DIF & 61 $\\times$ 61 & 25,448 & 4.2 & 4.9 & 16.2 & 19.1 \\\\\n\t\t\tkNN & DIF & 61 $\\times$ 61 & 25,448 & 2.8 & 4.9 & 10.8 & 19.1 \\\\\n\t\t\t\\hline\n\t\t\\end{tabular}\n\t\\end{center}\n\n\t\\label{table3}\n\\end{table}\n\n\\FloatBarrier\n\n\\subsection{Al-C-Ca-S-Si Problem}\n\nFor this second cement system, the emulation problem consists of predicting at each time step of the RT simulation the (output) Al, C, Ca, S, Si, H and O aqueous concentrations (mol/kgw) from the (input) total amounts of Al, C, Ca, S and Si (mol). Here we focus on advective-dispersive transport only while similarly as for cement system 1, the considered domain sizes are 61 $\\times$ 61 and 121 $\\times$ 121. Moreover, the simulation time period is set to 6 years. \n\nAs mentioned earlier, for this higher-dimensional problem it is observed that the used scikit-learn kNN implementation becomes prohibitively slow compared to HPx$_{\\rm{4C}}$. We found that to get a good emulation accuracy the kNN training base needs to contain 1,000,000 samples (or more). This training base's size together with a 5-dimensional search space leads to an HPx$_{\\rm py}$-kNN reactive transport simulation time that is comparable to that of HPx$_{\\rm{4C}}$. Therefore, for this second cement system we built a custom kNN regressor around another kNN implementation contained in the FAISS package \\citep{faiss2017}. Our used FAISS variant allows for GPU computing and is much faster than scikit-learn for this cement system, but is slightly less accurate due to the use of an approximate rather than exact nearest neighbor search \\citep[see][for details]{faiss2017}.\n\n\\subsubsection{Training the DNN}\n\\label{train_res2}\n\nBuilding a good training set to perform a kNN search and learn the weights and biases of our DNN turned out to be a complicated task in this case. This because to make useful kNN predictions and/or learn an useful DNN, the training set must be sufficiently representative of the geochemical conditions encountered during the reactive transport simulation one wish to perform with HPx$_{\\rm py}$-kNN and the trained HPx$_{\\rm py}$-DNN. In contrast to cement system 1, creating the training set by sampling the $X$-space with a controlled randomness between predefined lower and upper bounds did not prove successful. We tried that strategy by drawing as much as 4,000,000 5-dimensional $\\textbf{x}$ vectors from the $X$-space using a Sobol low-discrepancy sequence \\citep[][]{Sobol1967, Joe-Kuo2003}. Such low-discrepancy sampling scheme covers the 5-dimensional hypercube more uniformly than LHS. Despite a good performance on the test set (not shown), the resulting DNN accuracy in reactive transport mode was never deemed satisfying. In other words, no satisfying ``global\" or ``universal\" DNN emulator could be devised for this cement system. This is probably caused by the fact for this problem, the input (5 total amounts) and output (7 aqueous concentrations) spaces are quite nonlinearly related and both cover 6 to 10 orders of magnitudes depending on the considered element. Therefore, we resorted to the alternative training strategy detailed below. The latter basically tries to grasp the complex correlations and higher-order dependencies that exist between the elements of $\\textbf{x}$ (total amounts, input space) for a given reactive transport simulation setup, in order to produce a training set that honors these between-input relationships.\n\n\\begin{itemize}\n\t\\item Perform a ``cheap\" full reactive transport simulation under the transport conditions and geochemistry of interest and collect the resulting $\\textbf{x}$ -$\\textbf{y}$  pairs of examples (for the considered grid nodes and time steps). Computational demand controls what domain size and simulation time period can be used for this cheap calculation. We used a modest 16 $\\times$ 16 domain and a simulation time period of 10 years. The associated HPx$_{\\rm{4C}}$ runtimes is 180 s.\n\t\n\t\\item Fit a kernel density estimator (KDE) with a Gaussian kernel to the collected $\\textbf{x}$ vectors (encapsulated in the $\\textbf{X}$ array) and generate a fixed number of new input vectors, $\\textbf{x}_{KDE}$. Then run PHREEQC for the $\\textbf{X}_{KDE}$ set to get the corresponding output set, $\\textbf{Y}_{KDE}$. Now apply the correction for porosity described in section \\ref{train_res1} to the $\\textbf{X}_{KDE}$ set and form the training set by merging the ensemble of $\\textbf{x}$-$\\textbf{y}$ pairs with that of the $\\textbf{x}_{KDE}$-$\\textbf{y}_{KDE}$ pairs. The number of produced unique examples by the considered cheap HPx$_{\\rm{4C}}$ simulations varied between 10,000 and 50,000. The KDE-based enrichment of this dataset was deemed necessary to provide more input variability, thereby avoiding overfitting of the trained DNN and improving the kNN accuracy, while still honoring the complex between-input relationships. The number of KDE-generated samples was set as to obtain a total training set size of 1,000,000 examples. A key component of the approach is the bandwith parameter of the Gaussian KDE kernel which controls how much the KDE-generated samples depart from the original ensemble. After limited trial and error, we fixed the kernel bandwith to 0.0025 for the considered case studies.\n\t\n\\end{itemize}\n\nThe scatter plots in Figure \\ref{fig9} illustrate our training set creation procedure. The orange dots depict the pairwise relationships observed between the 5 elements of $\\textbf{x}$ in the cheap simulation. The red and cyan dots in Figure \\ref{fig9} represent the KDE-generated samples with the selected bandwidth, before and after applying the correction for porosity, respectively. Training of the DNN is achieved using the ensemble of original and KDE-corrected input points. We refer to this kind of dataset as RT-based, since it is based on an albeit cheap, full RT simulation. Furthermore, we refer to the obtained DNN and kNN emulators as ``local\" emulators, since as opposed to the emulators constructed for cement system 1, the current emulators are only valid for the input conditions encapsulated in the RT-based training set.\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1.5cm}\\includegraphics[width=50pc]{Figure9.png}\n\t\\caption{Scatter plots of the complex relationships between the five considered inputs for (1) the computationally cheap RT simulation of cement system 2 (orange dots) performed with the original HPx code, (2) the corresponding sampled points by kernel density estimation (KDE) using the selected bandwith (turquoise dots) and, (3) the same sampled KDE points after porosity correction (red dots).}\n\t\\label{fig9}\n\\end{figure}\n\nTraining performance of the DNN emulator is presented in Figure \\ref{fig10}. Similarly as for cement system 1, about 90\\% of the available data was used for the actual training of the DNN while the remaining 10\\% were used a validation set to contol overfitting. Lastly, performance is evaluated for both the trained DNN and kNN emulators using an independent test set of 10,000 examples. Overall, the accuracy of our ``local \" DNN emulator for this RT-inspired dataset is rather large with $Q_2$ values always greater or equal than 0.998. Training performance of the corresponding local kNN emulator is equally good (not shown). Regarding speedup, for this problem single-threaded PHREEQC achieves about 210 geochemical calculations per second on our used Intel\\textsuperscript{\\textregistered} i7 CPU while the GPU-based DNN and kNN emulators are both about 3000 times faster when predicting the 10,000 test points at once .\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure10.png}\n\t\\caption{1-1 plots of local DNN emulation performance obtained for system 2 when the local DNN is trained using 1,000,000 samples. The $x$-axis and $y$-axis present the original and emulated 10,000 independent test data points, respectively. The RMSE and $Q_2$ coefficient denote the root-mean-square-error and coefficient of determination in testing mode, respectively, between the original and emulated 10,000 test data points.}\n\t\\label{fig10}\n\\end{figure}\n\n\\subsubsection{Reactive transport simulation}\n\\label{prob2res}\nOur ``local\" DNN performs rather well when applied to the 61 $\\times$ 61 grid size and a time period of 6 years (Figures \\ref{fig11} - \\ref{fig12}). This for all components but C, for which some localized deviations appear between original and emulated 2D concentration profiles towards the end of the simulation period (Figure \\ref{fig12}). When applied to the 121 $\\times$ 121 grid, HPx$_{\\rm py}$-DNN produces additional discrepancies for O, Si and Al towards the end of the simulation period (Figures \\ref{fig13} - \\ref{fig14}). Yet most of the observed artifacts could probably be smoothed out by using post-filtering. The associated speedups are listed in Table \\ref{table4}. These speedups are larger to those obtained for cement system 1, with values between 8 and 9 when evaluated against HPx$_{\\rm{4C}}$. These speedups represent about 85 \\% to 90 \\% of the maximum possible speedup (Table \\ref{table3}). Overall, these findings indicate that for the considered problem, our RT-based training of a local DNN only works is the training set is sufficiently representative of the particular geochemical conditions encountered in the computationally demanding simulations, which is arguably not easy to achieve. This limitation is further discussed in section \\ref{discussion}.\n\nWe note a more uniform behavior for HPx$_{\\rm py}$-kNN across grid sizes than for HPx$_{\\rm py}$-DNN. Here the results for the 121  $\\times$ 121 grid are only slightly less accurate than those associated with the 61 $ \\times$ 61 grid (see Figures \\ref{fig15} - \\ref{fig16} where for brevity we only show concentration profiles for the C Al and S components). Furthermore, whenever observed the discrepancies between original and emulated profiles are more regularly scattered than for HPx$_{\\rm py}$-DNN. Note also that herein too, post-filtering could likely smooth out a large part of these deviations. In addition, owing to the use of a GPU to achieve the kNN calculations, the speedup provided by HPx$_{\\rm py}$-kNN are as large as those provided by HPx$_{\\rm py}$-DNN (Table \\ref{table4}).\n\nWith respect to the emulated solid amounts, the HPx$_{\\rm py}$-DNN results look visually good for the the 61 $\\times$ 61 grid. This is shown in Figure \\ref{fig17} for the C, Al and S chemical components, while emulation of the H, O Ca and Si chemical components is globally of similar quality (not shown). Nevertheless, significant deviations appear towards the end of the simulation period for every chemical component (see Figure \\ref{fig18} for the C, Al and S chemical components, emulation of the H, O Ca and Si chemical components shows the same level of mismatch). The HPx$_{\\rm py}$-kNN predictions are also fairly accurate for the 61 $\\times$ 61 (see Figure \\ref{fig19} for the C, Al and S chemical components, emulation of the H, O Ca and Si chemical components exhibits a globally similar quality) while some discrepancies show up at the end of the simulation (Figure \\ref{fig20}). However, the mismatch is less pronounced than for HPx$_{\\rm py}$-DNN. Overall, HPx$_{\\rm py}$-kNN appears to be somewhat more robust than HPx$_{\\rm py}$-DNN for this cement system, while providing the same (large) speedup.\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure11.png}\n\t\\caption{2D concentration profiles obtained for cement system 2 after 2 and 6 (final time step) years. RTM means the original HPx$_{\\rm{4C}}$ model and TM+DNN denotes the Hydrus transport model coupled with our DNN geochemical emulator (HPx$_{\\rm py}$-DNN). The first to fourth row present profiles for $Ca^{\\rm{conc}}$, $Si^{\\rm{conc}}$, $H^{\\rm{conc}}$, and $O^{\\rm{conc}}$, respectively. The considered grid size is 61 $\\times$ 61.}\n\t\\label{fig11}\n\\end{figure}\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure12.png}\n\t\\caption{2D concentration profiles obtained for cement system 2 after 2 and 6 (final time step) years. RTM means the original HPx$_{\\rm{4C}}$ model and TM+DNN denotes the Hydrus transport model coupled with our DNN geochemical emulator (HPx$_{\\rm py}$-DNN). The first to third row present profiles for $C^{\\rm{conc}}$, $Al^{\\rm{conc}}$, and $S^{\\rm{conc}}$, respectively. The considered grid size is 61 $\\times$ 61.}\n\t\\label{fig12}\n\\end{figure}\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure13.png}\n\t\\caption{2D concentration profiles obtained for cement system 2 after 2 and 6 (final time step) years. RTM means the original HPx$_{\\rm{4C}}$ model and TM+DNN denotes the Hydrus transport model coupled with our DNN geochemical emulator (HPx$_{\\rm py}$-DNN). The first to fourth row present profiles for $Ca^{\\rm{conc}}$, $Si^{\\rm{conc}}$, $H^{\\rm{conc}}$, and $O^{\\rm{conc}}$, respectively. The considered grid size is 121 $\\times$ 121.}\n\t\\label{fig13}\n\\end{figure}\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure14.png}\n\t\\caption{2D concentration profiles obtained for cement system 2 after 2 and 6 (final time step) years. RTM means the original HPx$_{\\rm{4C}}$ model and TM+DNN denotes the Hydrus transport model coupled with our DNN geochemical emulator (HPx$_{\\rm py}$-DNN). The first to third row present profiles for $C^{\\rm{conc}}$, $Al^{\\rm{conc}}$, and $S^{\\rm{conc}}$, respectively. The considered grid size is 121 $\\times$ 121.}\n\t\\label{fig14}\n\\end{figure}\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure15.png}\n\t\\caption{2D concentration profiles obtained for cement system 2 after 2 and 6 (final time step) years. RTM means the original HPx$_{\\rm{4C}}$ model and TM+kNN denotes the Hydrus transport model coupled with our kNN geochemical emulator (HPx$_{\\rm py}$-kNN).  The first to third row present profiles for $C^{\\rm{conc}}$, $Al^{\\rm{conc}}$, and $S^{\\rm{conc}}$, respectively. The considered grid size is 61 $\\times$ 61.}\n\t\\label{fig15}\n\\end{figure}\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure16.png}\n\t\\caption{2D concentration profiles obtained for cement system 2 after 2 and 6 (final time step) years. RTM means the original HPx$_{\\rm{4C}}$ model and TM+kNN denotes the Hydrus transport model coupled with our kNN geochemical emulator (HPx$_{\\rm py}$-kNN). The first to third row present profiles for $C^{\\rm{conc}}$, $Al^{\\rm{conc}}$, and $S^{\\rm{conc}}$, respectively. The considered grid size is 121 $\\times$ 121.}\n\t\\label{fig16}\n\\end{figure}\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure17.png}\n\t\\caption{2D solid amount profiles of the C, Al and S chemical components of cement system 2 after 2 and 6 (final time step) years. RTM means the original HPx$_{\\rm{4C}}$ model and TM+DNN denotes the Hydrus transport model coupled with our DNN geochemical emulator (HPx$_{\\rm py}$-DNN). The first to third row present profiles for $C^{\\rm{solid}}$, $Al^{\\rm{solid}}$, and $S^{\\rm{solid}}$, respectively The considered grid size is 61 $\\times$ 61.}\n\t\\label{fig17}\n\\end{figure}\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure18.png}\n\t\\caption{2D solid amount profiles of the C, Al and S chemical components of cement system 2 after 2 and 6 (final time step) years. RTM means the original HPx$_{\\rm{4C}}$ model and TM+DNN denotes the Hydrus transport model coupled with our DNN geochemical emulator (HPx$_{\\rm py}$-DNN). The first to third row present profiles for $C^{\\rm{solid}}$, $Al^{\\rm{solid}}$, and $S^{\\rm{solid}}$, respectively The considered grid size is 121 $\\times$ 121.}\n\t\\label{fig18}\n\\end{figure}\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure19.png}\n\t\\caption{2D solid amount profiles of the C, Al and S chemical components of cement system 2 after 2 and 6 (final time step) years. RTM means the original HPx$_{\\rm{4C}}$ model and TM+kNN denotes the Hydrus transport model coupled with our kNN geochemical emulator (HPx$_{\\rm py}$-kNN). The first to third row present profiles for $C^{\\rm{solid}}$, $Al^{\\rm{solid}}$, and $S^{\\rm{solid}}$, respectively The considered grid size is 61 $\\times$ 61.}\n\t\\label{fig19}\n\\end{figure}\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure20.png}\n\t\\caption{2D solid amount profiles of the C, Al and S chemical components of cement system 2 after 2 and 6 (final time step) years. RTM means the original HPx$_{\\rm{4C}}$ model and TM+kNN denotes the Hydrus transport model coupled with our kNN geochemical emulator (HPx$_{\\rm py}$-kNN). The first to third row present profiles for $C^{\\rm{solid}}$, $Al^{\\rm{solid}}$, and $S^{\\rm{solid}}$, respectively The considered grid size is 121 $\\times$ 121.}\n\t\\label{fig20}\n\\end{figure}\n\n\\begin{table}[h!]\n\t\\caption{Speedups offered by the KNN and DNN emulators in HPx$_{\\rm py}$ for the reactive transport simulations considered for cement system 2. The HPx$_{\\rm{4C}}$ calculations involve the parallelization of PHREEQC over our 4 CPUs. The HPx$_{\\rm{1C}}$ calculations are performed on a single CPU. Both the kNN and DNN predictions make use of a GPU. ML signifies the used machine learning method for emulation, BC denotes the type of flow boundary conditions and GS is the grid size. The maximum possible speedups associated with HPx$_{\\rm{4C}}$ and HP$_{\\rm{1C}}$,  Max SP HPx$_{\\rm{4C}}$ and  Max SP HPx$_{\\rm{1C}}$, correspond to an hypothetical situation where the geochemical calculations incur zero computational cost.}\n\n\t\\begin{center}\n\t\t\\begin{tabular}{cccccccc}\n\t\t\t\\hline\n\t\t\t& & & & & & & \\\\\n\t\t\tML & BC & GS & HPx$_{\\rm{4C}}$ time (s) & SP HPx$_{\\rm{4C}}$ & Max SP HPx$_{\\rm{4C}}$ & SP HPx$_{\\rm{1C}}$ & Max SP HPx$_{\\rm{1C}}$ \\\\\n\t\t\tDNN & ADV & 61 $\\times$ 61 & 21,415 & 8.2 & 9.0 & 30.3 & 33.1\\\\\n\t\t\tkNN & ADV & 61 $\\times$ 61 & 21,415 & 7.9 & 9.0 & 28.9 & 33.1\\\\\n\t\t\tDNN & ADV & 121 $\\times$ 121 & 199,841 & 8.2 & 9.5 & 29.9 & 32.8\\\\\n\t\t\tkNN & ADV & 121 $\\times$ 121 & 199,841 & 8.5 & 9.5 & 31.3 & 32.8\\\\\n\t\t\t\\hline\n\t\t\\end{tabular}\n\t\\end{center}\n\n\t\\label{table4}\n\\end{table}\n\n\\FloatBarrier\n\n3.2 Al-C-Ca-S-Si Problem\n\\subsection{Al-C-Ca-S-Si Problem}\n\nFor this second cement system, the emulation problem consists of predicting at each time step of the RT simulation the (output) Al, C, Ca, S, Si, H and O aqueous concentrations (mol/kgw) from the (input) total amounts of Al, C, Ca, S and Si (mol). Here we focus on advective-dispersive transport only while similarly as for cement system 1, the considered domain sizes are 61 $\\times$ 61 and 121 $\\times$ 121. Moreover, the simulation time period is set to 6 years. \n\nAs mentioned earlier, for this higher-dimensional problem it is observed that the used scikit-learn kNN implementation becomes prohibitively slow compared to HPx$_{\\rm{4C}}$. We found that to get a good emulation accuracy the kNN training base needs to contain 1,000,000 samples (or more). This training base's size together with a 5-dimensional search space leads to an HPx$_{\\rm py}$-kNN reactive transport simulation time that is comparable to that of HPx$_{\\rm{4C}}$. Therefore, for this second cement system we built a custom kNN regressor around another kNN implementation contained in the FAISS package \\citep{faiss2017}. Our used FAISS variant allows for GPU computing and is much faster than scikit-learn for this cement system, but is slightly less accurate due to the use of an approximate rather than exact nearest neighbor search \\citep[see][for details]{faiss2017}.\n\n\\subsubsection{Training the DNN}\n\\label{train_res2}\n\nBuilding a good training set to perform a kNN search and learn the weights and biases of our DNN turned out to be a complicated task in this case. This because to make useful kNN predictions and/or learn an useful DNN, the training set must be sufficiently representative of the geochemical conditions encountered during the reactive transport simulation one wish to perform with HPx$_{\\rm py}$-kNN and the trained HPx$_{\\rm py}$-DNN. In contrast to cement system 1, creating the training set by sampling the $X$-space with a controlled randomness between predefined lower and upper bounds did not prove successful. We tried that strategy by drawing as much as 4,000,000 5-dimensional $\\textbf{x}$ vectors from the $X$-space using a Sobol low-discrepancy sequence \\citep[][]{Sobol1967, Joe-Kuo2003}. Such low-discrepancy sampling scheme covers the 5-dimensional hypercube more uniformly than LHS. Despite a good performance on the test set (not shown), the resulting DNN accuracy in reactive transport mode was never deemed satisfying. In other words, no satisfying ``global\" or ``universal\" DNN emulator could be devised for this cement system. This is probably caused by the fact for this problem, the input (5 total amounts) and output (7 aqueous concentrations) spaces are quite nonlinearly related and both cover 6 to 10 orders of magnitudes depending on the considered element. Therefore, we resorted to the alternative training strategy detailed below. The latter basically tries to grasp the complex correlations and higher-order dependencies that exist between the elements of $\\textbf{x}$ (total amounts, input space) for a given reactive transport simulation setup, in order to produce a training set that honors these between-input relationships.\n\n\\begin{itemize}\n\t\\item Perform a ``cheap\" full reactive transport simulation under the transport conditions and geochemistry of interest and collect the resulting $\\textbf{x}$ -$\\textbf{y}$  pairs of examples (for the considered grid nodes and time steps). Computational demand controls what domain size and simulation time period can be used for this cheap calculation. We used a modest 16 $\\times$ 16 domain and a simulation time period of 10 years. The associated HPx$_{\\rm{4C}}$ runtimes is 180 s.\n\t\n\t\\item Fit a kernel density estimator (KDE) with a Gaussian kernel to the collected $\\textbf{x}$ vectors (encapsulated in the $\\textbf{X}$ array) and generate a fixed number of new input vectors, $\\textbf{x}_{KDE}$. Then run PHREEQC for the $\\textbf{X}_{KDE}$ set to get the corresponding output set, $\\textbf{Y}_{KDE}$. Now apply the correction for porosity described in section \\ref{train_res1} to the $\\textbf{X}_{KDE}$ set and form the training set by merging the ensemble of $\\textbf{x}$-$\\textbf{y}$ pairs with that of the $\\textbf{x}_{KDE}$-$\\textbf{y}_{KDE}$ pairs. The number of produced unique examples by the considered cheap HPx$_{\\rm{4C}}$ simulations varied between 10,000 and 50,000. The KDE-based enrichment of this dataset was deemed necessary to provide more input variability, thereby avoiding overfitting of the trained DNN and improving the kNN accuracy, while still honoring the complex between-input relationships. The number of KDE-generated samples was set as to obtain a total training set size of 1,000,000 examples. A key component of the approach is the bandwith parameter of the Gaussian KDE kernel which controls how much the KDE-generated samples depart from the original ensemble. After limited trial and error, we fixed the kernel bandwith to 0.0025 for the considered case studies.\n\t\n\\end{itemize}\n\nThe scatter plots in Figure \\ref{fig9} illustrate our training set creation procedure. The orange dots depict the pairwise relationships observed between the 5 elements of $\\textbf{x}$ in the cheap simulation. The red and cyan dots in Figure \\ref{fig9} represent the KDE-generated samples with the selected bandwidth, before and after applying the correction for porosity, respectively. Training of the DNN is achieved using the ensemble of original and KDE-corrected input points. We refer to this kind of dataset as RT-based, since it is based on an albeit cheap, full RT simulation. Furthermore, we refer to the obtained DNN and kNN emulators as ``local\" emulators, since as opposed to the emulators constructed for cement system 1, the current emulators are only valid for the input conditions encapsulated in the RT-based training set.\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1.5cm}\\includegraphics[width=50pc]{Figure9.png}\n\t\\caption{Scatter plots of the complex relationships between the five considered inputs for (1) the computationally cheap RT simulation of cement system 2 (orange dots) performed with the original HPx code, (2) the corresponding sampled points by kernel density estimation (KDE) using the selected bandwith (turquoise dots) and, (3) the same sampled KDE points after porosity correction (red dots).}\n\t\\label{fig9}\n\\end{figure}\n\nTraining performance of the DNN emulator is presented in Figure \\ref{fig10}. Similarly as for cement system 1, about 90\\% of the available data was used for the actual training of the DNN while the remaining 10\\% were used a validation set to contol overfitting. Lastly, performance is evaluated for both the trained DNN and kNN emulators using an independent test set of 10,000 examples. Overall, the accuracy of our ``local \" DNN emulator for this RT-inspired dataset is rather large with $Q_2$ values always greater or equal than 0.998. Training performance of the corresponding local kNN emulator is equally good (not shown). Regarding speedup, for this problem single-threaded PHREEQC achieves about 210 geochemical calculations per second on our used Intel\\textsuperscript{\\textregistered} i7 CPU while the GPU-based DNN and kNN emulators are both about 3000 times faster when predicting the 10,000 test points at once .\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure10.png}\n\t\\caption{1-1 plots of local DNN emulation performance obtained for system 2 when the local DNN is trained using 1,000,000 samples. The $x$-axis and $y$-axis present the original and emulated 10,000 independent test data points, respectively. The RMSE and $Q_2$ coefficient denote the root-mean-square-error and coefficient of determination in testing mode, respectively, between the original and emulated 10,000 test data points.}\n\t\\label{fig10}\n\\end{figure}\n\n\\subsubsection{Reactive transport simulation}\n\\label{prob2res}\nOur ``local\" DNN performs rather well when applied to the 61 $\\times$ 61 grid size and a time period of 6 years (Figures \\ref{fig11} - \\ref{fig12}). This for all components but C, for which some localized deviations appear between original and emulated 2D concentration profiles towards the end of the simulation period (Figure \\ref{fig12}). When applied to the 121 $\\times$ 121 grid, HPx$_{\\rm py}$-DNN produces additional discrepancies for O, Si and Al towards the end of the simulation period (Figures \\ref{fig13} - \\ref{fig14}). Yet most of the observed artifacts could probably be smoothed out by using post-filtering. The associated speedups are listed in Table \\ref{table4}. These speedups are larger to those obtained for cement system 1, with values between 8 and 9 when evaluated against HPx$_{\\rm{4C}}$. These speedups represent about 85 \\% to 90 \\% of the maximum possible speedup (Table \\ref{table3}). Overall, these findings indicate that for the considered problem, our RT-based training of a local DNN only works is the training set is sufficiently representative of the particular geochemical conditions encountered in the computationally demanding simulations, which is arguably not easy to achieve. This limitation is further discussed in section \\ref{discussion}.\n\nWe note a more uniform behavior for HPx$_{\\rm py}$-kNN across grid sizes than for HPx$_{\\rm py}$-DNN. Here the results for the 121  $\\times$ 121 grid are only slightly less accurate than those associated with the 61 $ \\times$ 61 grid (see Figures \\ref{fig15} - \\ref{fig16} where for brevity we only show concentration profiles for the C Al and S components). Furthermore, whenever observed the discrepancies between original and emulated profiles are more regularly scattered than for HPx$_{\\rm py}$-DNN. Note also that herein too, post-filtering could likely smooth out a large part of these deviations. In addition, owing to the use of a GPU to achieve the kNN calculations, the speedup provided by HPx$_{\\rm py}$-kNN are as large as those provided by HPx$_{\\rm py}$-DNN (Table \\ref{table4}).\n\nWith respect to the emulated solid amounts, the HPx$_{\\rm py}$-DNN results look visually good for the the 61 $\\times$ 61 grid. This is shown in Figure \\ref{fig17} for the C, Al and S chemical components, while emulation of the H, O Ca and Si chemical components is globally of similar quality (not shown). Nevertheless, significant deviations appear towards the end of the simulation period for every chemical component (see Figure \\ref{fig18} for the C, Al and S chemical components, emulation of the H, O Ca and Si chemical components shows the same level of mismatch). The HPx$_{\\rm py}$-kNN predictions are also fairly accurate for the 61 $\\times$ 61 (see Figure \\ref{fig19} for the C, Al and S chemical components, emulation of the H, O Ca and Si chemical components exhibits a globally similar quality) while some discrepancies show up at the end of the simulation (Figure \\ref{fig20}). However, the mismatch is less pronounced than for HPx$_{\\rm py}$-DNN. Overall, HPx$_{\\rm py}$-kNN appears to be somewhat more robust than HPx$_{\\rm py}$-DNN for this cement system, while providing the same (large) speedup.\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure11.png}\n\t\\caption{2D concentration profiles obtained for cement system 2 after 2 and 6 (final time step) years. RTM means the original HPx$_{\\rm{4C}}$ model and TM+DNN denotes the Hydrus transport model coupled with our DNN geochemical emulator (HPx$_{\\rm py}$-DNN). The first to fourth row present profiles for $Ca^{\\rm{conc}}$, $Si^{\\rm{conc}}$, $H^{\\rm{conc}}$, and $O^{\\rm{conc}}$, respectively. The considered grid size is 61 $\\times$ 61.}\n\t\\label{fig11}\n\\end{figure}\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure12.png}\n\t\\caption{2D concentration profiles obtained for cement system 2 after 2 and 6 (final time step) years. RTM means the original HPx$_{\\rm{4C}}$ model and TM+DNN denotes the Hydrus transport model coupled with our DNN geochemical emulator (HPx$_{\\rm py}$-DNN). The first to third row present profiles for $C^{\\rm{conc}}$, $Al^{\\rm{conc}}$, and $S^{\\rm{conc}}$, respectively. The considered grid size is 61 $\\times$ 61.}\n\t\\label{fig12}\n\\end{figure}\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure13.png}\n\t\\caption{2D concentration profiles obtained for cement system 2 after 2 and 6 (final time step) years. RTM means the original HPx$_{\\rm{4C}}$ model and TM+DNN denotes the Hydrus transport model coupled with our DNN geochemical emulator (HPx$_{\\rm py}$-DNN). The first to fourth row present profiles for $Ca^{\\rm{conc}}$, $Si^{\\rm{conc}}$, $H^{\\rm{conc}}$, and $O^{\\rm{conc}}$, respectively. The considered grid size is 121 $\\times$ 121.}\n\t\\label{fig13}\n\\end{figure}\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure14.png}\n\t\\caption{2D concentration profiles obtained for cement system 2 after 2 and 6 (final time step) years. RTM means the original HPx$_{\\rm{4C}}$ model and TM+DNN denotes the Hydrus transport model coupled with our DNN geochemical emulator (HPx$_{\\rm py}$-DNN). The first to third row present profiles for $C^{\\rm{conc}}$, $Al^{\\rm{conc}}$, and $S^{\\rm{conc}}$, respectively. The considered grid size is 121 $\\times$ 121.}\n\t\\label{fig14}\n\\end{figure}\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure15.png}\n\t\\caption{2D concentration profiles obtained for cement system 2 after 2 and 6 (final time step) years. RTM means the original HPx$_{\\rm{4C}}$ model and TM+kNN denotes the Hydrus transport model coupled with our kNN geochemical emulator (HPx$_{\\rm py}$-kNN).  The first to third row present profiles for $C^{\\rm{conc}}$, $Al^{\\rm{conc}}$, and $S^{\\rm{conc}}$, respectively. The considered grid size is 61 $\\times$ 61.}\n\t\\label{fig15}\n\\end{figure}\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure16.png}\n\t\\caption{2D concentration profiles obtained for cement system 2 after 2 and 6 (final time step) years. RTM means the original HPx$_{\\rm{4C}}$ model and TM+kNN denotes the Hydrus transport model coupled with our kNN geochemical emulator (HPx$_{\\rm py}$-kNN). The first to third row present profiles for $C^{\\rm{conc}}$, $Al^{\\rm{conc}}$, and $S^{\\rm{conc}}$, respectively. The considered grid size is 121 $\\times$ 121.}\n\t\\label{fig16}\n\\end{figure}\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure17.png}\n\t\\caption{2D solid amount profiles of the C, Al and S chemical components of cement system 2 after 2 and 6 (final time step) years. RTM means the original HPx$_{\\rm{4C}}$ model and TM+DNN denotes the Hydrus transport model coupled with our DNN geochemical emulator (HPx$_{\\rm py}$-DNN). The first to third row present profiles for $C^{\\rm{solid}}$, $Al^{\\rm{solid}}$, and $S^{\\rm{solid}}$, respectively The considered grid size is 61 $\\times$ 61.}\n\t\\label{fig17}\n\\end{figure}\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure18.png}\n\t\\caption{2D solid amount profiles of the C, Al and S chemical components of cement system 2 after 2 and 6 (final time step) years. RTM means the original HPx$_{\\rm{4C}}$ model and TM+DNN denotes the Hydrus transport model coupled with our DNN geochemical emulator (HPx$_{\\rm py}$-DNN). The first to third row present profiles for $C^{\\rm{solid}}$, $Al^{\\rm{solid}}$, and $S^{\\rm{solid}}$, respectively The considered grid size is 121 $\\times$ 121.}\n\t\\label{fig18}\n\\end{figure}\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure19.png}\n\t\\caption{2D solid amount profiles of the C, Al and S chemical components of cement system 2 after 2 and 6 (final time step) years. RTM means the original HPx$_{\\rm{4C}}$ model and TM+kNN denotes the Hydrus transport model coupled with our kNN geochemical emulator (HPx$_{\\rm py}$-kNN). The first to third row present profiles for $C^{\\rm{solid}}$, $Al^{\\rm{solid}}$, and $S^{\\rm{solid}}$, respectively The considered grid size is 61 $\\times$ 61.}\n\t\\label{fig19}\n\\end{figure}\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure20.png}\n\t\\caption{2D solid amount profiles of the C, Al and S chemical components of cement system 2 after 2 and 6 (final time step) years. RTM means the original HPx$_{\\rm{4C}}$ model and TM+kNN denotes the Hydrus transport model coupled with our kNN geochemical emulator (HPx$_{\\rm py}$-kNN). The first to third row present profiles for $C^{\\rm{solid}}$, $Al^{\\rm{solid}}$, and $S^{\\rm{solid}}$, respectively The considered grid size is 121 $\\times$ 121.}\n\t\\label{fig20}\n\\end{figure}\n\n\\begin{table}[h!]\n\t\\caption{Speedups offered by the KNN and DNN emulators in HPx$_{\\rm py}$ for the reactive transport simulations considered for cement system 2. The HPx$_{\\rm{4C}}$ calculations involve the parallelization of PHREEQC over our 4 CPUs. The HPx$_{\\rm{1C}}$ calculations are performed on a single CPU. Both the kNN and DNN predictions make use of a GPU. ML signifies the used machine learning method for emulation, BC denotes the type of flow boundary conditions and GS is the grid size. The maximum possible speedups associated with HPx$_{\\rm{4C}}$ and HP$_{\\rm{1C}}$,  Max SP HPx$_{\\rm{4C}}$ and  Max SP HPx$_{\\rm{1C}}$, correspond to an hypothetical situation where the geochemical calculations incur zero computational cost.}\n\n\t\\begin{center}\n\t\t\\begin{tabular}{cccccccc}\n\t\t\t\\hline\n\t\t\t& & & & & & & \\\\\n\t\t\tML & BC & GS & HPx$_{\\rm{4C}}$ time (s) & SP HPx$_{\\rm{4C}}$ & Max SP HPx$_{\\rm{4C}}$ & SP HPx$_{\\rm{1C}}$ & Max SP HPx$_{\\rm{1C}}$ \\\\\n\t\t\tDNN & ADV & 61 $\\times$ 61 & 21,415 & 8.2 & 9.0 & 30.3 & 33.1\\\\\n\t\t\tkNN & ADV & 61 $\\times$ 61 & 21,415 & 7.9 & 9.0 & 28.9 & 33.1\\\\\n\t\t\tDNN & ADV & 121 $\\times$ 121 & 199,841 & 8.2 & 9.5 & 29.9 & 32.8\\\\\n\t\t\tkNN & ADV & 121 $\\times$ 121 & 199,841 & 8.5 & 9.5 & 31.3 & 32.8\\\\\n\t\t\t\\hline\n\t\t\\end{tabular}\n\t\\end{center}\n\n\t\\label{table4}\n\\end{table}\n\n\\FloatBarrier\n\n3.2.2 Reactive transport simulation\n\\subsubsection{Reactive transport simulation}\n\\label{prob2res}\nOur ``local\" DNN performs rather well when applied to the 61 $\\times$ 61 grid size and a time period of 6 years (Figures \\ref{fig11} - \\ref{fig12}). This for all components but C, for which some localized deviations appear between original and emulated 2D concentration profiles towards the end of the simulation period (Figure \\ref{fig12}). When applied to the 121 $\\times$ 121 grid, HPx$_{\\rm py}$-DNN produces additional discrepancies for O, Si and Al towards the end of the simulation period (Figures \\ref{fig13} - \\ref{fig14}). Yet most of the observed artifacts could probably be smoothed out by using post-filtering. The associated speedups are listed in Table \\ref{table4}. These speedups are larger to those obtained for cement system 1, with values between 8 and 9 when evaluated against HPx$_{\\rm{4C}}$. These speedups represent about 85 \\% to 90 \\% of the maximum possible speedup (Table \\ref{table3}). Overall, these findings indicate that for the considered problem, our RT-based training of a local DNN only works is the training set is sufficiently representative of the particular geochemical conditions encountered in the computationally demanding simulations, which is arguably not easy to achieve. This limitation is further discussed in section \\ref{discussion}.\n\nWe note a more uniform behavior for HPx$_{\\rm py}$-kNN across grid sizes than for HPx$_{\\rm py}$-DNN. Here the results for the 121  $\\times$ 121 grid are only slightly less accurate than those associated with the 61 $ \\times$ 61 grid (see Figures \\ref{fig15} - \\ref{fig16} where for brevity we only show concentration profiles for the C Al and S components). Furthermore, whenever observed the discrepancies between original and emulated profiles are more regularly scattered than for HPx$_{\\rm py}$-DNN. Note also that herein too, post-filtering could likely smooth out a large part of these deviations. In addition, owing to the use of a GPU to achieve the kNN calculations, the speedup provided by HPx$_{\\rm py}$-kNN are as large as those provided by HPx$_{\\rm py}$-DNN (Table \\ref{table4}).\n\nWith respect to the emulated solid amounts, the HPx$_{\\rm py}$-DNN results look visually good for the the 61 $\\times$ 61 grid. This is shown in Figure \\ref{fig17} for the C, Al and S chemical components, while emulation of the H, O Ca and Si chemical components is globally of similar quality (not shown). Nevertheless, significant deviations appear towards the end of the simulation period for every chemical component (see Figure \\ref{fig18} for the C, Al and S chemical components, emulation of the H, O Ca and Si chemical components shows the same level of mismatch). The HPx$_{\\rm py}$-kNN predictions are also fairly accurate for the 61 $\\times$ 61 (see Figure \\ref{fig19} for the C, Al and S chemical components, emulation of the H, O Ca and Si chemical components exhibits a globally similar quality) while some discrepancies show up at the end of the simulation (Figure \\ref{fig20}). However, the mismatch is less pronounced than for HPx$_{\\rm py}$-DNN. Overall, HPx$_{\\rm py}$-kNN appears to be somewhat more robust than HPx$_{\\rm py}$-DNN for this cement system, while providing the same (large) speedup.\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure11.png}\n\t\\caption{2D concentration profiles obtained for cement system 2 after 2 and 6 (final time step) years. RTM means the original HPx$_{\\rm{4C}}$ model and TM+DNN denotes the Hydrus transport model coupled with our DNN geochemical emulator (HPx$_{\\rm py}$-DNN). The first to fourth row present profiles for $Ca^{\\rm{conc}}$, $Si^{\\rm{conc}}$, $H^{\\rm{conc}}$, and $O^{\\rm{conc}}$, respectively. The considered grid size is 61 $\\times$ 61.}\n\t\\label{fig11}\n\\end{figure}\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure12.png}\n\t\\caption{2D concentration profiles obtained for cement system 2 after 2 and 6 (final time step) years. RTM means the original HPx$_{\\rm{4C}}$ model and TM+DNN denotes the Hydrus transport model coupled with our DNN geochemical emulator (HPx$_{\\rm py}$-DNN). The first to third row present profiles for $C^{\\rm{conc}}$, $Al^{\\rm{conc}}$, and $S^{\\rm{conc}}$, respectively. The considered grid size is 61 $\\times$ 61.}\n\t\\label{fig12}\n\\end{figure}\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure13.png}\n\t\\caption{2D concentration profiles obtained for cement system 2 after 2 and 6 (final time step) years. RTM means the original HPx$_{\\rm{4C}}$ model and TM+DNN denotes the Hydrus transport model coupled with our DNN geochemical emulator (HPx$_{\\rm py}$-DNN). The first to fourth row present profiles for $Ca^{\\rm{conc}}$, $Si^{\\rm{conc}}$, $H^{\\rm{conc}}$, and $O^{\\rm{conc}}$, respectively. The considered grid size is 121 $\\times$ 121.}\n\t\\label{fig13}\n\\end{figure}\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure14.png}\n\t\\caption{2D concentration profiles obtained for cement system 2 after 2 and 6 (final time step) years. RTM means the original HPx$_{\\rm{4C}}$ model and TM+DNN denotes the Hydrus transport model coupled with our DNN geochemical emulator (HPx$_{\\rm py}$-DNN). The first to third row present profiles for $C^{\\rm{conc}}$, $Al^{\\rm{conc}}$, and $S^{\\rm{conc}}$, respectively. The considered grid size is 121 $\\times$ 121.}\n\t\\label{fig14}\n\\end{figure}\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure15.png}\n\t\\caption{2D concentration profiles obtained for cement system 2 after 2 and 6 (final time step) years. RTM means the original HPx$_{\\rm{4C}}$ model and TM+kNN denotes the Hydrus transport model coupled with our kNN geochemical emulator (HPx$_{\\rm py}$-kNN).  The first to third row present profiles for $C^{\\rm{conc}}$, $Al^{\\rm{conc}}$, and $S^{\\rm{conc}}$, respectively. The considered grid size is 61 $\\times$ 61.}\n\t\\label{fig15}\n\\end{figure}\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure16.png}\n\t\\caption{2D concentration profiles obtained for cement system 2 after 2 and 6 (final time step) years. RTM means the original HPx$_{\\rm{4C}}$ model and TM+kNN denotes the Hydrus transport model coupled with our kNN geochemical emulator (HPx$_{\\rm py}$-kNN). The first to third row present profiles for $C^{\\rm{conc}}$, $Al^{\\rm{conc}}$, and $S^{\\rm{conc}}$, respectively. The considered grid size is 121 $\\times$ 121.}\n\t\\label{fig16}\n\\end{figure}\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure17.png}\n\t\\caption{2D solid amount profiles of the C, Al and S chemical components of cement system 2 after 2 and 6 (final time step) years. RTM means the original HPx$_{\\rm{4C}}$ model and TM+DNN denotes the Hydrus transport model coupled with our DNN geochemical emulator (HPx$_{\\rm py}$-DNN). The first to third row present profiles for $C^{\\rm{solid}}$, $Al^{\\rm{solid}}$, and $S^{\\rm{solid}}$, respectively The considered grid size is 61 $\\times$ 61.}\n\t\\label{fig17}\n\\end{figure}\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure18.png}\n\t\\caption{2D solid amount profiles of the C, Al and S chemical components of cement system 2 after 2 and 6 (final time step) years. RTM means the original HPx$_{\\rm{4C}}$ model and TM+DNN denotes the Hydrus transport model coupled with our DNN geochemical emulator (HPx$_{\\rm py}$-DNN). The first to third row present profiles for $C^{\\rm{solid}}$, $Al^{\\rm{solid}}$, and $S^{\\rm{solid}}$, respectively The considered grid size is 121 $\\times$ 121.}\n\t\\label{fig18}\n\\end{figure}\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure19.png}\n\t\\caption{2D solid amount profiles of the C, Al and S chemical components of cement system 2 after 2 and 6 (final time step) years. RTM means the original HPx$_{\\rm{4C}}$ model and TM+kNN denotes the Hydrus transport model coupled with our kNN geochemical emulator (HPx$_{\\rm py}$-kNN). The first to third row present profiles for $C^{\\rm{solid}}$, $Al^{\\rm{solid}}$, and $S^{\\rm{solid}}$, respectively The considered grid size is 61 $\\times$ 61.}\n\t\\label{fig19}\n\\end{figure}\n\n\\begin{figure}[h!]\n\t\\noindent\\hspace{-1cm}\\includegraphics[width=45pc]{Figure20.png}\n\t\\caption{2D solid amount profiles of the C, Al and S chemical components of cement system 2 after 2 and 6 (final time step) years. RTM means the original HPx$_{\\rm{4C}}$ model and TM+kNN denotes the Hydrus transport model coupled with our kNN geochemical emulator (HPx$_{\\rm py}$-kNN). The first to third row present profiles for $C^{\\rm{solid}}$, $Al^{\\rm{solid}}$, and $S^{\\rm{solid}}$, respectively The considered grid size is 121 $\\times$ 121.}\n\t\\label{fig20}\n\\end{figure}\n\n\\begin{table}[h!]\n\t\\caption{Speedups offered by the KNN and DNN emulators in HPx$_{\\rm py}$ for the reactive transport simulations considered for cement system 2. The HPx$_{\\rm{4C}}$ calculations involve the parallelization of PHREEQC over our 4 CPUs. The HPx$_{\\rm{1C}}$ calculations are performed on a single CPU. Both the kNN and DNN predictions make use of a GPU. ML signifies the used machine learning method for emulation, BC denotes the type of flow boundary conditions and GS is the grid size. The maximum possible speedups associated with HPx$_{\\rm{4C}}$ and HP$_{\\rm{1C}}$,  Max SP HPx$_{\\rm{4C}}$ and  Max SP HPx$_{\\rm{1C}}$, correspond to an hypothetical situation where the geochemical calculations incur zero computational cost.}\n\n\t\\begin{center}\n\t\t\\begin{tabular}{cccccccc}\n\t\t\t\\hline\n\t\t\t& & & & & & & \\\\\n\t\t\tML & BC & GS & HPx$_{\\rm{4C}}$ time (s) & SP HPx$_{\\rm{4C}}$ & Max SP HPx$_{\\rm{4C}}$ & SP HPx$_{\\rm{1C}}$ & Max SP HPx$_{\\rm{1C}}$ \\\\\n\t\t\tDNN & ADV & 61 $\\times$ 61 & 21,415 & 8.2 & 9.0 & 30.3 & 33.1\\\\\n\t\t\tkNN & ADV & 61 $\\times$ 61 & 21,415 & 7.9 & 9.0 & 28.9 & 33.1\\\\\n\t\t\tDNN & ADV & 121 $\\times$ 121 & 199,841 & 8.2 & 9.5 & 29.9 & 32.8\\\\\n\t\t\tkNN & ADV & 121 $\\times$ 121 & 199,841 & 8.5 & 9.5 & 31.3 & 32.8\\\\\n\t\t\t\\hline\n\t\t\\end{tabular}\n\t\\end{center}\n\n\t\\label{table4}\n\\end{table}\n\n\\FloatBarrier\n\n",
        "label": "fig20",
        "Descriptive_question1": "Which chemical component is represented in the first row of figure_20?",
        "Descriptive_question2": "What is the grid size used in figure_20?",
        "Reasoning_question1": "How do the solid amount profiles of C, Al, and S in figure_20 compare between the RTM and TM+kNN models after 6 years?",
        "Reasoning_question2": "What trends can be observed in the changes of solid amounts for C, Al, and S from 2 years to 6 years in figure_20?",
        "Descriptive_answer1": "C",
        "Descriptive_answer2": "121 × 121",
        "Reasoning_answer1": "The solid amount profiles of C, Al, and S in figure_20 show that the TM+kNN model closely approximates the RTM (original HPx_4C) model after 6 years, with some slight discrepancies but overall similar spatial distribution and magnitudes across the grid. The TM+kNN emulated profiles maintain the characteristic patterns of the original RTM, indicating good accuracy of the kNN emulator for these chemical components.",
        "Reasoning_answer2": "From 2 years to 6 years, figure_20 shows an overall change in the solid amounts of C, Al, and S indicating progression of the reactive transport processes. The solid amounts of these components tend to decrease in certain regions, particularly towards the depletion zones, while other areas may show accumulation or little change, reflecting spatial heterogeneity. This trend suggests ongoing reaction and transport dynamics affecting the solid phases over the 4-year interval."
    },
    {
        "paper_id": "1605.00380.json",
        "image_id": "figure_4",
        "image_path": [
            "/home/yz979/scratch/chengye/scimolmo/test-images-tables/dataset_imagestables/1605.00380/images/delta_q_L_and_delta_RH_L_vs_T_ocean_iGCM_sim_and_theory_1.eps"
        ],
        "caption": "Changes over land in (a) surface-air specific humidity and (b) surface-air relative humidity between pairs of idealized GCM simulations with a subtropical continent.  The relative humidity changes are \nnormalized by the land surface-air temperature change.\nSolid black lines denote the simulated changes and the dashed lines represent the estimated changes using the ocean-influence box model (\\ref{eqn:box_model_3}). Pseudo relative humidities are shown (see text), but the blue line in (b) shows the change in the mean of the actual relative humidity for comparison. The red line in (a) indicates what the change in surface-air land specific humidity would be if the land pseudo relative humidity did not change (i.e. for each pair of simulations, the land specific humidity change if the land pseudo relative humidity is fixed at its value in the colder simulation).",
        "classify": "Chart",
        "section_info": "3 Ocean-influence box model\n\\section{Ocean-influence box model}\n\nFor the simplest version of our box model, the ``ocean-influence box model'', we assume that the influence of evapotranspiration on the boundary-layer moisture balance over land is negligible. Setting $E_{\\rm{L}}=0$ in (\\ref{eqn:new_box_model_2}), we find:\n\n\\begin{equation}\nq_{\\rm{L}} \\approx \\gamma q_{\\rm{O}}.\n\\label{eqn:box_model_2}\n\\end{equation}\n\nThe parameter $\\gamma$ may remain approximately constant under climate change even if there are changes in the mixing time scales $\\tau_1$ and $\\tau_2$. For example, if the overall tropical circulation and convective mass fluxes slow down with climate warming \\citep[e.g.,][]{held_soden_2006,vecchi_soden_2007} such that both mixing time scales increase by the same factor, then this will not cause $\\gamma$ to change. Assuming negligible changes in $\\gamma$, we can write:\n\n\\begin{equation}\n\\delta q_{\\rm{L}} \\approx \\gamma \\delta q_{\\rm{O}},\n\\label{eqn:box_model_3}\n\\end{equation}\n\nwhere $\\delta$ denotes the change in climate.\n\nThe same results would also follow (with a different definition of $\\gamma$) if the influence of evapotranspiration on land specific humidity, $q_{\\rm{E}}$, is not neglected but is instead assumed to scale with land specific humidity. \nNote that $q_{\\rm{E}}$ scaling with land specific humidity is not\nthe same as the evapotranspiration rate scaling with land specific humidity.\nIn particular, $q_E$ depends on  \n$(\\tau_1 \\tau_2)/(\\tau_1 + \\tau_2)$, and this factor would change even if\nboth $\\tau_1$ and $\\tau_2$ increase by the same factor.\n\nThe ocean-influence box model suggests a straightforward hypothesis - that the ratio of land to ocean specific humidity remains approximately constant as the climate changes or, equivalently, that fractional changes in specific humidity over land and ocean are equal: \n\n\\begin{equation}\n\\frac{\\delta q_{\\rm{L}}}{q_{\\rm{L}}} \\approx \\frac{\\delta q_{\\rm{O}}}{q_{\\rm{O}}}.\n\\label{eqn:constant_q_ratio}\n\\end{equation}\n\nBy contrast the fractional changes in saturation specific humidity depend on the local temperature change and will be bigger over land than ocean.\nTo the extent that (\\ref{eqn:constant_q_ratio}) holds, it is clear that if land warms more than ocean, and ocean relative humidity does not change greatly, then the land relative humidity will decrease.\n\nWe now assess the applicability of this ocean-influence box model result to idealized and comprehensive GCM simulations. \nWe use (\\ref{eqn:box_model_3}) to estimate the change in land relative humidity under climate change given the changes in land temperature\nand ocean specific humidity, and calculating $\\gamma$ as the the ratio of land to ocean specific humidities in the control climate.  \n\n\n\n\\begin{figure}[t]\n  \\centering\n  \\noindent\\includegraphics[width=19pc,angle=0]{delta_q_L_and_delta_RH_L_vs_T_ocean_iGCM_sim_and_theory_1.eps}\n  \\caption{Changes over land in (a) surface-air specific humidity and (b) surface-air relative humidity between pairs of idealized GCM simulations with a subtropical continent.  The relative humidity changes are \nnormalized by the land surface-air temperature change.\nSolid black lines denote the simulated changes and the dashed lines represent the estimated changes using the ocean-influence box model (\\ref{eqn:box_model_3}). Pseudo relative humidities are shown (see text), but the blue line in (b) shows the change in the mean of the actual relative humidity for comparison. The red line in (a) indicates what the change in surface-air land specific humidity would be if the land pseudo relative humidity did not change (i.e. for each pair of simulations, the land specific humidity change if the land pseudo relative humidity is fixed at its value in the colder simulation).}\n\\label{fig:delta_q_L_and_RH_L_theory_1}\n\\end{figure}\n\n\n\n\n\\subsection{Application of ocean-influence box model to idealized GCM simulations}\n\\label{sect:iGCM_analysis_theory_1}\n\nThe ocean-influence box model is first applied to idealized GCM simulations over a wide range of climates. The idealized GCM is similar to that of \\citet{frierson_2006} and \\citet{frierson_2007}, with specific details as in \\citet{byrne_ogorman_2013} and \\citet{ogorman_2008}. It is based on a spectral version of the GFDL dynamical core, with a two-stream gray radiation scheme, no cloud or water vapor radiative feedbacks, and the simplified moist convection scheme of \\citet{frierson_2007}. The simulations have a subtropical continent spanning $20^{\\circ}\\rm{N}$ to $40^{\\circ}\\rm{N}$ and $0^{\\circ}\\rm{E}$ to $120^{\\circ}\\rm{E}$, with a slab ocean elsewhere (Fig. \\ref{fig:continent_schematic}). The land surface hydrology is simulated using a simple bucket model \\citep{manabe_1969} and all other land surface properties are identical to those of the slab ocean. We vary the climate over a wide range of global-mean surface-air temperatures (between 260K and 317K) by changing the longwave optical thickness, which is analogous to varying the concentrations of $\\rm{CO}_2$ and other greenhouse gases. The longwave optical thickness is specified by $\\tau = \\alpha \\tau_{\\rm{ref}}$, where $\\tau_{\\rm{ref}}$ is a reference optical thickness distribution, and we analyze simulations with 10 different values of the parameter $\\alpha$\\footnote{Simulations are performed with the following $\\alpha$ values: 0.2, 0.4, 0.7, 1.0, 1.5, 2.0, 3.0, 4.0, 5.0, and 6.0.}. We present results based on time averages over 4000 days.\n\nWhen applying the box model to the simulations, we assume that the specific humidity is well-mixed in the boundary layer and use the surface-air specific humidities to represent the entire boundary layer. In the case of the idealized GCM, surface-air quantities are taken to be those of the lowest atmospheric level, $\\sigma = 0.989$, where $\\sigma=p/p_s$, and $p$ and $p_s$ are the pressure and surface pressure, respectively. The results are qualitatively similar when specific humidities are instead averaged between the surface and $\\sigma = 0.9$ (not shown). Land values are averaged (with area weighting) over the entire subtropical continent, and the ocean averages are taken over neighboring ocean at the same latitudes, i.e. from $20^{\\circ}\\rm{N}$ to $40^{\\circ}\\rm{N}$ and $120^{\\circ}\\rm{E}$ to $360^{\\circ}\\rm{E}$.\\footnote{\nOur results are almost identical if we calculate ocean averages using the ``control'' Southern Hemisphere as in \\citet{byrne_ogorman_2013}, i.e. ocean values averaged over $20^{\\circ}\\rm{S}$ to $40^{\\circ}\\rm{S}$ and $0^{\\circ}\\rm{E}$ to $120^{\\circ}\\rm{E}$. We choose to average over neighboring ocean in this study because the box model involves advection of moisture from ocean to land, and this naturally suggests averaging over ocean adjacent to the land continent.} \n\n\n\n\\begin{figure}[t]\n  \\centering\n  \\noindent\\includegraphics[width=19pc,angle=0]{gamma_and_epsilon_vs_T_ocean_nonlinear_new.eps}\n  \\caption{Parameters for the box models applied to the idealized GCM simulations. (a) The $\\gamma$ parameter for the ocean-influence box model (solid black line) and the full box model (dashed black line). (b) The $q_{\\rm{E }}$ parameter for the full box model (dashed black line) and the surface-air land specific humidity (red line) scaled by a factor of 0.25 so that is roughly matches the magnitude of $q_{\\rm{E}}$.}\n\\label{fig:gamma_and_epsilon}\n\\end{figure}\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\nTo apply the ocean-influence box model (\\ref{eqn:box_model_3}), we calculate the $\\gamma$ parameter (the ratio of land to ocean specific humidities) for each simulation (except the warmest). We then estimate the change in surface-air land specific humidity between pairs of nearest-neighbor simulations as a function of $\\gamma$ and the changes in ocean specific humidity, where $\\gamma$ is set to its value in the colder of the two simulations and assumed to be constant as the climate changes. \n\nLand surface-air specific humidity changes between the pairs of idealized GCM simulations, along with the estimates of these changes using (\\ref{eqn:box_model_3}), are plotted against the mid-point ocean temperature for each pair in Fig. \\ref{fig:delta_q_L_and_RH_L_theory_1}.  The simulated specific humidity changes are well-captured by the ocean-influence box model (\\ref{eqn:box_model_3}) over the full range of climates (Fig. \\ref{fig:delta_q_L_and_RH_L_theory_1}a).  \nThe increases in specific humidity, $\\delta q_{\\rm{L}}$, generally increase in magnitude as the climate warms (Fig. \\ref{fig:delta_q_L_and_RH_L_theory_1}a),\n\nand they are generally smaller than what would occur if land relative humidity remained constant (see the red line in Fig. \\ref{fig:delta_q_L_and_RH_L_theory_1}a). \nThe small deviations from the prediction of the ocean-influence box model (\\ref{eqn:box_model_3}) could be due to the influence of evapotranspiration, changes in circulation patterns, or changes in the ratio $\\lambda$ of free tropospheric to surface-air ocean specific humidity which is assumed to be constant in the box model. The parameter $\\lambda$ might be expected to increase with warming because the fractional rate of increase in saturation vapor pressure with temperature is higher at the lower temperatures that occur further up in the atmosphere, and\nbecause there is enhanced warming aloft at low latitudes in simulations of global warming \\citep[e.g.,][]{santer_et_al_2005},\nand such effects could be included in a more complicated box model. \n\nThe $\\gamma$ parameter is relatively constant over the wide range of climates simulated (Fig. \\ref{fig:gamma_and_epsilon}a) consistent with our neglect of changes in $\\gamma$ when deriving (\\ref{eqn:box_model_3}), with a mean value of 0.61 and minimum and maximum values of 0.56 and 0.68, respectively. Thus, for the subtropical continent in this idealized GCM, land specific humidity is approximately 60\\% of the neighboring ocean specific humidity.\n\nThe box model (\\ref{eqn:box_model_3}) predicts the changes in mean specific humidity which must be combined with the mean temperatures to estimate the relative humidity changes. However, because of the nonlinearity of the Clausius-Clapeyron relation, it is not possible to reproduce the mean relative humidity using the mean temperature and mean specific humidity. We instead use a pseudo relative humidity, defined in terms of the mean temperature, specific humidity, and pressure as $H(\\overline{T},\\overline{p},\\overline{q})$ where the bars denote time and spatial means, and $H(T,p,q)$ is the thermodynamic relationship between relative humidity, temperature, pressure and specific humidity. For convenience we will refer to this pseudo relative humidity as the relative humidity, but we also show the actual relative humidity changes for comparison in Figs. \\ref{fig:delta_q_L_and_RH_L_theory_1}b and \\ref{fig:delta_q_RH_CMIP5_sim_theory}b.\n\nThe box-model estimate of the relative humidity changes is somewhat less accurate than for specific humidity, but the decreases in relative humidity with warming and the decreasing magnitude of these changes as the climate warms are both captured (Fig. \\ref{fig:delta_q_L_and_RH_L_theory_1}b). \n\nGiven the simplicity of the ocean-influence box model, its ability to describe the behavior of land relative humidity in this idealized GCM is impressive. However, the geometry and surface properties of Earth's land masses are more varied and complex than the idealized continent considered, and factors such as orography or cloud feedbacks that are not included in the idealized GCM could alter the surface humidity response. Therefore, to investigate the changes in land relative humidity further, we turn to more comprehensive simulations from the CMIP5 archive. \n\n\n\n\\begin{figure}[t]\n  \\centering\n  \\noindent\\includegraphics[width=19pc,angle=0]{delta_SH_land_and_delta_RH_CMIP5_and_estimates_vs_lat.eps}\n  \\caption{Multimodel-mean changes between the historical and RCP8.5 simulations in zonal and time mean (a) surface-air land specific humidity and (b) surface-air land relative humidity normalized by the global-mean surface-air temperature change. Solid black lines denote the simulated changes and dashed lines denote the estimated changes using the box model (\\ref{eqn:box_model_3}). For (a), the red line indicates the change in surface-air land specific humidity for constant land pseudo relative humidity (i.e. for land pseudo relative humidity fixed at the values in the historical simulations). \nPseudo relative humidities are shown, but the blue line in (b) shows the simulated mean changes for the relative humidity variable outputted by the models for comparison.}\n\\label{fig:delta_q_RH_CMIP5_sim_theory}\n\\end{figure}\n\n\\subsection{Application of ocean-influence box model to CMIP5 simulations}\n\\label{sect:CMIP5_analysis_theory_1}\n\nWe apply the ocean-influence box model to changes in land surface-air relative humidity between 30 year time averages in the historical (1976-2005) and RCP8.5 (2070-2099) simulations from the CMIP5 archive \\citep{taylor_et_al_bams_2012}. We analyze 19 models in total\\footnote{The CMIP5 models considered are: ACCESS1-0, ACCESS1-3, BCC-CSM1-1, BCC-CSM1-1-M, BNU-ESM, CanESM2, CNRM-CM5, CSIRO-Mk3-6-0, GFDL-CM3, GFDL-ESM2M, INMCM4, IPSL-CM5A-LR, IPSL-CM5A-MR, IPSL-CM5B-LR, MIROC-ESM, MIROC-ESM-CHEM, MIROC5, MRI-CGCM3, and NorESM1-M. The variables used in this paper have the following names in the CMIP5 archive: evaporation (\\textit{evspsbl}), surface-air specific humidity (\\textit{huss}), surface-air temperature (\\textit{tas}), and surface-air relative humidity (\\textit{hurs}).}, and in each case the r1i1p1 ensemble member is used. As for the idealized GCM analysis, we assume moisture is well mixed in the boundary layer and take surface-air specific humidity to be representative of the boundary layer (using the average specific humidity between the surface and 900hPa gives similar results). \n\n\nThe specific humidities in the box model are identified with the zonal and time mean specific humidities (over land or ocean) for each latitude and for each of the (12) months of the year in the CMIP5 simulations.  We calculate $\\gamma$ as the ratio of the mean land and ocean specific humidities at each latitude and for each month of the year in the historical simulations. By computing $\\gamma$ in this way, we are assuming that the horizontal exchange of moisture between land and ocean, described by the box model, is taking place predominantly in the zonal direction. \nUsing the diagnosed $\\gamma$, and assuming it does not change as the climate warms, changes in mean surface-air land specific humidity are estimated for each latitude and each month of the year using (\\ref{eqn:box_model_3}) and the changes in mean ocean specific humidity. \n\n\n\n\\begin{figure}[t]\n  \\centering\n  \\noindent\\includegraphics[width=19pc,angle=0]{gamma_epsilon_zeta_vs_lat_both_theories_CMIP5.eps}\n  \\caption{Multimodel means in the CMIP5 simulations of the (a) $\\gamma$ parameter for the ocean-influence box model (solid black line) and for the regression approach including evapotranspiration (dashed black line), and (b) the regression coefficient $\\epsilon$. The parameter $\\gamma$ is evaluated based on the historical simulations for the ocean-influence box model, and $\\gamma$ and $\\epsilon$ are evaluated using equation \\ref{eqn:regression} for the regression approach.}\n\\label{fig:gamma_eps_zeta_vs_lat}\n\\end{figure}\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\nThe simulated and estimated annual mean changes in land specific humidity at each latitude are shown in Fig. \\ref{fig:delta_q_RH_CMIP5_sim_theory}a, and the magnitude and latitudinal variations of the changes are reasonably well captured by the ocean-influence box model, including the flat region in the Northern Hemisphere midlatitudes. The magnitude of the increases is underestimated at most latitudes which, as discussed in the case of the idealized GCM simulations could be partly due to increases in the parameter $\\lambda$ relating free-tropospheric specific humidity to ocean surface specific humidity, but other aspects of the ocean-influence box model such as neglecting the influence of evapotranspiration are also likely to play a role. The parameter $\\gamma$ (i.e. the ratio of land and ocean specific humidities) is shown in Fig.~\\ref{fig:gamma_eps_zeta_vs_lat}a. It has a global, annual and multimodel mean value of 0.74, which is somewhat larger than the\nvalue found in the idealized GCM simulations. This is not\nsurprising given that the land in the idealized simulations\nis a subtropical continent, which is generally drier relative\nto neighboring oceans than continents at lower or higher\nlatitudes. \n\n\n\nTogether with the simulated changes in monthly-mean surface-air land temperature, the estimated changes in specific humidity are used to estimate the land pseudo relative humidity changes. As for the idealized GCM analysis, it is necessary to compare pseudo-relative humidities because of the difficulty in converting time and zonal mean specific humidities estimated by the box model to relative humidities. The use of pseudo relative humidities also avoids the complication that different climate models use different saturation vapor pressure formulations. The changes in pseudo relative humidity are calculated for each month of the year before taking the annual mean for both the simulated changes and the changes estimated by the box model. The changes in pseudo relative humidity and model-outputted relative humidity are somewhat similar at lower latitudes but quite different at higher latitudes (cf. blue and black solid lines in Fig. \\ref{fig:delta_q_RH_CMIP5_sim_theory}b), where the differing computations of saturation vapor pressure over ice in the various models becomes important and there is larger variability. Nonetheless the pseudo relative humidity is a useful measure of subsaturation, and we will refer to pseudo relative humidity as relative humidity for simplicity.\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\nThe simulated changes in (pseudo) land relative humidity are quite well described by the ocean-influence box model in the Southern Hemisphere and at lower latitudes (Fig. \\ref{fig:delta_q_RH_CMIP5_sim_theory}b). The estimated and simulated global-mean land relative humidity changes in the various climate models are also correlated, with a correlation coefficient of 0.66. \nDue to the general underestimation of the specific humidity increases by the ocean-influence box model (Fig. \\ref{fig:delta_q_RH_CMIP5_sim_theory}a), the relative humidity decreases are overestimated (Fig. \\ref{fig:delta_q_RH_CMIP5_sim_theory}b), with a large discrepancy in the mid- to high-latitudes of the Northern Hemisphere. At these latitudes, there is more land than ocean and it is likely that changes in ocean specific humidity have a weak influence on the specific humidity in the interior of large continents, or that meridional moisture transports from ocean at other latitudes become more important. Changes in relative humidity in these inner continental regions are likely to be more strongly influenced by local evapotranspiration changes, and they could be influenced by\nshifts in the iceline or changes in soil moisture or vegetation, effects which are not considered in the ocean-influence box model. \n\n\n\n\n\n\n\n\n3.1 Application of ocean-influence box model to idealized GCM simulations\n\\subsection{Application of ocean-influence box model to idealized GCM simulations}\n\\label{sect:iGCM_analysis_theory_1}\n\nThe ocean-influence box model is first applied to idealized GCM simulations over a wide range of climates. The idealized GCM is similar to that of \\citet{frierson_2006} and \\citet{frierson_2007}, with specific details as in \\citet{byrne_ogorman_2013} and \\citet{ogorman_2008}. It is based on a spectral version of the GFDL dynamical core, with a two-stream gray radiation scheme, no cloud or water vapor radiative feedbacks, and the simplified moist convection scheme of \\citet{frierson_2007}. The simulations have a subtropical continent spanning $20^{\\circ}\\rm{N}$ to $40^{\\circ}\\rm{N}$ and $0^{\\circ}\\rm{E}$ to $120^{\\circ}\\rm{E}$, with a slab ocean elsewhere (Fig. \\ref{fig:continent_schematic}). The land surface hydrology is simulated using a simple bucket model \\citep{manabe_1969} and all other land surface properties are identical to those of the slab ocean. We vary the climate over a wide range of global-mean surface-air temperatures (between 260K and 317K) by changing the longwave optical thickness, which is analogous to varying the concentrations of $\\rm{CO}_2$ and other greenhouse gases. The longwave optical thickness is specified by $\\tau = \\alpha \\tau_{\\rm{ref}}$, where $\\tau_{\\rm{ref}}$ is a reference optical thickness distribution, and we analyze simulations with 10 different values of the parameter $\\alpha$\\footnote{Simulations are performed with the following $\\alpha$ values: 0.2, 0.4, 0.7, 1.0, 1.5, 2.0, 3.0, 4.0, 5.0, and 6.0.}. We present results based on time averages over 4000 days.\n\nWhen applying the box model to the simulations, we assume that the specific humidity is well-mixed in the boundary layer and use the surface-air specific humidities to represent the entire boundary layer. In the case of the idealized GCM, surface-air quantities are taken to be those of the lowest atmospheric level, $\\sigma = 0.989$, where $\\sigma=p/p_s$, and $p$ and $p_s$ are the pressure and surface pressure, respectively. The results are qualitatively similar when specific humidities are instead averaged between the surface and $\\sigma = 0.9$ (not shown). Land values are averaged (with area weighting) over the entire subtropical continent, and the ocean averages are taken over neighboring ocean at the same latitudes, i.e. from $20^{\\circ}\\rm{N}$ to $40^{\\circ}\\rm{N}$ and $120^{\\circ}\\rm{E}$ to $360^{\\circ}\\rm{E}$.\\footnote{\nOur results are almost identical if we calculate ocean averages using the ``control'' Southern Hemisphere as in \\citet{byrne_ogorman_2013}, i.e. ocean values averaged over $20^{\\circ}\\rm{S}$ to $40^{\\circ}\\rm{S}$ and $0^{\\circ}\\rm{E}$ to $120^{\\circ}\\rm{E}$. We choose to average over neighboring ocean in this study because the box model involves advection of moisture from ocean to land, and this naturally suggests averaging over ocean adjacent to the land continent.} \n\n\n\n\\begin{figure}[t]\n  \\centering\n  \\noindent\\includegraphics[width=19pc,angle=0]{gamma_and_epsilon_vs_T_ocean_nonlinear_new.eps}\n  \\caption{Parameters for the box models applied to the idealized GCM simulations. (a) The $\\gamma$ parameter for the ocean-influence box model (solid black line) and the full box model (dashed black line). (b) The $q_{\\rm{E }}$ parameter for the full box model (dashed black line) and the surface-air land specific humidity (red line) scaled by a factor of 0.25 so that is roughly matches the magnitude of $q_{\\rm{E}}$.}\n\\label{fig:gamma_and_epsilon}\n\\end{figure}\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\nTo apply the ocean-influence box model (\\ref{eqn:box_model_3}), we calculate the $\\gamma$ parameter (the ratio of land to ocean specific humidities) for each simulation (except the warmest). We then estimate the change in surface-air land specific humidity between pairs of nearest-neighbor simulations as a function of $\\gamma$ and the changes in ocean specific humidity, where $\\gamma$ is set to its value in the colder of the two simulations and assumed to be constant as the climate changes. \n\nLand surface-air specific humidity changes between the pairs of idealized GCM simulations, along with the estimates of these changes using (\\ref{eqn:box_model_3}), are plotted against the mid-point ocean temperature for each pair in Fig. \\ref{fig:delta_q_L_and_RH_L_theory_1}.  The simulated specific humidity changes are well-captured by the ocean-influence box model (\\ref{eqn:box_model_3}) over the full range of climates (Fig. \\ref{fig:delta_q_L_and_RH_L_theory_1}a).  \nThe increases in specific humidity, $\\delta q_{\\rm{L}}$, generally increase in magnitude as the climate warms (Fig. \\ref{fig:delta_q_L_and_RH_L_theory_1}a),\n\nand they are generally smaller than what would occur if land relative humidity remained constant (see the red line in Fig. \\ref{fig:delta_q_L_and_RH_L_theory_1}a). \nThe small deviations from the prediction of the ocean-influence box model (\\ref{eqn:box_model_3}) could be due to the influence of evapotranspiration, changes in circulation patterns, or changes in the ratio $\\lambda$ of free tropospheric to surface-air ocean specific humidity which is assumed to be constant in the box model. The parameter $\\lambda$ might be expected to increase with warming because the fractional rate of increase in saturation vapor pressure with temperature is higher at the lower temperatures that occur further up in the atmosphere, and\nbecause there is enhanced warming aloft at low latitudes in simulations of global warming \\citep[e.g.,][]{santer_et_al_2005},\nand such effects could be included in a more complicated box model. \n\nThe $\\gamma$ parameter is relatively constant over the wide range of climates simulated (Fig. \\ref{fig:gamma_and_epsilon}a) consistent with our neglect of changes in $\\gamma$ when deriving (\\ref{eqn:box_model_3}), with a mean value of 0.61 and minimum and maximum values of 0.56 and 0.68, respectively. Thus, for the subtropical continent in this idealized GCM, land specific humidity is approximately 60\\% of the neighboring ocean specific humidity.\n\nThe box model (\\ref{eqn:box_model_3}) predicts the changes in mean specific humidity which must be combined with the mean temperatures to estimate the relative humidity changes. However, because of the nonlinearity of the Clausius-Clapeyron relation, it is not possible to reproduce the mean relative humidity using the mean temperature and mean specific humidity. We instead use a pseudo relative humidity, defined in terms of the mean temperature, specific humidity, and pressure as $H(\\overline{T},\\overline{p},\\overline{q})$ where the bars denote time and spatial means, and $H(T,p,q)$ is the thermodynamic relationship between relative humidity, temperature, pressure and specific humidity. For convenience we will refer to this pseudo relative humidity as the relative humidity, but we also show the actual relative humidity changes for comparison in Figs. \\ref{fig:delta_q_L_and_RH_L_theory_1}b and \\ref{fig:delta_q_RH_CMIP5_sim_theory}b.\n\nThe box-model estimate of the relative humidity changes is somewhat less accurate than for specific humidity, but the decreases in relative humidity with warming and the decreasing magnitude of these changes as the climate warms are both captured (Fig. \\ref{fig:delta_q_L_and_RH_L_theory_1}b). \n\nGiven the simplicity of the ocean-influence box model, its ability to describe the behavior of land relative humidity in this idealized GCM is impressive. However, the geometry and surface properties of Earth's land masses are more varied and complex than the idealized continent considered, and factors such as orography or cloud feedbacks that are not included in the idealized GCM could alter the surface humidity response. Therefore, to investigate the changes in land relative humidity further, we turn to more comprehensive simulations from the CMIP5 archive. \n\n\n\n\\begin{figure}[t]\n  \\centering\n  \\noindent\\includegraphics[width=19pc,angle=0]{delta_SH_land_and_delta_RH_CMIP5_and_estimates_vs_lat.eps}\n  \\caption{Multimodel-mean changes between the historical and RCP8.5 simulations in zonal and time mean (a) surface-air land specific humidity and (b) surface-air land relative humidity normalized by the global-mean surface-air temperature change. Solid black lines denote the simulated changes and dashed lines denote the estimated changes using the box model (\\ref{eqn:box_model_3}). For (a), the red line indicates the change in surface-air land specific humidity for constant land pseudo relative humidity (i.e. for land pseudo relative humidity fixed at the values in the historical simulations). \nPseudo relative humidities are shown, but the blue line in (b) shows the simulated mean changes for the relative humidity variable outputted by the models for comparison.}\n\\label{fig:delta_q_RH_CMIP5_sim_theory}\n\\end{figure}\n\n4 Influence of evapotranspiration\n\\section{Influence of evapotranspiration}\n\\label{sect:full_model_RH_land}\n\n\n\n\nThe ocean-influence box model captures much (but not all) of the behavior in vastly more complex GCMs. However, the moisture balance of the land boundary layer is also affected by evapotranspiration, and changes in land surface properties, such as soil moisture or stomatal conductance, can affect evapotranspiration in the absence of any changes in the overlying atmosphere.  For example, changes in stomatal conductance under elevated $\\rm{CO}_2$ conditions has been shown to reduce both evapotranspiration and land relative humidity without changes in ocean humidity \\citep{andrews_et_al_2011}. \n\n\n\n\\begin{figure}[t]\n  \\centering\n  \\noindent\\includegraphics[width=19pc,angle=0]{delta_q_L_and_delta_RH_L_vs_T_ocean_iGCM_sim_and_theory_2_nonlinear_new.eps}\n  \\caption{As in Figure \\ref{fig:delta_q_L_and_RH_L_theory_1}, but here showing estimates of the surface-air specific and relative humidity changes from the full box model (\\ref{eqn:new_box_model_3}). The contributions due to ocean specific humidity changes (blue dashed lines) and evapotranspiration changes (green dashed lines) are also shown. The contributions to changes in relative humidity are calculated using (\\ref{eqn:appendix_3}). Pseudo relative humidities are shown in this figure (the changes in actual relative humidity are shown by the blue line in Fig.~\\ref{fig:delta_q_L_and_RH_L_theory_1}b.)}\n\\label{fig:delta_q_L_and_RH_L_theory_2}\n\\end{figure}\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\nWe turn to the full box model (\\ref{eqn:new_box_model_2}) which includes\nthe effects of evapotranspiration. We assume once more that changes in $\\gamma$ are negligible, \n\nsuch that:\n\n\\begin{equation}\n\\delta q_{\\rm{L}} \\approx \\gamma \\delta q_{\\rm{O}} + \\delta q_{\\rm{E}}.\n\\label{eqn:new_box_model_3}\n\\end{equation}\n\nThere are two terms contributing to changes in $q_{\\rm{L}}$ in (\\ref{eqn:new_box_model_3}): the term arising from changes in ocean specific humidity, $\\gamma \\delta q_{\\rm{O}}$, and an additional land evapotranspiration term, $\\delta q_{\\rm{E}}$. We next assess their relative importance in controlling changes in land humidity in the idealized GCM and CMIP5 simulations.\n\n\n\\subsection{Application of full box model to idealized GCM simulations}\n\\label{sect:theory_2_simulation_analysis}\n\nWe first examine the idealized GCM simulations with a subtropical continent. In contrast to the ocean-influence box model (\\ref{eqn:box_model_3}), for which the single parameter $\\gamma$ could be easily estimated in the control simulation in each case, the full model (\\ref{eqn:new_box_model_3}) has two parameters to be estimated, $\\gamma$ and $q_{\\rm{E}}$. To estimate these parameters, we perform an additional set of simulations with the same longwave optical thicknesses as in the 10 simulations described previously but with the evapotranspiration set to zero over land. \n\nSpecifying the land evapotranspiration in this way is analogous to drying out the soil. (Note that the change in evapotranspiration affects both the humidity of the atmosphere and the surface energy balance.)\nUsing these additional simulations with $E_{\\rm{L}} = 0$, we can estimate $\\gamma$ for each climate using (\\ref{eqn:new_box_model_2}): $\\gamma = q_{\\rm{L},E_{\\rm{L}}=0} / q_{\\rm{O},E_{\\rm{L}}=0}$.  The $\\gamma$ values obtained are smaller than those calculated from the control climate for the ocean-influence box model (Fig. \\ref{fig:gamma_and_epsilon}a) because the contribution of evapotranspiration to the land specific humidity is now also taken into account.  We then use these $\\gamma$ values to estimate $q_{\\rm{E}}$ for the original simulations with dynamic land surface hydrology \n: $q_{\\rm{E}} = q_{\\rm{L}} - \\gamma q_{\\rm{O}}$. The values of $q_{\\rm{E}}$ increase with warming except in hot climates (Fig. \\ref{fig:gamma_and_epsilon}b).\n\nInterestingly, the influence of evapotranspiration on land specific humidity as measured by $q_{\\rm{E}}$ roughly scales with the land specific humidity except in hot climates (compare the dashed black and solid red lines in Fig. \\ref{fig:gamma_and_epsilon}b), and this helps to explain why the ocean-influence box model\nis accurate even though evapotranspiration affects land specific humidity.\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\nWe then estimate the changes in land specific humidity between pairs of nearest-neighbor simulations from (\\ref{eqn:new_box_model_3}) with $\\gamma$ assumed to be constant as the climate changes.  The simulated and estimated changes in surface-air land specific humidity, along with the contributions due to changes in ocean specific humidity and land evapotranspiration are shown in Fig. \\ref{fig:delta_q_L_and_RH_L_theory_2}a. The full box model captures the basic behavior of the land specific humidity changes as a function of temperature, although it is less accurate in hot climates. The contribution from ocean specific humidity changes, $\\gamma \\delta q_{\\rm{O}}$, is larger than the contribution from land evapotranspiration for all climates (Fig. \\ref{fig:delta_q_L_and_RH_L_theory_2}a).  The changes in simulated land (pseudo) relative humidity are also well captured by the full box model (Fig. \\ref{fig:delta_q_L_and_RH_L_theory_2}b). \n\n\n\n\n\n\nBecause relative humidity depends on temperature as well as specific humidity, \nthere is no unique way to use the box model result (\\ref{eqn:new_box_model_3}) to decompose changes in land relative humidity into contributions due to ocean specific humidity and land evapotranspiration. However, a decomposition derived in appendix A (equation \\ref{eqn:appendix_3}) has several desirable properties. According to the decomposition, the contributions to the change in land relative humidity from evapotranspiration and ocean specific humidity are weighted according to their contribution to land specific humidity in the control climate. The change in ocean specific humidity leads to a decrease in land relative humidity if the fractional increase in ocean specific humidity is less than the fractional increase in saturation specific humidity over land. Similarly, evapotranspiration contributes to a decrease in land relative humidity if the fractional increase in $q_{\\rm{E}}$ is less than the fractional increase in saturation specific humidity over land.  (Note that the fractional change in $q_{\\rm{E}}$ is generally different from the fractional change in evapotranspiration.)  \n\nUsing this decomposition of the change in land relative humidity, we find that the land evapotranspiration contribution is of comparable importance to the ocean specific humidity contribution  for the idealized GCM simulations\n(Fig. \\ref{fig:delta_q_L_and_RH_L_theory_2}b). By contrast, we found that the\ncontribution of ocean specific humidity was more important than land evapotranspiration when land specific humidity changes were considered.  The discrepancy arises because, according to the decomposition (\\ref{eqn:appendix_3}), it is not the magnitude of a particular contribution to the change in specific humidity that matters for its contribution to the change in relative humidity, but rather how its fractional changes compare to the fractional changes in saturation specific humidity and how much it contributes to the land specific humidity in the control climate. \n\n\n\n\\subsection{Influence of evapotranspiration in CMIP5 simulations}\n\nWe now investigate how land evapotranspiration contributes to specific humidity changes in the CMIP5 simulations. \n\nWe need to estimate both $\\gamma$ and $q_{\\rm{E}}$ for the full box model, but there are no CMIP5 simulations analogous to the zero-evapotranspiration simulations with the idealized GCM described above. Instead, we estimate the influence of evapotranspiration and ocean specific humidity on land specific humidity using a multiple linear regression approach based on the intermodel scatter across the CMIP5 models. We use the regression relationship\n\n\\begin{equation}\n\\delta q_{\\rm{L}} = \\gamma \\delta q_{\\rm{O}} + \\epsilon \\delta E_{\\rm{L}} + \\zeta, \n\\label{eqn:regression}\n\\end{equation}\n\nwhich is motivated by the full box model (\\ref{eqn:new_box_model_2}), but note that $\\epsilon \\delta E_{\\rm{L}}$ will not generally equal $\\delta q_{\\rm{E}}$ to the extent that parameters such as $\\tau_1$ and $\\tau_2$ change with climate, and changes in these parameters will contribute to the remainder term $\\zeta$.  The variables $\\delta q_{\\rm{L}},\\delta q_{\\rm{O}}$ and  $\\delta E_{\\rm{L}}$ are identified as the zonal and time mean for each latitude and month of the year in each model.  The regression coefficients $\\gamma$, $\\epsilon$, and $\\zeta$ are then estimated using ordinary least squares regression for each latitude and month of the year. The annual means are shown for $\\gamma$ and $\\epsilon$ in Fig.~\\ref{fig:gamma_eps_zeta_vs_lat} and for $\\zeta$ in Fig.~\\ref{fig:delta_SH_land_CMIP5_theory_2}. The regression coefficient $\\gamma$ has a similar magnitude and latitudinal structure to the $\\gamma$ parameter calculated for the ocean-influence box model (Fig.~\\ref{fig:gamma_eps_zeta_vs_lat}a). The coefficient $\\epsilon$ is positive at all latitudes (Fig.~\\ref{fig:gamma_eps_zeta_vs_lat}b) indicating that enhanced evapotranspiration increases the land specific humidity, while the remainder term $\\zeta$ (Fig.~\\ref{fig:delta_SH_land_CMIP5_theory_2}) is negative at most latitudes. \n\n\n\n\\begin{figure}[t]\n  \\centering\n\n   \\noindent\\includegraphics[width=19pc,angle=0]{delta_SH_land_and_estimates_vs_lat_theory_2_v4.eps}\n  \\caption{Multimodel-mean changes in zonal- and time-mean surface-air land specific humidity (solid black line) in the CMIP5 simulations, and the contributions to these changes due to ocean specific humidity changes ($\\gamma \\delta q_{\\rm{O}}$; blue dashed line), land evapotranspiration changes ($\\epsilon \\delta E_{\\rm{L}}$; green dashed line), and the remainder term ($\\zeta$; red dashed line) as estimated using the regression relation (\\ref{eqn:regression}).}\n\\label{fig:delta_SH_land_CMIP5_theory_2}\n\\end{figure}\n\n\n\n\n\n\n\n\n\n\n\n\nBy construction, the regression relationship (\\ref{eqn:regression}) is exactly satisfied in the multimodel mean. Based on this relationship, the annual-mean contributions to changes in land specific humidity from changes in ocean specific humidity, changes in land evapotranspiration, and the remainder term are shown in Fig. \\ref{fig:delta_SH_land_CMIP5_theory_2}. At all latitudes, changes in land specific humidity are dominated by the ocean specific humidity contribution. The contribution due to changes in land evapotranspiration is positive and has its largest values in the Northern Hemisphere where the land fraction is greatest. \n\nIt is not possible to estimate the individual contributions to changes in land relative humidity for the CMIP5 simulations, as we did for the idealized GCM simulations. This is because the decomposition of relative humidity changes discussed in appendix A involves the individual contributions to land specific humidity in the control climate, and these are difficult to calculate using a regression approach. However, the results from the idealized GCM simulations suggest that evapotranspiration could be important for the changes in land relative humidity in the CMIP5 simulations, even though it is a second order influence for changes in land specific humidity. It would be worthwhile to estimate the land evapotranspiration contribution for full-complexity GCMs by performing simulations with specified land evapotranspiration rates as was done for the idealized GCM in this study.\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n",
        "label": "fig:delta_q_L_and_RH_L_theory_1",
        "Descriptive_question1": "What color is used to represent the simulated changes in surface-air specific humidity over land in figure_4?",
        "Descriptive_question2": "Which line style represents the estimated changes using the ocean-influence box model in figure_4?",
        "Reasoning_question1": "How do the simulated changes in surface-air specific humidity over land compare to the estimated changes using the ocean-influence box model in figure_4, and what might explain any discrepancies?",
        "Reasoning_question2": "What trend can be observed in the relative humidity changes over land with increasing temperature in figure_4, and how does this relate to the predictions of the ocean-influence box model?",
        "Descriptive_answer1": "black",
        "Descriptive_answer2": "dashed",
        "Reasoning_answer1": "The simulated changes in surface-air specific humidity over land, shown by the solid black lines, closely follow the estimated changes using the ocean-influence box model, represented by the dashed lines, over a wide range of climates. However, small deviations occur where the simulated changes are slightly higher or lower than the estimates. These discrepancies could be due to factors not included in the simple ocean-influence box model, such as the influence of evapotranspiration, changes in circulation patterns, or variations in parameters relating the free troposphere to surface humidity, which the model assumes to be constant.",
        "Reasoning_answer2": "The relative humidity changes over land show a decreasing trend with increasing temperature in figure_4b, indicating that relative humidity tends to drop as the climate warms. This decrease becomes less pronounced at higher temperatures. This observed trend aligns well with the ocean-influence box model predictions, which capture both the magnitude and the declining rate of relative humidity drop with warming. The model suggests this behavior arises because the ratio of land to ocean specific humidity remains roughly constant, so with land warming more than ocean and ocean relative humidity staying nearly steady, land relative humidity decreases."
    },
    {
        "paper_id": "1605.00380.json",
        "image_id": "figure_5",
        "image_path": [
            "/home/yz979/scratch/chengye/scimolmo/test-images-tables/dataset_imagestables/1605.00380/images/gamma_and_epsilon_vs_T_ocean_nonlinear_new.eps"
        ],
        "caption": "Parameters for the box models applied to the idealized GCM simulations. (a) The $\\gamma$ parameter for the ocean-influence box model (solid black line) and the full box model (dashed black line). (b) The $q_{\\rm{E }}$ parameter for the full box model (dashed black line) and the surface-air land specific humidity (red line) scaled by a factor of 0.25 so that is roughly matches the magnitude of $q_{\\rm{E}}$.",
        "classify": "Chart",
        "section_info": "3 Ocean-influence box model\n\\section{Ocean-influence box model}\n\nFor the simplest version of our box model, the ``ocean-influence box model'', we assume that the influence of evapotranspiration on the boundary-layer moisture balance over land is negligible. Setting $E_{\\rm{L}}=0$ in (\\ref{eqn:new_box_model_2}), we find:\n\n\\begin{equation}\nq_{\\rm{L}} \\approx \\gamma q_{\\rm{O}}.\n\\label{eqn:box_model_2}\n\\end{equation}\n\nThe parameter $\\gamma$ may remain approximately constant under climate change even if there are changes in the mixing time scales $\\tau_1$ and $\\tau_2$. For example, if the overall tropical circulation and convective mass fluxes slow down with climate warming \\citep[e.g.,][]{held_soden_2006,vecchi_soden_2007} such that both mixing time scales increase by the same factor, then this will not cause $\\gamma$ to change. Assuming negligible changes in $\\gamma$, we can write:\n\n\\begin{equation}\n\\delta q_{\\rm{L}} \\approx \\gamma \\delta q_{\\rm{O}},\n\\label{eqn:box_model_3}\n\\end{equation}\n\nwhere $\\delta$ denotes the change in climate.\n\nThe same results would also follow (with a different definition of $\\gamma$) if the influence of evapotranspiration on land specific humidity, $q_{\\rm{E}}$, is not neglected but is instead assumed to scale with land specific humidity. \nNote that $q_{\\rm{E}}$ scaling with land specific humidity is not\nthe same as the evapotranspiration rate scaling with land specific humidity.\nIn particular, $q_E$ depends on  \n$(\\tau_1 \\tau_2)/(\\tau_1 + \\tau_2)$, and this factor would change even if\nboth $\\tau_1$ and $\\tau_2$ increase by the same factor.\n\nThe ocean-influence box model suggests a straightforward hypothesis - that the ratio of land to ocean specific humidity remains approximately constant as the climate changes or, equivalently, that fractional changes in specific humidity over land and ocean are equal: \n\n\\begin{equation}\n\\frac{\\delta q_{\\rm{L}}}{q_{\\rm{L}}} \\approx \\frac{\\delta q_{\\rm{O}}}{q_{\\rm{O}}}.\n\\label{eqn:constant_q_ratio}\n\\end{equation}\n\nBy contrast the fractional changes in saturation specific humidity depend on the local temperature change and will be bigger over land than ocean.\nTo the extent that (\\ref{eqn:constant_q_ratio}) holds, it is clear that if land warms more than ocean, and ocean relative humidity does not change greatly, then the land relative humidity will decrease.\n\nWe now assess the applicability of this ocean-influence box model result to idealized and comprehensive GCM simulations. \nWe use (\\ref{eqn:box_model_3}) to estimate the change in land relative humidity under climate change given the changes in land temperature\nand ocean specific humidity, and calculating $\\gamma$ as the the ratio of land to ocean specific humidities in the control climate.  \n\n\n\n\\begin{figure}[t]\n  \\centering\n  \\noindent\\includegraphics[width=19pc,angle=0]{delta_q_L_and_delta_RH_L_vs_T_ocean_iGCM_sim_and_theory_1.eps}\n  \\caption{Changes over land in (a) surface-air specific humidity and (b) surface-air relative humidity between pairs of idealized GCM simulations with a subtropical continent.  The relative humidity changes are \nnormalized by the land surface-air temperature change.\nSolid black lines denote the simulated changes and the dashed lines represent the estimated changes using the ocean-influence box model (\\ref{eqn:box_model_3}). Pseudo relative humidities are shown (see text), but the blue line in (b) shows the change in the mean of the actual relative humidity for comparison. The red line in (a) indicates what the change in surface-air land specific humidity would be if the land pseudo relative humidity did not change (i.e. for each pair of simulations, the land specific humidity change if the land pseudo relative humidity is fixed at its value in the colder simulation).}\n\\label{fig:delta_q_L_and_RH_L_theory_1}\n\\end{figure}\n\n\n\n\n\\subsection{Application of ocean-influence box model to idealized GCM simulations}\n\\label{sect:iGCM_analysis_theory_1}\n\nThe ocean-influence box model is first applied to idealized GCM simulations over a wide range of climates. The idealized GCM is similar to that of \\citet{frierson_2006} and \\citet{frierson_2007}, with specific details as in \\citet{byrne_ogorman_2013} and \\citet{ogorman_2008}. It is based on a spectral version of the GFDL dynamical core, with a two-stream gray radiation scheme, no cloud or water vapor radiative feedbacks, and the simplified moist convection scheme of \\citet{frierson_2007}. The simulations have a subtropical continent spanning $20^{\\circ}\\rm{N}$ to $40^{\\circ}\\rm{N}$ and $0^{\\circ}\\rm{E}$ to $120^{\\circ}\\rm{E}$, with a slab ocean elsewhere (Fig. \\ref{fig:continent_schematic}). The land surface hydrology is simulated using a simple bucket model \\citep{manabe_1969} and all other land surface properties are identical to those of the slab ocean. We vary the climate over a wide range of global-mean surface-air temperatures (between 260K and 317K) by changing the longwave optical thickness, which is analogous to varying the concentrations of $\\rm{CO}_2$ and other greenhouse gases. The longwave optical thickness is specified by $\\tau = \\alpha \\tau_{\\rm{ref}}$, where $\\tau_{\\rm{ref}}$ is a reference optical thickness distribution, and we analyze simulations with 10 different values of the parameter $\\alpha$\\footnote{Simulations are performed with the following $\\alpha$ values: 0.2, 0.4, 0.7, 1.0, 1.5, 2.0, 3.0, 4.0, 5.0, and 6.0.}. We present results based on time averages over 4000 days.\n\nWhen applying the box model to the simulations, we assume that the specific humidity is well-mixed in the boundary layer and use the surface-air specific humidities to represent the entire boundary layer. In the case of the idealized GCM, surface-air quantities are taken to be those of the lowest atmospheric level, $\\sigma = 0.989$, where $\\sigma=p/p_s$, and $p$ and $p_s$ are the pressure and surface pressure, respectively. The results are qualitatively similar when specific humidities are instead averaged between the surface and $\\sigma = 0.9$ (not shown). Land values are averaged (with area weighting) over the entire subtropical continent, and the ocean averages are taken over neighboring ocean at the same latitudes, i.e. from $20^{\\circ}\\rm{N}$ to $40^{\\circ}\\rm{N}$ and $120^{\\circ}\\rm{E}$ to $360^{\\circ}\\rm{E}$.\\footnote{\nOur results are almost identical if we calculate ocean averages using the ``control'' Southern Hemisphere as in \\citet{byrne_ogorman_2013}, i.e. ocean values averaged over $20^{\\circ}\\rm{S}$ to $40^{\\circ}\\rm{S}$ and $0^{\\circ}\\rm{E}$ to $120^{\\circ}\\rm{E}$. We choose to average over neighboring ocean in this study because the box model involves advection of moisture from ocean to land, and this naturally suggests averaging over ocean adjacent to the land continent.} \n\n\n\n\\begin{figure}[t]\n  \\centering\n  \\noindent\\includegraphics[width=19pc,angle=0]{gamma_and_epsilon_vs_T_ocean_nonlinear_new.eps}\n  \\caption{Parameters for the box models applied to the idealized GCM simulations. (a) The $\\gamma$ parameter for the ocean-influence box model (solid black line) and the full box model (dashed black line). (b) The $q_{\\rm{E }}$ parameter for the full box model (dashed black line) and the surface-air land specific humidity (red line) scaled by a factor of 0.25 so that is roughly matches the magnitude of $q_{\\rm{E}}$.}\n\\label{fig:gamma_and_epsilon}\n\\end{figure}\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\nTo apply the ocean-influence box model (\\ref{eqn:box_model_3}), we calculate the $\\gamma$ parameter (the ratio of land to ocean specific humidities) for each simulation (except the warmest). We then estimate the change in surface-air land specific humidity between pairs of nearest-neighbor simulations as a function of $\\gamma$ and the changes in ocean specific humidity, where $\\gamma$ is set to its value in the colder of the two simulations and assumed to be constant as the climate changes. \n\nLand surface-air specific humidity changes between the pairs of idealized GCM simulations, along with the estimates of these changes using (\\ref{eqn:box_model_3}), are plotted against the mid-point ocean temperature for each pair in Fig. \\ref{fig:delta_q_L_and_RH_L_theory_1}.  The simulated specific humidity changes are well-captured by the ocean-influence box model (\\ref{eqn:box_model_3}) over the full range of climates (Fig. \\ref{fig:delta_q_L_and_RH_L_theory_1}a).  \nThe increases in specific humidity, $\\delta q_{\\rm{L}}$, generally increase in magnitude as the climate warms (Fig. \\ref{fig:delta_q_L_and_RH_L_theory_1}a),\n\nand they are generally smaller than what would occur if land relative humidity remained constant (see the red line in Fig. \\ref{fig:delta_q_L_and_RH_L_theory_1}a). \nThe small deviations from the prediction of the ocean-influence box model (\\ref{eqn:box_model_3}) could be due to the influence of evapotranspiration, changes in circulation patterns, or changes in the ratio $\\lambda$ of free tropospheric to surface-air ocean specific humidity which is assumed to be constant in the box model. The parameter $\\lambda$ might be expected to increase with warming because the fractional rate of increase in saturation vapor pressure with temperature is higher at the lower temperatures that occur further up in the atmosphere, and\nbecause there is enhanced warming aloft at low latitudes in simulations of global warming \\citep[e.g.,][]{santer_et_al_2005},\nand such effects could be included in a more complicated box model. \n\nThe $\\gamma$ parameter is relatively constant over the wide range of climates simulated (Fig. \\ref{fig:gamma_and_epsilon}a) consistent with our neglect of changes in $\\gamma$ when deriving (\\ref{eqn:box_model_3}), with a mean value of 0.61 and minimum and maximum values of 0.56 and 0.68, respectively. Thus, for the subtropical continent in this idealized GCM, land specific humidity is approximately 60\\% of the neighboring ocean specific humidity.\n\nThe box model (\\ref{eqn:box_model_3}) predicts the changes in mean specific humidity which must be combined with the mean temperatures to estimate the relative humidity changes. However, because of the nonlinearity of the Clausius-Clapeyron relation, it is not possible to reproduce the mean relative humidity using the mean temperature and mean specific humidity. We instead use a pseudo relative humidity, defined in terms of the mean temperature, specific humidity, and pressure as $H(\\overline{T},\\overline{p},\\overline{q})$ where the bars denote time and spatial means, and $H(T,p,q)$ is the thermodynamic relationship between relative humidity, temperature, pressure and specific humidity. For convenience we will refer to this pseudo relative humidity as the relative humidity, but we also show the actual relative humidity changes for comparison in Figs. \\ref{fig:delta_q_L_and_RH_L_theory_1}b and \\ref{fig:delta_q_RH_CMIP5_sim_theory}b.\n\nThe box-model estimate of the relative humidity changes is somewhat less accurate than for specific humidity, but the decreases in relative humidity with warming and the decreasing magnitude of these changes as the climate warms are both captured (Fig. \\ref{fig:delta_q_L_and_RH_L_theory_1}b). \n\nGiven the simplicity of the ocean-influence box model, its ability to describe the behavior of land relative humidity in this idealized GCM is impressive. However, the geometry and surface properties of Earth's land masses are more varied and complex than the idealized continent considered, and factors such as orography or cloud feedbacks that are not included in the idealized GCM could alter the surface humidity response. Therefore, to investigate the changes in land relative humidity further, we turn to more comprehensive simulations from the CMIP5 archive. \n\n\n\n\\begin{figure}[t]\n  \\centering\n  \\noindent\\includegraphics[width=19pc,angle=0]{delta_SH_land_and_delta_RH_CMIP5_and_estimates_vs_lat.eps}\n  \\caption{Multimodel-mean changes between the historical and RCP8.5 simulations in zonal and time mean (a) surface-air land specific humidity and (b) surface-air land relative humidity normalized by the global-mean surface-air temperature change. Solid black lines denote the simulated changes and dashed lines denote the estimated changes using the box model (\\ref{eqn:box_model_3}). For (a), the red line indicates the change in surface-air land specific humidity for constant land pseudo relative humidity (i.e. for land pseudo relative humidity fixed at the values in the historical simulations). \nPseudo relative humidities are shown, but the blue line in (b) shows the simulated mean changes for the relative humidity variable outputted by the models for comparison.}\n\\label{fig:delta_q_RH_CMIP5_sim_theory}\n\\end{figure}\n\n\\subsection{Application of ocean-influence box model to CMIP5 simulations}\n\\label{sect:CMIP5_analysis_theory_1}\n\nWe apply the ocean-influence box model to changes in land surface-air relative humidity between 30 year time averages in the historical (1976-2005) and RCP8.5 (2070-2099) simulations from the CMIP5 archive \\citep{taylor_et_al_bams_2012}. We analyze 19 models in total\\footnote{The CMIP5 models considered are: ACCESS1-0, ACCESS1-3, BCC-CSM1-1, BCC-CSM1-1-M, BNU-ESM, CanESM2, CNRM-CM5, CSIRO-Mk3-6-0, GFDL-CM3, GFDL-ESM2M, INMCM4, IPSL-CM5A-LR, IPSL-CM5A-MR, IPSL-CM5B-LR, MIROC-ESM, MIROC-ESM-CHEM, MIROC5, MRI-CGCM3, and NorESM1-M. The variables used in this paper have the following names in the CMIP5 archive: evaporation (\\textit{evspsbl}), surface-air specific humidity (\\textit{huss}), surface-air temperature (\\textit{tas}), and surface-air relative humidity (\\textit{hurs}).}, and in each case the r1i1p1 ensemble member is used. As for the idealized GCM analysis, we assume moisture is well mixed in the boundary layer and take surface-air specific humidity to be representative of the boundary layer (using the average specific humidity between the surface and 900hPa gives similar results). \n\n\nThe specific humidities in the box model are identified with the zonal and time mean specific humidities (over land or ocean) for each latitude and for each of the (12) months of the year in the CMIP5 simulations.  We calculate $\\gamma$ as the ratio of the mean land and ocean specific humidities at each latitude and for each month of the year in the historical simulations. By computing $\\gamma$ in this way, we are assuming that the horizontal exchange of moisture between land and ocean, described by the box model, is taking place predominantly in the zonal direction. \nUsing the diagnosed $\\gamma$, and assuming it does not change as the climate warms, changes in mean surface-air land specific humidity are estimated for each latitude and each month of the year using (\\ref{eqn:box_model_3}) and the changes in mean ocean specific humidity. \n\n\n\n\\begin{figure}[t]\n  \\centering\n  \\noindent\\includegraphics[width=19pc,angle=0]{gamma_epsilon_zeta_vs_lat_both_theories_CMIP5.eps}\n  \\caption{Multimodel means in the CMIP5 simulations of the (a) $\\gamma$ parameter for the ocean-influence box model (solid black line) and for the regression approach including evapotranspiration (dashed black line), and (b) the regression coefficient $\\epsilon$. The parameter $\\gamma$ is evaluated based on the historical simulations for the ocean-influence box model, and $\\gamma$ and $\\epsilon$ are evaluated using equation \\ref{eqn:regression} for the regression approach.}\n\\label{fig:gamma_eps_zeta_vs_lat}\n\\end{figure}\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\nThe simulated and estimated annual mean changes in land specific humidity at each latitude are shown in Fig. \\ref{fig:delta_q_RH_CMIP5_sim_theory}a, and the magnitude and latitudinal variations of the changes are reasonably well captured by the ocean-influence box model, including the flat region in the Northern Hemisphere midlatitudes. The magnitude of the increases is underestimated at most latitudes which, as discussed in the case of the idealized GCM simulations could be partly due to increases in the parameter $\\lambda$ relating free-tropospheric specific humidity to ocean surface specific humidity, but other aspects of the ocean-influence box model such as neglecting the influence of evapotranspiration are also likely to play a role. The parameter $\\gamma$ (i.e. the ratio of land and ocean specific humidities) is shown in Fig.~\\ref{fig:gamma_eps_zeta_vs_lat}a. It has a global, annual and multimodel mean value of 0.74, which is somewhat larger than the\nvalue found in the idealized GCM simulations. This is not\nsurprising given that the land in the idealized simulations\nis a subtropical continent, which is generally drier relative\nto neighboring oceans than continents at lower or higher\nlatitudes. \n\n\n\nTogether with the simulated changes in monthly-mean surface-air land temperature, the estimated changes in specific humidity are used to estimate the land pseudo relative humidity changes. As for the idealized GCM analysis, it is necessary to compare pseudo-relative humidities because of the difficulty in converting time and zonal mean specific humidities estimated by the box model to relative humidities. The use of pseudo relative humidities also avoids the complication that different climate models use different saturation vapor pressure formulations. The changes in pseudo relative humidity are calculated for each month of the year before taking the annual mean for both the simulated changes and the changes estimated by the box model. The changes in pseudo relative humidity and model-outputted relative humidity are somewhat similar at lower latitudes but quite different at higher latitudes (cf. blue and black solid lines in Fig. \\ref{fig:delta_q_RH_CMIP5_sim_theory}b), where the differing computations of saturation vapor pressure over ice in the various models becomes important and there is larger variability. Nonetheless the pseudo relative humidity is a useful measure of subsaturation, and we will refer to pseudo relative humidity as relative humidity for simplicity.\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\nThe simulated changes in (pseudo) land relative humidity are quite well described by the ocean-influence box model in the Southern Hemisphere and at lower latitudes (Fig. \\ref{fig:delta_q_RH_CMIP5_sim_theory}b). The estimated and simulated global-mean land relative humidity changes in the various climate models are also correlated, with a correlation coefficient of 0.66. \nDue to the general underestimation of the specific humidity increases by the ocean-influence box model (Fig. \\ref{fig:delta_q_RH_CMIP5_sim_theory}a), the relative humidity decreases are overestimated (Fig. \\ref{fig:delta_q_RH_CMIP5_sim_theory}b), with a large discrepancy in the mid- to high-latitudes of the Northern Hemisphere. At these latitudes, there is more land than ocean and it is likely that changes in ocean specific humidity have a weak influence on the specific humidity in the interior of large continents, or that meridional moisture transports from ocean at other latitudes become more important. Changes in relative humidity in these inner continental regions are likely to be more strongly influenced by local evapotranspiration changes, and they could be influenced by\nshifts in the iceline or changes in soil moisture or vegetation, effects which are not considered in the ocean-influence box model. \n\n\n\n\n\n\n\n\n3.1 Application of ocean-influence box model to idealized GCM simulations\n\\subsection{Application of ocean-influence box model to idealized GCM simulations}\n\\label{sect:iGCM_analysis_theory_1}\n\nThe ocean-influence box model is first applied to idealized GCM simulations over a wide range of climates. The idealized GCM is similar to that of \\citet{frierson_2006} and \\citet{frierson_2007}, with specific details as in \\citet{byrne_ogorman_2013} and \\citet{ogorman_2008}. It is based on a spectral version of the GFDL dynamical core, with a two-stream gray radiation scheme, no cloud or water vapor radiative feedbacks, and the simplified moist convection scheme of \\citet{frierson_2007}. The simulations have a subtropical continent spanning $20^{\\circ}\\rm{N}$ to $40^{\\circ}\\rm{N}$ and $0^{\\circ}\\rm{E}$ to $120^{\\circ}\\rm{E}$, with a slab ocean elsewhere (Fig. \\ref{fig:continent_schematic}). The land surface hydrology is simulated using a simple bucket model \\citep{manabe_1969} and all other land surface properties are identical to those of the slab ocean. We vary the climate over a wide range of global-mean surface-air temperatures (between 260K and 317K) by changing the longwave optical thickness, which is analogous to varying the concentrations of $\\rm{CO}_2$ and other greenhouse gases. The longwave optical thickness is specified by $\\tau = \\alpha \\tau_{\\rm{ref}}$, where $\\tau_{\\rm{ref}}$ is a reference optical thickness distribution, and we analyze simulations with 10 different values of the parameter $\\alpha$\\footnote{Simulations are performed with the following $\\alpha$ values: 0.2, 0.4, 0.7, 1.0, 1.5, 2.0, 3.0, 4.0, 5.0, and 6.0.}. We present results based on time averages over 4000 days.\n\nWhen applying the box model to the simulations, we assume that the specific humidity is well-mixed in the boundary layer and use the surface-air specific humidities to represent the entire boundary layer. In the case of the idealized GCM, surface-air quantities are taken to be those of the lowest atmospheric level, $\\sigma = 0.989$, where $\\sigma=p/p_s$, and $p$ and $p_s$ are the pressure and surface pressure, respectively. The results are qualitatively similar when specific humidities are instead averaged between the surface and $\\sigma = 0.9$ (not shown). Land values are averaged (with area weighting) over the entire subtropical continent, and the ocean averages are taken over neighboring ocean at the same latitudes, i.e. from $20^{\\circ}\\rm{N}$ to $40^{\\circ}\\rm{N}$ and $120^{\\circ}\\rm{E}$ to $360^{\\circ}\\rm{E}$.\\footnote{\nOur results are almost identical if we calculate ocean averages using the ``control'' Southern Hemisphere as in \\citet{byrne_ogorman_2013}, i.e. ocean values averaged over $20^{\\circ}\\rm{S}$ to $40^{\\circ}\\rm{S}$ and $0^{\\circ}\\rm{E}$ to $120^{\\circ}\\rm{E}$. We choose to average over neighboring ocean in this study because the box model involves advection of moisture from ocean to land, and this naturally suggests averaging over ocean adjacent to the land continent.} \n\n\n\n\\begin{figure}[t]\n  \\centering\n  \\noindent\\includegraphics[width=19pc,angle=0]{gamma_and_epsilon_vs_T_ocean_nonlinear_new.eps}\n  \\caption{Parameters for the box models applied to the idealized GCM simulations. (a) The $\\gamma$ parameter for the ocean-influence box model (solid black line) and the full box model (dashed black line). (b) The $q_{\\rm{E }}$ parameter for the full box model (dashed black line) and the surface-air land specific humidity (red line) scaled by a factor of 0.25 so that is roughly matches the magnitude of $q_{\\rm{E}}$.}\n\\label{fig:gamma_and_epsilon}\n\\end{figure}\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\nTo apply the ocean-influence box model (\\ref{eqn:box_model_3}), we calculate the $\\gamma$ parameter (the ratio of land to ocean specific humidities) for each simulation (except the warmest). We then estimate the change in surface-air land specific humidity between pairs of nearest-neighbor simulations as a function of $\\gamma$ and the changes in ocean specific humidity, where $\\gamma$ is set to its value in the colder of the two simulations and assumed to be constant as the climate changes. \n\nLand surface-air specific humidity changes between the pairs of idealized GCM simulations, along with the estimates of these changes using (\\ref{eqn:box_model_3}), are plotted against the mid-point ocean temperature for each pair in Fig. \\ref{fig:delta_q_L_and_RH_L_theory_1}.  The simulated specific humidity changes are well-captured by the ocean-influence box model (\\ref{eqn:box_model_3}) over the full range of climates (Fig. \\ref{fig:delta_q_L_and_RH_L_theory_1}a).  \nThe increases in specific humidity, $\\delta q_{\\rm{L}}$, generally increase in magnitude as the climate warms (Fig. \\ref{fig:delta_q_L_and_RH_L_theory_1}a),\n\nand they are generally smaller than what would occur if land relative humidity remained constant (see the red line in Fig. \\ref{fig:delta_q_L_and_RH_L_theory_1}a). \nThe small deviations from the prediction of the ocean-influence box model (\\ref{eqn:box_model_3}) could be due to the influence of evapotranspiration, changes in circulation patterns, or changes in the ratio $\\lambda$ of free tropospheric to surface-air ocean specific humidity which is assumed to be constant in the box model. The parameter $\\lambda$ might be expected to increase with warming because the fractional rate of increase in saturation vapor pressure with temperature is higher at the lower temperatures that occur further up in the atmosphere, and\nbecause there is enhanced warming aloft at low latitudes in simulations of global warming \\citep[e.g.,][]{santer_et_al_2005},\nand such effects could be included in a more complicated box model. \n\nThe $\\gamma$ parameter is relatively constant over the wide range of climates simulated (Fig. \\ref{fig:gamma_and_epsilon}a) consistent with our neglect of changes in $\\gamma$ when deriving (\\ref{eqn:box_model_3}), with a mean value of 0.61 and minimum and maximum values of 0.56 and 0.68, respectively. Thus, for the subtropical continent in this idealized GCM, land specific humidity is approximately 60\\% of the neighboring ocean specific humidity.\n\nThe box model (\\ref{eqn:box_model_3}) predicts the changes in mean specific humidity which must be combined with the mean temperatures to estimate the relative humidity changes. However, because of the nonlinearity of the Clausius-Clapeyron relation, it is not possible to reproduce the mean relative humidity using the mean temperature and mean specific humidity. We instead use a pseudo relative humidity, defined in terms of the mean temperature, specific humidity, and pressure as $H(\\overline{T},\\overline{p},\\overline{q})$ where the bars denote time and spatial means, and $H(T,p,q)$ is the thermodynamic relationship between relative humidity, temperature, pressure and specific humidity. For convenience we will refer to this pseudo relative humidity as the relative humidity, but we also show the actual relative humidity changes for comparison in Figs. \\ref{fig:delta_q_L_and_RH_L_theory_1}b and \\ref{fig:delta_q_RH_CMIP5_sim_theory}b.\n\nThe box-model estimate of the relative humidity changes is somewhat less accurate than for specific humidity, but the decreases in relative humidity with warming and the decreasing magnitude of these changes as the climate warms are both captured (Fig. \\ref{fig:delta_q_L_and_RH_L_theory_1}b). \n\nGiven the simplicity of the ocean-influence box model, its ability to describe the behavior of land relative humidity in this idealized GCM is impressive. However, the geometry and surface properties of Earth's land masses are more varied and complex than the idealized continent considered, and factors such as orography or cloud feedbacks that are not included in the idealized GCM could alter the surface humidity response. Therefore, to investigate the changes in land relative humidity further, we turn to more comprehensive simulations from the CMIP5 archive. \n\n\n\n\\begin{figure}[t]\n  \\centering\n  \\noindent\\includegraphics[width=19pc,angle=0]{delta_SH_land_and_delta_RH_CMIP5_and_estimates_vs_lat.eps}\n  \\caption{Multimodel-mean changes between the historical and RCP8.5 simulations in zonal and time mean (a) surface-air land specific humidity and (b) surface-air land relative humidity normalized by the global-mean surface-air temperature change. Solid black lines denote the simulated changes and dashed lines denote the estimated changes using the box model (\\ref{eqn:box_model_3}). For (a), the red line indicates the change in surface-air land specific humidity for constant land pseudo relative humidity (i.e. for land pseudo relative humidity fixed at the values in the historical simulations). \nPseudo relative humidities are shown, but the blue line in (b) shows the simulated mean changes for the relative humidity variable outputted by the models for comparison.}\n\\label{fig:delta_q_RH_CMIP5_sim_theory}\n\\end{figure}\n\n4 Influence of evapotranspiration\n\\section{Influence of evapotranspiration}\n\\label{sect:full_model_RH_land}\n\n\n\n\nThe ocean-influence box model captures much (but not all) of the behavior in vastly more complex GCMs. However, the moisture balance of the land boundary layer is also affected by evapotranspiration, and changes in land surface properties, such as soil moisture or stomatal conductance, can affect evapotranspiration in the absence of any changes in the overlying atmosphere.  For example, changes in stomatal conductance under elevated $\\rm{CO}_2$ conditions has been shown to reduce both evapotranspiration and land relative humidity without changes in ocean humidity \\citep{andrews_et_al_2011}. \n\n\n\n\\begin{figure}[t]\n  \\centering\n  \\noindent\\includegraphics[width=19pc,angle=0]{delta_q_L_and_delta_RH_L_vs_T_ocean_iGCM_sim_and_theory_2_nonlinear_new.eps}\n  \\caption{As in Figure \\ref{fig:delta_q_L_and_RH_L_theory_1}, but here showing estimates of the surface-air specific and relative humidity changes from the full box model (\\ref{eqn:new_box_model_3}). The contributions due to ocean specific humidity changes (blue dashed lines) and evapotranspiration changes (green dashed lines) are also shown. The contributions to changes in relative humidity are calculated using (\\ref{eqn:appendix_3}). Pseudo relative humidities are shown in this figure (the changes in actual relative humidity are shown by the blue line in Fig.~\\ref{fig:delta_q_L_and_RH_L_theory_1}b.)}\n\\label{fig:delta_q_L_and_RH_L_theory_2}\n\\end{figure}\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\nWe turn to the full box model (\\ref{eqn:new_box_model_2}) which includes\nthe effects of evapotranspiration. We assume once more that changes in $\\gamma$ are negligible, \n\nsuch that:\n\n\\begin{equation}\n\\delta q_{\\rm{L}} \\approx \\gamma \\delta q_{\\rm{O}} + \\delta q_{\\rm{E}}.\n\\label{eqn:new_box_model_3}\n\\end{equation}\n\nThere are two terms contributing to changes in $q_{\\rm{L}}$ in (\\ref{eqn:new_box_model_3}): the term arising from changes in ocean specific humidity, $\\gamma \\delta q_{\\rm{O}}$, and an additional land evapotranspiration term, $\\delta q_{\\rm{E}}$. We next assess their relative importance in controlling changes in land humidity in the idealized GCM and CMIP5 simulations.\n\n\n\\subsection{Application of full box model to idealized GCM simulations}\n\\label{sect:theory_2_simulation_analysis}\n\nWe first examine the idealized GCM simulations with a subtropical continent. In contrast to the ocean-influence box model (\\ref{eqn:box_model_3}), for which the single parameter $\\gamma$ could be easily estimated in the control simulation in each case, the full model (\\ref{eqn:new_box_model_3}) has two parameters to be estimated, $\\gamma$ and $q_{\\rm{E}}$. To estimate these parameters, we perform an additional set of simulations with the same longwave optical thicknesses as in the 10 simulations described previously but with the evapotranspiration set to zero over land. \n\nSpecifying the land evapotranspiration in this way is analogous to drying out the soil. (Note that the change in evapotranspiration affects both the humidity of the atmosphere and the surface energy balance.)\nUsing these additional simulations with $E_{\\rm{L}} = 0$, we can estimate $\\gamma$ for each climate using (\\ref{eqn:new_box_model_2}): $\\gamma = q_{\\rm{L},E_{\\rm{L}}=0} / q_{\\rm{O},E_{\\rm{L}}=0}$.  The $\\gamma$ values obtained are smaller than those calculated from the control climate for the ocean-influence box model (Fig. \\ref{fig:gamma_and_epsilon}a) because the contribution of evapotranspiration to the land specific humidity is now also taken into account.  We then use these $\\gamma$ values to estimate $q_{\\rm{E}}$ for the original simulations with dynamic land surface hydrology \n: $q_{\\rm{E}} = q_{\\rm{L}} - \\gamma q_{\\rm{O}}$. The values of $q_{\\rm{E}}$ increase with warming except in hot climates (Fig. \\ref{fig:gamma_and_epsilon}b).\n\nInterestingly, the influence of evapotranspiration on land specific humidity as measured by $q_{\\rm{E}}$ roughly scales with the land specific humidity except in hot climates (compare the dashed black and solid red lines in Fig. \\ref{fig:gamma_and_epsilon}b), and this helps to explain why the ocean-influence box model\nis accurate even though evapotranspiration affects land specific humidity.\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\nWe then estimate the changes in land specific humidity between pairs of nearest-neighbor simulations from (\\ref{eqn:new_box_model_3}) with $\\gamma$ assumed to be constant as the climate changes.  The simulated and estimated changes in surface-air land specific humidity, along with the contributions due to changes in ocean specific humidity and land evapotranspiration are shown in Fig. \\ref{fig:delta_q_L_and_RH_L_theory_2}a. The full box model captures the basic behavior of the land specific humidity changes as a function of temperature, although it is less accurate in hot climates. The contribution from ocean specific humidity changes, $\\gamma \\delta q_{\\rm{O}}$, is larger than the contribution from land evapotranspiration for all climates (Fig. \\ref{fig:delta_q_L_and_RH_L_theory_2}a).  The changes in simulated land (pseudo) relative humidity are also well captured by the full box model (Fig. \\ref{fig:delta_q_L_and_RH_L_theory_2}b). \n\n\n\n\n\n\nBecause relative humidity depends on temperature as well as specific humidity, \nthere is no unique way to use the box model result (\\ref{eqn:new_box_model_3}) to decompose changes in land relative humidity into contributions due to ocean specific humidity and land evapotranspiration. However, a decomposition derived in appendix A (equation \\ref{eqn:appendix_3}) has several desirable properties. According to the decomposition, the contributions to the change in land relative humidity from evapotranspiration and ocean specific humidity are weighted according to their contribution to land specific humidity in the control climate. The change in ocean specific humidity leads to a decrease in land relative humidity if the fractional increase in ocean specific humidity is less than the fractional increase in saturation specific humidity over land. Similarly, evapotranspiration contributes to a decrease in land relative humidity if the fractional increase in $q_{\\rm{E}}$ is less than the fractional increase in saturation specific humidity over land.  (Note that the fractional change in $q_{\\rm{E}}$ is generally different from the fractional change in evapotranspiration.)  \n\nUsing this decomposition of the change in land relative humidity, we find that the land evapotranspiration contribution is of comparable importance to the ocean specific humidity contribution  for the idealized GCM simulations\n(Fig. \\ref{fig:delta_q_L_and_RH_L_theory_2}b). By contrast, we found that the\ncontribution of ocean specific humidity was more important than land evapotranspiration when land specific humidity changes were considered.  The discrepancy arises because, according to the decomposition (\\ref{eqn:appendix_3}), it is not the magnitude of a particular contribution to the change in specific humidity that matters for its contribution to the change in relative humidity, but rather how its fractional changes compare to the fractional changes in saturation specific humidity and how much it contributes to the land specific humidity in the control climate. \n\n\n\n\\subsection{Influence of evapotranspiration in CMIP5 simulations}\n\nWe now investigate how land evapotranspiration contributes to specific humidity changes in the CMIP5 simulations. \n\nWe need to estimate both $\\gamma$ and $q_{\\rm{E}}$ for the full box model, but there are no CMIP5 simulations analogous to the zero-evapotranspiration simulations with the idealized GCM described above. Instead, we estimate the influence of evapotranspiration and ocean specific humidity on land specific humidity using a multiple linear regression approach based on the intermodel scatter across the CMIP5 models. We use the regression relationship\n\n\\begin{equation}\n\\delta q_{\\rm{L}} = \\gamma \\delta q_{\\rm{O}} + \\epsilon \\delta E_{\\rm{L}} + \\zeta, \n\\label{eqn:regression}\n\\end{equation}\n\nwhich is motivated by the full box model (\\ref{eqn:new_box_model_2}), but note that $\\epsilon \\delta E_{\\rm{L}}$ will not generally equal $\\delta q_{\\rm{E}}$ to the extent that parameters such as $\\tau_1$ and $\\tau_2$ change with climate, and changes in these parameters will contribute to the remainder term $\\zeta$.  The variables $\\delta q_{\\rm{L}},\\delta q_{\\rm{O}}$ and  $\\delta E_{\\rm{L}}$ are identified as the zonal and time mean for each latitude and month of the year in each model.  The regression coefficients $\\gamma$, $\\epsilon$, and $\\zeta$ are then estimated using ordinary least squares regression for each latitude and month of the year. The annual means are shown for $\\gamma$ and $\\epsilon$ in Fig.~\\ref{fig:gamma_eps_zeta_vs_lat} and for $\\zeta$ in Fig.~\\ref{fig:delta_SH_land_CMIP5_theory_2}. The regression coefficient $\\gamma$ has a similar magnitude and latitudinal structure to the $\\gamma$ parameter calculated for the ocean-influence box model (Fig.~\\ref{fig:gamma_eps_zeta_vs_lat}a). The coefficient $\\epsilon$ is positive at all latitudes (Fig.~\\ref{fig:gamma_eps_zeta_vs_lat}b) indicating that enhanced evapotranspiration increases the land specific humidity, while the remainder term $\\zeta$ (Fig.~\\ref{fig:delta_SH_land_CMIP5_theory_2}) is negative at most latitudes. \n\n\n\n\\begin{figure}[t]\n  \\centering\n\n   \\noindent\\includegraphics[width=19pc,angle=0]{delta_SH_land_and_estimates_vs_lat_theory_2_v4.eps}\n  \\caption{Multimodel-mean changes in zonal- and time-mean surface-air land specific humidity (solid black line) in the CMIP5 simulations, and the contributions to these changes due to ocean specific humidity changes ($\\gamma \\delta q_{\\rm{O}}$; blue dashed line), land evapotranspiration changes ($\\epsilon \\delta E_{\\rm{L}}$; green dashed line), and the remainder term ($\\zeta$; red dashed line) as estimated using the regression relation (\\ref{eqn:regression}).}\n\\label{fig:delta_SH_land_CMIP5_theory_2}\n\\end{figure}\n\n\n\n\n\n\n\n\n\n\n\n\nBy construction, the regression relationship (\\ref{eqn:regression}) is exactly satisfied in the multimodel mean. Based on this relationship, the annual-mean contributions to changes in land specific humidity from changes in ocean specific humidity, changes in land evapotranspiration, and the remainder term are shown in Fig. \\ref{fig:delta_SH_land_CMIP5_theory_2}. At all latitudes, changes in land specific humidity are dominated by the ocean specific humidity contribution. The contribution due to changes in land evapotranspiration is positive and has its largest values in the Northern Hemisphere where the land fraction is greatest. \n\nIt is not possible to estimate the individual contributions to changes in land relative humidity for the CMIP5 simulations, as we did for the idealized GCM simulations. This is because the decomposition of relative humidity changes discussed in appendix A involves the individual contributions to land specific humidity in the control climate, and these are difficult to calculate using a regression approach. However, the results from the idealized GCM simulations suggest that evapotranspiration could be important for the changes in land relative humidity in the CMIP5 simulations, even though it is a second order influence for changes in land specific humidity. It would be worthwhile to estimate the land evapotranspiration contribution for full-complexity GCMs by performing simulations with specified land evapotranspiration rates as was done for the idealized GCM in this study.\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n4.1 Application of full box model to idealized GCM simulations\n\\subsection{Application of full box model to idealized GCM simulations}\n\\label{sect:theory_2_simulation_analysis}\n\nWe first examine the idealized GCM simulations with a subtropical continent. In contrast to the ocean-influence box model (\\ref{eqn:box_model_3}), for which the single parameter $\\gamma$ could be easily estimated in the control simulation in each case, the full model (\\ref{eqn:new_box_model_3}) has two parameters to be estimated, $\\gamma$ and $q_{\\rm{E}}$. To estimate these parameters, we perform an additional set of simulations with the same longwave optical thicknesses as in the 10 simulations described previously but with the evapotranspiration set to zero over land. \n\nSpecifying the land evapotranspiration in this way is analogous to drying out the soil. (Note that the change in evapotranspiration affects both the humidity of the atmosphere and the surface energy balance.)\nUsing these additional simulations with $E_{\\rm{L}} = 0$, we can estimate $\\gamma$ for each climate using (\\ref{eqn:new_box_model_2}): $\\gamma = q_{\\rm{L},E_{\\rm{L}}=0} / q_{\\rm{O},E_{\\rm{L}}=0}$.  The $\\gamma$ values obtained are smaller than those calculated from the control climate for the ocean-influence box model (Fig. \\ref{fig:gamma_and_epsilon}a) because the contribution of evapotranspiration to the land specific humidity is now also taken into account.  We then use these $\\gamma$ values to estimate $q_{\\rm{E}}$ for the original simulations with dynamic land surface hydrology \n: $q_{\\rm{E}} = q_{\\rm{L}} - \\gamma q_{\\rm{O}}$. The values of $q_{\\rm{E}}$ increase with warming except in hot climates (Fig. \\ref{fig:gamma_and_epsilon}b).\n\nInterestingly, the influence of evapotranspiration on land specific humidity as measured by $q_{\\rm{E}}$ roughly scales with the land specific humidity except in hot climates (compare the dashed black and solid red lines in Fig. \\ref{fig:gamma_and_epsilon}b), and this helps to explain why the ocean-influence box model\nis accurate even though evapotranspiration affects land specific humidity.\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\nWe then estimate the changes in land specific humidity between pairs of nearest-neighbor simulations from (\\ref{eqn:new_box_model_3}) with $\\gamma$ assumed to be constant as the climate changes.  The simulated and estimated changes in surface-air land specific humidity, along with the contributions due to changes in ocean specific humidity and land evapotranspiration are shown in Fig. \\ref{fig:delta_q_L_and_RH_L_theory_2}a. The full box model captures the basic behavior of the land specific humidity changes as a function of temperature, although it is less accurate in hot climates. The contribution from ocean specific humidity changes, $\\gamma \\delta q_{\\rm{O}}$, is larger than the contribution from land evapotranspiration for all climates (Fig. \\ref{fig:delta_q_L_and_RH_L_theory_2}a).  The changes in simulated land (pseudo) relative humidity are also well captured by the full box model (Fig. \\ref{fig:delta_q_L_and_RH_L_theory_2}b). \n\n\n\n\n\n\nBecause relative humidity depends on temperature as well as specific humidity, \nthere is no unique way to use the box model result (\\ref{eqn:new_box_model_3}) to decompose changes in land relative humidity into contributions due to ocean specific humidity and land evapotranspiration. However, a decomposition derived in appendix A (equation \\ref{eqn:appendix_3}) has several desirable properties. According to the decomposition, the contributions to the change in land relative humidity from evapotranspiration and ocean specific humidity are weighted according to their contribution to land specific humidity in the control climate. The change in ocean specific humidity leads to a decrease in land relative humidity if the fractional increase in ocean specific humidity is less than the fractional increase in saturation specific humidity over land. Similarly, evapotranspiration contributes to a decrease in land relative humidity if the fractional increase in $q_{\\rm{E}}$ is less than the fractional increase in saturation specific humidity over land.  (Note that the fractional change in $q_{\\rm{E}}$ is generally different from the fractional change in evapotranspiration.)  \n\nUsing this decomposition of the change in land relative humidity, we find that the land evapotranspiration contribution is of comparable importance to the ocean specific humidity contribution  for the idealized GCM simulations\n(Fig. \\ref{fig:delta_q_L_and_RH_L_theory_2}b). By contrast, we found that the\ncontribution of ocean specific humidity was more important than land evapotranspiration when land specific humidity changes were considered.  The discrepancy arises because, according to the decomposition (\\ref{eqn:appendix_3}), it is not the magnitude of a particular contribution to the change in specific humidity that matters for its contribution to the change in relative humidity, but rather how its fractional changes compare to the fractional changes in saturation specific humidity and how much it contributes to the land specific humidity in the control climate. \n\n\n\n",
        "label": "fig:gamma_and_epsilon",
        "Descriptive_question1": "What is the mean value of the gamma parameter for the ocean-influence box model in figure_5?",
        "Descriptive_question2": "What color is used for the surface-air land specific humidity line scaled by a factor of 0.25 in figure_5?",
        "Reasoning_question1": "Why might the gamma parameter for the ocean-influence box model remain relatively constant across different climates in figure_5?",
        "Reasoning_question2": "How does the relationship between the q_E parameter and the scaled surface-air land specific humidity in figure_5 help explain the accuracy of the ocean-influence box model despite the influence of evapotranspiration?",
        "Descriptive_answer1": "0.61",
        "Descriptive_answer2": "red",
        "Reasoning_answer1": "The gamma parameter represents the ratio of land to ocean specific humidities and is related to mixing time scales in the atmospheric boundary layer. If climate change causes the tropical circulation and convective fluxes to slow down uniformly, increasing mixing timescales by the same factor, gamma remains approximately constant. Thus, gamma's constancy across climates in Figure 5 suggests consistent relative moisture exchange dynamics between ocean and land, supporting the underlying model assumption of negligible changes in gamma.",
        "Reasoning_answer2": "In Figure 5, the q_E parameter (dashed black line) roughly scales with the land specific humidity scaled by 0.25 (red line), indicating that evapotranspiration contributes proportionally to the land moisture content. This proportional scaling means that evapotranspiration effects do not cause large deviations in land specific humidity relative to the ocean. Therefore, the ocean-influence box model remains accurate despite evapotranspiration influences because evapotranspiration's contribution scales with land humidity and is implicitly captured within the overall moisture balance."
    },
    {
        "paper_id": "1605.00380.json",
        "image_id": "figure_6",
        "image_path": [
            "/home/yz979/scratch/chengye/scimolmo/test-images-tables/dataset_imagestables/1605.00380/images/delta_SH_land_and_delta_RH_CMIP5_and_estimates_vs_lat.eps"
        ],
        "caption": "Multimodel-mean changes between the historical and RCP8.5 simulations in zonal and time mean (a) surface-air land specific humidity and (b) surface-air land relative humidity normalized by the global-mean surface-air temperature change. Solid black lines denote the simulated changes and dashed lines denote the estimated changes using the box model (\\ref{eqn:box_model_3}). For (a), the red line indicates the change in surface-air land specific humidity for constant land pseudo relative humidity (i.e. for land pseudo relative humidity fixed at the values in the historical simulations). \nPseudo relative humidities are shown, but the blue line in (b) shows the simulated mean changes for the relative humidity variable outputted by the models for comparison.",
        "classify": "Chart",
        "section_info": "3 Ocean-influence box model\n\\section{Ocean-influence box model}\n\nFor the simplest version of our box model, the ``ocean-influence box model'', we assume that the influence of evapotranspiration on the boundary-layer moisture balance over land is negligible. Setting $E_{\\rm{L}}=0$ in (\\ref{eqn:new_box_model_2}), we find:\n\n\\begin{equation}\nq_{\\rm{L}} \\approx \\gamma q_{\\rm{O}}.\n\\label{eqn:box_model_2}\n\\end{equation}\n\nThe parameter $\\gamma$ may remain approximately constant under climate change even if there are changes in the mixing time scales $\\tau_1$ and $\\tau_2$. For example, if the overall tropical circulation and convective mass fluxes slow down with climate warming \\citep[e.g.,][]{held_soden_2006,vecchi_soden_2007} such that both mixing time scales increase by the same factor, then this will not cause $\\gamma$ to change. Assuming negligible changes in $\\gamma$, we can write:\n\n\\begin{equation}\n\\delta q_{\\rm{L}} \\approx \\gamma \\delta q_{\\rm{O}},\n\\label{eqn:box_model_3}\n\\end{equation}\n\nwhere $\\delta$ denotes the change in climate.\n\nThe same results would also follow (with a different definition of $\\gamma$) if the influence of evapotranspiration on land specific humidity, $q_{\\rm{E}}$, is not neglected but is instead assumed to scale with land specific humidity. \nNote that $q_{\\rm{E}}$ scaling with land specific humidity is not\nthe same as the evapotranspiration rate scaling with land specific humidity.\nIn particular, $q_E$ depends on  \n$(\\tau_1 \\tau_2)/(\\tau_1 + \\tau_2)$, and this factor would change even if\nboth $\\tau_1$ and $\\tau_2$ increase by the same factor.\n\nThe ocean-influence box model suggests a straightforward hypothesis - that the ratio of land to ocean specific humidity remains approximately constant as the climate changes or, equivalently, that fractional changes in specific humidity over land and ocean are equal: \n\n\\begin{equation}\n\\frac{\\delta q_{\\rm{L}}}{q_{\\rm{L}}} \\approx \\frac{\\delta q_{\\rm{O}}}{q_{\\rm{O}}}.\n\\label{eqn:constant_q_ratio}\n\\end{equation}\n\nBy contrast the fractional changes in saturation specific humidity depend on the local temperature change and will be bigger over land than ocean.\nTo the extent that (\\ref{eqn:constant_q_ratio}) holds, it is clear that if land warms more than ocean, and ocean relative humidity does not change greatly, then the land relative humidity will decrease.\n\nWe now assess the applicability of this ocean-influence box model result to idealized and comprehensive GCM simulations. \nWe use (\\ref{eqn:box_model_3}) to estimate the change in land relative humidity under climate change given the changes in land temperature\nand ocean specific humidity, and calculating $\\gamma$ as the the ratio of land to ocean specific humidities in the control climate.  \n\n\n\n\\begin{figure}[t]\n  \\centering\n  \\noindent\\includegraphics[width=19pc,angle=0]{delta_q_L_and_delta_RH_L_vs_T_ocean_iGCM_sim_and_theory_1.eps}\n  \\caption{Changes over land in (a) surface-air specific humidity and (b) surface-air relative humidity between pairs of idealized GCM simulations with a subtropical continent.  The relative humidity changes are \nnormalized by the land surface-air temperature change.\nSolid black lines denote the simulated changes and the dashed lines represent the estimated changes using the ocean-influence box model (\\ref{eqn:box_model_3}). Pseudo relative humidities are shown (see text), but the blue line in (b) shows the change in the mean of the actual relative humidity for comparison. The red line in (a) indicates what the change in surface-air land specific humidity would be if the land pseudo relative humidity did not change (i.e. for each pair of simulations, the land specific humidity change if the land pseudo relative humidity is fixed at its value in the colder simulation).}\n\\label{fig:delta_q_L_and_RH_L_theory_1}\n\\end{figure}\n\n\n\n\n\\subsection{Application of ocean-influence box model to idealized GCM simulations}\n\\label{sect:iGCM_analysis_theory_1}\n\nThe ocean-influence box model is first applied to idealized GCM simulations over a wide range of climates. The idealized GCM is similar to that of \\citet{frierson_2006} and \\citet{frierson_2007}, with specific details as in \\citet{byrne_ogorman_2013} and \\citet{ogorman_2008}. It is based on a spectral version of the GFDL dynamical core, with a two-stream gray radiation scheme, no cloud or water vapor radiative feedbacks, and the simplified moist convection scheme of \\citet{frierson_2007}. The simulations have a subtropical continent spanning $20^{\\circ}\\rm{N}$ to $40^{\\circ}\\rm{N}$ and $0^{\\circ}\\rm{E}$ to $120^{\\circ}\\rm{E}$, with a slab ocean elsewhere (Fig. \\ref{fig:continent_schematic}). The land surface hydrology is simulated using a simple bucket model \\citep{manabe_1969} and all other land surface properties are identical to those of the slab ocean. We vary the climate over a wide range of global-mean surface-air temperatures (between 260K and 317K) by changing the longwave optical thickness, which is analogous to varying the concentrations of $\\rm{CO}_2$ and other greenhouse gases. The longwave optical thickness is specified by $\\tau = \\alpha \\tau_{\\rm{ref}}$, where $\\tau_{\\rm{ref}}$ is a reference optical thickness distribution, and we analyze simulations with 10 different values of the parameter $\\alpha$\\footnote{Simulations are performed with the following $\\alpha$ values: 0.2, 0.4, 0.7, 1.0, 1.5, 2.0, 3.0, 4.0, 5.0, and 6.0.}. We present results based on time averages over 4000 days.\n\nWhen applying the box model to the simulations, we assume that the specific humidity is well-mixed in the boundary layer and use the surface-air specific humidities to represent the entire boundary layer. In the case of the idealized GCM, surface-air quantities are taken to be those of the lowest atmospheric level, $\\sigma = 0.989$, where $\\sigma=p/p_s$, and $p$ and $p_s$ are the pressure and surface pressure, respectively. The results are qualitatively similar when specific humidities are instead averaged between the surface and $\\sigma = 0.9$ (not shown). Land values are averaged (with area weighting) over the entire subtropical continent, and the ocean averages are taken over neighboring ocean at the same latitudes, i.e. from $20^{\\circ}\\rm{N}$ to $40^{\\circ}\\rm{N}$ and $120^{\\circ}\\rm{E}$ to $360^{\\circ}\\rm{E}$.\\footnote{\nOur results are almost identical if we calculate ocean averages using the ``control'' Southern Hemisphere as in \\citet{byrne_ogorman_2013}, i.e. ocean values averaged over $20^{\\circ}\\rm{S}$ to $40^{\\circ}\\rm{S}$ and $0^{\\circ}\\rm{E}$ to $120^{\\circ}\\rm{E}$. We choose to average over neighboring ocean in this study because the box model involves advection of moisture from ocean to land, and this naturally suggests averaging over ocean adjacent to the land continent.} \n\n\n\n\\begin{figure}[t]\n  \\centering\n  \\noindent\\includegraphics[width=19pc,angle=0]{gamma_and_epsilon_vs_T_ocean_nonlinear_new.eps}\n  \\caption{Parameters for the box models applied to the idealized GCM simulations. (a) The $\\gamma$ parameter for the ocean-influence box model (solid black line) and the full box model (dashed black line). (b) The $q_{\\rm{E }}$ parameter for the full box model (dashed black line) and the surface-air land specific humidity (red line) scaled by a factor of 0.25 so that is roughly matches the magnitude of $q_{\\rm{E}}$.}\n\\label{fig:gamma_and_epsilon}\n\\end{figure}\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\nTo apply the ocean-influence box model (\\ref{eqn:box_model_3}), we calculate the $\\gamma$ parameter (the ratio of land to ocean specific humidities) for each simulation (except the warmest). We then estimate the change in surface-air land specific humidity between pairs of nearest-neighbor simulations as a function of $\\gamma$ and the changes in ocean specific humidity, where $\\gamma$ is set to its value in the colder of the two simulations and assumed to be constant as the climate changes. \n\nLand surface-air specific humidity changes between the pairs of idealized GCM simulations, along with the estimates of these changes using (\\ref{eqn:box_model_3}), are plotted against the mid-point ocean temperature for each pair in Fig. \\ref{fig:delta_q_L_and_RH_L_theory_1}.  The simulated specific humidity changes are well-captured by the ocean-influence box model (\\ref{eqn:box_model_3}) over the full range of climates (Fig. \\ref{fig:delta_q_L_and_RH_L_theory_1}a).  \nThe increases in specific humidity, $\\delta q_{\\rm{L}}$, generally increase in magnitude as the climate warms (Fig. \\ref{fig:delta_q_L_and_RH_L_theory_1}a),\n\nand they are generally smaller than what would occur if land relative humidity remained constant (see the red line in Fig. \\ref{fig:delta_q_L_and_RH_L_theory_1}a). \nThe small deviations from the prediction of the ocean-influence box model (\\ref{eqn:box_model_3}) could be due to the influence of evapotranspiration, changes in circulation patterns, or changes in the ratio $\\lambda$ of free tropospheric to surface-air ocean specific humidity which is assumed to be constant in the box model. The parameter $\\lambda$ might be expected to increase with warming because the fractional rate of increase in saturation vapor pressure with temperature is higher at the lower temperatures that occur further up in the atmosphere, and\nbecause there is enhanced warming aloft at low latitudes in simulations of global warming \\citep[e.g.,][]{santer_et_al_2005},\nand such effects could be included in a more complicated box model. \n\nThe $\\gamma$ parameter is relatively constant over the wide range of climates simulated (Fig. \\ref{fig:gamma_and_epsilon}a) consistent with our neglect of changes in $\\gamma$ when deriving (\\ref{eqn:box_model_3}), with a mean value of 0.61 and minimum and maximum values of 0.56 and 0.68, respectively. Thus, for the subtropical continent in this idealized GCM, land specific humidity is approximately 60\\% of the neighboring ocean specific humidity.\n\nThe box model (\\ref{eqn:box_model_3}) predicts the changes in mean specific humidity which must be combined with the mean temperatures to estimate the relative humidity changes. However, because of the nonlinearity of the Clausius-Clapeyron relation, it is not possible to reproduce the mean relative humidity using the mean temperature and mean specific humidity. We instead use a pseudo relative humidity, defined in terms of the mean temperature, specific humidity, and pressure as $H(\\overline{T},\\overline{p},\\overline{q})$ where the bars denote time and spatial means, and $H(T,p,q)$ is the thermodynamic relationship between relative humidity, temperature, pressure and specific humidity. For convenience we will refer to this pseudo relative humidity as the relative humidity, but we also show the actual relative humidity changes for comparison in Figs. \\ref{fig:delta_q_L_and_RH_L_theory_1}b and \\ref{fig:delta_q_RH_CMIP5_sim_theory}b.\n\nThe box-model estimate of the relative humidity changes is somewhat less accurate than for specific humidity, but the decreases in relative humidity with warming and the decreasing magnitude of these changes as the climate warms are both captured (Fig. \\ref{fig:delta_q_L_and_RH_L_theory_1}b). \n\nGiven the simplicity of the ocean-influence box model, its ability to describe the behavior of land relative humidity in this idealized GCM is impressive. However, the geometry and surface properties of Earth's land masses are more varied and complex than the idealized continent considered, and factors such as orography or cloud feedbacks that are not included in the idealized GCM could alter the surface humidity response. Therefore, to investigate the changes in land relative humidity further, we turn to more comprehensive simulations from the CMIP5 archive. \n\n\n\n\\begin{figure}[t]\n  \\centering\n  \\noindent\\includegraphics[width=19pc,angle=0]{delta_SH_land_and_delta_RH_CMIP5_and_estimates_vs_lat.eps}\n  \\caption{Multimodel-mean changes between the historical and RCP8.5 simulations in zonal and time mean (a) surface-air land specific humidity and (b) surface-air land relative humidity normalized by the global-mean surface-air temperature change. Solid black lines denote the simulated changes and dashed lines denote the estimated changes using the box model (\\ref{eqn:box_model_3}). For (a), the red line indicates the change in surface-air land specific humidity for constant land pseudo relative humidity (i.e. for land pseudo relative humidity fixed at the values in the historical simulations). \nPseudo relative humidities are shown, but the blue line in (b) shows the simulated mean changes for the relative humidity variable outputted by the models for comparison.}\n\\label{fig:delta_q_RH_CMIP5_sim_theory}\n\\end{figure}\n\n\\subsection{Application of ocean-influence box model to CMIP5 simulations}\n\\label{sect:CMIP5_analysis_theory_1}\n\nWe apply the ocean-influence box model to changes in land surface-air relative humidity between 30 year time averages in the historical (1976-2005) and RCP8.5 (2070-2099) simulations from the CMIP5 archive \\citep{taylor_et_al_bams_2012}. We analyze 19 models in total\\footnote{The CMIP5 models considered are: ACCESS1-0, ACCESS1-3, BCC-CSM1-1, BCC-CSM1-1-M, BNU-ESM, CanESM2, CNRM-CM5, CSIRO-Mk3-6-0, GFDL-CM3, GFDL-ESM2M, INMCM4, IPSL-CM5A-LR, IPSL-CM5A-MR, IPSL-CM5B-LR, MIROC-ESM, MIROC-ESM-CHEM, MIROC5, MRI-CGCM3, and NorESM1-M. The variables used in this paper have the following names in the CMIP5 archive: evaporation (\\textit{evspsbl}), surface-air specific humidity (\\textit{huss}), surface-air temperature (\\textit{tas}), and surface-air relative humidity (\\textit{hurs}).}, and in each case the r1i1p1 ensemble member is used. As for the idealized GCM analysis, we assume moisture is well mixed in the boundary layer and take surface-air specific humidity to be representative of the boundary layer (using the average specific humidity between the surface and 900hPa gives similar results). \n\n\nThe specific humidities in the box model are identified with the zonal and time mean specific humidities (over land or ocean) for each latitude and for each of the (12) months of the year in the CMIP5 simulations.  We calculate $\\gamma$ as the ratio of the mean land and ocean specific humidities at each latitude and for each month of the year in the historical simulations. By computing $\\gamma$ in this way, we are assuming that the horizontal exchange of moisture between land and ocean, described by the box model, is taking place predominantly in the zonal direction. \nUsing the diagnosed $\\gamma$, and assuming it does not change as the climate warms, changes in mean surface-air land specific humidity are estimated for each latitude and each month of the year using (\\ref{eqn:box_model_3}) and the changes in mean ocean specific humidity. \n\n\n\n\\begin{figure}[t]\n  \\centering\n  \\noindent\\includegraphics[width=19pc,angle=0]{gamma_epsilon_zeta_vs_lat_both_theories_CMIP5.eps}\n  \\caption{Multimodel means in the CMIP5 simulations of the (a) $\\gamma$ parameter for the ocean-influence box model (solid black line) and for the regression approach including evapotranspiration (dashed black line), and (b) the regression coefficient $\\epsilon$. The parameter $\\gamma$ is evaluated based on the historical simulations for the ocean-influence box model, and $\\gamma$ and $\\epsilon$ are evaluated using equation \\ref{eqn:regression} for the regression approach.}\n\\label{fig:gamma_eps_zeta_vs_lat}\n\\end{figure}\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\nThe simulated and estimated annual mean changes in land specific humidity at each latitude are shown in Fig. \\ref{fig:delta_q_RH_CMIP5_sim_theory}a, and the magnitude and latitudinal variations of the changes are reasonably well captured by the ocean-influence box model, including the flat region in the Northern Hemisphere midlatitudes. The magnitude of the increases is underestimated at most latitudes which, as discussed in the case of the idealized GCM simulations could be partly due to increases in the parameter $\\lambda$ relating free-tropospheric specific humidity to ocean surface specific humidity, but other aspects of the ocean-influence box model such as neglecting the influence of evapotranspiration are also likely to play a role. The parameter $\\gamma$ (i.e. the ratio of land and ocean specific humidities) is shown in Fig.~\\ref{fig:gamma_eps_zeta_vs_lat}a. It has a global, annual and multimodel mean value of 0.74, which is somewhat larger than the\nvalue found in the idealized GCM simulations. This is not\nsurprising given that the land in the idealized simulations\nis a subtropical continent, which is generally drier relative\nto neighboring oceans than continents at lower or higher\nlatitudes. \n\n\n\nTogether with the simulated changes in monthly-mean surface-air land temperature, the estimated changes in specific humidity are used to estimate the land pseudo relative humidity changes. As for the idealized GCM analysis, it is necessary to compare pseudo-relative humidities because of the difficulty in converting time and zonal mean specific humidities estimated by the box model to relative humidities. The use of pseudo relative humidities also avoids the complication that different climate models use different saturation vapor pressure formulations. The changes in pseudo relative humidity are calculated for each month of the year before taking the annual mean for both the simulated changes and the changes estimated by the box model. The changes in pseudo relative humidity and model-outputted relative humidity are somewhat similar at lower latitudes but quite different at higher latitudes (cf. blue and black solid lines in Fig. \\ref{fig:delta_q_RH_CMIP5_sim_theory}b), where the differing computations of saturation vapor pressure over ice in the various models becomes important and there is larger variability. Nonetheless the pseudo relative humidity is a useful measure of subsaturation, and we will refer to pseudo relative humidity as relative humidity for simplicity.\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\nThe simulated changes in (pseudo) land relative humidity are quite well described by the ocean-influence box model in the Southern Hemisphere and at lower latitudes (Fig. \\ref{fig:delta_q_RH_CMIP5_sim_theory}b). The estimated and simulated global-mean land relative humidity changes in the various climate models are also correlated, with a correlation coefficient of 0.66. \nDue to the general underestimation of the specific humidity increases by the ocean-influence box model (Fig. \\ref{fig:delta_q_RH_CMIP5_sim_theory}a), the relative humidity decreases are overestimated (Fig. \\ref{fig:delta_q_RH_CMIP5_sim_theory}b), with a large discrepancy in the mid- to high-latitudes of the Northern Hemisphere. At these latitudes, there is more land than ocean and it is likely that changes in ocean specific humidity have a weak influence on the specific humidity in the interior of large continents, or that meridional moisture transports from ocean at other latitudes become more important. Changes in relative humidity in these inner continental regions are likely to be more strongly influenced by local evapotranspiration changes, and they could be influenced by\nshifts in the iceline or changes in soil moisture or vegetation, effects which are not considered in the ocean-influence box model. \n\n\n\n\n\n\n\n\n3.1 Application of ocean-influence box model to idealized GCM simulations\n\\subsection{Application of ocean-influence box model to idealized GCM simulations}\n\\label{sect:iGCM_analysis_theory_1}\n\nThe ocean-influence box model is first applied to idealized GCM simulations over a wide range of climates. The idealized GCM is similar to that of \\citet{frierson_2006} and \\citet{frierson_2007}, with specific details as in \\citet{byrne_ogorman_2013} and \\citet{ogorman_2008}. It is based on a spectral version of the GFDL dynamical core, with a two-stream gray radiation scheme, no cloud or water vapor radiative feedbacks, and the simplified moist convection scheme of \\citet{frierson_2007}. The simulations have a subtropical continent spanning $20^{\\circ}\\rm{N}$ to $40^{\\circ}\\rm{N}$ and $0^{\\circ}\\rm{E}$ to $120^{\\circ}\\rm{E}$, with a slab ocean elsewhere (Fig. \\ref{fig:continent_schematic}). The land surface hydrology is simulated using a simple bucket model \\citep{manabe_1969} and all other land surface properties are identical to those of the slab ocean. We vary the climate over a wide range of global-mean surface-air temperatures (between 260K and 317K) by changing the longwave optical thickness, which is analogous to varying the concentrations of $\\rm{CO}_2$ and other greenhouse gases. The longwave optical thickness is specified by $\\tau = \\alpha \\tau_{\\rm{ref}}$, where $\\tau_{\\rm{ref}}$ is a reference optical thickness distribution, and we analyze simulations with 10 different values of the parameter $\\alpha$\\footnote{Simulations are performed with the following $\\alpha$ values: 0.2, 0.4, 0.7, 1.0, 1.5, 2.0, 3.0, 4.0, 5.0, and 6.0.}. We present results based on time averages over 4000 days.\n\nWhen applying the box model to the simulations, we assume that the specific humidity is well-mixed in the boundary layer and use the surface-air specific humidities to represent the entire boundary layer. In the case of the idealized GCM, surface-air quantities are taken to be those of the lowest atmospheric level, $\\sigma = 0.989$, where $\\sigma=p/p_s$, and $p$ and $p_s$ are the pressure and surface pressure, respectively. The results are qualitatively similar when specific humidities are instead averaged between the surface and $\\sigma = 0.9$ (not shown). Land values are averaged (with area weighting) over the entire subtropical continent, and the ocean averages are taken over neighboring ocean at the same latitudes, i.e. from $20^{\\circ}\\rm{N}$ to $40^{\\circ}\\rm{N}$ and $120^{\\circ}\\rm{E}$ to $360^{\\circ}\\rm{E}$.\\footnote{\nOur results are almost identical if we calculate ocean averages using the ``control'' Southern Hemisphere as in \\citet{byrne_ogorman_2013}, i.e. ocean values averaged over $20^{\\circ}\\rm{S}$ to $40^{\\circ}\\rm{S}$ and $0^{\\circ}\\rm{E}$ to $120^{\\circ}\\rm{E}$. We choose to average over neighboring ocean in this study because the box model involves advection of moisture from ocean to land, and this naturally suggests averaging over ocean adjacent to the land continent.} \n\n\n\n\\begin{figure}[t]\n  \\centering\n  \\noindent\\includegraphics[width=19pc,angle=0]{gamma_and_epsilon_vs_T_ocean_nonlinear_new.eps}\n  \\caption{Parameters for the box models applied to the idealized GCM simulations. (a) The $\\gamma$ parameter for the ocean-influence box model (solid black line) and the full box model (dashed black line). (b) The $q_{\\rm{E }}$ parameter for the full box model (dashed black line) and the surface-air land specific humidity (red line) scaled by a factor of 0.25 so that is roughly matches the magnitude of $q_{\\rm{E}}$.}\n\\label{fig:gamma_and_epsilon}\n\\end{figure}\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\nTo apply the ocean-influence box model (\\ref{eqn:box_model_3}), we calculate the $\\gamma$ parameter (the ratio of land to ocean specific humidities) for each simulation (except the warmest). We then estimate the change in surface-air land specific humidity between pairs of nearest-neighbor simulations as a function of $\\gamma$ and the changes in ocean specific humidity, where $\\gamma$ is set to its value in the colder of the two simulations and assumed to be constant as the climate changes. \n\nLand surface-air specific humidity changes between the pairs of idealized GCM simulations, along with the estimates of these changes using (\\ref{eqn:box_model_3}), are plotted against the mid-point ocean temperature for each pair in Fig. \\ref{fig:delta_q_L_and_RH_L_theory_1}.  The simulated specific humidity changes are well-captured by the ocean-influence box model (\\ref{eqn:box_model_3}) over the full range of climates (Fig. \\ref{fig:delta_q_L_and_RH_L_theory_1}a).  \nThe increases in specific humidity, $\\delta q_{\\rm{L}}$, generally increase in magnitude as the climate warms (Fig. \\ref{fig:delta_q_L_and_RH_L_theory_1}a),\n\nand they are generally smaller than what would occur if land relative humidity remained constant (see the red line in Fig. \\ref{fig:delta_q_L_and_RH_L_theory_1}a). \nThe small deviations from the prediction of the ocean-influence box model (\\ref{eqn:box_model_3}) could be due to the influence of evapotranspiration, changes in circulation patterns, or changes in the ratio $\\lambda$ of free tropospheric to surface-air ocean specific humidity which is assumed to be constant in the box model. The parameter $\\lambda$ might be expected to increase with warming because the fractional rate of increase in saturation vapor pressure with temperature is higher at the lower temperatures that occur further up in the atmosphere, and\nbecause there is enhanced warming aloft at low latitudes in simulations of global warming \\citep[e.g.,][]{santer_et_al_2005},\nand such effects could be included in a more complicated box model. \n\nThe $\\gamma$ parameter is relatively constant over the wide range of climates simulated (Fig. \\ref{fig:gamma_and_epsilon}a) consistent with our neglect of changes in $\\gamma$ when deriving (\\ref{eqn:box_model_3}), with a mean value of 0.61 and minimum and maximum values of 0.56 and 0.68, respectively. Thus, for the subtropical continent in this idealized GCM, land specific humidity is approximately 60\\% of the neighboring ocean specific humidity.\n\nThe box model (\\ref{eqn:box_model_3}) predicts the changes in mean specific humidity which must be combined with the mean temperatures to estimate the relative humidity changes. However, because of the nonlinearity of the Clausius-Clapeyron relation, it is not possible to reproduce the mean relative humidity using the mean temperature and mean specific humidity. We instead use a pseudo relative humidity, defined in terms of the mean temperature, specific humidity, and pressure as $H(\\overline{T},\\overline{p},\\overline{q})$ where the bars denote time and spatial means, and $H(T,p,q)$ is the thermodynamic relationship between relative humidity, temperature, pressure and specific humidity. For convenience we will refer to this pseudo relative humidity as the relative humidity, but we also show the actual relative humidity changes for comparison in Figs. \\ref{fig:delta_q_L_and_RH_L_theory_1}b and \\ref{fig:delta_q_RH_CMIP5_sim_theory}b.\n\nThe box-model estimate of the relative humidity changes is somewhat less accurate than for specific humidity, but the decreases in relative humidity with warming and the decreasing magnitude of these changes as the climate warms are both captured (Fig. \\ref{fig:delta_q_L_and_RH_L_theory_1}b). \n\nGiven the simplicity of the ocean-influence box model, its ability to describe the behavior of land relative humidity in this idealized GCM is impressive. However, the geometry and surface properties of Earth's land masses are more varied and complex than the idealized continent considered, and factors such as orography or cloud feedbacks that are not included in the idealized GCM could alter the surface humidity response. Therefore, to investigate the changes in land relative humidity further, we turn to more comprehensive simulations from the CMIP5 archive. \n\n\n\n\\begin{figure}[t]\n  \\centering\n  \\noindent\\includegraphics[width=19pc,angle=0]{delta_SH_land_and_delta_RH_CMIP5_and_estimates_vs_lat.eps}\n  \\caption{Multimodel-mean changes between the historical and RCP8.5 simulations in zonal and time mean (a) surface-air land specific humidity and (b) surface-air land relative humidity normalized by the global-mean surface-air temperature change. Solid black lines denote the simulated changes and dashed lines denote the estimated changes using the box model (\\ref{eqn:box_model_3}). For (a), the red line indicates the change in surface-air land specific humidity for constant land pseudo relative humidity (i.e. for land pseudo relative humidity fixed at the values in the historical simulations). \nPseudo relative humidities are shown, but the blue line in (b) shows the simulated mean changes for the relative humidity variable outputted by the models for comparison.}\n\\label{fig:delta_q_RH_CMIP5_sim_theory}\n\\end{figure}\n\n3.2 Application of ocean-influence box model to CMIP5 simulations\n\\subsection{Application of ocean-influence box model to CMIP5 simulations}\n\\label{sect:CMIP5_analysis_theory_1}\n\nWe apply the ocean-influence box model to changes in land surface-air relative humidity between 30 year time averages in the historical (1976-2005) and RCP8.5 (2070-2099) simulations from the CMIP5 archive \\citep{taylor_et_al_bams_2012}. We analyze 19 models in total\\footnote{The CMIP5 models considered are: ACCESS1-0, ACCESS1-3, BCC-CSM1-1, BCC-CSM1-1-M, BNU-ESM, CanESM2, CNRM-CM5, CSIRO-Mk3-6-0, GFDL-CM3, GFDL-ESM2M, INMCM4, IPSL-CM5A-LR, IPSL-CM5A-MR, IPSL-CM5B-LR, MIROC-ESM, MIROC-ESM-CHEM, MIROC5, MRI-CGCM3, and NorESM1-M. The variables used in this paper have the following names in the CMIP5 archive: evaporation (\\textit{evspsbl}), surface-air specific humidity (\\textit{huss}), surface-air temperature (\\textit{tas}), and surface-air relative humidity (\\textit{hurs}).}, and in each case the r1i1p1 ensemble member is used. As for the idealized GCM analysis, we assume moisture is well mixed in the boundary layer and take surface-air specific humidity to be representative of the boundary layer (using the average specific humidity between the surface and 900hPa gives similar results). \n\n\nThe specific humidities in the box model are identified with the zonal and time mean specific humidities (over land or ocean) for each latitude and for each of the (12) months of the year in the CMIP5 simulations.  We calculate $\\gamma$ as the ratio of the mean land and ocean specific humidities at each latitude and for each month of the year in the historical simulations. By computing $\\gamma$ in this way, we are assuming that the horizontal exchange of moisture between land and ocean, described by the box model, is taking place predominantly in the zonal direction. \nUsing the diagnosed $\\gamma$, and assuming it does not change as the climate warms, changes in mean surface-air land specific humidity are estimated for each latitude and each month of the year using (\\ref{eqn:box_model_3}) and the changes in mean ocean specific humidity. \n\n\n\n\\begin{figure}[t]\n  \\centering\n  \\noindent\\includegraphics[width=19pc,angle=0]{gamma_epsilon_zeta_vs_lat_both_theories_CMIP5.eps}\n  \\caption{Multimodel means in the CMIP5 simulations of the (a) $\\gamma$ parameter for the ocean-influence box model (solid black line) and for the regression approach including evapotranspiration (dashed black line), and (b) the regression coefficient $\\epsilon$. The parameter $\\gamma$ is evaluated based on the historical simulations for the ocean-influence box model, and $\\gamma$ and $\\epsilon$ are evaluated using equation \\ref{eqn:regression} for the regression approach.}\n\\label{fig:gamma_eps_zeta_vs_lat}\n\\end{figure}\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\nThe simulated and estimated annual mean changes in land specific humidity at each latitude are shown in Fig. \\ref{fig:delta_q_RH_CMIP5_sim_theory}a, and the magnitude and latitudinal variations of the changes are reasonably well captured by the ocean-influence box model, including the flat region in the Northern Hemisphere midlatitudes. The magnitude of the increases is underestimated at most latitudes which, as discussed in the case of the idealized GCM simulations could be partly due to increases in the parameter $\\lambda$ relating free-tropospheric specific humidity to ocean surface specific humidity, but other aspects of the ocean-influence box model such as neglecting the influence of evapotranspiration are also likely to play a role. The parameter $\\gamma$ (i.e. the ratio of land and ocean specific humidities) is shown in Fig.~\\ref{fig:gamma_eps_zeta_vs_lat}a. It has a global, annual and multimodel mean value of 0.74, which is somewhat larger than the\nvalue found in the idealized GCM simulations. This is not\nsurprising given that the land in the idealized simulations\nis a subtropical continent, which is generally drier relative\nto neighboring oceans than continents at lower or higher\nlatitudes. \n\n\n\nTogether with the simulated changes in monthly-mean surface-air land temperature, the estimated changes in specific humidity are used to estimate the land pseudo relative humidity changes. As for the idealized GCM analysis, it is necessary to compare pseudo-relative humidities because of the difficulty in converting time and zonal mean specific humidities estimated by the box model to relative humidities. The use of pseudo relative humidities also avoids the complication that different climate models use different saturation vapor pressure formulations. The changes in pseudo relative humidity are calculated for each month of the year before taking the annual mean for both the simulated changes and the changes estimated by the box model. The changes in pseudo relative humidity and model-outputted relative humidity are somewhat similar at lower latitudes but quite different at higher latitudes (cf. blue and black solid lines in Fig. \\ref{fig:delta_q_RH_CMIP5_sim_theory}b), where the differing computations of saturation vapor pressure over ice in the various models becomes important and there is larger variability. Nonetheless the pseudo relative humidity is a useful measure of subsaturation, and we will refer to pseudo relative humidity as relative humidity for simplicity.\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\nThe simulated changes in (pseudo) land relative humidity are quite well described by the ocean-influence box model in the Southern Hemisphere and at lower latitudes (Fig. \\ref{fig:delta_q_RH_CMIP5_sim_theory}b). The estimated and simulated global-mean land relative humidity changes in the various climate models are also correlated, with a correlation coefficient of 0.66. \nDue to the general underestimation of the specific humidity increases by the ocean-influence box model (Fig. \\ref{fig:delta_q_RH_CMIP5_sim_theory}a), the relative humidity decreases are overestimated (Fig. \\ref{fig:delta_q_RH_CMIP5_sim_theory}b), with a large discrepancy in the mid- to high-latitudes of the Northern Hemisphere. At these latitudes, there is more land than ocean and it is likely that changes in ocean specific humidity have a weak influence on the specific humidity in the interior of large continents, or that meridional moisture transports from ocean at other latitudes become more important. Changes in relative humidity in these inner continental regions are likely to be more strongly influenced by local evapotranspiration changes, and they could be influenced by\nshifts in the iceline or changes in soil moisture or vegetation, effects which are not considered in the ocean-influence box model. \n\n\n\n\n\n\n\n\n",
        "label": "fig:delta_q_RH_CMIP5_sim_theory",
        "Descriptive_question1": "What color is used for the line representing simulated changes in surface-air land specific humidity in figure_6?",
        "Descriptive_question2": "What type of line is used to denote estimated changes using the box model in figure_6?",
        "Reasoning_question1": "Why might the ocean-influence box model underestimate the increases in surface-air land specific humidity at most latitudes in figure_6?",
        "Reasoning_question2": "How does the discrepancy between simulated and estimated relative humidity changes in the Northern Hemisphere mid- to high-latitudes in figure_6 reflect the limitations of the ocean-influence box model?",
        "Descriptive_answer1": "black",
        "Descriptive_answer2": "dashed",
        "Reasoning_answer1": "The ocean-influence box model might underestimate the increases in surface-air land specific humidity because it neglects certain processes such as the influence of evapotranspiration, changes in circulation patterns, and changes in the ratio of free tropospheric to surface-air ocean specific humidity. These factors could contribute to greater moisture increases over land that the simpler model fails to capture.",
        "Reasoning_answer2": "The larger discrepancy in the Northern Hemisphere mid- to high-latitudes between simulated and estimated relative humidity changes suggests that the ocean-influence box model's assumption of negligible local evapotranspiration and predominant zonal moisture exchange from ocean may not hold well. Interior continental regions with more land area are likely influenced more by local evapotranspiration, soil moisture, vegetation changes, and meridional moisture transport, which the model does not consider, highlighting its limitations in these regions."
    },
    {
        "paper_id": "1605.00380.json",
        "image_id": "figure_7",
        "image_path": [
            "/home/yz979/scratch/chengye/scimolmo/test-images-tables/dataset_imagestables/1605.00380/images/gamma_epsilon_zeta_vs_lat_both_theories_CMIP5.eps"
        ],
        "caption": "Multimodel means in the CMIP5 simulations of the (a) $\\gamma$ parameter for the ocean-influence box model (solid black line) and for the regression approach including evapotranspiration (dashed black line), and (b) the regression coefficient $\\epsilon$. The parameter $\\gamma$ is evaluated based on the historical simulations for the ocean-influence box model, and $\\gamma$ and $\\epsilon$ are evaluated using equation \\ref{eqn:regression} for the regression approach.",
        "classify": "Chart",
        "section_info": "3 Ocean-influence box model\n\\section{Ocean-influence box model}\n\nFor the simplest version of our box model, the ``ocean-influence box model'', we assume that the influence of evapotranspiration on the boundary-layer moisture balance over land is negligible. Setting $E_{\\rm{L}}=0$ in (\\ref{eqn:new_box_model_2}), we find:\n\n\\begin{equation}\nq_{\\rm{L}} \\approx \\gamma q_{\\rm{O}}.\n\\label{eqn:box_model_2}\n\\end{equation}\n\nThe parameter $\\gamma$ may remain approximately constant under climate change even if there are changes in the mixing time scales $\\tau_1$ and $\\tau_2$. For example, if the overall tropical circulation and convective mass fluxes slow down with climate warming \\citep[e.g.,][]{held_soden_2006,vecchi_soden_2007} such that both mixing time scales increase by the same factor, then this will not cause $\\gamma$ to change. Assuming negligible changes in $\\gamma$, we can write:\n\n\\begin{equation}\n\\delta q_{\\rm{L}} \\approx \\gamma \\delta q_{\\rm{O}},\n\\label{eqn:box_model_3}\n\\end{equation}\n\nwhere $\\delta$ denotes the change in climate.\n\nThe same results would also follow (with a different definition of $\\gamma$) if the influence of evapotranspiration on land specific humidity, $q_{\\rm{E}}$, is not neglected but is instead assumed to scale with land specific humidity. \nNote that $q_{\\rm{E}}$ scaling with land specific humidity is not\nthe same as the evapotranspiration rate scaling with land specific humidity.\nIn particular, $q_E$ depends on  \n$(\\tau_1 \\tau_2)/(\\tau_1 + \\tau_2)$, and this factor would change even if\nboth $\\tau_1$ and $\\tau_2$ increase by the same factor.\n\nThe ocean-influence box model suggests a straightforward hypothesis - that the ratio of land to ocean specific humidity remains approximately constant as the climate changes or, equivalently, that fractional changes in specific humidity over land and ocean are equal: \n\n\\begin{equation}\n\\frac{\\delta q_{\\rm{L}}}{q_{\\rm{L}}} \\approx \\frac{\\delta q_{\\rm{O}}}{q_{\\rm{O}}}.\n\\label{eqn:constant_q_ratio}\n\\end{equation}\n\nBy contrast the fractional changes in saturation specific humidity depend on the local temperature change and will be bigger over land than ocean.\nTo the extent that (\\ref{eqn:constant_q_ratio}) holds, it is clear that if land warms more than ocean, and ocean relative humidity does not change greatly, then the land relative humidity will decrease.\n\nWe now assess the applicability of this ocean-influence box model result to idealized and comprehensive GCM simulations. \nWe use (\\ref{eqn:box_model_3}) to estimate the change in land relative humidity under climate change given the changes in land temperature\nand ocean specific humidity, and calculating $\\gamma$ as the the ratio of land to ocean specific humidities in the control climate.  \n\n\n\n\\begin{figure}[t]\n  \\centering\n  \\noindent\\includegraphics[width=19pc,angle=0]{delta_q_L_and_delta_RH_L_vs_T_ocean_iGCM_sim_and_theory_1.eps}\n  \\caption{Changes over land in (a) surface-air specific humidity and (b) surface-air relative humidity between pairs of idealized GCM simulations with a subtropical continent.  The relative humidity changes are \nnormalized by the land surface-air temperature change.\nSolid black lines denote the simulated changes and the dashed lines represent the estimated changes using the ocean-influence box model (\\ref{eqn:box_model_3}). Pseudo relative humidities are shown (see text), but the blue line in (b) shows the change in the mean of the actual relative humidity for comparison. The red line in (a) indicates what the change in surface-air land specific humidity would be if the land pseudo relative humidity did not change (i.e. for each pair of simulations, the land specific humidity change if the land pseudo relative humidity is fixed at its value in the colder simulation).}\n\\label{fig:delta_q_L_and_RH_L_theory_1}\n\\end{figure}\n\n\n\n\n\\subsection{Application of ocean-influence box model to idealized GCM simulations}\n\\label{sect:iGCM_analysis_theory_1}\n\nThe ocean-influence box model is first applied to idealized GCM simulations over a wide range of climates. The idealized GCM is similar to that of \\citet{frierson_2006} and \\citet{frierson_2007}, with specific details as in \\citet{byrne_ogorman_2013} and \\citet{ogorman_2008}. It is based on a spectral version of the GFDL dynamical core, with a two-stream gray radiation scheme, no cloud or water vapor radiative feedbacks, and the simplified moist convection scheme of \\citet{frierson_2007}. The simulations have a subtropical continent spanning $20^{\\circ}\\rm{N}$ to $40^{\\circ}\\rm{N}$ and $0^{\\circ}\\rm{E}$ to $120^{\\circ}\\rm{E}$, with a slab ocean elsewhere (Fig. \\ref{fig:continent_schematic}). The land surface hydrology is simulated using a simple bucket model \\citep{manabe_1969} and all other land surface properties are identical to those of the slab ocean. We vary the climate over a wide range of global-mean surface-air temperatures (between 260K and 317K) by changing the longwave optical thickness, which is analogous to varying the concentrations of $\\rm{CO}_2$ and other greenhouse gases. The longwave optical thickness is specified by $\\tau = \\alpha \\tau_{\\rm{ref}}$, where $\\tau_{\\rm{ref}}$ is a reference optical thickness distribution, and we analyze simulations with 10 different values of the parameter $\\alpha$\\footnote{Simulations are performed with the following $\\alpha$ values: 0.2, 0.4, 0.7, 1.0, 1.5, 2.0, 3.0, 4.0, 5.0, and 6.0.}. We present results based on time averages over 4000 days.\n\nWhen applying the box model to the simulations, we assume that the specific humidity is well-mixed in the boundary layer and use the surface-air specific humidities to represent the entire boundary layer. In the case of the idealized GCM, surface-air quantities are taken to be those of the lowest atmospheric level, $\\sigma = 0.989$, where $\\sigma=p/p_s$, and $p$ and $p_s$ are the pressure and surface pressure, respectively. The results are qualitatively similar when specific humidities are instead averaged between the surface and $\\sigma = 0.9$ (not shown). Land values are averaged (with area weighting) over the entire subtropical continent, and the ocean averages are taken over neighboring ocean at the same latitudes, i.e. from $20^{\\circ}\\rm{N}$ to $40^{\\circ}\\rm{N}$ and $120^{\\circ}\\rm{E}$ to $360^{\\circ}\\rm{E}$.\\footnote{\nOur results are almost identical if we calculate ocean averages using the ``control'' Southern Hemisphere as in \\citet{byrne_ogorman_2013}, i.e. ocean values averaged over $20^{\\circ}\\rm{S}$ to $40^{\\circ}\\rm{S}$ and $0^{\\circ}\\rm{E}$ to $120^{\\circ}\\rm{E}$. We choose to average over neighboring ocean in this study because the box model involves advection of moisture from ocean to land, and this naturally suggests averaging over ocean adjacent to the land continent.} \n\n\n\n\\begin{figure}[t]\n  \\centering\n  \\noindent\\includegraphics[width=19pc,angle=0]{gamma_and_epsilon_vs_T_ocean_nonlinear_new.eps}\n  \\caption{Parameters for the box models applied to the idealized GCM simulations. (a) The $\\gamma$ parameter for the ocean-influence box model (solid black line) and the full box model (dashed black line). (b) The $q_{\\rm{E }}$ parameter for the full box model (dashed black line) and the surface-air land specific humidity (red line) scaled by a factor of 0.25 so that is roughly matches the magnitude of $q_{\\rm{E}}$.}\n\\label{fig:gamma_and_epsilon}\n\\end{figure}\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\nTo apply the ocean-influence box model (\\ref{eqn:box_model_3}), we calculate the $\\gamma$ parameter (the ratio of land to ocean specific humidities) for each simulation (except the warmest). We then estimate the change in surface-air land specific humidity between pairs of nearest-neighbor simulations as a function of $\\gamma$ and the changes in ocean specific humidity, where $\\gamma$ is set to its value in the colder of the two simulations and assumed to be constant as the climate changes. \n\nLand surface-air specific humidity changes between the pairs of idealized GCM simulations, along with the estimates of these changes using (\\ref{eqn:box_model_3}), are plotted against the mid-point ocean temperature for each pair in Fig. \\ref{fig:delta_q_L_and_RH_L_theory_1}.  The simulated specific humidity changes are well-captured by the ocean-influence box model (\\ref{eqn:box_model_3}) over the full range of climates (Fig. \\ref{fig:delta_q_L_and_RH_L_theory_1}a).  \nThe increases in specific humidity, $\\delta q_{\\rm{L}}$, generally increase in magnitude as the climate warms (Fig. \\ref{fig:delta_q_L_and_RH_L_theory_1}a),\n\nand they are generally smaller than what would occur if land relative humidity remained constant (see the red line in Fig. \\ref{fig:delta_q_L_and_RH_L_theory_1}a). \nThe small deviations from the prediction of the ocean-influence box model (\\ref{eqn:box_model_3}) could be due to the influence of evapotranspiration, changes in circulation patterns, or changes in the ratio $\\lambda$ of free tropospheric to surface-air ocean specific humidity which is assumed to be constant in the box model. The parameter $\\lambda$ might be expected to increase with warming because the fractional rate of increase in saturation vapor pressure with temperature is higher at the lower temperatures that occur further up in the atmosphere, and\nbecause there is enhanced warming aloft at low latitudes in simulations of global warming \\citep[e.g.,][]{santer_et_al_2005},\nand such effects could be included in a more complicated box model. \n\nThe $\\gamma$ parameter is relatively constant over the wide range of climates simulated (Fig. \\ref{fig:gamma_and_epsilon}a) consistent with our neglect of changes in $\\gamma$ when deriving (\\ref{eqn:box_model_3}), with a mean value of 0.61 and minimum and maximum values of 0.56 and 0.68, respectively. Thus, for the subtropical continent in this idealized GCM, land specific humidity is approximately 60\\% of the neighboring ocean specific humidity.\n\nThe box model (\\ref{eqn:box_model_3}) predicts the changes in mean specific humidity which must be combined with the mean temperatures to estimate the relative humidity changes. However, because of the nonlinearity of the Clausius-Clapeyron relation, it is not possible to reproduce the mean relative humidity using the mean temperature and mean specific humidity. We instead use a pseudo relative humidity, defined in terms of the mean temperature, specific humidity, and pressure as $H(\\overline{T},\\overline{p},\\overline{q})$ where the bars denote time and spatial means, and $H(T,p,q)$ is the thermodynamic relationship between relative humidity, temperature, pressure and specific humidity. For convenience we will refer to this pseudo relative humidity as the relative humidity, but we also show the actual relative humidity changes for comparison in Figs. \\ref{fig:delta_q_L_and_RH_L_theory_1}b and \\ref{fig:delta_q_RH_CMIP5_sim_theory}b.\n\nThe box-model estimate of the relative humidity changes is somewhat less accurate than for specific humidity, but the decreases in relative humidity with warming and the decreasing magnitude of these changes as the climate warms are both captured (Fig. \\ref{fig:delta_q_L_and_RH_L_theory_1}b). \n\nGiven the simplicity of the ocean-influence box model, its ability to describe the behavior of land relative humidity in this idealized GCM is impressive. However, the geometry and surface properties of Earth's land masses are more varied and complex than the idealized continent considered, and factors such as orography or cloud feedbacks that are not included in the idealized GCM could alter the surface humidity response. Therefore, to investigate the changes in land relative humidity further, we turn to more comprehensive simulations from the CMIP5 archive. \n\n\n\n\\begin{figure}[t]\n  \\centering\n  \\noindent\\includegraphics[width=19pc,angle=0]{delta_SH_land_and_delta_RH_CMIP5_and_estimates_vs_lat.eps}\n  \\caption{Multimodel-mean changes between the historical and RCP8.5 simulations in zonal and time mean (a) surface-air land specific humidity and (b) surface-air land relative humidity normalized by the global-mean surface-air temperature change. Solid black lines denote the simulated changes and dashed lines denote the estimated changes using the box model (\\ref{eqn:box_model_3}). For (a), the red line indicates the change in surface-air land specific humidity for constant land pseudo relative humidity (i.e. for land pseudo relative humidity fixed at the values in the historical simulations). \nPseudo relative humidities are shown, but the blue line in (b) shows the simulated mean changes for the relative humidity variable outputted by the models for comparison.}\n\\label{fig:delta_q_RH_CMIP5_sim_theory}\n\\end{figure}\n\n\\subsection{Application of ocean-influence box model to CMIP5 simulations}\n\\label{sect:CMIP5_analysis_theory_1}\n\nWe apply the ocean-influence box model to changes in land surface-air relative humidity between 30 year time averages in the historical (1976-2005) and RCP8.5 (2070-2099) simulations from the CMIP5 archive \\citep{taylor_et_al_bams_2012}. We analyze 19 models in total\\footnote{The CMIP5 models considered are: ACCESS1-0, ACCESS1-3, BCC-CSM1-1, BCC-CSM1-1-M, BNU-ESM, CanESM2, CNRM-CM5, CSIRO-Mk3-6-0, GFDL-CM3, GFDL-ESM2M, INMCM4, IPSL-CM5A-LR, IPSL-CM5A-MR, IPSL-CM5B-LR, MIROC-ESM, MIROC-ESM-CHEM, MIROC5, MRI-CGCM3, and NorESM1-M. The variables used in this paper have the following names in the CMIP5 archive: evaporation (\\textit{evspsbl}), surface-air specific humidity (\\textit{huss}), surface-air temperature (\\textit{tas}), and surface-air relative humidity (\\textit{hurs}).}, and in each case the r1i1p1 ensemble member is used. As for the idealized GCM analysis, we assume moisture is well mixed in the boundary layer and take surface-air specific humidity to be representative of the boundary layer (using the average specific humidity between the surface and 900hPa gives similar results). \n\n\nThe specific humidities in the box model are identified with the zonal and time mean specific humidities (over land or ocean) for each latitude and for each of the (12) months of the year in the CMIP5 simulations.  We calculate $\\gamma$ as the ratio of the mean land and ocean specific humidities at each latitude and for each month of the year in the historical simulations. By computing $\\gamma$ in this way, we are assuming that the horizontal exchange of moisture between land and ocean, described by the box model, is taking place predominantly in the zonal direction. \nUsing the diagnosed $\\gamma$, and assuming it does not change as the climate warms, changes in mean surface-air land specific humidity are estimated for each latitude and each month of the year using (\\ref{eqn:box_model_3}) and the changes in mean ocean specific humidity. \n\n\n\n\\begin{figure}[t]\n  \\centering\n  \\noindent\\includegraphics[width=19pc,angle=0]{gamma_epsilon_zeta_vs_lat_both_theories_CMIP5.eps}\n  \\caption{Multimodel means in the CMIP5 simulations of the (a) $\\gamma$ parameter for the ocean-influence box model (solid black line) and for the regression approach including evapotranspiration (dashed black line), and (b) the regression coefficient $\\epsilon$. The parameter $\\gamma$ is evaluated based on the historical simulations for the ocean-influence box model, and $\\gamma$ and $\\epsilon$ are evaluated using equation \\ref{eqn:regression} for the regression approach.}\n\\label{fig:gamma_eps_zeta_vs_lat}\n\\end{figure}\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\nThe simulated and estimated annual mean changes in land specific humidity at each latitude are shown in Fig. \\ref{fig:delta_q_RH_CMIP5_sim_theory}a, and the magnitude and latitudinal variations of the changes are reasonably well captured by the ocean-influence box model, including the flat region in the Northern Hemisphere midlatitudes. The magnitude of the increases is underestimated at most latitudes which, as discussed in the case of the idealized GCM simulations could be partly due to increases in the parameter $\\lambda$ relating free-tropospheric specific humidity to ocean surface specific humidity, but other aspects of the ocean-influence box model such as neglecting the influence of evapotranspiration are also likely to play a role. The parameter $\\gamma$ (i.e. the ratio of land and ocean specific humidities) is shown in Fig.~\\ref{fig:gamma_eps_zeta_vs_lat}a. It has a global, annual and multimodel mean value of 0.74, which is somewhat larger than the\nvalue found in the idealized GCM simulations. This is not\nsurprising given that the land in the idealized simulations\nis a subtropical continent, which is generally drier relative\nto neighboring oceans than continents at lower or higher\nlatitudes. \n\n\n\nTogether with the simulated changes in monthly-mean surface-air land temperature, the estimated changes in specific humidity are used to estimate the land pseudo relative humidity changes. As for the idealized GCM analysis, it is necessary to compare pseudo-relative humidities because of the difficulty in converting time and zonal mean specific humidities estimated by the box model to relative humidities. The use of pseudo relative humidities also avoids the complication that different climate models use different saturation vapor pressure formulations. The changes in pseudo relative humidity are calculated for each month of the year before taking the annual mean for both the simulated changes and the changes estimated by the box model. The changes in pseudo relative humidity and model-outputted relative humidity are somewhat similar at lower latitudes but quite different at higher latitudes (cf. blue and black solid lines in Fig. \\ref{fig:delta_q_RH_CMIP5_sim_theory}b), where the differing computations of saturation vapor pressure over ice in the various models becomes important and there is larger variability. Nonetheless the pseudo relative humidity is a useful measure of subsaturation, and we will refer to pseudo relative humidity as relative humidity for simplicity.\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\nThe simulated changes in (pseudo) land relative humidity are quite well described by the ocean-influence box model in the Southern Hemisphere and at lower latitudes (Fig. \\ref{fig:delta_q_RH_CMIP5_sim_theory}b). The estimated and simulated global-mean land relative humidity changes in the various climate models are also correlated, with a correlation coefficient of 0.66. \nDue to the general underestimation of the specific humidity increases by the ocean-influence box model (Fig. \\ref{fig:delta_q_RH_CMIP5_sim_theory}a), the relative humidity decreases are overestimated (Fig. \\ref{fig:delta_q_RH_CMIP5_sim_theory}b), with a large discrepancy in the mid- to high-latitudes of the Northern Hemisphere. At these latitudes, there is more land than ocean and it is likely that changes in ocean specific humidity have a weak influence on the specific humidity in the interior of large continents, or that meridional moisture transports from ocean at other latitudes become more important. Changes in relative humidity in these inner continental regions are likely to be more strongly influenced by local evapotranspiration changes, and they could be influenced by\nshifts in the iceline or changes in soil moisture or vegetation, effects which are not considered in the ocean-influence box model. \n\n\n\n\n\n\n\n\n3.2 Application of ocean-influence box model to CMIP5 simulations\n\\subsection{Application of ocean-influence box model to CMIP5 simulations}\n\\label{sect:CMIP5_analysis_theory_1}\n\nWe apply the ocean-influence box model to changes in land surface-air relative humidity between 30 year time averages in the historical (1976-2005) and RCP8.5 (2070-2099) simulations from the CMIP5 archive \\citep{taylor_et_al_bams_2012}. We analyze 19 models in total\\footnote{The CMIP5 models considered are: ACCESS1-0, ACCESS1-3, BCC-CSM1-1, BCC-CSM1-1-M, BNU-ESM, CanESM2, CNRM-CM5, CSIRO-Mk3-6-0, GFDL-CM3, GFDL-ESM2M, INMCM4, IPSL-CM5A-LR, IPSL-CM5A-MR, IPSL-CM5B-LR, MIROC-ESM, MIROC-ESM-CHEM, MIROC5, MRI-CGCM3, and NorESM1-M. The variables used in this paper have the following names in the CMIP5 archive: evaporation (\\textit{evspsbl}), surface-air specific humidity (\\textit{huss}), surface-air temperature (\\textit{tas}), and surface-air relative humidity (\\textit{hurs}).}, and in each case the r1i1p1 ensemble member is used. As for the idealized GCM analysis, we assume moisture is well mixed in the boundary layer and take surface-air specific humidity to be representative of the boundary layer (using the average specific humidity between the surface and 900hPa gives similar results). \n\n\nThe specific humidities in the box model are identified with the zonal and time mean specific humidities (over land or ocean) for each latitude and for each of the (12) months of the year in the CMIP5 simulations.  We calculate $\\gamma$ as the ratio of the mean land and ocean specific humidities at each latitude and for each month of the year in the historical simulations. By computing $\\gamma$ in this way, we are assuming that the horizontal exchange of moisture between land and ocean, described by the box model, is taking place predominantly in the zonal direction. \nUsing the diagnosed $\\gamma$, and assuming it does not change as the climate warms, changes in mean surface-air land specific humidity are estimated for each latitude and each month of the year using (\\ref{eqn:box_model_3}) and the changes in mean ocean specific humidity. \n\n\n\n\\begin{figure}[t]\n  \\centering\n  \\noindent\\includegraphics[width=19pc,angle=0]{gamma_epsilon_zeta_vs_lat_both_theories_CMIP5.eps}\n  \\caption{Multimodel means in the CMIP5 simulations of the (a) $\\gamma$ parameter for the ocean-influence box model (solid black line) and for the regression approach including evapotranspiration (dashed black line), and (b) the regression coefficient $\\epsilon$. The parameter $\\gamma$ is evaluated based on the historical simulations for the ocean-influence box model, and $\\gamma$ and $\\epsilon$ are evaluated using equation \\ref{eqn:regression} for the regression approach.}\n\\label{fig:gamma_eps_zeta_vs_lat}\n\\end{figure}\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\nThe simulated and estimated annual mean changes in land specific humidity at each latitude are shown in Fig. \\ref{fig:delta_q_RH_CMIP5_sim_theory}a, and the magnitude and latitudinal variations of the changes are reasonably well captured by the ocean-influence box model, including the flat region in the Northern Hemisphere midlatitudes. The magnitude of the increases is underestimated at most latitudes which, as discussed in the case of the idealized GCM simulations could be partly due to increases in the parameter $\\lambda$ relating free-tropospheric specific humidity to ocean surface specific humidity, but other aspects of the ocean-influence box model such as neglecting the influence of evapotranspiration are also likely to play a role. The parameter $\\gamma$ (i.e. the ratio of land and ocean specific humidities) is shown in Fig.~\\ref{fig:gamma_eps_zeta_vs_lat}a. It has a global, annual and multimodel mean value of 0.74, which is somewhat larger than the\nvalue found in the idealized GCM simulations. This is not\nsurprising given that the land in the idealized simulations\nis a subtropical continent, which is generally drier relative\nto neighboring oceans than continents at lower or higher\nlatitudes. \n\n\n\nTogether with the simulated changes in monthly-mean surface-air land temperature, the estimated changes in specific humidity are used to estimate the land pseudo relative humidity changes. As for the idealized GCM analysis, it is necessary to compare pseudo-relative humidities because of the difficulty in converting time and zonal mean specific humidities estimated by the box model to relative humidities. The use of pseudo relative humidities also avoids the complication that different climate models use different saturation vapor pressure formulations. The changes in pseudo relative humidity are calculated for each month of the year before taking the annual mean for both the simulated changes and the changes estimated by the box model. The changes in pseudo relative humidity and model-outputted relative humidity are somewhat similar at lower latitudes but quite different at higher latitudes (cf. blue and black solid lines in Fig. \\ref{fig:delta_q_RH_CMIP5_sim_theory}b), where the differing computations of saturation vapor pressure over ice in the various models becomes important and there is larger variability. Nonetheless the pseudo relative humidity is a useful measure of subsaturation, and we will refer to pseudo relative humidity as relative humidity for simplicity.\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\nThe simulated changes in (pseudo) land relative humidity are quite well described by the ocean-influence box model in the Southern Hemisphere and at lower latitudes (Fig. \\ref{fig:delta_q_RH_CMIP5_sim_theory}b). The estimated and simulated global-mean land relative humidity changes in the various climate models are also correlated, with a correlation coefficient of 0.66. \nDue to the general underestimation of the specific humidity increases by the ocean-influence box model (Fig. \\ref{fig:delta_q_RH_CMIP5_sim_theory}a), the relative humidity decreases are overestimated (Fig. \\ref{fig:delta_q_RH_CMIP5_sim_theory}b), with a large discrepancy in the mid- to high-latitudes of the Northern Hemisphere. At these latitudes, there is more land than ocean and it is likely that changes in ocean specific humidity have a weak influence on the specific humidity in the interior of large continents, or that meridional moisture transports from ocean at other latitudes become more important. Changes in relative humidity in these inner continental regions are likely to be more strongly influenced by local evapotranspiration changes, and they could be influenced by\nshifts in the iceline or changes in soil moisture or vegetation, effects which are not considered in the ocean-influence box model. \n\n\n\n\n\n\n\n\n4 Influence of evapotranspiration\n\\section{Influence of evapotranspiration}\n\\label{sect:full_model_RH_land}\n\n\n\n\nThe ocean-influence box model captures much (but not all) of the behavior in vastly more complex GCMs. However, the moisture balance of the land boundary layer is also affected by evapotranspiration, and changes in land surface properties, such as soil moisture or stomatal conductance, can affect evapotranspiration in the absence of any changes in the overlying atmosphere.  For example, changes in stomatal conductance under elevated $\\rm{CO}_2$ conditions has been shown to reduce both evapotranspiration and land relative humidity without changes in ocean humidity \\citep{andrews_et_al_2011}. \n\n\n\n\\begin{figure}[t]\n  \\centering\n  \\noindent\\includegraphics[width=19pc,angle=0]{delta_q_L_and_delta_RH_L_vs_T_ocean_iGCM_sim_and_theory_2_nonlinear_new.eps}\n  \\caption{As in Figure \\ref{fig:delta_q_L_and_RH_L_theory_1}, but here showing estimates of the surface-air specific and relative humidity changes from the full box model (\\ref{eqn:new_box_model_3}). The contributions due to ocean specific humidity changes (blue dashed lines) and evapotranspiration changes (green dashed lines) are also shown. The contributions to changes in relative humidity are calculated using (\\ref{eqn:appendix_3}). Pseudo relative humidities are shown in this figure (the changes in actual relative humidity are shown by the blue line in Fig.~\\ref{fig:delta_q_L_and_RH_L_theory_1}b.)}\n\\label{fig:delta_q_L_and_RH_L_theory_2}\n\\end{figure}\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\nWe turn to the full box model (\\ref{eqn:new_box_model_2}) which includes\nthe effects of evapotranspiration. We assume once more that changes in $\\gamma$ are negligible, \n\nsuch that:\n\n\\begin{equation}\n\\delta q_{\\rm{L}} \\approx \\gamma \\delta q_{\\rm{O}} + \\delta q_{\\rm{E}}.\n\\label{eqn:new_box_model_3}\n\\end{equation}\n\nThere are two terms contributing to changes in $q_{\\rm{L}}$ in (\\ref{eqn:new_box_model_3}): the term arising from changes in ocean specific humidity, $\\gamma \\delta q_{\\rm{O}}$, and an additional land evapotranspiration term, $\\delta q_{\\rm{E}}$. We next assess their relative importance in controlling changes in land humidity in the idealized GCM and CMIP5 simulations.\n\n\n\\subsection{Application of full box model to idealized GCM simulations}\n\\label{sect:theory_2_simulation_analysis}\n\nWe first examine the idealized GCM simulations with a subtropical continent. In contrast to the ocean-influence box model (\\ref{eqn:box_model_3}), for which the single parameter $\\gamma$ could be easily estimated in the control simulation in each case, the full model (\\ref{eqn:new_box_model_3}) has two parameters to be estimated, $\\gamma$ and $q_{\\rm{E}}$. To estimate these parameters, we perform an additional set of simulations with the same longwave optical thicknesses as in the 10 simulations described previously but with the evapotranspiration set to zero over land. \n\nSpecifying the land evapotranspiration in this way is analogous to drying out the soil. (Note that the change in evapotranspiration affects both the humidity of the atmosphere and the surface energy balance.)\nUsing these additional simulations with $E_{\\rm{L}} = 0$, we can estimate $\\gamma$ for each climate using (\\ref{eqn:new_box_model_2}): $\\gamma = q_{\\rm{L},E_{\\rm{L}}=0} / q_{\\rm{O},E_{\\rm{L}}=0}$.  The $\\gamma$ values obtained are smaller than those calculated from the control climate for the ocean-influence box model (Fig. \\ref{fig:gamma_and_epsilon}a) because the contribution of evapotranspiration to the land specific humidity is now also taken into account.  We then use these $\\gamma$ values to estimate $q_{\\rm{E}}$ for the original simulations with dynamic land surface hydrology \n: $q_{\\rm{E}} = q_{\\rm{L}} - \\gamma q_{\\rm{O}}$. The values of $q_{\\rm{E}}$ increase with warming except in hot climates (Fig. \\ref{fig:gamma_and_epsilon}b).\n\nInterestingly, the influence of evapotranspiration on land specific humidity as measured by $q_{\\rm{E}}$ roughly scales with the land specific humidity except in hot climates (compare the dashed black and solid red lines in Fig. \\ref{fig:gamma_and_epsilon}b), and this helps to explain why the ocean-influence box model\nis accurate even though evapotranspiration affects land specific humidity.\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\nWe then estimate the changes in land specific humidity between pairs of nearest-neighbor simulations from (\\ref{eqn:new_box_model_3}) with $\\gamma$ assumed to be constant as the climate changes.  The simulated and estimated changes in surface-air land specific humidity, along with the contributions due to changes in ocean specific humidity and land evapotranspiration are shown in Fig. \\ref{fig:delta_q_L_and_RH_L_theory_2}a. The full box model captures the basic behavior of the land specific humidity changes as a function of temperature, although it is less accurate in hot climates. The contribution from ocean specific humidity changes, $\\gamma \\delta q_{\\rm{O}}$, is larger than the contribution from land evapotranspiration for all climates (Fig. \\ref{fig:delta_q_L_and_RH_L_theory_2}a).  The changes in simulated land (pseudo) relative humidity are also well captured by the full box model (Fig. \\ref{fig:delta_q_L_and_RH_L_theory_2}b). \n\n\n\n\n\n\nBecause relative humidity depends on temperature as well as specific humidity, \nthere is no unique way to use the box model result (\\ref{eqn:new_box_model_3}) to decompose changes in land relative humidity into contributions due to ocean specific humidity and land evapotranspiration. However, a decomposition derived in appendix A (equation \\ref{eqn:appendix_3}) has several desirable properties. According to the decomposition, the contributions to the change in land relative humidity from evapotranspiration and ocean specific humidity are weighted according to their contribution to land specific humidity in the control climate. The change in ocean specific humidity leads to a decrease in land relative humidity if the fractional increase in ocean specific humidity is less than the fractional increase in saturation specific humidity over land. Similarly, evapotranspiration contributes to a decrease in land relative humidity if the fractional increase in $q_{\\rm{E}}$ is less than the fractional increase in saturation specific humidity over land.  (Note that the fractional change in $q_{\\rm{E}}$ is generally different from the fractional change in evapotranspiration.)  \n\nUsing this decomposition of the change in land relative humidity, we find that the land evapotranspiration contribution is of comparable importance to the ocean specific humidity contribution  for the idealized GCM simulations\n(Fig. \\ref{fig:delta_q_L_and_RH_L_theory_2}b). By contrast, we found that the\ncontribution of ocean specific humidity was more important than land evapotranspiration when land specific humidity changes were considered.  The discrepancy arises because, according to the decomposition (\\ref{eqn:appendix_3}), it is not the magnitude of a particular contribution to the change in specific humidity that matters for its contribution to the change in relative humidity, but rather how its fractional changes compare to the fractional changes in saturation specific humidity and how much it contributes to the land specific humidity in the control climate. \n\n\n\n\\subsection{Influence of evapotranspiration in CMIP5 simulations}\n\nWe now investigate how land evapotranspiration contributes to specific humidity changes in the CMIP5 simulations. \n\nWe need to estimate both $\\gamma$ and $q_{\\rm{E}}$ for the full box model, but there are no CMIP5 simulations analogous to the zero-evapotranspiration simulations with the idealized GCM described above. Instead, we estimate the influence of evapotranspiration and ocean specific humidity on land specific humidity using a multiple linear regression approach based on the intermodel scatter across the CMIP5 models. We use the regression relationship\n\n\\begin{equation}\n\\delta q_{\\rm{L}} = \\gamma \\delta q_{\\rm{O}} + \\epsilon \\delta E_{\\rm{L}} + \\zeta, \n\\label{eqn:regression}\n\\end{equation}\n\nwhich is motivated by the full box model (\\ref{eqn:new_box_model_2}), but note that $\\epsilon \\delta E_{\\rm{L}}$ will not generally equal $\\delta q_{\\rm{E}}$ to the extent that parameters such as $\\tau_1$ and $\\tau_2$ change with climate, and changes in these parameters will contribute to the remainder term $\\zeta$.  The variables $\\delta q_{\\rm{L}},\\delta q_{\\rm{O}}$ and  $\\delta E_{\\rm{L}}$ are identified as the zonal and time mean for each latitude and month of the year in each model.  The regression coefficients $\\gamma$, $\\epsilon$, and $\\zeta$ are then estimated using ordinary least squares regression for each latitude and month of the year. The annual means are shown for $\\gamma$ and $\\epsilon$ in Fig.~\\ref{fig:gamma_eps_zeta_vs_lat} and for $\\zeta$ in Fig.~\\ref{fig:delta_SH_land_CMIP5_theory_2}. The regression coefficient $\\gamma$ has a similar magnitude and latitudinal structure to the $\\gamma$ parameter calculated for the ocean-influence box model (Fig.~\\ref{fig:gamma_eps_zeta_vs_lat}a). The coefficient $\\epsilon$ is positive at all latitudes (Fig.~\\ref{fig:gamma_eps_zeta_vs_lat}b) indicating that enhanced evapotranspiration increases the land specific humidity, while the remainder term $\\zeta$ (Fig.~\\ref{fig:delta_SH_land_CMIP5_theory_2}) is negative at most latitudes. \n\n\n\n\\begin{figure}[t]\n  \\centering\n\n   \\noindent\\includegraphics[width=19pc,angle=0]{delta_SH_land_and_estimates_vs_lat_theory_2_v4.eps}\n  \\caption{Multimodel-mean changes in zonal- and time-mean surface-air land specific humidity (solid black line) in the CMIP5 simulations, and the contributions to these changes due to ocean specific humidity changes ($\\gamma \\delta q_{\\rm{O}}$; blue dashed line), land evapotranspiration changes ($\\epsilon \\delta E_{\\rm{L}}$; green dashed line), and the remainder term ($\\zeta$; red dashed line) as estimated using the regression relation (\\ref{eqn:regression}).}\n\\label{fig:delta_SH_land_CMIP5_theory_2}\n\\end{figure}\n\n\n\n\n\n\n\n\n\n\n\n\nBy construction, the regression relationship (\\ref{eqn:regression}) is exactly satisfied in the multimodel mean. Based on this relationship, the annual-mean contributions to changes in land specific humidity from changes in ocean specific humidity, changes in land evapotranspiration, and the remainder term are shown in Fig. \\ref{fig:delta_SH_land_CMIP5_theory_2}. At all latitudes, changes in land specific humidity are dominated by the ocean specific humidity contribution. The contribution due to changes in land evapotranspiration is positive and has its largest values in the Northern Hemisphere where the land fraction is greatest. \n\nIt is not possible to estimate the individual contributions to changes in land relative humidity for the CMIP5 simulations, as we did for the idealized GCM simulations. This is because the decomposition of relative humidity changes discussed in appendix A involves the individual contributions to land specific humidity in the control climate, and these are difficult to calculate using a regression approach. However, the results from the idealized GCM simulations suggest that evapotranspiration could be important for the changes in land relative humidity in the CMIP5 simulations, even though it is a second order influence for changes in land specific humidity. It would be worthwhile to estimate the land evapotranspiration contribution for full-complexity GCMs by performing simulations with specified land evapotranspiration rates as was done for the idealized GCM in this study.\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n4.2 Influence of evapotranspiration in CMIP5 simulations\n\\subsection{Influence of evapotranspiration in CMIP5 simulations}\n\nWe now investigate how land evapotranspiration contributes to specific humidity changes in the CMIP5 simulations. \n\nWe need to estimate both $\\gamma$ and $q_{\\rm{E}}$ for the full box model, but there are no CMIP5 simulations analogous to the zero-evapotranspiration simulations with the idealized GCM described above. Instead, we estimate the influence of evapotranspiration and ocean specific humidity on land specific humidity using a multiple linear regression approach based on the intermodel scatter across the CMIP5 models. We use the regression relationship\n\n\\begin{equation}\n\\delta q_{\\rm{L}} = \\gamma \\delta q_{\\rm{O}} + \\epsilon \\delta E_{\\rm{L}} + \\zeta, \n\\label{eqn:regression}\n\\end{equation}\n\nwhich is motivated by the full box model (\\ref{eqn:new_box_model_2}), but note that $\\epsilon \\delta E_{\\rm{L}}$ will not generally equal $\\delta q_{\\rm{E}}$ to the extent that parameters such as $\\tau_1$ and $\\tau_2$ change with climate, and changes in these parameters will contribute to the remainder term $\\zeta$.  The variables $\\delta q_{\\rm{L}},\\delta q_{\\rm{O}}$ and  $\\delta E_{\\rm{L}}$ are identified as the zonal and time mean for each latitude and month of the year in each model.  The regression coefficients $\\gamma$, $\\epsilon$, and $\\zeta$ are then estimated using ordinary least squares regression for each latitude and month of the year. The annual means are shown for $\\gamma$ and $\\epsilon$ in Fig.~\\ref{fig:gamma_eps_zeta_vs_lat} and for $\\zeta$ in Fig.~\\ref{fig:delta_SH_land_CMIP5_theory_2}. The regression coefficient $\\gamma$ has a similar magnitude and latitudinal structure to the $\\gamma$ parameter calculated for the ocean-influence box model (Fig.~\\ref{fig:gamma_eps_zeta_vs_lat}a). The coefficient $\\epsilon$ is positive at all latitudes (Fig.~\\ref{fig:gamma_eps_zeta_vs_lat}b) indicating that enhanced evapotranspiration increases the land specific humidity, while the remainder term $\\zeta$ (Fig.~\\ref{fig:delta_SH_land_CMIP5_theory_2}) is negative at most latitudes. \n\n\n\n\\begin{figure}[t]\n  \\centering\n\n   \\noindent\\includegraphics[width=19pc,angle=0]{delta_SH_land_and_estimates_vs_lat_theory_2_v4.eps}\n  \\caption{Multimodel-mean changes in zonal- and time-mean surface-air land specific humidity (solid black line) in the CMIP5 simulations, and the contributions to these changes due to ocean specific humidity changes ($\\gamma \\delta q_{\\rm{O}}$; blue dashed line), land evapotranspiration changes ($\\epsilon \\delta E_{\\rm{L}}$; green dashed line), and the remainder term ($\\zeta$; red dashed line) as estimated using the regression relation (\\ref{eqn:regression}).}\n\\label{fig:delta_SH_land_CMIP5_theory_2}\n\\end{figure}\n\n\n\n\n\n\n\n\n\n\n\n\nBy construction, the regression relationship (\\ref{eqn:regression}) is exactly satisfied in the multimodel mean. Based on this relationship, the annual-mean contributions to changes in land specific humidity from changes in ocean specific humidity, changes in land evapotranspiration, and the remainder term are shown in Fig. \\ref{fig:delta_SH_land_CMIP5_theory_2}. At all latitudes, changes in land specific humidity are dominated by the ocean specific humidity contribution. The contribution due to changes in land evapotranspiration is positive and has its largest values in the Northern Hemisphere where the land fraction is greatest. \n\nIt is not possible to estimate the individual contributions to changes in land relative humidity for the CMIP5 simulations, as we did for the idealized GCM simulations. This is because the decomposition of relative humidity changes discussed in appendix A involves the individual contributions to land specific humidity in the control climate, and these are difficult to calculate using a regression approach. However, the results from the idealized GCM simulations suggest that evapotranspiration could be important for the changes in land relative humidity in the CMIP5 simulations, even though it is a second order influence for changes in land specific humidity. It would be worthwhile to estimate the land evapotranspiration contribution for full-complexity GCMs by performing simulations with specified land evapotranspiration rates as was done for the idealized GCM in this study.\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n",
        "label": "fig:gamma_eps_zeta_vs_lat",
        "Descriptive_question1": "What is the global, annual, and multimodel mean value of the gamma parameter in the ocean-influence box model as shown in figure_7?",
        "Descriptive_question2": "What color is used for the solid line representing the gamma parameter for the ocean-influence box model in figure_7?",
        "Reasoning_question1": "How does the gamma parameter for the ocean-influence box model compare to the regression approach including evapotranspiration across different latitudes in figure_7, and what might explain any observed differences?",
        "Reasoning_question2": "What trends can be inferred about the regression coefficient epsilon across latitudes in figure_7, and how might these trends relate to the influence of evapotranspiration on land specific humidity?",
        "Descriptive_answer1": "0.74",
        "Descriptive_answer2": "black",
        "Reasoning_answer1": "In figure_7(a), the solid black line represents the gamma parameter for the ocean-influence box model, while the dashed black line represents the gamma parameter estimated from the regression approach that includes evapotranspiration. Across different latitudes, the gamma parameter values from the ocean-influence box model are generally similar but slightly differ from those obtained from the regression approach. These differences can be attributed to the fact that the ocean-influence box model neglects evapotranspiration, whereas the regression approach accounts for it, thereby capturing additional processes influencing land specific humidity and leading to variations in gamma.",
        "Reasoning_answer2": "Figure_7(b) shows the regression coefficient epsilon, which is positive across all latitudes, indicating a consistent positive influence of evapotranspiration on land specific humidity. The value of epsilon varies with latitude, generally showing higher values where land fraction and evapotranspiration are greater (likely in mid-latitudes). This trend suggests that evapotranspiration contributes to increasing land specific humidity, and its impact varies regionally depending on the local land characteristics and evapotranspiration rates."
    },
    {
        "paper_id": "1605.00380.json",
        "image_id": "figure_8",
        "image_path": [
            "/home/yz979/scratch/chengye/scimolmo/test-images-tables/dataset_imagestables/1605.00380/images/delta_q_L_and_delta_RH_L_vs_T_ocean_iGCM_sim_and_theory_2_nonlinear_new.eps"
        ],
        "caption": "As in Figure \\ref{fig:delta_q_L_and_RH_L_theory_1}, but here showing estimates of the surface-air specific and relative humidity changes from the full box model (\\ref{eqn:new_box_model_3}). The contributions due to ocean specific humidity changes (blue dashed lines) and evapotranspiration changes (green dashed lines) are also shown. The contributions to changes in relative humidity are calculated using (\\ref{eqn:appendix_3}). Pseudo relative humidities are shown in this figure (the changes in actual relative humidity are shown by the blue line in Fig.~\\ref{fig:delta_q_L_and_RH_L_theory_1}b.)",
        "classify": "Chart",
        "section_info": "4 Influence of evapotranspiration\n\\section{Influence of evapotranspiration}\n\\label{sect:full_model_RH_land}\n\n\n\n\nThe ocean-influence box model captures much (but not all) of the behavior in vastly more complex GCMs. However, the moisture balance of the land boundary layer is also affected by evapotranspiration, and changes in land surface properties, such as soil moisture or stomatal conductance, can affect evapotranspiration in the absence of any changes in the overlying atmosphere.  For example, changes in stomatal conductance under elevated $\\rm{CO}_2$ conditions has been shown to reduce both evapotranspiration and land relative humidity without changes in ocean humidity \\citep{andrews_et_al_2011}. \n\n\n\n\\begin{figure}[t]\n  \\centering\n  \\noindent\\includegraphics[width=19pc,angle=0]{delta_q_L_and_delta_RH_L_vs_T_ocean_iGCM_sim_and_theory_2_nonlinear_new.eps}\n  \\caption{As in Figure \\ref{fig:delta_q_L_and_RH_L_theory_1}, but here showing estimates of the surface-air specific and relative humidity changes from the full box model (\\ref{eqn:new_box_model_3}). The contributions due to ocean specific humidity changes (blue dashed lines) and evapotranspiration changes (green dashed lines) are also shown. The contributions to changes in relative humidity are calculated using (\\ref{eqn:appendix_3}). Pseudo relative humidities are shown in this figure (the changes in actual relative humidity are shown by the blue line in Fig.~\\ref{fig:delta_q_L_and_RH_L_theory_1}b.)}\n\\label{fig:delta_q_L_and_RH_L_theory_2}\n\\end{figure}\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\nWe turn to the full box model (\\ref{eqn:new_box_model_2}) which includes\nthe effects of evapotranspiration. We assume once more that changes in $\\gamma$ are negligible, \n\nsuch that:\n\n\\begin{equation}\n\\delta q_{\\rm{L}} \\approx \\gamma \\delta q_{\\rm{O}} + \\delta q_{\\rm{E}}.\n\\label{eqn:new_box_model_3}\n\\end{equation}\n\nThere are two terms contributing to changes in $q_{\\rm{L}}$ in (\\ref{eqn:new_box_model_3}): the term arising from changes in ocean specific humidity, $\\gamma \\delta q_{\\rm{O}}$, and an additional land evapotranspiration term, $\\delta q_{\\rm{E}}$. We next assess their relative importance in controlling changes in land humidity in the idealized GCM and CMIP5 simulations.\n\n\n\\subsection{Application of full box model to idealized GCM simulations}\n\\label{sect:theory_2_simulation_analysis}\n\nWe first examine the idealized GCM simulations with a subtropical continent. In contrast to the ocean-influence box model (\\ref{eqn:box_model_3}), for which the single parameter $\\gamma$ could be easily estimated in the control simulation in each case, the full model (\\ref{eqn:new_box_model_3}) has two parameters to be estimated, $\\gamma$ and $q_{\\rm{E}}$. To estimate these parameters, we perform an additional set of simulations with the same longwave optical thicknesses as in the 10 simulations described previously but with the evapotranspiration set to zero over land. \n\nSpecifying the land evapotranspiration in this way is analogous to drying out the soil. (Note that the change in evapotranspiration affects both the humidity of the atmosphere and the surface energy balance.)\nUsing these additional simulations with $E_{\\rm{L}} = 0$, we can estimate $\\gamma$ for each climate using (\\ref{eqn:new_box_model_2}): $\\gamma = q_{\\rm{L},E_{\\rm{L}}=0} / q_{\\rm{O},E_{\\rm{L}}=0}$.  The $\\gamma$ values obtained are smaller than those calculated from the control climate for the ocean-influence box model (Fig. \\ref{fig:gamma_and_epsilon}a) because the contribution of evapotranspiration to the land specific humidity is now also taken into account.  We then use these $\\gamma$ values to estimate $q_{\\rm{E}}$ for the original simulations with dynamic land surface hydrology \n: $q_{\\rm{E}} = q_{\\rm{L}} - \\gamma q_{\\rm{O}}$. The values of $q_{\\rm{E}}$ increase with warming except in hot climates (Fig. \\ref{fig:gamma_and_epsilon}b).\n\nInterestingly, the influence of evapotranspiration on land specific humidity as measured by $q_{\\rm{E}}$ roughly scales with the land specific humidity except in hot climates (compare the dashed black and solid red lines in Fig. \\ref{fig:gamma_and_epsilon}b), and this helps to explain why the ocean-influence box model\nis accurate even though evapotranspiration affects land specific humidity.\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\nWe then estimate the changes in land specific humidity between pairs of nearest-neighbor simulations from (\\ref{eqn:new_box_model_3}) with $\\gamma$ assumed to be constant as the climate changes.  The simulated and estimated changes in surface-air land specific humidity, along with the contributions due to changes in ocean specific humidity and land evapotranspiration are shown in Fig. \\ref{fig:delta_q_L_and_RH_L_theory_2}a. The full box model captures the basic behavior of the land specific humidity changes as a function of temperature, although it is less accurate in hot climates. The contribution from ocean specific humidity changes, $\\gamma \\delta q_{\\rm{O}}$, is larger than the contribution from land evapotranspiration for all climates (Fig. \\ref{fig:delta_q_L_and_RH_L_theory_2}a).  The changes in simulated land (pseudo) relative humidity are also well captured by the full box model (Fig. \\ref{fig:delta_q_L_and_RH_L_theory_2}b). \n\n\n\n\n\n\nBecause relative humidity depends on temperature as well as specific humidity, \nthere is no unique way to use the box model result (\\ref{eqn:new_box_model_3}) to decompose changes in land relative humidity into contributions due to ocean specific humidity and land evapotranspiration. However, a decomposition derived in appendix A (equation \\ref{eqn:appendix_3}) has several desirable properties. According to the decomposition, the contributions to the change in land relative humidity from evapotranspiration and ocean specific humidity are weighted according to their contribution to land specific humidity in the control climate. The change in ocean specific humidity leads to a decrease in land relative humidity if the fractional increase in ocean specific humidity is less than the fractional increase in saturation specific humidity over land. Similarly, evapotranspiration contributes to a decrease in land relative humidity if the fractional increase in $q_{\\rm{E}}$ is less than the fractional increase in saturation specific humidity over land.  (Note that the fractional change in $q_{\\rm{E}}$ is generally different from the fractional change in evapotranspiration.)  \n\nUsing this decomposition of the change in land relative humidity, we find that the land evapotranspiration contribution is of comparable importance to the ocean specific humidity contribution  for the idealized GCM simulations\n(Fig. \\ref{fig:delta_q_L_and_RH_L_theory_2}b). By contrast, we found that the\ncontribution of ocean specific humidity was more important than land evapotranspiration when land specific humidity changes were considered.  The discrepancy arises because, according to the decomposition (\\ref{eqn:appendix_3}), it is not the magnitude of a particular contribution to the change in specific humidity that matters for its contribution to the change in relative humidity, but rather how its fractional changes compare to the fractional changes in saturation specific humidity and how much it contributes to the land specific humidity in the control climate. \n\n\n\n\\subsection{Influence of evapotranspiration in CMIP5 simulations}\n\nWe now investigate how land evapotranspiration contributes to specific humidity changes in the CMIP5 simulations. \n\nWe need to estimate both $\\gamma$ and $q_{\\rm{E}}$ for the full box model, but there are no CMIP5 simulations analogous to the zero-evapotranspiration simulations with the idealized GCM described above. Instead, we estimate the influence of evapotranspiration and ocean specific humidity on land specific humidity using a multiple linear regression approach based on the intermodel scatter across the CMIP5 models. We use the regression relationship\n\n\\begin{equation}\n\\delta q_{\\rm{L}} = \\gamma \\delta q_{\\rm{O}} + \\epsilon \\delta E_{\\rm{L}} + \\zeta, \n\\label{eqn:regression}\n\\end{equation}\n\nwhich is motivated by the full box model (\\ref{eqn:new_box_model_2}), but note that $\\epsilon \\delta E_{\\rm{L}}$ will not generally equal $\\delta q_{\\rm{E}}$ to the extent that parameters such as $\\tau_1$ and $\\tau_2$ change with climate, and changes in these parameters will contribute to the remainder term $\\zeta$.  The variables $\\delta q_{\\rm{L}},\\delta q_{\\rm{O}}$ and  $\\delta E_{\\rm{L}}$ are identified as the zonal and time mean for each latitude and month of the year in each model.  The regression coefficients $\\gamma$, $\\epsilon$, and $\\zeta$ are then estimated using ordinary least squares regression for each latitude and month of the year. The annual means are shown for $\\gamma$ and $\\epsilon$ in Fig.~\\ref{fig:gamma_eps_zeta_vs_lat} and for $\\zeta$ in Fig.~\\ref{fig:delta_SH_land_CMIP5_theory_2}. The regression coefficient $\\gamma$ has a similar magnitude and latitudinal structure to the $\\gamma$ parameter calculated for the ocean-influence box model (Fig.~\\ref{fig:gamma_eps_zeta_vs_lat}a). The coefficient $\\epsilon$ is positive at all latitudes (Fig.~\\ref{fig:gamma_eps_zeta_vs_lat}b) indicating that enhanced evapotranspiration increases the land specific humidity, while the remainder term $\\zeta$ (Fig.~\\ref{fig:delta_SH_land_CMIP5_theory_2}) is negative at most latitudes. \n\n\n\n\\begin{figure}[t]\n  \\centering\n\n   \\noindent\\includegraphics[width=19pc,angle=0]{delta_SH_land_and_estimates_vs_lat_theory_2_v4.eps}\n  \\caption{Multimodel-mean changes in zonal- and time-mean surface-air land specific humidity (solid black line) in the CMIP5 simulations, and the contributions to these changes due to ocean specific humidity changes ($\\gamma \\delta q_{\\rm{O}}$; blue dashed line), land evapotranspiration changes ($\\epsilon \\delta E_{\\rm{L}}$; green dashed line), and the remainder term ($\\zeta$; red dashed line) as estimated using the regression relation (\\ref{eqn:regression}).}\n\\label{fig:delta_SH_land_CMIP5_theory_2}\n\\end{figure}\n\n\n\n\n\n\n\n\n\n\n\n\nBy construction, the regression relationship (\\ref{eqn:regression}) is exactly satisfied in the multimodel mean. Based on this relationship, the annual-mean contributions to changes in land specific humidity from changes in ocean specific humidity, changes in land evapotranspiration, and the remainder term are shown in Fig. \\ref{fig:delta_SH_land_CMIP5_theory_2}. At all latitudes, changes in land specific humidity are dominated by the ocean specific humidity contribution. The contribution due to changes in land evapotranspiration is positive and has its largest values in the Northern Hemisphere where the land fraction is greatest. \n\nIt is not possible to estimate the individual contributions to changes in land relative humidity for the CMIP5 simulations, as we did for the idealized GCM simulations. This is because the decomposition of relative humidity changes discussed in appendix A involves the individual contributions to land specific humidity in the control climate, and these are difficult to calculate using a regression approach. However, the results from the idealized GCM simulations suggest that evapotranspiration could be important for the changes in land relative humidity in the CMIP5 simulations, even though it is a second order influence for changes in land specific humidity. It would be worthwhile to estimate the land evapotranspiration contribution for full-complexity GCMs by performing simulations with specified land evapotranspiration rates as was done for the idealized GCM in this study.\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n4.1 Application of full box model to idealized GCM simulations\n\\subsection{Application of full box model to idealized GCM simulations}\n\\label{sect:theory_2_simulation_analysis}\n\nWe first examine the idealized GCM simulations with a subtropical continent. In contrast to the ocean-influence box model (\\ref{eqn:box_model_3}), for which the single parameter $\\gamma$ could be easily estimated in the control simulation in each case, the full model (\\ref{eqn:new_box_model_3}) has two parameters to be estimated, $\\gamma$ and $q_{\\rm{E}}$. To estimate these parameters, we perform an additional set of simulations with the same longwave optical thicknesses as in the 10 simulations described previously but with the evapotranspiration set to zero over land. \n\nSpecifying the land evapotranspiration in this way is analogous to drying out the soil. (Note that the change in evapotranspiration affects both the humidity of the atmosphere and the surface energy balance.)\nUsing these additional simulations with $E_{\\rm{L}} = 0$, we can estimate $\\gamma$ for each climate using (\\ref{eqn:new_box_model_2}): $\\gamma = q_{\\rm{L},E_{\\rm{L}}=0} / q_{\\rm{O},E_{\\rm{L}}=0}$.  The $\\gamma$ values obtained are smaller than those calculated from the control climate for the ocean-influence box model (Fig. \\ref{fig:gamma_and_epsilon}a) because the contribution of evapotranspiration to the land specific humidity is now also taken into account.  We then use these $\\gamma$ values to estimate $q_{\\rm{E}}$ for the original simulations with dynamic land surface hydrology \n: $q_{\\rm{E}} = q_{\\rm{L}} - \\gamma q_{\\rm{O}}$. The values of $q_{\\rm{E}}$ increase with warming except in hot climates (Fig. \\ref{fig:gamma_and_epsilon}b).\n\nInterestingly, the influence of evapotranspiration on land specific humidity as measured by $q_{\\rm{E}}$ roughly scales with the land specific humidity except in hot climates (compare the dashed black and solid red lines in Fig. \\ref{fig:gamma_and_epsilon}b), and this helps to explain why the ocean-influence box model\nis accurate even though evapotranspiration affects land specific humidity.\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\nWe then estimate the changes in land specific humidity between pairs of nearest-neighbor simulations from (\\ref{eqn:new_box_model_3}) with $\\gamma$ assumed to be constant as the climate changes.  The simulated and estimated changes in surface-air land specific humidity, along with the contributions due to changes in ocean specific humidity and land evapotranspiration are shown in Fig. \\ref{fig:delta_q_L_and_RH_L_theory_2}a. The full box model captures the basic behavior of the land specific humidity changes as a function of temperature, although it is less accurate in hot climates. The contribution from ocean specific humidity changes, $\\gamma \\delta q_{\\rm{O}}$, is larger than the contribution from land evapotranspiration for all climates (Fig. \\ref{fig:delta_q_L_and_RH_L_theory_2}a).  The changes in simulated land (pseudo) relative humidity are also well captured by the full box model (Fig. \\ref{fig:delta_q_L_and_RH_L_theory_2}b). \n\n\n\n\n\n\nBecause relative humidity depends on temperature as well as specific humidity, \nthere is no unique way to use the box model result (\\ref{eqn:new_box_model_3}) to decompose changes in land relative humidity into contributions due to ocean specific humidity and land evapotranspiration. However, a decomposition derived in appendix A (equation \\ref{eqn:appendix_3}) has several desirable properties. According to the decomposition, the contributions to the change in land relative humidity from evapotranspiration and ocean specific humidity are weighted according to their contribution to land specific humidity in the control climate. The change in ocean specific humidity leads to a decrease in land relative humidity if the fractional increase in ocean specific humidity is less than the fractional increase in saturation specific humidity over land. Similarly, evapotranspiration contributes to a decrease in land relative humidity if the fractional increase in $q_{\\rm{E}}$ is less than the fractional increase in saturation specific humidity over land.  (Note that the fractional change in $q_{\\rm{E}}$ is generally different from the fractional change in evapotranspiration.)  \n\nUsing this decomposition of the change in land relative humidity, we find that the land evapotranspiration contribution is of comparable importance to the ocean specific humidity contribution  for the idealized GCM simulations\n(Fig. \\ref{fig:delta_q_L_and_RH_L_theory_2}b). By contrast, we found that the\ncontribution of ocean specific humidity was more important than land evapotranspiration when land specific humidity changes were considered.  The discrepancy arises because, according to the decomposition (\\ref{eqn:appendix_3}), it is not the magnitude of a particular contribution to the change in specific humidity that matters for its contribution to the change in relative humidity, but rather how its fractional changes compare to the fractional changes in saturation specific humidity and how much it contributes to the land specific humidity in the control climate. \n\n\n\n",
        "label": "fig:delta_q_L_and_RH_L_theory_2",
        "Descriptive_question1": "What color represents the contributions due to ocean specific humidity changes in figure_8?",
        "Descriptive_question2": "What type of lines are used for evapotranspiration changes in figure_8?",
        "Reasoning_question1": "How does the contribution of ocean specific humidity compare to evapotranspiration in affecting land specific humidity changes across different climates in figure_8?",
        "Reasoning_question2": "Why might the land evapotranspiration contribution be of comparable importance to ocean specific humidity for changes in land relative humidity, despite being less significant for specific humidity changes in figure_8?",
        "Descriptive_answer1": "blue",
        "Descriptive_answer2": "green dashed",
        "Reasoning_answer1": "The figure shows that the contribution from ocean specific humidity changes (blue dashed lines) is consistently larger than the contribution from land evapotranspiration changes (green dashed lines) across all climates, indicating ocean humidity has a dominant effect on land specific humidity changes.",
        "Reasoning_answer2": "Although evapotranspiration contributes less to changes in land specific humidity compared to ocean humidity, its fractional changes relative to saturation specific humidity and its weighting based on the control climate's land specific humidity make its impact on land relative humidity comparable to ocean humidity. This is because relative humidity depends on both specific humidity and temperature, so contributions to relative humidity changes depend on fractional changes and baseline contributions, not just absolute magnitude."
    },
    {
        "paper_id": "1605.00380.json",
        "image_id": "figure_9",
        "image_path": [
            "/home/yz979/scratch/chengye/scimolmo/test-images-tables/dataset_imagestables/1605.00380/images/delta_SH_land_and_estimates_vs_lat_theory_2_v4.eps"
        ],
        "caption": "Multimodel-mean changes in zonal- and time-mean surface-air land specific humidity (solid black line) in the CMIP5 simulations, and the contributions to these changes due to ocean specific humidity changes ($\\gamma \\delta q_{\\rm{O}}$; blue dashed line), land evapotranspiration changes ($\\epsilon \\delta E_{\\rm{L}}$; green dashed line), and the remainder term ($\\zeta$; red dashed line) as estimated using the regression relation (\\ref{eqn:regression}).",
        "classify": "Chart",
        "section_info": "4 Influence of evapotranspiration\n\\section{Influence of evapotranspiration}\n\\label{sect:full_model_RH_land}\n\n\n\n\nThe ocean-influence box model captures much (but not all) of the behavior in vastly more complex GCMs. However, the moisture balance of the land boundary layer is also affected by evapotranspiration, and changes in land surface properties, such as soil moisture or stomatal conductance, can affect evapotranspiration in the absence of any changes in the overlying atmosphere.  For example, changes in stomatal conductance under elevated $\\rm{CO}_2$ conditions has been shown to reduce both evapotranspiration and land relative humidity without changes in ocean humidity \\citep{andrews_et_al_2011}. \n\n\n\n\\begin{figure}[t]\n  \\centering\n  \\noindent\\includegraphics[width=19pc,angle=0]{delta_q_L_and_delta_RH_L_vs_T_ocean_iGCM_sim_and_theory_2_nonlinear_new.eps}\n  \\caption{As in Figure \\ref{fig:delta_q_L_and_RH_L_theory_1}, but here showing estimates of the surface-air specific and relative humidity changes from the full box model (\\ref{eqn:new_box_model_3}). The contributions due to ocean specific humidity changes (blue dashed lines) and evapotranspiration changes (green dashed lines) are also shown. The contributions to changes in relative humidity are calculated using (\\ref{eqn:appendix_3}). Pseudo relative humidities are shown in this figure (the changes in actual relative humidity are shown by the blue line in Fig.~\\ref{fig:delta_q_L_and_RH_L_theory_1}b.)}\n\\label{fig:delta_q_L_and_RH_L_theory_2}\n\\end{figure}\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\nWe turn to the full box model (\\ref{eqn:new_box_model_2}) which includes\nthe effects of evapotranspiration. We assume once more that changes in $\\gamma$ are negligible, \n\nsuch that:\n\n\\begin{equation}\n\\delta q_{\\rm{L}} \\approx \\gamma \\delta q_{\\rm{O}} + \\delta q_{\\rm{E}}.\n\\label{eqn:new_box_model_3}\n\\end{equation}\n\nThere are two terms contributing to changes in $q_{\\rm{L}}$ in (\\ref{eqn:new_box_model_3}): the term arising from changes in ocean specific humidity, $\\gamma \\delta q_{\\rm{O}}$, and an additional land evapotranspiration term, $\\delta q_{\\rm{E}}$. We next assess their relative importance in controlling changes in land humidity in the idealized GCM and CMIP5 simulations.\n\n\n\\subsection{Application of full box model to idealized GCM simulations}\n\\label{sect:theory_2_simulation_analysis}\n\nWe first examine the idealized GCM simulations with a subtropical continent. In contrast to the ocean-influence box model (\\ref{eqn:box_model_3}), for which the single parameter $\\gamma$ could be easily estimated in the control simulation in each case, the full model (\\ref{eqn:new_box_model_3}) has two parameters to be estimated, $\\gamma$ and $q_{\\rm{E}}$. To estimate these parameters, we perform an additional set of simulations with the same longwave optical thicknesses as in the 10 simulations described previously but with the evapotranspiration set to zero over land. \n\nSpecifying the land evapotranspiration in this way is analogous to drying out the soil. (Note that the change in evapotranspiration affects both the humidity of the atmosphere and the surface energy balance.)\nUsing these additional simulations with $E_{\\rm{L}} = 0$, we can estimate $\\gamma$ for each climate using (\\ref{eqn:new_box_model_2}): $\\gamma = q_{\\rm{L},E_{\\rm{L}}=0} / q_{\\rm{O},E_{\\rm{L}}=0}$.  The $\\gamma$ values obtained are smaller than those calculated from the control climate for the ocean-influence box model (Fig. \\ref{fig:gamma_and_epsilon}a) because the contribution of evapotranspiration to the land specific humidity is now also taken into account.  We then use these $\\gamma$ values to estimate $q_{\\rm{E}}$ for the original simulations with dynamic land surface hydrology \n: $q_{\\rm{E}} = q_{\\rm{L}} - \\gamma q_{\\rm{O}}$. The values of $q_{\\rm{E}}$ increase with warming except in hot climates (Fig. \\ref{fig:gamma_and_epsilon}b).\n\nInterestingly, the influence of evapotranspiration on land specific humidity as measured by $q_{\\rm{E}}$ roughly scales with the land specific humidity except in hot climates (compare the dashed black and solid red lines in Fig. \\ref{fig:gamma_and_epsilon}b), and this helps to explain why the ocean-influence box model\nis accurate even though evapotranspiration affects land specific humidity.\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\nWe then estimate the changes in land specific humidity between pairs of nearest-neighbor simulations from (\\ref{eqn:new_box_model_3}) with $\\gamma$ assumed to be constant as the climate changes.  The simulated and estimated changes in surface-air land specific humidity, along with the contributions due to changes in ocean specific humidity and land evapotranspiration are shown in Fig. \\ref{fig:delta_q_L_and_RH_L_theory_2}a. The full box model captures the basic behavior of the land specific humidity changes as a function of temperature, although it is less accurate in hot climates. The contribution from ocean specific humidity changes, $\\gamma \\delta q_{\\rm{O}}$, is larger than the contribution from land evapotranspiration for all climates (Fig. \\ref{fig:delta_q_L_and_RH_L_theory_2}a).  The changes in simulated land (pseudo) relative humidity are also well captured by the full box model (Fig. \\ref{fig:delta_q_L_and_RH_L_theory_2}b). \n\n\n\n\n\n\nBecause relative humidity depends on temperature as well as specific humidity, \nthere is no unique way to use the box model result (\\ref{eqn:new_box_model_3}) to decompose changes in land relative humidity into contributions due to ocean specific humidity and land evapotranspiration. However, a decomposition derived in appendix A (equation \\ref{eqn:appendix_3}) has several desirable properties. According to the decomposition, the contributions to the change in land relative humidity from evapotranspiration and ocean specific humidity are weighted according to their contribution to land specific humidity in the control climate. The change in ocean specific humidity leads to a decrease in land relative humidity if the fractional increase in ocean specific humidity is less than the fractional increase in saturation specific humidity over land. Similarly, evapotranspiration contributes to a decrease in land relative humidity if the fractional increase in $q_{\\rm{E}}$ is less than the fractional increase in saturation specific humidity over land.  (Note that the fractional change in $q_{\\rm{E}}$ is generally different from the fractional change in evapotranspiration.)  \n\nUsing this decomposition of the change in land relative humidity, we find that the land evapotranspiration contribution is of comparable importance to the ocean specific humidity contribution  for the idealized GCM simulations\n(Fig. \\ref{fig:delta_q_L_and_RH_L_theory_2}b). By contrast, we found that the\ncontribution of ocean specific humidity was more important than land evapotranspiration when land specific humidity changes were considered.  The discrepancy arises because, according to the decomposition (\\ref{eqn:appendix_3}), it is not the magnitude of a particular contribution to the change in specific humidity that matters for its contribution to the change in relative humidity, but rather how its fractional changes compare to the fractional changes in saturation specific humidity and how much it contributes to the land specific humidity in the control climate. \n\n\n\n\\subsection{Influence of evapotranspiration in CMIP5 simulations}\n\nWe now investigate how land evapotranspiration contributes to specific humidity changes in the CMIP5 simulations. \n\nWe need to estimate both $\\gamma$ and $q_{\\rm{E}}$ for the full box model, but there are no CMIP5 simulations analogous to the zero-evapotranspiration simulations with the idealized GCM described above. Instead, we estimate the influence of evapotranspiration and ocean specific humidity on land specific humidity using a multiple linear regression approach based on the intermodel scatter across the CMIP5 models. We use the regression relationship\n\n\\begin{equation}\n\\delta q_{\\rm{L}} = \\gamma \\delta q_{\\rm{O}} + \\epsilon \\delta E_{\\rm{L}} + \\zeta, \n\\label{eqn:regression}\n\\end{equation}\n\nwhich is motivated by the full box model (\\ref{eqn:new_box_model_2}), but note that $\\epsilon \\delta E_{\\rm{L}}$ will not generally equal $\\delta q_{\\rm{E}}$ to the extent that parameters such as $\\tau_1$ and $\\tau_2$ change with climate, and changes in these parameters will contribute to the remainder term $\\zeta$.  The variables $\\delta q_{\\rm{L}},\\delta q_{\\rm{O}}$ and  $\\delta E_{\\rm{L}}$ are identified as the zonal and time mean for each latitude and month of the year in each model.  The regression coefficients $\\gamma$, $\\epsilon$, and $\\zeta$ are then estimated using ordinary least squares regression for each latitude and month of the year. The annual means are shown for $\\gamma$ and $\\epsilon$ in Fig.~\\ref{fig:gamma_eps_zeta_vs_lat} and for $\\zeta$ in Fig.~\\ref{fig:delta_SH_land_CMIP5_theory_2}. The regression coefficient $\\gamma$ has a similar magnitude and latitudinal structure to the $\\gamma$ parameter calculated for the ocean-influence box model (Fig.~\\ref{fig:gamma_eps_zeta_vs_lat}a). The coefficient $\\epsilon$ is positive at all latitudes (Fig.~\\ref{fig:gamma_eps_zeta_vs_lat}b) indicating that enhanced evapotranspiration increases the land specific humidity, while the remainder term $\\zeta$ (Fig.~\\ref{fig:delta_SH_land_CMIP5_theory_2}) is negative at most latitudes. \n\n\n\n\\begin{figure}[t]\n  \\centering\n\n   \\noindent\\includegraphics[width=19pc,angle=0]{delta_SH_land_and_estimates_vs_lat_theory_2_v4.eps}\n  \\caption{Multimodel-mean changes in zonal- and time-mean surface-air land specific humidity (solid black line) in the CMIP5 simulations, and the contributions to these changes due to ocean specific humidity changes ($\\gamma \\delta q_{\\rm{O}}$; blue dashed line), land evapotranspiration changes ($\\epsilon \\delta E_{\\rm{L}}$; green dashed line), and the remainder term ($\\zeta$; red dashed line) as estimated using the regression relation (\\ref{eqn:regression}).}\n\\label{fig:delta_SH_land_CMIP5_theory_2}\n\\end{figure}\n\n\n\n\n\n\n\n\n\n\n\n\nBy construction, the regression relationship (\\ref{eqn:regression}) is exactly satisfied in the multimodel mean. Based on this relationship, the annual-mean contributions to changes in land specific humidity from changes in ocean specific humidity, changes in land evapotranspiration, and the remainder term are shown in Fig. \\ref{fig:delta_SH_land_CMIP5_theory_2}. At all latitudes, changes in land specific humidity are dominated by the ocean specific humidity contribution. The contribution due to changes in land evapotranspiration is positive and has its largest values in the Northern Hemisphere where the land fraction is greatest. \n\nIt is not possible to estimate the individual contributions to changes in land relative humidity for the CMIP5 simulations, as we did for the idealized GCM simulations. This is because the decomposition of relative humidity changes discussed in appendix A involves the individual contributions to land specific humidity in the control climate, and these are difficult to calculate using a regression approach. However, the results from the idealized GCM simulations suggest that evapotranspiration could be important for the changes in land relative humidity in the CMIP5 simulations, even though it is a second order influence for changes in land specific humidity. It would be worthwhile to estimate the land evapotranspiration contribution for full-complexity GCMs by performing simulations with specified land evapotranspiration rates as was done for the idealized GCM in this study.\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n4.2 Influence of evapotranspiration in CMIP5 simulations\n\\subsection{Influence of evapotranspiration in CMIP5 simulations}\n\nWe now investigate how land evapotranspiration contributes to specific humidity changes in the CMIP5 simulations. \n\nWe need to estimate both $\\gamma$ and $q_{\\rm{E}}$ for the full box model, but there are no CMIP5 simulations analogous to the zero-evapotranspiration simulations with the idealized GCM described above. Instead, we estimate the influence of evapotranspiration and ocean specific humidity on land specific humidity using a multiple linear regression approach based on the intermodel scatter across the CMIP5 models. We use the regression relationship\n\n\\begin{equation}\n\\delta q_{\\rm{L}} = \\gamma \\delta q_{\\rm{O}} + \\epsilon \\delta E_{\\rm{L}} + \\zeta, \n\\label{eqn:regression}\n\\end{equation}\n\nwhich is motivated by the full box model (\\ref{eqn:new_box_model_2}), but note that $\\epsilon \\delta E_{\\rm{L}}$ will not generally equal $\\delta q_{\\rm{E}}$ to the extent that parameters such as $\\tau_1$ and $\\tau_2$ change with climate, and changes in these parameters will contribute to the remainder term $\\zeta$.  The variables $\\delta q_{\\rm{L}},\\delta q_{\\rm{O}}$ and  $\\delta E_{\\rm{L}}$ are identified as the zonal and time mean for each latitude and month of the year in each model.  The regression coefficients $\\gamma$, $\\epsilon$, and $\\zeta$ are then estimated using ordinary least squares regression for each latitude and month of the year. The annual means are shown for $\\gamma$ and $\\epsilon$ in Fig.~\\ref{fig:gamma_eps_zeta_vs_lat} and for $\\zeta$ in Fig.~\\ref{fig:delta_SH_land_CMIP5_theory_2}. The regression coefficient $\\gamma$ has a similar magnitude and latitudinal structure to the $\\gamma$ parameter calculated for the ocean-influence box model (Fig.~\\ref{fig:gamma_eps_zeta_vs_lat}a). The coefficient $\\epsilon$ is positive at all latitudes (Fig.~\\ref{fig:gamma_eps_zeta_vs_lat}b) indicating that enhanced evapotranspiration increases the land specific humidity, while the remainder term $\\zeta$ (Fig.~\\ref{fig:delta_SH_land_CMIP5_theory_2}) is negative at most latitudes. \n\n\n\n\\begin{figure}[t]\n  \\centering\n\n   \\noindent\\includegraphics[width=19pc,angle=0]{delta_SH_land_and_estimates_vs_lat_theory_2_v4.eps}\n  \\caption{Multimodel-mean changes in zonal- and time-mean surface-air land specific humidity (solid black line) in the CMIP5 simulations, and the contributions to these changes due to ocean specific humidity changes ($\\gamma \\delta q_{\\rm{O}}$; blue dashed line), land evapotranspiration changes ($\\epsilon \\delta E_{\\rm{L}}$; green dashed line), and the remainder term ($\\zeta$; red dashed line) as estimated using the regression relation (\\ref{eqn:regression}).}\n\\label{fig:delta_SH_land_CMIP5_theory_2}\n\\end{figure}\n\n\n\n\n\n\n\n\n\n\n\n\nBy construction, the regression relationship (\\ref{eqn:regression}) is exactly satisfied in the multimodel mean. Based on this relationship, the annual-mean contributions to changes in land specific humidity from changes in ocean specific humidity, changes in land evapotranspiration, and the remainder term are shown in Fig. \\ref{fig:delta_SH_land_CMIP5_theory_2}. At all latitudes, changes in land specific humidity are dominated by the ocean specific humidity contribution. The contribution due to changes in land evapotranspiration is positive and has its largest values in the Northern Hemisphere where the land fraction is greatest. \n\nIt is not possible to estimate the individual contributions to changes in land relative humidity for the CMIP5 simulations, as we did for the idealized GCM simulations. This is because the decomposition of relative humidity changes discussed in appendix A involves the individual contributions to land specific humidity in the control climate, and these are difficult to calculate using a regression approach. However, the results from the idealized GCM simulations suggest that evapotranspiration could be important for the changes in land relative humidity in the CMIP5 simulations, even though it is a second order influence for changes in land specific humidity. It would be worthwhile to estimate the land evapotranspiration contribution for full-complexity GCMs by performing simulations with specified land evapotranspiration rates as was done for the idealized GCM in this study.\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n",
        "label": "fig:delta_SH_land_CMIP5_theory_2",
        "Descriptive_question1": "What color represents the ocean specific humidity changes in figure 9?",
        "Descriptive_question2": "Which line style is used for the remainder term in figure 9?",
        "Reasoning_question1": "Why might the ocean specific humidity contribution dominate over land evapotranspiration in influencing land specific humidity changes across all latitudes in figure 9?",
        "Reasoning_question2": "How does the contribution of land evapotranspiration to land specific humidity changes vary between the Northern and Southern Hemispheres in figure 9, and what might explain this variation?",
        "Descriptive_answer1": "blue",
        "Descriptive_answer2": "red dashed line",
        "Reasoning_answer1": "The ocean specific humidity contribution dominates because the ocean covers a vast area and controls the atmospheric moisture content, influencing the baseline humidity that reaches the land surfaces. Since ocean humidity changes directly affect the moisture available to the atmosphere, they have a larger and more direct impact on land specific humidity changes than evapotranspiration, which is limited by land surface properties and varies with vegetation and soil moisture conditions.",
        "Reasoning_answer2": "The contribution of land evapotranspiration to land specific humidity changes is larger in the Northern Hemisphere than in the Southern Hemisphere. This is likely because the Northern Hemisphere has a greater land fraction, providing more area for evapotranspiration to occur, thus influencing land humidity more strongly there compared to the Southern Hemisphere, which has more ocean and less land area."
    },
    {
        "paper_id": "1908.06407.json",
        "image_id": "figure_3",
        "image_path": [
            "/home/yz979/scratch/chengye/scimolmo/test-images-tables/dataset_imagestables/1908.06407/images/raw_data.png"
        ],
        "caption": "Example of raw data collected from the accelerometer and gyroscope.",
        "classify": "Chart",
        "section_info": "3 Smart Chair Sensing Platform\n\\section{Smart Chair Sensing Platform}\\label{Smart Chair Sensing Platform}\n\n\\subsection{Smart Chair Design}\\label{SmartChairDesign}\n\nWhile designing the smart chair we rely on the Internet of Things (IoT) paradigm~\\cite{iot-2,iot1} - a number of sensors~\\cite{sensors} are connected to a global network with intelligent capabilities. The smart chair platform consists of two units: a sensing unit  for data collection and a server for data processing. The block diagram of the platform is shown in Fig.\\ref{scheme}.\n\n\\begin{figure}[!b]\n\t\\centerline{\\includegraphics[width=\\linewidth]{pic/system_architecture.png}}\n\t\\caption{Block diagram of proposed platform.}\n\t\\label{scheme}\n\\end{figure}\n\nThe core element of the sensing unit is RaspberryPi-3 with Motion Processing Unit (MPU) 9250 which collects the data from the accelerometer, magnetometer and gyroscope via I2C protocol \\cite{mpu_9250}.\nThe data are collected every 0.01 s. This solution can be further extended by adding extra sensors, e.g. pressure and temperature ones. Upon collecting the data, the sensing unit makes a request and sends it to a server over the WiFi wireless channel. The data are sent to the server via HTTP protocol every second in the JSON format. Data processing is realized on the server using the machine learning algorithms described in more details in the next section. All the machine learning algorithms are implemented using Python programming language. The smart chair experimental testbed is shown in Fig.~\\ref{sensing_system}. It shoukd be noted here that the sensing unit is fixed at the bottom of a chair.\n\n\\begin{figure}[!b]\n\t\\centering\n\t\\begin{subfigure}[]{0.18\\textwidth}\n\t\t\\centerline{\\includegraphics[height=4.1cm]{pic/system_anatomy.jpg}}\n\t\t\\caption{Anatomy of sensing system.}\n\t\t\\label{sensors}\n\n\t\\end{subfigure}\n\t\\begin{subfigure}[]{0.3\\textwidth}\n\t\t\\centerline{\\includegraphics[height=4.1cm]{pic/smart_bottom.jpg}}\n\t\t\\caption{sensing module fixed on the bottom of a chair.}\n\t\t\\label{chair}\n\t\\end{subfigure}\n\t\\caption{Smart chair experimental testbed.}\n\t\\label{sensing_system}\n\\end{figure}\n\nThe cloud is an application, the front-end between the database, microcontroller and machine learning algorithms. The cloud application is developed in Java with Spring framework \\cite{spring_framework}. This program allows us to run the application in different types of a cloud platform, for example, Cloud Foundry, Hysterix, Amazon Cloud \\cite{spring_boot}. Also, Spring Data has an opportunity of using access technologies in different databases. The platform, therefore, can be connected to MySQL, PostgreSQL, Oracle databases, MongoDB \\cite{spring_data}.\n\n\n\\subsection{Data Collection and Preprocessing}\\label{DataCollection and Pre-processing}\n\nThe data collection procedure is crucial for further data analysis. It is essential to properly markdown the data for future convenient preprocessing.\n\nIn this experiment we collect the data from 19 persons playing CS:GO. Each CS:GO session lasts around 35 minutes and the sesnsors’ data are collectes every 0.01 s.\n\nIn our case we have 3 types of data. The accelerometer  provides acceleration, the magnetometer provides the angles between the sensor own axes X, Y, Z and the lines of force of the Earth magnetic field, the gyroscope provides the angular velocity. In this work, we present the data as parts of time series.\n\nExample of raw data from the accelerometer and gyroscope are shown in Fig.~\\ref{raw_data}. Axis \\textit{z} corresponds to the vertical axis, \\textit{y} corresponds to the axis passing through the player and the monitor, \\textit{x} corresponds to the axis parallel to the gaming table.\n\n\\begin{figure}[!bt]\n\t\\centerline{\\includegraphics[width=\\linewidth]{pic/raw_data.png}}\n\t\\caption{Example of raw data collected from the accelerometer and gyroscope.}\n\t\\label{raw_data}\n\\end{figure}\n\nWe created two main feature groups based on the raw data. The first group is the portion of time when the data from thr sensor are deviating from the mean for more than 3 standard deviations. This means that a person is actively changing the position and moving. It can be named the active movement and considered for \\textit{x}, \\textit{y}, \\textit{z} components of the accelerometer and gyroscope, resulting in 6 features in total.\n\nThe second group of six features is the mean dispersion of the sensors’ data when a person is not actively moving. It corresponds to the subtle oscillations by foot, head or other parts of the body. Besides, it is able to catch the slow and constant rotations on the chair.\n\nAn additional feature we used is the portion of time when the person leans on the back of the chair. It is extracted from the accelerometer data: if \\textit{z}-coordinate is lower than a threshold, then the angle between \\textit{z} and the vertical axes is too low and the person is leaning on the back of the chair.\n\n3.2 Data Collection and Preprocessing\n\\subsection{Data Collection and Preprocessing}\\label{DataCollection and Pre-processing}\n\nThe data collection procedure is crucial for further data analysis. It is essential to properly markdown the data for future convenient preprocessing.\n\nIn this experiment we collect the data from 19 persons playing CS:GO. Each CS:GO session lasts around 35 minutes and the sesnsors’ data are collectes every 0.01 s.\n\nIn our case we have 3 types of data. The accelerometer  provides acceleration, the magnetometer provides the angles between the sensor own axes X, Y, Z and the lines of force of the Earth magnetic field, the gyroscope provides the angular velocity. In this work, we present the data as parts of time series.\n\nExample of raw data from the accelerometer and gyroscope are shown in Fig.~\\ref{raw_data}. Axis \\textit{z} corresponds to the vertical axis, \\textit{y} corresponds to the axis passing through the player and the monitor, \\textit{x} corresponds to the axis parallel to the gaming table.\n\n\\begin{figure}[!bt]\n\t\\centerline{\\includegraphics[width=\\linewidth]{pic/raw_data.png}}\n\t\\caption{Example of raw data collected from the accelerometer and gyroscope.}\n\t\\label{raw_data}\n\\end{figure}\n\nWe created two main feature groups based on the raw data. The first group is the portion of time when the data from thr sensor are deviating from the mean for more than 3 standard deviations. This means that a person is actively changing the position and moving. It can be named the active movement and considered for \\textit{x}, \\textit{y}, \\textit{z} components of the accelerometer and gyroscope, resulting in 6 features in total.\n\nThe second group of six features is the mean dispersion of the sensors’ data when a person is not actively moving. It corresponds to the subtle oscillations by foot, head or other parts of the body. Besides, it is able to catch the slow and constant rotations on the chair.\n\nAn additional feature we used is the portion of time when the person leans on the back of the chair. It is extracted from the accelerometer data: if \\textit{z}-coordinate is lower than a threshold, then the angle between \\textit{z} and the vertical axes is too low and the person is leaning on the back of the chair.\n\n",
        "label": "raw_data",
        "Descriptive_question1": "Which axis in figure_3 corresponds to the vertical direction?",
        "Descriptive_question2": "What type of data is collected from the gyroscope in figure_3?",
        "Reasoning_question1": "Based on the data trends in figure_3, what can you infer about the player's movement intensity along the different axes during the gaming session?",
        "Reasoning_question2": "How might the differences in accelerometer and gyroscope data in figure_3 reflect the player's posture or behavior while playing CS:GO?",
        "Descriptive_answer1": "z",
        "Descriptive_answer2": "angular velocity",
        "Reasoning_answer1": "In figure_3, the accelerometer data exhibit varying levels of intensity across different axes over time, with the z-axis showing distinct peaks indicating vertical movements, while x and y axes show less pronounced fluctuations. The gyroscope data similarly vary, but their patterns suggest more rotational movements along the axes. This implies that the player engages in more vertical movement and rotation rather than lateral shifts, indicating moderate movement intensity primarily involving posture adjustments rather than large-scale displacement.",
        "Reasoning_answer2": "The accelerometer measures linear acceleration including gravity, so when the player leans back, the z-axis readings decrease, indicating change in posture. The gyroscope measures angular velocity, so variations in its data reflect rotations of the chair or body. Thus, differences between accelerometer and gyroscope signals suggest periods when the player adjusts posture by leaning or twisting, inferring a dynamic but seated behavior consistent with gaming activities."
    },
    {
        "paper_id": "1908.06407.json",
        "image_id": "figure_4",
        "image_path": [
            "/home/yz979/scratch/chengye/scimolmo/test-images-tables/dataset_imagestables/1908.06407/images/heatmap.png"
        ],
        "caption": "Heat map for features.",
        "classify": "Chart",
        "section_info": "4 Machine Learning\n\\section{Machine Learning}\\label{Machine Learning}\n\n\\subsection{Training Data}\nWe asked 19 participants (9 professional athletes and 10 amateur players) to estimate their skill against the low/high scale. After encoding the low skill to \\textit{0} and the high skill to \\textit{1} we got the binary target for machine learning models.\n\nFrom preprocessing step (see Section III-B) we have 13 features. Correlations between them and the target are represented in Fig.~\\ref{heatmap}. The detailed description is provided in Table~\\ref{heatmap_table}.\n\n\\begin{figure}[!bt]\n\t\\centerline{\\includegraphics[width=\\linewidth]{pic/heatmap.png}}\n\t\\caption{Heat map for features.}\n\t\\label{heatmap}\n\\end{figure}\n\n\n\\begin{table}[]\n\t\\caption{Description of features on the heat map.}\n\t\\label{heatmap_table}\n\t\\begin{center}\n\t\t\\begin{tabular}{|p{0.6cm}|p{7.4cm}|}\n\t\t\t\\hline\n\t\t\t\\textbf{Name}  & \\textbf{Description}                                                                        \\\\\\hline\n\t\t\tSkill & Player’s skill in CS:GO. 0 or 1 (low or high)                                      \\\\\\hline\n\t\t\taxn   & Active movement along x-axis. Intensity of rectilinear motion to the right/left.   \\\\\\hline\n\t\t\tayn   & Active movement along y-axis.Intensity of rectilinear motion to/from table.        \\\\\\hline\n\t\t\tazn   & Active movement along z-axis. Intensity of rectilinear motion up/down.             \\\\\\hline\n\t\t\tgxn   & Active rotation on x-axis. Frequency of approaching to/distancing from a monitor.  \\\\\\hline\n\t\t\tgyn   & Active rotation on y-axis. Intensity of swaying to the right/left side of a chair. \\\\\\hline\n\t\t\tgzn   & Active rotation on z-axis. Intensity of rotations on a vertical axis.              \\\\\\hline\n\t\t\tlb    & Portion of time when player leans to the back of a chair.                          \\\\\\hline\n\t\t\taxo   & Intensity of subtle rectilinear oscillations parallel to table.                    \\\\\\hline\n\t\t\tayo   & Intensity of subtle rectilinear oscillations to/from table.                        \\\\\\hline\n\t\t\tazo   & Intensity of subtle rectilinear oscillations up/down.                              \\\\\\hline\n\t\t\tgxo   & Intensity of subtle approaching to/distancing from a monitor.                      \\\\\\hline\n\t\t\tgyo   & Intensity of subtle swaying to the right/left side of a chair.                     \\\\\\hline\n\t\t\tgzo   & Intensity of subtle rotations on a vertical axis.                                 \\\\\\hline\n\t\t\\end{tabular}\n\t\\end{center}\n\\end{table}\n\nWe observe that the professional athletes perform less active movements during the game (rows/columns 2-7). Less intensive active movements could be connected with the higher concentration on the game. On the other hand, some subtle motions (rows/columns 9-14) are more characteristic of the athletes. It includes the rotational motion on the y-axis and the rectilinear motion along the x-axis. The first corresponds to the point on how much the person is swaying from right to left; the second means that the person moves in rectilinear way from right to left. Leaning on the back of the chair is more representative of the amateur players.\n\nIn order to increase the amount of data we divided the log of  each player into 3-minutes sessions. As a result, our dataset includes 154 sessions from 19 persons. We have fitted several machine learning models based on 13 features to predict the player skill.\n\n\\subsection{Machine Learning Algorithms}\\label{ml_algorithms}\n\\subsubsection{Logistic regression}\n\nIt is the classifier which uses the logistic function to calculate probability from margin and maximizing the likelihood function,  \n$L$, \ngiven the observations \\cite{ng_lectures}:\n\n\\begin{equation}\nL(\\theta) = \\text{Pr} (Y|X; \\theta) = \\prod_i \\text{Pr}(y_i | x_i; \\theta),\n\\label{lr}\n\\end{equation}\nwhere $X$ is a design matrix, $Y$ is a vector of targets, $\\theta$ is the vector of model parameters. Probability of being from class 1 is determined by sigmoid function \\eqref{sigmoid}:\n\n\\begin{equation}\n\\text{Pr}(y_i | x_i; \\theta) = \\frac1{1 + e^{-\\theta^\\top x_i}}.\n\\label{sigmoid}\n\\end{equation}\n\nIn our experiment the main advantages of the logistic regression are stability, easy interpretation and good approximation.\n\n\\subsubsection{Support Vector Machine (SVM)}\n\nWe also used SVM classifier with soft-margins \\cite{svm_classic}. This method tries to separete classes by a hyperplane so that the gap between them is maximum possible:\n\n\\begin{align}\n\\label{eqn:eqlabel}\n\\begin{split}\n\\min_{w, \\xi}~& \\frac12 w^\\top w + \\frac\\gamma{2}\\sum_ {i=1}^n \\xi_i,\n\\\\\n\\text{subject to~}&y_i(w^\\top x_i + b) \\geq 1 - \\xi_i,\\\\\n& \\xi_i \\geq 0,  i \\in 1 \\dots n,\n\\end{split}\n\\end{align}\nwhere $x_i$ is a feature vector, $y_i$ is a scalar target, $w$ and $b$ are the parameters determining the separation hyperplane and its width, $\\xi_k$ is a slack variable, $\\gamma$ is the parameter which determines the tradeoff between the maximum margin and the minimum classification error.\n\nThe main advantage of this method is, again, stability due to  maximization of the gap between different classes.\n\n\\subsubsection{Nearest neighbors}\n\nFor the given input, $k$-nearest neighbors classifier searches for the $k$ nearest neighbors in the feature space from the training set and returns the most popular label among them \\cite{cover1967nearest}. In our problem we used the number of neighbours equal to 5, which provides the maximum ROC AUC.\n\nIn our case this simple algorithm can predict the unknown player performance taking into account performance of similar players.\n\n\\subsubsection{Random Forest}\n\nA random forest is the classifier represented by an ensemble of tree-structured classifiers. Random forest can handle complex dependencies in data, but this may lead to overfitting. According to experiments the optimal maximum tree depth in our problem is 4. \n\nFor our problem random forest can learn logical rules to distinguish the high-skilled and low-skilled players and catch more complex patterns in their behaviours.\n\n\n4.1 Training Data\n\\subsection{Training Data}\nWe asked 19 participants (9 professional athletes and 10 amateur players) to estimate their skill against the low/high scale. After encoding the low skill to \\textit{0} and the high skill to \\textit{1} we got the binary target for machine learning models.\n\nFrom preprocessing step (see Section III-B) we have 13 features. Correlations between them and the target are represented in Fig.~\\ref{heatmap}. The detailed description is provided in Table~\\ref{heatmap_table}.\n\n\\begin{figure}[!bt]\n\t\\centerline{\\includegraphics[width=\\linewidth]{pic/heatmap.png}}\n\t\\caption{Heat map for features.}\n\t\\label{heatmap}\n\\end{figure}\n\n\n\\begin{table}[]\n\t\\caption{Description of features on the heat map.}\n\t\\label{heatmap_table}\n\t\\begin{center}\n\t\t\\begin{tabular}{|p{0.6cm}|p{7.4cm}|}\n\t\t\t\\hline\n\t\t\t\\textbf{Name}  & \\textbf{Description}                                                                        \\\\\\hline\n\t\t\tSkill & Player’s skill in CS:GO. 0 or 1 (low or high)                                      \\\\\\hline\n\t\t\taxn   & Active movement along x-axis. Intensity of rectilinear motion to the right/left.   \\\\\\hline\n\t\t\tayn   & Active movement along y-axis.Intensity of rectilinear motion to/from table.        \\\\\\hline\n\t\t\tazn   & Active movement along z-axis. Intensity of rectilinear motion up/down.             \\\\\\hline\n\t\t\tgxn   & Active rotation on x-axis. Frequency of approaching to/distancing from a monitor.  \\\\\\hline\n\t\t\tgyn   & Active rotation on y-axis. Intensity of swaying to the right/left side of a chair. \\\\\\hline\n\t\t\tgzn   & Active rotation on z-axis. Intensity of rotations on a vertical axis.              \\\\\\hline\n\t\t\tlb    & Portion of time when player leans to the back of a chair.                          \\\\\\hline\n\t\t\taxo   & Intensity of subtle rectilinear oscillations parallel to table.                    \\\\\\hline\n\t\t\tayo   & Intensity of subtle rectilinear oscillations to/from table.                        \\\\\\hline\n\t\t\tazo   & Intensity of subtle rectilinear oscillations up/down.                              \\\\\\hline\n\t\t\tgxo   & Intensity of subtle approaching to/distancing from a monitor.                      \\\\\\hline\n\t\t\tgyo   & Intensity of subtle swaying to the right/left side of a chair.                     \\\\\\hline\n\t\t\tgzo   & Intensity of subtle rotations on a vertical axis.                                 \\\\\\hline\n\t\t\\end{tabular}\n\t\\end{center}\n\\end{table}\n\nWe observe that the professional athletes perform less active movements during the game (rows/columns 2-7). Less intensive active movements could be connected with the higher concentration on the game. On the other hand, some subtle motions (rows/columns 9-14) are more characteristic of the athletes. It includes the rotational motion on the y-axis and the rectilinear motion along the x-axis. The first corresponds to the point on how much the person is swaying from right to left; the second means that the person moves in rectilinear way from right to left. Leaning on the back of the chair is more representative of the amateur players.\n\nIn order to increase the amount of data we divided the log of  each player into 3-minutes sessions. As a result, our dataset includes 154 sessions from 19 persons. We have fitted several machine learning models based on 13 features to predict the player skill.\n\n5 Evaluation\n\\section{Evaluation}\\label{Evaluation}\n\nWe can not use sessions from the same player for both training and testing stage, because our model should be able to predict the performance for new participants.\nThus, to estimate the model performance correctly we trained models, described in \\ref{ml_algorithms}, on all people except 4-5 out of 19 and then validated on them. For more stable results the scores were calculated 100 times and averaged.\n\nWe used ROC AUC score \\cite{fan2006understanding} as an evaluation metric, for it nicely represents how well the  classes are separated by a model. The maximum possible value is 1, while the random guess get 0.5. Scores for different algorithms are shown in Table~\\ref{table_scores}. Linear models, such as Logistic Regression and SVM perform better than KNN and Random Forest. It can be explained that the real dependence between the player skill and extracted features is similar to the linear dependency.\nBesides, it is possible that KNN and Random Forest are too complex for our problem, for they tend to overfit for small data.\n\nThe mean ROC AUC score for all of the algorithms is more or equal to 0.8, which means that the eSport athlete performance can be successfully predicted by machine learning models. As an illustrative example, ROC curve for Logistic Regression is shown in Fig.~\\ref{roc_auc}.\n\n\\begin{table}[!bt]\n\t\\caption{Models performance on predicting the player skill: a comparative study.}\n\t\\label{table_scores}\n\t\\begin{center}\n\t\t\\begin{tabular}{|l|l|l|}\n\t\t\\hline\n\t\t\\textbf{Method}                 & \\textbf{AUC, mean} & \\textbf{AUC, std} \\\\\\hline\n\t\tLogistic Regression    & 0.85      & 0.14     \\\\\\hline\n\t\tSupport Vector Machine & \\textbf{0.86}      & 0.13     \\\\\\hline\n\t\tKNN, 5 neighbours      & 0.80      & 0.13     \\\\\\hline\n\t\tRandom Forest, depth 4 & 0.82      & 0.16  \\\\\\hline  \n\t\t\\end{tabular}\n\t\\end{center}\n\\end{table}\n\n\nTo figure out which characteristics define the player performance we used coefficients in the logistic regression. It is a stable and reliable estimation of feature importance. Positive coefficients correspond to the high-skilled players behavior, while negative coefficients are typical for the  low-skilled players behavior. Large absolute value means that feature is more important. Fig.~\\ref{feature_importance} demonstrates that the motion to right and to left is the most characteristic of professional athletes. Active rotations on the chair are also important factors, but typically refer to the amateur players.\nThese results are consistent with the recap from the heatmap shown in Fig.~\\ref{heatmap}.\n\n\\begin{figure}[!tb]\n\t\\centerline{\\includegraphics[width=\\linewidth]{pic/roc_auc_curve.png}}\n\t\\caption{ROC AUC curve for logistic regression.}\n\t\\label{roc_auc}\n\\end{figure}\n\n\\begin{figure*}[!hbt]\n\t\\centerline{\\includegraphics[width=\\linewidth]{pic/feature_importance.png}}\n\t\\caption{Feature importance as coefficients in the logistic regression.}\n\t\\label{feature_importance}\n\\end{figure*}\n\n\n \n\n\n\n\n\n\n\n",
        "label": "heatmap",
        "Descriptive_question1": "What is the total number of features represented in the heat map in figure_4?",
        "Descriptive_question2": "Which feature in figure_4 corresponds to leaning on the back of the chair?",
        "Reasoning_question1": "Based on the heat map in figure_4, what might explain the difference in active movement intensity between professional and amateur players?",
        "Reasoning_question2": "How do subtle motions in figure_4 correlate with player skill levels, and what could this imply about concentration during gameplay?",
        "Descriptive_answer1": "13",
        "Descriptive_answer2": "lb",
        "Reasoning_answer1": "The heat map and accompanying description indicate that professional athletes perform less active movements during the game compared to amateur players (features axn, ayn, azn, gxn, gyn, gzn). This reduced intensity in active movement among professionals could be explained by higher concentration levels during gameplay, leading to more controlled and deliberate movements.",
        "Reasoning_answer2": "Subtle motions (features axo, ayo, azo, gxo, gyo, gzo) are more characteristic of professional athletes, showing positive correlation with high skill. This suggests that while professionals reduce large active movements, they exhibit more subtle motions, which could imply a state of focused attention with minimal but precise body adjustments, reflecting higher concentration during gameplay."
    },
    {
        "paper_id": "1908.06407.json",
        "image_id": "figure_5",
        "image_path": [
            "/home/yz979/scratch/chengye/scimolmo/test-images-tables/dataset_imagestables/1908.06407/images/roc_auc_curve.png"
        ],
        "caption": "ROC AUC curve for logistic regression.",
        "classify": "Chart",
        "section_info": "5 Evaluation\n\\section{Evaluation}\\label{Evaluation}\n\nWe can not use sessions from the same player for both training and testing stage, because our model should be able to predict the performance for new participants.\nThus, to estimate the model performance correctly we trained models, described in \\ref{ml_algorithms}, on all people except 4-5 out of 19 and then validated on them. For more stable results the scores were calculated 100 times and averaged.\n\nWe used ROC AUC score \\cite{fan2006understanding} as an evaluation metric, for it nicely represents how well the  classes are separated by a model. The maximum possible value is 1, while the random guess get 0.5. Scores for different algorithms are shown in Table~\\ref{table_scores}. Linear models, such as Logistic Regression and SVM perform better than KNN and Random Forest. It can be explained that the real dependence between the player skill and extracted features is similar to the linear dependency.\nBesides, it is possible that KNN and Random Forest are too complex for our problem, for they tend to overfit for small data.\n\nThe mean ROC AUC score for all of the algorithms is more or equal to 0.8, which means that the eSport athlete performance can be successfully predicted by machine learning models. As an illustrative example, ROC curve for Logistic Regression is shown in Fig.~\\ref{roc_auc}.\n\n\\begin{table}[!bt]\n\t\\caption{Models performance on predicting the player skill: a comparative study.}\n\t\\label{table_scores}\n\t\\begin{center}\n\t\t\\begin{tabular}{|l|l|l|}\n\t\t\\hline\n\t\t\\textbf{Method}                 & \\textbf{AUC, mean} & \\textbf{AUC, std} \\\\\\hline\n\t\tLogistic Regression    & 0.85      & 0.14     \\\\\\hline\n\t\tSupport Vector Machine & \\textbf{0.86}      & 0.13     \\\\\\hline\n\t\tKNN, 5 neighbours      & 0.80      & 0.13     \\\\\\hline\n\t\tRandom Forest, depth 4 & 0.82      & 0.16  \\\\\\hline  \n\t\t\\end{tabular}\n\t\\end{center}\n\\end{table}\n\n\nTo figure out which characteristics define the player performance we used coefficients in the logistic regression. It is a stable and reliable estimation of feature importance. Positive coefficients correspond to the high-skilled players behavior, while negative coefficients are typical for the  low-skilled players behavior. Large absolute value means that feature is more important. Fig.~\\ref{feature_importance} demonstrates that the motion to right and to left is the most characteristic of professional athletes. Active rotations on the chair are also important factors, but typically refer to the amateur players.\nThese results are consistent with the recap from the heatmap shown in Fig.~\\ref{heatmap}.\n\n\\begin{figure}[!tb]\n\t\\centerline{\\includegraphics[width=\\linewidth]{pic/roc_auc_curve.png}}\n\t\\caption{ROC AUC curve for logistic regression.}\n\t\\label{roc_auc}\n\\end{figure}\n\n\\begin{figure*}[!hbt]\n\t\\centerline{\\includegraphics[width=\\linewidth]{pic/feature_importance.png}}\n\t\\caption{Feature importance as coefficients in the logistic regression.}\n\t\\label{feature_importance}\n\\end{figure*}\n\n\n \n\n\n\n\n\n\n\n",
        "label": "roc_auc",
        "Descriptive_question1": "What is the title of figure_5?",
        "Descriptive_question2": "Which model is represented in figure_5?",
        "Reasoning_question1": "What does the ROC AUC curve in figure_5 indicate about the performance of logistic regression in separating classes?",
        "Reasoning_question2": "How does the ROC AUC curve in figure_5 support the claim that logistic regression performs better than other models like KNN or Random Forest?",
        "Descriptive_answer1": "ROC AUC curve for logistic regression",
        "Descriptive_answer2": "Logistic Regression",
        "Reasoning_answer1": "The ROC AUC curve demonstrates how effectively logistic regression distinguishes between classes; since the curve is shown as an illustrative example and the mean AUC is 0.85 (which is well above the random guess baseline of 0.5), it indicates the model separates classes quite well.",
        "Reasoning_answer2": "Since the table shows logistic regression having a higher mean AUC (0.85) compared to KNN (0.80) and Random Forest (0.82), the ROC AUC curve visually representing logistic regression's performance supports this superior class separation ability, confirming its better predictive power over the other models."
    },
    {
        "paper_id": "1908.06407.json",
        "image_id": "figure_6",
        "image_path": [
            "/home/yz979/scratch/chengye/scimolmo/test-images-tables/dataset_imagestables/1908.06407/images/feature_importance.png"
        ],
        "caption": "Feature importance as coefficients in the logistic regression.",
        "classify": "Chart",
        "section_info": "5 Evaluation\n\\section{Evaluation}\\label{Evaluation}\n\nWe can not use sessions from the same player for both training and testing stage, because our model should be able to predict the performance for new participants.\nThus, to estimate the model performance correctly we trained models, described in \\ref{ml_algorithms}, on all people except 4-5 out of 19 and then validated on them. For more stable results the scores were calculated 100 times and averaged.\n\nWe used ROC AUC score \\cite{fan2006understanding} as an evaluation metric, for it nicely represents how well the  classes are separated by a model. The maximum possible value is 1, while the random guess get 0.5. Scores for different algorithms are shown in Table~\\ref{table_scores}. Linear models, such as Logistic Regression and SVM perform better than KNN and Random Forest. It can be explained that the real dependence between the player skill and extracted features is similar to the linear dependency.\nBesides, it is possible that KNN and Random Forest are too complex for our problem, for they tend to overfit for small data.\n\nThe mean ROC AUC score for all of the algorithms is more or equal to 0.8, which means that the eSport athlete performance can be successfully predicted by machine learning models. As an illustrative example, ROC curve for Logistic Regression is shown in Fig.~\\ref{roc_auc}.\n\n\\begin{table}[!bt]\n\t\\caption{Models performance on predicting the player skill: a comparative study.}\n\t\\label{table_scores}\n\t\\begin{center}\n\t\t\\begin{tabular}{|l|l|l|}\n\t\t\\hline\n\t\t\\textbf{Method}                 & \\textbf{AUC, mean} & \\textbf{AUC, std} \\\\\\hline\n\t\tLogistic Regression    & 0.85      & 0.14     \\\\\\hline\n\t\tSupport Vector Machine & \\textbf{0.86}      & 0.13     \\\\\\hline\n\t\tKNN, 5 neighbours      & 0.80      & 0.13     \\\\\\hline\n\t\tRandom Forest, depth 4 & 0.82      & 0.16  \\\\\\hline  \n\t\t\\end{tabular}\n\t\\end{center}\n\\end{table}\n\n\nTo figure out which characteristics define the player performance we used coefficients in the logistic regression. It is a stable and reliable estimation of feature importance. Positive coefficients correspond to the high-skilled players behavior, while negative coefficients are typical for the  low-skilled players behavior. Large absolute value means that feature is more important. Fig.~\\ref{feature_importance} demonstrates that the motion to right and to left is the most characteristic of professional athletes. Active rotations on the chair are also important factors, but typically refer to the amateur players.\nThese results are consistent with the recap from the heatmap shown in Fig.~\\ref{heatmap}.\n\n\\begin{figure}[!tb]\n\t\\centerline{\\includegraphics[width=\\linewidth]{pic/roc_auc_curve.png}}\n\t\\caption{ROC AUC curve for logistic regression.}\n\t\\label{roc_auc}\n\\end{figure}\n\n\\begin{figure*}[!hbt]\n\t\\centerline{\\includegraphics[width=\\linewidth]{pic/feature_importance.png}}\n\t\\caption{Feature importance as coefficients in the logistic regression.}\n\t\\label{feature_importance}\n\\end{figure*}\n\n\n \n\n\n\n\n\n\n\n",
        "label": "feature_importance",
        "Descriptive_question1": "What is the most important feature for high-skilled players in figure_6?",
        "Descriptive_question2": "Which direction of motion has a positive coefficient in figure_6?",
        "Reasoning_question1": "Why might motion to the right and left be more characteristic of professional athletes according to figure_6?",
        "Reasoning_question2": "How does the importance of chair rotations compare to motion features for player skill prediction in figure_6?",
        "Descriptive_answer1": "motion to right",
        "Descriptive_answer2": "motion to left",
        "Reasoning_answer1": "Motion to the right and left have the largest positive coefficients in logistic regression, indicating these movements are strong indicators of high skill. This suggests professional players frequently exhibit lateral motion, reflecting refined and purposeful body movements associated with their expertise.",
        "Reasoning_answer2": "Chair rotations have smaller negative coefficients compared to the strong positive coefficients of motion to right and left. This implies rotations are more typical of lower-skilled, amateur players and are less important or even negatively associated with high skill prediction, highlighting that motion features are more predictive of player skill."
    },
    {
        "paper_id": "1511.03757.json",
        "image_id": "figure_2",
        "image_path": [
            "/home/yz979/scratch/chengye/scimolmo/test-images-tables/dataset_imagestables/1511.03757/images/fig2.eps"
        ],
        "caption": "(Color online)\nKohn-Sham band structure of SrVO$_3$ in the LDA.\nThe energy is measured from the Fermi level.\n",
        "classify": "Chart",
        "section_info": "6 Results\n\\section{Results}\n\\label{results}\n\n\nThe ternary 3d$^1$ transition metal oxide SrVO$_3$ has become\none of the ``drosophila compounds'' of correlated systems.\nIt is a correlated metal\nthat has been characterized using various experimental \n\\cite{Onoda, inoue, maiti, maiti2, sekiyama, takizawa, yoshida2010}.\nand theoretical techniques, see e.g.\n\\cite{nekrasov, anisimov, pavarini-d1, liebsch, lechermann-d1, trimarchi, aichhorn09, karolak, lee, held-optics, tomczak-gwdmft, gatti-guzzo, sakuma14, TomczakPRB2014}.\nA review of most of the available experimental and theoretical \ndata has been given recently in Ref.~\\onlinecite{TomczakPRB2014}.\nSrVO$_3$ displays Fermi liquid behavior up to\nremarkably high temperatures of the order of $200$ K \\cite{Onoda, inoue},\nwith a moderate mass enhancement of the order \nof $2$ \\cite{yoshida2010, vandermarel}.\nDetailed spectroscopic investigations have made it an\nideal test compound for modern many-body calculations, and \nmore and more refined dynamical mean field-based studies \nare available.\nThe majority of studies so far have focused on the t$_{2g}$-manifold\nwhich forms the states close to the Fermi level, and those will\nalso be the focus in the present investigations. \nNote however that this restriction quite severely limits the\nrange of validity of the low-energy description. Indeed, as\nshown recently~\\cite{tomczak-gwdmft, TomczakPRB2014} at\nenergies of about 2 to 3 eV above the Fermi level\nthe spectral properties are largely determined by the e$_g$\nstates. This has in particular led to a reinterpretation of\nan inverse photoemission feature at about 2.5 eV that was \nfrequently interpreted as an upper Hubbard band of t$_{2g}$\ncharacter in the earlier literature.\nHere, we use the compound only as an illustration of \nthe principles of constructing effective low-energy models,\nwithout being concerned with a description of spectra beyond\nthe pure t$_{2g}$ part.\n\n\nThe LDA band structure of SrVO$_3$ is shown in Fig.~\\ref{lda}.\nOne clearly distinguishes the three-fold degenerate manifold\nof t$_{2g}$ bands close to the Fermi level (chosen as the\nzero of energy), followed in the unoccupied part of the spectrum\nby the two e$_g$ bands. At about -2.5 eV the filled O-2p derived\nbands are visible.\n\nWhen a standard GW calculation is performed, see Fig.~\\ref{fullGW},\nthe bandwidth of the t$_{2g}$-states is reduced from 2.5 eV to\n2.1 eV, while the overall shape of the dispersion remains similar\nto the LDA one.\nThis is in agreement with previous GW calculations in the literature\n\\cite{tomczak-gwdmft, TomczakPRB2014, gatti-guzzo, sakuma14}.\n\n\n\nThe calculation is based on the full-potential linear muffin-tin orbitals (FP-LMTO) implementation.\n8$\\times$8$\\times$8 \\bm{k}-mesh is employed in both the DFT/LDA and the GW calculations.\nThe calculational details are the same as in Ref.~\\onlinecite{Hirayama}.\n\n\nWe now turn to a discussion of the band structure corresponding to \nthe Hamiltonians to be used as input for subsequent many-body \ncalculations for this material.\nWe will proceed step by step to analyze the influence of the various\ncorrection terms, with respect to the DFT starting point\n(which will be overlaid to the respective band structures).\n\n\n\n\\begin{figure}[h]\n\\centering \n\\includegraphics[clip,width=0.45\\textwidth ]{fig2.eps} \n\\caption{(Color online)\nKohn-Sham band structure of SrVO$_3$ in the LDA.\nThe energy is measured from the Fermi level.\n}\n\\label{lda}\n\\end{figure} \n\n\n\\begin{figure}[h]\n\\centering \n\\includegraphics[clip,width=0.45\\textwidth ]{fig3.eps} \n\\caption{(Color online)\nBand structure  of SrVO$_3$ in the one-shot GW approximation.\nFor comparison, the LDA band structure is also given\n(black dotted line).\nThe energy is measured from the Fermi level.\n}\n\\label{fullGW}\n\\end{figure} \n\n\n\n\n\n\n\\begin{figure}[h]\n\\centering \n\\includegraphics[clip,width=0.45\\textwidth ]{fig4.eps} \n\\caption{(Color online)\nBand structure of SrVO$_3$ corresponding to the LDA Hamiltonian\nfrom which the LDA exchange-correlation potential has been subtracted.\nFor comparison, the LDA band structure is also given\n(black dotted line).\nThe energy is measured from the Fermi level.\n}\n\\label{lda-vxc}\n\\end{figure} \n\n\n\\begin{figure}[h]\n\\centering \n\\includegraphics[clip,width=0.45\\textwidth ]{fig5.eps} \n\\caption{(Color online)\nBand structure  of SrVO$_3$ in the Hartree approximation.\nFor comparison, the LDA band structure is also given\n(black dotted line).\nThe energy is measured from the Fermi level.\n}\n\\label{hartree}\n\\end{figure} \n\n\nFigure~\\ref{lda-vxc} \ndisplays the band structure corresponding to\nthe LDA Hamiltonian from which the \nLDA exchange-correlation\npotential has been subtracted,  \nthat is, the eigenvalues of $H_{\\rm LDA}-V_{\\rm xc}$ where\nthese operators are evaluated for the self-consistent LDA\ndensity.\nThe subtraction of $V_{\\rm xc}$ widens the band structure from \nthe LDA bandwidth of 2.5 eV \nto more than twice this value: the new band\nwidth is 5.4 eV.\nThis indicates that the exchange-correlation potential $V_{\\rm xc}$ \nof the LDA is responsible for a substantial amount of \nthe band-narrowing effect\narising from electronic correlations, stressing the need \nto subtract the double counting in a consistent manner. \nFigure~\\ref{lda-vxc} is regarded as the starting point for the\nfollowing considerations.\n\nFor simple materials (see e.g. the calculations on Li in Ref.~\\onlinecite{Lithium})\nit has been noted in the literature that the Hartree band structure\nis close to the DFT one. This raises the question of the origin of the\nsubstantial band widening in the present case. Indeed, the present\ncalculation shown in Fig.~\\ref{lda-vxc} differs from a Hartree calculation\nonly \nby the fact that Fig.~\\ref{lda-vxc} is evaluated for the converged LDA density.\nWe have performed a test calculation where we plot the Hartree band\nstructure calculated for the converged Hartree density. The result\nis shown in Fig.~\\ref{hartree}. As seen from this plot, while the\nband is not fully as wide as in Fig.~\\ref{lda-vxc}, a substantial \nwidening is already present at this stage.\n\n\n\n\\begin{figure}[h]\n\\centering \n\\includegraphics[clip,width=0.45\\textwidth ]{fig6.eps} \n\\caption{(Color online)\nBand structure  of $H_{\\rm LDA}-V_{\\rm xc}+\\Delta \\Sigma_{\\rm H}(\\omega=0)$ of SrVO$_3$.\nFor comparison, the LDA band structure is also given\n(black dotted line).\nThe energy is measured from the Fermi level.\n}\n\\label{lda-vxc-sigH(0)}\n\\end{figure} \n\n\n\nStarting from the Hamiltonian without V$_{xc}$ (see the \ndispersion in Fig.~\\ref{lda-vxc}), we first take into account \nthe static part of the corrective self-energy $\\Delta \\Sigma_{\\rm H}$:\nIn Fig.~\\ref{lda-vxc-sigH(0)}, we plot the dispersion corresponding to\n$H_{\\rm LDA}-V_{\\rm xc}+\\Delta \\Sigma_{\\rm H}(\\omega=0)$. \nCompared to the dispersion of Fig.~\\ref{lda-vxc}, the overall band structure \nis narrowed to 3.7 eV. This value is, however, still considerably larger\nthan the LDA bandwidth (Fig.~\\ref{lda}). Although\n the electronic correlations coupling the H- and \nthe L space are included in Fig.~\\ref{lda-vxc-sigH(0)}, and narrow the band with respect to the case\nwhere V$_{xc}$ is taken out, correlations within\nthe L space are not included. \nThe LDA, on the other hand, at least partially\nincludes correlations within the L space, \nand these are very effective in narrowing the band.\nInterestingly, the bottom of the occupied band is quite\nexactly at the LDA value, and the remaining widening is\npurely in the unoccupied part.\n\n\n\\begin{figure*}[htb]\n\\begin{center} \n\\includegraphics[width=0.9\\textwidth ]{fig7.eps} \n\\end{center} \n\\caption{(Color online)\nFrequency dependence of $-V_{xc}+\\Delta \\Sigma_{\\rm H}$ for SrVO$_3$.\n(a) and (b) are the real and imaginary parts at the $\\Gamma$ point, \nrespectively,\nand (c) and (d) ((e) and (f)) are those at $X$ ($R$).\nFor comparison, the full GW self-energies are also given\n(black dotted line).\n}\n\\label{sigH}\n\\end{figure*} \n\n\n\n\\begin{figure}[h]\n\\centering \n\\includegraphics[clip,width=0.45\\textwidth ]{fig8.eps} \n\\caption{(Color online)\nBand structure  of $Z_{\\rm H}[H_{\\rm LDA}-V_{\\rm xc}+\\Delta\\Sigma_{\\rm H}(\\omega=0)]$ of SrVO$_3$.\nFor comparison, the LDA band structure is also given\n(black dotted line).\nThe energy is measured from the Fermi level.\n}\n\\label{ZH_SigmaH}\n\\end{figure} \n\n\nTo go further and in particular to analyze the dynamic\nbehavior of $\\Delta \\Sigma_{\\rm H}$ \nwe plot in Fig.~\\ref{sigH} \nthe self-energy corrections $\\Delta \\Sigma_{\\rm H}$ (with $\\Sigma$ (see Eq.~(\\ref{full-self-energy}))\nfor comparison) for the real and imaginary parts\nat several representative choices of momenta.\nThe frequency dependence is smooth around the Fermi level.\nIn particular, in contrast to the full self-energy,\nthere are no poles in $\\Delta \\Sigma_{\\rm H}$ \nin the low-energy region, thanks to the\nexclusion of fluctuations within the L-space.\nThe frequency dependence of the real part indicates that the \nlinearization \n$\\displaystyle{\\Delta \\Sigma_{\\rm H}(\\omega) \\sim \\Delta \\Sigma_{\\rm H}(\\omega=0)+[d\\Delta \\Sigma_{\\rm H}(\\omega)/d\\omega]_{\\omega=0}\\omega }$\noffers a reasonably good approximation. \nIn fact, the behavior of the constrained self-energy\nis much closer to linearity than that of \nthe full GW self-energy $\\Sigma$.\nThis is easily understood by the fact that the low-energy excitations are excluded in the present constrained self-energy $\\Delta\\Sigma_{\\rm H}$ analogously to an insulator, thus eliminating the strong frequency dependence of the constrained self-energy near the Fermi level. This is one of the consequences of the \ncontrollability and an advantage of the present MACE scheme, as \nmentioned in the introduction.\nIn the following discussion, we employ this linearized approximation.\nThen the renormalization factor $Z_{\\rm H}$ defined by Eq.~(\\ref{ZH})\nis interpreted as that from the contribution of the H-space. \nAfter \nthis renormalization factor is taken into account,\nthe effective Hamiltonian is given by $Z_{\\rm H}[H_{\\rm LDA}-V_{\\rm xc}+\\Delta\\Sigma_{\\rm H}(\\omega=0)]$ (see Fig.~\\ref{ZH_SigmaH}). \nInterestingly the renormalization factor $Z_{\\rm H}$=0.92 stays\nclose to one, so that\nthe bandwidth is only slightly reduced (to\n$3.4$ eV instead of the $3.7$ eV above). \nWe also note that $Z_{\\rm H}$ has weak momentum dependence\nas we reveal in the following.\n\n\n\n\\begin{figure*}[htb]\n\\begin{center} \n\\includegraphics[width=0.9\\textwidth ]{fig9.eps} \n\\end{center} \n\\caption{(Color online)\n{\\bf minor modification}\nFrequency dependence of $-V_{\\rm xc}+\\Delta \\Sigma_{\\rm H}$ of SrVO$_3$.\n(a) and (b) are the real and imaginary parts of the local\ncomponent, respectively.\nThe black dotted line is $\\Sigma$ defined in Eq.~(\\ref{full-self-energy}) in the full GW approximation for comparison.\n(c) and (d) are the real and imaginary \nparts at several $k$ points.\n}\n\\label{sigH_nonlocal}\n\\end{figure*} \n\n\n\nFigure~\\ref{sigH_nonlocal} \nshows the local and nonlocal parts of $\\Delta \\Sigma_{\\rm H}$.\nThe frequency \ndependence\nof the local part (Figs.~\\ref{sigH_nonlocal} (a)(b)) \nis quite similar \nto the  frequency-dependent self-energies at various momenta\nin Fig.~\\ref{sigH}.\nOne immediately reads off an interesting property, which is\nakin to what has been found for the full GW self-energy in\nRef.~\\onlinecite{Tomczak-PRL2012}, namely that the nonlocal part of \n$\\Delta \\Sigma_{\\rm H}$ shows only weak frequency dependence \nas shown in Figs.~\\ref{sigH_nonlocal} (c)(d). \nThis explains why $Z_{\\rm H}$ \nis only weakly momentum-dependent.\nOn the other hand, $\\Delta \\Sigma_{\\rm H}^{\\rm nonlocal} (0)$ at $R$ is $\\sim 1$ eV larger than \nat $\\Gamma$, which causes the band widening effect.\n $\\Delta \\Sigma_{\\rm H}$\ncan thus to a good approximation be decomposed into a\nfrequency-dependent local part and a static nonlocal one:\n$\\Delta \\Sigma_{\\rm H}(k,\\omega) = \\Delta \\Sigma^{\\rm local}_{\\rm H}(\\omega) + \\Delta \\Sigma^{\\rm nonlocal}_{\\rm H}(k)$,\nwith $ \\Delta \\Sigma_{\\rm H}^{\\rm local}(\\omega) = \\sum_q  \\Delta \\Sigma_{\\rm H}(q,\\omega)$\n\nSince in the simple case of SrVO$_3$ where the t$_{2g}$ states are\ndegenerate, static local operators are scalar and are compensated \nby a chemical potential shift such that the correct particle\nnumber is obtained. The above band structure corresponding to\n$H_{\\rm LDA} - V_{\\rm xc} + \\Delta \\Sigma_{\\rm H}(k, \\omega=0)$ \nis thus identical to that of\n$H_{\\rm LDA} - V_{\\rm xc} + \\Delta \\Sigma^{\\rm nonlocal}_{\\rm H}(k)$.\nThe local dynamical part of $\\Delta \\Sigma_{\\rm H}$ then results in a \nnarrowing of this band structure by a factor $Z_{\\rm H} = 0.92$.\n\n\n\n\\begin{figure}[h]\n\\centering \n\\includegraphics[clip,width=0.45\\textwidth ]{fig10.eps} \n\\caption{(Color online)\n(a) Frequency dependence of the on-site effective interaction.\nPanel (b) is a low-energy zoom of panel (a). \n}\n\\label{W}\n\\end{figure} \n\n\nBefore turning to a discussion of the low-energy correction for the GW treatment\n$\\Delta \\Sigma_{\\rm L} =G^{(0)}(W-W_{\\rm L})$, we analyze the effective\ninteraction $W_{\\rm L}$ shown in Fig.~\\ref{W} \nin comparison to $W$ and $W_{\\rm H}$.\nThe fully screened Coulomb interaction $W$ displays the familiar\nshape of an interaction that is strongly screened at low energies\n(with a value of $0.88$ eV at $\\omega=0$), while retrieving the\nvalue of the bare Coulomb interaction $v$ ($\\sim 16.0$ eV) \nat high energy.\nThe crossover from the bare to the  \nscreened values takes place at the plasma energy of \nabout $15$ eV. This behavior has been discussed before\n\\cite{Review_SB2014}; we note in particular that the plasma \nfrequency is known from electron energy loss spectroscopy \nmeasurements of the related SrTiO$_3$ compound \\cite{vandermarel,kohiki}.\nThe partially screened interaction\n$W_{\\rm H}$ constructed within cRPA\nconverges to $W$ at high energies, but displays\nweaker screening effects at low energies (with a value of $3.5$ eV at $\\omega=0$),\nsince intra-t$_{2g}$ screening channels are excluded.\nAs was already anticipated in Sec.~\\ref{Outline},  $W_{\\rm L}$ can be interpreted as the screened interaction of \na low-energy model where a static interaction of value $W_{\\rm H}(\\omega=0)$\nhas been imposed as the bare interaction. Since now {\\it only} intra-t$_{2g}$\nscreening channels are active, screening takes place only at low\nenergies, where the scale is given by the t$_{2g}$ bandwidth.\nAlso plotted is the difference $W_{\\rm GW}^{\\rm dyn}=W-W_{d}$: except at low energies\nwhere the t$_{2g}$ screening channels come into play, its frequency\ndependence is essentially given by that of $W$, while the\nhigh-energy limit is the bare Coulomb interaction $v$ reduced\nby $W_{\\rm H}(0)$.\n\n\n\\begin{figure*}[htb]\n\\begin{center} \n\\includegraphics[width=0.9\\textwidth ]{fig11.eps} \n\\end{center} \n\\caption{(Color online)\nFrequency dependence of $-V_{\\rm xc}+ G^{(1)}W_{\\rm H}^{\\rm dyn}$ of SrVO$_3$.\n(a) and (b) are the real and imaginary local parts, respectively.\n(c) and (d) are the real and imaginary \nparts at several $k$ points.\n}\n\\label{DeltaSigma_L_Pert}\n\\end{figure*} \n\n\n\n\\begin{figure*}[htb]\n\\begin{center} \n\\includegraphics[width=0.9\\textwidth ]{fig12.eps} \n\\end{center} \n\\caption{(Color online)\nFrequency dependence of $-V_{\\rm xc}+ G^{(0)}W_{\\rm GW}^{\\rm dyn}$ of SrVO$_3$.\n(a) and (b) are the real and imaginary local parts, respectively.\n(c) and (d) are the real and imaginary \nparts at several $k$ points.\n}\n\\label{DeltaSigma_L_GW}\n\\end{figure*} \n\n\n\nThe frequency dependence of the real and imaginary parts of the low-energy self-energy $\\Delta \\Sigma_{\\rm L} = G^{(1)}W_{\\rm H}^{\\rm dyn}$ for the direct perturbative treatment is illustrated in Fig.~\\ref{DeltaSigma_L_Pert}. \n$\\Delta \\Sigma_{\\rm L} = G^{(0)}W_{\\rm GW}^{\\rm dyn}$ used\nfor the GW treatment is shown in Fig.~\\ref{DeltaSigma_L_GW}\nfor several choices of momenta.\n\\\\\nThe results for the perturbative and the GW treatment are nearly \nidentical.\nThe renormalization factor of $\\Delta \\Sigma_{\\rm L}^{\\rm nonlocal}$ \n(and $\\Delta \\Sigma_{\\rm L}$) is $\\sim 0.77$.\nThe zero-frequency shift\n${\\rm Re}\\Delta \\Sigma_{\\rm L}^{\\rm nonlocal} (0)$ at $R$ is $\\sim 3$ eV \nlarger than \nat $\\Gamma$.\nAgain, one sees how $\\Sigma_{\\rm L}$ separates into local dynamical\nand nonlocal static parts: \n$\\Delta \\Sigma_{\\rm L} = \\Delta \\Sigma^{\\rm nonlocal}_{\\rm L}(k) + \n\\Delta \\Sigma^{\\rm local}_{\\rm L}(\\omega)$.\nThis is expected since such a separation has been found within\nthe full GW calculation \\cite{TomczakPRB2014}, and is even\nmore plausible for a GW treatment within the t$_{2g}$ subspace.\n\n\n\n\\begin{figure}[h]\n\\centering \n\\includegraphics[clip,width=0.45\\textwidth ]{fig13.eps} \n\\caption{(Color online)\nBand structure  of $[\\epsilon_{\\rm DFT}-V_{\\rm xc}+\\Delta\\Sigma_{\\rm H}+\\Delta \\Sigma_{\\rm L}^{\\rm nonlocal}]Z_{\\rm HL}$ of SrVO$_3$ in the direct perturbative treatment.\nFor comparison, the LDA band structure is also given\n(black dotted line).\nThe energy is measured from the Fermi level.\n}\n\\label{DFT+Sigma_HL_Pert}\n\\end{figure} \n\n\n\n\\begin{figure}[h]\n\\centering \n\\includegraphics[clip,width=0.45\\textwidth ]{fig14.eps} \n\\caption{(Color online)\nBand structure  of $[\\epsilon_{\\rm DFT}-V_{\\rm xc}+\\Delta\\Sigma_{\\rm H}+\\Delta \\Sigma_{\\rm L}^{\\rm nonlocal}]Z_{\\rm HL}$ of SrVO$_3$ in the GW treatment.\nFor comparison, the LDA band structure is also given\n(black dotted line).\nThe energy is measured from the Fermi level.\n}\n\\label{DFT+Sigma_HL_GW}\n\\end{figure} \n\n\n\nThe corresponding band structures \nare given by \n $[\\epsilon_{\\rm DFT}+\\Delta\\Sigma_{\\rm H}+\\Delta \\Sigma_{\\rm L}^{\\rm nonlocal}]Z_{\\rm HL}$ in Fig.~\\ref{DFT+Sigma_HL_Pert} \nfor the perturbative treatment (see Eq.~(\\ref{DeltaSigmaLnonlocalPert})), and in Fig.~\\ref{DFT+Sigma_HL_GW} \nfor the GW treatment,\nwhere $\\Delta \\Sigma_{\\rm L}^{\\rm nonlocal}$ is defined in Eq.~(\\ref{DeltaSigLnonlocalGW}) and $Z_{\\rm HL}$ is around 0.92.\n$Z_{\\rm HL}$ is practically the same as $Z_{\\rm H}$, meaning that the nonlocal dynamical correction \nin the H-space is small, which justifies the GW perturbative treatment for the nonlocal part.\nOne finds that \nthe nonlocal part $\\Delta \\Sigma_{\\rm L}=G^{(0)}_{ll}(W-W_{\\rm L})$ narrows\nthe empty states but widens the occupied ones, resulting\nin an overall bandwidth of 3.2 eV.\nThe difference between the direct perturbation and the GW treatment is small.\n\n\n\n\\begin{figure}[h]\n\\centering \n\\includegraphics[clip,width=0.45\\textwidth ]{fig15.eps} \n\\caption{(Color online)\nBand structure  of $[\\epsilon_{\\rm DFT}-V_{\\rm xc}+\\Delta\\Sigma_{\\rm H}+\\Delta\\Sigma_{\\rm L}^{\\rm nonlocal}]Z_{\\rm HL}Z_B$ of SrVO$_3$ in the GW treatment.\nFor comparison, the LDA band structure is also given\n(black dotted line).\nThe energy is measured from the Fermi level.}\n\\label{DFT+DeltaSigmaL_GW_ZHL_ZB}\n\\end{figure} \n\n\nAfter including the effect of the local self-energy by the nonperturbative \nCasula trick, we show the dispersion\ngiven by \n$[\\epsilon_{\\rm DFT}-V_{\\rm xc}+\\Delta\\Sigma_{\\rm H}+\\Delta\\Sigma_{\\rm L}^{\\rm nonlocal}]Z_{\\rm HL}Z_B$ for the GW-type treatment in Fig.~\\ref{DFT+DeltaSigmaL_GW_ZHL_ZB}: \nThe GW-like treatment gives a band dispersion\nclose to the LDA one. \nThe $Z_B$ factor corresponding to the local dynamical part of \n$\\Delta \\Sigma_{\\rm L}$ amounts to $Z_B=0.7$.\nThe nonperturbative treatment \nresults in an LDA-like band dispersion for the empty states,\nbut a slightly narrower bandwidth in the occupied part.\nThe low-energy effective Hamiltonian at this level of treatment has a \nfrequency independent effective interaction both with local \nand nonlocal interaction given by the Fourier transform of \n$W_{\\rm H}(q,\\omega=0)$, which contains both direct and exchange interactions. \n\n\n\\begin{figure}[h]\n\\centering \n\\includegraphics[clip,width=0.45\\textwidth ]{fig16.eps} \n\\caption{(Color online)\nBand structure  of $[\\epsilon_{\\rm DFT}+\\Sigma_{\\rm corr}^{\\rm GW}]Z_{\\rm corr}Z_B$ of SrVO$_3$.\nFor comparison, the LDA band structure is also given\n(black dotted line).\nThe energy is measured from the Fermi level.}\n\\label{DFT+Sigma_corr_GW_Z}\n\\end{figure} \n\n\n\nWe finally show the dispersion\ngiven by Eq.~(\\ref{Teff_final}), namely\n$[\\epsilon_{\\rm DFT}+\\Sigma_{\\rm corr}^{\\rm GW}]Z_{\\rm corr}Z_B$ in Fig.~\\ref{DFT+Sigma_corr_GW_Z}. \nThe result shows an LDA-like band dispersion for the occupied states,\nbut a slightly wider bandwidth in the empty part, resulting in a bandwidth 14\\% wider in total than the LDA bandwidth. \nThe effective interaction of the low-energy effective Hamiltonian \nat this final level has\nfrequency independent local and nonlocal interactions \ngiven by the Fourier transform of $W_r(q,\\omega=0)$. \n\n\n\n\\begin{figure}[h]\n\\centering \n\\includegraphics[clip,width=0.45\\textwidth ]{fig17.eps} \n\\caption{(Color online)\nBand structure  of $[\\epsilon_{\\rm DFT}-V_{\\rm xc}+\\Sigma-(G^{(0)}W_{\\rm GW}^{\\rm dyn})^{\\rm nonlocal}]Z'Z_B$ of SrVO$_3$\n(Z' is the renormalization factor of $\\Sigma-(G^{(0)}W_{\\rm GW}^{\\rm dyn})^{\\rm nonlocal}$).\nFor comparison, the LDA band structure is also given\n(black dotted line).\nThe energy is measured from the Fermi level.\n}\n\\label{GW+ZB}\n\\end{figure} \n\n\n\nThe resulting effective Hamiltonian in the L-space \nconsists of single-particle and two-particle (interaction) parts.\nThe effective interaction is the same as in previous estimates \nby the cRPA in the literature, while the single-particle dispersion \nis revised after removing the double counting and taking into \naccount the frequency dependence of the effective interaction. The \nfinal effective bandwidth for SrVO$_3$ is,\nafter partial cancellation, \nslightly (14\\%) larger than the LDA bandwidth.  \nThis is a reasonable result because the LDA takes into account all \ncorrelations though insufficiently, while the present scheme by the constrained self-energy excludes the \ncorrelation effects arising from the L-space.\nAlthough the bandwidth derived for the effective model is slightly \nlarger than that of the LDA,  it is clear that the correlation effects \nare stronger than in the LDA or standard GW when the {\\it ab initio} \nmodel is solved by an accurate solver. In fact, if the effective \ninteraction contained in the final effective model is treated by \nthe GW scheme, one obtains the dispersion illustrated in \nFig.~\\ref{GW+ZB}, \nwhich is given from the self energy of the whole GW calculation $\\Sigma$ \nby correcting the local dynamical part $W-W_{\\rm L}$ by $Z_B$. \nThis indicates that even an insufficient treatment of the local \nstatic interaction by the GW scheme gives a dispersion with the bandwidth ($\\sim 1.9$ eV) narrower \nthan the LDA ($\\sim 2.5$ eV, Fig.~\\ref{lda}) and GW ($\\sim 2.1$ eV, Fig.~\\ref{fullGW}) dispersions.  \nA slightly ($\\sim 14$\\%) wider dispersion than that of the LDA bands obtained for the \neffective Hamiltonian accompanied with frequency independent effective \nCoulomb interactions without the exchange part may account for the slight \noverestimate of correlation effects in the literature mentioned in \nthe introduction.\nOur scheme offers an optimized way for the  derivation of \n{\\it ab initio} models for the L-space after eliminating the H-space\nin  a systematic fashion.\n\n\nOur findings of the band widening are consistent with studies\nbased on the combined GW+DMFT method in the literature \n\\cite{TomczakPRB2014}: There, it was argued that within GW+DMFT\nthe best effective Hamiltonian that DMFT should be performed on,\nis a one-body Hamiltonian corrected by the nonlocal part of the\nGW self-energy. The corresponding spectral function \n(see e.g.\nFigure 5 of Ref.~\\onlinecite{TomczakPRB2014}) displays \na broadening similar to \nFig.~\\ref{DFT+Sigma_HL_GW}.\nMost interestingly, our present calculations confirm a pronounced\nasymmetry observed in Ref.~\\onlinecite{TomczakPRB2014}, namely a stronger widening\neffect in the unoccupied part of the spectrum than in the occupied\none.\n\n",
        "label": "lda",
        "Descriptive_question1": "What is the energy reference point in figure_2?",
        "Descriptive_question2": "Which bands are visible at about -2.5 eV in figure_2?",
        "Reasoning_question1": "Why are the t_{2g} bands significant in the context of the Fermi level in figure_2?",
        "Reasoning_question2": "How does the positioning of the e_g bands above the Fermi level in figure_2 impact the interpretation of spectral properties at higher energies?",
        "Descriptive_answer1": "Fermi level",
        "Descriptive_answer2": "O-2p derived",
        "Reasoning_answer1": "The t_{2g} bands are three-fold degenerate and situated close to the Fermi level, indicating that they dominate the low-energy electronic states and thus play a crucial role in determining the metallic and correlated behavior of SrVO_3 near the Fermi energy.",
        "Reasoning_answer2": "Since the e_g bands appear at energies about 2 to 3 eV above the Fermi level, they influence the spectral properties at higher energies. This means that features observed in spectroscopic experiments at these energies are largely determined by e_g states rather than t_{2g} states, leading to a reinterpretation of prior spectral features previously attributed to t_{2g} upper Hubbard bands."
    },
    {
        "paper_id": "1511.03757.json",
        "image_id": "figure_3",
        "image_path": [
            "/home/yz979/scratch/chengye/scimolmo/test-images-tables/dataset_imagestables/1511.03757/images/fig3.eps"
        ],
        "caption": "(Color online)\nBand structure  of SrVO$_3$ in the one-shot GW approximation.\nFor comparison, the LDA band structure is also given\n(black dotted line).\nThe energy is measured from the Fermi level.\n",
        "classify": "Chart",
        "section_info": "6 Results\n\\section{Results}\n\\label{results}\n\n\nThe ternary 3d$^1$ transition metal oxide SrVO$_3$ has become\none of the ``drosophila compounds'' of correlated systems.\nIt is a correlated metal\nthat has been characterized using various experimental \n\\cite{Onoda, inoue, maiti, maiti2, sekiyama, takizawa, yoshida2010}.\nand theoretical techniques, see e.g.\n\\cite{nekrasov, anisimov, pavarini-d1, liebsch, lechermann-d1, trimarchi, aichhorn09, karolak, lee, held-optics, tomczak-gwdmft, gatti-guzzo, sakuma14, TomczakPRB2014}.\nA review of most of the available experimental and theoretical \ndata has been given recently in Ref.~\\onlinecite{TomczakPRB2014}.\nSrVO$_3$ displays Fermi liquid behavior up to\nremarkably high temperatures of the order of $200$ K \\cite{Onoda, inoue},\nwith a moderate mass enhancement of the order \nof $2$ \\cite{yoshida2010, vandermarel}.\nDetailed spectroscopic investigations have made it an\nideal test compound for modern many-body calculations, and \nmore and more refined dynamical mean field-based studies \nare available.\nThe majority of studies so far have focused on the t$_{2g}$-manifold\nwhich forms the states close to the Fermi level, and those will\nalso be the focus in the present investigations. \nNote however that this restriction quite severely limits the\nrange of validity of the low-energy description. Indeed, as\nshown recently~\\cite{tomczak-gwdmft, TomczakPRB2014} at\nenergies of about 2 to 3 eV above the Fermi level\nthe spectral properties are largely determined by the e$_g$\nstates. This has in particular led to a reinterpretation of\nan inverse photoemission feature at about 2.5 eV that was \nfrequently interpreted as an upper Hubbard band of t$_{2g}$\ncharacter in the earlier literature.\nHere, we use the compound only as an illustration of \nthe principles of constructing effective low-energy models,\nwithout being concerned with a description of spectra beyond\nthe pure t$_{2g}$ part.\n\n\nThe LDA band structure of SrVO$_3$ is shown in Fig.~\\ref{lda}.\nOne clearly distinguishes the three-fold degenerate manifold\nof t$_{2g}$ bands close to the Fermi level (chosen as the\nzero of energy), followed in the unoccupied part of the spectrum\nby the two e$_g$ bands. At about -2.5 eV the filled O-2p derived\nbands are visible.\n\nWhen a standard GW calculation is performed, see Fig.~\\ref{fullGW},\nthe bandwidth of the t$_{2g}$-states is reduced from 2.5 eV to\n2.1 eV, while the overall shape of the dispersion remains similar\nto the LDA one.\nThis is in agreement with previous GW calculations in the literature\n\\cite{tomczak-gwdmft, TomczakPRB2014, gatti-guzzo, sakuma14}.\n\n\n\nThe calculation is based on the full-potential linear muffin-tin orbitals (FP-LMTO) implementation.\n8$\\times$8$\\times$8 \\bm{k}-mesh is employed in both the DFT/LDA and the GW calculations.\nThe calculational details are the same as in Ref.~\\onlinecite{Hirayama}.\n\n\nWe now turn to a discussion of the band structure corresponding to \nthe Hamiltonians to be used as input for subsequent many-body \ncalculations for this material.\nWe will proceed step by step to analyze the influence of the various\ncorrection terms, with respect to the DFT starting point\n(which will be overlaid to the respective band structures).\n\n\n\n\\begin{figure}[h]\n\\centering \n\\includegraphics[clip,width=0.45\\textwidth ]{fig2.eps} \n\\caption{(Color online)\nKohn-Sham band structure of SrVO$_3$ in the LDA.\nThe energy is measured from the Fermi level.\n}\n\\label{lda}\n\\end{figure} \n\n\n\\begin{figure}[h]\n\\centering \n\\includegraphics[clip,width=0.45\\textwidth ]{fig3.eps} \n\\caption{(Color online)\nBand structure  of SrVO$_3$ in the one-shot GW approximation.\nFor comparison, the LDA band structure is also given\n(black dotted line).\nThe energy is measured from the Fermi level.\n}\n\\label{fullGW}\n\\end{figure} \n\n\n\n\n\n\n\\begin{figure}[h]\n\\centering \n\\includegraphics[clip,width=0.45\\textwidth ]{fig4.eps} \n\\caption{(Color online)\nBand structure of SrVO$_3$ corresponding to the LDA Hamiltonian\nfrom which the LDA exchange-correlation potential has been subtracted.\nFor comparison, the LDA band structure is also given\n(black dotted line).\nThe energy is measured from the Fermi level.\n}\n\\label{lda-vxc}\n\\end{figure} \n\n\n\\begin{figure}[h]\n\\centering \n\\includegraphics[clip,width=0.45\\textwidth ]{fig5.eps} \n\\caption{(Color online)\nBand structure  of SrVO$_3$ in the Hartree approximation.\nFor comparison, the LDA band structure is also given\n(black dotted line).\nThe energy is measured from the Fermi level.\n}\n\\label{hartree}\n\\end{figure} \n\n\nFigure~\\ref{lda-vxc} \ndisplays the band structure corresponding to\nthe LDA Hamiltonian from which the \nLDA exchange-correlation\npotential has been subtracted,  \nthat is, the eigenvalues of $H_{\\rm LDA}-V_{\\rm xc}$ where\nthese operators are evaluated for the self-consistent LDA\ndensity.\nThe subtraction of $V_{\\rm xc}$ widens the band structure from \nthe LDA bandwidth of 2.5 eV \nto more than twice this value: the new band\nwidth is 5.4 eV.\nThis indicates that the exchange-correlation potential $V_{\\rm xc}$ \nof the LDA is responsible for a substantial amount of \nthe band-narrowing effect\narising from electronic correlations, stressing the need \nto subtract the double counting in a consistent manner. \nFigure~\\ref{lda-vxc} is regarded as the starting point for the\nfollowing considerations.\n\nFor simple materials (see e.g. the calculations on Li in Ref.~\\onlinecite{Lithium})\nit has been noted in the literature that the Hartree band structure\nis close to the DFT one. This raises the question of the origin of the\nsubstantial band widening in the present case. Indeed, the present\ncalculation shown in Fig.~\\ref{lda-vxc} differs from a Hartree calculation\nonly \nby the fact that Fig.~\\ref{lda-vxc} is evaluated for the converged LDA density.\nWe have performed a test calculation where we plot the Hartree band\nstructure calculated for the converged Hartree density. The result\nis shown in Fig.~\\ref{hartree}. As seen from this plot, while the\nband is not fully as wide as in Fig.~\\ref{lda-vxc}, a substantial \nwidening is already present at this stage.\n\n\n\n\\begin{figure}[h]\n\\centering \n\\includegraphics[clip,width=0.45\\textwidth ]{fig6.eps} \n\\caption{(Color online)\nBand structure  of $H_{\\rm LDA}-V_{\\rm xc}+\\Delta \\Sigma_{\\rm H}(\\omega=0)$ of SrVO$_3$.\nFor comparison, the LDA band structure is also given\n(black dotted line).\nThe energy is measured from the Fermi level.\n}\n\\label{lda-vxc-sigH(0)}\n\\end{figure} \n\n\n\nStarting from the Hamiltonian without V$_{xc}$ (see the \ndispersion in Fig.~\\ref{lda-vxc}), we first take into account \nthe static part of the corrective self-energy $\\Delta \\Sigma_{\\rm H}$:\nIn Fig.~\\ref{lda-vxc-sigH(0)}, we plot the dispersion corresponding to\n$H_{\\rm LDA}-V_{\\rm xc}+\\Delta \\Sigma_{\\rm H}(\\omega=0)$. \nCompared to the dispersion of Fig.~\\ref{lda-vxc}, the overall band structure \nis narrowed to 3.7 eV. This value is, however, still considerably larger\nthan the LDA bandwidth (Fig.~\\ref{lda}). Although\n the electronic correlations coupling the H- and \nthe L space are included in Fig.~\\ref{lda-vxc-sigH(0)}, and narrow the band with respect to the case\nwhere V$_{xc}$ is taken out, correlations within\nthe L space are not included. \nThe LDA, on the other hand, at least partially\nincludes correlations within the L space, \nand these are very effective in narrowing the band.\nInterestingly, the bottom of the occupied band is quite\nexactly at the LDA value, and the remaining widening is\npurely in the unoccupied part.\n\n\n\\begin{figure*}[htb]\n\\begin{center} \n\\includegraphics[width=0.9\\textwidth ]{fig7.eps} \n\\end{center} \n\\caption{(Color online)\nFrequency dependence of $-V_{xc}+\\Delta \\Sigma_{\\rm H}$ for SrVO$_3$.\n(a) and (b) are the real and imaginary parts at the $\\Gamma$ point, \nrespectively,\nand (c) and (d) ((e) and (f)) are those at $X$ ($R$).\nFor comparison, the full GW self-energies are also given\n(black dotted line).\n}\n\\label{sigH}\n\\end{figure*} \n\n\n\n\\begin{figure}[h]\n\\centering \n\\includegraphics[clip,width=0.45\\textwidth ]{fig8.eps} \n\\caption{(Color online)\nBand structure  of $Z_{\\rm H}[H_{\\rm LDA}-V_{\\rm xc}+\\Delta\\Sigma_{\\rm H}(\\omega=0)]$ of SrVO$_3$.\nFor comparison, the LDA band structure is also given\n(black dotted line).\nThe energy is measured from the Fermi level.\n}\n\\label{ZH_SigmaH}\n\\end{figure} \n\n\nTo go further and in particular to analyze the dynamic\nbehavior of $\\Delta \\Sigma_{\\rm H}$ \nwe plot in Fig.~\\ref{sigH} \nthe self-energy corrections $\\Delta \\Sigma_{\\rm H}$ (with $\\Sigma$ (see Eq.~(\\ref{full-self-energy}))\nfor comparison) for the real and imaginary parts\nat several representative choices of momenta.\nThe frequency dependence is smooth around the Fermi level.\nIn particular, in contrast to the full self-energy,\nthere are no poles in $\\Delta \\Sigma_{\\rm H}$ \nin the low-energy region, thanks to the\nexclusion of fluctuations within the L-space.\nThe frequency dependence of the real part indicates that the \nlinearization \n$\\displaystyle{\\Delta \\Sigma_{\\rm H}(\\omega) \\sim \\Delta \\Sigma_{\\rm H}(\\omega=0)+[d\\Delta \\Sigma_{\\rm H}(\\omega)/d\\omega]_{\\omega=0}\\omega }$\noffers a reasonably good approximation. \nIn fact, the behavior of the constrained self-energy\nis much closer to linearity than that of \nthe full GW self-energy $\\Sigma$.\nThis is easily understood by the fact that the low-energy excitations are excluded in the present constrained self-energy $\\Delta\\Sigma_{\\rm H}$ analogously to an insulator, thus eliminating the strong frequency dependence of the constrained self-energy near the Fermi level. This is one of the consequences of the \ncontrollability and an advantage of the present MACE scheme, as \nmentioned in the introduction.\nIn the following discussion, we employ this linearized approximation.\nThen the renormalization factor $Z_{\\rm H}$ defined by Eq.~(\\ref{ZH})\nis interpreted as that from the contribution of the H-space. \nAfter \nthis renormalization factor is taken into account,\nthe effective Hamiltonian is given by $Z_{\\rm H}[H_{\\rm LDA}-V_{\\rm xc}+\\Delta\\Sigma_{\\rm H}(\\omega=0)]$ (see Fig.~\\ref{ZH_SigmaH}). \nInterestingly the renormalization factor $Z_{\\rm H}$=0.92 stays\nclose to one, so that\nthe bandwidth is only slightly reduced (to\n$3.4$ eV instead of the $3.7$ eV above). \nWe also note that $Z_{\\rm H}$ has weak momentum dependence\nas we reveal in the following.\n\n\n\n\\begin{figure*}[htb]\n\\begin{center} \n\\includegraphics[width=0.9\\textwidth ]{fig9.eps} \n\\end{center} \n\\caption{(Color online)\n{\\bf minor modification}\nFrequency dependence of $-V_{\\rm xc}+\\Delta \\Sigma_{\\rm H}$ of SrVO$_3$.\n(a) and (b) are the real and imaginary parts of the local\ncomponent, respectively.\nThe black dotted line is $\\Sigma$ defined in Eq.~(\\ref{full-self-energy}) in the full GW approximation for comparison.\n(c) and (d) are the real and imaginary \nparts at several $k$ points.\n}\n\\label{sigH_nonlocal}\n\\end{figure*} \n\n\n\nFigure~\\ref{sigH_nonlocal} \nshows the local and nonlocal parts of $\\Delta \\Sigma_{\\rm H}$.\nThe frequency \ndependence\nof the local part (Figs.~\\ref{sigH_nonlocal} (a)(b)) \nis quite similar \nto the  frequency-dependent self-energies at various momenta\nin Fig.~\\ref{sigH}.\nOne immediately reads off an interesting property, which is\nakin to what has been found for the full GW self-energy in\nRef.~\\onlinecite{Tomczak-PRL2012}, namely that the nonlocal part of \n$\\Delta \\Sigma_{\\rm H}$ shows only weak frequency dependence \nas shown in Figs.~\\ref{sigH_nonlocal} (c)(d). \nThis explains why $Z_{\\rm H}$ \nis only weakly momentum-dependent.\nOn the other hand, $\\Delta \\Sigma_{\\rm H}^{\\rm nonlocal} (0)$ at $R$ is $\\sim 1$ eV larger than \nat $\\Gamma$, which causes the band widening effect.\n $\\Delta \\Sigma_{\\rm H}$\ncan thus to a good approximation be decomposed into a\nfrequency-dependent local part and a static nonlocal one:\n$\\Delta \\Sigma_{\\rm H}(k,\\omega) = \\Delta \\Sigma^{\\rm local}_{\\rm H}(\\omega) + \\Delta \\Sigma^{\\rm nonlocal}_{\\rm H}(k)$,\nwith $ \\Delta \\Sigma_{\\rm H}^{\\rm local}(\\omega) = \\sum_q  \\Delta \\Sigma_{\\rm H}(q,\\omega)$\n\nSince in the simple case of SrVO$_3$ where the t$_{2g}$ states are\ndegenerate, static local operators are scalar and are compensated \nby a chemical potential shift such that the correct particle\nnumber is obtained. The above band structure corresponding to\n$H_{\\rm LDA} - V_{\\rm xc} + \\Delta \\Sigma_{\\rm H}(k, \\omega=0)$ \nis thus identical to that of\n$H_{\\rm LDA} - V_{\\rm xc} + \\Delta \\Sigma^{\\rm nonlocal}_{\\rm H}(k)$.\nThe local dynamical part of $\\Delta \\Sigma_{\\rm H}$ then results in a \nnarrowing of this band structure by a factor $Z_{\\rm H} = 0.92$.\n\n\n\n\\begin{figure}[h]\n\\centering \n\\includegraphics[clip,width=0.45\\textwidth ]{fig10.eps} \n\\caption{(Color online)\n(a) Frequency dependence of the on-site effective interaction.\nPanel (b) is a low-energy zoom of panel (a). \n}\n\\label{W}\n\\end{figure} \n\n\nBefore turning to a discussion of the low-energy correction for the GW treatment\n$\\Delta \\Sigma_{\\rm L} =G^{(0)}(W-W_{\\rm L})$, we analyze the effective\ninteraction $W_{\\rm L}$ shown in Fig.~\\ref{W} \nin comparison to $W$ and $W_{\\rm H}$.\nThe fully screened Coulomb interaction $W$ displays the familiar\nshape of an interaction that is strongly screened at low energies\n(with a value of $0.88$ eV at $\\omega=0$), while retrieving the\nvalue of the bare Coulomb interaction $v$ ($\\sim 16.0$ eV) \nat high energy.\nThe crossover from the bare to the  \nscreened values takes place at the plasma energy of \nabout $15$ eV. This behavior has been discussed before\n\\cite{Review_SB2014}; we note in particular that the plasma \nfrequency is known from electron energy loss spectroscopy \nmeasurements of the related SrTiO$_3$ compound \\cite{vandermarel,kohiki}.\nThe partially screened interaction\n$W_{\\rm H}$ constructed within cRPA\nconverges to $W$ at high energies, but displays\nweaker screening effects at low energies (with a value of $3.5$ eV at $\\omega=0$),\nsince intra-t$_{2g}$ screening channels are excluded.\nAs was already anticipated in Sec.~\\ref{Outline},  $W_{\\rm L}$ can be interpreted as the screened interaction of \na low-energy model where a static interaction of value $W_{\\rm H}(\\omega=0)$\nhas been imposed as the bare interaction. Since now {\\it only} intra-t$_{2g}$\nscreening channels are active, screening takes place only at low\nenergies, where the scale is given by the t$_{2g}$ bandwidth.\nAlso plotted is the difference $W_{\\rm GW}^{\\rm dyn}=W-W_{d}$: except at low energies\nwhere the t$_{2g}$ screening channels come into play, its frequency\ndependence is essentially given by that of $W$, while the\nhigh-energy limit is the bare Coulomb interaction $v$ reduced\nby $W_{\\rm H}(0)$.\n\n\n\\begin{figure*}[htb]\n\\begin{center} \n\\includegraphics[width=0.9\\textwidth ]{fig11.eps} \n\\end{center} \n\\caption{(Color online)\nFrequency dependence of $-V_{\\rm xc}+ G^{(1)}W_{\\rm H}^{\\rm dyn}$ of SrVO$_3$.\n(a) and (b) are the real and imaginary local parts, respectively.\n(c) and (d) are the real and imaginary \nparts at several $k$ points.\n}\n\\label{DeltaSigma_L_Pert}\n\\end{figure*} \n\n\n\n\\begin{figure*}[htb]\n\\begin{center} \n\\includegraphics[width=0.9\\textwidth ]{fig12.eps} \n\\end{center} \n\\caption{(Color online)\nFrequency dependence of $-V_{\\rm xc}+ G^{(0)}W_{\\rm GW}^{\\rm dyn}$ of SrVO$_3$.\n(a) and (b) are the real and imaginary local parts, respectively.\n(c) and (d) are the real and imaginary \nparts at several $k$ points.\n}\n\\label{DeltaSigma_L_GW}\n\\end{figure*} \n\n\n\nThe frequency dependence of the real and imaginary parts of the low-energy self-energy $\\Delta \\Sigma_{\\rm L} = G^{(1)}W_{\\rm H}^{\\rm dyn}$ for the direct perturbative treatment is illustrated in Fig.~\\ref{DeltaSigma_L_Pert}. \n$\\Delta \\Sigma_{\\rm L} = G^{(0)}W_{\\rm GW}^{\\rm dyn}$ used\nfor the GW treatment is shown in Fig.~\\ref{DeltaSigma_L_GW}\nfor several choices of momenta.\n\\\\\nThe results for the perturbative and the GW treatment are nearly \nidentical.\nThe renormalization factor of $\\Delta \\Sigma_{\\rm L}^{\\rm nonlocal}$ \n(and $\\Delta \\Sigma_{\\rm L}$) is $\\sim 0.77$.\nThe zero-frequency shift\n${\\rm Re}\\Delta \\Sigma_{\\rm L}^{\\rm nonlocal} (0)$ at $R$ is $\\sim 3$ eV \nlarger than \nat $\\Gamma$.\nAgain, one sees how $\\Sigma_{\\rm L}$ separates into local dynamical\nand nonlocal static parts: \n$\\Delta \\Sigma_{\\rm L} = \\Delta \\Sigma^{\\rm nonlocal}_{\\rm L}(k) + \n\\Delta \\Sigma^{\\rm local}_{\\rm L}(\\omega)$.\nThis is expected since such a separation has been found within\nthe full GW calculation \\cite{TomczakPRB2014}, and is even\nmore plausible for a GW treatment within the t$_{2g}$ subspace.\n\n\n\n\\begin{figure}[h]\n\\centering \n\\includegraphics[clip,width=0.45\\textwidth ]{fig13.eps} \n\\caption{(Color online)\nBand structure  of $[\\epsilon_{\\rm DFT}-V_{\\rm xc}+\\Delta\\Sigma_{\\rm H}+\\Delta \\Sigma_{\\rm L}^{\\rm nonlocal}]Z_{\\rm HL}$ of SrVO$_3$ in the direct perturbative treatment.\nFor comparison, the LDA band structure is also given\n(black dotted line).\nThe energy is measured from the Fermi level.\n}\n\\label{DFT+Sigma_HL_Pert}\n\\end{figure} \n\n\n\n\\begin{figure}[h]\n\\centering \n\\includegraphics[clip,width=0.45\\textwidth ]{fig14.eps} \n\\caption{(Color online)\nBand structure  of $[\\epsilon_{\\rm DFT}-V_{\\rm xc}+\\Delta\\Sigma_{\\rm H}+\\Delta \\Sigma_{\\rm L}^{\\rm nonlocal}]Z_{\\rm HL}$ of SrVO$_3$ in the GW treatment.\nFor comparison, the LDA band structure is also given\n(black dotted line).\nThe energy is measured from the Fermi level.\n}\n\\label{DFT+Sigma_HL_GW}\n\\end{figure} \n\n\n\nThe corresponding band structures \nare given by \n $[\\epsilon_{\\rm DFT}+\\Delta\\Sigma_{\\rm H}+\\Delta \\Sigma_{\\rm L}^{\\rm nonlocal}]Z_{\\rm HL}$ in Fig.~\\ref{DFT+Sigma_HL_Pert} \nfor the perturbative treatment (see Eq.~(\\ref{DeltaSigmaLnonlocalPert})), and in Fig.~\\ref{DFT+Sigma_HL_GW} \nfor the GW treatment,\nwhere $\\Delta \\Sigma_{\\rm L}^{\\rm nonlocal}$ is defined in Eq.~(\\ref{DeltaSigLnonlocalGW}) and $Z_{\\rm HL}$ is around 0.92.\n$Z_{\\rm HL}$ is practically the same as $Z_{\\rm H}$, meaning that the nonlocal dynamical correction \nin the H-space is small, which justifies the GW perturbative treatment for the nonlocal part.\nOne finds that \nthe nonlocal part $\\Delta \\Sigma_{\\rm L}=G^{(0)}_{ll}(W-W_{\\rm L})$ narrows\nthe empty states but widens the occupied ones, resulting\nin an overall bandwidth of 3.2 eV.\nThe difference between the direct perturbation and the GW treatment is small.\n\n\n\n\\begin{figure}[h]\n\\centering \n\\includegraphics[clip,width=0.45\\textwidth ]{fig15.eps} \n\\caption{(Color online)\nBand structure  of $[\\epsilon_{\\rm DFT}-V_{\\rm xc}+\\Delta\\Sigma_{\\rm H}+\\Delta\\Sigma_{\\rm L}^{\\rm nonlocal}]Z_{\\rm HL}Z_B$ of SrVO$_3$ in the GW treatment.\nFor comparison, the LDA band structure is also given\n(black dotted line).\nThe energy is measured from the Fermi level.}\n\\label{DFT+DeltaSigmaL_GW_ZHL_ZB}\n\\end{figure} \n\n\nAfter including the effect of the local self-energy by the nonperturbative \nCasula trick, we show the dispersion\ngiven by \n$[\\epsilon_{\\rm DFT}-V_{\\rm xc}+\\Delta\\Sigma_{\\rm H}+\\Delta\\Sigma_{\\rm L}^{\\rm nonlocal}]Z_{\\rm HL}Z_B$ for the GW-type treatment in Fig.~\\ref{DFT+DeltaSigmaL_GW_ZHL_ZB}: \nThe GW-like treatment gives a band dispersion\nclose to the LDA one. \nThe $Z_B$ factor corresponding to the local dynamical part of \n$\\Delta \\Sigma_{\\rm L}$ amounts to $Z_B=0.7$.\nThe nonperturbative treatment \nresults in an LDA-like band dispersion for the empty states,\nbut a slightly narrower bandwidth in the occupied part.\nThe low-energy effective Hamiltonian at this level of treatment has a \nfrequency independent effective interaction both with local \nand nonlocal interaction given by the Fourier transform of \n$W_{\\rm H}(q,\\omega=0)$, which contains both direct and exchange interactions. \n\n\n\\begin{figure}[h]\n\\centering \n\\includegraphics[clip,width=0.45\\textwidth ]{fig16.eps} \n\\caption{(Color online)\nBand structure  of $[\\epsilon_{\\rm DFT}+\\Sigma_{\\rm corr}^{\\rm GW}]Z_{\\rm corr}Z_B$ of SrVO$_3$.\nFor comparison, the LDA band structure is also given\n(black dotted line).\nThe energy is measured from the Fermi level.}\n\\label{DFT+Sigma_corr_GW_Z}\n\\end{figure} \n\n\n\nWe finally show the dispersion\ngiven by Eq.~(\\ref{Teff_final}), namely\n$[\\epsilon_{\\rm DFT}+\\Sigma_{\\rm corr}^{\\rm GW}]Z_{\\rm corr}Z_B$ in Fig.~\\ref{DFT+Sigma_corr_GW_Z}. \nThe result shows an LDA-like band dispersion for the occupied states,\nbut a slightly wider bandwidth in the empty part, resulting in a bandwidth 14\\% wider in total than the LDA bandwidth. \nThe effective interaction of the low-energy effective Hamiltonian \nat this final level has\nfrequency independent local and nonlocal interactions \ngiven by the Fourier transform of $W_r(q,\\omega=0)$. \n\n\n\n\\begin{figure}[h]\n\\centering \n\\includegraphics[clip,width=0.45\\textwidth ]{fig17.eps} \n\\caption{(Color online)\nBand structure  of $[\\epsilon_{\\rm DFT}-V_{\\rm xc}+\\Sigma-(G^{(0)}W_{\\rm GW}^{\\rm dyn})^{\\rm nonlocal}]Z'Z_B$ of SrVO$_3$\n(Z' is the renormalization factor of $\\Sigma-(G^{(0)}W_{\\rm GW}^{\\rm dyn})^{\\rm nonlocal}$).\nFor comparison, the LDA band structure is also given\n(black dotted line).\nThe energy is measured from the Fermi level.\n}\n\\label{GW+ZB}\n\\end{figure} \n\n\n\nThe resulting effective Hamiltonian in the L-space \nconsists of single-particle and two-particle (interaction) parts.\nThe effective interaction is the same as in previous estimates \nby the cRPA in the literature, while the single-particle dispersion \nis revised after removing the double counting and taking into \naccount the frequency dependence of the effective interaction. The \nfinal effective bandwidth for SrVO$_3$ is,\nafter partial cancellation, \nslightly (14\\%) larger than the LDA bandwidth.  \nThis is a reasonable result because the LDA takes into account all \ncorrelations though insufficiently, while the present scheme by the constrained self-energy excludes the \ncorrelation effects arising from the L-space.\nAlthough the bandwidth derived for the effective model is slightly \nlarger than that of the LDA,  it is clear that the correlation effects \nare stronger than in the LDA or standard GW when the {\\it ab initio} \nmodel is solved by an accurate solver. In fact, if the effective \ninteraction contained in the final effective model is treated by \nthe GW scheme, one obtains the dispersion illustrated in \nFig.~\\ref{GW+ZB}, \nwhich is given from the self energy of the whole GW calculation $\\Sigma$ \nby correcting the local dynamical part $W-W_{\\rm L}$ by $Z_B$. \nThis indicates that even an insufficient treatment of the local \nstatic interaction by the GW scheme gives a dispersion with the bandwidth ($\\sim 1.9$ eV) narrower \nthan the LDA ($\\sim 2.5$ eV, Fig.~\\ref{lda}) and GW ($\\sim 2.1$ eV, Fig.~\\ref{fullGW}) dispersions.  \nA slightly ($\\sim 14$\\%) wider dispersion than that of the LDA bands obtained for the \neffective Hamiltonian accompanied with frequency independent effective \nCoulomb interactions without the exchange part may account for the slight \noverestimate of correlation effects in the literature mentioned in \nthe introduction.\nOur scheme offers an optimized way for the  derivation of \n{\\it ab initio} models for the L-space after eliminating the H-space\nin  a systematic fashion.\n\n\nOur findings of the band widening are consistent with studies\nbased on the combined GW+DMFT method in the literature \n\\cite{TomczakPRB2014}: There, it was argued that within GW+DMFT\nthe best effective Hamiltonian that DMFT should be performed on,\nis a one-body Hamiltonian corrected by the nonlocal part of the\nGW self-energy. The corresponding spectral function \n(see e.g.\nFigure 5 of Ref.~\\onlinecite{TomczakPRB2014}) displays \na broadening similar to \nFig.~\\ref{DFT+Sigma_HL_GW}.\nMost interestingly, our present calculations confirm a pronounced\nasymmetry observed in Ref.~\\onlinecite{TomczakPRB2014}, namely a stronger widening\neffect in the unoccupied part of the spectrum than in the occupied\none.\n\n",
        "label": "fullGW",
        "Descriptive_question1": "What is the bandwidth of the t2g-states in the one-shot GW approximation for SrVO3 in figure_3?",
        "Descriptive_question2": "What type of line represents the LDA band structure in figure_3?",
        "Reasoning_question1": "How does the bandwidth of the t2g-states in the one-shot GW approximation compare to the LDA bandwidth in figure_3, and what might this indicate about electronic correlations?",
        "Reasoning_question2": "Why might the overall shape of the dispersion in the one-shot GW approximation remain similar to the LDA in figure_3 despite the reduction in bandwidth?",
        "Descriptive_answer1": "2.1 eV",
        "Descriptive_answer2": "black dotted line",
        "Reasoning_answer1": "The bandwidth of the t2g-states in the one-shot GW approximation is 2.1 eV, which is narrower than the LDA bandwidth of 2.5 eV. This indicates that electronic correlations, captured beyond LDA by GW, lead to a reduction in bandwidth, reflecting band narrowing effects due to many-body interactions.",
        "Reasoning_answer2": "The overall shape of the dispersion remains similar to the LDA because the GW approximation modifies the electronic self-energy primarily by adjusting the bandwidth (band narrowing) rather than drastically altering the band dispersion features. This suggests that the quasiparticle dispersion relations in GW retain similar qualitative features to LDA but include improved treatment of electronic correlations affecting band widths."
    },
    {
        "paper_id": "1511.03757.json",
        "image_id": "figure_4",
        "image_path": [
            "/home/yz979/scratch/chengye/scimolmo/test-images-tables/dataset_imagestables/1511.03757/images/fig4.eps"
        ],
        "caption": "(Color online)\nBand structure of SrVO$_3$ corresponding to the LDA Hamiltonian\nfrom which the LDA exchange-correlation potential has been subtracted.\nFor comparison, the LDA band structure is also given\n(black dotted line).\nThe energy is measured from the Fermi level.\n",
        "classify": "Chart",
        "section_info": "6 Results\n\\section{Results}\n\\label{results}\n\n\nThe ternary 3d$^1$ transition metal oxide SrVO$_3$ has become\none of the ``drosophila compounds'' of correlated systems.\nIt is a correlated metal\nthat has been characterized using various experimental \n\\cite{Onoda, inoue, maiti, maiti2, sekiyama, takizawa, yoshida2010}.\nand theoretical techniques, see e.g.\n\\cite{nekrasov, anisimov, pavarini-d1, liebsch, lechermann-d1, trimarchi, aichhorn09, karolak, lee, held-optics, tomczak-gwdmft, gatti-guzzo, sakuma14, TomczakPRB2014}.\nA review of most of the available experimental and theoretical \ndata has been given recently in Ref.~\\onlinecite{TomczakPRB2014}.\nSrVO$_3$ displays Fermi liquid behavior up to\nremarkably high temperatures of the order of $200$ K \\cite{Onoda, inoue},\nwith a moderate mass enhancement of the order \nof $2$ \\cite{yoshida2010, vandermarel}.\nDetailed spectroscopic investigations have made it an\nideal test compound for modern many-body calculations, and \nmore and more refined dynamical mean field-based studies \nare available.\nThe majority of studies so far have focused on the t$_{2g}$-manifold\nwhich forms the states close to the Fermi level, and those will\nalso be the focus in the present investigations. \nNote however that this restriction quite severely limits the\nrange of validity of the low-energy description. Indeed, as\nshown recently~\\cite{tomczak-gwdmft, TomczakPRB2014} at\nenergies of about 2 to 3 eV above the Fermi level\nthe spectral properties are largely determined by the e$_g$\nstates. This has in particular led to a reinterpretation of\nan inverse photoemission feature at about 2.5 eV that was \nfrequently interpreted as an upper Hubbard band of t$_{2g}$\ncharacter in the earlier literature.\nHere, we use the compound only as an illustration of \nthe principles of constructing effective low-energy models,\nwithout being concerned with a description of spectra beyond\nthe pure t$_{2g}$ part.\n\n\nThe LDA band structure of SrVO$_3$ is shown in Fig.~\\ref{lda}.\nOne clearly distinguishes the three-fold degenerate manifold\nof t$_{2g}$ bands close to the Fermi level (chosen as the\nzero of energy), followed in the unoccupied part of the spectrum\nby the two e$_g$ bands. At about -2.5 eV the filled O-2p derived\nbands are visible.\n\nWhen a standard GW calculation is performed, see Fig.~\\ref{fullGW},\nthe bandwidth of the t$_{2g}$-states is reduced from 2.5 eV to\n2.1 eV, while the overall shape of the dispersion remains similar\nto the LDA one.\nThis is in agreement with previous GW calculations in the literature\n\\cite{tomczak-gwdmft, TomczakPRB2014, gatti-guzzo, sakuma14}.\n\n\n\nThe calculation is based on the full-potential linear muffin-tin orbitals (FP-LMTO) implementation.\n8$\\times$8$\\times$8 \\bm{k}-mesh is employed in both the DFT/LDA and the GW calculations.\nThe calculational details are the same as in Ref.~\\onlinecite{Hirayama}.\n\n\nWe now turn to a discussion of the band structure corresponding to \nthe Hamiltonians to be used as input for subsequent many-body \ncalculations for this material.\nWe will proceed step by step to analyze the influence of the various\ncorrection terms, with respect to the DFT starting point\n(which will be overlaid to the respective band structures).\n\n\n\n\\begin{figure}[h]\n\\centering \n\\includegraphics[clip,width=0.45\\textwidth ]{fig2.eps} \n\\caption{(Color online)\nKohn-Sham band structure of SrVO$_3$ in the LDA.\nThe energy is measured from the Fermi level.\n}\n\\label{lda}\n\\end{figure} \n\n\n\\begin{figure}[h]\n\\centering \n\\includegraphics[clip,width=0.45\\textwidth ]{fig3.eps} \n\\caption{(Color online)\nBand structure  of SrVO$_3$ in the one-shot GW approximation.\nFor comparison, the LDA band structure is also given\n(black dotted line).\nThe energy is measured from the Fermi level.\n}\n\\label{fullGW}\n\\end{figure} \n\n\n\n\n\n\n\\begin{figure}[h]\n\\centering \n\\includegraphics[clip,width=0.45\\textwidth ]{fig4.eps} \n\\caption{(Color online)\nBand structure of SrVO$_3$ corresponding to the LDA Hamiltonian\nfrom which the LDA exchange-correlation potential has been subtracted.\nFor comparison, the LDA band structure is also given\n(black dotted line).\nThe energy is measured from the Fermi level.\n}\n\\label{lda-vxc}\n\\end{figure} \n\n\n\\begin{figure}[h]\n\\centering \n\\includegraphics[clip,width=0.45\\textwidth ]{fig5.eps} \n\\caption{(Color online)\nBand structure  of SrVO$_3$ in the Hartree approximation.\nFor comparison, the LDA band structure is also given\n(black dotted line).\nThe energy is measured from the Fermi level.\n}\n\\label{hartree}\n\\end{figure} \n\n\nFigure~\\ref{lda-vxc} \ndisplays the band structure corresponding to\nthe LDA Hamiltonian from which the \nLDA exchange-correlation\npotential has been subtracted,  \nthat is, the eigenvalues of $H_{\\rm LDA}-V_{\\rm xc}$ where\nthese operators are evaluated for the self-consistent LDA\ndensity.\nThe subtraction of $V_{\\rm xc}$ widens the band structure from \nthe LDA bandwidth of 2.5 eV \nto more than twice this value: the new band\nwidth is 5.4 eV.\nThis indicates that the exchange-correlation potential $V_{\\rm xc}$ \nof the LDA is responsible for a substantial amount of \nthe band-narrowing effect\narising from electronic correlations, stressing the need \nto subtract the double counting in a consistent manner. \nFigure~\\ref{lda-vxc} is regarded as the starting point for the\nfollowing considerations.\n\nFor simple materials (see e.g. the calculations on Li in Ref.~\\onlinecite{Lithium})\nit has been noted in the literature that the Hartree band structure\nis close to the DFT one. This raises the question of the origin of the\nsubstantial band widening in the present case. Indeed, the present\ncalculation shown in Fig.~\\ref{lda-vxc} differs from a Hartree calculation\nonly \nby the fact that Fig.~\\ref{lda-vxc} is evaluated for the converged LDA density.\nWe have performed a test calculation where we plot the Hartree band\nstructure calculated for the converged Hartree density. The result\nis shown in Fig.~\\ref{hartree}. As seen from this plot, while the\nband is not fully as wide as in Fig.~\\ref{lda-vxc}, a substantial \nwidening is already present at this stage.\n\n\n\n\\begin{figure}[h]\n\\centering \n\\includegraphics[clip,width=0.45\\textwidth ]{fig6.eps} \n\\caption{(Color online)\nBand structure  of $H_{\\rm LDA}-V_{\\rm xc}+\\Delta \\Sigma_{\\rm H}(\\omega=0)$ of SrVO$_3$.\nFor comparison, the LDA band structure is also given\n(black dotted line).\nThe energy is measured from the Fermi level.\n}\n\\label{lda-vxc-sigH(0)}\n\\end{figure} \n\n\n\nStarting from the Hamiltonian without V$_{xc}$ (see the \ndispersion in Fig.~\\ref{lda-vxc}), we first take into account \nthe static part of the corrective self-energy $\\Delta \\Sigma_{\\rm H}$:\nIn Fig.~\\ref{lda-vxc-sigH(0)}, we plot the dispersion corresponding to\n$H_{\\rm LDA}-V_{\\rm xc}+\\Delta \\Sigma_{\\rm H}(\\omega=0)$. \nCompared to the dispersion of Fig.~\\ref{lda-vxc}, the overall band structure \nis narrowed to 3.7 eV. This value is, however, still considerably larger\nthan the LDA bandwidth (Fig.~\\ref{lda}). Although\n the electronic correlations coupling the H- and \nthe L space are included in Fig.~\\ref{lda-vxc-sigH(0)}, and narrow the band with respect to the case\nwhere V$_{xc}$ is taken out, correlations within\nthe L space are not included. \nThe LDA, on the other hand, at least partially\nincludes correlations within the L space, \nand these are very effective in narrowing the band.\nInterestingly, the bottom of the occupied band is quite\nexactly at the LDA value, and the remaining widening is\npurely in the unoccupied part.\n\n\n\\begin{figure*}[htb]\n\\begin{center} \n\\includegraphics[width=0.9\\textwidth ]{fig7.eps} \n\\end{center} \n\\caption{(Color online)\nFrequency dependence of $-V_{xc}+\\Delta \\Sigma_{\\rm H}$ for SrVO$_3$.\n(a) and (b) are the real and imaginary parts at the $\\Gamma$ point, \nrespectively,\nand (c) and (d) ((e) and (f)) are those at $X$ ($R$).\nFor comparison, the full GW self-energies are also given\n(black dotted line).\n}\n\\label{sigH}\n\\end{figure*} \n\n\n\n\\begin{figure}[h]\n\\centering \n\\includegraphics[clip,width=0.45\\textwidth ]{fig8.eps} \n\\caption{(Color online)\nBand structure  of $Z_{\\rm H}[H_{\\rm LDA}-V_{\\rm xc}+\\Delta\\Sigma_{\\rm H}(\\omega=0)]$ of SrVO$_3$.\nFor comparison, the LDA band structure is also given\n(black dotted line).\nThe energy is measured from the Fermi level.\n}\n\\label{ZH_SigmaH}\n\\end{figure} \n\n\nTo go further and in particular to analyze the dynamic\nbehavior of $\\Delta \\Sigma_{\\rm H}$ \nwe plot in Fig.~\\ref{sigH} \nthe self-energy corrections $\\Delta \\Sigma_{\\rm H}$ (with $\\Sigma$ (see Eq.~(\\ref{full-self-energy}))\nfor comparison) for the real and imaginary parts\nat several representative choices of momenta.\nThe frequency dependence is smooth around the Fermi level.\nIn particular, in contrast to the full self-energy,\nthere are no poles in $\\Delta \\Sigma_{\\rm H}$ \nin the low-energy region, thanks to the\nexclusion of fluctuations within the L-space.\nThe frequency dependence of the real part indicates that the \nlinearization \n$\\displaystyle{\\Delta \\Sigma_{\\rm H}(\\omega) \\sim \\Delta \\Sigma_{\\rm H}(\\omega=0)+[d\\Delta \\Sigma_{\\rm H}(\\omega)/d\\omega]_{\\omega=0}\\omega }$\noffers a reasonably good approximation. \nIn fact, the behavior of the constrained self-energy\nis much closer to linearity than that of \nthe full GW self-energy $\\Sigma$.\nThis is easily understood by the fact that the low-energy excitations are excluded in the present constrained self-energy $\\Delta\\Sigma_{\\rm H}$ analogously to an insulator, thus eliminating the strong frequency dependence of the constrained self-energy near the Fermi level. This is one of the consequences of the \ncontrollability and an advantage of the present MACE scheme, as \nmentioned in the introduction.\nIn the following discussion, we employ this linearized approximation.\nThen the renormalization factor $Z_{\\rm H}$ defined by Eq.~(\\ref{ZH})\nis interpreted as that from the contribution of the H-space. \nAfter \nthis renormalization factor is taken into account,\nthe effective Hamiltonian is given by $Z_{\\rm H}[H_{\\rm LDA}-V_{\\rm xc}+\\Delta\\Sigma_{\\rm H}(\\omega=0)]$ (see Fig.~\\ref{ZH_SigmaH}). \nInterestingly the renormalization factor $Z_{\\rm H}$=0.92 stays\nclose to one, so that\nthe bandwidth is only slightly reduced (to\n$3.4$ eV instead of the $3.7$ eV above). \nWe also note that $Z_{\\rm H}$ has weak momentum dependence\nas we reveal in the following.\n\n\n\n\\begin{figure*}[htb]\n\\begin{center} \n\\includegraphics[width=0.9\\textwidth ]{fig9.eps} \n\\end{center} \n\\caption{(Color online)\n{\\bf minor modification}\nFrequency dependence of $-V_{\\rm xc}+\\Delta \\Sigma_{\\rm H}$ of SrVO$_3$.\n(a) and (b) are the real and imaginary parts of the local\ncomponent, respectively.\nThe black dotted line is $\\Sigma$ defined in Eq.~(\\ref{full-self-energy}) in the full GW approximation for comparison.\n(c) and (d) are the real and imaginary \nparts at several $k$ points.\n}\n\\label{sigH_nonlocal}\n\\end{figure*} \n\n\n\nFigure~\\ref{sigH_nonlocal} \nshows the local and nonlocal parts of $\\Delta \\Sigma_{\\rm H}$.\nThe frequency \ndependence\nof the local part (Figs.~\\ref{sigH_nonlocal} (a)(b)) \nis quite similar \nto the  frequency-dependent self-energies at various momenta\nin Fig.~\\ref{sigH}.\nOne immediately reads off an interesting property, which is\nakin to what has been found for the full GW self-energy in\nRef.~\\onlinecite{Tomczak-PRL2012}, namely that the nonlocal part of \n$\\Delta \\Sigma_{\\rm H}$ shows only weak frequency dependence \nas shown in Figs.~\\ref{sigH_nonlocal} (c)(d). \nThis explains why $Z_{\\rm H}$ \nis only weakly momentum-dependent.\nOn the other hand, $\\Delta \\Sigma_{\\rm H}^{\\rm nonlocal} (0)$ at $R$ is $\\sim 1$ eV larger than \nat $\\Gamma$, which causes the band widening effect.\n $\\Delta \\Sigma_{\\rm H}$\ncan thus to a good approximation be decomposed into a\nfrequency-dependent local part and a static nonlocal one:\n$\\Delta \\Sigma_{\\rm H}(k,\\omega) = \\Delta \\Sigma^{\\rm local}_{\\rm H}(\\omega) + \\Delta \\Sigma^{\\rm nonlocal}_{\\rm H}(k)$,\nwith $ \\Delta \\Sigma_{\\rm H}^{\\rm local}(\\omega) = \\sum_q  \\Delta \\Sigma_{\\rm H}(q,\\omega)$\n\nSince in the simple case of SrVO$_3$ where the t$_{2g}$ states are\ndegenerate, static local operators are scalar and are compensated \nby a chemical potential shift such that the correct particle\nnumber is obtained. The above band structure corresponding to\n$H_{\\rm LDA} - V_{\\rm xc} + \\Delta \\Sigma_{\\rm H}(k, \\omega=0)$ \nis thus identical to that of\n$H_{\\rm LDA} - V_{\\rm xc} + \\Delta \\Sigma^{\\rm nonlocal}_{\\rm H}(k)$.\nThe local dynamical part of $\\Delta \\Sigma_{\\rm H}$ then results in a \nnarrowing of this band structure by a factor $Z_{\\rm H} = 0.92$.\n\n\n\n\\begin{figure}[h]\n\\centering \n\\includegraphics[clip,width=0.45\\textwidth ]{fig10.eps} \n\\caption{(Color online)\n(a) Frequency dependence of the on-site effective interaction.\nPanel (b) is a low-energy zoom of panel (a). \n}\n\\label{W}\n\\end{figure} \n\n\nBefore turning to a discussion of the low-energy correction for the GW treatment\n$\\Delta \\Sigma_{\\rm L} =G^{(0)}(W-W_{\\rm L})$, we analyze the effective\ninteraction $W_{\\rm L}$ shown in Fig.~\\ref{W} \nin comparison to $W$ and $W_{\\rm H}$.\nThe fully screened Coulomb interaction $W$ displays the familiar\nshape of an interaction that is strongly screened at low energies\n(with a value of $0.88$ eV at $\\omega=0$), while retrieving the\nvalue of the bare Coulomb interaction $v$ ($\\sim 16.0$ eV) \nat high energy.\nThe crossover from the bare to the  \nscreened values takes place at the plasma energy of \nabout $15$ eV. This behavior has been discussed before\n\\cite{Review_SB2014}; we note in particular that the plasma \nfrequency is known from electron energy loss spectroscopy \nmeasurements of the related SrTiO$_3$ compound \\cite{vandermarel,kohiki}.\nThe partially screened interaction\n$W_{\\rm H}$ constructed within cRPA\nconverges to $W$ at high energies, but displays\nweaker screening effects at low energies (with a value of $3.5$ eV at $\\omega=0$),\nsince intra-t$_{2g}$ screening channels are excluded.\nAs was already anticipated in Sec.~\\ref{Outline},  $W_{\\rm L}$ can be interpreted as the screened interaction of \na low-energy model where a static interaction of value $W_{\\rm H}(\\omega=0)$\nhas been imposed as the bare interaction. Since now {\\it only} intra-t$_{2g}$\nscreening channels are active, screening takes place only at low\nenergies, where the scale is given by the t$_{2g}$ bandwidth.\nAlso plotted is the difference $W_{\\rm GW}^{\\rm dyn}=W-W_{d}$: except at low energies\nwhere the t$_{2g}$ screening channels come into play, its frequency\ndependence is essentially given by that of $W$, while the\nhigh-energy limit is the bare Coulomb interaction $v$ reduced\nby $W_{\\rm H}(0)$.\n\n\n\\begin{figure*}[htb]\n\\begin{center} \n\\includegraphics[width=0.9\\textwidth ]{fig11.eps} \n\\end{center} \n\\caption{(Color online)\nFrequency dependence of $-V_{\\rm xc}+ G^{(1)}W_{\\rm H}^{\\rm dyn}$ of SrVO$_3$.\n(a) and (b) are the real and imaginary local parts, respectively.\n(c) and (d) are the real and imaginary \nparts at several $k$ points.\n}\n\\label{DeltaSigma_L_Pert}\n\\end{figure*} \n\n\n\n\\begin{figure*}[htb]\n\\begin{center} \n\\includegraphics[width=0.9\\textwidth ]{fig12.eps} \n\\end{center} \n\\caption{(Color online)\nFrequency dependence of $-V_{\\rm xc}+ G^{(0)}W_{\\rm GW}^{\\rm dyn}$ of SrVO$_3$.\n(a) and (b) are the real and imaginary local parts, respectively.\n(c) and (d) are the real and imaginary \nparts at several $k$ points.\n}\n\\label{DeltaSigma_L_GW}\n\\end{figure*} \n\n\n\nThe frequency dependence of the real and imaginary parts of the low-energy self-energy $\\Delta \\Sigma_{\\rm L} = G^{(1)}W_{\\rm H}^{\\rm dyn}$ for the direct perturbative treatment is illustrated in Fig.~\\ref{DeltaSigma_L_Pert}. \n$\\Delta \\Sigma_{\\rm L} = G^{(0)}W_{\\rm GW}^{\\rm dyn}$ used\nfor the GW treatment is shown in Fig.~\\ref{DeltaSigma_L_GW}\nfor several choices of momenta.\n\\\\\nThe results for the perturbative and the GW treatment are nearly \nidentical.\nThe renormalization factor of $\\Delta \\Sigma_{\\rm L}^{\\rm nonlocal}$ \n(and $\\Delta \\Sigma_{\\rm L}$) is $\\sim 0.77$.\nThe zero-frequency shift\n${\\rm Re}\\Delta \\Sigma_{\\rm L}^{\\rm nonlocal} (0)$ at $R$ is $\\sim 3$ eV \nlarger than \nat $\\Gamma$.\nAgain, one sees how $\\Sigma_{\\rm L}$ separates into local dynamical\nand nonlocal static parts: \n$\\Delta \\Sigma_{\\rm L} = \\Delta \\Sigma^{\\rm nonlocal}_{\\rm L}(k) + \n\\Delta \\Sigma^{\\rm local}_{\\rm L}(\\omega)$.\nThis is expected since such a separation has been found within\nthe full GW calculation \\cite{TomczakPRB2014}, and is even\nmore plausible for a GW treatment within the t$_{2g}$ subspace.\n\n\n\n\\begin{figure}[h]\n\\centering \n\\includegraphics[clip,width=0.45\\textwidth ]{fig13.eps} \n\\caption{(Color online)\nBand structure  of $[\\epsilon_{\\rm DFT}-V_{\\rm xc}+\\Delta\\Sigma_{\\rm H}+\\Delta \\Sigma_{\\rm L}^{\\rm nonlocal}]Z_{\\rm HL}$ of SrVO$_3$ in the direct perturbative treatment.\nFor comparison, the LDA band structure is also given\n(black dotted line).\nThe energy is measured from the Fermi level.\n}\n\\label{DFT+Sigma_HL_Pert}\n\\end{figure} \n\n\n\n\\begin{figure}[h]\n\\centering \n\\includegraphics[clip,width=0.45\\textwidth ]{fig14.eps} \n\\caption{(Color online)\nBand structure  of $[\\epsilon_{\\rm DFT}-V_{\\rm xc}+\\Delta\\Sigma_{\\rm H}+\\Delta \\Sigma_{\\rm L}^{\\rm nonlocal}]Z_{\\rm HL}$ of SrVO$_3$ in the GW treatment.\nFor comparison, the LDA band structure is also given\n(black dotted line).\nThe energy is measured from the Fermi level.\n}\n\\label{DFT+Sigma_HL_GW}\n\\end{figure} \n\n\n\nThe corresponding band structures \nare given by \n $[\\epsilon_{\\rm DFT}+\\Delta\\Sigma_{\\rm H}+\\Delta \\Sigma_{\\rm L}^{\\rm nonlocal}]Z_{\\rm HL}$ in Fig.~\\ref{DFT+Sigma_HL_Pert} \nfor the perturbative treatment (see Eq.~(\\ref{DeltaSigmaLnonlocalPert})), and in Fig.~\\ref{DFT+Sigma_HL_GW} \nfor the GW treatment,\nwhere $\\Delta \\Sigma_{\\rm L}^{\\rm nonlocal}$ is defined in Eq.~(\\ref{DeltaSigLnonlocalGW}) and $Z_{\\rm HL}$ is around 0.92.\n$Z_{\\rm HL}$ is practically the same as $Z_{\\rm H}$, meaning that the nonlocal dynamical correction \nin the H-space is small, which justifies the GW perturbative treatment for the nonlocal part.\nOne finds that \nthe nonlocal part $\\Delta \\Sigma_{\\rm L}=G^{(0)}_{ll}(W-W_{\\rm L})$ narrows\nthe empty states but widens the occupied ones, resulting\nin an overall bandwidth of 3.2 eV.\nThe difference between the direct perturbation and the GW treatment is small.\n\n\n\n\\begin{figure}[h]\n\\centering \n\\includegraphics[clip,width=0.45\\textwidth ]{fig15.eps} \n\\caption{(Color online)\nBand structure  of $[\\epsilon_{\\rm DFT}-V_{\\rm xc}+\\Delta\\Sigma_{\\rm H}+\\Delta\\Sigma_{\\rm L}^{\\rm nonlocal}]Z_{\\rm HL}Z_B$ of SrVO$_3$ in the GW treatment.\nFor comparison, the LDA band structure is also given\n(black dotted line).\nThe energy is measured from the Fermi level.}\n\\label{DFT+DeltaSigmaL_GW_ZHL_ZB}\n\\end{figure} \n\n\nAfter including the effect of the local self-energy by the nonperturbative \nCasula trick, we show the dispersion\ngiven by \n$[\\epsilon_{\\rm DFT}-V_{\\rm xc}+\\Delta\\Sigma_{\\rm H}+\\Delta\\Sigma_{\\rm L}^{\\rm nonlocal}]Z_{\\rm HL}Z_B$ for the GW-type treatment in Fig.~\\ref{DFT+DeltaSigmaL_GW_ZHL_ZB}: \nThe GW-like treatment gives a band dispersion\nclose to the LDA one. \nThe $Z_B$ factor corresponding to the local dynamical part of \n$\\Delta \\Sigma_{\\rm L}$ amounts to $Z_B=0.7$.\nThe nonperturbative treatment \nresults in an LDA-like band dispersion for the empty states,\nbut a slightly narrower bandwidth in the occupied part.\nThe low-energy effective Hamiltonian at this level of treatment has a \nfrequency independent effective interaction both with local \nand nonlocal interaction given by the Fourier transform of \n$W_{\\rm H}(q,\\omega=0)$, which contains both direct and exchange interactions. \n\n\n\\begin{figure}[h]\n\\centering \n\\includegraphics[clip,width=0.45\\textwidth ]{fig16.eps} \n\\caption{(Color online)\nBand structure  of $[\\epsilon_{\\rm DFT}+\\Sigma_{\\rm corr}^{\\rm GW}]Z_{\\rm corr}Z_B$ of SrVO$_3$.\nFor comparison, the LDA band structure is also given\n(black dotted line).\nThe energy is measured from the Fermi level.}\n\\label{DFT+Sigma_corr_GW_Z}\n\\end{figure} \n\n\n\nWe finally show the dispersion\ngiven by Eq.~(\\ref{Teff_final}), namely\n$[\\epsilon_{\\rm DFT}+\\Sigma_{\\rm corr}^{\\rm GW}]Z_{\\rm corr}Z_B$ in Fig.~\\ref{DFT+Sigma_corr_GW_Z}. \nThe result shows an LDA-like band dispersion for the occupied states,\nbut a slightly wider bandwidth in the empty part, resulting in a bandwidth 14\\% wider in total than the LDA bandwidth. \nThe effective interaction of the low-energy effective Hamiltonian \nat this final level has\nfrequency independent local and nonlocal interactions \ngiven by the Fourier transform of $W_r(q,\\omega=0)$. \n\n\n\n\\begin{figure}[h]\n\\centering \n\\includegraphics[clip,width=0.45\\textwidth ]{fig17.eps} \n\\caption{(Color online)\nBand structure  of $[\\epsilon_{\\rm DFT}-V_{\\rm xc}+\\Sigma-(G^{(0)}W_{\\rm GW}^{\\rm dyn})^{\\rm nonlocal}]Z'Z_B$ of SrVO$_3$\n(Z' is the renormalization factor of $\\Sigma-(G^{(0)}W_{\\rm GW}^{\\rm dyn})^{\\rm nonlocal}$).\nFor comparison, the LDA band structure is also given\n(black dotted line).\nThe energy is measured from the Fermi level.\n}\n\\label{GW+ZB}\n\\end{figure} \n\n\n\nThe resulting effective Hamiltonian in the L-space \nconsists of single-particle and two-particle (interaction) parts.\nThe effective interaction is the same as in previous estimates \nby the cRPA in the literature, while the single-particle dispersion \nis revised after removing the double counting and taking into \naccount the frequency dependence of the effective interaction. The \nfinal effective bandwidth for SrVO$_3$ is,\nafter partial cancellation, \nslightly (14\\%) larger than the LDA bandwidth.  \nThis is a reasonable result because the LDA takes into account all \ncorrelations though insufficiently, while the present scheme by the constrained self-energy excludes the \ncorrelation effects arising from the L-space.\nAlthough the bandwidth derived for the effective model is slightly \nlarger than that of the LDA,  it is clear that the correlation effects \nare stronger than in the LDA or standard GW when the {\\it ab initio} \nmodel is solved by an accurate solver. In fact, if the effective \ninteraction contained in the final effective model is treated by \nthe GW scheme, one obtains the dispersion illustrated in \nFig.~\\ref{GW+ZB}, \nwhich is given from the self energy of the whole GW calculation $\\Sigma$ \nby correcting the local dynamical part $W-W_{\\rm L}$ by $Z_B$. \nThis indicates that even an insufficient treatment of the local \nstatic interaction by the GW scheme gives a dispersion with the bandwidth ($\\sim 1.9$ eV) narrower \nthan the LDA ($\\sim 2.5$ eV, Fig.~\\ref{lda}) and GW ($\\sim 2.1$ eV, Fig.~\\ref{fullGW}) dispersions.  \nA slightly ($\\sim 14$\\%) wider dispersion than that of the LDA bands obtained for the \neffective Hamiltonian accompanied with frequency independent effective \nCoulomb interactions without the exchange part may account for the slight \noverestimate of correlation effects in the literature mentioned in \nthe introduction.\nOur scheme offers an optimized way for the  derivation of \n{\\it ab initio} models for the L-space after eliminating the H-space\nin  a systematic fashion.\n\n\nOur findings of the band widening are consistent with studies\nbased on the combined GW+DMFT method in the literature \n\\cite{TomczakPRB2014}: There, it was argued that within GW+DMFT\nthe best effective Hamiltonian that DMFT should be performed on,\nis a one-body Hamiltonian corrected by the nonlocal part of the\nGW self-energy. The corresponding spectral function \n(see e.g.\nFigure 5 of Ref.~\\onlinecite{TomczakPRB2014}) displays \na broadening similar to \nFig.~\\ref{DFT+Sigma_HL_GW}.\nMost interestingly, our present calculations confirm a pronounced\nasymmetry observed in Ref.~\\onlinecite{TomczakPRB2014}, namely a stronger widening\neffect in the unoccupied part of the spectrum than in the occupied\none.\n\n",
        "label": "lda-vxc",
        "Descriptive_question1": "What is the bandwidth of the band structure in figure_4 after subtracting the LDA exchange-correlation potential?",
        "Descriptive_question2": "What type of line represents the LDA band structure in figure_4?",
        "Reasoning_question1": "Why does the band structure in figure_4 show a significant widening compared to the LDA bandwidth, and what does this indicate about the role of the exchange-correlation potential?",
        "Reasoning_question2": "How does the subtraction of the LDA exchange-correlation potential in figure_4 affect the electronic correlations compared to the standard LDA band structure?",
        "Descriptive_answer1": "5.4 eV",
        "Descriptive_answer2": "black dotted line",
        "Reasoning_answer1": "The band structure in figure_4 corresponds to the LDA Hamiltonian with the LDA exchange-correlation potential subtracted. This subtraction removes a key part of the band-narrowing effect caused by electronic correlations included in the exchange-correlation potential. Consequently, the bandwidth expands from the original 2.5 eV in the LDA to 5.4 eV when this potential is removed. This significant widening indicates that the LDA exchange-correlation potential is responsible for a considerable amount of the band-narrowing observed in the standard LDA calculation, highlighting its role in capturing electronic correlation effects that reduce the bandwidth.",
        "Reasoning_answer2": "Subtracting the LDA exchange-correlation potential essentially removes the correlations accounted for within the LDA, resulting in a band structure that is significantly wider, as seen in figure_4. This means that the LDA exchange-correlation potential acts to effectively narrow the bands by incorporating electron-electron interactions. Therefore, the resultant band structure after subtraction reflects a less correlated system with diminished renormalization effects on the bands, indicating that the standard LDA band structure partially accounts for electronic correlations that influence the bandwidth and effective mass of charge carriers."
    },
    {
        "paper_id": "1511.03757.json",
        "image_id": "figure_5",
        "image_path": [
            "/home/yz979/scratch/chengye/scimolmo/test-images-tables/dataset_imagestables/1511.03757/images/fig5.eps"
        ],
        "caption": "(Color online)\nBand structure  of SrVO$_3$ in the Hartree approximation.\nFor comparison, the LDA band structure is also given\n(black dotted line).\nThe energy is measured from the Fermi level.\n",
        "classify": "Chart",
        "section_info": "6 Results\n\\section{Results}\n\\label{results}\n\n\nThe ternary 3d$^1$ transition metal oxide SrVO$_3$ has become\none of the ``drosophila compounds'' of correlated systems.\nIt is a correlated metal\nthat has been characterized using various experimental \n\\cite{Onoda, inoue, maiti, maiti2, sekiyama, takizawa, yoshida2010}.\nand theoretical techniques, see e.g.\n\\cite{nekrasov, anisimov, pavarini-d1, liebsch, lechermann-d1, trimarchi, aichhorn09, karolak, lee, held-optics, tomczak-gwdmft, gatti-guzzo, sakuma14, TomczakPRB2014}.\nA review of most of the available experimental and theoretical \ndata has been given recently in Ref.~\\onlinecite{TomczakPRB2014}.\nSrVO$_3$ displays Fermi liquid behavior up to\nremarkably high temperatures of the order of $200$ K \\cite{Onoda, inoue},\nwith a moderate mass enhancement of the order \nof $2$ \\cite{yoshida2010, vandermarel}.\nDetailed spectroscopic investigations have made it an\nideal test compound for modern many-body calculations, and \nmore and more refined dynamical mean field-based studies \nare available.\nThe majority of studies so far have focused on the t$_{2g}$-manifold\nwhich forms the states close to the Fermi level, and those will\nalso be the focus in the present investigations. \nNote however that this restriction quite severely limits the\nrange of validity of the low-energy description. Indeed, as\nshown recently~\\cite{tomczak-gwdmft, TomczakPRB2014} at\nenergies of about 2 to 3 eV above the Fermi level\nthe spectral properties are largely determined by the e$_g$\nstates. This has in particular led to a reinterpretation of\nan inverse photoemission feature at about 2.5 eV that was \nfrequently interpreted as an upper Hubbard band of t$_{2g}$\ncharacter in the earlier literature.\nHere, we use the compound only as an illustration of \nthe principles of constructing effective low-energy models,\nwithout being concerned with a description of spectra beyond\nthe pure t$_{2g}$ part.\n\n\nThe LDA band structure of SrVO$_3$ is shown in Fig.~\\ref{lda}.\nOne clearly distinguishes the three-fold degenerate manifold\nof t$_{2g}$ bands close to the Fermi level (chosen as the\nzero of energy), followed in the unoccupied part of the spectrum\nby the two e$_g$ bands. At about -2.5 eV the filled O-2p derived\nbands are visible.\n\nWhen a standard GW calculation is performed, see Fig.~\\ref{fullGW},\nthe bandwidth of the t$_{2g}$-states is reduced from 2.5 eV to\n2.1 eV, while the overall shape of the dispersion remains similar\nto the LDA one.\nThis is in agreement with previous GW calculations in the literature\n\\cite{tomczak-gwdmft, TomczakPRB2014, gatti-guzzo, sakuma14}.\n\n\n\nThe calculation is based on the full-potential linear muffin-tin orbitals (FP-LMTO) implementation.\n8$\\times$8$\\times$8 \\bm{k}-mesh is employed in both the DFT/LDA and the GW calculations.\nThe calculational details are the same as in Ref.~\\onlinecite{Hirayama}.\n\n\nWe now turn to a discussion of the band structure corresponding to \nthe Hamiltonians to be used as input for subsequent many-body \ncalculations for this material.\nWe will proceed step by step to analyze the influence of the various\ncorrection terms, with respect to the DFT starting point\n(which will be overlaid to the respective band structures).\n\n\n\n\\begin{figure}[h]\n\\centering \n\\includegraphics[clip,width=0.45\\textwidth ]{fig2.eps} \n\\caption{(Color online)\nKohn-Sham band structure of SrVO$_3$ in the LDA.\nThe energy is measured from the Fermi level.\n}\n\\label{lda}\n\\end{figure} \n\n\n\\begin{figure}[h]\n\\centering \n\\includegraphics[clip,width=0.45\\textwidth ]{fig3.eps} \n\\caption{(Color online)\nBand structure  of SrVO$_3$ in the one-shot GW approximation.\nFor comparison, the LDA band structure is also given\n(black dotted line).\nThe energy is measured from the Fermi level.\n}\n\\label{fullGW}\n\\end{figure} \n\n\n\n\n\n\n\\begin{figure}[h]\n\\centering \n\\includegraphics[clip,width=0.45\\textwidth ]{fig4.eps} \n\\caption{(Color online)\nBand structure of SrVO$_3$ corresponding to the LDA Hamiltonian\nfrom which the LDA exchange-correlation potential has been subtracted.\nFor comparison, the LDA band structure is also given\n(black dotted line).\nThe energy is measured from the Fermi level.\n}\n\\label{lda-vxc}\n\\end{figure} \n\n\n\\begin{figure}[h]\n\\centering \n\\includegraphics[clip,width=0.45\\textwidth ]{fig5.eps} \n\\caption{(Color online)\nBand structure  of SrVO$_3$ in the Hartree approximation.\nFor comparison, the LDA band structure is also given\n(black dotted line).\nThe energy is measured from the Fermi level.\n}\n\\label{hartree}\n\\end{figure} \n\n\nFigure~\\ref{lda-vxc} \ndisplays the band structure corresponding to\nthe LDA Hamiltonian from which the \nLDA exchange-correlation\npotential has been subtracted,  \nthat is, the eigenvalues of $H_{\\rm LDA}-V_{\\rm xc}$ where\nthese operators are evaluated for the self-consistent LDA\ndensity.\nThe subtraction of $V_{\\rm xc}$ widens the band structure from \nthe LDA bandwidth of 2.5 eV \nto more than twice this value: the new band\nwidth is 5.4 eV.\nThis indicates that the exchange-correlation potential $V_{\\rm xc}$ \nof the LDA is responsible for a substantial amount of \nthe band-narrowing effect\narising from electronic correlations, stressing the need \nto subtract the double counting in a consistent manner. \nFigure~\\ref{lda-vxc} is regarded as the starting point for the\nfollowing considerations.\n\nFor simple materials (see e.g. the calculations on Li in Ref.~\\onlinecite{Lithium})\nit has been noted in the literature that the Hartree band structure\nis close to the DFT one. This raises the question of the origin of the\nsubstantial band widening in the present case. Indeed, the present\ncalculation shown in Fig.~\\ref{lda-vxc} differs from a Hartree calculation\nonly \nby the fact that Fig.~\\ref{lda-vxc} is evaluated for the converged LDA density.\nWe have performed a test calculation where we plot the Hartree band\nstructure calculated for the converged Hartree density. The result\nis shown in Fig.~\\ref{hartree}. As seen from this plot, while the\nband is not fully as wide as in Fig.~\\ref{lda-vxc}, a substantial \nwidening is already present at this stage.\n\n\n\n\\begin{figure}[h]\n\\centering \n\\includegraphics[clip,width=0.45\\textwidth ]{fig6.eps} \n\\caption{(Color online)\nBand structure  of $H_{\\rm LDA}-V_{\\rm xc}+\\Delta \\Sigma_{\\rm H}(\\omega=0)$ of SrVO$_3$.\nFor comparison, the LDA band structure is also given\n(black dotted line).\nThe energy is measured from the Fermi level.\n}\n\\label{lda-vxc-sigH(0)}\n\\end{figure} \n\n\n\nStarting from the Hamiltonian without V$_{xc}$ (see the \ndispersion in Fig.~\\ref{lda-vxc}), we first take into account \nthe static part of the corrective self-energy $\\Delta \\Sigma_{\\rm H}$:\nIn Fig.~\\ref{lda-vxc-sigH(0)}, we plot the dispersion corresponding to\n$H_{\\rm LDA}-V_{\\rm xc}+\\Delta \\Sigma_{\\rm H}(\\omega=0)$. \nCompared to the dispersion of Fig.~\\ref{lda-vxc}, the overall band structure \nis narrowed to 3.7 eV. This value is, however, still considerably larger\nthan the LDA bandwidth (Fig.~\\ref{lda}). Although\n the electronic correlations coupling the H- and \nthe L space are included in Fig.~\\ref{lda-vxc-sigH(0)}, and narrow the band with respect to the case\nwhere V$_{xc}$ is taken out, correlations within\nthe L space are not included. \nThe LDA, on the other hand, at least partially\nincludes correlations within the L space, \nand these are very effective in narrowing the band.\nInterestingly, the bottom of the occupied band is quite\nexactly at the LDA value, and the remaining widening is\npurely in the unoccupied part.\n\n\n\\begin{figure*}[htb]\n\\begin{center} \n\\includegraphics[width=0.9\\textwidth ]{fig7.eps} \n\\end{center} \n\\caption{(Color online)\nFrequency dependence of $-V_{xc}+\\Delta \\Sigma_{\\rm H}$ for SrVO$_3$.\n(a) and (b) are the real and imaginary parts at the $\\Gamma$ point, \nrespectively,\nand (c) and (d) ((e) and (f)) are those at $X$ ($R$).\nFor comparison, the full GW self-energies are also given\n(black dotted line).\n}\n\\label{sigH}\n\\end{figure*} \n\n\n\n\\begin{figure}[h]\n\\centering \n\\includegraphics[clip,width=0.45\\textwidth ]{fig8.eps} \n\\caption{(Color online)\nBand structure  of $Z_{\\rm H}[H_{\\rm LDA}-V_{\\rm xc}+\\Delta\\Sigma_{\\rm H}(\\omega=0)]$ of SrVO$_3$.\nFor comparison, the LDA band structure is also given\n(black dotted line).\nThe energy is measured from the Fermi level.\n}\n\\label{ZH_SigmaH}\n\\end{figure} \n\n\nTo go further and in particular to analyze the dynamic\nbehavior of $\\Delta \\Sigma_{\\rm H}$ \nwe plot in Fig.~\\ref{sigH} \nthe self-energy corrections $\\Delta \\Sigma_{\\rm H}$ (with $\\Sigma$ (see Eq.~(\\ref{full-self-energy}))\nfor comparison) for the real and imaginary parts\nat several representative choices of momenta.\nThe frequency dependence is smooth around the Fermi level.\nIn particular, in contrast to the full self-energy,\nthere are no poles in $\\Delta \\Sigma_{\\rm H}$ \nin the low-energy region, thanks to the\nexclusion of fluctuations within the L-space.\nThe frequency dependence of the real part indicates that the \nlinearization \n$\\displaystyle{\\Delta \\Sigma_{\\rm H}(\\omega) \\sim \\Delta \\Sigma_{\\rm H}(\\omega=0)+[d\\Delta \\Sigma_{\\rm H}(\\omega)/d\\omega]_{\\omega=0}\\omega }$\noffers a reasonably good approximation. \nIn fact, the behavior of the constrained self-energy\nis much closer to linearity than that of \nthe full GW self-energy $\\Sigma$.\nThis is easily understood by the fact that the low-energy excitations are excluded in the present constrained self-energy $\\Delta\\Sigma_{\\rm H}$ analogously to an insulator, thus eliminating the strong frequency dependence of the constrained self-energy near the Fermi level. This is one of the consequences of the \ncontrollability and an advantage of the present MACE scheme, as \nmentioned in the introduction.\nIn the following discussion, we employ this linearized approximation.\nThen the renormalization factor $Z_{\\rm H}$ defined by Eq.~(\\ref{ZH})\nis interpreted as that from the contribution of the H-space. \nAfter \nthis renormalization factor is taken into account,\nthe effective Hamiltonian is given by $Z_{\\rm H}[H_{\\rm LDA}-V_{\\rm xc}+\\Delta\\Sigma_{\\rm H}(\\omega=0)]$ (see Fig.~\\ref{ZH_SigmaH}). \nInterestingly the renormalization factor $Z_{\\rm H}$=0.92 stays\nclose to one, so that\nthe bandwidth is only slightly reduced (to\n$3.4$ eV instead of the $3.7$ eV above). \nWe also note that $Z_{\\rm H}$ has weak momentum dependence\nas we reveal in the following.\n\n\n\n\\begin{figure*}[htb]\n\\begin{center} \n\\includegraphics[width=0.9\\textwidth ]{fig9.eps} \n\\end{center} \n\\caption{(Color online)\n{\\bf minor modification}\nFrequency dependence of $-V_{\\rm xc}+\\Delta \\Sigma_{\\rm H}$ of SrVO$_3$.\n(a) and (b) are the real and imaginary parts of the local\ncomponent, respectively.\nThe black dotted line is $\\Sigma$ defined in Eq.~(\\ref{full-self-energy}) in the full GW approximation for comparison.\n(c) and (d) are the real and imaginary \nparts at several $k$ points.\n}\n\\label{sigH_nonlocal}\n\\end{figure*} \n\n\n\nFigure~\\ref{sigH_nonlocal} \nshows the local and nonlocal parts of $\\Delta \\Sigma_{\\rm H}$.\nThe frequency \ndependence\nof the local part (Figs.~\\ref{sigH_nonlocal} (a)(b)) \nis quite similar \nto the  frequency-dependent self-energies at various momenta\nin Fig.~\\ref{sigH}.\nOne immediately reads off an interesting property, which is\nakin to what has been found for the full GW self-energy in\nRef.~\\onlinecite{Tomczak-PRL2012}, namely that the nonlocal part of \n$\\Delta \\Sigma_{\\rm H}$ shows only weak frequency dependence \nas shown in Figs.~\\ref{sigH_nonlocal} (c)(d). \nThis explains why $Z_{\\rm H}$ \nis only weakly momentum-dependent.\nOn the other hand, $\\Delta \\Sigma_{\\rm H}^{\\rm nonlocal} (0)$ at $R$ is $\\sim 1$ eV larger than \nat $\\Gamma$, which causes the band widening effect.\n $\\Delta \\Sigma_{\\rm H}$\ncan thus to a good approximation be decomposed into a\nfrequency-dependent local part and a static nonlocal one:\n$\\Delta \\Sigma_{\\rm H}(k,\\omega) = \\Delta \\Sigma^{\\rm local}_{\\rm H}(\\omega) + \\Delta \\Sigma^{\\rm nonlocal}_{\\rm H}(k)$,\nwith $ \\Delta \\Sigma_{\\rm H}^{\\rm local}(\\omega) = \\sum_q  \\Delta \\Sigma_{\\rm H}(q,\\omega)$\n\nSince in the simple case of SrVO$_3$ where the t$_{2g}$ states are\ndegenerate, static local operators are scalar and are compensated \nby a chemical potential shift such that the correct particle\nnumber is obtained. The above band structure corresponding to\n$H_{\\rm LDA} - V_{\\rm xc} + \\Delta \\Sigma_{\\rm H}(k, \\omega=0)$ \nis thus identical to that of\n$H_{\\rm LDA} - V_{\\rm xc} + \\Delta \\Sigma^{\\rm nonlocal}_{\\rm H}(k)$.\nThe local dynamical part of $\\Delta \\Sigma_{\\rm H}$ then results in a \nnarrowing of this band structure by a factor $Z_{\\rm H} = 0.92$.\n\n\n\n\\begin{figure}[h]\n\\centering \n\\includegraphics[clip,width=0.45\\textwidth ]{fig10.eps} \n\\caption{(Color online)\n(a) Frequency dependence of the on-site effective interaction.\nPanel (b) is a low-energy zoom of panel (a). \n}\n\\label{W}\n\\end{figure} \n\n\nBefore turning to a discussion of the low-energy correction for the GW treatment\n$\\Delta \\Sigma_{\\rm L} =G^{(0)}(W-W_{\\rm L})$, we analyze the effective\ninteraction $W_{\\rm L}$ shown in Fig.~\\ref{W} \nin comparison to $W$ and $W_{\\rm H}$.\nThe fully screened Coulomb interaction $W$ displays the familiar\nshape of an interaction that is strongly screened at low energies\n(with a value of $0.88$ eV at $\\omega=0$), while retrieving the\nvalue of the bare Coulomb interaction $v$ ($\\sim 16.0$ eV) \nat high energy.\nThe crossover from the bare to the  \nscreened values takes place at the plasma energy of \nabout $15$ eV. This behavior has been discussed before\n\\cite{Review_SB2014}; we note in particular that the plasma \nfrequency is known from electron energy loss spectroscopy \nmeasurements of the related SrTiO$_3$ compound \\cite{vandermarel,kohiki}.\nThe partially screened interaction\n$W_{\\rm H}$ constructed within cRPA\nconverges to $W$ at high energies, but displays\nweaker screening effects at low energies (with a value of $3.5$ eV at $\\omega=0$),\nsince intra-t$_{2g}$ screening channels are excluded.\nAs was already anticipated in Sec.~\\ref{Outline},  $W_{\\rm L}$ can be interpreted as the screened interaction of \na low-energy model where a static interaction of value $W_{\\rm H}(\\omega=0)$\nhas been imposed as the bare interaction. Since now {\\it only} intra-t$_{2g}$\nscreening channels are active, screening takes place only at low\nenergies, where the scale is given by the t$_{2g}$ bandwidth.\nAlso plotted is the difference $W_{\\rm GW}^{\\rm dyn}=W-W_{d}$: except at low energies\nwhere the t$_{2g}$ screening channels come into play, its frequency\ndependence is essentially given by that of $W$, while the\nhigh-energy limit is the bare Coulomb interaction $v$ reduced\nby $W_{\\rm H}(0)$.\n\n\n\\begin{figure*}[htb]\n\\begin{center} \n\\includegraphics[width=0.9\\textwidth ]{fig11.eps} \n\\end{center} \n\\caption{(Color online)\nFrequency dependence of $-V_{\\rm xc}+ G^{(1)}W_{\\rm H}^{\\rm dyn}$ of SrVO$_3$.\n(a) and (b) are the real and imaginary local parts, respectively.\n(c) and (d) are the real and imaginary \nparts at several $k$ points.\n}\n\\label{DeltaSigma_L_Pert}\n\\end{figure*} \n\n\n\n\\begin{figure*}[htb]\n\\begin{center} \n\\includegraphics[width=0.9\\textwidth ]{fig12.eps} \n\\end{center} \n\\caption{(Color online)\nFrequency dependence of $-V_{\\rm xc}+ G^{(0)}W_{\\rm GW}^{\\rm dyn}$ of SrVO$_3$.\n(a) and (b) are the real and imaginary local parts, respectively.\n(c) and (d) are the real and imaginary \nparts at several $k$ points.\n}\n\\label{DeltaSigma_L_GW}\n\\end{figure*} \n\n\n\nThe frequency dependence of the real and imaginary parts of the low-energy self-energy $\\Delta \\Sigma_{\\rm L} = G^{(1)}W_{\\rm H}^{\\rm dyn}$ for the direct perturbative treatment is illustrated in Fig.~\\ref{DeltaSigma_L_Pert}. \n$\\Delta \\Sigma_{\\rm L} = G^{(0)}W_{\\rm GW}^{\\rm dyn}$ used\nfor the GW treatment is shown in Fig.~\\ref{DeltaSigma_L_GW}\nfor several choices of momenta.\n\\\\\nThe results for the perturbative and the GW treatment are nearly \nidentical.\nThe renormalization factor of $\\Delta \\Sigma_{\\rm L}^{\\rm nonlocal}$ \n(and $\\Delta \\Sigma_{\\rm L}$) is $\\sim 0.77$.\nThe zero-frequency shift\n${\\rm Re}\\Delta \\Sigma_{\\rm L}^{\\rm nonlocal} (0)$ at $R$ is $\\sim 3$ eV \nlarger than \nat $\\Gamma$.\nAgain, one sees how $\\Sigma_{\\rm L}$ separates into local dynamical\nand nonlocal static parts: \n$\\Delta \\Sigma_{\\rm L} = \\Delta \\Sigma^{\\rm nonlocal}_{\\rm L}(k) + \n\\Delta \\Sigma^{\\rm local}_{\\rm L}(\\omega)$.\nThis is expected since such a separation has been found within\nthe full GW calculation \\cite{TomczakPRB2014}, and is even\nmore plausible for a GW treatment within the t$_{2g}$ subspace.\n\n\n\n\\begin{figure}[h]\n\\centering \n\\includegraphics[clip,width=0.45\\textwidth ]{fig13.eps} \n\\caption{(Color online)\nBand structure  of $[\\epsilon_{\\rm DFT}-V_{\\rm xc}+\\Delta\\Sigma_{\\rm H}+\\Delta \\Sigma_{\\rm L}^{\\rm nonlocal}]Z_{\\rm HL}$ of SrVO$_3$ in the direct perturbative treatment.\nFor comparison, the LDA band structure is also given\n(black dotted line).\nThe energy is measured from the Fermi level.\n}\n\\label{DFT+Sigma_HL_Pert}\n\\end{figure} \n\n\n\n\\begin{figure}[h]\n\\centering \n\\includegraphics[clip,width=0.45\\textwidth ]{fig14.eps} \n\\caption{(Color online)\nBand structure  of $[\\epsilon_{\\rm DFT}-V_{\\rm xc}+\\Delta\\Sigma_{\\rm H}+\\Delta \\Sigma_{\\rm L}^{\\rm nonlocal}]Z_{\\rm HL}$ of SrVO$_3$ in the GW treatment.\nFor comparison, the LDA band structure is also given\n(black dotted line).\nThe energy is measured from the Fermi level.\n}\n\\label{DFT+Sigma_HL_GW}\n\\end{figure} \n\n\n\nThe corresponding band structures \nare given by \n $[\\epsilon_{\\rm DFT}+\\Delta\\Sigma_{\\rm H}+\\Delta \\Sigma_{\\rm L}^{\\rm nonlocal}]Z_{\\rm HL}$ in Fig.~\\ref{DFT+Sigma_HL_Pert} \nfor the perturbative treatment (see Eq.~(\\ref{DeltaSigmaLnonlocalPert})), and in Fig.~\\ref{DFT+Sigma_HL_GW} \nfor the GW treatment,\nwhere $\\Delta \\Sigma_{\\rm L}^{\\rm nonlocal}$ is defined in Eq.~(\\ref{DeltaSigLnonlocalGW}) and $Z_{\\rm HL}$ is around 0.92.\n$Z_{\\rm HL}$ is practically the same as $Z_{\\rm H}$, meaning that the nonlocal dynamical correction \nin the H-space is small, which justifies the GW perturbative treatment for the nonlocal part.\nOne finds that \nthe nonlocal part $\\Delta \\Sigma_{\\rm L}=G^{(0)}_{ll}(W-W_{\\rm L})$ narrows\nthe empty states but widens the occupied ones, resulting\nin an overall bandwidth of 3.2 eV.\nThe difference between the direct perturbation and the GW treatment is small.\n\n\n\n\\begin{figure}[h]\n\\centering \n\\includegraphics[clip,width=0.45\\textwidth ]{fig15.eps} \n\\caption{(Color online)\nBand structure  of $[\\epsilon_{\\rm DFT}-V_{\\rm xc}+\\Delta\\Sigma_{\\rm H}+\\Delta\\Sigma_{\\rm L}^{\\rm nonlocal}]Z_{\\rm HL}Z_B$ of SrVO$_3$ in the GW treatment.\nFor comparison, the LDA band structure is also given\n(black dotted line).\nThe energy is measured from the Fermi level.}\n\\label{DFT+DeltaSigmaL_GW_ZHL_ZB}\n\\end{figure} \n\n\nAfter including the effect of the local self-energy by the nonperturbative \nCasula trick, we show the dispersion\ngiven by \n$[\\epsilon_{\\rm DFT}-V_{\\rm xc}+\\Delta\\Sigma_{\\rm H}+\\Delta\\Sigma_{\\rm L}^{\\rm nonlocal}]Z_{\\rm HL}Z_B$ for the GW-type treatment in Fig.~\\ref{DFT+DeltaSigmaL_GW_ZHL_ZB}: \nThe GW-like treatment gives a band dispersion\nclose to the LDA one. \nThe $Z_B$ factor corresponding to the local dynamical part of \n$\\Delta \\Sigma_{\\rm L}$ amounts to $Z_B=0.7$.\nThe nonperturbative treatment \nresults in an LDA-like band dispersion for the empty states,\nbut a slightly narrower bandwidth in the occupied part.\nThe low-energy effective Hamiltonian at this level of treatment has a \nfrequency independent effective interaction both with local \nand nonlocal interaction given by the Fourier transform of \n$W_{\\rm H}(q,\\omega=0)$, which contains both direct and exchange interactions. \n\n\n\\begin{figure}[h]\n\\centering \n\\includegraphics[clip,width=0.45\\textwidth ]{fig16.eps} \n\\caption{(Color online)\nBand structure  of $[\\epsilon_{\\rm DFT}+\\Sigma_{\\rm corr}^{\\rm GW}]Z_{\\rm corr}Z_B$ of SrVO$_3$.\nFor comparison, the LDA band structure is also given\n(black dotted line).\nThe energy is measured from the Fermi level.}\n\\label{DFT+Sigma_corr_GW_Z}\n\\end{figure} \n\n\n\nWe finally show the dispersion\ngiven by Eq.~(\\ref{Teff_final}), namely\n$[\\epsilon_{\\rm DFT}+\\Sigma_{\\rm corr}^{\\rm GW}]Z_{\\rm corr}Z_B$ in Fig.~\\ref{DFT+Sigma_corr_GW_Z}. \nThe result shows an LDA-like band dispersion for the occupied states,\nbut a slightly wider bandwidth in the empty part, resulting in a bandwidth 14\\% wider in total than the LDA bandwidth. \nThe effective interaction of the low-energy effective Hamiltonian \nat this final level has\nfrequency independent local and nonlocal interactions \ngiven by the Fourier transform of $W_r(q,\\omega=0)$. \n\n\n\n\\begin{figure}[h]\n\\centering \n\\includegraphics[clip,width=0.45\\textwidth ]{fig17.eps} \n\\caption{(Color online)\nBand structure  of $[\\epsilon_{\\rm DFT}-V_{\\rm xc}+\\Sigma-(G^{(0)}W_{\\rm GW}^{\\rm dyn})^{\\rm nonlocal}]Z'Z_B$ of SrVO$_3$\n(Z' is the renormalization factor of $\\Sigma-(G^{(0)}W_{\\rm GW}^{\\rm dyn})^{\\rm nonlocal}$).\nFor comparison, the LDA band structure is also given\n(black dotted line).\nThe energy is measured from the Fermi level.\n}\n\\label{GW+ZB}\n\\end{figure} \n\n\n\nThe resulting effective Hamiltonian in the L-space \nconsists of single-particle and two-particle (interaction) parts.\nThe effective interaction is the same as in previous estimates \nby the cRPA in the literature, while the single-particle dispersion \nis revised after removing the double counting and taking into \naccount the frequency dependence of the effective interaction. The \nfinal effective bandwidth for SrVO$_3$ is,\nafter partial cancellation, \nslightly (14\\%) larger than the LDA bandwidth.  \nThis is a reasonable result because the LDA takes into account all \ncorrelations though insufficiently, while the present scheme by the constrained self-energy excludes the \ncorrelation effects arising from the L-space.\nAlthough the bandwidth derived for the effective model is slightly \nlarger than that of the LDA,  it is clear that the correlation effects \nare stronger than in the LDA or standard GW when the {\\it ab initio} \nmodel is solved by an accurate solver. In fact, if the effective \ninteraction contained in the final effective model is treated by \nthe GW scheme, one obtains the dispersion illustrated in \nFig.~\\ref{GW+ZB}, \nwhich is given from the self energy of the whole GW calculation $\\Sigma$ \nby correcting the local dynamical part $W-W_{\\rm L}$ by $Z_B$. \nThis indicates that even an insufficient treatment of the local \nstatic interaction by the GW scheme gives a dispersion with the bandwidth ($\\sim 1.9$ eV) narrower \nthan the LDA ($\\sim 2.5$ eV, Fig.~\\ref{lda}) and GW ($\\sim 2.1$ eV, Fig.~\\ref{fullGW}) dispersions.  \nA slightly ($\\sim 14$\\%) wider dispersion than that of the LDA bands obtained for the \neffective Hamiltonian accompanied with frequency independent effective \nCoulomb interactions without the exchange part may account for the slight \noverestimate of correlation effects in the literature mentioned in \nthe introduction.\nOur scheme offers an optimized way for the  derivation of \n{\\it ab initio} models for the L-space after eliminating the H-space\nin  a systematic fashion.\n\n\nOur findings of the band widening are consistent with studies\nbased on the combined GW+DMFT method in the literature \n\\cite{TomczakPRB2014}: There, it was argued that within GW+DMFT\nthe best effective Hamiltonian that DMFT should be performed on,\nis a one-body Hamiltonian corrected by the nonlocal part of the\nGW self-energy. The corresponding spectral function \n(see e.g.\nFigure 5 of Ref.~\\onlinecite{TomczakPRB2014}) displays \na broadening similar to \nFig.~\\ref{DFT+Sigma_HL_GW}.\nMost interestingly, our present calculations confirm a pronounced\nasymmetry observed in Ref.~\\onlinecite{TomczakPRB2014}, namely a stronger widening\neffect in the unoccupied part of the spectrum than in the occupied\none.\n\n",
        "label": "hartree",
        "Descriptive_question1": "What approximation is used for the band structure of SrVO$_3$ in figure_5?",
        "Descriptive_question2": "What type of line represents the LDA band structure in figure_5?",
        "Reasoning_question1": "How does the bandwidth of the Hartree approximation compare to the LDA band structure in figure_5, and what might this suggest about electronic correlations?",
        "Reasoning_question2": "Why might the Hartree band structure in figure_5 show substantial widening compared to a typical Hartree calculation, and what role does the density used in the calculation play in this effect?",
        "Descriptive_answer1": "Hartree approximation",
        "Descriptive_answer2": "black dotted line",
        "Reasoning_answer1": "The bandwidth of the Hartree approximation in figure_5 is wider than that of the LDA band structure, indicating that the Hartree approximation does not fully include electronic correlations that tend to narrow the bandwidth. This suggests that electronic correlations, partially accounted for in LDA but missing in the Hartree approximation, play a significant role in band narrowing of SrVO$_3$.",
        "Reasoning_answer2": "The Hartree band structure in figure_5 shows substantial widening because it is calculated for the converged Hartree density, as opposed to a simpler Hartree calculation at the LDA density. Using the converged Hartree density leads to a redistribution of charge and potential that increases band dispersion, resulting in wider bands. This highlights the importance of the density used in the calculation, as it influences band structure changes beyond just the approximation level."
    },
    {
        "paper_id": "1511.03757.json",
        "image_id": "figure_6",
        "image_path": [
            "/home/yz979/scratch/chengye/scimolmo/test-images-tables/dataset_imagestables/1511.03757/images/fig6.eps"
        ],
        "caption": "(Color online)\nBand structure  of $H_{\\rm LDA}-V_{\\rm xc}+\\Delta \\Sigma_{\\rm H}(\\omega=0)$ of SrVO$_3$.\nFor comparison, the LDA band structure is also given\n(black dotted line).\nThe energy is measured from the Fermi level.\n",
        "classify": "Chart",
        "section_info": "6 Results\n\\section{Results}\n\\label{results}\n\n\nThe ternary 3d$^1$ transition metal oxide SrVO$_3$ has become\none of the ``drosophila compounds'' of correlated systems.\nIt is a correlated metal\nthat has been characterized using various experimental \n\\cite{Onoda, inoue, maiti, maiti2, sekiyama, takizawa, yoshida2010}.\nand theoretical techniques, see e.g.\n\\cite{nekrasov, anisimov, pavarini-d1, liebsch, lechermann-d1, trimarchi, aichhorn09, karolak, lee, held-optics, tomczak-gwdmft, gatti-guzzo, sakuma14, TomczakPRB2014}.\nA review of most of the available experimental and theoretical \ndata has been given recently in Ref.~\\onlinecite{TomczakPRB2014}.\nSrVO$_3$ displays Fermi liquid behavior up to\nremarkably high temperatures of the order of $200$ K \\cite{Onoda, inoue},\nwith a moderate mass enhancement of the order \nof $2$ \\cite{yoshida2010, vandermarel}.\nDetailed spectroscopic investigations have made it an\nideal test compound for modern many-body calculations, and \nmore and more refined dynamical mean field-based studies \nare available.\nThe majority of studies so far have focused on the t$_{2g}$-manifold\nwhich forms the states close to the Fermi level, and those will\nalso be the focus in the present investigations. \nNote however that this restriction quite severely limits the\nrange of validity of the low-energy description. Indeed, as\nshown recently~\\cite{tomczak-gwdmft, TomczakPRB2014} at\nenergies of about 2 to 3 eV above the Fermi level\nthe spectral properties are largely determined by the e$_g$\nstates. This has in particular led to a reinterpretation of\nan inverse photoemission feature at about 2.5 eV that was \nfrequently interpreted as an upper Hubbard band of t$_{2g}$\ncharacter in the earlier literature.\nHere, we use the compound only as an illustration of \nthe principles of constructing effective low-energy models,\nwithout being concerned with a description of spectra beyond\nthe pure t$_{2g}$ part.\n\n\nThe LDA band structure of SrVO$_3$ is shown in Fig.~\\ref{lda}.\nOne clearly distinguishes the three-fold degenerate manifold\nof t$_{2g}$ bands close to the Fermi level (chosen as the\nzero of energy), followed in the unoccupied part of the spectrum\nby the two e$_g$ bands. At about -2.5 eV the filled O-2p derived\nbands are visible.\n\nWhen a standard GW calculation is performed, see Fig.~\\ref{fullGW},\nthe bandwidth of the t$_{2g}$-states is reduced from 2.5 eV to\n2.1 eV, while the overall shape of the dispersion remains similar\nto the LDA one.\nThis is in agreement with previous GW calculations in the literature\n\\cite{tomczak-gwdmft, TomczakPRB2014, gatti-guzzo, sakuma14}.\n\n\n\nThe calculation is based on the full-potential linear muffin-tin orbitals (FP-LMTO) implementation.\n8$\\times$8$\\times$8 \\bm{k}-mesh is employed in both the DFT/LDA and the GW calculations.\nThe calculational details are the same as in Ref.~\\onlinecite{Hirayama}.\n\n\nWe now turn to a discussion of the band structure corresponding to \nthe Hamiltonians to be used as input for subsequent many-body \ncalculations for this material.\nWe will proceed step by step to analyze the influence of the various\ncorrection terms, with respect to the DFT starting point\n(which will be overlaid to the respective band structures).\n\n\n\n\\begin{figure}[h]\n\\centering \n\\includegraphics[clip,width=0.45\\textwidth ]{fig2.eps} \n\\caption{(Color online)\nKohn-Sham band structure of SrVO$_3$ in the LDA.\nThe energy is measured from the Fermi level.\n}\n\\label{lda}\n\\end{figure} \n\n\n\\begin{figure}[h]\n\\centering \n\\includegraphics[clip,width=0.45\\textwidth ]{fig3.eps} \n\\caption{(Color online)\nBand structure  of SrVO$_3$ in the one-shot GW approximation.\nFor comparison, the LDA band structure is also given\n(black dotted line).\nThe energy is measured from the Fermi level.\n}\n\\label{fullGW}\n\\end{figure} \n\n\n\n\n\n\n\\begin{figure}[h]\n\\centering \n\\includegraphics[clip,width=0.45\\textwidth ]{fig4.eps} \n\\caption{(Color online)\nBand structure of SrVO$_3$ corresponding to the LDA Hamiltonian\nfrom which the LDA exchange-correlation potential has been subtracted.\nFor comparison, the LDA band structure is also given\n(black dotted line).\nThe energy is measured from the Fermi level.\n}\n\\label{lda-vxc}\n\\end{figure} \n\n\n\\begin{figure}[h]\n\\centering \n\\includegraphics[clip,width=0.45\\textwidth ]{fig5.eps} \n\\caption{(Color online)\nBand structure  of SrVO$_3$ in the Hartree approximation.\nFor comparison, the LDA band structure is also given\n(black dotted line).\nThe energy is measured from the Fermi level.\n}\n\\label{hartree}\n\\end{figure} \n\n\nFigure~\\ref{lda-vxc} \ndisplays the band structure corresponding to\nthe LDA Hamiltonian from which the \nLDA exchange-correlation\npotential has been subtracted,  \nthat is, the eigenvalues of $H_{\\rm LDA}-V_{\\rm xc}$ where\nthese operators are evaluated for the self-consistent LDA\ndensity.\nThe subtraction of $V_{\\rm xc}$ widens the band structure from \nthe LDA bandwidth of 2.5 eV \nto more than twice this value: the new band\nwidth is 5.4 eV.\nThis indicates that the exchange-correlation potential $V_{\\rm xc}$ \nof the LDA is responsible for a substantial amount of \nthe band-narrowing effect\narising from electronic correlations, stressing the need \nto subtract the double counting in a consistent manner. \nFigure~\\ref{lda-vxc} is regarded as the starting point for the\nfollowing considerations.\n\nFor simple materials (see e.g. the calculations on Li in Ref.~\\onlinecite{Lithium})\nit has been noted in the literature that the Hartree band structure\nis close to the DFT one. This raises the question of the origin of the\nsubstantial band widening in the present case. Indeed, the present\ncalculation shown in Fig.~\\ref{lda-vxc} differs from a Hartree calculation\nonly \nby the fact that Fig.~\\ref{lda-vxc} is evaluated for the converged LDA density.\nWe have performed a test calculation where we plot the Hartree band\nstructure calculated for the converged Hartree density. The result\nis shown in Fig.~\\ref{hartree}. As seen from this plot, while the\nband is not fully as wide as in Fig.~\\ref{lda-vxc}, a substantial \nwidening is already present at this stage.\n\n\n\n\\begin{figure}[h]\n\\centering \n\\includegraphics[clip,width=0.45\\textwidth ]{fig6.eps} \n\\caption{(Color online)\nBand structure  of $H_{\\rm LDA}-V_{\\rm xc}+\\Delta \\Sigma_{\\rm H}(\\omega=0)$ of SrVO$_3$.\nFor comparison, the LDA band structure is also given\n(black dotted line).\nThe energy is measured from the Fermi level.\n}\n\\label{lda-vxc-sigH(0)}\n\\end{figure} \n\n\n\nStarting from the Hamiltonian without V$_{xc}$ (see the \ndispersion in Fig.~\\ref{lda-vxc}), we first take into account \nthe static part of the corrective self-energy $\\Delta \\Sigma_{\\rm H}$:\nIn Fig.~\\ref{lda-vxc-sigH(0)}, we plot the dispersion corresponding to\n$H_{\\rm LDA}-V_{\\rm xc}+\\Delta \\Sigma_{\\rm H}(\\omega=0)$. \nCompared to the dispersion of Fig.~\\ref{lda-vxc}, the overall band structure \nis narrowed to 3.7 eV. This value is, however, still considerably larger\nthan the LDA bandwidth (Fig.~\\ref{lda}). Although\n the electronic correlations coupling the H- and \nthe L space are included in Fig.~\\ref{lda-vxc-sigH(0)}, and narrow the band with respect to the case\nwhere V$_{xc}$ is taken out, correlations within\nthe L space are not included. \nThe LDA, on the other hand, at least partially\nincludes correlations within the L space, \nand these are very effective in narrowing the band.\nInterestingly, the bottom of the occupied band is quite\nexactly at the LDA value, and the remaining widening is\npurely in the unoccupied part.\n\n\n\\begin{figure*}[htb]\n\\begin{center} \n\\includegraphics[width=0.9\\textwidth ]{fig7.eps} \n\\end{center} \n\\caption{(Color online)\nFrequency dependence of $-V_{xc}+\\Delta \\Sigma_{\\rm H}$ for SrVO$_3$.\n(a) and (b) are the real and imaginary parts at the $\\Gamma$ point, \nrespectively,\nand (c) and (d) ((e) and (f)) are those at $X$ ($R$).\nFor comparison, the full GW self-energies are also given\n(black dotted line).\n}\n\\label{sigH}\n\\end{figure*} \n\n\n\n\\begin{figure}[h]\n\\centering \n\\includegraphics[clip,width=0.45\\textwidth ]{fig8.eps} \n\\caption{(Color online)\nBand structure  of $Z_{\\rm H}[H_{\\rm LDA}-V_{\\rm xc}+\\Delta\\Sigma_{\\rm H}(\\omega=0)]$ of SrVO$_3$.\nFor comparison, the LDA band structure is also given\n(black dotted line).\nThe energy is measured from the Fermi level.\n}\n\\label{ZH_SigmaH}\n\\end{figure} \n\n\nTo go further and in particular to analyze the dynamic\nbehavior of $\\Delta \\Sigma_{\\rm H}$ \nwe plot in Fig.~\\ref{sigH} \nthe self-energy corrections $\\Delta \\Sigma_{\\rm H}$ (with $\\Sigma$ (see Eq.~(\\ref{full-self-energy}))\nfor comparison) for the real and imaginary parts\nat several representative choices of momenta.\nThe frequency dependence is smooth around the Fermi level.\nIn particular, in contrast to the full self-energy,\nthere are no poles in $\\Delta \\Sigma_{\\rm H}$ \nin the low-energy region, thanks to the\nexclusion of fluctuations within the L-space.\nThe frequency dependence of the real part indicates that the \nlinearization \n$\\displaystyle{\\Delta \\Sigma_{\\rm H}(\\omega) \\sim \\Delta \\Sigma_{\\rm H}(\\omega=0)+[d\\Delta \\Sigma_{\\rm H}(\\omega)/d\\omega]_{\\omega=0}\\omega }$\noffers a reasonably good approximation. \nIn fact, the behavior of the constrained self-energy\nis much closer to linearity than that of \nthe full GW self-energy $\\Sigma$.\nThis is easily understood by the fact that the low-energy excitations are excluded in the present constrained self-energy $\\Delta\\Sigma_{\\rm H}$ analogously to an insulator, thus eliminating the strong frequency dependence of the constrained self-energy near the Fermi level. This is one of the consequences of the \ncontrollability and an advantage of the present MACE scheme, as \nmentioned in the introduction.\nIn the following discussion, we employ this linearized approximation.\nThen the renormalization factor $Z_{\\rm H}$ defined by Eq.~(\\ref{ZH})\nis interpreted as that from the contribution of the H-space. \nAfter \nthis renormalization factor is taken into account,\nthe effective Hamiltonian is given by $Z_{\\rm H}[H_{\\rm LDA}-V_{\\rm xc}+\\Delta\\Sigma_{\\rm H}(\\omega=0)]$ (see Fig.~\\ref{ZH_SigmaH}). \nInterestingly the renormalization factor $Z_{\\rm H}$=0.92 stays\nclose to one, so that\nthe bandwidth is only slightly reduced (to\n$3.4$ eV instead of the $3.7$ eV above). \nWe also note that $Z_{\\rm H}$ has weak momentum dependence\nas we reveal in the following.\n\n\n\n\\begin{figure*}[htb]\n\\begin{center} \n\\includegraphics[width=0.9\\textwidth ]{fig9.eps} \n\\end{center} \n\\caption{(Color online)\n{\\bf minor modification}\nFrequency dependence of $-V_{\\rm xc}+\\Delta \\Sigma_{\\rm H}$ of SrVO$_3$.\n(a) and (b) are the real and imaginary parts of the local\ncomponent, respectively.\nThe black dotted line is $\\Sigma$ defined in Eq.~(\\ref{full-self-energy}) in the full GW approximation for comparison.\n(c) and (d) are the real and imaginary \nparts at several $k$ points.\n}\n\\label{sigH_nonlocal}\n\\end{figure*} \n\n\n\nFigure~\\ref{sigH_nonlocal} \nshows the local and nonlocal parts of $\\Delta \\Sigma_{\\rm H}$.\nThe frequency \ndependence\nof the local part (Figs.~\\ref{sigH_nonlocal} (a)(b)) \nis quite similar \nto the  frequency-dependent self-energies at various momenta\nin Fig.~\\ref{sigH}.\nOne immediately reads off an interesting property, which is\nakin to what has been found for the full GW self-energy in\nRef.~\\onlinecite{Tomczak-PRL2012}, namely that the nonlocal part of \n$\\Delta \\Sigma_{\\rm H}$ shows only weak frequency dependence \nas shown in Figs.~\\ref{sigH_nonlocal} (c)(d). \nThis explains why $Z_{\\rm H}$ \nis only weakly momentum-dependent.\nOn the other hand, $\\Delta \\Sigma_{\\rm H}^{\\rm nonlocal} (0)$ at $R$ is $\\sim 1$ eV larger than \nat $\\Gamma$, which causes the band widening effect.\n $\\Delta \\Sigma_{\\rm H}$\ncan thus to a good approximation be decomposed into a\nfrequency-dependent local part and a static nonlocal one:\n$\\Delta \\Sigma_{\\rm H}(k,\\omega) = \\Delta \\Sigma^{\\rm local}_{\\rm H}(\\omega) + \\Delta \\Sigma^{\\rm nonlocal}_{\\rm H}(k)$,\nwith $ \\Delta \\Sigma_{\\rm H}^{\\rm local}(\\omega) = \\sum_q  \\Delta \\Sigma_{\\rm H}(q,\\omega)$\n\nSince in the simple case of SrVO$_3$ where the t$_{2g}$ states are\ndegenerate, static local operators are scalar and are compensated \nby a chemical potential shift such that the correct particle\nnumber is obtained. The above band structure corresponding to\n$H_{\\rm LDA} - V_{\\rm xc} + \\Delta \\Sigma_{\\rm H}(k, \\omega=0)$ \nis thus identical to that of\n$H_{\\rm LDA} - V_{\\rm xc} + \\Delta \\Sigma^{\\rm nonlocal}_{\\rm H}(k)$.\nThe local dynamical part of $\\Delta \\Sigma_{\\rm H}$ then results in a \nnarrowing of this band structure by a factor $Z_{\\rm H} = 0.92$.\n\n\n\n\\begin{figure}[h]\n\\centering \n\\includegraphics[clip,width=0.45\\textwidth ]{fig10.eps} \n\\caption{(Color online)\n(a) Frequency dependence of the on-site effective interaction.\nPanel (b) is a low-energy zoom of panel (a). \n}\n\\label{W}\n\\end{figure} \n\n\nBefore turning to a discussion of the low-energy correction for the GW treatment\n$\\Delta \\Sigma_{\\rm L} =G^{(0)}(W-W_{\\rm L})$, we analyze the effective\ninteraction $W_{\\rm L}$ shown in Fig.~\\ref{W} \nin comparison to $W$ and $W_{\\rm H}$.\nThe fully screened Coulomb interaction $W$ displays the familiar\nshape of an interaction that is strongly screened at low energies\n(with a value of $0.88$ eV at $\\omega=0$), while retrieving the\nvalue of the bare Coulomb interaction $v$ ($\\sim 16.0$ eV) \nat high energy.\nThe crossover from the bare to the  \nscreened values takes place at the plasma energy of \nabout $15$ eV. This behavior has been discussed before\n\\cite{Review_SB2014}; we note in particular that the plasma \nfrequency is known from electron energy loss spectroscopy \nmeasurements of the related SrTiO$_3$ compound \\cite{vandermarel,kohiki}.\nThe partially screened interaction\n$W_{\\rm H}$ constructed within cRPA\nconverges to $W$ at high energies, but displays\nweaker screening effects at low energies (with a value of $3.5$ eV at $\\omega=0$),\nsince intra-t$_{2g}$ screening channels are excluded.\nAs was already anticipated in Sec.~\\ref{Outline},  $W_{\\rm L}$ can be interpreted as the screened interaction of \na low-energy model where a static interaction of value $W_{\\rm H}(\\omega=0)$\nhas been imposed as the bare interaction. Since now {\\it only} intra-t$_{2g}$\nscreening channels are active, screening takes place only at low\nenergies, where the scale is given by the t$_{2g}$ bandwidth.\nAlso plotted is the difference $W_{\\rm GW}^{\\rm dyn}=W-W_{d}$: except at low energies\nwhere the t$_{2g}$ screening channels come into play, its frequency\ndependence is essentially given by that of $W$, while the\nhigh-energy limit is the bare Coulomb interaction $v$ reduced\nby $W_{\\rm H}(0)$.\n\n\n\\begin{figure*}[htb]\n\\begin{center} \n\\includegraphics[width=0.9\\textwidth ]{fig11.eps} \n\\end{center} \n\\caption{(Color online)\nFrequency dependence of $-V_{\\rm xc}+ G^{(1)}W_{\\rm H}^{\\rm dyn}$ of SrVO$_3$.\n(a) and (b) are the real and imaginary local parts, respectively.\n(c) and (d) are the real and imaginary \nparts at several $k$ points.\n}\n\\label{DeltaSigma_L_Pert}\n\\end{figure*} \n\n\n\n\\begin{figure*}[htb]\n\\begin{center} \n\\includegraphics[width=0.9\\textwidth ]{fig12.eps} \n\\end{center} \n\\caption{(Color online)\nFrequency dependence of $-V_{\\rm xc}+ G^{(0)}W_{\\rm GW}^{\\rm dyn}$ of SrVO$_3$.\n(a) and (b) are the real and imaginary local parts, respectively.\n(c) and (d) are the real and imaginary \nparts at several $k$ points.\n}\n\\label{DeltaSigma_L_GW}\n\\end{figure*} \n\n\n\nThe frequency dependence of the real and imaginary parts of the low-energy self-energy $\\Delta \\Sigma_{\\rm L} = G^{(1)}W_{\\rm H}^{\\rm dyn}$ for the direct perturbative treatment is illustrated in Fig.~\\ref{DeltaSigma_L_Pert}. \n$\\Delta \\Sigma_{\\rm L} = G^{(0)}W_{\\rm GW}^{\\rm dyn}$ used\nfor the GW treatment is shown in Fig.~\\ref{DeltaSigma_L_GW}\nfor several choices of momenta.\n\\\\\nThe results for the perturbative and the GW treatment are nearly \nidentical.\nThe renormalization factor of $\\Delta \\Sigma_{\\rm L}^{\\rm nonlocal}$ \n(and $\\Delta \\Sigma_{\\rm L}$) is $\\sim 0.77$.\nThe zero-frequency shift\n${\\rm Re}\\Delta \\Sigma_{\\rm L}^{\\rm nonlocal} (0)$ at $R$ is $\\sim 3$ eV \nlarger than \nat $\\Gamma$.\nAgain, one sees how $\\Sigma_{\\rm L}$ separates into local dynamical\nand nonlocal static parts: \n$\\Delta \\Sigma_{\\rm L} = \\Delta \\Sigma^{\\rm nonlocal}_{\\rm L}(k) + \n\\Delta \\Sigma^{\\rm local}_{\\rm L}(\\omega)$.\nThis is expected since such a separation has been found within\nthe full GW calculation \\cite{TomczakPRB2014}, and is even\nmore plausible for a GW treatment within the t$_{2g}$ subspace.\n\n\n\n\\begin{figure}[h]\n\\centering \n\\includegraphics[clip,width=0.45\\textwidth ]{fig13.eps} \n\\caption{(Color online)\nBand structure  of $[\\epsilon_{\\rm DFT}-V_{\\rm xc}+\\Delta\\Sigma_{\\rm H}+\\Delta \\Sigma_{\\rm L}^{\\rm nonlocal}]Z_{\\rm HL}$ of SrVO$_3$ in the direct perturbative treatment.\nFor comparison, the LDA band structure is also given\n(black dotted line).\nThe energy is measured from the Fermi level.\n}\n\\label{DFT+Sigma_HL_Pert}\n\\end{figure} \n\n\n\n\\begin{figure}[h]\n\\centering \n\\includegraphics[clip,width=0.45\\textwidth ]{fig14.eps} \n\\caption{(Color online)\nBand structure  of $[\\epsilon_{\\rm DFT}-V_{\\rm xc}+\\Delta\\Sigma_{\\rm H}+\\Delta \\Sigma_{\\rm L}^{\\rm nonlocal}]Z_{\\rm HL}$ of SrVO$_3$ in the GW treatment.\nFor comparison, the LDA band structure is also given\n(black dotted line).\nThe energy is measured from the Fermi level.\n}\n\\label{DFT+Sigma_HL_GW}\n\\end{figure} \n\n\n\nThe corresponding band structures \nare given by \n $[\\epsilon_{\\rm DFT}+\\Delta\\Sigma_{\\rm H}+\\Delta \\Sigma_{\\rm L}^{\\rm nonlocal}]Z_{\\rm HL}$ in Fig.~\\ref{DFT+Sigma_HL_Pert} \nfor the perturbative treatment (see Eq.~(\\ref{DeltaSigmaLnonlocalPert})), and in Fig.~\\ref{DFT+Sigma_HL_GW} \nfor the GW treatment,\nwhere $\\Delta \\Sigma_{\\rm L}^{\\rm nonlocal}$ is defined in Eq.~(\\ref{DeltaSigLnonlocalGW}) and $Z_{\\rm HL}$ is around 0.92.\n$Z_{\\rm HL}$ is practically the same as $Z_{\\rm H}$, meaning that the nonlocal dynamical correction \nin the H-space is small, which justifies the GW perturbative treatment for the nonlocal part.\nOne finds that \nthe nonlocal part $\\Delta \\Sigma_{\\rm L}=G^{(0)}_{ll}(W-W_{\\rm L})$ narrows\nthe empty states but widens the occupied ones, resulting\nin an overall bandwidth of 3.2 eV.\nThe difference between the direct perturbation and the GW treatment is small.\n\n\n\n\\begin{figure}[h]\n\\centering \n\\includegraphics[clip,width=0.45\\textwidth ]{fig15.eps} \n\\caption{(Color online)\nBand structure  of $[\\epsilon_{\\rm DFT}-V_{\\rm xc}+\\Delta\\Sigma_{\\rm H}+\\Delta\\Sigma_{\\rm L}^{\\rm nonlocal}]Z_{\\rm HL}Z_B$ of SrVO$_3$ in the GW treatment.\nFor comparison, the LDA band structure is also given\n(black dotted line).\nThe energy is measured from the Fermi level.}\n\\label{DFT+DeltaSigmaL_GW_ZHL_ZB}\n\\end{figure} \n\n\nAfter including the effect of the local self-energy by the nonperturbative \nCasula trick, we show the dispersion\ngiven by \n$[\\epsilon_{\\rm DFT}-V_{\\rm xc}+\\Delta\\Sigma_{\\rm H}+\\Delta\\Sigma_{\\rm L}^{\\rm nonlocal}]Z_{\\rm HL}Z_B$ for the GW-type treatment in Fig.~\\ref{DFT+DeltaSigmaL_GW_ZHL_ZB}: \nThe GW-like treatment gives a band dispersion\nclose to the LDA one. \nThe $Z_B$ factor corresponding to the local dynamical part of \n$\\Delta \\Sigma_{\\rm L}$ amounts to $Z_B=0.7$.\nThe nonperturbative treatment \nresults in an LDA-like band dispersion for the empty states,\nbut a slightly narrower bandwidth in the occupied part.\nThe low-energy effective Hamiltonian at this level of treatment has a \nfrequency independent effective interaction both with local \nand nonlocal interaction given by the Fourier transform of \n$W_{\\rm H}(q,\\omega=0)$, which contains both direct and exchange interactions. \n\n\n\\begin{figure}[h]\n\\centering \n\\includegraphics[clip,width=0.45\\textwidth ]{fig16.eps} \n\\caption{(Color online)\nBand structure  of $[\\epsilon_{\\rm DFT}+\\Sigma_{\\rm corr}^{\\rm GW}]Z_{\\rm corr}Z_B$ of SrVO$_3$.\nFor comparison, the LDA band structure is also given\n(black dotted line).\nThe energy is measured from the Fermi level.}\n\\label{DFT+Sigma_corr_GW_Z}\n\\end{figure} \n\n\n\nWe finally show the dispersion\ngiven by Eq.~(\\ref{Teff_final}), namely\n$[\\epsilon_{\\rm DFT}+\\Sigma_{\\rm corr}^{\\rm GW}]Z_{\\rm corr}Z_B$ in Fig.~\\ref{DFT+Sigma_corr_GW_Z}. \nThe result shows an LDA-like band dispersion for the occupied states,\nbut a slightly wider bandwidth in the empty part, resulting in a bandwidth 14\\% wider in total than the LDA bandwidth. \nThe effective interaction of the low-energy effective Hamiltonian \nat this final level has\nfrequency independent local and nonlocal interactions \ngiven by the Fourier transform of $W_r(q,\\omega=0)$. \n\n\n\n\\begin{figure}[h]\n\\centering \n\\includegraphics[clip,width=0.45\\textwidth ]{fig17.eps} \n\\caption{(Color online)\nBand structure  of $[\\epsilon_{\\rm DFT}-V_{\\rm xc}+\\Sigma-(G^{(0)}W_{\\rm GW}^{\\rm dyn})^{\\rm nonlocal}]Z'Z_B$ of SrVO$_3$\n(Z' is the renormalization factor of $\\Sigma-(G^{(0)}W_{\\rm GW}^{\\rm dyn})^{\\rm nonlocal}$).\nFor comparison, the LDA band structure is also given\n(black dotted line).\nThe energy is measured from the Fermi level.\n}\n\\label{GW+ZB}\n\\end{figure} \n\n\n\nThe resulting effective Hamiltonian in the L-space \nconsists of single-particle and two-particle (interaction) parts.\nThe effective interaction is the same as in previous estimates \nby the cRPA in the literature, while the single-particle dispersion \nis revised after removing the double counting and taking into \naccount the frequency dependence of the effective interaction. The \nfinal effective bandwidth for SrVO$_3$ is,\nafter partial cancellation, \nslightly (14\\%) larger than the LDA bandwidth.  \nThis is a reasonable result because the LDA takes into account all \ncorrelations though insufficiently, while the present scheme by the constrained self-energy excludes the \ncorrelation effects arising from the L-space.\nAlthough the bandwidth derived for the effective model is slightly \nlarger than that of the LDA,  it is clear that the correlation effects \nare stronger than in the LDA or standard GW when the {\\it ab initio} \nmodel is solved by an accurate solver. In fact, if the effective \ninteraction contained in the final effective model is treated by \nthe GW scheme, one obtains the dispersion illustrated in \nFig.~\\ref{GW+ZB}, \nwhich is given from the self energy of the whole GW calculation $\\Sigma$ \nby correcting the local dynamical part $W-W_{\\rm L}$ by $Z_B$. \nThis indicates that even an insufficient treatment of the local \nstatic interaction by the GW scheme gives a dispersion with the bandwidth ($\\sim 1.9$ eV) narrower \nthan the LDA ($\\sim 2.5$ eV, Fig.~\\ref{lda}) and GW ($\\sim 2.1$ eV, Fig.~\\ref{fullGW}) dispersions.  \nA slightly ($\\sim 14$\\%) wider dispersion than that of the LDA bands obtained for the \neffective Hamiltonian accompanied with frequency independent effective \nCoulomb interactions without the exchange part may account for the slight \noverestimate of correlation effects in the literature mentioned in \nthe introduction.\nOur scheme offers an optimized way for the  derivation of \n{\\it ab initio} models for the L-space after eliminating the H-space\nin  a systematic fashion.\n\n\nOur findings of the band widening are consistent with studies\nbased on the combined GW+DMFT method in the literature \n\\cite{TomczakPRB2014}: There, it was argued that within GW+DMFT\nthe best effective Hamiltonian that DMFT should be performed on,\nis a one-body Hamiltonian corrected by the nonlocal part of the\nGW self-energy. The corresponding spectral function \n(see e.g.\nFigure 5 of Ref.~\\onlinecite{TomczakPRB2014}) displays \na broadening similar to \nFig.~\\ref{DFT+Sigma_HL_GW}.\nMost interestingly, our present calculations confirm a pronounced\nasymmetry observed in Ref.~\\onlinecite{TomczakPRB2014}, namely a stronger widening\neffect in the unoccupied part of the spectrum than in the occupied\none.\n\n",
        "label": "lda-vxc-sigH(0)",
        "Descriptive_question1": "What is the bandwidth value of the band structure in figure_6 for SrVO₃ after including the static corrective self-energy?",
        "Descriptive_question2": "What type of line represents the LDA band structure in figure_6?",
        "Reasoning_question1": "Why does the band structure in figure_6 show a narrowing to 3.7 eV compared to the structure without the exchange-correlation potential?",
        "Reasoning_question2": "How does the inclusion of the static part of the corrective self-energy in figure_6 affect the comparison between the occupied and unoccupied parts of the band structure relative to the LDA values?",
        "Descriptive_answer1": "3.7 eV",
        "Descriptive_answer2": "black dotted line",
        "Reasoning_answer1": "The band structure in figure_6 narrows to 3.7 eV because the static part of the corrective self-energy, ΔΣ_H(ω=0), is taken into account. This inclusion captures electronic correlations coupling the high-energy (H) and low-energy (L) spaces, which partially compensate for the large band widening observed when the LDA exchange-correlation potential (V_xc) is subtracted. Hence, the band narrows from the wider 5.4 eV (when V_xc is removed) down to 3.7 eV, showing that ΔΣ_H(ω=0) introduces correlation effects that reduce the bandwidth toward the physical value.",
        "Reasoning_answer2": "The static corrective self-energy narrows the overall band compared to the scenario without V_xc, but it affects the occupied and unoccupied parts differently. Notably, the bottom of the occupied band aligns precisely with the LDA values, indicating that the occupied states remain at similar energies. In contrast, the unoccupied part remains wider than in LDA, reflecting that the band widening due to ΔΣ_H(ω=0) predominantly occurs in the unoccupied states. This asymmetry implies that the static correction preserves the occupied band energies but leads to a higher energy shift and band broadening in the unoccupied region compared to LDA."
    },
    {
        "paper_id": "1511.03757.json",
        "image_id": "figure_7",
        "image_path": [
            "/home/yz979/scratch/chengye/scimolmo/test-images-tables/dataset_imagestables/1511.03757/images/fig7.eps"
        ],
        "caption": "(Color online)\nFrequency dependence of $-V_{xc}+\\Delta \\Sigma_{\\rm H}$ for SrVO$_3$.\n(a) and (b) are the real and imaginary parts at the $\\Gamma$ point, \nrespectively,\nand (c) and (d) ((e) and (f)) are those at $X$ ($R$).\nFor comparison, the full GW self-energies are also given\n(black dotted line).\n",
        "classify": "Chart",
        "section_info": "6 Results\n\\section{Results}\n\\label{results}\n\n\nThe ternary 3d$^1$ transition metal oxide SrVO$_3$ has become\none of the ``drosophila compounds'' of correlated systems.\nIt is a correlated metal\nthat has been characterized using various experimental \n\\cite{Onoda, inoue, maiti, maiti2, sekiyama, takizawa, yoshida2010}.\nand theoretical techniques, see e.g.\n\\cite{nekrasov, anisimov, pavarini-d1, liebsch, lechermann-d1, trimarchi, aichhorn09, karolak, lee, held-optics, tomczak-gwdmft, gatti-guzzo, sakuma14, TomczakPRB2014}.\nA review of most of the available experimental and theoretical \ndata has been given recently in Ref.~\\onlinecite{TomczakPRB2014}.\nSrVO$_3$ displays Fermi liquid behavior up to\nremarkably high temperatures of the order of $200$ K \\cite{Onoda, inoue},\nwith a moderate mass enhancement of the order \nof $2$ \\cite{yoshida2010, vandermarel}.\nDetailed spectroscopic investigations have made it an\nideal test compound for modern many-body calculations, and \nmore and more refined dynamical mean field-based studies \nare available.\nThe majority of studies so far have focused on the t$_{2g}$-manifold\nwhich forms the states close to the Fermi level, and those will\nalso be the focus in the present investigations. \nNote however that this restriction quite severely limits the\nrange of validity of the low-energy description. Indeed, as\nshown recently~\\cite{tomczak-gwdmft, TomczakPRB2014} at\nenergies of about 2 to 3 eV above the Fermi level\nthe spectral properties are largely determined by the e$_g$\nstates. This has in particular led to a reinterpretation of\nan inverse photoemission feature at about 2.5 eV that was \nfrequently interpreted as an upper Hubbard band of t$_{2g}$\ncharacter in the earlier literature.\nHere, we use the compound only as an illustration of \nthe principles of constructing effective low-energy models,\nwithout being concerned with a description of spectra beyond\nthe pure t$_{2g}$ part.\n\n\nThe LDA band structure of SrVO$_3$ is shown in Fig.~\\ref{lda}.\nOne clearly distinguishes the three-fold degenerate manifold\nof t$_{2g}$ bands close to the Fermi level (chosen as the\nzero of energy), followed in the unoccupied part of the spectrum\nby the two e$_g$ bands. At about -2.5 eV the filled O-2p derived\nbands are visible.\n\nWhen a standard GW calculation is performed, see Fig.~\\ref{fullGW},\nthe bandwidth of the t$_{2g}$-states is reduced from 2.5 eV to\n2.1 eV, while the overall shape of the dispersion remains similar\nto the LDA one.\nThis is in agreement with previous GW calculations in the literature\n\\cite{tomczak-gwdmft, TomczakPRB2014, gatti-guzzo, sakuma14}.\n\n\n\nThe calculation is based on the full-potential linear muffin-tin orbitals (FP-LMTO) implementation.\n8$\\times$8$\\times$8 \\bm{k}-mesh is employed in both the DFT/LDA and the GW calculations.\nThe calculational details are the same as in Ref.~\\onlinecite{Hirayama}.\n\n\nWe now turn to a discussion of the band structure corresponding to \nthe Hamiltonians to be used as input for subsequent many-body \ncalculations for this material.\nWe will proceed step by step to analyze the influence of the various\ncorrection terms, with respect to the DFT starting point\n(which will be overlaid to the respective band structures).\n\n\n\n\\begin{figure}[h]\n\\centering \n\\includegraphics[clip,width=0.45\\textwidth ]{fig2.eps} \n\\caption{(Color online)\nKohn-Sham band structure of SrVO$_3$ in the LDA.\nThe energy is measured from the Fermi level.\n}\n\\label{lda}\n\\end{figure} \n\n\n\\begin{figure}[h]\n\\centering \n\\includegraphics[clip,width=0.45\\textwidth ]{fig3.eps} \n\\caption{(Color online)\nBand structure  of SrVO$_3$ in the one-shot GW approximation.\nFor comparison, the LDA band structure is also given\n(black dotted line).\nThe energy is measured from the Fermi level.\n}\n\\label{fullGW}\n\\end{figure} \n\n\n\n\n\n\n\\begin{figure}[h]\n\\centering \n\\includegraphics[clip,width=0.45\\textwidth ]{fig4.eps} \n\\caption{(Color online)\nBand structure of SrVO$_3$ corresponding to the LDA Hamiltonian\nfrom which the LDA exchange-correlation potential has been subtracted.\nFor comparison, the LDA band structure is also given\n(black dotted line).\nThe energy is measured from the Fermi level.\n}\n\\label{lda-vxc}\n\\end{figure} \n\n\n\\begin{figure}[h]\n\\centering \n\\includegraphics[clip,width=0.45\\textwidth ]{fig5.eps} \n\\caption{(Color online)\nBand structure  of SrVO$_3$ in the Hartree approximation.\nFor comparison, the LDA band structure is also given\n(black dotted line).\nThe energy is measured from the Fermi level.\n}\n\\label{hartree}\n\\end{figure} \n\n\nFigure~\\ref{lda-vxc} \ndisplays the band structure corresponding to\nthe LDA Hamiltonian from which the \nLDA exchange-correlation\npotential has been subtracted,  \nthat is, the eigenvalues of $H_{\\rm LDA}-V_{\\rm xc}$ where\nthese operators are evaluated for the self-consistent LDA\ndensity.\nThe subtraction of $V_{\\rm xc}$ widens the band structure from \nthe LDA bandwidth of 2.5 eV \nto more than twice this value: the new band\nwidth is 5.4 eV.\nThis indicates that the exchange-correlation potential $V_{\\rm xc}$ \nof the LDA is responsible for a substantial amount of \nthe band-narrowing effect\narising from electronic correlations, stressing the need \nto subtract the double counting in a consistent manner. \nFigure~\\ref{lda-vxc} is regarded as the starting point for the\nfollowing considerations.\n\nFor simple materials (see e.g. the calculations on Li in Ref.~\\onlinecite{Lithium})\nit has been noted in the literature that the Hartree band structure\nis close to the DFT one. This raises the question of the origin of the\nsubstantial band widening in the present case. Indeed, the present\ncalculation shown in Fig.~\\ref{lda-vxc} differs from a Hartree calculation\nonly \nby the fact that Fig.~\\ref{lda-vxc} is evaluated for the converged LDA density.\nWe have performed a test calculation where we plot the Hartree band\nstructure calculated for the converged Hartree density. The result\nis shown in Fig.~\\ref{hartree}. As seen from this plot, while the\nband is not fully as wide as in Fig.~\\ref{lda-vxc}, a substantial \nwidening is already present at this stage.\n\n\n\n\\begin{figure}[h]\n\\centering \n\\includegraphics[clip,width=0.45\\textwidth ]{fig6.eps} \n\\caption{(Color online)\nBand structure  of $H_{\\rm LDA}-V_{\\rm xc}+\\Delta \\Sigma_{\\rm H}(\\omega=0)$ of SrVO$_3$.\nFor comparison, the LDA band structure is also given\n(black dotted line).\nThe energy is measured from the Fermi level.\n}\n\\label{lda-vxc-sigH(0)}\n\\end{figure} \n\n\n\nStarting from the Hamiltonian without V$_{xc}$ (see the \ndispersion in Fig.~\\ref{lda-vxc}), we first take into account \nthe static part of the corrective self-energy $\\Delta \\Sigma_{\\rm H}$:\nIn Fig.~\\ref{lda-vxc-sigH(0)}, we plot the dispersion corresponding to\n$H_{\\rm LDA}-V_{\\rm xc}+\\Delta \\Sigma_{\\rm H}(\\omega=0)$. \nCompared to the dispersion of Fig.~\\ref{lda-vxc}, the overall band structure \nis narrowed to 3.7 eV. This value is, however, still considerably larger\nthan the LDA bandwidth (Fig.~\\ref{lda}). Although\n the electronic correlations coupling the H- and \nthe L space are included in Fig.~\\ref{lda-vxc-sigH(0)}, and narrow the band with respect to the case\nwhere V$_{xc}$ is taken out, correlations within\nthe L space are not included. \nThe LDA, on the other hand, at least partially\nincludes correlations within the L space, \nand these are very effective in narrowing the band.\nInterestingly, the bottom of the occupied band is quite\nexactly at the LDA value, and the remaining widening is\npurely in the unoccupied part.\n\n\n\\begin{figure*}[htb]\n\\begin{center} \n\\includegraphics[width=0.9\\textwidth ]{fig7.eps} \n\\end{center} \n\\caption{(Color online)\nFrequency dependence of $-V_{xc}+\\Delta \\Sigma_{\\rm H}$ for SrVO$_3$.\n(a) and (b) are the real and imaginary parts at the $\\Gamma$ point, \nrespectively,\nand (c) and (d) ((e) and (f)) are those at $X$ ($R$).\nFor comparison, the full GW self-energies are also given\n(black dotted line).\n}\n\\label{sigH}\n\\end{figure*} \n\n\n\n\\begin{figure}[h]\n\\centering \n\\includegraphics[clip,width=0.45\\textwidth ]{fig8.eps} \n\\caption{(Color online)\nBand structure  of $Z_{\\rm H}[H_{\\rm LDA}-V_{\\rm xc}+\\Delta\\Sigma_{\\rm H}(\\omega=0)]$ of SrVO$_3$.\nFor comparison, the LDA band structure is also given\n(black dotted line).\nThe energy is measured from the Fermi level.\n}\n\\label{ZH_SigmaH}\n\\end{figure} \n\n\nTo go further and in particular to analyze the dynamic\nbehavior of $\\Delta \\Sigma_{\\rm H}$ \nwe plot in Fig.~\\ref{sigH} \nthe self-energy corrections $\\Delta \\Sigma_{\\rm H}$ (with $\\Sigma$ (see Eq.~(\\ref{full-self-energy}))\nfor comparison) for the real and imaginary parts\nat several representative choices of momenta.\nThe frequency dependence is smooth around the Fermi level.\nIn particular, in contrast to the full self-energy,\nthere are no poles in $\\Delta \\Sigma_{\\rm H}$ \nin the low-energy region, thanks to the\nexclusion of fluctuations within the L-space.\nThe frequency dependence of the real part indicates that the \nlinearization \n$\\displaystyle{\\Delta \\Sigma_{\\rm H}(\\omega) \\sim \\Delta \\Sigma_{\\rm H}(\\omega=0)+[d\\Delta \\Sigma_{\\rm H}(\\omega)/d\\omega]_{\\omega=0}\\omega }$\noffers a reasonably good approximation. \nIn fact, the behavior of the constrained self-energy\nis much closer to linearity than that of \nthe full GW self-energy $\\Sigma$.\nThis is easily understood by the fact that the low-energy excitations are excluded in the present constrained self-energy $\\Delta\\Sigma_{\\rm H}$ analogously to an insulator, thus eliminating the strong frequency dependence of the constrained self-energy near the Fermi level. This is one of the consequences of the \ncontrollability and an advantage of the present MACE scheme, as \nmentioned in the introduction.\nIn the following discussion, we employ this linearized approximation.\nThen the renormalization factor $Z_{\\rm H}$ defined by Eq.~(\\ref{ZH})\nis interpreted as that from the contribution of the H-space. \nAfter \nthis renormalization factor is taken into account,\nthe effective Hamiltonian is given by $Z_{\\rm H}[H_{\\rm LDA}-V_{\\rm xc}+\\Delta\\Sigma_{\\rm H}(\\omega=0)]$ (see Fig.~\\ref{ZH_SigmaH}). \nInterestingly the renormalization factor $Z_{\\rm H}$=0.92 stays\nclose to one, so that\nthe bandwidth is only slightly reduced (to\n$3.4$ eV instead of the $3.7$ eV above). \nWe also note that $Z_{\\rm H}$ has weak momentum dependence\nas we reveal in the following.\n\n\n\n\\begin{figure*}[htb]\n\\begin{center} \n\\includegraphics[width=0.9\\textwidth ]{fig9.eps} \n\\end{center} \n\\caption{(Color online)\n{\\bf minor modification}\nFrequency dependence of $-V_{\\rm xc}+\\Delta \\Sigma_{\\rm H}$ of SrVO$_3$.\n(a) and (b) are the real and imaginary parts of the local\ncomponent, respectively.\nThe black dotted line is $\\Sigma$ defined in Eq.~(\\ref{full-self-energy}) in the full GW approximation for comparison.\n(c) and (d) are the real and imaginary \nparts at several $k$ points.\n}\n\\label{sigH_nonlocal}\n\\end{figure*} \n\n\n\nFigure~\\ref{sigH_nonlocal} \nshows the local and nonlocal parts of $\\Delta \\Sigma_{\\rm H}$.\nThe frequency \ndependence\nof the local part (Figs.~\\ref{sigH_nonlocal} (a)(b)) \nis quite similar \nto the  frequency-dependent self-energies at various momenta\nin Fig.~\\ref{sigH}.\nOne immediately reads off an interesting property, which is\nakin to what has been found for the full GW self-energy in\nRef.~\\onlinecite{Tomczak-PRL2012}, namely that the nonlocal part of \n$\\Delta \\Sigma_{\\rm H}$ shows only weak frequency dependence \nas shown in Figs.~\\ref{sigH_nonlocal} (c)(d). \nThis explains why $Z_{\\rm H}$ \nis only weakly momentum-dependent.\nOn the other hand, $\\Delta \\Sigma_{\\rm H}^{\\rm nonlocal} (0)$ at $R$ is $\\sim 1$ eV larger than \nat $\\Gamma$, which causes the band widening effect.\n $\\Delta \\Sigma_{\\rm H}$\ncan thus to a good approximation be decomposed into a\nfrequency-dependent local part and a static nonlocal one:\n$\\Delta \\Sigma_{\\rm H}(k,\\omega) = \\Delta \\Sigma^{\\rm local}_{\\rm H}(\\omega) + \\Delta \\Sigma^{\\rm nonlocal}_{\\rm H}(k)$,\nwith $ \\Delta \\Sigma_{\\rm H}^{\\rm local}(\\omega) = \\sum_q  \\Delta \\Sigma_{\\rm H}(q,\\omega)$\n\nSince in the simple case of SrVO$_3$ where the t$_{2g}$ states are\ndegenerate, static local operators are scalar and are compensated \nby a chemical potential shift such that the correct particle\nnumber is obtained. The above band structure corresponding to\n$H_{\\rm LDA} - V_{\\rm xc} + \\Delta \\Sigma_{\\rm H}(k, \\omega=0)$ \nis thus identical to that of\n$H_{\\rm LDA} - V_{\\rm xc} + \\Delta \\Sigma^{\\rm nonlocal}_{\\rm H}(k)$.\nThe local dynamical part of $\\Delta \\Sigma_{\\rm H}$ then results in a \nnarrowing of this band structure by a factor $Z_{\\rm H} = 0.92$.\n\n\n\n\\begin{figure}[h]\n\\centering \n\\includegraphics[clip,width=0.45\\textwidth ]{fig10.eps} \n\\caption{(Color online)\n(a) Frequency dependence of the on-site effective interaction.\nPanel (b) is a low-energy zoom of panel (a). \n}\n\\label{W}\n\\end{figure} \n\n\nBefore turning to a discussion of the low-energy correction for the GW treatment\n$\\Delta \\Sigma_{\\rm L} =G^{(0)}(W-W_{\\rm L})$, we analyze the effective\ninteraction $W_{\\rm L}$ shown in Fig.~\\ref{W} \nin comparison to $W$ and $W_{\\rm H}$.\nThe fully screened Coulomb interaction $W$ displays the familiar\nshape of an interaction that is strongly screened at low energies\n(with a value of $0.88$ eV at $\\omega=0$), while retrieving the\nvalue of the bare Coulomb interaction $v$ ($\\sim 16.0$ eV) \nat high energy.\nThe crossover from the bare to the  \nscreened values takes place at the plasma energy of \nabout $15$ eV. This behavior has been discussed before\n\\cite{Review_SB2014}; we note in particular that the plasma \nfrequency is known from electron energy loss spectroscopy \nmeasurements of the related SrTiO$_3$ compound \\cite{vandermarel,kohiki}.\nThe partially screened interaction\n$W_{\\rm H}$ constructed within cRPA\nconverges to $W$ at high energies, but displays\nweaker screening effects at low energies (with a value of $3.5$ eV at $\\omega=0$),\nsince intra-t$_{2g}$ screening channels are excluded.\nAs was already anticipated in Sec.~\\ref{Outline},  $W_{\\rm L}$ can be interpreted as the screened interaction of \na low-energy model where a static interaction of value $W_{\\rm H}(\\omega=0)$\nhas been imposed as the bare interaction. Since now {\\it only} intra-t$_{2g}$\nscreening channels are active, screening takes place only at low\nenergies, where the scale is given by the t$_{2g}$ bandwidth.\nAlso plotted is the difference $W_{\\rm GW}^{\\rm dyn}=W-W_{d}$: except at low energies\nwhere the t$_{2g}$ screening channels come into play, its frequency\ndependence is essentially given by that of $W$, while the\nhigh-energy limit is the bare Coulomb interaction $v$ reduced\nby $W_{\\rm H}(0)$.\n\n\n\\begin{figure*}[htb]\n\\begin{center} \n\\includegraphics[width=0.9\\textwidth ]{fig11.eps} \n\\end{center} \n\\caption{(Color online)\nFrequency dependence of $-V_{\\rm xc}+ G^{(1)}W_{\\rm H}^{\\rm dyn}$ of SrVO$_3$.\n(a) and (b) are the real and imaginary local parts, respectively.\n(c) and (d) are the real and imaginary \nparts at several $k$ points.\n}\n\\label{DeltaSigma_L_Pert}\n\\end{figure*} \n\n\n\n\\begin{figure*}[htb]\n\\begin{center} \n\\includegraphics[width=0.9\\textwidth ]{fig12.eps} \n\\end{center} \n\\caption{(Color online)\nFrequency dependence of $-V_{\\rm xc}+ G^{(0)}W_{\\rm GW}^{\\rm dyn}$ of SrVO$_3$.\n(a) and (b) are the real and imaginary local parts, respectively.\n(c) and (d) are the real and imaginary \nparts at several $k$ points.\n}\n\\label{DeltaSigma_L_GW}\n\\end{figure*} \n\n\n\nThe frequency dependence of the real and imaginary parts of the low-energy self-energy $\\Delta \\Sigma_{\\rm L} = G^{(1)}W_{\\rm H}^{\\rm dyn}$ for the direct perturbative treatment is illustrated in Fig.~\\ref{DeltaSigma_L_Pert}. \n$\\Delta \\Sigma_{\\rm L} = G^{(0)}W_{\\rm GW}^{\\rm dyn}$ used\nfor the GW treatment is shown in Fig.~\\ref{DeltaSigma_L_GW}\nfor several choices of momenta.\n\\\\\nThe results for the perturbative and the GW treatment are nearly \nidentical.\nThe renormalization factor of $\\Delta \\Sigma_{\\rm L}^{\\rm nonlocal}$ \n(and $\\Delta \\Sigma_{\\rm L}$) is $\\sim 0.77$.\nThe zero-frequency shift\n${\\rm Re}\\Delta \\Sigma_{\\rm L}^{\\rm nonlocal} (0)$ at $R$ is $\\sim 3$ eV \nlarger than \nat $\\Gamma$.\nAgain, one sees how $\\Sigma_{\\rm L}$ separates into local dynamical\nand nonlocal static parts: \n$\\Delta \\Sigma_{\\rm L} = \\Delta \\Sigma^{\\rm nonlocal}_{\\rm L}(k) + \n\\Delta \\Sigma^{\\rm local}_{\\rm L}(\\omega)$.\nThis is expected since such a separation has been found within\nthe full GW calculation \\cite{TomczakPRB2014}, and is even\nmore plausible for a GW treatment within the t$_{2g}$ subspace.\n\n\n\n\\begin{figure}[h]\n\\centering \n\\includegraphics[clip,width=0.45\\textwidth ]{fig13.eps} \n\\caption{(Color online)\nBand structure  of $[\\epsilon_{\\rm DFT}-V_{\\rm xc}+\\Delta\\Sigma_{\\rm H}+\\Delta \\Sigma_{\\rm L}^{\\rm nonlocal}]Z_{\\rm HL}$ of SrVO$_3$ in the direct perturbative treatment.\nFor comparison, the LDA band structure is also given\n(black dotted line).\nThe energy is measured from the Fermi level.\n}\n\\label{DFT+Sigma_HL_Pert}\n\\end{figure} \n\n\n\n\\begin{figure}[h]\n\\centering \n\\includegraphics[clip,width=0.45\\textwidth ]{fig14.eps} \n\\caption{(Color online)\nBand structure  of $[\\epsilon_{\\rm DFT}-V_{\\rm xc}+\\Delta\\Sigma_{\\rm H}+\\Delta \\Sigma_{\\rm L}^{\\rm nonlocal}]Z_{\\rm HL}$ of SrVO$_3$ in the GW treatment.\nFor comparison, the LDA band structure is also given\n(black dotted line).\nThe energy is measured from the Fermi level.\n}\n\\label{DFT+Sigma_HL_GW}\n\\end{figure} \n\n\n\nThe corresponding band structures \nare given by \n $[\\epsilon_{\\rm DFT}+\\Delta\\Sigma_{\\rm H}+\\Delta \\Sigma_{\\rm L}^{\\rm nonlocal}]Z_{\\rm HL}$ in Fig.~\\ref{DFT+Sigma_HL_Pert} \nfor the perturbative treatment (see Eq.~(\\ref{DeltaSigmaLnonlocalPert})), and in Fig.~\\ref{DFT+Sigma_HL_GW} \nfor the GW treatment,\nwhere $\\Delta \\Sigma_{\\rm L}^{\\rm nonlocal}$ is defined in Eq.~(\\ref{DeltaSigLnonlocalGW}) and $Z_{\\rm HL}$ is around 0.92.\n$Z_{\\rm HL}$ is practically the same as $Z_{\\rm H}$, meaning that the nonlocal dynamical correction \nin the H-space is small, which justifies the GW perturbative treatment for the nonlocal part.\nOne finds that \nthe nonlocal part $\\Delta \\Sigma_{\\rm L}=G^{(0)}_{ll}(W-W_{\\rm L})$ narrows\nthe empty states but widens the occupied ones, resulting\nin an overall bandwidth of 3.2 eV.\nThe difference between the direct perturbation and the GW treatment is small.\n\n\n\n\\begin{figure}[h]\n\\centering \n\\includegraphics[clip,width=0.45\\textwidth ]{fig15.eps} \n\\caption{(Color online)\nBand structure  of $[\\epsilon_{\\rm DFT}-V_{\\rm xc}+\\Delta\\Sigma_{\\rm H}+\\Delta\\Sigma_{\\rm L}^{\\rm nonlocal}]Z_{\\rm HL}Z_B$ of SrVO$_3$ in the GW treatment.\nFor comparison, the LDA band structure is also given\n(black dotted line).\nThe energy is measured from the Fermi level.}\n\\label{DFT+DeltaSigmaL_GW_ZHL_ZB}\n\\end{figure} \n\n\nAfter including the effect of the local self-energy by the nonperturbative \nCasula trick, we show the dispersion\ngiven by \n$[\\epsilon_{\\rm DFT}-V_{\\rm xc}+\\Delta\\Sigma_{\\rm H}+\\Delta\\Sigma_{\\rm L}^{\\rm nonlocal}]Z_{\\rm HL}Z_B$ for the GW-type treatment in Fig.~\\ref{DFT+DeltaSigmaL_GW_ZHL_ZB}: \nThe GW-like treatment gives a band dispersion\nclose to the LDA one. \nThe $Z_B$ factor corresponding to the local dynamical part of \n$\\Delta \\Sigma_{\\rm L}$ amounts to $Z_B=0.7$.\nThe nonperturbative treatment \nresults in an LDA-like band dispersion for the empty states,\nbut a slightly narrower bandwidth in the occupied part.\nThe low-energy effective Hamiltonian at this level of treatment has a \nfrequency independent effective interaction both with local \nand nonlocal interaction given by the Fourier transform of \n$W_{\\rm H}(q,\\omega=0)$, which contains both direct and exchange interactions. \n\n\n\\begin{figure}[h]\n\\centering \n\\includegraphics[clip,width=0.45\\textwidth ]{fig16.eps} \n\\caption{(Color online)\nBand structure  of $[\\epsilon_{\\rm DFT}+\\Sigma_{\\rm corr}^{\\rm GW}]Z_{\\rm corr}Z_B$ of SrVO$_3$.\nFor comparison, the LDA band structure is also given\n(black dotted line).\nThe energy is measured from the Fermi level.}\n\\label{DFT+Sigma_corr_GW_Z}\n\\end{figure} \n\n\n\nWe finally show the dispersion\ngiven by Eq.~(\\ref{Teff_final}), namely\n$[\\epsilon_{\\rm DFT}+\\Sigma_{\\rm corr}^{\\rm GW}]Z_{\\rm corr}Z_B$ in Fig.~\\ref{DFT+Sigma_corr_GW_Z}. \nThe result shows an LDA-like band dispersion for the occupied states,\nbut a slightly wider bandwidth in the empty part, resulting in a bandwidth 14\\% wider in total than the LDA bandwidth. \nThe effective interaction of the low-energy effective Hamiltonian \nat this final level has\nfrequency independent local and nonlocal interactions \ngiven by the Fourier transform of $W_r(q,\\omega=0)$. \n\n\n\n\\begin{figure}[h]\n\\centering \n\\includegraphics[clip,width=0.45\\textwidth ]{fig17.eps} \n\\caption{(Color online)\nBand structure  of $[\\epsilon_{\\rm DFT}-V_{\\rm xc}+\\Sigma-(G^{(0)}W_{\\rm GW}^{\\rm dyn})^{\\rm nonlocal}]Z'Z_B$ of SrVO$_3$\n(Z' is the renormalization factor of $\\Sigma-(G^{(0)}W_{\\rm GW}^{\\rm dyn})^{\\rm nonlocal}$).\nFor comparison, the LDA band structure is also given\n(black dotted line).\nThe energy is measured from the Fermi level.\n}\n\\label{GW+ZB}\n\\end{figure} \n\n\n\nThe resulting effective Hamiltonian in the L-space \nconsists of single-particle and two-particle (interaction) parts.\nThe effective interaction is the same as in previous estimates \nby the cRPA in the literature, while the single-particle dispersion \nis revised after removing the double counting and taking into \naccount the frequency dependence of the effective interaction. The \nfinal effective bandwidth for SrVO$_3$ is,\nafter partial cancellation, \nslightly (14\\%) larger than the LDA bandwidth.  \nThis is a reasonable result because the LDA takes into account all \ncorrelations though insufficiently, while the present scheme by the constrained self-energy excludes the \ncorrelation effects arising from the L-space.\nAlthough the bandwidth derived for the effective model is slightly \nlarger than that of the LDA,  it is clear that the correlation effects \nare stronger than in the LDA or standard GW when the {\\it ab initio} \nmodel is solved by an accurate solver. In fact, if the effective \ninteraction contained in the final effective model is treated by \nthe GW scheme, one obtains the dispersion illustrated in \nFig.~\\ref{GW+ZB}, \nwhich is given from the self energy of the whole GW calculation $\\Sigma$ \nby correcting the local dynamical part $W-W_{\\rm L}$ by $Z_B$. \nThis indicates that even an insufficient treatment of the local \nstatic interaction by the GW scheme gives a dispersion with the bandwidth ($\\sim 1.9$ eV) narrower \nthan the LDA ($\\sim 2.5$ eV, Fig.~\\ref{lda}) and GW ($\\sim 2.1$ eV, Fig.~\\ref{fullGW}) dispersions.  \nA slightly ($\\sim 14$\\%) wider dispersion than that of the LDA bands obtained for the \neffective Hamiltonian accompanied with frequency independent effective \nCoulomb interactions without the exchange part may account for the slight \noverestimate of correlation effects in the literature mentioned in \nthe introduction.\nOur scheme offers an optimized way for the  derivation of \n{\\it ab initio} models for the L-space after eliminating the H-space\nin  a systematic fashion.\n\n\nOur findings of the band widening are consistent with studies\nbased on the combined GW+DMFT method in the literature \n\\cite{TomczakPRB2014}: There, it was argued that within GW+DMFT\nthe best effective Hamiltonian that DMFT should be performed on,\nis a one-body Hamiltonian corrected by the nonlocal part of the\nGW self-energy. The corresponding spectral function \n(see e.g.\nFigure 5 of Ref.~\\onlinecite{TomczakPRB2014}) displays \na broadening similar to \nFig.~\\ref{DFT+Sigma_HL_GW}.\nMost interestingly, our present calculations confirm a pronounced\nasymmetry observed in Ref.~\\onlinecite{TomczakPRB2014}, namely a stronger widening\neffect in the unoccupied part of the spectrum than in the occupied\none.\n\n",
        "label": "sigH",
        "Descriptive_question1": "What is the label of the point associated with panels (c) and (d) in figure_7?",
        "Descriptive_question2": "What type of line represents the full GW self-energies in figure_7?",
        "Reasoning_question1": "How does the frequency dependence of the real part of -Vxc + ΔΣH at the Γ point in figure_7 compare to that at the X point, and what might this imply about the electronic correlations at these points?",
        "Reasoning_question2": "Based on the comparison with the full GW self-energies shown in figure_7, what can be inferred about the behavior of ΔΣH in the low-energy region across different momentum points?",
        "Descriptive_answer1": "X",
        "Descriptive_answer2": "black dotted line",
        "Reasoning_answer1": "By examining figure_7, the real part of -V_{xc} + \\Delta \\Sigma_{\\rm H} at the \\Gamma point (panel (a)) shows a smooth and more linear frequency dependence around zero energy compared to that at the X point (panel (c)), which might exhibit slightly different curvature or features. This suggests that at the \\Gamma point, electronic correlations captured by \\Delta \\Sigma_{\\rm H} have a more straightforward, less complicated dynamic behavior, implying that local interactions or fluctuations are more uniform or weaker here. At the X point, deviations from linearity could indicate stronger or more complex correlations or screening effects in that momentum region.",
        "Reasoning_answer2": "From figure_7, the full GW self-energies represented by the black dotted lines show poles and significant frequency dependence in the low-energy region at all momentum points. In contrast, \\Delta \\Sigma_{\\rm H}, which excludes low-energy fluctuations within the L-space, shows a smooth frequency dependence without poles at low energies across all momenta (\\Gamma, X, and R points). This indicates that \\Delta \\Sigma_{\\rm H} effectively removes low-energy correlation effects, reflecting only higher-energy processes and resulting in more controllable self-energy behavior in the low-energy region."
    },
    {
        "paper_id": "1511.03757.json",
        "image_id": "figure_8",
        "image_path": [
            "/home/yz979/scratch/chengye/scimolmo/test-images-tables/dataset_imagestables/1511.03757/images/fig8.eps"
        ],
        "caption": "(Color online)\nBand structure  of $Z_{\\rm H}[H_{\\rm LDA}-V_{\\rm xc}+\\Delta\\Sigma_{\\rm H}(\\omega=0)]$ of SrVO$_3$.\nFor comparison, the LDA band structure is also given\n(black dotted line).\nThe energy is measured from the Fermi level.\n",
        "classify": "Chart",
        "section_info": "6 Results\n\\section{Results}\n\\label{results}\n\n\nThe ternary 3d$^1$ transition metal oxide SrVO$_3$ has become\none of the ``drosophila compounds'' of correlated systems.\nIt is a correlated metal\nthat has been characterized using various experimental \n\\cite{Onoda, inoue, maiti, maiti2, sekiyama, takizawa, yoshida2010}.\nand theoretical techniques, see e.g.\n\\cite{nekrasov, anisimov, pavarini-d1, liebsch, lechermann-d1, trimarchi, aichhorn09, karolak, lee, held-optics, tomczak-gwdmft, gatti-guzzo, sakuma14, TomczakPRB2014}.\nA review of most of the available experimental and theoretical \ndata has been given recently in Ref.~\\onlinecite{TomczakPRB2014}.\nSrVO$_3$ displays Fermi liquid behavior up to\nremarkably high temperatures of the order of $200$ K \\cite{Onoda, inoue},\nwith a moderate mass enhancement of the order \nof $2$ \\cite{yoshida2010, vandermarel}.\nDetailed spectroscopic investigations have made it an\nideal test compound for modern many-body calculations, and \nmore and more refined dynamical mean field-based studies \nare available.\nThe majority of studies so far have focused on the t$_{2g}$-manifold\nwhich forms the states close to the Fermi level, and those will\nalso be the focus in the present investigations. \nNote however that this restriction quite severely limits the\nrange of validity of the low-energy description. Indeed, as\nshown recently~\\cite{tomczak-gwdmft, TomczakPRB2014} at\nenergies of about 2 to 3 eV above the Fermi level\nthe spectral properties are largely determined by the e$_g$\nstates. This has in particular led to a reinterpretation of\nan inverse photoemission feature at about 2.5 eV that was \nfrequently interpreted as an upper Hubbard band of t$_{2g}$\ncharacter in the earlier literature.\nHere, we use the compound only as an illustration of \nthe principles of constructing effective low-energy models,\nwithout being concerned with a description of spectra beyond\nthe pure t$_{2g}$ part.\n\n\nThe LDA band structure of SrVO$_3$ is shown in Fig.~\\ref{lda}.\nOne clearly distinguishes the three-fold degenerate manifold\nof t$_{2g}$ bands close to the Fermi level (chosen as the\nzero of energy), followed in the unoccupied part of the spectrum\nby the two e$_g$ bands. At about -2.5 eV the filled O-2p derived\nbands are visible.\n\nWhen a standard GW calculation is performed, see Fig.~\\ref{fullGW},\nthe bandwidth of the t$_{2g}$-states is reduced from 2.5 eV to\n2.1 eV, while the overall shape of the dispersion remains similar\nto the LDA one.\nThis is in agreement with previous GW calculations in the literature\n\\cite{tomczak-gwdmft, TomczakPRB2014, gatti-guzzo, sakuma14}.\n\n\n\nThe calculation is based on the full-potential linear muffin-tin orbitals (FP-LMTO) implementation.\n8$\\times$8$\\times$8 \\bm{k}-mesh is employed in both the DFT/LDA and the GW calculations.\nThe calculational details are the same as in Ref.~\\onlinecite{Hirayama}.\n\n\nWe now turn to a discussion of the band structure corresponding to \nthe Hamiltonians to be used as input for subsequent many-body \ncalculations for this material.\nWe will proceed step by step to analyze the influence of the various\ncorrection terms, with respect to the DFT starting point\n(which will be overlaid to the respective band structures).\n\n\n\n\\begin{figure}[h]\n\\centering \n\\includegraphics[clip,width=0.45\\textwidth ]{fig2.eps} \n\\caption{(Color online)\nKohn-Sham band structure of SrVO$_3$ in the LDA.\nThe energy is measured from the Fermi level.\n}\n\\label{lda}\n\\end{figure} \n\n\n\\begin{figure}[h]\n\\centering \n\\includegraphics[clip,width=0.45\\textwidth ]{fig3.eps} \n\\caption{(Color online)\nBand structure  of SrVO$_3$ in the one-shot GW approximation.\nFor comparison, the LDA band structure is also given\n(black dotted line).\nThe energy is measured from the Fermi level.\n}\n\\label{fullGW}\n\\end{figure} \n\n\n\n\n\n\n\\begin{figure}[h]\n\\centering \n\\includegraphics[clip,width=0.45\\textwidth ]{fig4.eps} \n\\caption{(Color online)\nBand structure of SrVO$_3$ corresponding to the LDA Hamiltonian\nfrom which the LDA exchange-correlation potential has been subtracted.\nFor comparison, the LDA band structure is also given\n(black dotted line).\nThe energy is measured from the Fermi level.\n}\n\\label{lda-vxc}\n\\end{figure} \n\n\n\\begin{figure}[h]\n\\centering \n\\includegraphics[clip,width=0.45\\textwidth ]{fig5.eps} \n\\caption{(Color online)\nBand structure  of SrVO$_3$ in the Hartree approximation.\nFor comparison, the LDA band structure is also given\n(black dotted line).\nThe energy is measured from the Fermi level.\n}\n\\label{hartree}\n\\end{figure} \n\n\nFigure~\\ref{lda-vxc} \ndisplays the band structure corresponding to\nthe LDA Hamiltonian from which the \nLDA exchange-correlation\npotential has been subtracted,  \nthat is, the eigenvalues of $H_{\\rm LDA}-V_{\\rm xc}$ where\nthese operators are evaluated for the self-consistent LDA\ndensity.\nThe subtraction of $V_{\\rm xc}$ widens the band structure from \nthe LDA bandwidth of 2.5 eV \nto more than twice this value: the new band\nwidth is 5.4 eV.\nThis indicates that the exchange-correlation potential $V_{\\rm xc}$ \nof the LDA is responsible for a substantial amount of \nthe band-narrowing effect\narising from electronic correlations, stressing the need \nto subtract the double counting in a consistent manner. \nFigure~\\ref{lda-vxc} is regarded as the starting point for the\nfollowing considerations.\n\nFor simple materials (see e.g. the calculations on Li in Ref.~\\onlinecite{Lithium})\nit has been noted in the literature that the Hartree band structure\nis close to the DFT one. This raises the question of the origin of the\nsubstantial band widening in the present case. Indeed, the present\ncalculation shown in Fig.~\\ref{lda-vxc} differs from a Hartree calculation\nonly \nby the fact that Fig.~\\ref{lda-vxc} is evaluated for the converged LDA density.\nWe have performed a test calculation where we plot the Hartree band\nstructure calculated for the converged Hartree density. The result\nis shown in Fig.~\\ref{hartree}. As seen from this plot, while the\nband is not fully as wide as in Fig.~\\ref{lda-vxc}, a substantial \nwidening is already present at this stage.\n\n\n\n\\begin{figure}[h]\n\\centering \n\\includegraphics[clip,width=0.45\\textwidth ]{fig6.eps} \n\\caption{(Color online)\nBand structure  of $H_{\\rm LDA}-V_{\\rm xc}+\\Delta \\Sigma_{\\rm H}(\\omega=0)$ of SrVO$_3$.\nFor comparison, the LDA band structure is also given\n(black dotted line).\nThe energy is measured from the Fermi level.\n}\n\\label{lda-vxc-sigH(0)}\n\\end{figure} \n\n\n\nStarting from the Hamiltonian without V$_{xc}$ (see the \ndispersion in Fig.~\\ref{lda-vxc}), we first take into account \nthe static part of the corrective self-energy $\\Delta \\Sigma_{\\rm H}$:\nIn Fig.~\\ref{lda-vxc-sigH(0)}, we plot the dispersion corresponding to\n$H_{\\rm LDA}-V_{\\rm xc}+\\Delta \\Sigma_{\\rm H}(\\omega=0)$. \nCompared to the dispersion of Fig.~\\ref{lda-vxc}, the overall band structure \nis narrowed to 3.7 eV. This value is, however, still considerably larger\nthan the LDA bandwidth (Fig.~\\ref{lda}). Although\n the electronic correlations coupling the H- and \nthe L space are included in Fig.~\\ref{lda-vxc-sigH(0)}, and narrow the band with respect to the case\nwhere V$_{xc}$ is taken out, correlations within\nthe L space are not included. \nThe LDA, on the other hand, at least partially\nincludes correlations within the L space, \nand these are very effective in narrowing the band.\nInterestingly, the bottom of the occupied band is quite\nexactly at the LDA value, and the remaining widening is\npurely in the unoccupied part.\n\n\n\\begin{figure*}[htb]\n\\begin{center} \n\\includegraphics[width=0.9\\textwidth ]{fig7.eps} \n\\end{center} \n\\caption{(Color online)\nFrequency dependence of $-V_{xc}+\\Delta \\Sigma_{\\rm H}$ for SrVO$_3$.\n(a) and (b) are the real and imaginary parts at the $\\Gamma$ point, \nrespectively,\nand (c) and (d) ((e) and (f)) are those at $X$ ($R$).\nFor comparison, the full GW self-energies are also given\n(black dotted line).\n}\n\\label{sigH}\n\\end{figure*} \n\n\n\n\\begin{figure}[h]\n\\centering \n\\includegraphics[clip,width=0.45\\textwidth ]{fig8.eps} \n\\caption{(Color online)\nBand structure  of $Z_{\\rm H}[H_{\\rm LDA}-V_{\\rm xc}+\\Delta\\Sigma_{\\rm H}(\\omega=0)]$ of SrVO$_3$.\nFor comparison, the LDA band structure is also given\n(black dotted line).\nThe energy is measured from the Fermi level.\n}\n\\label{ZH_SigmaH}\n\\end{figure} \n\n\nTo go further and in particular to analyze the dynamic\nbehavior of $\\Delta \\Sigma_{\\rm H}$ \nwe plot in Fig.~\\ref{sigH} \nthe self-energy corrections $\\Delta \\Sigma_{\\rm H}$ (with $\\Sigma$ (see Eq.~(\\ref{full-self-energy}))\nfor comparison) for the real and imaginary parts\nat several representative choices of momenta.\nThe frequency dependence is smooth around the Fermi level.\nIn particular, in contrast to the full self-energy,\nthere are no poles in $\\Delta \\Sigma_{\\rm H}$ \nin the low-energy region, thanks to the\nexclusion of fluctuations within the L-space.\nThe frequency dependence of the real part indicates that the \nlinearization \n$\\displaystyle{\\Delta \\Sigma_{\\rm H}(\\omega) \\sim \\Delta \\Sigma_{\\rm H}(\\omega=0)+[d\\Delta \\Sigma_{\\rm H}(\\omega)/d\\omega]_{\\omega=0}\\omega }$\noffers a reasonably good approximation. \nIn fact, the behavior of the constrained self-energy\nis much closer to linearity than that of \nthe full GW self-energy $\\Sigma$.\nThis is easily understood by the fact that the low-energy excitations are excluded in the present constrained self-energy $\\Delta\\Sigma_{\\rm H}$ analogously to an insulator, thus eliminating the strong frequency dependence of the constrained self-energy near the Fermi level. This is one of the consequences of the \ncontrollability and an advantage of the present MACE scheme, as \nmentioned in the introduction.\nIn the following discussion, we employ this linearized approximation.\nThen the renormalization factor $Z_{\\rm H}$ defined by Eq.~(\\ref{ZH})\nis interpreted as that from the contribution of the H-space. \nAfter \nthis renormalization factor is taken into account,\nthe effective Hamiltonian is given by $Z_{\\rm H}[H_{\\rm LDA}-V_{\\rm xc}+\\Delta\\Sigma_{\\rm H}(\\omega=0)]$ (see Fig.~\\ref{ZH_SigmaH}). \nInterestingly the renormalization factor $Z_{\\rm H}$=0.92 stays\nclose to one, so that\nthe bandwidth is only slightly reduced (to\n$3.4$ eV instead of the $3.7$ eV above). \nWe also note that $Z_{\\rm H}$ has weak momentum dependence\nas we reveal in the following.\n\n\n\n\\begin{figure*}[htb]\n\\begin{center} \n\\includegraphics[width=0.9\\textwidth ]{fig9.eps} \n\\end{center} \n\\caption{(Color online)\n{\\bf minor modification}\nFrequency dependence of $-V_{\\rm xc}+\\Delta \\Sigma_{\\rm H}$ of SrVO$_3$.\n(a) and (b) are the real and imaginary parts of the local\ncomponent, respectively.\nThe black dotted line is $\\Sigma$ defined in Eq.~(\\ref{full-self-energy}) in the full GW approximation for comparison.\n(c) and (d) are the real and imaginary \nparts at several $k$ points.\n}\n\\label{sigH_nonlocal}\n\\end{figure*} \n\n\n\nFigure~\\ref{sigH_nonlocal} \nshows the local and nonlocal parts of $\\Delta \\Sigma_{\\rm H}$.\nThe frequency \ndependence\nof the local part (Figs.~\\ref{sigH_nonlocal} (a)(b)) \nis quite similar \nto the  frequency-dependent self-energies at various momenta\nin Fig.~\\ref{sigH}.\nOne immediately reads off an interesting property, which is\nakin to what has been found for the full GW self-energy in\nRef.~\\onlinecite{Tomczak-PRL2012}, namely that the nonlocal part of \n$\\Delta \\Sigma_{\\rm H}$ shows only weak frequency dependence \nas shown in Figs.~\\ref{sigH_nonlocal} (c)(d). \nThis explains why $Z_{\\rm H}$ \nis only weakly momentum-dependent.\nOn the other hand, $\\Delta \\Sigma_{\\rm H}^{\\rm nonlocal} (0)$ at $R$ is $\\sim 1$ eV larger than \nat $\\Gamma$, which causes the band widening effect.\n $\\Delta \\Sigma_{\\rm H}$\ncan thus to a good approximation be decomposed into a\nfrequency-dependent local part and a static nonlocal one:\n$\\Delta \\Sigma_{\\rm H}(k,\\omega) = \\Delta \\Sigma^{\\rm local}_{\\rm H}(\\omega) + \\Delta \\Sigma^{\\rm nonlocal}_{\\rm H}(k)$,\nwith $ \\Delta \\Sigma_{\\rm H}^{\\rm local}(\\omega) = \\sum_q  \\Delta \\Sigma_{\\rm H}(q,\\omega)$\n\nSince in the simple case of SrVO$_3$ where the t$_{2g}$ states are\ndegenerate, static local operators are scalar and are compensated \nby a chemical potential shift such that the correct particle\nnumber is obtained. The above band structure corresponding to\n$H_{\\rm LDA} - V_{\\rm xc} + \\Delta \\Sigma_{\\rm H}(k, \\omega=0)$ \nis thus identical to that of\n$H_{\\rm LDA} - V_{\\rm xc} + \\Delta \\Sigma^{\\rm nonlocal}_{\\rm H}(k)$.\nThe local dynamical part of $\\Delta \\Sigma_{\\rm H}$ then results in a \nnarrowing of this band structure by a factor $Z_{\\rm H} = 0.92$.\n\n\n\n\\begin{figure}[h]\n\\centering \n\\includegraphics[clip,width=0.45\\textwidth ]{fig10.eps} \n\\caption{(Color online)\n(a) Frequency dependence of the on-site effective interaction.\nPanel (b) is a low-energy zoom of panel (a). \n}\n\\label{W}\n\\end{figure} \n\n\nBefore turning to a discussion of the low-energy correction for the GW treatment\n$\\Delta \\Sigma_{\\rm L} =G^{(0)}(W-W_{\\rm L})$, we analyze the effective\ninteraction $W_{\\rm L}$ shown in Fig.~\\ref{W} \nin comparison to $W$ and $W_{\\rm H}$.\nThe fully screened Coulomb interaction $W$ displays the familiar\nshape of an interaction that is strongly screened at low energies\n(with a value of $0.88$ eV at $\\omega=0$), while retrieving the\nvalue of the bare Coulomb interaction $v$ ($\\sim 16.0$ eV) \nat high energy.\nThe crossover from the bare to the  \nscreened values takes place at the plasma energy of \nabout $15$ eV. This behavior has been discussed before\n\\cite{Review_SB2014}; we note in particular that the plasma \nfrequency is known from electron energy loss spectroscopy \nmeasurements of the related SrTiO$_3$ compound \\cite{vandermarel,kohiki}.\nThe partially screened interaction\n$W_{\\rm H}$ constructed within cRPA\nconverges to $W$ at high energies, but displays\nweaker screening effects at low energies (with a value of $3.5$ eV at $\\omega=0$),\nsince intra-t$_{2g}$ screening channels are excluded.\nAs was already anticipated in Sec.~\\ref{Outline},  $W_{\\rm L}$ can be interpreted as the screened interaction of \na low-energy model where a static interaction of value $W_{\\rm H}(\\omega=0)$\nhas been imposed as the bare interaction. Since now {\\it only} intra-t$_{2g}$\nscreening channels are active, screening takes place only at low\nenergies, where the scale is given by the t$_{2g}$ bandwidth.\nAlso plotted is the difference $W_{\\rm GW}^{\\rm dyn}=W-W_{d}$: except at low energies\nwhere the t$_{2g}$ screening channels come into play, its frequency\ndependence is essentially given by that of $W$, while the\nhigh-energy limit is the bare Coulomb interaction $v$ reduced\nby $W_{\\rm H}(0)$.\n\n\n\\begin{figure*}[htb]\n\\begin{center} \n\\includegraphics[width=0.9\\textwidth ]{fig11.eps} \n\\end{center} \n\\caption{(Color online)\nFrequency dependence of $-V_{\\rm xc}+ G^{(1)}W_{\\rm H}^{\\rm dyn}$ of SrVO$_3$.\n(a) and (b) are the real and imaginary local parts, respectively.\n(c) and (d) are the real and imaginary \nparts at several $k$ points.\n}\n\\label{DeltaSigma_L_Pert}\n\\end{figure*} \n\n\n\n\\begin{figure*}[htb]\n\\begin{center} \n\\includegraphics[width=0.9\\textwidth ]{fig12.eps} \n\\end{center} \n\\caption{(Color online)\nFrequency dependence of $-V_{\\rm xc}+ G^{(0)}W_{\\rm GW}^{\\rm dyn}$ of SrVO$_3$.\n(a) and (b) are the real and imaginary local parts, respectively.\n(c) and (d) are the real and imaginary \nparts at several $k$ points.\n}\n\\label{DeltaSigma_L_GW}\n\\end{figure*} \n\n\n\nThe frequency dependence of the real and imaginary parts of the low-energy self-energy $\\Delta \\Sigma_{\\rm L} = G^{(1)}W_{\\rm H}^{\\rm dyn}$ for the direct perturbative treatment is illustrated in Fig.~\\ref{DeltaSigma_L_Pert}. \n$\\Delta \\Sigma_{\\rm L} = G^{(0)}W_{\\rm GW}^{\\rm dyn}$ used\nfor the GW treatment is shown in Fig.~\\ref{DeltaSigma_L_GW}\nfor several choices of momenta.\n\\\\\nThe results for the perturbative and the GW treatment are nearly \nidentical.\nThe renormalization factor of $\\Delta \\Sigma_{\\rm L}^{\\rm nonlocal}$ \n(and $\\Delta \\Sigma_{\\rm L}$) is $\\sim 0.77$.\nThe zero-frequency shift\n${\\rm Re}\\Delta \\Sigma_{\\rm L}^{\\rm nonlocal} (0)$ at $R$ is $\\sim 3$ eV \nlarger than \nat $\\Gamma$.\nAgain, one sees how $\\Sigma_{\\rm L}$ separates into local dynamical\nand nonlocal static parts: \n$\\Delta \\Sigma_{\\rm L} = \\Delta \\Sigma^{\\rm nonlocal}_{\\rm L}(k) + \n\\Delta \\Sigma^{\\rm local}_{\\rm L}(\\omega)$.\nThis is expected since such a separation has been found within\nthe full GW calculation \\cite{TomczakPRB2014}, and is even\nmore plausible for a GW treatment within the t$_{2g}$ subspace.\n\n\n\n\\begin{figure}[h]\n\\centering \n\\includegraphics[clip,width=0.45\\textwidth ]{fig13.eps} \n\\caption{(Color online)\nBand structure  of $[\\epsilon_{\\rm DFT}-V_{\\rm xc}+\\Delta\\Sigma_{\\rm H}+\\Delta \\Sigma_{\\rm L}^{\\rm nonlocal}]Z_{\\rm HL}$ of SrVO$_3$ in the direct perturbative treatment.\nFor comparison, the LDA band structure is also given\n(black dotted line).\nThe energy is measured from the Fermi level.\n}\n\\label{DFT+Sigma_HL_Pert}\n\\end{figure} \n\n\n\n\\begin{figure}[h]\n\\centering \n\\includegraphics[clip,width=0.45\\textwidth ]{fig14.eps} \n\\caption{(Color online)\nBand structure  of $[\\epsilon_{\\rm DFT}-V_{\\rm xc}+\\Delta\\Sigma_{\\rm H}+\\Delta \\Sigma_{\\rm L}^{\\rm nonlocal}]Z_{\\rm HL}$ of SrVO$_3$ in the GW treatment.\nFor comparison, the LDA band structure is also given\n(black dotted line).\nThe energy is measured from the Fermi level.\n}\n\\label{DFT+Sigma_HL_GW}\n\\end{figure} \n\n\n\nThe corresponding band structures \nare given by \n $[\\epsilon_{\\rm DFT}+\\Delta\\Sigma_{\\rm H}+\\Delta \\Sigma_{\\rm L}^{\\rm nonlocal}]Z_{\\rm HL}$ in Fig.~\\ref{DFT+Sigma_HL_Pert} \nfor the perturbative treatment (see Eq.~(\\ref{DeltaSigmaLnonlocalPert})), and in Fig.~\\ref{DFT+Sigma_HL_GW} \nfor the GW treatment,\nwhere $\\Delta \\Sigma_{\\rm L}^{\\rm nonlocal}$ is defined in Eq.~(\\ref{DeltaSigLnonlocalGW}) and $Z_{\\rm HL}$ is around 0.92.\n$Z_{\\rm HL}$ is practically the same as $Z_{\\rm H}$, meaning that the nonlocal dynamical correction \nin the H-space is small, which justifies the GW perturbative treatment for the nonlocal part.\nOne finds that \nthe nonlocal part $\\Delta \\Sigma_{\\rm L}=G^{(0)}_{ll}(W-W_{\\rm L})$ narrows\nthe empty states but widens the occupied ones, resulting\nin an overall bandwidth of 3.2 eV.\nThe difference between the direct perturbation and the GW treatment is small.\n\n\n\n\\begin{figure}[h]\n\\centering \n\\includegraphics[clip,width=0.45\\textwidth ]{fig15.eps} \n\\caption{(Color online)\nBand structure  of $[\\epsilon_{\\rm DFT}-V_{\\rm xc}+\\Delta\\Sigma_{\\rm H}+\\Delta\\Sigma_{\\rm L}^{\\rm nonlocal}]Z_{\\rm HL}Z_B$ of SrVO$_3$ in the GW treatment.\nFor comparison, the LDA band structure is also given\n(black dotted line).\nThe energy is measured from the Fermi level.}\n\\label{DFT+DeltaSigmaL_GW_ZHL_ZB}\n\\end{figure} \n\n\nAfter including the effect of the local self-energy by the nonperturbative \nCasula trick, we show the dispersion\ngiven by \n$[\\epsilon_{\\rm DFT}-V_{\\rm xc}+\\Delta\\Sigma_{\\rm H}+\\Delta\\Sigma_{\\rm L}^{\\rm nonlocal}]Z_{\\rm HL}Z_B$ for the GW-type treatment in Fig.~\\ref{DFT+DeltaSigmaL_GW_ZHL_ZB}: \nThe GW-like treatment gives a band dispersion\nclose to the LDA one. \nThe $Z_B$ factor corresponding to the local dynamical part of \n$\\Delta \\Sigma_{\\rm L}$ amounts to $Z_B=0.7$.\nThe nonperturbative treatment \nresults in an LDA-like band dispersion for the empty states,\nbut a slightly narrower bandwidth in the occupied part.\nThe low-energy effective Hamiltonian at this level of treatment has a \nfrequency independent effective interaction both with local \nand nonlocal interaction given by the Fourier transform of \n$W_{\\rm H}(q,\\omega=0)$, which contains both direct and exchange interactions. \n\n\n\\begin{figure}[h]\n\\centering \n\\includegraphics[clip,width=0.45\\textwidth ]{fig16.eps} \n\\caption{(Color online)\nBand structure  of $[\\epsilon_{\\rm DFT}+\\Sigma_{\\rm corr}^{\\rm GW}]Z_{\\rm corr}Z_B$ of SrVO$_3$.\nFor comparison, the LDA band structure is also given\n(black dotted line).\nThe energy is measured from the Fermi level.}\n\\label{DFT+Sigma_corr_GW_Z}\n\\end{figure} \n\n\n\nWe finally show the dispersion\ngiven by Eq.~(\\ref{Teff_final}), namely\n$[\\epsilon_{\\rm DFT}+\\Sigma_{\\rm corr}^{\\rm GW}]Z_{\\rm corr}Z_B$ in Fig.~\\ref{DFT+Sigma_corr_GW_Z}. \nThe result shows an LDA-like band dispersion for the occupied states,\nbut a slightly wider bandwidth in the empty part, resulting in a bandwidth 14\\% wider in total than the LDA bandwidth. \nThe effective interaction of the low-energy effective Hamiltonian \nat this final level has\nfrequency independent local and nonlocal interactions \ngiven by the Fourier transform of $W_r(q,\\omega=0)$. \n\n\n\n\\begin{figure}[h]\n\\centering \n\\includegraphics[clip,width=0.45\\textwidth ]{fig17.eps} \n\\caption{(Color online)\nBand structure  of $[\\epsilon_{\\rm DFT}-V_{\\rm xc}+\\Sigma-(G^{(0)}W_{\\rm GW}^{\\rm dyn})^{\\rm nonlocal}]Z'Z_B$ of SrVO$_3$\n(Z' is the renormalization factor of $\\Sigma-(G^{(0)}W_{\\rm GW}^{\\rm dyn})^{\\rm nonlocal}$).\nFor comparison, the LDA band structure is also given\n(black dotted line).\nThe energy is measured from the Fermi level.\n}\n\\label{GW+ZB}\n\\end{figure} \n\n\n\nThe resulting effective Hamiltonian in the L-space \nconsists of single-particle and two-particle (interaction) parts.\nThe effective interaction is the same as in previous estimates \nby the cRPA in the literature, while the single-particle dispersion \nis revised after removing the double counting and taking into \naccount the frequency dependence of the effective interaction. The \nfinal effective bandwidth for SrVO$_3$ is,\nafter partial cancellation, \nslightly (14\\%) larger than the LDA bandwidth.  \nThis is a reasonable result because the LDA takes into account all \ncorrelations though insufficiently, while the present scheme by the constrained self-energy excludes the \ncorrelation effects arising from the L-space.\nAlthough the bandwidth derived for the effective model is slightly \nlarger than that of the LDA,  it is clear that the correlation effects \nare stronger than in the LDA or standard GW when the {\\it ab initio} \nmodel is solved by an accurate solver. In fact, if the effective \ninteraction contained in the final effective model is treated by \nthe GW scheme, one obtains the dispersion illustrated in \nFig.~\\ref{GW+ZB}, \nwhich is given from the self energy of the whole GW calculation $\\Sigma$ \nby correcting the local dynamical part $W-W_{\\rm L}$ by $Z_B$. \nThis indicates that even an insufficient treatment of the local \nstatic interaction by the GW scheme gives a dispersion with the bandwidth ($\\sim 1.9$ eV) narrower \nthan the LDA ($\\sim 2.5$ eV, Fig.~\\ref{lda}) and GW ($\\sim 2.1$ eV, Fig.~\\ref{fullGW}) dispersions.  \nA slightly ($\\sim 14$\\%) wider dispersion than that of the LDA bands obtained for the \neffective Hamiltonian accompanied with frequency independent effective \nCoulomb interactions without the exchange part may account for the slight \noverestimate of correlation effects in the literature mentioned in \nthe introduction.\nOur scheme offers an optimized way for the  derivation of \n{\\it ab initio} models for the L-space after eliminating the H-space\nin  a systematic fashion.\n\n\nOur findings of the band widening are consistent with studies\nbased on the combined GW+DMFT method in the literature \n\\cite{TomczakPRB2014}: There, it was argued that within GW+DMFT\nthe best effective Hamiltonian that DMFT should be performed on,\nis a one-body Hamiltonian corrected by the nonlocal part of the\nGW self-energy. The corresponding spectral function \n(see e.g.\nFigure 5 of Ref.~\\onlinecite{TomczakPRB2014}) displays \na broadening similar to \nFig.~\\ref{DFT+Sigma_HL_GW}.\nMost interestingly, our present calculations confirm a pronounced\nasymmetry observed in Ref.~\\onlinecite{TomczakPRB2014}, namely a stronger widening\neffect in the unoccupied part of the spectrum than in the occupied\none.\n\n",
        "label": "ZH_SigmaH",
        "Descriptive_question1": "What is the bandwidth of the band structure shown in figure_8 for SrVO₃ after renormalization?",
        "Descriptive_question2": "What color is used for the LDA band structure in figure_8?",
        "Reasoning_question1": "Why does the bandwidth in figure_8 reduce slightly to 3.4 eV after applying the renormalization factor Z_H?",
        "Reasoning_question2": "How does the band structure in figure_8 compare to the LDA band structure in terms of the occupied and unoccupied parts of the spectrum?",
        "Descriptive_answer1": "3.4 eV",
        "Descriptive_answer2": "black dotted line",
        "Reasoning_answer1": "The renormalization factor Z_H=0.92, which is close to one, accounts for the linear frequency dependence of the constrained self-energy ΔΣ_H at low energies, representing correlations from the H-space. Applying this factor slightly reduces the bandwidth from 3.7 eV (after removing V_xc and adding ΔΣ_H(0)) to 3.4 eV, reflecting a moderate narrowing due to dynamical correlation effects captured by Z_H.",
        "Reasoning_answer2": "Compared to the LDA band structure, the band in figure_8 shows a widening compared to the original LDA bandwidth of 2.5 eV, with the occupied part bottom matching the LDA value but the unoccupied part notably widened. This asymmetry reflects that the static part of the Hamiltonian without V_xc and with ΔΣ_H widens the bands mainly in the unoccupied states; the LDA partially includes correlations within the L space that narrow the band, especially in the occupied region, whereas figure_8 includes the effects from the H-space leading to a broader unoccupied spectrum."
    },
    {
        "paper_id": "1511.03757.json",
        "image_id": "figure_9",
        "image_path": [
            "/home/yz979/scratch/chengye/scimolmo/test-images-tables/dataset_imagestables/1511.03757/images/fig9.eps"
        ],
        "caption": "(Color online)\n{\\bf minor modification}\nFrequency dependence of $-V_{\\rm xc}+\\Delta \\Sigma_{\\rm H}$ of SrVO$_3$.\n(a) and (b) are the real and imaginary parts of the local\ncomponent, respectively.\nThe black dotted line is $\\Sigma$ defined in Eq.~(\\ref{full-self-energy}) in the full GW approximation for comparison.\n(c) and (d) are the real and imaginary \nparts at several $k$ points.\n",
        "classify": "Chart",
        "section_info": "6 Results\n\\section{Results}\n\\label{results}\n\n\nThe ternary 3d$^1$ transition metal oxide SrVO$_3$ has become\none of the ``drosophila compounds'' of correlated systems.\nIt is a correlated metal\nthat has been characterized using various experimental \n\\cite{Onoda, inoue, maiti, maiti2, sekiyama, takizawa, yoshida2010}.\nand theoretical techniques, see e.g.\n\\cite{nekrasov, anisimov, pavarini-d1, liebsch, lechermann-d1, trimarchi, aichhorn09, karolak, lee, held-optics, tomczak-gwdmft, gatti-guzzo, sakuma14, TomczakPRB2014}.\nA review of most of the available experimental and theoretical \ndata has been given recently in Ref.~\\onlinecite{TomczakPRB2014}.\nSrVO$_3$ displays Fermi liquid behavior up to\nremarkably high temperatures of the order of $200$ K \\cite{Onoda, inoue},\nwith a moderate mass enhancement of the order \nof $2$ \\cite{yoshida2010, vandermarel}.\nDetailed spectroscopic investigations have made it an\nideal test compound for modern many-body calculations, and \nmore and more refined dynamical mean field-based studies \nare available.\nThe majority of studies so far have focused on the t$_{2g}$-manifold\nwhich forms the states close to the Fermi level, and those will\nalso be the focus in the present investigations. \nNote however that this restriction quite severely limits the\nrange of validity of the low-energy description. Indeed, as\nshown recently~\\cite{tomczak-gwdmft, TomczakPRB2014} at\nenergies of about 2 to 3 eV above the Fermi level\nthe spectral properties are largely determined by the e$_g$\nstates. This has in particular led to a reinterpretation of\nan inverse photoemission feature at about 2.5 eV that was \nfrequently interpreted as an upper Hubbard band of t$_{2g}$\ncharacter in the earlier literature.\nHere, we use the compound only as an illustration of \nthe principles of constructing effective low-energy models,\nwithout being concerned with a description of spectra beyond\nthe pure t$_{2g}$ part.\n\n\nThe LDA band structure of SrVO$_3$ is shown in Fig.~\\ref{lda}.\nOne clearly distinguishes the three-fold degenerate manifold\nof t$_{2g}$ bands close to the Fermi level (chosen as the\nzero of energy), followed in the unoccupied part of the spectrum\nby the two e$_g$ bands. At about -2.5 eV the filled O-2p derived\nbands are visible.\n\nWhen a standard GW calculation is performed, see Fig.~\\ref{fullGW},\nthe bandwidth of the t$_{2g}$-states is reduced from 2.5 eV to\n2.1 eV, while the overall shape of the dispersion remains similar\nto the LDA one.\nThis is in agreement with previous GW calculations in the literature\n\\cite{tomczak-gwdmft, TomczakPRB2014, gatti-guzzo, sakuma14}.\n\n\n\nThe calculation is based on the full-potential linear muffin-tin orbitals (FP-LMTO) implementation.\n8$\\times$8$\\times$8 \\bm{k}-mesh is employed in both the DFT/LDA and the GW calculations.\nThe calculational details are the same as in Ref.~\\onlinecite{Hirayama}.\n\n\nWe now turn to a discussion of the band structure corresponding to \nthe Hamiltonians to be used as input for subsequent many-body \ncalculations for this material.\nWe will proceed step by step to analyze the influence of the various\ncorrection terms, with respect to the DFT starting point\n(which will be overlaid to the respective band structures).\n\n\n\n\\begin{figure}[h]\n\\centering \n\\includegraphics[clip,width=0.45\\textwidth ]{fig2.eps} \n\\caption{(Color online)\nKohn-Sham band structure of SrVO$_3$ in the LDA.\nThe energy is measured from the Fermi level.\n}\n\\label{lda}\n\\end{figure} \n\n\n\\begin{figure}[h]\n\\centering \n\\includegraphics[clip,width=0.45\\textwidth ]{fig3.eps} \n\\caption{(Color online)\nBand structure  of SrVO$_3$ in the one-shot GW approximation.\nFor comparison, the LDA band structure is also given\n(black dotted line).\nThe energy is measured from the Fermi level.\n}\n\\label{fullGW}\n\\end{figure} \n\n\n\n\n\n\n\\begin{figure}[h]\n\\centering \n\\includegraphics[clip,width=0.45\\textwidth ]{fig4.eps} \n\\caption{(Color online)\nBand structure of SrVO$_3$ corresponding to the LDA Hamiltonian\nfrom which the LDA exchange-correlation potential has been subtracted.\nFor comparison, the LDA band structure is also given\n(black dotted line).\nThe energy is measured from the Fermi level.\n}\n\\label{lda-vxc}\n\\end{figure} \n\n\n\\begin{figure}[h]\n\\centering \n\\includegraphics[clip,width=0.45\\textwidth ]{fig5.eps} \n\\caption{(Color online)\nBand structure  of SrVO$_3$ in the Hartree approximation.\nFor comparison, the LDA band structure is also given\n(black dotted line).\nThe energy is measured from the Fermi level.\n}\n\\label{hartree}\n\\end{figure} \n\n\nFigure~\\ref{lda-vxc} \ndisplays the band structure corresponding to\nthe LDA Hamiltonian from which the \nLDA exchange-correlation\npotential has been subtracted,  \nthat is, the eigenvalues of $H_{\\rm LDA}-V_{\\rm xc}$ where\nthese operators are evaluated for the self-consistent LDA\ndensity.\nThe subtraction of $V_{\\rm xc}$ widens the band structure from \nthe LDA bandwidth of 2.5 eV \nto more than twice this value: the new band\nwidth is 5.4 eV.\nThis indicates that the exchange-correlation potential $V_{\\rm xc}$ \nof the LDA is responsible for a substantial amount of \nthe band-narrowing effect\narising from electronic correlations, stressing the need \nto subtract the double counting in a consistent manner. \nFigure~\\ref{lda-vxc} is regarded as the starting point for the\nfollowing considerations.\n\nFor simple materials (see e.g. the calculations on Li in Ref.~\\onlinecite{Lithium})\nit has been noted in the literature that the Hartree band structure\nis close to the DFT one. This raises the question of the origin of the\nsubstantial band widening in the present case. Indeed, the present\ncalculation shown in Fig.~\\ref{lda-vxc} differs from a Hartree calculation\nonly \nby the fact that Fig.~\\ref{lda-vxc} is evaluated for the converged LDA density.\nWe have performed a test calculation where we plot the Hartree band\nstructure calculated for the converged Hartree density. The result\nis shown in Fig.~\\ref{hartree}. As seen from this plot, while the\nband is not fully as wide as in Fig.~\\ref{lda-vxc}, a substantial \nwidening is already present at this stage.\n\n\n\n\\begin{figure}[h]\n\\centering \n\\includegraphics[clip,width=0.45\\textwidth ]{fig6.eps} \n\\caption{(Color online)\nBand structure  of $H_{\\rm LDA}-V_{\\rm xc}+\\Delta \\Sigma_{\\rm H}(\\omega=0)$ of SrVO$_3$.\nFor comparison, the LDA band structure is also given\n(black dotted line).\nThe energy is measured from the Fermi level.\n}\n\\label{lda-vxc-sigH(0)}\n\\end{figure} \n\n\n\nStarting from the Hamiltonian without V$_{xc}$ (see the \ndispersion in Fig.~\\ref{lda-vxc}), we first take into account \nthe static part of the corrective self-energy $\\Delta \\Sigma_{\\rm H}$:\nIn Fig.~\\ref{lda-vxc-sigH(0)}, we plot the dispersion corresponding to\n$H_{\\rm LDA}-V_{\\rm xc}+\\Delta \\Sigma_{\\rm H}(\\omega=0)$. \nCompared to the dispersion of Fig.~\\ref{lda-vxc}, the overall band structure \nis narrowed to 3.7 eV. This value is, however, still considerably larger\nthan the LDA bandwidth (Fig.~\\ref{lda}). Although\n the electronic correlations coupling the H- and \nthe L space are included in Fig.~\\ref{lda-vxc-sigH(0)}, and narrow the band with respect to the case\nwhere V$_{xc}$ is taken out, correlations within\nthe L space are not included. \nThe LDA, on the other hand, at least partially\nincludes correlations within the L space, \nand these are very effective in narrowing the band.\nInterestingly, the bottom of the occupied band is quite\nexactly at the LDA value, and the remaining widening is\npurely in the unoccupied part.\n\n\n\\begin{figure*}[htb]\n\\begin{center} \n\\includegraphics[width=0.9\\textwidth ]{fig7.eps} \n\\end{center} \n\\caption{(Color online)\nFrequency dependence of $-V_{xc}+\\Delta \\Sigma_{\\rm H}$ for SrVO$_3$.\n(a) and (b) are the real and imaginary parts at the $\\Gamma$ point, \nrespectively,\nand (c) and (d) ((e) and (f)) are those at $X$ ($R$).\nFor comparison, the full GW self-energies are also given\n(black dotted line).\n}\n\\label{sigH}\n\\end{figure*} \n\n\n\n\\begin{figure}[h]\n\\centering \n\\includegraphics[clip,width=0.45\\textwidth ]{fig8.eps} \n\\caption{(Color online)\nBand structure  of $Z_{\\rm H}[H_{\\rm LDA}-V_{\\rm xc}+\\Delta\\Sigma_{\\rm H}(\\omega=0)]$ of SrVO$_3$.\nFor comparison, the LDA band structure is also given\n(black dotted line).\nThe energy is measured from the Fermi level.\n}\n\\label{ZH_SigmaH}\n\\end{figure} \n\n\nTo go further and in particular to analyze the dynamic\nbehavior of $\\Delta \\Sigma_{\\rm H}$ \nwe plot in Fig.~\\ref{sigH} \nthe self-energy corrections $\\Delta \\Sigma_{\\rm H}$ (with $\\Sigma$ (see Eq.~(\\ref{full-self-energy}))\nfor comparison) for the real and imaginary parts\nat several representative choices of momenta.\nThe frequency dependence is smooth around the Fermi level.\nIn particular, in contrast to the full self-energy,\nthere are no poles in $\\Delta \\Sigma_{\\rm H}$ \nin the low-energy region, thanks to the\nexclusion of fluctuations within the L-space.\nThe frequency dependence of the real part indicates that the \nlinearization \n$\\displaystyle{\\Delta \\Sigma_{\\rm H}(\\omega) \\sim \\Delta \\Sigma_{\\rm H}(\\omega=0)+[d\\Delta \\Sigma_{\\rm H}(\\omega)/d\\omega]_{\\omega=0}\\omega }$\noffers a reasonably good approximation. \nIn fact, the behavior of the constrained self-energy\nis much closer to linearity than that of \nthe full GW self-energy $\\Sigma$.\nThis is easily understood by the fact that the low-energy excitations are excluded in the present constrained self-energy $\\Delta\\Sigma_{\\rm H}$ analogously to an insulator, thus eliminating the strong frequency dependence of the constrained self-energy near the Fermi level. This is one of the consequences of the \ncontrollability and an advantage of the present MACE scheme, as \nmentioned in the introduction.\nIn the following discussion, we employ this linearized approximation.\nThen the renormalization factor $Z_{\\rm H}$ defined by Eq.~(\\ref{ZH})\nis interpreted as that from the contribution of the H-space. \nAfter \nthis renormalization factor is taken into account,\nthe effective Hamiltonian is given by $Z_{\\rm H}[H_{\\rm LDA}-V_{\\rm xc}+\\Delta\\Sigma_{\\rm H}(\\omega=0)]$ (see Fig.~\\ref{ZH_SigmaH}). \nInterestingly the renormalization factor $Z_{\\rm H}$=0.92 stays\nclose to one, so that\nthe bandwidth is only slightly reduced (to\n$3.4$ eV instead of the $3.7$ eV above). \nWe also note that $Z_{\\rm H}$ has weak momentum dependence\nas we reveal in the following.\n\n\n\n\\begin{figure*}[htb]\n\\begin{center} \n\\includegraphics[width=0.9\\textwidth ]{fig9.eps} \n\\end{center} \n\\caption{(Color online)\n{\\bf minor modification}\nFrequency dependence of $-V_{\\rm xc}+\\Delta \\Sigma_{\\rm H}$ of SrVO$_3$.\n(a) and (b) are the real and imaginary parts of the local\ncomponent, respectively.\nThe black dotted line is $\\Sigma$ defined in Eq.~(\\ref{full-self-energy}) in the full GW approximation for comparison.\n(c) and (d) are the real and imaginary \nparts at several $k$ points.\n}\n\\label{sigH_nonlocal}\n\\end{figure*} \n\n\n\nFigure~\\ref{sigH_nonlocal} \nshows the local and nonlocal parts of $\\Delta \\Sigma_{\\rm H}$.\nThe frequency \ndependence\nof the local part (Figs.~\\ref{sigH_nonlocal} (a)(b)) \nis quite similar \nto the  frequency-dependent self-energies at various momenta\nin Fig.~\\ref{sigH}.\nOne immediately reads off an interesting property, which is\nakin to what has been found for the full GW self-energy in\nRef.~\\onlinecite{Tomczak-PRL2012}, namely that the nonlocal part of \n$\\Delta \\Sigma_{\\rm H}$ shows only weak frequency dependence \nas shown in Figs.~\\ref{sigH_nonlocal} (c)(d). \nThis explains why $Z_{\\rm H}$ \nis only weakly momentum-dependent.\nOn the other hand, $\\Delta \\Sigma_{\\rm H}^{\\rm nonlocal} (0)$ at $R$ is $\\sim 1$ eV larger than \nat $\\Gamma$, which causes the band widening effect.\n $\\Delta \\Sigma_{\\rm H}$\ncan thus to a good approximation be decomposed into a\nfrequency-dependent local part and a static nonlocal one:\n$\\Delta \\Sigma_{\\rm H}(k,\\omega) = \\Delta \\Sigma^{\\rm local}_{\\rm H}(\\omega) + \\Delta \\Sigma^{\\rm nonlocal}_{\\rm H}(k)$,\nwith $ \\Delta \\Sigma_{\\rm H}^{\\rm local}(\\omega) = \\sum_q  \\Delta \\Sigma_{\\rm H}(q,\\omega)$\n\nSince in the simple case of SrVO$_3$ where the t$_{2g}$ states are\ndegenerate, static local operators are scalar and are compensated \nby a chemical potential shift such that the correct particle\nnumber is obtained. The above band structure corresponding to\n$H_{\\rm LDA} - V_{\\rm xc} + \\Delta \\Sigma_{\\rm H}(k, \\omega=0)$ \nis thus identical to that of\n$H_{\\rm LDA} - V_{\\rm xc} + \\Delta \\Sigma^{\\rm nonlocal}_{\\rm H}(k)$.\nThe local dynamical part of $\\Delta \\Sigma_{\\rm H}$ then results in a \nnarrowing of this band structure by a factor $Z_{\\rm H} = 0.92$.\n\n\n\n\\begin{figure}[h]\n\\centering \n\\includegraphics[clip,width=0.45\\textwidth ]{fig10.eps} \n\\caption{(Color online)\n(a) Frequency dependence of the on-site effective interaction.\nPanel (b) is a low-energy zoom of panel (a). \n}\n\\label{W}\n\\end{figure} \n\n\nBefore turning to a discussion of the low-energy correction for the GW treatment\n$\\Delta \\Sigma_{\\rm L} =G^{(0)}(W-W_{\\rm L})$, we analyze the effective\ninteraction $W_{\\rm L}$ shown in Fig.~\\ref{W} \nin comparison to $W$ and $W_{\\rm H}$.\nThe fully screened Coulomb interaction $W$ displays the familiar\nshape of an interaction that is strongly screened at low energies\n(with a value of $0.88$ eV at $\\omega=0$), while retrieving the\nvalue of the bare Coulomb interaction $v$ ($\\sim 16.0$ eV) \nat high energy.\nThe crossover from the bare to the  \nscreened values takes place at the plasma energy of \nabout $15$ eV. This behavior has been discussed before\n\\cite{Review_SB2014}; we note in particular that the plasma \nfrequency is known from electron energy loss spectroscopy \nmeasurements of the related SrTiO$_3$ compound \\cite{vandermarel,kohiki}.\nThe partially screened interaction\n$W_{\\rm H}$ constructed within cRPA\nconverges to $W$ at high energies, but displays\nweaker screening effects at low energies (with a value of $3.5$ eV at $\\omega=0$),\nsince intra-t$_{2g}$ screening channels are excluded.\nAs was already anticipated in Sec.~\\ref{Outline},  $W_{\\rm L}$ can be interpreted as the screened interaction of \na low-energy model where a static interaction of value $W_{\\rm H}(\\omega=0)$\nhas been imposed as the bare interaction. Since now {\\it only} intra-t$_{2g}$\nscreening channels are active, screening takes place only at low\nenergies, where the scale is given by the t$_{2g}$ bandwidth.\nAlso plotted is the difference $W_{\\rm GW}^{\\rm dyn}=W-W_{d}$: except at low energies\nwhere the t$_{2g}$ screening channels come into play, its frequency\ndependence is essentially given by that of $W$, while the\nhigh-energy limit is the bare Coulomb interaction $v$ reduced\nby $W_{\\rm H}(0)$.\n\n\n\\begin{figure*}[htb]\n\\begin{center} \n\\includegraphics[width=0.9\\textwidth ]{fig11.eps} \n\\end{center} \n\\caption{(Color online)\nFrequency dependence of $-V_{\\rm xc}+ G^{(1)}W_{\\rm H}^{\\rm dyn}$ of SrVO$_3$.\n(a) and (b) are the real and imaginary local parts, respectively.\n(c) and (d) are the real and imaginary \nparts at several $k$ points.\n}\n\\label{DeltaSigma_L_Pert}\n\\end{figure*} \n\n\n\n\\begin{figure*}[htb]\n\\begin{center} \n\\includegraphics[width=0.9\\textwidth ]{fig12.eps} \n\\end{center} \n\\caption{(Color online)\nFrequency dependence of $-V_{\\rm xc}+ G^{(0)}W_{\\rm GW}^{\\rm dyn}$ of SrVO$_3$.\n(a) and (b) are the real and imaginary local parts, respectively.\n(c) and (d) are the real and imaginary \nparts at several $k$ points.\n}\n\\label{DeltaSigma_L_GW}\n\\end{figure*} \n\n\n\nThe frequency dependence of the real and imaginary parts of the low-energy self-energy $\\Delta \\Sigma_{\\rm L} = G^{(1)}W_{\\rm H}^{\\rm dyn}$ for the direct perturbative treatment is illustrated in Fig.~\\ref{DeltaSigma_L_Pert}. \n$\\Delta \\Sigma_{\\rm L} = G^{(0)}W_{\\rm GW}^{\\rm dyn}$ used\nfor the GW treatment is shown in Fig.~\\ref{DeltaSigma_L_GW}\nfor several choices of momenta.\n\\\\\nThe results for the perturbative and the GW treatment are nearly \nidentical.\nThe renormalization factor of $\\Delta \\Sigma_{\\rm L}^{\\rm nonlocal}$ \n(and $\\Delta \\Sigma_{\\rm L}$) is $\\sim 0.77$.\nThe zero-frequency shift\n${\\rm Re}\\Delta \\Sigma_{\\rm L}^{\\rm nonlocal} (0)$ at $R$ is $\\sim 3$ eV \nlarger than \nat $\\Gamma$.\nAgain, one sees how $\\Sigma_{\\rm L}$ separates into local dynamical\nand nonlocal static parts: \n$\\Delta \\Sigma_{\\rm L} = \\Delta \\Sigma^{\\rm nonlocal}_{\\rm L}(k) + \n\\Delta \\Sigma^{\\rm local}_{\\rm L}(\\omega)$.\nThis is expected since such a separation has been found within\nthe full GW calculation \\cite{TomczakPRB2014}, and is even\nmore plausible for a GW treatment within the t$_{2g}$ subspace.\n\n\n\n\\begin{figure}[h]\n\\centering \n\\includegraphics[clip,width=0.45\\textwidth ]{fig13.eps} \n\\caption{(Color online)\nBand structure  of $[\\epsilon_{\\rm DFT}-V_{\\rm xc}+\\Delta\\Sigma_{\\rm H}+\\Delta \\Sigma_{\\rm L}^{\\rm nonlocal}]Z_{\\rm HL}$ of SrVO$_3$ in the direct perturbative treatment.\nFor comparison, the LDA band structure is also given\n(black dotted line).\nThe energy is measured from the Fermi level.\n}\n\\label{DFT+Sigma_HL_Pert}\n\\end{figure} \n\n\n\n\\begin{figure}[h]\n\\centering \n\\includegraphics[clip,width=0.45\\textwidth ]{fig14.eps} \n\\caption{(Color online)\nBand structure  of $[\\epsilon_{\\rm DFT}-V_{\\rm xc}+\\Delta\\Sigma_{\\rm H}+\\Delta \\Sigma_{\\rm L}^{\\rm nonlocal}]Z_{\\rm HL}$ of SrVO$_3$ in the GW treatment.\nFor comparison, the LDA band structure is also given\n(black dotted line).\nThe energy is measured from the Fermi level.\n}\n\\label{DFT+Sigma_HL_GW}\n\\end{figure} \n\n\n\nThe corresponding band structures \nare given by \n $[\\epsilon_{\\rm DFT}+\\Delta\\Sigma_{\\rm H}+\\Delta \\Sigma_{\\rm L}^{\\rm nonlocal}]Z_{\\rm HL}$ in Fig.~\\ref{DFT+Sigma_HL_Pert} \nfor the perturbative treatment (see Eq.~(\\ref{DeltaSigmaLnonlocalPert})), and in Fig.~\\ref{DFT+Sigma_HL_GW} \nfor the GW treatment,\nwhere $\\Delta \\Sigma_{\\rm L}^{\\rm nonlocal}$ is defined in Eq.~(\\ref{DeltaSigLnonlocalGW}) and $Z_{\\rm HL}$ is around 0.92.\n$Z_{\\rm HL}$ is practically the same as $Z_{\\rm H}$, meaning that the nonlocal dynamical correction \nin the H-space is small, which justifies the GW perturbative treatment for the nonlocal part.\nOne finds that \nthe nonlocal part $\\Delta \\Sigma_{\\rm L}=G^{(0)}_{ll}(W-W_{\\rm L})$ narrows\nthe empty states but widens the occupied ones, resulting\nin an overall bandwidth of 3.2 eV.\nThe difference between the direct perturbation and the GW treatment is small.\n\n\n\n\\begin{figure}[h]\n\\centering \n\\includegraphics[clip,width=0.45\\textwidth ]{fig15.eps} \n\\caption{(Color online)\nBand structure  of $[\\epsilon_{\\rm DFT}-V_{\\rm xc}+\\Delta\\Sigma_{\\rm H}+\\Delta\\Sigma_{\\rm L}^{\\rm nonlocal}]Z_{\\rm HL}Z_B$ of SrVO$_3$ in the GW treatment.\nFor comparison, the LDA band structure is also given\n(black dotted line).\nThe energy is measured from the Fermi level.}\n\\label{DFT+DeltaSigmaL_GW_ZHL_ZB}\n\\end{figure} \n\n\nAfter including the effect of the local self-energy by the nonperturbative \nCasula trick, we show the dispersion\ngiven by \n$[\\epsilon_{\\rm DFT}-V_{\\rm xc}+\\Delta\\Sigma_{\\rm H}+\\Delta\\Sigma_{\\rm L}^{\\rm nonlocal}]Z_{\\rm HL}Z_B$ for the GW-type treatment in Fig.~\\ref{DFT+DeltaSigmaL_GW_ZHL_ZB}: \nThe GW-like treatment gives a band dispersion\nclose to the LDA one. \nThe $Z_B$ factor corresponding to the local dynamical part of \n$\\Delta \\Sigma_{\\rm L}$ amounts to $Z_B=0.7$.\nThe nonperturbative treatment \nresults in an LDA-like band dispersion for the empty states,\nbut a slightly narrower bandwidth in the occupied part.\nThe low-energy effective Hamiltonian at this level of treatment has a \nfrequency independent effective interaction both with local \nand nonlocal interaction given by the Fourier transform of \n$W_{\\rm H}(q,\\omega=0)$, which contains both direct and exchange interactions. \n\n\n\\begin{figure}[h]\n\\centering \n\\includegraphics[clip,width=0.45\\textwidth ]{fig16.eps} \n\\caption{(Color online)\nBand structure  of $[\\epsilon_{\\rm DFT}+\\Sigma_{\\rm corr}^{\\rm GW}]Z_{\\rm corr}Z_B$ of SrVO$_3$.\nFor comparison, the LDA band structure is also given\n(black dotted line).\nThe energy is measured from the Fermi level.}\n\\label{DFT+Sigma_corr_GW_Z}\n\\end{figure} \n\n\n\nWe finally show the dispersion\ngiven by Eq.~(\\ref{Teff_final}), namely\n$[\\epsilon_{\\rm DFT}+\\Sigma_{\\rm corr}^{\\rm GW}]Z_{\\rm corr}Z_B$ in Fig.~\\ref{DFT+Sigma_corr_GW_Z}. \nThe result shows an LDA-like band dispersion for the occupied states,\nbut a slightly wider bandwidth in the empty part, resulting in a bandwidth 14\\% wider in total than the LDA bandwidth. \nThe effective interaction of the low-energy effective Hamiltonian \nat this final level has\nfrequency independent local and nonlocal interactions \ngiven by the Fourier transform of $W_r(q,\\omega=0)$. \n\n\n\n\\begin{figure}[h]\n\\centering \n\\includegraphics[clip,width=0.45\\textwidth ]{fig17.eps} \n\\caption{(Color online)\nBand structure  of $[\\epsilon_{\\rm DFT}-V_{\\rm xc}+\\Sigma-(G^{(0)}W_{\\rm GW}^{\\rm dyn})^{\\rm nonlocal}]Z'Z_B$ of SrVO$_3$\n(Z' is the renormalization factor of $\\Sigma-(G^{(0)}W_{\\rm GW}^{\\rm dyn})^{\\rm nonlocal}$).\nFor comparison, the LDA band structure is also given\n(black dotted line).\nThe energy is measured from the Fermi level.\n}\n\\label{GW+ZB}\n\\end{figure} \n\n\n\nThe resulting effective Hamiltonian in the L-space \nconsists of single-particle and two-particle (interaction) parts.\nThe effective interaction is the same as in previous estimates \nby the cRPA in the literature, while the single-particle dispersion \nis revised after removing the double counting and taking into \naccount the frequency dependence of the effective interaction. The \nfinal effective bandwidth for SrVO$_3$ is,\nafter partial cancellation, \nslightly (14\\%) larger than the LDA bandwidth.  \nThis is a reasonable result because the LDA takes into account all \ncorrelations though insufficiently, while the present scheme by the constrained self-energy excludes the \ncorrelation effects arising from the L-space.\nAlthough the bandwidth derived for the effective model is slightly \nlarger than that of the LDA,  it is clear that the correlation effects \nare stronger than in the LDA or standard GW when the {\\it ab initio} \nmodel is solved by an accurate solver. In fact, if the effective \ninteraction contained in the final effective model is treated by \nthe GW scheme, one obtains the dispersion illustrated in \nFig.~\\ref{GW+ZB}, \nwhich is given from the self energy of the whole GW calculation $\\Sigma$ \nby correcting the local dynamical part $W-W_{\\rm L}$ by $Z_B$. \nThis indicates that even an insufficient treatment of the local \nstatic interaction by the GW scheme gives a dispersion with the bandwidth ($\\sim 1.9$ eV) narrower \nthan the LDA ($\\sim 2.5$ eV, Fig.~\\ref{lda}) and GW ($\\sim 2.1$ eV, Fig.~\\ref{fullGW}) dispersions.  \nA slightly ($\\sim 14$\\%) wider dispersion than that of the LDA bands obtained for the \neffective Hamiltonian accompanied with frequency independent effective \nCoulomb interactions without the exchange part may account for the slight \noverestimate of correlation effects in the literature mentioned in \nthe introduction.\nOur scheme offers an optimized way for the  derivation of \n{\\it ab initio} models for the L-space after eliminating the H-space\nin  a systematic fashion.\n\n\nOur findings of the band widening are consistent with studies\nbased on the combined GW+DMFT method in the literature \n\\cite{TomczakPRB2014}: There, it was argued that within GW+DMFT\nthe best effective Hamiltonian that DMFT should be performed on,\nis a one-body Hamiltonian corrected by the nonlocal part of the\nGW self-energy. The corresponding spectral function \n(see e.g.\nFigure 5 of Ref.~\\onlinecite{TomczakPRB2014}) displays \na broadening similar to \nFig.~\\ref{DFT+Sigma_HL_GW}.\nMost interestingly, our present calculations confirm a pronounced\nasymmetry observed in Ref.~\\onlinecite{TomczakPRB2014}, namely a stronger widening\neffect in the unoccupied part of the spectrum than in the occupied\none.\n\n",
        "label": "sigH_nonlocal",
        "Descriptive_question1": "What do panels (a) and (b) in figure_9 represent?",
        "Descriptive_question2": "What is represented by the black dotted line in figure_9?",
        "Reasoning_question1": "How does the frequency dependence of the local component in figure_9 compare to the full GW approximation?",
        "Reasoning_question2": "What can be inferred about the nonlocal part of the self-energy from figure_9, and how does it affect the band widening effect at different k points?",
        "Descriptive_answer1": "real, imaginary",
        "Descriptive_answer2": "full GW self-energy",
        "Reasoning_answer1": "In figure_9, the local component shown in panels (a) and (b) displays the real and imaginary parts of the self-energy that correspond to the constrained self-energy \\(-V_{xc} + \\Delta \\Sigma_H\\). When compared to the black dotted line, which represents the full GW self-energy \\(\\Sigma\\), the local component shows a smoother frequency dependence with fewer features, since low-energy fluctuations are excluded, resulting in a more linear and less oscillatory behavior around the Fermi level. This indicates that the local component of the constrained self-energy behaves more like an insulator without poles at low energies, unlike the full GW self-energy which has poles and more complex frequency dependence.",
        "Reasoning_answer2": "Figure_9 parts (c) and (d) represent the real and imaginary parts of the self-energy at different momentum points \\(k\\). The nonlocal part of the self-energy shows a dependence on momentum, with the value of \\(\\Delta \\Sigma_H^{nonlocal}(0)\\) at point R being approximately 1 eV larger than at point \\(\\Gamma\\). This difference implies that the nonlocal self-energy causes band widening, especially at higher k points like R compared to \\(\\Gamma\\). The frequency dependence of this nonlocal part is weak, meaning it acts as a static correction that shifts the bands differently at various k points, thereby causing widening of the band structure due to momentum-dependent energy shifts."
    },
    {
        "paper_id": "1511.03757.json",
        "image_id": "figure_10",
        "image_path": [
            "/home/yz979/scratch/chengye/scimolmo/test-images-tables/dataset_imagestables/1511.03757/images/fig10.eps"
        ],
        "caption": "(Color online)\n(a) Frequency dependence of the on-site effective interaction.\nPanel (b) is a low-energy zoom of panel (a). \n",
        "classify": "Chart",
        "section_info": "2 Outline of derivation of low-energy effective Hamiltonian\n\\section{Outline of derivation of low-energy effective Hamiltonian}\n\\label{Outline}\n\n\\subsection{General strategy}\n\\label{general}\nWe decompose the full Hilbert space into low-energy (the L space)\nand high-energy subspaces (the H part) thanks to the hierarchical\nstructure of strongly correlated electron systems as \ndescribed  in Sec.~\\ref{Introduction}.\nOur construction for the decomposition and bridging \nbetween the two parts\nwill be based on the GW scheme, since this allows\nfor a well-defined way to avoid double-counting. Indeed, at the\nDFT level, after identifying the H- and L-spaces,\nthe partial trace summation and the elimination of the H-space \ncan be performed by means of the cRPA~\\cite{crpa,Review2010}, as \ndescribed below. However, when the low-energy part is \nsolved by a refined many-body solver, some part of the interactions are\ncounted twice since the initial DFT calculation\nalready contains the correlation effects for the L part.\nIndeed, the DFT considers the exchange correlation contribution \nwithout distinguishing the L- and H-spaces and it is impossible\nto disentangle the two spaces at this level.\nThe GW scheme, instead, allows for the subtraction of the double counting\nby calculating a constrained self-energy as constructed in\nRef.~\\onlinecite{Hirayama}.\nThis constrained self-energy incorporates the interactions in the\nform of a self-energy \nfrom which the contribution of the L part \nhas been excluded.\n\nAfter eliminating \nthe H-space, the L-space is expressed \nby single-, two-, three-particles and even higher terms.  However, the effective many-body \ninteraction higher than the two-particle channel is expected to be small \nif the target L-space is isolated from the H-space. This is \ntrue in typical strongly correlated systems, and motivates a perturbative\ntreatment of the H-L-coupling. \nIn this article, \nwe ignore multi-particle effective interactions of higher order than the two-body terms. \n\nThe single-particle (kinetic energy) terms are modified (renormalized) \nby the constrained self energy. \nThe two-particle (effective Coulomb interaction) terms are represented \nas the partially screened interaction obtained from the cRPA~\\cite{crpa}. \nIn general, the self-energy and the screened interaction \nare frequency-dependent, thus not allowing for a representation \nin a Hamiltonian form.\n\nAs mentioned above, in this paper, we focus on methods that derive\nthe low-energy effective models described in the form of Hamiltonians \n\\begin{eqnarray}\nH_{\\rm eff}=\\sum_q T_{\\rm eff}(q)c^{\\dagger}_qc_q+\\sum_{q,k,p} W_{\\rm H}(q)c^{\\dagger}_k c_{k+q}c^{\\dagger}_p c_{p-q},\n\\label{zeroth-Heff}\n\\end{eqnarray}\nwhere \nthe renormalized single particle dispersion $T_{\\rm eff}(q)$ after incorporating the self-energy effect and the effective\ninteraction $W_{\\rm H}(q)$ screened by the H part constitute the model for the \nelectrons in the L part represented by the creation (annihilation) operators for the\nelectron, $c_k^{\\dagger} (c_k)$ at momentum $k$. Here, for simplicity,\nspin and orbital indices are omitted.\n\nOur task at this stage is thus to map the model with frequency dependent\nsingle- and two-particle terms onto a frequency-independent Hamiltonian \nin a controlled way. For this mapping, we propose a scheme that combines \nthe merits of the works by Hirayama {\\it et al.}~\\cite{Hirayama} and \nCasula {\\it et al.}~\\cite{casula-ZB}: In a step-by-step procedure,\nwe include the influence of the H-space into the L-space and eliminate\nthe frequency dependence by taking into account its effect on the\nHamiltonian in the form of an effective renormalization of the parameters. \n\n\nWe remark that, in practice, \nthe derived effective low-energy Hamiltonian satisfies the following principle:\nIf one solved the effective low-energy Hamiltonian within the GW-type perturbative \ntreatment instead of the accurate low-energy solver, that would yield the same result \nas the solution obtained by the same perturbative scheme starting from the full space including the H and L parts. \nThis is called the ``chain rule\", which justifies the effective Hamiltonian as that for the L part.\n\n\nRenormalized single-particle Hamiltonian\nOur starting point for the single-particle part is the DFT band dispersion\ndenoted by $\\epsilon_{\\rm DFT}(q)$ and the corresponding \nKohn-Sham Hamiltonian $H^{(0)}=\\sum_q \\epsilon_{\\rm DFT}(q)c^{\\dagger}_qc_q$.\nHere, $c^{\\dagger}_q (c_q)$, is the creation (annihilation) operator of an \nelectron with wavevector $q$. We have suppressed the spin and band \nindices for simplicity.\nThen, the single particle Green's function \n$G_{\\rm DFT}$ reads\n\\begin{eqnarray}\nG_{\\rm DFT}(q,\\omega)=1/[\\omega-\\epsilon_{\\rm DFT}].\n\\end{eqnarray}\n\n\nOn the DFT level, electronic correlations are taken into\naccount in the form of an effective exchange correlation potential \n$V_{\\rm xc}(q)$. As discussed above, treating the electron correlation \neffects in the L-space explicitly within the low-energy solver would\nlead to a double counting of electronic correlation in the L-space.\nTo avoid the double counting, $V_{\\rm xc}(q)$ is subtracted and replaced\nby a corrective self-energy $\\Delta \\Sigma (q,\\omega)$.\nBy incorporating $\\Delta \\Sigma$, the effective single-particle part reads \n\\begin{eqnarray}\nT_{\\rm eff}(q,\\omega)=\\epsilon_{\\rm DFT}-V_{\\rm xc}(q)+\\Delta \\Sigma (q,\\omega).\n\\label{Teff(q,w)}\n\\end{eqnarray}\n\n\\subsection{H-space contribution to self-energy: constrained self-energy}\n$\\Delta \\Sigma$ comes from two contributions; $\\Delta\\Sigma=\\Delta\\Sigma_{\\rm H}+\\Delta\\Sigma_{\\rm L}$. \n$\\Delta\\Sigma_{\\rm H}$ is the contribution to the self-energy from the H space, while\n$\\Delta\\Sigma_{\\rm L}$ is from the frequency dependent part of the effective interaction\nincorporated into the self-energy, which is the constrained self-energy effect within the L space obtained by excluding the self-energy arising from the static effective interaction $W_{\\rm H}(q)$. \nHere, we sketch the idea for $\\Delta\\Sigma_{\\rm H}$ and discuss $\\Delta\\Sigma_{\\rm L}$ in the next subsection.\nA specific form for the correction $\\Delta\\Sigma_{\\rm H}$, dubbed ``constrained self-energy'',\nwas already derived in Ref.~\\onlinecite{Hirayama}, based on a restricted GW calculation.\nThe basic prescription is to add only the self-energy arising from the contribution \nof the H-space, by excluding the part stemming purely from the L-space. \nThe reason why one should exclude the self-energy stemming from the \nL-space is that this part is more accurately calculated within the \nlow-energy solver afterwards. \n\n\n\n\n\\subsection{Renormalization to self-energy from frequency-dependent partially-screened interaction }\nThe cRPA~\\cite{crpa} was \nproposed as a means to calculate the effective local Coulomb interactions\nto be used in the L-space from a systematic first-principles procedure.\nIt can be understood as a way of tracing out the H-space for deriving the effective interaction, while\nkeeping track of the resulting renormalization of the L-space\ndegrees of freedom.\nThe tracing out of the H-space by the standard cRPA results in an effective interaction\nfor the two-particle part of the model in the L-space in the form\n\\begin{eqnarray}\nW_{\\rm H} (q,\\omega) = \\frac{v(q)}{1- P_{\\rm H}(q,\\omega)v(q)},\n\\label{Wr_crpa}\n\\end{eqnarray}\nwhere the wave-number ($q$) dependent bare Coulomb interaction $v$ \nis partially screened by the partial polarization $P_{\\rm H}$. \nHere, $P_{\\rm H}$ is defined in terms of the total polarization $P$ by excluding \nthe intra-L-space polarization $P_{\\rm L}$: $P_{\\rm H}\\equiv P-P_{\\rm L}$.\n$P_{\\rm L}$ involves only screening processes within the L-space.\n\nHere, $W_{\\rm H}$ is frequency dependent as\nschematically illustrated in Fig.~\\ref{Cartoon_W}.\nHowever, most many-body calculations in the literature that use the effective \ninteractions from the cRPA method or similar schemes neglect this\nfrequency-dependence (exceptions are \nRefs.~\\onlinecite{werner12},\n\\onlinecite{casula-ZB},\n\\onlinecite{vanroeke2014}, \n\\onlinecite{Hirayama},\n\\onlinecite{casula-prb},\n\\onlinecite{werner-LCO}, \n\\onlinecite{cafe2as2},\n\\onlinecite{sakuma2013}, \n\\onlinecite{sakuma14},\n\\onlinecite{tomczak-gwdmft},\n\\onlinecite{TomczakPRB2014}),\nand use only the zero-frequency value of the\ninteraction $W_{\\rm H}(q,\\omega=0)$ for the construction of the low-energy effective Hamiltonian~\\cite{crpa,Hirayama}.\n\n\nWe note that this static limit $W_{\\rm H} (q,\\omega=0) $ obtained by using Eq.~(\\ref{Wr_crpa}) in fact\nsatisfies the above mentioned chain rule: \nThe whole dynamical interaction emerging when \none solves the whole H- and L-space degrees of freedom by RPA is the\nusual fully screened interaction \n$W(q,\\omega)$ given by\n\\begin{eqnarray}\nW (q,\\omega) = \\frac{v(q)}{1- P(q,\\omega)v(q)},\n\\label{full_rpa}\n\\end{eqnarray}\nas is depicted in Fig.~\\ref{Cartoon_W}.\n\nOn the other hand, \nif we calculate the screening by the RPA within the L-space by regarding as if $W_{\\rm H} (q,\\omega=0)$ \nwould be the bare interaction, \nthis leads to a screened interaction\n\\begin{eqnarray}\nW_{\\rm L} (q,\\omega) = \\frac{W_{\\rm H}(q,\\omega=0)}{1- P_{\\rm L}(q,\\omega)W_{\\rm H}(q,\\omega=0)},\n\\label{W_L}\n\\end{eqnarray}\nwhich is depicted schematically in Fig.~\\ref{Cartoon_W}.\nHere, $P_{\\rm L}$ is the RPA polarization in the low-energy\nsubspace.\nNote that $W_{\\rm L} (q,\\omega\\rightarrow \\infty) = W_{\\rm H}(q,\\omega=0)$.\nThen the chain rule\n$W_{\\rm L} (q,\\omega) =W(q,\\omega=0)$ can be proven~\\cite{crpa}.\n\n\nHowever, the static $W_{\\rm H}(q,\\omega=0)$ amounts to neglecting the frequency dependent part \n\\begin{eqnarray}\nW^{\\rm dyn}_{\\rm H}(q,\\omega)\\equiv W_{\\rm H}(q,\\omega)- W_{\\rm H}(q,\\omega=0)\n\\label{Wrdyn}\n\\end{eqnarray}\ndepicted by the vertical hatching in Fig.~\\ref{Cartoon_W}.\nIn this paper, we will take into account the \ncontribution of this dynamical part as the renormalization to the \nkinetic energy part, either as a perturbative\nself-energy or in a nonperturbative fashion.\nIn the effective low-energy model, we then keep $W_{\\rm H}(q,\\omega=0)$ for the effective interaction. \nIn the case of the perturbative treatment, for example, \nthe contribution to the self-energy $\\Delta \\Sigma_{\\rm L}$ is \n\\begin{eqnarray}\n\\Delta \\Sigma_{\\rm L}^{\\rm Pert}=G_{\\rm L} W^{\\rm dyn}_{\\rm H}(q,\\omega)\n\\label{DeltaSigmaH}\n\\end{eqnarray}\nas was formulated in Ref.~\\onlinecite{Hirayama} and we review in detail in the next section.\n\n\n\\begin{figure}[h]\n\\centering \n\\includegraphics[clip,width=0.45\\textwidth ]{fig1.eps} \n\\caption{(Color online)\nSchematic frequency dependence of effective interaction screened\nfrom bare interaction $v$, and  \nobtained from full RPA (GW) ($W$), cRPA ($W_{\\rm H}$) and screened interaction by RPA ($W_{\\rm L}$) within low-energy effective model at the effective interaction $U=W_{\\rm H}(\\omega=0)$.\nThis is only a qualitative feature and more realistic dependence is seen in Fig.~\\ref{W}.\n}\n\\label{Cartoon_W}\n\\end{figure} \n\nWe also remark that the dynamical part to be considered can be improved\nfrom Eq.~(\\ref{Wrdyn}) in a more consistent manner:\nSince the screening on the RPA level within the L-space is $W_L$ in Eq.~(\\ref{W_L}), \none realizes that the dynamical part of the interaction\nignored when we use the low-energy solver is   \n\\begin{eqnarray}\nW^{\\rm dyn}_{\\rm GW}(q,\\omega)\\equiv W(q,\\omega) - W_{\\rm L} (q,\\omega)\n\\label{WdynGW}\n\\end{eqnarray}\nas depicted as the horizontal hatching in Fig.~\\ref{Cartoon_W}. \n\nThen we need to take into account the renormalization (namely, self-energy effect) originating from $W^{\\rm dyn}_{\\rm GW}(q,\\omega)$ instead of $W^{\\rm dyn}_{\\rm H}(q,\\omega)$.\nThe perturbative contribution to the self-energy then replaces Eq.~(\\ref{DeltaSigmaH}) with  \n\\begin{eqnarray}\n\\Delta \\Sigma_{\\rm L}^{\\rm GW}=G_{\\rm L} W^{\\rm dyn}_{\\rm GW}(q,\\omega).\n\\label{DeltaSigmaGW}\n\\end{eqnarray}\nUsing $W^{\\rm dyn}_{\\rm GW}(q,\\omega)$ in Eq.~(\\ref{DeltaSigmaGW}) replacing $W^{\\rm dyn}_{\\rm H}(q,\\omega)$ is expected to improve the self-energy, because it takes into account the missing part of the\nL-space dynamics on the GW level and satisfies the chain rule \neven for the self-energy as we show in the following.\n\n\nA conventional one-shot GW calculation in the full space gives\nfull self-energy from the fully screened Coulomb interaction $W$ (Eq.~(\\ref{full_rpa})) as\n\\begin{eqnarray}\n \\Sigma = G_{\\rm L} W\n \\label{full-self-energy}\n\\end{eqnarray}\n(Note: we write here and in the following\nsymbolically $GW$. Depending\non if the calculation is done on the real/Matsubara\naxis, a factor -1 or i has to be added.)\nOn the other hand, the self-energy within the L-space at the one-shot GW level\nis \n\\begin{eqnarray}\n \\Sigma_{\\rm L} = G_{\\rm L} W_{\\rm L}.\n \\label{L-self-energy}\n\\end{eqnarray}\nThen $\\Sigma= \\Delta \\Sigma_{\\rm L}^{\\rm GW}+\\Sigma_{\\rm L}$\nis obviously satisfied, which is the chain rule for the self-energy.\nIn this article, we compare the results calculated from the two choices, Eq.~(\\ref{Wrdyn}) and Eq.~(\\ref{WdynGW}) in examples to gain into physical insights.\n\n\n\n\n\nThe effective model is then given by \n\\begin{eqnarray} \nH_{\\rm eff}&=&\\sum_q T_{\\rm eff}(q,\\omega)c^{\\dagger}_qc_q \\nonumber \\\\\n&+&\\sum_{q,k,p} W_{\\rm H}(q,0)c^{\\dagger}_k c_{k+q}c^{\\dagger}_p c_{p-q},\n\\label{Heff_with_w}\n\\end{eqnarray}\n\nwith $T_{\\rm eff}$ given by Eq.~(\\ref{Teff(q,w)}) by employing either $\\Delta \\Sigma_{\\rm L}^{\\rm Pert}$ or $\\Delta \\Sigma_{\\rm L}^{\\rm GW}$ for $\\Delta \\Sigma_{\\rm L}$ contained in $\\Delta \\Sigma$ in Eq.~(\\ref{Teff(q,w)}).\nEquation~(\\ref{Heff_with_w}) still contains the frequency dependence in $T_{\\rm eff}$,\nwhich should be incorporated in the frequency-independent form by including the renormalization effect.\n\n\nThen, on top of the zero-frequency limit $T_{\\rm eff}(q,\\omega=0)$, \nto incorporate the effects of the frequency-dependence,  \nwe implement the following procedure: \nFirst, the frequency dependence in the nonlocal part of $W_{\\rm H}$ \nis taken into account perturbatively. \nThis is done by constructing a self-energy\n$\\Delta \\Sigma (q,\\omega)$ along the lines of \nRef.~\\onlinecite{Hirayama}. This \nproposal employs $\\Delta \\Sigma_{\\rm L}(q,\\omega)=G_{\\rm L}W^{\\rm dyn}$,\nwhere $W^{\\rm dyn}$ is either $W^{\\rm dyn}_{\\rm H}$ or  $W^{\\rm dyn}_{\\rm GW}$. Thus incorporated renormalized \nsingle-particle part is linearized in \n$\\omega$ as $T_{\\rm eff}(q,\\omega=0)-[d\\Delta \\Sigma (q,\\omega)/d\\omega]\\omega$,\nand the $\\omega$ dependence is absorbed into the renormalization factor  \n$Z_{\\rm corr}=1/[1-d\\Delta \\Sigma (q,\\omega)/d\\omega|_{\\omega=0}]$,\nwhere the dispersion $T_{\\rm eff}^{(0)}(q)$ is replaced with $Z_{\\rm corr}T_{\\rm eff}^{(0)}(q)$. \n\nThe effects of the local part of the interaction are taken into \naccount following the proposal by Casula {\\it et al.}~\\cite{casula-ZB}.\nThere it was shown that -- in the antiadiabatic limit --\na model with frequency-dependent interactions\ncan be mapped onto a model with static interactions and a renormalized\none-body Hamiltonian. The renormalization factor $Z_B$ can be explicitly\nobtained from the frequency-dependence of the interaction. We will \ngive the explicit form in the next section.\n\nAs a result, the single-particle part of the effective Hamiltonian \nEq.~(\\ref{zeroth-Heff}) as defined by its hopping $T_{\\rm eff}^{(0)}(q)$ \nis replaced by a single-particle Hamiltonian with effective hopping \n\\begin{eqnarray}\nT_{\\rm eff}^{(1)}(q)=[\\epsilon_{\\rm DFT}-V_{\\rm xc} +\n\\Delta \\Sigma (q,\\omega=0)]Z_{\\rm corr}Z_B,\n\\end{eqnarray}\nwhich replaces $T_{\\rm eff}(q,\\omega)$ in Eq.~(\\ref{Heff_with_w}) and the Hamiltonian form (\\ref{zeroth-Heff}) is obtained.\n\n\n\n2.3 Renormalization to self-energy from frequency-dependent partially-screened interaction\n\\subsection{Renormalization to self-energy from frequency-dependent partially-screened interaction }\nThe cRPA~\\cite{crpa} was \nproposed as a means to calculate the effective local Coulomb interactions\nto be used in the L-space from a systematic first-principles procedure.\nIt can be understood as a way of tracing out the H-space for deriving the effective interaction, while\nkeeping track of the resulting renormalization of the L-space\ndegrees of freedom.\nThe tracing out of the H-space by the standard cRPA results in an effective interaction\nfor the two-particle part of the model in the L-space in the form\n\\begin{eqnarray}\nW_{\\rm H} (q,\\omega) = \\frac{v(q)}{1- P_{\\rm H}(q,\\omega)v(q)},\n\\label{Wr_crpa}\n\\end{eqnarray}\nwhere the wave-number ($q$) dependent bare Coulomb interaction $v$ \nis partially screened by the partial polarization $P_{\\rm H}$. \nHere, $P_{\\rm H}$ is defined in terms of the total polarization $P$ by excluding \nthe intra-L-space polarization $P_{\\rm L}$: $P_{\\rm H}\\equiv P-P_{\\rm L}$.\n$P_{\\rm L}$ involves only screening processes within the L-space.\n\nHere, $W_{\\rm H}$ is frequency dependent as\nschematically illustrated in Fig.~\\ref{Cartoon_W}.\nHowever, most many-body calculations in the literature that use the effective \ninteractions from the cRPA method or similar schemes neglect this\nfrequency-dependence (exceptions are \nRefs.~\\onlinecite{werner12},\n\\onlinecite{casula-ZB},\n\\onlinecite{vanroeke2014}, \n\\onlinecite{Hirayama},\n\\onlinecite{casula-prb},\n\\onlinecite{werner-LCO}, \n\\onlinecite{cafe2as2},\n\\onlinecite{sakuma2013}, \n\\onlinecite{sakuma14},\n\\onlinecite{tomczak-gwdmft},\n\\onlinecite{TomczakPRB2014}),\nand use only the zero-frequency value of the\ninteraction $W_{\\rm H}(q,\\omega=0)$ for the construction of the low-energy effective Hamiltonian~\\cite{crpa,Hirayama}.\n\n\nWe note that this static limit $W_{\\rm H} (q,\\omega=0) $ obtained by using Eq.~(\\ref{Wr_crpa}) in fact\nsatisfies the above mentioned chain rule: \nThe whole dynamical interaction emerging when \none solves the whole H- and L-space degrees of freedom by RPA is the\nusual fully screened interaction \n$W(q,\\omega)$ given by\n\\begin{eqnarray}\nW (q,\\omega) = \\frac{v(q)}{1- P(q,\\omega)v(q)},\n\\label{full_rpa}\n\\end{eqnarray}\nas is depicted in Fig.~\\ref{Cartoon_W}.\n\nOn the other hand, \nif we calculate the screening by the RPA within the L-space by regarding as if $W_{\\rm H} (q,\\omega=0)$ \nwould be the bare interaction, \nthis leads to a screened interaction\n\\begin{eqnarray}\nW_{\\rm L} (q,\\omega) = \\frac{W_{\\rm H}(q,\\omega=0)}{1- P_{\\rm L}(q,\\omega)W_{\\rm H}(q,\\omega=0)},\n\\label{W_L}\n\\end{eqnarray}\nwhich is depicted schematically in Fig.~\\ref{Cartoon_W}.\nHere, $P_{\\rm L}$ is the RPA polarization in the low-energy\nsubspace.\nNote that $W_{\\rm L} (q,\\omega\\rightarrow \\infty) = W_{\\rm H}(q,\\omega=0)$.\nThen the chain rule\n$W_{\\rm L} (q,\\omega) =W(q,\\omega=0)$ can be proven~\\cite{crpa}.\n\n\nHowever, the static $W_{\\rm H}(q,\\omega=0)$ amounts to neglecting the frequency dependent part \n\\begin{eqnarray}\nW^{\\rm dyn}_{\\rm H}(q,\\omega)\\equiv W_{\\rm H}(q,\\omega)- W_{\\rm H}(q,\\omega=0)\n\\label{Wrdyn}\n\\end{eqnarray}\ndepicted by the vertical hatching in Fig.~\\ref{Cartoon_W}.\nIn this paper, we will take into account the \ncontribution of this dynamical part as the renormalization to the \nkinetic energy part, either as a perturbative\nself-energy or in a nonperturbative fashion.\nIn the effective low-energy model, we then keep $W_{\\rm H}(q,\\omega=0)$ for the effective interaction. \nIn the case of the perturbative treatment, for example, \nthe contribution to the self-energy $\\Delta \\Sigma_{\\rm L}$ is \n\\begin{eqnarray}\n\\Delta \\Sigma_{\\rm L}^{\\rm Pert}=G_{\\rm L} W^{\\rm dyn}_{\\rm H}(q,\\omega)\n\\label{DeltaSigmaH}\n\\end{eqnarray}\nas was formulated in Ref.~\\onlinecite{Hirayama} and we review in detail in the next section.\n\n\n\\begin{figure}[h]\n\\centering \n\\includegraphics[clip,width=0.45\\textwidth ]{fig1.eps} \n\\caption{(Color online)\nSchematic frequency dependence of effective interaction screened\nfrom bare interaction $v$, and  \nobtained from full RPA (GW) ($W$), cRPA ($W_{\\rm H}$) and screened interaction by RPA ($W_{\\rm L}$) within low-energy effective model at the effective interaction $U=W_{\\rm H}(\\omega=0)$.\nThis is only a qualitative feature and more realistic dependence is seen in Fig.~\\ref{W}.\n}\n\\label{Cartoon_W}\n\\end{figure} \n\nWe also remark that the dynamical part to be considered can be improved\nfrom Eq.~(\\ref{Wrdyn}) in a more consistent manner:\nSince the screening on the RPA level within the L-space is $W_L$ in Eq.~(\\ref{W_L}), \none realizes that the dynamical part of the interaction\nignored when we use the low-energy solver is   \n\\begin{eqnarray}\nW^{\\rm dyn}_{\\rm GW}(q,\\omega)\\equiv W(q,\\omega) - W_{\\rm L} (q,\\omega)\n\\label{WdynGW}\n\\end{eqnarray}\nas depicted as the horizontal hatching in Fig.~\\ref{Cartoon_W}. \n\nThen we need to take into account the renormalization (namely, self-energy effect) originating from $W^{\\rm dyn}_{\\rm GW}(q,\\omega)$ instead of $W^{\\rm dyn}_{\\rm H}(q,\\omega)$.\nThe perturbative contribution to the self-energy then replaces Eq.~(\\ref{DeltaSigmaH}) with  \n\\begin{eqnarray}\n\\Delta \\Sigma_{\\rm L}^{\\rm GW}=G_{\\rm L} W^{\\rm dyn}_{\\rm GW}(q,\\omega).\n\\label{DeltaSigmaGW}\n\\end{eqnarray}\nUsing $W^{\\rm dyn}_{\\rm GW}(q,\\omega)$ in Eq.~(\\ref{DeltaSigmaGW}) replacing $W^{\\rm dyn}_{\\rm H}(q,\\omega)$ is expected to improve the self-energy, because it takes into account the missing part of the\nL-space dynamics on the GW level and satisfies the chain rule \neven for the self-energy as we show in the following.\n\n\nA conventional one-shot GW calculation in the full space gives\nfull self-energy from the fully screened Coulomb interaction $W$ (Eq.~(\\ref{full_rpa})) as\n\\begin{eqnarray}\n \\Sigma = G_{\\rm L} W\n \\label{full-self-energy}\n\\end{eqnarray}\n(Note: we write here and in the following\nsymbolically $GW$. Depending\non if the calculation is done on the real/Matsubara\naxis, a factor -1 or i has to be added.)\nOn the other hand, the self-energy within the L-space at the one-shot GW level\nis \n\\begin{eqnarray}\n \\Sigma_{\\rm L} = G_{\\rm L} W_{\\rm L}.\n \\label{L-self-energy}\n\\end{eqnarray}\nThen $\\Sigma= \\Delta \\Sigma_{\\rm L}^{\\rm GW}+\\Sigma_{\\rm L}$\nis obviously satisfied, which is the chain rule for the self-energy.\nIn this article, we compare the results calculated from the two choices, Eq.~(\\ref{Wrdyn}) and Eq.~(\\ref{WdynGW}) in examples to gain into physical insights.\n\n\n\n\n\nThe effective model is then given by \n\\begin{eqnarray} \nH_{\\rm eff}&=&\\sum_q T_{\\rm eff}(q,\\omega)c^{\\dagger}_qc_q \\nonumber \\\\\n&+&\\sum_{q,k,p} W_{\\rm H}(q,0)c^{\\dagger}_k c_{k+q}c^{\\dagger}_p c_{p-q},\n\\label{Heff_with_w}\n\\end{eqnarray}\n\nwith $T_{\\rm eff}$ given by Eq.~(\\ref{Teff(q,w)}) by employing either $\\Delta \\Sigma_{\\rm L}^{\\rm Pert}$ or $\\Delta \\Sigma_{\\rm L}^{\\rm GW}$ for $\\Delta \\Sigma_{\\rm L}$ contained in $\\Delta \\Sigma$ in Eq.~(\\ref{Teff(q,w)}).\nEquation~(\\ref{Heff_with_w}) still contains the frequency dependence in $T_{\\rm eff}$,\nwhich should be incorporated in the frequency-independent form by including the renormalization effect.\n\n\nThen, on top of the zero-frequency limit $T_{\\rm eff}(q,\\omega=0)$, \nto incorporate the effects of the frequency-dependence,  \nwe implement the following procedure: \nFirst, the frequency dependence in the nonlocal part of $W_{\\rm H}$ \nis taken into account perturbatively. \nThis is done by constructing a self-energy\n$\\Delta \\Sigma (q,\\omega)$ along the lines of \nRef.~\\onlinecite{Hirayama}. This \nproposal employs $\\Delta \\Sigma_{\\rm L}(q,\\omega)=G_{\\rm L}W^{\\rm dyn}$,\nwhere $W^{\\rm dyn}$ is either $W^{\\rm dyn}_{\\rm H}$ or  $W^{\\rm dyn}_{\\rm GW}$. Thus incorporated renormalized \nsingle-particle part is linearized in \n$\\omega$ as $T_{\\rm eff}(q,\\omega=0)-[d\\Delta \\Sigma (q,\\omega)/d\\omega]\\omega$,\nand the $\\omega$ dependence is absorbed into the renormalization factor  \n$Z_{\\rm corr}=1/[1-d\\Delta \\Sigma (q,\\omega)/d\\omega|_{\\omega=0}]$,\nwhere the dispersion $T_{\\rm eff}^{(0)}(q)$ is replaced with $Z_{\\rm corr}T_{\\rm eff}^{(0)}(q)$. \n\nThe effects of the local part of the interaction are taken into \naccount following the proposal by Casula {\\it et al.}~\\cite{casula-ZB}.\nThere it was shown that -- in the antiadiabatic limit --\na model with frequency-dependent interactions\ncan be mapped onto a model with static interactions and a renormalized\none-body Hamiltonian. The renormalization factor $Z_B$ can be explicitly\nobtained from the frequency-dependence of the interaction. We will \ngive the explicit form in the next section.\n\nAs a result, the single-particle part of the effective Hamiltonian \nEq.~(\\ref{zeroth-Heff}) as defined by its hopping $T_{\\rm eff}^{(0)}(q)$ \nis replaced by a single-particle Hamiltonian with effective hopping \n\\begin{eqnarray}\nT_{\\rm eff}^{(1)}(q)=[\\epsilon_{\\rm DFT}-V_{\\rm xc} +\n\\Delta \\Sigma (q,\\omega=0)]Z_{\\rm corr}Z_B,\n\\end{eqnarray}\nwhich replaces $T_{\\rm eff}(q,\\omega)$ in Eq.~(\\ref{Heff_with_w}) and the Hamiltonian form (\\ref{zeroth-Heff}) is obtained.\n\n\n\n6 Results\n\\section{Results}\n\\label{results}\n\n\nThe ternary 3d$^1$ transition metal oxide SrVO$_3$ has become\none of the ``drosophila compounds'' of correlated systems.\nIt is a correlated metal\nthat has been characterized using various experimental \n\\cite{Onoda, inoue, maiti, maiti2, sekiyama, takizawa, yoshida2010}.\nand theoretical techniques, see e.g.\n\\cite{nekrasov, anisimov, pavarini-d1, liebsch, lechermann-d1, trimarchi, aichhorn09, karolak, lee, held-optics, tomczak-gwdmft, gatti-guzzo, sakuma14, TomczakPRB2014}.\nA review of most of the available experimental and theoretical \ndata has been given recently in Ref.~\\onlinecite{TomczakPRB2014}.\nSrVO$_3$ displays Fermi liquid behavior up to\nremarkably high temperatures of the order of $200$ K \\cite{Onoda, inoue},\nwith a moderate mass enhancement of the order \nof $2$ \\cite{yoshida2010, vandermarel}.\nDetailed spectroscopic investigations have made it an\nideal test compound for modern many-body calculations, and \nmore and more refined dynamical mean field-based studies \nare available.\nThe majority of studies so far have focused on the t$_{2g}$-manifold\nwhich forms the states close to the Fermi level, and those will\nalso be the focus in the present investigations. \nNote however that this restriction quite severely limits the\nrange of validity of the low-energy description. Indeed, as\nshown recently~\\cite{tomczak-gwdmft, TomczakPRB2014} at\nenergies of about 2 to 3 eV above the Fermi level\nthe spectral properties are largely determined by the e$_g$\nstates. This has in particular led to a reinterpretation of\nan inverse photoemission feature at about 2.5 eV that was \nfrequently interpreted as an upper Hubbard band of t$_{2g}$\ncharacter in the earlier literature.\nHere, we use the compound only as an illustration of \nthe principles of constructing effective low-energy models,\nwithout being concerned with a description of spectra beyond\nthe pure t$_{2g}$ part.\n\n\nThe LDA band structure of SrVO$_3$ is shown in Fig.~\\ref{lda}.\nOne clearly distinguishes the three-fold degenerate manifold\nof t$_{2g}$ bands close to the Fermi level (chosen as the\nzero of energy), followed in the unoccupied part of the spectrum\nby the two e$_g$ bands. At about -2.5 eV the filled O-2p derived\nbands are visible.\n\nWhen a standard GW calculation is performed, see Fig.~\\ref{fullGW},\nthe bandwidth of the t$_{2g}$-states is reduced from 2.5 eV to\n2.1 eV, while the overall shape of the dispersion remains similar\nto the LDA one.\nThis is in agreement with previous GW calculations in the literature\n\\cite{tomczak-gwdmft, TomczakPRB2014, gatti-guzzo, sakuma14}.\n\n\n\nThe calculation is based on the full-potential linear muffin-tin orbitals (FP-LMTO) implementation.\n8$\\times$8$\\times$8 \\bm{k}-mesh is employed in both the DFT/LDA and the GW calculations.\nThe calculational details are the same as in Ref.~\\onlinecite{Hirayama}.\n\n\nWe now turn to a discussion of the band structure corresponding to \nthe Hamiltonians to be used as input for subsequent many-body \ncalculations for this material.\nWe will proceed step by step to analyze the influence of the various\ncorrection terms, with respect to the DFT starting point\n(which will be overlaid to the respective band structures).\n\n\n\n\\begin{figure}[h]\n\\centering \n\\includegraphics[clip,width=0.45\\textwidth ]{fig2.eps} \n\\caption{(Color online)\nKohn-Sham band structure of SrVO$_3$ in the LDA.\nThe energy is measured from the Fermi level.\n}\n\\label{lda}\n\\end{figure} \n\n\n\\begin{figure}[h]\n\\centering \n\\includegraphics[clip,width=0.45\\textwidth ]{fig3.eps} \n\\caption{(Color online)\nBand structure  of SrVO$_3$ in the one-shot GW approximation.\nFor comparison, the LDA band structure is also given\n(black dotted line).\nThe energy is measured from the Fermi level.\n}\n\\label{fullGW}\n\\end{figure} \n\n\n\n\n\n\n\\begin{figure}[h]\n\\centering \n\\includegraphics[clip,width=0.45\\textwidth ]{fig4.eps} \n\\caption{(Color online)\nBand structure of SrVO$_3$ corresponding to the LDA Hamiltonian\nfrom which the LDA exchange-correlation potential has been subtracted.\nFor comparison, the LDA band structure is also given\n(black dotted line).\nThe energy is measured from the Fermi level.\n}\n\\label{lda-vxc}\n\\end{figure} \n\n\n\\begin{figure}[h]\n\\centering \n\\includegraphics[clip,width=0.45\\textwidth ]{fig5.eps} \n\\caption{(Color online)\nBand structure  of SrVO$_3$ in the Hartree approximation.\nFor comparison, the LDA band structure is also given\n(black dotted line).\nThe energy is measured from the Fermi level.\n}\n\\label{hartree}\n\\end{figure} \n\n\nFigure~\\ref{lda-vxc} \ndisplays the band structure corresponding to\nthe LDA Hamiltonian from which the \nLDA exchange-correlation\npotential has been subtracted,  \nthat is, the eigenvalues of $H_{\\rm LDA}-V_{\\rm xc}$ where\nthese operators are evaluated for the self-consistent LDA\ndensity.\nThe subtraction of $V_{\\rm xc}$ widens the band structure from \nthe LDA bandwidth of 2.5 eV \nto more than twice this value: the new band\nwidth is 5.4 eV.\nThis indicates that the exchange-correlation potential $V_{\\rm xc}$ \nof the LDA is responsible for a substantial amount of \nthe band-narrowing effect\narising from electronic correlations, stressing the need \nto subtract the double counting in a consistent manner. \nFigure~\\ref{lda-vxc} is regarded as the starting point for the\nfollowing considerations.\n\nFor simple materials (see e.g. the calculations on Li in Ref.~\\onlinecite{Lithium})\nit has been noted in the literature that the Hartree band structure\nis close to the DFT one. This raises the question of the origin of the\nsubstantial band widening in the present case. Indeed, the present\ncalculation shown in Fig.~\\ref{lda-vxc} differs from a Hartree calculation\nonly \nby the fact that Fig.~\\ref{lda-vxc} is evaluated for the converged LDA density.\nWe have performed a test calculation where we plot the Hartree band\nstructure calculated for the converged Hartree density. The result\nis shown in Fig.~\\ref{hartree}. As seen from this plot, while the\nband is not fully as wide as in Fig.~\\ref{lda-vxc}, a substantial \nwidening is already present at this stage.\n\n\n\n\\begin{figure}[h]\n\\centering \n\\includegraphics[clip,width=0.45\\textwidth ]{fig6.eps} \n\\caption{(Color online)\nBand structure  of $H_{\\rm LDA}-V_{\\rm xc}+\\Delta \\Sigma_{\\rm H}(\\omega=0)$ of SrVO$_3$.\nFor comparison, the LDA band structure is also given\n(black dotted line).\nThe energy is measured from the Fermi level.\n}\n\\label{lda-vxc-sigH(0)}\n\\end{figure} \n\n\n\nStarting from the Hamiltonian without V$_{xc}$ (see the \ndispersion in Fig.~\\ref{lda-vxc}), we first take into account \nthe static part of the corrective self-energy $\\Delta \\Sigma_{\\rm H}$:\nIn Fig.~\\ref{lda-vxc-sigH(0)}, we plot the dispersion corresponding to\n$H_{\\rm LDA}-V_{\\rm xc}+\\Delta \\Sigma_{\\rm H}(\\omega=0)$. \nCompared to the dispersion of Fig.~\\ref{lda-vxc}, the overall band structure \nis narrowed to 3.7 eV. This value is, however, still considerably larger\nthan the LDA bandwidth (Fig.~\\ref{lda}). Although\n the electronic correlations coupling the H- and \nthe L space are included in Fig.~\\ref{lda-vxc-sigH(0)}, and narrow the band with respect to the case\nwhere V$_{xc}$ is taken out, correlations within\nthe L space are not included. \nThe LDA, on the other hand, at least partially\nincludes correlations within the L space, \nand these are very effective in narrowing the band.\nInterestingly, the bottom of the occupied band is quite\nexactly at the LDA value, and the remaining widening is\npurely in the unoccupied part.\n\n\n\\begin{figure*}[htb]\n\\begin{center} \n\\includegraphics[width=0.9\\textwidth ]{fig7.eps} \n\\end{center} \n\\caption{(Color online)\nFrequency dependence of $-V_{xc}+\\Delta \\Sigma_{\\rm H}$ for SrVO$_3$.\n(a) and (b) are the real and imaginary parts at the $\\Gamma$ point, \nrespectively,\nand (c) and (d) ((e) and (f)) are those at $X$ ($R$).\nFor comparison, the full GW self-energies are also given\n(black dotted line).\n}\n\\label{sigH}\n\\end{figure*} \n\n\n\n\\begin{figure}[h]\n\\centering \n\\includegraphics[clip,width=0.45\\textwidth ]{fig8.eps} \n\\caption{(Color online)\nBand structure  of $Z_{\\rm H}[H_{\\rm LDA}-V_{\\rm xc}+\\Delta\\Sigma_{\\rm H}(\\omega=0)]$ of SrVO$_3$.\nFor comparison, the LDA band structure is also given\n(black dotted line).\nThe energy is measured from the Fermi level.\n}\n\\label{ZH_SigmaH}\n\\end{figure} \n\n\nTo go further and in particular to analyze the dynamic\nbehavior of $\\Delta \\Sigma_{\\rm H}$ \nwe plot in Fig.~\\ref{sigH} \nthe self-energy corrections $\\Delta \\Sigma_{\\rm H}$ (with $\\Sigma$ (see Eq.~(\\ref{full-self-energy}))\nfor comparison) for the real and imaginary parts\nat several representative choices of momenta.\nThe frequency dependence is smooth around the Fermi level.\nIn particular, in contrast to the full self-energy,\nthere are no poles in $\\Delta \\Sigma_{\\rm H}$ \nin the low-energy region, thanks to the\nexclusion of fluctuations within the L-space.\nThe frequency dependence of the real part indicates that the \nlinearization \n$\\displaystyle{\\Delta \\Sigma_{\\rm H}(\\omega) \\sim \\Delta \\Sigma_{\\rm H}(\\omega=0)+[d\\Delta \\Sigma_{\\rm H}(\\omega)/d\\omega]_{\\omega=0}\\omega }$\noffers a reasonably good approximation. \nIn fact, the behavior of the constrained self-energy\nis much closer to linearity than that of \nthe full GW self-energy $\\Sigma$.\nThis is easily understood by the fact that the low-energy excitations are excluded in the present constrained self-energy $\\Delta\\Sigma_{\\rm H}$ analogously to an insulator, thus eliminating the strong frequency dependence of the constrained self-energy near the Fermi level. This is one of the consequences of the \ncontrollability and an advantage of the present MACE scheme, as \nmentioned in the introduction.\nIn the following discussion, we employ this linearized approximation.\nThen the renormalization factor $Z_{\\rm H}$ defined by Eq.~(\\ref{ZH})\nis interpreted as that from the contribution of the H-space. \nAfter \nthis renormalization factor is taken into account,\nthe effective Hamiltonian is given by $Z_{\\rm H}[H_{\\rm LDA}-V_{\\rm xc}+\\Delta\\Sigma_{\\rm H}(\\omega=0)]$ (see Fig.~\\ref{ZH_SigmaH}). \nInterestingly the renormalization factor $Z_{\\rm H}$=0.92 stays\nclose to one, so that\nthe bandwidth is only slightly reduced (to\n$3.4$ eV instead of the $3.7$ eV above). \nWe also note that $Z_{\\rm H}$ has weak momentum dependence\nas we reveal in the following.\n\n\n\n\\begin{figure*}[htb]\n\\begin{center} \n\\includegraphics[width=0.9\\textwidth ]{fig9.eps} \n\\end{center} \n\\caption{(Color online)\n{\\bf minor modification}\nFrequency dependence of $-V_{\\rm xc}+\\Delta \\Sigma_{\\rm H}$ of SrVO$_3$.\n(a) and (b) are the real and imaginary parts of the local\ncomponent, respectively.\nThe black dotted line is $\\Sigma$ defined in Eq.~(\\ref{full-self-energy}) in the full GW approximation for comparison.\n(c) and (d) are the real and imaginary \nparts at several $k$ points.\n}\n\\label{sigH_nonlocal}\n\\end{figure*} \n\n\n\nFigure~\\ref{sigH_nonlocal} \nshows the local and nonlocal parts of $\\Delta \\Sigma_{\\rm H}$.\nThe frequency \ndependence\nof the local part (Figs.~\\ref{sigH_nonlocal} (a)(b)) \nis quite similar \nto the  frequency-dependent self-energies at various momenta\nin Fig.~\\ref{sigH}.\nOne immediately reads off an interesting property, which is\nakin to what has been found for the full GW self-energy in\nRef.~\\onlinecite{Tomczak-PRL2012}, namely that the nonlocal part of \n$\\Delta \\Sigma_{\\rm H}$ shows only weak frequency dependence \nas shown in Figs.~\\ref{sigH_nonlocal} (c)(d). \nThis explains why $Z_{\\rm H}$ \nis only weakly momentum-dependent.\nOn the other hand, $\\Delta \\Sigma_{\\rm H}^{\\rm nonlocal} (0)$ at $R$ is $\\sim 1$ eV larger than \nat $\\Gamma$, which causes the band widening effect.\n $\\Delta \\Sigma_{\\rm H}$\ncan thus to a good approximation be decomposed into a\nfrequency-dependent local part and a static nonlocal one:\n$\\Delta \\Sigma_{\\rm H}(k,\\omega) = \\Delta \\Sigma^{\\rm local}_{\\rm H}(\\omega) + \\Delta \\Sigma^{\\rm nonlocal}_{\\rm H}(k)$,\nwith $ \\Delta \\Sigma_{\\rm H}^{\\rm local}(\\omega) = \\sum_q  \\Delta \\Sigma_{\\rm H}(q,\\omega)$\n\nSince in the simple case of SrVO$_3$ where the t$_{2g}$ states are\ndegenerate, static local operators are scalar and are compensated \nby a chemical potential shift such that the correct particle\nnumber is obtained. The above band structure corresponding to\n$H_{\\rm LDA} - V_{\\rm xc} + \\Delta \\Sigma_{\\rm H}(k, \\omega=0)$ \nis thus identical to that of\n$H_{\\rm LDA} - V_{\\rm xc} + \\Delta \\Sigma^{\\rm nonlocal}_{\\rm H}(k)$.\nThe local dynamical part of $\\Delta \\Sigma_{\\rm H}$ then results in a \nnarrowing of this band structure by a factor $Z_{\\rm H} = 0.92$.\n\n\n\n\\begin{figure}[h]\n\\centering \n\\includegraphics[clip,width=0.45\\textwidth ]{fig10.eps} \n\\caption{(Color online)\n(a) Frequency dependence of the on-site effective interaction.\nPanel (b) is a low-energy zoom of panel (a). \n}\n\\label{W}\n\\end{figure} \n\n\nBefore turning to a discussion of the low-energy correction for the GW treatment\n$\\Delta \\Sigma_{\\rm L} =G^{(0)}(W-W_{\\rm L})$, we analyze the effective\ninteraction $W_{\\rm L}$ shown in Fig.~\\ref{W} \nin comparison to $W$ and $W_{\\rm H}$.\nThe fully screened Coulomb interaction $W$ displays the familiar\nshape of an interaction that is strongly screened at low energies\n(with a value of $0.88$ eV at $\\omega=0$), while retrieving the\nvalue of the bare Coulomb interaction $v$ ($\\sim 16.0$ eV) \nat high energy.\nThe crossover from the bare to the  \nscreened values takes place at the plasma energy of \nabout $15$ eV. This behavior has been discussed before\n\\cite{Review_SB2014}; we note in particular that the plasma \nfrequency is known from electron energy loss spectroscopy \nmeasurements of the related SrTiO$_3$ compound \\cite{vandermarel,kohiki}.\nThe partially screened interaction\n$W_{\\rm H}$ constructed within cRPA\nconverges to $W$ at high energies, but displays\nweaker screening effects at low energies (with a value of $3.5$ eV at $\\omega=0$),\nsince intra-t$_{2g}$ screening channels are excluded.\nAs was already anticipated in Sec.~\\ref{Outline},  $W_{\\rm L}$ can be interpreted as the screened interaction of \na low-energy model where a static interaction of value $W_{\\rm H}(\\omega=0)$\nhas been imposed as the bare interaction. Since now {\\it only} intra-t$_{2g}$\nscreening channels are active, screening takes place only at low\nenergies, where the scale is given by the t$_{2g}$ bandwidth.\nAlso plotted is the difference $W_{\\rm GW}^{\\rm dyn}=W-W_{d}$: except at low energies\nwhere the t$_{2g}$ screening channels come into play, its frequency\ndependence is essentially given by that of $W$, while the\nhigh-energy limit is the bare Coulomb interaction $v$ reduced\nby $W_{\\rm H}(0)$.\n\n\n\\begin{figure*}[htb]\n\\begin{center} \n\\includegraphics[width=0.9\\textwidth ]{fig11.eps} \n\\end{center} \n\\caption{(Color online)\nFrequency dependence of $-V_{\\rm xc}+ G^{(1)}W_{\\rm H}^{\\rm dyn}$ of SrVO$_3$.\n(a) and (b) are the real and imaginary local parts, respectively.\n(c) and (d) are the real and imaginary \nparts at several $k$ points.\n}\n\\label{DeltaSigma_L_Pert}\n\\end{figure*} \n\n\n\n\\begin{figure*}[htb]\n\\begin{center} \n\\includegraphics[width=0.9\\textwidth ]{fig12.eps} \n\\end{center} \n\\caption{(Color online)\nFrequency dependence of $-V_{\\rm xc}+ G^{(0)}W_{\\rm GW}^{\\rm dyn}$ of SrVO$_3$.\n(a) and (b) are the real and imaginary local parts, respectively.\n(c) and (d) are the real and imaginary \nparts at several $k$ points.\n}\n\\label{DeltaSigma_L_GW}\n\\end{figure*} \n\n\n\nThe frequency dependence of the real and imaginary parts of the low-energy self-energy $\\Delta \\Sigma_{\\rm L} = G^{(1)}W_{\\rm H}^{\\rm dyn}$ for the direct perturbative treatment is illustrated in Fig.~\\ref{DeltaSigma_L_Pert}. \n$\\Delta \\Sigma_{\\rm L} = G^{(0)}W_{\\rm GW}^{\\rm dyn}$ used\nfor the GW treatment is shown in Fig.~\\ref{DeltaSigma_L_GW}\nfor several choices of momenta.\n\\\\\nThe results for the perturbative and the GW treatment are nearly \nidentical.\nThe renormalization factor of $\\Delta \\Sigma_{\\rm L}^{\\rm nonlocal}$ \n(and $\\Delta \\Sigma_{\\rm L}$) is $\\sim 0.77$.\nThe zero-frequency shift\n${\\rm Re}\\Delta \\Sigma_{\\rm L}^{\\rm nonlocal} (0)$ at $R$ is $\\sim 3$ eV \nlarger than \nat $\\Gamma$.\nAgain, one sees how $\\Sigma_{\\rm L}$ separates into local dynamical\nand nonlocal static parts: \n$\\Delta \\Sigma_{\\rm L} = \\Delta \\Sigma^{\\rm nonlocal}_{\\rm L}(k) + \n\\Delta \\Sigma^{\\rm local}_{\\rm L}(\\omega)$.\nThis is expected since such a separation has been found within\nthe full GW calculation \\cite{TomczakPRB2014}, and is even\nmore plausible for a GW treatment within the t$_{2g}$ subspace.\n\n\n\n\\begin{figure}[h]\n\\centering \n\\includegraphics[clip,width=0.45\\textwidth ]{fig13.eps} \n\\caption{(Color online)\nBand structure  of $[\\epsilon_{\\rm DFT}-V_{\\rm xc}+\\Delta\\Sigma_{\\rm H}+\\Delta \\Sigma_{\\rm L}^{\\rm nonlocal}]Z_{\\rm HL}$ of SrVO$_3$ in the direct perturbative treatment.\nFor comparison, the LDA band structure is also given\n(black dotted line).\nThe energy is measured from the Fermi level.\n}\n\\label{DFT+Sigma_HL_Pert}\n\\end{figure} \n\n\n\n\\begin{figure}[h]\n\\centering \n\\includegraphics[clip,width=0.45\\textwidth ]{fig14.eps} \n\\caption{(Color online)\nBand structure  of $[\\epsilon_{\\rm DFT}-V_{\\rm xc}+\\Delta\\Sigma_{\\rm H}+\\Delta \\Sigma_{\\rm L}^{\\rm nonlocal}]Z_{\\rm HL}$ of SrVO$_3$ in the GW treatment.\nFor comparison, the LDA band structure is also given\n(black dotted line).\nThe energy is measured from the Fermi level.\n}\n\\label{DFT+Sigma_HL_GW}\n\\end{figure} \n\n\n\nThe corresponding band structures \nare given by \n $[\\epsilon_{\\rm DFT}+\\Delta\\Sigma_{\\rm H}+\\Delta \\Sigma_{\\rm L}^{\\rm nonlocal}]Z_{\\rm HL}$ in Fig.~\\ref{DFT+Sigma_HL_Pert} \nfor the perturbative treatment (see Eq.~(\\ref{DeltaSigmaLnonlocalPert})), and in Fig.~\\ref{DFT+Sigma_HL_GW} \nfor the GW treatment,\nwhere $\\Delta \\Sigma_{\\rm L}^{\\rm nonlocal}$ is defined in Eq.~(\\ref{DeltaSigLnonlocalGW}) and $Z_{\\rm HL}$ is around 0.92.\n$Z_{\\rm HL}$ is practically the same as $Z_{\\rm H}$, meaning that the nonlocal dynamical correction \nin the H-space is small, which justifies the GW perturbative treatment for the nonlocal part.\nOne finds that \nthe nonlocal part $\\Delta \\Sigma_{\\rm L}=G^{(0)}_{ll}(W-W_{\\rm L})$ narrows\nthe empty states but widens the occupied ones, resulting\nin an overall bandwidth of 3.2 eV.\nThe difference between the direct perturbation and the GW treatment is small.\n\n\n\n\\begin{figure}[h]\n\\centering \n\\includegraphics[clip,width=0.45\\textwidth ]{fig15.eps} \n\\caption{(Color online)\nBand structure  of $[\\epsilon_{\\rm DFT}-V_{\\rm xc}+\\Delta\\Sigma_{\\rm H}+\\Delta\\Sigma_{\\rm L}^{\\rm nonlocal}]Z_{\\rm HL}Z_B$ of SrVO$_3$ in the GW treatment.\nFor comparison, the LDA band structure is also given\n(black dotted line).\nThe energy is measured from the Fermi level.}\n\\label{DFT+DeltaSigmaL_GW_ZHL_ZB}\n\\end{figure} \n\n\nAfter including the effect of the local self-energy by the nonperturbative \nCasula trick, we show the dispersion\ngiven by \n$[\\epsilon_{\\rm DFT}-V_{\\rm xc}+\\Delta\\Sigma_{\\rm H}+\\Delta\\Sigma_{\\rm L}^{\\rm nonlocal}]Z_{\\rm HL}Z_B$ for the GW-type treatment in Fig.~\\ref{DFT+DeltaSigmaL_GW_ZHL_ZB}: \nThe GW-like treatment gives a band dispersion\nclose to the LDA one. \nThe $Z_B$ factor corresponding to the local dynamical part of \n$\\Delta \\Sigma_{\\rm L}$ amounts to $Z_B=0.7$.\nThe nonperturbative treatment \nresults in an LDA-like band dispersion for the empty states,\nbut a slightly narrower bandwidth in the occupied part.\nThe low-energy effective Hamiltonian at this level of treatment has a \nfrequency independent effective interaction both with local \nand nonlocal interaction given by the Fourier transform of \n$W_{\\rm H}(q,\\omega=0)$, which contains both direct and exchange interactions. \n\n\n\\begin{figure}[h]\n\\centering \n\\includegraphics[clip,width=0.45\\textwidth ]{fig16.eps} \n\\caption{(Color online)\nBand structure  of $[\\epsilon_{\\rm DFT}+\\Sigma_{\\rm corr}^{\\rm GW}]Z_{\\rm corr}Z_B$ of SrVO$_3$.\nFor comparison, the LDA band structure is also given\n(black dotted line).\nThe energy is measured from the Fermi level.}\n\\label{DFT+Sigma_corr_GW_Z}\n\\end{figure} \n\n\n\nWe finally show the dispersion\ngiven by Eq.~(\\ref{Teff_final}), namely\n$[\\epsilon_{\\rm DFT}+\\Sigma_{\\rm corr}^{\\rm GW}]Z_{\\rm corr}Z_B$ in Fig.~\\ref{DFT+Sigma_corr_GW_Z}. \nThe result shows an LDA-like band dispersion for the occupied states,\nbut a slightly wider bandwidth in the empty part, resulting in a bandwidth 14\\% wider in total than the LDA bandwidth. \nThe effective interaction of the low-energy effective Hamiltonian \nat this final level has\nfrequency independent local and nonlocal interactions \ngiven by the Fourier transform of $W_r(q,\\omega=0)$. \n\n\n\n\\begin{figure}[h]\n\\centering \n\\includegraphics[clip,width=0.45\\textwidth ]{fig17.eps} \n\\caption{(Color online)\nBand structure  of $[\\epsilon_{\\rm DFT}-V_{\\rm xc}+\\Sigma-(G^{(0)}W_{\\rm GW}^{\\rm dyn})^{\\rm nonlocal}]Z'Z_B$ of SrVO$_3$\n(Z' is the renormalization factor of $\\Sigma-(G^{(0)}W_{\\rm GW}^{\\rm dyn})^{\\rm nonlocal}$).\nFor comparison, the LDA band structure is also given\n(black dotted line).\nThe energy is measured from the Fermi level.\n}\n\\label{GW+ZB}\n\\end{figure} \n\n\n\nThe resulting effective Hamiltonian in the L-space \nconsists of single-particle and two-particle (interaction) parts.\nThe effective interaction is the same as in previous estimates \nby the cRPA in the literature, while the single-particle dispersion \nis revised after removing the double counting and taking into \naccount the frequency dependence of the effective interaction. The \nfinal effective bandwidth for SrVO$_3$ is,\nafter partial cancellation, \nslightly (14\\%) larger than the LDA bandwidth.  \nThis is a reasonable result because the LDA takes into account all \ncorrelations though insufficiently, while the present scheme by the constrained self-energy excludes the \ncorrelation effects arising from the L-space.\nAlthough the bandwidth derived for the effective model is slightly \nlarger than that of the LDA,  it is clear that the correlation effects \nare stronger than in the LDA or standard GW when the {\\it ab initio} \nmodel is solved by an accurate solver. In fact, if the effective \ninteraction contained in the final effective model is treated by \nthe GW scheme, one obtains the dispersion illustrated in \nFig.~\\ref{GW+ZB}, \nwhich is given from the self energy of the whole GW calculation $\\Sigma$ \nby correcting the local dynamical part $W-W_{\\rm L}$ by $Z_B$. \nThis indicates that even an insufficient treatment of the local \nstatic interaction by the GW scheme gives a dispersion with the bandwidth ($\\sim 1.9$ eV) narrower \nthan the LDA ($\\sim 2.5$ eV, Fig.~\\ref{lda}) and GW ($\\sim 2.1$ eV, Fig.~\\ref{fullGW}) dispersions.  \nA slightly ($\\sim 14$\\%) wider dispersion than that of the LDA bands obtained for the \neffective Hamiltonian accompanied with frequency independent effective \nCoulomb interactions without the exchange part may account for the slight \noverestimate of correlation effects in the literature mentioned in \nthe introduction.\nOur scheme offers an optimized way for the  derivation of \n{\\it ab initio} models for the L-space after eliminating the H-space\nin  a systematic fashion.\n\n\nOur findings of the band widening are consistent with studies\nbased on the combined GW+DMFT method in the literature \n\\cite{TomczakPRB2014}: There, it was argued that within GW+DMFT\nthe best effective Hamiltonian that DMFT should be performed on,\nis a one-body Hamiltonian corrected by the nonlocal part of the\nGW self-energy. The corresponding spectral function \n(see e.g.\nFigure 5 of Ref.~\\onlinecite{TomczakPRB2014}) displays \na broadening similar to \nFig.~\\ref{DFT+Sigma_HL_GW}.\nMost interestingly, our present calculations confirm a pronounced\nasymmetry observed in Ref.~\\onlinecite{TomczakPRB2014}, namely a stronger widening\neffect in the unoccupied part of the spectrum than in the occupied\none.\n\n",
        "label": "W",
        "Descriptive_question1": "What is the value of the on-site effective interaction at zero frequency in figure_10?",
        "Descriptive_question2": "Which panel in figure_10 provides a low-energy zoom of the frequency dependence?",
        "Reasoning_question1": "How does the screening behavior of the partially screened interaction differ from the fully screened interaction at low energies in figure_10?",
        "Reasoning_question2": "What can be inferred about the impact of intra-t2g screening channels on the effective interaction at low energies in figure_10?",
        "Descriptive_answer1": "3.5 eV",
        "Descriptive_answer2": "(b)",
        "Reasoning_answer1": "The fully screened interaction (W) shows strong screening at low energies, resulting in a much smaller on-site effective interaction (~0.88 eV at zero frequency). In contrast, the partially screened interaction (W_H) exhibits weaker screening effects at low energies, maintaining a higher interaction value (~3.5 eV at zero frequency). This difference arises because W_H excludes certain screening channels (intra-t2g), leading to less reduction in effective interaction compared to W.",
        "Reasoning_answer2": "Since the partially screened interaction W_H excludes intra-t2g screening channels, these channels are responsible for significant screening at low energies in the fully screened interaction W. The absence of intra-t2g screening in W_H results in a higher effective interaction at low energies, indicating that intra-t2g channels play a crucial role in screening the Coulomb interaction within the low-energy manifold."
    },
    {
        "paper_id": "1511.03757.json",
        "image_id": "figure_11",
        "image_path": [
            "/home/yz979/scratch/chengye/scimolmo/test-images-tables/dataset_imagestables/1511.03757/images/fig11.eps"
        ],
        "caption": "(Color online)\nFrequency dependence of $-V_{\\rm xc}+ G^{(1)}W_{\\rm H}^{\\rm dyn}$ of SrVO$_3$.\n(a) and (b) are the real and imaginary local parts, respectively.\n(c) and (d) are the real and imaginary \nparts at several $k$ points.\n",
        "classify": "Chart",
        "section_info": "6 Results\n\\section{Results}\n\\label{results}\n\n\nThe ternary 3d$^1$ transition metal oxide SrVO$_3$ has become\none of the ``drosophila compounds'' of correlated systems.\nIt is a correlated metal\nthat has been characterized using various experimental \n\\cite{Onoda, inoue, maiti, maiti2, sekiyama, takizawa, yoshida2010}.\nand theoretical techniques, see e.g.\n\\cite{nekrasov, anisimov, pavarini-d1, liebsch, lechermann-d1, trimarchi, aichhorn09, karolak, lee, held-optics, tomczak-gwdmft, gatti-guzzo, sakuma14, TomczakPRB2014}.\nA review of most of the available experimental and theoretical \ndata has been given recently in Ref.~\\onlinecite{TomczakPRB2014}.\nSrVO$_3$ displays Fermi liquid behavior up to\nremarkably high temperatures of the order of $200$ K \\cite{Onoda, inoue},\nwith a moderate mass enhancement of the order \nof $2$ \\cite{yoshida2010, vandermarel}.\nDetailed spectroscopic investigations have made it an\nideal test compound for modern many-body calculations, and \nmore and more refined dynamical mean field-based studies \nare available.\nThe majority of studies so far have focused on the t$_{2g}$-manifold\nwhich forms the states close to the Fermi level, and those will\nalso be the focus in the present investigations. \nNote however that this restriction quite severely limits the\nrange of validity of the low-energy description. Indeed, as\nshown recently~\\cite{tomczak-gwdmft, TomczakPRB2014} at\nenergies of about 2 to 3 eV above the Fermi level\nthe spectral properties are largely determined by the e$_g$\nstates. This has in particular led to a reinterpretation of\nan inverse photoemission feature at about 2.5 eV that was \nfrequently interpreted as an upper Hubbard band of t$_{2g}$\ncharacter in the earlier literature.\nHere, we use the compound only as an illustration of \nthe principles of constructing effective low-energy models,\nwithout being concerned with a description of spectra beyond\nthe pure t$_{2g}$ part.\n\n\nThe LDA band structure of SrVO$_3$ is shown in Fig.~\\ref{lda}.\nOne clearly distinguishes the three-fold degenerate manifold\nof t$_{2g}$ bands close to the Fermi level (chosen as the\nzero of energy), followed in the unoccupied part of the spectrum\nby the two e$_g$ bands. At about -2.5 eV the filled O-2p derived\nbands are visible.\n\nWhen a standard GW calculation is performed, see Fig.~\\ref{fullGW},\nthe bandwidth of the t$_{2g}$-states is reduced from 2.5 eV to\n2.1 eV, while the overall shape of the dispersion remains similar\nto the LDA one.\nThis is in agreement with previous GW calculations in the literature\n\\cite{tomczak-gwdmft, TomczakPRB2014, gatti-guzzo, sakuma14}.\n\n\n\nThe calculation is based on the full-potential linear muffin-tin orbitals (FP-LMTO) implementation.\n8$\\times$8$\\times$8 \\bm{k}-mesh is employed in both the DFT/LDA and the GW calculations.\nThe calculational details are the same as in Ref.~\\onlinecite{Hirayama}.\n\n\nWe now turn to a discussion of the band structure corresponding to \nthe Hamiltonians to be used as input for subsequent many-body \ncalculations for this material.\nWe will proceed step by step to analyze the influence of the various\ncorrection terms, with respect to the DFT starting point\n(which will be overlaid to the respective band structures).\n\n\n\n\\begin{figure}[h]\n\\centering \n\\includegraphics[clip,width=0.45\\textwidth ]{fig2.eps} \n\\caption{(Color online)\nKohn-Sham band structure of SrVO$_3$ in the LDA.\nThe energy is measured from the Fermi level.\n}\n\\label{lda}\n\\end{figure} \n\n\n\\begin{figure}[h]\n\\centering \n\\includegraphics[clip,width=0.45\\textwidth ]{fig3.eps} \n\\caption{(Color online)\nBand structure  of SrVO$_3$ in the one-shot GW approximation.\nFor comparison, the LDA band structure is also given\n(black dotted line).\nThe energy is measured from the Fermi level.\n}\n\\label{fullGW}\n\\end{figure} \n\n\n\n\n\n\n\\begin{figure}[h]\n\\centering \n\\includegraphics[clip,width=0.45\\textwidth ]{fig4.eps} \n\\caption{(Color online)\nBand structure of SrVO$_3$ corresponding to the LDA Hamiltonian\nfrom which the LDA exchange-correlation potential has been subtracted.\nFor comparison, the LDA band structure is also given\n(black dotted line).\nThe energy is measured from the Fermi level.\n}\n\\label{lda-vxc}\n\\end{figure} \n\n\n\\begin{figure}[h]\n\\centering \n\\includegraphics[clip,width=0.45\\textwidth ]{fig5.eps} \n\\caption{(Color online)\nBand structure  of SrVO$_3$ in the Hartree approximation.\nFor comparison, the LDA band structure is also given\n(black dotted line).\nThe energy is measured from the Fermi level.\n}\n\\label{hartree}\n\\end{figure} \n\n\nFigure~\\ref{lda-vxc} \ndisplays the band structure corresponding to\nthe LDA Hamiltonian from which the \nLDA exchange-correlation\npotential has been subtracted,  \nthat is, the eigenvalues of $H_{\\rm LDA}-V_{\\rm xc}$ where\nthese operators are evaluated for the self-consistent LDA\ndensity.\nThe subtraction of $V_{\\rm xc}$ widens the band structure from \nthe LDA bandwidth of 2.5 eV \nto more than twice this value: the new band\nwidth is 5.4 eV.\nThis indicates that the exchange-correlation potential $V_{\\rm xc}$ \nof the LDA is responsible for a substantial amount of \nthe band-narrowing effect\narising from electronic correlations, stressing the need \nto subtract the double counting in a consistent manner. \nFigure~\\ref{lda-vxc} is regarded as the starting point for the\nfollowing considerations.\n\nFor simple materials (see e.g. the calculations on Li in Ref.~\\onlinecite{Lithium})\nit has been noted in the literature that the Hartree band structure\nis close to the DFT one. This raises the question of the origin of the\nsubstantial band widening in the present case. Indeed, the present\ncalculation shown in Fig.~\\ref{lda-vxc} differs from a Hartree calculation\nonly \nby the fact that Fig.~\\ref{lda-vxc} is evaluated for the converged LDA density.\nWe have performed a test calculation where we plot the Hartree band\nstructure calculated for the converged Hartree density. The result\nis shown in Fig.~\\ref{hartree}. As seen from this plot, while the\nband is not fully as wide as in Fig.~\\ref{lda-vxc}, a substantial \nwidening is already present at this stage.\n\n\n\n\\begin{figure}[h]\n\\centering \n\\includegraphics[clip,width=0.45\\textwidth ]{fig6.eps} \n\\caption{(Color online)\nBand structure  of $H_{\\rm LDA}-V_{\\rm xc}+\\Delta \\Sigma_{\\rm H}(\\omega=0)$ of SrVO$_3$.\nFor comparison, the LDA band structure is also given\n(black dotted line).\nThe energy is measured from the Fermi level.\n}\n\\label{lda-vxc-sigH(0)}\n\\end{figure} \n\n\n\nStarting from the Hamiltonian without V$_{xc}$ (see the \ndispersion in Fig.~\\ref{lda-vxc}), we first take into account \nthe static part of the corrective self-energy $\\Delta \\Sigma_{\\rm H}$:\nIn Fig.~\\ref{lda-vxc-sigH(0)}, we plot the dispersion corresponding to\n$H_{\\rm LDA}-V_{\\rm xc}+\\Delta \\Sigma_{\\rm H}(\\omega=0)$. \nCompared to the dispersion of Fig.~\\ref{lda-vxc}, the overall band structure \nis narrowed to 3.7 eV. This value is, however, still considerably larger\nthan the LDA bandwidth (Fig.~\\ref{lda}). Although\n the electronic correlations coupling the H- and \nthe L space are included in Fig.~\\ref{lda-vxc-sigH(0)}, and narrow the band with respect to the case\nwhere V$_{xc}$ is taken out, correlations within\nthe L space are not included. \nThe LDA, on the other hand, at least partially\nincludes correlations within the L space, \nand these are very effective in narrowing the band.\nInterestingly, the bottom of the occupied band is quite\nexactly at the LDA value, and the remaining widening is\npurely in the unoccupied part.\n\n\n\\begin{figure*}[htb]\n\\begin{center} \n\\includegraphics[width=0.9\\textwidth ]{fig7.eps} \n\\end{center} \n\\caption{(Color online)\nFrequency dependence of $-V_{xc}+\\Delta \\Sigma_{\\rm H}$ for SrVO$_3$.\n(a) and (b) are the real and imaginary parts at the $\\Gamma$ point, \nrespectively,\nand (c) and (d) ((e) and (f)) are those at $X$ ($R$).\nFor comparison, the full GW self-energies are also given\n(black dotted line).\n}\n\\label{sigH}\n\\end{figure*} \n\n\n\n\\begin{figure}[h]\n\\centering \n\\includegraphics[clip,width=0.45\\textwidth ]{fig8.eps} \n\\caption{(Color online)\nBand structure  of $Z_{\\rm H}[H_{\\rm LDA}-V_{\\rm xc}+\\Delta\\Sigma_{\\rm H}(\\omega=0)]$ of SrVO$_3$.\nFor comparison, the LDA band structure is also given\n(black dotted line).\nThe energy is measured from the Fermi level.\n}\n\\label{ZH_SigmaH}\n\\end{figure} \n\n\nTo go further and in particular to analyze the dynamic\nbehavior of $\\Delta \\Sigma_{\\rm H}$ \nwe plot in Fig.~\\ref{sigH} \nthe self-energy corrections $\\Delta \\Sigma_{\\rm H}$ (with $\\Sigma$ (see Eq.~(\\ref{full-self-energy}))\nfor comparison) for the real and imaginary parts\nat several representative choices of momenta.\nThe frequency dependence is smooth around the Fermi level.\nIn particular, in contrast to the full self-energy,\nthere are no poles in $\\Delta \\Sigma_{\\rm H}$ \nin the low-energy region, thanks to the\nexclusion of fluctuations within the L-space.\nThe frequency dependence of the real part indicates that the \nlinearization \n$\\displaystyle{\\Delta \\Sigma_{\\rm H}(\\omega) \\sim \\Delta \\Sigma_{\\rm H}(\\omega=0)+[d\\Delta \\Sigma_{\\rm H}(\\omega)/d\\omega]_{\\omega=0}\\omega }$\noffers a reasonably good approximation. \nIn fact, the behavior of the constrained self-energy\nis much closer to linearity than that of \nthe full GW self-energy $\\Sigma$.\nThis is easily understood by the fact that the low-energy excitations are excluded in the present constrained self-energy $\\Delta\\Sigma_{\\rm H}$ analogously to an insulator, thus eliminating the strong frequency dependence of the constrained self-energy near the Fermi level. This is one of the consequences of the \ncontrollability and an advantage of the present MACE scheme, as \nmentioned in the introduction.\nIn the following discussion, we employ this linearized approximation.\nThen the renormalization factor $Z_{\\rm H}$ defined by Eq.~(\\ref{ZH})\nis interpreted as that from the contribution of the H-space. \nAfter \nthis renormalization factor is taken into account,\nthe effective Hamiltonian is given by $Z_{\\rm H}[H_{\\rm LDA}-V_{\\rm xc}+\\Delta\\Sigma_{\\rm H}(\\omega=0)]$ (see Fig.~\\ref{ZH_SigmaH}). \nInterestingly the renormalization factor $Z_{\\rm H}$=0.92 stays\nclose to one, so that\nthe bandwidth is only slightly reduced (to\n$3.4$ eV instead of the $3.7$ eV above). \nWe also note that $Z_{\\rm H}$ has weak momentum dependence\nas we reveal in the following.\n\n\n\n\\begin{figure*}[htb]\n\\begin{center} \n\\includegraphics[width=0.9\\textwidth ]{fig9.eps} \n\\end{center} \n\\caption{(Color online)\n{\\bf minor modification}\nFrequency dependence of $-V_{\\rm xc}+\\Delta \\Sigma_{\\rm H}$ of SrVO$_3$.\n(a) and (b) are the real and imaginary parts of the local\ncomponent, respectively.\nThe black dotted line is $\\Sigma$ defined in Eq.~(\\ref{full-self-energy}) in the full GW approximation for comparison.\n(c) and (d) are the real and imaginary \nparts at several $k$ points.\n}\n\\label{sigH_nonlocal}\n\\end{figure*} \n\n\n\nFigure~\\ref{sigH_nonlocal} \nshows the local and nonlocal parts of $\\Delta \\Sigma_{\\rm H}$.\nThe frequency \ndependence\nof the local part (Figs.~\\ref{sigH_nonlocal} (a)(b)) \nis quite similar \nto the  frequency-dependent self-energies at various momenta\nin Fig.~\\ref{sigH}.\nOne immediately reads off an interesting property, which is\nakin to what has been found for the full GW self-energy in\nRef.~\\onlinecite{Tomczak-PRL2012}, namely that the nonlocal part of \n$\\Delta \\Sigma_{\\rm H}$ shows only weak frequency dependence \nas shown in Figs.~\\ref{sigH_nonlocal} (c)(d). \nThis explains why $Z_{\\rm H}$ \nis only weakly momentum-dependent.\nOn the other hand, $\\Delta \\Sigma_{\\rm H}^{\\rm nonlocal} (0)$ at $R$ is $\\sim 1$ eV larger than \nat $\\Gamma$, which causes the band widening effect.\n $\\Delta \\Sigma_{\\rm H}$\ncan thus to a good approximation be decomposed into a\nfrequency-dependent local part and a static nonlocal one:\n$\\Delta \\Sigma_{\\rm H}(k,\\omega) = \\Delta \\Sigma^{\\rm local}_{\\rm H}(\\omega) + \\Delta \\Sigma^{\\rm nonlocal}_{\\rm H}(k)$,\nwith $ \\Delta \\Sigma_{\\rm H}^{\\rm local}(\\omega) = \\sum_q  \\Delta \\Sigma_{\\rm H}(q,\\omega)$\n\nSince in the simple case of SrVO$_3$ where the t$_{2g}$ states are\ndegenerate, static local operators are scalar and are compensated \nby a chemical potential shift such that the correct particle\nnumber is obtained. The above band structure corresponding to\n$H_{\\rm LDA} - V_{\\rm xc} + \\Delta \\Sigma_{\\rm H}(k, \\omega=0)$ \nis thus identical to that of\n$H_{\\rm LDA} - V_{\\rm xc} + \\Delta \\Sigma^{\\rm nonlocal}_{\\rm H}(k)$.\nThe local dynamical part of $\\Delta \\Sigma_{\\rm H}$ then results in a \nnarrowing of this band structure by a factor $Z_{\\rm H} = 0.92$.\n\n\n\n\\begin{figure}[h]\n\\centering \n\\includegraphics[clip,width=0.45\\textwidth ]{fig10.eps} \n\\caption{(Color online)\n(a) Frequency dependence of the on-site effective interaction.\nPanel (b) is a low-energy zoom of panel (a). \n}\n\\label{W}\n\\end{figure} \n\n\nBefore turning to a discussion of the low-energy correction for the GW treatment\n$\\Delta \\Sigma_{\\rm L} =G^{(0)}(W-W_{\\rm L})$, we analyze the effective\ninteraction $W_{\\rm L}$ shown in Fig.~\\ref{W} \nin comparison to $W$ and $W_{\\rm H}$.\nThe fully screened Coulomb interaction $W$ displays the familiar\nshape of an interaction that is strongly screened at low energies\n(with a value of $0.88$ eV at $\\omega=0$), while retrieving the\nvalue of the bare Coulomb interaction $v$ ($\\sim 16.0$ eV) \nat high energy.\nThe crossover from the bare to the  \nscreened values takes place at the plasma energy of \nabout $15$ eV. This behavior has been discussed before\n\\cite{Review_SB2014}; we note in particular that the plasma \nfrequency is known from electron energy loss spectroscopy \nmeasurements of the related SrTiO$_3$ compound \\cite{vandermarel,kohiki}.\nThe partially screened interaction\n$W_{\\rm H}$ constructed within cRPA\nconverges to $W$ at high energies, but displays\nweaker screening effects at low energies (with a value of $3.5$ eV at $\\omega=0$),\nsince intra-t$_{2g}$ screening channels are excluded.\nAs was already anticipated in Sec.~\\ref{Outline},  $W_{\\rm L}$ can be interpreted as the screened interaction of \na low-energy model where a static interaction of value $W_{\\rm H}(\\omega=0)$\nhas been imposed as the bare interaction. Since now {\\it only} intra-t$_{2g}$\nscreening channels are active, screening takes place only at low\nenergies, where the scale is given by the t$_{2g}$ bandwidth.\nAlso plotted is the difference $W_{\\rm GW}^{\\rm dyn}=W-W_{d}$: except at low energies\nwhere the t$_{2g}$ screening channels come into play, its frequency\ndependence is essentially given by that of $W$, while the\nhigh-energy limit is the bare Coulomb interaction $v$ reduced\nby $W_{\\rm H}(0)$.\n\n\n\\begin{figure*}[htb]\n\\begin{center} \n\\includegraphics[width=0.9\\textwidth ]{fig11.eps} \n\\end{center} \n\\caption{(Color online)\nFrequency dependence of $-V_{\\rm xc}+ G^{(1)}W_{\\rm H}^{\\rm dyn}$ of SrVO$_3$.\n(a) and (b) are the real and imaginary local parts, respectively.\n(c) and (d) are the real and imaginary \nparts at several $k$ points.\n}\n\\label{DeltaSigma_L_Pert}\n\\end{figure*} \n\n\n\n\\begin{figure*}[htb]\n\\begin{center} \n\\includegraphics[width=0.9\\textwidth ]{fig12.eps} \n\\end{center} \n\\caption{(Color online)\nFrequency dependence of $-V_{\\rm xc}+ G^{(0)}W_{\\rm GW}^{\\rm dyn}$ of SrVO$_3$.\n(a) and (b) are the real and imaginary local parts, respectively.\n(c) and (d) are the real and imaginary \nparts at several $k$ points.\n}\n\\label{DeltaSigma_L_GW}\n\\end{figure*} \n\n\n\nThe frequency dependence of the real and imaginary parts of the low-energy self-energy $\\Delta \\Sigma_{\\rm L} = G^{(1)}W_{\\rm H}^{\\rm dyn}$ for the direct perturbative treatment is illustrated in Fig.~\\ref{DeltaSigma_L_Pert}. \n$\\Delta \\Sigma_{\\rm L} = G^{(0)}W_{\\rm GW}^{\\rm dyn}$ used\nfor the GW treatment is shown in Fig.~\\ref{DeltaSigma_L_GW}\nfor several choices of momenta.\n\\\\\nThe results for the perturbative and the GW treatment are nearly \nidentical.\nThe renormalization factor of $\\Delta \\Sigma_{\\rm L}^{\\rm nonlocal}$ \n(and $\\Delta \\Sigma_{\\rm L}$) is $\\sim 0.77$.\nThe zero-frequency shift\n${\\rm Re}\\Delta \\Sigma_{\\rm L}^{\\rm nonlocal} (0)$ at $R$ is $\\sim 3$ eV \nlarger than \nat $\\Gamma$.\nAgain, one sees how $\\Sigma_{\\rm L}$ separates into local dynamical\nand nonlocal static parts: \n$\\Delta \\Sigma_{\\rm L} = \\Delta \\Sigma^{\\rm nonlocal}_{\\rm L}(k) + \n\\Delta \\Sigma^{\\rm local}_{\\rm L}(\\omega)$.\nThis is expected since such a separation has been found within\nthe full GW calculation \\cite{TomczakPRB2014}, and is even\nmore plausible for a GW treatment within the t$_{2g}$ subspace.\n\n\n\n\\begin{figure}[h]\n\\centering \n\\includegraphics[clip,width=0.45\\textwidth ]{fig13.eps} \n\\caption{(Color online)\nBand structure  of $[\\epsilon_{\\rm DFT}-V_{\\rm xc}+\\Delta\\Sigma_{\\rm H}+\\Delta \\Sigma_{\\rm L}^{\\rm nonlocal}]Z_{\\rm HL}$ of SrVO$_3$ in the direct perturbative treatment.\nFor comparison, the LDA band structure is also given\n(black dotted line).\nThe energy is measured from the Fermi level.\n}\n\\label{DFT+Sigma_HL_Pert}\n\\end{figure} \n\n\n\n\\begin{figure}[h]\n\\centering \n\\includegraphics[clip,width=0.45\\textwidth ]{fig14.eps} \n\\caption{(Color online)\nBand structure  of $[\\epsilon_{\\rm DFT}-V_{\\rm xc}+\\Delta\\Sigma_{\\rm H}+\\Delta \\Sigma_{\\rm L}^{\\rm nonlocal}]Z_{\\rm HL}$ of SrVO$_3$ in the GW treatment.\nFor comparison, the LDA band structure is also given\n(black dotted line).\nThe energy is measured from the Fermi level.\n}\n\\label{DFT+Sigma_HL_GW}\n\\end{figure} \n\n\n\nThe corresponding band structures \nare given by \n $[\\epsilon_{\\rm DFT}+\\Delta\\Sigma_{\\rm H}+\\Delta \\Sigma_{\\rm L}^{\\rm nonlocal}]Z_{\\rm HL}$ in Fig.~\\ref{DFT+Sigma_HL_Pert} \nfor the perturbative treatment (see Eq.~(\\ref{DeltaSigmaLnonlocalPert})), and in Fig.~\\ref{DFT+Sigma_HL_GW} \nfor the GW treatment,\nwhere $\\Delta \\Sigma_{\\rm L}^{\\rm nonlocal}$ is defined in Eq.~(\\ref{DeltaSigLnonlocalGW}) and $Z_{\\rm HL}$ is around 0.92.\n$Z_{\\rm HL}$ is practically the same as $Z_{\\rm H}$, meaning that the nonlocal dynamical correction \nin the H-space is small, which justifies the GW perturbative treatment for the nonlocal part.\nOne finds that \nthe nonlocal part $\\Delta \\Sigma_{\\rm L}=G^{(0)}_{ll}(W-W_{\\rm L})$ narrows\nthe empty states but widens the occupied ones, resulting\nin an overall bandwidth of 3.2 eV.\nThe difference between the direct perturbation and the GW treatment is small.\n\n\n\n\\begin{figure}[h]\n\\centering \n\\includegraphics[clip,width=0.45\\textwidth ]{fig15.eps} \n\\caption{(Color online)\nBand structure  of $[\\epsilon_{\\rm DFT}-V_{\\rm xc}+\\Delta\\Sigma_{\\rm H}+\\Delta\\Sigma_{\\rm L}^{\\rm nonlocal}]Z_{\\rm HL}Z_B$ of SrVO$_3$ in the GW treatment.\nFor comparison, the LDA band structure is also given\n(black dotted line).\nThe energy is measured from the Fermi level.}\n\\label{DFT+DeltaSigmaL_GW_ZHL_ZB}\n\\end{figure} \n\n\nAfter including the effect of the local self-energy by the nonperturbative \nCasula trick, we show the dispersion\ngiven by \n$[\\epsilon_{\\rm DFT}-V_{\\rm xc}+\\Delta\\Sigma_{\\rm H}+\\Delta\\Sigma_{\\rm L}^{\\rm nonlocal}]Z_{\\rm HL}Z_B$ for the GW-type treatment in Fig.~\\ref{DFT+DeltaSigmaL_GW_ZHL_ZB}: \nThe GW-like treatment gives a band dispersion\nclose to the LDA one. \nThe $Z_B$ factor corresponding to the local dynamical part of \n$\\Delta \\Sigma_{\\rm L}$ amounts to $Z_B=0.7$.\nThe nonperturbative treatment \nresults in an LDA-like band dispersion for the empty states,\nbut a slightly narrower bandwidth in the occupied part.\nThe low-energy effective Hamiltonian at this level of treatment has a \nfrequency independent effective interaction both with local \nand nonlocal interaction given by the Fourier transform of \n$W_{\\rm H}(q,\\omega=0)$, which contains both direct and exchange interactions. \n\n\n\\begin{figure}[h]\n\\centering \n\\includegraphics[clip,width=0.45\\textwidth ]{fig16.eps} \n\\caption{(Color online)\nBand structure  of $[\\epsilon_{\\rm DFT}+\\Sigma_{\\rm corr}^{\\rm GW}]Z_{\\rm corr}Z_B$ of SrVO$_3$.\nFor comparison, the LDA band structure is also given\n(black dotted line).\nThe energy is measured from the Fermi level.}\n\\label{DFT+Sigma_corr_GW_Z}\n\\end{figure} \n\n\n\nWe finally show the dispersion\ngiven by Eq.~(\\ref{Teff_final}), namely\n$[\\epsilon_{\\rm DFT}+\\Sigma_{\\rm corr}^{\\rm GW}]Z_{\\rm corr}Z_B$ in Fig.~\\ref{DFT+Sigma_corr_GW_Z}. \nThe result shows an LDA-like band dispersion for the occupied states,\nbut a slightly wider bandwidth in the empty part, resulting in a bandwidth 14\\% wider in total than the LDA bandwidth. \nThe effective interaction of the low-energy effective Hamiltonian \nat this final level has\nfrequency independent local and nonlocal interactions \ngiven by the Fourier transform of $W_r(q,\\omega=0)$. \n\n\n\n\\begin{figure}[h]\n\\centering \n\\includegraphics[clip,width=0.45\\textwidth ]{fig17.eps} \n\\caption{(Color online)\nBand structure  of $[\\epsilon_{\\rm DFT}-V_{\\rm xc}+\\Sigma-(G^{(0)}W_{\\rm GW}^{\\rm dyn})^{\\rm nonlocal}]Z'Z_B$ of SrVO$_3$\n(Z' is the renormalization factor of $\\Sigma-(G^{(0)}W_{\\rm GW}^{\\rm dyn})^{\\rm nonlocal}$).\nFor comparison, the LDA band structure is also given\n(black dotted line).\nThe energy is measured from the Fermi level.\n}\n\\label{GW+ZB}\n\\end{figure} \n\n\n\nThe resulting effective Hamiltonian in the L-space \nconsists of single-particle and two-particle (interaction) parts.\nThe effective interaction is the same as in previous estimates \nby the cRPA in the literature, while the single-particle dispersion \nis revised after removing the double counting and taking into \naccount the frequency dependence of the effective interaction. The \nfinal effective bandwidth for SrVO$_3$ is,\nafter partial cancellation, \nslightly (14\\%) larger than the LDA bandwidth.  \nThis is a reasonable result because the LDA takes into account all \ncorrelations though insufficiently, while the present scheme by the constrained self-energy excludes the \ncorrelation effects arising from the L-space.\nAlthough the bandwidth derived for the effective model is slightly \nlarger than that of the LDA,  it is clear that the correlation effects \nare stronger than in the LDA or standard GW when the {\\it ab initio} \nmodel is solved by an accurate solver. In fact, if the effective \ninteraction contained in the final effective model is treated by \nthe GW scheme, one obtains the dispersion illustrated in \nFig.~\\ref{GW+ZB}, \nwhich is given from the self energy of the whole GW calculation $\\Sigma$ \nby correcting the local dynamical part $W-W_{\\rm L}$ by $Z_B$. \nThis indicates that even an insufficient treatment of the local \nstatic interaction by the GW scheme gives a dispersion with the bandwidth ($\\sim 1.9$ eV) narrower \nthan the LDA ($\\sim 2.5$ eV, Fig.~\\ref{lda}) and GW ($\\sim 2.1$ eV, Fig.~\\ref{fullGW}) dispersions.  \nA slightly ($\\sim 14$\\%) wider dispersion than that of the LDA bands obtained for the \neffective Hamiltonian accompanied with frequency independent effective \nCoulomb interactions without the exchange part may account for the slight \noverestimate of correlation effects in the literature mentioned in \nthe introduction.\nOur scheme offers an optimized way for the  derivation of \n{\\it ab initio} models for the L-space after eliminating the H-space\nin  a systematic fashion.\n\n\nOur findings of the band widening are consistent with studies\nbased on the combined GW+DMFT method in the literature \n\\cite{TomczakPRB2014}: There, it was argued that within GW+DMFT\nthe best effective Hamiltonian that DMFT should be performed on,\nis a one-body Hamiltonian corrected by the nonlocal part of the\nGW self-energy. The corresponding spectral function \n(see e.g.\nFigure 5 of Ref.~\\onlinecite{TomczakPRB2014}) displays \na broadening similar to \nFig.~\\ref{DFT+Sigma_HL_GW}.\nMost interestingly, our present calculations confirm a pronounced\nasymmetry observed in Ref.~\\onlinecite{TomczakPRB2014}, namely a stronger widening\neffect in the unoccupied part of the spectrum than in the occupied\none.\n\n",
        "label": "DeltaSigma_L_Pert",
        "Descriptive_question1": "What does panel (a) in figure_11 represent?",
        "Descriptive_question2": "What does panel (c) in figure_11 show?",
        "Reasoning_question1": "Why might the real part of the local component in figure_11 (a) show a specific frequency dependence compared to the imaginary part in (b)?",
        "Reasoning_question2": "How does the variation in the real and imaginary parts at different k points in figure_11 (c) and (d) reflect the material properties of SrVO3?",
        "Descriptive_answer1": "Real local part",
        "Descriptive_answer2": "Real parts at several k points",
        "Reasoning_answer1": "The real part in panel (a) represents the dispersive component of the self-energy correction and typically shows systematic frequency dependence reflecting changes in effective electron interactions; in contrast, the imaginary part in (b) represents scattering and lifetime effects, which vary differently with frequency, leading to distinct patterns in their frequency dependence.",
        "Reasoning_answer2": "The variation of the real and imaginary parts at different k points in (c) and (d) indicates momentum-dependent electronic interactions in SrVO3; such dispersion reflects how electronic correlations and screening differ across the Brillouin zone, which affects quasiparticle lifetimes and band renormalization, highlighting the anisotropic nature of the electronic structure in SrVO3."
    },
    {
        "paper_id": "1511.03757.json",
        "image_id": "figure_12",
        "image_path": [
            "/home/yz979/scratch/chengye/scimolmo/test-images-tables/dataset_imagestables/1511.03757/images/fig12.eps"
        ],
        "caption": "(Color online)\nFrequency dependence of $-V_{\\rm xc}+ G^{(0)}W_{\\rm GW}^{\\rm dyn}$ of SrVO$_3$.\n(a) and (b) are the real and imaginary local parts, respectively.\n(c) and (d) are the real and imaginary \nparts at several $k$ points.\n",
        "classify": "Chart",
        "section_info": "6 Results\n\\section{Results}\n\\label{results}\n\n\nThe ternary 3d$^1$ transition metal oxide SrVO$_3$ has become\none of the ``drosophila compounds'' of correlated systems.\nIt is a correlated metal\nthat has been characterized using various experimental \n\\cite{Onoda, inoue, maiti, maiti2, sekiyama, takizawa, yoshida2010}.\nand theoretical techniques, see e.g.\n\\cite{nekrasov, anisimov, pavarini-d1, liebsch, lechermann-d1, trimarchi, aichhorn09, karolak, lee, held-optics, tomczak-gwdmft, gatti-guzzo, sakuma14, TomczakPRB2014}.\nA review of most of the available experimental and theoretical \ndata has been given recently in Ref.~\\onlinecite{TomczakPRB2014}.\nSrVO$_3$ displays Fermi liquid behavior up to\nremarkably high temperatures of the order of $200$ K \\cite{Onoda, inoue},\nwith a moderate mass enhancement of the order \nof $2$ \\cite{yoshida2010, vandermarel}.\nDetailed spectroscopic investigations have made it an\nideal test compound for modern many-body calculations, and \nmore and more refined dynamical mean field-based studies \nare available.\nThe majority of studies so far have focused on the t$_{2g}$-manifold\nwhich forms the states close to the Fermi level, and those will\nalso be the focus in the present investigations. \nNote however that this restriction quite severely limits the\nrange of validity of the low-energy description. Indeed, as\nshown recently~\\cite{tomczak-gwdmft, TomczakPRB2014} at\nenergies of about 2 to 3 eV above the Fermi level\nthe spectral properties are largely determined by the e$_g$\nstates. This has in particular led to a reinterpretation of\nan inverse photoemission feature at about 2.5 eV that was \nfrequently interpreted as an upper Hubbard band of t$_{2g}$\ncharacter in the earlier literature.\nHere, we use the compound only as an illustration of \nthe principles of constructing effective low-energy models,\nwithout being concerned with a description of spectra beyond\nthe pure t$_{2g}$ part.\n\n\nThe LDA band structure of SrVO$_3$ is shown in Fig.~\\ref{lda}.\nOne clearly distinguishes the three-fold degenerate manifold\nof t$_{2g}$ bands close to the Fermi level (chosen as the\nzero of energy), followed in the unoccupied part of the spectrum\nby the two e$_g$ bands. At about -2.5 eV the filled O-2p derived\nbands are visible.\n\nWhen a standard GW calculation is performed, see Fig.~\\ref{fullGW},\nthe bandwidth of the t$_{2g}$-states is reduced from 2.5 eV to\n2.1 eV, while the overall shape of the dispersion remains similar\nto the LDA one.\nThis is in agreement with previous GW calculations in the literature\n\\cite{tomczak-gwdmft, TomczakPRB2014, gatti-guzzo, sakuma14}.\n\n\n\nThe calculation is based on the full-potential linear muffin-tin orbitals (FP-LMTO) implementation.\n8$\\times$8$\\times$8 \\bm{k}-mesh is employed in both the DFT/LDA and the GW calculations.\nThe calculational details are the same as in Ref.~\\onlinecite{Hirayama}.\n\n\nWe now turn to a discussion of the band structure corresponding to \nthe Hamiltonians to be used as input for subsequent many-body \ncalculations for this material.\nWe will proceed step by step to analyze the influence of the various\ncorrection terms, with respect to the DFT starting point\n(which will be overlaid to the respective band structures).\n\n\n\n\\begin{figure}[h]\n\\centering \n\\includegraphics[clip,width=0.45\\textwidth ]{fig2.eps} \n\\caption{(Color online)\nKohn-Sham band structure of SrVO$_3$ in the LDA.\nThe energy is measured from the Fermi level.\n}\n\\label{lda}\n\\end{figure} \n\n\n\\begin{figure}[h]\n\\centering \n\\includegraphics[clip,width=0.45\\textwidth ]{fig3.eps} \n\\caption{(Color online)\nBand structure  of SrVO$_3$ in the one-shot GW approximation.\nFor comparison, the LDA band structure is also given\n(black dotted line).\nThe energy is measured from the Fermi level.\n}\n\\label{fullGW}\n\\end{figure} \n\n\n\n\n\n\n\\begin{figure}[h]\n\\centering \n\\includegraphics[clip,width=0.45\\textwidth ]{fig4.eps} \n\\caption{(Color online)\nBand structure of SrVO$_3$ corresponding to the LDA Hamiltonian\nfrom which the LDA exchange-correlation potential has been subtracted.\nFor comparison, the LDA band structure is also given\n(black dotted line).\nThe energy is measured from the Fermi level.\n}\n\\label{lda-vxc}\n\\end{figure} \n\n\n\\begin{figure}[h]\n\\centering \n\\includegraphics[clip,width=0.45\\textwidth ]{fig5.eps} \n\\caption{(Color online)\nBand structure  of SrVO$_3$ in the Hartree approximation.\nFor comparison, the LDA band structure is also given\n(black dotted line).\nThe energy is measured from the Fermi level.\n}\n\\label{hartree}\n\\end{figure} \n\n\nFigure~\\ref{lda-vxc} \ndisplays the band structure corresponding to\nthe LDA Hamiltonian from which the \nLDA exchange-correlation\npotential has been subtracted,  \nthat is, the eigenvalues of $H_{\\rm LDA}-V_{\\rm xc}$ where\nthese operators are evaluated for the self-consistent LDA\ndensity.\nThe subtraction of $V_{\\rm xc}$ widens the band structure from \nthe LDA bandwidth of 2.5 eV \nto more than twice this value: the new band\nwidth is 5.4 eV.\nThis indicates that the exchange-correlation potential $V_{\\rm xc}$ \nof the LDA is responsible for a substantial amount of \nthe band-narrowing effect\narising from electronic correlations, stressing the need \nto subtract the double counting in a consistent manner. \nFigure~\\ref{lda-vxc} is regarded as the starting point for the\nfollowing considerations.\n\nFor simple materials (see e.g. the calculations on Li in Ref.~\\onlinecite{Lithium})\nit has been noted in the literature that the Hartree band structure\nis close to the DFT one. This raises the question of the origin of the\nsubstantial band widening in the present case. Indeed, the present\ncalculation shown in Fig.~\\ref{lda-vxc} differs from a Hartree calculation\nonly \nby the fact that Fig.~\\ref{lda-vxc} is evaluated for the converged LDA density.\nWe have performed a test calculation where we plot the Hartree band\nstructure calculated for the converged Hartree density. The result\nis shown in Fig.~\\ref{hartree}. As seen from this plot, while the\nband is not fully as wide as in Fig.~\\ref{lda-vxc}, a substantial \nwidening is already present at this stage.\n\n\n\n\\begin{figure}[h]\n\\centering \n\\includegraphics[clip,width=0.45\\textwidth ]{fig6.eps} \n\\caption{(Color online)\nBand structure  of $H_{\\rm LDA}-V_{\\rm xc}+\\Delta \\Sigma_{\\rm H}(\\omega=0)$ of SrVO$_3$.\nFor comparison, the LDA band structure is also given\n(black dotted line).\nThe energy is measured from the Fermi level.\n}\n\\label{lda-vxc-sigH(0)}\n\\end{figure} \n\n\n\nStarting from the Hamiltonian without V$_{xc}$ (see the \ndispersion in Fig.~\\ref{lda-vxc}), we first take into account \nthe static part of the corrective self-energy $\\Delta \\Sigma_{\\rm H}$:\nIn Fig.~\\ref{lda-vxc-sigH(0)}, we plot the dispersion corresponding to\n$H_{\\rm LDA}-V_{\\rm xc}+\\Delta \\Sigma_{\\rm H}(\\omega=0)$. \nCompared to the dispersion of Fig.~\\ref{lda-vxc}, the overall band structure \nis narrowed to 3.7 eV. This value is, however, still considerably larger\nthan the LDA bandwidth (Fig.~\\ref{lda}). Although\n the electronic correlations coupling the H- and \nthe L space are included in Fig.~\\ref{lda-vxc-sigH(0)}, and narrow the band with respect to the case\nwhere V$_{xc}$ is taken out, correlations within\nthe L space are not included. \nThe LDA, on the other hand, at least partially\nincludes correlations within the L space, \nand these are very effective in narrowing the band.\nInterestingly, the bottom of the occupied band is quite\nexactly at the LDA value, and the remaining widening is\npurely in the unoccupied part.\n\n\n\\begin{figure*}[htb]\n\\begin{center} \n\\includegraphics[width=0.9\\textwidth ]{fig7.eps} \n\\end{center} \n\\caption{(Color online)\nFrequency dependence of $-V_{xc}+\\Delta \\Sigma_{\\rm H}$ for SrVO$_3$.\n(a) and (b) are the real and imaginary parts at the $\\Gamma$ point, \nrespectively,\nand (c) and (d) ((e) and (f)) are those at $X$ ($R$).\nFor comparison, the full GW self-energies are also given\n(black dotted line).\n}\n\\label{sigH}\n\\end{figure*} \n\n\n\n\\begin{figure}[h]\n\\centering \n\\includegraphics[clip,width=0.45\\textwidth ]{fig8.eps} \n\\caption{(Color online)\nBand structure  of $Z_{\\rm H}[H_{\\rm LDA}-V_{\\rm xc}+\\Delta\\Sigma_{\\rm H}(\\omega=0)]$ of SrVO$_3$.\nFor comparison, the LDA band structure is also given\n(black dotted line).\nThe energy is measured from the Fermi level.\n}\n\\label{ZH_SigmaH}\n\\end{figure} \n\n\nTo go further and in particular to analyze the dynamic\nbehavior of $\\Delta \\Sigma_{\\rm H}$ \nwe plot in Fig.~\\ref{sigH} \nthe self-energy corrections $\\Delta \\Sigma_{\\rm H}$ (with $\\Sigma$ (see Eq.~(\\ref{full-self-energy}))\nfor comparison) for the real and imaginary parts\nat several representative choices of momenta.\nThe frequency dependence is smooth around the Fermi level.\nIn particular, in contrast to the full self-energy,\nthere are no poles in $\\Delta \\Sigma_{\\rm H}$ \nin the low-energy region, thanks to the\nexclusion of fluctuations within the L-space.\nThe frequency dependence of the real part indicates that the \nlinearization \n$\\displaystyle{\\Delta \\Sigma_{\\rm H}(\\omega) \\sim \\Delta \\Sigma_{\\rm H}(\\omega=0)+[d\\Delta \\Sigma_{\\rm H}(\\omega)/d\\omega]_{\\omega=0}\\omega }$\noffers a reasonably good approximation. \nIn fact, the behavior of the constrained self-energy\nis much closer to linearity than that of \nthe full GW self-energy $\\Sigma$.\nThis is easily understood by the fact that the low-energy excitations are excluded in the present constrained self-energy $\\Delta\\Sigma_{\\rm H}$ analogously to an insulator, thus eliminating the strong frequency dependence of the constrained self-energy near the Fermi level. This is one of the consequences of the \ncontrollability and an advantage of the present MACE scheme, as \nmentioned in the introduction.\nIn the following discussion, we employ this linearized approximation.\nThen the renormalization factor $Z_{\\rm H}$ defined by Eq.~(\\ref{ZH})\nis interpreted as that from the contribution of the H-space. \nAfter \nthis renormalization factor is taken into account,\nthe effective Hamiltonian is given by $Z_{\\rm H}[H_{\\rm LDA}-V_{\\rm xc}+\\Delta\\Sigma_{\\rm H}(\\omega=0)]$ (see Fig.~\\ref{ZH_SigmaH}). \nInterestingly the renormalization factor $Z_{\\rm H}$=0.92 stays\nclose to one, so that\nthe bandwidth is only slightly reduced (to\n$3.4$ eV instead of the $3.7$ eV above). \nWe also note that $Z_{\\rm H}$ has weak momentum dependence\nas we reveal in the following.\n\n\n\n\\begin{figure*}[htb]\n\\begin{center} \n\\includegraphics[width=0.9\\textwidth ]{fig9.eps} \n\\end{center} \n\\caption{(Color online)\n{\\bf minor modification}\nFrequency dependence of $-V_{\\rm xc}+\\Delta \\Sigma_{\\rm H}$ of SrVO$_3$.\n(a) and (b) are the real and imaginary parts of the local\ncomponent, respectively.\nThe black dotted line is $\\Sigma$ defined in Eq.~(\\ref{full-self-energy}) in the full GW approximation for comparison.\n(c) and (d) are the real and imaginary \nparts at several $k$ points.\n}\n\\label{sigH_nonlocal}\n\\end{figure*} \n\n\n\nFigure~\\ref{sigH_nonlocal} \nshows the local and nonlocal parts of $\\Delta \\Sigma_{\\rm H}$.\nThe frequency \ndependence\nof the local part (Figs.~\\ref{sigH_nonlocal} (a)(b)) \nis quite similar \nto the  frequency-dependent self-energies at various momenta\nin Fig.~\\ref{sigH}.\nOne immediately reads off an interesting property, which is\nakin to what has been found for the full GW self-energy in\nRef.~\\onlinecite{Tomczak-PRL2012}, namely that the nonlocal part of \n$\\Delta \\Sigma_{\\rm H}$ shows only weak frequency dependence \nas shown in Figs.~\\ref{sigH_nonlocal} (c)(d). \nThis explains why $Z_{\\rm H}$ \nis only weakly momentum-dependent.\nOn the other hand, $\\Delta \\Sigma_{\\rm H}^{\\rm nonlocal} (0)$ at $R$ is $\\sim 1$ eV larger than \nat $\\Gamma$, which causes the band widening effect.\n $\\Delta \\Sigma_{\\rm H}$\ncan thus to a good approximation be decomposed into a\nfrequency-dependent local part and a static nonlocal one:\n$\\Delta \\Sigma_{\\rm H}(k,\\omega) = \\Delta \\Sigma^{\\rm local}_{\\rm H}(\\omega) + \\Delta \\Sigma^{\\rm nonlocal}_{\\rm H}(k)$,\nwith $ \\Delta \\Sigma_{\\rm H}^{\\rm local}(\\omega) = \\sum_q  \\Delta \\Sigma_{\\rm H}(q,\\omega)$\n\nSince in the simple case of SrVO$_3$ where the t$_{2g}$ states are\ndegenerate, static local operators are scalar and are compensated \nby a chemical potential shift such that the correct particle\nnumber is obtained. The above band structure corresponding to\n$H_{\\rm LDA} - V_{\\rm xc} + \\Delta \\Sigma_{\\rm H}(k, \\omega=0)$ \nis thus identical to that of\n$H_{\\rm LDA} - V_{\\rm xc} + \\Delta \\Sigma^{\\rm nonlocal}_{\\rm H}(k)$.\nThe local dynamical part of $\\Delta \\Sigma_{\\rm H}$ then results in a \nnarrowing of this band structure by a factor $Z_{\\rm H} = 0.92$.\n\n\n\n\\begin{figure}[h]\n\\centering \n\\includegraphics[clip,width=0.45\\textwidth ]{fig10.eps} \n\\caption{(Color online)\n(a) Frequency dependence of the on-site effective interaction.\nPanel (b) is a low-energy zoom of panel (a). \n}\n\\label{W}\n\\end{figure} \n\n\nBefore turning to a discussion of the low-energy correction for the GW treatment\n$\\Delta \\Sigma_{\\rm L} =G^{(0)}(W-W_{\\rm L})$, we analyze the effective\ninteraction $W_{\\rm L}$ shown in Fig.~\\ref{W} \nin comparison to $W$ and $W_{\\rm H}$.\nThe fully screened Coulomb interaction $W$ displays the familiar\nshape of an interaction that is strongly screened at low energies\n(with a value of $0.88$ eV at $\\omega=0$), while retrieving the\nvalue of the bare Coulomb interaction $v$ ($\\sim 16.0$ eV) \nat high energy.\nThe crossover from the bare to the  \nscreened values takes place at the plasma energy of \nabout $15$ eV. This behavior has been discussed before\n\\cite{Review_SB2014}; we note in particular that the plasma \nfrequency is known from electron energy loss spectroscopy \nmeasurements of the related SrTiO$_3$ compound \\cite{vandermarel,kohiki}.\nThe partially screened interaction\n$W_{\\rm H}$ constructed within cRPA\nconverges to $W$ at high energies, but displays\nweaker screening effects at low energies (with a value of $3.5$ eV at $\\omega=0$),\nsince intra-t$_{2g}$ screening channels are excluded.\nAs was already anticipated in Sec.~\\ref{Outline},  $W_{\\rm L}$ can be interpreted as the screened interaction of \na low-energy model where a static interaction of value $W_{\\rm H}(\\omega=0)$\nhas been imposed as the bare interaction. Since now {\\it only} intra-t$_{2g}$\nscreening channels are active, screening takes place only at low\nenergies, where the scale is given by the t$_{2g}$ bandwidth.\nAlso plotted is the difference $W_{\\rm GW}^{\\rm dyn}=W-W_{d}$: except at low energies\nwhere the t$_{2g}$ screening channels come into play, its frequency\ndependence is essentially given by that of $W$, while the\nhigh-energy limit is the bare Coulomb interaction $v$ reduced\nby $W_{\\rm H}(0)$.\n\n\n\\begin{figure*}[htb]\n\\begin{center} \n\\includegraphics[width=0.9\\textwidth ]{fig11.eps} \n\\end{center} \n\\caption{(Color online)\nFrequency dependence of $-V_{\\rm xc}+ G^{(1)}W_{\\rm H}^{\\rm dyn}$ of SrVO$_3$.\n(a) and (b) are the real and imaginary local parts, respectively.\n(c) and (d) are the real and imaginary \nparts at several $k$ points.\n}\n\\label{DeltaSigma_L_Pert}\n\\end{figure*} \n\n\n\n\\begin{figure*}[htb]\n\\begin{center} \n\\includegraphics[width=0.9\\textwidth ]{fig12.eps} \n\\end{center} \n\\caption{(Color online)\nFrequency dependence of $-V_{\\rm xc}+ G^{(0)}W_{\\rm GW}^{\\rm dyn}$ of SrVO$_3$.\n(a) and (b) are the real and imaginary local parts, respectively.\n(c) and (d) are the real and imaginary \nparts at several $k$ points.\n}\n\\label{DeltaSigma_L_GW}\n\\end{figure*} \n\n\n\nThe frequency dependence of the real and imaginary parts of the low-energy self-energy $\\Delta \\Sigma_{\\rm L} = G^{(1)}W_{\\rm H}^{\\rm dyn}$ for the direct perturbative treatment is illustrated in Fig.~\\ref{DeltaSigma_L_Pert}. \n$\\Delta \\Sigma_{\\rm L} = G^{(0)}W_{\\rm GW}^{\\rm dyn}$ used\nfor the GW treatment is shown in Fig.~\\ref{DeltaSigma_L_GW}\nfor several choices of momenta.\n\\\\\nThe results for the perturbative and the GW treatment are nearly \nidentical.\nThe renormalization factor of $\\Delta \\Sigma_{\\rm L}^{\\rm nonlocal}$ \n(and $\\Delta \\Sigma_{\\rm L}$) is $\\sim 0.77$.\nThe zero-frequency shift\n${\\rm Re}\\Delta \\Sigma_{\\rm L}^{\\rm nonlocal} (0)$ at $R$ is $\\sim 3$ eV \nlarger than \nat $\\Gamma$.\nAgain, one sees how $\\Sigma_{\\rm L}$ separates into local dynamical\nand nonlocal static parts: \n$\\Delta \\Sigma_{\\rm L} = \\Delta \\Sigma^{\\rm nonlocal}_{\\rm L}(k) + \n\\Delta \\Sigma^{\\rm local}_{\\rm L}(\\omega)$.\nThis is expected since such a separation has been found within\nthe full GW calculation \\cite{TomczakPRB2014}, and is even\nmore plausible for a GW treatment within the t$_{2g}$ subspace.\n\n\n\n\\begin{figure}[h]\n\\centering \n\\includegraphics[clip,width=0.45\\textwidth ]{fig13.eps} \n\\caption{(Color online)\nBand structure  of $[\\epsilon_{\\rm DFT}-V_{\\rm xc}+\\Delta\\Sigma_{\\rm H}+\\Delta \\Sigma_{\\rm L}^{\\rm nonlocal}]Z_{\\rm HL}$ of SrVO$_3$ in the direct perturbative treatment.\nFor comparison, the LDA band structure is also given\n(black dotted line).\nThe energy is measured from the Fermi level.\n}\n\\label{DFT+Sigma_HL_Pert}\n\\end{figure} \n\n\n\n\\begin{figure}[h]\n\\centering \n\\includegraphics[clip,width=0.45\\textwidth ]{fig14.eps} \n\\caption{(Color online)\nBand structure  of $[\\epsilon_{\\rm DFT}-V_{\\rm xc}+\\Delta\\Sigma_{\\rm H}+\\Delta \\Sigma_{\\rm L}^{\\rm nonlocal}]Z_{\\rm HL}$ of SrVO$_3$ in the GW treatment.\nFor comparison, the LDA band structure is also given\n(black dotted line).\nThe energy is measured from the Fermi level.\n}\n\\label{DFT+Sigma_HL_GW}\n\\end{figure} \n\n\n\nThe corresponding band structures \nare given by \n $[\\epsilon_{\\rm DFT}+\\Delta\\Sigma_{\\rm H}+\\Delta \\Sigma_{\\rm L}^{\\rm nonlocal}]Z_{\\rm HL}$ in Fig.~\\ref{DFT+Sigma_HL_Pert} \nfor the perturbative treatment (see Eq.~(\\ref{DeltaSigmaLnonlocalPert})), and in Fig.~\\ref{DFT+Sigma_HL_GW} \nfor the GW treatment,\nwhere $\\Delta \\Sigma_{\\rm L}^{\\rm nonlocal}$ is defined in Eq.~(\\ref{DeltaSigLnonlocalGW}) and $Z_{\\rm HL}$ is around 0.92.\n$Z_{\\rm HL}$ is practically the same as $Z_{\\rm H}$, meaning that the nonlocal dynamical correction \nin the H-space is small, which justifies the GW perturbative treatment for the nonlocal part.\nOne finds that \nthe nonlocal part $\\Delta \\Sigma_{\\rm L}=G^{(0)}_{ll}(W-W_{\\rm L})$ narrows\nthe empty states but widens the occupied ones, resulting\nin an overall bandwidth of 3.2 eV.\nThe difference between the direct perturbation and the GW treatment is small.\n\n\n\n\\begin{figure}[h]\n\\centering \n\\includegraphics[clip,width=0.45\\textwidth ]{fig15.eps} \n\\caption{(Color online)\nBand structure  of $[\\epsilon_{\\rm DFT}-V_{\\rm xc}+\\Delta\\Sigma_{\\rm H}+\\Delta\\Sigma_{\\rm L}^{\\rm nonlocal}]Z_{\\rm HL}Z_B$ of SrVO$_3$ in the GW treatment.\nFor comparison, the LDA band structure is also given\n(black dotted line).\nThe energy is measured from the Fermi level.}\n\\label{DFT+DeltaSigmaL_GW_ZHL_ZB}\n\\end{figure} \n\n\nAfter including the effect of the local self-energy by the nonperturbative \nCasula trick, we show the dispersion\ngiven by \n$[\\epsilon_{\\rm DFT}-V_{\\rm xc}+\\Delta\\Sigma_{\\rm H}+\\Delta\\Sigma_{\\rm L}^{\\rm nonlocal}]Z_{\\rm HL}Z_B$ for the GW-type treatment in Fig.~\\ref{DFT+DeltaSigmaL_GW_ZHL_ZB}: \nThe GW-like treatment gives a band dispersion\nclose to the LDA one. \nThe $Z_B$ factor corresponding to the local dynamical part of \n$\\Delta \\Sigma_{\\rm L}$ amounts to $Z_B=0.7$.\nThe nonperturbative treatment \nresults in an LDA-like band dispersion for the empty states,\nbut a slightly narrower bandwidth in the occupied part.\nThe low-energy effective Hamiltonian at this level of treatment has a \nfrequency independent effective interaction both with local \nand nonlocal interaction given by the Fourier transform of \n$W_{\\rm H}(q,\\omega=0)$, which contains both direct and exchange interactions. \n\n\n\\begin{figure}[h]\n\\centering \n\\includegraphics[clip,width=0.45\\textwidth ]{fig16.eps} \n\\caption{(Color online)\nBand structure  of $[\\epsilon_{\\rm DFT}+\\Sigma_{\\rm corr}^{\\rm GW}]Z_{\\rm corr}Z_B$ of SrVO$_3$.\nFor comparison, the LDA band structure is also given\n(black dotted line).\nThe energy is measured from the Fermi level.}\n\\label{DFT+Sigma_corr_GW_Z}\n\\end{figure} \n\n\n\nWe finally show the dispersion\ngiven by Eq.~(\\ref{Teff_final}), namely\n$[\\epsilon_{\\rm DFT}+\\Sigma_{\\rm corr}^{\\rm GW}]Z_{\\rm corr}Z_B$ in Fig.~\\ref{DFT+Sigma_corr_GW_Z}. \nThe result shows an LDA-like band dispersion for the occupied states,\nbut a slightly wider bandwidth in the empty part, resulting in a bandwidth 14\\% wider in total than the LDA bandwidth. \nThe effective interaction of the low-energy effective Hamiltonian \nat this final level has\nfrequency independent local and nonlocal interactions \ngiven by the Fourier transform of $W_r(q,\\omega=0)$. \n\n\n\n\\begin{figure}[h]\n\\centering \n\\includegraphics[clip,width=0.45\\textwidth ]{fig17.eps} \n\\caption{(Color online)\nBand structure  of $[\\epsilon_{\\rm DFT}-V_{\\rm xc}+\\Sigma-(G^{(0)}W_{\\rm GW}^{\\rm dyn})^{\\rm nonlocal}]Z'Z_B$ of SrVO$_3$\n(Z' is the renormalization factor of $\\Sigma-(G^{(0)}W_{\\rm GW}^{\\rm dyn})^{\\rm nonlocal}$).\nFor comparison, the LDA band structure is also given\n(black dotted line).\nThe energy is measured from the Fermi level.\n}\n\\label{GW+ZB}\n\\end{figure} \n\n\n\nThe resulting effective Hamiltonian in the L-space \nconsists of single-particle and two-particle (interaction) parts.\nThe effective interaction is the same as in previous estimates \nby the cRPA in the literature, while the single-particle dispersion \nis revised after removing the double counting and taking into \naccount the frequency dependence of the effective interaction. The \nfinal effective bandwidth for SrVO$_3$ is,\nafter partial cancellation, \nslightly (14\\%) larger than the LDA bandwidth.  \nThis is a reasonable result because the LDA takes into account all \ncorrelations though insufficiently, while the present scheme by the constrained self-energy excludes the \ncorrelation effects arising from the L-space.\nAlthough the bandwidth derived for the effective model is slightly \nlarger than that of the LDA,  it is clear that the correlation effects \nare stronger than in the LDA or standard GW when the {\\it ab initio} \nmodel is solved by an accurate solver. In fact, if the effective \ninteraction contained in the final effective model is treated by \nthe GW scheme, one obtains the dispersion illustrated in \nFig.~\\ref{GW+ZB}, \nwhich is given from the self energy of the whole GW calculation $\\Sigma$ \nby correcting the local dynamical part $W-W_{\\rm L}$ by $Z_B$. \nThis indicates that even an insufficient treatment of the local \nstatic interaction by the GW scheme gives a dispersion with the bandwidth ($\\sim 1.9$ eV) narrower \nthan the LDA ($\\sim 2.5$ eV, Fig.~\\ref{lda}) and GW ($\\sim 2.1$ eV, Fig.~\\ref{fullGW}) dispersions.  \nA slightly ($\\sim 14$\\%) wider dispersion than that of the LDA bands obtained for the \neffective Hamiltonian accompanied with frequency independent effective \nCoulomb interactions without the exchange part may account for the slight \noverestimate of correlation effects in the literature mentioned in \nthe introduction.\nOur scheme offers an optimized way for the  derivation of \n{\\it ab initio} models for the L-space after eliminating the H-space\nin  a systematic fashion.\n\n\nOur findings of the band widening are consistent with studies\nbased on the combined GW+DMFT method in the literature \n\\cite{TomczakPRB2014}: There, it was argued that within GW+DMFT\nthe best effective Hamiltonian that DMFT should be performed on,\nis a one-body Hamiltonian corrected by the nonlocal part of the\nGW self-energy. The corresponding spectral function \n(see e.g.\nFigure 5 of Ref.~\\onlinecite{TomczakPRB2014}) displays \na broadening similar to \nFig.~\\ref{DFT+Sigma_HL_GW}.\nMost interestingly, our present calculations confirm a pronounced\nasymmetry observed in Ref.~\\onlinecite{TomczakPRB2014}, namely a stronger widening\neffect in the unoccupied part of the spectrum than in the occupied\none.\n\n",
        "label": "DeltaSigma_L_GW",
        "Descriptive_question1": "What does panel (a) in figure_12 represent?",
        "Descriptive_question2": "What is shown in panel (c) of figure_12?",
        "Reasoning_question1": "How does the frequency dependence of the real part in panel (a) of figure_12 compare to that in panel (c), and what might this indicate about the local versus k-point specific behavior?",
        "Reasoning_question2": "What can be inferred about the electronic correlations in SrVO_3 from the differences between the imaginary parts in panels (b) and (d) of figure_12?",
        "Descriptive_answer1": "real local parts",
        "Descriptive_answer2": "real parts at several k points",
        "Reasoning_answer1": "The real part in panel (a) shows a smooth frequency dependence as it represents the local self-energy correction averaged over momentum, indicating a more uniform behavior across the Brillouin zone. In contrast, panel (c) displays the real parts at several specific k points, which reveal variation and slight deviation from the local behavior, reflecting momentum-dependent self-energy effects. This suggests that while local correlations are relatively uniform in energy dependence, nonlocal or momentum-specific correlations introduce additional k-dependent dispersion features.",
        "Reasoning_answer2": "The imaginary parts in panels (b) and (d) show distinct differences: panel (b) displays the imaginary part of the local self-energy, which is smoother and represents averaged scattering processes affecting the electron lifetime uniformly in space. Panel (d), depicting the imaginary parts at several k points, shows variation and possibly sharper features, indicating that electronic correlations have momentum-dependent scattering processes, leading to k-dependent lifetime effects. This underscores the presence of nonlocal correlation effects in SrVO_3 that affect different momentum states differently, contributing to anisotropic electronic behavior."
    },
    {
        "paper_id": "1511.03757.json",
        "image_id": "figure_13",
        "image_path": [
            "/home/yz979/scratch/chengye/scimolmo/test-images-tables/dataset_imagestables/1511.03757/images/fig13.eps"
        ],
        "caption": "(Color online)\nBand structure  of $[\\epsilon_{\\rm DFT}-V_{\\rm xc}+\\Delta\\Sigma_{\\rm H}+\\Delta \\Sigma_{\\rm L}^{\\rm nonlocal}]Z_{\\rm HL}$ of SrVO$_3$ in the direct perturbative treatment.\nFor comparison, the LDA band structure is also given\n(black dotted line).\nThe energy is measured from the Fermi level.\n",
        "classify": "Chart",
        "section_info": "6 Results\n\\section{Results}\n\\label{results}\n\n\nThe ternary 3d$^1$ transition metal oxide SrVO$_3$ has become\none of the ``drosophila compounds'' of correlated systems.\nIt is a correlated metal\nthat has been characterized using various experimental \n\\cite{Onoda, inoue, maiti, maiti2, sekiyama, takizawa, yoshida2010}.\nand theoretical techniques, see e.g.\n\\cite{nekrasov, anisimov, pavarini-d1, liebsch, lechermann-d1, trimarchi, aichhorn09, karolak, lee, held-optics, tomczak-gwdmft, gatti-guzzo, sakuma14, TomczakPRB2014}.\nA review of most of the available experimental and theoretical \ndata has been given recently in Ref.~\\onlinecite{TomczakPRB2014}.\nSrVO$_3$ displays Fermi liquid behavior up to\nremarkably high temperatures of the order of $200$ K \\cite{Onoda, inoue},\nwith a moderate mass enhancement of the order \nof $2$ \\cite{yoshida2010, vandermarel}.\nDetailed spectroscopic investigations have made it an\nideal test compound for modern many-body calculations, and \nmore and more refined dynamical mean field-based studies \nare available.\nThe majority of studies so far have focused on the t$_{2g}$-manifold\nwhich forms the states close to the Fermi level, and those will\nalso be the focus in the present investigations. \nNote however that this restriction quite severely limits the\nrange of validity of the low-energy description. Indeed, as\nshown recently~\\cite{tomczak-gwdmft, TomczakPRB2014} at\nenergies of about 2 to 3 eV above the Fermi level\nthe spectral properties are largely determined by the e$_g$\nstates. This has in particular led to a reinterpretation of\nan inverse photoemission feature at about 2.5 eV that was \nfrequently interpreted as an upper Hubbard band of t$_{2g}$\ncharacter in the earlier literature.\nHere, we use the compound only as an illustration of \nthe principles of constructing effective low-energy models,\nwithout being concerned with a description of spectra beyond\nthe pure t$_{2g}$ part.\n\n\nThe LDA band structure of SrVO$_3$ is shown in Fig.~\\ref{lda}.\nOne clearly distinguishes the three-fold degenerate manifold\nof t$_{2g}$ bands close to the Fermi level (chosen as the\nzero of energy), followed in the unoccupied part of the spectrum\nby the two e$_g$ bands. At about -2.5 eV the filled O-2p derived\nbands are visible.\n\nWhen a standard GW calculation is performed, see Fig.~\\ref{fullGW},\nthe bandwidth of the t$_{2g}$-states is reduced from 2.5 eV to\n2.1 eV, while the overall shape of the dispersion remains similar\nto the LDA one.\nThis is in agreement with previous GW calculations in the literature\n\\cite{tomczak-gwdmft, TomczakPRB2014, gatti-guzzo, sakuma14}.\n\n\n\nThe calculation is based on the full-potential linear muffin-tin orbitals (FP-LMTO) implementation.\n8$\\times$8$\\times$8 \\bm{k}-mesh is employed in both the DFT/LDA and the GW calculations.\nThe calculational details are the same as in Ref.~\\onlinecite{Hirayama}.\n\n\nWe now turn to a discussion of the band structure corresponding to \nthe Hamiltonians to be used as input for subsequent many-body \ncalculations for this material.\nWe will proceed step by step to analyze the influence of the various\ncorrection terms, with respect to the DFT starting point\n(which will be overlaid to the respective band structures).\n\n\n\n\\begin{figure}[h]\n\\centering \n\\includegraphics[clip,width=0.45\\textwidth ]{fig2.eps} \n\\caption{(Color online)\nKohn-Sham band structure of SrVO$_3$ in the LDA.\nThe energy is measured from the Fermi level.\n}\n\\label{lda}\n\\end{figure} \n\n\n\\begin{figure}[h]\n\\centering \n\\includegraphics[clip,width=0.45\\textwidth ]{fig3.eps} \n\\caption{(Color online)\nBand structure  of SrVO$_3$ in the one-shot GW approximation.\nFor comparison, the LDA band structure is also given\n(black dotted line).\nThe energy is measured from the Fermi level.\n}\n\\label{fullGW}\n\\end{figure} \n\n\n\n\n\n\n\\begin{figure}[h]\n\\centering \n\\includegraphics[clip,width=0.45\\textwidth ]{fig4.eps} \n\\caption{(Color online)\nBand structure of SrVO$_3$ corresponding to the LDA Hamiltonian\nfrom which the LDA exchange-correlation potential has been subtracted.\nFor comparison, the LDA band structure is also given\n(black dotted line).\nThe energy is measured from the Fermi level.\n}\n\\label{lda-vxc}\n\\end{figure} \n\n\n\\begin{figure}[h]\n\\centering \n\\includegraphics[clip,width=0.45\\textwidth ]{fig5.eps} \n\\caption{(Color online)\nBand structure  of SrVO$_3$ in the Hartree approximation.\nFor comparison, the LDA band structure is also given\n(black dotted line).\nThe energy is measured from the Fermi level.\n}\n\\label{hartree}\n\\end{figure} \n\n\nFigure~\\ref{lda-vxc} \ndisplays the band structure corresponding to\nthe LDA Hamiltonian from which the \nLDA exchange-correlation\npotential has been subtracted,  \nthat is, the eigenvalues of $H_{\\rm LDA}-V_{\\rm xc}$ where\nthese operators are evaluated for the self-consistent LDA\ndensity.\nThe subtraction of $V_{\\rm xc}$ widens the band structure from \nthe LDA bandwidth of 2.5 eV \nto more than twice this value: the new band\nwidth is 5.4 eV.\nThis indicates that the exchange-correlation potential $V_{\\rm xc}$ \nof the LDA is responsible for a substantial amount of \nthe band-narrowing effect\narising from electronic correlations, stressing the need \nto subtract the double counting in a consistent manner. \nFigure~\\ref{lda-vxc} is regarded as the starting point for the\nfollowing considerations.\n\nFor simple materials (see e.g. the calculations on Li in Ref.~\\onlinecite{Lithium})\nit has been noted in the literature that the Hartree band structure\nis close to the DFT one. This raises the question of the origin of the\nsubstantial band widening in the present case. Indeed, the present\ncalculation shown in Fig.~\\ref{lda-vxc} differs from a Hartree calculation\nonly \nby the fact that Fig.~\\ref{lda-vxc} is evaluated for the converged LDA density.\nWe have performed a test calculation where we plot the Hartree band\nstructure calculated for the converged Hartree density. The result\nis shown in Fig.~\\ref{hartree}. As seen from this plot, while the\nband is not fully as wide as in Fig.~\\ref{lda-vxc}, a substantial \nwidening is already present at this stage.\n\n\n\n\\begin{figure}[h]\n\\centering \n\\includegraphics[clip,width=0.45\\textwidth ]{fig6.eps} \n\\caption{(Color online)\nBand structure  of $H_{\\rm LDA}-V_{\\rm xc}+\\Delta \\Sigma_{\\rm H}(\\omega=0)$ of SrVO$_3$.\nFor comparison, the LDA band structure is also given\n(black dotted line).\nThe energy is measured from the Fermi level.\n}\n\\label{lda-vxc-sigH(0)}\n\\end{figure} \n\n\n\nStarting from the Hamiltonian without V$_{xc}$ (see the \ndispersion in Fig.~\\ref{lda-vxc}), we first take into account \nthe static part of the corrective self-energy $\\Delta \\Sigma_{\\rm H}$:\nIn Fig.~\\ref{lda-vxc-sigH(0)}, we plot the dispersion corresponding to\n$H_{\\rm LDA}-V_{\\rm xc}+\\Delta \\Sigma_{\\rm H}(\\omega=0)$. \nCompared to the dispersion of Fig.~\\ref{lda-vxc}, the overall band structure \nis narrowed to 3.7 eV. This value is, however, still considerably larger\nthan the LDA bandwidth (Fig.~\\ref{lda}). Although\n the electronic correlations coupling the H- and \nthe L space are included in Fig.~\\ref{lda-vxc-sigH(0)}, and narrow the band with respect to the case\nwhere V$_{xc}$ is taken out, correlations within\nthe L space are not included. \nThe LDA, on the other hand, at least partially\nincludes correlations within the L space, \nand these are very effective in narrowing the band.\nInterestingly, the bottom of the occupied band is quite\nexactly at the LDA value, and the remaining widening is\npurely in the unoccupied part.\n\n\n\\begin{figure*}[htb]\n\\begin{center} \n\\includegraphics[width=0.9\\textwidth ]{fig7.eps} \n\\end{center} \n\\caption{(Color online)\nFrequency dependence of $-V_{xc}+\\Delta \\Sigma_{\\rm H}$ for SrVO$_3$.\n(a) and (b) are the real and imaginary parts at the $\\Gamma$ point, \nrespectively,\nand (c) and (d) ((e) and (f)) are those at $X$ ($R$).\nFor comparison, the full GW self-energies are also given\n(black dotted line).\n}\n\\label{sigH}\n\\end{figure*} \n\n\n\n\\begin{figure}[h]\n\\centering \n\\includegraphics[clip,width=0.45\\textwidth ]{fig8.eps} \n\\caption{(Color online)\nBand structure  of $Z_{\\rm H}[H_{\\rm LDA}-V_{\\rm xc}+\\Delta\\Sigma_{\\rm H}(\\omega=0)]$ of SrVO$_3$.\nFor comparison, the LDA band structure is also given\n(black dotted line).\nThe energy is measured from the Fermi level.\n}\n\\label{ZH_SigmaH}\n\\end{figure} \n\n\nTo go further and in particular to analyze the dynamic\nbehavior of $\\Delta \\Sigma_{\\rm H}$ \nwe plot in Fig.~\\ref{sigH} \nthe self-energy corrections $\\Delta \\Sigma_{\\rm H}$ (with $\\Sigma$ (see Eq.~(\\ref{full-self-energy}))\nfor comparison) for the real and imaginary parts\nat several representative choices of momenta.\nThe frequency dependence is smooth around the Fermi level.\nIn particular, in contrast to the full self-energy,\nthere are no poles in $\\Delta \\Sigma_{\\rm H}$ \nin the low-energy region, thanks to the\nexclusion of fluctuations within the L-space.\nThe frequency dependence of the real part indicates that the \nlinearization \n$\\displaystyle{\\Delta \\Sigma_{\\rm H}(\\omega) \\sim \\Delta \\Sigma_{\\rm H}(\\omega=0)+[d\\Delta \\Sigma_{\\rm H}(\\omega)/d\\omega]_{\\omega=0}\\omega }$\noffers a reasonably good approximation. \nIn fact, the behavior of the constrained self-energy\nis much closer to linearity than that of \nthe full GW self-energy $\\Sigma$.\nThis is easily understood by the fact that the low-energy excitations are excluded in the present constrained self-energy $\\Delta\\Sigma_{\\rm H}$ analogously to an insulator, thus eliminating the strong frequency dependence of the constrained self-energy near the Fermi level. This is one of the consequences of the \ncontrollability and an advantage of the present MACE scheme, as \nmentioned in the introduction.\nIn the following discussion, we employ this linearized approximation.\nThen the renormalization factor $Z_{\\rm H}$ defined by Eq.~(\\ref{ZH})\nis interpreted as that from the contribution of the H-space. \nAfter \nthis renormalization factor is taken into account,\nthe effective Hamiltonian is given by $Z_{\\rm H}[H_{\\rm LDA}-V_{\\rm xc}+\\Delta\\Sigma_{\\rm H}(\\omega=0)]$ (see Fig.~\\ref{ZH_SigmaH}). \nInterestingly the renormalization factor $Z_{\\rm H}$=0.92 stays\nclose to one, so that\nthe bandwidth is only slightly reduced (to\n$3.4$ eV instead of the $3.7$ eV above). \nWe also note that $Z_{\\rm H}$ has weak momentum dependence\nas we reveal in the following.\n\n\n\n\\begin{figure*}[htb]\n\\begin{center} \n\\includegraphics[width=0.9\\textwidth ]{fig9.eps} \n\\end{center} \n\\caption{(Color online)\n{\\bf minor modification}\nFrequency dependence of $-V_{\\rm xc}+\\Delta \\Sigma_{\\rm H}$ of SrVO$_3$.\n(a) and (b) are the real and imaginary parts of the local\ncomponent, respectively.\nThe black dotted line is $\\Sigma$ defined in Eq.~(\\ref{full-self-energy}) in the full GW approximation for comparison.\n(c) and (d) are the real and imaginary \nparts at several $k$ points.\n}\n\\label{sigH_nonlocal}\n\\end{figure*} \n\n\n\nFigure~\\ref{sigH_nonlocal} \nshows the local and nonlocal parts of $\\Delta \\Sigma_{\\rm H}$.\nThe frequency \ndependence\nof the local part (Figs.~\\ref{sigH_nonlocal} (a)(b)) \nis quite similar \nto the  frequency-dependent self-energies at various momenta\nin Fig.~\\ref{sigH}.\nOne immediately reads off an interesting property, which is\nakin to what has been found for the full GW self-energy in\nRef.~\\onlinecite{Tomczak-PRL2012}, namely that the nonlocal part of \n$\\Delta \\Sigma_{\\rm H}$ shows only weak frequency dependence \nas shown in Figs.~\\ref{sigH_nonlocal} (c)(d). \nThis explains why $Z_{\\rm H}$ \nis only weakly momentum-dependent.\nOn the other hand, $\\Delta \\Sigma_{\\rm H}^{\\rm nonlocal} (0)$ at $R$ is $\\sim 1$ eV larger than \nat $\\Gamma$, which causes the band widening effect.\n $\\Delta \\Sigma_{\\rm H}$\ncan thus to a good approximation be decomposed into a\nfrequency-dependent local part and a static nonlocal one:\n$\\Delta \\Sigma_{\\rm H}(k,\\omega) = \\Delta \\Sigma^{\\rm local}_{\\rm H}(\\omega) + \\Delta \\Sigma^{\\rm nonlocal}_{\\rm H}(k)$,\nwith $ \\Delta \\Sigma_{\\rm H}^{\\rm local}(\\omega) = \\sum_q  \\Delta \\Sigma_{\\rm H}(q,\\omega)$\n\nSince in the simple case of SrVO$_3$ where the t$_{2g}$ states are\ndegenerate, static local operators are scalar and are compensated \nby a chemical potential shift such that the correct particle\nnumber is obtained. The above band structure corresponding to\n$H_{\\rm LDA} - V_{\\rm xc} + \\Delta \\Sigma_{\\rm H}(k, \\omega=0)$ \nis thus identical to that of\n$H_{\\rm LDA} - V_{\\rm xc} + \\Delta \\Sigma^{\\rm nonlocal}_{\\rm H}(k)$.\nThe local dynamical part of $\\Delta \\Sigma_{\\rm H}$ then results in a \nnarrowing of this band structure by a factor $Z_{\\rm H} = 0.92$.\n\n\n\n\\begin{figure}[h]\n\\centering \n\\includegraphics[clip,width=0.45\\textwidth ]{fig10.eps} \n\\caption{(Color online)\n(a) Frequency dependence of the on-site effective interaction.\nPanel (b) is a low-energy zoom of panel (a). \n}\n\\label{W}\n\\end{figure} \n\n\nBefore turning to a discussion of the low-energy correction for the GW treatment\n$\\Delta \\Sigma_{\\rm L} =G^{(0)}(W-W_{\\rm L})$, we analyze the effective\ninteraction $W_{\\rm L}$ shown in Fig.~\\ref{W} \nin comparison to $W$ and $W_{\\rm H}$.\nThe fully screened Coulomb interaction $W$ displays the familiar\nshape of an interaction that is strongly screened at low energies\n(with a value of $0.88$ eV at $\\omega=0$), while retrieving the\nvalue of the bare Coulomb interaction $v$ ($\\sim 16.0$ eV) \nat high energy.\nThe crossover from the bare to the  \nscreened values takes place at the plasma energy of \nabout $15$ eV. This behavior has been discussed before\n\\cite{Review_SB2014}; we note in particular that the plasma \nfrequency is known from electron energy loss spectroscopy \nmeasurements of the related SrTiO$_3$ compound \\cite{vandermarel,kohiki}.\nThe partially screened interaction\n$W_{\\rm H}$ constructed within cRPA\nconverges to $W$ at high energies, but displays\nweaker screening effects at low energies (with a value of $3.5$ eV at $\\omega=0$),\nsince intra-t$_{2g}$ screening channels are excluded.\nAs was already anticipated in Sec.~\\ref{Outline},  $W_{\\rm L}$ can be interpreted as the screened interaction of \na low-energy model where a static interaction of value $W_{\\rm H}(\\omega=0)$\nhas been imposed as the bare interaction. Since now {\\it only} intra-t$_{2g}$\nscreening channels are active, screening takes place only at low\nenergies, where the scale is given by the t$_{2g}$ bandwidth.\nAlso plotted is the difference $W_{\\rm GW}^{\\rm dyn}=W-W_{d}$: except at low energies\nwhere the t$_{2g}$ screening channels come into play, its frequency\ndependence is essentially given by that of $W$, while the\nhigh-energy limit is the bare Coulomb interaction $v$ reduced\nby $W_{\\rm H}(0)$.\n\n\n\\begin{figure*}[htb]\n\\begin{center} \n\\includegraphics[width=0.9\\textwidth ]{fig11.eps} \n\\end{center} \n\\caption{(Color online)\nFrequency dependence of $-V_{\\rm xc}+ G^{(1)}W_{\\rm H}^{\\rm dyn}$ of SrVO$_3$.\n(a) and (b) are the real and imaginary local parts, respectively.\n(c) and (d) are the real and imaginary \nparts at several $k$ points.\n}\n\\label{DeltaSigma_L_Pert}\n\\end{figure*} \n\n\n\n\\begin{figure*}[htb]\n\\begin{center} \n\\includegraphics[width=0.9\\textwidth ]{fig12.eps} \n\\end{center} \n\\caption{(Color online)\nFrequency dependence of $-V_{\\rm xc}+ G^{(0)}W_{\\rm GW}^{\\rm dyn}$ of SrVO$_3$.\n(a) and (b) are the real and imaginary local parts, respectively.\n(c) and (d) are the real and imaginary \nparts at several $k$ points.\n}\n\\label{DeltaSigma_L_GW}\n\\end{figure*} \n\n\n\nThe frequency dependence of the real and imaginary parts of the low-energy self-energy $\\Delta \\Sigma_{\\rm L} = G^{(1)}W_{\\rm H}^{\\rm dyn}$ for the direct perturbative treatment is illustrated in Fig.~\\ref{DeltaSigma_L_Pert}. \n$\\Delta \\Sigma_{\\rm L} = G^{(0)}W_{\\rm GW}^{\\rm dyn}$ used\nfor the GW treatment is shown in Fig.~\\ref{DeltaSigma_L_GW}\nfor several choices of momenta.\n\\\\\nThe results for the perturbative and the GW treatment are nearly \nidentical.\nThe renormalization factor of $\\Delta \\Sigma_{\\rm L}^{\\rm nonlocal}$ \n(and $\\Delta \\Sigma_{\\rm L}$) is $\\sim 0.77$.\nThe zero-frequency shift\n${\\rm Re}\\Delta \\Sigma_{\\rm L}^{\\rm nonlocal} (0)$ at $R$ is $\\sim 3$ eV \nlarger than \nat $\\Gamma$.\nAgain, one sees how $\\Sigma_{\\rm L}$ separates into local dynamical\nand nonlocal static parts: \n$\\Delta \\Sigma_{\\rm L} = \\Delta \\Sigma^{\\rm nonlocal}_{\\rm L}(k) + \n\\Delta \\Sigma^{\\rm local}_{\\rm L}(\\omega)$.\nThis is expected since such a separation has been found within\nthe full GW calculation \\cite{TomczakPRB2014}, and is even\nmore plausible for a GW treatment within the t$_{2g}$ subspace.\n\n\n\n\\begin{figure}[h]\n\\centering \n\\includegraphics[clip,width=0.45\\textwidth ]{fig13.eps} \n\\caption{(Color online)\nBand structure  of $[\\epsilon_{\\rm DFT}-V_{\\rm xc}+\\Delta\\Sigma_{\\rm H}+\\Delta \\Sigma_{\\rm L}^{\\rm nonlocal}]Z_{\\rm HL}$ of SrVO$_3$ in the direct perturbative treatment.\nFor comparison, the LDA band structure is also given\n(black dotted line).\nThe energy is measured from the Fermi level.\n}\n\\label{DFT+Sigma_HL_Pert}\n\\end{figure} \n\n\n\n\\begin{figure}[h]\n\\centering \n\\includegraphics[clip,width=0.45\\textwidth ]{fig14.eps} \n\\caption{(Color online)\nBand structure  of $[\\epsilon_{\\rm DFT}-V_{\\rm xc}+\\Delta\\Sigma_{\\rm H}+\\Delta \\Sigma_{\\rm L}^{\\rm nonlocal}]Z_{\\rm HL}$ of SrVO$_3$ in the GW treatment.\nFor comparison, the LDA band structure is also given\n(black dotted line).\nThe energy is measured from the Fermi level.\n}\n\\label{DFT+Sigma_HL_GW}\n\\end{figure} \n\n\n\nThe corresponding band structures \nare given by \n $[\\epsilon_{\\rm DFT}+\\Delta\\Sigma_{\\rm H}+\\Delta \\Sigma_{\\rm L}^{\\rm nonlocal}]Z_{\\rm HL}$ in Fig.~\\ref{DFT+Sigma_HL_Pert} \nfor the perturbative treatment (see Eq.~(\\ref{DeltaSigmaLnonlocalPert})), and in Fig.~\\ref{DFT+Sigma_HL_GW} \nfor the GW treatment,\nwhere $\\Delta \\Sigma_{\\rm L}^{\\rm nonlocal}$ is defined in Eq.~(\\ref{DeltaSigLnonlocalGW}) and $Z_{\\rm HL}$ is around 0.92.\n$Z_{\\rm HL}$ is practically the same as $Z_{\\rm H}$, meaning that the nonlocal dynamical correction \nin the H-space is small, which justifies the GW perturbative treatment for the nonlocal part.\nOne finds that \nthe nonlocal part $\\Delta \\Sigma_{\\rm L}=G^{(0)}_{ll}(W-W_{\\rm L})$ narrows\nthe empty states but widens the occupied ones, resulting\nin an overall bandwidth of 3.2 eV.\nThe difference between the direct perturbation and the GW treatment is small.\n\n\n\n\\begin{figure}[h]\n\\centering \n\\includegraphics[clip,width=0.45\\textwidth ]{fig15.eps} \n\\caption{(Color online)\nBand structure  of $[\\epsilon_{\\rm DFT}-V_{\\rm xc}+\\Delta\\Sigma_{\\rm H}+\\Delta\\Sigma_{\\rm L}^{\\rm nonlocal}]Z_{\\rm HL}Z_B$ of SrVO$_3$ in the GW treatment.\nFor comparison, the LDA band structure is also given\n(black dotted line).\nThe energy is measured from the Fermi level.}\n\\label{DFT+DeltaSigmaL_GW_ZHL_ZB}\n\\end{figure} \n\n\nAfter including the effect of the local self-energy by the nonperturbative \nCasula trick, we show the dispersion\ngiven by \n$[\\epsilon_{\\rm DFT}-V_{\\rm xc}+\\Delta\\Sigma_{\\rm H}+\\Delta\\Sigma_{\\rm L}^{\\rm nonlocal}]Z_{\\rm HL}Z_B$ for the GW-type treatment in Fig.~\\ref{DFT+DeltaSigmaL_GW_ZHL_ZB}: \nThe GW-like treatment gives a band dispersion\nclose to the LDA one. \nThe $Z_B$ factor corresponding to the local dynamical part of \n$\\Delta \\Sigma_{\\rm L}$ amounts to $Z_B=0.7$.\nThe nonperturbative treatment \nresults in an LDA-like band dispersion for the empty states,\nbut a slightly narrower bandwidth in the occupied part.\nThe low-energy effective Hamiltonian at this level of treatment has a \nfrequency independent effective interaction both with local \nand nonlocal interaction given by the Fourier transform of \n$W_{\\rm H}(q,\\omega=0)$, which contains both direct and exchange interactions. \n\n\n\\begin{figure}[h]\n\\centering \n\\includegraphics[clip,width=0.45\\textwidth ]{fig16.eps} \n\\caption{(Color online)\nBand structure  of $[\\epsilon_{\\rm DFT}+\\Sigma_{\\rm corr}^{\\rm GW}]Z_{\\rm corr}Z_B$ of SrVO$_3$.\nFor comparison, the LDA band structure is also given\n(black dotted line).\nThe energy is measured from the Fermi level.}\n\\label{DFT+Sigma_corr_GW_Z}\n\\end{figure} \n\n\n\nWe finally show the dispersion\ngiven by Eq.~(\\ref{Teff_final}), namely\n$[\\epsilon_{\\rm DFT}+\\Sigma_{\\rm corr}^{\\rm GW}]Z_{\\rm corr}Z_B$ in Fig.~\\ref{DFT+Sigma_corr_GW_Z}. \nThe result shows an LDA-like band dispersion for the occupied states,\nbut a slightly wider bandwidth in the empty part, resulting in a bandwidth 14\\% wider in total than the LDA bandwidth. \nThe effective interaction of the low-energy effective Hamiltonian \nat this final level has\nfrequency independent local and nonlocal interactions \ngiven by the Fourier transform of $W_r(q,\\omega=0)$. \n\n\n\n\\begin{figure}[h]\n\\centering \n\\includegraphics[clip,width=0.45\\textwidth ]{fig17.eps} \n\\caption{(Color online)\nBand structure  of $[\\epsilon_{\\rm DFT}-V_{\\rm xc}+\\Sigma-(G^{(0)}W_{\\rm GW}^{\\rm dyn})^{\\rm nonlocal}]Z'Z_B$ of SrVO$_3$\n(Z' is the renormalization factor of $\\Sigma-(G^{(0)}W_{\\rm GW}^{\\rm dyn})^{\\rm nonlocal}$).\nFor comparison, the LDA band structure is also given\n(black dotted line).\nThe energy is measured from the Fermi level.\n}\n\\label{GW+ZB}\n\\end{figure} \n\n\n\nThe resulting effective Hamiltonian in the L-space \nconsists of single-particle and two-particle (interaction) parts.\nThe effective interaction is the same as in previous estimates \nby the cRPA in the literature, while the single-particle dispersion \nis revised after removing the double counting and taking into \naccount the frequency dependence of the effective interaction. The \nfinal effective bandwidth for SrVO$_3$ is,\nafter partial cancellation, \nslightly (14\\%) larger than the LDA bandwidth.  \nThis is a reasonable result because the LDA takes into account all \ncorrelations though insufficiently, while the present scheme by the constrained self-energy excludes the \ncorrelation effects arising from the L-space.\nAlthough the bandwidth derived for the effective model is slightly \nlarger than that of the LDA,  it is clear that the correlation effects \nare stronger than in the LDA or standard GW when the {\\it ab initio} \nmodel is solved by an accurate solver. In fact, if the effective \ninteraction contained in the final effective model is treated by \nthe GW scheme, one obtains the dispersion illustrated in \nFig.~\\ref{GW+ZB}, \nwhich is given from the self energy of the whole GW calculation $\\Sigma$ \nby correcting the local dynamical part $W-W_{\\rm L}$ by $Z_B$. \nThis indicates that even an insufficient treatment of the local \nstatic interaction by the GW scheme gives a dispersion with the bandwidth ($\\sim 1.9$ eV) narrower \nthan the LDA ($\\sim 2.5$ eV, Fig.~\\ref{lda}) and GW ($\\sim 2.1$ eV, Fig.~\\ref{fullGW}) dispersions.  \nA slightly ($\\sim 14$\\%) wider dispersion than that of the LDA bands obtained for the \neffective Hamiltonian accompanied with frequency independent effective \nCoulomb interactions without the exchange part may account for the slight \noverestimate of correlation effects in the literature mentioned in \nthe introduction.\nOur scheme offers an optimized way for the  derivation of \n{\\it ab initio} models for the L-space after eliminating the H-space\nin  a systematic fashion.\n\n\nOur findings of the band widening are consistent with studies\nbased on the combined GW+DMFT method in the literature \n\\cite{TomczakPRB2014}: There, it was argued that within GW+DMFT\nthe best effective Hamiltonian that DMFT should be performed on,\nis a one-body Hamiltonian corrected by the nonlocal part of the\nGW self-energy. The corresponding spectral function \n(see e.g.\nFigure 5 of Ref.~\\onlinecite{TomczakPRB2014}) displays \na broadening similar to \nFig.~\\ref{DFT+Sigma_HL_GW}.\nMost interestingly, our present calculations confirm a pronounced\nasymmetry observed in Ref.~\\onlinecite{TomczakPRB2014}, namely a stronger widening\neffect in the unoccupied part of the spectrum than in the occupied\none.\n\n",
        "label": "DFT+Sigma_HL_Pert",
        "Descriptive_question1": "What is the overall bandwidth of the band structure shown in figure_13 for SrVO₃ in the direct perturbative treatment?",
        "Descriptive_question2": "What color is used for the LDA band structure in figure_13 for comparison with the perturbative treatment of SrVO₃?",
        "Reasoning_question1": "How does the inclusion of nonlocal corrections in figure_13 affect the band structure of SrVO₃ compared to the LDA band structure, particularly in terms of the empty and occupied states?",
        "Reasoning_question2": "Why might the bandwidth in figure_13 for SrVO₃ in the direct perturbative treatment be wider than the LDA bandwidth, and what does this suggest about the correlation effects in the perturbative approach?",
        "Descriptive_answer1": "3.2 eV",
        "Descriptive_answer2": "black dotted line",
        "Reasoning_answer1": "The inclusion of nonlocal corrections in figure_13 results in a band structure where the empty states are narrowed and the occupied states are widened compared to the LDA band structure. This leads to an overall bandwidth of 3.2 eV, which differs from the LDA bandwidth by affecting the distribution of energy states asymmetrically between empty and occupied states.",
        "Reasoning_answer2": "The bandwidth is wider than the LDA bandwidth because the perturbative treatment includes the nonlocal self-energy corrections that partially counteract the band narrowing effects included in the LDA. This suggests that the perturbative approach accounts for correlation effects associated with the separation of the low-energy space but excludes certain local dynamical correlations, resulting in an effective band widening after partial cancellation of various effects."
    },
    {
        "paper_id": "1511.03757.json",
        "image_id": "figure_14",
        "image_path": [
            "/home/yz979/scratch/chengye/scimolmo/test-images-tables/dataset_imagestables/1511.03757/images/fig14.eps"
        ],
        "caption": "(Color online)\nBand structure  of $[\\epsilon_{\\rm DFT}-V_{\\rm xc}+\\Delta\\Sigma_{\\rm H}+\\Delta \\Sigma_{\\rm L}^{\\rm nonlocal}]Z_{\\rm HL}$ of SrVO$_3$ in the GW treatment.\nFor comparison, the LDA band structure is also given\n(black dotted line).\nThe energy is measured from the Fermi level.\n",
        "classify": "Chart",
        "section_info": "6 Results\n\\section{Results}\n\\label{results}\n\n\nThe ternary 3d$^1$ transition metal oxide SrVO$_3$ has become\none of the ``drosophila compounds'' of correlated systems.\nIt is a correlated metal\nthat has been characterized using various experimental \n\\cite{Onoda, inoue, maiti, maiti2, sekiyama, takizawa, yoshida2010}.\nand theoretical techniques, see e.g.\n\\cite{nekrasov, anisimov, pavarini-d1, liebsch, lechermann-d1, trimarchi, aichhorn09, karolak, lee, held-optics, tomczak-gwdmft, gatti-guzzo, sakuma14, TomczakPRB2014}.\nA review of most of the available experimental and theoretical \ndata has been given recently in Ref.~\\onlinecite{TomczakPRB2014}.\nSrVO$_3$ displays Fermi liquid behavior up to\nremarkably high temperatures of the order of $200$ K \\cite{Onoda, inoue},\nwith a moderate mass enhancement of the order \nof $2$ \\cite{yoshida2010, vandermarel}.\nDetailed spectroscopic investigations have made it an\nideal test compound for modern many-body calculations, and \nmore and more refined dynamical mean field-based studies \nare available.\nThe majority of studies so far have focused on the t$_{2g}$-manifold\nwhich forms the states close to the Fermi level, and those will\nalso be the focus in the present investigations. \nNote however that this restriction quite severely limits the\nrange of validity of the low-energy description. Indeed, as\nshown recently~\\cite{tomczak-gwdmft, TomczakPRB2014} at\nenergies of about 2 to 3 eV above the Fermi level\nthe spectral properties are largely determined by the e$_g$\nstates. This has in particular led to a reinterpretation of\nan inverse photoemission feature at about 2.5 eV that was \nfrequently interpreted as an upper Hubbard band of t$_{2g}$\ncharacter in the earlier literature.\nHere, we use the compound only as an illustration of \nthe principles of constructing effective low-energy models,\nwithout being concerned with a description of spectra beyond\nthe pure t$_{2g}$ part.\n\n\nThe LDA band structure of SrVO$_3$ is shown in Fig.~\\ref{lda}.\nOne clearly distinguishes the three-fold degenerate manifold\nof t$_{2g}$ bands close to the Fermi level (chosen as the\nzero of energy), followed in the unoccupied part of the spectrum\nby the two e$_g$ bands. At about -2.5 eV the filled O-2p derived\nbands are visible.\n\nWhen a standard GW calculation is performed, see Fig.~\\ref{fullGW},\nthe bandwidth of the t$_{2g}$-states is reduced from 2.5 eV to\n2.1 eV, while the overall shape of the dispersion remains similar\nto the LDA one.\nThis is in agreement with previous GW calculations in the literature\n\\cite{tomczak-gwdmft, TomczakPRB2014, gatti-guzzo, sakuma14}.\n\n\n\nThe calculation is based on the full-potential linear muffin-tin orbitals (FP-LMTO) implementation.\n8$\\times$8$\\times$8 \\bm{k}-mesh is employed in both the DFT/LDA and the GW calculations.\nThe calculational details are the same as in Ref.~\\onlinecite{Hirayama}.\n\n\nWe now turn to a discussion of the band structure corresponding to \nthe Hamiltonians to be used as input for subsequent many-body \ncalculations for this material.\nWe will proceed step by step to analyze the influence of the various\ncorrection terms, with respect to the DFT starting point\n(which will be overlaid to the respective band structures).\n\n\n\n\\begin{figure}[h]\n\\centering \n\\includegraphics[clip,width=0.45\\textwidth ]{fig2.eps} \n\\caption{(Color online)\nKohn-Sham band structure of SrVO$_3$ in the LDA.\nThe energy is measured from the Fermi level.\n}\n\\label{lda}\n\\end{figure} \n\n\n\\begin{figure}[h]\n\\centering \n\\includegraphics[clip,width=0.45\\textwidth ]{fig3.eps} \n\\caption{(Color online)\nBand structure  of SrVO$_3$ in the one-shot GW approximation.\nFor comparison, the LDA band structure is also given\n(black dotted line).\nThe energy is measured from the Fermi level.\n}\n\\label{fullGW}\n\\end{figure} \n\n\n\n\n\n\n\\begin{figure}[h]\n\\centering \n\\includegraphics[clip,width=0.45\\textwidth ]{fig4.eps} \n\\caption{(Color online)\nBand structure of SrVO$_3$ corresponding to the LDA Hamiltonian\nfrom which the LDA exchange-correlation potential has been subtracted.\nFor comparison, the LDA band structure is also given\n(black dotted line).\nThe energy is measured from the Fermi level.\n}\n\\label{lda-vxc}\n\\end{figure} \n\n\n\\begin{figure}[h]\n\\centering \n\\includegraphics[clip,width=0.45\\textwidth ]{fig5.eps} \n\\caption{(Color online)\nBand structure  of SrVO$_3$ in the Hartree approximation.\nFor comparison, the LDA band structure is also given\n(black dotted line).\nThe energy is measured from the Fermi level.\n}\n\\label{hartree}\n\\end{figure} \n\n\nFigure~\\ref{lda-vxc} \ndisplays the band structure corresponding to\nthe LDA Hamiltonian from which the \nLDA exchange-correlation\npotential has been subtracted,  \nthat is, the eigenvalues of $H_{\\rm LDA}-V_{\\rm xc}$ where\nthese operators are evaluated for the self-consistent LDA\ndensity.\nThe subtraction of $V_{\\rm xc}$ widens the band structure from \nthe LDA bandwidth of 2.5 eV \nto more than twice this value: the new band\nwidth is 5.4 eV.\nThis indicates that the exchange-correlation potential $V_{\\rm xc}$ \nof the LDA is responsible for a substantial amount of \nthe band-narrowing effect\narising from electronic correlations, stressing the need \nto subtract the double counting in a consistent manner. \nFigure~\\ref{lda-vxc} is regarded as the starting point for the\nfollowing considerations.\n\nFor simple materials (see e.g. the calculations on Li in Ref.~\\onlinecite{Lithium})\nit has been noted in the literature that the Hartree band structure\nis close to the DFT one. This raises the question of the origin of the\nsubstantial band widening in the present case. Indeed, the present\ncalculation shown in Fig.~\\ref{lda-vxc} differs from a Hartree calculation\nonly \nby the fact that Fig.~\\ref{lda-vxc} is evaluated for the converged LDA density.\nWe have performed a test calculation where we plot the Hartree band\nstructure calculated for the converged Hartree density. The result\nis shown in Fig.~\\ref{hartree}. As seen from this plot, while the\nband is not fully as wide as in Fig.~\\ref{lda-vxc}, a substantial \nwidening is already present at this stage.\n\n\n\n\\begin{figure}[h]\n\\centering \n\\includegraphics[clip,width=0.45\\textwidth ]{fig6.eps} \n\\caption{(Color online)\nBand structure  of $H_{\\rm LDA}-V_{\\rm xc}+\\Delta \\Sigma_{\\rm H}(\\omega=0)$ of SrVO$_3$.\nFor comparison, the LDA band structure is also given\n(black dotted line).\nThe energy is measured from the Fermi level.\n}\n\\label{lda-vxc-sigH(0)}\n\\end{figure} \n\n\n\nStarting from the Hamiltonian without V$_{xc}$ (see the \ndispersion in Fig.~\\ref{lda-vxc}), we first take into account \nthe static part of the corrective self-energy $\\Delta \\Sigma_{\\rm H}$:\nIn Fig.~\\ref{lda-vxc-sigH(0)}, we plot the dispersion corresponding to\n$H_{\\rm LDA}-V_{\\rm xc}+\\Delta \\Sigma_{\\rm H}(\\omega=0)$. \nCompared to the dispersion of Fig.~\\ref{lda-vxc}, the overall band structure \nis narrowed to 3.7 eV. This value is, however, still considerably larger\nthan the LDA bandwidth (Fig.~\\ref{lda}). Although\n the electronic correlations coupling the H- and \nthe L space are included in Fig.~\\ref{lda-vxc-sigH(0)}, and narrow the band with respect to the case\nwhere V$_{xc}$ is taken out, correlations within\nthe L space are not included. \nThe LDA, on the other hand, at least partially\nincludes correlations within the L space, \nand these are very effective in narrowing the band.\nInterestingly, the bottom of the occupied band is quite\nexactly at the LDA value, and the remaining widening is\npurely in the unoccupied part.\n\n\n\\begin{figure*}[htb]\n\\begin{center} \n\\includegraphics[width=0.9\\textwidth ]{fig7.eps} \n\\end{center} \n\\caption{(Color online)\nFrequency dependence of $-V_{xc}+\\Delta \\Sigma_{\\rm H}$ for SrVO$_3$.\n(a) and (b) are the real and imaginary parts at the $\\Gamma$ point, \nrespectively,\nand (c) and (d) ((e) and (f)) are those at $X$ ($R$).\nFor comparison, the full GW self-energies are also given\n(black dotted line).\n}\n\\label{sigH}\n\\end{figure*} \n\n\n\n\\begin{figure}[h]\n\\centering \n\\includegraphics[clip,width=0.45\\textwidth ]{fig8.eps} \n\\caption{(Color online)\nBand structure  of $Z_{\\rm H}[H_{\\rm LDA}-V_{\\rm xc}+\\Delta\\Sigma_{\\rm H}(\\omega=0)]$ of SrVO$_3$.\nFor comparison, the LDA band structure is also given\n(black dotted line).\nThe energy is measured from the Fermi level.\n}\n\\label{ZH_SigmaH}\n\\end{figure} \n\n\nTo go further and in particular to analyze the dynamic\nbehavior of $\\Delta \\Sigma_{\\rm H}$ \nwe plot in Fig.~\\ref{sigH} \nthe self-energy corrections $\\Delta \\Sigma_{\\rm H}$ (with $\\Sigma$ (see Eq.~(\\ref{full-self-energy}))\nfor comparison) for the real and imaginary parts\nat several representative choices of momenta.\nThe frequency dependence is smooth around the Fermi level.\nIn particular, in contrast to the full self-energy,\nthere are no poles in $\\Delta \\Sigma_{\\rm H}$ \nin the low-energy region, thanks to the\nexclusion of fluctuations within the L-space.\nThe frequency dependence of the real part indicates that the \nlinearization \n$\\displaystyle{\\Delta \\Sigma_{\\rm H}(\\omega) \\sim \\Delta \\Sigma_{\\rm H}(\\omega=0)+[d\\Delta \\Sigma_{\\rm H}(\\omega)/d\\omega]_{\\omega=0}\\omega }$\noffers a reasonably good approximation. \nIn fact, the behavior of the constrained self-energy\nis much closer to linearity than that of \nthe full GW self-energy $\\Sigma$.\nThis is easily understood by the fact that the low-energy excitations are excluded in the present constrained self-energy $\\Delta\\Sigma_{\\rm H}$ analogously to an insulator, thus eliminating the strong frequency dependence of the constrained self-energy near the Fermi level. This is one of the consequences of the \ncontrollability and an advantage of the present MACE scheme, as \nmentioned in the introduction.\nIn the following discussion, we employ this linearized approximation.\nThen the renormalization factor $Z_{\\rm H}$ defined by Eq.~(\\ref{ZH})\nis interpreted as that from the contribution of the H-space. \nAfter \nthis renormalization factor is taken into account,\nthe effective Hamiltonian is given by $Z_{\\rm H}[H_{\\rm LDA}-V_{\\rm xc}+\\Delta\\Sigma_{\\rm H}(\\omega=0)]$ (see Fig.~\\ref{ZH_SigmaH}). \nInterestingly the renormalization factor $Z_{\\rm H}$=0.92 stays\nclose to one, so that\nthe bandwidth is only slightly reduced (to\n$3.4$ eV instead of the $3.7$ eV above). \nWe also note that $Z_{\\rm H}$ has weak momentum dependence\nas we reveal in the following.\n\n\n\n\\begin{figure*}[htb]\n\\begin{center} \n\\includegraphics[width=0.9\\textwidth ]{fig9.eps} \n\\end{center} \n\\caption{(Color online)\n{\\bf minor modification}\nFrequency dependence of $-V_{\\rm xc}+\\Delta \\Sigma_{\\rm H}$ of SrVO$_3$.\n(a) and (b) are the real and imaginary parts of the local\ncomponent, respectively.\nThe black dotted line is $\\Sigma$ defined in Eq.~(\\ref{full-self-energy}) in the full GW approximation for comparison.\n(c) and (d) are the real and imaginary \nparts at several $k$ points.\n}\n\\label{sigH_nonlocal}\n\\end{figure*} \n\n\n\nFigure~\\ref{sigH_nonlocal} \nshows the local and nonlocal parts of $\\Delta \\Sigma_{\\rm H}$.\nThe frequency \ndependence\nof the local part (Figs.~\\ref{sigH_nonlocal} (a)(b)) \nis quite similar \nto the  frequency-dependent self-energies at various momenta\nin Fig.~\\ref{sigH}.\nOne immediately reads off an interesting property, which is\nakin to what has been found for the full GW self-energy in\nRef.~\\onlinecite{Tomczak-PRL2012}, namely that the nonlocal part of \n$\\Delta \\Sigma_{\\rm H}$ shows only weak frequency dependence \nas shown in Figs.~\\ref{sigH_nonlocal} (c)(d). \nThis explains why $Z_{\\rm H}$ \nis only weakly momentum-dependent.\nOn the other hand, $\\Delta \\Sigma_{\\rm H}^{\\rm nonlocal} (0)$ at $R$ is $\\sim 1$ eV larger than \nat $\\Gamma$, which causes the band widening effect.\n $\\Delta \\Sigma_{\\rm H}$\ncan thus to a good approximation be decomposed into a\nfrequency-dependent local part and a static nonlocal one:\n$\\Delta \\Sigma_{\\rm H}(k,\\omega) = \\Delta \\Sigma^{\\rm local}_{\\rm H}(\\omega) + \\Delta \\Sigma^{\\rm nonlocal}_{\\rm H}(k)$,\nwith $ \\Delta \\Sigma_{\\rm H}^{\\rm local}(\\omega) = \\sum_q  \\Delta \\Sigma_{\\rm H}(q,\\omega)$\n\nSince in the simple case of SrVO$_3$ where the t$_{2g}$ states are\ndegenerate, static local operators are scalar and are compensated \nby a chemical potential shift such that the correct particle\nnumber is obtained. The above band structure corresponding to\n$H_{\\rm LDA} - V_{\\rm xc} + \\Delta \\Sigma_{\\rm H}(k, \\omega=0)$ \nis thus identical to that of\n$H_{\\rm LDA} - V_{\\rm xc} + \\Delta \\Sigma^{\\rm nonlocal}_{\\rm H}(k)$.\nThe local dynamical part of $\\Delta \\Sigma_{\\rm H}$ then results in a \nnarrowing of this band structure by a factor $Z_{\\rm H} = 0.92$.\n\n\n\n\\begin{figure}[h]\n\\centering \n\\includegraphics[clip,width=0.45\\textwidth ]{fig10.eps} \n\\caption{(Color online)\n(a) Frequency dependence of the on-site effective interaction.\nPanel (b) is a low-energy zoom of panel (a). \n}\n\\label{W}\n\\end{figure} \n\n\nBefore turning to a discussion of the low-energy correction for the GW treatment\n$\\Delta \\Sigma_{\\rm L} =G^{(0)}(W-W_{\\rm L})$, we analyze the effective\ninteraction $W_{\\rm L}$ shown in Fig.~\\ref{W} \nin comparison to $W$ and $W_{\\rm H}$.\nThe fully screened Coulomb interaction $W$ displays the familiar\nshape of an interaction that is strongly screened at low energies\n(with a value of $0.88$ eV at $\\omega=0$), while retrieving the\nvalue of the bare Coulomb interaction $v$ ($\\sim 16.0$ eV) \nat high energy.\nThe crossover from the bare to the  \nscreened values takes place at the plasma energy of \nabout $15$ eV. This behavior has been discussed before\n\\cite{Review_SB2014}; we note in particular that the plasma \nfrequency is known from electron energy loss spectroscopy \nmeasurements of the related SrTiO$_3$ compound \\cite{vandermarel,kohiki}.\nThe partially screened interaction\n$W_{\\rm H}$ constructed within cRPA\nconverges to $W$ at high energies, but displays\nweaker screening effects at low energies (with a value of $3.5$ eV at $\\omega=0$),\nsince intra-t$_{2g}$ screening channels are excluded.\nAs was already anticipated in Sec.~\\ref{Outline},  $W_{\\rm L}$ can be interpreted as the screened interaction of \na low-energy model where a static interaction of value $W_{\\rm H}(\\omega=0)$\nhas been imposed as the bare interaction. Since now {\\it only} intra-t$_{2g}$\nscreening channels are active, screening takes place only at low\nenergies, where the scale is given by the t$_{2g}$ bandwidth.\nAlso plotted is the difference $W_{\\rm GW}^{\\rm dyn}=W-W_{d}$: except at low energies\nwhere the t$_{2g}$ screening channels come into play, its frequency\ndependence is essentially given by that of $W$, while the\nhigh-energy limit is the bare Coulomb interaction $v$ reduced\nby $W_{\\rm H}(0)$.\n\n\n\\begin{figure*}[htb]\n\\begin{center} \n\\includegraphics[width=0.9\\textwidth ]{fig11.eps} \n\\end{center} \n\\caption{(Color online)\nFrequency dependence of $-V_{\\rm xc}+ G^{(1)}W_{\\rm H}^{\\rm dyn}$ of SrVO$_3$.\n(a) and (b) are the real and imaginary local parts, respectively.\n(c) and (d) are the real and imaginary \nparts at several $k$ points.\n}\n\\label{DeltaSigma_L_Pert}\n\\end{figure*} \n\n\n\n\\begin{figure*}[htb]\n\\begin{center} \n\\includegraphics[width=0.9\\textwidth ]{fig12.eps} \n\\end{center} \n\\caption{(Color online)\nFrequency dependence of $-V_{\\rm xc}+ G^{(0)}W_{\\rm GW}^{\\rm dyn}$ of SrVO$_3$.\n(a) and (b) are the real and imaginary local parts, respectively.\n(c) and (d) are the real and imaginary \nparts at several $k$ points.\n}\n\\label{DeltaSigma_L_GW}\n\\end{figure*} \n\n\n\nThe frequency dependence of the real and imaginary parts of the low-energy self-energy $\\Delta \\Sigma_{\\rm L} = G^{(1)}W_{\\rm H}^{\\rm dyn}$ for the direct perturbative treatment is illustrated in Fig.~\\ref{DeltaSigma_L_Pert}. \n$\\Delta \\Sigma_{\\rm L} = G^{(0)}W_{\\rm GW}^{\\rm dyn}$ used\nfor the GW treatment is shown in Fig.~\\ref{DeltaSigma_L_GW}\nfor several choices of momenta.\n\\\\\nThe results for the perturbative and the GW treatment are nearly \nidentical.\nThe renormalization factor of $\\Delta \\Sigma_{\\rm L}^{\\rm nonlocal}$ \n(and $\\Delta \\Sigma_{\\rm L}$) is $\\sim 0.77$.\nThe zero-frequency shift\n${\\rm Re}\\Delta \\Sigma_{\\rm L}^{\\rm nonlocal} (0)$ at $R$ is $\\sim 3$ eV \nlarger than \nat $\\Gamma$.\nAgain, one sees how $\\Sigma_{\\rm L}$ separates into local dynamical\nand nonlocal static parts: \n$\\Delta \\Sigma_{\\rm L} = \\Delta \\Sigma^{\\rm nonlocal}_{\\rm L}(k) + \n\\Delta \\Sigma^{\\rm local}_{\\rm L}(\\omega)$.\nThis is expected since such a separation has been found within\nthe full GW calculation \\cite{TomczakPRB2014}, and is even\nmore plausible for a GW treatment within the t$_{2g}$ subspace.\n\n\n\n\\begin{figure}[h]\n\\centering \n\\includegraphics[clip,width=0.45\\textwidth ]{fig13.eps} \n\\caption{(Color online)\nBand structure  of $[\\epsilon_{\\rm DFT}-V_{\\rm xc}+\\Delta\\Sigma_{\\rm H}+\\Delta \\Sigma_{\\rm L}^{\\rm nonlocal}]Z_{\\rm HL}$ of SrVO$_3$ in the direct perturbative treatment.\nFor comparison, the LDA band structure is also given\n(black dotted line).\nThe energy is measured from the Fermi level.\n}\n\\label{DFT+Sigma_HL_Pert}\n\\end{figure} \n\n\n\n\\begin{figure}[h]\n\\centering \n\\includegraphics[clip,width=0.45\\textwidth ]{fig14.eps} \n\\caption{(Color online)\nBand structure  of $[\\epsilon_{\\rm DFT}-V_{\\rm xc}+\\Delta\\Sigma_{\\rm H}+\\Delta \\Sigma_{\\rm L}^{\\rm nonlocal}]Z_{\\rm HL}$ of SrVO$_3$ in the GW treatment.\nFor comparison, the LDA band structure is also given\n(black dotted line).\nThe energy is measured from the Fermi level.\n}\n\\label{DFT+Sigma_HL_GW}\n\\end{figure} \n\n\n\nThe corresponding band structures \nare given by \n $[\\epsilon_{\\rm DFT}+\\Delta\\Sigma_{\\rm H}+\\Delta \\Sigma_{\\rm L}^{\\rm nonlocal}]Z_{\\rm HL}$ in Fig.~\\ref{DFT+Sigma_HL_Pert} \nfor the perturbative treatment (see Eq.~(\\ref{DeltaSigmaLnonlocalPert})), and in Fig.~\\ref{DFT+Sigma_HL_GW} \nfor the GW treatment,\nwhere $\\Delta \\Sigma_{\\rm L}^{\\rm nonlocal}$ is defined in Eq.~(\\ref{DeltaSigLnonlocalGW}) and $Z_{\\rm HL}$ is around 0.92.\n$Z_{\\rm HL}$ is practically the same as $Z_{\\rm H}$, meaning that the nonlocal dynamical correction \nin the H-space is small, which justifies the GW perturbative treatment for the nonlocal part.\nOne finds that \nthe nonlocal part $\\Delta \\Sigma_{\\rm L}=G^{(0)}_{ll}(W-W_{\\rm L})$ narrows\nthe empty states but widens the occupied ones, resulting\nin an overall bandwidth of 3.2 eV.\nThe difference between the direct perturbation and the GW treatment is small.\n\n\n\n\\begin{figure}[h]\n\\centering \n\\includegraphics[clip,width=0.45\\textwidth ]{fig15.eps} \n\\caption{(Color online)\nBand structure  of $[\\epsilon_{\\rm DFT}-V_{\\rm xc}+\\Delta\\Sigma_{\\rm H}+\\Delta\\Sigma_{\\rm L}^{\\rm nonlocal}]Z_{\\rm HL}Z_B$ of SrVO$_3$ in the GW treatment.\nFor comparison, the LDA band structure is also given\n(black dotted line).\nThe energy is measured from the Fermi level.}\n\\label{DFT+DeltaSigmaL_GW_ZHL_ZB}\n\\end{figure} \n\n\nAfter including the effect of the local self-energy by the nonperturbative \nCasula trick, we show the dispersion\ngiven by \n$[\\epsilon_{\\rm DFT}-V_{\\rm xc}+\\Delta\\Sigma_{\\rm H}+\\Delta\\Sigma_{\\rm L}^{\\rm nonlocal}]Z_{\\rm HL}Z_B$ for the GW-type treatment in Fig.~\\ref{DFT+DeltaSigmaL_GW_ZHL_ZB}: \nThe GW-like treatment gives a band dispersion\nclose to the LDA one. \nThe $Z_B$ factor corresponding to the local dynamical part of \n$\\Delta \\Sigma_{\\rm L}$ amounts to $Z_B=0.7$.\nThe nonperturbative treatment \nresults in an LDA-like band dispersion for the empty states,\nbut a slightly narrower bandwidth in the occupied part.\nThe low-energy effective Hamiltonian at this level of treatment has a \nfrequency independent effective interaction both with local \nand nonlocal interaction given by the Fourier transform of \n$W_{\\rm H}(q,\\omega=0)$, which contains both direct and exchange interactions. \n\n\n\\begin{figure}[h]\n\\centering \n\\includegraphics[clip,width=0.45\\textwidth ]{fig16.eps} \n\\caption{(Color online)\nBand structure  of $[\\epsilon_{\\rm DFT}+\\Sigma_{\\rm corr}^{\\rm GW}]Z_{\\rm corr}Z_B$ of SrVO$_3$.\nFor comparison, the LDA band structure is also given\n(black dotted line).\nThe energy is measured from the Fermi level.}\n\\label{DFT+Sigma_corr_GW_Z}\n\\end{figure} \n\n\n\nWe finally show the dispersion\ngiven by Eq.~(\\ref{Teff_final}), namely\n$[\\epsilon_{\\rm DFT}+\\Sigma_{\\rm corr}^{\\rm GW}]Z_{\\rm corr}Z_B$ in Fig.~\\ref{DFT+Sigma_corr_GW_Z}. \nThe result shows an LDA-like band dispersion for the occupied states,\nbut a slightly wider bandwidth in the empty part, resulting in a bandwidth 14\\% wider in total than the LDA bandwidth. \nThe effective interaction of the low-energy effective Hamiltonian \nat this final level has\nfrequency independent local and nonlocal interactions \ngiven by the Fourier transform of $W_r(q,\\omega=0)$. \n\n\n\n\\begin{figure}[h]\n\\centering \n\\includegraphics[clip,width=0.45\\textwidth ]{fig17.eps} \n\\caption{(Color online)\nBand structure  of $[\\epsilon_{\\rm DFT}-V_{\\rm xc}+\\Sigma-(G^{(0)}W_{\\rm GW}^{\\rm dyn})^{\\rm nonlocal}]Z'Z_B$ of SrVO$_3$\n(Z' is the renormalization factor of $\\Sigma-(G^{(0)}W_{\\rm GW}^{\\rm dyn})^{\\rm nonlocal}$).\nFor comparison, the LDA band structure is also given\n(black dotted line).\nThe energy is measured from the Fermi level.\n}\n\\label{GW+ZB}\n\\end{figure} \n\n\n\nThe resulting effective Hamiltonian in the L-space \nconsists of single-particle and two-particle (interaction) parts.\nThe effective interaction is the same as in previous estimates \nby the cRPA in the literature, while the single-particle dispersion \nis revised after removing the double counting and taking into \naccount the frequency dependence of the effective interaction. The \nfinal effective bandwidth for SrVO$_3$ is,\nafter partial cancellation, \nslightly (14\\%) larger than the LDA bandwidth.  \nThis is a reasonable result because the LDA takes into account all \ncorrelations though insufficiently, while the present scheme by the constrained self-energy excludes the \ncorrelation effects arising from the L-space.\nAlthough the bandwidth derived for the effective model is slightly \nlarger than that of the LDA,  it is clear that the correlation effects \nare stronger than in the LDA or standard GW when the {\\it ab initio} \nmodel is solved by an accurate solver. In fact, if the effective \ninteraction contained in the final effective model is treated by \nthe GW scheme, one obtains the dispersion illustrated in \nFig.~\\ref{GW+ZB}, \nwhich is given from the self energy of the whole GW calculation $\\Sigma$ \nby correcting the local dynamical part $W-W_{\\rm L}$ by $Z_B$. \nThis indicates that even an insufficient treatment of the local \nstatic interaction by the GW scheme gives a dispersion with the bandwidth ($\\sim 1.9$ eV) narrower \nthan the LDA ($\\sim 2.5$ eV, Fig.~\\ref{lda}) and GW ($\\sim 2.1$ eV, Fig.~\\ref{fullGW}) dispersions.  \nA slightly ($\\sim 14$\\%) wider dispersion than that of the LDA bands obtained for the \neffective Hamiltonian accompanied with frequency independent effective \nCoulomb interactions without the exchange part may account for the slight \noverestimate of correlation effects in the literature mentioned in \nthe introduction.\nOur scheme offers an optimized way for the  derivation of \n{\\it ab initio} models for the L-space after eliminating the H-space\nin  a systematic fashion.\n\n\nOur findings of the band widening are consistent with studies\nbased on the combined GW+DMFT method in the literature \n\\cite{TomczakPRB2014}: There, it was argued that within GW+DMFT\nthe best effective Hamiltonian that DMFT should be performed on,\nis a one-body Hamiltonian corrected by the nonlocal part of the\nGW self-energy. The corresponding spectral function \n(see e.g.\nFigure 5 of Ref.~\\onlinecite{TomczakPRB2014}) displays \na broadening similar to \nFig.~\\ref{DFT+Sigma_HL_GW}.\nMost interestingly, our present calculations confirm a pronounced\nasymmetry observed in Ref.~\\onlinecite{TomczakPRB2014}, namely a stronger widening\neffect in the unoccupied part of the spectrum than in the occupied\none.\n\n",
        "label": "DFT+Sigma_HL_GW",
        "Descriptive_question1": "What is the overall bandwidth of the band structure shown in figure_14 for SrVO$_3$ in the GW treatment?",
        "Descriptive_question2": "What color is used for the LDA band structure in figure_14?",
        "Reasoning_question1": "How does the bandwidth of the GW treatment in figure_14 compare to the LDA bandwidth, and what might this indicate about the correlation effects in SrVO$_3$?",
        "Reasoning_question2": "Why might the band structure in figure_14 show a stronger widening effect in the unoccupied part of the spectrum compared to the occupied part for SrVO$_3$ in the GW treatment?",
        "Descriptive_answer1": "3.2 eV",
        "Descriptive_answer2": "black dotted line",
        "Reasoning_answer1": "The bandwidth in figure_14 is 3.2 eV, which is slightly wider (about 14% larger according to the text) than the LDA bandwidth of 2.5 eV. This suggests that the effective Hamiltonian including the GW corrections and removing double counting of correlations results in a bandwidth increase compared to LDA. It indicates that the LDA underestimates the bandwidth and correlations, and including GW corrections accounts for some correlations, although the final effective model still excludes certain correlations from the low-energy space, resulting in a slightly wider band overall.",
        "Reasoning_answer2": "The stronger widening effect in the unoccupied part compared to the occupied part arises because the nonlocal part of the low-energy self-energy corrections tend to narrow the empty states while widening the occupied ones, as mentioned in the text. Additionally, the exchange effects and frequency dependencies of the self-energies impact the unoccupied bands more strongly, causing an asymmetry in the band widening seen in figure_14. This reflects different correlation and screening effects for occupied and unoccupied states in SrVO3 when treated with GW corrections."
    },
    {
        "paper_id": "1511.03757.json",
        "image_id": "figure_15",
        "image_path": [
            "/home/yz979/scratch/chengye/scimolmo/test-images-tables/dataset_imagestables/1511.03757/images/fig15.eps"
        ],
        "caption": "(Color online)\nBand structure  of $[\\epsilon_{\\rm DFT}-V_{\\rm xc}+\\Delta\\Sigma_{\\rm H}+\\Delta\\Sigma_{\\rm L}^{\\rm nonlocal}]Z_{\\rm HL}Z_B$ of SrVO$_3$ in the GW treatment.\nFor comparison, the LDA band structure is also given\n(black dotted line).\nThe energy is measured from the Fermi level.",
        "classify": "Chart",
        "section_info": "6 Results\n\\section{Results}\n\\label{results}\n\n\nThe ternary 3d$^1$ transition metal oxide SrVO$_3$ has become\none of the ``drosophila compounds'' of correlated systems.\nIt is a correlated metal\nthat has been characterized using various experimental \n\\cite{Onoda, inoue, maiti, maiti2, sekiyama, takizawa, yoshida2010}.\nand theoretical techniques, see e.g.\n\\cite{nekrasov, anisimov, pavarini-d1, liebsch, lechermann-d1, trimarchi, aichhorn09, karolak, lee, held-optics, tomczak-gwdmft, gatti-guzzo, sakuma14, TomczakPRB2014}.\nA review of most of the available experimental and theoretical \ndata has been given recently in Ref.~\\onlinecite{TomczakPRB2014}.\nSrVO$_3$ displays Fermi liquid behavior up to\nremarkably high temperatures of the order of $200$ K \\cite{Onoda, inoue},\nwith a moderate mass enhancement of the order \nof $2$ \\cite{yoshida2010, vandermarel}.\nDetailed spectroscopic investigations have made it an\nideal test compound for modern many-body calculations, and \nmore and more refined dynamical mean field-based studies \nare available.\nThe majority of studies so far have focused on the t$_{2g}$-manifold\nwhich forms the states close to the Fermi level, and those will\nalso be the focus in the present investigations. \nNote however that this restriction quite severely limits the\nrange of validity of the low-energy description. Indeed, as\nshown recently~\\cite{tomczak-gwdmft, TomczakPRB2014} at\nenergies of about 2 to 3 eV above the Fermi level\nthe spectral properties are largely determined by the e$_g$\nstates. This has in particular led to a reinterpretation of\nan inverse photoemission feature at about 2.5 eV that was \nfrequently interpreted as an upper Hubbard band of t$_{2g}$\ncharacter in the earlier literature.\nHere, we use the compound only as an illustration of \nthe principles of constructing effective low-energy models,\nwithout being concerned with a description of spectra beyond\nthe pure t$_{2g}$ part.\n\n\nThe LDA band structure of SrVO$_3$ is shown in Fig.~\\ref{lda}.\nOne clearly distinguishes the three-fold degenerate manifold\nof t$_{2g}$ bands close to the Fermi level (chosen as the\nzero of energy), followed in the unoccupied part of the spectrum\nby the two e$_g$ bands. At about -2.5 eV the filled O-2p derived\nbands are visible.\n\nWhen a standard GW calculation is performed, see Fig.~\\ref{fullGW},\nthe bandwidth of the t$_{2g}$-states is reduced from 2.5 eV to\n2.1 eV, while the overall shape of the dispersion remains similar\nto the LDA one.\nThis is in agreement with previous GW calculations in the literature\n\\cite{tomczak-gwdmft, TomczakPRB2014, gatti-guzzo, sakuma14}.\n\n\n\nThe calculation is based on the full-potential linear muffin-tin orbitals (FP-LMTO) implementation.\n8$\\times$8$\\times$8 \\bm{k}-mesh is employed in both the DFT/LDA and the GW calculations.\nThe calculational details are the same as in Ref.~\\onlinecite{Hirayama}.\n\n\nWe now turn to a discussion of the band structure corresponding to \nthe Hamiltonians to be used as input for subsequent many-body \ncalculations for this material.\nWe will proceed step by step to analyze the influence of the various\ncorrection terms, with respect to the DFT starting point\n(which will be overlaid to the respective band structures).\n\n\n\n\\begin{figure}[h]\n\\centering \n\\includegraphics[clip,width=0.45\\textwidth ]{fig2.eps} \n\\caption{(Color online)\nKohn-Sham band structure of SrVO$_3$ in the LDA.\nThe energy is measured from the Fermi level.\n}\n\\label{lda}\n\\end{figure} \n\n\n\\begin{figure}[h]\n\\centering \n\\includegraphics[clip,width=0.45\\textwidth ]{fig3.eps} \n\\caption{(Color online)\nBand structure  of SrVO$_3$ in the one-shot GW approximation.\nFor comparison, the LDA band structure is also given\n(black dotted line).\nThe energy is measured from the Fermi level.\n}\n\\label{fullGW}\n\\end{figure} \n\n\n\n\n\n\n\\begin{figure}[h]\n\\centering \n\\includegraphics[clip,width=0.45\\textwidth ]{fig4.eps} \n\\caption{(Color online)\nBand structure of SrVO$_3$ corresponding to the LDA Hamiltonian\nfrom which the LDA exchange-correlation potential has been subtracted.\nFor comparison, the LDA band structure is also given\n(black dotted line).\nThe energy is measured from the Fermi level.\n}\n\\label{lda-vxc}\n\\end{figure} \n\n\n\\begin{figure}[h]\n\\centering \n\\includegraphics[clip,width=0.45\\textwidth ]{fig5.eps} \n\\caption{(Color online)\nBand structure  of SrVO$_3$ in the Hartree approximation.\nFor comparison, the LDA band structure is also given\n(black dotted line).\nThe energy is measured from the Fermi level.\n}\n\\label{hartree}\n\\end{figure} \n\n\nFigure~\\ref{lda-vxc} \ndisplays the band structure corresponding to\nthe LDA Hamiltonian from which the \nLDA exchange-correlation\npotential has been subtracted,  \nthat is, the eigenvalues of $H_{\\rm LDA}-V_{\\rm xc}$ where\nthese operators are evaluated for the self-consistent LDA\ndensity.\nThe subtraction of $V_{\\rm xc}$ widens the band structure from \nthe LDA bandwidth of 2.5 eV \nto more than twice this value: the new band\nwidth is 5.4 eV.\nThis indicates that the exchange-correlation potential $V_{\\rm xc}$ \nof the LDA is responsible for a substantial amount of \nthe band-narrowing effect\narising from electronic correlations, stressing the need \nto subtract the double counting in a consistent manner. \nFigure~\\ref{lda-vxc} is regarded as the starting point for the\nfollowing considerations.\n\nFor simple materials (see e.g. the calculations on Li in Ref.~\\onlinecite{Lithium})\nit has been noted in the literature that the Hartree band structure\nis close to the DFT one. This raises the question of the origin of the\nsubstantial band widening in the present case. Indeed, the present\ncalculation shown in Fig.~\\ref{lda-vxc} differs from a Hartree calculation\nonly \nby the fact that Fig.~\\ref{lda-vxc} is evaluated for the converged LDA density.\nWe have performed a test calculation where we plot the Hartree band\nstructure calculated for the converged Hartree density. The result\nis shown in Fig.~\\ref{hartree}. As seen from this plot, while the\nband is not fully as wide as in Fig.~\\ref{lda-vxc}, a substantial \nwidening is already present at this stage.\n\n\n\n\\begin{figure}[h]\n\\centering \n\\includegraphics[clip,width=0.45\\textwidth ]{fig6.eps} \n\\caption{(Color online)\nBand structure  of $H_{\\rm LDA}-V_{\\rm xc}+\\Delta \\Sigma_{\\rm H}(\\omega=0)$ of SrVO$_3$.\nFor comparison, the LDA band structure is also given\n(black dotted line).\nThe energy is measured from the Fermi level.\n}\n\\label{lda-vxc-sigH(0)}\n\\end{figure} \n\n\n\nStarting from the Hamiltonian without V$_{xc}$ (see the \ndispersion in Fig.~\\ref{lda-vxc}), we first take into account \nthe static part of the corrective self-energy $\\Delta \\Sigma_{\\rm H}$:\nIn Fig.~\\ref{lda-vxc-sigH(0)}, we plot the dispersion corresponding to\n$H_{\\rm LDA}-V_{\\rm xc}+\\Delta \\Sigma_{\\rm H}(\\omega=0)$. \nCompared to the dispersion of Fig.~\\ref{lda-vxc}, the overall band structure \nis narrowed to 3.7 eV. This value is, however, still considerably larger\nthan the LDA bandwidth (Fig.~\\ref{lda}). Although\n the electronic correlations coupling the H- and \nthe L space are included in Fig.~\\ref{lda-vxc-sigH(0)}, and narrow the band with respect to the case\nwhere V$_{xc}$ is taken out, correlations within\nthe L space are not included. \nThe LDA, on the other hand, at least partially\nincludes correlations within the L space, \nand these are very effective in narrowing the band.\nInterestingly, the bottom of the occupied band is quite\nexactly at the LDA value, and the remaining widening is\npurely in the unoccupied part.\n\n\n\\begin{figure*}[htb]\n\\begin{center} \n\\includegraphics[width=0.9\\textwidth ]{fig7.eps} \n\\end{center} \n\\caption{(Color online)\nFrequency dependence of $-V_{xc}+\\Delta \\Sigma_{\\rm H}$ for SrVO$_3$.\n(a) and (b) are the real and imaginary parts at the $\\Gamma$ point, \nrespectively,\nand (c) and (d) ((e) and (f)) are those at $X$ ($R$).\nFor comparison, the full GW self-energies are also given\n(black dotted line).\n}\n\\label{sigH}\n\\end{figure*} \n\n\n\n\\begin{figure}[h]\n\\centering \n\\includegraphics[clip,width=0.45\\textwidth ]{fig8.eps} \n\\caption{(Color online)\nBand structure  of $Z_{\\rm H}[H_{\\rm LDA}-V_{\\rm xc}+\\Delta\\Sigma_{\\rm H}(\\omega=0)]$ of SrVO$_3$.\nFor comparison, the LDA band structure is also given\n(black dotted line).\nThe energy is measured from the Fermi level.\n}\n\\label{ZH_SigmaH}\n\\end{figure} \n\n\nTo go further and in particular to analyze the dynamic\nbehavior of $\\Delta \\Sigma_{\\rm H}$ \nwe plot in Fig.~\\ref{sigH} \nthe self-energy corrections $\\Delta \\Sigma_{\\rm H}$ (with $\\Sigma$ (see Eq.~(\\ref{full-self-energy}))\nfor comparison) for the real and imaginary parts\nat several representative choices of momenta.\nThe frequency dependence is smooth around the Fermi level.\nIn particular, in contrast to the full self-energy,\nthere are no poles in $\\Delta \\Sigma_{\\rm H}$ \nin the low-energy region, thanks to the\nexclusion of fluctuations within the L-space.\nThe frequency dependence of the real part indicates that the \nlinearization \n$\\displaystyle{\\Delta \\Sigma_{\\rm H}(\\omega) \\sim \\Delta \\Sigma_{\\rm H}(\\omega=0)+[d\\Delta \\Sigma_{\\rm H}(\\omega)/d\\omega]_{\\omega=0}\\omega }$\noffers a reasonably good approximation. \nIn fact, the behavior of the constrained self-energy\nis much closer to linearity than that of \nthe full GW self-energy $\\Sigma$.\nThis is easily understood by the fact that the low-energy excitations are excluded in the present constrained self-energy $\\Delta\\Sigma_{\\rm H}$ analogously to an insulator, thus eliminating the strong frequency dependence of the constrained self-energy near the Fermi level. This is one of the consequences of the \ncontrollability and an advantage of the present MACE scheme, as \nmentioned in the introduction.\nIn the following discussion, we employ this linearized approximation.\nThen the renormalization factor $Z_{\\rm H}$ defined by Eq.~(\\ref{ZH})\nis interpreted as that from the contribution of the H-space. \nAfter \nthis renormalization factor is taken into account,\nthe effective Hamiltonian is given by $Z_{\\rm H}[H_{\\rm LDA}-V_{\\rm xc}+\\Delta\\Sigma_{\\rm H}(\\omega=0)]$ (see Fig.~\\ref{ZH_SigmaH}). \nInterestingly the renormalization factor $Z_{\\rm H}$=0.92 stays\nclose to one, so that\nthe bandwidth is only slightly reduced (to\n$3.4$ eV instead of the $3.7$ eV above). \nWe also note that $Z_{\\rm H}$ has weak momentum dependence\nas we reveal in the following.\n\n\n\n\\begin{figure*}[htb]\n\\begin{center} \n\\includegraphics[width=0.9\\textwidth ]{fig9.eps} \n\\end{center} \n\\caption{(Color online)\n{\\bf minor modification}\nFrequency dependence of $-V_{\\rm xc}+\\Delta \\Sigma_{\\rm H}$ of SrVO$_3$.\n(a) and (b) are the real and imaginary parts of the local\ncomponent, respectively.\nThe black dotted line is $\\Sigma$ defined in Eq.~(\\ref{full-self-energy}) in the full GW approximation for comparison.\n(c) and (d) are the real and imaginary \nparts at several $k$ points.\n}\n\\label{sigH_nonlocal}\n\\end{figure*} \n\n\n\nFigure~\\ref{sigH_nonlocal} \nshows the local and nonlocal parts of $\\Delta \\Sigma_{\\rm H}$.\nThe frequency \ndependence\nof the local part (Figs.~\\ref{sigH_nonlocal} (a)(b)) \nis quite similar \nto the  frequency-dependent self-energies at various momenta\nin Fig.~\\ref{sigH}.\nOne immediately reads off an interesting property, which is\nakin to what has been found for the full GW self-energy in\nRef.~\\onlinecite{Tomczak-PRL2012}, namely that the nonlocal part of \n$\\Delta \\Sigma_{\\rm H}$ shows only weak frequency dependence \nas shown in Figs.~\\ref{sigH_nonlocal} (c)(d). \nThis explains why $Z_{\\rm H}$ \nis only weakly momentum-dependent.\nOn the other hand, $\\Delta \\Sigma_{\\rm H}^{\\rm nonlocal} (0)$ at $R$ is $\\sim 1$ eV larger than \nat $\\Gamma$, which causes the band widening effect.\n $\\Delta \\Sigma_{\\rm H}$\ncan thus to a good approximation be decomposed into a\nfrequency-dependent local part and a static nonlocal one:\n$\\Delta \\Sigma_{\\rm H}(k,\\omega) = \\Delta \\Sigma^{\\rm local}_{\\rm H}(\\omega) + \\Delta \\Sigma^{\\rm nonlocal}_{\\rm H}(k)$,\nwith $ \\Delta \\Sigma_{\\rm H}^{\\rm local}(\\omega) = \\sum_q  \\Delta \\Sigma_{\\rm H}(q,\\omega)$\n\nSince in the simple case of SrVO$_3$ where the t$_{2g}$ states are\ndegenerate, static local operators are scalar and are compensated \nby a chemical potential shift such that the correct particle\nnumber is obtained. The above band structure corresponding to\n$H_{\\rm LDA} - V_{\\rm xc} + \\Delta \\Sigma_{\\rm H}(k, \\omega=0)$ \nis thus identical to that of\n$H_{\\rm LDA} - V_{\\rm xc} + \\Delta \\Sigma^{\\rm nonlocal}_{\\rm H}(k)$.\nThe local dynamical part of $\\Delta \\Sigma_{\\rm H}$ then results in a \nnarrowing of this band structure by a factor $Z_{\\rm H} = 0.92$.\n\n\n\n\\begin{figure}[h]\n\\centering \n\\includegraphics[clip,width=0.45\\textwidth ]{fig10.eps} \n\\caption{(Color online)\n(a) Frequency dependence of the on-site effective interaction.\nPanel (b) is a low-energy zoom of panel (a). \n}\n\\label{W}\n\\end{figure} \n\n\nBefore turning to a discussion of the low-energy correction for the GW treatment\n$\\Delta \\Sigma_{\\rm L} =G^{(0)}(W-W_{\\rm L})$, we analyze the effective\ninteraction $W_{\\rm L}$ shown in Fig.~\\ref{W} \nin comparison to $W$ and $W_{\\rm H}$.\nThe fully screened Coulomb interaction $W$ displays the familiar\nshape of an interaction that is strongly screened at low energies\n(with a value of $0.88$ eV at $\\omega=0$), while retrieving the\nvalue of the bare Coulomb interaction $v$ ($\\sim 16.0$ eV) \nat high energy.\nThe crossover from the bare to the  \nscreened values takes place at the plasma energy of \nabout $15$ eV. This behavior has been discussed before\n\\cite{Review_SB2014}; we note in particular that the plasma \nfrequency is known from electron energy loss spectroscopy \nmeasurements of the related SrTiO$_3$ compound \\cite{vandermarel,kohiki}.\nThe partially screened interaction\n$W_{\\rm H}$ constructed within cRPA\nconverges to $W$ at high energies, but displays\nweaker screening effects at low energies (with a value of $3.5$ eV at $\\omega=0$),\nsince intra-t$_{2g}$ screening channels are excluded.\nAs was already anticipated in Sec.~\\ref{Outline},  $W_{\\rm L}$ can be interpreted as the screened interaction of \na low-energy model where a static interaction of value $W_{\\rm H}(\\omega=0)$\nhas been imposed as the bare interaction. Since now {\\it only} intra-t$_{2g}$\nscreening channels are active, screening takes place only at low\nenergies, where the scale is given by the t$_{2g}$ bandwidth.\nAlso plotted is the difference $W_{\\rm GW}^{\\rm dyn}=W-W_{d}$: except at low energies\nwhere the t$_{2g}$ screening channels come into play, its frequency\ndependence is essentially given by that of $W$, while the\nhigh-energy limit is the bare Coulomb interaction $v$ reduced\nby $W_{\\rm H}(0)$.\n\n\n\\begin{figure*}[htb]\n\\begin{center} \n\\includegraphics[width=0.9\\textwidth ]{fig11.eps} \n\\end{center} \n\\caption{(Color online)\nFrequency dependence of $-V_{\\rm xc}+ G^{(1)}W_{\\rm H}^{\\rm dyn}$ of SrVO$_3$.\n(a) and (b) are the real and imaginary local parts, respectively.\n(c) and (d) are the real and imaginary \nparts at several $k$ points.\n}\n\\label{DeltaSigma_L_Pert}\n\\end{figure*} \n\n\n\n\\begin{figure*}[htb]\n\\begin{center} \n\\includegraphics[width=0.9\\textwidth ]{fig12.eps} \n\\end{center} \n\\caption{(Color online)\nFrequency dependence of $-V_{\\rm xc}+ G^{(0)}W_{\\rm GW}^{\\rm dyn}$ of SrVO$_3$.\n(a) and (b) are the real and imaginary local parts, respectively.\n(c) and (d) are the real and imaginary \nparts at several $k$ points.\n}\n\\label{DeltaSigma_L_GW}\n\\end{figure*} \n\n\n\nThe frequency dependence of the real and imaginary parts of the low-energy self-energy $\\Delta \\Sigma_{\\rm L} = G^{(1)}W_{\\rm H}^{\\rm dyn}$ for the direct perturbative treatment is illustrated in Fig.~\\ref{DeltaSigma_L_Pert}. \n$\\Delta \\Sigma_{\\rm L} = G^{(0)}W_{\\rm GW}^{\\rm dyn}$ used\nfor the GW treatment is shown in Fig.~\\ref{DeltaSigma_L_GW}\nfor several choices of momenta.\n\\\\\nThe results for the perturbative and the GW treatment are nearly \nidentical.\nThe renormalization factor of $\\Delta \\Sigma_{\\rm L}^{\\rm nonlocal}$ \n(and $\\Delta \\Sigma_{\\rm L}$) is $\\sim 0.77$.\nThe zero-frequency shift\n${\\rm Re}\\Delta \\Sigma_{\\rm L}^{\\rm nonlocal} (0)$ at $R$ is $\\sim 3$ eV \nlarger than \nat $\\Gamma$.\nAgain, one sees how $\\Sigma_{\\rm L}$ separates into local dynamical\nand nonlocal static parts: \n$\\Delta \\Sigma_{\\rm L} = \\Delta \\Sigma^{\\rm nonlocal}_{\\rm L}(k) + \n\\Delta \\Sigma^{\\rm local}_{\\rm L}(\\omega)$.\nThis is expected since such a separation has been found within\nthe full GW calculation \\cite{TomczakPRB2014}, and is even\nmore plausible for a GW treatment within the t$_{2g}$ subspace.\n\n\n\n\\begin{figure}[h]\n\\centering \n\\includegraphics[clip,width=0.45\\textwidth ]{fig13.eps} \n\\caption{(Color online)\nBand structure  of $[\\epsilon_{\\rm DFT}-V_{\\rm xc}+\\Delta\\Sigma_{\\rm H}+\\Delta \\Sigma_{\\rm L}^{\\rm nonlocal}]Z_{\\rm HL}$ of SrVO$_3$ in the direct perturbative treatment.\nFor comparison, the LDA band structure is also given\n(black dotted line).\nThe energy is measured from the Fermi level.\n}\n\\label{DFT+Sigma_HL_Pert}\n\\end{figure} \n\n\n\n\\begin{figure}[h]\n\\centering \n\\includegraphics[clip,width=0.45\\textwidth ]{fig14.eps} \n\\caption{(Color online)\nBand structure  of $[\\epsilon_{\\rm DFT}-V_{\\rm xc}+\\Delta\\Sigma_{\\rm H}+\\Delta \\Sigma_{\\rm L}^{\\rm nonlocal}]Z_{\\rm HL}$ of SrVO$_3$ in the GW treatment.\nFor comparison, the LDA band structure is also given\n(black dotted line).\nThe energy is measured from the Fermi level.\n}\n\\label{DFT+Sigma_HL_GW}\n\\end{figure} \n\n\n\nThe corresponding band structures \nare given by \n $[\\epsilon_{\\rm DFT}+\\Delta\\Sigma_{\\rm H}+\\Delta \\Sigma_{\\rm L}^{\\rm nonlocal}]Z_{\\rm HL}$ in Fig.~\\ref{DFT+Sigma_HL_Pert} \nfor the perturbative treatment (see Eq.~(\\ref{DeltaSigmaLnonlocalPert})), and in Fig.~\\ref{DFT+Sigma_HL_GW} \nfor the GW treatment,\nwhere $\\Delta \\Sigma_{\\rm L}^{\\rm nonlocal}$ is defined in Eq.~(\\ref{DeltaSigLnonlocalGW}) and $Z_{\\rm HL}$ is around 0.92.\n$Z_{\\rm HL}$ is practically the same as $Z_{\\rm H}$, meaning that the nonlocal dynamical correction \nin the H-space is small, which justifies the GW perturbative treatment for the nonlocal part.\nOne finds that \nthe nonlocal part $\\Delta \\Sigma_{\\rm L}=G^{(0)}_{ll}(W-W_{\\rm L})$ narrows\nthe empty states but widens the occupied ones, resulting\nin an overall bandwidth of 3.2 eV.\nThe difference between the direct perturbation and the GW treatment is small.\n\n\n\n\\begin{figure}[h]\n\\centering \n\\includegraphics[clip,width=0.45\\textwidth ]{fig15.eps} \n\\caption{(Color online)\nBand structure  of $[\\epsilon_{\\rm DFT}-V_{\\rm xc}+\\Delta\\Sigma_{\\rm H}+\\Delta\\Sigma_{\\rm L}^{\\rm nonlocal}]Z_{\\rm HL}Z_B$ of SrVO$_3$ in the GW treatment.\nFor comparison, the LDA band structure is also given\n(black dotted line).\nThe energy is measured from the Fermi level.}\n\\label{DFT+DeltaSigmaL_GW_ZHL_ZB}\n\\end{figure} \n\n\nAfter including the effect of the local self-energy by the nonperturbative \nCasula trick, we show the dispersion\ngiven by \n$[\\epsilon_{\\rm DFT}-V_{\\rm xc}+\\Delta\\Sigma_{\\rm H}+\\Delta\\Sigma_{\\rm L}^{\\rm nonlocal}]Z_{\\rm HL}Z_B$ for the GW-type treatment in Fig.~\\ref{DFT+DeltaSigmaL_GW_ZHL_ZB}: \nThe GW-like treatment gives a band dispersion\nclose to the LDA one. \nThe $Z_B$ factor corresponding to the local dynamical part of \n$\\Delta \\Sigma_{\\rm L}$ amounts to $Z_B=0.7$.\nThe nonperturbative treatment \nresults in an LDA-like band dispersion for the empty states,\nbut a slightly narrower bandwidth in the occupied part.\nThe low-energy effective Hamiltonian at this level of treatment has a \nfrequency independent effective interaction both with local \nand nonlocal interaction given by the Fourier transform of \n$W_{\\rm H}(q,\\omega=0)$, which contains both direct and exchange interactions. \n\n\n\\begin{figure}[h]\n\\centering \n\\includegraphics[clip,width=0.45\\textwidth ]{fig16.eps} \n\\caption{(Color online)\nBand structure  of $[\\epsilon_{\\rm DFT}+\\Sigma_{\\rm corr}^{\\rm GW}]Z_{\\rm corr}Z_B$ of SrVO$_3$.\nFor comparison, the LDA band structure is also given\n(black dotted line).\nThe energy is measured from the Fermi level.}\n\\label{DFT+Sigma_corr_GW_Z}\n\\end{figure} \n\n\n\nWe finally show the dispersion\ngiven by Eq.~(\\ref{Teff_final}), namely\n$[\\epsilon_{\\rm DFT}+\\Sigma_{\\rm corr}^{\\rm GW}]Z_{\\rm corr}Z_B$ in Fig.~\\ref{DFT+Sigma_corr_GW_Z}. \nThe result shows an LDA-like band dispersion for the occupied states,\nbut a slightly wider bandwidth in the empty part, resulting in a bandwidth 14\\% wider in total than the LDA bandwidth. \nThe effective interaction of the low-energy effective Hamiltonian \nat this final level has\nfrequency independent local and nonlocal interactions \ngiven by the Fourier transform of $W_r(q,\\omega=0)$. \n\n\n\n\\begin{figure}[h]\n\\centering \n\\includegraphics[clip,width=0.45\\textwidth ]{fig17.eps} \n\\caption{(Color online)\nBand structure  of $[\\epsilon_{\\rm DFT}-V_{\\rm xc}+\\Sigma-(G^{(0)}W_{\\rm GW}^{\\rm dyn})^{\\rm nonlocal}]Z'Z_B$ of SrVO$_3$\n(Z' is the renormalization factor of $\\Sigma-(G^{(0)}W_{\\rm GW}^{\\rm dyn})^{\\rm nonlocal}$).\nFor comparison, the LDA band structure is also given\n(black dotted line).\nThe energy is measured from the Fermi level.\n}\n\\label{GW+ZB}\n\\end{figure} \n\n\n\nThe resulting effective Hamiltonian in the L-space \nconsists of single-particle and two-particle (interaction) parts.\nThe effective interaction is the same as in previous estimates \nby the cRPA in the literature, while the single-particle dispersion \nis revised after removing the double counting and taking into \naccount the frequency dependence of the effective interaction. The \nfinal effective bandwidth for SrVO$_3$ is,\nafter partial cancellation, \nslightly (14\\%) larger than the LDA bandwidth.  \nThis is a reasonable result because the LDA takes into account all \ncorrelations though insufficiently, while the present scheme by the constrained self-energy excludes the \ncorrelation effects arising from the L-space.\nAlthough the bandwidth derived for the effective model is slightly \nlarger than that of the LDA,  it is clear that the correlation effects \nare stronger than in the LDA or standard GW when the {\\it ab initio} \nmodel is solved by an accurate solver. In fact, if the effective \ninteraction contained in the final effective model is treated by \nthe GW scheme, one obtains the dispersion illustrated in \nFig.~\\ref{GW+ZB}, \nwhich is given from the self energy of the whole GW calculation $\\Sigma$ \nby correcting the local dynamical part $W-W_{\\rm L}$ by $Z_B$. \nThis indicates that even an insufficient treatment of the local \nstatic interaction by the GW scheme gives a dispersion with the bandwidth ($\\sim 1.9$ eV) narrower \nthan the LDA ($\\sim 2.5$ eV, Fig.~\\ref{lda}) and GW ($\\sim 2.1$ eV, Fig.~\\ref{fullGW}) dispersions.  \nA slightly ($\\sim 14$\\%) wider dispersion than that of the LDA bands obtained for the \neffective Hamiltonian accompanied with frequency independent effective \nCoulomb interactions without the exchange part may account for the slight \noverestimate of correlation effects in the literature mentioned in \nthe introduction.\nOur scheme offers an optimized way for the  derivation of \n{\\it ab initio} models for the L-space after eliminating the H-space\nin  a systematic fashion.\n\n\nOur findings of the band widening are consistent with studies\nbased on the combined GW+DMFT method in the literature \n\\cite{TomczakPRB2014}: There, it was argued that within GW+DMFT\nthe best effective Hamiltonian that DMFT should be performed on,\nis a one-body Hamiltonian corrected by the nonlocal part of the\nGW self-energy. The corresponding spectral function \n(see e.g.\nFigure 5 of Ref.~\\onlinecite{TomczakPRB2014}) displays \na broadening similar to \nFig.~\\ref{DFT+Sigma_HL_GW}.\nMost interestingly, our present calculations confirm a pronounced\nasymmetry observed in Ref.~\\onlinecite{TomczakPRB2014}, namely a stronger widening\neffect in the unoccupied part of the spectrum than in the occupied\none.\n\n",
        "label": "DFT+DeltaSigmaL_GW_ZHL_ZB",
        "Descriptive_question1": "What is the bandwidth of the t_{2g} states in the GW treatment shown in figure_15?",
        "Descriptive_question2": "What color is used for the LDA band structure line in figure_15?",
        "Reasoning_question1": "How does the bandwidth of the GW treatment in figure_15 compare to the LDA bandwidth, and what might this indicate about electronic correlations?",
        "Reasoning_question2": "Why might the band dispersion in the unoccupied part of the spectrum in figure_15 show a different behavior compared to the occupied part, based on the provided context?",
        "Descriptive_answer1": "3.2 eV",
        "Descriptive_answer2": "black dotted line",
        "Reasoning_answer1": "The bandwidth of the t_{2g} states in the GW treatment (figure_15) is 3.2 eV, which is slightly wider than the LDA bandwidth of 2.5 eV. This indicates that the GW treatment, by correcting for double counting and including nonlocal self-energy effects, results in a band structure with an overall slightly increased bandwidth compared to LDA. This suggests that while LDA captures some correlation effects, it underestimates the bandwidth, and the GW treatment reveals stronger correlations lead to a slight band widening after partial cancellation of effects.",
        "Reasoning_answer2": "The different behavior of the band dispersion in the unoccupied part versus the occupied part, as seen in figure_15, may be due to the nonlocal part of the low-energy self-energy corrections which tend to narrow the empty (unoccupied) states but widen the occupied ones. Additionally, the electronic correlations within the low-energy space affect occupied and unoccupied bands differently, leading to asymmetry. This is consistent with the discussion that the band widening effect is stronger in the unoccupied part, reflecting the interplay between nonlocal self-energy effects and frequency-dependent screening that differentially modify the electron and hole parts of the spectral function."
    },
    {
        "paper_id": "1511.03757.json",
        "image_id": "figure_16",
        "image_path": [
            "/home/yz979/scratch/chengye/scimolmo/test-images-tables/dataset_imagestables/1511.03757/images/fig16.eps"
        ],
        "caption": "(Color online)\nBand structure  of $[\\epsilon_{\\rm DFT}+\\Sigma_{\\rm corr}^{\\rm GW}]Z_{\\rm corr}Z_B$ of SrVO$_3$.\nFor comparison, the LDA band structure is also given\n(black dotted line).\nThe energy is measured from the Fermi level.",
        "classify": "Chart",
        "section_info": "6 Results\n\\section{Results}\n\\label{results}\n\n\nThe ternary 3d$^1$ transition metal oxide SrVO$_3$ has become\none of the ``drosophila compounds'' of correlated systems.\nIt is a correlated metal\nthat has been characterized using various experimental \n\\cite{Onoda, inoue, maiti, maiti2, sekiyama, takizawa, yoshida2010}.\nand theoretical techniques, see e.g.\n\\cite{nekrasov, anisimov, pavarini-d1, liebsch, lechermann-d1, trimarchi, aichhorn09, karolak, lee, held-optics, tomczak-gwdmft, gatti-guzzo, sakuma14, TomczakPRB2014}.\nA review of most of the available experimental and theoretical \ndata has been given recently in Ref.~\\onlinecite{TomczakPRB2014}.\nSrVO$_3$ displays Fermi liquid behavior up to\nremarkably high temperatures of the order of $200$ K \\cite{Onoda, inoue},\nwith a moderate mass enhancement of the order \nof $2$ \\cite{yoshida2010, vandermarel}.\nDetailed spectroscopic investigations have made it an\nideal test compound for modern many-body calculations, and \nmore and more refined dynamical mean field-based studies \nare available.\nThe majority of studies so far have focused on the t$_{2g}$-manifold\nwhich forms the states close to the Fermi level, and those will\nalso be the focus in the present investigations. \nNote however that this restriction quite severely limits the\nrange of validity of the low-energy description. Indeed, as\nshown recently~\\cite{tomczak-gwdmft, TomczakPRB2014} at\nenergies of about 2 to 3 eV above the Fermi level\nthe spectral properties are largely determined by the e$_g$\nstates. This has in particular led to a reinterpretation of\nan inverse photoemission feature at about 2.5 eV that was \nfrequently interpreted as an upper Hubbard band of t$_{2g}$\ncharacter in the earlier literature.\nHere, we use the compound only as an illustration of \nthe principles of constructing effective low-energy models,\nwithout being concerned with a description of spectra beyond\nthe pure t$_{2g}$ part.\n\n\nThe LDA band structure of SrVO$_3$ is shown in Fig.~\\ref{lda}.\nOne clearly distinguishes the three-fold degenerate manifold\nof t$_{2g}$ bands close to the Fermi level (chosen as the\nzero of energy), followed in the unoccupied part of the spectrum\nby the two e$_g$ bands. At about -2.5 eV the filled O-2p derived\nbands are visible.\n\nWhen a standard GW calculation is performed, see Fig.~\\ref{fullGW},\nthe bandwidth of the t$_{2g}$-states is reduced from 2.5 eV to\n2.1 eV, while the overall shape of the dispersion remains similar\nto the LDA one.\nThis is in agreement with previous GW calculations in the literature\n\\cite{tomczak-gwdmft, TomczakPRB2014, gatti-guzzo, sakuma14}.\n\n\n\nThe calculation is based on the full-potential linear muffin-tin orbitals (FP-LMTO) implementation.\n8$\\times$8$\\times$8 \\bm{k}-mesh is employed in both the DFT/LDA and the GW calculations.\nThe calculational details are the same as in Ref.~\\onlinecite{Hirayama}.\n\n\nWe now turn to a discussion of the band structure corresponding to \nthe Hamiltonians to be used as input for subsequent many-body \ncalculations for this material.\nWe will proceed step by step to analyze the influence of the various\ncorrection terms, with respect to the DFT starting point\n(which will be overlaid to the respective band structures).\n\n\n\n\\begin{figure}[h]\n\\centering \n\\includegraphics[clip,width=0.45\\textwidth ]{fig2.eps} \n\\caption{(Color online)\nKohn-Sham band structure of SrVO$_3$ in the LDA.\nThe energy is measured from the Fermi level.\n}\n\\label{lda}\n\\end{figure} \n\n\n\\begin{figure}[h]\n\\centering \n\\includegraphics[clip,width=0.45\\textwidth ]{fig3.eps} \n\\caption{(Color online)\nBand structure  of SrVO$_3$ in the one-shot GW approximation.\nFor comparison, the LDA band structure is also given\n(black dotted line).\nThe energy is measured from the Fermi level.\n}\n\\label{fullGW}\n\\end{figure} \n\n\n\n\n\n\n\\begin{figure}[h]\n\\centering \n\\includegraphics[clip,width=0.45\\textwidth ]{fig4.eps} \n\\caption{(Color online)\nBand structure of SrVO$_3$ corresponding to the LDA Hamiltonian\nfrom which the LDA exchange-correlation potential has been subtracted.\nFor comparison, the LDA band structure is also given\n(black dotted line).\nThe energy is measured from the Fermi level.\n}\n\\label{lda-vxc}\n\\end{figure} \n\n\n\\begin{figure}[h]\n\\centering \n\\includegraphics[clip,width=0.45\\textwidth ]{fig5.eps} \n\\caption{(Color online)\nBand structure  of SrVO$_3$ in the Hartree approximation.\nFor comparison, the LDA band structure is also given\n(black dotted line).\nThe energy is measured from the Fermi level.\n}\n\\label{hartree}\n\\end{figure} \n\n\nFigure~\\ref{lda-vxc} \ndisplays the band structure corresponding to\nthe LDA Hamiltonian from which the \nLDA exchange-correlation\npotential has been subtracted,  \nthat is, the eigenvalues of $H_{\\rm LDA}-V_{\\rm xc}$ where\nthese operators are evaluated for the self-consistent LDA\ndensity.\nThe subtraction of $V_{\\rm xc}$ widens the band structure from \nthe LDA bandwidth of 2.5 eV \nto more than twice this value: the new band\nwidth is 5.4 eV.\nThis indicates that the exchange-correlation potential $V_{\\rm xc}$ \nof the LDA is responsible for a substantial amount of \nthe band-narrowing effect\narising from electronic correlations, stressing the need \nto subtract the double counting in a consistent manner. \nFigure~\\ref{lda-vxc} is regarded as the starting point for the\nfollowing considerations.\n\nFor simple materials (see e.g. the calculations on Li in Ref.~\\onlinecite{Lithium})\nit has been noted in the literature that the Hartree band structure\nis close to the DFT one. This raises the question of the origin of the\nsubstantial band widening in the present case. Indeed, the present\ncalculation shown in Fig.~\\ref{lda-vxc} differs from a Hartree calculation\nonly \nby the fact that Fig.~\\ref{lda-vxc} is evaluated for the converged LDA density.\nWe have performed a test calculation where we plot the Hartree band\nstructure calculated for the converged Hartree density. The result\nis shown in Fig.~\\ref{hartree}. As seen from this plot, while the\nband is not fully as wide as in Fig.~\\ref{lda-vxc}, a substantial \nwidening is already present at this stage.\n\n\n\n\\begin{figure}[h]\n\\centering \n\\includegraphics[clip,width=0.45\\textwidth ]{fig6.eps} \n\\caption{(Color online)\nBand structure  of $H_{\\rm LDA}-V_{\\rm xc}+\\Delta \\Sigma_{\\rm H}(\\omega=0)$ of SrVO$_3$.\nFor comparison, the LDA band structure is also given\n(black dotted line).\nThe energy is measured from the Fermi level.\n}\n\\label{lda-vxc-sigH(0)}\n\\end{figure} \n\n\n\nStarting from the Hamiltonian without V$_{xc}$ (see the \ndispersion in Fig.~\\ref{lda-vxc}), we first take into account \nthe static part of the corrective self-energy $\\Delta \\Sigma_{\\rm H}$:\nIn Fig.~\\ref{lda-vxc-sigH(0)}, we plot the dispersion corresponding to\n$H_{\\rm LDA}-V_{\\rm xc}+\\Delta \\Sigma_{\\rm H}(\\omega=0)$. \nCompared to the dispersion of Fig.~\\ref{lda-vxc}, the overall band structure \nis narrowed to 3.7 eV. This value is, however, still considerably larger\nthan the LDA bandwidth (Fig.~\\ref{lda}). Although\n the electronic correlations coupling the H- and \nthe L space are included in Fig.~\\ref{lda-vxc-sigH(0)}, and narrow the band with respect to the case\nwhere V$_{xc}$ is taken out, correlations within\nthe L space are not included. \nThe LDA, on the other hand, at least partially\nincludes correlations within the L space, \nand these are very effective in narrowing the band.\nInterestingly, the bottom of the occupied band is quite\nexactly at the LDA value, and the remaining widening is\npurely in the unoccupied part.\n\n\n\\begin{figure*}[htb]\n\\begin{center} \n\\includegraphics[width=0.9\\textwidth ]{fig7.eps} \n\\end{center} \n\\caption{(Color online)\nFrequency dependence of $-V_{xc}+\\Delta \\Sigma_{\\rm H}$ for SrVO$_3$.\n(a) and (b) are the real and imaginary parts at the $\\Gamma$ point, \nrespectively,\nand (c) and (d) ((e) and (f)) are those at $X$ ($R$).\nFor comparison, the full GW self-energies are also given\n(black dotted line).\n}\n\\label{sigH}\n\\end{figure*} \n\n\n\n\\begin{figure}[h]\n\\centering \n\\includegraphics[clip,width=0.45\\textwidth ]{fig8.eps} \n\\caption{(Color online)\nBand structure  of $Z_{\\rm H}[H_{\\rm LDA}-V_{\\rm xc}+\\Delta\\Sigma_{\\rm H}(\\omega=0)]$ of SrVO$_3$.\nFor comparison, the LDA band structure is also given\n(black dotted line).\nThe energy is measured from the Fermi level.\n}\n\\label{ZH_SigmaH}\n\\end{figure} \n\n\nTo go further and in particular to analyze the dynamic\nbehavior of $\\Delta \\Sigma_{\\rm H}$ \nwe plot in Fig.~\\ref{sigH} \nthe self-energy corrections $\\Delta \\Sigma_{\\rm H}$ (with $\\Sigma$ (see Eq.~(\\ref{full-self-energy}))\nfor comparison) for the real and imaginary parts\nat several representative choices of momenta.\nThe frequency dependence is smooth around the Fermi level.\nIn particular, in contrast to the full self-energy,\nthere are no poles in $\\Delta \\Sigma_{\\rm H}$ \nin the low-energy region, thanks to the\nexclusion of fluctuations within the L-space.\nThe frequency dependence of the real part indicates that the \nlinearization \n$\\displaystyle{\\Delta \\Sigma_{\\rm H}(\\omega) \\sim \\Delta \\Sigma_{\\rm H}(\\omega=0)+[d\\Delta \\Sigma_{\\rm H}(\\omega)/d\\omega]_{\\omega=0}\\omega }$\noffers a reasonably good approximation. \nIn fact, the behavior of the constrained self-energy\nis much closer to linearity than that of \nthe full GW self-energy $\\Sigma$.\nThis is easily understood by the fact that the low-energy excitations are excluded in the present constrained self-energy $\\Delta\\Sigma_{\\rm H}$ analogously to an insulator, thus eliminating the strong frequency dependence of the constrained self-energy near the Fermi level. This is one of the consequences of the \ncontrollability and an advantage of the present MACE scheme, as \nmentioned in the introduction.\nIn the following discussion, we employ this linearized approximation.\nThen the renormalization factor $Z_{\\rm H}$ defined by Eq.~(\\ref{ZH})\nis interpreted as that from the contribution of the H-space. \nAfter \nthis renormalization factor is taken into account,\nthe effective Hamiltonian is given by $Z_{\\rm H}[H_{\\rm LDA}-V_{\\rm xc}+\\Delta\\Sigma_{\\rm H}(\\omega=0)]$ (see Fig.~\\ref{ZH_SigmaH}). \nInterestingly the renormalization factor $Z_{\\rm H}$=0.92 stays\nclose to one, so that\nthe bandwidth is only slightly reduced (to\n$3.4$ eV instead of the $3.7$ eV above). \nWe also note that $Z_{\\rm H}$ has weak momentum dependence\nas we reveal in the following.\n\n\n\n\\begin{figure*}[htb]\n\\begin{center} \n\\includegraphics[width=0.9\\textwidth ]{fig9.eps} \n\\end{center} \n\\caption{(Color online)\n{\\bf minor modification}\nFrequency dependence of $-V_{\\rm xc}+\\Delta \\Sigma_{\\rm H}$ of SrVO$_3$.\n(a) and (b) are the real and imaginary parts of the local\ncomponent, respectively.\nThe black dotted line is $\\Sigma$ defined in Eq.~(\\ref{full-self-energy}) in the full GW approximation for comparison.\n(c) and (d) are the real and imaginary \nparts at several $k$ points.\n}\n\\label{sigH_nonlocal}\n\\end{figure*} \n\n\n\nFigure~\\ref{sigH_nonlocal} \nshows the local and nonlocal parts of $\\Delta \\Sigma_{\\rm H}$.\nThe frequency \ndependence\nof the local part (Figs.~\\ref{sigH_nonlocal} (a)(b)) \nis quite similar \nto the  frequency-dependent self-energies at various momenta\nin Fig.~\\ref{sigH}.\nOne immediately reads off an interesting property, which is\nakin to what has been found for the full GW self-energy in\nRef.~\\onlinecite{Tomczak-PRL2012}, namely that the nonlocal part of \n$\\Delta \\Sigma_{\\rm H}$ shows only weak frequency dependence \nas shown in Figs.~\\ref{sigH_nonlocal} (c)(d). \nThis explains why $Z_{\\rm H}$ \nis only weakly momentum-dependent.\nOn the other hand, $\\Delta \\Sigma_{\\rm H}^{\\rm nonlocal} (0)$ at $R$ is $\\sim 1$ eV larger than \nat $\\Gamma$, which causes the band widening effect.\n $\\Delta \\Sigma_{\\rm H}$\ncan thus to a good approximation be decomposed into a\nfrequency-dependent local part and a static nonlocal one:\n$\\Delta \\Sigma_{\\rm H}(k,\\omega) = \\Delta \\Sigma^{\\rm local}_{\\rm H}(\\omega) + \\Delta \\Sigma^{\\rm nonlocal}_{\\rm H}(k)$,\nwith $ \\Delta \\Sigma_{\\rm H}^{\\rm local}(\\omega) = \\sum_q  \\Delta \\Sigma_{\\rm H}(q,\\omega)$\n\nSince in the simple case of SrVO$_3$ where the t$_{2g}$ states are\ndegenerate, static local operators are scalar and are compensated \nby a chemical potential shift such that the correct particle\nnumber is obtained. The above band structure corresponding to\n$H_{\\rm LDA} - V_{\\rm xc} + \\Delta \\Sigma_{\\rm H}(k, \\omega=0)$ \nis thus identical to that of\n$H_{\\rm LDA} - V_{\\rm xc} + \\Delta \\Sigma^{\\rm nonlocal}_{\\rm H}(k)$.\nThe local dynamical part of $\\Delta \\Sigma_{\\rm H}$ then results in a \nnarrowing of this band structure by a factor $Z_{\\rm H} = 0.92$.\n\n\n\n\\begin{figure}[h]\n\\centering \n\\includegraphics[clip,width=0.45\\textwidth ]{fig10.eps} \n\\caption{(Color online)\n(a) Frequency dependence of the on-site effective interaction.\nPanel (b) is a low-energy zoom of panel (a). \n}\n\\label{W}\n\\end{figure} \n\n\nBefore turning to a discussion of the low-energy correction for the GW treatment\n$\\Delta \\Sigma_{\\rm L} =G^{(0)}(W-W_{\\rm L})$, we analyze the effective\ninteraction $W_{\\rm L}$ shown in Fig.~\\ref{W} \nin comparison to $W$ and $W_{\\rm H}$.\nThe fully screened Coulomb interaction $W$ displays the familiar\nshape of an interaction that is strongly screened at low energies\n(with a value of $0.88$ eV at $\\omega=0$), while retrieving the\nvalue of the bare Coulomb interaction $v$ ($\\sim 16.0$ eV) \nat high energy.\nThe crossover from the bare to the  \nscreened values takes place at the plasma energy of \nabout $15$ eV. This behavior has been discussed before\n\\cite{Review_SB2014}; we note in particular that the plasma \nfrequency is known from electron energy loss spectroscopy \nmeasurements of the related SrTiO$_3$ compound \\cite{vandermarel,kohiki}.\nThe partially screened interaction\n$W_{\\rm H}$ constructed within cRPA\nconverges to $W$ at high energies, but displays\nweaker screening effects at low energies (with a value of $3.5$ eV at $\\omega=0$),\nsince intra-t$_{2g}$ screening channels are excluded.\nAs was already anticipated in Sec.~\\ref{Outline},  $W_{\\rm L}$ can be interpreted as the screened interaction of \na low-energy model where a static interaction of value $W_{\\rm H}(\\omega=0)$\nhas been imposed as the bare interaction. Since now {\\it only} intra-t$_{2g}$\nscreening channels are active, screening takes place only at low\nenergies, where the scale is given by the t$_{2g}$ bandwidth.\nAlso plotted is the difference $W_{\\rm GW}^{\\rm dyn}=W-W_{d}$: except at low energies\nwhere the t$_{2g}$ screening channels come into play, its frequency\ndependence is essentially given by that of $W$, while the\nhigh-energy limit is the bare Coulomb interaction $v$ reduced\nby $W_{\\rm H}(0)$.\n\n\n\\begin{figure*}[htb]\n\\begin{center} \n\\includegraphics[width=0.9\\textwidth ]{fig11.eps} \n\\end{center} \n\\caption{(Color online)\nFrequency dependence of $-V_{\\rm xc}+ G^{(1)}W_{\\rm H}^{\\rm dyn}$ of SrVO$_3$.\n(a) and (b) are the real and imaginary local parts, respectively.\n(c) and (d) are the real and imaginary \nparts at several $k$ points.\n}\n\\label{DeltaSigma_L_Pert}\n\\end{figure*} \n\n\n\n\\begin{figure*}[htb]\n\\begin{center} \n\\includegraphics[width=0.9\\textwidth ]{fig12.eps} \n\\end{center} \n\\caption{(Color online)\nFrequency dependence of $-V_{\\rm xc}+ G^{(0)}W_{\\rm GW}^{\\rm dyn}$ of SrVO$_3$.\n(a) and (b) are the real and imaginary local parts, respectively.\n(c) and (d) are the real and imaginary \nparts at several $k$ points.\n}\n\\label{DeltaSigma_L_GW}\n\\end{figure*} \n\n\n\nThe frequency dependence of the real and imaginary parts of the low-energy self-energy $\\Delta \\Sigma_{\\rm L} = G^{(1)}W_{\\rm H}^{\\rm dyn}$ for the direct perturbative treatment is illustrated in Fig.~\\ref{DeltaSigma_L_Pert}. \n$\\Delta \\Sigma_{\\rm L} = G^{(0)}W_{\\rm GW}^{\\rm dyn}$ used\nfor the GW treatment is shown in Fig.~\\ref{DeltaSigma_L_GW}\nfor several choices of momenta.\n\\\\\nThe results for the perturbative and the GW treatment are nearly \nidentical.\nThe renormalization factor of $\\Delta \\Sigma_{\\rm L}^{\\rm nonlocal}$ \n(and $\\Delta \\Sigma_{\\rm L}$) is $\\sim 0.77$.\nThe zero-frequency shift\n${\\rm Re}\\Delta \\Sigma_{\\rm L}^{\\rm nonlocal} (0)$ at $R$ is $\\sim 3$ eV \nlarger than \nat $\\Gamma$.\nAgain, one sees how $\\Sigma_{\\rm L}$ separates into local dynamical\nand nonlocal static parts: \n$\\Delta \\Sigma_{\\rm L} = \\Delta \\Sigma^{\\rm nonlocal}_{\\rm L}(k) + \n\\Delta \\Sigma^{\\rm local}_{\\rm L}(\\omega)$.\nThis is expected since such a separation has been found within\nthe full GW calculation \\cite{TomczakPRB2014}, and is even\nmore plausible for a GW treatment within the t$_{2g}$ subspace.\n\n\n\n\\begin{figure}[h]\n\\centering \n\\includegraphics[clip,width=0.45\\textwidth ]{fig13.eps} \n\\caption{(Color online)\nBand structure  of $[\\epsilon_{\\rm DFT}-V_{\\rm xc}+\\Delta\\Sigma_{\\rm H}+\\Delta \\Sigma_{\\rm L}^{\\rm nonlocal}]Z_{\\rm HL}$ of SrVO$_3$ in the direct perturbative treatment.\nFor comparison, the LDA band structure is also given\n(black dotted line).\nThe energy is measured from the Fermi level.\n}\n\\label{DFT+Sigma_HL_Pert}\n\\end{figure} \n\n\n\n\\begin{figure}[h]\n\\centering \n\\includegraphics[clip,width=0.45\\textwidth ]{fig14.eps} \n\\caption{(Color online)\nBand structure  of $[\\epsilon_{\\rm DFT}-V_{\\rm xc}+\\Delta\\Sigma_{\\rm H}+\\Delta \\Sigma_{\\rm L}^{\\rm nonlocal}]Z_{\\rm HL}$ of SrVO$_3$ in the GW treatment.\nFor comparison, the LDA band structure is also given\n(black dotted line).\nThe energy is measured from the Fermi level.\n}\n\\label{DFT+Sigma_HL_GW}\n\\end{figure} \n\n\n\nThe corresponding band structures \nare given by \n $[\\epsilon_{\\rm DFT}+\\Delta\\Sigma_{\\rm H}+\\Delta \\Sigma_{\\rm L}^{\\rm nonlocal}]Z_{\\rm HL}$ in Fig.~\\ref{DFT+Sigma_HL_Pert} \nfor the perturbative treatment (see Eq.~(\\ref{DeltaSigmaLnonlocalPert})), and in Fig.~\\ref{DFT+Sigma_HL_GW} \nfor the GW treatment,\nwhere $\\Delta \\Sigma_{\\rm L}^{\\rm nonlocal}$ is defined in Eq.~(\\ref{DeltaSigLnonlocalGW}) and $Z_{\\rm HL}$ is around 0.92.\n$Z_{\\rm HL}$ is practically the same as $Z_{\\rm H}$, meaning that the nonlocal dynamical correction \nin the H-space is small, which justifies the GW perturbative treatment for the nonlocal part.\nOne finds that \nthe nonlocal part $\\Delta \\Sigma_{\\rm L}=G^{(0)}_{ll}(W-W_{\\rm L})$ narrows\nthe empty states but widens the occupied ones, resulting\nin an overall bandwidth of 3.2 eV.\nThe difference between the direct perturbation and the GW treatment is small.\n\n\n\n\\begin{figure}[h]\n\\centering \n\\includegraphics[clip,width=0.45\\textwidth ]{fig15.eps} \n\\caption{(Color online)\nBand structure  of $[\\epsilon_{\\rm DFT}-V_{\\rm xc}+\\Delta\\Sigma_{\\rm H}+\\Delta\\Sigma_{\\rm L}^{\\rm nonlocal}]Z_{\\rm HL}Z_B$ of SrVO$_3$ in the GW treatment.\nFor comparison, the LDA band structure is also given\n(black dotted line).\nThe energy is measured from the Fermi level.}\n\\label{DFT+DeltaSigmaL_GW_ZHL_ZB}\n\\end{figure} \n\n\nAfter including the effect of the local self-energy by the nonperturbative \nCasula trick, we show the dispersion\ngiven by \n$[\\epsilon_{\\rm DFT}-V_{\\rm xc}+\\Delta\\Sigma_{\\rm H}+\\Delta\\Sigma_{\\rm L}^{\\rm nonlocal}]Z_{\\rm HL}Z_B$ for the GW-type treatment in Fig.~\\ref{DFT+DeltaSigmaL_GW_ZHL_ZB}: \nThe GW-like treatment gives a band dispersion\nclose to the LDA one. \nThe $Z_B$ factor corresponding to the local dynamical part of \n$\\Delta \\Sigma_{\\rm L}$ amounts to $Z_B=0.7$.\nThe nonperturbative treatment \nresults in an LDA-like band dispersion for the empty states,\nbut a slightly narrower bandwidth in the occupied part.\nThe low-energy effective Hamiltonian at this level of treatment has a \nfrequency independent effective interaction both with local \nand nonlocal interaction given by the Fourier transform of \n$W_{\\rm H}(q,\\omega=0)$, which contains both direct and exchange interactions. \n\n\n\\begin{figure}[h]\n\\centering \n\\includegraphics[clip,width=0.45\\textwidth ]{fig16.eps} \n\\caption{(Color online)\nBand structure  of $[\\epsilon_{\\rm DFT}+\\Sigma_{\\rm corr}^{\\rm GW}]Z_{\\rm corr}Z_B$ of SrVO$_3$.\nFor comparison, the LDA band structure is also given\n(black dotted line).\nThe energy is measured from the Fermi level.}\n\\label{DFT+Sigma_corr_GW_Z}\n\\end{figure} \n\n\n\nWe finally show the dispersion\ngiven by Eq.~(\\ref{Teff_final}), namely\n$[\\epsilon_{\\rm DFT}+\\Sigma_{\\rm corr}^{\\rm GW}]Z_{\\rm corr}Z_B$ in Fig.~\\ref{DFT+Sigma_corr_GW_Z}. \nThe result shows an LDA-like band dispersion for the occupied states,\nbut a slightly wider bandwidth in the empty part, resulting in a bandwidth 14\\% wider in total than the LDA bandwidth. \nThe effective interaction of the low-energy effective Hamiltonian \nat this final level has\nfrequency independent local and nonlocal interactions \ngiven by the Fourier transform of $W_r(q,\\omega=0)$. \n\n\n\n\\begin{figure}[h]\n\\centering \n\\includegraphics[clip,width=0.45\\textwidth ]{fig17.eps} \n\\caption{(Color online)\nBand structure  of $[\\epsilon_{\\rm DFT}-V_{\\rm xc}+\\Sigma-(G^{(0)}W_{\\rm GW}^{\\rm dyn})^{\\rm nonlocal}]Z'Z_B$ of SrVO$_3$\n(Z' is the renormalization factor of $\\Sigma-(G^{(0)}W_{\\rm GW}^{\\rm dyn})^{\\rm nonlocal}$).\nFor comparison, the LDA band structure is also given\n(black dotted line).\nThe energy is measured from the Fermi level.\n}\n\\label{GW+ZB}\n\\end{figure} \n\n\n\nThe resulting effective Hamiltonian in the L-space \nconsists of single-particle and two-particle (interaction) parts.\nThe effective interaction is the same as in previous estimates \nby the cRPA in the literature, while the single-particle dispersion \nis revised after removing the double counting and taking into \naccount the frequency dependence of the effective interaction. The \nfinal effective bandwidth for SrVO$_3$ is,\nafter partial cancellation, \nslightly (14\\%) larger than the LDA bandwidth.  \nThis is a reasonable result because the LDA takes into account all \ncorrelations though insufficiently, while the present scheme by the constrained self-energy excludes the \ncorrelation effects arising from the L-space.\nAlthough the bandwidth derived for the effective model is slightly \nlarger than that of the LDA,  it is clear that the correlation effects \nare stronger than in the LDA or standard GW when the {\\it ab initio} \nmodel is solved by an accurate solver. In fact, if the effective \ninteraction contained in the final effective model is treated by \nthe GW scheme, one obtains the dispersion illustrated in \nFig.~\\ref{GW+ZB}, \nwhich is given from the self energy of the whole GW calculation $\\Sigma$ \nby correcting the local dynamical part $W-W_{\\rm L}$ by $Z_B$. \nThis indicates that even an insufficient treatment of the local \nstatic interaction by the GW scheme gives a dispersion with the bandwidth ($\\sim 1.9$ eV) narrower \nthan the LDA ($\\sim 2.5$ eV, Fig.~\\ref{lda}) and GW ($\\sim 2.1$ eV, Fig.~\\ref{fullGW}) dispersions.  \nA slightly ($\\sim 14$\\%) wider dispersion than that of the LDA bands obtained for the \neffective Hamiltonian accompanied with frequency independent effective \nCoulomb interactions without the exchange part may account for the slight \noverestimate of correlation effects in the literature mentioned in \nthe introduction.\nOur scheme offers an optimized way for the  derivation of \n{\\it ab initio} models for the L-space after eliminating the H-space\nin  a systematic fashion.\n\n\nOur findings of the band widening are consistent with studies\nbased on the combined GW+DMFT method in the literature \n\\cite{TomczakPRB2014}: There, it was argued that within GW+DMFT\nthe best effective Hamiltonian that DMFT should be performed on,\nis a one-body Hamiltonian corrected by the nonlocal part of the\nGW self-energy. The corresponding spectral function \n(see e.g.\nFigure 5 of Ref.~\\onlinecite{TomczakPRB2014}) displays \na broadening similar to \nFig.~\\ref{DFT+Sigma_HL_GW}.\nMost interestingly, our present calculations confirm a pronounced\nasymmetry observed in Ref.~\\onlinecite{TomczakPRB2014}, namely a stronger widening\neffect in the unoccupied part of the spectrum than in the occupied\none.\n\n",
        "label": "DFT+Sigma_corr_GW_Z",
        "Descriptive_question1": "What is the bandwidth comparison shown in figure_16 for SrVO₃?",
        "Descriptive_question2": "What type of line represents the LDA band structure in figure_16?",
        "Reasoning_question1": "Why does the band structure in figure_16 show a slightly wider bandwidth in the empty part compared to the LDA band structure for SrVO₃?",
        "Reasoning_question2": "How does the effective interaction in the low-energy effective Hamiltonian impact the band dispersion shown in figure_16 for SrVO₃?",
        "Descriptive_answer1": "14% wider",
        "Descriptive_answer2": "black dotted line",
        "Reasoning_answer1": "The band structure in figure_16 shows a slightly wider bandwidth in the empty part compared to the LDA because after removing double counting and considering frequency dependence of interactions, the effective Hamiltonian accounts for correlation effects outside the low-energy space (L-space). The partial cancellation of band widening and narrowing effects results in an overall 14% wider bandwidth than LDA, with the empty states specifically slightly broadened due to the nonlocal self-energy corrections.",
        "Reasoning_answer2": "The effective interaction in the low-energy Hamiltonian is frequency independent and includes local and nonlocal parts given by the Fourier transform of W_r(q,ω=0). This leads to partial cancellation between band widening from nonlocal corrections and narrowing from local dynamical effects after applying renormalization factors Z_corr and Z_B. As a result, the band dispersion in figure_16 is close to LDA with a slightly wider bandwidth, reflecting an optimized balance of correlation effects in the effective model."
    },
    {
        "paper_id": "1511.03757.json",
        "image_id": "figure_17",
        "image_path": [
            "/home/yz979/scratch/chengye/scimolmo/test-images-tables/dataset_imagestables/1511.03757/images/fig17.eps"
        ],
        "caption": "(Color online)\nBand structure  of $[\\epsilon_{\\rm DFT}-V_{\\rm xc}+\\Sigma-(G^{(0)}W_{\\rm GW}^{\\rm dyn})^{\\rm nonlocal}]Z'Z_B$ of SrVO$_3$\n(Z' is the renormalization factor of $\\Sigma-(G^{(0)}W_{\\rm GW}^{\\rm dyn})^{\\rm nonlocal}$).\nFor comparison, the LDA band structure is also given\n(black dotted line).\nThe energy is measured from the Fermi level.\n",
        "classify": "Chart",
        "section_info": "6 Results\n\\section{Results}\n\\label{results}\n\n\nThe ternary 3d$^1$ transition metal oxide SrVO$_3$ has become\none of the ``drosophila compounds'' of correlated systems.\nIt is a correlated metal\nthat has been characterized using various experimental \n\\cite{Onoda, inoue, maiti, maiti2, sekiyama, takizawa, yoshida2010}.\nand theoretical techniques, see e.g.\n\\cite{nekrasov, anisimov, pavarini-d1, liebsch, lechermann-d1, trimarchi, aichhorn09, karolak, lee, held-optics, tomczak-gwdmft, gatti-guzzo, sakuma14, TomczakPRB2014}.\nA review of most of the available experimental and theoretical \ndata has been given recently in Ref.~\\onlinecite{TomczakPRB2014}.\nSrVO$_3$ displays Fermi liquid behavior up to\nremarkably high temperatures of the order of $200$ K \\cite{Onoda, inoue},\nwith a moderate mass enhancement of the order \nof $2$ \\cite{yoshida2010, vandermarel}.\nDetailed spectroscopic investigations have made it an\nideal test compound for modern many-body calculations, and \nmore and more refined dynamical mean field-based studies \nare available.\nThe majority of studies so far have focused on the t$_{2g}$-manifold\nwhich forms the states close to the Fermi level, and those will\nalso be the focus in the present investigations. \nNote however that this restriction quite severely limits the\nrange of validity of the low-energy description. Indeed, as\nshown recently~\\cite{tomczak-gwdmft, TomczakPRB2014} at\nenergies of about 2 to 3 eV above the Fermi level\nthe spectral properties are largely determined by the e$_g$\nstates. This has in particular led to a reinterpretation of\nan inverse photoemission feature at about 2.5 eV that was \nfrequently interpreted as an upper Hubbard band of t$_{2g}$\ncharacter in the earlier literature.\nHere, we use the compound only as an illustration of \nthe principles of constructing effective low-energy models,\nwithout being concerned with a description of spectra beyond\nthe pure t$_{2g}$ part.\n\n\nThe LDA band structure of SrVO$_3$ is shown in Fig.~\\ref{lda}.\nOne clearly distinguishes the three-fold degenerate manifold\nof t$_{2g}$ bands close to the Fermi level (chosen as the\nzero of energy), followed in the unoccupied part of the spectrum\nby the two e$_g$ bands. At about -2.5 eV the filled O-2p derived\nbands are visible.\n\nWhen a standard GW calculation is performed, see Fig.~\\ref{fullGW},\nthe bandwidth of the t$_{2g}$-states is reduced from 2.5 eV to\n2.1 eV, while the overall shape of the dispersion remains similar\nto the LDA one.\nThis is in agreement with previous GW calculations in the literature\n\\cite{tomczak-gwdmft, TomczakPRB2014, gatti-guzzo, sakuma14}.\n\n\n\nThe calculation is based on the full-potential linear muffin-tin orbitals (FP-LMTO) implementation.\n8$\\times$8$\\times$8 \\bm{k}-mesh is employed in both the DFT/LDA and the GW calculations.\nThe calculational details are the same as in Ref.~\\onlinecite{Hirayama}.\n\n\nWe now turn to a discussion of the band structure corresponding to \nthe Hamiltonians to be used as input for subsequent many-body \ncalculations for this material.\nWe will proceed step by step to analyze the influence of the various\ncorrection terms, with respect to the DFT starting point\n(which will be overlaid to the respective band structures).\n\n\n\n\\begin{figure}[h]\n\\centering \n\\includegraphics[clip,width=0.45\\textwidth ]{fig2.eps} \n\\caption{(Color online)\nKohn-Sham band structure of SrVO$_3$ in the LDA.\nThe energy is measured from the Fermi level.\n}\n\\label{lda}\n\\end{figure} \n\n\n\\begin{figure}[h]\n\\centering \n\\includegraphics[clip,width=0.45\\textwidth ]{fig3.eps} \n\\caption{(Color online)\nBand structure  of SrVO$_3$ in the one-shot GW approximation.\nFor comparison, the LDA band structure is also given\n(black dotted line).\nThe energy is measured from the Fermi level.\n}\n\\label{fullGW}\n\\end{figure} \n\n\n\n\n\n\n\\begin{figure}[h]\n\\centering \n\\includegraphics[clip,width=0.45\\textwidth ]{fig4.eps} \n\\caption{(Color online)\nBand structure of SrVO$_3$ corresponding to the LDA Hamiltonian\nfrom which the LDA exchange-correlation potential has been subtracted.\nFor comparison, the LDA band structure is also given\n(black dotted line).\nThe energy is measured from the Fermi level.\n}\n\\label{lda-vxc}\n\\end{figure} \n\n\n\\begin{figure}[h]\n\\centering \n\\includegraphics[clip,width=0.45\\textwidth ]{fig5.eps} \n\\caption{(Color online)\nBand structure  of SrVO$_3$ in the Hartree approximation.\nFor comparison, the LDA band structure is also given\n(black dotted line).\nThe energy is measured from the Fermi level.\n}\n\\label{hartree}\n\\end{figure} \n\n\nFigure~\\ref{lda-vxc} \ndisplays the band structure corresponding to\nthe LDA Hamiltonian from which the \nLDA exchange-correlation\npotential has been subtracted,  \nthat is, the eigenvalues of $H_{\\rm LDA}-V_{\\rm xc}$ where\nthese operators are evaluated for the self-consistent LDA\ndensity.\nThe subtraction of $V_{\\rm xc}$ widens the band structure from \nthe LDA bandwidth of 2.5 eV \nto more than twice this value: the new band\nwidth is 5.4 eV.\nThis indicates that the exchange-correlation potential $V_{\\rm xc}$ \nof the LDA is responsible for a substantial amount of \nthe band-narrowing effect\narising from electronic correlations, stressing the need \nto subtract the double counting in a consistent manner. \nFigure~\\ref{lda-vxc} is regarded as the starting point for the\nfollowing considerations.\n\nFor simple materials (see e.g. the calculations on Li in Ref.~\\onlinecite{Lithium})\nit has been noted in the literature that the Hartree band structure\nis close to the DFT one. This raises the question of the origin of the\nsubstantial band widening in the present case. Indeed, the present\ncalculation shown in Fig.~\\ref{lda-vxc} differs from a Hartree calculation\nonly \nby the fact that Fig.~\\ref{lda-vxc} is evaluated for the converged LDA density.\nWe have performed a test calculation where we plot the Hartree band\nstructure calculated for the converged Hartree density. The result\nis shown in Fig.~\\ref{hartree}. As seen from this plot, while the\nband is not fully as wide as in Fig.~\\ref{lda-vxc}, a substantial \nwidening is already present at this stage.\n\n\n\n\\begin{figure}[h]\n\\centering \n\\includegraphics[clip,width=0.45\\textwidth ]{fig6.eps} \n\\caption{(Color online)\nBand structure  of $H_{\\rm LDA}-V_{\\rm xc}+\\Delta \\Sigma_{\\rm H}(\\omega=0)$ of SrVO$_3$.\nFor comparison, the LDA band structure is also given\n(black dotted line).\nThe energy is measured from the Fermi level.\n}\n\\label{lda-vxc-sigH(0)}\n\\end{figure} \n\n\n\nStarting from the Hamiltonian without V$_{xc}$ (see the \ndispersion in Fig.~\\ref{lda-vxc}), we first take into account \nthe static part of the corrective self-energy $\\Delta \\Sigma_{\\rm H}$:\nIn Fig.~\\ref{lda-vxc-sigH(0)}, we plot the dispersion corresponding to\n$H_{\\rm LDA}-V_{\\rm xc}+\\Delta \\Sigma_{\\rm H}(\\omega=0)$. \nCompared to the dispersion of Fig.~\\ref{lda-vxc}, the overall band structure \nis narrowed to 3.7 eV. This value is, however, still considerably larger\nthan the LDA bandwidth (Fig.~\\ref{lda}). Although\n the electronic correlations coupling the H- and \nthe L space are included in Fig.~\\ref{lda-vxc-sigH(0)}, and narrow the band with respect to the case\nwhere V$_{xc}$ is taken out, correlations within\nthe L space are not included. \nThe LDA, on the other hand, at least partially\nincludes correlations within the L space, \nand these are very effective in narrowing the band.\nInterestingly, the bottom of the occupied band is quite\nexactly at the LDA value, and the remaining widening is\npurely in the unoccupied part.\n\n\n\\begin{figure*}[htb]\n\\begin{center} \n\\includegraphics[width=0.9\\textwidth ]{fig7.eps} \n\\end{center} \n\\caption{(Color online)\nFrequency dependence of $-V_{xc}+\\Delta \\Sigma_{\\rm H}$ for SrVO$_3$.\n(a) and (b) are the real and imaginary parts at the $\\Gamma$ point, \nrespectively,\nand (c) and (d) ((e) and (f)) are those at $X$ ($R$).\nFor comparison, the full GW self-energies are also given\n(black dotted line).\n}\n\\label{sigH}\n\\end{figure*} \n\n\n\n\\begin{figure}[h]\n\\centering \n\\includegraphics[clip,width=0.45\\textwidth ]{fig8.eps} \n\\caption{(Color online)\nBand structure  of $Z_{\\rm H}[H_{\\rm LDA}-V_{\\rm xc}+\\Delta\\Sigma_{\\rm H}(\\omega=0)]$ of SrVO$_3$.\nFor comparison, the LDA band structure is also given\n(black dotted line).\nThe energy is measured from the Fermi level.\n}\n\\label{ZH_SigmaH}\n\\end{figure} \n\n\nTo go further and in particular to analyze the dynamic\nbehavior of $\\Delta \\Sigma_{\\rm H}$ \nwe plot in Fig.~\\ref{sigH} \nthe self-energy corrections $\\Delta \\Sigma_{\\rm H}$ (with $\\Sigma$ (see Eq.~(\\ref{full-self-energy}))\nfor comparison) for the real and imaginary parts\nat several representative choices of momenta.\nThe frequency dependence is smooth around the Fermi level.\nIn particular, in contrast to the full self-energy,\nthere are no poles in $\\Delta \\Sigma_{\\rm H}$ \nin the low-energy region, thanks to the\nexclusion of fluctuations within the L-space.\nThe frequency dependence of the real part indicates that the \nlinearization \n$\\displaystyle{\\Delta \\Sigma_{\\rm H}(\\omega) \\sim \\Delta \\Sigma_{\\rm H}(\\omega=0)+[d\\Delta \\Sigma_{\\rm H}(\\omega)/d\\omega]_{\\omega=0}\\omega }$\noffers a reasonably good approximation. \nIn fact, the behavior of the constrained self-energy\nis much closer to linearity than that of \nthe full GW self-energy $\\Sigma$.\nThis is easily understood by the fact that the low-energy excitations are excluded in the present constrained self-energy $\\Delta\\Sigma_{\\rm H}$ analogously to an insulator, thus eliminating the strong frequency dependence of the constrained self-energy near the Fermi level. This is one of the consequences of the \ncontrollability and an advantage of the present MACE scheme, as \nmentioned in the introduction.\nIn the following discussion, we employ this linearized approximation.\nThen the renormalization factor $Z_{\\rm H}$ defined by Eq.~(\\ref{ZH})\nis interpreted as that from the contribution of the H-space. \nAfter \nthis renormalization factor is taken into account,\nthe effective Hamiltonian is given by $Z_{\\rm H}[H_{\\rm LDA}-V_{\\rm xc}+\\Delta\\Sigma_{\\rm H}(\\omega=0)]$ (see Fig.~\\ref{ZH_SigmaH}). \nInterestingly the renormalization factor $Z_{\\rm H}$=0.92 stays\nclose to one, so that\nthe bandwidth is only slightly reduced (to\n$3.4$ eV instead of the $3.7$ eV above). \nWe also note that $Z_{\\rm H}$ has weak momentum dependence\nas we reveal in the following.\n\n\n\n\\begin{figure*}[htb]\n\\begin{center} \n\\includegraphics[width=0.9\\textwidth ]{fig9.eps} \n\\end{center} \n\\caption{(Color online)\n{\\bf minor modification}\nFrequency dependence of $-V_{\\rm xc}+\\Delta \\Sigma_{\\rm H}$ of SrVO$_3$.\n(a) and (b) are the real and imaginary parts of the local\ncomponent, respectively.\nThe black dotted line is $\\Sigma$ defined in Eq.~(\\ref{full-self-energy}) in the full GW approximation for comparison.\n(c) and (d) are the real and imaginary \nparts at several $k$ points.\n}\n\\label{sigH_nonlocal}\n\\end{figure*} \n\n\n\nFigure~\\ref{sigH_nonlocal} \nshows the local and nonlocal parts of $\\Delta \\Sigma_{\\rm H}$.\nThe frequency \ndependence\nof the local part (Figs.~\\ref{sigH_nonlocal} (a)(b)) \nis quite similar \nto the  frequency-dependent self-energies at various momenta\nin Fig.~\\ref{sigH}.\nOne immediately reads off an interesting property, which is\nakin to what has been found for the full GW self-energy in\nRef.~\\onlinecite{Tomczak-PRL2012}, namely that the nonlocal part of \n$\\Delta \\Sigma_{\\rm H}$ shows only weak frequency dependence \nas shown in Figs.~\\ref{sigH_nonlocal} (c)(d). \nThis explains why $Z_{\\rm H}$ \nis only weakly momentum-dependent.\nOn the other hand, $\\Delta \\Sigma_{\\rm H}^{\\rm nonlocal} (0)$ at $R$ is $\\sim 1$ eV larger than \nat $\\Gamma$, which causes the band widening effect.\n $\\Delta \\Sigma_{\\rm H}$\ncan thus to a good approximation be decomposed into a\nfrequency-dependent local part and a static nonlocal one:\n$\\Delta \\Sigma_{\\rm H}(k,\\omega) = \\Delta \\Sigma^{\\rm local}_{\\rm H}(\\omega) + \\Delta \\Sigma^{\\rm nonlocal}_{\\rm H}(k)$,\nwith $ \\Delta \\Sigma_{\\rm H}^{\\rm local}(\\omega) = \\sum_q  \\Delta \\Sigma_{\\rm H}(q,\\omega)$\n\nSince in the simple case of SrVO$_3$ where the t$_{2g}$ states are\ndegenerate, static local operators are scalar and are compensated \nby a chemical potential shift such that the correct particle\nnumber is obtained. The above band structure corresponding to\n$H_{\\rm LDA} - V_{\\rm xc} + \\Delta \\Sigma_{\\rm H}(k, \\omega=0)$ \nis thus identical to that of\n$H_{\\rm LDA} - V_{\\rm xc} + \\Delta \\Sigma^{\\rm nonlocal}_{\\rm H}(k)$.\nThe local dynamical part of $\\Delta \\Sigma_{\\rm H}$ then results in a \nnarrowing of this band structure by a factor $Z_{\\rm H} = 0.92$.\n\n\n\n\\begin{figure}[h]\n\\centering \n\\includegraphics[clip,width=0.45\\textwidth ]{fig10.eps} \n\\caption{(Color online)\n(a) Frequency dependence of the on-site effective interaction.\nPanel (b) is a low-energy zoom of panel (a). \n}\n\\label{W}\n\\end{figure} \n\n\nBefore turning to a discussion of the low-energy correction for the GW treatment\n$\\Delta \\Sigma_{\\rm L} =G^{(0)}(W-W_{\\rm L})$, we analyze the effective\ninteraction $W_{\\rm L}$ shown in Fig.~\\ref{W} \nin comparison to $W$ and $W_{\\rm H}$.\nThe fully screened Coulomb interaction $W$ displays the familiar\nshape of an interaction that is strongly screened at low energies\n(with a value of $0.88$ eV at $\\omega=0$), while retrieving the\nvalue of the bare Coulomb interaction $v$ ($\\sim 16.0$ eV) \nat high energy.\nThe crossover from the bare to the  \nscreened values takes place at the plasma energy of \nabout $15$ eV. This behavior has been discussed before\n\\cite{Review_SB2014}; we note in particular that the plasma \nfrequency is known from electron energy loss spectroscopy \nmeasurements of the related SrTiO$_3$ compound \\cite{vandermarel,kohiki}.\nThe partially screened interaction\n$W_{\\rm H}$ constructed within cRPA\nconverges to $W$ at high energies, but displays\nweaker screening effects at low energies (with a value of $3.5$ eV at $\\omega=0$),\nsince intra-t$_{2g}$ screening channels are excluded.\nAs was already anticipated in Sec.~\\ref{Outline},  $W_{\\rm L}$ can be interpreted as the screened interaction of \na low-energy model where a static interaction of value $W_{\\rm H}(\\omega=0)$\nhas been imposed as the bare interaction. Since now {\\it only} intra-t$_{2g}$\nscreening channels are active, screening takes place only at low\nenergies, where the scale is given by the t$_{2g}$ bandwidth.\nAlso plotted is the difference $W_{\\rm GW}^{\\rm dyn}=W-W_{d}$: except at low energies\nwhere the t$_{2g}$ screening channels come into play, its frequency\ndependence is essentially given by that of $W$, while the\nhigh-energy limit is the bare Coulomb interaction $v$ reduced\nby $W_{\\rm H}(0)$.\n\n\n\\begin{figure*}[htb]\n\\begin{center} \n\\includegraphics[width=0.9\\textwidth ]{fig11.eps} \n\\end{center} \n\\caption{(Color online)\nFrequency dependence of $-V_{\\rm xc}+ G^{(1)}W_{\\rm H}^{\\rm dyn}$ of SrVO$_3$.\n(a) and (b) are the real and imaginary local parts, respectively.\n(c) and (d) are the real and imaginary \nparts at several $k$ points.\n}\n\\label{DeltaSigma_L_Pert}\n\\end{figure*} \n\n\n\n\\begin{figure*}[htb]\n\\begin{center} \n\\includegraphics[width=0.9\\textwidth ]{fig12.eps} \n\\end{center} \n\\caption{(Color online)\nFrequency dependence of $-V_{\\rm xc}+ G^{(0)}W_{\\rm GW}^{\\rm dyn}$ of SrVO$_3$.\n(a) and (b) are the real and imaginary local parts, respectively.\n(c) and (d) are the real and imaginary \nparts at several $k$ points.\n}\n\\label{DeltaSigma_L_GW}\n\\end{figure*} \n\n\n\nThe frequency dependence of the real and imaginary parts of the low-energy self-energy $\\Delta \\Sigma_{\\rm L} = G^{(1)}W_{\\rm H}^{\\rm dyn}$ for the direct perturbative treatment is illustrated in Fig.~\\ref{DeltaSigma_L_Pert}. \n$\\Delta \\Sigma_{\\rm L} = G^{(0)}W_{\\rm GW}^{\\rm dyn}$ used\nfor the GW treatment is shown in Fig.~\\ref{DeltaSigma_L_GW}\nfor several choices of momenta.\n\\\\\nThe results for the perturbative and the GW treatment are nearly \nidentical.\nThe renormalization factor of $\\Delta \\Sigma_{\\rm L}^{\\rm nonlocal}$ \n(and $\\Delta \\Sigma_{\\rm L}$) is $\\sim 0.77$.\nThe zero-frequency shift\n${\\rm Re}\\Delta \\Sigma_{\\rm L}^{\\rm nonlocal} (0)$ at $R$ is $\\sim 3$ eV \nlarger than \nat $\\Gamma$.\nAgain, one sees how $\\Sigma_{\\rm L}$ separates into local dynamical\nand nonlocal static parts: \n$\\Delta \\Sigma_{\\rm L} = \\Delta \\Sigma^{\\rm nonlocal}_{\\rm L}(k) + \n\\Delta \\Sigma^{\\rm local}_{\\rm L}(\\omega)$.\nThis is expected since such a separation has been found within\nthe full GW calculation \\cite{TomczakPRB2014}, and is even\nmore plausible for a GW treatment within the t$_{2g}$ subspace.\n\n\n\n\\begin{figure}[h]\n\\centering \n\\includegraphics[clip,width=0.45\\textwidth ]{fig13.eps} \n\\caption{(Color online)\nBand structure  of $[\\epsilon_{\\rm DFT}-V_{\\rm xc}+\\Delta\\Sigma_{\\rm H}+\\Delta \\Sigma_{\\rm L}^{\\rm nonlocal}]Z_{\\rm HL}$ of SrVO$_3$ in the direct perturbative treatment.\nFor comparison, the LDA band structure is also given\n(black dotted line).\nThe energy is measured from the Fermi level.\n}\n\\label{DFT+Sigma_HL_Pert}\n\\end{figure} \n\n\n\n\\begin{figure}[h]\n\\centering \n\\includegraphics[clip,width=0.45\\textwidth ]{fig14.eps} \n\\caption{(Color online)\nBand structure  of $[\\epsilon_{\\rm DFT}-V_{\\rm xc}+\\Delta\\Sigma_{\\rm H}+\\Delta \\Sigma_{\\rm L}^{\\rm nonlocal}]Z_{\\rm HL}$ of SrVO$_3$ in the GW treatment.\nFor comparison, the LDA band structure is also given\n(black dotted line).\nThe energy is measured from the Fermi level.\n}\n\\label{DFT+Sigma_HL_GW}\n\\end{figure} \n\n\n\nThe corresponding band structures \nare given by \n $[\\epsilon_{\\rm DFT}+\\Delta\\Sigma_{\\rm H}+\\Delta \\Sigma_{\\rm L}^{\\rm nonlocal}]Z_{\\rm HL}$ in Fig.~\\ref{DFT+Sigma_HL_Pert} \nfor the perturbative treatment (see Eq.~(\\ref{DeltaSigmaLnonlocalPert})), and in Fig.~\\ref{DFT+Sigma_HL_GW} \nfor the GW treatment,\nwhere $\\Delta \\Sigma_{\\rm L}^{\\rm nonlocal}$ is defined in Eq.~(\\ref{DeltaSigLnonlocalGW}) and $Z_{\\rm HL}$ is around 0.92.\n$Z_{\\rm HL}$ is practically the same as $Z_{\\rm H}$, meaning that the nonlocal dynamical correction \nin the H-space is small, which justifies the GW perturbative treatment for the nonlocal part.\nOne finds that \nthe nonlocal part $\\Delta \\Sigma_{\\rm L}=G^{(0)}_{ll}(W-W_{\\rm L})$ narrows\nthe empty states but widens the occupied ones, resulting\nin an overall bandwidth of 3.2 eV.\nThe difference between the direct perturbation and the GW treatment is small.\n\n\n\n\\begin{figure}[h]\n\\centering \n\\includegraphics[clip,width=0.45\\textwidth ]{fig15.eps} \n\\caption{(Color online)\nBand structure  of $[\\epsilon_{\\rm DFT}-V_{\\rm xc}+\\Delta\\Sigma_{\\rm H}+\\Delta\\Sigma_{\\rm L}^{\\rm nonlocal}]Z_{\\rm HL}Z_B$ of SrVO$_3$ in the GW treatment.\nFor comparison, the LDA band structure is also given\n(black dotted line).\nThe energy is measured from the Fermi level.}\n\\label{DFT+DeltaSigmaL_GW_ZHL_ZB}\n\\end{figure} \n\n\nAfter including the effect of the local self-energy by the nonperturbative \nCasula trick, we show the dispersion\ngiven by \n$[\\epsilon_{\\rm DFT}-V_{\\rm xc}+\\Delta\\Sigma_{\\rm H}+\\Delta\\Sigma_{\\rm L}^{\\rm nonlocal}]Z_{\\rm HL}Z_B$ for the GW-type treatment in Fig.~\\ref{DFT+DeltaSigmaL_GW_ZHL_ZB}: \nThe GW-like treatment gives a band dispersion\nclose to the LDA one. \nThe $Z_B$ factor corresponding to the local dynamical part of \n$\\Delta \\Sigma_{\\rm L}$ amounts to $Z_B=0.7$.\nThe nonperturbative treatment \nresults in an LDA-like band dispersion for the empty states,\nbut a slightly narrower bandwidth in the occupied part.\nThe low-energy effective Hamiltonian at this level of treatment has a \nfrequency independent effective interaction both with local \nand nonlocal interaction given by the Fourier transform of \n$W_{\\rm H}(q,\\omega=0)$, which contains both direct and exchange interactions. \n\n\n\\begin{figure}[h]\n\\centering \n\\includegraphics[clip,width=0.45\\textwidth ]{fig16.eps} \n\\caption{(Color online)\nBand structure  of $[\\epsilon_{\\rm DFT}+\\Sigma_{\\rm corr}^{\\rm GW}]Z_{\\rm corr}Z_B$ of SrVO$_3$.\nFor comparison, the LDA band structure is also given\n(black dotted line).\nThe energy is measured from the Fermi level.}\n\\label{DFT+Sigma_corr_GW_Z}\n\\end{figure} \n\n\n\nWe finally show the dispersion\ngiven by Eq.~(\\ref{Teff_final}), namely\n$[\\epsilon_{\\rm DFT}+\\Sigma_{\\rm corr}^{\\rm GW}]Z_{\\rm corr}Z_B$ in Fig.~\\ref{DFT+Sigma_corr_GW_Z}. \nThe result shows an LDA-like band dispersion for the occupied states,\nbut a slightly wider bandwidth in the empty part, resulting in a bandwidth 14\\% wider in total than the LDA bandwidth. \nThe effective interaction of the low-energy effective Hamiltonian \nat this final level has\nfrequency independent local and nonlocal interactions \ngiven by the Fourier transform of $W_r(q,\\omega=0)$. \n\n\n\n\\begin{figure}[h]\n\\centering \n\\includegraphics[clip,width=0.45\\textwidth ]{fig17.eps} \n\\caption{(Color online)\nBand structure  of $[\\epsilon_{\\rm DFT}-V_{\\rm xc}+\\Sigma-(G^{(0)}W_{\\rm GW}^{\\rm dyn})^{\\rm nonlocal}]Z'Z_B$ of SrVO$_3$\n(Z' is the renormalization factor of $\\Sigma-(G^{(0)}W_{\\rm GW}^{\\rm dyn})^{\\rm nonlocal}$).\nFor comparison, the LDA band structure is also given\n(black dotted line).\nThe energy is measured from the Fermi level.\n}\n\\label{GW+ZB}\n\\end{figure} \n\n\n\nThe resulting effective Hamiltonian in the L-space \nconsists of single-particle and two-particle (interaction) parts.\nThe effective interaction is the same as in previous estimates \nby the cRPA in the literature, while the single-particle dispersion \nis revised after removing the double counting and taking into \naccount the frequency dependence of the effective interaction. The \nfinal effective bandwidth for SrVO$_3$ is,\nafter partial cancellation, \nslightly (14\\%) larger than the LDA bandwidth.  \nThis is a reasonable result because the LDA takes into account all \ncorrelations though insufficiently, while the present scheme by the constrained self-energy excludes the \ncorrelation effects arising from the L-space.\nAlthough the bandwidth derived for the effective model is slightly \nlarger than that of the LDA,  it is clear that the correlation effects \nare stronger than in the LDA or standard GW when the {\\it ab initio} \nmodel is solved by an accurate solver. In fact, if the effective \ninteraction contained in the final effective model is treated by \nthe GW scheme, one obtains the dispersion illustrated in \nFig.~\\ref{GW+ZB}, \nwhich is given from the self energy of the whole GW calculation $\\Sigma$ \nby correcting the local dynamical part $W-W_{\\rm L}$ by $Z_B$. \nThis indicates that even an insufficient treatment of the local \nstatic interaction by the GW scheme gives a dispersion with the bandwidth ($\\sim 1.9$ eV) narrower \nthan the LDA ($\\sim 2.5$ eV, Fig.~\\ref{lda}) and GW ($\\sim 2.1$ eV, Fig.~\\ref{fullGW}) dispersions.  \nA slightly ($\\sim 14$\\%) wider dispersion than that of the LDA bands obtained for the \neffective Hamiltonian accompanied with frequency independent effective \nCoulomb interactions without the exchange part may account for the slight \noverestimate of correlation effects in the literature mentioned in \nthe introduction.\nOur scheme offers an optimized way for the  derivation of \n{\\it ab initio} models for the L-space after eliminating the H-space\nin  a systematic fashion.\n\n\nOur findings of the band widening are consistent with studies\nbased on the combined GW+DMFT method in the literature \n\\cite{TomczakPRB2014}: There, it was argued that within GW+DMFT\nthe best effective Hamiltonian that DMFT should be performed on,\nis a one-body Hamiltonian corrected by the nonlocal part of the\nGW self-energy. The corresponding spectral function \n(see e.g.\nFigure 5 of Ref.~\\onlinecite{TomczakPRB2014}) displays \na broadening similar to \nFig.~\\ref{DFT+Sigma_HL_GW}.\nMost interestingly, our present calculations confirm a pronounced\nasymmetry observed in Ref.~\\onlinecite{TomczakPRB2014}, namely a stronger widening\neffect in the unoccupied part of the spectrum than in the occupied\none.\n\n",
        "label": "GW+ZB",
        "Descriptive_question1": "What is the bandwidth value shown for the GW scheme in figure_17?",
        "Descriptive_question2": "What color is used for the LDA band structure line in figure_17?",
        "Reasoning_question1": "Why might the bandwidth in figure_17 be narrower than both LDA and standard GW dispersions for SrVO3?",
        "Reasoning_question2": "How does the asymmetry in band widening between occupied and unoccupied parts of the spectrum in figure_17 compare to findings from GW+DMFT studies?",
        "Descriptive_answer1": "not directly given",
        "Descriptive_answer2": "black dotted line",
        "Reasoning_answer1": "The bandwidth in figure_17 is narrower than both LDA and standard GW dispersions because the figure represents an effective Hamiltonian incorporating the full GW self-energy corrected by subtracting the local dynamical part of the screened interaction and renormalized by Z' and Z_B factors. This correction effectively accounts for local correlations beyond standard GW, leading to a more accurate treatment of electron interactions that narrows the bandwidth more than the LDA or GW alone. Additionally, the inclusion of these renormalization factors reduces the bandwidth further, reflecting stronger electron correlation effects.",
        "Reasoning_answer2": "The asymmetry in band widening in figure_17, showing a stronger narrowing in the unoccupied part compared to the occupied part, aligns with findings from GW+DMFT studies. Both approaches observe a pronounced asymmetry where the occupied states maintain a bandwidth similar to or slightly wider than the LDA, while the unoccupied states experience stronger narrowing. This consistent observation suggests that local dynamical correlation effects impact unoccupied states more, confirming the reliability of the modeling approach in figure_17 relative to GW+DMFT results."
    },
    {
        "paper_id": "1704.00974.json",
        "image_id": "figure_3",
        "image_path": [
            "/home/yz979/scratch/chengye/scimolmo/test-images-tables/dataset_imagestables/1704.00974/images/Fig3-efield-spectra.png"
        ],
        "caption": "Contour plot of the second derivatives of absorption spectra versus applied voltage. The spectra were recorded by white light excitation at $T = 1.3$~K on a Cu$_2$O crystal slab with $[110]$ orientation. The light was polarized along the $[1\\bar10]$ direction in the left panel and along the $[001]$ direction in the right panel. The mid panel shows a close-up of the section of the contour plot in the left panel, marked there by the solid box, in order to highlight the anticrossing at the first resonance involving the state of the $n$ = 6 multiplet showing the strongest field dispersion to higher energies and the state of the $n$ = 7 multiplet with the strongest dispersion to lower energies. The weak equidistant vertical stripes in the left and right panel are artifacts of taking the 2nd derivative.",
        "classify": "Chart",
        "section_info": "3 Scaling of exciton properties\n\\section{Scaling of exciton properties}\\label{sec:scaling}\n\nBefore discussing the different scaling laws, we first present briefly the comprehensive set of data from which the laws are extracted. These data comprise spectra recorded in magnetic and electric field, from which also the understanding of the zero field level spectrum can be deepened. Figure~\\ref{Fig1-bfield-spectra} shows a contour plot of exciton absorption spectra versus the magnetic field applied in the Faraday geometry. At zero field the hydrogen-like series of exciton states is seen, out of which a complex Zeeman splitting pattern arises. Better insight into the details of these spectra can be obtained by taking their second derivatives from which weak features become well resolved. The corresponding contour plot of the second derivatives is shown in Fig.~\\ref{Fig2-bfield-spectra-2ndderivative}: in panel (a) for the states with $n>4$ up to $B$ = 7~T, while in panel (b) a close-up of the low-field range up to 2~T is shown for $n>6$. At zero field the levels belonging to a particular principal quantum number are spread over an energy range which becomes the narrower, the higher $n$ is. However, in magnetic field the splitting of a multiplet becomes larger for higher $n$, because higher angular momentum states limited by $l \\leq n-1$ contribute. \n\nDue to the multitude of observed levels in particular at medium field strengths it appears difficult to extract generally valid scaling laws for the dependences on the principal quantum number there. For that purpose we have to restrict our analysis to ranges, where states can be identified well. This is the case \nat low fields up to the point, where resonances of states belonging to exciton multiplets with different principal quantum numbers occur.  \nSimilarly, states can be quite well identified in high magnetic fields, where the dominant observed lines tend to cluster around transitions that correspond to those between Landau levels, see Fig.~\\ref{Fig1-bfield-spectra}. In fact, the close-up of Fig.~\\ref{Fig1-bfield-spectra} shown in the Appendix~\\ref{app:ll} as Fig.~\\ref{Fig-Appendix-LandauLevels} reveals transitions which can be associated with Landau level quantum numbers up to 79, which arise in magnetic field from $P$-excitons with $n$=79 in zero field, see also discussion below in Sec.~\\ref{sec:magn}. The corresponding average radius of this exciton wavefunction would be 10.4~$\\mu$m, which is squeezed in magnetic field, thereby enhancing the oscillator strength so that these highly excited states become visible. The $n$=79 state can be observed starting from 0.5~T, where the Landau level extension is about 300~nm.\n\n\\begin{figure*}[ht]\n\\includegraphics[width=1\\textwidth]{Fig2-bfield-spectra-2ndderivative.pdf}\n\\caption{Top panel: Same data as in Fig.~\\ref{Fig1-bfield-spectra}, but in form of a contour plot of the second derivative of the absorption spectra versus magnetic field for $n \\geq 5$. Lower panel: Close-up of the states for $n \\geq 7$ in $B$ up to 2~T. $T$ = 1.3~K. The scales on the right give the strength of the features  in arbitrary units. The weak equidistant vertical stripes which are apparent mostly at low energies in the upper panel, are artifacts of taking the 2nd derivative.}\\label{Fig2-bfield-spectra-2ndderivative}\n\\end{figure*}\n\nExciton level splitting can be also induced by applying an electric field and exploiting the Stark effect. Corresponding spectra are shown in Fig.~\\ref{Fig3-efield-spectra} where their second derivatives are plotted as function of the voltage applied to the sample for the excitons with $n \\geq 5$. Overall, the number of lines observed there remains smaller than in magnetic field because already for pretty low voltages below 10~V the exciton states with $n >$10 are subject to field-induced dissociation. Still, for identifying scaling laws, one has to restrict also here to the low field strength regime or to lines with  dominant oscillator strength compared to the other features. As discussed above, variation of the linear polarization of the exciting light allows one to vary the number of detected spectral lines. In the left panel, the light polarization was $\\hat{\\bm e} \\parallel[1\\bar{1}0]$ and in the right panel the polarization was $\\hat{\\bm e}\\parallel[001]$. In the former case, the quadrupolar-active, even exciton states (e.g., $S$- and $D$-excitons) are forbidden, while in the latter case they are allowed, giving the spectra a more complex appearance.\n\n\\subsection{Zero-field behavior}\\label{sec:zero}\n\n\\begin{figure*}[ht]\n\\includegraphics[width=1\\linewidth]{Fig3-efield-spectra.png}\n\\caption{Contour plot of the second derivatives of absorption spectra versus applied voltage. The spectra were recorded by white light excitation at $T = 1.3$~K on a Cu$_2$O crystal slab with $[110]$ orientation. The light was polarized along the $[1\\bar10]$ direction in the left panel and along the $[001]$ direction in the right panel. The mid panel shows a close-up of the section of the contour plot in the left panel, marked there by the solid box, in order to highlight the anticrossing at the first resonance involving the state of the $n$ = 6 multiplet showing the strongest field dispersion to higher energies and the state of the $n$ = 7 multiplet with the strongest dispersion to lower energies. The weak equidistant vertical stripes in the left and right panel are artifacts of taking the 2nd derivative.}\\label{Fig3-efield-spectra}\n\\end{figure*}\n\nThe exciton energies in zero external field can be described well by the quantum defect formalism of Eq.~\\eqref{qdef}~\\cite{PhysRevB.93.075203}. We recall that in Cu$_2$O the origin of the quantum defect is quite different from that in many-electron Rydberg atoms and is related with the complex valence band structure: The separation between the topmost $\\Gamma_7^+$ and the closest $\\Gamma_8^+$ valence subbands is on the same order of magnitude as the exciton Rydberg energy $\\mathcal R$. Hence, the mixing of these bands via the off-diagonal elements of the Luttinger Hamiltonian gives rise to the deviation of the excitonic series from a hydrogenic one and to the fine structure of the exciton energy spectrum. Moreover, the exchange interaction between the electron and the hole also contributes significantly to the exciton state splitting and mixing~\\cite{PhysRevB.23.2731,PhysRevLett.115.027402,PhysRevB.93.195203,efield}. For excitons with principal quantum numbers $n\\gtrsim 4$ estimates show that the mixing of the valence subbands can be treated perturbatively and the effective Hamiltonian can be recast in the following form~[cf.~\\cite{PhysRevB.93.075203,efield}]:\n\\begin{multline}\n\\label{H:eff}\n\\mathcal H = \\frac{p^2}{2\\mu} - \\frac{e^2}{\\varepsilon r} + \\mathcal A p^4 +\\\\\n \\frac{e^2}{\\varepsilon r^3} \\left[\\mathcal B_e(\\bm l\\cdot \\bm s_e) + \\mathcal B_h(\\bm l\\cdot \\bm s_h)\\right] + \\\\\n\\mathcal C\\delta(\\bm r) (\\bm s_e \\cdot \\bm s_h) + \\mathcal H_{cubic} + \\mathcal H_{sd}.\n\\end{multline}\nHere and through the rest of the manuscript, we consider excitons with center of mass wavevector $\\bm K=0$. In Eq.~\\eqref{H:eff} the first two terms, subsequently denoted in short by $\\mathcal H_0$, describe the standard hydrogen-like problem with the relative motion momentum $\\bm p$, the relative motion coordinate $\\bm r$, the electron charge $e$ and the dielectric constant $\\varepsilon$. The quartic term $\\mathcal Ap^4$ describes the nonparabolicity of the kinetic energy with $\\mathcal A$ being a constant. The parameter $\\mathcal A$ can be estimated in the spherical approximation of the Hamiltonian via the Luttinger parameter $\\gamma_2$ as $\\mathcal A \\approx 2\\gamma_2^2/(m_0 ^2\\Delta)$, where $m_0$ is the free electron mass and $\\Delta$ is the splitting between the $\\Gamma_8^+$ and $\\Gamma_7^+$ bands. The second line of Eq.~\\eqref{H:eff} describes the spin-orbit interaction, where $\\bm l=\\hbar^{-1}[\\bm r\\times \\bm p]$ is the angular momentum operator, $\\bm s_e$ ($\\bm s_h$) are the electron (hole) spin-$1/2$ operators acting on the basis functions of the $\\Gamma_6^+$ and $\\Gamma_7^+$ representations, respectively, and $\\mathcal B_e$ ($\\mathcal B_h$) are constants. In the third line, the term $\\mathcal C\\delta(\\bm r) (\\bm s_e \\cdot \\bm s_h)$ describes the short-range electron-hole exchange interaction with the exchange constant $C$, and the additional terms $\\mathcal H_{cubic}$ and $\\mathcal H_{sd}$ account for the cubic anisotropy and the $S-D$-exciton mixing, respectively. In the following the contributions $\\mathcal H_{cubic}$ and $\\mathcal H_{sd}$ are neglected. The Hamiltonian~\\eqref{H:eff} produces quantum defects that are in reasonable agreement with the experiment. To that end we first solve the hydrogenic problem and obtain $\\Psi_{nlm}(\\bm r)$, which are the eigenfunctions of $\\mathcal H_0$. The remaining terms are treated perturbatively. To that end, we introduce the Hamiltonian that is within the applied approximations the extension of the Hamiltonian beyond the hydrogen model:\n\\begin{equation}\n\\label{Hd}\n\\mathcal H_d =\\mathcal A p^4 +\n \\frac{e^2}{\\varepsilon r^3} \\left[\\mathcal B_e(\\bm l\\cdot \\bm s_e) + \\mathcal B_h(\\bm l\\cdot \\bm s_h)\\right] + \n\\mathcal C\\delta(\\bm r) (\\bm s_e \\cdot \\bm s_h),\n\\end{equation}\nand is responsible for the quantum defect appearance.\nRetaining first-order perturbation theory in $\\mathcal H_d$ the contributions leading in the inverse principal quantum number we arrive at~\\cite{ll4_eng}\n\\begin{subequations}\n\\label{defects}\n\\begin{align}\n&\\langle p^4\\rangle_{nlm} = \\left(\\frac{\\hbar}{a_B}\\right)^4 \\frac{4}{n^3(l+1)}, \\label{non:par}\\\\\n&\\left\\langle \\frac{1}{r^3} \\right\\rangle_{nlm} = \\frac{1}{a_B^3} \\frac{1}{n^3l(l+1/2)(l+1)},\\\\\n& \\langle \\delta(\\bm r)\\rangle_{nlm} =  \\frac{\\delta^K_{l,0}}{\\pi} \\frac{1}{a_B n^3},\n\\end{align}\n\\end{subequations} \nwhere $\\langle \\ldots \\rangle_{nlm}$ denotes the quantum-mechanical average of the corresponding quantity over the exciton state $\\Psi_{nlm}(\\bm r)$, $a_B = h^2\\varepsilon/(\\mu e^2)\\approx1.11$~nm is the exciton Bohr radius for the $P$-excitons, and $\\delta^K_{a,b}$ is the Kronecker $\\delta$-symbol. \n\nThe analysis of the experimental data shows that the exciton quantum defects for large $n$ quickly converge to constant values for fixed angular momentum $l$. This observation is supported by Eqs.~\\eqref{defects}, which show that each contribution to the energy deviation of the levels within a multiplet from the simple hydrogen formula scales as $1/n^3$, in agreement with the quantum defect description, Eq.~\\eqref{qdef}. Furthermore, we found experimentally that with increasing $l$ the quantum defects drop continuously from finite positive values towards zero, again, in full agreement with Eqs.~\\eqref{defects}. The $S$-excitons demonstrate the largest quantum defect $\\delta_{n,l=0} \\approx 0.65$, for the $P$-excitons it is about $0.34$ and for the $D$- and $F$-excitons it is reduced to $0.18$ and $0.12$, respectively, in the large-$n$ limit. Theoretical estimates of the non-parabolicity contribution to the quantum defects, Eq.~\\eqref{non:par}, for $\\gamma_2=0.8$~\\cite{PhysRevLett.115.027402,PhysRevB.93.195203} yield $\\delta_{n,l=0}=0.87$, $\\delta_{n,l=1}=0.43$, $\\delta_{n,l=2} =0.28$ and $\\delta_{n,l=3} = 0.21$ in reasonably good agreement with experiment. It supports our conjecture that the dominant contribution to quantum defects arises from the $p^4$ nonparabolic term in the valence band dispersion.\n\n\\begin{figure}[h]\n\\includegraphics[width=\\columnwidth]{Fig4-multipletsplitting.pdf}\n\\caption{Energy spread of the exciton states within a multiplet with fixed principal quantum number versus $n$ in a double logarithmic representation. The line gives a fit to the data after a $n^{-3}$ scaling law, following the theoretical considerations in the text, see Eq.~\\eqref{defects}.} \\label{Fig4-multipletsplitting}\n\\end{figure}\n\nThe consequence of the quantum defect series for a particular principal quantum number is that the associated states are spread over a finite energy range with the low (high) angular momentum states on the low (high) energy side, as clearly seen in the spectra in Fig.~\\ref{Fig2-bfield-spectra-2ndderivative} and in Fig.~\\ref{Fig3-efield-spectra}. The widths of these multiplets were determined from the data in zero field and also in weak external fields, applied to optically activate excitons that are dark without field or to enhance their visibility so that their energies could be extrapolated to zero field with high accuracy. The widths of the multiplets are plotted in Fig.~\\ref{Fig4-multipletsplitting} as function of $n$ in double-logarithmic representation and show a strong drop with increasing principal quantum number from $3$ meV for $n=3$ to less than 0.1 meV for $n$ exceeding 10. The data follow the $1/n^3$ scaling law expected from the quantum defect description and corroborated by the calculations according to Eqs.~\\eqref{defects}, as shown by the red line fit.\n\nThe definition of the Rydberg exciton regime is to some extent arbitrary. We take it here as the regime in which the principal quantum number exceeds $n=5$, because the exciton extension given by twice the average exciton radius exceeds then 100~nm, which is roughly two orders of magnitude larger than the ground state wave function extension. Correspondingly the data in Fig.~\\ref{Fig4-multipletsplitting} and also in all subsequent scaling plots were fitted for $n \\geq 6$, but the plots of the fits were extended also towards lower $n$ in case corresponding data were available like for the multiplet splitting just discussed.\n\n\n\\subsection{Scaling in magnetic field}\\label{sec:magn}\n\nThe magnetic field behavior of the excitons in the Rydberg regime has been assessed in detail in Refs. \\cite{bfield,Aszmann:2016aa}. Entering this regime or $n \\gtrsim 6$, the density of states becomes so large that a precise assignment of the observed states is difficult, in particular, because here avoided crossings and therefore state mixings dominate. Hence statistical methods have been applied by analyzing the distribution of energy separations between nearest exciton levels at fixed transition energies. For $n > 6$ clear signatures of quantum chaos were found due to the dominance of anticrossings as can be seen also from the spectra in Fig.~\\ref{Fig2-bfield-spectra-2ndderivative} in the energy range above about $2.170$~eV. Here we focus on different features: (i) an estimate of the size of the excitons, and (ii) the magnetic field induced crossing of states with adjacent $n$.\n\n\\textit{Exciton size evaluation.} So far, the Rydberg exciton size has been determined somewhat ``indirectly'' from their principal quantum number $n$ using the formula for the average radius of an orbital in the hydrogen model. This average radius is given by~\\cite{ll3_eng}\n\\begin{equation}\n\\label{extension}\n\\langle r\\rangle_{nlm}  = \\frac{a_B}{2} \\left[ 3n^2 - l \\left( l+1 \\right) \\right] \\approx \\frac{3 a_B}{2} n^2  \n\\end{equation}\nwith the approximation being well suited for $n \\gtrsim 6$ for the $P$-excitons. The magnetic field introduces an independent length scale given by the magnetic length, $\\ell_c = \\sqrt{\\hbar c/ eB}$ characterizing the extension of the magnetic confinement potential, which competes with the Coulomb interaction: At low magnetic fields, the main $P$-exciton lines in Fig.~\\ref{Fig1-bfield-spectra} show a rather weak dependence on magnetic field, corresponding to a diamagnetic shift $\\propto B^2$, which changes with increasing field to a stronger shift that can be roughly approximated by a $B$-linear dependence. In order to assess this transition theoretically, it is sufficient to present the effective Hamiltonian of the exciton in the magnetic field $\\bm B\\parallel z$ in the form: \n\\begin{equation}\n\\label{HB:simple}\n\\mathcal H_B = \\mathcal H_0 + \\frac{\\hbar e B}{2\\mu c}l_z + \\frac{\\hbar^2e^2B^2}{8\\mu c^2} (x^2+y^2).\n\\end{equation}\nThe terms responsible for the quantum defect are not of importance here. We also disregard the electron and hole Zeeman spin splittings. For strong magnetic fields one can neglect in first approximation the Coulomb interaction and approximate the states by the electron-hole Landau levels. More precisely, even at strong magnetic fields the Coulomb interaction provides bound magnetoexciton states with the binding energy scaling as ${\\mathcal R}\\ln(a_B/\\ell_c)$~\\cite{ll3_eng}. With increasing field the changeover from an exciton behavior dominated by the Coulomb interaction at low fields to a behavior with dominant magnetic confinement at high fields, where Landau levels are formed, occurs. \n\nCorrespondingly, an exciton resonance in the optical spectrum transforms to a good approximation into a transition between electron and hole Landau levels. Roughly, $P$-excitons with principal quantum number $n$ transform into a transition between Landau levels with quantum number $n$~\\cite{Yafet1956137,PhysRev.188.1294}. The extension of these Landau levels in real space is given by\n\\begin{equation}\n\\label{lcn}\n\\ell_{c,n}  \\approx \\ell_{c} n^{1/2} = \\sqrt{ \\frac{\\hbar c n}{eB} } \n\\end{equation}\nfor not too small $n$. The transition takes place for \n\\[\n\\ell_{c,n} \\sim \\langle r \\rangle_{nlm},\n\\]\ni.e., where the Landau level extension is about equal to the Coulomb extension, which can be achieved by increasing the magnetic field to a particular crossover field strength $B_{c,n}$. From this crossover field one obtains $\\ell_{c,n} = 25.6~$nm$ \\sqrt{n} / \\sqrt{B_{c,n} [\\mathrm T]}$, where $B_{c,n}$ should be inserted in units of Tesla. In the experiment we can determine $B_{c,n}$ as the field strength at which the quadratic field dependence of the $P$-exciton energies changes into a linear one. Note that the crossover fields and the resulting Landau level extensions can be only approximately determined in that way, but they are nevertheless sufficient to test their scaling with $n$. From the considerations above we expect a scaling with $n^{-3}$ for $B_{c,n}$.\n\nFrom the spectra it is confirmed that the transition to Landau-level like behavior occurs at continuously decreasing magnetic fields with increasing principal quantum number. In the high field regime, where also quantum chaos was detected for $n > 6$ no single strong transition is observed for a particular $n$, but bunches of lines which cluster around the Landau level transitions appear. Technically, to determine the crossover field we have taken the center of the line multiplet around a Landau transition and have extrapolated its energy linearly to lower fields, so that we could determine the crossing point $B_{c,n}$ with the quadratic diamagnetic shift of the $P$-exciton line. The $B_{c,n}$ determined in that way are shown by the red circles in Fig.~\\ref{Fig5-lengthscales}, demonstrating the strong drop with increasing $n$. The dotted line gives a $n^{-3}$ fit to the data from which excellent agreement is seen.\n\nFrom these fields $\\ell_{c,n} ( B_{c,n} )$ can be calculated. The Landau level extensions determined in that way are shown in Fig.~\\ref{Fig5-lengthscales} by the black squares and compared to the average exciton radius $r_{n,l}$ according to the hydrogenic formula. For example, for $n=6$ the hydrogenic formula~\\eqref{extension} gives an average radius of 60~nm. The changeover field strength, on the other hand, is $1.3 \\pm 0.3$~T, from which we obtain $\\ell_{c,6}(B_{c,6}) \\approx 55\\pm 8$~nm which are surprisingly close values. For all other $n$ we find similarly good agreement between $\\ell_{c,n}$ and $\\left( r_{n,l} \\right)$, confirming the expected $n^2$ scaling of the wave function extension: $\\ell_{c,n} \\propto (n/B_{c,n})^{1/2} \\propto \\sqrt{n/n^{-3}}$. Note that the inaccuracy in determining $B_{c,n}$ translates through the square-root connection in moderate inaccuracies of the Landau level extension $\\ell_{c,n}$ that are indicated by the error bars. For high $n$ sufficient accuracy was achieved by recording the spectra in 5~mT steps which in combination with the steeper slope of the Landau level transitions facilitates the separation from the low-field behavior. While the observed agreement with the hydrogenic formula may be expected, it also validates the applied exciton description using, for example, a uniform dielectric screening over large length scales. It also reassures the huge extension of the highly excited Rydberg exciton, for which we estimate for $n=20$ an average radius of about 0.7~$\\mu$m.\n\n\\begin{figure}[h]\n\\includegraphics[width=\\linewidth]{Fig5-lengthscales.pdf}\n\\caption{Red circles: crossover field strength $B_{c,n}$ from diamagnetic to Landau level-like behavior as function of principal quantum number. The red dotted line is a fit to the data using a $n^{-3}$ dependence. Black squares: Landau level radius $\\ell_{c,n}$, Eq.~\\eqref{lcn}, estimated from $B_{c,n}$ as function of $n$. The black solid line gives the average radius of the orbital according to the hydrogenic formula, Eq.~\\eqref{extension}.} \\label{Fig5-lengthscales}\n\\end{figure}\n\n\\emph{Magnetic field-induced crossings.} For the field-induced resonances we focus on the first one between exciton states belonging to the multiplets with principal quantum numbers $n$ and $n+1$, as shown in detail in the bottom panel of Fig.~\\ref{Fig2-bfield-spectra-2ndderivative} for field strengths up to $2$ T.  At these resonances we observe systematic crossings, in contrast to the general trend of anticrossings in the spectra. From lower to higher energies we clearly see that the field strength, $B_r$, at which these crossings occur shifts strongly to lower values. This can be expected from (i) the reduced splitting between the exciton states at zero field, (ii) the enhanced field-induced splitting of each of them involving larger angular momenta, and (iii) the stronger diamagnetic shift of each multiplet for higher $n$.\n\nFrom the data we can determine the resonant field strengths $B_r$ versus the principal quantum number. This dependence is shown in Fig.~\\ref{Fig6-resonantmagneticfields} using a double-logarithmic representation. The resonant fields decrease from $B_r=2$~T for $n=6$ to $B_r=0.04$~T for $n=16$.\nFitting the observed data for the Rydberg exciton regime with a power law form reveals a dependence as $n^{-4}$, unlike for Rydberg atoms, where one finds a $n^{-6}$-dependence~\\cite{gallagher2005rydberg}.\n\n\\begin{figure}[h]\n\\includegraphics[width=\\linewidth]{Fig6-resonantmagneticfields.pdf}\n\\caption{Resonant magnetic field strength, $B_r$, at which the first crossing between states belonging to the exciton multiplets $n$ and $n+1$ occurs, as function of the principal quantum number $n$ in a double logarithmic presentation. Symbols give the experimental data, the line is a fit to these data following a $n^{-4}$ law, Eq.~\\eqref{Bc}.}\\label{Fig6-resonantmagneticfields}\n\\end{figure}\n\nThis scaling can be well understood by applying the simplified hydrogen-like model, which is to a good approximation justified because of the quite small impact of the quantum defect on the large-$n$ Rydberg states, as already demonstrated by the $n^{-3}$ scaling of the multiplet width in Fig.~\\ref{Fig4-multipletsplitting}. Using atom-like excitonic units, i.e., $2\\mathcal R = \\mu e^4/(\\varepsilon^2\\hbar^2)$ as the unit of energy, the Bohr radius $a_B$ as the unit of length, and $\\hbar c/(ea_B^2)$ as the unit of magnetic field, and \n\\begin{equation}\n\\label{r2}\n\\langle r^2\\rangle_{nlm} = \\frac{n^2}{2}[5n^2+1-3l(l+1)] \\propto n^4\n\\end{equation}\nfor sufficiently large $n$, we obtain for the level energies to a good approximation:\n\\begin{equation}\n\\label{Enl:B}\nE_{nl} \\approx -\\frac{1}{2n^2} + \\frac{m B}{2} + A_l n^4 B^2.\n\\end{equation}\nHere $A_l$ is a constant that depends on the angular momentum $l$ of the studied level. Neglecting the diamagnetic shift $\\propto B^2$, we obtain for the resonance field of the sublevels $n,l=n-1,m=n-1$ and $n+1, l=n, m=-n$, which are involved in the crossing (corresponding to the states with the maximal $m$ of the $n$th multiplet and with the minimal $m$  of the $n+1$st multiplet), in perturbation approach:\n\\begin{equation}\n\\label{Bc}\nB_r(n) \\propto \\frac{1}{n^4}.\n\\end{equation}\nIt is noteworthy that for the crossing of these states the diamagnetic contribution is vanishingly small, because $A_l n^4 B_r^2(n)\\propto 1/n^4 \\ll mB \\propto 1/n^3$ in Eq.~\\eqref{Enl:B}.  This is different from the atomic physics case where one, as a rule, considers states with magnetic quantum numbers, $m=0$ or $1$, as only these states can be observed in single photon absorption out of an $s$-state. Correspondingly, in the atomic case the paramagnetic term $\\propto mB/2$ in Eq.~\\eqref{Enl:B} is not important and the diamagnetic term $\\propto A_l n^4 B^2$ dominates~\\cite{gallagher2005rydberg}. For atoms one finds therefore a $n^{-6}$ dependence of the resonance field strength. \n\n\\subsection{Electric field}\n\nLet us turn now to the scaling of exciton properties in electric field. As outlined in Sec.~\\ref{sec:exper}, we recorded spectra for two different configurations, taken on a $[110]$ oriented crystal. For the exciting light propagating along the same $[110]$-direction the linear light polarization was chosen either along $[1\\bar10]$ (left panel of Fig.~\\ref{Fig3-efield-spectra}) or along $[001]$ (right panel of the same figure). In the first case the quadrupolar transitions are forbidden so that it is easy to follow the dispersion of the $P$-excitons, which is the first major point of this subsection. In the latter case, they are allowed, so that $S$- and $D$-excitons appear, allowing a more comprehensive insight into the different states within an exciton multiplet~\\cite{efield}. This is exploited here to determine the fields at which states from different multiplets come in resonance, representing the second major point of this part. The same configuration was also used to study the polarizability of the $S$-excitons which are well separated on the low energy flanks of the $P$-excitons. Finally we also address exciton ionization processes, for which we use again the first configuration (without quadrupolar transitions) to determine the ionization field strength as well as the linewidth of $P$-excitons.\n\n\\emph{Scaling of $S$- and $P$-exciton polarizability.} For the $P$-exciton dispersion, we have to distinguish between the low-$n$ and high-$n$ states. For low $n$ the effect of the quantum defect, namely the lifting of level degeneracy, is particularly relevant. In electric field, this leads to the observation of a quadratic Stark effect for the non-degenerate states, because in centrosymmetric crystals any non-degenerate excitonic state has no electric dipole moment without electric field application. The dipole moment is induced by the electric field which subsequently orients it, leading to the quadratic energy shift. This is in contrast to the linear Stark effect obtained by degenerate perturbation theory for a multiplet of levels each having the same energy. In this case, the degenerate states in the multiplet that become coupled by the electric field are linearly combined such that an electric dipole moment is established. This dipole moment only has to be oriented by the field leading to linear energy shifts with increasing field. This situation is to a good approximation relevant for the high $n$-range. Strictly speaking, the exciton differentiation between small and large values of the principal quantum number as well as between non-degenerate and degenerate states depends on the experimental resolution: For each $n$, independent of its value, there is a range of small fields in which a quadratic Stark shift occurs due to level splitting in the crystal. Roughly, this is the range in which the associated quantum defect exceeds the field induced energy shift. This range of quadratic Stark effect decreases, however, strongly with increasing $n$.\n\n\\begin{figure}[h]\n\\includegraphics[width=\\linewidth]{Fig7-polarizabilitySP.pdf}\n\\caption{Polarizability $\\alpha_{n,l}$ of the $S$- (open triangles) and $P$-excitons (closed squares) versus principal quantum number $n$. The line is a fit to the data following the expected $n^7$ scaling law, Eq.~\\eqref{pol:scaling}.}\\label{Fig7-polarizabilitySP}\n\\end{figure}\n\nThe quadratic Stark effect is described by\n\\begin{equation}\n\\label{quadratic}\n\\Delta E_{nl}^{(2)} = - \\alpha_{nl} F^2,\n\\end{equation}\nwhere $\\alpha_{nl}$ is the polarizability of the exciton state $(n,l)$. Although a quadratic Stark effect is already expected for the hydrogenic model with $\\alpha_{nl} \\propto n^6$ due to the field induced mixing of the multiplets with different principal quantum numbers~\\cite{ll3_eng}, the presence of the quantum defect leads to an important change of the polarizability scaling with $n$: Mixing becomes possible for states within a multiplet of particular $n$, which are split due to the $\\mathcal H_d$ part of the Hamiltonian, Eq.~\\eqref{Hd}. The scaling relation of the polarizability can be understood making use of second order perturbation theory to evaluate the quadratic Stark shift: Taking into account the dipole moment operator matrix elements between neighbouring states with the same $n$ that scale as $e\\langle r\\rangle_{nlm} \\propto n^2$, Eq.~\\eqref{extension}, and the scaling of the energy gap between these states $\\propto 1/n^3$, see Fig.~\\ref{Fig4-multipletsplitting} and Sec.~\\ref{sec:zero}, the polarizability scales as\n\\begin{equation}\n\\label{pol:scaling}\n\\alpha_{nl} \\propto n^7,\n\\end{equation}\nin accordance with the behavior typically observed for Rydberg atoms~\\cite{PhysRevA.62.042703,RevModPhys.82.2313}.\n\nHere, we observe for the $P$-excitons in Fig.~\\ref{Fig3-efield-spectra} a quadratic Stark effect for $n \\leq$ 13, in agreement with Eq.~\\eqref{quadratic}, while the splitting pattern for the higher-$n$ excitons approaches the linear Stark fan of hydrogen within the experimental resolution. In Fig.~\\ref{Fig3-efield-spectra}(a) the shift of the $P$-excitons to lower energies with increasing field can be well resolved and increases drastically with increasing principal quantum number, see the blue-colored feature of lowest energy in each $n$-manifold. From the data we can assess the polarizability $\\alpha_{n,P}$ ($l=1$) which is shown by the solid squares in Fig.~\\ref{Fig7-polarizabilitySP} as function of $n$ in a double-logarithmic plot. The polarizability increases from about 5 $\\mu$eV/V$^2$ for $n$ = 5 to about 2000 $\\mu$eV/V$^2$ for $n$ = 12. The solid lines give the fit to these data by a power law scaling with the seventh power of $n$, in accordance with the theoretical expectations, Eq.~\\eqref{pol:scaling}, from which a good description of the data is obtained. In Fig.~\\ref{Fig3-efield-spectra}(b), due to the chosen polarization configuration, also the $S$-excitons can be observed, which show also a quadratic Stark effect. Deriving also for them the polarizability, which is possible for $n \\leq 10$, one obtains the data shown by the open triangles in Fig.~\\ref{Fig7-polarizabilitySP}. Within the experimental error no difference in polarizability between $S$- and $P$-excitons can be resolved.\n\n\\emph{Scaling of electric field induced resonances.} Next we turn to the first resonances occurring  with increasing electric field between states of adjacent principal quantum numbers. Similar to the magnetic field case, we find also here a strong shift of the resonance voltage $U_r$ to lower values with increasing $n$. Looking at the resonances in closer detail, we find, however, that at these resonances levels do not cross but systematically avoid each other, as shown in the mid panel of Fig.~\\ref{Fig3-efield-spectra}. Extrapolating the dispersions, one can determine the resonance voltages which are shown in Fig.~\\ref{Fig8-resonancevoltages} in a double logarithmic representation versus $n$. $U_r$ decreases from 8~V for $n=5$ to about 40~mV for $n=13$. The data are in reasonably good accord with a power law scaling like $n^{-5}$ which is in line with the results on Rydberg atoms. Resonance voltage and field strength $F_r$ are connected by a simple proportionality relation for the chosen capacitor geometry by which the field is applied. However, care needs to be exercised in doing this conversion using just the nominal geometric and dielectric parameters, because surface charges, charged defects etc. can lead to depolarization effects in the crystal, so that there may be some discrepancy between the nominally calculated and the actually present field strength by a factor of $3 \\ldots 5$, as discussed in detail in Ref.~\\cite{efield}.\n\n\\begin{figure}[h]\n\\includegraphics[width=\\linewidth]{Fig8-resonancevoltages.pdf}\n\\caption{Principal quantum number $n$-dependence of the resonance voltage $U_r$ at which the first resonance of levels belonging to the multiplets with principal quantum numbers $n$ and $n+1$ occurs. At these resonances avoided crossings are observed. The red line shows a fit according to a $n^{-3}$ scaling law, Eq.~\\eqref{Fc}.}\\label{Fig8-resonancevoltages}\n\\end{figure}\n\nThis behavior can be understood again using the hydrogen-like model, because the quantum defect is not essential for the evaluation of the resonant fields. In the presence of an electric field the simplified Hamiltonian of the exciton reads:\n\\begin{equation}\n\\mathcal H_E = H_0 + eFz,\n\\end{equation}\nwhere we assumed that the field $\\bm F$ is applied along the $z$-direction. In first-order perturbation theory:\n\\begin{equation}\n\\label{Enl:F}\n\\frac{E_{nl}}{2\\mathcal R} = -\\frac{1}{2n^2} + \\frac{3}{2}\\tilde Fn(n_1-n_2),\n\\end{equation}\nwhere $\\tilde F = Fa_B/(2\\mathcal R)$ is the reduced electric field. Here $n_1$ and $n_2$ are the parabolic quantum numbers being non-negative integers with $n=n_1+n_2+|m|+1$. The resonance field strength, $F_r$, can be evaluated from Eq.~\\eqref{Enl:F} putting $n_1-n_2 \\approx \\pm n$ for $n\\gg 1$ and for the states with highest and lowest energy of the manifold with the result~\\cite{gallagher2005rydberg}:\n\\begin{equation}\n\\label{Fc}\nF_{r} \\propto \\frac{1}{3n^5}.\n\\end{equation}\nNote that the power-law dependence is different from the magnetic field case. This is because the electric dipole moment increases as $n^2$~\\cite{ll3_eng}, whereas the magnetic field induced splitting is determined by the paramagnetic contribution, which scales linearly with $n$ for the lowest and highest energy states in a multiplet.\n\nWe have also determined the energy splittings at these anticrossings as function of $n$ which are shown in Fig.~\\ref{Fig9-anticrossingenergies}. The data show a strong drop with $n$, even though one has to emphasize the splitting magnitude is only slightly larger than the linewidths of the involved transitions, resulting in considerable error bars, despite of which the drop from about 80 $\\mu$eV splitting for $n$ = 5 to about 10 $\\mu$eV for $n$ = 10 can be determined. The data can be reasonably well described by a $n^{-4}$ fit following also from perturbation theory. In order to estimate the anticrossing energy we follow Refs.~\\cite{gallagher2005rydberg,popov} and evaluate the matrix elements of the ``quantum defect'' Hamiltonian~\\eqref{Hd} on the eigenfunctions of the hydrogen atom in the parabolic coordinates $\\Psi_{nn_1n_2m}(\\bm r)$, where $n_1$, $n_2$, and $m$ are the parabolic quantum numbers. The avoided crossing energy reads\n\\begin{equation}\n\\label{anti:stark}\n\\delta E_{anti} = 2\\left|\\int d\\bm r \\Psi_{n'n_1'n_2'm}^*(\\bm r) \\mathcal H_d \\Psi_{nn_1n_2m}(\\bm r)  \\right|.\n\\end{equation}\nFor the hydrogenic states with adjacent principal quantum numbers $n'=n\\pm 1$ we have~\\cite{gallagher2005rydberg}\n\\[\n\\int d\\bm r \\Psi_{n'lm}^*(\\bm r) \\mathcal H_d \\Psi_{nlm}(\\bm r) \\sim \\frac{\\mathcal R\\delta_{nl}}{n^3}.\n\\]\nFinally, taking into account that \n\\[\n\\sum_l \\int d\\bm r \\Psi_{n'n_1'n_2'm}^*(\\bm r)\\Psi_{nlm}(\\bm r) \\propto \\frac{1}{n-m},\n\\]\nwe obtain the scaling law\n\\begin{equation}\n\\label{anti:stark:sc}\n\\delta E_{anti} \\sim \\frac{\\mathcal R \\delta_{nl}}{n^3(n-m)} \\propto 1/n^4.\n\\end{equation}\nUsing for a rough estimate $\\delta_{nl}=0.5$ (Sec.~\\ref{sec:zero}) we obtain for the $n=5$ and $n=6$ multiplets an anticrossing energy of about $70$~$\\mu$eV in reasonable agreement with the data in Fig.~\\ref{Fig9-anticrossingenergies}.\n\n\\begin{figure}[h]\n\\includegraphics[width=\\linewidth]{Fig9-anticrossingenergies.pdf}\n\\caption{Black squares: Dependence of the energy splittings due to anticrossings at the first resonances of states from the exciton multiplets $n$ and $n+1$, induced by the electric field, on the principal quantum number. The red line gives a fit to the data according to a $n^{-4}$ dependence, Eq.~\\eqref{anti:stark:sc}.}\n\\label{Fig9-anticrossingenergies}\n\\end{figure} \n\n\\emph{Crossing vs. anticrossing}. To understand the origin of crossing or anticrossing at the first resonance of states arising from the multiplets with principal quantum number $n$ and $n+1$ in magnetic and electric field, respectively, we have to consider the problem in more detail: Both electric and magnetic field preserve the axial symmetry of the ``hydrogenic'' problem, therefore the $z$-component of the angular momentum or ``magnetic'' quantum number $m$ is a good quantum number. In magnetic field the first crossing occurs between the states with $m=n-1$ (the maximal $m$ in the manifold with $n$) and $m=-n$ (the minimal $m$ in the manifold with $n+1$). The difference of the angular momentum components of these states is\n\\begin{equation}\n\\label{Delta:lz:B}\n\\Delta l_z = 2n+1.\n\\end{equation}\nThis is an odd number $1$, $3$, \\ldots, so that the parity of excitonic states which cross in magnetic field is different. In the centrosymmetric point group $O_h$ the states are characterized by a definite parity as well and, therefore, states with odd $\\Delta l_z$ do not mix. Hence, one may expect that the first crossing in magnetic field is indeed an \\emph{allowed} crossing. In electric field the situation is different. One can readily check by perturbative calculations that the first resonance occurs for states with $m=0$, e.g., $|n=2S\\rangle + |n=2P_z\\rangle$ and $|3S\\rangle - |3P_z\\rangle$. These states have the same symmetry and, hence, the crossing of these states is \\emph{avoided}.\n\nHowever, one has to be careful when using this line of argumentation to discuss the crossings/anticrossings of \\emph{observed} states in the transmission spectra. In fact, the consideration above neglects the symmetry of the two-particle Bloch function of the exciton~\\cite{PhysRevLett.115.027402}. All observed excitonic states (in a given geometry and polarization) interact with light and, hence, have the \\emph{same} symmetry, $\\Gamma_4^-$. For example, the states observed in the geometry where light with $[1\\bar10] \\parallel y$ polarization propagates along the $[110] \\parallel z$ direction, couple to the $y$-component of the electric field and therefore transform as this $y$-coordinate. Hence, at least the light-matter interaction, i.e., the polariton effect~\\cite{excitons:RS,PSSB:PSSB2221730104,PhysRevB.94.045205}, can convert allowed crossings into avoided ones. However, the polariton effect is relatively weak in cuprous oxide crystals where the direct interband transitions are forbidden at the $\\Gamma$-point of the Brillouin zone, leading to a small oscillator strength of allowed exciton transitions~\\cite{NIKITINE1961292,PSSB:PSSB2221730104}. The quantitative measure for the strength of the polariton effect is the longitudinal-transverse splitting, $\\hbar\\omega_{nlm,LT}$, of the exciton state with quantum numbers $(n,l,m)$. Provided that the longitudinal-transverse splitting is smaller than the non-radiative damping of the excitonic states, \n\\[\n\\hbar\\omega_{nlm,LT} < \\hbar \\Gamma_{nlm},\n\\]\nthe light-matter interaction is in the weak coupling regime so that polariton modes are not observable, and also the anticrossings are hidden in the optical spectra.\n\n\\emph{Ionization in electric field.} A particularly appealing feature of Rydberg excitons in comparison to Rydberg atoms is the fact that the true high electric field regime, in which the interaction strength with the field exceeds the Coulomb interaction, can be reached easily in the laboratory, while for atoms it is much harder to enter this limit. This is mostly due to the different Rydberg energy in cuprous oxide, $\\sim90$~meV, which is more than two orders smaller than the hydrogen atom Rydberg energy. From Fig.~\\ref{Fig3-efield-spectra} one sees that for elevated electric field strengths the excitons dissociate: The absorption lines fade away and finally disappear with increasing field. The fields required for dissociation are the smaller, the higher the principal quantum number of the involved exciton state is. A series of spectra recorded for the $n=11$ multiplet is shown in Fig.~\\ref{Fig10-highresolutionspectrainefieldn=11}, from which the drop of the $P$-exciton resonance with increasing applied voltage can be quantitatively assessed. Simultaneously with the line drop the width of the resonance increases.\n\nOn the high energy side of the $P$-exciton another peak can be seen, which can be associated with the $F$-excitons that can hardly be identified without applied voltage, but become more and more pronounced with increasing voltage. At higher energies even further weak features appear. This increase of visibility is partly related to the transfer of oscillator strength from the $P$ to the $F$ and other higher angular momenta excitons. It is somewhat surprising that the $F$-exciton absorption is still growing even when the $P$-exciton absorption has basically completely vanished. A potential explanation is the action of the centrifugal barrier in the kinetic energy which varies with the angular momentum of the involved state. Due to its action the particles are kept away from the field axis $z$, making these states more robust with respect to field application with increasing $l$. However, the complex phenomenology associated with these excitons has not yet fully been developed so that we will not discuss it here in further detail. \n\n\\begin{figure}[h]\n\\includegraphics[width=\\linewidth]{Fig10-highresolutionspectrainefieldn=11.pdf}\n\\caption{High resolution absorption spectra of the $n=11$ exciton multiplet recorded with a frequency stabilized laser for different applied voltages.}\\label{Fig10-highresolutionspectrainefieldn=11}\n\\end{figure}\n\nTo assess the $P$-exciton behavior more quantitatively, we have to consider the impact of the electric field in more detail. The associated potential leads to a tilting of the Coulomb potential between electron and hole along the field direction. Due to this tilting eventually tunneling of carriers through the potential can occur for sufficiently high applied voltages. Ultimately in the high field regime the exciton state is moved into the state continuum so that it can no longer be observed. This behavior determines both the absorption strength (given by the area below a resonance) and its linewidth (given by the lifetime of the state): At low fields where tunneling is still not relevant, both quantities should stay constant, while in the tunneling regime we expect an exponential dependence for both the spectrum area and linewidth. \n\nFor determining the ionization field strength we have calculated the area below each resonance. For that purpose we have fitted each resonance with the Toyozawa formula~\\cite{toyozawa,ueno,jolk} \n\\begin{equation}\n\\label{toyozawa}\n\\alpha_n(E)=C_n\\frac{\\frac{\\Gamma_n}{2}+2q_n(E-E_n)}{\\left(\\frac{\\Gamma_n}{2}\\right)^2+(E-E_n)^2}\n\\end{equation}\nfrom which we obtain also the linewidth. Here $E_n$ and $\\Gamma_n$ are the energy and the damping of the state $n$. $q_n$ describes the asymmetry of the observed resonance and $C_n$ gives its amplitude. Here we have restricted to the voltage regime in which the $P$-exciton line represents a well isolated line that is not influenced by adjacent absorption features. This is the regime in which the emerging $F$-excitons are still much weaker in comparison. Once higher angular momentum excitons become prominent, the spectra on the high energy flank of each $P$-exciton show pronounced modulations which we attribute to interactions with the electron-hole continuum that are not fully understood so far. This modulation also aggravates a lineshape analysis so that we refrain from a line analysis in this regime. In Fig.~\\ref{Fig10-highresolutionspectrainefieldn=11} this regime corresponds to voltages higher than 0.25~V and is decreasing for higher $n$. Then the area below each absorption peak indeed shows to a good approximation an exponential drop with applied voltage as soon as effects of the applied voltage set in, up to the point where the line disappears. \n\nFrom this dependence we have determined the effective ionization voltage taken as the voltage at which the area below the resonance has dropped to $1/e$. \nThe drop has in fact two origins: (1) the redistribution of oscillator strength to other excitons due to state mixing; (2) the actual dissociation of the excitons. As all states are affected similarly by these two factors, we still use the voltage determined in the described way as characteristic for ionization. This voltage is plotted in Fig.~\\ref{Fig11-ionizationvoltage} as function of the principal quantum number. The data can be well described by a dependence proportional to $n^{-4}$. This is the dependence that is also expected by setting Eq.~\\eqref{Enl:F} for the exciton energy $E_{n,l}$ in electric field to zero. \n\n\\begin{figure}[h]\n\\includegraphics[width=\\linewidth]{Fig11-ionizationvoltage.pdf}\n\\caption{Dependence of the dissociation voltage, determined by taking the $1/e$ value of the area below a resonance, on the principal quantum number. The solid line shows a fit to the data by a $n^{-4}$ dependence.}\\label{Fig11-ionizationvoltage}\n\\end{figure}\n\n\\emph{Exciton linewidth.} Finally, we turn to the dependence of the linewidth on the electric field strength, for which  one would expect -- as described above -- a transition for increasing voltage from being constant for negligible tunneling to strongly increasing as soon as tunneling can occur until ultimately the potential barrier is lowered to an extent that the excitons become unbound, which is given by the field strength at which the energy in Eq.~\\eqref{Enl:F} is zero. The experimental data for the excitons from $n=10$ up to $n=16$ are shown in Fig.~\\ref{Fig12-linewidthsvsvoltage}. At zero electric field we observe the well known decrease of the linewidth with increasing $n$ that has been described already in Ref.~\\cite{Kazimierczuk:2014yq}.  For sufficiently low excitation powers such that effects like power broadening can be disregarded, the linewidth dependence on the principal quantum number can be described by a $n^{-3}$ law due to the corresponding scaling of the transition rates for photon and phonon emission. \n\nWhen applying the electric field, we find that the low lying excitons in the series indeed follow roughly the expected dependence (dashed line in Fig.~\\ref{Fig12-linewidthsvsvoltage}) which according to the theoretical calculations should be given by~\\cite{merkulov_ion,excitons:RS}\n\\begin{equation}\n\\label{Gamma}\n\\Gamma \\propto \\exp{\\left(-\\frac{2}{3n^3F}\\right)},\n\\end{equation}\nsee Appendix~\\ref{app:ion} for details. Somewhat surprisingly, the linewidth shows a different behavior for the high lying excitons in the shown set of states. Their linewidth stays constant within the experimental accuracy over the range of fields where they can be observed until they disappear.\n\n\\begin{figure}[h]\n\\includegraphics[width=\\linewidth]{Fig12-linewidthsvsvoltage.pdf}\n\\caption{Dependence of the linewidth of the $P$-exciton resonances on the applied voltage for different principal quantum numbers from $n=10$ up to $n=16$. The dashed lines show a fit to the data of the excitons $n=10$ and $n=11$ according to Eq.~\\eqref{Gamma}. For clarity in the left (right) panel the even (odd) $n$ states are shown.}\\label{Fig12-linewidthsvsvoltage}\n\\end{figure}\n\nThis behavior can be understood as follows. The exponent becomes significant at the critical field\n\\begin{equation}\n\\label{Fg}\nF_\\Gamma \\sim \\frac{1}{n^3}.\n\\end{equation}\nSince this field is parametrically larger than the ionization field $F_i$ for the states with large $n$, the increase in the linewidth is not seen, because of faster ionization of the state.\n\n\n\\begin{table*}[t]\n\\caption{\\label{tab:comparison}Comparison of scaling laws with principal quantum number $n$ for Rydberg atoms and Rydberg excitons.}\n\\begin{ruledtabular}\n\\begin{tabular}{ccc}\n & Rydberg atoms & Rydberg excitons \\\\ \\hline\n\\emph{Zero field} & & \\\\\nMultiplet splitting due to quantum defect & $\\propto n^{-3}$ (except of hydrogen) & $\\propto n^{-3}$ \\\\ \\hline \n\\emph{Electric field} & & \\\\ \nPolarizability & $\\propto n^{-7}$ ($\\propto n^{-6}$ for hydrogen) & $\\propto n^{-7}$ \\\\\nResonance field of states from multiplets $n$ and $n+1$&$\\propto n^{-5}$&$\\propto n^{-5}$\\\\ \nAnticrossing energy at first resonance & $\\propto n^{-4}$ & $\\propto n^{-4}$ \\\\\nIonization voltage & $\\propto n^{-4}$ & $\\propto n^{-4}$ \\\\\n\\hline\n\\emph{Magnetic field} & & \\\\ \nCrossover field to magnetoexciton & $\\propto n^{-3}$  & $\\propto n^{-3}$ \\\\\nResonance field of states from multiplets $n$ and $n+1$&$\\propto n^{-6}$&$\\propto n^{-4}$\\\\ \n\\end{tabular}\n\\end{ruledtabular}\n\\end{table*}\n\n3.1 Zero-field behavior\n\\subsection{Zero-field behavior}\\label{sec:zero}\n\n\\begin{figure*}[ht]\n\\includegraphics[width=1\\linewidth]{Fig3-efield-spectra.png}\n\\caption{Contour plot of the second derivatives of absorption spectra versus applied voltage. The spectra were recorded by white light excitation at $T = 1.3$~K on a Cu$_2$O crystal slab with $[110]$ orientation. The light was polarized along the $[1\\bar10]$ direction in the left panel and along the $[001]$ direction in the right panel. The mid panel shows a close-up of the section of the contour plot in the left panel, marked there by the solid box, in order to highlight the anticrossing at the first resonance involving the state of the $n$ = 6 multiplet showing the strongest field dispersion to higher energies and the state of the $n$ = 7 multiplet with the strongest dispersion to lower energies. The weak equidistant vertical stripes in the left and right panel are artifacts of taking the 2nd derivative.}\\label{Fig3-efield-spectra}\n\\end{figure*}\n\nThe exciton energies in zero external field can be described well by the quantum defect formalism of Eq.~\\eqref{qdef}~\\cite{PhysRevB.93.075203}. We recall that in Cu$_2$O the origin of the quantum defect is quite different from that in many-electron Rydberg atoms and is related with the complex valence band structure: The separation between the topmost $\\Gamma_7^+$ and the closest $\\Gamma_8^+$ valence subbands is on the same order of magnitude as the exciton Rydberg energy $\\mathcal R$. Hence, the mixing of these bands via the off-diagonal elements of the Luttinger Hamiltonian gives rise to the deviation of the excitonic series from a hydrogenic one and to the fine structure of the exciton energy spectrum. Moreover, the exchange interaction between the electron and the hole also contributes significantly to the exciton state splitting and mixing~\\cite{PhysRevB.23.2731,PhysRevLett.115.027402,PhysRevB.93.195203,efield}. For excitons with principal quantum numbers $n\\gtrsim 4$ estimates show that the mixing of the valence subbands can be treated perturbatively and the effective Hamiltonian can be recast in the following form~[cf.~\\cite{PhysRevB.93.075203,efield}]:\n\\begin{multline}\n\\label{H:eff}\n\\mathcal H = \\frac{p^2}{2\\mu} - \\frac{e^2}{\\varepsilon r} + \\mathcal A p^4 +\\\\\n \\frac{e^2}{\\varepsilon r^3} \\left[\\mathcal B_e(\\bm l\\cdot \\bm s_e) + \\mathcal B_h(\\bm l\\cdot \\bm s_h)\\right] + \\\\\n\\mathcal C\\delta(\\bm r) (\\bm s_e \\cdot \\bm s_h) + \\mathcal H_{cubic} + \\mathcal H_{sd}.\n\\end{multline}\nHere and through the rest of the manuscript, we consider excitons with center of mass wavevector $\\bm K=0$. In Eq.~\\eqref{H:eff} the first two terms, subsequently denoted in short by $\\mathcal H_0$, describe the standard hydrogen-like problem with the relative motion momentum $\\bm p$, the relative motion coordinate $\\bm r$, the electron charge $e$ and the dielectric constant $\\varepsilon$. The quartic term $\\mathcal Ap^4$ describes the nonparabolicity of the kinetic energy with $\\mathcal A$ being a constant. The parameter $\\mathcal A$ can be estimated in the spherical approximation of the Hamiltonian via the Luttinger parameter $\\gamma_2$ as $\\mathcal A \\approx 2\\gamma_2^2/(m_0 ^2\\Delta)$, where $m_0$ is the free electron mass and $\\Delta$ is the splitting between the $\\Gamma_8^+$ and $\\Gamma_7^+$ bands. The second line of Eq.~\\eqref{H:eff} describes the spin-orbit interaction, where $\\bm l=\\hbar^{-1}[\\bm r\\times \\bm p]$ is the angular momentum operator, $\\bm s_e$ ($\\bm s_h$) are the electron (hole) spin-$1/2$ operators acting on the basis functions of the $\\Gamma_6^+$ and $\\Gamma_7^+$ representations, respectively, and $\\mathcal B_e$ ($\\mathcal B_h$) are constants. In the third line, the term $\\mathcal C\\delta(\\bm r) (\\bm s_e \\cdot \\bm s_h)$ describes the short-range electron-hole exchange interaction with the exchange constant $C$, and the additional terms $\\mathcal H_{cubic}$ and $\\mathcal H_{sd}$ account for the cubic anisotropy and the $S-D$-exciton mixing, respectively. In the following the contributions $\\mathcal H_{cubic}$ and $\\mathcal H_{sd}$ are neglected. The Hamiltonian~\\eqref{H:eff} produces quantum defects that are in reasonable agreement with the experiment. To that end we first solve the hydrogenic problem and obtain $\\Psi_{nlm}(\\bm r)$, which are the eigenfunctions of $\\mathcal H_0$. The remaining terms are treated perturbatively. To that end, we introduce the Hamiltonian that is within the applied approximations the extension of the Hamiltonian beyond the hydrogen model:\n\\begin{equation}\n\\label{Hd}\n\\mathcal H_d =\\mathcal A p^4 +\n \\frac{e^2}{\\varepsilon r^3} \\left[\\mathcal B_e(\\bm l\\cdot \\bm s_e) + \\mathcal B_h(\\bm l\\cdot \\bm s_h)\\right] + \n\\mathcal C\\delta(\\bm r) (\\bm s_e \\cdot \\bm s_h),\n\\end{equation}\nand is responsible for the quantum defect appearance.\nRetaining first-order perturbation theory in $\\mathcal H_d$ the contributions leading in the inverse principal quantum number we arrive at~\\cite{ll4_eng}\n\\begin{subequations}\n\\label{defects}\n\\begin{align}\n&\\langle p^4\\rangle_{nlm} = \\left(\\frac{\\hbar}{a_B}\\right)^4 \\frac{4}{n^3(l+1)}, \\label{non:par}\\\\\n&\\left\\langle \\frac{1}{r^3} \\right\\rangle_{nlm} = \\frac{1}{a_B^3} \\frac{1}{n^3l(l+1/2)(l+1)},\\\\\n& \\langle \\delta(\\bm r)\\rangle_{nlm} =  \\frac{\\delta^K_{l,0}}{\\pi} \\frac{1}{a_B n^3},\n\\end{align}\n\\end{subequations} \nwhere $\\langle \\ldots \\rangle_{nlm}$ denotes the quantum-mechanical average of the corresponding quantity over the exciton state $\\Psi_{nlm}(\\bm r)$, $a_B = h^2\\varepsilon/(\\mu e^2)\\approx1.11$~nm is the exciton Bohr radius for the $P$-excitons, and $\\delta^K_{a,b}$ is the Kronecker $\\delta$-symbol. \n\nThe analysis of the experimental data shows that the exciton quantum defects for large $n$ quickly converge to constant values for fixed angular momentum $l$. This observation is supported by Eqs.~\\eqref{defects}, which show that each contribution to the energy deviation of the levels within a multiplet from the simple hydrogen formula scales as $1/n^3$, in agreement with the quantum defect description, Eq.~\\eqref{qdef}. Furthermore, we found experimentally that with increasing $l$ the quantum defects drop continuously from finite positive values towards zero, again, in full agreement with Eqs.~\\eqref{defects}. The $S$-excitons demonstrate the largest quantum defect $\\delta_{n,l=0} \\approx 0.65$, for the $P$-excitons it is about $0.34$ and for the $D$- and $F$-excitons it is reduced to $0.18$ and $0.12$, respectively, in the large-$n$ limit. Theoretical estimates of the non-parabolicity contribution to the quantum defects, Eq.~\\eqref{non:par}, for $\\gamma_2=0.8$~\\cite{PhysRevLett.115.027402,PhysRevB.93.195203} yield $\\delta_{n,l=0}=0.87$, $\\delta_{n,l=1}=0.43$, $\\delta_{n,l=2} =0.28$ and $\\delta_{n,l=3} = 0.21$ in reasonably good agreement with experiment. It supports our conjecture that the dominant contribution to quantum defects arises from the $p^4$ nonparabolic term in the valence band dispersion.\n\n\\begin{figure}[h]\n\\includegraphics[width=\\columnwidth]{Fig4-multipletsplitting.pdf}\n\\caption{Energy spread of the exciton states within a multiplet with fixed principal quantum number versus $n$ in a double logarithmic representation. The line gives a fit to the data after a $n^{-3}$ scaling law, following the theoretical considerations in the text, see Eq.~\\eqref{defects}.} \\label{Fig4-multipletsplitting}\n\\end{figure}\n\nThe consequence of the quantum defect series for a particular principal quantum number is that the associated states are spread over a finite energy range with the low (high) angular momentum states on the low (high) energy side, as clearly seen in the spectra in Fig.~\\ref{Fig2-bfield-spectra-2ndderivative} and in Fig.~\\ref{Fig3-efield-spectra}. The widths of these multiplets were determined from the data in zero field and also in weak external fields, applied to optically activate excitons that are dark without field or to enhance their visibility so that their energies could be extrapolated to zero field with high accuracy. The widths of the multiplets are plotted in Fig.~\\ref{Fig4-multipletsplitting} as function of $n$ in double-logarithmic representation and show a strong drop with increasing principal quantum number from $3$ meV for $n=3$ to less than 0.1 meV for $n$ exceeding 10. The data follow the $1/n^3$ scaling law expected from the quantum defect description and corroborated by the calculations according to Eqs.~\\eqref{defects}, as shown by the red line fit.\n\nThe definition of the Rydberg exciton regime is to some extent arbitrary. We take it here as the regime in which the principal quantum number exceeds $n=5$, because the exciton extension given by twice the average exciton radius exceeds then 100~nm, which is roughly two orders of magnitude larger than the ground state wave function extension. Correspondingly the data in Fig.~\\ref{Fig4-multipletsplitting} and also in all subsequent scaling plots were fitted for $n \\geq 6$, but the plots of the fits were extended also towards lower $n$ in case corresponding data were available like for the multiplet splitting just discussed.\n\n\n3.3 Electric field\n\\subsection{Electric field}\n\nLet us turn now to the scaling of exciton properties in electric field. As outlined in Sec.~\\ref{sec:exper}, we recorded spectra for two different configurations, taken on a $[110]$ oriented crystal. For the exciting light propagating along the same $[110]$-direction the linear light polarization was chosen either along $[1\\bar10]$ (left panel of Fig.~\\ref{Fig3-efield-spectra}) or along $[001]$ (right panel of the same figure). In the first case the quadrupolar transitions are forbidden so that it is easy to follow the dispersion of the $P$-excitons, which is the first major point of this subsection. In the latter case, they are allowed, so that $S$- and $D$-excitons appear, allowing a more comprehensive insight into the different states within an exciton multiplet~\\cite{efield}. This is exploited here to determine the fields at which states from different multiplets come in resonance, representing the second major point of this part. The same configuration was also used to study the polarizability of the $S$-excitons which are well separated on the low energy flanks of the $P$-excitons. Finally we also address exciton ionization processes, for which we use again the first configuration (without quadrupolar transitions) to determine the ionization field strength as well as the linewidth of $P$-excitons.\n\n\\emph{Scaling of $S$- and $P$-exciton polarizability.} For the $P$-exciton dispersion, we have to distinguish between the low-$n$ and high-$n$ states. For low $n$ the effect of the quantum defect, namely the lifting of level degeneracy, is particularly relevant. In electric field, this leads to the observation of a quadratic Stark effect for the non-degenerate states, because in centrosymmetric crystals any non-degenerate excitonic state has no electric dipole moment without electric field application. The dipole moment is induced by the electric field which subsequently orients it, leading to the quadratic energy shift. This is in contrast to the linear Stark effect obtained by degenerate perturbation theory for a multiplet of levels each having the same energy. In this case, the degenerate states in the multiplet that become coupled by the electric field are linearly combined such that an electric dipole moment is established. This dipole moment only has to be oriented by the field leading to linear energy shifts with increasing field. This situation is to a good approximation relevant for the high $n$-range. Strictly speaking, the exciton differentiation between small and large values of the principal quantum number as well as between non-degenerate and degenerate states depends on the experimental resolution: For each $n$, independent of its value, there is a range of small fields in which a quadratic Stark shift occurs due to level splitting in the crystal. Roughly, this is the range in which the associated quantum defect exceeds the field induced energy shift. This range of quadratic Stark effect decreases, however, strongly with increasing $n$.\n\n\\begin{figure}[h]\n\\includegraphics[width=\\linewidth]{Fig7-polarizabilitySP.pdf}\n\\caption{Polarizability $\\alpha_{n,l}$ of the $S$- (open triangles) and $P$-excitons (closed squares) versus principal quantum number $n$. The line is a fit to the data following the expected $n^7$ scaling law, Eq.~\\eqref{pol:scaling}.}\\label{Fig7-polarizabilitySP}\n\\end{figure}\n\nThe quadratic Stark effect is described by\n\\begin{equation}\n\\label{quadratic}\n\\Delta E_{nl}^{(2)} = - \\alpha_{nl} F^2,\n\\end{equation}\nwhere $\\alpha_{nl}$ is the polarizability of the exciton state $(n,l)$. Although a quadratic Stark effect is already expected for the hydrogenic model with $\\alpha_{nl} \\propto n^6$ due to the field induced mixing of the multiplets with different principal quantum numbers~\\cite{ll3_eng}, the presence of the quantum defect leads to an important change of the polarizability scaling with $n$: Mixing becomes possible for states within a multiplet of particular $n$, which are split due to the $\\mathcal H_d$ part of the Hamiltonian, Eq.~\\eqref{Hd}. The scaling relation of the polarizability can be understood making use of second order perturbation theory to evaluate the quadratic Stark shift: Taking into account the dipole moment operator matrix elements between neighbouring states with the same $n$ that scale as $e\\langle r\\rangle_{nlm} \\propto n^2$, Eq.~\\eqref{extension}, and the scaling of the energy gap between these states $\\propto 1/n^3$, see Fig.~\\ref{Fig4-multipletsplitting} and Sec.~\\ref{sec:zero}, the polarizability scales as\n\\begin{equation}\n\\label{pol:scaling}\n\\alpha_{nl} \\propto n^7,\n\\end{equation}\nin accordance with the behavior typically observed for Rydberg atoms~\\cite{PhysRevA.62.042703,RevModPhys.82.2313}.\n\nHere, we observe for the $P$-excitons in Fig.~\\ref{Fig3-efield-spectra} a quadratic Stark effect for $n \\leq$ 13, in agreement with Eq.~\\eqref{quadratic}, while the splitting pattern for the higher-$n$ excitons approaches the linear Stark fan of hydrogen within the experimental resolution. In Fig.~\\ref{Fig3-efield-spectra}(a) the shift of the $P$-excitons to lower energies with increasing field can be well resolved and increases drastically with increasing principal quantum number, see the blue-colored feature of lowest energy in each $n$-manifold. From the data we can assess the polarizability $\\alpha_{n,P}$ ($l=1$) which is shown by the solid squares in Fig.~\\ref{Fig7-polarizabilitySP} as function of $n$ in a double-logarithmic plot. The polarizability increases from about 5 $\\mu$eV/V$^2$ for $n$ = 5 to about 2000 $\\mu$eV/V$^2$ for $n$ = 12. The solid lines give the fit to these data by a power law scaling with the seventh power of $n$, in accordance with the theoretical expectations, Eq.~\\eqref{pol:scaling}, from which a good description of the data is obtained. In Fig.~\\ref{Fig3-efield-spectra}(b), due to the chosen polarization configuration, also the $S$-excitons can be observed, which show also a quadratic Stark effect. Deriving also for them the polarizability, which is possible for $n \\leq 10$, one obtains the data shown by the open triangles in Fig.~\\ref{Fig7-polarizabilitySP}. Within the experimental error no difference in polarizability between $S$- and $P$-excitons can be resolved.\n\n\\emph{Scaling of electric field induced resonances.} Next we turn to the first resonances occurring  with increasing electric field between states of adjacent principal quantum numbers. Similar to the magnetic field case, we find also here a strong shift of the resonance voltage $U_r$ to lower values with increasing $n$. Looking at the resonances in closer detail, we find, however, that at these resonances levels do not cross but systematically avoid each other, as shown in the mid panel of Fig.~\\ref{Fig3-efield-spectra}. Extrapolating the dispersions, one can determine the resonance voltages which are shown in Fig.~\\ref{Fig8-resonancevoltages} in a double logarithmic representation versus $n$. $U_r$ decreases from 8~V for $n=5$ to about 40~mV for $n=13$. The data are in reasonably good accord with a power law scaling like $n^{-5}$ which is in line with the results on Rydberg atoms. Resonance voltage and field strength $F_r$ are connected by a simple proportionality relation for the chosen capacitor geometry by which the field is applied. However, care needs to be exercised in doing this conversion using just the nominal geometric and dielectric parameters, because surface charges, charged defects etc. can lead to depolarization effects in the crystal, so that there may be some discrepancy between the nominally calculated and the actually present field strength by a factor of $3 \\ldots 5$, as discussed in detail in Ref.~\\cite{efield}.\n\n\\begin{figure}[h]\n\\includegraphics[width=\\linewidth]{Fig8-resonancevoltages.pdf}\n\\caption{Principal quantum number $n$-dependence of the resonance voltage $U_r$ at which the first resonance of levels belonging to the multiplets with principal quantum numbers $n$ and $n+1$ occurs. At these resonances avoided crossings are observed. The red line shows a fit according to a $n^{-3}$ scaling law, Eq.~\\eqref{Fc}.}\\label{Fig8-resonancevoltages}\n\\end{figure}\n\nThis behavior can be understood again using the hydrogen-like model, because the quantum defect is not essential for the evaluation of the resonant fields. In the presence of an electric field the simplified Hamiltonian of the exciton reads:\n\\begin{equation}\n\\mathcal H_E = H_0 + eFz,\n\\end{equation}\nwhere we assumed that the field $\\bm F$ is applied along the $z$-direction. In first-order perturbation theory:\n\\begin{equation}\n\\label{Enl:F}\n\\frac{E_{nl}}{2\\mathcal R} = -\\frac{1}{2n^2} + \\frac{3}{2}\\tilde Fn(n_1-n_2),\n\\end{equation}\nwhere $\\tilde F = Fa_B/(2\\mathcal R)$ is the reduced electric field. Here $n_1$ and $n_2$ are the parabolic quantum numbers being non-negative integers with $n=n_1+n_2+|m|+1$. The resonance field strength, $F_r$, can be evaluated from Eq.~\\eqref{Enl:F} putting $n_1-n_2 \\approx \\pm n$ for $n\\gg 1$ and for the states with highest and lowest energy of the manifold with the result~\\cite{gallagher2005rydberg}:\n\\begin{equation}\n\\label{Fc}\nF_{r} \\propto \\frac{1}{3n^5}.\n\\end{equation}\nNote that the power-law dependence is different from the magnetic field case. This is because the electric dipole moment increases as $n^2$~\\cite{ll3_eng}, whereas the magnetic field induced splitting is determined by the paramagnetic contribution, which scales linearly with $n$ for the lowest and highest energy states in a multiplet.\n\nWe have also determined the energy splittings at these anticrossings as function of $n$ which are shown in Fig.~\\ref{Fig9-anticrossingenergies}. The data show a strong drop with $n$, even though one has to emphasize the splitting magnitude is only slightly larger than the linewidths of the involved transitions, resulting in considerable error bars, despite of which the drop from about 80 $\\mu$eV splitting for $n$ = 5 to about 10 $\\mu$eV for $n$ = 10 can be determined. The data can be reasonably well described by a $n^{-4}$ fit following also from perturbation theory. In order to estimate the anticrossing energy we follow Refs.~\\cite{gallagher2005rydberg,popov} and evaluate the matrix elements of the ``quantum defect'' Hamiltonian~\\eqref{Hd} on the eigenfunctions of the hydrogen atom in the parabolic coordinates $\\Psi_{nn_1n_2m}(\\bm r)$, where $n_1$, $n_2$, and $m$ are the parabolic quantum numbers. The avoided crossing energy reads\n\\begin{equation}\n\\label{anti:stark}\n\\delta E_{anti} = 2\\left|\\int d\\bm r \\Psi_{n'n_1'n_2'm}^*(\\bm r) \\mathcal H_d \\Psi_{nn_1n_2m}(\\bm r)  \\right|.\n\\end{equation}\nFor the hydrogenic states with adjacent principal quantum numbers $n'=n\\pm 1$ we have~\\cite{gallagher2005rydberg}\n\\[\n\\int d\\bm r \\Psi_{n'lm}^*(\\bm r) \\mathcal H_d \\Psi_{nlm}(\\bm r) \\sim \\frac{\\mathcal R\\delta_{nl}}{n^3}.\n\\]\nFinally, taking into account that \n\\[\n\\sum_l \\int d\\bm r \\Psi_{n'n_1'n_2'm}^*(\\bm r)\\Psi_{nlm}(\\bm r) \\propto \\frac{1}{n-m},\n\\]\nwe obtain the scaling law\n\\begin{equation}\n\\label{anti:stark:sc}\n\\delta E_{anti} \\sim \\frac{\\mathcal R \\delta_{nl}}{n^3(n-m)} \\propto 1/n^4.\n\\end{equation}\nUsing for a rough estimate $\\delta_{nl}=0.5$ (Sec.~\\ref{sec:zero}) we obtain for the $n=5$ and $n=6$ multiplets an anticrossing energy of about $70$~$\\mu$eV in reasonable agreement with the data in Fig.~\\ref{Fig9-anticrossingenergies}.\n\n\\begin{figure}[h]\n\\includegraphics[width=\\linewidth]{Fig9-anticrossingenergies.pdf}\n\\caption{Black squares: Dependence of the energy splittings due to anticrossings at the first resonances of states from the exciton multiplets $n$ and $n+1$, induced by the electric field, on the principal quantum number. The red line gives a fit to the data according to a $n^{-4}$ dependence, Eq.~\\eqref{anti:stark:sc}.}\n\\label{Fig9-anticrossingenergies}\n\\end{figure} \n\n\\emph{Crossing vs. anticrossing}. To understand the origin of crossing or anticrossing at the first resonance of states arising from the multiplets with principal quantum number $n$ and $n+1$ in magnetic and electric field, respectively, we have to consider the problem in more detail: Both electric and magnetic field preserve the axial symmetry of the ``hydrogenic'' problem, therefore the $z$-component of the angular momentum or ``magnetic'' quantum number $m$ is a good quantum number. In magnetic field the first crossing occurs between the states with $m=n-1$ (the maximal $m$ in the manifold with $n$) and $m=-n$ (the minimal $m$ in the manifold with $n+1$). The difference of the angular momentum components of these states is\n\\begin{equation}\n\\label{Delta:lz:B}\n\\Delta l_z = 2n+1.\n\\end{equation}\nThis is an odd number $1$, $3$, \\ldots, so that the parity of excitonic states which cross in magnetic field is different. In the centrosymmetric point group $O_h$ the states are characterized by a definite parity as well and, therefore, states with odd $\\Delta l_z$ do not mix. Hence, one may expect that the first crossing in magnetic field is indeed an \\emph{allowed} crossing. In electric field the situation is different. One can readily check by perturbative calculations that the first resonance occurs for states with $m=0$, e.g., $|n=2S\\rangle + |n=2P_z\\rangle$ and $|3S\\rangle - |3P_z\\rangle$. These states have the same symmetry and, hence, the crossing of these states is \\emph{avoided}.\n\nHowever, one has to be careful when using this line of argumentation to discuss the crossings/anticrossings of \\emph{observed} states in the transmission spectra. In fact, the consideration above neglects the symmetry of the two-particle Bloch function of the exciton~\\cite{PhysRevLett.115.027402}. All observed excitonic states (in a given geometry and polarization) interact with light and, hence, have the \\emph{same} symmetry, $\\Gamma_4^-$. For example, the states observed in the geometry where light with $[1\\bar10] \\parallel y$ polarization propagates along the $[110] \\parallel z$ direction, couple to the $y$-component of the electric field and therefore transform as this $y$-coordinate. Hence, at least the light-matter interaction, i.e., the polariton effect~\\cite{excitons:RS,PSSB:PSSB2221730104,PhysRevB.94.045205}, can convert allowed crossings into avoided ones. However, the polariton effect is relatively weak in cuprous oxide crystals where the direct interband transitions are forbidden at the $\\Gamma$-point of the Brillouin zone, leading to a small oscillator strength of allowed exciton transitions~\\cite{NIKITINE1961292,PSSB:PSSB2221730104}. The quantitative measure for the strength of the polariton effect is the longitudinal-transverse splitting, $\\hbar\\omega_{nlm,LT}$, of the exciton state with quantum numbers $(n,l,m)$. Provided that the longitudinal-transverse splitting is smaller than the non-radiative damping of the excitonic states, \n\\[\n\\hbar\\omega_{nlm,LT} < \\hbar \\Gamma_{nlm},\n\\]\nthe light-matter interaction is in the weak coupling regime so that polariton modes are not observable, and also the anticrossings are hidden in the optical spectra.\n\n\\emph{Ionization in electric field.} A particularly appealing feature of Rydberg excitons in comparison to Rydberg atoms is the fact that the true high electric field regime, in which the interaction strength with the field exceeds the Coulomb interaction, can be reached easily in the laboratory, while for atoms it is much harder to enter this limit. This is mostly due to the different Rydberg energy in cuprous oxide, $\\sim90$~meV, which is more than two orders smaller than the hydrogen atom Rydberg energy. From Fig.~\\ref{Fig3-efield-spectra} one sees that for elevated electric field strengths the excitons dissociate: The absorption lines fade away and finally disappear with increasing field. The fields required for dissociation are the smaller, the higher the principal quantum number of the involved exciton state is. A series of spectra recorded for the $n=11$ multiplet is shown in Fig.~\\ref{Fig10-highresolutionspectrainefieldn=11}, from which the drop of the $P$-exciton resonance with increasing applied voltage can be quantitatively assessed. Simultaneously with the line drop the width of the resonance increases.\n\nOn the high energy side of the $P$-exciton another peak can be seen, which can be associated with the $F$-excitons that can hardly be identified without applied voltage, but become more and more pronounced with increasing voltage. At higher energies even further weak features appear. This increase of visibility is partly related to the transfer of oscillator strength from the $P$ to the $F$ and other higher angular momenta excitons. It is somewhat surprising that the $F$-exciton absorption is still growing even when the $P$-exciton absorption has basically completely vanished. A potential explanation is the action of the centrifugal barrier in the kinetic energy which varies with the angular momentum of the involved state. Due to its action the particles are kept away from the field axis $z$, making these states more robust with respect to field application with increasing $l$. However, the complex phenomenology associated with these excitons has not yet fully been developed so that we will not discuss it here in further detail. \n\n\\begin{figure}[h]\n\\includegraphics[width=\\linewidth]{Fig10-highresolutionspectrainefieldn=11.pdf}\n\\caption{High resolution absorption spectra of the $n=11$ exciton multiplet recorded with a frequency stabilized laser for different applied voltages.}\\label{Fig10-highresolutionspectrainefieldn=11}\n\\end{figure}\n\nTo assess the $P$-exciton behavior more quantitatively, we have to consider the impact of the electric field in more detail. The associated potential leads to a tilting of the Coulomb potential between electron and hole along the field direction. Due to this tilting eventually tunneling of carriers through the potential can occur for sufficiently high applied voltages. Ultimately in the high field regime the exciton state is moved into the state continuum so that it can no longer be observed. This behavior determines both the absorption strength (given by the area below a resonance) and its linewidth (given by the lifetime of the state): At low fields where tunneling is still not relevant, both quantities should stay constant, while in the tunneling regime we expect an exponential dependence for both the spectrum area and linewidth. \n\nFor determining the ionization field strength we have calculated the area below each resonance. For that purpose we have fitted each resonance with the Toyozawa formula~\\cite{toyozawa,ueno,jolk} \n\\begin{equation}\n\\label{toyozawa}\n\\alpha_n(E)=C_n\\frac{\\frac{\\Gamma_n}{2}+2q_n(E-E_n)}{\\left(\\frac{\\Gamma_n}{2}\\right)^2+(E-E_n)^2}\n\\end{equation}\nfrom which we obtain also the linewidth. Here $E_n$ and $\\Gamma_n$ are the energy and the damping of the state $n$. $q_n$ describes the asymmetry of the observed resonance and $C_n$ gives its amplitude. Here we have restricted to the voltage regime in which the $P$-exciton line represents a well isolated line that is not influenced by adjacent absorption features. This is the regime in which the emerging $F$-excitons are still much weaker in comparison. Once higher angular momentum excitons become prominent, the spectra on the high energy flank of each $P$-exciton show pronounced modulations which we attribute to interactions with the electron-hole continuum that are not fully understood so far. This modulation also aggravates a lineshape analysis so that we refrain from a line analysis in this regime. In Fig.~\\ref{Fig10-highresolutionspectrainefieldn=11} this regime corresponds to voltages higher than 0.25~V and is decreasing for higher $n$. Then the area below each absorption peak indeed shows to a good approximation an exponential drop with applied voltage as soon as effects of the applied voltage set in, up to the point where the line disappears. \n\nFrom this dependence we have determined the effective ionization voltage taken as the voltage at which the area below the resonance has dropped to $1/e$. \nThe drop has in fact two origins: (1) the redistribution of oscillator strength to other excitons due to state mixing; (2) the actual dissociation of the excitons. As all states are affected similarly by these two factors, we still use the voltage determined in the described way as characteristic for ionization. This voltage is plotted in Fig.~\\ref{Fig11-ionizationvoltage} as function of the principal quantum number. The data can be well described by a dependence proportional to $n^{-4}$. This is the dependence that is also expected by setting Eq.~\\eqref{Enl:F} for the exciton energy $E_{n,l}$ in electric field to zero. \n\n\\begin{figure}[h]\n\\includegraphics[width=\\linewidth]{Fig11-ionizationvoltage.pdf}\n\\caption{Dependence of the dissociation voltage, determined by taking the $1/e$ value of the area below a resonance, on the principal quantum number. The solid line shows a fit to the data by a $n^{-4}$ dependence.}\\label{Fig11-ionizationvoltage}\n\\end{figure}\n\n\\emph{Exciton linewidth.} Finally, we turn to the dependence of the linewidth on the electric field strength, for which  one would expect -- as described above -- a transition for increasing voltage from being constant for negligible tunneling to strongly increasing as soon as tunneling can occur until ultimately the potential barrier is lowered to an extent that the excitons become unbound, which is given by the field strength at which the energy in Eq.~\\eqref{Enl:F} is zero. The experimental data for the excitons from $n=10$ up to $n=16$ are shown in Fig.~\\ref{Fig12-linewidthsvsvoltage}. At zero electric field we observe the well known decrease of the linewidth with increasing $n$ that has been described already in Ref.~\\cite{Kazimierczuk:2014yq}.  For sufficiently low excitation powers such that effects like power broadening can be disregarded, the linewidth dependence on the principal quantum number can be described by a $n^{-3}$ law due to the corresponding scaling of the transition rates for photon and phonon emission. \n\nWhen applying the electric field, we find that the low lying excitons in the series indeed follow roughly the expected dependence (dashed line in Fig.~\\ref{Fig12-linewidthsvsvoltage}) which according to the theoretical calculations should be given by~\\cite{merkulov_ion,excitons:RS}\n\\begin{equation}\n\\label{Gamma}\n\\Gamma \\propto \\exp{\\left(-\\frac{2}{3n^3F}\\right)},\n\\end{equation}\nsee Appendix~\\ref{app:ion} for details. Somewhat surprisingly, the linewidth shows a different behavior for the high lying excitons in the shown set of states. Their linewidth stays constant within the experimental accuracy over the range of fields where they can be observed until they disappear.\n\n\\begin{figure}[h]\n\\includegraphics[width=\\linewidth]{Fig12-linewidthsvsvoltage.pdf}\n\\caption{Dependence of the linewidth of the $P$-exciton resonances on the applied voltage for different principal quantum numbers from $n=10$ up to $n=16$. The dashed lines show a fit to the data of the excitons $n=10$ and $n=11$ according to Eq.~\\eqref{Gamma}. For clarity in the left (right) panel the even (odd) $n$ states are shown.}\\label{Fig12-linewidthsvsvoltage}\n\\end{figure}\n\nThis behavior can be understood as follows. The exponent becomes significant at the critical field\n\\begin{equation}\n\\label{Fg}\nF_\\Gamma \\sim \\frac{1}{n^3}.\n\\end{equation}\nSince this field is parametrically larger than the ionization field $F_i$ for the states with large $n$, the increase in the linewidth is not seen, because of faster ionization of the state.\n\n\n\\begin{table*}[t]\n\\caption{\\label{tab:comparison}Comparison of scaling laws with principal quantum number $n$ for Rydberg atoms and Rydberg excitons.}\n\\begin{ruledtabular}\n\\begin{tabular}{ccc}\n & Rydberg atoms & Rydberg excitons \\\\ \\hline\n\\emph{Zero field} & & \\\\\nMultiplet splitting due to quantum defect & $\\propto n^{-3}$ (except of hydrogen) & $\\propto n^{-3}$ \\\\ \\hline \n\\emph{Electric field} & & \\\\ \nPolarizability & $\\propto n^{-7}$ ($\\propto n^{-6}$ for hydrogen) & $\\propto n^{-7}$ \\\\\nResonance field of states from multiplets $n$ and $n+1$&$\\propto n^{-5}$&$\\propto n^{-5}$\\\\ \nAnticrossing energy at first resonance & $\\propto n^{-4}$ & $\\propto n^{-4}$ \\\\\nIonization voltage & $\\propto n^{-4}$ & $\\propto n^{-4}$ \\\\\n\\hline\n\\emph{Magnetic field} & & \\\\ \nCrossover field to magnetoexciton & $\\propto n^{-3}$  & $\\propto n^{-3}$ \\\\\nResonance field of states from multiplets $n$ and $n+1$&$\\propto n^{-6}$&$\\propto n^{-4}$\\\\ \n\\end{tabular}\n\\end{ruledtabular}\n\\end{table*}\n\n",
        "label": "Fig3-efield-spectra",
        "Descriptive_question1": "What is the temperature at which the spectra were recorded in figure_3?",
        "Descriptive_question2": "What is the orientation of the Cu2O crystal slab shown in figure_3?",
        "Reasoning_question1": "Why do the spectra in figure_3 show fewer lines compared to magnetic field spectra, and how does this impact the observation of scaling laws?",
        "Reasoning_question2": "What can be inferred from the anticrossing highlighted in the mid panel of figure_3 regarding the interaction between states of the n=6 and n=7 multiplets?",
        "Descriptive_answer1": "1.3 K",
        "Descriptive_answer2": "[110]",
        "Reasoning_answer1": "The spectra in figure_3 show fewer lines compared to magnetic field spectra because, in electric field, exciton states with principal quantum number n > 10 are subject to field-induced dissociation even at relatively low voltages below 10 V. This leads to a smaller number of observable exciton lines, as opposed to magnetic field spectra where a multitude of levels can be seen. Consequently, the restriction to low field strengths or to lines with dominant oscillator strength becomes necessary to identify scaling laws, making the analysis more limited than in the magnetic field case.",
        "Reasoning_answer2": "The anticrossing highlighted in the mid panel of figure_3 involves states of the n=6 and n=7 multiplets that show the strongest field dispersion to higher and lower energies, respectively. This anticrossing indicates an interaction or coupling between these states, leading to avoided crossing behavior rather than a simple crossing. This implies mixing of the excitonic states from adjacent principal quantum numbers in the electric field, consistent with the observed avoided crossings due to same symmetry states interacting and resulting in energy level repulsion."
    },
    {
        "paper_id": "1704.00974.json",
        "image_id": "figure_13",
        "image_path": [
            "/home/yz979/scratch/chengye/scimolmo/test-images-tables/dataset_imagestables/1704.00974/images/Fig13-Appendix-LandauLevels.png"
        ],
        "caption": "Close-up of the black boxed region of the magnetic field spectra in Fig.~\\ref{Fig1-bfield-spectra}, which allows one to resolve higher Landau level transitions. Zooming in even further allows detection of transitions associated with Landau level quantum numbers of more than 70.",
        "classify": "Chart",
        "section_info": "2 Experiment\n\\section{Experiment}\\label{sec:exper}\n\nThe experiments were performed on bulk-like cuprous oxide crystal slabs that were cut and polished from a natural rock.  \nThe samples were placed strain-free in a sample holder that allowed application of an electric field \\cite{Brandt07}. The holder was placed at $T = 1.3$~K in the liquid helium insert of an optical cryostat equipped with a magnet coil for fields up to 7 T. Both electric and magnetic field were applied in longitudinal field configuration, i.e. they were oriented along the optical axis normal to the crystal slab plane. Further technical details can be found in Refs.~\\cite{efield,bfield}.\n\nThe absorption was measured, depending on the required spectral resolution, by using either a broadband white light source or a frequency stabilized dye laser with neV-linewidth for excitation. For the white light source the bandwidth was reduced to the wavelength range of interest by a double monochromator. \nAfter transmission through the crystal the white light was dispersed by another double monochromator and detected by a Si-charge coupled device camera. The spectral resolution provided is less than 10~$\\mu$eV which is sufficient for exciton states with $n < 20$ to study the effects of interest here. \nWhen the laser was used to obtain a particularly high spectral resolution, its output was stabilized by a noise eater. After transmission through the sample it was detected by a avalanche photodiode. The power density of the exciting light was chosen low enough that the excitation of dressed states as discussed in Ref.~\\cite{PhysRevLett.117.133003} can be neglected and the experimentally observed spectral lines correspond to resonant absorption peaks.\n\nRecently we showed that the absorption spectra of cuprous oxide crystals vary with the light polarization in combination with the chosen crystal orientation. For example, in electric field applied to a $[110]$ oriented crystal, the quadrupole allowed transitions to the $S$- and $D$-excitons which can be observed for light polarized linearly along the $[001]$-direction are absent for light polarized along the $[1\\bar10]$ direction, simplifying the spectra significantly~\\cite{efield}.  Depending on the problem to be studied, we chose the appropriate polarization configuration. If the task was to address the whole multiplet belonging to a particular $n$ for which as many states as possible need to be resolved in a manifold, we used the $[001]$-polarization, while the $[1\\bar10]$-polarization was chosen when, for example, the field dispersion of the $P$-excitons was to be measured with high accuracy.\n\nFor the subsequent discussion several remarks are due. For atoms, in a constant external field the spherical symmetry is reduced, but the rotational symmetry about the field is maintained, so that the angular momentum component along the field direction, $m$, remains a good quantum number. In a longitudinal field configuration, where the optical axis is oriented along the field, well defined selection rules hold in electric dipole approximation for single photon absorption: Starting from an electron in a $s$-shell, only transitions to $p$-shell states are possible due to the restriction $\\Delta l = \\pm 1$, whereby all states in this state triplet can be excited leaving the magnetic quantum number unchanged, $\\Delta m=0$, or changing it by unity, $\\Delta m = \\pm 1$, for linearly or circularly polarized light, respectively. Other states of the Rydberg manifold with higher angular momenta $l > 1$, for example, cannot be excited out of a $s$-shell state.\n\nThe situation is different for the yellow excitons in cuprous oxide, where the dominant dipole-allowed excitons are those with $P$-type envelope wavefunction. The absorption of light takes place through excitation of an exciton in a particular quantum state out of the ground state of the crystal. Due to the reduction from continuous to discrete symmetry oscillator strength is shuffled from the $P$-states to other odd exciton states, namely the $F$-, $H$-, $\\ldots$ excitons, even though it remains at least two orders of magnitude smaller compared to the $P$-states. Furthermore, the $S$-exciton transitions are allowed in quadrupole approximation, and, through the exchange coupling, also the $D$-excitons become accessible. Thereby, if allowed by the transition matrix elements, the observation of states is facilitated or enhanced at the expense of the $P$-excitons, as described quantitatively in recent studies of low lying excitons~\\cite{efield,bfield}.\n\n\\begin{figure*}[ht]\n\\includegraphics[width=\\textwidth]{Fig1-bfield-spectra.png}\n\\caption{Contour plot of absorption spectra versus magnetic field applied in the Faraday configuration (light propagation along the magnetic field), recorded at $T$ = 1.3~K on a Cu$_2$O crystal slab. The dashed line indicates the band gap at zero field. Landau level quantum numbers that can be approximately assigned to a bunch of clustered transitions are indicated at the right and top axes. The black frame indicates the area from which a close-up is shown in the Appendix \\ref{app:ll} as Fig.~\\ref{Fig-Appendix-LandauLevels}. The scale on the right shows gives the strength of the absorption features in arbitrary units.}\\label{Fig1-bfield-spectra}\n\\end{figure*}\n\nIn more detail, a magnetic field enhances the visibility of the weak states that are accessible through the crystal symmetry-induced mixing of excitons with odd parity envelope as this mixing is strongly increased by the field. Quadrupole-allowed excitons with even parity  \nremain weak~\\cite{bfield}. By contrast, in electric field the excitons with opposite parity become mixed from which the even parity excitons profit in oscillator strength. Hence, the $S$-, $D$-, \\ldots excitons become more pronounced in the absorption spectra. As a consequence of this mixing by the crystal structure and by the external field, the majority of states within an exciton multiplet becomes easily detectable, in contrast to atoms. This allows for access also to high angular momentum states by single photon absorption. \n\n3 Scaling of exciton properties\n\\section{Scaling of exciton properties}\\label{sec:scaling}\n\nBefore discussing the different scaling laws, we first present briefly the comprehensive set of data from which the laws are extracted. These data comprise spectra recorded in magnetic and electric field, from which also the understanding of the zero field level spectrum can be deepened. Figure~\\ref{Fig1-bfield-spectra} shows a contour plot of exciton absorption spectra versus the magnetic field applied in the Faraday geometry. At zero field the hydrogen-like series of exciton states is seen, out of which a complex Zeeman splitting pattern arises. Better insight into the details of these spectra can be obtained by taking their second derivatives from which weak features become well resolved. The corresponding contour plot of the second derivatives is shown in Fig.~\\ref{Fig2-bfield-spectra-2ndderivative}: in panel (a) for the states with $n>4$ up to $B$ = 7~T, while in panel (b) a close-up of the low-field range up to 2~T is shown for $n>6$. At zero field the levels belonging to a particular principal quantum number are spread over an energy range which becomes the narrower, the higher $n$ is. However, in magnetic field the splitting of a multiplet becomes larger for higher $n$, because higher angular momentum states limited by $l \\leq n-1$ contribute. \n\nDue to the multitude of observed levels in particular at medium field strengths it appears difficult to extract generally valid scaling laws for the dependences on the principal quantum number there. For that purpose we have to restrict our analysis to ranges, where states can be identified well. This is the case \nat low fields up to the point, where resonances of states belonging to exciton multiplets with different principal quantum numbers occur.  \nSimilarly, states can be quite well identified in high magnetic fields, where the dominant observed lines tend to cluster around transitions that correspond to those between Landau levels, see Fig.~\\ref{Fig1-bfield-spectra}. In fact, the close-up of Fig.~\\ref{Fig1-bfield-spectra} shown in the Appendix~\\ref{app:ll} as Fig.~\\ref{Fig-Appendix-LandauLevels} reveals transitions which can be associated with Landau level quantum numbers up to 79, which arise in magnetic field from $P$-excitons with $n$=79 in zero field, see also discussion below in Sec.~\\ref{sec:magn}. The corresponding average radius of this exciton wavefunction would be 10.4~$\\mu$m, which is squeezed in magnetic field, thereby enhancing the oscillator strength so that these highly excited states become visible. The $n$=79 state can be observed starting from 0.5~T, where the Landau level extension is about 300~nm.\n\n\\begin{figure*}[ht]\n\\includegraphics[width=1\\textwidth]{Fig2-bfield-spectra-2ndderivative.pdf}\n\\caption{Top panel: Same data as in Fig.~\\ref{Fig1-bfield-spectra}, but in form of a contour plot of the second derivative of the absorption spectra versus magnetic field for $n \\geq 5$. Lower panel: Close-up of the states for $n \\geq 7$ in $B$ up to 2~T. $T$ = 1.3~K. The scales on the right give the strength of the features  in arbitrary units. The weak equidistant vertical stripes which are apparent mostly at low energies in the upper panel, are artifacts of taking the 2nd derivative.}\\label{Fig2-bfield-spectra-2ndderivative}\n\\end{figure*}\n\nExciton level splitting can be also induced by applying an electric field and exploiting the Stark effect. Corresponding spectra are shown in Fig.~\\ref{Fig3-efield-spectra} where their second derivatives are plotted as function of the voltage applied to the sample for the excitons with $n \\geq 5$. Overall, the number of lines observed there remains smaller than in magnetic field because already for pretty low voltages below 10~V the exciton states with $n >$10 are subject to field-induced dissociation. Still, for identifying scaling laws, one has to restrict also here to the low field strength regime or to lines with  dominant oscillator strength compared to the other features. As discussed above, variation of the linear polarization of the exciting light allows one to vary the number of detected spectral lines. In the left panel, the light polarization was $\\hat{\\bm e} \\parallel[1\\bar{1}0]$ and in the right panel the polarization was $\\hat{\\bm e}\\parallel[001]$. In the former case, the quadrupolar-active, even exciton states (e.g., $S$- and $D$-excitons) are forbidden, while in the latter case they are allowed, giving the spectra a more complex appearance.\n\n\\subsection{Zero-field behavior}\\label{sec:zero}\n\n\\begin{figure*}[ht]\n\\includegraphics[width=1\\linewidth]{Fig3-efield-spectra.png}\n\\caption{Contour plot of the second derivatives of absorption spectra versus applied voltage. The spectra were recorded by white light excitation at $T = 1.3$~K on a Cu$_2$O crystal slab with $[110]$ orientation. The light was polarized along the $[1\\bar10]$ direction in the left panel and along the $[001]$ direction in the right panel. The mid panel shows a close-up of the section of the contour plot in the left panel, marked there by the solid box, in order to highlight the anticrossing at the first resonance involving the state of the $n$ = 6 multiplet showing the strongest field dispersion to higher energies and the state of the $n$ = 7 multiplet with the strongest dispersion to lower energies. The weak equidistant vertical stripes in the left and right panel are artifacts of taking the 2nd derivative.}\\label{Fig3-efield-spectra}\n\\end{figure*}\n\nThe exciton energies in zero external field can be described well by the quantum defect formalism of Eq.~\\eqref{qdef}~\\cite{PhysRevB.93.075203}. We recall that in Cu$_2$O the origin of the quantum defect is quite different from that in many-electron Rydberg atoms and is related with the complex valence band structure: The separation between the topmost $\\Gamma_7^+$ and the closest $\\Gamma_8^+$ valence subbands is on the same order of magnitude as the exciton Rydberg energy $\\mathcal R$. Hence, the mixing of these bands via the off-diagonal elements of the Luttinger Hamiltonian gives rise to the deviation of the excitonic series from a hydrogenic one and to the fine structure of the exciton energy spectrum. Moreover, the exchange interaction between the electron and the hole also contributes significantly to the exciton state splitting and mixing~\\cite{PhysRevB.23.2731,PhysRevLett.115.027402,PhysRevB.93.195203,efield}. For excitons with principal quantum numbers $n\\gtrsim 4$ estimates show that the mixing of the valence subbands can be treated perturbatively and the effective Hamiltonian can be recast in the following form~[cf.~\\cite{PhysRevB.93.075203,efield}]:\n\\begin{multline}\n\\label{H:eff}\n\\mathcal H = \\frac{p^2}{2\\mu} - \\frac{e^2}{\\varepsilon r} + \\mathcal A p^4 +\\\\\n \\frac{e^2}{\\varepsilon r^3} \\left[\\mathcal B_e(\\bm l\\cdot \\bm s_e) + \\mathcal B_h(\\bm l\\cdot \\bm s_h)\\right] + \\\\\n\\mathcal C\\delta(\\bm r) (\\bm s_e \\cdot \\bm s_h) + \\mathcal H_{cubic} + \\mathcal H_{sd}.\n\\end{multline}\nHere and through the rest of the manuscript, we consider excitons with center of mass wavevector $\\bm K=0$. In Eq.~\\eqref{H:eff} the first two terms, subsequently denoted in short by $\\mathcal H_0$, describe the standard hydrogen-like problem with the relative motion momentum $\\bm p$, the relative motion coordinate $\\bm r$, the electron charge $e$ and the dielectric constant $\\varepsilon$. The quartic term $\\mathcal Ap^4$ describes the nonparabolicity of the kinetic energy with $\\mathcal A$ being a constant. The parameter $\\mathcal A$ can be estimated in the spherical approximation of the Hamiltonian via the Luttinger parameter $\\gamma_2$ as $\\mathcal A \\approx 2\\gamma_2^2/(m_0 ^2\\Delta)$, where $m_0$ is the free electron mass and $\\Delta$ is the splitting between the $\\Gamma_8^+$ and $\\Gamma_7^+$ bands. The second line of Eq.~\\eqref{H:eff} describes the spin-orbit interaction, where $\\bm l=\\hbar^{-1}[\\bm r\\times \\bm p]$ is the angular momentum operator, $\\bm s_e$ ($\\bm s_h$) are the electron (hole) spin-$1/2$ operators acting on the basis functions of the $\\Gamma_6^+$ and $\\Gamma_7^+$ representations, respectively, and $\\mathcal B_e$ ($\\mathcal B_h$) are constants. In the third line, the term $\\mathcal C\\delta(\\bm r) (\\bm s_e \\cdot \\bm s_h)$ describes the short-range electron-hole exchange interaction with the exchange constant $C$, and the additional terms $\\mathcal H_{cubic}$ and $\\mathcal H_{sd}$ account for the cubic anisotropy and the $S-D$-exciton mixing, respectively. In the following the contributions $\\mathcal H_{cubic}$ and $\\mathcal H_{sd}$ are neglected. The Hamiltonian~\\eqref{H:eff} produces quantum defects that are in reasonable agreement with the experiment. To that end we first solve the hydrogenic problem and obtain $\\Psi_{nlm}(\\bm r)$, which are the eigenfunctions of $\\mathcal H_0$. The remaining terms are treated perturbatively. To that end, we introduce the Hamiltonian that is within the applied approximations the extension of the Hamiltonian beyond the hydrogen model:\n\\begin{equation}\n\\label{Hd}\n\\mathcal H_d =\\mathcal A p^4 +\n \\frac{e^2}{\\varepsilon r^3} \\left[\\mathcal B_e(\\bm l\\cdot \\bm s_e) + \\mathcal B_h(\\bm l\\cdot \\bm s_h)\\right] + \n\\mathcal C\\delta(\\bm r) (\\bm s_e \\cdot \\bm s_h),\n\\end{equation}\nand is responsible for the quantum defect appearance.\nRetaining first-order perturbation theory in $\\mathcal H_d$ the contributions leading in the inverse principal quantum number we arrive at~\\cite{ll4_eng}\n\\begin{subequations}\n\\label{defects}\n\\begin{align}\n&\\langle p^4\\rangle_{nlm} = \\left(\\frac{\\hbar}{a_B}\\right)^4 \\frac{4}{n^3(l+1)}, \\label{non:par}\\\\\n&\\left\\langle \\frac{1}{r^3} \\right\\rangle_{nlm} = \\frac{1}{a_B^3} \\frac{1}{n^3l(l+1/2)(l+1)},\\\\\n& \\langle \\delta(\\bm r)\\rangle_{nlm} =  \\frac{\\delta^K_{l,0}}{\\pi} \\frac{1}{a_B n^3},\n\\end{align}\n\\end{subequations} \nwhere $\\langle \\ldots \\rangle_{nlm}$ denotes the quantum-mechanical average of the corresponding quantity over the exciton state $\\Psi_{nlm}(\\bm r)$, $a_B = h^2\\varepsilon/(\\mu e^2)\\approx1.11$~nm is the exciton Bohr radius for the $P$-excitons, and $\\delta^K_{a,b}$ is the Kronecker $\\delta$-symbol. \n\nThe analysis of the experimental data shows that the exciton quantum defects for large $n$ quickly converge to constant values for fixed angular momentum $l$. This observation is supported by Eqs.~\\eqref{defects}, which show that each contribution to the energy deviation of the levels within a multiplet from the simple hydrogen formula scales as $1/n^3$, in agreement with the quantum defect description, Eq.~\\eqref{qdef}. Furthermore, we found experimentally that with increasing $l$ the quantum defects drop continuously from finite positive values towards zero, again, in full agreement with Eqs.~\\eqref{defects}. The $S$-excitons demonstrate the largest quantum defect $\\delta_{n,l=0} \\approx 0.65$, for the $P$-excitons it is about $0.34$ and for the $D$- and $F$-excitons it is reduced to $0.18$ and $0.12$, respectively, in the large-$n$ limit. Theoretical estimates of the non-parabolicity contribution to the quantum defects, Eq.~\\eqref{non:par}, for $\\gamma_2=0.8$~\\cite{PhysRevLett.115.027402,PhysRevB.93.195203} yield $\\delta_{n,l=0}=0.87$, $\\delta_{n,l=1}=0.43$, $\\delta_{n,l=2} =0.28$ and $\\delta_{n,l=3} = 0.21$ in reasonably good agreement with experiment. It supports our conjecture that the dominant contribution to quantum defects arises from the $p^4$ nonparabolic term in the valence band dispersion.\n\n\\begin{figure}[h]\n\\includegraphics[width=\\columnwidth]{Fig4-multipletsplitting.pdf}\n\\caption{Energy spread of the exciton states within a multiplet with fixed principal quantum number versus $n$ in a double logarithmic representation. The line gives a fit to the data after a $n^{-3}$ scaling law, following the theoretical considerations in the text, see Eq.~\\eqref{defects}.} \\label{Fig4-multipletsplitting}\n\\end{figure}\n\nThe consequence of the quantum defect series for a particular principal quantum number is that the associated states are spread over a finite energy range with the low (high) angular momentum states on the low (high) energy side, as clearly seen in the spectra in Fig.~\\ref{Fig2-bfield-spectra-2ndderivative} and in Fig.~\\ref{Fig3-efield-spectra}. The widths of these multiplets were determined from the data in zero field and also in weak external fields, applied to optically activate excitons that are dark without field or to enhance their visibility so that their energies could be extrapolated to zero field with high accuracy. The widths of the multiplets are plotted in Fig.~\\ref{Fig4-multipletsplitting} as function of $n$ in double-logarithmic representation and show a strong drop with increasing principal quantum number from $3$ meV for $n=3$ to less than 0.1 meV for $n$ exceeding 10. The data follow the $1/n^3$ scaling law expected from the quantum defect description and corroborated by the calculations according to Eqs.~\\eqref{defects}, as shown by the red line fit.\n\nThe definition of the Rydberg exciton regime is to some extent arbitrary. We take it here as the regime in which the principal quantum number exceeds $n=5$, because the exciton extension given by twice the average exciton radius exceeds then 100~nm, which is roughly two orders of magnitude larger than the ground state wave function extension. Correspondingly the data in Fig.~\\ref{Fig4-multipletsplitting} and also in all subsequent scaling plots were fitted for $n \\geq 6$, but the plots of the fits were extended also towards lower $n$ in case corresponding data were available like for the multiplet splitting just discussed.\n\n\n\\subsection{Scaling in magnetic field}\\label{sec:magn}\n\nThe magnetic field behavior of the excitons in the Rydberg regime has been assessed in detail in Refs. \\cite{bfield,Aszmann:2016aa}. Entering this regime or $n \\gtrsim 6$, the density of states becomes so large that a precise assignment of the observed states is difficult, in particular, because here avoided crossings and therefore state mixings dominate. Hence statistical methods have been applied by analyzing the distribution of energy separations between nearest exciton levels at fixed transition energies. For $n > 6$ clear signatures of quantum chaos were found due to the dominance of anticrossings as can be seen also from the spectra in Fig.~\\ref{Fig2-bfield-spectra-2ndderivative} in the energy range above about $2.170$~eV. Here we focus on different features: (i) an estimate of the size of the excitons, and (ii) the magnetic field induced crossing of states with adjacent $n$.\n\n\\textit{Exciton size evaluation.} So far, the Rydberg exciton size has been determined somewhat ``indirectly'' from their principal quantum number $n$ using the formula for the average radius of an orbital in the hydrogen model. This average radius is given by~\\cite{ll3_eng}\n\\begin{equation}\n\\label{extension}\n\\langle r\\rangle_{nlm}  = \\frac{a_B}{2} \\left[ 3n^2 - l \\left( l+1 \\right) \\right] \\approx \\frac{3 a_B}{2} n^2  \n\\end{equation}\nwith the approximation being well suited for $n \\gtrsim 6$ for the $P$-excitons. The magnetic field introduces an independent length scale given by the magnetic length, $\\ell_c = \\sqrt{\\hbar c/ eB}$ characterizing the extension of the magnetic confinement potential, which competes with the Coulomb interaction: At low magnetic fields, the main $P$-exciton lines in Fig.~\\ref{Fig1-bfield-spectra} show a rather weak dependence on magnetic field, corresponding to a diamagnetic shift $\\propto B^2$, which changes with increasing field to a stronger shift that can be roughly approximated by a $B$-linear dependence. In order to assess this transition theoretically, it is sufficient to present the effective Hamiltonian of the exciton in the magnetic field $\\bm B\\parallel z$ in the form: \n\\begin{equation}\n\\label{HB:simple}\n\\mathcal H_B = \\mathcal H_0 + \\frac{\\hbar e B}{2\\mu c}l_z + \\frac{\\hbar^2e^2B^2}{8\\mu c^2} (x^2+y^2).\n\\end{equation}\nThe terms responsible for the quantum defect are not of importance here. We also disregard the electron and hole Zeeman spin splittings. For strong magnetic fields one can neglect in first approximation the Coulomb interaction and approximate the states by the electron-hole Landau levels. More precisely, even at strong magnetic fields the Coulomb interaction provides bound magnetoexciton states with the binding energy scaling as ${\\mathcal R}\\ln(a_B/\\ell_c)$~\\cite{ll3_eng}. With increasing field the changeover from an exciton behavior dominated by the Coulomb interaction at low fields to a behavior with dominant magnetic confinement at high fields, where Landau levels are formed, occurs. \n\nCorrespondingly, an exciton resonance in the optical spectrum transforms to a good approximation into a transition between electron and hole Landau levels. Roughly, $P$-excitons with principal quantum number $n$ transform into a transition between Landau levels with quantum number $n$~\\cite{Yafet1956137,PhysRev.188.1294}. The extension of these Landau levels in real space is given by\n\\begin{equation}\n\\label{lcn}\n\\ell_{c,n}  \\approx \\ell_{c} n^{1/2} = \\sqrt{ \\frac{\\hbar c n}{eB} } \n\\end{equation}\nfor not too small $n$. The transition takes place for \n\\[\n\\ell_{c,n} \\sim \\langle r \\rangle_{nlm},\n\\]\ni.e., where the Landau level extension is about equal to the Coulomb extension, which can be achieved by increasing the magnetic field to a particular crossover field strength $B_{c,n}$. From this crossover field one obtains $\\ell_{c,n} = 25.6~$nm$ \\sqrt{n} / \\sqrt{B_{c,n} [\\mathrm T]}$, where $B_{c,n}$ should be inserted in units of Tesla. In the experiment we can determine $B_{c,n}$ as the field strength at which the quadratic field dependence of the $P$-exciton energies changes into a linear one. Note that the crossover fields and the resulting Landau level extensions can be only approximately determined in that way, but they are nevertheless sufficient to test their scaling with $n$. From the considerations above we expect a scaling with $n^{-3}$ for $B_{c,n}$.\n\nFrom the spectra it is confirmed that the transition to Landau-level like behavior occurs at continuously decreasing magnetic fields with increasing principal quantum number. In the high field regime, where also quantum chaos was detected for $n > 6$ no single strong transition is observed for a particular $n$, but bunches of lines which cluster around the Landau level transitions appear. Technically, to determine the crossover field we have taken the center of the line multiplet around a Landau transition and have extrapolated its energy linearly to lower fields, so that we could determine the crossing point $B_{c,n}$ with the quadratic diamagnetic shift of the $P$-exciton line. The $B_{c,n}$ determined in that way are shown by the red circles in Fig.~\\ref{Fig5-lengthscales}, demonstrating the strong drop with increasing $n$. The dotted line gives a $n^{-3}$ fit to the data from which excellent agreement is seen.\n\nFrom these fields $\\ell_{c,n} ( B_{c,n} )$ can be calculated. The Landau level extensions determined in that way are shown in Fig.~\\ref{Fig5-lengthscales} by the black squares and compared to the average exciton radius $r_{n,l}$ according to the hydrogenic formula. For example, for $n=6$ the hydrogenic formula~\\eqref{extension} gives an average radius of 60~nm. The changeover field strength, on the other hand, is $1.3 \\pm 0.3$~T, from which we obtain $\\ell_{c,6}(B_{c,6}) \\approx 55\\pm 8$~nm which are surprisingly close values. For all other $n$ we find similarly good agreement between $\\ell_{c,n}$ and $\\left( r_{n,l} \\right)$, confirming the expected $n^2$ scaling of the wave function extension: $\\ell_{c,n} \\propto (n/B_{c,n})^{1/2} \\propto \\sqrt{n/n^{-3}}$. Note that the inaccuracy in determining $B_{c,n}$ translates through the square-root connection in moderate inaccuracies of the Landau level extension $\\ell_{c,n}$ that are indicated by the error bars. For high $n$ sufficient accuracy was achieved by recording the spectra in 5~mT steps which in combination with the steeper slope of the Landau level transitions facilitates the separation from the low-field behavior. While the observed agreement with the hydrogenic formula may be expected, it also validates the applied exciton description using, for example, a uniform dielectric screening over large length scales. It also reassures the huge extension of the highly excited Rydberg exciton, for which we estimate for $n=20$ an average radius of about 0.7~$\\mu$m.\n\n\\begin{figure}[h]\n\\includegraphics[width=\\linewidth]{Fig5-lengthscales.pdf}\n\\caption{Red circles: crossover field strength $B_{c,n}$ from diamagnetic to Landau level-like behavior as function of principal quantum number. The red dotted line is a fit to the data using a $n^{-3}$ dependence. Black squares: Landau level radius $\\ell_{c,n}$, Eq.~\\eqref{lcn}, estimated from $B_{c,n}$ as function of $n$. The black solid line gives the average radius of the orbital according to the hydrogenic formula, Eq.~\\eqref{extension}.} \\label{Fig5-lengthscales}\n\\end{figure}\n\n\\emph{Magnetic field-induced crossings.} For the field-induced resonances we focus on the first one between exciton states belonging to the multiplets with principal quantum numbers $n$ and $n+1$, as shown in detail in the bottom panel of Fig.~\\ref{Fig2-bfield-spectra-2ndderivative} for field strengths up to $2$ T.  At these resonances we observe systematic crossings, in contrast to the general trend of anticrossings in the spectra. From lower to higher energies we clearly see that the field strength, $B_r$, at which these crossings occur shifts strongly to lower values. This can be expected from (i) the reduced splitting between the exciton states at zero field, (ii) the enhanced field-induced splitting of each of them involving larger angular momenta, and (iii) the stronger diamagnetic shift of each multiplet for higher $n$.\n\nFrom the data we can determine the resonant field strengths $B_r$ versus the principal quantum number. This dependence is shown in Fig.~\\ref{Fig6-resonantmagneticfields} using a double-logarithmic representation. The resonant fields decrease from $B_r=2$~T for $n=6$ to $B_r=0.04$~T for $n=16$.\nFitting the observed data for the Rydberg exciton regime with a power law form reveals a dependence as $n^{-4}$, unlike for Rydberg atoms, where one finds a $n^{-6}$-dependence~\\cite{gallagher2005rydberg}.\n\n\\begin{figure}[h]\n\\includegraphics[width=\\linewidth]{Fig6-resonantmagneticfields.pdf}\n\\caption{Resonant magnetic field strength, $B_r$, at which the first crossing between states belonging to the exciton multiplets $n$ and $n+1$ occurs, as function of the principal quantum number $n$ in a double logarithmic presentation. Symbols give the experimental data, the line is a fit to these data following a $n^{-4}$ law, Eq.~\\eqref{Bc}.}\\label{Fig6-resonantmagneticfields}\n\\end{figure}\n\nThis scaling can be well understood by applying the simplified hydrogen-like model, which is to a good approximation justified because of the quite small impact of the quantum defect on the large-$n$ Rydberg states, as already demonstrated by the $n^{-3}$ scaling of the multiplet width in Fig.~\\ref{Fig4-multipletsplitting}. Using atom-like excitonic units, i.e., $2\\mathcal R = \\mu e^4/(\\varepsilon^2\\hbar^2)$ as the unit of energy, the Bohr radius $a_B$ as the unit of length, and $\\hbar c/(ea_B^2)$ as the unit of magnetic field, and \n\\begin{equation}\n\\label{r2}\n\\langle r^2\\rangle_{nlm} = \\frac{n^2}{2}[5n^2+1-3l(l+1)] \\propto n^4\n\\end{equation}\nfor sufficiently large $n$, we obtain for the level energies to a good approximation:\n\\begin{equation}\n\\label{Enl:B}\nE_{nl} \\approx -\\frac{1}{2n^2} + \\frac{m B}{2} + A_l n^4 B^2.\n\\end{equation}\nHere $A_l$ is a constant that depends on the angular momentum $l$ of the studied level. Neglecting the diamagnetic shift $\\propto B^2$, we obtain for the resonance field of the sublevels $n,l=n-1,m=n-1$ and $n+1, l=n, m=-n$, which are involved in the crossing (corresponding to the states with the maximal $m$ of the $n$th multiplet and with the minimal $m$  of the $n+1$st multiplet), in perturbation approach:\n\\begin{equation}\n\\label{Bc}\nB_r(n) \\propto \\frac{1}{n^4}.\n\\end{equation}\nIt is noteworthy that for the crossing of these states the diamagnetic contribution is vanishingly small, because $A_l n^4 B_r^2(n)\\propto 1/n^4 \\ll mB \\propto 1/n^3$ in Eq.~\\eqref{Enl:B}.  This is different from the atomic physics case where one, as a rule, considers states with magnetic quantum numbers, $m=0$ or $1$, as only these states can be observed in single photon absorption out of an $s$-state. Correspondingly, in the atomic case the paramagnetic term $\\propto mB/2$ in Eq.~\\eqref{Enl:B} is not important and the diamagnetic term $\\propto A_l n^4 B^2$ dominates~\\cite{gallagher2005rydberg}. For atoms one finds therefore a $n^{-6}$ dependence of the resonance field strength. \n\n\\subsection{Electric field}\n\nLet us turn now to the scaling of exciton properties in electric field. As outlined in Sec.~\\ref{sec:exper}, we recorded spectra for two different configurations, taken on a $[110]$ oriented crystal. For the exciting light propagating along the same $[110]$-direction the linear light polarization was chosen either along $[1\\bar10]$ (left panel of Fig.~\\ref{Fig3-efield-spectra}) or along $[001]$ (right panel of the same figure). In the first case the quadrupolar transitions are forbidden so that it is easy to follow the dispersion of the $P$-excitons, which is the first major point of this subsection. In the latter case, they are allowed, so that $S$- and $D$-excitons appear, allowing a more comprehensive insight into the different states within an exciton multiplet~\\cite{efield}. This is exploited here to determine the fields at which states from different multiplets come in resonance, representing the second major point of this part. The same configuration was also used to study the polarizability of the $S$-excitons which are well separated on the low energy flanks of the $P$-excitons. Finally we also address exciton ionization processes, for which we use again the first configuration (without quadrupolar transitions) to determine the ionization field strength as well as the linewidth of $P$-excitons.\n\n\\emph{Scaling of $S$- and $P$-exciton polarizability.} For the $P$-exciton dispersion, we have to distinguish between the low-$n$ and high-$n$ states. For low $n$ the effect of the quantum defect, namely the lifting of level degeneracy, is particularly relevant. In electric field, this leads to the observation of a quadratic Stark effect for the non-degenerate states, because in centrosymmetric crystals any non-degenerate excitonic state has no electric dipole moment without electric field application. The dipole moment is induced by the electric field which subsequently orients it, leading to the quadratic energy shift. This is in contrast to the linear Stark effect obtained by degenerate perturbation theory for a multiplet of levels each having the same energy. In this case, the degenerate states in the multiplet that become coupled by the electric field are linearly combined such that an electric dipole moment is established. This dipole moment only has to be oriented by the field leading to linear energy shifts with increasing field. This situation is to a good approximation relevant for the high $n$-range. Strictly speaking, the exciton differentiation between small and large values of the principal quantum number as well as between non-degenerate and degenerate states depends on the experimental resolution: For each $n$, independent of its value, there is a range of small fields in which a quadratic Stark shift occurs due to level splitting in the crystal. Roughly, this is the range in which the associated quantum defect exceeds the field induced energy shift. This range of quadratic Stark effect decreases, however, strongly with increasing $n$.\n\n\\begin{figure}[h]\n\\includegraphics[width=\\linewidth]{Fig7-polarizabilitySP.pdf}\n\\caption{Polarizability $\\alpha_{n,l}$ of the $S$- (open triangles) and $P$-excitons (closed squares) versus principal quantum number $n$. The line is a fit to the data following the expected $n^7$ scaling law, Eq.~\\eqref{pol:scaling}.}\\label{Fig7-polarizabilitySP}\n\\end{figure}\n\nThe quadratic Stark effect is described by\n\\begin{equation}\n\\label{quadratic}\n\\Delta E_{nl}^{(2)} = - \\alpha_{nl} F^2,\n\\end{equation}\nwhere $\\alpha_{nl}$ is the polarizability of the exciton state $(n,l)$. Although a quadratic Stark effect is already expected for the hydrogenic model with $\\alpha_{nl} \\propto n^6$ due to the field induced mixing of the multiplets with different principal quantum numbers~\\cite{ll3_eng}, the presence of the quantum defect leads to an important change of the polarizability scaling with $n$: Mixing becomes possible for states within a multiplet of particular $n$, which are split due to the $\\mathcal H_d$ part of the Hamiltonian, Eq.~\\eqref{Hd}. The scaling relation of the polarizability can be understood making use of second order perturbation theory to evaluate the quadratic Stark shift: Taking into account the dipole moment operator matrix elements between neighbouring states with the same $n$ that scale as $e\\langle r\\rangle_{nlm} \\propto n^2$, Eq.~\\eqref{extension}, and the scaling of the energy gap between these states $\\propto 1/n^3$, see Fig.~\\ref{Fig4-multipletsplitting} and Sec.~\\ref{sec:zero}, the polarizability scales as\n\\begin{equation}\n\\label{pol:scaling}\n\\alpha_{nl} \\propto n^7,\n\\end{equation}\nin accordance with the behavior typically observed for Rydberg atoms~\\cite{PhysRevA.62.042703,RevModPhys.82.2313}.\n\nHere, we observe for the $P$-excitons in Fig.~\\ref{Fig3-efield-spectra} a quadratic Stark effect for $n \\leq$ 13, in agreement with Eq.~\\eqref{quadratic}, while the splitting pattern for the higher-$n$ excitons approaches the linear Stark fan of hydrogen within the experimental resolution. In Fig.~\\ref{Fig3-efield-spectra}(a) the shift of the $P$-excitons to lower energies with increasing field can be well resolved and increases drastically with increasing principal quantum number, see the blue-colored feature of lowest energy in each $n$-manifold. From the data we can assess the polarizability $\\alpha_{n,P}$ ($l=1$) which is shown by the solid squares in Fig.~\\ref{Fig7-polarizabilitySP} as function of $n$ in a double-logarithmic plot. The polarizability increases from about 5 $\\mu$eV/V$^2$ for $n$ = 5 to about 2000 $\\mu$eV/V$^2$ for $n$ = 12. The solid lines give the fit to these data by a power law scaling with the seventh power of $n$, in accordance with the theoretical expectations, Eq.~\\eqref{pol:scaling}, from which a good description of the data is obtained. In Fig.~\\ref{Fig3-efield-spectra}(b), due to the chosen polarization configuration, also the $S$-excitons can be observed, which show also a quadratic Stark effect. Deriving also for them the polarizability, which is possible for $n \\leq 10$, one obtains the data shown by the open triangles in Fig.~\\ref{Fig7-polarizabilitySP}. Within the experimental error no difference in polarizability between $S$- and $P$-excitons can be resolved.\n\n\\emph{Scaling of electric field induced resonances.} Next we turn to the first resonances occurring  with increasing electric field between states of adjacent principal quantum numbers. Similar to the magnetic field case, we find also here a strong shift of the resonance voltage $U_r$ to lower values with increasing $n$. Looking at the resonances in closer detail, we find, however, that at these resonances levels do not cross but systematically avoid each other, as shown in the mid panel of Fig.~\\ref{Fig3-efield-spectra}. Extrapolating the dispersions, one can determine the resonance voltages which are shown in Fig.~\\ref{Fig8-resonancevoltages} in a double logarithmic representation versus $n$. $U_r$ decreases from 8~V for $n=5$ to about 40~mV for $n=13$. The data are in reasonably good accord with a power law scaling like $n^{-5}$ which is in line with the results on Rydberg atoms. Resonance voltage and field strength $F_r$ are connected by a simple proportionality relation for the chosen capacitor geometry by which the field is applied. However, care needs to be exercised in doing this conversion using just the nominal geometric and dielectric parameters, because surface charges, charged defects etc. can lead to depolarization effects in the crystal, so that there may be some discrepancy between the nominally calculated and the actually present field strength by a factor of $3 \\ldots 5$, as discussed in detail in Ref.~\\cite{efield}.\n\n\\begin{figure}[h]\n\\includegraphics[width=\\linewidth]{Fig8-resonancevoltages.pdf}\n\\caption{Principal quantum number $n$-dependence of the resonance voltage $U_r$ at which the first resonance of levels belonging to the multiplets with principal quantum numbers $n$ and $n+1$ occurs. At these resonances avoided crossings are observed. The red line shows a fit according to a $n^{-3}$ scaling law, Eq.~\\eqref{Fc}.}\\label{Fig8-resonancevoltages}\n\\end{figure}\n\nThis behavior can be understood again using the hydrogen-like model, because the quantum defect is not essential for the evaluation of the resonant fields. In the presence of an electric field the simplified Hamiltonian of the exciton reads:\n\\begin{equation}\n\\mathcal H_E = H_0 + eFz,\n\\end{equation}\nwhere we assumed that the field $\\bm F$ is applied along the $z$-direction. In first-order perturbation theory:\n\\begin{equation}\n\\label{Enl:F}\n\\frac{E_{nl}}{2\\mathcal R} = -\\frac{1}{2n^2} + \\frac{3}{2}\\tilde Fn(n_1-n_2),\n\\end{equation}\nwhere $\\tilde F = Fa_B/(2\\mathcal R)$ is the reduced electric field. Here $n_1$ and $n_2$ are the parabolic quantum numbers being non-negative integers with $n=n_1+n_2+|m|+1$. The resonance field strength, $F_r$, can be evaluated from Eq.~\\eqref{Enl:F} putting $n_1-n_2 \\approx \\pm n$ for $n\\gg 1$ and for the states with highest and lowest energy of the manifold with the result~\\cite{gallagher2005rydberg}:\n\\begin{equation}\n\\label{Fc}\nF_{r} \\propto \\frac{1}{3n^5}.\n\\end{equation}\nNote that the power-law dependence is different from the magnetic field case. This is because the electric dipole moment increases as $n^2$~\\cite{ll3_eng}, whereas the magnetic field induced splitting is determined by the paramagnetic contribution, which scales linearly with $n$ for the lowest and highest energy states in a multiplet.\n\nWe have also determined the energy splittings at these anticrossings as function of $n$ which are shown in Fig.~\\ref{Fig9-anticrossingenergies}. The data show a strong drop with $n$, even though one has to emphasize the splitting magnitude is only slightly larger than the linewidths of the involved transitions, resulting in considerable error bars, despite of which the drop from about 80 $\\mu$eV splitting for $n$ = 5 to about 10 $\\mu$eV for $n$ = 10 can be determined. The data can be reasonably well described by a $n^{-4}$ fit following also from perturbation theory. In order to estimate the anticrossing energy we follow Refs.~\\cite{gallagher2005rydberg,popov} and evaluate the matrix elements of the ``quantum defect'' Hamiltonian~\\eqref{Hd} on the eigenfunctions of the hydrogen atom in the parabolic coordinates $\\Psi_{nn_1n_2m}(\\bm r)$, where $n_1$, $n_2$, and $m$ are the parabolic quantum numbers. The avoided crossing energy reads\n\\begin{equation}\n\\label{anti:stark}\n\\delta E_{anti} = 2\\left|\\int d\\bm r \\Psi_{n'n_1'n_2'm}^*(\\bm r) \\mathcal H_d \\Psi_{nn_1n_2m}(\\bm r)  \\right|.\n\\end{equation}\nFor the hydrogenic states with adjacent principal quantum numbers $n'=n\\pm 1$ we have~\\cite{gallagher2005rydberg}\n\\[\n\\int d\\bm r \\Psi_{n'lm}^*(\\bm r) \\mathcal H_d \\Psi_{nlm}(\\bm r) \\sim \\frac{\\mathcal R\\delta_{nl}}{n^3}.\n\\]\nFinally, taking into account that \n\\[\n\\sum_l \\int d\\bm r \\Psi_{n'n_1'n_2'm}^*(\\bm r)\\Psi_{nlm}(\\bm r) \\propto \\frac{1}{n-m},\n\\]\nwe obtain the scaling law\n\\begin{equation}\n\\label{anti:stark:sc}\n\\delta E_{anti} \\sim \\frac{\\mathcal R \\delta_{nl}}{n^3(n-m)} \\propto 1/n^4.\n\\end{equation}\nUsing for a rough estimate $\\delta_{nl}=0.5$ (Sec.~\\ref{sec:zero}) we obtain for the $n=5$ and $n=6$ multiplets an anticrossing energy of about $70$~$\\mu$eV in reasonable agreement with the data in Fig.~\\ref{Fig9-anticrossingenergies}.\n\n\\begin{figure}[h]\n\\includegraphics[width=\\linewidth]{Fig9-anticrossingenergies.pdf}\n\\caption{Black squares: Dependence of the energy splittings due to anticrossings at the first resonances of states from the exciton multiplets $n$ and $n+1$, induced by the electric field, on the principal quantum number. The red line gives a fit to the data according to a $n^{-4}$ dependence, Eq.~\\eqref{anti:stark:sc}.}\n\\label{Fig9-anticrossingenergies}\n\\end{figure} \n\n\\emph{Crossing vs. anticrossing}. To understand the origin of crossing or anticrossing at the first resonance of states arising from the multiplets with principal quantum number $n$ and $n+1$ in magnetic and electric field, respectively, we have to consider the problem in more detail: Both electric and magnetic field preserve the axial symmetry of the ``hydrogenic'' problem, therefore the $z$-component of the angular momentum or ``magnetic'' quantum number $m$ is a good quantum number. In magnetic field the first crossing occurs between the states with $m=n-1$ (the maximal $m$ in the manifold with $n$) and $m=-n$ (the minimal $m$ in the manifold with $n+1$). The difference of the angular momentum components of these states is\n\\begin{equation}\n\\label{Delta:lz:B}\n\\Delta l_z = 2n+1.\n\\end{equation}\nThis is an odd number $1$, $3$, \\ldots, so that the parity of excitonic states which cross in magnetic field is different. In the centrosymmetric point group $O_h$ the states are characterized by a definite parity as well and, therefore, states with odd $\\Delta l_z$ do not mix. Hence, one may expect that the first crossing in magnetic field is indeed an \\emph{allowed} crossing. In electric field the situation is different. One can readily check by perturbative calculations that the first resonance occurs for states with $m=0$, e.g., $|n=2S\\rangle + |n=2P_z\\rangle$ and $|3S\\rangle - |3P_z\\rangle$. These states have the same symmetry and, hence, the crossing of these states is \\emph{avoided}.\n\nHowever, one has to be careful when using this line of argumentation to discuss the crossings/anticrossings of \\emph{observed} states in the transmission spectra. In fact, the consideration above neglects the symmetry of the two-particle Bloch function of the exciton~\\cite{PhysRevLett.115.027402}. All observed excitonic states (in a given geometry and polarization) interact with light and, hence, have the \\emph{same} symmetry, $\\Gamma_4^-$. For example, the states observed in the geometry where light with $[1\\bar10] \\parallel y$ polarization propagates along the $[110] \\parallel z$ direction, couple to the $y$-component of the electric field and therefore transform as this $y$-coordinate. Hence, at least the light-matter interaction, i.e., the polariton effect~\\cite{excitons:RS,PSSB:PSSB2221730104,PhysRevB.94.045205}, can convert allowed crossings into avoided ones. However, the polariton effect is relatively weak in cuprous oxide crystals where the direct interband transitions are forbidden at the $\\Gamma$-point of the Brillouin zone, leading to a small oscillator strength of allowed exciton transitions~\\cite{NIKITINE1961292,PSSB:PSSB2221730104}. The quantitative measure for the strength of the polariton effect is the longitudinal-transverse splitting, $\\hbar\\omega_{nlm,LT}$, of the exciton state with quantum numbers $(n,l,m)$. Provided that the longitudinal-transverse splitting is smaller than the non-radiative damping of the excitonic states, \n\\[\n\\hbar\\omega_{nlm,LT} < \\hbar \\Gamma_{nlm},\n\\]\nthe light-matter interaction is in the weak coupling regime so that polariton modes are not observable, and also the anticrossings are hidden in the optical spectra.\n\n\\emph{Ionization in electric field.} A particularly appealing feature of Rydberg excitons in comparison to Rydberg atoms is the fact that the true high electric field regime, in which the interaction strength with the field exceeds the Coulomb interaction, can be reached easily in the laboratory, while for atoms it is much harder to enter this limit. This is mostly due to the different Rydberg energy in cuprous oxide, $\\sim90$~meV, which is more than two orders smaller than the hydrogen atom Rydberg energy. From Fig.~\\ref{Fig3-efield-spectra} one sees that for elevated electric field strengths the excitons dissociate: The absorption lines fade away and finally disappear with increasing field. The fields required for dissociation are the smaller, the higher the principal quantum number of the involved exciton state is. A series of spectra recorded for the $n=11$ multiplet is shown in Fig.~\\ref{Fig10-highresolutionspectrainefieldn=11}, from which the drop of the $P$-exciton resonance with increasing applied voltage can be quantitatively assessed. Simultaneously with the line drop the width of the resonance increases.\n\nOn the high energy side of the $P$-exciton another peak can be seen, which can be associated with the $F$-excitons that can hardly be identified without applied voltage, but become more and more pronounced with increasing voltage. At higher energies even further weak features appear. This increase of visibility is partly related to the transfer of oscillator strength from the $P$ to the $F$ and other higher angular momenta excitons. It is somewhat surprising that the $F$-exciton absorption is still growing even when the $P$-exciton absorption has basically completely vanished. A potential explanation is the action of the centrifugal barrier in the kinetic energy which varies with the angular momentum of the involved state. Due to its action the particles are kept away from the field axis $z$, making these states more robust with respect to field application with increasing $l$. However, the complex phenomenology associated with these excitons has not yet fully been developed so that we will not discuss it here in further detail. \n\n\\begin{figure}[h]\n\\includegraphics[width=\\linewidth]{Fig10-highresolutionspectrainefieldn=11.pdf}\n\\caption{High resolution absorption spectra of the $n=11$ exciton multiplet recorded with a frequency stabilized laser for different applied voltages.}\\label{Fig10-highresolutionspectrainefieldn=11}\n\\end{figure}\n\nTo assess the $P$-exciton behavior more quantitatively, we have to consider the impact of the electric field in more detail. The associated potential leads to a tilting of the Coulomb potential between electron and hole along the field direction. Due to this tilting eventually tunneling of carriers through the potential can occur for sufficiently high applied voltages. Ultimately in the high field regime the exciton state is moved into the state continuum so that it can no longer be observed. This behavior determines both the absorption strength (given by the area below a resonance) and its linewidth (given by the lifetime of the state): At low fields where tunneling is still not relevant, both quantities should stay constant, while in the tunneling regime we expect an exponential dependence for both the spectrum area and linewidth. \n\nFor determining the ionization field strength we have calculated the area below each resonance. For that purpose we have fitted each resonance with the Toyozawa formula~\\cite{toyozawa,ueno,jolk} \n\\begin{equation}\n\\label{toyozawa}\n\\alpha_n(E)=C_n\\frac{\\frac{\\Gamma_n}{2}+2q_n(E-E_n)}{\\left(\\frac{\\Gamma_n}{2}\\right)^2+(E-E_n)^2}\n\\end{equation}\nfrom which we obtain also the linewidth. Here $E_n$ and $\\Gamma_n$ are the energy and the damping of the state $n$. $q_n$ describes the asymmetry of the observed resonance and $C_n$ gives its amplitude. Here we have restricted to the voltage regime in which the $P$-exciton line represents a well isolated line that is not influenced by adjacent absorption features. This is the regime in which the emerging $F$-excitons are still much weaker in comparison. Once higher angular momentum excitons become prominent, the spectra on the high energy flank of each $P$-exciton show pronounced modulations which we attribute to interactions with the electron-hole continuum that are not fully understood so far. This modulation also aggravates a lineshape analysis so that we refrain from a line analysis in this regime. In Fig.~\\ref{Fig10-highresolutionspectrainefieldn=11} this regime corresponds to voltages higher than 0.25~V and is decreasing for higher $n$. Then the area below each absorption peak indeed shows to a good approximation an exponential drop with applied voltage as soon as effects of the applied voltage set in, up to the point where the line disappears. \n\nFrom this dependence we have determined the effective ionization voltage taken as the voltage at which the area below the resonance has dropped to $1/e$. \nThe drop has in fact two origins: (1) the redistribution of oscillator strength to other excitons due to state mixing; (2) the actual dissociation of the excitons. As all states are affected similarly by these two factors, we still use the voltage determined in the described way as characteristic for ionization. This voltage is plotted in Fig.~\\ref{Fig11-ionizationvoltage} as function of the principal quantum number. The data can be well described by a dependence proportional to $n^{-4}$. This is the dependence that is also expected by setting Eq.~\\eqref{Enl:F} for the exciton energy $E_{n,l}$ in electric field to zero. \n\n\\begin{figure}[h]\n\\includegraphics[width=\\linewidth]{Fig11-ionizationvoltage.pdf}\n\\caption{Dependence of the dissociation voltage, determined by taking the $1/e$ value of the area below a resonance, on the principal quantum number. The solid line shows a fit to the data by a $n^{-4}$ dependence.}\\label{Fig11-ionizationvoltage}\n\\end{figure}\n\n\\emph{Exciton linewidth.} Finally, we turn to the dependence of the linewidth on the electric field strength, for which  one would expect -- as described above -- a transition for increasing voltage from being constant for negligible tunneling to strongly increasing as soon as tunneling can occur until ultimately the potential barrier is lowered to an extent that the excitons become unbound, which is given by the field strength at which the energy in Eq.~\\eqref{Enl:F} is zero. The experimental data for the excitons from $n=10$ up to $n=16$ are shown in Fig.~\\ref{Fig12-linewidthsvsvoltage}. At zero electric field we observe the well known decrease of the linewidth with increasing $n$ that has been described already in Ref.~\\cite{Kazimierczuk:2014yq}.  For sufficiently low excitation powers such that effects like power broadening can be disregarded, the linewidth dependence on the principal quantum number can be described by a $n^{-3}$ law due to the corresponding scaling of the transition rates for photon and phonon emission. \n\nWhen applying the electric field, we find that the low lying excitons in the series indeed follow roughly the expected dependence (dashed line in Fig.~\\ref{Fig12-linewidthsvsvoltage}) which according to the theoretical calculations should be given by~\\cite{merkulov_ion,excitons:RS}\n\\begin{equation}\n\\label{Gamma}\n\\Gamma \\propto \\exp{\\left(-\\frac{2}{3n^3F}\\right)},\n\\end{equation}\nsee Appendix~\\ref{app:ion} for details. Somewhat surprisingly, the linewidth shows a different behavior for the high lying excitons in the shown set of states. Their linewidth stays constant within the experimental accuracy over the range of fields where they can be observed until they disappear.\n\n\\begin{figure}[h]\n\\includegraphics[width=\\linewidth]{Fig12-linewidthsvsvoltage.pdf}\n\\caption{Dependence of the linewidth of the $P$-exciton resonances on the applied voltage for different principal quantum numbers from $n=10$ up to $n=16$. The dashed lines show a fit to the data of the excitons $n=10$ and $n=11$ according to Eq.~\\eqref{Gamma}. For clarity in the left (right) panel the even (odd) $n$ states are shown.}\\label{Fig12-linewidthsvsvoltage}\n\\end{figure}\n\nThis behavior can be understood as follows. The exponent becomes significant at the critical field\n\\begin{equation}\n\\label{Fg}\nF_\\Gamma \\sim \\frac{1}{n^3}.\n\\end{equation}\nSince this field is parametrically larger than the ionization field $F_i$ for the states with large $n$, the increase in the linewidth is not seen, because of faster ionization of the state.\n\n\n\\begin{table*}[t]\n\\caption{\\label{tab:comparison}Comparison of scaling laws with principal quantum number $n$ for Rydberg atoms and Rydberg excitons.}\n\\begin{ruledtabular}\n\\begin{tabular}{ccc}\n & Rydberg atoms & Rydberg excitons \\\\ \\hline\n\\emph{Zero field} & & \\\\\nMultiplet splitting due to quantum defect & $\\propto n^{-3}$ (except of hydrogen) & $\\propto n^{-3}$ \\\\ \\hline \n\\emph{Electric field} & & \\\\ \nPolarizability & $\\propto n^{-7}$ ($\\propto n^{-6}$ for hydrogen) & $\\propto n^{-7}$ \\\\\nResonance field of states from multiplets $n$ and $n+1$&$\\propto n^{-5}$&$\\propto n^{-5}$\\\\ \nAnticrossing energy at first resonance & $\\propto n^{-4}$ & $\\propto n^{-4}$ \\\\\nIonization voltage & $\\propto n^{-4}$ & $\\propto n^{-4}$ \\\\\n\\hline\n\\emph{Magnetic field} & & \\\\ \nCrossover field to magnetoexciton & $\\propto n^{-3}$  & $\\propto n^{-3}$ \\\\\nResonance field of states from multiplets $n$ and $n+1$&$\\propto n^{-6}$&$\\propto n^{-4}$\\\\ \n\\end{tabular}\n\\end{ruledtabular}\n\\end{table*}\n\n5 Landau level transitions\n\\section{Landau level transitions}\\label{app:ll}\n\nFigure~\\ref{Fig-Appendix-LandauLevels} zooms into the region of absorption spectra indicated by the black box in Fig.~\\ref{Fig1-bfield-spectra}, from which the resonances that can be in lowest approximation assigned to transitions between electron and hole Landau levels with high quantum number can be resolved in more detail. This assignment is only approximate as for the corresponding energy range the system shows quantum chaotic behavior with multiple anticrossings, so that no strict $\\bm B$-linear behavior of spectral line energies with some Coulomb related modification can be identified. Still the transitions bunch around these pure Landau level transitions so that a corresponding identification becomes possible.\n\n\\begin{figure*}[ht]\n\\includegraphics[width=\\textwidth]{Fig13-Appendix-LandauLevels.png}\n\\caption{Close-up of the black boxed region of the magnetic field spectra in Fig.~\\ref{Fig1-bfield-spectra}, which allows one to resolve higher Landau level transitions. Zooming in even further allows detection of transitions associated with Landau level quantum numbers of more than 70.}\\label{Fig-Appendix-LandauLevels}\n\\end{figure*}\n\nFrom the representation in Fig.\\ref{Fig-Appendix-LandauLevels} we can identify transitions up to Landau level quantum number 57. Zooming in further reveals transitions up to quantum number $n$=79. Transitions associated with a particular Landau level quantum number $n$ arise with increasing magnetic field from the $B=0$ transitions associated with $P$-excitons with the same principal quantum number $n$. Experiments have shown that under the chosen experimental conditions it is very hard to observe $P$-excitons with principal quantum number higher than \n25 at $T$=1.2~K without magnetic field, not only because the energy spacing is small well below 1~meV, but also because of the decreasing oscillator strength with increasing $n$. Field application increases the splitting between the levels and enhances the oscillator strength through squeezing the exciton wave function. However, at $B$ = 0.5~T the extension of the $n$=73 Landau level orbit is still about 300~nm.\n\n",
        "label": "Fig-Appendix-LandauLevels",
        "Descriptive_question1": "What is the highest Landau level quantum number that can be resolved in figure_13?",
        "Descriptive_question2": "At what magnetic field strength does the extension of the n=73 Landau level orbit measure approximately 300 nm in figure_13?",
        "Reasoning_question1": "Why are transitions associated with Landau level quantum numbers up to 79 observable in figure_13 only after applying a magnetic field, despite the difficulty in observing high principal quantum number P-excitons at zero field?",
        "Reasoning_question2": "How does the application of a magnetic field influence the visibility of higher Landau level transitions in figure_13 compared to zero-field conditions, and what physical mechanisms are responsible for this effect?",
        "Descriptive_answer1": "79",
        "Descriptive_answer2": "0.5 T",
        "Reasoning_answer1": "Transitions associated with Landau level quantum numbers up to 79 become observable only upon applying a magnetic field because, at zero field, high principal quantum number P-excitons (above n=25) are difficult to detect due to their small energy spacing (less than 1 meV) and weak oscillator strength. The magnetic field increases the splitting between levels and squeezes the exciton wave function, enhancing the oscillator strength, thereby making these otherwise weak and closely spaced transitions resolvable.",
        "Reasoning_answer2": "The magnetic field enhances the visibility of higher Landau level transitions by increasing the splitting between the exciton energy levels and squeezing (compressing) the exciton wave functions. This squeezing enhances the oscillator strength of the excitons, making transitions more pronounced in the absorption spectra. Consequently, transitions that are hard to detect at zero field due to weak oscillator strength and small energy spacing become observable and distinguishable in magnetic field, allowing detection of Landau levels up to very high quantum numbers, as seen in figure_13."
    },
    {
        "paper_id": "2302.00973.json",
        "image_id": "figure_2",
        "image_path": [
            "/home/yz979/scratch/chengye/scimolmo/test-images-tables/dataset_imagestables/2302.00973/images/handdrawing.png"
        ],
        "caption": "Comparison of the hand drawings of Parkinson's disease (PD) patient and healthy control (HC) subject under a reference drawing (RD).",
        "classify": "Chart",
        "section_info": "2 Material\n\\section{Material} \\label{sec:dataset}\n\nIn this research, a data set is considered. The data set, here and later referred to as DraWritePD, was acquired from $49$ participants, with a mean age of $74.1$ years and a similar gender distribution. Within the group of patients with PD, the age deviation was approximately $3.35$ years, while within the group of subjects with HC, the age deviation was $4.55$ years, making both groups very similar. \n\nData acquisition was performed with an iPad Pro $9.7$ inch ($2016$) equipped with an Apple Pencil. As shown in Fig.\\ref{fig:drawing curves}, participants were asked to mimic the reference pattern to draw. During this process, the iPad Pro scanned the Apple Pencil signal at $240$ points per second. As shown in Table \\ref{tab:dataset description}, for each scan, the device captures six time sequence parameters: azimuth ($a$); altitude ($l$); pressure ($p$); timestamp ($t$); x-Axis ($x$); y-Axis ($y$).\n\n\\renewcommand{\\arraystretch}{1.5}\n\\begin{table}[ht] \n\t\\centering\n\t\\scriptsize \n\t\\caption{Description of the DraWritePD dataset}\n\t\\setlength{\\tabcolsep}{1.5mm}{\n\t\\begin{tabular}{| c || c| c | c| c | c | c|}\n\t\t\\hline \n\t\tParameter & azimuth &\taltitude & pressure & timestamp  & x-Axis & y-Axis\\\\\n\t\t\\hline\n\t\tNotation & $a$ & $l$ & $p$ & $t$ & $x$ & $y$\\\\\n\t\t\\hline\n\t\tQuantitation & rad & rad & psi & sec & mm & mm\\\\\n\t\t\\hline\n\t\\end{tabular}}\n\t\\label{tab:dataset description}\n\\end{table}\n\nOriginally, the dLAST consisted of $3$ tests: \\textit{$\\Pi$}\\textit{$\\Lambda$}, \\textit{$\\Pi$} and \\textit{sin wave}, each with $3$ exercises: \\textit{continue}, \\textit{copy} and \\textit{trace} applied to each test. The general task was to complete a set of tests consisting of $9$ different drawing exercises. To optimise the testing procedure within the frameworks of the present contribution, only the \\textit{$\\Pi$} test was considered. Specifically, the graphical plots of \\textit{$\\Pi$} test are demonstrated in Fig. \\ref{fig:drawing curves}.\n\nThe data acquisition process was carried out under strict privacy law guidance. The Research Ethics Committee approved the study of the University of Tartu (No.$1275T-9$).\n\n\n",
        "label": "fig:drawing curves",
        "Descriptive_question1": "What does figure_2 compare in terms of drawing subjects?",
        "Descriptive_question2": "What type of test is demonstrated in figure_2?",
        "Reasoning_question1": "How might the differences in drawing patterns in figure_2 between Parkinson's disease patients and healthy control subjects reflect the motor symptoms of Parkinson's disease?",
        "Reasoning_question2": "What could be inferred about the precision of hand movements in Parkinson's disease patients compared to healthy controls based on the drawing comparisons shown in figure_2?",
        "Descriptive_answer1": "PD patient, HC subject",
        "Descriptive_answer2": "Pi test",
        "Reasoning_answer1": "The differences in drawing patterns between Parkinson's disease (PD) patients and healthy controls (HC) in figure_2 likely reflect the motor symptoms of Parkinson's disease, such as tremors, rigidity, and bradykinesia. These symptoms can cause PD patients to draw less smoothly and with more irregularities compared to healthy individuals. The visible discrepancies in the drawings suggest impairments in fine motor control and coordination that are characteristic of PD.",
        "Reasoning_answer2": "The drawing comparison indicates that Parkinson's disease patients have decreased precision in hand movements compared to healthy controls. The less smooth and possibly deviated lines drawn by PD patients reflect difficulties in maintaining steady and accurate hand motions, which is consistent with the motor impairments caused by PD. This suggests that PD patients have impaired fine motor skills affecting their ability to replicate reference drawings accurately."
    },
    {
        "paper_id": "2302.00973.json",
        "image_id": "figure_4",
        "image_path": [
            "/home/yz979/scratch/chengye/scimolmo/test-images-tables/dataset_imagestables/2302.00973/images/sliding-window.png",
            "/home/yz979/scratch/chengye/scimolmo/test-images-tables/dataset_imagestables/2302.00973/images/characteristic.png"
        ],
        "caption": "The effect of hyper-parameters in data pre-processing on the model, where (a) is the effect of transforming different raw parameter signals into velocity features in Feature engineering and selection on the model, and (b) is the effect of \\textit{window size} in Segmentation on the model.",
        "classify": "Chart",
        "section_info": "3 Methodology\n\\section{Methodology} \\label{sec:methodology}\n\nIn this section, we illustrate the proposed LSTM-CNN model for the diagnosis of PD. The framework starts with preprocessing methods for standardising the data.  Regarding the model architecture, we propose a lightweight hybrid model that is composed of an LSTM block cascaded with a CNN-based classifier. We will demonstrate in detail the performance and efficiency of the proposed LSTM-CNN model.\n\n\\begin{figure}[!t]\n\t\\centering  \n\t\\includegraphics[width=0.5\\textwidth ]{Figures/seg.png}\n\t\\caption{The scheme of data segmentation with temporal overlap region used in the framework. The \\emph{window size} and \\emph{stride size} control the length and overlap of the patch.}  \n\t\\label{fig:segmentation}\n\\end{figure}\n\n\n\\subsection{Data pre-processing}\n\nIn many practical applications, the raw data may not be perfectly collected --- for example, some data may be missing, or inevitably contain some abnormal features that are not appropriate for direct use in training a regression or a classification model. In this situation, it is always necessary to use some pre-processing methods to standardise the data or improve their quality. Considering the characteristics of the DraWritePD data set, we employ the following preprocessing methods to ensure that our data are suitable for training a diagnostic model.\n\n\\textbf{Normalisation} \n As explained in Section \\ref{sec:dataset}, the DraWritePD data set includes parameters with different ranges. It is necessary to rescale or normalise the data. The Min-Max normalisation technique is used to linearly convert each individual parameter signal in the range from $0$ to $1$. \n \n\\textbf{Feature engineering and selection}\nAfter the normalising procedure, the feature engineering process proposed in \\cite{DROTAR2016} suggests considering the kinematic parameters of the movements of the tip of the pen. Kinematic parameters of fine motor movements (observed during writing and drawing activities) would reflect tremor, freezing, and other symptoms caused by progressing PD \\cite{rosenblum2013handwriting}, and we have demonstrated this conclusion through experiments (see Fig. \\ref{fig:parameter} (a)). Within the framework of the present studies, main attention is paid to the projections of the velocity of x- and y-coordinate parameter signals on the coordinate axis given by Eq.(1), while keeping the other parameters constant.\n\\begin{equation}\nv_x^t= (x_t-x_{t-1})/\\delta_t, \\quad  v_y^t= (y_t-y_{t-1})/\\delta_t,\n\\end{equation}\nwhere ($x_{t-1}$,$y_{t-1}$) and ($x_t$,$y_t$) are the x- and y-coordinate position information of two adjacent time points, and $\\delta_t$ is the time interval between two contiguous sampling data points. The use of velocity also has the advantage of converting the non-stationary $x,y$ coordinate-based features into stationary ones, leading to a more tractable classification under the proposed model. \n\n\\begin{figure}[t]\n    \\centering\n    \\subfigure[]{\n        \\includegraphics[width=0.2\\textwidth]{Figures/characteristic.png}\n    }\n    \\subfigure[]{\n        \\includegraphics[width=0.2\\textwidth]{Figures/sliding-window.png}\n    }\n    \\caption{The effect of hyper-parameters in data pre-processing on the model, where (a) is the effect of transforming different raw parameter signals into velocity features in Feature engineering and selection on the model, and (b) is the effect of \\textit{window size} in Segmentation on the model.} \n\t\\label{fig:parameter}\n\\end{figure} \n\n\\textbf{Segmentation} \nAfterwards, data segmentation is adopted in the proposed framework to generate suitable data samples for model training. The approach of using a sliding window to randomly select a local patch is frequently used in time-series classification and has been demonstrated to be useful in improving model performance. Fig. \\ref{fig:segmentation} shows an example based on the DraWritePD data set. The multichannel time-series data are cropped into small patches with a slight overlap to preserve the temporal information. The parameters \\textit{window size} ($w$) and \\textit{stride size} ($s$) control the length and overlap of the resampling patches. The choice of \\textit{window size} and \\textit{stride size} depends on the concrete applications. Experiments illustrated by Fig. \\ref{fig:parameter} (b) demonstrate that the \\textit{$w$=128} provides balance between the performance and the efficiency under the proposed LSTM-CNN architecture. It is also worth noting that such segmentation generates more data samples for training, which is crucial for limited or small data sampling cases, including the proposed method based on the LSTM-CNN deep learning architecture. \n\n\n\\subsection{Model Structure}\n\nThe proposed model structure is a hybrid of the LSTM and CNN models. Such a combination helps to explore the advantages of both the LSTM and CNN models. In this LSTM-CNN model, convolution operators are also reduced to $1$D instances to reduce the computational cost. For clarity, we introduce the model in Fig. \\ref{fig:lstmcnn} according to the characteristics of the LSTM and CNN blocks.\n\n\\begin{figure*}[!t]\n\t\\centering  \n\t\\includegraphics[width=\\textwidth ]{Figures/model2.png}\n\t\\caption{The network structure of the proposed LSTM-CNN model.} \n\t\\label{fig:lstmcnn}\n\\end{figure*}\n\n\\textbf{LSTM Block} \nThe LSTM unit is a subtype of the recurrent neural network (RNN). It was originally proposed as an efficient and scalable building block for analysing complex sequential data or time-series data. The LSTM unit contains a group of special memory cells and is capable of extracting temporal features of data based on the memory of historical information, giving a great advantage over CNN in the extraction of sequence data. For each memory cell, the input data is first sent to different gates, including the input gate, the forget gate, and the output gate, to control the behaviour of each memory cell, and then the output is sent as input at the corresponding moment of the next memory cell. For more details on the structure and properties of LSTM, we refer to \\cite{hochreiter1997long}. In our LSTM-CNN model, the LSTM block consists of a series of cascaded LSTM units. Experiments show that, in order to improve efficiency while maintaining model performance, we can choose only one LSTM unit to construct the LSTM block. Furthermore, the output of each LSTM unit is concatenated with the original input data and sent to the subsequent CNN block for robust classification, in which the LSTM output is dimension-expanded (represented as batch size, $1$, window size, and feature dimension) to accommodate the shape of the input convolutional layer. \n\n\\textbf{CNN Block} \nThe CNN unit is suitable for classification or recognition tasks due to its ability to learn discriminate representations. We explore the performance of convolutional networks and follow the architecture and suggestions in \\cite{krizhevsky2012imagenet}. As shown in Fig. \\ref{fig:lstmcnn}, the proposed model contains two CNN units, where each consists of a $1$D temporal convolutional layer, a rectified linear unit (ReLU) layer and a 1D max pool layer. Specifically, the $1$D convolutional layer is the most important component due to its unique feature extraction ability, where in the first convolution unit, the $16$ convolution kernels are used for feature extraction, and in the second unit, the $32$ convolution kernels are used for deeper feature extraction operations on the output of the feature by the upper layer. The size of each convolution kernel is $3\\times1$, and the sliding step size of the convolution window is fixed to $2$ throughout all experiments. Next, a ReLU layer is used to activate its output. A maximum pool layer is used after the ReLU layer to perform the downsampling operation and reduce parameters while maintaining dominant features, with a sampling kernel size of $2\\times1$ and a stride size of $2$. Notice that all operators are reduced to cases of $1$ -D for efficiency.\n\n\\textbf{FC Block}\nThe fully connected (FC) block --- consisting of a fully connected layer, a ReLU activation layer and a dropout layer --- is used in the proposed model. During the training phase, the dropout layer temporarily removes nodes from the network with a probability of $0.5$. For stochastic gradient descent, since it is randomly dropped, each mini-batch is training a different network to prevent model overfitting and improve model performance. Furthermore, a fully connected layer is deployed after the dropout layer to convert the previous output, the value of which represents the probabilities belonging to each class.\n\n\n3.1 Data pre-processing\n\\subsection{Data pre-processing}\n\nIn many practical applications, the raw data may not be perfectly collected --- for example, some data may be missing, or inevitably contain some abnormal features that are not appropriate for direct use in training a regression or a classification model. In this situation, it is always necessary to use some pre-processing methods to standardise the data or improve their quality. Considering the characteristics of the DraWritePD data set, we employ the following preprocessing methods to ensure that our data are suitable for training a diagnostic model.\n\n\\textbf{Normalisation} \n As explained in Section \\ref{sec:dataset}, the DraWritePD data set includes parameters with different ranges. It is necessary to rescale or normalise the data. The Min-Max normalisation technique is used to linearly convert each individual parameter signal in the range from $0$ to $1$. \n \n\\textbf{Feature engineering and selection}\nAfter the normalising procedure, the feature engineering process proposed in \\cite{DROTAR2016} suggests considering the kinematic parameters of the movements of the tip of the pen. Kinematic parameters of fine motor movements (observed during writing and drawing activities) would reflect tremor, freezing, and other symptoms caused by progressing PD \\cite{rosenblum2013handwriting}, and we have demonstrated this conclusion through experiments (see Fig. \\ref{fig:parameter} (a)). Within the framework of the present studies, main attention is paid to the projections of the velocity of x- and y-coordinate parameter signals on the coordinate axis given by Eq.(1), while keeping the other parameters constant.\n\\begin{equation}\nv_x^t= (x_t-x_{t-1})/\\delta_t, \\quad  v_y^t= (y_t-y_{t-1})/\\delta_t,\n\\end{equation}\nwhere ($x_{t-1}$,$y_{t-1}$) and ($x_t$,$y_t$) are the x- and y-coordinate position information of two adjacent time points, and $\\delta_t$ is the time interval between two contiguous sampling data points. The use of velocity also has the advantage of converting the non-stationary $x,y$ coordinate-based features into stationary ones, leading to a more tractable classification under the proposed model. \n\n\\begin{figure}[t]\n    \\centering\n    \\subfigure[]{\n        \\includegraphics[width=0.2\\textwidth]{Figures/characteristic.png}\n    }\n    \\subfigure[]{\n        \\includegraphics[width=0.2\\textwidth]{Figures/sliding-window.png}\n    }\n    \\caption{The effect of hyper-parameters in data pre-processing on the model, where (a) is the effect of transforming different raw parameter signals into velocity features in Feature engineering and selection on the model, and (b) is the effect of \\textit{window size} in Segmentation on the model.} \n\t\\label{fig:parameter}\n\\end{figure} \n\n\\textbf{Segmentation} \nAfterwards, data segmentation is adopted in the proposed framework to generate suitable data samples for model training. The approach of using a sliding window to randomly select a local patch is frequently used in time-series classification and has been demonstrated to be useful in improving model performance. Fig. \\ref{fig:segmentation} shows an example based on the DraWritePD data set. The multichannel time-series data are cropped into small patches with a slight overlap to preserve the temporal information. The parameters \\textit{window size} ($w$) and \\textit{stride size} ($s$) control the length and overlap of the resampling patches. The choice of \\textit{window size} and \\textit{stride size} depends on the concrete applications. Experiments illustrated by Fig. \\ref{fig:parameter} (b) demonstrate that the \\textit{$w$=128} provides balance between the performance and the efficiency under the proposed LSTM-CNN architecture. It is also worth noting that such segmentation generates more data samples for training, which is crucial for limited or small data sampling cases, including the proposed method based on the LSTM-CNN deep learning architecture. \n\n\n4 Experimental Results\n\\section{Experimental Results}\\label{sec:results}\n\nIn this section, we evaluate and analyse the performance of the proposed LSTM-CNN model using the DraWritePD data set.  The model runs on the desktop PC with an Intel(R) Core(TM) $3.60$ GHz($8$ CPU), $32$GB RAM, and an NVIDIA RTX$3070$Ti GPU with $8$ GB memory. \n\n\\subsection{Dataset}\n\nThe DraWritePD set contains $157$ pieces of sequence data from $29$ subjects with HC and $20$ patients with PD. The raw sequence data needs to be cropped into patches before being fed to the LSTM-CNN model. The class imbalance problem may occur during the segmentation due to the different lengths of the sequence data. As shown in Fig. \\ref{fig:segmentation}, a nonuniform sampling strategy with varying stride size is adopted to impose the number of generated patch data in each class to be the same. The statistics of the training and testing dataset is listed in Table \\ref{tab:dataste}.\n\n\\renewcommand{\\arraystretch}{1.5}\n\\begin{table}[ht]\n\t\\centering\n\t\\scriptsize \n\t\\caption{The information of training set and testing set.}\n    \\setlength{\\tabcolsep}{4mm}{\n\t\\begin{tabular}{|c||c|c|c|c|}\n\t\t\\hline \n\t    \\multirow{2}*{}  &  \\multicolumn{2}{c|}{Training set} & \\multicolumn{2}{c|}{Testing set} \\\\\n\t    \\cline{2-5}\n\t\t                 &       HC       &         PD        &         HC       &       PD      \\\\\n\t\t\\hline\n\t\t\\hline\n\t\tParticipant     &       25       &         16        &         4        &        4      \\\\\n\t\t\\hline\n\t\tSequence set (S)      &       80       &         51        &         15       &       11      \\\\\n\t\t\\hline\n\t\tPatch set (P)      &     16166      &       16836       &       3670       &     3319      \\\\\n\t\t\\hline\n\t\\end{tabular}}\n\t\\label{tab:dataste}\n\\end{table}\n\nDuring the training phase, the patch data set was randomly divided in the $8:2$ ratio into a training patch data set and a validation patch data set. During the testing phase, the proposed model was first evaluated based on the testing patch data set and then applied to the testing sequence data set, where the predicted result of each raw sequence data set was determined by the majority vote of the prediction result of the patch data. To clarify, we denote the patch data set testing as $P$ and the raw sequence data set as $S$, and independently evaluate the performance of the proposed model on the two cases. \n\n\n\\subsection{Experimental Setup}\nIn order to fully exploit the performance of the proposed LSTM-CNN model, we use a cross-validation strategy to optimally choose the parameters. Adam \\cite{kingma2014adam} optimiser is used to train the model, and the initial learning rate is set to $0.001$. Furthermore, the cross-entropy loss function is used for model fitting and the batch size is set to $64$. The proposed model is completed in $200$ epochs with the loss curve shown in Fig. \\ref{fig:train_val_curve}. We use the metrics: accuracy, precision, recall, specificity, $F_1$ score, and Matthews correlation coefficient (MCC) for evaluation, where the latter has been adopted by many existing methods to describe the different aspects of the performance of a classifier \\cite{baldi2000assessing}. Once the training phase is completed, the one with the best fitness value is chosen for testing. Moreover, the length of segmented patches and the choice of feature selection are also discussed to interpret their roles in determining the model performance. As shown in Fig.~\\ref{fig:parameter}, the model achieves the best classification result when the window size is $128$ by using the ($v_x,v_y$) velocity characteristics. The model performance is eventually tested on both the original sequences dataset ($S$) and the segmented patches dataset ($P$).\n\n\n\\subsection{Quantitative Evaluation and Comparison}\n\n\\renewcommand{\\arraystretch}{1.5}\n\\begin{table*}[ht]\n\t\\centering\n\t\\scriptsize \n\t\\caption{Quantitative comparison of different classification methods.}\n \\setlength{\\tabcolsep}{3mm}{\n\t\\begin{tabular}{ | c || c || c | c | c | c | c | c |}\n  \\hline\n  \\multirow{2}*{Model} & \\multirow{2}*{Inference time (s)} &  \\multicolumn{6}{c|}{Metric} \\\\\n\t\\cline{3-8}\n\t\t~ & ~ &  Accuracy (P/S) & Precision (P/S) & Recall (P/S) & Specificity (P/S) & $F_1$ score (P/S) & MCC (P/S) \\\\\n\t\t\\hline\n        \\hline\n        LR & 0.034 & 0.8061 / 0.9231  & 0.8559 / 0.8462 & 0.8565 / \\textbf{1.00} & 0.7018 / 0.8667 & 0.8562 / 0.9167 & 0.5585 / 0.8563  \\\\\n        \\hline\n        SVM & 6.060 &  0.8371 / 0.8846  & 0.8657 / 0.7857 & 0.8977 / \\textbf{1.00} & 0.7119 / 0.8000 & 0.8814 / 0.8800 & 0.6229 / 0.7928  \\\\\n        \\hline\n        RF & 9.526 &  0.8339 / 0.8462  & 0.9015 / 0.8889 & 0.8412 / 0.7273 & 0.8088 / 0.9333 & 0.8729 / 0.8000 & 0.6368 / 0.6860  \\\\\n        \\hline\n        LGB & 0.161 & 0.7889 / 0.8077   & 0.9183 / 0.8750 & 0.7538 / 0.6364 & 0.8613 / 0.9333 & 0.8280 / 0.7368 & 0.5800 / 0.6098  \\\\\n        \\hline\n        \n        MLP & 4.095 & 0.8274 / 0.8846  & 0.8598 / 0.8333 & 0.8921 / 0.9091 & 0.6891 / 0.8667 & 0.8756 / 0.8696 & 0.5950 / 0.7688  \\\\\n        \\hline\n        AlexNet & 4.143 &  0.7872 / 0.8846  & 0.9093 / \\textbf{1.00} & 0.7606 / 0.7273 & 0.8425 / \\textbf{1.00} & 0.8284 / 0.8421 & 0.5662 / 0.7785  \\\\\n        \\hline\n        \\hline\n        LSTM-CNN (Ours) & 4.212 &  0.7994 / \\textbf{0.9615}  & 0.8883 / \\textbf{1.00} & 0.8039 / 0.9091 & 0.7900 / \\textbf{1.00} & 0.8440 / \\textbf{0.9524} & 0.5720 / \\textbf{0.9232}  \\\\\n        \\hline     \n\t\\end{tabular}}\n\t\\label{tab:model structure}\n\\end{table*}\n\nWe provide a quantitative comparison to demonstrate the effectiveness and advantages of the proposed LSTM-CNN model. First, we compare it with some traditional machine learning (ML)-based classifiers, which include the Logistic Regression (LR), Support Vector Machine (SVM), Random Forest (RF), and LightGBM (LGB) \\cite{ke2017lightgbm}. For each classifier, we take $10$-fold cross-validations and the grid search algorithm to optimise the parameters and to ensure the robustness of the results. The training and validation of all these ML classifiers are run under Python's scikit-learn library \\cite{pedregosa2011scikit}. As shown in Table \\ref{tab:model structure}, our model has obvious advantages in most classification metrics,  in terms of accuracy increased by $3.8$\\%, $F_1$ score increased by $3.5$\\%, and MCC increased by $6.6$\\%. Regarding the efficiency of the model, our method outperforms $1.8$ seconds and $5.3$ seconds compared to the SVM model and the RF model, although it is slower than the optimised LR model and the LGB model.\n\nAdditionally, we also compare the performance of different neural network models on this task. The multilayer perceptron (MLP) is the basic model and its structure consists of two fully connected layers. AlexNet adopts a convolutional neural network structure similar to AlexNet\\cite{krizhevsky2012imagenet}, but to adapt to the size of the input data, the internal parameters are modified. In our work, to improve the efficiency of the LSTM-CNN, in addition to the $1$D convolution operation in the CNN block, an LSTM block is added, which contains a concatenation operation. Additionally, as shown in Fig.\\ref{fig:train_val_curve}, the average of multiple experimental results (N=$10$) is used as the model result. Finally, let us point out that our methods achieve optimal results in all metrics with the accuracy rate being $96.15\\%$, the $F_1$ score being $95.24\\%$, and the MCC being $92.32\\%$. Specifically, in the $S$ testing set, only one sequence data from the PD category is misclassified, and the remaining 25 sequence data are correctly classified.\n\nIn summary, the model proposed in this article could not only achieve high recognition accuracy, but also significantly simplify the structure of the model and improve the efficiency of deep learning models in the diagnosis of PD.\n\n\\begin{figure}[t]\n    \\centering\n    \\subfigure[]{\n        \\includegraphics[width=0.22\\textwidth]{Figures/train-acc-loss.png}\n        }\n    \\subfigure[]{\n        \\includegraphics[width=0.22\\textwidth]{Figures/val-acc-loss.png}\n    }\n    \\caption{The accuracy and loss curves of the LSTM-CNN model on the training set(a) and the validation set(b), respectively, where the solid curve represents the average of multiple experiments, and the shaded part represents the range of the results of multiple experiments (N=$10$).} \n\t\\label{fig:train_val_curve}\n\\end{figure}\n\n4.2 Experimental Setup\n\\subsection{Experimental Setup}\nIn order to fully exploit the performance of the proposed LSTM-CNN model, we use a cross-validation strategy to optimally choose the parameters. Adam \\cite{kingma2014adam} optimiser is used to train the model, and the initial learning rate is set to $0.001$. Furthermore, the cross-entropy loss function is used for model fitting and the batch size is set to $64$. The proposed model is completed in $200$ epochs with the loss curve shown in Fig. \\ref{fig:train_val_curve}. We use the metrics: accuracy, precision, recall, specificity, $F_1$ score, and Matthews correlation coefficient (MCC) for evaluation, where the latter has been adopted by many existing methods to describe the different aspects of the performance of a classifier \\cite{baldi2000assessing}. Once the training phase is completed, the one with the best fitness value is chosen for testing. Moreover, the length of segmented patches and the choice of feature selection are also discussed to interpret their roles in determining the model performance. As shown in Fig.~\\ref{fig:parameter}, the model achieves the best classification result when the window size is $128$ by using the ($v_x,v_y$) velocity characteristics. The model performance is eventually tested on both the original sequences dataset ($S$) and the segmented patches dataset ($P$).\n\n\n",
        "label": "fig:parameter",
        "Descriptive_question1": "What is the optimal window size for segmentation shown in figure_4 (b)?",
        "Descriptive_question2": "What type of features are analyzed in figure_4 (a) for their effect on the model?",
        "Reasoning_question1": "Why might the window size of 128 provide a balance between performance and efficiency in figure_4 (b)?",
        "Reasoning_question2": "How does the transformation of raw parameter signals into velocity features in figure_4 (a) potentially improve the model's classification performance?",
        "Descriptive_answer1": "128",
        "Descriptive_answer2": "velocity features",
        "Reasoning_answer1": "A window size of 128 likely balances the trade-off between including sufficient temporal information for classification and maintaining computational efficiency. Smaller window sizes may not capture enough temporal context, reducing performance, while larger windows increase computational load and may introduce redundant data. Thus, 128 provides enough data for accurate recognition while keeping the model efficient.",
        "Reasoning_answer2": "Transforming raw parameter signals into velocity features converts non-stationary position data into stationary velocity data, which are more stable and can better reflect fine motor characteristics affected by Parkinson's disease. This transformation highlights relevant movement dynamics, improving the tractability and accuracy of the classification model."
    },
    {
        "paper_id": "2302.00973.json",
        "image_id": "figure_6",
        "image_path": [
            "/home/yz979/scratch/chengye/scimolmo/test-images-tables/dataset_imagestables/2302.00973/images/val-acc-loss.png",
            "/home/yz979/scratch/chengye/scimolmo/test-images-tables/dataset_imagestables/2302.00973/images/train-acc-loss.png"
        ],
        "caption": "The accuracy and loss curves of the LSTM-CNN model on the training set(a) and the validation set(b), respectively, where the solid curve represents the average of multiple experiments, and the shaded part represents the range of the results of multiple experiments (N=$10$).",
        "classify": "Chart",
        "section_info": "4 Experimental Results\n\\section{Experimental Results}\\label{sec:results}\n\nIn this section, we evaluate and analyse the performance of the proposed LSTM-CNN model using the DraWritePD data set.  The model runs on the desktop PC with an Intel(R) Core(TM) $3.60$ GHz($8$ CPU), $32$GB RAM, and an NVIDIA RTX$3070$Ti GPU with $8$ GB memory. \n\n\\subsection{Dataset}\n\nThe DraWritePD set contains $157$ pieces of sequence data from $29$ subjects with HC and $20$ patients with PD. The raw sequence data needs to be cropped into patches before being fed to the LSTM-CNN model. The class imbalance problem may occur during the segmentation due to the different lengths of the sequence data. As shown in Fig. \\ref{fig:segmentation}, a nonuniform sampling strategy with varying stride size is adopted to impose the number of generated patch data in each class to be the same. The statistics of the training and testing dataset is listed in Table \\ref{tab:dataste}.\n\n\\renewcommand{\\arraystretch}{1.5}\n\\begin{table}[ht]\n\t\\centering\n\t\\scriptsize \n\t\\caption{The information of training set and testing set.}\n    \\setlength{\\tabcolsep}{4mm}{\n\t\\begin{tabular}{|c||c|c|c|c|}\n\t\t\\hline \n\t    \\multirow{2}*{}  &  \\multicolumn{2}{c|}{Training set} & \\multicolumn{2}{c|}{Testing set} \\\\\n\t    \\cline{2-5}\n\t\t                 &       HC       &         PD        &         HC       &       PD      \\\\\n\t\t\\hline\n\t\t\\hline\n\t\tParticipant     &       25       &         16        &         4        &        4      \\\\\n\t\t\\hline\n\t\tSequence set (S)      &       80       &         51        &         15       &       11      \\\\\n\t\t\\hline\n\t\tPatch set (P)      &     16166      &       16836       &       3670       &     3319      \\\\\n\t\t\\hline\n\t\\end{tabular}}\n\t\\label{tab:dataste}\n\\end{table}\n\nDuring the training phase, the patch data set was randomly divided in the $8:2$ ratio into a training patch data set and a validation patch data set. During the testing phase, the proposed model was first evaluated based on the testing patch data set and then applied to the testing sequence data set, where the predicted result of each raw sequence data set was determined by the majority vote of the prediction result of the patch data. To clarify, we denote the patch data set testing as $P$ and the raw sequence data set as $S$, and independently evaluate the performance of the proposed model on the two cases. \n\n\n\\subsection{Experimental Setup}\nIn order to fully exploit the performance of the proposed LSTM-CNN model, we use a cross-validation strategy to optimally choose the parameters. Adam \\cite{kingma2014adam} optimiser is used to train the model, and the initial learning rate is set to $0.001$. Furthermore, the cross-entropy loss function is used for model fitting and the batch size is set to $64$. The proposed model is completed in $200$ epochs with the loss curve shown in Fig. \\ref{fig:train_val_curve}. We use the metrics: accuracy, precision, recall, specificity, $F_1$ score, and Matthews correlation coefficient (MCC) for evaluation, where the latter has been adopted by many existing methods to describe the different aspects of the performance of a classifier \\cite{baldi2000assessing}. Once the training phase is completed, the one with the best fitness value is chosen for testing. Moreover, the length of segmented patches and the choice of feature selection are also discussed to interpret their roles in determining the model performance. As shown in Fig.~\\ref{fig:parameter}, the model achieves the best classification result when the window size is $128$ by using the ($v_x,v_y$) velocity characteristics. The model performance is eventually tested on both the original sequences dataset ($S$) and the segmented patches dataset ($P$).\n\n\n\\subsection{Quantitative Evaluation and Comparison}\n\n\\renewcommand{\\arraystretch}{1.5}\n\\begin{table*}[ht]\n\t\\centering\n\t\\scriptsize \n\t\\caption{Quantitative comparison of different classification methods.}\n \\setlength{\\tabcolsep}{3mm}{\n\t\\begin{tabular}{ | c || c || c | c | c | c | c | c |}\n  \\hline\n  \\multirow{2}*{Model} & \\multirow{2}*{Inference time (s)} &  \\multicolumn{6}{c|}{Metric} \\\\\n\t\\cline{3-8}\n\t\t~ & ~ &  Accuracy (P/S) & Precision (P/S) & Recall (P/S) & Specificity (P/S) & $F_1$ score (P/S) & MCC (P/S) \\\\\n\t\t\\hline\n        \\hline\n        LR & 0.034 & 0.8061 / 0.9231  & 0.8559 / 0.8462 & 0.8565 / \\textbf{1.00} & 0.7018 / 0.8667 & 0.8562 / 0.9167 & 0.5585 / 0.8563  \\\\\n        \\hline\n        SVM & 6.060 &  0.8371 / 0.8846  & 0.8657 / 0.7857 & 0.8977 / \\textbf{1.00} & 0.7119 / 0.8000 & 0.8814 / 0.8800 & 0.6229 / 0.7928  \\\\\n        \\hline\n        RF & 9.526 &  0.8339 / 0.8462  & 0.9015 / 0.8889 & 0.8412 / 0.7273 & 0.8088 / 0.9333 & 0.8729 / 0.8000 & 0.6368 / 0.6860  \\\\\n        \\hline\n        LGB & 0.161 & 0.7889 / 0.8077   & 0.9183 / 0.8750 & 0.7538 / 0.6364 & 0.8613 / 0.9333 & 0.8280 / 0.7368 & 0.5800 / 0.6098  \\\\\n        \\hline\n        \n        MLP & 4.095 & 0.8274 / 0.8846  & 0.8598 / 0.8333 & 0.8921 / 0.9091 & 0.6891 / 0.8667 & 0.8756 / 0.8696 & 0.5950 / 0.7688  \\\\\n        \\hline\n        AlexNet & 4.143 &  0.7872 / 0.8846  & 0.9093 / \\textbf{1.00} & 0.7606 / 0.7273 & 0.8425 / \\textbf{1.00} & 0.8284 / 0.8421 & 0.5662 / 0.7785  \\\\\n        \\hline\n        \\hline\n        LSTM-CNN (Ours) & 4.212 &  0.7994 / \\textbf{0.9615}  & 0.8883 / \\textbf{1.00} & 0.8039 / 0.9091 & 0.7900 / \\textbf{1.00} & 0.8440 / \\textbf{0.9524} & 0.5720 / \\textbf{0.9232}  \\\\\n        \\hline     \n\t\\end{tabular}}\n\t\\label{tab:model structure}\n\\end{table*}\n\nWe provide a quantitative comparison to demonstrate the effectiveness and advantages of the proposed LSTM-CNN model. First, we compare it with some traditional machine learning (ML)-based classifiers, which include the Logistic Regression (LR), Support Vector Machine (SVM), Random Forest (RF), and LightGBM (LGB) \\cite{ke2017lightgbm}. For each classifier, we take $10$-fold cross-validations and the grid search algorithm to optimise the parameters and to ensure the robustness of the results. The training and validation of all these ML classifiers are run under Python's scikit-learn library \\cite{pedregosa2011scikit}. As shown in Table \\ref{tab:model structure}, our model has obvious advantages in most classification metrics,  in terms of accuracy increased by $3.8$\\%, $F_1$ score increased by $3.5$\\%, and MCC increased by $6.6$\\%. Regarding the efficiency of the model, our method outperforms $1.8$ seconds and $5.3$ seconds compared to the SVM model and the RF model, although it is slower than the optimised LR model and the LGB model.\n\nAdditionally, we also compare the performance of different neural network models on this task. The multilayer perceptron (MLP) is the basic model and its structure consists of two fully connected layers. AlexNet adopts a convolutional neural network structure similar to AlexNet\\cite{krizhevsky2012imagenet}, but to adapt to the size of the input data, the internal parameters are modified. In our work, to improve the efficiency of the LSTM-CNN, in addition to the $1$D convolution operation in the CNN block, an LSTM block is added, which contains a concatenation operation. Additionally, as shown in Fig.\\ref{fig:train_val_curve}, the average of multiple experimental results (N=$10$) is used as the model result. Finally, let us point out that our methods achieve optimal results in all metrics with the accuracy rate being $96.15\\%$, the $F_1$ score being $95.24\\%$, and the MCC being $92.32\\%$. Specifically, in the $S$ testing set, only one sequence data from the PD category is misclassified, and the remaining 25 sequence data are correctly classified.\n\nIn summary, the model proposed in this article could not only achieve high recognition accuracy, but also significantly simplify the structure of the model and improve the efficiency of deep learning models in the diagnosis of PD.\n\n\\begin{figure}[t]\n    \\centering\n    \\subfigure[]{\n        \\includegraphics[width=0.22\\textwidth]{Figures/train-acc-loss.png}\n        }\n    \\subfigure[]{\n        \\includegraphics[width=0.22\\textwidth]{Figures/val-acc-loss.png}\n    }\n    \\caption{The accuracy and loss curves of the LSTM-CNN model on the training set(a) and the validation set(b), respectively, where the solid curve represents the average of multiple experiments, and the shaded part represents the range of the results of multiple experiments (N=$10$).} \n\t\\label{fig:train_val_curve}\n\\end{figure}\n\n4.2 Experimental Setup\n\\subsection{Experimental Setup}\nIn order to fully exploit the performance of the proposed LSTM-CNN model, we use a cross-validation strategy to optimally choose the parameters. Adam \\cite{kingma2014adam} optimiser is used to train the model, and the initial learning rate is set to $0.001$. Furthermore, the cross-entropy loss function is used for model fitting and the batch size is set to $64$. The proposed model is completed in $200$ epochs with the loss curve shown in Fig. \\ref{fig:train_val_curve}. We use the metrics: accuracy, precision, recall, specificity, $F_1$ score, and Matthews correlation coefficient (MCC) for evaluation, where the latter has been adopted by many existing methods to describe the different aspects of the performance of a classifier \\cite{baldi2000assessing}. Once the training phase is completed, the one with the best fitness value is chosen for testing. Moreover, the length of segmented patches and the choice of feature selection are also discussed to interpret their roles in determining the model performance. As shown in Fig.~\\ref{fig:parameter}, the model achieves the best classification result when the window size is $128$ by using the ($v_x,v_y$) velocity characteristics. The model performance is eventually tested on both the original sequences dataset ($S$) and the segmented patches dataset ($P$).\n\n\n4.3 Quantitative Evaluation and Comparison\n\\subsection{Quantitative Evaluation and Comparison}\n\n\\renewcommand{\\arraystretch}{1.5}\n\\begin{table*}[ht]\n\t\\centering\n\t\\scriptsize \n\t\\caption{Quantitative comparison of different classification methods.}\n \\setlength{\\tabcolsep}{3mm}{\n\t\\begin{tabular}{ | c || c || c | c | c | c | c | c |}\n  \\hline\n  \\multirow{2}*{Model} & \\multirow{2}*{Inference time (s)} &  \\multicolumn{6}{c|}{Metric} \\\\\n\t\\cline{3-8}\n\t\t~ & ~ &  Accuracy (P/S) & Precision (P/S) & Recall (P/S) & Specificity (P/S) & $F_1$ score (P/S) & MCC (P/S) \\\\\n\t\t\\hline\n        \\hline\n        LR & 0.034 & 0.8061 / 0.9231  & 0.8559 / 0.8462 & 0.8565 / \\textbf{1.00} & 0.7018 / 0.8667 & 0.8562 / 0.9167 & 0.5585 / 0.8563  \\\\\n        \\hline\n        SVM & 6.060 &  0.8371 / 0.8846  & 0.8657 / 0.7857 & 0.8977 / \\textbf{1.00} & 0.7119 / 0.8000 & 0.8814 / 0.8800 & 0.6229 / 0.7928  \\\\\n        \\hline\n        RF & 9.526 &  0.8339 / 0.8462  & 0.9015 / 0.8889 & 0.8412 / 0.7273 & 0.8088 / 0.9333 & 0.8729 / 0.8000 & 0.6368 / 0.6860  \\\\\n        \\hline\n        LGB & 0.161 & 0.7889 / 0.8077   & 0.9183 / 0.8750 & 0.7538 / 0.6364 & 0.8613 / 0.9333 & 0.8280 / 0.7368 & 0.5800 / 0.6098  \\\\\n        \\hline\n        \n        MLP & 4.095 & 0.8274 / 0.8846  & 0.8598 / 0.8333 & 0.8921 / 0.9091 & 0.6891 / 0.8667 & 0.8756 / 0.8696 & 0.5950 / 0.7688  \\\\\n        \\hline\n        AlexNet & 4.143 &  0.7872 / 0.8846  & 0.9093 / \\textbf{1.00} & 0.7606 / 0.7273 & 0.8425 / \\textbf{1.00} & 0.8284 / 0.8421 & 0.5662 / 0.7785  \\\\\n        \\hline\n        \\hline\n        LSTM-CNN (Ours) & 4.212 &  0.7994 / \\textbf{0.9615}  & 0.8883 / \\textbf{1.00} & 0.8039 / 0.9091 & 0.7900 / \\textbf{1.00} & 0.8440 / \\textbf{0.9524} & 0.5720 / \\textbf{0.9232}  \\\\\n        \\hline     \n\t\\end{tabular}}\n\t\\label{tab:model structure}\n\\end{table*}\n\nWe provide a quantitative comparison to demonstrate the effectiveness and advantages of the proposed LSTM-CNN model. First, we compare it with some traditional machine learning (ML)-based classifiers, which include the Logistic Regression (LR), Support Vector Machine (SVM), Random Forest (RF), and LightGBM (LGB) \\cite{ke2017lightgbm}. For each classifier, we take $10$-fold cross-validations and the grid search algorithm to optimise the parameters and to ensure the robustness of the results. The training and validation of all these ML classifiers are run under Python's scikit-learn library \\cite{pedregosa2011scikit}. As shown in Table \\ref{tab:model structure}, our model has obvious advantages in most classification metrics,  in terms of accuracy increased by $3.8$\\%, $F_1$ score increased by $3.5$\\%, and MCC increased by $6.6$\\%. Regarding the efficiency of the model, our method outperforms $1.8$ seconds and $5.3$ seconds compared to the SVM model and the RF model, although it is slower than the optimised LR model and the LGB model.\n\nAdditionally, we also compare the performance of different neural network models on this task. The multilayer perceptron (MLP) is the basic model and its structure consists of two fully connected layers. AlexNet adopts a convolutional neural network structure similar to AlexNet\\cite{krizhevsky2012imagenet}, but to adapt to the size of the input data, the internal parameters are modified. In our work, to improve the efficiency of the LSTM-CNN, in addition to the $1$D convolution operation in the CNN block, an LSTM block is added, which contains a concatenation operation. Additionally, as shown in Fig.\\ref{fig:train_val_curve}, the average of multiple experimental results (N=$10$) is used as the model result. Finally, let us point out that our methods achieve optimal results in all metrics with the accuracy rate being $96.15\\%$, the $F_1$ score being $95.24\\%$, and the MCC being $92.32\\%$. Specifically, in the $S$ testing set, only one sequence data from the PD category is misclassified, and the remaining 25 sequence data are correctly classified.\n\nIn summary, the model proposed in this article could not only achieve high recognition accuracy, but also significantly simplify the structure of the model and improve the efficiency of deep learning models in the diagnosis of PD.\n\n\\begin{figure}[t]\n    \\centering\n    \\subfigure[]{\n        \\includegraphics[width=0.22\\textwidth]{Figures/train-acc-loss.png}\n        }\n    \\subfigure[]{\n        \\includegraphics[width=0.22\\textwidth]{Figures/val-acc-loss.png}\n    }\n    \\caption{The accuracy and loss curves of the LSTM-CNN model on the training set(a) and the validation set(b), respectively, where the solid curve represents the average of multiple experiments, and the shaded part represents the range of the results of multiple experiments (N=$10$).} \n\t\\label{fig:train_val_curve}\n\\end{figure}\n\n",
        "label": "fig:train_val_curve",
        "Descriptive_question1": "What does the solid curve represent in figure_6?",
        "Descriptive_question2": "What does the shaded part indicate in figure_6?",
        "Reasoning_question1": "How does the accuracy trend of the LSTM-CNN model differ between the training and validation sets as shown in figure_6?",
        "Reasoning_question2": "What can be inferred about the stability of the LSTM-CNN model's performance from the shaded parts in figure_6?",
        "Descriptive_answer1": "average",
        "Descriptive_answer2": "range",
        "Reasoning_answer1": "In figure_6, the accuracy curve of the LSTM-CNN model on the training set (a) shows a continuous increase and converges to a high accuracy value as the number of epochs increases, indicating effective learning and fitting of the training data. Meanwhile, the validation set accuracy curve (b) also increases but at a slightly slower rate and stabilizes at a slightly lower value compared to the training set, reflecting some generalization from the training data but with less overfitting. This difference in trends indicates that the model learns well on training data and maintains good, though slightly lower, performance on unseen validation data.",
        "Reasoning_answer2": "The shaded parts in figure_6 represent the range of the results from multiple experiments (N=10), indicating variability or consistency in model performance across runs. For both training (a) and validation (b) sets, the shaded regions are relatively narrow as epochs increase, suggesting the LSTM-CNN model's performance is stable and reproducible with low variance. The narrowing of the shaded part over epochs shows that the model converges consistently, indicating robust and reliable performance."
    },
    {
        "paper_id": "1704.01165.json",
        "image_id": "figure_5",
        "image_path": [
            "/home/yz979/scratch/chengye/scimolmo/test-images-tables/dataset_imagestables/1704.01165/images/2lm.eps"
        ],
        "caption": "Total fraction of water (ice + water in hydrated silicates) remaining after the main sequence, RGB and AGB stellar evolution phases, for a 2 M$_{\\odot}$ progenitor with reduced iron abundances. The retention of water is shown as a function of the minor planet's initial orbital distance, composition, radius and formation time.",
        "classify": "Chart",
        "section_info": "3 Results\n\\section{Results}\\label{S:Results}\nIn this section we discuss the bulk amount of water surviving in the planetary system, as a function of our free model parameters. Figs. \\ref{fig:Masses}-\\ref{fig:Metallicities} show the final fraction of water, based on the end states of the production runs discussed in Section \\ref{S:Model} (i.e., when the star reaches the WD stage). We present the \\emph{total} fraction of retained water, defined as water ice + water in hydrated silicates, which ultimately contributes hydrogen and oxygen when accreting onto polluted WD atmospheres. Each panel consists of three subplots, each representing a different choice for the initial composition. Within each subplot there are multiple lines, depicting the final water fraction as a function of the initial orbital distance. Each line is characterized by a specific color and width, as well as a style. The line width decreases with the size of the object, so thin lines represent large objects, and each line style corresponds to a different formation time. \n\nFig. \\ref{fig:Masses} summarizes the main results from this study. Note that Panel \\ref{fig:1sm} (upper-left) is nearly identical to the equivalent panel in our previous study (Fig. 4(a) from MP16, having the same stellar mass, however a slightly higher metallicity compared to [Fe/H]=0). All the other panels however indicate a notable change in water retention as the progenitor mass increases. \n\nAnother important model parameter is the initial orbital distance. The general trend in the data suggests that minor planets at a greater distance from the star can better retain their water. This is true, as one might naturally expect, for the overwhelming majority of cases. There could be exceptions to this rule, however they require special circumstances. Specifically, these exceptions occur for rock-rich minor planets around 2 M$_{\\odot}$ progenitors, as seen in Panel \\ref{fig:2sm}. The rightmost subplot shows the water retention for a rock/ice mass ratio of 3. Note the solid blue line that shows the water retention for minor planets with a radius of 25 km and a formation time of 3 Myr. It can be seen that water retention decreases from 175 AU to 20 AU, however it then sharply increases again as the initial orbital distance shortens. The explanation in this case is that at a distance of less than 20 AU, the surface temperature of the minor planet increases. This change in the boundary condition also imposes higher internal temperatures, which in turn allow for some small fraction of the internal rock to hydrate (react with liquid water) during the minor planet's early evolution. Hence, below 20 AU the retention of water actually increases towards the star. This trend is again reversed below 7 AU, since the surface temperature is so high that ice is expelled even prior to attaining hydration temperatures. We thus get peak water retention at 7 AU for this particular combination of parameters. All other minor planets on the same subplot which have a radius larger than 25 km, will also sublimate all their internal ice below a distance of 20 AU. Nevertheless, in these larger objects water retention may behave differently. For example, in a minor planet with a radius of 100 km, water retention below 20 AU actually decreases. The reason in this case is that the peak internal temperatures during the first few Myr of evolution are much higher (due to the larger radius), and may even exceed the point of rock dehydration, in which the rock exudes some of the water it had previously absorbed. The change in surface boundary condition at closer distances again imposes higher internal temperatures, which is why more dehydration occurs towards the star (hence the reduction in water retention).\n\nWe are thus faced with the recognition that water retention depends on the particular combination of model parameters. The early evolution of a minor planet determines much of its water retention outcome: its internal structure, whether or not it is differentiated, and how much of its rock is hydrated or dehydrated. During the main sequence and post-main sequence, the luminosity of the star then determines if ice is removed via sublimation, either partially or completely. Note that here we only consider water ice, although other, far less abundant yet more volatile ices may also exist, especially in the smallest minor planets.\n\nFigs. \\ref{fig:2-7-3-3} - \\ref{fig:6.4-5-5-2} demonstrate exactly how water retention is affected by various combinations of parameters. Fig. \\ref{fig:2-7-3-3} shows the compositional cross-section of a minor planet with a radius of 100 km, following its first 10 Myr of evolution around a 2 M$_{\\odot}$ star (since this is an animated figure, one may view the evolution, in addition to the end state which is depicted by the still image). In this case the formation time is short and the composition is rock-rich, so the minor planet quickly differentiates. Almost all the anhydrous rock becomes hydrated, embedding nearly half of the initial water content onto it. The remaining water forms a very thin ice-rich crust which later sublimates via insolation from the star. For progenitor star masses of 3 M$_{\\odot}$ and above, these stars are so massive that most or even all the ice inside large minor planets close to the star, is sublimated prior to reaching hydration temperatures. Fig. \\ref{fig:3-12-4-3} shows the compositional cross-section of a minor planet with a radius of 100 km, following its first 1.7 Myr of evolution around a 3 M$_{\\odot}$ star. In this particular example some hydration still occurs, however a considerable fraction of the rock remains anhydrous (in the animation it can be seen that outside-in sublimation of ice from the mantle and inside-out differentiation operate on competing time-scales). In some very extreme cases water can be retained internally even though the minor planet is close to the star, but not as a result of rock hydration. This occurs only for in minor planets with a long formation time and a water-rich composition, around extremely massive stars (5-6.4 M$_{\\odot}$, see pink dashed line in Panels \\ref{fig:5sm} and \\ref{fig:64sm}). In this heating regime the star reaches the WD phase so quickly that despite its intense luminosity some fraction of the internal ice remains. E.g., Fig. \\ref{fig:6.4-5-5-2} shows the compositional cross-section of a minor planet with a radius of 100 km, following its entire 64 Myr evolution to the WD stage, around a 6.4 M$_{\\odot}$ star. It can be seen that the inner core retains its original icy composition, simply because heat does not have sufficient time to reach the centre. We note that forming a water-rich minor planet at close distances around such massive stars is probably unrealistic, given their short lifetimes and their distant snow lines \\citep{KennedyKenyon-2008}. Therefore this combination of parameters is perhaps only plausible for a minor planet that has migrated inward.\n\n\\begin{figure}\n\\begin{center}\n    \\label{fig:2-7-3-3}\\includegraphics[scale=1]{movie1.eps}\n\t\\caption{Animated figure (duration - 20 s) featuring the first 10 Myr of evolution (from a total of 1.35 Gyr) of a 100 km radius minor planet around a 2 M$_{\\odot}$ progenitor at 7 AU, with a 3 Myr formation time and 75\\% initial rock fraction. Colour interpretation: {\\it black} (pores); {\\it white} (water ice); {\\it blue} (liquid water); {\\it brown} (anhydrous rock); and {\\it olive} (hydrated rock). The animation shows the differentiation of an initially homogeneous body into a hydrous rocky core underlying a thin ice-enriched crust, from which the ice subsequently sublimates.}\n\\end{center}\n\\end{figure}\n\n\\begin{figure}\n\t\\begin{center}{\\label{fig:3-12-4-3}\\includegraphics[scale=1]{movie2.eps}}\n\t\\caption{Animated figure (duration - 22 s) featuring the first 1.7 Myr of evolution (from a total of 470 Myr) of a 100 km radius minor planet around a 3 M$_{\\odot}$ progenitor at 12 AU, with a 4 Myr formation time and 75\\% initial rock fraction. Colour interpretation: {\\it black} (pores); {\\it white} (water ice); {\\it blue} (liquid water); {\\it brown} (anhydrous rock); and {\\it olive} (hydrated rock). The animation shows the differentiation of an initially homogeneous body into a smaller hydrous rocky core underlying a thicker ice-enriched crust (both compared to Fig. \\ref{fig:2-7-3-3}). Here the inside-out migration of water and outside-in sublimation of water ice from the mantle, occur on similar time scales.}\n\\end{center}\n\\end{figure}\n\n\\begin{figure}\n\t\\begin{center}{\\label{fig:6.4-5-5-2}\\includegraphics[scale=1]{movie3.eps}}\n\t\\caption{Animated figure (duration - 21 s) featuring the full 64 Myr evolution of a 100 km radius minor planet around a 6.4 M$_{\\odot}$ progenitor at 5 AU, with a 5 Myr formation time and 67\\% initial rock fraction. Colour interpretation: {\\it black} (pores); {\\it white} (water ice); {\\it blue} (liquid water); {\\it brown} (anhydrous rock); and {\\it olive} (hydrated rock). The animation shows the gradual ablation of ice from the surface inwards. This body never differentiates, and its host star's evolution is too short to remove all its internal ice.}\n\\end{center}\n\\end{figure}\n\nAnother trend in the data suggests that while more massive stars have a shorter lifetime, and a higher final/initial mass ratio (hence minor planets experience more orbital expansion), their high luminosities clearly dominate the fate of water. We define the outer bound of water retention as the distance beyond which the initial amount of water is fully retained. Examining the smallest minor planet in our sample (radius=1 km), it is immediately evident from these plots that the outer bound of water retention increases steadily with progenitor mass. \n\nThe inner bound of water retention is similarly defined as the minimal distance above which water retention is greater than zero. The behaviour of the inner bound of water retention is a bit more complex compared to the former, as illustrated by Figs. \\ref{fig:Masses} and \\ref{fig:RetentionVsMass}, however the general trend is also that the inner bound increases with progenitor mass. In 1 M$_{\\odot}$ progenitors, the exact fraction of water retained at the inner bound (3 AU) as a result of silicate hydration can be much larger than zero, and it depends on the precise parameters of the minor planet. More massive progenitors than about 2 M$_{\\odot}$ already cannot retain any water at 3 AU, which only illustrates the importance of progenitor mass. \n\n\\begin{figure}\n\t\\begin{center}\n\t\t\\subfigure[Iron abundance {[Fe/H]}=-1 - 0.61 M$_{\\odot}$ WD - 916 Myr evolution]{\\label{fig:2lm}\\includegraphics[scale=0.5]{2lm.eps}}\n\t\t\\subfigure[Iron abundance {[Fe/H]}=-2 - 0.66 M$_{\\odot}$ WD - 767 Myr evolution]{\\label{fig:2vlm}\\includegraphics[scale=0.5]{2vlm.eps}}\n\t\t\\subfigure{\\includegraphics[scale=0.5]{legend.eps}}\n\t\\end{center}\n\t\\caption{Total fraction of water (ice + water in hydrated silicates) remaining after the main sequence, RGB and AGB stellar evolution phases, for a 2 M$_{\\odot}$ progenitor with reduced iron abundances. The retention of water is shown as a function of the minor planet's initial orbital distance, composition, radius and formation time.}\n\t\\label{fig:Metallicities}\n\\end{figure}\n\nWe also investigate the effect of metallicity on water retention. We limit the investigation to 2 M$_{\\odot}$ progenitor stars, which approximately correspond to the peak mass in the WD mass distribution (see Section \\ref{S:Intro}). In Fig. \\ref{fig:Metallicities} we investigate their water retention for [Fe/H]=-1, and [Fe/H]=-2, that is, one order of magnitude and two orders of magnitude reduction in metallicity compared with Panel \\ref{fig:2sm} of Fig. \\ref{fig:Masses}. Clearly the progenitor metallicity is of lesser importance than the progenitor mass. For example, in Panel \\ref{fig:2vlm} the final WD mass is 0.66 M$_{\\odot}$, more than but similar to the resulting WD mass in Panel \\ref{fig:3sm} (0.65 M$_{\\odot}$), and the evolution time of the former is longer than that of the latter. So while an extremely metal-poor progenitor can easily result in a comparable WD mass to a considerably more massive progenitor, it nevertheless does not affect water retention nearly as much, despite having a longer lifetime. \n\n\n",
        "label": "fig:Metallicities",
        "Descriptive_question1": "What is the iron abundance value for the first subplot in figure_5?",
        "Descriptive_question2": "What is the final white dwarf mass in the second subplot of figure_5?",
        "Reasoning_question1": "How does the water retention trend vary with initial orbital distance for minor planets around a 2 M⊙ progenitor in figure_5, and what might explain any anomalies observed at closer distances?",
        "Reasoning_question2": "Comparing the two subplots in figure_5, what can be inferred about the impact of differing iron abundances ([Fe/H]=-1 vs. [Fe/H]=-2) on water retention for minor planets around a 2 M⊙ progenitor?",
        "Descriptive_answer1": "-1",
        "Descriptive_answer2": "0.66 M$_{\\odot}$",
        "Reasoning_answer1": "In figure_5, for minor planets around a 2 M$_{\\odot}$ progenitor, water retention generally increases with greater initial orbital distance, as planets farther from the star can better retain their water. However, an anomaly occurs closer to the star where, for rock-rich compositions (rock/ice mass ratio of 3) with a radius of 25 km and formation time of 3 Myr, water retention decreases from 175 AU to 20 AU but then sharply increases again at distances less than 20 AU. This is explained by the higher surface temperatures at closer distances inducing internal temperatures that allow rock hydration, increasing water retention. This trend reverses again below 7 AU, where high temperatures cause ice sublimation, reducing water retention. Thus, water retention depends on a balance between sublimation and hydration processes affected by orbital distance and minor planet properties.",
        "Reasoning_answer2": "Comparing the two subplots in figure_5 with iron abundances [Fe/H]=-1 and [Fe/H]=-2 for a 2 M$_{\\odot}$ progenitor, it is evident that the metallicity level (iron abundance) has a lesser effect on water retention than the progenitor mass. Both subplots show similar patterns in water retention despite the order of magnitude difference in metallicity and a slight difference in the resulting white dwarf mass (0.61 M$_{\\odot}$ vs. 0.66 M$_{\\odot}$). Therefore, although lower metallicity slightly influences stellar evolution and lifetime, its impact on water retention in minor planets is negligible compared to the dominant factors of progenitor mass and minor planet characteristics."
    },
    {
        "paper_id": "1704.01165.json",
        "image_id": "figure_6",
        "image_path": [
            "/home/yz979/scratch/chengye/scimolmo/test-images-tables/dataset_imagestables/1704.01165/images/FinalVsWD1.eps"
        ],
        "caption": "Water retention as a function of WD mass. The top and bottom solid lines in each panel mark the outer and inner water retention bounds respectively, expressed in terms of the minor planet's final orbital distance. Above the top solid line minor planets retain their original water content, and below the bottom solid line no water survives. The grey area in between the solid lines represents partial water retention, where 50$\\%$ water retention is marked by the dashed line.",
        "classify": "Chart",
        "section_info": "3 Results\n\\section{Results}\\label{S:Results}\nIn this section we discuss the bulk amount of water surviving in the planetary system, as a function of our free model parameters. Figs. \\ref{fig:Masses}-\\ref{fig:Metallicities} show the final fraction of water, based on the end states of the production runs discussed in Section \\ref{S:Model} (i.e., when the star reaches the WD stage). We present the \\emph{total} fraction of retained water, defined as water ice + water in hydrated silicates, which ultimately contributes hydrogen and oxygen when accreting onto polluted WD atmospheres. Each panel consists of three subplots, each representing a different choice for the initial composition. Within each subplot there are multiple lines, depicting the final water fraction as a function of the initial orbital distance. Each line is characterized by a specific color and width, as well as a style. The line width decreases with the size of the object, so thin lines represent large objects, and each line style corresponds to a different formation time. \n\nFig. \\ref{fig:Masses} summarizes the main results from this study. Note that Panel \\ref{fig:1sm} (upper-left) is nearly identical to the equivalent panel in our previous study (Fig. 4(a) from MP16, having the same stellar mass, however a slightly higher metallicity compared to [Fe/H]=0). All the other panels however indicate a notable change in water retention as the progenitor mass increases. \n\nAnother important model parameter is the initial orbital distance. The general trend in the data suggests that minor planets at a greater distance from the star can better retain their water. This is true, as one might naturally expect, for the overwhelming majority of cases. There could be exceptions to this rule, however they require special circumstances. Specifically, these exceptions occur for rock-rich minor planets around 2 M$_{\\odot}$ progenitors, as seen in Panel \\ref{fig:2sm}. The rightmost subplot shows the water retention for a rock/ice mass ratio of 3. Note the solid blue line that shows the water retention for minor planets with a radius of 25 km and a formation time of 3 Myr. It can be seen that water retention decreases from 175 AU to 20 AU, however it then sharply increases again as the initial orbital distance shortens. The explanation in this case is that at a distance of less than 20 AU, the surface temperature of the minor planet increases. This change in the boundary condition also imposes higher internal temperatures, which in turn allow for some small fraction of the internal rock to hydrate (react with liquid water) during the minor planet's early evolution. Hence, below 20 AU the retention of water actually increases towards the star. This trend is again reversed below 7 AU, since the surface temperature is so high that ice is expelled even prior to attaining hydration temperatures. We thus get peak water retention at 7 AU for this particular combination of parameters. All other minor planets on the same subplot which have a radius larger than 25 km, will also sublimate all their internal ice below a distance of 20 AU. Nevertheless, in these larger objects water retention may behave differently. For example, in a minor planet with a radius of 100 km, water retention below 20 AU actually decreases. The reason in this case is that the peak internal temperatures during the first few Myr of evolution are much higher (due to the larger radius), and may even exceed the point of rock dehydration, in which the rock exudes some of the water it had previously absorbed. The change in surface boundary condition at closer distances again imposes higher internal temperatures, which is why more dehydration occurs towards the star (hence the reduction in water retention).\n\nWe are thus faced with the recognition that water retention depends on the particular combination of model parameters. The early evolution of a minor planet determines much of its water retention outcome: its internal structure, whether or not it is differentiated, and how much of its rock is hydrated or dehydrated. During the main sequence and post-main sequence, the luminosity of the star then determines if ice is removed via sublimation, either partially or completely. Note that here we only consider water ice, although other, far less abundant yet more volatile ices may also exist, especially in the smallest minor planets.\n\nFigs. \\ref{fig:2-7-3-3} - \\ref{fig:6.4-5-5-2} demonstrate exactly how water retention is affected by various combinations of parameters. Fig. \\ref{fig:2-7-3-3} shows the compositional cross-section of a minor planet with a radius of 100 km, following its first 10 Myr of evolution around a 2 M$_{\\odot}$ star (since this is an animated figure, one may view the evolution, in addition to the end state which is depicted by the still image). In this case the formation time is short and the composition is rock-rich, so the minor planet quickly differentiates. Almost all the anhydrous rock becomes hydrated, embedding nearly half of the initial water content onto it. The remaining water forms a very thin ice-rich crust which later sublimates via insolation from the star. For progenitor star masses of 3 M$_{\\odot}$ and above, these stars are so massive that most or even all the ice inside large minor planets close to the star, is sublimated prior to reaching hydration temperatures. Fig. \\ref{fig:3-12-4-3} shows the compositional cross-section of a minor planet with a radius of 100 km, following its first 1.7 Myr of evolution around a 3 M$_{\\odot}$ star. In this particular example some hydration still occurs, however a considerable fraction of the rock remains anhydrous (in the animation it can be seen that outside-in sublimation of ice from the mantle and inside-out differentiation operate on competing time-scales). In some very extreme cases water can be retained internally even though the minor planet is close to the star, but not as a result of rock hydration. This occurs only for in minor planets with a long formation time and a water-rich composition, around extremely massive stars (5-6.4 M$_{\\odot}$, see pink dashed line in Panels \\ref{fig:5sm} and \\ref{fig:64sm}). In this heating regime the star reaches the WD phase so quickly that despite its intense luminosity some fraction of the internal ice remains. E.g., Fig. \\ref{fig:6.4-5-5-2} shows the compositional cross-section of a minor planet with a radius of 100 km, following its entire 64 Myr evolution to the WD stage, around a 6.4 M$_{\\odot}$ star. It can be seen that the inner core retains its original icy composition, simply because heat does not have sufficient time to reach the centre. We note that forming a water-rich minor planet at close distances around such massive stars is probably unrealistic, given their short lifetimes and their distant snow lines \\citep{KennedyKenyon-2008}. Therefore this combination of parameters is perhaps only plausible for a minor planet that has migrated inward.\n\n\\begin{figure}\n\\begin{center}\n    \\label{fig:2-7-3-3}\\includegraphics[scale=1]{movie1.eps}\n\t\\caption{Animated figure (duration - 20 s) featuring the first 10 Myr of evolution (from a total of 1.35 Gyr) of a 100 km radius minor planet around a 2 M$_{\\odot}$ progenitor at 7 AU, with a 3 Myr formation time and 75\\% initial rock fraction. Colour interpretation: {\\it black} (pores); {\\it white} (water ice); {\\it blue} (liquid water); {\\it brown} (anhydrous rock); and {\\it olive} (hydrated rock). The animation shows the differentiation of an initially homogeneous body into a hydrous rocky core underlying a thin ice-enriched crust, from which the ice subsequently sublimates.}\n\\end{center}\n\\end{figure}\n\n\\begin{figure}\n\t\\begin{center}{\\label{fig:3-12-4-3}\\includegraphics[scale=1]{movie2.eps}}\n\t\\caption{Animated figure (duration - 22 s) featuring the first 1.7 Myr of evolution (from a total of 470 Myr) of a 100 km radius minor planet around a 3 M$_{\\odot}$ progenitor at 12 AU, with a 4 Myr formation time and 75\\% initial rock fraction. Colour interpretation: {\\it black} (pores); {\\it white} (water ice); {\\it blue} (liquid water); {\\it brown} (anhydrous rock); and {\\it olive} (hydrated rock). The animation shows the differentiation of an initially homogeneous body into a smaller hydrous rocky core underlying a thicker ice-enriched crust (both compared to Fig. \\ref{fig:2-7-3-3}). Here the inside-out migration of water and outside-in sublimation of water ice from the mantle, occur on similar time scales.}\n\\end{center}\n\\end{figure}\n\n\\begin{figure}\n\t\\begin{center}{\\label{fig:6.4-5-5-2}\\includegraphics[scale=1]{movie3.eps}}\n\t\\caption{Animated figure (duration - 21 s) featuring the full 64 Myr evolution of a 100 km radius minor planet around a 6.4 M$_{\\odot}$ progenitor at 5 AU, with a 5 Myr formation time and 67\\% initial rock fraction. Colour interpretation: {\\it black} (pores); {\\it white} (water ice); {\\it blue} (liquid water); {\\it brown} (anhydrous rock); and {\\it olive} (hydrated rock). The animation shows the gradual ablation of ice from the surface inwards. This body never differentiates, and its host star's evolution is too short to remove all its internal ice.}\n\\end{center}\n\\end{figure}\n\nAnother trend in the data suggests that while more massive stars have a shorter lifetime, and a higher final/initial mass ratio (hence minor planets experience more orbital expansion), their high luminosities clearly dominate the fate of water. We define the outer bound of water retention as the distance beyond which the initial amount of water is fully retained. Examining the smallest minor planet in our sample (radius=1 km), it is immediately evident from these plots that the outer bound of water retention increases steadily with progenitor mass. \n\nThe inner bound of water retention is similarly defined as the minimal distance above which water retention is greater than zero. The behaviour of the inner bound of water retention is a bit more complex compared to the former, as illustrated by Figs. \\ref{fig:Masses} and \\ref{fig:RetentionVsMass}, however the general trend is also that the inner bound increases with progenitor mass. In 1 M$_{\\odot}$ progenitors, the exact fraction of water retained at the inner bound (3 AU) as a result of silicate hydration can be much larger than zero, and it depends on the precise parameters of the minor planet. More massive progenitors than about 2 M$_{\\odot}$ already cannot retain any water at 3 AU, which only illustrates the importance of progenitor mass. \n\n\\begin{figure}\n\t\\begin{center}\n\t\t\\subfigure[Iron abundance {[Fe/H]}=-1 - 0.61 M$_{\\odot}$ WD - 916 Myr evolution]{\\label{fig:2lm}\\includegraphics[scale=0.5]{2lm.eps}}\n\t\t\\subfigure[Iron abundance {[Fe/H]}=-2 - 0.66 M$_{\\odot}$ WD - 767 Myr evolution]{\\label{fig:2vlm}\\includegraphics[scale=0.5]{2vlm.eps}}\n\t\t\\subfigure{\\includegraphics[scale=0.5]{legend.eps}}\n\t\\end{center}\n\t\\caption{Total fraction of water (ice + water in hydrated silicates) remaining after the main sequence, RGB and AGB stellar evolution phases, for a 2 M$_{\\odot}$ progenitor with reduced iron abundances. The retention of water is shown as a function of the minor planet's initial orbital distance, composition, radius and formation time.}\n\t\\label{fig:Metallicities}\n\\end{figure}\n\nWe also investigate the effect of metallicity on water retention. We limit the investigation to 2 M$_{\\odot}$ progenitor stars, which approximately correspond to the peak mass in the WD mass distribution (see Section \\ref{S:Intro}). In Fig. \\ref{fig:Metallicities} we investigate their water retention for [Fe/H]=-1, and [Fe/H]=-2, that is, one order of magnitude and two orders of magnitude reduction in metallicity compared with Panel \\ref{fig:2sm} of Fig. \\ref{fig:Masses}. Clearly the progenitor metallicity is of lesser importance than the progenitor mass. For example, in Panel \\ref{fig:2vlm} the final WD mass is 0.66 M$_{\\odot}$, more than but similar to the resulting WD mass in Panel \\ref{fig:3sm} (0.65 M$_{\\odot}$), and the evolution time of the former is longer than that of the latter. So while an extremely metal-poor progenitor can easily result in a comparable WD mass to a considerably more massive progenitor, it nevertheless does not affect water retention nearly as much, despite having a longer lifetime. \n\n\n4 Discussion\n\\section{Discussion}\\label{S:discussion}\nIn this study we investigate a wide range of progenitor masses, relevant to G, F, A and B type stars, and also a range of metallicities, from solar abundance down to 10$^{-2}$ solar abundance. The results in Section \\ref{S:Results} indicate that the progenitor mass is inversely correlated with water retention, and that water-bearing planetary systems around very massive stars in the range 5-6.4 M$_{\\odot}$, if in fact they exist, cannot retain much of their water even to Kuiper Belt distances. Less massive progenitors in the mass range 3-3.6 M$_{\\odot}$ result in intermediate water retention in their planetary systems. The innermost minor planets that retain water do not, however, include large quantities of water in the form of hydrated silicates (at least up to the size investigated in our sample). The least massive, yet most common WD progenitors in the mass range 1-2 M$_{\\odot}$, allow for the highest degree of water retention in their respective systems, including plenty of hydrated silicates in the interiors of large minor planets at all distances. This is particularly true for 1 M$_{\\odot}$ that can retain hydrated silicates even down to a minimal initial orbital distance of 3 AU.\n\n\\begin{figure*}\n\t\\begin{center}\n\t\t\\subfigure[Minor planet radius = 1 km]{\\label{fig:fin1}\\includegraphics[scale=0.5]{FinalVsWD1.eps}}\n\t\t\\subfigure[Minor planet radius = 5 km]{\\label{fig:fin5}\\includegraphics[scale=0.5]{FinalVsWD5.eps}}\n\t\t\\subfigure[Minor planet radius = 25 km]{\\label{fig:fin25}\\includegraphics[scale=0.5]{FinalVsWD25.eps}}\n\t\t\\subfigure[Minor planet radius = 50 km]{\\label{fig:fin50}\\includegraphics[scale=0.5]{FinalVsWD50.eps}}\n\t\t\\subfigure[Minor planet radius = 100 km]{\\label{fig:fin100}\\includegraphics[scale=0.5]{FinalVsWD100.eps}}\n\t\\end{center}\n\t\\caption{Water retention as a function of WD mass. The top and bottom solid lines in each panel mark the outer and inner water retention bounds respectively, expressed in terms of the minor planet's final orbital distance. Above the top solid line minor planets retain their original water content, and below the bottom solid line no water survives. The grey area in between the solid lines represents partial water retention, where 50$\\%$ water retention is marked by the dashed line.}\n\t\\label{fig:RetentionVsMass}\n\\end{figure*}\n\nFig. \\ref{fig:RetentionVsMass} shows the dependence of water retention (in terms of the final orbital distance of a minor planet, that is, after orbital expansion) on the WD mass. Panels \\ref{fig:fin1}-\\ref{fig:fin100} refer to minor planets of increasing size. Note that here we consider approximate mean values, averaging over different formation times and compositions, given same-size minor planets. Our analysis of different progenitor masses clearly indicates that there is a water retention outer bound, a distance beyond which water is fully retained. Similarly, the water retention inner bound is the minimal distance above which water retention is greater than zero. Here we find both the outer and inner bounds (top and bottom solid lines respectively) to generally increase with WD mass. The only exception is for the inner bound in Panel \\ref{fig:fin100} which is shown to decrease for very massive WDs, however this is only true for the \\emph{mean} value, whereas we recall that icy minor planets with short distance orbits and long formation times are not very likely.\n\t\nRemarkably, in very massive WDs, one expects to find comets with full or even partial water retention only at distances of thousands, or hundreds of AU, respectively. This find has implications for WD pollution, since any delivery mechanism for these exo-comets onto the WD \\citep{VerasEtAl-2014b,StoneEtAl-2015,CaiazzoHeyl-2017} has to be compatible with such great distances. Much larger minor planets (radius $>$ 25 km), primarily around WDs that are 0.6 M$_{\\odot}$ or less, can retain a large fraction of their water at relatively close distances (e.g., the 50$\\%$ retention dashed line is approximately less than $\\sim$100 AU given these conditions), which could also be significant in constraining their delivery mechanism \\citep{DebesEtAl-2012,BonsorEtAl-2011,FrewenHansen-2014,DebesSigurdsson-2002,MustillEtAl-2014,VerasGansicke-2015,VerasEtAl-2016,PayneEtAl-2016,PetrovichMunoz-2016,KratterPerets-2012,PeretsKratter-2012,ShapeeThompson-2013,MichaelyPerets-2014,HamersPortegiesZwart-2016}. \n\nGiven these results, we predict a marked difference in the water pollution of WDs of various masses. Minor planets around less massive WDs should be able to retain more water, and specifically at closer distances to the WD. If everything else is equal, both of these results imply that low-mass progenitor stars should have a higher rate of water-bearing minor planets perturbed onto the WD. This effect might be detectable in the future. For example, note Fig. 5 in \\cite{RaddiEtAl-2015} which plots the total mass of hydrogen in the convection zones of helium-dominated white dwarfs as function of T$_{eff}$, or cooling age. One of the possible interpretations for the increase in trace hydrogen with cooling age is the long-term accretion of water-bearing minor planets that contribute hydrogen which does not diffuse out of the WD atmosphere. If this interpretation is correct, then in the future (as more data points become available) a richer statistics might enable us to identify different tracks along this plot, for various WD masses. I.e., we would expect a steeper slope for tracks that bin low-mass WDs.\n\nFinally we show that compared to the variations in progenitor mass, even a two order of magnitude reduction in stellar metallicity results in much smaller differences in water retention. We conclude that future studies should be less concerned with the direct effect of metallicity on water retention.\n\n",
        "label": "fig:RetentionVsMass",
        "Descriptive_question1": "What do the top and bottom solid lines represent in figure_6?",
        "Descriptive_question2": "What does the dashed line indicate in figure_6?",
        "Reasoning_question1": "How does the water retention behavior change with increasing WD mass in figure_6, and what might be the underlying reasons for this trend?",
        "Reasoning_question2": "Why might the inner bound of water retention decrease for very massive WDs in the panel for 100 km radius minor planets in figure_6, despite the general trend of increasing bounds with WD mass?",
        "Descriptive_answer1": "outer and inner water retention bounds",
        "Descriptive_answer2": "50% water retention",
        "Reasoning_answer1": "As WD mass increases, both the outer and inner water retention bounds in figure_6 generally move outward, meaning that minor planets need to be at greater distances from the WD to retain their water. This trend likely arises because more massive WDs originate from more luminous progenitor stars which produce greater heating, thereby sublimating or depleting water at closer distances. Hence, higher WD mass corresponds to harsher environments resulting in less water retention at smaller orbital distances.",
        "Reasoning_answer2": "The decrease in the inner bound of water retention for very massive WDs in the 100 km radius panel may be due to the complex thermal evolution of larger minor planets. Larger objects experience higher internal temperatures early on, potentially causing dehydration of hydrated rock. However, for very massive WDs, the rapid progression to the WD phase limits the time heat can penetrate the deepest regions, allowing some water retention in inner cores even at relatively closer distances. Thus, the interplay between short stellar lifetimes, large minor planet size, and internal thermal diffusion can cause this exception to the typical trend."
    },
    {
        "paper_id": "2107.07634.json",
        "image_id": "figure_4",
        "image_path": [
            "/home/yz979/scratch/chengye/scimolmo/test-images-tables/dataset_imagestables/2107.07634/images/det_curve_checker_edc_v2.png"
        ],
        "caption": "DET curves for structured evaluation set. The vertical dotted line indicates an operating point.",
        "classify": "Chart",
        "section_info": "4 Experimental evaluation\n\\section{Experimental evaluation}\n\\label{sec:exp}\n\n\n\nWe evaluated the effectiveness of the proposed approach on a KWS task, and compared its performance with a self-attention phonetic decoder with/without the conventional multi-task learning and a BLSTM decoder with the conventional multi-task learning. Although we used our internal datasets in experiments,  our proposed approach is easily applicable to any public ASR and KWS datasets.\n\n\\subsection{Data}\nOur ASR training data consisted of approximately 3 million utterances of transcribed near-field speech signals recorded with devices such as smart phones. Then data augmentation was performed by convolving room impulse responses (RIRs) with speech signals. The RIRs were recorded in various rooms with smart speakers with six microphones. Additionally, echo residuals were added to the augmented data. As a result, we obtained approximately 9 million augmented utterances consisting of the near-field signals, simulated far-field signals, and simulated far-field signals with the echo residuals. The KWS data consisted of approximately $65k$ false triggers and $300k$ true triggers spoken by anonymous speakers, which were triggered by a reference voice triggering system. The audio signals were recorded with smart speakers. The KWS data were combined with the augmented ASR dataset, and utterances were randomly sampled from the combined dataset for mini-batch training.\n\nFor evaluation, we used two different datasets. The first  is a \\emph{structured} dataset, where positive samples containing a keyword phrase were internally collected in controlled conditions from 100 participants, approximately evenly divided between males and females. Each subject spoke the keyword phrase followed by prompted voice commands to a smart speaker. The recordings were made in four different acoustic conditions: quiet, external noise from TV or kitchen appliances, music playing from the device at medium volume, and music playing at loud volume. 13000 such positive utterances were collected. For negative data, we used a set of 2000 hours of audio recordings which did not contain the keyword phrase by playing podcasts, audiobooks, TV play-back, etc. These negative audio samples were also recorded by the same smart speaker. The negative audio data allowed us to compute false accept (FA) per hour. The second dataset, called take home evaluation set, is a more realistic and challenging  dataset collected at home by employees. Each of the 80 participants volunteered to use the smart speaker daily for two weeks. By applying extra audio logging on device and personal review by the user, audio below the usual on-device trigger threshold was collected. This setup allowed us to collect challenging negative data, which was similar to the keyword phrase. We collected 7896 positive and 20919 negative audio samples\\footnote{The amount of the dataset has been increased by additional participants compared to the evaluation dataset used in \\cite{adya2020hybrid} and \\cite{9053577}, so the result reported in this paper is not directly comparable.} for evaluation. This dataset allowed us to compute the absolute number of false accepts (FAs).\n\n\\subsection{Two-stage approach for efficient KWS}\n\\begin{figure}[t]\n  \\centering\n\n\n  \\includegraphics[scale=0.45]{TwoPassKWS_v2.png}\n  \\caption{A two-stage approach for efficient KWS \\cite{gruenstein2017cascade,sigtia2018}. A 1st pass light-weight KWS system is always-on and takes streaming audio signals, where a DNN-HMM system is used to obtain a KWS score and an alignment for an audio segment containing a keyword. Once the 1st pass KWS score exceeds a threshold,  the audio segment is passed to a bigger KWS model (so-called checker) and a KWS score is re-computed.}\n\n  \\label{fig:TwoPass}\n\\end{figure}\n\nWe used a two-stage approach for efficient KWS \\cite{gruenstein2017cascade,sigtia2018} as shown in figure \\ref{fig:TwoPass}. A light-weight model was always-on and first detected candidate audio segments from streaming audio inputs. Once the segments were detected, a bigger model (so-called checker) was turned on and checked if the segments actually contained the keyword phrase or not. This two-stage approach greatly reduces compute cost and battery consumption on-device. For the 1st pass model, we used five layers of fully-connected neural networks with 64 hidden units as the acoustic model.  We used 20 target classes for the acoustic model; 18 phoneme classes for the keyword, one for silence and one for other speech.  We computed a 13-dimensional MFCC feature at a rate of 100 frames per second,  and supplied 19 consecutive frames to the acoustic model.  The confidence scores for KWS and alignments to extract audio segments were obtained using an HMM.  Given keyword start and end times from the HMM alignment,  we used $(start~time - 0.5)$ seconds  and $(end~time + 0.3)$ seconds for segmentation to ensure that the segment contained the detected keyword portion. The 1st pass threshold was set to obtain approximately 21 FA/hr on the structured evaluation dataset. We used the same 1st pass system for all the experiments and evaluated the effectiveness of our proposed model as the checker.\n\n\n\\subsection{Model training}\nFor a baseline phoneme classifier, we used a self-attention based acoustic model. The model consisted of 6 layers of Transformer blocks, each of which had a multi-head self-attention layer with 256 hidden dimension and 4 heads, followed by a feedforward neural network with 1024 hidden units. Finally, outputs from the Transformer blocks were projected to 54-dimensional logits for phonetic and blank labels by a linear layer. The baseline model was trained with the CTC loss\\footnote{In \\cite{adya2020hybrid}, the vanilla Transformer decoder was also trained along with the self-attention encoder using cross entropy loss, and used as a regularizer during training. We omitted the regularization just because of simplicity in our experiments. The regularization can be applied to all the approaches in our experiments including the proposed approach.}. The same architecture was also used for the conventional multi-task learning \\cite{9053577} by splitting the last layer into 54 outputs for the phonetic CTC loss and three discriminative outputs for a positive class, a negative class and a blank label for the phrase level CTC loss. Regarding the proposed approach, we used the same self-attention phoneme classifier for the phonetic encoder. The cross attention decoder consisted of a Transformer decoder block (i.e., $P=1$) which had the same configuration as the Transformer blocks of the encoder except the cross attention block. The dimension of the query vector and the length of the query sequence were set at 256 and 4, respectively. The last linear layer projected the reshaped $1024 (256\\times4)$-dimensional vector to two logits for positive and negative classes. The encoder and the decoder were jointly trained using the phonetic CTC loss and the phrase level cross entropy loss (see Section \\ref{sec:proposed}). We also explored a BLSTM decoder by replacing the cross attention decoder by a layer of BLSTMs with 256 hidden units followed by a linear layer which processed a concatenated BLSTM outputs at the first and last frame to predict logits. The scaling factor $\\alpha$ in Eq. (\\ref{eq:MTL}) for the multi-task learning was experimentally set at $10$. $40$-dimensional log mel-filter bank features $\\pm$ 3 context frames were used as inputs. In addition, we sub-sampled the features once per three frames to reduce computational complexity.\n\n All models were trained using the Adam optimizer \\cite{kingma2014adam}. The learning rate was first increased linearly to $0.0008$ until epoch $2$, then linearly decayed to $0.00056$ until epoch $16$. Finally the learning rate was exponentially decreased until the last epoch which was set at $28$. We used 16 GPUs for training and the batch size was 128 at each GPU. \n\n\n\n\n\n\\subsection{Results}\n\n\\begin{figure}[t]\n  \\centering\n\n\n  \\includegraphics[width=\\linewidth]{det_curve_checker_edc_v2.png}\n  \\caption{DET curves for structured evaluation set. The vertical dotted line indicates an operating point.}\n\n  \\label{fig:det_edc}\n\\end{figure}\n\n\\begin{figure}[t]\n  \\centering\n\n\n  \\includegraphics[width=\\linewidth]{det_curve_checker_thk_v2.png}\n  \\caption{DET curves for take home evaluation set. The vertical dotted line indicates an operating point.}\n\n  \\label{fig:det_thk}\n\\end{figure}\n\n\\begin{table*}[t]\n  \\caption{False reject ratios for structured evaluation set [$\\%$] at an operating point of 1 FA/100 hrs, and for take home evaluation set at an operating point of 100 FAs.}\n\n  \\label{tab:FRRs}\n   \\centering\n\n\\begin{tabular}{cccccc}\n  \\toprule\n &  MTL & Branch & Structured evaluation set&  Take home evaluation set & Avg.\\\\\n  \\midrule\n Phoneme classifier &  &Phonetic& 20.26 & 27.72 & 23.99\\\\ \\midrule\n Conventional MTL \\cite{9053577}& \\checkmark &\\begin{tabular}[c]{@{}c@{}}Phonetic\\\\ Phrase\\end{tabular}& \\begin{tabular}[c]{@{}c@{}}5.00 \\\\\\textbf{3.49}\\end{tabular}& \\begin{tabular}[c]{@{}c@{}}14.11 \\\\10.11\\end{tabular} &\\begin{tabular}[c]{@{}c@{}}9.56 \\\\6.80\\end{tabular}\\\\ \\midrule\n BLSTM decoder & \\checkmark &\\begin{tabular}[c]{@{}c@{}}Phonetic\\\\ Phrase\\end{tabular}& \\begin{tabular}[c]{@{}c@{}}5.02 \\\\4.76\\end{tabular} & \\begin{tabular}[c]{@{}c@{}}12.36 \\\\8.89\\end{tabular} &\\begin{tabular}[c]{@{}c@{}} 8.69\\\\6.83\\end{tabular}\\\\ \\midrule\n Cross attention decoder & \\checkmark &\\begin{tabular}[c]{@{}c@{}}Phonetic\\\\ Phrase\\end{tabular}& \\begin{tabular}[c]{@{}c@{}}4.64 \\\\3.82\\end{tabular} & \\begin{tabular}[c]{@{}c@{}}13.21 \\\\\\textbf{8.17}\\end{tabular} &\\begin{tabular}[c]{@{}c@{}} 8.93\\\\\\textbf{6.00}\\end{tabular}\\\\\n     \\bottomrule\n\\end{tabular}\n\n\\end{table*}\n\n\n\nFigures \\ref{fig:det_edc} and \\ref{fig:det_thk} show detection error tradeoff (DET) curves for all models evaluated on the structured evaluation dataset and take home evaluation dataset, respectively. The horizontal axis represents FA/hr for the structured dataset or the  absolute number of FAs for take home dataset. The vertical axis represents FRRs. Table \\ref{tab:FRRs} shows FRRs obtained with the baseline and proposed models at operating points. In the case of multi-task learning, results from both the phonetic and  phrase branches were reported. First, multi-task learning significantly improved the FRRs compared to the phoneme classifier which was trained only on the ASR data. This result shows the effectiveness of using both the ASR and the KWS data for KWS model training. Second, the phrase branch always yielded better results than the phonetic branch, presumably because the phrase branch was directly optimized for the target task. Note that although the performance of the phonetic branch was not as good as the phrase branch, the phonetic branch has an advantage of flexibility where the keyword phrase is configurable at test time.\n\n Lastly, the proposed cross attention decoder with the phrase branch yielded the best performance and achieved a $12\\%$ relative reduction in the FRRs  compared to the conventional multi-task learning and the BLSTM decoder. The cross attention decoder has another advantage over the BLSTM decoder, which is less training time and less runtime cost as reported in \\cite{adya2020hybrid}.\n\nEven though the proposed decoder can effectively learn from the KWS training data\\footnote{Cross validation loss with the conventional multi-task learning was $1.5\\times$ higher than the loss with the cross attention decoder.}, the proposed approach with the phrase branch did not outperform the conventional multi-task learning for the structured evaluation set. This performance degradation could be because of mismatched conditions/distributions between the KWS training data and the structured evaluation dataset that was recorded in the controlled conditions.\n\n4.4 Results\n\\subsection{Results}\n\n\\begin{figure}[t]\n  \\centering\n\n\n  \\includegraphics[width=\\linewidth]{det_curve_checker_edc_v2.png}\n  \\caption{DET curves for structured evaluation set. The vertical dotted line indicates an operating point.}\n\n  \\label{fig:det_edc}\n\\end{figure}\n\n\\begin{figure}[t]\n  \\centering\n\n\n  \\includegraphics[width=\\linewidth]{det_curve_checker_thk_v2.png}\n  \\caption{DET curves for take home evaluation set. The vertical dotted line indicates an operating point.}\n\n  \\label{fig:det_thk}\n\\end{figure}\n\n\\begin{table*}[t]\n  \\caption{False reject ratios for structured evaluation set [$\\%$] at an operating point of 1 FA/100 hrs, and for take home evaluation set at an operating point of 100 FAs.}\n\n  \\label{tab:FRRs}\n   \\centering\n\n\\begin{tabular}{cccccc}\n  \\toprule\n &  MTL & Branch & Structured evaluation set&  Take home evaluation set & Avg.\\\\\n  \\midrule\n Phoneme classifier &  &Phonetic& 20.26 & 27.72 & 23.99\\\\ \\midrule\n Conventional MTL \\cite{9053577}& \\checkmark &\\begin{tabular}[c]{@{}c@{}}Phonetic\\\\ Phrase\\end{tabular}& \\begin{tabular}[c]{@{}c@{}}5.00 \\\\\\textbf{3.49}\\end{tabular}& \\begin{tabular}[c]{@{}c@{}}14.11 \\\\10.11\\end{tabular} &\\begin{tabular}[c]{@{}c@{}}9.56 \\\\6.80\\end{tabular}\\\\ \\midrule\n BLSTM decoder & \\checkmark &\\begin{tabular}[c]{@{}c@{}}Phonetic\\\\ Phrase\\end{tabular}& \\begin{tabular}[c]{@{}c@{}}5.02 \\\\4.76\\end{tabular} & \\begin{tabular}[c]{@{}c@{}}12.36 \\\\8.89\\end{tabular} &\\begin{tabular}[c]{@{}c@{}} 8.69\\\\6.83\\end{tabular}\\\\ \\midrule\n Cross attention decoder & \\checkmark &\\begin{tabular}[c]{@{}c@{}}Phonetic\\\\ Phrase\\end{tabular}& \\begin{tabular}[c]{@{}c@{}}4.64 \\\\3.82\\end{tabular} & \\begin{tabular}[c]{@{}c@{}}13.21 \\\\\\textbf{8.17}\\end{tabular} &\\begin{tabular}[c]{@{}c@{}} 8.93\\\\\\textbf{6.00}\\end{tabular}\\\\\n     \\bottomrule\n\\end{tabular}\n\n\\end{table*}\n\n\n\nFigures \\ref{fig:det_edc} and \\ref{fig:det_thk} show detection error tradeoff (DET) curves for all models evaluated on the structured evaluation dataset and take home evaluation dataset, respectively. The horizontal axis represents FA/hr for the structured dataset or the  absolute number of FAs for take home dataset. The vertical axis represents FRRs. Table \\ref{tab:FRRs} shows FRRs obtained with the baseline and proposed models at operating points. In the case of multi-task learning, results from both the phonetic and  phrase branches were reported. First, multi-task learning significantly improved the FRRs compared to the phoneme classifier which was trained only on the ASR data. This result shows the effectiveness of using both the ASR and the KWS data for KWS model training. Second, the phrase branch always yielded better results than the phonetic branch, presumably because the phrase branch was directly optimized for the target task. Note that although the performance of the phonetic branch was not as good as the phrase branch, the phonetic branch has an advantage of flexibility where the keyword phrase is configurable at test time.\n\n Lastly, the proposed cross attention decoder with the phrase branch yielded the best performance and achieved a $12\\%$ relative reduction in the FRRs  compared to the conventional multi-task learning and the BLSTM decoder. The cross attention decoder has another advantage over the BLSTM decoder, which is less training time and less runtime cost as reported in \\cite{adya2020hybrid}.\n\nEven though the proposed decoder can effectively learn from the KWS training data\\footnote{Cross validation loss with the conventional multi-task learning was $1.5\\times$ higher than the loss with the cross attention decoder.}, the proposed approach with the phrase branch did not outperform the conventional multi-task learning for the structured evaluation set. This performance degradation could be because of mismatched conditions/distributions between the KWS training data and the structured evaluation dataset that was recorded in the controlled conditions.\n\n",
        "label": "fig:det_edc",
        "Descriptive_question1": "What does the vertical dotted line represent in figure_4?",
        "Descriptive_question2": "What type of curves are shown in figure_4?",
        "Reasoning_question1": "Why might the cross attention decoder not have outperformed the conventional multi-task learning in figure_4 for the structured evaluation set?",
        "Reasoning_question2": "What trend can be inferred from the DET curves in figure_4 regarding the performance of different models on the structured evaluation set?",
        "Descriptive_answer1": "operating point",
        "Descriptive_answer2": "DET curves",
        "Reasoning_answer1": "The cross attention decoder with the phrase branch did not outperform the conventional multi-task learning for the structured evaluation set possibly due to mismatched conditions or distributions between the KWS training data and the controlled conditions under which the structured evaluation dataset was recorded. This mismatch could hinder the cross attention decoder's ability to generalize effectively to the structured dataset's conditions.",
        "Reasoning_answer2": "The DET curves in figure_4 suggest that multi-task learning improves performance over a phoneme classifier alone, and that the phrase branch generally yields better false reject ratios than the phonetic branch. Among the models using the phrase branch, the cross attention decoder shows the best performance, indicating it achieves lower false reject ratios at comparable false accept rates, thus highlighting its effectiveness as the checker model in the two-stage KWS setup."
    },
    {
        "paper_id": "2107.07634.json",
        "image_id": "figure_5",
        "image_path": [
            "/home/yz979/scratch/chengye/scimolmo/test-images-tables/dataset_imagestables/2107.07634/images/det_curve_checker_thk_v2.png"
        ],
        "caption": "DET curves for take home evaluation set. The vertical dotted line indicates an operating point.",
        "classify": "Chart",
        "section_info": "4 Experimental evaluation\n\\section{Experimental evaluation}\n\\label{sec:exp}\n\n\n\nWe evaluated the effectiveness of the proposed approach on a KWS task, and compared its performance with a self-attention phonetic decoder with/without the conventional multi-task learning and a BLSTM decoder with the conventional multi-task learning. Although we used our internal datasets in experiments,  our proposed approach is easily applicable to any public ASR and KWS datasets.\n\n\\subsection{Data}\nOur ASR training data consisted of approximately 3 million utterances of transcribed near-field speech signals recorded with devices such as smart phones. Then data augmentation was performed by convolving room impulse responses (RIRs) with speech signals. The RIRs were recorded in various rooms with smart speakers with six microphones. Additionally, echo residuals were added to the augmented data. As a result, we obtained approximately 9 million augmented utterances consisting of the near-field signals, simulated far-field signals, and simulated far-field signals with the echo residuals. The KWS data consisted of approximately $65k$ false triggers and $300k$ true triggers spoken by anonymous speakers, which were triggered by a reference voice triggering system. The audio signals were recorded with smart speakers. The KWS data were combined with the augmented ASR dataset, and utterances were randomly sampled from the combined dataset for mini-batch training.\n\nFor evaluation, we used two different datasets. The first  is a \\emph{structured} dataset, where positive samples containing a keyword phrase were internally collected in controlled conditions from 100 participants, approximately evenly divided between males and females. Each subject spoke the keyword phrase followed by prompted voice commands to a smart speaker. The recordings were made in four different acoustic conditions: quiet, external noise from TV or kitchen appliances, music playing from the device at medium volume, and music playing at loud volume. 13000 such positive utterances were collected. For negative data, we used a set of 2000 hours of audio recordings which did not contain the keyword phrase by playing podcasts, audiobooks, TV play-back, etc. These negative audio samples were also recorded by the same smart speaker. The negative audio data allowed us to compute false accept (FA) per hour. The second dataset, called take home evaluation set, is a more realistic and challenging  dataset collected at home by employees. Each of the 80 participants volunteered to use the smart speaker daily for two weeks. By applying extra audio logging on device and personal review by the user, audio below the usual on-device trigger threshold was collected. This setup allowed us to collect challenging negative data, which was similar to the keyword phrase. We collected 7896 positive and 20919 negative audio samples\\footnote{The amount of the dataset has been increased by additional participants compared to the evaluation dataset used in \\cite{adya2020hybrid} and \\cite{9053577}, so the result reported in this paper is not directly comparable.} for evaluation. This dataset allowed us to compute the absolute number of false accepts (FAs).\n\n\\subsection{Two-stage approach for efficient KWS}\n\\begin{figure}[t]\n  \\centering\n\n\n  \\includegraphics[scale=0.45]{TwoPassKWS_v2.png}\n  \\caption{A two-stage approach for efficient KWS \\cite{gruenstein2017cascade,sigtia2018}. A 1st pass light-weight KWS system is always-on and takes streaming audio signals, where a DNN-HMM system is used to obtain a KWS score and an alignment for an audio segment containing a keyword. Once the 1st pass KWS score exceeds a threshold,  the audio segment is passed to a bigger KWS model (so-called checker) and a KWS score is re-computed.}\n\n  \\label{fig:TwoPass}\n\\end{figure}\n\nWe used a two-stage approach for efficient KWS \\cite{gruenstein2017cascade,sigtia2018} as shown in figure \\ref{fig:TwoPass}. A light-weight model was always-on and first detected candidate audio segments from streaming audio inputs. Once the segments were detected, a bigger model (so-called checker) was turned on and checked if the segments actually contained the keyword phrase or not. This two-stage approach greatly reduces compute cost and battery consumption on-device. For the 1st pass model, we used five layers of fully-connected neural networks with 64 hidden units as the acoustic model.  We used 20 target classes for the acoustic model; 18 phoneme classes for the keyword, one for silence and one for other speech.  We computed a 13-dimensional MFCC feature at a rate of 100 frames per second,  and supplied 19 consecutive frames to the acoustic model.  The confidence scores for KWS and alignments to extract audio segments were obtained using an HMM.  Given keyword start and end times from the HMM alignment,  we used $(start~time - 0.5)$ seconds  and $(end~time + 0.3)$ seconds for segmentation to ensure that the segment contained the detected keyword portion. The 1st pass threshold was set to obtain approximately 21 FA/hr on the structured evaluation dataset. We used the same 1st pass system for all the experiments and evaluated the effectiveness of our proposed model as the checker.\n\n\n\\subsection{Model training}\nFor a baseline phoneme classifier, we used a self-attention based acoustic model. The model consisted of 6 layers of Transformer blocks, each of which had a multi-head self-attention layer with 256 hidden dimension and 4 heads, followed by a feedforward neural network with 1024 hidden units. Finally, outputs from the Transformer blocks were projected to 54-dimensional logits for phonetic and blank labels by a linear layer. The baseline model was trained with the CTC loss\\footnote{In \\cite{adya2020hybrid}, the vanilla Transformer decoder was also trained along with the self-attention encoder using cross entropy loss, and used as a regularizer during training. We omitted the regularization just because of simplicity in our experiments. The regularization can be applied to all the approaches in our experiments including the proposed approach.}. The same architecture was also used for the conventional multi-task learning \\cite{9053577} by splitting the last layer into 54 outputs for the phonetic CTC loss and three discriminative outputs for a positive class, a negative class and a blank label for the phrase level CTC loss. Regarding the proposed approach, we used the same self-attention phoneme classifier for the phonetic encoder. The cross attention decoder consisted of a Transformer decoder block (i.e., $P=1$) which had the same configuration as the Transformer blocks of the encoder except the cross attention block. The dimension of the query vector and the length of the query sequence were set at 256 and 4, respectively. The last linear layer projected the reshaped $1024 (256\\times4)$-dimensional vector to two logits for positive and negative classes. The encoder and the decoder were jointly trained using the phonetic CTC loss and the phrase level cross entropy loss (see Section \\ref{sec:proposed}). We also explored a BLSTM decoder by replacing the cross attention decoder by a layer of BLSTMs with 256 hidden units followed by a linear layer which processed a concatenated BLSTM outputs at the first and last frame to predict logits. The scaling factor $\\alpha$ in Eq. (\\ref{eq:MTL}) for the multi-task learning was experimentally set at $10$. $40$-dimensional log mel-filter bank features $\\pm$ 3 context frames were used as inputs. In addition, we sub-sampled the features once per three frames to reduce computational complexity.\n\n All models were trained using the Adam optimizer \\cite{kingma2014adam}. The learning rate was first increased linearly to $0.0008$ until epoch $2$, then linearly decayed to $0.00056$ until epoch $16$. Finally the learning rate was exponentially decreased until the last epoch which was set at $28$. We used 16 GPUs for training and the batch size was 128 at each GPU. \n\n\n\n\n\n\\subsection{Results}\n\n\\begin{figure}[t]\n  \\centering\n\n\n  \\includegraphics[width=\\linewidth]{det_curve_checker_edc_v2.png}\n  \\caption{DET curves for structured evaluation set. The vertical dotted line indicates an operating point.}\n\n  \\label{fig:det_edc}\n\\end{figure}\n\n\\begin{figure}[t]\n  \\centering\n\n\n  \\includegraphics[width=\\linewidth]{det_curve_checker_thk_v2.png}\n  \\caption{DET curves for take home evaluation set. The vertical dotted line indicates an operating point.}\n\n  \\label{fig:det_thk}\n\\end{figure}\n\n\\begin{table*}[t]\n  \\caption{False reject ratios for structured evaluation set [$\\%$] at an operating point of 1 FA/100 hrs, and for take home evaluation set at an operating point of 100 FAs.}\n\n  \\label{tab:FRRs}\n   \\centering\n\n\\begin{tabular}{cccccc}\n  \\toprule\n &  MTL & Branch & Structured evaluation set&  Take home evaluation set & Avg.\\\\\n  \\midrule\n Phoneme classifier &  &Phonetic& 20.26 & 27.72 & 23.99\\\\ \\midrule\n Conventional MTL \\cite{9053577}& \\checkmark &\\begin{tabular}[c]{@{}c@{}}Phonetic\\\\ Phrase\\end{tabular}& \\begin{tabular}[c]{@{}c@{}}5.00 \\\\\\textbf{3.49}\\end{tabular}& \\begin{tabular}[c]{@{}c@{}}14.11 \\\\10.11\\end{tabular} &\\begin{tabular}[c]{@{}c@{}}9.56 \\\\6.80\\end{tabular}\\\\ \\midrule\n BLSTM decoder & \\checkmark &\\begin{tabular}[c]{@{}c@{}}Phonetic\\\\ Phrase\\end{tabular}& \\begin{tabular}[c]{@{}c@{}}5.02 \\\\4.76\\end{tabular} & \\begin{tabular}[c]{@{}c@{}}12.36 \\\\8.89\\end{tabular} &\\begin{tabular}[c]{@{}c@{}} 8.69\\\\6.83\\end{tabular}\\\\ \\midrule\n Cross attention decoder & \\checkmark &\\begin{tabular}[c]{@{}c@{}}Phonetic\\\\ Phrase\\end{tabular}& \\begin{tabular}[c]{@{}c@{}}4.64 \\\\3.82\\end{tabular} & \\begin{tabular}[c]{@{}c@{}}13.21 \\\\\\textbf{8.17}\\end{tabular} &\\begin{tabular}[c]{@{}c@{}} 8.93\\\\\\textbf{6.00}\\end{tabular}\\\\\n     \\bottomrule\n\\end{tabular}\n\n\\end{table*}\n\n\n\nFigures \\ref{fig:det_edc} and \\ref{fig:det_thk} show detection error tradeoff (DET) curves for all models evaluated on the structured evaluation dataset and take home evaluation dataset, respectively. The horizontal axis represents FA/hr for the structured dataset or the  absolute number of FAs for take home dataset. The vertical axis represents FRRs. Table \\ref{tab:FRRs} shows FRRs obtained with the baseline and proposed models at operating points. In the case of multi-task learning, results from both the phonetic and  phrase branches were reported. First, multi-task learning significantly improved the FRRs compared to the phoneme classifier which was trained only on the ASR data. This result shows the effectiveness of using both the ASR and the KWS data for KWS model training. Second, the phrase branch always yielded better results than the phonetic branch, presumably because the phrase branch was directly optimized for the target task. Note that although the performance of the phonetic branch was not as good as the phrase branch, the phonetic branch has an advantage of flexibility where the keyword phrase is configurable at test time.\n\n Lastly, the proposed cross attention decoder with the phrase branch yielded the best performance and achieved a $12\\%$ relative reduction in the FRRs  compared to the conventional multi-task learning and the BLSTM decoder. The cross attention decoder has another advantage over the BLSTM decoder, which is less training time and less runtime cost as reported in \\cite{adya2020hybrid}.\n\nEven though the proposed decoder can effectively learn from the KWS training data\\footnote{Cross validation loss with the conventional multi-task learning was $1.5\\times$ higher than the loss with the cross attention decoder.}, the proposed approach with the phrase branch did not outperform the conventional multi-task learning for the structured evaluation set. This performance degradation could be because of mismatched conditions/distributions between the KWS training data and the structured evaluation dataset that was recorded in the controlled conditions.\n\n4.4 Results\n\\subsection{Results}\n\n\\begin{figure}[t]\n  \\centering\n\n\n  \\includegraphics[width=\\linewidth]{det_curve_checker_edc_v2.png}\n  \\caption{DET curves for structured evaluation set. The vertical dotted line indicates an operating point.}\n\n  \\label{fig:det_edc}\n\\end{figure}\n\n\\begin{figure}[t]\n  \\centering\n\n\n  \\includegraphics[width=\\linewidth]{det_curve_checker_thk_v2.png}\n  \\caption{DET curves for take home evaluation set. The vertical dotted line indicates an operating point.}\n\n  \\label{fig:det_thk}\n\\end{figure}\n\n\\begin{table*}[t]\n  \\caption{False reject ratios for structured evaluation set [$\\%$] at an operating point of 1 FA/100 hrs, and for take home evaluation set at an operating point of 100 FAs.}\n\n  \\label{tab:FRRs}\n   \\centering\n\n\\begin{tabular}{cccccc}\n  \\toprule\n &  MTL & Branch & Structured evaluation set&  Take home evaluation set & Avg.\\\\\n  \\midrule\n Phoneme classifier &  &Phonetic& 20.26 & 27.72 & 23.99\\\\ \\midrule\n Conventional MTL \\cite{9053577}& \\checkmark &\\begin{tabular}[c]{@{}c@{}}Phonetic\\\\ Phrase\\end{tabular}& \\begin{tabular}[c]{@{}c@{}}5.00 \\\\\\textbf{3.49}\\end{tabular}& \\begin{tabular}[c]{@{}c@{}}14.11 \\\\10.11\\end{tabular} &\\begin{tabular}[c]{@{}c@{}}9.56 \\\\6.80\\end{tabular}\\\\ \\midrule\n BLSTM decoder & \\checkmark &\\begin{tabular}[c]{@{}c@{}}Phonetic\\\\ Phrase\\end{tabular}& \\begin{tabular}[c]{@{}c@{}}5.02 \\\\4.76\\end{tabular} & \\begin{tabular}[c]{@{}c@{}}12.36 \\\\8.89\\end{tabular} &\\begin{tabular}[c]{@{}c@{}} 8.69\\\\6.83\\end{tabular}\\\\ \\midrule\n Cross attention decoder & \\checkmark &\\begin{tabular}[c]{@{}c@{}}Phonetic\\\\ Phrase\\end{tabular}& \\begin{tabular}[c]{@{}c@{}}4.64 \\\\3.82\\end{tabular} & \\begin{tabular}[c]{@{}c@{}}13.21 \\\\\\textbf{8.17}\\end{tabular} &\\begin{tabular}[c]{@{}c@{}} 8.93\\\\\\textbf{6.00}\\end{tabular}\\\\\n     \\bottomrule\n\\end{tabular}\n\n\\end{table*}\n\n\n\nFigures \\ref{fig:det_edc} and \\ref{fig:det_thk} show detection error tradeoff (DET) curves for all models evaluated on the structured evaluation dataset and take home evaluation dataset, respectively. The horizontal axis represents FA/hr for the structured dataset or the  absolute number of FAs for take home dataset. The vertical axis represents FRRs. Table \\ref{tab:FRRs} shows FRRs obtained with the baseline and proposed models at operating points. In the case of multi-task learning, results from both the phonetic and  phrase branches were reported. First, multi-task learning significantly improved the FRRs compared to the phoneme classifier which was trained only on the ASR data. This result shows the effectiveness of using both the ASR and the KWS data for KWS model training. Second, the phrase branch always yielded better results than the phonetic branch, presumably because the phrase branch was directly optimized for the target task. Note that although the performance of the phonetic branch was not as good as the phrase branch, the phonetic branch has an advantage of flexibility where the keyword phrase is configurable at test time.\n\n Lastly, the proposed cross attention decoder with the phrase branch yielded the best performance and achieved a $12\\%$ relative reduction in the FRRs  compared to the conventional multi-task learning and the BLSTM decoder. The cross attention decoder has another advantage over the BLSTM decoder, which is less training time and less runtime cost as reported in \\cite{adya2020hybrid}.\n\nEven though the proposed decoder can effectively learn from the KWS training data\\footnote{Cross validation loss with the conventional multi-task learning was $1.5\\times$ higher than the loss with the cross attention decoder.}, the proposed approach with the phrase branch did not outperform the conventional multi-task learning for the structured evaluation set. This performance degradation could be because of mismatched conditions/distributions between the KWS training data and the structured evaluation dataset that was recorded in the controlled conditions.\n\n",
        "label": "fig:det_thk",
        "Descriptive_question1": "What does the vertical dotted line represent in figure_5?",
        "Descriptive_question2": "What is plotted on the vertical axis of figure_5?",
        "Reasoning_question1": "How does the performance of the cross attention decoder compare to other models at the operating point in figure_5?",
        "Reasoning_question2": "What trend can be observed in figure_5 regarding the trade-off between false accepts and false rejects for the take home evaluation set?",
        "Descriptive_answer1": "operating point",
        "Descriptive_answer2": "FRRs",
        "Reasoning_answer1": "At the operating point indicated by the vertical dotted line, the cross attention decoder shows the lowest false reject ratios among the models presented, indicating the best performance for the take home evaluation set. This is supported by the table data where the cross attention decoder with the phrase branch has a false reject ratio of 8.17%, which is lower than the conventional multi-task learning and BLSTM decoder values.",
        "Reasoning_answer2": "Figure_5 shows a clear trade-off pattern where as the absolute number of false accepts increases along the horizontal axis, the false reject ratios decrease on the vertical axis. This indicates that allowing more false accepts reduces the chance of missing true keywords, illustrating the typical detection error trade-off. The different models follow this trend, but the cross attention decoder achieves a lower false reject ratio for the same number of false accepts, demonstrating better trade-off performance."
    },
    {
        "paper_id": "1908.06383.json",
        "image_id": "figure_2",
        "image_path": [
            "/home/yz979/scratch/chengye/scimolmo/test-images-tables/dataset_imagestables/1908.06383/images/fig02.eps"
        ],
        "caption": "Ladders of zeroes  plotted according to the leading terms of  expansion (\\ref{3.4}). \t(a) Accumulation of zeroes (corresponding to eigenvalues) to the segment $[-\\Tht(\\g),\\Tht(\\g)]$ as the gain-to-loss distance $\\ell$ grows. Here the gain-and-loss amplitude $\\gamma=40$ and $\\ell=50$ (curve 1), 100 (curve 2) and  150 (curve 3). The zeroes are shown with crosses; thin solid lines are to guide the eye. (b) Transformation of the ladder of eigenvalues (curve 1) to a ladder of resonances (curve 3) under the change of the gain-and-loss amplitude. Here $\\ell=100$ and $\\gamma=40$ (curve 1) and $\\gamma = 50$ (curve 3). Curve 2 corresponds to $\\gamma=9\\pi^2/2\\approx  44.413$. In this case the imaginary part of term $\\propto \\ell^{-3}$ in  expansion (\\ref{3.4}) vanishes, and we obtain a ladder of coexisting ``nearly spectral singularities'' with Im$\\,k=O(\\ell^{-5})$.",
        "classify": "Chart",
        "section_info": "4 Behavior of zeroes and ladders of resonances and eigenvalues\n\\section{\nBehavior of zeroes and ladders of resonances and eigenvalues}\n\\label{sec:zeros}\n\nIn this section we employ the results of the previous section to describe how the zeroes of the function $F$ depend  on $\\g$ and $\\ell$.\n\n\\subsection{Location of zeroes and dependence of gain-and-loss amplitude}\n\nAccording Remark~\\ref{rm1}, as $\\g=0$, there is a single zero $k=0$. It corresponds to a resonance and the associated solution to equation (\\ref{2.1}) is the constant function. As soon as $\\g$ is positive, no matter how small it is, there arise {\\sl infinitely many} zeroes of the function $F$, see Lemma~\\ref{lm3.1}, and we still have a resonance at $k=0$ but now the associated solution of equation (\\ref{2.1}) is non-constant. Despite   \\textcolor{black}{the potential $V$ is a {\\sl regular} perturbation as $\\g$ is small}, it \\textcolor{black}{influences}   the general spectral picture quite essentially producing at once infinitely many zeroes. All these zeroes are located symmetrically with respect to the imaginary axis and are continuous in $\\g$ and $\\ell$. For the corresponding eigenvalues and resonances given by $\\l=k^2$, this property   means that we deal with complex-conjugate pairs of resonances and eigenvalues, as it is usual for $\\mathcal{PT}$-symmetric systems.\n\nBy Lemma~\\ref{lm3.2}, \nfor small $\\g$, we have one more pure imaginary zero close to $k=0$. This zero is holomorphic in $\\g$ and the leading terms of its Taylor series are given by (\\ref{3.1}).\nSince $(\\ell+\\tfrac{1}{3})\\g>0$, this zero corresponds to a resonance. The corresponding value $\\l$\nis negative and   \\textcolor{black}{lies close or next to the bottom} of the essential spectrum.\n\n\nLemmata~\\ref{lm3.1} and~\\ref{lm3.3} state the following important facts. The zeroes can accumulate at infinity only, that is, each circle of a fixed radius in the complex plane contains only finitely many zeroes. Except for zeroes in the circle $|k|<\\sqrt{\\g} r$, where $r$ is introduced in (\\ref{3.5a}), all other zeroes are located in the upper half-plane in the exponential sector\n\\begin{equation*}\n\\sqrt{\\g}e^{(\\ell+1)\\IM k}<|k|<\\frac{47}{25}\n \\sqrt{\\g}e^{(\\ell+1)\\IM k},\\qquad |k|<\\frac{76}{73}\\g e^{(\\ell+\\frac{9}{8})|k|}.\n\\end{equation*}\nThe second inequality deserves a special consideration. Let $\\g$ be small enough and consider the zeroes outside the circumference $|k|=\\sqrt{\\g} r$. Then the inequality implies\n\\begin{equation*}\n\\frac{39}{20}<\\frac{76}{73}\\g^\\frac{1}{2} e^{(\\ell+\\frac{9}{8})|k|}\\qquad \\Rightarrow\\qquad |k|>\\frac{1}{\\ell+\\frac{9}{8}}\\ln \\frac{2847\\g^{-\\frac{1}{2}}}{1520}\n\\end{equation*}\nand hence,  except the zeroes discussed in Lemma~\\ref{lm3.2}, all other zeroes are located far from $k=0$ at distances of order $O(|\\ln\\g|)$. This means that for small $\\g$, infinitely many zeroes emerge from infinity in the upper half-plane and only two of them come from zero and   are  located in the upper half-plane, too.\n\nSince all zeroes with $\\IM k \\leq 0$   are located in the aforementioned circle $|k|<\\sqrt{\\g} r$,   for all   $\\g>0$ and $\\ell\\geqslant 0$, there are only {\\sl finitely many} such zeroes.  They \ncorrespond to  the eigenvalues (bound states), and for them and inequality (\\ref{3.0b}) should be satisfied as well.  All eigenvalues   are complex-valued since according Lemma~\\ref{lm3.5}, there are no pure imaginary zeroes in lower complex half-plane.  Real zeroes \ncorrespond to spectral singularities. Thus,  for all $\\ell\\geqslant 0$ and $\\g>0$ the system has infinitely many resonances  and   finitely many bound states and spectral singularities.    The size of the aforementioned circle, in which the zeroes with  $\\IM k \\leq 0$  are located, increases proportionally to $\\g$ (see the definition of $r$ in (\\ref{3.5a})). Hence, by increasing $\\g$ we have more chances to get the zeroes in the lower half-plane and as we shall discuss below, this is indeed the case.\n\n\n\\subsection{Large distance regime and ladders of eigenvalues and resonances}\\label{sec:ladders}\n\nHere we discuss   the behavior of the zeroes  as $\\ell$ grows. Lemma~\\ref{lm3.7} states that  below the line $\\IM k=-\\frac{\\ln \\frac{\\ell\\sqrt{C_1}}{\\sqrt{C_2}}}{2(1-\\ln 2)\\ell}$, the function $F$ possesses the same number zeroes as the functions $F_-$ and $F_+$ do and these  zeroes of $F$ converge to the zeroes of $F_\\pm$ as $\\ell\\to+\\infty$, see (\\ref{3.27}). As $\\ell\\to+\\infty$, the distance from the aforementioned line to the real axis is of order $O(\\ell^{-1}\\ln\\ell)$ and is small. A natural question whether there can be  some other zeroes in the lower half-plane above this line is answered in Lemma~\\ref{lm3.4} and the answer is positive. According to this lemma,  making $\\ell$ large enough, we \\textcolor{black}{certainly} have $2N+1$ zeroes in the vicinity of the real axis, where $N:=\\lfloor \\tfrac{1}{2}+\\tfrac{2\\Tht(\\g)}{\\pi}\\ell\\rfloor$, $\\lfloor\\cdot\\rfloor$ is the integer part of a number. All these zeroes are located in the circles\n$\\big\\{k:\\, |k-\\tfrac{\\pi n}{2\\ell}|<\\tfrac{\\pi}{4\\ell}\\big\\}$, exactly one zero in each circle, and all these circles are inside a fixed circle $\\{k:\\,|k|<\\Tht(\\g)\\}$. As $\\ell$ grows, the number  of such zeroes increases as well but all of them \\textcolor{black}{remain} in the circle $\\{k:\\,|k|<\\Tht(\\g)\\}$. As $\\g$ grows, the size of the latter circle increases as $O(\\sqrt{\\g})$.   For large $\\ell$, these zeroes are approximately given by asymptotic formulae (\\ref{3.4}). As these formulae show, if $\\sin\\sqrt{2\\g}<0$, the zeroes are located in the upper half-plane and correspond to  resonances.  The described behavior is similar to that of Wannier-Stark resonances. Namely, for gain-and-loss amplitude we again have a ladder of resonances accumulating in the vicinity certain zone in the essential spectrum.  As $\\sin\\sqrt{2\\g}>0$, these zeroes are in the lower half-plane and correspond to eigenvalues. The   number of these zeroes  guaranteed by  Lemma~\\ref{lm3.4}  grows as $O(\\ell)$ as $\\ell\\to+\\infty$, while the  distances between the neighbouring zeroes are of order $O(\\ell^{-1})$. This means that we have either resonances or eigenvalues accumulating to the segment $[-\\Tht(\\g),\\Tht(\\g)]$ on the real axis, see Figure~\\ref{fig:accum}(a,b). It is especially interesting to notice that by means of a continuous  change  of the gain-and-loss amplitude one can transform a ladder of resonances  to a ladder of eigenvalues or vice versa as shown in figure~\\ref{fig:accum}(b). Such a transformation is obviously not possible for  Wannier-Stark ladders   in  self-adjoint operators where complex eigenvalues cannt exist.  The peculiar regime  $\\sin\\sqrt{2\\g}=0$ corresponds to the situation when the leading order of imaginary part of expansion (\\ref{3.4}) vanishes [curve~2 in figure~\\ref{fig:accum}(b)]. A   delicate analysis shows that in this case the imaginary parts of zeroes described by  Lemma~\\ref{lm3.4} are  nonzero and  amount to  $O(\\ell^{-5})$. Thus, for $\\sin\\sqrt{2\\g}=0$ we have a ladder of coexisting  ``nearly spectral singularities'' with extremely small by nevertheless nonzero imaginary parts (genuine spectral singularities with exactly zero imaginary parts  will be discussed in section~\\ref{sec:ss}).\n\n\\begin{figure}\n\t\\centering\n\\includegraphics[width=0.99\\columnwidth]{fig02.eps}\n\\caption{Ladders of zeroes  plotted according to the leading terms of  expansion (\\ref{3.4}). \t(a) Accumulation of zeroes (corresponding to eigenvalues) to the segment $[-\\Tht(\\g),\\Tht(\\g)]$ as the gain-to-loss distance $\\ell$ grows. Here the gain-and-loss amplitude $\\gamma=40$ and $\\ell=50$ (curve 1), 100 (curve 2) and  150 (curve 3). The zeroes are shown with crosses; thin solid lines are to guide the eye. (b) Transformation of the ladder of eigenvalues (curve 1) to a ladder of resonances (curve 3) under the change of the gain-and-loss amplitude. Here $\\ell=100$ and $\\gamma=40$ (curve 1) and $\\gamma = 50$ (curve 3). Curve 2 corresponds to $\\gamma=9\\pi^2/2\\approx  44.413$. In this case the imaginary part of term $\\propto \\ell^{-3}$ in  expansion (\\ref{3.4}) vanishes, and we obtain a ladder of coexisting ``nearly spectral singularities'' with Im$\\,k=O(\\ell^{-5})$.}\n\\label{fig:accum}\n\\end{figure}\n\n\nIt is interesting to compare this spectral picture with that in the limiting situation $\\ell=+\\infty$ and the first issue is which operator are to be regarded as limiting ones. Such limiting operator are to be treated in the norm resolvent sense, that is, we should find an approximation for the resolvent of our operator as $\\ell\\to+\\infty$.\nAccording the results of \\cite{UMJ},  as $\\ell\\to+\\infty$, the solution of the equation\n\\begin{equation*}\n-u''+Vu-\\l u=f\\quad\\text{in}\\quad\\mathds{R},\\quad u\\in H^2(\\mathds{R}),\\quad f(x):=f_-(x+\\ell)+f_0(x)+f_+(x-\\ell),\\quad f_0,\\,f_\\pm\\in L_2(\\mathds{R}),\n\\end{equation*}\nis approximated in the norm resolvent sense up to a small error by $u_-(x+\\ell)+u_0(x)+u_+(x-\\ell)$. Here $u_0, u_\\pm\\in H^2(\\mathds{R})$ solve the equations\n\\begin{equation*}\n-u_0''-\\l u_0=f_0,\\quad -u_\\pm''+V_\\pm u_\\pm -\\l u_\\pm=f_\\pm\\quad\\text{in}\\quad \\mathds{R},\n\\end{equation*}\nwhere the potentials $V_\\pm$ were introduced in (\\ref{3.23}).\nHence, the limiting operator is in fact a direct sum of three operators\n\n$\\Op_-\\oplus\\Op_0\\oplus\\Op_+$, $ \\Op_0:=-\\frac{d^2\\ }{dx^2}$,\n\n$\\Op_\\pm:=-\\frac{d^2\\ }{dx^2}+V_-$.\n\nThe essential spectra of all three operators are $[0,+\\infty)$. The operator $\\Op_0$ has the only resonance at $k=0$, while the resonances and the eigenvalues of the operators $\\Op_\\pm$ are expressed via the zeroes of the functions $F_\\pm$ by the formula $\\l=k^2$, see Lemma~\\ref{lm3.6}.  In view of such approximation, it would be natural to expect that as $\\ell\\to+\\infty$, the above discussed zeroes $k_n(\\ell,\\g)$   converge to similar real zeroes  of the functions $F_\\pm$ or to zero.\n{\\sl This is not true!} Indeed, by Lemma~\\ref{lm3.6}, each of the functions $F_\\pm$ has at most {\\sl one}  real zero, while the number of the zeroes $k_n(\\ell,\\g)$ in the vicinity of this segment {\\sl increases} as $\\ell\\to+\\infty$. Thus, regarding the case $\\ell=+\\infty$ and eigenvalues equations for the operators $\\Op_\\pm$ as limiting, for sufficiently large finite $\\ell$ we get a {\\sl large number} of {\\sl eigenvalues or resonances emerging from internal points} of the zone $[0,\\Tht^2(g)]$ in essential spectrum. And at most three points (including zero)  are spectral singularities for the limiting equations. In other words, here we have eigenvalues emerging from ordinary points in the essential spectrum not being spectral singularities for the unperturbed operators. And while in the case of Wannier-Stark resonances their existence is due to the eigenvalues of the operator $-\\frac{d^2\\ }{dx^2}+\\e x$ accumulating on the real line as $\\e\\to+0$, this surely not the case in our model.\n\n\n\nHere we just make one more important observation. The presence of the mentioned zero-width resonances, or real zeroes of $F$, could be regarded as an explanation or the reason for the existence of the above described ladder of resonances and eigenvalues. However, {\\sl this is not the case!} Indeed, for $\\frac{\\pi^2}{2} < \\gamma< \\gamma_*\\approx  11.561$, the function $F$ {\\sl does have} a ladder of resonances, but as we shall show in the next section, it {\\sl has no} real zeroes. So, our ladder of resonances or eigenvalues exists due to some other reasons and  the existence of the zero-width resonances is mostly the implication of the presence of such ladder and its behavior as $\\g$ varies.\n\n\n4.2 Large distance regime and ladders of eigenvalues and resonances\n\\subsection{Large distance regime and ladders of eigenvalues and resonances}\\label{sec:ladders}\n\nHere we discuss   the behavior of the zeroes  as $\\ell$ grows. Lemma~\\ref{lm3.7} states that  below the line $\\IM k=-\\frac{\\ln \\frac{\\ell\\sqrt{C_1}}{\\sqrt{C_2}}}{2(1-\\ln 2)\\ell}$, the function $F$ possesses the same number zeroes as the functions $F_-$ and $F_+$ do and these  zeroes of $F$ converge to the zeroes of $F_\\pm$ as $\\ell\\to+\\infty$, see (\\ref{3.27}). As $\\ell\\to+\\infty$, the distance from the aforementioned line to the real axis is of order $O(\\ell^{-1}\\ln\\ell)$ and is small. A natural question whether there can be  some other zeroes in the lower half-plane above this line is answered in Lemma~\\ref{lm3.4} and the answer is positive. According to this lemma,  making $\\ell$ large enough, we \\textcolor{black}{certainly} have $2N+1$ zeroes in the vicinity of the real axis, where $N:=\\lfloor \\tfrac{1}{2}+\\tfrac{2\\Tht(\\g)}{\\pi}\\ell\\rfloor$, $\\lfloor\\cdot\\rfloor$ is the integer part of a number. All these zeroes are located in the circles\n$\\big\\{k:\\, |k-\\tfrac{\\pi n}{2\\ell}|<\\tfrac{\\pi}{4\\ell}\\big\\}$, exactly one zero in each circle, and all these circles are inside a fixed circle $\\{k:\\,|k|<\\Tht(\\g)\\}$. As $\\ell$ grows, the number  of such zeroes increases as well but all of them \\textcolor{black}{remain} in the circle $\\{k:\\,|k|<\\Tht(\\g)\\}$. As $\\g$ grows, the size of the latter circle increases as $O(\\sqrt{\\g})$.   For large $\\ell$, these zeroes are approximately given by asymptotic formulae (\\ref{3.4}). As these formulae show, if $\\sin\\sqrt{2\\g}<0$, the zeroes are located in the upper half-plane and correspond to  resonances.  The described behavior is similar to that of Wannier-Stark resonances. Namely, for gain-and-loss amplitude we again have a ladder of resonances accumulating in the vicinity certain zone in the essential spectrum.  As $\\sin\\sqrt{2\\g}>0$, these zeroes are in the lower half-plane and correspond to eigenvalues. The   number of these zeroes  guaranteed by  Lemma~\\ref{lm3.4}  grows as $O(\\ell)$ as $\\ell\\to+\\infty$, while the  distances between the neighbouring zeroes are of order $O(\\ell^{-1})$. This means that we have either resonances or eigenvalues accumulating to the segment $[-\\Tht(\\g),\\Tht(\\g)]$ on the real axis, see Figure~\\ref{fig:accum}(a,b). It is especially interesting to notice that by means of a continuous  change  of the gain-and-loss amplitude one can transform a ladder of resonances  to a ladder of eigenvalues or vice versa as shown in figure~\\ref{fig:accum}(b). Such a transformation is obviously not possible for  Wannier-Stark ladders   in  self-adjoint operators where complex eigenvalues cannt exist.  The peculiar regime  $\\sin\\sqrt{2\\g}=0$ corresponds to the situation when the leading order of imaginary part of expansion (\\ref{3.4}) vanishes [curve~2 in figure~\\ref{fig:accum}(b)]. A   delicate analysis shows that in this case the imaginary parts of zeroes described by  Lemma~\\ref{lm3.4} are  nonzero and  amount to  $O(\\ell^{-5})$. Thus, for $\\sin\\sqrt{2\\g}=0$ we have a ladder of coexisting  ``nearly spectral singularities'' with extremely small by nevertheless nonzero imaginary parts (genuine spectral singularities with exactly zero imaginary parts  will be discussed in section~\\ref{sec:ss}).\n\n\\begin{figure}\n\t\\centering\n\\includegraphics[width=0.99\\columnwidth]{fig02.eps}\n\\caption{Ladders of zeroes  plotted according to the leading terms of  expansion (\\ref{3.4}). \t(a) Accumulation of zeroes (corresponding to eigenvalues) to the segment $[-\\Tht(\\g),\\Tht(\\g)]$ as the gain-to-loss distance $\\ell$ grows. Here the gain-and-loss amplitude $\\gamma=40$ and $\\ell=50$ (curve 1), 100 (curve 2) and  150 (curve 3). The zeroes are shown with crosses; thin solid lines are to guide the eye. (b) Transformation of the ladder of eigenvalues (curve 1) to a ladder of resonances (curve 3) under the change of the gain-and-loss amplitude. Here $\\ell=100$ and $\\gamma=40$ (curve 1) and $\\gamma = 50$ (curve 3). Curve 2 corresponds to $\\gamma=9\\pi^2/2\\approx  44.413$. In this case the imaginary part of term $\\propto \\ell^{-3}$ in  expansion (\\ref{3.4}) vanishes, and we obtain a ladder of coexisting ``nearly spectral singularities'' with Im$\\,k=O(\\ell^{-5})$.}\n\\label{fig:accum}\n\\end{figure}\n\n\nIt is interesting to compare this spectral picture with that in the limiting situation $\\ell=+\\infty$ and the first issue is which operator are to be regarded as limiting ones. Such limiting operator are to be treated in the norm resolvent sense, that is, we should find an approximation for the resolvent of our operator as $\\ell\\to+\\infty$.\nAccording the results of \\cite{UMJ},  as $\\ell\\to+\\infty$, the solution of the equation\n\\begin{equation*}\n-u''+Vu-\\l u=f\\quad\\text{in}\\quad\\mathds{R},\\quad u\\in H^2(\\mathds{R}),\\quad f(x):=f_-(x+\\ell)+f_0(x)+f_+(x-\\ell),\\quad f_0,\\,f_\\pm\\in L_2(\\mathds{R}),\n\\end{equation*}\nis approximated in the norm resolvent sense up to a small error by $u_-(x+\\ell)+u_0(x)+u_+(x-\\ell)$. Here $u_0, u_\\pm\\in H^2(\\mathds{R})$ solve the equations\n\\begin{equation*}\n-u_0''-\\l u_0=f_0,\\quad -u_\\pm''+V_\\pm u_\\pm -\\l u_\\pm=f_\\pm\\quad\\text{in}\\quad \\mathds{R},\n\\end{equation*}\nwhere the potentials $V_\\pm$ were introduced in (\\ref{3.23}).\nHence, the limiting operator is in fact a direct sum of three operators\n\n$\\Op_-\\oplus\\Op_0\\oplus\\Op_+$, $ \\Op_0:=-\\frac{d^2\\ }{dx^2}$,\n\n$\\Op_\\pm:=-\\frac{d^2\\ }{dx^2}+V_-$.\n\nThe essential spectra of all three operators are $[0,+\\infty)$. The operator $\\Op_0$ has the only resonance at $k=0$, while the resonances and the eigenvalues of the operators $\\Op_\\pm$ are expressed via the zeroes of the functions $F_\\pm$ by the formula $\\l=k^2$, see Lemma~\\ref{lm3.6}.  In view of such approximation, it would be natural to expect that as $\\ell\\to+\\infty$, the above discussed zeroes $k_n(\\ell,\\g)$   converge to similar real zeroes  of the functions $F_\\pm$ or to zero.\n{\\sl This is not true!} Indeed, by Lemma~\\ref{lm3.6}, each of the functions $F_\\pm$ has at most {\\sl one}  real zero, while the number of the zeroes $k_n(\\ell,\\g)$ in the vicinity of this segment {\\sl increases} as $\\ell\\to+\\infty$. Thus, regarding the case $\\ell=+\\infty$ and eigenvalues equations for the operators $\\Op_\\pm$ as limiting, for sufficiently large finite $\\ell$ we get a {\\sl large number} of {\\sl eigenvalues or resonances emerging from internal points} of the zone $[0,\\Tht^2(g)]$ in essential spectrum. And at most three points (including zero)  are spectral singularities for the limiting equations. In other words, here we have eigenvalues emerging from ordinary points in the essential spectrum not being spectral singularities for the unperturbed operators. And while in the case of Wannier-Stark resonances their existence is due to the eigenvalues of the operator $-\\frac{d^2\\ }{dx^2}+\\e x$ accumulating on the real line as $\\e\\to+0$, this surely not the case in our model.\n\n\n\nHere we just make one more important observation. The presence of the mentioned zero-width resonances, or real zeroes of $F$, could be regarded as an explanation or the reason for the existence of the above described ladder of resonances and eigenvalues. However, {\\sl this is not the case!} Indeed, for $\\frac{\\pi^2}{2} < \\gamma< \\gamma_*\\approx  11.561$, the function $F$ {\\sl does have} a ladder of resonances, but as we shall show in the next section, it {\\sl has no} real zeroes. So, our ladder of resonances or eigenvalues exists due to some other reasons and  the existence of the zero-width resonances is mostly the implication of the presence of such ladder and its behavior as $\\g$ varies.\n\n\n",
        "label": "fig:accum",
        "Descriptive_question1": "What is the gain-and-loss amplitude value for curve 1 in figure_2, panel (a)?",
        "Descriptive_question2": "What symbol is used to represent zeroes in figure_2, panel (a)?",
        "Reasoning_question1": "How does the accumulation of zeroes to the segment [-Tht(g), Tht(g)] in figure_2, panel (a), change as the gain-to-loss distance increases from 50 to 150?",
        "Reasoning_question2": "What can be inferred about the transformation of eigenvalues to resonances in figure_2, panel (b), when the gain-and-loss amplitude changes from 40 to 50?",
        "Descriptive_answer1": "40",
        "Descriptive_answer2": "crosses",
        "Reasoning_answer1": "In figure_2 panel (a), as the gain-to-loss distance \\(\\ell\\) increases from 50 (curve 1) to 150 (curve 3), the number of zeroes accumulating to the segment \\([-\\Theta(\\gamma),\\Theta(\\gamma)]\\) on the real axis also increases. The zeroes become denser and more closely spaced along this segment, indicating that the ladder of eigenvalues grows with increasing \\(\\ell\\). This is consistent with the discussion that the number of zeroes in this region grows proportionally to \\(\\ell\\), while the spacing between them decreases accordingly.",
        "Reasoning_answer2": "Figure_2 panel (b) shows that when the gain-and-loss amplitude \\(\\gamma\\) increases from 40 (curve 1) to 50 (curve 3) with \\(\\ell=100\\) fixed, the ladder of eigenvalues (initially located in the lower half-plane) transforms continuously into a ladder of resonances (located in the upper half-plane). This indicates that changing \\(\\gamma\\) can cause the zeroes to move across the real axis from lower to upper half-plane, converting eigenvalues into resonances. The intermediate curve 2 corresponds to a special value of \\(\\gamma = 9\\pi^2/2 \\approx 44.413\\) where the imaginary part at leading order vanishes, producing \"nearly spectral singularities.\" This demonstrates a controllable transition between eigenvalues and resonances by tuning \\(\\gamma\\)."
    },
    {
        "paper_id": "1908.06383.json",
        "image_id": "figure_3",
        "image_path": [
            "/home/yz979/scratch/chengye/scimolmo/test-images-tables/dataset_imagestables/1908.06383/images/fig03.eps"
        ],
        "caption": "Illustration for proof of Lemma~\\ref{lm5.3}. Graphs of the function $g(u, \\beta)$ for $\\b=\\pi$ (a), $\\b=\\b_*\\approx  4.808438$ (b), and $\\b=5$ (c). Notice broken vertical axes in (b) and (c). ",
        "classify": "Chart",
        "section_info": "5 Spectral singularities\n\\section{Spectral singularities}\n\\label{sec:ss}\n\\subsection{General analytical expressions}\n\nIn this section we study real zeroes of the function $F$ corresponding to spectral singularities, i.e. to  zero-width resonances.\nFor such zeroes, equation (\\ref{2.10}) is a pair of two real equations for one real variable $k$ and two parameters $\\ell$ and $\\g$.\nThanks to the symmetry of the zeroes with respect to the imaginary axis, it is sufficient to find only positive real resonances since the negative ones are located symmetrically with respect to the origin. Similar to the proof of Lemma~\\ref{lm3.6}, for real positive $k$  we make  change (\\ref{4.1})\nand rewrite equation (\\ref{2.10}) in the following form:\n\\begin{align*}\n(1-2u^{-4})\\cos\\b u^{-1}&+(2u^4-1)\\cosh\\b u + 2i \\sqrt{1-u^4}\\left(u^2\\sinh\\b u - u^{-4}\\sin\\b u^{-1}\\right)\n\\\\\n&-e^{-2i\\b\\ell\\sqrt{u^{-2}-u^2}} \\left(\\cosh(\\b u)-\\cos\\b u^{-1}\\right)=0,\\qquad \\b=\\sqrt{2\\g}.\n\\end{align*}\nTaking the real and imaginary part of this equation and multiplying the equation by $u^4$, we obtain:\n\\begin{equation}\\label{4.3}\n\\begin{aligned}\n&(u^4-2)\\cos\\b u^{-1}+u^4(2u^4-1)\\cosh\\b u\n=u^4\\cos\\left(2\\b\\ell\\sqrt{u^{-2}-u^2}\\right)\\left(\\cosh \\b u-\\cos\\b u^{-1}\\right),\n\\\\\n&2\\sqrt{1-u^4}\\Big(u^6\\sinh\\b u-\\sin\\b u^{-1}\\big)\n=-u^4\\sin\\left(2\\b\\ell\\sqrt{u^{-2}-u^2}\\right)\\left(\\cosh \\b u-\\cos\\b u^{-1}\\right).\n\\end{aligned}\n\\end{equation}\nThis is a system of two real equations with three real variables. If we are given $(\\b,\\ell)$ and we try to find $u$, the system is overdetermined and does not necessary have a root. In other words, it is solvable with respect to $u$ only if $(\\b,\\ell)$ are located on some (solvability) curves. In order to avoid working with an overdetermined system, in what follows we regard (\\ref{4.3}) as a system for two unknown variable with one parameter.\n\nTo find the curves in $(\\b,\\ell)$ plane, on which equations (\\ref{4.3}) are solvable with respect to $u$, we shall regard $u$ as a parameter and $(\\b,\\ell)$ as unknown variables. We take the sum of squares of equations (\\ref{4.3}) and divide the result by $(1-u^4)$. We also divide equations (\\ref{4.3}). This leads us to a   pair  of equations:\n\\begin{align}\\label{4.4}\n&u^4(1-u^4) \\cosh\\b u\\cos\\b u^{-1}-2u^6 \\sinh\\b u\\sin\\b u^{-1} +1-u^{12}=0,\n\\\\\n&\\frac{2\\sqrt{1-u^4}(u^6\\sinh\\b u-\\sin\\b u^{-1})}{(2-u^4)\\cos\\b u^{-1}+u^4(1-2u^4)\\cosh\\b u}=\\tan \\Big( 2\\b\\ell\\sqrt{u^{-2}-u^2}\\Big).\\nonumber\n\\end{align}\nThe second equation can be solved explicitly with respect to $\\ell$:\n\\begin{equation}\\label{4.10}\n\\ell=\\frac{1}{2\\b\\sqrt{u^{-2}-u^2}} \\left(\\arctan \\frac{2\\sqrt{1-u^4}(u^6\\sinh\\b u-\\sin\\b u^{-1})}{(2-u^4)\\cos\\b u^{-1}+u^4(1-2u^4)\\cosh\\b u}+\\pi n\\right),\n\\end{equation}\nwhere $n\\in\\mathds{N}$ is an arbitrary natural number. As the next lemma states, to make equations (\\ref{4.4}), (\\ref{4.10}) equivalent to (\\ref{4.3}), we should also assume that\n\\begin{equation}\\label{4.5}\n(-1)^n=\\sign \\big((u^4-2)\\cos\\b u^{-1}+u^4(2u^4-1)\\cosh\\b u\\big).\n\\end{equation}\n\n\\begin{lemma}\\label{lm4.1}\nEquations (\\ref{4.3}) are equivalent to (\\ref{4.4}), (\\ref{4.10}), (\\ref{4.5}).\n\\end{lemma}\n\n\\begin{proof}\nWe rewrite shortly equations (\\ref{4.3}) as\n$A_1=B\\cos\\a$, $ A_2=-B\\sin\\a$,\nwhere $A_1$, $A_2$ are the left hand sides in (\\ref{4.3}), $\\a=2\\b\\ell\\sqrt{u^{-2}-u^2}$ and $B=u^4(\\cosh\\b u-\\cos\\b u^{-1})$. Then equations (\\ref{4.4}), (\\ref{4.10}) become\n$A_1^2+A_2^2=B^2$, $\\a=-\\arctan\\frac{A_2}{A_1}+\\pi n$.\nWe have:\n\\begin{equation*}\nB\\cos \\a=(-1)^n\\cos\\arctan\\frac{A_2}{A_1}=\\frac{(-1)^n}{\\sqrt{1+\\frac{A_2^2}{A_1^2}}} = \\frac{(-1)^n|A_1|}{\\sqrt{A_1^2+A_2^2}}\n\\end{equation*}\nand we get the first equation $A_1=B\\cos\\a$  provided condition (\\ref{4.5}) is satisfied. In the same way we check that the latter condition also ensures the second equation $A_2=-B\\sin\\a$.\n\\end{proof}\n\n\nEquation (\\ref{4.4}) is transcendental, and we can not solve it analytically.\nNevertheless, for each $u\\in(0,1)$, this is an equation only for a single variable $\\b$, not for two as equations (\\ref{4.3}). So, we propose the following algorithm of recovering the aforementioned solvability curves: choose $u\\in(0,1)$, then solve equation (\\ref{4.4}) and recover the sequence of  distances  $\\ell$ by formula (\\ref{4.10}) with different integer $n$. Then the gain-and-loss amplitude $\\gamma$ and the corresponding wavenumber $k$ can be readily recovered from $\\beta$ and  $u$.  In a similar way, one can first fix some value of $\\beta$ (i.e., fix the gain-and-loss strength) and then   solve equation (\\ref{4.4}) with respect to $u$ and   recover   $\\ell$ by (\\ref{4.10}). Equation (\\ref{4.4})  is well-behaved, and for each $\\beta$  all its zeros  $u$ can be easily found numerically.\n\nAlternatively, as explained below  in Section~\\ref{sec:general}, the values  corresponding to  spectral singularities can be found systematically by means of the numerical continuation from the limit $\\ell=0$. However, in this case   equation (\\ref{4.4}) is still useful because it allows one to check that all spectral singularities have been found for the given value of the gain-and-loss $\\gamma$.\n\n\n\\subsection{\nAbsence of   spectral singularities}\n\\label{sec:gap}\n\nFor $u=0$ and $u=1$, the left-hand-side of equation (\\ref{4.4}) is equal respectively to $1$ and $-2\\sinh\\beta\\sin\\beta$. Then a sufficient condition for the existence of a spectral singularity at the given gain-and-loss amplitude $\\gamma$ is $\\sin\\beta=\\sin\\sqrt{2\\gamma}>0$. At the same time, it is also possible to establish sufficient conditions that forbid the existence of   spectral singularities in a certain interval of parameters. In this subsection we prove the existence of two ``forbidden gaps'' for the roots of equation (\\ref{4.4}). The first one exists for all $\\b\\geqslant 0$ and it states that there is no roots in certain interval. The second gap is a certain interval of values of $\\b$, for which equation (\\ref{4.4}) has no zeroes at all.\n\nFor the convenience, by $g(u,\\b)$ we denote the left hand side of equation (\\ref{4.4}). The first ``forbidden gap'' is described in the following lemma.\n\n\\begin{lemma}\\label{lm5.1}\nFor all $\\b\\geqslant 0$, equation (\\ref{4.4}) has no roots in the interval $\\big[0,(1+\\tfrac{\\b}{4})^{-1}\\big)$.\n\\end{lemma}\n\\begin{proof}\nEmploying a standard inequality $a\\cos\\a+b\\sin\\a\\leqslant \\sqrt{a^2+b^2}$, we estimate the first two term in equation (\\ref{4.4}) as\n\\begin{equation*}\nu^4(1-u^4) \\cosh\\b u\\cos\\b u^{-1}-2u^6 \\sinh\\b u\\sin\\b u^{-1} \\leqslant \\sqrt{u^8(1-u^4)^2\\cosh^2 \\b u+4u^{12}\\sinh^2\\b u}.\n\\end{equation*}\nHence, equation (\\ref{4.4}) surely has no roots for values of $u$ satisfying\n\\begin{equation*}\n \\sqrt{u^8(1-u^4)^2\\cosh^2 \\b u+4u^{12}\\sinh^2\\b u} <1-u^{12}.\n\\end{equation*}\nExpressing $\\cosh^2 \\b u$ via $\\sinh^2 \\b u$ and simplifying this inequality, we obtain $ u^4(1+u^4)\\cosh \\b u <1+u^{12}$ and hence,\n\\begin{equation}\\label{3.37}\n\\cosh \\b u-1 <\\frac{1-2u^4+u^8}{u^4},\\qquad \\sqrt{2}\\sinh \\b u\n<u^{-2}-u^2.\n\\end{equation}\nThanks to the estimate $\\sqrt{2}\\sinh 2\\ln u^{-1}\\leqslant u^{-2}-u^2$, $u\\in(0,1]$, the last inequality in (\\ref{3.37}) holds once $\\frac{\\b}{4}<u^{-1}\\ln u^{-1}$. It is easy to confirm that this inequality is true once $u^{-1}>1+\\tfrac{\\b}{4}$.\nThe proof is complete.\n\\end{proof}\n\n\nThe next lemma is   auxiliary  and will be employed in studying the second forbidden zone.\n\n\\begin{lemma}\\label{lm5.2}\nThe function $g(u,\\pi)$ is positive on $[0,1)$.\n\\end{lemma}\n\n\\begin{proof}\nWe have $g(0,\\pi)=1$ and by Lemma~\\ref{lm5.1}, it is positive for $u<(1+\\tfrac{\\pi}{4})^{-1}$. This is why in what follows we consider only the values $u\\geqslant (1+\\tfrac{\\pi}{4})^{-1}$. For such values of $u$ we have\n$\\pi\\leqslant \\pi u^{-1}\\leqslant \\pi (1+\\tfrac{\\pi}{4})<1.79\\pi$.\nAs\n$\\tfrac{3\\pi}{2} \\leqslant \\pi u^{-1}\\leqslant \\pi (1+\\tfrac{\\pi}{4})$,\nthe function $\\sin\\b u^{-1}$ is negative, while $\\cos \\b u^{-1}$ is positive. Hence, for such values of $u$, the function $g(u,\\pi)$ is positive. It remains to consider the values $\\frac{2}{3}<u\\leqslant 1$.\n\nFor such values of $u$ we first observe the following simple estimates:\n\\begin{equation*}\n\\frac{1-u^4}{1-u}\\cos\\pi u^{-1}\\geqslant -4,\\qquad -\\frac{\\sin \\pi u^{-1}}{1-u}=\\frac{\\sin \\pi (u^{-1}-1)}{1-u}\\geqslant \\pi u.\n\\end{equation*}\nThen the function $g$ satisfies the estimate:\n\\begin{equation}\\label{3.12}\n\\frac{g(u,\\pi)}{u^4(1-u)}\\geqslant g_1(u)+g_2(u),\\qquad g_1(u):=-4\\cosh\\pi u +2\\pi u^3 \\sinh \\pi u,\\qquad g_2(u):=\\frac{1-u^{12}}{u^4(1-u)}.\n\\end{equation}\nThe function $g_1(u)$ is monotonically increasing in $u\\in[\\tfrac{2}{3},1]$ since\n\\begin{equation*}\ng_1'(u)=2\\pi^2 u^3\\cosh\\pi u+(6\\pi u^2-4\\pi)\\sinh\\pi u>2\\pi(\\pi u^3+6u^2-2)\\sinh\\pi u>2\\pi\\sinh\\pi u>0.\n\\end{equation*}\nHence,\n$g_1(u)\\geqslant g_1\\left(\\frac{2}{3}\\right)>-9.05$.\nFor the function $g_2$ we have the following representation and estimate:\n\\begin{equation*}\ng_2(u)=1+\\sum\\limits_{j=1}^{4}(u^j+u^{-j})+\\sum\\limits_{j=5}^{8} u^j\\geqslant 9 +\\sum\\limits_{j=5}^{8}\\left(\\frac{2}{3}\\right)^j>9.31.\n\\end{equation*}\nTwo last estimates and (\\ref{3.12}) imply the positivity of the function $g$ for $u\\in[\\tfrac{2}{3},1)$.\n\\end{proof}\n\n\nDenote\n\\begin{align*}\ng_*(u,\\b):=&\\b u^3(1-3u^4)\\cosh\\b u \\sin \\b u^{-1}+\\b u^5 (3-u^4)\\sinh\\b u \\cos \\b u^{-1}\n\\\\\n&-2 u^4(1+u^4) \\cosh\\b u \\cos \\b u^{-1} -6(1+u^{12}).\n\\end{align*}\n\n\nThe next lemma states the existence of the second forbidden zone.\n\n\\begin{lemma}\\label{lm5.3}\nEquation (\\ref{4.4}) has no roots as $\\pi<\\b<\\b_*<5$, where $(u_*, \\b_*)$ is  the root of the system of the equations\n\\begin{equation}\\label{3.13}\ng(u,\\b)=0,\\qquad g_*(u,\\b)=0,\\qquad u\\in[0,1],\\qquad \\pi<\\b<5,\n\\end{equation}\nwith minimal possible $\\b$.\nTheir approximate values are\n\\begin{equation}\\label{3.14}\n\\b_*=4.808438,\\qquad u_*=0.611772.\n\\end{equation}\n\\end{lemma}\n\n\\begin{proof}\nThe function $g(u,\\pi)$ is positive on $[0,1)$ and $g(1,\\pi)=0$, see Figure~\\ref{fig:forbidden}a. As $\\pi<\\b<2\\pi$, we have $g(0,\\b)=1>0$ and $g(1,\\b)=-2\\sin\\b\\sinh\\b>0$. Hence, for $\\b$ close enough to $\\pi$, the function $g(u,\\b)$ is positive for all $u\\in[0,1]$. At the same time, we have $g(0.65,5)<-0.617<0$ and therefore, for $\\b=5$, equation (\\ref{4.4}) possesses at least two roots, one in $(0,0.65)$ and another in $(0.65,1)$. cf. Figure~\\ref{fig:forbidden}c. We also observe that the function $g$ is jointly continuous in $(u,\\b)$. The above facts means that as $\\b$ grows from $\\pi$ to $5$, at some value $\\b=\\b_*$, the graph of the function $g$ is still located in the upper half-plane but touches the $u$-axis at some point $u=u_*$,\nsee Figure~\\ref{fig:forbidden}b. The function $g(u,\\b)$ is positive as $\\pi<\\b<\\b_*$ and $u\\in[0,1]$. Then the point $u=u_*$ is obviously the global minimum of $g$ and hence, $(u_*,\\b_*)$ is a solution to the system of equations $g(u,\\b)=0$, $\\frac{\\p g}{\\p u}(u,\\b)=0$. It is easy to check that $g_*=\\frac{\\p g}{\\p u}-6 g$ and hence, $(u_*,\\b_*)$ solves system (\\ref{3.13}). These roots can be found numerically and this gives (\\ref{3.14}). The proof is complete.\n\\end{proof}\n\n\n\\begin{figure}\n\t\\centering\n\\includegraphics[width=0.99\\columnwidth]{fig03.eps}\n\\caption{Illustration for proof of Lemma~\\ref{lm5.3}. Graphs of the function $g(u, \\beta)$ for $\\b=\\pi$ (a), $\\b=\\b_*\\approx  4.808438$ (b), and $\\b=5$ (c). Notice broken vertical axes in (b) and (c). }\n\\label{fig:forbidden}\n\\end{figure}\n\nReturning from the auxiliary variable $\\beta$ to the gain-and-loss amplitude $\\gamma=\\beta^2/2$, from   Lemma~\\ref{lm5.3}  we deduce  the following important result:\n\\begin{equation}\n\\label{eq:gap}\n\\textrm{there is no spectral singularities for\\quad }  \\frac{\\pi^2}{2} < \\gamma< \\gamma_*\\approx  11.561.\n\\end{equation}\n\n\n\n\\subsection{Creating a  spectral singularity  at a given wavenumber}\n\n\\begin{table}\n\t\\centering\n\t\\begin{tabular}{c|cccc}\n\t\t$n$ & 0 & 1& 2&\\\\\\hline\n\t\t$\\gamma_\\star^{(n)}$ & 2.071&13.307&27.783 &\\\\[2mm]\n\t\t$k_\\star^{(n)}$ & 1.065&4.318 &7.529 &\n\t\\end{tabular}\n\t\\caption{Approximate values of gain-and-loss amplitudes $\\gamma=\\gamma_\\star^{(n)}$  and wavenumbers   $k=k_\\star^{(n)}$, $n=0, 1, \\ldots$,  corresponding to spectral singularities with lowest $\\gamma$ in the limit $\\ell=0$, see Table~I in  \\cite{Mostafazadeh2009}. \\label{tbl:1}}\n\\end{table}\n\n\nFor $\\ell=0$ spectral singularity can only be obtained for some isolated values of the wavenumber $k$ and the gain-and-loss amplitude $\\gamma$ \\cite{Mostafazadeh2009}. Several lowest values of $\\gamma$ corresponding to the spectral singularities  and the associated wavenumbers  $k$ are listed in Table~\\ref{tbl:1}.  An  important advantage of the more general system with nonzero gain-to-loss disctance    $\\ell>0$ consists  in the possibility to create a spectral   singularity   at any wavenumber $k$ given beforehand. Indeed, let us  return back to equations (\\ref{4.4}), (\\ref{4.10}) and    discuss the following   issue: given a point $k$ on the real axis, how to choose $\\b$ and $\\ell$ to have a  resonance at this point? Equations (\\ref{4.4})--(\\ref{4.10}) allow us to answer  easily this question.\n\nWe fix $k>0$ and we find the associated value of $u$ by resolving (\\ref{4.1}):\n\\begin{equation}\\label{4.12}\nu^{-2}-u^2=4k^2\\b^{-2},\\qquad\n\nu=\\b R^{-1}, \\quad \\mbox{where\\ } R = \\sqrt{2k^2+\\sqrt{4k^4+\\b^4}}.\n\\end{equation}\nWe divide equation (\\ref{4.4}) by $u^6$ and substitute then the above formulae and\n\\begin{equation*}\n\\frac{1-u^{12}}{u^6}=\\frac{1-u^4}{u^2}\\frac{1+u^4+u^8}{u^4}=(u^{-2}-u^2)\\big((u^{-2}-u^2)^2+3\n\\big).\n\\end{equation*}\nThis gives the equation:\n\\begin{equation}\\label{4.11}\n\\begin{aligned}\n2\\b^4 k^2&\\cosh(\\b^2R^{-1})  \\cos R\n\n\n - \\b^6 \\sinh(\\b^2R^{-1})  \\sin R\n  +2k^2(16k^4+3\\b^4)=0.\n\\end{aligned}\n\\end{equation}\nAn algorithm for creating a resonance at a prescribed point  $k$ is as follows. Given $k>0$, we first solve equation (\\ref{4.11}) with respect to $\\b$ and we also find $u$ by (\\ref{4.12}). Then needed values of  $\\ell$ are determined by (\\ref{4.10}), (\\ref{4.5}).\n\n\nIn order to illustrate this  algorithm,  we us consider a finite interval of wavenumbers $k\\in (0, k_1]$, where  we set $k_1 = 10$ for the numerics reported on in what follows. We scan   the chosen interval  with a sufficiently small step ($\\Delta k=0.01$) and for each value of $k$ solve equation (\\ref{4.11}) numerically  using the simple dichotomy method. While for each $k$  equation (\\ref{4.11}) might have several roots $\\beta$, in our numerical procedure we always choose the  minimal positive root, i.e., the one which allows to achieve the spectral singularity with given $k$ at the smallest possible value  of the gain-and-loss amplitude $\\gamma=\\gamma_\\textrm{min}$. Next, we choose the minimal positive distance $\\ell_\\textrm{min}$ which satisfies the conditions   (\\ref{4.10}), (\\ref{4.5}) and then we use the periodicity in $\\ell$ to generate the sequence of larger gain-to-loss distances $\\ell_n = \\ell_\\textrm{min} + n\\pi/(2k)$, $n=1,2\\ldots$ [see   (\\ref{eq:periodic})]. The resulting dependencies $\\gamma_\\textrm{min}(k)$ and $\\ell_\\textrm{min}(k)$, $\\ell_n(k)$   are shown in figure~\\ref{fig:min}. The minimal gain-and-loss amplitude $\\g_{min}(k)$  and the minimal gain-to-loss distance $\\ell_{min}$ are   discontinuous,  which means that the small variation  in  the wavenumuber $k$  might require  a significant  change either in   $\\gamma$ or in $\\ell$.  It is especially important that the values of the  distance  $\\ell$ are generically different from zero, which points out explicitly    that the new degree of freedom offered by the nonzero gain-to-loss distance    is   important for the achieving a spectral singularity  at   the given wavenumber $k$.\n\n\n\\begin{figure}\n\t\\centering\n\t\\includegraphics[width=0.8\\columnwidth]{fig04.eps}\n\t\\caption{(a) Minimal value of the gain-and-loss amplitude $\\gamma_\\textrm{min}$  which corresponds to   a spectral singularity with the given value of the wavenumber $k$. (b)  Minimal gain-to-loss distance  $\\ell_\\textrm{min}$  which corresponds to   a spectral singularity with the  $k$ and $\\gamma$ from the  left panel (bold curves) and larger distances $\\ell_n$ obtained using the periodicity in $\\ell$ (thin \\textcolor{black}{dotted} curves).}\n\t\\label{fig:min}\n\\end{figure}\n\n\n\\subsection{$\\PT$-symmetry breaking laser-antilaser threshold}\n\nA particularly important characteristics of any $\\PT$-symmetric structure is the $\\PT$ symmetry breaking threshold, i.e., the amplitude of the gain-and-loss\ncorresponding to the  ``phase transition'' from the purely real to complex spectrum.  The best studied scenario of the phase transition  \nis the collision of two real discrete  eigenvalues at an exceptional point with the subsequent splitting in a complex-conjugate pair. However, in systems with nonempty continuous spectrum,  the phase transition can also occur through   the splitting of a self-dual spectral singularity, which results in a bifurcation of a complex-conjugate pair from an interior point of the continuum \\cite{Yang17,KZ17,Konotop2019}. At the moment corresponding to the formation of the spectral singularity, the system operates in the CPA-laser regime \\cite{Longhi10}. Thus, in such a system, the $\\PT$-symmetry breaking threshold at the same time corresponds to the CPA-laser threshold.\n\nLemma~\\ref{lm3.3} guarantees that the spectrum of our system is real for sufficiently small gain-and-loss amplitudes $\\gamma$. Additionally, according to Lemma~\\ref{lm3.5}, the spectrum  does not have any real discrete eigenvalue. Hence, the $\\PT$-symmetry breaking is expected to occur through the emergence of a self-dual spectral singularity. In order to identify the $\\PT$-symmetry breaking threshold in our system, we start from the limit $\\ell=0$, there the phase transition takes place at  $\\gamma_\\star^{(0)} \\approx 2.072$, see \\cite{Mostafazadeh2009,KZ17} and Table~\\ref{tbl:1}. Thus, the spectrum with $\\ell=0$ is purely real and continuous for $\\gamma \\in [0, \\gamma_\\star]$, while the  increase of the gain-and-loss just above $\\gamma_\\star^{(0)}$ leads to the bifurcation of a complex-conjugate pair from an interior point  of the continuum. The spectral singularity forming at  $\\gamma_\\star^{(0)}$ takes place at wavenumber $k=k_\\star^{(0)} \\approx  1.065$. Respectively, the complex-conjugate pair of eigenvalues bifurcates from $\\lambda_0 =  [k_\\star^{(0)}]^2$. (Notice that the further increase of $\\gamma$ above the next threshold values listed in Table~\\ref{tbl:1} leads to the formation of new spectral singularities and, respectively, to bifurcations of new complex-conjugate pairs in the spectrum.)\n\nNext, we use the numerical continuation in $\\ell$ in order to continue the known solution at $\\ell=0$ to the domain $\\ell>0$.  The obtained dependence of the threshold value  of the gain-and-loss amplitude on distance $\\ell$ is shown in figure~\\ref{fig:threshold}(a), where we observe that the phase transition threshold decreases monotonically with the growth of $\\ell$. This means that introducing an additional space   between the gain and loss, one can decrease the $\\PT$-symmetry breaking  threshold, i.e. achieve the laser-antilaser operation at \\textcolor{black}{lower} gain-and-loss amplitudes than in a waveguide with the adjacent gain and loss.\n\n\n\\begin{figure}\n\t\\centering\n\t\\includegraphics[width=0.85\\columnwidth]{fig05.eps}\n\t\\caption{(a) $\\PT$-symmetry breaking threshold $\\gamma_\\star^{(0)}$ \\textit{vs} the distance between the gain and loss $\\ell$. The spectrum is purely real and continuous for $\\gamma\\leq \\gamma_\\star$, but acquires a pair of complex conjugate eigenvalues  as the gain-and-loss amplitude exceeds the threshold $\\gamma_\\star^{(0)}$. (b) Values of the wavevector $k_\\star^{(0)}$ corresponding to the dependence in (a).}\n\t\\label{fig:threshold}\n\\end{figure}\n\n\\subsection{General picture of spectral singularities}\n\\label{sec:general}\n\n\\begin{figure}\n\t\\centering\n\t\\includegraphics[width=0.99\\columnwidth]{fig06.eps}\n\t\\caption{(a) Values of the gain-and-loss amplitude $\\gamma$ and distance  $\\ell$, for which   spectral singularities occur. (b) Corresponding  wavenumbers   $k$. \\textcolor{black}{Panel (c) magnifies   the region  $(\\ell, \\gamma)\\in [2,4]\\times[10,50]$ from   (a), and panel  (d) is the  magnification of corresponding curves from panel (b).}}\n\t\\label{fig:general}\n\\end{figure}\n\nLet us now turn to description of the general picture of spectral singularities. In order to construct systematically different solutions, we again start from the limit $\\ell=0$, where the values of $\\gamma$ and $k$ corresponding to spectral singularities are known, see Table~\\ref{tbl:1}. Then we use the periodicity of function $e^{-4ik\\ell}$ in $\\ell$ in order to construct new branches of solutions having no counterparts in the limit $\\ell=0$, cf. equation (\\ref{eq:periodic}). This procedure results in a fairly complicated picture containing a multitude of spectral singularities, some part of which (corresponding to  relatively  small   values of the gain-and-loss) is shown in figure~\\ref{fig:general}(a,b) as the curves on the plane $\\gamma$ {\\it vs} $\\ell$ and $k$ {\\it vs} $\\ell$.\n\n\nLet us describe the structure of the found solutions using the  diagram $\\gamma$ {\\it vs} $\\ell$ in figure~\\ref{fig:general}(a). The multitude of curves shown in this plot can be divided into three groups (plotted with red, blue and green curves) which have been obtained by means of the continuation from three different solutions in the limit $\\ell=0$. There is a well-visible vertical gap between red and blue curves, which corresponds to the ``forbidden'' values of the gain-and-loss amplitudes $\\gamma$ found above in (\\ref{eq:gap}). At the same time,  there is no gap between blue and green curves, which  results in a multitude of intersections between these curves.\nThe first group of spectral singularities,  corresponding to red curves in Figure~\\ref{fig:general}(a),   is obtained through  the continuation from the  spectral singularity  in the limit $\\ell=0$ with the smallest gain-and-loss amplitude, i.e.  from values $\\gamma_\\star^{(0)}$ and $k_\\star^{(0)}$ in Table~\\ref{tbl:1}.  The leftmost (bold)  curve in this group which originates in the limit $\\ell=0$  is the $\\PT$-symmetry breaking threshold which was already shown in figure~\\ref{fig:threshold}(a). For values of $\\gamma$ above this curve, there is always one or more (but finitely many, see Section~\\ref{sec:zeros})   complex-conjugate pairs of eigenvalues in the spectrum.  Several curves situated to the right from the bold red curve are obtained using the fact that if $\\gamma$ and $k$ are solutions in the limit $\\ell=0$, then the same values of $\\gamma$ and $k$ also correspond to a spectral singularity with $\\ell_n = (n\\pi)/(2k)$, $n=1,2,\\ldots$. Once a single solution with a new distance   $\\ell_n$ is obtained, a new branch of solutions can be constructed using numerical continuation in $\\gamma$ or in $\\ell$. In the limit $\\ell \\to \\infty$ all the red curves in figure~\\ref{fig:general}(a) approach  the asymptotic value $\\gamma=0$ or at $\\gamma_*= \\pi^2/2\\approx 4.935$. Notice that, except for the bold line demarcating the $\\PT$-symmetry breaking threshold, none of the red curves can be continued to the limit $\\ell\\to 0$.\n\nThe multitude of red curves  in figure~\\ref{fig:general}(a) demonstrates explicitly how new spectral singularities emerge with the increase of $\\ell$. Indeed, drawing an imaginary horizontal line, say, at $\\gamma=3$ [see the vertical axis  tick  in figure~\\ref{fig:general}(a)], we observe that with the increase of $\\ell$ this line   intersects more and more red curves. Each intersection corresponds to the values of $\\gamma$ and $\\ell$ at which   some  root $k$   crosses the real axis and goes down from the upper to lower complex half-plane. Thus,  a finite-width resonance transforms  to a complex eigenvalue through the spectral singularity (we recall again that  in view of   $\\PT$ symmetry  any root $k\\ne 0$ crosses the real line simultaneously with its counterpart $-\\bar{k}$, i.e.  the corresponding   spectral singularity is self-dual).\nIn order to illustrate this process, in Figure~\\ref{fig:g=3}  we show the evolution of three numerically found complex zeros of function $F(k, \\gamma, \\ell)$  under   the increase of $\\ell$.  In this figure the imaginary part of each complex zero changes sign from positive to negative and then asymptotically approaches zero (remaining negative). In the limit of large $\\ell$,  this behavior agrees  with expansion (\\ref{3.4}) of Lemma~\\ref{lm3.4} , where, for the chosen value of $\\gamma$,   we have $\\sin\\sqrt{2\\gamma}>0$.   Thus the growing distance   $\\ell$ results in a sequence of self-dual spectral singularities and in the increasing number of complex-conjugate eigenvalues in the spectrum.\n\n \\begin{figure}\n \t\\centering\n \t\\includegraphics[width=0.8\\columnwidth]{fig07.eps}\n \t\\caption{(a,b) Real and imaginary parts of three complex zeros of function $F$ for fixed $\\gamma=3$ and increasing $\\ell$. For each curve, the imaginary part is positive for small $\\ell$ and becomes negative for all sufficiently large $\\ell$. \\textcolor{black}{For each three shown eigenvalues, its real and \timaginary parts are of the same colour in both panels.}}\n \t\\label{fig:g=3}\n \\end{figure}\n\n\nThe second group of spectral singularities [blue curves in figure~\\ref{fig:general}(a)] was obtained by means of the continuation from the  next solution in the limit $\\ell=0$, i.e.  from $\\gamma_\\star^{(1)}$ and $k_\\star^{(1)}$ in Table~\\ref{tbl:1}. Again, one of the curves [the leftmost bold curve in figure~\\ref{fig:general}(a)] was obtained through the direct continuation from the limit $\\ell=0$, while other blue curves were generated  using the periodicity of function $F(k, \\gamma, \\ell)$  in $\\ell$ and cannot be continued to the limit $\\ell\\to0$. In the limit $\\ell\\to\\infty$ the blue curves approach the horizontal asymptotes  $\\gamma=2\\pi^2\\approx19.739$ and $\\gamma=9\\pi^2/2\\approx 44.413$. In  the $(\\gamma, \\ell)$-plane  the  gain-and-loss amplitudes corresponding to the   group of blue curves are well separated from  those for the red curves: indeed, all red curves are bounded from above by the asymptote $\\gamma= \\pi^2/2\\approx 4.935$, while all blue curves are bounded from below by $\\gamma_*\\approx 11.561$, see (\\ref{eq:gap}). Thus, the  emergence of new spectral singularities with the increase of $\\ell$ is sensitive to the value of the gain-and-loss amplitude and does not occur for     the gain-and-loss amplitudes lying in the  gap  between the red and blue curves.\n\nIn comparison with the red curves discussed above, the curves from the blue group in figure~\\ref{fig:general}(a) feature more complicated behavior and, in particular, can intersect each other (and also intersect the curves from the next, third group of green curves discussed below). The intersections between the blue curves occur for the gain-and-loss amplitudes in the interval $11.561 \\lessapprox   \\gamma <  9\\pi^2/2\\approx 19.739$, where $\\sin\\sqrt{2\\gamma}$ is negative.  \\textcolor{black}{At first glance,} this might seem to contradict to  the  expansion (\\ref{3.4}) of Lemma~\\ref{lm3.4},  which  suggests that in this case the multitude of complex zeros   accumulate in the upper complex half-plane with the growth of $\\ell$. However, this apparent contradiction is resolved if we trace the behavior of the complex roots more closely.   Indeed, choosing for an example $\\gamma =16$ [see the vertical axis tick in  figure~\\ref{fig:general}(a)] and computing several  complex roots under the increase of $\\ell$, we observe that each considered root first goes down from the upper half-plane to the lower one but then again returns to the upper half-plane, see Figure~\\ref{fig:g=16}(b) and (b$_1$). Thus,  in  this  interval of the gain-and-loss amplitudes the increase of $\\ell$  results  either in the transformation from the resonance to the eigenvalue or to the opposite process,  i.e. to the disappearance of the complex-conjugate pair.  Respectively, the intersection between the two blue curves corresponds to  two coexisting spectral singularities, i.e. to the moment  when one complex-conjugate pair of eigenvalues disappears and another pair (with different $k$) emerges. For sufficiently large $\\ell$ the imaginary part  of each   considered root  remains positive and approaches zero, in accordance with expansion (\\ref{3.4}). Thus, in this interval of the gain-and-loss strengths, the limit $\\ell\\to\\infty$ the spectrum contains only a finite number of complex-conjugate eigenvalues, which correspond to complex zeros $k$ whose behavior is not covered   by expansion  (\\ref{3.4}).\n\n \\begin{figure}\n\t\\centering\n\t\\includegraphics[width=0.99\\columnwidth]{fig08.eps}\n\t\\caption{(a,b) Real and imaginary parts for three   complex zeros of function $F$ for fixed $\\gamma=16$ and increasing $\\ell$. Panel b$_1$ is the magnification of some region of (b) and  shows more clearly that imaginary parts of all   three shown eigenvalues are positive for all sufficiently large  $\\ell$. \\textcolor{black}{For each three shown eigenvalues, its real and \timaginary parts are of the same colour in both panels.}}\n\t\\label{fig:g=16}\n\\end{figure}\n\nThe third  group of spectral singularities [green curves in figure~\\ref{fig:general}(a)] was obtained by means of the continuation from  the solution $\\gamma_\\star^{(2)}$ and $k_\\star^{(2)}$ in Table~\\ref{tbl:1}. In view of the very complicated structure of the overall resulting picture, we only show a section of these curves corresponding to relatively small   values of the gain-and-loss amplitudes $\\gamma$. Quite interestingly, there is no gap between the blue and green groups of curves, which results in the multitude of intersections between blue and green curves, \\textcolor{black}{see Fig.~\\ref{fig:general}(a) and the magnified view in Fig.~\\ref{fig:general}(c)}. These   intersections suggest a possibility of   simultaneous emergence of two complex-conjugate pairs of eigenvalues from two different interior points of the continuous spectra.\n\nConsidering further solutions $\\gamma_\\star^{(n)}$, $k_\\star^{(n)}$, $n=3,4,\\ldots$, in the limit $\\ell=0$ one can construct new groups of spectral singularities  with larger values of $\\gamma$, which are not shown in Figure~\\ref{fig:general}.\n\n\n5.2 Absence of   spectral singularities\n\\subsection{\nAbsence of   spectral singularities}\n\\label{sec:gap}\n\nFor $u=0$ and $u=1$, the left-hand-side of equation (\\ref{4.4}) is equal respectively to $1$ and $-2\\sinh\\beta\\sin\\beta$. Then a sufficient condition for the existence of a spectral singularity at the given gain-and-loss amplitude $\\gamma$ is $\\sin\\beta=\\sin\\sqrt{2\\gamma}>0$. At the same time, it is also possible to establish sufficient conditions that forbid the existence of   spectral singularities in a certain interval of parameters. In this subsection we prove the existence of two ``forbidden gaps'' for the roots of equation (\\ref{4.4}). The first one exists for all $\\b\\geqslant 0$ and it states that there is no roots in certain interval. The second gap is a certain interval of values of $\\b$, for which equation (\\ref{4.4}) has no zeroes at all.\n\nFor the convenience, by $g(u,\\b)$ we denote the left hand side of equation (\\ref{4.4}). The first ``forbidden gap'' is described in the following lemma.\n\n\\begin{lemma}\\label{lm5.1}\nFor all $\\b\\geqslant 0$, equation (\\ref{4.4}) has no roots in the interval $\\big[0,(1+\\tfrac{\\b}{4})^{-1}\\big)$.\n\\end{lemma}\n\\begin{proof}\nEmploying a standard inequality $a\\cos\\a+b\\sin\\a\\leqslant \\sqrt{a^2+b^2}$, we estimate the first two term in equation (\\ref{4.4}) as\n\\begin{equation*}\nu^4(1-u^4) \\cosh\\b u\\cos\\b u^{-1}-2u^6 \\sinh\\b u\\sin\\b u^{-1} \\leqslant \\sqrt{u^8(1-u^4)^2\\cosh^2 \\b u+4u^{12}\\sinh^2\\b u}.\n\\end{equation*}\nHence, equation (\\ref{4.4}) surely has no roots for values of $u$ satisfying\n\\begin{equation*}\n \\sqrt{u^8(1-u^4)^2\\cosh^2 \\b u+4u^{12}\\sinh^2\\b u} <1-u^{12}.\n\\end{equation*}\nExpressing $\\cosh^2 \\b u$ via $\\sinh^2 \\b u$ and simplifying this inequality, we obtain $ u^4(1+u^4)\\cosh \\b u <1+u^{12}$ and hence,\n\\begin{equation}\\label{3.37}\n\\cosh \\b u-1 <\\frac{1-2u^4+u^8}{u^4},\\qquad \\sqrt{2}\\sinh \\b u\n<u^{-2}-u^2.\n\\end{equation}\nThanks to the estimate $\\sqrt{2}\\sinh 2\\ln u^{-1}\\leqslant u^{-2}-u^2$, $u\\in(0,1]$, the last inequality in (\\ref{3.37}) holds once $\\frac{\\b}{4}<u^{-1}\\ln u^{-1}$. It is easy to confirm that this inequality is true once $u^{-1}>1+\\tfrac{\\b}{4}$.\nThe proof is complete.\n\\end{proof}\n\n\nThe next lemma is   auxiliary  and will be employed in studying the second forbidden zone.\n\n\\begin{lemma}\\label{lm5.2}\nThe function $g(u,\\pi)$ is positive on $[0,1)$.\n\\end{lemma}\n\n\\begin{proof}\nWe have $g(0,\\pi)=1$ and by Lemma~\\ref{lm5.1}, it is positive for $u<(1+\\tfrac{\\pi}{4})^{-1}$. This is why in what follows we consider only the values $u\\geqslant (1+\\tfrac{\\pi}{4})^{-1}$. For such values of $u$ we have\n$\\pi\\leqslant \\pi u^{-1}\\leqslant \\pi (1+\\tfrac{\\pi}{4})<1.79\\pi$.\nAs\n$\\tfrac{3\\pi}{2} \\leqslant \\pi u^{-1}\\leqslant \\pi (1+\\tfrac{\\pi}{4})$,\nthe function $\\sin\\b u^{-1}$ is negative, while $\\cos \\b u^{-1}$ is positive. Hence, for such values of $u$, the function $g(u,\\pi)$ is positive. It remains to consider the values $\\frac{2}{3}<u\\leqslant 1$.\n\nFor such values of $u$ we first observe the following simple estimates:\n\\begin{equation*}\n\\frac{1-u^4}{1-u}\\cos\\pi u^{-1}\\geqslant -4,\\qquad -\\frac{\\sin \\pi u^{-1}}{1-u}=\\frac{\\sin \\pi (u^{-1}-1)}{1-u}\\geqslant \\pi u.\n\\end{equation*}\nThen the function $g$ satisfies the estimate:\n\\begin{equation}\\label{3.12}\n\\frac{g(u,\\pi)}{u^4(1-u)}\\geqslant g_1(u)+g_2(u),\\qquad g_1(u):=-4\\cosh\\pi u +2\\pi u^3 \\sinh \\pi u,\\qquad g_2(u):=\\frac{1-u^{12}}{u^4(1-u)}.\n\\end{equation}\nThe function $g_1(u)$ is monotonically increasing in $u\\in[\\tfrac{2}{3},1]$ since\n\\begin{equation*}\ng_1'(u)=2\\pi^2 u^3\\cosh\\pi u+(6\\pi u^2-4\\pi)\\sinh\\pi u>2\\pi(\\pi u^3+6u^2-2)\\sinh\\pi u>2\\pi\\sinh\\pi u>0.\n\\end{equation*}\nHence,\n$g_1(u)\\geqslant g_1\\left(\\frac{2}{3}\\right)>-9.05$.\nFor the function $g_2$ we have the following representation and estimate:\n\\begin{equation*}\ng_2(u)=1+\\sum\\limits_{j=1}^{4}(u^j+u^{-j})+\\sum\\limits_{j=5}^{8} u^j\\geqslant 9 +\\sum\\limits_{j=5}^{8}\\left(\\frac{2}{3}\\right)^j>9.31.\n\\end{equation*}\nTwo last estimates and (\\ref{3.12}) imply the positivity of the function $g$ for $u\\in[\\tfrac{2}{3},1)$.\n\\end{proof}\n\n\nDenote\n\\begin{align*}\ng_*(u,\\b):=&\\b u^3(1-3u^4)\\cosh\\b u \\sin \\b u^{-1}+\\b u^5 (3-u^4)\\sinh\\b u \\cos \\b u^{-1}\n\\\\\n&-2 u^4(1+u^4) \\cosh\\b u \\cos \\b u^{-1} -6(1+u^{12}).\n\\end{align*}\n\n\nThe next lemma states the existence of the second forbidden zone.\n\n\\begin{lemma}\\label{lm5.3}\nEquation (\\ref{4.4}) has no roots as $\\pi<\\b<\\b_*<5$, where $(u_*, \\b_*)$ is  the root of the system of the equations\n\\begin{equation}\\label{3.13}\ng(u,\\b)=0,\\qquad g_*(u,\\b)=0,\\qquad u\\in[0,1],\\qquad \\pi<\\b<5,\n\\end{equation}\nwith minimal possible $\\b$.\nTheir approximate values are\n\\begin{equation}\\label{3.14}\n\\b_*=4.808438,\\qquad u_*=0.611772.\n\\end{equation}\n\\end{lemma}\n\n\\begin{proof}\nThe function $g(u,\\pi)$ is positive on $[0,1)$ and $g(1,\\pi)=0$, see Figure~\\ref{fig:forbidden}a. As $\\pi<\\b<2\\pi$, we have $g(0,\\b)=1>0$ and $g(1,\\b)=-2\\sin\\b\\sinh\\b>0$. Hence, for $\\b$ close enough to $\\pi$, the function $g(u,\\b)$ is positive for all $u\\in[0,1]$. At the same time, we have $g(0.65,5)<-0.617<0$ and therefore, for $\\b=5$, equation (\\ref{4.4}) possesses at least two roots, one in $(0,0.65)$ and another in $(0.65,1)$. cf. Figure~\\ref{fig:forbidden}c. We also observe that the function $g$ is jointly continuous in $(u,\\b)$. The above facts means that as $\\b$ grows from $\\pi$ to $5$, at some value $\\b=\\b_*$, the graph of the function $g$ is still located in the upper half-plane but touches the $u$-axis at some point $u=u_*$,\nsee Figure~\\ref{fig:forbidden}b. The function $g(u,\\b)$ is positive as $\\pi<\\b<\\b_*$ and $u\\in[0,1]$. Then the point $u=u_*$ is obviously the global minimum of $g$ and hence, $(u_*,\\b_*)$ is a solution to the system of equations $g(u,\\b)=0$, $\\frac{\\p g}{\\p u}(u,\\b)=0$. It is easy to check that $g_*=\\frac{\\p g}{\\p u}-6 g$ and hence, $(u_*,\\b_*)$ solves system (\\ref{3.13}). These roots can be found numerically and this gives (\\ref{3.14}). The proof is complete.\n\\end{proof}\n\n\n\\begin{figure}\n\t\\centering\n\\includegraphics[width=0.99\\columnwidth]{fig03.eps}\n\\caption{Illustration for proof of Lemma~\\ref{lm5.3}. Graphs of the function $g(u, \\beta)$ for $\\b=\\pi$ (a), $\\b=\\b_*\\approx  4.808438$ (b), and $\\b=5$ (c). Notice broken vertical axes in (b) and (c). }\n\\label{fig:forbidden}\n\\end{figure}\n\nReturning from the auxiliary variable $\\beta$ to the gain-and-loss amplitude $\\gamma=\\beta^2/2$, from   Lemma~\\ref{lm5.3}  we deduce  the following important result:\n\\begin{equation}\n\\label{eq:gap}\n\\textrm{there is no spectral singularities for\\quad }  \\frac{\\pi^2}{2} < \\gamma< \\gamma_*\\approx  11.561.\n\\end{equation}\n\n\n\n",
        "label": "fig:forbidden",
        "Descriptive_question1": "What is the value of beta in figure_3 panel (a)?",
        "Descriptive_question2": "What feature is noted about the vertical axes in figure_3 panels (b) and (c)?",
        "Reasoning_question1": "How does the behavior of the function g(u, β) change across the panels in figure_3 as beta increases from π to 5?",
        "Reasoning_question2": "What does the touching of the u-axis by the graph of g(u, β) at β ≈ 4.808438 in figure_3 panel (b) imply about the existence of spectral singularities?",
        "Descriptive_answer1": "\\beta=\\pi",
        "Descriptive_answer2": "Broken vertical axes",
        "Reasoning_answer1": "As \\beta increases from \\pi in panel (a) through \\beta_* \\approx 4.808438 in panel (b) to 5 in panel (c), the function g(u, \\beta) changes from being positive on the entire interval [0,1) at \\beta=\\pi (no zeros), to touching the u-axis (g(u, \\beta)=0) at some internal point u_* in panel (b), indicating the emergence of a root. At \\beta=5 in panel (c), g(u, \\beta) crosses the u-axis twice, demonstrating the presence of two roots within the interval (0,1). This transition illustrates the creation of zeros of g(u, \\beta) as \\beta crosses the critical value \\beta_*, marking the boundary of the forbidden gap where spectral singularities can occur.",
        "Reasoning_answer2": "The touching of the u-axis by the graph of g(u, \\beta) at \\beta \\approx 4.808438 in panel (b) represents a critical point where the function achieves a zero with zero slope, i.e., a minimum that equals zero. This indicates that at this specific value \\beta=\\beta_*, the equation g(u, \\beta)=0 has a root, signifying the boundary of a 'forbidden zone.' For \\beta values below \\beta_*, no roots exist (no spectral singularities), while for values above \\beta_* , roots emerge. Therefore, this touching point marks the onset of existence of spectral singularities and defines a gap in \\beta where no spectral singularities can occur."
    },
    {
        "paper_id": "1908.06383.json",
        "image_id": "figure_4",
        "image_path": [
            "/home/yz979/scratch/chengye/scimolmo/test-images-tables/dataset_imagestables/1908.06383/images/fig04.eps"
        ],
        "caption": "(a) Minimal value of the gain-and-loss amplitude $\\gamma_\\textrm{min}$  which corresponds to   a spectral singularity with the given value of the wavenumber $k$. (b)  Minimal gain-to-loss distance  $\\ell_\\textrm{min}$  which corresponds to   a spectral singularity with the  $k$ and $\\gamma$ from the  left panel (bold curves) and larger distances $\\ell_n$ obtained using the periodicity in $\\ell$ (thin \\textcolor{black}{dotted} curves).",
        "classify": "Chart",
        "section_info": "5 Spectral singularities\n\\section{Spectral singularities}\n\\label{sec:ss}\n\\subsection{General analytical expressions}\n\nIn this section we study real zeroes of the function $F$ corresponding to spectral singularities, i.e. to  zero-width resonances.\nFor such zeroes, equation (\\ref{2.10}) is a pair of two real equations for one real variable $k$ and two parameters $\\ell$ and $\\g$.\nThanks to the symmetry of the zeroes with respect to the imaginary axis, it is sufficient to find only positive real resonances since the negative ones are located symmetrically with respect to the origin. Similar to the proof of Lemma~\\ref{lm3.6}, for real positive $k$  we make  change (\\ref{4.1})\nand rewrite equation (\\ref{2.10}) in the following form:\n\\begin{align*}\n(1-2u^{-4})\\cos\\b u^{-1}&+(2u^4-1)\\cosh\\b u + 2i \\sqrt{1-u^4}\\left(u^2\\sinh\\b u - u^{-4}\\sin\\b u^{-1}\\right)\n\\\\\n&-e^{-2i\\b\\ell\\sqrt{u^{-2}-u^2}} \\left(\\cosh(\\b u)-\\cos\\b u^{-1}\\right)=0,\\qquad \\b=\\sqrt{2\\g}.\n\\end{align*}\nTaking the real and imaginary part of this equation and multiplying the equation by $u^4$, we obtain:\n\\begin{equation}\\label{4.3}\n\\begin{aligned}\n&(u^4-2)\\cos\\b u^{-1}+u^4(2u^4-1)\\cosh\\b u\n=u^4\\cos\\left(2\\b\\ell\\sqrt{u^{-2}-u^2}\\right)\\left(\\cosh \\b u-\\cos\\b u^{-1}\\right),\n\\\\\n&2\\sqrt{1-u^4}\\Big(u^6\\sinh\\b u-\\sin\\b u^{-1}\\big)\n=-u^4\\sin\\left(2\\b\\ell\\sqrt{u^{-2}-u^2}\\right)\\left(\\cosh \\b u-\\cos\\b u^{-1}\\right).\n\\end{aligned}\n\\end{equation}\nThis is a system of two real equations with three real variables. If we are given $(\\b,\\ell)$ and we try to find $u$, the system is overdetermined and does not necessary have a root. In other words, it is solvable with respect to $u$ only if $(\\b,\\ell)$ are located on some (solvability) curves. In order to avoid working with an overdetermined system, in what follows we regard (\\ref{4.3}) as a system for two unknown variable with one parameter.\n\nTo find the curves in $(\\b,\\ell)$ plane, on which equations (\\ref{4.3}) are solvable with respect to $u$, we shall regard $u$ as a parameter and $(\\b,\\ell)$ as unknown variables. We take the sum of squares of equations (\\ref{4.3}) and divide the result by $(1-u^4)$. We also divide equations (\\ref{4.3}). This leads us to a   pair  of equations:\n\\begin{align}\\label{4.4}\n&u^4(1-u^4) \\cosh\\b u\\cos\\b u^{-1}-2u^6 \\sinh\\b u\\sin\\b u^{-1} +1-u^{12}=0,\n\\\\\n&\\frac{2\\sqrt{1-u^4}(u^6\\sinh\\b u-\\sin\\b u^{-1})}{(2-u^4)\\cos\\b u^{-1}+u^4(1-2u^4)\\cosh\\b u}=\\tan \\Big( 2\\b\\ell\\sqrt{u^{-2}-u^2}\\Big).\\nonumber\n\\end{align}\nThe second equation can be solved explicitly with respect to $\\ell$:\n\\begin{equation}\\label{4.10}\n\\ell=\\frac{1}{2\\b\\sqrt{u^{-2}-u^2}} \\left(\\arctan \\frac{2\\sqrt{1-u^4}(u^6\\sinh\\b u-\\sin\\b u^{-1})}{(2-u^4)\\cos\\b u^{-1}+u^4(1-2u^4)\\cosh\\b u}+\\pi n\\right),\n\\end{equation}\nwhere $n\\in\\mathds{N}$ is an arbitrary natural number. As the next lemma states, to make equations (\\ref{4.4}), (\\ref{4.10}) equivalent to (\\ref{4.3}), we should also assume that\n\\begin{equation}\\label{4.5}\n(-1)^n=\\sign \\big((u^4-2)\\cos\\b u^{-1}+u^4(2u^4-1)\\cosh\\b u\\big).\n\\end{equation}\n\n\\begin{lemma}\\label{lm4.1}\nEquations (\\ref{4.3}) are equivalent to (\\ref{4.4}), (\\ref{4.10}), (\\ref{4.5}).\n\\end{lemma}\n\n\\begin{proof}\nWe rewrite shortly equations (\\ref{4.3}) as\n$A_1=B\\cos\\a$, $ A_2=-B\\sin\\a$,\nwhere $A_1$, $A_2$ are the left hand sides in (\\ref{4.3}), $\\a=2\\b\\ell\\sqrt{u^{-2}-u^2}$ and $B=u^4(\\cosh\\b u-\\cos\\b u^{-1})$. Then equations (\\ref{4.4}), (\\ref{4.10}) become\n$A_1^2+A_2^2=B^2$, $\\a=-\\arctan\\frac{A_2}{A_1}+\\pi n$.\nWe have:\n\\begin{equation*}\nB\\cos \\a=(-1)^n\\cos\\arctan\\frac{A_2}{A_1}=\\frac{(-1)^n}{\\sqrt{1+\\frac{A_2^2}{A_1^2}}} = \\frac{(-1)^n|A_1|}{\\sqrt{A_1^2+A_2^2}}\n\\end{equation*}\nand we get the first equation $A_1=B\\cos\\a$  provided condition (\\ref{4.5}) is satisfied. In the same way we check that the latter condition also ensures the second equation $A_2=-B\\sin\\a$.\n\\end{proof}\n\n\nEquation (\\ref{4.4}) is transcendental, and we can not solve it analytically.\nNevertheless, for each $u\\in(0,1)$, this is an equation only for a single variable $\\b$, not for two as equations (\\ref{4.3}). So, we propose the following algorithm of recovering the aforementioned solvability curves: choose $u\\in(0,1)$, then solve equation (\\ref{4.4}) and recover the sequence of  distances  $\\ell$ by formula (\\ref{4.10}) with different integer $n$. Then the gain-and-loss amplitude $\\gamma$ and the corresponding wavenumber $k$ can be readily recovered from $\\beta$ and  $u$.  In a similar way, one can first fix some value of $\\beta$ (i.e., fix the gain-and-loss strength) and then   solve equation (\\ref{4.4}) with respect to $u$ and   recover   $\\ell$ by (\\ref{4.10}). Equation (\\ref{4.4})  is well-behaved, and for each $\\beta$  all its zeros  $u$ can be easily found numerically.\n\nAlternatively, as explained below  in Section~\\ref{sec:general}, the values  corresponding to  spectral singularities can be found systematically by means of the numerical continuation from the limit $\\ell=0$. However, in this case   equation (\\ref{4.4}) is still useful because it allows one to check that all spectral singularities have been found for the given value of the gain-and-loss $\\gamma$.\n\n\n\\subsection{\nAbsence of   spectral singularities}\n\\label{sec:gap}\n\nFor $u=0$ and $u=1$, the left-hand-side of equation (\\ref{4.4}) is equal respectively to $1$ and $-2\\sinh\\beta\\sin\\beta$. Then a sufficient condition for the existence of a spectral singularity at the given gain-and-loss amplitude $\\gamma$ is $\\sin\\beta=\\sin\\sqrt{2\\gamma}>0$. At the same time, it is also possible to establish sufficient conditions that forbid the existence of   spectral singularities in a certain interval of parameters. In this subsection we prove the existence of two ``forbidden gaps'' for the roots of equation (\\ref{4.4}). The first one exists for all $\\b\\geqslant 0$ and it states that there is no roots in certain interval. The second gap is a certain interval of values of $\\b$, for which equation (\\ref{4.4}) has no zeroes at all.\n\nFor the convenience, by $g(u,\\b)$ we denote the left hand side of equation (\\ref{4.4}). The first ``forbidden gap'' is described in the following lemma.\n\n\\begin{lemma}\\label{lm5.1}\nFor all $\\b\\geqslant 0$, equation (\\ref{4.4}) has no roots in the interval $\\big[0,(1+\\tfrac{\\b}{4})^{-1}\\big)$.\n\\end{lemma}\n\\begin{proof}\nEmploying a standard inequality $a\\cos\\a+b\\sin\\a\\leqslant \\sqrt{a^2+b^2}$, we estimate the first two term in equation (\\ref{4.4}) as\n\\begin{equation*}\nu^4(1-u^4) \\cosh\\b u\\cos\\b u^{-1}-2u^6 \\sinh\\b u\\sin\\b u^{-1} \\leqslant \\sqrt{u^8(1-u^4)^2\\cosh^2 \\b u+4u^{12}\\sinh^2\\b u}.\n\\end{equation*}\nHence, equation (\\ref{4.4}) surely has no roots for values of $u$ satisfying\n\\begin{equation*}\n \\sqrt{u^8(1-u^4)^2\\cosh^2 \\b u+4u^{12}\\sinh^2\\b u} <1-u^{12}.\n\\end{equation*}\nExpressing $\\cosh^2 \\b u$ via $\\sinh^2 \\b u$ and simplifying this inequality, we obtain $ u^4(1+u^4)\\cosh \\b u <1+u^{12}$ and hence,\n\\begin{equation}\\label{3.37}\n\\cosh \\b u-1 <\\frac{1-2u^4+u^8}{u^4},\\qquad \\sqrt{2}\\sinh \\b u\n<u^{-2}-u^2.\n\\end{equation}\nThanks to the estimate $\\sqrt{2}\\sinh 2\\ln u^{-1}\\leqslant u^{-2}-u^2$, $u\\in(0,1]$, the last inequality in (\\ref{3.37}) holds once $\\frac{\\b}{4}<u^{-1}\\ln u^{-1}$. It is easy to confirm that this inequality is true once $u^{-1}>1+\\tfrac{\\b}{4}$.\nThe proof is complete.\n\\end{proof}\n\n\nThe next lemma is   auxiliary  and will be employed in studying the second forbidden zone.\n\n\\begin{lemma}\\label{lm5.2}\nThe function $g(u,\\pi)$ is positive on $[0,1)$.\n\\end{lemma}\n\n\\begin{proof}\nWe have $g(0,\\pi)=1$ and by Lemma~\\ref{lm5.1}, it is positive for $u<(1+\\tfrac{\\pi}{4})^{-1}$. This is why in what follows we consider only the values $u\\geqslant (1+\\tfrac{\\pi}{4})^{-1}$. For such values of $u$ we have\n$\\pi\\leqslant \\pi u^{-1}\\leqslant \\pi (1+\\tfrac{\\pi}{4})<1.79\\pi$.\nAs\n$\\tfrac{3\\pi}{2} \\leqslant \\pi u^{-1}\\leqslant \\pi (1+\\tfrac{\\pi}{4})$,\nthe function $\\sin\\b u^{-1}$ is negative, while $\\cos \\b u^{-1}$ is positive. Hence, for such values of $u$, the function $g(u,\\pi)$ is positive. It remains to consider the values $\\frac{2}{3}<u\\leqslant 1$.\n\nFor such values of $u$ we first observe the following simple estimates:\n\\begin{equation*}\n\\frac{1-u^4}{1-u}\\cos\\pi u^{-1}\\geqslant -4,\\qquad -\\frac{\\sin \\pi u^{-1}}{1-u}=\\frac{\\sin \\pi (u^{-1}-1)}{1-u}\\geqslant \\pi u.\n\\end{equation*}\nThen the function $g$ satisfies the estimate:\n\\begin{equation}\\label{3.12}\n\\frac{g(u,\\pi)}{u^4(1-u)}\\geqslant g_1(u)+g_2(u),\\qquad g_1(u):=-4\\cosh\\pi u +2\\pi u^3 \\sinh \\pi u,\\qquad g_2(u):=\\frac{1-u^{12}}{u^4(1-u)}.\n\\end{equation}\nThe function $g_1(u)$ is monotonically increasing in $u\\in[\\tfrac{2}{3},1]$ since\n\\begin{equation*}\ng_1'(u)=2\\pi^2 u^3\\cosh\\pi u+(6\\pi u^2-4\\pi)\\sinh\\pi u>2\\pi(\\pi u^3+6u^2-2)\\sinh\\pi u>2\\pi\\sinh\\pi u>0.\n\\end{equation*}\nHence,\n$g_1(u)\\geqslant g_1\\left(\\frac{2}{3}\\right)>-9.05$.\nFor the function $g_2$ we have the following representation and estimate:\n\\begin{equation*}\ng_2(u)=1+\\sum\\limits_{j=1}^{4}(u^j+u^{-j})+\\sum\\limits_{j=5}^{8} u^j\\geqslant 9 +\\sum\\limits_{j=5}^{8}\\left(\\frac{2}{3}\\right)^j>9.31.\n\\end{equation*}\nTwo last estimates and (\\ref{3.12}) imply the positivity of the function $g$ for $u\\in[\\tfrac{2}{3},1)$.\n\\end{proof}\n\n\nDenote\n\\begin{align*}\ng_*(u,\\b):=&\\b u^3(1-3u^4)\\cosh\\b u \\sin \\b u^{-1}+\\b u^5 (3-u^4)\\sinh\\b u \\cos \\b u^{-1}\n\\\\\n&-2 u^4(1+u^4) \\cosh\\b u \\cos \\b u^{-1} -6(1+u^{12}).\n\\end{align*}\n\n\nThe next lemma states the existence of the second forbidden zone.\n\n\\begin{lemma}\\label{lm5.3}\nEquation (\\ref{4.4}) has no roots as $\\pi<\\b<\\b_*<5$, where $(u_*, \\b_*)$ is  the root of the system of the equations\n\\begin{equation}\\label{3.13}\ng(u,\\b)=0,\\qquad g_*(u,\\b)=0,\\qquad u\\in[0,1],\\qquad \\pi<\\b<5,\n\\end{equation}\nwith minimal possible $\\b$.\nTheir approximate values are\n\\begin{equation}\\label{3.14}\n\\b_*=4.808438,\\qquad u_*=0.611772.\n\\end{equation}\n\\end{lemma}\n\n\\begin{proof}\nThe function $g(u,\\pi)$ is positive on $[0,1)$ and $g(1,\\pi)=0$, see Figure~\\ref{fig:forbidden}a. As $\\pi<\\b<2\\pi$, we have $g(0,\\b)=1>0$ and $g(1,\\b)=-2\\sin\\b\\sinh\\b>0$. Hence, for $\\b$ close enough to $\\pi$, the function $g(u,\\b)$ is positive for all $u\\in[0,1]$. At the same time, we have $g(0.65,5)<-0.617<0$ and therefore, for $\\b=5$, equation (\\ref{4.4}) possesses at least two roots, one in $(0,0.65)$ and another in $(0.65,1)$. cf. Figure~\\ref{fig:forbidden}c. We also observe that the function $g$ is jointly continuous in $(u,\\b)$. The above facts means that as $\\b$ grows from $\\pi$ to $5$, at some value $\\b=\\b_*$, the graph of the function $g$ is still located in the upper half-plane but touches the $u$-axis at some point $u=u_*$,\nsee Figure~\\ref{fig:forbidden}b. The function $g(u,\\b)$ is positive as $\\pi<\\b<\\b_*$ and $u\\in[0,1]$. Then the point $u=u_*$ is obviously the global minimum of $g$ and hence, $(u_*,\\b_*)$ is a solution to the system of equations $g(u,\\b)=0$, $\\frac{\\p g}{\\p u}(u,\\b)=0$. It is easy to check that $g_*=\\frac{\\p g}{\\p u}-6 g$ and hence, $(u_*,\\b_*)$ solves system (\\ref{3.13}). These roots can be found numerically and this gives (\\ref{3.14}). The proof is complete.\n\\end{proof}\n\n\n\\begin{figure}\n\t\\centering\n\\includegraphics[width=0.99\\columnwidth]{fig03.eps}\n\\caption{Illustration for proof of Lemma~\\ref{lm5.3}. Graphs of the function $g(u, \\beta)$ for $\\b=\\pi$ (a), $\\b=\\b_*\\approx  4.808438$ (b), and $\\b=5$ (c). Notice broken vertical axes in (b) and (c). }\n\\label{fig:forbidden}\n\\end{figure}\n\nReturning from the auxiliary variable $\\beta$ to the gain-and-loss amplitude $\\gamma=\\beta^2/2$, from   Lemma~\\ref{lm5.3}  we deduce  the following important result:\n\\begin{equation}\n\\label{eq:gap}\n\\textrm{there is no spectral singularities for\\quad }  \\frac{\\pi^2}{2} < \\gamma< \\gamma_*\\approx  11.561.\n\\end{equation}\n\n\n\n\\subsection{Creating a  spectral singularity  at a given wavenumber}\n\n\\begin{table}\n\t\\centering\n\t\\begin{tabular}{c|cccc}\n\t\t$n$ & 0 & 1& 2&\\\\\\hline\n\t\t$\\gamma_\\star^{(n)}$ & 2.071&13.307&27.783 &\\\\[2mm]\n\t\t$k_\\star^{(n)}$ & 1.065&4.318 &7.529 &\n\t\\end{tabular}\n\t\\caption{Approximate values of gain-and-loss amplitudes $\\gamma=\\gamma_\\star^{(n)}$  and wavenumbers   $k=k_\\star^{(n)}$, $n=0, 1, \\ldots$,  corresponding to spectral singularities with lowest $\\gamma$ in the limit $\\ell=0$, see Table~I in  \\cite{Mostafazadeh2009}. \\label{tbl:1}}\n\\end{table}\n\n\nFor $\\ell=0$ spectral singularity can only be obtained for some isolated values of the wavenumber $k$ and the gain-and-loss amplitude $\\gamma$ \\cite{Mostafazadeh2009}. Several lowest values of $\\gamma$ corresponding to the spectral singularities  and the associated wavenumbers  $k$ are listed in Table~\\ref{tbl:1}.  An  important advantage of the more general system with nonzero gain-to-loss disctance    $\\ell>0$ consists  in the possibility to create a spectral   singularity   at any wavenumber $k$ given beforehand. Indeed, let us  return back to equations (\\ref{4.4}), (\\ref{4.10}) and    discuss the following   issue: given a point $k$ on the real axis, how to choose $\\b$ and $\\ell$ to have a  resonance at this point? Equations (\\ref{4.4})--(\\ref{4.10}) allow us to answer  easily this question.\n\nWe fix $k>0$ and we find the associated value of $u$ by resolving (\\ref{4.1}):\n\\begin{equation}\\label{4.12}\nu^{-2}-u^2=4k^2\\b^{-2},\\qquad\n\nu=\\b R^{-1}, \\quad \\mbox{where\\ } R = \\sqrt{2k^2+\\sqrt{4k^4+\\b^4}}.\n\\end{equation}\nWe divide equation (\\ref{4.4}) by $u^6$ and substitute then the above formulae and\n\\begin{equation*}\n\\frac{1-u^{12}}{u^6}=\\frac{1-u^4}{u^2}\\frac{1+u^4+u^8}{u^4}=(u^{-2}-u^2)\\big((u^{-2}-u^2)^2+3\n\\big).\n\\end{equation*}\nThis gives the equation:\n\\begin{equation}\\label{4.11}\n\\begin{aligned}\n2\\b^4 k^2&\\cosh(\\b^2R^{-1})  \\cos R\n\n\n - \\b^6 \\sinh(\\b^2R^{-1})  \\sin R\n  +2k^2(16k^4+3\\b^4)=0.\n\\end{aligned}\n\\end{equation}\nAn algorithm for creating a resonance at a prescribed point  $k$ is as follows. Given $k>0$, we first solve equation (\\ref{4.11}) with respect to $\\b$ and we also find $u$ by (\\ref{4.12}). Then needed values of  $\\ell$ are determined by (\\ref{4.10}), (\\ref{4.5}).\n\n\nIn order to illustrate this  algorithm,  we us consider a finite interval of wavenumbers $k\\in (0, k_1]$, where  we set $k_1 = 10$ for the numerics reported on in what follows. We scan   the chosen interval  with a sufficiently small step ($\\Delta k=0.01$) and for each value of $k$ solve equation (\\ref{4.11}) numerically  using the simple dichotomy method. While for each $k$  equation (\\ref{4.11}) might have several roots $\\beta$, in our numerical procedure we always choose the  minimal positive root, i.e., the one which allows to achieve the spectral singularity with given $k$ at the smallest possible value  of the gain-and-loss amplitude $\\gamma=\\gamma_\\textrm{min}$. Next, we choose the minimal positive distance $\\ell_\\textrm{min}$ which satisfies the conditions   (\\ref{4.10}), (\\ref{4.5}) and then we use the periodicity in $\\ell$ to generate the sequence of larger gain-to-loss distances $\\ell_n = \\ell_\\textrm{min} + n\\pi/(2k)$, $n=1,2\\ldots$ [see   (\\ref{eq:periodic})]. The resulting dependencies $\\gamma_\\textrm{min}(k)$ and $\\ell_\\textrm{min}(k)$, $\\ell_n(k)$   are shown in figure~\\ref{fig:min}. The minimal gain-and-loss amplitude $\\g_{min}(k)$  and the minimal gain-to-loss distance $\\ell_{min}$ are   discontinuous,  which means that the small variation  in  the wavenumuber $k$  might require  a significant  change either in   $\\gamma$ or in $\\ell$.  It is especially important that the values of the  distance  $\\ell$ are generically different from zero, which points out explicitly    that the new degree of freedom offered by the nonzero gain-to-loss distance    is   important for the achieving a spectral singularity  at   the given wavenumber $k$.\n\n\n\\begin{figure}\n\t\\centering\n\t\\includegraphics[width=0.8\\columnwidth]{fig04.eps}\n\t\\caption{(a) Minimal value of the gain-and-loss amplitude $\\gamma_\\textrm{min}$  which corresponds to   a spectral singularity with the given value of the wavenumber $k$. (b)  Minimal gain-to-loss distance  $\\ell_\\textrm{min}$  which corresponds to   a spectral singularity with the  $k$ and $\\gamma$ from the  left panel (bold curves) and larger distances $\\ell_n$ obtained using the periodicity in $\\ell$ (thin \\textcolor{black}{dotted} curves).}\n\t\\label{fig:min}\n\\end{figure}\n\n\n\\subsection{$\\PT$-symmetry breaking laser-antilaser threshold}\n\nA particularly important characteristics of any $\\PT$-symmetric structure is the $\\PT$ symmetry breaking threshold, i.e., the amplitude of the gain-and-loss\ncorresponding to the  ``phase transition'' from the purely real to complex spectrum.  The best studied scenario of the phase transition  \nis the collision of two real discrete  eigenvalues at an exceptional point with the subsequent splitting in a complex-conjugate pair. However, in systems with nonempty continuous spectrum,  the phase transition can also occur through   the splitting of a self-dual spectral singularity, which results in a bifurcation of a complex-conjugate pair from an interior point of the continuum \\cite{Yang17,KZ17,Konotop2019}. At the moment corresponding to the formation of the spectral singularity, the system operates in the CPA-laser regime \\cite{Longhi10}. Thus, in such a system, the $\\PT$-symmetry breaking threshold at the same time corresponds to the CPA-laser threshold.\n\nLemma~\\ref{lm3.3} guarantees that the spectrum of our system is real for sufficiently small gain-and-loss amplitudes $\\gamma$. Additionally, according to Lemma~\\ref{lm3.5}, the spectrum  does not have any real discrete eigenvalue. Hence, the $\\PT$-symmetry breaking is expected to occur through the emergence of a self-dual spectral singularity. In order to identify the $\\PT$-symmetry breaking threshold in our system, we start from the limit $\\ell=0$, there the phase transition takes place at  $\\gamma_\\star^{(0)} \\approx 2.072$, see \\cite{Mostafazadeh2009,KZ17} and Table~\\ref{tbl:1}. Thus, the spectrum with $\\ell=0$ is purely real and continuous for $\\gamma \\in [0, \\gamma_\\star]$, while the  increase of the gain-and-loss just above $\\gamma_\\star^{(0)}$ leads to the bifurcation of a complex-conjugate pair from an interior point  of the continuum. The spectral singularity forming at  $\\gamma_\\star^{(0)}$ takes place at wavenumber $k=k_\\star^{(0)} \\approx  1.065$. Respectively, the complex-conjugate pair of eigenvalues bifurcates from $\\lambda_0 =  [k_\\star^{(0)}]^2$. (Notice that the further increase of $\\gamma$ above the next threshold values listed in Table~\\ref{tbl:1} leads to the formation of new spectral singularities and, respectively, to bifurcations of new complex-conjugate pairs in the spectrum.)\n\nNext, we use the numerical continuation in $\\ell$ in order to continue the known solution at $\\ell=0$ to the domain $\\ell>0$.  The obtained dependence of the threshold value  of the gain-and-loss amplitude on distance $\\ell$ is shown in figure~\\ref{fig:threshold}(a), where we observe that the phase transition threshold decreases monotonically with the growth of $\\ell$. This means that introducing an additional space   between the gain and loss, one can decrease the $\\PT$-symmetry breaking  threshold, i.e. achieve the laser-antilaser operation at \\textcolor{black}{lower} gain-and-loss amplitudes than in a waveguide with the adjacent gain and loss.\n\n\n\\begin{figure}\n\t\\centering\n\t\\includegraphics[width=0.85\\columnwidth]{fig05.eps}\n\t\\caption{(a) $\\PT$-symmetry breaking threshold $\\gamma_\\star^{(0)}$ \\textit{vs} the distance between the gain and loss $\\ell$. The spectrum is purely real and continuous for $\\gamma\\leq \\gamma_\\star$, but acquires a pair of complex conjugate eigenvalues  as the gain-and-loss amplitude exceeds the threshold $\\gamma_\\star^{(0)}$. (b) Values of the wavevector $k_\\star^{(0)}$ corresponding to the dependence in (a).}\n\t\\label{fig:threshold}\n\\end{figure}\n\n\\subsection{General picture of spectral singularities}\n\\label{sec:general}\n\n\\begin{figure}\n\t\\centering\n\t\\includegraphics[width=0.99\\columnwidth]{fig06.eps}\n\t\\caption{(a) Values of the gain-and-loss amplitude $\\gamma$ and distance  $\\ell$, for which   spectral singularities occur. (b) Corresponding  wavenumbers   $k$. \\textcolor{black}{Panel (c) magnifies   the region  $(\\ell, \\gamma)\\in [2,4]\\times[10,50]$ from   (a), and panel  (d) is the  magnification of corresponding curves from panel (b).}}\n\t\\label{fig:general}\n\\end{figure}\n\nLet us now turn to description of the general picture of spectral singularities. In order to construct systematically different solutions, we again start from the limit $\\ell=0$, where the values of $\\gamma$ and $k$ corresponding to spectral singularities are known, see Table~\\ref{tbl:1}. Then we use the periodicity of function $e^{-4ik\\ell}$ in $\\ell$ in order to construct new branches of solutions having no counterparts in the limit $\\ell=0$, cf. equation (\\ref{eq:periodic}). This procedure results in a fairly complicated picture containing a multitude of spectral singularities, some part of which (corresponding to  relatively  small   values of the gain-and-loss) is shown in figure~\\ref{fig:general}(a,b) as the curves on the plane $\\gamma$ {\\it vs} $\\ell$ and $k$ {\\it vs} $\\ell$.\n\n\nLet us describe the structure of the found solutions using the  diagram $\\gamma$ {\\it vs} $\\ell$ in figure~\\ref{fig:general}(a). The multitude of curves shown in this plot can be divided into three groups (plotted with red, blue and green curves) which have been obtained by means of the continuation from three different solutions in the limit $\\ell=0$. There is a well-visible vertical gap between red and blue curves, which corresponds to the ``forbidden'' values of the gain-and-loss amplitudes $\\gamma$ found above in (\\ref{eq:gap}). At the same time,  there is no gap between blue and green curves, which  results in a multitude of intersections between these curves.\nThe first group of spectral singularities,  corresponding to red curves in Figure~\\ref{fig:general}(a),   is obtained through  the continuation from the  spectral singularity  in the limit $\\ell=0$ with the smallest gain-and-loss amplitude, i.e.  from values $\\gamma_\\star^{(0)}$ and $k_\\star^{(0)}$ in Table~\\ref{tbl:1}.  The leftmost (bold)  curve in this group which originates in the limit $\\ell=0$  is the $\\PT$-symmetry breaking threshold which was already shown in figure~\\ref{fig:threshold}(a). For values of $\\gamma$ above this curve, there is always one or more (but finitely many, see Section~\\ref{sec:zeros})   complex-conjugate pairs of eigenvalues in the spectrum.  Several curves situated to the right from the bold red curve are obtained using the fact that if $\\gamma$ and $k$ are solutions in the limit $\\ell=0$, then the same values of $\\gamma$ and $k$ also correspond to a spectral singularity with $\\ell_n = (n\\pi)/(2k)$, $n=1,2,\\ldots$. Once a single solution with a new distance   $\\ell_n$ is obtained, a new branch of solutions can be constructed using numerical continuation in $\\gamma$ or in $\\ell$. In the limit $\\ell \\to \\infty$ all the red curves in figure~\\ref{fig:general}(a) approach  the asymptotic value $\\gamma=0$ or at $\\gamma_*= \\pi^2/2\\approx 4.935$. Notice that, except for the bold line demarcating the $\\PT$-symmetry breaking threshold, none of the red curves can be continued to the limit $\\ell\\to 0$.\n\nThe multitude of red curves  in figure~\\ref{fig:general}(a) demonstrates explicitly how new spectral singularities emerge with the increase of $\\ell$. Indeed, drawing an imaginary horizontal line, say, at $\\gamma=3$ [see the vertical axis  tick  in figure~\\ref{fig:general}(a)], we observe that with the increase of $\\ell$ this line   intersects more and more red curves. Each intersection corresponds to the values of $\\gamma$ and $\\ell$ at which   some  root $k$   crosses the real axis and goes down from the upper to lower complex half-plane. Thus,  a finite-width resonance transforms  to a complex eigenvalue through the spectral singularity (we recall again that  in view of   $\\PT$ symmetry  any root $k\\ne 0$ crosses the real line simultaneously with its counterpart $-\\bar{k}$, i.e.  the corresponding   spectral singularity is self-dual).\nIn order to illustrate this process, in Figure~\\ref{fig:g=3}  we show the evolution of three numerically found complex zeros of function $F(k, \\gamma, \\ell)$  under   the increase of $\\ell$.  In this figure the imaginary part of each complex zero changes sign from positive to negative and then asymptotically approaches zero (remaining negative). In the limit of large $\\ell$,  this behavior agrees  with expansion (\\ref{3.4}) of Lemma~\\ref{lm3.4} , where, for the chosen value of $\\gamma$,   we have $\\sin\\sqrt{2\\gamma}>0$.   Thus the growing distance   $\\ell$ results in a sequence of self-dual spectral singularities and in the increasing number of complex-conjugate eigenvalues in the spectrum.\n\n \\begin{figure}\n \t\\centering\n \t\\includegraphics[width=0.8\\columnwidth]{fig07.eps}\n \t\\caption{(a,b) Real and imaginary parts of three complex zeros of function $F$ for fixed $\\gamma=3$ and increasing $\\ell$. For each curve, the imaginary part is positive for small $\\ell$ and becomes negative for all sufficiently large $\\ell$. \\textcolor{black}{For each three shown eigenvalues, its real and \timaginary parts are of the same colour in both panels.}}\n \t\\label{fig:g=3}\n \\end{figure}\n\n\nThe second group of spectral singularities [blue curves in figure~\\ref{fig:general}(a)] was obtained by means of the continuation from the  next solution in the limit $\\ell=0$, i.e.  from $\\gamma_\\star^{(1)}$ and $k_\\star^{(1)}$ in Table~\\ref{tbl:1}. Again, one of the curves [the leftmost bold curve in figure~\\ref{fig:general}(a)] was obtained through the direct continuation from the limit $\\ell=0$, while other blue curves were generated  using the periodicity of function $F(k, \\gamma, \\ell)$  in $\\ell$ and cannot be continued to the limit $\\ell\\to0$. In the limit $\\ell\\to\\infty$ the blue curves approach the horizontal asymptotes  $\\gamma=2\\pi^2\\approx19.739$ and $\\gamma=9\\pi^2/2\\approx 44.413$. In  the $(\\gamma, \\ell)$-plane  the  gain-and-loss amplitudes corresponding to the   group of blue curves are well separated from  those for the red curves: indeed, all red curves are bounded from above by the asymptote $\\gamma= \\pi^2/2\\approx 4.935$, while all blue curves are bounded from below by $\\gamma_*\\approx 11.561$, see (\\ref{eq:gap}). Thus, the  emergence of new spectral singularities with the increase of $\\ell$ is sensitive to the value of the gain-and-loss amplitude and does not occur for     the gain-and-loss amplitudes lying in the  gap  between the red and blue curves.\n\nIn comparison with the red curves discussed above, the curves from the blue group in figure~\\ref{fig:general}(a) feature more complicated behavior and, in particular, can intersect each other (and also intersect the curves from the next, third group of green curves discussed below). The intersections between the blue curves occur for the gain-and-loss amplitudes in the interval $11.561 \\lessapprox   \\gamma <  9\\pi^2/2\\approx 19.739$, where $\\sin\\sqrt{2\\gamma}$ is negative.  \\textcolor{black}{At first glance,} this might seem to contradict to  the  expansion (\\ref{3.4}) of Lemma~\\ref{lm3.4},  which  suggests that in this case the multitude of complex zeros   accumulate in the upper complex half-plane with the growth of $\\ell$. However, this apparent contradiction is resolved if we trace the behavior of the complex roots more closely.   Indeed, choosing for an example $\\gamma =16$ [see the vertical axis tick in  figure~\\ref{fig:general}(a)] and computing several  complex roots under the increase of $\\ell$, we observe that each considered root first goes down from the upper half-plane to the lower one but then again returns to the upper half-plane, see Figure~\\ref{fig:g=16}(b) and (b$_1$). Thus,  in  this  interval of the gain-and-loss amplitudes the increase of $\\ell$  results  either in the transformation from the resonance to the eigenvalue or to the opposite process,  i.e. to the disappearance of the complex-conjugate pair.  Respectively, the intersection between the two blue curves corresponds to  two coexisting spectral singularities, i.e. to the moment  when one complex-conjugate pair of eigenvalues disappears and another pair (with different $k$) emerges. For sufficiently large $\\ell$ the imaginary part  of each   considered root  remains positive and approaches zero, in accordance with expansion (\\ref{3.4}). Thus, in this interval of the gain-and-loss strengths, the limit $\\ell\\to\\infty$ the spectrum contains only a finite number of complex-conjugate eigenvalues, which correspond to complex zeros $k$ whose behavior is not covered   by expansion  (\\ref{3.4}).\n\n \\begin{figure}\n\t\\centering\n\t\\includegraphics[width=0.99\\columnwidth]{fig08.eps}\n\t\\caption{(a,b) Real and imaginary parts for three   complex zeros of function $F$ for fixed $\\gamma=16$ and increasing $\\ell$. Panel b$_1$ is the magnification of some region of (b) and  shows more clearly that imaginary parts of all   three shown eigenvalues are positive for all sufficiently large  $\\ell$. \\textcolor{black}{For each three shown eigenvalues, its real and \timaginary parts are of the same colour in both panels.}}\n\t\\label{fig:g=16}\n\\end{figure}\n\nThe third  group of spectral singularities [green curves in figure~\\ref{fig:general}(a)] was obtained by means of the continuation from  the solution $\\gamma_\\star^{(2)}$ and $k_\\star^{(2)}$ in Table~\\ref{tbl:1}. In view of the very complicated structure of the overall resulting picture, we only show a section of these curves corresponding to relatively small   values of the gain-and-loss amplitudes $\\gamma$. Quite interestingly, there is no gap between the blue and green groups of curves, which results in the multitude of intersections between blue and green curves, \\textcolor{black}{see Fig.~\\ref{fig:general}(a) and the magnified view in Fig.~\\ref{fig:general}(c)}. These   intersections suggest a possibility of   simultaneous emergence of two complex-conjugate pairs of eigenvalues from two different interior points of the continuous spectra.\n\nConsidering further solutions $\\gamma_\\star^{(n)}$, $k_\\star^{(n)}$, $n=3,4,\\ldots$, in the limit $\\ell=0$ one can construct new groups of spectral singularities  with larger values of $\\gamma$, which are not shown in Figure~\\ref{fig:general}.\n\n\n5.3 Creating a  spectral singularity  at a given wavenumber\n\\subsection{Creating a  spectral singularity  at a given wavenumber}\n\n\\begin{table}\n\t\\centering\n\t\\begin{tabular}{c|cccc}\n\t\t$n$ & 0 & 1& 2&\\\\\\hline\n\t\t$\\gamma_\\star^{(n)}$ & 2.071&13.307&27.783 &\\\\[2mm]\n\t\t$k_\\star^{(n)}$ & 1.065&4.318 &7.529 &\n\t\\end{tabular}\n\t\\caption{Approximate values of gain-and-loss amplitudes $\\gamma=\\gamma_\\star^{(n)}$  and wavenumbers   $k=k_\\star^{(n)}$, $n=0, 1, \\ldots$,  corresponding to spectral singularities with lowest $\\gamma$ in the limit $\\ell=0$, see Table~I in  \\cite{Mostafazadeh2009}. \\label{tbl:1}}\n\\end{table}\n\n\nFor $\\ell=0$ spectral singularity can only be obtained for some isolated values of the wavenumber $k$ and the gain-and-loss amplitude $\\gamma$ \\cite{Mostafazadeh2009}. Several lowest values of $\\gamma$ corresponding to the spectral singularities  and the associated wavenumbers  $k$ are listed in Table~\\ref{tbl:1}.  An  important advantage of the more general system with nonzero gain-to-loss disctance    $\\ell>0$ consists  in the possibility to create a spectral   singularity   at any wavenumber $k$ given beforehand. Indeed, let us  return back to equations (\\ref{4.4}), (\\ref{4.10}) and    discuss the following   issue: given a point $k$ on the real axis, how to choose $\\b$ and $\\ell$ to have a  resonance at this point? Equations (\\ref{4.4})--(\\ref{4.10}) allow us to answer  easily this question.\n\nWe fix $k>0$ and we find the associated value of $u$ by resolving (\\ref{4.1}):\n\\begin{equation}\\label{4.12}\nu^{-2}-u^2=4k^2\\b^{-2},\\qquad\n\nu=\\b R^{-1}, \\quad \\mbox{where\\ } R = \\sqrt{2k^2+\\sqrt{4k^4+\\b^4}}.\n\\end{equation}\nWe divide equation (\\ref{4.4}) by $u^6$ and substitute then the above formulae and\n\\begin{equation*}\n\\frac{1-u^{12}}{u^6}=\\frac{1-u^4}{u^2}\\frac{1+u^4+u^8}{u^4}=(u^{-2}-u^2)\\big((u^{-2}-u^2)^2+3\n\\big).\n\\end{equation*}\nThis gives the equation:\n\\begin{equation}\\label{4.11}\n\\begin{aligned}\n2\\b^4 k^2&\\cosh(\\b^2R^{-1})  \\cos R\n\n\n - \\b^6 \\sinh(\\b^2R^{-1})  \\sin R\n  +2k^2(16k^4+3\\b^4)=0.\n\\end{aligned}\n\\end{equation}\nAn algorithm for creating a resonance at a prescribed point  $k$ is as follows. Given $k>0$, we first solve equation (\\ref{4.11}) with respect to $\\b$ and we also find $u$ by (\\ref{4.12}). Then needed values of  $\\ell$ are determined by (\\ref{4.10}), (\\ref{4.5}).\n\n\nIn order to illustrate this  algorithm,  we us consider a finite interval of wavenumbers $k\\in (0, k_1]$, where  we set $k_1 = 10$ for the numerics reported on in what follows. We scan   the chosen interval  with a sufficiently small step ($\\Delta k=0.01$) and for each value of $k$ solve equation (\\ref{4.11}) numerically  using the simple dichotomy method. While for each $k$  equation (\\ref{4.11}) might have several roots $\\beta$, in our numerical procedure we always choose the  minimal positive root, i.e., the one which allows to achieve the spectral singularity with given $k$ at the smallest possible value  of the gain-and-loss amplitude $\\gamma=\\gamma_\\textrm{min}$. Next, we choose the minimal positive distance $\\ell_\\textrm{min}$ which satisfies the conditions   (\\ref{4.10}), (\\ref{4.5}) and then we use the periodicity in $\\ell$ to generate the sequence of larger gain-to-loss distances $\\ell_n = \\ell_\\textrm{min} + n\\pi/(2k)$, $n=1,2\\ldots$ [see   (\\ref{eq:periodic})]. The resulting dependencies $\\gamma_\\textrm{min}(k)$ and $\\ell_\\textrm{min}(k)$, $\\ell_n(k)$   are shown in figure~\\ref{fig:min}. The minimal gain-and-loss amplitude $\\g_{min}(k)$  and the minimal gain-to-loss distance $\\ell_{min}$ are   discontinuous,  which means that the small variation  in  the wavenumuber $k$  might require  a significant  change either in   $\\gamma$ or in $\\ell$.  It is especially important that the values of the  distance  $\\ell$ are generically different from zero, which points out explicitly    that the new degree of freedom offered by the nonzero gain-to-loss distance    is   important for the achieving a spectral singularity  at   the given wavenumber $k$.\n\n\n\\begin{figure}\n\t\\centering\n\t\\includegraphics[width=0.8\\columnwidth]{fig04.eps}\n\t\\caption{(a) Minimal value of the gain-and-loss amplitude $\\gamma_\\textrm{min}$  which corresponds to   a spectral singularity with the given value of the wavenumber $k$. (b)  Minimal gain-to-loss distance  $\\ell_\\textrm{min}$  which corresponds to   a spectral singularity with the  $k$ and $\\gamma$ from the  left panel (bold curves) and larger distances $\\ell_n$ obtained using the periodicity in $\\ell$ (thin \\textcolor{black}{dotted} curves).}\n\t\\label{fig:min}\n\\end{figure}\n\n\n",
        "label": "fig:min",
        "Descriptive_question1": "What is the label of the y-axis in panel (a) of figure_4?",
        "Descriptive_question2": "What type of curves represent larger distances in panel (b) of figure_4?",
        "Reasoning_question1": "Why might the minimal gain-and-loss amplitude shown in panel (a) of figure_4 be discontinuous with respect to the wavenumber k?",
        "Reasoning_question2": "How does the periodicity in gain-to-loss distance, as depicted in panel (b) of figure_4, affect the ability to achieve spectral singularities at different wavenumbers?",
        "Descriptive_answer1": "Minimal value",
        "Descriptive_answer2": "dotted",
        "Reasoning_answer1": "The minimal gain-and-loss amplitude in panel (a) of figure_4 is discontinuous with respect to the wavenumber k because small variations in k can result in requiring significantly different values of the gain-and-loss amplitude γ to achieve a spectral singularity. This indicates that spectral singularities occur only for discrete parameter combinations, leading to abrupt changes or jumps in the minimal amplitude as k varies.",
        "Reasoning_answer2": "The periodicity in gain-to-loss distance ℓ shown in panel (b) means that spectral singularities can be achieved not only at the minimal distance ℓ_min but also at distances ℓ_n = ℓ_min + nπ/(2k) for integers n. This expands the range of available ℓ values for achieving spectral singularities at a given wavenumber k, enabling multiple distinct distances to produce spectral singularities, thereby offering greater flexibility in system design."
    },
    {
        "paper_id": "1908.06383.json",
        "image_id": "figure_5",
        "image_path": [
            "/home/yz979/scratch/chengye/scimolmo/test-images-tables/dataset_imagestables/1908.06383/images/fig05.eps"
        ],
        "caption": "(a) $\\PT$-symmetry breaking threshold $\\gamma_\\star^{(0)}$ \\textit{vs} the distance between the gain and loss $\\ell$. The spectrum is purely real and continuous for $\\gamma\\leq \\gamma_\\star$, but acquires a pair of complex conjugate eigenvalues  as the gain-and-loss amplitude exceeds the threshold $\\gamma_\\star^{(0)}$. (b) Values of the wavevector $k_\\star^{(0)}$ corresponding to the dependence in (a).",
        "classify": "Chart",
        "section_info": "5 Spectral singularities\n\\section{Spectral singularities}\n\\label{sec:ss}\n\\subsection{General analytical expressions}\n\nIn this section we study real zeroes of the function $F$ corresponding to spectral singularities, i.e. to  zero-width resonances.\nFor such zeroes, equation (\\ref{2.10}) is a pair of two real equations for one real variable $k$ and two parameters $\\ell$ and $\\g$.\nThanks to the symmetry of the zeroes with respect to the imaginary axis, it is sufficient to find only positive real resonances since the negative ones are located symmetrically with respect to the origin. Similar to the proof of Lemma~\\ref{lm3.6}, for real positive $k$  we make  change (\\ref{4.1})\nand rewrite equation (\\ref{2.10}) in the following form:\n\\begin{align*}\n(1-2u^{-4})\\cos\\b u^{-1}&+(2u^4-1)\\cosh\\b u + 2i \\sqrt{1-u^4}\\left(u^2\\sinh\\b u - u^{-4}\\sin\\b u^{-1}\\right)\n\\\\\n&-e^{-2i\\b\\ell\\sqrt{u^{-2}-u^2}} \\left(\\cosh(\\b u)-\\cos\\b u^{-1}\\right)=0,\\qquad \\b=\\sqrt{2\\g}.\n\\end{align*}\nTaking the real and imaginary part of this equation and multiplying the equation by $u^4$, we obtain:\n\\begin{equation}\\label{4.3}\n\\begin{aligned}\n&(u^4-2)\\cos\\b u^{-1}+u^4(2u^4-1)\\cosh\\b u\n=u^4\\cos\\left(2\\b\\ell\\sqrt{u^{-2}-u^2}\\right)\\left(\\cosh \\b u-\\cos\\b u^{-1}\\right),\n\\\\\n&2\\sqrt{1-u^4}\\Big(u^6\\sinh\\b u-\\sin\\b u^{-1}\\big)\n=-u^4\\sin\\left(2\\b\\ell\\sqrt{u^{-2}-u^2}\\right)\\left(\\cosh \\b u-\\cos\\b u^{-1}\\right).\n\\end{aligned}\n\\end{equation}\nThis is a system of two real equations with three real variables. If we are given $(\\b,\\ell)$ and we try to find $u$, the system is overdetermined and does not necessary have a root. In other words, it is solvable with respect to $u$ only if $(\\b,\\ell)$ are located on some (solvability) curves. In order to avoid working with an overdetermined system, in what follows we regard (\\ref{4.3}) as a system for two unknown variable with one parameter.\n\nTo find the curves in $(\\b,\\ell)$ plane, on which equations (\\ref{4.3}) are solvable with respect to $u$, we shall regard $u$ as a parameter and $(\\b,\\ell)$ as unknown variables. We take the sum of squares of equations (\\ref{4.3}) and divide the result by $(1-u^4)$. We also divide equations (\\ref{4.3}). This leads us to a   pair  of equations:\n\\begin{align}\\label{4.4}\n&u^4(1-u^4) \\cosh\\b u\\cos\\b u^{-1}-2u^6 \\sinh\\b u\\sin\\b u^{-1} +1-u^{12}=0,\n\\\\\n&\\frac{2\\sqrt{1-u^4}(u^6\\sinh\\b u-\\sin\\b u^{-1})}{(2-u^4)\\cos\\b u^{-1}+u^4(1-2u^4)\\cosh\\b u}=\\tan \\Big( 2\\b\\ell\\sqrt{u^{-2}-u^2}\\Big).\\nonumber\n\\end{align}\nThe second equation can be solved explicitly with respect to $\\ell$:\n\\begin{equation}\\label{4.10}\n\\ell=\\frac{1}{2\\b\\sqrt{u^{-2}-u^2}} \\left(\\arctan \\frac{2\\sqrt{1-u^4}(u^6\\sinh\\b u-\\sin\\b u^{-1})}{(2-u^4)\\cos\\b u^{-1}+u^4(1-2u^4)\\cosh\\b u}+\\pi n\\right),\n\\end{equation}\nwhere $n\\in\\mathds{N}$ is an arbitrary natural number. As the next lemma states, to make equations (\\ref{4.4}), (\\ref{4.10}) equivalent to (\\ref{4.3}), we should also assume that\n\\begin{equation}\\label{4.5}\n(-1)^n=\\sign \\big((u^4-2)\\cos\\b u^{-1}+u^4(2u^4-1)\\cosh\\b u\\big).\n\\end{equation}\n\n\\begin{lemma}\\label{lm4.1}\nEquations (\\ref{4.3}) are equivalent to (\\ref{4.4}), (\\ref{4.10}), (\\ref{4.5}).\n\\end{lemma}\n\n\\begin{proof}\nWe rewrite shortly equations (\\ref{4.3}) as\n$A_1=B\\cos\\a$, $ A_2=-B\\sin\\a$,\nwhere $A_1$, $A_2$ are the left hand sides in (\\ref{4.3}), $\\a=2\\b\\ell\\sqrt{u^{-2}-u^2}$ and $B=u^4(\\cosh\\b u-\\cos\\b u^{-1})$. Then equations (\\ref{4.4}), (\\ref{4.10}) become\n$A_1^2+A_2^2=B^2$, $\\a=-\\arctan\\frac{A_2}{A_1}+\\pi n$.\nWe have:\n\\begin{equation*}\nB\\cos \\a=(-1)^n\\cos\\arctan\\frac{A_2}{A_1}=\\frac{(-1)^n}{\\sqrt{1+\\frac{A_2^2}{A_1^2}}} = \\frac{(-1)^n|A_1|}{\\sqrt{A_1^2+A_2^2}}\n\\end{equation*}\nand we get the first equation $A_1=B\\cos\\a$  provided condition (\\ref{4.5}) is satisfied. In the same way we check that the latter condition also ensures the second equation $A_2=-B\\sin\\a$.\n\\end{proof}\n\n\nEquation (\\ref{4.4}) is transcendental, and we can not solve it analytically.\nNevertheless, for each $u\\in(0,1)$, this is an equation only for a single variable $\\b$, not for two as equations (\\ref{4.3}). So, we propose the following algorithm of recovering the aforementioned solvability curves: choose $u\\in(0,1)$, then solve equation (\\ref{4.4}) and recover the sequence of  distances  $\\ell$ by formula (\\ref{4.10}) with different integer $n$. Then the gain-and-loss amplitude $\\gamma$ and the corresponding wavenumber $k$ can be readily recovered from $\\beta$ and  $u$.  In a similar way, one can first fix some value of $\\beta$ (i.e., fix the gain-and-loss strength) and then   solve equation (\\ref{4.4}) with respect to $u$ and   recover   $\\ell$ by (\\ref{4.10}). Equation (\\ref{4.4})  is well-behaved, and for each $\\beta$  all its zeros  $u$ can be easily found numerically.\n\nAlternatively, as explained below  in Section~\\ref{sec:general}, the values  corresponding to  spectral singularities can be found systematically by means of the numerical continuation from the limit $\\ell=0$. However, in this case   equation (\\ref{4.4}) is still useful because it allows one to check that all spectral singularities have been found for the given value of the gain-and-loss $\\gamma$.\n\n\n\\subsection{\nAbsence of   spectral singularities}\n\\label{sec:gap}\n\nFor $u=0$ and $u=1$, the left-hand-side of equation (\\ref{4.4}) is equal respectively to $1$ and $-2\\sinh\\beta\\sin\\beta$. Then a sufficient condition for the existence of a spectral singularity at the given gain-and-loss amplitude $\\gamma$ is $\\sin\\beta=\\sin\\sqrt{2\\gamma}>0$. At the same time, it is also possible to establish sufficient conditions that forbid the existence of   spectral singularities in a certain interval of parameters. In this subsection we prove the existence of two ``forbidden gaps'' for the roots of equation (\\ref{4.4}). The first one exists for all $\\b\\geqslant 0$ and it states that there is no roots in certain interval. The second gap is a certain interval of values of $\\b$, for which equation (\\ref{4.4}) has no zeroes at all.\n\nFor the convenience, by $g(u,\\b)$ we denote the left hand side of equation (\\ref{4.4}). The first ``forbidden gap'' is described in the following lemma.\n\n\\begin{lemma}\\label{lm5.1}\nFor all $\\b\\geqslant 0$, equation (\\ref{4.4}) has no roots in the interval $\\big[0,(1+\\tfrac{\\b}{4})^{-1}\\big)$.\n\\end{lemma}\n\\begin{proof}\nEmploying a standard inequality $a\\cos\\a+b\\sin\\a\\leqslant \\sqrt{a^2+b^2}$, we estimate the first two term in equation (\\ref{4.4}) as\n\\begin{equation*}\nu^4(1-u^4) \\cosh\\b u\\cos\\b u^{-1}-2u^6 \\sinh\\b u\\sin\\b u^{-1} \\leqslant \\sqrt{u^8(1-u^4)^2\\cosh^2 \\b u+4u^{12}\\sinh^2\\b u}.\n\\end{equation*}\nHence, equation (\\ref{4.4}) surely has no roots for values of $u$ satisfying\n\\begin{equation*}\n \\sqrt{u^8(1-u^4)^2\\cosh^2 \\b u+4u^{12}\\sinh^2\\b u} <1-u^{12}.\n\\end{equation*}\nExpressing $\\cosh^2 \\b u$ via $\\sinh^2 \\b u$ and simplifying this inequality, we obtain $ u^4(1+u^4)\\cosh \\b u <1+u^{12}$ and hence,\n\\begin{equation}\\label{3.37}\n\\cosh \\b u-1 <\\frac{1-2u^4+u^8}{u^4},\\qquad \\sqrt{2}\\sinh \\b u\n<u^{-2}-u^2.\n\\end{equation}\nThanks to the estimate $\\sqrt{2}\\sinh 2\\ln u^{-1}\\leqslant u^{-2}-u^2$, $u\\in(0,1]$, the last inequality in (\\ref{3.37}) holds once $\\frac{\\b}{4}<u^{-1}\\ln u^{-1}$. It is easy to confirm that this inequality is true once $u^{-1}>1+\\tfrac{\\b}{4}$.\nThe proof is complete.\n\\end{proof}\n\n\nThe next lemma is   auxiliary  and will be employed in studying the second forbidden zone.\n\n\\begin{lemma}\\label{lm5.2}\nThe function $g(u,\\pi)$ is positive on $[0,1)$.\n\\end{lemma}\n\n\\begin{proof}\nWe have $g(0,\\pi)=1$ and by Lemma~\\ref{lm5.1}, it is positive for $u<(1+\\tfrac{\\pi}{4})^{-1}$. This is why in what follows we consider only the values $u\\geqslant (1+\\tfrac{\\pi}{4})^{-1}$. For such values of $u$ we have\n$\\pi\\leqslant \\pi u^{-1}\\leqslant \\pi (1+\\tfrac{\\pi}{4})<1.79\\pi$.\nAs\n$\\tfrac{3\\pi}{2} \\leqslant \\pi u^{-1}\\leqslant \\pi (1+\\tfrac{\\pi}{4})$,\nthe function $\\sin\\b u^{-1}$ is negative, while $\\cos \\b u^{-1}$ is positive. Hence, for such values of $u$, the function $g(u,\\pi)$ is positive. It remains to consider the values $\\frac{2}{3}<u\\leqslant 1$.\n\nFor such values of $u$ we first observe the following simple estimates:\n\\begin{equation*}\n\\frac{1-u^4}{1-u}\\cos\\pi u^{-1}\\geqslant -4,\\qquad -\\frac{\\sin \\pi u^{-1}}{1-u}=\\frac{\\sin \\pi (u^{-1}-1)}{1-u}\\geqslant \\pi u.\n\\end{equation*}\nThen the function $g$ satisfies the estimate:\n\\begin{equation}\\label{3.12}\n\\frac{g(u,\\pi)}{u^4(1-u)}\\geqslant g_1(u)+g_2(u),\\qquad g_1(u):=-4\\cosh\\pi u +2\\pi u^3 \\sinh \\pi u,\\qquad g_2(u):=\\frac{1-u^{12}}{u^4(1-u)}.\n\\end{equation}\nThe function $g_1(u)$ is monotonically increasing in $u\\in[\\tfrac{2}{3},1]$ since\n\\begin{equation*}\ng_1'(u)=2\\pi^2 u^3\\cosh\\pi u+(6\\pi u^2-4\\pi)\\sinh\\pi u>2\\pi(\\pi u^3+6u^2-2)\\sinh\\pi u>2\\pi\\sinh\\pi u>0.\n\\end{equation*}\nHence,\n$g_1(u)\\geqslant g_1\\left(\\frac{2}{3}\\right)>-9.05$.\nFor the function $g_2$ we have the following representation and estimate:\n\\begin{equation*}\ng_2(u)=1+\\sum\\limits_{j=1}^{4}(u^j+u^{-j})+\\sum\\limits_{j=5}^{8} u^j\\geqslant 9 +\\sum\\limits_{j=5}^{8}\\left(\\frac{2}{3}\\right)^j>9.31.\n\\end{equation*}\nTwo last estimates and (\\ref{3.12}) imply the positivity of the function $g$ for $u\\in[\\tfrac{2}{3},1)$.\n\\end{proof}\n\n\nDenote\n\\begin{align*}\ng_*(u,\\b):=&\\b u^3(1-3u^4)\\cosh\\b u \\sin \\b u^{-1}+\\b u^5 (3-u^4)\\sinh\\b u \\cos \\b u^{-1}\n\\\\\n&-2 u^4(1+u^4) \\cosh\\b u \\cos \\b u^{-1} -6(1+u^{12}).\n\\end{align*}\n\n\nThe next lemma states the existence of the second forbidden zone.\n\n\\begin{lemma}\\label{lm5.3}\nEquation (\\ref{4.4}) has no roots as $\\pi<\\b<\\b_*<5$, where $(u_*, \\b_*)$ is  the root of the system of the equations\n\\begin{equation}\\label{3.13}\ng(u,\\b)=0,\\qquad g_*(u,\\b)=0,\\qquad u\\in[0,1],\\qquad \\pi<\\b<5,\n\\end{equation}\nwith minimal possible $\\b$.\nTheir approximate values are\n\\begin{equation}\\label{3.14}\n\\b_*=4.808438,\\qquad u_*=0.611772.\n\\end{equation}\n\\end{lemma}\n\n\\begin{proof}\nThe function $g(u,\\pi)$ is positive on $[0,1)$ and $g(1,\\pi)=0$, see Figure~\\ref{fig:forbidden}a. As $\\pi<\\b<2\\pi$, we have $g(0,\\b)=1>0$ and $g(1,\\b)=-2\\sin\\b\\sinh\\b>0$. Hence, for $\\b$ close enough to $\\pi$, the function $g(u,\\b)$ is positive for all $u\\in[0,1]$. At the same time, we have $g(0.65,5)<-0.617<0$ and therefore, for $\\b=5$, equation (\\ref{4.4}) possesses at least two roots, one in $(0,0.65)$ and another in $(0.65,1)$. cf. Figure~\\ref{fig:forbidden}c. We also observe that the function $g$ is jointly continuous in $(u,\\b)$. The above facts means that as $\\b$ grows from $\\pi$ to $5$, at some value $\\b=\\b_*$, the graph of the function $g$ is still located in the upper half-plane but touches the $u$-axis at some point $u=u_*$,\nsee Figure~\\ref{fig:forbidden}b. The function $g(u,\\b)$ is positive as $\\pi<\\b<\\b_*$ and $u\\in[0,1]$. Then the point $u=u_*$ is obviously the global minimum of $g$ and hence, $(u_*,\\b_*)$ is a solution to the system of equations $g(u,\\b)=0$, $\\frac{\\p g}{\\p u}(u,\\b)=0$. It is easy to check that $g_*=\\frac{\\p g}{\\p u}-6 g$ and hence, $(u_*,\\b_*)$ solves system (\\ref{3.13}). These roots can be found numerically and this gives (\\ref{3.14}). The proof is complete.\n\\end{proof}\n\n\n\\begin{figure}\n\t\\centering\n\\includegraphics[width=0.99\\columnwidth]{fig03.eps}\n\\caption{Illustration for proof of Lemma~\\ref{lm5.3}. Graphs of the function $g(u, \\beta)$ for $\\b=\\pi$ (a), $\\b=\\b_*\\approx  4.808438$ (b), and $\\b=5$ (c). Notice broken vertical axes in (b) and (c). }\n\\label{fig:forbidden}\n\\end{figure}\n\nReturning from the auxiliary variable $\\beta$ to the gain-and-loss amplitude $\\gamma=\\beta^2/2$, from   Lemma~\\ref{lm5.3}  we deduce  the following important result:\n\\begin{equation}\n\\label{eq:gap}\n\\textrm{there is no spectral singularities for\\quad }  \\frac{\\pi^2}{2} < \\gamma< \\gamma_*\\approx  11.561.\n\\end{equation}\n\n\n\n\\subsection{Creating a  spectral singularity  at a given wavenumber}\n\n\\begin{table}\n\t\\centering\n\t\\begin{tabular}{c|cccc}\n\t\t$n$ & 0 & 1& 2&\\\\\\hline\n\t\t$\\gamma_\\star^{(n)}$ & 2.071&13.307&27.783 &\\\\[2mm]\n\t\t$k_\\star^{(n)}$ & 1.065&4.318 &7.529 &\n\t\\end{tabular}\n\t\\caption{Approximate values of gain-and-loss amplitudes $\\gamma=\\gamma_\\star^{(n)}$  and wavenumbers   $k=k_\\star^{(n)}$, $n=0, 1, \\ldots$,  corresponding to spectral singularities with lowest $\\gamma$ in the limit $\\ell=0$, see Table~I in  \\cite{Mostafazadeh2009}. \\label{tbl:1}}\n\\end{table}\n\n\nFor $\\ell=0$ spectral singularity can only be obtained for some isolated values of the wavenumber $k$ and the gain-and-loss amplitude $\\gamma$ \\cite{Mostafazadeh2009}. Several lowest values of $\\gamma$ corresponding to the spectral singularities  and the associated wavenumbers  $k$ are listed in Table~\\ref{tbl:1}.  An  important advantage of the more general system with nonzero gain-to-loss disctance    $\\ell>0$ consists  in the possibility to create a spectral   singularity   at any wavenumber $k$ given beforehand. Indeed, let us  return back to equations (\\ref{4.4}), (\\ref{4.10}) and    discuss the following   issue: given a point $k$ on the real axis, how to choose $\\b$ and $\\ell$ to have a  resonance at this point? Equations (\\ref{4.4})--(\\ref{4.10}) allow us to answer  easily this question.\n\nWe fix $k>0$ and we find the associated value of $u$ by resolving (\\ref{4.1}):\n\\begin{equation}\\label{4.12}\nu^{-2}-u^2=4k^2\\b^{-2},\\qquad\n\nu=\\b R^{-1}, \\quad \\mbox{where\\ } R = \\sqrt{2k^2+\\sqrt{4k^4+\\b^4}}.\n\\end{equation}\nWe divide equation (\\ref{4.4}) by $u^6$ and substitute then the above formulae and\n\\begin{equation*}\n\\frac{1-u^{12}}{u^6}=\\frac{1-u^4}{u^2}\\frac{1+u^4+u^8}{u^4}=(u^{-2}-u^2)\\big((u^{-2}-u^2)^2+3\n\\big).\n\\end{equation*}\nThis gives the equation:\n\\begin{equation}\\label{4.11}\n\\begin{aligned}\n2\\b^4 k^2&\\cosh(\\b^2R^{-1})  \\cos R\n\n\n - \\b^6 \\sinh(\\b^2R^{-1})  \\sin R\n  +2k^2(16k^4+3\\b^4)=0.\n\\end{aligned}\n\\end{equation}\nAn algorithm for creating a resonance at a prescribed point  $k$ is as follows. Given $k>0$, we first solve equation (\\ref{4.11}) with respect to $\\b$ and we also find $u$ by (\\ref{4.12}). Then needed values of  $\\ell$ are determined by (\\ref{4.10}), (\\ref{4.5}).\n\n\nIn order to illustrate this  algorithm,  we us consider a finite interval of wavenumbers $k\\in (0, k_1]$, where  we set $k_1 = 10$ for the numerics reported on in what follows. We scan   the chosen interval  with a sufficiently small step ($\\Delta k=0.01$) and for each value of $k$ solve equation (\\ref{4.11}) numerically  using the simple dichotomy method. While for each $k$  equation (\\ref{4.11}) might have several roots $\\beta$, in our numerical procedure we always choose the  minimal positive root, i.e., the one which allows to achieve the spectral singularity with given $k$ at the smallest possible value  of the gain-and-loss amplitude $\\gamma=\\gamma_\\textrm{min}$. Next, we choose the minimal positive distance $\\ell_\\textrm{min}$ which satisfies the conditions   (\\ref{4.10}), (\\ref{4.5}) and then we use the periodicity in $\\ell$ to generate the sequence of larger gain-to-loss distances $\\ell_n = \\ell_\\textrm{min} + n\\pi/(2k)$, $n=1,2\\ldots$ [see   (\\ref{eq:periodic})]. The resulting dependencies $\\gamma_\\textrm{min}(k)$ and $\\ell_\\textrm{min}(k)$, $\\ell_n(k)$   are shown in figure~\\ref{fig:min}. The minimal gain-and-loss amplitude $\\g_{min}(k)$  and the minimal gain-to-loss distance $\\ell_{min}$ are   discontinuous,  which means that the small variation  in  the wavenumuber $k$  might require  a significant  change either in   $\\gamma$ or in $\\ell$.  It is especially important that the values of the  distance  $\\ell$ are generically different from zero, which points out explicitly    that the new degree of freedom offered by the nonzero gain-to-loss distance    is   important for the achieving a spectral singularity  at   the given wavenumber $k$.\n\n\n\\begin{figure}\n\t\\centering\n\t\\includegraphics[width=0.8\\columnwidth]{fig04.eps}\n\t\\caption{(a) Minimal value of the gain-and-loss amplitude $\\gamma_\\textrm{min}$  which corresponds to   a spectral singularity with the given value of the wavenumber $k$. (b)  Minimal gain-to-loss distance  $\\ell_\\textrm{min}$  which corresponds to   a spectral singularity with the  $k$ and $\\gamma$ from the  left panel (bold curves) and larger distances $\\ell_n$ obtained using the periodicity in $\\ell$ (thin \\textcolor{black}{dotted} curves).}\n\t\\label{fig:min}\n\\end{figure}\n\n\n\\subsection{$\\PT$-symmetry breaking laser-antilaser threshold}\n\nA particularly important characteristics of any $\\PT$-symmetric structure is the $\\PT$ symmetry breaking threshold, i.e., the amplitude of the gain-and-loss\ncorresponding to the  ``phase transition'' from the purely real to complex spectrum.  The best studied scenario of the phase transition  \nis the collision of two real discrete  eigenvalues at an exceptional point with the subsequent splitting in a complex-conjugate pair. However, in systems with nonempty continuous spectrum,  the phase transition can also occur through   the splitting of a self-dual spectral singularity, which results in a bifurcation of a complex-conjugate pair from an interior point of the continuum \\cite{Yang17,KZ17,Konotop2019}. At the moment corresponding to the formation of the spectral singularity, the system operates in the CPA-laser regime \\cite{Longhi10}. Thus, in such a system, the $\\PT$-symmetry breaking threshold at the same time corresponds to the CPA-laser threshold.\n\nLemma~\\ref{lm3.3} guarantees that the spectrum of our system is real for sufficiently small gain-and-loss amplitudes $\\gamma$. Additionally, according to Lemma~\\ref{lm3.5}, the spectrum  does not have any real discrete eigenvalue. Hence, the $\\PT$-symmetry breaking is expected to occur through the emergence of a self-dual spectral singularity. In order to identify the $\\PT$-symmetry breaking threshold in our system, we start from the limit $\\ell=0$, there the phase transition takes place at  $\\gamma_\\star^{(0)} \\approx 2.072$, see \\cite{Mostafazadeh2009,KZ17} and Table~\\ref{tbl:1}. Thus, the spectrum with $\\ell=0$ is purely real and continuous for $\\gamma \\in [0, \\gamma_\\star]$, while the  increase of the gain-and-loss just above $\\gamma_\\star^{(0)}$ leads to the bifurcation of a complex-conjugate pair from an interior point  of the continuum. The spectral singularity forming at  $\\gamma_\\star^{(0)}$ takes place at wavenumber $k=k_\\star^{(0)} \\approx  1.065$. Respectively, the complex-conjugate pair of eigenvalues bifurcates from $\\lambda_0 =  [k_\\star^{(0)}]^2$. (Notice that the further increase of $\\gamma$ above the next threshold values listed in Table~\\ref{tbl:1} leads to the formation of new spectral singularities and, respectively, to bifurcations of new complex-conjugate pairs in the spectrum.)\n\nNext, we use the numerical continuation in $\\ell$ in order to continue the known solution at $\\ell=0$ to the domain $\\ell>0$.  The obtained dependence of the threshold value  of the gain-and-loss amplitude on distance $\\ell$ is shown in figure~\\ref{fig:threshold}(a), where we observe that the phase transition threshold decreases monotonically with the growth of $\\ell$. This means that introducing an additional space   between the gain and loss, one can decrease the $\\PT$-symmetry breaking  threshold, i.e. achieve the laser-antilaser operation at \\textcolor{black}{lower} gain-and-loss amplitudes than in a waveguide with the adjacent gain and loss.\n\n\n\\begin{figure}\n\t\\centering\n\t\\includegraphics[width=0.85\\columnwidth]{fig05.eps}\n\t\\caption{(a) $\\PT$-symmetry breaking threshold $\\gamma_\\star^{(0)}$ \\textit{vs} the distance between the gain and loss $\\ell$. The spectrum is purely real and continuous for $\\gamma\\leq \\gamma_\\star$, but acquires a pair of complex conjugate eigenvalues  as the gain-and-loss amplitude exceeds the threshold $\\gamma_\\star^{(0)}$. (b) Values of the wavevector $k_\\star^{(0)}$ corresponding to the dependence in (a).}\n\t\\label{fig:threshold}\n\\end{figure}\n\n\\subsection{General picture of spectral singularities}\n\\label{sec:general}\n\n\\begin{figure}\n\t\\centering\n\t\\includegraphics[width=0.99\\columnwidth]{fig06.eps}\n\t\\caption{(a) Values of the gain-and-loss amplitude $\\gamma$ and distance  $\\ell$, for which   spectral singularities occur. (b) Corresponding  wavenumbers   $k$. \\textcolor{black}{Panel (c) magnifies   the region  $(\\ell, \\gamma)\\in [2,4]\\times[10,50]$ from   (a), and panel  (d) is the  magnification of corresponding curves from panel (b).}}\n\t\\label{fig:general}\n\\end{figure}\n\nLet us now turn to description of the general picture of spectral singularities. In order to construct systematically different solutions, we again start from the limit $\\ell=0$, where the values of $\\gamma$ and $k$ corresponding to spectral singularities are known, see Table~\\ref{tbl:1}. Then we use the periodicity of function $e^{-4ik\\ell}$ in $\\ell$ in order to construct new branches of solutions having no counterparts in the limit $\\ell=0$, cf. equation (\\ref{eq:periodic}). This procedure results in a fairly complicated picture containing a multitude of spectral singularities, some part of which (corresponding to  relatively  small   values of the gain-and-loss) is shown in figure~\\ref{fig:general}(a,b) as the curves on the plane $\\gamma$ {\\it vs} $\\ell$ and $k$ {\\it vs} $\\ell$.\n\n\nLet us describe the structure of the found solutions using the  diagram $\\gamma$ {\\it vs} $\\ell$ in figure~\\ref{fig:general}(a). The multitude of curves shown in this plot can be divided into three groups (plotted with red, blue and green curves) which have been obtained by means of the continuation from three different solutions in the limit $\\ell=0$. There is a well-visible vertical gap between red and blue curves, which corresponds to the ``forbidden'' values of the gain-and-loss amplitudes $\\gamma$ found above in (\\ref{eq:gap}). At the same time,  there is no gap between blue and green curves, which  results in a multitude of intersections between these curves.\nThe first group of spectral singularities,  corresponding to red curves in Figure~\\ref{fig:general}(a),   is obtained through  the continuation from the  spectral singularity  in the limit $\\ell=0$ with the smallest gain-and-loss amplitude, i.e.  from values $\\gamma_\\star^{(0)}$ and $k_\\star^{(0)}$ in Table~\\ref{tbl:1}.  The leftmost (bold)  curve in this group which originates in the limit $\\ell=0$  is the $\\PT$-symmetry breaking threshold which was already shown in figure~\\ref{fig:threshold}(a). For values of $\\gamma$ above this curve, there is always one or more (but finitely many, see Section~\\ref{sec:zeros})   complex-conjugate pairs of eigenvalues in the spectrum.  Several curves situated to the right from the bold red curve are obtained using the fact that if $\\gamma$ and $k$ are solutions in the limit $\\ell=0$, then the same values of $\\gamma$ and $k$ also correspond to a spectral singularity with $\\ell_n = (n\\pi)/(2k)$, $n=1,2,\\ldots$. Once a single solution with a new distance   $\\ell_n$ is obtained, a new branch of solutions can be constructed using numerical continuation in $\\gamma$ or in $\\ell$. In the limit $\\ell \\to \\infty$ all the red curves in figure~\\ref{fig:general}(a) approach  the asymptotic value $\\gamma=0$ or at $\\gamma_*= \\pi^2/2\\approx 4.935$. Notice that, except for the bold line demarcating the $\\PT$-symmetry breaking threshold, none of the red curves can be continued to the limit $\\ell\\to 0$.\n\nThe multitude of red curves  in figure~\\ref{fig:general}(a) demonstrates explicitly how new spectral singularities emerge with the increase of $\\ell$. Indeed, drawing an imaginary horizontal line, say, at $\\gamma=3$ [see the vertical axis  tick  in figure~\\ref{fig:general}(a)], we observe that with the increase of $\\ell$ this line   intersects more and more red curves. Each intersection corresponds to the values of $\\gamma$ and $\\ell$ at which   some  root $k$   crosses the real axis and goes down from the upper to lower complex half-plane. Thus,  a finite-width resonance transforms  to a complex eigenvalue through the spectral singularity (we recall again that  in view of   $\\PT$ symmetry  any root $k\\ne 0$ crosses the real line simultaneously with its counterpart $-\\bar{k}$, i.e.  the corresponding   spectral singularity is self-dual).\nIn order to illustrate this process, in Figure~\\ref{fig:g=3}  we show the evolution of three numerically found complex zeros of function $F(k, \\gamma, \\ell)$  under   the increase of $\\ell$.  In this figure the imaginary part of each complex zero changes sign from positive to negative and then asymptotically approaches zero (remaining negative). In the limit of large $\\ell$,  this behavior agrees  with expansion (\\ref{3.4}) of Lemma~\\ref{lm3.4} , where, for the chosen value of $\\gamma$,   we have $\\sin\\sqrt{2\\gamma}>0$.   Thus the growing distance   $\\ell$ results in a sequence of self-dual spectral singularities and in the increasing number of complex-conjugate eigenvalues in the spectrum.\n\n \\begin{figure}\n \t\\centering\n \t\\includegraphics[width=0.8\\columnwidth]{fig07.eps}\n \t\\caption{(a,b) Real and imaginary parts of three complex zeros of function $F$ for fixed $\\gamma=3$ and increasing $\\ell$. For each curve, the imaginary part is positive for small $\\ell$ and becomes negative for all sufficiently large $\\ell$. \\textcolor{black}{For each three shown eigenvalues, its real and \timaginary parts are of the same colour in both panels.}}\n \t\\label{fig:g=3}\n \\end{figure}\n\n\nThe second group of spectral singularities [blue curves in figure~\\ref{fig:general}(a)] was obtained by means of the continuation from the  next solution in the limit $\\ell=0$, i.e.  from $\\gamma_\\star^{(1)}$ and $k_\\star^{(1)}$ in Table~\\ref{tbl:1}. Again, one of the curves [the leftmost bold curve in figure~\\ref{fig:general}(a)] was obtained through the direct continuation from the limit $\\ell=0$, while other blue curves were generated  using the periodicity of function $F(k, \\gamma, \\ell)$  in $\\ell$ and cannot be continued to the limit $\\ell\\to0$. In the limit $\\ell\\to\\infty$ the blue curves approach the horizontal asymptotes  $\\gamma=2\\pi^2\\approx19.739$ and $\\gamma=9\\pi^2/2\\approx 44.413$. In  the $(\\gamma, \\ell)$-plane  the  gain-and-loss amplitudes corresponding to the   group of blue curves are well separated from  those for the red curves: indeed, all red curves are bounded from above by the asymptote $\\gamma= \\pi^2/2\\approx 4.935$, while all blue curves are bounded from below by $\\gamma_*\\approx 11.561$, see (\\ref{eq:gap}). Thus, the  emergence of new spectral singularities with the increase of $\\ell$ is sensitive to the value of the gain-and-loss amplitude and does not occur for     the gain-and-loss amplitudes lying in the  gap  between the red and blue curves.\n\nIn comparison with the red curves discussed above, the curves from the blue group in figure~\\ref{fig:general}(a) feature more complicated behavior and, in particular, can intersect each other (and also intersect the curves from the next, third group of green curves discussed below). The intersections between the blue curves occur for the gain-and-loss amplitudes in the interval $11.561 \\lessapprox   \\gamma <  9\\pi^2/2\\approx 19.739$, where $\\sin\\sqrt{2\\gamma}$ is negative.  \\textcolor{black}{At first glance,} this might seem to contradict to  the  expansion (\\ref{3.4}) of Lemma~\\ref{lm3.4},  which  suggests that in this case the multitude of complex zeros   accumulate in the upper complex half-plane with the growth of $\\ell$. However, this apparent contradiction is resolved if we trace the behavior of the complex roots more closely.   Indeed, choosing for an example $\\gamma =16$ [see the vertical axis tick in  figure~\\ref{fig:general}(a)] and computing several  complex roots under the increase of $\\ell$, we observe that each considered root first goes down from the upper half-plane to the lower one but then again returns to the upper half-plane, see Figure~\\ref{fig:g=16}(b) and (b$_1$). Thus,  in  this  interval of the gain-and-loss amplitudes the increase of $\\ell$  results  either in the transformation from the resonance to the eigenvalue or to the opposite process,  i.e. to the disappearance of the complex-conjugate pair.  Respectively, the intersection between the two blue curves corresponds to  two coexisting spectral singularities, i.e. to the moment  when one complex-conjugate pair of eigenvalues disappears and another pair (with different $k$) emerges. For sufficiently large $\\ell$ the imaginary part  of each   considered root  remains positive and approaches zero, in accordance with expansion (\\ref{3.4}). Thus, in this interval of the gain-and-loss strengths, the limit $\\ell\\to\\infty$ the spectrum contains only a finite number of complex-conjugate eigenvalues, which correspond to complex zeros $k$ whose behavior is not covered   by expansion  (\\ref{3.4}).\n\n \\begin{figure}\n\t\\centering\n\t\\includegraphics[width=0.99\\columnwidth]{fig08.eps}\n\t\\caption{(a,b) Real and imaginary parts for three   complex zeros of function $F$ for fixed $\\gamma=16$ and increasing $\\ell$. Panel b$_1$ is the magnification of some region of (b) and  shows more clearly that imaginary parts of all   three shown eigenvalues are positive for all sufficiently large  $\\ell$. \\textcolor{black}{For each three shown eigenvalues, its real and \timaginary parts are of the same colour in both panels.}}\n\t\\label{fig:g=16}\n\\end{figure}\n\nThe third  group of spectral singularities [green curves in figure~\\ref{fig:general}(a)] was obtained by means of the continuation from  the solution $\\gamma_\\star^{(2)}$ and $k_\\star^{(2)}$ in Table~\\ref{tbl:1}. In view of the very complicated structure of the overall resulting picture, we only show a section of these curves corresponding to relatively small   values of the gain-and-loss amplitudes $\\gamma$. Quite interestingly, there is no gap between the blue and green groups of curves, which results in the multitude of intersections between blue and green curves, \\textcolor{black}{see Fig.~\\ref{fig:general}(a) and the magnified view in Fig.~\\ref{fig:general}(c)}. These   intersections suggest a possibility of   simultaneous emergence of two complex-conjugate pairs of eigenvalues from two different interior points of the continuous spectra.\n\nConsidering further solutions $\\gamma_\\star^{(n)}$, $k_\\star^{(n)}$, $n=3,4,\\ldots$, in the limit $\\ell=0$ one can construct new groups of spectral singularities  with larger values of $\\gamma$, which are not shown in Figure~\\ref{fig:general}.\n\n\n5.4 $\\PT$-symmetry breaking laser-antilaser threshold\n\\subsection{$\\PT$-symmetry breaking laser-antilaser threshold}\n\nA particularly important characteristics of any $\\PT$-symmetric structure is the $\\PT$ symmetry breaking threshold, i.e., the amplitude of the gain-and-loss\ncorresponding to the  ``phase transition'' from the purely real to complex spectrum.  The best studied scenario of the phase transition  \nis the collision of two real discrete  eigenvalues at an exceptional point with the subsequent splitting in a complex-conjugate pair. However, in systems with nonempty continuous spectrum,  the phase transition can also occur through   the splitting of a self-dual spectral singularity, which results in a bifurcation of a complex-conjugate pair from an interior point of the continuum \\cite{Yang17,KZ17,Konotop2019}. At the moment corresponding to the formation of the spectral singularity, the system operates in the CPA-laser regime \\cite{Longhi10}. Thus, in such a system, the $\\PT$-symmetry breaking threshold at the same time corresponds to the CPA-laser threshold.\n\nLemma~\\ref{lm3.3} guarantees that the spectrum of our system is real for sufficiently small gain-and-loss amplitudes $\\gamma$. Additionally, according to Lemma~\\ref{lm3.5}, the spectrum  does not have any real discrete eigenvalue. Hence, the $\\PT$-symmetry breaking is expected to occur through the emergence of a self-dual spectral singularity. In order to identify the $\\PT$-symmetry breaking threshold in our system, we start from the limit $\\ell=0$, there the phase transition takes place at  $\\gamma_\\star^{(0)} \\approx 2.072$, see \\cite{Mostafazadeh2009,KZ17} and Table~\\ref{tbl:1}. Thus, the spectrum with $\\ell=0$ is purely real and continuous for $\\gamma \\in [0, \\gamma_\\star]$, while the  increase of the gain-and-loss just above $\\gamma_\\star^{(0)}$ leads to the bifurcation of a complex-conjugate pair from an interior point  of the continuum. The spectral singularity forming at  $\\gamma_\\star^{(0)}$ takes place at wavenumber $k=k_\\star^{(0)} \\approx  1.065$. Respectively, the complex-conjugate pair of eigenvalues bifurcates from $\\lambda_0 =  [k_\\star^{(0)}]^2$. (Notice that the further increase of $\\gamma$ above the next threshold values listed in Table~\\ref{tbl:1} leads to the formation of new spectral singularities and, respectively, to bifurcations of new complex-conjugate pairs in the spectrum.)\n\nNext, we use the numerical continuation in $\\ell$ in order to continue the known solution at $\\ell=0$ to the domain $\\ell>0$.  The obtained dependence of the threshold value  of the gain-and-loss amplitude on distance $\\ell$ is shown in figure~\\ref{fig:threshold}(a), where we observe that the phase transition threshold decreases monotonically with the growth of $\\ell$. This means that introducing an additional space   between the gain and loss, one can decrease the $\\PT$-symmetry breaking  threshold, i.e. achieve the laser-antilaser operation at \\textcolor{black}{lower} gain-and-loss amplitudes than in a waveguide with the adjacent gain and loss.\n\n\n\\begin{figure}\n\t\\centering\n\t\\includegraphics[width=0.85\\columnwidth]{fig05.eps}\n\t\\caption{(a) $\\PT$-symmetry breaking threshold $\\gamma_\\star^{(0)}$ \\textit{vs} the distance between the gain and loss $\\ell$. The spectrum is purely real and continuous for $\\gamma\\leq \\gamma_\\star$, but acquires a pair of complex conjugate eigenvalues  as the gain-and-loss amplitude exceeds the threshold $\\gamma_\\star^{(0)}$. (b) Values of the wavevector $k_\\star^{(0)}$ corresponding to the dependence in (a).}\n\t\\label{fig:threshold}\n\\end{figure}\n\n5.5 General picture of spectral singularities\n\\subsection{General picture of spectral singularities}\n\\label{sec:general}\n\n\\begin{figure}\n\t\\centering\n\t\\includegraphics[width=0.99\\columnwidth]{fig06.eps}\n\t\\caption{(a) Values of the gain-and-loss amplitude $\\gamma$ and distance  $\\ell$, for which   spectral singularities occur. (b) Corresponding  wavenumbers   $k$. \\textcolor{black}{Panel (c) magnifies   the region  $(\\ell, \\gamma)\\in [2,4]\\times[10,50]$ from   (a), and panel  (d) is the  magnification of corresponding curves from panel (b).}}\n\t\\label{fig:general}\n\\end{figure}\n\nLet us now turn to description of the general picture of spectral singularities. In order to construct systematically different solutions, we again start from the limit $\\ell=0$, where the values of $\\gamma$ and $k$ corresponding to spectral singularities are known, see Table~\\ref{tbl:1}. Then we use the periodicity of function $e^{-4ik\\ell}$ in $\\ell$ in order to construct new branches of solutions having no counterparts in the limit $\\ell=0$, cf. equation (\\ref{eq:periodic}). This procedure results in a fairly complicated picture containing a multitude of spectral singularities, some part of which (corresponding to  relatively  small   values of the gain-and-loss) is shown in figure~\\ref{fig:general}(a,b) as the curves on the plane $\\gamma$ {\\it vs} $\\ell$ and $k$ {\\it vs} $\\ell$.\n\n\nLet us describe the structure of the found solutions using the  diagram $\\gamma$ {\\it vs} $\\ell$ in figure~\\ref{fig:general}(a). The multitude of curves shown in this plot can be divided into three groups (plotted with red, blue and green curves) which have been obtained by means of the continuation from three different solutions in the limit $\\ell=0$. There is a well-visible vertical gap between red and blue curves, which corresponds to the ``forbidden'' values of the gain-and-loss amplitudes $\\gamma$ found above in (\\ref{eq:gap}). At the same time,  there is no gap between blue and green curves, which  results in a multitude of intersections between these curves.\nThe first group of spectral singularities,  corresponding to red curves in Figure~\\ref{fig:general}(a),   is obtained through  the continuation from the  spectral singularity  in the limit $\\ell=0$ with the smallest gain-and-loss amplitude, i.e.  from values $\\gamma_\\star^{(0)}$ and $k_\\star^{(0)}$ in Table~\\ref{tbl:1}.  The leftmost (bold)  curve in this group which originates in the limit $\\ell=0$  is the $\\PT$-symmetry breaking threshold which was already shown in figure~\\ref{fig:threshold}(a). For values of $\\gamma$ above this curve, there is always one or more (but finitely many, see Section~\\ref{sec:zeros})   complex-conjugate pairs of eigenvalues in the spectrum.  Several curves situated to the right from the bold red curve are obtained using the fact that if $\\gamma$ and $k$ are solutions in the limit $\\ell=0$, then the same values of $\\gamma$ and $k$ also correspond to a spectral singularity with $\\ell_n = (n\\pi)/(2k)$, $n=1,2,\\ldots$. Once a single solution with a new distance   $\\ell_n$ is obtained, a new branch of solutions can be constructed using numerical continuation in $\\gamma$ or in $\\ell$. In the limit $\\ell \\to \\infty$ all the red curves in figure~\\ref{fig:general}(a) approach  the asymptotic value $\\gamma=0$ or at $\\gamma_*= \\pi^2/2\\approx 4.935$. Notice that, except for the bold line demarcating the $\\PT$-symmetry breaking threshold, none of the red curves can be continued to the limit $\\ell\\to 0$.\n\nThe multitude of red curves  in figure~\\ref{fig:general}(a) demonstrates explicitly how new spectral singularities emerge with the increase of $\\ell$. Indeed, drawing an imaginary horizontal line, say, at $\\gamma=3$ [see the vertical axis  tick  in figure~\\ref{fig:general}(a)], we observe that with the increase of $\\ell$ this line   intersects more and more red curves. Each intersection corresponds to the values of $\\gamma$ and $\\ell$ at which   some  root $k$   crosses the real axis and goes down from the upper to lower complex half-plane. Thus,  a finite-width resonance transforms  to a complex eigenvalue through the spectral singularity (we recall again that  in view of   $\\PT$ symmetry  any root $k\\ne 0$ crosses the real line simultaneously with its counterpart $-\\bar{k}$, i.e.  the corresponding   spectral singularity is self-dual).\nIn order to illustrate this process, in Figure~\\ref{fig:g=3}  we show the evolution of three numerically found complex zeros of function $F(k, \\gamma, \\ell)$  under   the increase of $\\ell$.  In this figure the imaginary part of each complex zero changes sign from positive to negative and then asymptotically approaches zero (remaining negative). In the limit of large $\\ell$,  this behavior agrees  with expansion (\\ref{3.4}) of Lemma~\\ref{lm3.4} , where, for the chosen value of $\\gamma$,   we have $\\sin\\sqrt{2\\gamma}>0$.   Thus the growing distance   $\\ell$ results in a sequence of self-dual spectral singularities and in the increasing number of complex-conjugate eigenvalues in the spectrum.\n\n \\begin{figure}\n \t\\centering\n \t\\includegraphics[width=0.8\\columnwidth]{fig07.eps}\n \t\\caption{(a,b) Real and imaginary parts of three complex zeros of function $F$ for fixed $\\gamma=3$ and increasing $\\ell$. For each curve, the imaginary part is positive for small $\\ell$ and becomes negative for all sufficiently large $\\ell$. \\textcolor{black}{For each three shown eigenvalues, its real and \timaginary parts are of the same colour in both panels.}}\n \t\\label{fig:g=3}\n \\end{figure}\n\n\nThe second group of spectral singularities [blue curves in figure~\\ref{fig:general}(a)] was obtained by means of the continuation from the  next solution in the limit $\\ell=0$, i.e.  from $\\gamma_\\star^{(1)}$ and $k_\\star^{(1)}$ in Table~\\ref{tbl:1}. Again, one of the curves [the leftmost bold curve in figure~\\ref{fig:general}(a)] was obtained through the direct continuation from the limit $\\ell=0$, while other blue curves were generated  using the periodicity of function $F(k, \\gamma, \\ell)$  in $\\ell$ and cannot be continued to the limit $\\ell\\to0$. In the limit $\\ell\\to\\infty$ the blue curves approach the horizontal asymptotes  $\\gamma=2\\pi^2\\approx19.739$ and $\\gamma=9\\pi^2/2\\approx 44.413$. In  the $(\\gamma, \\ell)$-plane  the  gain-and-loss amplitudes corresponding to the   group of blue curves are well separated from  those for the red curves: indeed, all red curves are bounded from above by the asymptote $\\gamma= \\pi^2/2\\approx 4.935$, while all blue curves are bounded from below by $\\gamma_*\\approx 11.561$, see (\\ref{eq:gap}). Thus, the  emergence of new spectral singularities with the increase of $\\ell$ is sensitive to the value of the gain-and-loss amplitude and does not occur for     the gain-and-loss amplitudes lying in the  gap  between the red and blue curves.\n\nIn comparison with the red curves discussed above, the curves from the blue group in figure~\\ref{fig:general}(a) feature more complicated behavior and, in particular, can intersect each other (and also intersect the curves from the next, third group of green curves discussed below). The intersections between the blue curves occur for the gain-and-loss amplitudes in the interval $11.561 \\lessapprox   \\gamma <  9\\pi^2/2\\approx 19.739$, where $\\sin\\sqrt{2\\gamma}$ is negative.  \\textcolor{black}{At first glance,} this might seem to contradict to  the  expansion (\\ref{3.4}) of Lemma~\\ref{lm3.4},  which  suggests that in this case the multitude of complex zeros   accumulate in the upper complex half-plane with the growth of $\\ell$. However, this apparent contradiction is resolved if we trace the behavior of the complex roots more closely.   Indeed, choosing for an example $\\gamma =16$ [see the vertical axis tick in  figure~\\ref{fig:general}(a)] and computing several  complex roots under the increase of $\\ell$, we observe that each considered root first goes down from the upper half-plane to the lower one but then again returns to the upper half-plane, see Figure~\\ref{fig:g=16}(b) and (b$_1$). Thus,  in  this  interval of the gain-and-loss amplitudes the increase of $\\ell$  results  either in the transformation from the resonance to the eigenvalue or to the opposite process,  i.e. to the disappearance of the complex-conjugate pair.  Respectively, the intersection between the two blue curves corresponds to  two coexisting spectral singularities, i.e. to the moment  when one complex-conjugate pair of eigenvalues disappears and another pair (with different $k$) emerges. For sufficiently large $\\ell$ the imaginary part  of each   considered root  remains positive and approaches zero, in accordance with expansion (\\ref{3.4}). Thus, in this interval of the gain-and-loss strengths, the limit $\\ell\\to\\infty$ the spectrum contains only a finite number of complex-conjugate eigenvalues, which correspond to complex zeros $k$ whose behavior is not covered   by expansion  (\\ref{3.4}).\n\n \\begin{figure}\n\t\\centering\n\t\\includegraphics[width=0.99\\columnwidth]{fig08.eps}\n\t\\caption{(a,b) Real and imaginary parts for three   complex zeros of function $F$ for fixed $\\gamma=16$ and increasing $\\ell$. Panel b$_1$ is the magnification of some region of (b) and  shows more clearly that imaginary parts of all   three shown eigenvalues are positive for all sufficiently large  $\\ell$. \\textcolor{black}{For each three shown eigenvalues, its real and \timaginary parts are of the same colour in both panels.}}\n\t\\label{fig:g=16}\n\\end{figure}\n\nThe third  group of spectral singularities [green curves in figure~\\ref{fig:general}(a)] was obtained by means of the continuation from  the solution $\\gamma_\\star^{(2)}$ and $k_\\star^{(2)}$ in Table~\\ref{tbl:1}. In view of the very complicated structure of the overall resulting picture, we only show a section of these curves corresponding to relatively small   values of the gain-and-loss amplitudes $\\gamma$. Quite interestingly, there is no gap between the blue and green groups of curves, which results in the multitude of intersections between blue and green curves, \\textcolor{black}{see Fig.~\\ref{fig:general}(a) and the magnified view in Fig.~\\ref{fig:general}(c)}. These   intersections suggest a possibility of   simultaneous emergence of two complex-conjugate pairs of eigenvalues from two different interior points of the continuous spectra.\n\nConsidering further solutions $\\gamma_\\star^{(n)}$, $k_\\star^{(n)}$, $n=3,4,\\ldots$, in the limit $\\ell=0$ one can construct new groups of spectral singularities  with larger values of $\\gamma$, which are not shown in Figure~\\ref{fig:general}.\n\n\n",
        "label": "fig:threshold",
        "Descriptive_question1": "What is the approximate value of the PT-symmetry breaking threshold γ_star^(0) at ℓ = 0 in figure_5?",
        "Descriptive_question2": "What is the approximate wavevector k_star^(0) value corresponding to ℓ = 0 in figure_5?",
        "Reasoning_question1": "How does the PT-symmetry breaking threshold γ_star^(0) change as the distance between gain and loss (ℓ) increases in figure_5, and what might this imply about achieving laser-antilaser operation?",
        "Reasoning_question2": "Based on figure_5, what can be inferred about the relationship between the wavevector k_star^(0) and the distance ℓ, and how might this affect the system's spectral behavior?",
        "Descriptive_answer1": "2.072",
        "Descriptive_answer2": "1.065",
        "Reasoning_answer1": "As the distance ℓ between the gain and loss increases, the PT-symmetry breaking threshold γ_star^(0) decreases monotonically, as observed in figure_5(a). This implies that by introducing a spatial separation between gain and loss regions, the system requires a lower gain-and-loss amplitude to reach the threshold where PT-symmetry breaks, facilitating laser-antilaser operation at lower gain levels compared to having adjacent gain and loss.",
        "Reasoning_answer2": "Figure_5(b) shows that the wavevector k_star^(0) also varies with distance ℓ, exhibiting a nontrivial dependence rather than remaining constant. Since k_star^(0) corresponds to the spectral singularity where complex-conjugate eigenvalues bifurcate, its variation with ℓ suggests that the spectral properties and resonance conditions of the system are sensitive to the gain-loss separation. Therefore, varying ℓ can tune the spectral position of critical points, influencing how the spectrum transitions to complex eigenvalues and affecting the system's overall spectral behavior."
    },
    {
        "paper_id": "1908.06383.json",
        "image_id": "figure_6",
        "image_path": [
            "/home/yz979/scratch/chengye/scimolmo/test-images-tables/dataset_imagestables/1908.06383/images/fig06.eps"
        ],
        "caption": "(a) Values of the gain-and-loss amplitude $\\gamma$ and distance  $\\ell$, for which   spectral singularities occur. (b) Corresponding  wavenumbers   $k$. \\textcolor{black}{Panel (c) magnifies   the region  $(\\ell, \\gamma)\\in [2,4]\\times[10,50]$ from   (a), and panel  (d) is the  magnification of corresponding curves from panel (b).}",
        "classify": "Chart",
        "section_info": "5 Spectral singularities\n\\section{Spectral singularities}\n\\label{sec:ss}\n\\subsection{General analytical expressions}\n\nIn this section we study real zeroes of the function $F$ corresponding to spectral singularities, i.e. to  zero-width resonances.\nFor such zeroes, equation (\\ref{2.10}) is a pair of two real equations for one real variable $k$ and two parameters $\\ell$ and $\\g$.\nThanks to the symmetry of the zeroes with respect to the imaginary axis, it is sufficient to find only positive real resonances since the negative ones are located symmetrically with respect to the origin. Similar to the proof of Lemma~\\ref{lm3.6}, for real positive $k$  we make  change (\\ref{4.1})\nand rewrite equation (\\ref{2.10}) in the following form:\n\\begin{align*}\n(1-2u^{-4})\\cos\\b u^{-1}&+(2u^4-1)\\cosh\\b u + 2i \\sqrt{1-u^4}\\left(u^2\\sinh\\b u - u^{-4}\\sin\\b u^{-1}\\right)\n\\\\\n&-e^{-2i\\b\\ell\\sqrt{u^{-2}-u^2}} \\left(\\cosh(\\b u)-\\cos\\b u^{-1}\\right)=0,\\qquad \\b=\\sqrt{2\\g}.\n\\end{align*}\nTaking the real and imaginary part of this equation and multiplying the equation by $u^4$, we obtain:\n\\begin{equation}\\label{4.3}\n\\begin{aligned}\n&(u^4-2)\\cos\\b u^{-1}+u^4(2u^4-1)\\cosh\\b u\n=u^4\\cos\\left(2\\b\\ell\\sqrt{u^{-2}-u^2}\\right)\\left(\\cosh \\b u-\\cos\\b u^{-1}\\right),\n\\\\\n&2\\sqrt{1-u^4}\\Big(u^6\\sinh\\b u-\\sin\\b u^{-1}\\big)\n=-u^4\\sin\\left(2\\b\\ell\\sqrt{u^{-2}-u^2}\\right)\\left(\\cosh \\b u-\\cos\\b u^{-1}\\right).\n\\end{aligned}\n\\end{equation}\nThis is a system of two real equations with three real variables. If we are given $(\\b,\\ell)$ and we try to find $u$, the system is overdetermined and does not necessary have a root. In other words, it is solvable with respect to $u$ only if $(\\b,\\ell)$ are located on some (solvability) curves. In order to avoid working with an overdetermined system, in what follows we regard (\\ref{4.3}) as a system for two unknown variable with one parameter.\n\nTo find the curves in $(\\b,\\ell)$ plane, on which equations (\\ref{4.3}) are solvable with respect to $u$, we shall regard $u$ as a parameter and $(\\b,\\ell)$ as unknown variables. We take the sum of squares of equations (\\ref{4.3}) and divide the result by $(1-u^4)$. We also divide equations (\\ref{4.3}). This leads us to a   pair  of equations:\n\\begin{align}\\label{4.4}\n&u^4(1-u^4) \\cosh\\b u\\cos\\b u^{-1}-2u^6 \\sinh\\b u\\sin\\b u^{-1} +1-u^{12}=0,\n\\\\\n&\\frac{2\\sqrt{1-u^4}(u^6\\sinh\\b u-\\sin\\b u^{-1})}{(2-u^4)\\cos\\b u^{-1}+u^4(1-2u^4)\\cosh\\b u}=\\tan \\Big( 2\\b\\ell\\sqrt{u^{-2}-u^2}\\Big).\\nonumber\n\\end{align}\nThe second equation can be solved explicitly with respect to $\\ell$:\n\\begin{equation}\\label{4.10}\n\\ell=\\frac{1}{2\\b\\sqrt{u^{-2}-u^2}} \\left(\\arctan \\frac{2\\sqrt{1-u^4}(u^6\\sinh\\b u-\\sin\\b u^{-1})}{(2-u^4)\\cos\\b u^{-1}+u^4(1-2u^4)\\cosh\\b u}+\\pi n\\right),\n\\end{equation}\nwhere $n\\in\\mathds{N}$ is an arbitrary natural number. As the next lemma states, to make equations (\\ref{4.4}), (\\ref{4.10}) equivalent to (\\ref{4.3}), we should also assume that\n\\begin{equation}\\label{4.5}\n(-1)^n=\\sign \\big((u^4-2)\\cos\\b u^{-1}+u^4(2u^4-1)\\cosh\\b u\\big).\n\\end{equation}\n\n\\begin{lemma}\\label{lm4.1}\nEquations (\\ref{4.3}) are equivalent to (\\ref{4.4}), (\\ref{4.10}), (\\ref{4.5}).\n\\end{lemma}\n\n\\begin{proof}\nWe rewrite shortly equations (\\ref{4.3}) as\n$A_1=B\\cos\\a$, $ A_2=-B\\sin\\a$,\nwhere $A_1$, $A_2$ are the left hand sides in (\\ref{4.3}), $\\a=2\\b\\ell\\sqrt{u^{-2}-u^2}$ and $B=u^4(\\cosh\\b u-\\cos\\b u^{-1})$. Then equations (\\ref{4.4}), (\\ref{4.10}) become\n$A_1^2+A_2^2=B^2$, $\\a=-\\arctan\\frac{A_2}{A_1}+\\pi n$.\nWe have:\n\\begin{equation*}\nB\\cos \\a=(-1)^n\\cos\\arctan\\frac{A_2}{A_1}=\\frac{(-1)^n}{\\sqrt{1+\\frac{A_2^2}{A_1^2}}} = \\frac{(-1)^n|A_1|}{\\sqrt{A_1^2+A_2^2}}\n\\end{equation*}\nand we get the first equation $A_1=B\\cos\\a$  provided condition (\\ref{4.5}) is satisfied. In the same way we check that the latter condition also ensures the second equation $A_2=-B\\sin\\a$.\n\\end{proof}\n\n\nEquation (\\ref{4.4}) is transcendental, and we can not solve it analytically.\nNevertheless, for each $u\\in(0,1)$, this is an equation only for a single variable $\\b$, not for two as equations (\\ref{4.3}). So, we propose the following algorithm of recovering the aforementioned solvability curves: choose $u\\in(0,1)$, then solve equation (\\ref{4.4}) and recover the sequence of  distances  $\\ell$ by formula (\\ref{4.10}) with different integer $n$. Then the gain-and-loss amplitude $\\gamma$ and the corresponding wavenumber $k$ can be readily recovered from $\\beta$ and  $u$.  In a similar way, one can first fix some value of $\\beta$ (i.e., fix the gain-and-loss strength) and then   solve equation (\\ref{4.4}) with respect to $u$ and   recover   $\\ell$ by (\\ref{4.10}). Equation (\\ref{4.4})  is well-behaved, and for each $\\beta$  all its zeros  $u$ can be easily found numerically.\n\nAlternatively, as explained below  in Section~\\ref{sec:general}, the values  corresponding to  spectral singularities can be found systematically by means of the numerical continuation from the limit $\\ell=0$. However, in this case   equation (\\ref{4.4}) is still useful because it allows one to check that all spectral singularities have been found for the given value of the gain-and-loss $\\gamma$.\n\n\n\\subsection{\nAbsence of   spectral singularities}\n\\label{sec:gap}\n\nFor $u=0$ and $u=1$, the left-hand-side of equation (\\ref{4.4}) is equal respectively to $1$ and $-2\\sinh\\beta\\sin\\beta$. Then a sufficient condition for the existence of a spectral singularity at the given gain-and-loss amplitude $\\gamma$ is $\\sin\\beta=\\sin\\sqrt{2\\gamma}>0$. At the same time, it is also possible to establish sufficient conditions that forbid the existence of   spectral singularities in a certain interval of parameters. In this subsection we prove the existence of two ``forbidden gaps'' for the roots of equation (\\ref{4.4}). The first one exists for all $\\b\\geqslant 0$ and it states that there is no roots in certain interval. The second gap is a certain interval of values of $\\b$, for which equation (\\ref{4.4}) has no zeroes at all.\n\nFor the convenience, by $g(u,\\b)$ we denote the left hand side of equation (\\ref{4.4}). The first ``forbidden gap'' is described in the following lemma.\n\n\\begin{lemma}\\label{lm5.1}\nFor all $\\b\\geqslant 0$, equation (\\ref{4.4}) has no roots in the interval $\\big[0,(1+\\tfrac{\\b}{4})^{-1}\\big)$.\n\\end{lemma}\n\\begin{proof}\nEmploying a standard inequality $a\\cos\\a+b\\sin\\a\\leqslant \\sqrt{a^2+b^2}$, we estimate the first two term in equation (\\ref{4.4}) as\n\\begin{equation*}\nu^4(1-u^4) \\cosh\\b u\\cos\\b u^{-1}-2u^6 \\sinh\\b u\\sin\\b u^{-1} \\leqslant \\sqrt{u^8(1-u^4)^2\\cosh^2 \\b u+4u^{12}\\sinh^2\\b u}.\n\\end{equation*}\nHence, equation (\\ref{4.4}) surely has no roots for values of $u$ satisfying\n\\begin{equation*}\n \\sqrt{u^8(1-u^4)^2\\cosh^2 \\b u+4u^{12}\\sinh^2\\b u} <1-u^{12}.\n\\end{equation*}\nExpressing $\\cosh^2 \\b u$ via $\\sinh^2 \\b u$ and simplifying this inequality, we obtain $ u^4(1+u^4)\\cosh \\b u <1+u^{12}$ and hence,\n\\begin{equation}\\label{3.37}\n\\cosh \\b u-1 <\\frac{1-2u^4+u^8}{u^4},\\qquad \\sqrt{2}\\sinh \\b u\n<u^{-2}-u^2.\n\\end{equation}\nThanks to the estimate $\\sqrt{2}\\sinh 2\\ln u^{-1}\\leqslant u^{-2}-u^2$, $u\\in(0,1]$, the last inequality in (\\ref{3.37}) holds once $\\frac{\\b}{4}<u^{-1}\\ln u^{-1}$. It is easy to confirm that this inequality is true once $u^{-1}>1+\\tfrac{\\b}{4}$.\nThe proof is complete.\n\\end{proof}\n\n\nThe next lemma is   auxiliary  and will be employed in studying the second forbidden zone.\n\n\\begin{lemma}\\label{lm5.2}\nThe function $g(u,\\pi)$ is positive on $[0,1)$.\n\\end{lemma}\n\n\\begin{proof}\nWe have $g(0,\\pi)=1$ and by Lemma~\\ref{lm5.1}, it is positive for $u<(1+\\tfrac{\\pi}{4})^{-1}$. This is why in what follows we consider only the values $u\\geqslant (1+\\tfrac{\\pi}{4})^{-1}$. For such values of $u$ we have\n$\\pi\\leqslant \\pi u^{-1}\\leqslant \\pi (1+\\tfrac{\\pi}{4})<1.79\\pi$.\nAs\n$\\tfrac{3\\pi}{2} \\leqslant \\pi u^{-1}\\leqslant \\pi (1+\\tfrac{\\pi}{4})$,\nthe function $\\sin\\b u^{-1}$ is negative, while $\\cos \\b u^{-1}$ is positive. Hence, for such values of $u$, the function $g(u,\\pi)$ is positive. It remains to consider the values $\\frac{2}{3}<u\\leqslant 1$.\n\nFor such values of $u$ we first observe the following simple estimates:\n\\begin{equation*}\n\\frac{1-u^4}{1-u}\\cos\\pi u^{-1}\\geqslant -4,\\qquad -\\frac{\\sin \\pi u^{-1}}{1-u}=\\frac{\\sin \\pi (u^{-1}-1)}{1-u}\\geqslant \\pi u.\n\\end{equation*}\nThen the function $g$ satisfies the estimate:\n\\begin{equation}\\label{3.12}\n\\frac{g(u,\\pi)}{u^4(1-u)}\\geqslant g_1(u)+g_2(u),\\qquad g_1(u):=-4\\cosh\\pi u +2\\pi u^3 \\sinh \\pi u,\\qquad g_2(u):=\\frac{1-u^{12}}{u^4(1-u)}.\n\\end{equation}\nThe function $g_1(u)$ is monotonically increasing in $u\\in[\\tfrac{2}{3},1]$ since\n\\begin{equation*}\ng_1'(u)=2\\pi^2 u^3\\cosh\\pi u+(6\\pi u^2-4\\pi)\\sinh\\pi u>2\\pi(\\pi u^3+6u^2-2)\\sinh\\pi u>2\\pi\\sinh\\pi u>0.\n\\end{equation*}\nHence,\n$g_1(u)\\geqslant g_1\\left(\\frac{2}{3}\\right)>-9.05$.\nFor the function $g_2$ we have the following representation and estimate:\n\\begin{equation*}\ng_2(u)=1+\\sum\\limits_{j=1}^{4}(u^j+u^{-j})+\\sum\\limits_{j=5}^{8} u^j\\geqslant 9 +\\sum\\limits_{j=5}^{8}\\left(\\frac{2}{3}\\right)^j>9.31.\n\\end{equation*}\nTwo last estimates and (\\ref{3.12}) imply the positivity of the function $g$ for $u\\in[\\tfrac{2}{3},1)$.\n\\end{proof}\n\n\nDenote\n\\begin{align*}\ng_*(u,\\b):=&\\b u^3(1-3u^4)\\cosh\\b u \\sin \\b u^{-1}+\\b u^5 (3-u^4)\\sinh\\b u \\cos \\b u^{-1}\n\\\\\n&-2 u^4(1+u^4) \\cosh\\b u \\cos \\b u^{-1} -6(1+u^{12}).\n\\end{align*}\n\n\nThe next lemma states the existence of the second forbidden zone.\n\n\\begin{lemma}\\label{lm5.3}\nEquation (\\ref{4.4}) has no roots as $\\pi<\\b<\\b_*<5$, where $(u_*, \\b_*)$ is  the root of the system of the equations\n\\begin{equation}\\label{3.13}\ng(u,\\b)=0,\\qquad g_*(u,\\b)=0,\\qquad u\\in[0,1],\\qquad \\pi<\\b<5,\n\\end{equation}\nwith minimal possible $\\b$.\nTheir approximate values are\n\\begin{equation}\\label{3.14}\n\\b_*=4.808438,\\qquad u_*=0.611772.\n\\end{equation}\n\\end{lemma}\n\n\\begin{proof}\nThe function $g(u,\\pi)$ is positive on $[0,1)$ and $g(1,\\pi)=0$, see Figure~\\ref{fig:forbidden}a. As $\\pi<\\b<2\\pi$, we have $g(0,\\b)=1>0$ and $g(1,\\b)=-2\\sin\\b\\sinh\\b>0$. Hence, for $\\b$ close enough to $\\pi$, the function $g(u,\\b)$ is positive for all $u\\in[0,1]$. At the same time, we have $g(0.65,5)<-0.617<0$ and therefore, for $\\b=5$, equation (\\ref{4.4}) possesses at least two roots, one in $(0,0.65)$ and another in $(0.65,1)$. cf. Figure~\\ref{fig:forbidden}c. We also observe that the function $g$ is jointly continuous in $(u,\\b)$. The above facts means that as $\\b$ grows from $\\pi$ to $5$, at some value $\\b=\\b_*$, the graph of the function $g$ is still located in the upper half-plane but touches the $u$-axis at some point $u=u_*$,\nsee Figure~\\ref{fig:forbidden}b. The function $g(u,\\b)$ is positive as $\\pi<\\b<\\b_*$ and $u\\in[0,1]$. Then the point $u=u_*$ is obviously the global minimum of $g$ and hence, $(u_*,\\b_*)$ is a solution to the system of equations $g(u,\\b)=0$, $\\frac{\\p g}{\\p u}(u,\\b)=0$. It is easy to check that $g_*=\\frac{\\p g}{\\p u}-6 g$ and hence, $(u_*,\\b_*)$ solves system (\\ref{3.13}). These roots can be found numerically and this gives (\\ref{3.14}). The proof is complete.\n\\end{proof}\n\n\n\\begin{figure}\n\t\\centering\n\\includegraphics[width=0.99\\columnwidth]{fig03.eps}\n\\caption{Illustration for proof of Lemma~\\ref{lm5.3}. Graphs of the function $g(u, \\beta)$ for $\\b=\\pi$ (a), $\\b=\\b_*\\approx  4.808438$ (b), and $\\b=5$ (c). Notice broken vertical axes in (b) and (c). }\n\\label{fig:forbidden}\n\\end{figure}\n\nReturning from the auxiliary variable $\\beta$ to the gain-and-loss amplitude $\\gamma=\\beta^2/2$, from   Lemma~\\ref{lm5.3}  we deduce  the following important result:\n\\begin{equation}\n\\label{eq:gap}\n\\textrm{there is no spectral singularities for\\quad }  \\frac{\\pi^2}{2} < \\gamma< \\gamma_*\\approx  11.561.\n\\end{equation}\n\n\n\n\\subsection{Creating a  spectral singularity  at a given wavenumber}\n\n\\begin{table}\n\t\\centering\n\t\\begin{tabular}{c|cccc}\n\t\t$n$ & 0 & 1& 2&\\\\\\hline\n\t\t$\\gamma_\\star^{(n)}$ & 2.071&13.307&27.783 &\\\\[2mm]\n\t\t$k_\\star^{(n)}$ & 1.065&4.318 &7.529 &\n\t\\end{tabular}\n\t\\caption{Approximate values of gain-and-loss amplitudes $\\gamma=\\gamma_\\star^{(n)}$  and wavenumbers   $k=k_\\star^{(n)}$, $n=0, 1, \\ldots$,  corresponding to spectral singularities with lowest $\\gamma$ in the limit $\\ell=0$, see Table~I in  \\cite{Mostafazadeh2009}. \\label{tbl:1}}\n\\end{table}\n\n\nFor $\\ell=0$ spectral singularity can only be obtained for some isolated values of the wavenumber $k$ and the gain-and-loss amplitude $\\gamma$ \\cite{Mostafazadeh2009}. Several lowest values of $\\gamma$ corresponding to the spectral singularities  and the associated wavenumbers  $k$ are listed in Table~\\ref{tbl:1}.  An  important advantage of the more general system with nonzero gain-to-loss disctance    $\\ell>0$ consists  in the possibility to create a spectral   singularity   at any wavenumber $k$ given beforehand. Indeed, let us  return back to equations (\\ref{4.4}), (\\ref{4.10}) and    discuss the following   issue: given a point $k$ on the real axis, how to choose $\\b$ and $\\ell$ to have a  resonance at this point? Equations (\\ref{4.4})--(\\ref{4.10}) allow us to answer  easily this question.\n\nWe fix $k>0$ and we find the associated value of $u$ by resolving (\\ref{4.1}):\n\\begin{equation}\\label{4.12}\nu^{-2}-u^2=4k^2\\b^{-2},\\qquad\n\nu=\\b R^{-1}, \\quad \\mbox{where\\ } R = \\sqrt{2k^2+\\sqrt{4k^4+\\b^4}}.\n\\end{equation}\nWe divide equation (\\ref{4.4}) by $u^6$ and substitute then the above formulae and\n\\begin{equation*}\n\\frac{1-u^{12}}{u^6}=\\frac{1-u^4}{u^2}\\frac{1+u^4+u^8}{u^4}=(u^{-2}-u^2)\\big((u^{-2}-u^2)^2+3\n\\big).\n\\end{equation*}\nThis gives the equation:\n\\begin{equation}\\label{4.11}\n\\begin{aligned}\n2\\b^4 k^2&\\cosh(\\b^2R^{-1})  \\cos R\n\n\n - \\b^6 \\sinh(\\b^2R^{-1})  \\sin R\n  +2k^2(16k^4+3\\b^4)=0.\n\\end{aligned}\n\\end{equation}\nAn algorithm for creating a resonance at a prescribed point  $k$ is as follows. Given $k>0$, we first solve equation (\\ref{4.11}) with respect to $\\b$ and we also find $u$ by (\\ref{4.12}). Then needed values of  $\\ell$ are determined by (\\ref{4.10}), (\\ref{4.5}).\n\n\nIn order to illustrate this  algorithm,  we us consider a finite interval of wavenumbers $k\\in (0, k_1]$, where  we set $k_1 = 10$ for the numerics reported on in what follows. We scan   the chosen interval  with a sufficiently small step ($\\Delta k=0.01$) and for each value of $k$ solve equation (\\ref{4.11}) numerically  using the simple dichotomy method. While for each $k$  equation (\\ref{4.11}) might have several roots $\\beta$, in our numerical procedure we always choose the  minimal positive root, i.e., the one which allows to achieve the spectral singularity with given $k$ at the smallest possible value  of the gain-and-loss amplitude $\\gamma=\\gamma_\\textrm{min}$. Next, we choose the minimal positive distance $\\ell_\\textrm{min}$ which satisfies the conditions   (\\ref{4.10}), (\\ref{4.5}) and then we use the periodicity in $\\ell$ to generate the sequence of larger gain-to-loss distances $\\ell_n = \\ell_\\textrm{min} + n\\pi/(2k)$, $n=1,2\\ldots$ [see   (\\ref{eq:periodic})]. The resulting dependencies $\\gamma_\\textrm{min}(k)$ and $\\ell_\\textrm{min}(k)$, $\\ell_n(k)$   are shown in figure~\\ref{fig:min}. The minimal gain-and-loss amplitude $\\g_{min}(k)$  and the minimal gain-to-loss distance $\\ell_{min}$ are   discontinuous,  which means that the small variation  in  the wavenumuber $k$  might require  a significant  change either in   $\\gamma$ or in $\\ell$.  It is especially important that the values of the  distance  $\\ell$ are generically different from zero, which points out explicitly    that the new degree of freedom offered by the nonzero gain-to-loss distance    is   important for the achieving a spectral singularity  at   the given wavenumber $k$.\n\n\n\\begin{figure}\n\t\\centering\n\t\\includegraphics[width=0.8\\columnwidth]{fig04.eps}\n\t\\caption{(a) Minimal value of the gain-and-loss amplitude $\\gamma_\\textrm{min}$  which corresponds to   a spectral singularity with the given value of the wavenumber $k$. (b)  Minimal gain-to-loss distance  $\\ell_\\textrm{min}$  which corresponds to   a spectral singularity with the  $k$ and $\\gamma$ from the  left panel (bold curves) and larger distances $\\ell_n$ obtained using the periodicity in $\\ell$ (thin \\textcolor{black}{dotted} curves).}\n\t\\label{fig:min}\n\\end{figure}\n\n\n\\subsection{$\\PT$-symmetry breaking laser-antilaser threshold}\n\nA particularly important characteristics of any $\\PT$-symmetric structure is the $\\PT$ symmetry breaking threshold, i.e., the amplitude of the gain-and-loss\ncorresponding to the  ``phase transition'' from the purely real to complex spectrum.  The best studied scenario of the phase transition  \nis the collision of two real discrete  eigenvalues at an exceptional point with the subsequent splitting in a complex-conjugate pair. However, in systems with nonempty continuous spectrum,  the phase transition can also occur through   the splitting of a self-dual spectral singularity, which results in a bifurcation of a complex-conjugate pair from an interior point of the continuum \\cite{Yang17,KZ17,Konotop2019}. At the moment corresponding to the formation of the spectral singularity, the system operates in the CPA-laser regime \\cite{Longhi10}. Thus, in such a system, the $\\PT$-symmetry breaking threshold at the same time corresponds to the CPA-laser threshold.\n\nLemma~\\ref{lm3.3} guarantees that the spectrum of our system is real for sufficiently small gain-and-loss amplitudes $\\gamma$. Additionally, according to Lemma~\\ref{lm3.5}, the spectrum  does not have any real discrete eigenvalue. Hence, the $\\PT$-symmetry breaking is expected to occur through the emergence of a self-dual spectral singularity. In order to identify the $\\PT$-symmetry breaking threshold in our system, we start from the limit $\\ell=0$, there the phase transition takes place at  $\\gamma_\\star^{(0)} \\approx 2.072$, see \\cite{Mostafazadeh2009,KZ17} and Table~\\ref{tbl:1}. Thus, the spectrum with $\\ell=0$ is purely real and continuous for $\\gamma \\in [0, \\gamma_\\star]$, while the  increase of the gain-and-loss just above $\\gamma_\\star^{(0)}$ leads to the bifurcation of a complex-conjugate pair from an interior point  of the continuum. The spectral singularity forming at  $\\gamma_\\star^{(0)}$ takes place at wavenumber $k=k_\\star^{(0)} \\approx  1.065$. Respectively, the complex-conjugate pair of eigenvalues bifurcates from $\\lambda_0 =  [k_\\star^{(0)}]^2$. (Notice that the further increase of $\\gamma$ above the next threshold values listed in Table~\\ref{tbl:1} leads to the formation of new spectral singularities and, respectively, to bifurcations of new complex-conjugate pairs in the spectrum.)\n\nNext, we use the numerical continuation in $\\ell$ in order to continue the known solution at $\\ell=0$ to the domain $\\ell>0$.  The obtained dependence of the threshold value  of the gain-and-loss amplitude on distance $\\ell$ is shown in figure~\\ref{fig:threshold}(a), where we observe that the phase transition threshold decreases monotonically with the growth of $\\ell$. This means that introducing an additional space   between the gain and loss, one can decrease the $\\PT$-symmetry breaking  threshold, i.e. achieve the laser-antilaser operation at \\textcolor{black}{lower} gain-and-loss amplitudes than in a waveguide with the adjacent gain and loss.\n\n\n\\begin{figure}\n\t\\centering\n\t\\includegraphics[width=0.85\\columnwidth]{fig05.eps}\n\t\\caption{(a) $\\PT$-symmetry breaking threshold $\\gamma_\\star^{(0)}$ \\textit{vs} the distance between the gain and loss $\\ell$. The spectrum is purely real and continuous for $\\gamma\\leq \\gamma_\\star$, but acquires a pair of complex conjugate eigenvalues  as the gain-and-loss amplitude exceeds the threshold $\\gamma_\\star^{(0)}$. (b) Values of the wavevector $k_\\star^{(0)}$ corresponding to the dependence in (a).}\n\t\\label{fig:threshold}\n\\end{figure}\n\n\\subsection{General picture of spectral singularities}\n\\label{sec:general}\n\n\\begin{figure}\n\t\\centering\n\t\\includegraphics[width=0.99\\columnwidth]{fig06.eps}\n\t\\caption{(a) Values of the gain-and-loss amplitude $\\gamma$ and distance  $\\ell$, for which   spectral singularities occur. (b) Corresponding  wavenumbers   $k$. \\textcolor{black}{Panel (c) magnifies   the region  $(\\ell, \\gamma)\\in [2,4]\\times[10,50]$ from   (a), and panel  (d) is the  magnification of corresponding curves from panel (b).}}\n\t\\label{fig:general}\n\\end{figure}\n\nLet us now turn to description of the general picture of spectral singularities. In order to construct systematically different solutions, we again start from the limit $\\ell=0$, where the values of $\\gamma$ and $k$ corresponding to spectral singularities are known, see Table~\\ref{tbl:1}. Then we use the periodicity of function $e^{-4ik\\ell}$ in $\\ell$ in order to construct new branches of solutions having no counterparts in the limit $\\ell=0$, cf. equation (\\ref{eq:periodic}). This procedure results in a fairly complicated picture containing a multitude of spectral singularities, some part of which (corresponding to  relatively  small   values of the gain-and-loss) is shown in figure~\\ref{fig:general}(a,b) as the curves on the plane $\\gamma$ {\\it vs} $\\ell$ and $k$ {\\it vs} $\\ell$.\n\n\nLet us describe the structure of the found solutions using the  diagram $\\gamma$ {\\it vs} $\\ell$ in figure~\\ref{fig:general}(a). The multitude of curves shown in this plot can be divided into three groups (plotted with red, blue and green curves) which have been obtained by means of the continuation from three different solutions in the limit $\\ell=0$. There is a well-visible vertical gap between red and blue curves, which corresponds to the ``forbidden'' values of the gain-and-loss amplitudes $\\gamma$ found above in (\\ref{eq:gap}). At the same time,  there is no gap between blue and green curves, which  results in a multitude of intersections between these curves.\nThe first group of spectral singularities,  corresponding to red curves in Figure~\\ref{fig:general}(a),   is obtained through  the continuation from the  spectral singularity  in the limit $\\ell=0$ with the smallest gain-and-loss amplitude, i.e.  from values $\\gamma_\\star^{(0)}$ and $k_\\star^{(0)}$ in Table~\\ref{tbl:1}.  The leftmost (bold)  curve in this group which originates in the limit $\\ell=0$  is the $\\PT$-symmetry breaking threshold which was already shown in figure~\\ref{fig:threshold}(a). For values of $\\gamma$ above this curve, there is always one or more (but finitely many, see Section~\\ref{sec:zeros})   complex-conjugate pairs of eigenvalues in the spectrum.  Several curves situated to the right from the bold red curve are obtained using the fact that if $\\gamma$ and $k$ are solutions in the limit $\\ell=0$, then the same values of $\\gamma$ and $k$ also correspond to a spectral singularity with $\\ell_n = (n\\pi)/(2k)$, $n=1,2,\\ldots$. Once a single solution with a new distance   $\\ell_n$ is obtained, a new branch of solutions can be constructed using numerical continuation in $\\gamma$ or in $\\ell$. In the limit $\\ell \\to \\infty$ all the red curves in figure~\\ref{fig:general}(a) approach  the asymptotic value $\\gamma=0$ or at $\\gamma_*= \\pi^2/2\\approx 4.935$. Notice that, except for the bold line demarcating the $\\PT$-symmetry breaking threshold, none of the red curves can be continued to the limit $\\ell\\to 0$.\n\nThe multitude of red curves  in figure~\\ref{fig:general}(a) demonstrates explicitly how new spectral singularities emerge with the increase of $\\ell$. Indeed, drawing an imaginary horizontal line, say, at $\\gamma=3$ [see the vertical axis  tick  in figure~\\ref{fig:general}(a)], we observe that with the increase of $\\ell$ this line   intersects more and more red curves. Each intersection corresponds to the values of $\\gamma$ and $\\ell$ at which   some  root $k$   crosses the real axis and goes down from the upper to lower complex half-plane. Thus,  a finite-width resonance transforms  to a complex eigenvalue through the spectral singularity (we recall again that  in view of   $\\PT$ symmetry  any root $k\\ne 0$ crosses the real line simultaneously with its counterpart $-\\bar{k}$, i.e.  the corresponding   spectral singularity is self-dual).\nIn order to illustrate this process, in Figure~\\ref{fig:g=3}  we show the evolution of three numerically found complex zeros of function $F(k, \\gamma, \\ell)$  under   the increase of $\\ell$.  In this figure the imaginary part of each complex zero changes sign from positive to negative and then asymptotically approaches zero (remaining negative). In the limit of large $\\ell$,  this behavior agrees  with expansion (\\ref{3.4}) of Lemma~\\ref{lm3.4} , where, for the chosen value of $\\gamma$,   we have $\\sin\\sqrt{2\\gamma}>0$.   Thus the growing distance   $\\ell$ results in a sequence of self-dual spectral singularities and in the increasing number of complex-conjugate eigenvalues in the spectrum.\n\n \\begin{figure}\n \t\\centering\n \t\\includegraphics[width=0.8\\columnwidth]{fig07.eps}\n \t\\caption{(a,b) Real and imaginary parts of three complex zeros of function $F$ for fixed $\\gamma=3$ and increasing $\\ell$. For each curve, the imaginary part is positive for small $\\ell$ and becomes negative for all sufficiently large $\\ell$. \\textcolor{black}{For each three shown eigenvalues, its real and \timaginary parts are of the same colour in both panels.}}\n \t\\label{fig:g=3}\n \\end{figure}\n\n\nThe second group of spectral singularities [blue curves in figure~\\ref{fig:general}(a)] was obtained by means of the continuation from the  next solution in the limit $\\ell=0$, i.e.  from $\\gamma_\\star^{(1)}$ and $k_\\star^{(1)}$ in Table~\\ref{tbl:1}. Again, one of the curves [the leftmost bold curve in figure~\\ref{fig:general}(a)] was obtained through the direct continuation from the limit $\\ell=0$, while other blue curves were generated  using the periodicity of function $F(k, \\gamma, \\ell)$  in $\\ell$ and cannot be continued to the limit $\\ell\\to0$. In the limit $\\ell\\to\\infty$ the blue curves approach the horizontal asymptotes  $\\gamma=2\\pi^2\\approx19.739$ and $\\gamma=9\\pi^2/2\\approx 44.413$. In  the $(\\gamma, \\ell)$-plane  the  gain-and-loss amplitudes corresponding to the   group of blue curves are well separated from  those for the red curves: indeed, all red curves are bounded from above by the asymptote $\\gamma= \\pi^2/2\\approx 4.935$, while all blue curves are bounded from below by $\\gamma_*\\approx 11.561$, see (\\ref{eq:gap}). Thus, the  emergence of new spectral singularities with the increase of $\\ell$ is sensitive to the value of the gain-and-loss amplitude and does not occur for     the gain-and-loss amplitudes lying in the  gap  between the red and blue curves.\n\nIn comparison with the red curves discussed above, the curves from the blue group in figure~\\ref{fig:general}(a) feature more complicated behavior and, in particular, can intersect each other (and also intersect the curves from the next, third group of green curves discussed below). The intersections between the blue curves occur for the gain-and-loss amplitudes in the interval $11.561 \\lessapprox   \\gamma <  9\\pi^2/2\\approx 19.739$, where $\\sin\\sqrt{2\\gamma}$ is negative.  \\textcolor{black}{At first glance,} this might seem to contradict to  the  expansion (\\ref{3.4}) of Lemma~\\ref{lm3.4},  which  suggests that in this case the multitude of complex zeros   accumulate in the upper complex half-plane with the growth of $\\ell$. However, this apparent contradiction is resolved if we trace the behavior of the complex roots more closely.   Indeed, choosing for an example $\\gamma =16$ [see the vertical axis tick in  figure~\\ref{fig:general}(a)] and computing several  complex roots under the increase of $\\ell$, we observe that each considered root first goes down from the upper half-plane to the lower one but then again returns to the upper half-plane, see Figure~\\ref{fig:g=16}(b) and (b$_1$). Thus,  in  this  interval of the gain-and-loss amplitudes the increase of $\\ell$  results  either in the transformation from the resonance to the eigenvalue or to the opposite process,  i.e. to the disappearance of the complex-conjugate pair.  Respectively, the intersection between the two blue curves corresponds to  two coexisting spectral singularities, i.e. to the moment  when one complex-conjugate pair of eigenvalues disappears and another pair (with different $k$) emerges. For sufficiently large $\\ell$ the imaginary part  of each   considered root  remains positive and approaches zero, in accordance with expansion (\\ref{3.4}). Thus, in this interval of the gain-and-loss strengths, the limit $\\ell\\to\\infty$ the spectrum contains only a finite number of complex-conjugate eigenvalues, which correspond to complex zeros $k$ whose behavior is not covered   by expansion  (\\ref{3.4}).\n\n \\begin{figure}\n\t\\centering\n\t\\includegraphics[width=0.99\\columnwidth]{fig08.eps}\n\t\\caption{(a,b) Real and imaginary parts for three   complex zeros of function $F$ for fixed $\\gamma=16$ and increasing $\\ell$. Panel b$_1$ is the magnification of some region of (b) and  shows more clearly that imaginary parts of all   three shown eigenvalues are positive for all sufficiently large  $\\ell$. \\textcolor{black}{For each three shown eigenvalues, its real and \timaginary parts are of the same colour in both panels.}}\n\t\\label{fig:g=16}\n\\end{figure}\n\nThe third  group of spectral singularities [green curves in figure~\\ref{fig:general}(a)] was obtained by means of the continuation from  the solution $\\gamma_\\star^{(2)}$ and $k_\\star^{(2)}$ in Table~\\ref{tbl:1}. In view of the very complicated structure of the overall resulting picture, we only show a section of these curves corresponding to relatively small   values of the gain-and-loss amplitudes $\\gamma$. Quite interestingly, there is no gap between the blue and green groups of curves, which results in the multitude of intersections between blue and green curves, \\textcolor{black}{see Fig.~\\ref{fig:general}(a) and the magnified view in Fig.~\\ref{fig:general}(c)}. These   intersections suggest a possibility of   simultaneous emergence of two complex-conjugate pairs of eigenvalues from two different interior points of the continuous spectra.\n\nConsidering further solutions $\\gamma_\\star^{(n)}$, $k_\\star^{(n)}$, $n=3,4,\\ldots$, in the limit $\\ell=0$ one can construct new groups of spectral singularities  with larger values of $\\gamma$, which are not shown in Figure~\\ref{fig:general}.\n\n\n5.5 General picture of spectral singularities\n\\subsection{General picture of spectral singularities}\n\\label{sec:general}\n\n\\begin{figure}\n\t\\centering\n\t\\includegraphics[width=0.99\\columnwidth]{fig06.eps}\n\t\\caption{(a) Values of the gain-and-loss amplitude $\\gamma$ and distance  $\\ell$, for which   spectral singularities occur. (b) Corresponding  wavenumbers   $k$. \\textcolor{black}{Panel (c) magnifies   the region  $(\\ell, \\gamma)\\in [2,4]\\times[10,50]$ from   (a), and panel  (d) is the  magnification of corresponding curves from panel (b).}}\n\t\\label{fig:general}\n\\end{figure}\n\nLet us now turn to description of the general picture of spectral singularities. In order to construct systematically different solutions, we again start from the limit $\\ell=0$, where the values of $\\gamma$ and $k$ corresponding to spectral singularities are known, see Table~\\ref{tbl:1}. Then we use the periodicity of function $e^{-4ik\\ell}$ in $\\ell$ in order to construct new branches of solutions having no counterparts in the limit $\\ell=0$, cf. equation (\\ref{eq:periodic}). This procedure results in a fairly complicated picture containing a multitude of spectral singularities, some part of which (corresponding to  relatively  small   values of the gain-and-loss) is shown in figure~\\ref{fig:general}(a,b) as the curves on the plane $\\gamma$ {\\it vs} $\\ell$ and $k$ {\\it vs} $\\ell$.\n\n\nLet us describe the structure of the found solutions using the  diagram $\\gamma$ {\\it vs} $\\ell$ in figure~\\ref{fig:general}(a). The multitude of curves shown in this plot can be divided into three groups (plotted with red, blue and green curves) which have been obtained by means of the continuation from three different solutions in the limit $\\ell=0$. There is a well-visible vertical gap between red and blue curves, which corresponds to the ``forbidden'' values of the gain-and-loss amplitudes $\\gamma$ found above in (\\ref{eq:gap}). At the same time,  there is no gap between blue and green curves, which  results in a multitude of intersections between these curves.\nThe first group of spectral singularities,  corresponding to red curves in Figure~\\ref{fig:general}(a),   is obtained through  the continuation from the  spectral singularity  in the limit $\\ell=0$ with the smallest gain-and-loss amplitude, i.e.  from values $\\gamma_\\star^{(0)}$ and $k_\\star^{(0)}$ in Table~\\ref{tbl:1}.  The leftmost (bold)  curve in this group which originates in the limit $\\ell=0$  is the $\\PT$-symmetry breaking threshold which was already shown in figure~\\ref{fig:threshold}(a). For values of $\\gamma$ above this curve, there is always one or more (but finitely many, see Section~\\ref{sec:zeros})   complex-conjugate pairs of eigenvalues in the spectrum.  Several curves situated to the right from the bold red curve are obtained using the fact that if $\\gamma$ and $k$ are solutions in the limit $\\ell=0$, then the same values of $\\gamma$ and $k$ also correspond to a spectral singularity with $\\ell_n = (n\\pi)/(2k)$, $n=1,2,\\ldots$. Once a single solution with a new distance   $\\ell_n$ is obtained, a new branch of solutions can be constructed using numerical continuation in $\\gamma$ or in $\\ell$. In the limit $\\ell \\to \\infty$ all the red curves in figure~\\ref{fig:general}(a) approach  the asymptotic value $\\gamma=0$ or at $\\gamma_*= \\pi^2/2\\approx 4.935$. Notice that, except for the bold line demarcating the $\\PT$-symmetry breaking threshold, none of the red curves can be continued to the limit $\\ell\\to 0$.\n\nThe multitude of red curves  in figure~\\ref{fig:general}(a) demonstrates explicitly how new spectral singularities emerge with the increase of $\\ell$. Indeed, drawing an imaginary horizontal line, say, at $\\gamma=3$ [see the vertical axis  tick  in figure~\\ref{fig:general}(a)], we observe that with the increase of $\\ell$ this line   intersects more and more red curves. Each intersection corresponds to the values of $\\gamma$ and $\\ell$ at which   some  root $k$   crosses the real axis and goes down from the upper to lower complex half-plane. Thus,  a finite-width resonance transforms  to a complex eigenvalue through the spectral singularity (we recall again that  in view of   $\\PT$ symmetry  any root $k\\ne 0$ crosses the real line simultaneously with its counterpart $-\\bar{k}$, i.e.  the corresponding   spectral singularity is self-dual).\nIn order to illustrate this process, in Figure~\\ref{fig:g=3}  we show the evolution of three numerically found complex zeros of function $F(k, \\gamma, \\ell)$  under   the increase of $\\ell$.  In this figure the imaginary part of each complex zero changes sign from positive to negative and then asymptotically approaches zero (remaining negative). In the limit of large $\\ell$,  this behavior agrees  with expansion (\\ref{3.4}) of Lemma~\\ref{lm3.4} , where, for the chosen value of $\\gamma$,   we have $\\sin\\sqrt{2\\gamma}>0$.   Thus the growing distance   $\\ell$ results in a sequence of self-dual spectral singularities and in the increasing number of complex-conjugate eigenvalues in the spectrum.\n\n \\begin{figure}\n \t\\centering\n \t\\includegraphics[width=0.8\\columnwidth]{fig07.eps}\n \t\\caption{(a,b) Real and imaginary parts of three complex zeros of function $F$ for fixed $\\gamma=3$ and increasing $\\ell$. For each curve, the imaginary part is positive for small $\\ell$ and becomes negative for all sufficiently large $\\ell$. \\textcolor{black}{For each three shown eigenvalues, its real and \timaginary parts are of the same colour in both panels.}}\n \t\\label{fig:g=3}\n \\end{figure}\n\n\nThe second group of spectral singularities [blue curves in figure~\\ref{fig:general}(a)] was obtained by means of the continuation from the  next solution in the limit $\\ell=0$, i.e.  from $\\gamma_\\star^{(1)}$ and $k_\\star^{(1)}$ in Table~\\ref{tbl:1}. Again, one of the curves [the leftmost bold curve in figure~\\ref{fig:general}(a)] was obtained through the direct continuation from the limit $\\ell=0$, while other blue curves were generated  using the periodicity of function $F(k, \\gamma, \\ell)$  in $\\ell$ and cannot be continued to the limit $\\ell\\to0$. In the limit $\\ell\\to\\infty$ the blue curves approach the horizontal asymptotes  $\\gamma=2\\pi^2\\approx19.739$ and $\\gamma=9\\pi^2/2\\approx 44.413$. In  the $(\\gamma, \\ell)$-plane  the  gain-and-loss amplitudes corresponding to the   group of blue curves are well separated from  those for the red curves: indeed, all red curves are bounded from above by the asymptote $\\gamma= \\pi^2/2\\approx 4.935$, while all blue curves are bounded from below by $\\gamma_*\\approx 11.561$, see (\\ref{eq:gap}). Thus, the  emergence of new spectral singularities with the increase of $\\ell$ is sensitive to the value of the gain-and-loss amplitude and does not occur for     the gain-and-loss amplitudes lying in the  gap  between the red and blue curves.\n\nIn comparison with the red curves discussed above, the curves from the blue group in figure~\\ref{fig:general}(a) feature more complicated behavior and, in particular, can intersect each other (and also intersect the curves from the next, third group of green curves discussed below). The intersections between the blue curves occur for the gain-and-loss amplitudes in the interval $11.561 \\lessapprox   \\gamma <  9\\pi^2/2\\approx 19.739$, where $\\sin\\sqrt{2\\gamma}$ is negative.  \\textcolor{black}{At first glance,} this might seem to contradict to  the  expansion (\\ref{3.4}) of Lemma~\\ref{lm3.4},  which  suggests that in this case the multitude of complex zeros   accumulate in the upper complex half-plane with the growth of $\\ell$. However, this apparent contradiction is resolved if we trace the behavior of the complex roots more closely.   Indeed, choosing for an example $\\gamma =16$ [see the vertical axis tick in  figure~\\ref{fig:general}(a)] and computing several  complex roots under the increase of $\\ell$, we observe that each considered root first goes down from the upper half-plane to the lower one but then again returns to the upper half-plane, see Figure~\\ref{fig:g=16}(b) and (b$_1$). Thus,  in  this  interval of the gain-and-loss amplitudes the increase of $\\ell$  results  either in the transformation from the resonance to the eigenvalue or to the opposite process,  i.e. to the disappearance of the complex-conjugate pair.  Respectively, the intersection between the two blue curves corresponds to  two coexisting spectral singularities, i.e. to the moment  when one complex-conjugate pair of eigenvalues disappears and another pair (with different $k$) emerges. For sufficiently large $\\ell$ the imaginary part  of each   considered root  remains positive and approaches zero, in accordance with expansion (\\ref{3.4}). Thus, in this interval of the gain-and-loss strengths, the limit $\\ell\\to\\infty$ the spectrum contains only a finite number of complex-conjugate eigenvalues, which correspond to complex zeros $k$ whose behavior is not covered   by expansion  (\\ref{3.4}).\n\n \\begin{figure}\n\t\\centering\n\t\\includegraphics[width=0.99\\columnwidth]{fig08.eps}\n\t\\caption{(a,b) Real and imaginary parts for three   complex zeros of function $F$ for fixed $\\gamma=16$ and increasing $\\ell$. Panel b$_1$ is the magnification of some region of (b) and  shows more clearly that imaginary parts of all   three shown eigenvalues are positive for all sufficiently large  $\\ell$. \\textcolor{black}{For each three shown eigenvalues, its real and \timaginary parts are of the same colour in both panels.}}\n\t\\label{fig:g=16}\n\\end{figure}\n\nThe third  group of spectral singularities [green curves in figure~\\ref{fig:general}(a)] was obtained by means of the continuation from  the solution $\\gamma_\\star^{(2)}$ and $k_\\star^{(2)}$ in Table~\\ref{tbl:1}. In view of the very complicated structure of the overall resulting picture, we only show a section of these curves corresponding to relatively small   values of the gain-and-loss amplitudes $\\gamma$. Quite interestingly, there is no gap between the blue and green groups of curves, which results in the multitude of intersections between blue and green curves, \\textcolor{black}{see Fig.~\\ref{fig:general}(a) and the magnified view in Fig.~\\ref{fig:general}(c)}. These   intersections suggest a possibility of   simultaneous emergence of two complex-conjugate pairs of eigenvalues from two different interior points of the continuous spectra.\n\nConsidering further solutions $\\gamma_\\star^{(n)}$, $k_\\star^{(n)}$, $n=3,4,\\ldots$, in the limit $\\ell=0$ one can construct new groups of spectral singularities  with larger values of $\\gamma$, which are not shown in Figure~\\ref{fig:general}.\n\n\n",
        "label": "fig:general",
        "Descriptive_question1": "What is the range of the gain-and-loss amplitude γ shown in panel (a) of figure_6?",
        "Descriptive_question2": "What is the range of the distance ℓ in panel (c) of figure_6?",
        "Reasoning_question1": "What does the vertical gap between the red and blue curves in panel (a) of figure_6 indicate about the occurrence of spectral singularities?",
        "Reasoning_question2": "How does the magnification in panel (c) of figure_6 help in understanding the relationship between γ and ℓ compared to panel (a)?",
        "Descriptive_answer1": "0 to 50",
        "Descriptive_answer2": "2 to 4",
        "Reasoning_answer1": "The vertical gap between the red and blue curves indicates a 'forbidden' range of the gain-and-loss amplitude γ in which spectral singularities do not occur. This gap corresponds to the interval \\( \\frac{\\pi^2}{2} < \\gamma < \\gamma_* \\approx 11.561 \\) found analytically as values where equation (4.4) has no roots. Hence, spectral singularities cannot occur for gain-and-loss amplitudes in this range, separating the two groups of spectral singularities (red and blue) in the \\(\\gamma\\) vs \\(\\ell\\) plane.",
        "Reasoning_answer2": "Panel (c) provides a magnified view of the region \\((\\ell, \\gamma) \\in [2,4] \\times [10,50]\\), allowing better visualization of the detailed structure and interactions among spectral singularity curves within this domain. While panel (a) shows an overall wide range with many curves, panel (c) zooms in to clarify the complex behavior, such as intersections between blue and green curves and cumulative effects, facilitating a more detailed understanding of how spectral singularities depend sensitively on both \\(\\gamma\\) and \\(\\ell\\) in this narrower parameter range."
    },
    {
        "paper_id": "1908.06383.json",
        "image_id": "figure_7",
        "image_path": [
            "/home/yz979/scratch/chengye/scimolmo/test-images-tables/dataset_imagestables/1908.06383/images/fig07.eps"
        ],
        "caption": "(a,b) Real and imaginary parts of three complex zeros of function $F$ for fixed $\\gamma=3$ and increasing $\\ell$. For each curve, the imaginary part is positive for small $\\ell$ and becomes negative for all sufficiently large $\\ell$. \\textcolor{black}{For each three shown eigenvalues, its real and \timaginary parts are of the same colour in both panels.}",
        "classify": "Chart",
        "section_info": "5 Spectral singularities\n\\section{Spectral singularities}\n\\label{sec:ss}\n\\subsection{General analytical expressions}\n\nIn this section we study real zeroes of the function $F$ corresponding to spectral singularities, i.e. to  zero-width resonances.\nFor such zeroes, equation (\\ref{2.10}) is a pair of two real equations for one real variable $k$ and two parameters $\\ell$ and $\\g$.\nThanks to the symmetry of the zeroes with respect to the imaginary axis, it is sufficient to find only positive real resonances since the negative ones are located symmetrically with respect to the origin. Similar to the proof of Lemma~\\ref{lm3.6}, for real positive $k$  we make  change (\\ref{4.1})\nand rewrite equation (\\ref{2.10}) in the following form:\n\\begin{align*}\n(1-2u^{-4})\\cos\\b u^{-1}&+(2u^4-1)\\cosh\\b u + 2i \\sqrt{1-u^4}\\left(u^2\\sinh\\b u - u^{-4}\\sin\\b u^{-1}\\right)\n\\\\\n&-e^{-2i\\b\\ell\\sqrt{u^{-2}-u^2}} \\left(\\cosh(\\b u)-\\cos\\b u^{-1}\\right)=0,\\qquad \\b=\\sqrt{2\\g}.\n\\end{align*}\nTaking the real and imaginary part of this equation and multiplying the equation by $u^4$, we obtain:\n\\begin{equation}\\label{4.3}\n\\begin{aligned}\n&(u^4-2)\\cos\\b u^{-1}+u^4(2u^4-1)\\cosh\\b u\n=u^4\\cos\\left(2\\b\\ell\\sqrt{u^{-2}-u^2}\\right)\\left(\\cosh \\b u-\\cos\\b u^{-1}\\right),\n\\\\\n&2\\sqrt{1-u^4}\\Big(u^6\\sinh\\b u-\\sin\\b u^{-1}\\big)\n=-u^4\\sin\\left(2\\b\\ell\\sqrt{u^{-2}-u^2}\\right)\\left(\\cosh \\b u-\\cos\\b u^{-1}\\right).\n\\end{aligned}\n\\end{equation}\nThis is a system of two real equations with three real variables. If we are given $(\\b,\\ell)$ and we try to find $u$, the system is overdetermined and does not necessary have a root. In other words, it is solvable with respect to $u$ only if $(\\b,\\ell)$ are located on some (solvability) curves. In order to avoid working with an overdetermined system, in what follows we regard (\\ref{4.3}) as a system for two unknown variable with one parameter.\n\nTo find the curves in $(\\b,\\ell)$ plane, on which equations (\\ref{4.3}) are solvable with respect to $u$, we shall regard $u$ as a parameter and $(\\b,\\ell)$ as unknown variables. We take the sum of squares of equations (\\ref{4.3}) and divide the result by $(1-u^4)$. We also divide equations (\\ref{4.3}). This leads us to a   pair  of equations:\n\\begin{align}\\label{4.4}\n&u^4(1-u^4) \\cosh\\b u\\cos\\b u^{-1}-2u^6 \\sinh\\b u\\sin\\b u^{-1} +1-u^{12}=0,\n\\\\\n&\\frac{2\\sqrt{1-u^4}(u^6\\sinh\\b u-\\sin\\b u^{-1})}{(2-u^4)\\cos\\b u^{-1}+u^4(1-2u^4)\\cosh\\b u}=\\tan \\Big( 2\\b\\ell\\sqrt{u^{-2}-u^2}\\Big).\\nonumber\n\\end{align}\nThe second equation can be solved explicitly with respect to $\\ell$:\n\\begin{equation}\\label{4.10}\n\\ell=\\frac{1}{2\\b\\sqrt{u^{-2}-u^2}} \\left(\\arctan \\frac{2\\sqrt{1-u^4}(u^6\\sinh\\b u-\\sin\\b u^{-1})}{(2-u^4)\\cos\\b u^{-1}+u^4(1-2u^4)\\cosh\\b u}+\\pi n\\right),\n\\end{equation}\nwhere $n\\in\\mathds{N}$ is an arbitrary natural number. As the next lemma states, to make equations (\\ref{4.4}), (\\ref{4.10}) equivalent to (\\ref{4.3}), we should also assume that\n\\begin{equation}\\label{4.5}\n(-1)^n=\\sign \\big((u^4-2)\\cos\\b u^{-1}+u^4(2u^4-1)\\cosh\\b u\\big).\n\\end{equation}\n\n\\begin{lemma}\\label{lm4.1}\nEquations (\\ref{4.3}) are equivalent to (\\ref{4.4}), (\\ref{4.10}), (\\ref{4.5}).\n\\end{lemma}\n\n\\begin{proof}\nWe rewrite shortly equations (\\ref{4.3}) as\n$A_1=B\\cos\\a$, $ A_2=-B\\sin\\a$,\nwhere $A_1$, $A_2$ are the left hand sides in (\\ref{4.3}), $\\a=2\\b\\ell\\sqrt{u^{-2}-u^2}$ and $B=u^4(\\cosh\\b u-\\cos\\b u^{-1})$. Then equations (\\ref{4.4}), (\\ref{4.10}) become\n$A_1^2+A_2^2=B^2$, $\\a=-\\arctan\\frac{A_2}{A_1}+\\pi n$.\nWe have:\n\\begin{equation*}\nB\\cos \\a=(-1)^n\\cos\\arctan\\frac{A_2}{A_1}=\\frac{(-1)^n}{\\sqrt{1+\\frac{A_2^2}{A_1^2}}} = \\frac{(-1)^n|A_1|}{\\sqrt{A_1^2+A_2^2}}\n\\end{equation*}\nand we get the first equation $A_1=B\\cos\\a$  provided condition (\\ref{4.5}) is satisfied. In the same way we check that the latter condition also ensures the second equation $A_2=-B\\sin\\a$.\n\\end{proof}\n\n\nEquation (\\ref{4.4}) is transcendental, and we can not solve it analytically.\nNevertheless, for each $u\\in(0,1)$, this is an equation only for a single variable $\\b$, not for two as equations (\\ref{4.3}). So, we propose the following algorithm of recovering the aforementioned solvability curves: choose $u\\in(0,1)$, then solve equation (\\ref{4.4}) and recover the sequence of  distances  $\\ell$ by formula (\\ref{4.10}) with different integer $n$. Then the gain-and-loss amplitude $\\gamma$ and the corresponding wavenumber $k$ can be readily recovered from $\\beta$ and  $u$.  In a similar way, one can first fix some value of $\\beta$ (i.e., fix the gain-and-loss strength) and then   solve equation (\\ref{4.4}) with respect to $u$ and   recover   $\\ell$ by (\\ref{4.10}). Equation (\\ref{4.4})  is well-behaved, and for each $\\beta$  all its zeros  $u$ can be easily found numerically.\n\nAlternatively, as explained below  in Section~\\ref{sec:general}, the values  corresponding to  spectral singularities can be found systematically by means of the numerical continuation from the limit $\\ell=0$. However, in this case   equation (\\ref{4.4}) is still useful because it allows one to check that all spectral singularities have been found for the given value of the gain-and-loss $\\gamma$.\n\n\n\\subsection{\nAbsence of   spectral singularities}\n\\label{sec:gap}\n\nFor $u=0$ and $u=1$, the left-hand-side of equation (\\ref{4.4}) is equal respectively to $1$ and $-2\\sinh\\beta\\sin\\beta$. Then a sufficient condition for the existence of a spectral singularity at the given gain-and-loss amplitude $\\gamma$ is $\\sin\\beta=\\sin\\sqrt{2\\gamma}>0$. At the same time, it is also possible to establish sufficient conditions that forbid the existence of   spectral singularities in a certain interval of parameters. In this subsection we prove the existence of two ``forbidden gaps'' for the roots of equation (\\ref{4.4}). The first one exists for all $\\b\\geqslant 0$ and it states that there is no roots in certain interval. The second gap is a certain interval of values of $\\b$, for which equation (\\ref{4.4}) has no zeroes at all.\n\nFor the convenience, by $g(u,\\b)$ we denote the left hand side of equation (\\ref{4.4}). The first ``forbidden gap'' is described in the following lemma.\n\n\\begin{lemma}\\label{lm5.1}\nFor all $\\b\\geqslant 0$, equation (\\ref{4.4}) has no roots in the interval $\\big[0,(1+\\tfrac{\\b}{4})^{-1}\\big)$.\n\\end{lemma}\n\\begin{proof}\nEmploying a standard inequality $a\\cos\\a+b\\sin\\a\\leqslant \\sqrt{a^2+b^2}$, we estimate the first two term in equation (\\ref{4.4}) as\n\\begin{equation*}\nu^4(1-u^4) \\cosh\\b u\\cos\\b u^{-1}-2u^6 \\sinh\\b u\\sin\\b u^{-1} \\leqslant \\sqrt{u^8(1-u^4)^2\\cosh^2 \\b u+4u^{12}\\sinh^2\\b u}.\n\\end{equation*}\nHence, equation (\\ref{4.4}) surely has no roots for values of $u$ satisfying\n\\begin{equation*}\n \\sqrt{u^8(1-u^4)^2\\cosh^2 \\b u+4u^{12}\\sinh^2\\b u} <1-u^{12}.\n\\end{equation*}\nExpressing $\\cosh^2 \\b u$ via $\\sinh^2 \\b u$ and simplifying this inequality, we obtain $ u^4(1+u^4)\\cosh \\b u <1+u^{12}$ and hence,\n\\begin{equation}\\label{3.37}\n\\cosh \\b u-1 <\\frac{1-2u^4+u^8}{u^4},\\qquad \\sqrt{2}\\sinh \\b u\n<u^{-2}-u^2.\n\\end{equation}\nThanks to the estimate $\\sqrt{2}\\sinh 2\\ln u^{-1}\\leqslant u^{-2}-u^2$, $u\\in(0,1]$, the last inequality in (\\ref{3.37}) holds once $\\frac{\\b}{4}<u^{-1}\\ln u^{-1}$. It is easy to confirm that this inequality is true once $u^{-1}>1+\\tfrac{\\b}{4}$.\nThe proof is complete.\n\\end{proof}\n\n\nThe next lemma is   auxiliary  and will be employed in studying the second forbidden zone.\n\n\\begin{lemma}\\label{lm5.2}\nThe function $g(u,\\pi)$ is positive on $[0,1)$.\n\\end{lemma}\n\n\\begin{proof}\nWe have $g(0,\\pi)=1$ and by Lemma~\\ref{lm5.1}, it is positive for $u<(1+\\tfrac{\\pi}{4})^{-1}$. This is why in what follows we consider only the values $u\\geqslant (1+\\tfrac{\\pi}{4})^{-1}$. For such values of $u$ we have\n$\\pi\\leqslant \\pi u^{-1}\\leqslant \\pi (1+\\tfrac{\\pi}{4})<1.79\\pi$.\nAs\n$\\tfrac{3\\pi}{2} \\leqslant \\pi u^{-1}\\leqslant \\pi (1+\\tfrac{\\pi}{4})$,\nthe function $\\sin\\b u^{-1}$ is negative, while $\\cos \\b u^{-1}$ is positive. Hence, for such values of $u$, the function $g(u,\\pi)$ is positive. It remains to consider the values $\\frac{2}{3}<u\\leqslant 1$.\n\nFor such values of $u$ we first observe the following simple estimates:\n\\begin{equation*}\n\\frac{1-u^4}{1-u}\\cos\\pi u^{-1}\\geqslant -4,\\qquad -\\frac{\\sin \\pi u^{-1}}{1-u}=\\frac{\\sin \\pi (u^{-1}-1)}{1-u}\\geqslant \\pi u.\n\\end{equation*}\nThen the function $g$ satisfies the estimate:\n\\begin{equation}\\label{3.12}\n\\frac{g(u,\\pi)}{u^4(1-u)}\\geqslant g_1(u)+g_2(u),\\qquad g_1(u):=-4\\cosh\\pi u +2\\pi u^3 \\sinh \\pi u,\\qquad g_2(u):=\\frac{1-u^{12}}{u^4(1-u)}.\n\\end{equation}\nThe function $g_1(u)$ is monotonically increasing in $u\\in[\\tfrac{2}{3},1]$ since\n\\begin{equation*}\ng_1'(u)=2\\pi^2 u^3\\cosh\\pi u+(6\\pi u^2-4\\pi)\\sinh\\pi u>2\\pi(\\pi u^3+6u^2-2)\\sinh\\pi u>2\\pi\\sinh\\pi u>0.\n\\end{equation*}\nHence,\n$g_1(u)\\geqslant g_1\\left(\\frac{2}{3}\\right)>-9.05$.\nFor the function $g_2$ we have the following representation and estimate:\n\\begin{equation*}\ng_2(u)=1+\\sum\\limits_{j=1}^{4}(u^j+u^{-j})+\\sum\\limits_{j=5}^{8} u^j\\geqslant 9 +\\sum\\limits_{j=5}^{8}\\left(\\frac{2}{3}\\right)^j>9.31.\n\\end{equation*}\nTwo last estimates and (\\ref{3.12}) imply the positivity of the function $g$ for $u\\in[\\tfrac{2}{3},1)$.\n\\end{proof}\n\n\nDenote\n\\begin{align*}\ng_*(u,\\b):=&\\b u^3(1-3u^4)\\cosh\\b u \\sin \\b u^{-1}+\\b u^5 (3-u^4)\\sinh\\b u \\cos \\b u^{-1}\n\\\\\n&-2 u^4(1+u^4) \\cosh\\b u \\cos \\b u^{-1} -6(1+u^{12}).\n\\end{align*}\n\n\nThe next lemma states the existence of the second forbidden zone.\n\n\\begin{lemma}\\label{lm5.3}\nEquation (\\ref{4.4}) has no roots as $\\pi<\\b<\\b_*<5$, where $(u_*, \\b_*)$ is  the root of the system of the equations\n\\begin{equation}\\label{3.13}\ng(u,\\b)=0,\\qquad g_*(u,\\b)=0,\\qquad u\\in[0,1],\\qquad \\pi<\\b<5,\n\\end{equation}\nwith minimal possible $\\b$.\nTheir approximate values are\n\\begin{equation}\\label{3.14}\n\\b_*=4.808438,\\qquad u_*=0.611772.\n\\end{equation}\n\\end{lemma}\n\n\\begin{proof}\nThe function $g(u,\\pi)$ is positive on $[0,1)$ and $g(1,\\pi)=0$, see Figure~\\ref{fig:forbidden}a. As $\\pi<\\b<2\\pi$, we have $g(0,\\b)=1>0$ and $g(1,\\b)=-2\\sin\\b\\sinh\\b>0$. Hence, for $\\b$ close enough to $\\pi$, the function $g(u,\\b)$ is positive for all $u\\in[0,1]$. At the same time, we have $g(0.65,5)<-0.617<0$ and therefore, for $\\b=5$, equation (\\ref{4.4}) possesses at least two roots, one in $(0,0.65)$ and another in $(0.65,1)$. cf. Figure~\\ref{fig:forbidden}c. We also observe that the function $g$ is jointly continuous in $(u,\\b)$. The above facts means that as $\\b$ grows from $\\pi$ to $5$, at some value $\\b=\\b_*$, the graph of the function $g$ is still located in the upper half-plane but touches the $u$-axis at some point $u=u_*$,\nsee Figure~\\ref{fig:forbidden}b. The function $g(u,\\b)$ is positive as $\\pi<\\b<\\b_*$ and $u\\in[0,1]$. Then the point $u=u_*$ is obviously the global minimum of $g$ and hence, $(u_*,\\b_*)$ is a solution to the system of equations $g(u,\\b)=0$, $\\frac{\\p g}{\\p u}(u,\\b)=0$. It is easy to check that $g_*=\\frac{\\p g}{\\p u}-6 g$ and hence, $(u_*,\\b_*)$ solves system (\\ref{3.13}). These roots can be found numerically and this gives (\\ref{3.14}). The proof is complete.\n\\end{proof}\n\n\n\\begin{figure}\n\t\\centering\n\\includegraphics[width=0.99\\columnwidth]{fig03.eps}\n\\caption{Illustration for proof of Lemma~\\ref{lm5.3}. Graphs of the function $g(u, \\beta)$ for $\\b=\\pi$ (a), $\\b=\\b_*\\approx  4.808438$ (b), and $\\b=5$ (c). Notice broken vertical axes in (b) and (c). }\n\\label{fig:forbidden}\n\\end{figure}\n\nReturning from the auxiliary variable $\\beta$ to the gain-and-loss amplitude $\\gamma=\\beta^2/2$, from   Lemma~\\ref{lm5.3}  we deduce  the following important result:\n\\begin{equation}\n\\label{eq:gap}\n\\textrm{there is no spectral singularities for\\quad }  \\frac{\\pi^2}{2} < \\gamma< \\gamma_*\\approx  11.561.\n\\end{equation}\n\n\n\n\\subsection{Creating a  spectral singularity  at a given wavenumber}\n\n\\begin{table}\n\t\\centering\n\t\\begin{tabular}{c|cccc}\n\t\t$n$ & 0 & 1& 2&\\\\\\hline\n\t\t$\\gamma_\\star^{(n)}$ & 2.071&13.307&27.783 &\\\\[2mm]\n\t\t$k_\\star^{(n)}$ & 1.065&4.318 &7.529 &\n\t\\end{tabular}\n\t\\caption{Approximate values of gain-and-loss amplitudes $\\gamma=\\gamma_\\star^{(n)}$  and wavenumbers   $k=k_\\star^{(n)}$, $n=0, 1, \\ldots$,  corresponding to spectral singularities with lowest $\\gamma$ in the limit $\\ell=0$, see Table~I in  \\cite{Mostafazadeh2009}. \\label{tbl:1}}\n\\end{table}\n\n\nFor $\\ell=0$ spectral singularity can only be obtained for some isolated values of the wavenumber $k$ and the gain-and-loss amplitude $\\gamma$ \\cite{Mostafazadeh2009}. Several lowest values of $\\gamma$ corresponding to the spectral singularities  and the associated wavenumbers  $k$ are listed in Table~\\ref{tbl:1}.  An  important advantage of the more general system with nonzero gain-to-loss disctance    $\\ell>0$ consists  in the possibility to create a spectral   singularity   at any wavenumber $k$ given beforehand. Indeed, let us  return back to equations (\\ref{4.4}), (\\ref{4.10}) and    discuss the following   issue: given a point $k$ on the real axis, how to choose $\\b$ and $\\ell$ to have a  resonance at this point? Equations (\\ref{4.4})--(\\ref{4.10}) allow us to answer  easily this question.\n\nWe fix $k>0$ and we find the associated value of $u$ by resolving (\\ref{4.1}):\n\\begin{equation}\\label{4.12}\nu^{-2}-u^2=4k^2\\b^{-2},\\qquad\n\nu=\\b R^{-1}, \\quad \\mbox{where\\ } R = \\sqrt{2k^2+\\sqrt{4k^4+\\b^4}}.\n\\end{equation}\nWe divide equation (\\ref{4.4}) by $u^6$ and substitute then the above formulae and\n\\begin{equation*}\n\\frac{1-u^{12}}{u^6}=\\frac{1-u^4}{u^2}\\frac{1+u^4+u^8}{u^4}=(u^{-2}-u^2)\\big((u^{-2}-u^2)^2+3\n\\big).\n\\end{equation*}\nThis gives the equation:\n\\begin{equation}\\label{4.11}\n\\begin{aligned}\n2\\b^4 k^2&\\cosh(\\b^2R^{-1})  \\cos R\n\n\n - \\b^6 \\sinh(\\b^2R^{-1})  \\sin R\n  +2k^2(16k^4+3\\b^4)=0.\n\\end{aligned}\n\\end{equation}\nAn algorithm for creating a resonance at a prescribed point  $k$ is as follows. Given $k>0$, we first solve equation (\\ref{4.11}) with respect to $\\b$ and we also find $u$ by (\\ref{4.12}). Then needed values of  $\\ell$ are determined by (\\ref{4.10}), (\\ref{4.5}).\n\n\nIn order to illustrate this  algorithm,  we us consider a finite interval of wavenumbers $k\\in (0, k_1]$, where  we set $k_1 = 10$ for the numerics reported on in what follows. We scan   the chosen interval  with a sufficiently small step ($\\Delta k=0.01$) and for each value of $k$ solve equation (\\ref{4.11}) numerically  using the simple dichotomy method. While for each $k$  equation (\\ref{4.11}) might have several roots $\\beta$, in our numerical procedure we always choose the  minimal positive root, i.e., the one which allows to achieve the spectral singularity with given $k$ at the smallest possible value  of the gain-and-loss amplitude $\\gamma=\\gamma_\\textrm{min}$. Next, we choose the minimal positive distance $\\ell_\\textrm{min}$ which satisfies the conditions   (\\ref{4.10}), (\\ref{4.5}) and then we use the periodicity in $\\ell$ to generate the sequence of larger gain-to-loss distances $\\ell_n = \\ell_\\textrm{min} + n\\pi/(2k)$, $n=1,2\\ldots$ [see   (\\ref{eq:periodic})]. The resulting dependencies $\\gamma_\\textrm{min}(k)$ and $\\ell_\\textrm{min}(k)$, $\\ell_n(k)$   are shown in figure~\\ref{fig:min}. The minimal gain-and-loss amplitude $\\g_{min}(k)$  and the minimal gain-to-loss distance $\\ell_{min}$ are   discontinuous,  which means that the small variation  in  the wavenumuber $k$  might require  a significant  change either in   $\\gamma$ or in $\\ell$.  It is especially important that the values of the  distance  $\\ell$ are generically different from zero, which points out explicitly    that the new degree of freedom offered by the nonzero gain-to-loss distance    is   important for the achieving a spectral singularity  at   the given wavenumber $k$.\n\n\n\\begin{figure}\n\t\\centering\n\t\\includegraphics[width=0.8\\columnwidth]{fig04.eps}\n\t\\caption{(a) Minimal value of the gain-and-loss amplitude $\\gamma_\\textrm{min}$  which corresponds to   a spectral singularity with the given value of the wavenumber $k$. (b)  Minimal gain-to-loss distance  $\\ell_\\textrm{min}$  which corresponds to   a spectral singularity with the  $k$ and $\\gamma$ from the  left panel (bold curves) and larger distances $\\ell_n$ obtained using the periodicity in $\\ell$ (thin \\textcolor{black}{dotted} curves).}\n\t\\label{fig:min}\n\\end{figure}\n\n\n\\subsection{$\\PT$-symmetry breaking laser-antilaser threshold}\n\nA particularly important characteristics of any $\\PT$-symmetric structure is the $\\PT$ symmetry breaking threshold, i.e., the amplitude of the gain-and-loss\ncorresponding to the  ``phase transition'' from the purely real to complex spectrum.  The best studied scenario of the phase transition  \nis the collision of two real discrete  eigenvalues at an exceptional point with the subsequent splitting in a complex-conjugate pair. However, in systems with nonempty continuous spectrum,  the phase transition can also occur through   the splitting of a self-dual spectral singularity, which results in a bifurcation of a complex-conjugate pair from an interior point of the continuum \\cite{Yang17,KZ17,Konotop2019}. At the moment corresponding to the formation of the spectral singularity, the system operates in the CPA-laser regime \\cite{Longhi10}. Thus, in such a system, the $\\PT$-symmetry breaking threshold at the same time corresponds to the CPA-laser threshold.\n\nLemma~\\ref{lm3.3} guarantees that the spectrum of our system is real for sufficiently small gain-and-loss amplitudes $\\gamma$. Additionally, according to Lemma~\\ref{lm3.5}, the spectrum  does not have any real discrete eigenvalue. Hence, the $\\PT$-symmetry breaking is expected to occur through the emergence of a self-dual spectral singularity. In order to identify the $\\PT$-symmetry breaking threshold in our system, we start from the limit $\\ell=0$, there the phase transition takes place at  $\\gamma_\\star^{(0)} \\approx 2.072$, see \\cite{Mostafazadeh2009,KZ17} and Table~\\ref{tbl:1}. Thus, the spectrum with $\\ell=0$ is purely real and continuous for $\\gamma \\in [0, \\gamma_\\star]$, while the  increase of the gain-and-loss just above $\\gamma_\\star^{(0)}$ leads to the bifurcation of a complex-conjugate pair from an interior point  of the continuum. The spectral singularity forming at  $\\gamma_\\star^{(0)}$ takes place at wavenumber $k=k_\\star^{(0)} \\approx  1.065$. Respectively, the complex-conjugate pair of eigenvalues bifurcates from $\\lambda_0 =  [k_\\star^{(0)}]^2$. (Notice that the further increase of $\\gamma$ above the next threshold values listed in Table~\\ref{tbl:1} leads to the formation of new spectral singularities and, respectively, to bifurcations of new complex-conjugate pairs in the spectrum.)\n\nNext, we use the numerical continuation in $\\ell$ in order to continue the known solution at $\\ell=0$ to the domain $\\ell>0$.  The obtained dependence of the threshold value  of the gain-and-loss amplitude on distance $\\ell$ is shown in figure~\\ref{fig:threshold}(a), where we observe that the phase transition threshold decreases monotonically with the growth of $\\ell$. This means that introducing an additional space   between the gain and loss, one can decrease the $\\PT$-symmetry breaking  threshold, i.e. achieve the laser-antilaser operation at \\textcolor{black}{lower} gain-and-loss amplitudes than in a waveguide with the adjacent gain and loss.\n\n\n\\begin{figure}\n\t\\centering\n\t\\includegraphics[width=0.85\\columnwidth]{fig05.eps}\n\t\\caption{(a) $\\PT$-symmetry breaking threshold $\\gamma_\\star^{(0)}$ \\textit{vs} the distance between the gain and loss $\\ell$. The spectrum is purely real and continuous for $\\gamma\\leq \\gamma_\\star$, but acquires a pair of complex conjugate eigenvalues  as the gain-and-loss amplitude exceeds the threshold $\\gamma_\\star^{(0)}$. (b) Values of the wavevector $k_\\star^{(0)}$ corresponding to the dependence in (a).}\n\t\\label{fig:threshold}\n\\end{figure}\n\n\\subsection{General picture of spectral singularities}\n\\label{sec:general}\n\n\\begin{figure}\n\t\\centering\n\t\\includegraphics[width=0.99\\columnwidth]{fig06.eps}\n\t\\caption{(a) Values of the gain-and-loss amplitude $\\gamma$ and distance  $\\ell$, for which   spectral singularities occur. (b) Corresponding  wavenumbers   $k$. \\textcolor{black}{Panel (c) magnifies   the region  $(\\ell, \\gamma)\\in [2,4]\\times[10,50]$ from   (a), and panel  (d) is the  magnification of corresponding curves from panel (b).}}\n\t\\label{fig:general}\n\\end{figure}\n\nLet us now turn to description of the general picture of spectral singularities. In order to construct systematically different solutions, we again start from the limit $\\ell=0$, where the values of $\\gamma$ and $k$ corresponding to spectral singularities are known, see Table~\\ref{tbl:1}. Then we use the periodicity of function $e^{-4ik\\ell}$ in $\\ell$ in order to construct new branches of solutions having no counterparts in the limit $\\ell=0$, cf. equation (\\ref{eq:periodic}). This procedure results in a fairly complicated picture containing a multitude of spectral singularities, some part of which (corresponding to  relatively  small   values of the gain-and-loss) is shown in figure~\\ref{fig:general}(a,b) as the curves on the plane $\\gamma$ {\\it vs} $\\ell$ and $k$ {\\it vs} $\\ell$.\n\n\nLet us describe the structure of the found solutions using the  diagram $\\gamma$ {\\it vs} $\\ell$ in figure~\\ref{fig:general}(a). The multitude of curves shown in this plot can be divided into three groups (plotted with red, blue and green curves) which have been obtained by means of the continuation from three different solutions in the limit $\\ell=0$. There is a well-visible vertical gap between red and blue curves, which corresponds to the ``forbidden'' values of the gain-and-loss amplitudes $\\gamma$ found above in (\\ref{eq:gap}). At the same time,  there is no gap between blue and green curves, which  results in a multitude of intersections between these curves.\nThe first group of spectral singularities,  corresponding to red curves in Figure~\\ref{fig:general}(a),   is obtained through  the continuation from the  spectral singularity  in the limit $\\ell=0$ with the smallest gain-and-loss amplitude, i.e.  from values $\\gamma_\\star^{(0)}$ and $k_\\star^{(0)}$ in Table~\\ref{tbl:1}.  The leftmost (bold)  curve in this group which originates in the limit $\\ell=0$  is the $\\PT$-symmetry breaking threshold which was already shown in figure~\\ref{fig:threshold}(a). For values of $\\gamma$ above this curve, there is always one or more (but finitely many, see Section~\\ref{sec:zeros})   complex-conjugate pairs of eigenvalues in the spectrum.  Several curves situated to the right from the bold red curve are obtained using the fact that if $\\gamma$ and $k$ are solutions in the limit $\\ell=0$, then the same values of $\\gamma$ and $k$ also correspond to a spectral singularity with $\\ell_n = (n\\pi)/(2k)$, $n=1,2,\\ldots$. Once a single solution with a new distance   $\\ell_n$ is obtained, a new branch of solutions can be constructed using numerical continuation in $\\gamma$ or in $\\ell$. In the limit $\\ell \\to \\infty$ all the red curves in figure~\\ref{fig:general}(a) approach  the asymptotic value $\\gamma=0$ or at $\\gamma_*= \\pi^2/2\\approx 4.935$. Notice that, except for the bold line demarcating the $\\PT$-symmetry breaking threshold, none of the red curves can be continued to the limit $\\ell\\to 0$.\n\nThe multitude of red curves  in figure~\\ref{fig:general}(a) demonstrates explicitly how new spectral singularities emerge with the increase of $\\ell$. Indeed, drawing an imaginary horizontal line, say, at $\\gamma=3$ [see the vertical axis  tick  in figure~\\ref{fig:general}(a)], we observe that with the increase of $\\ell$ this line   intersects more and more red curves. Each intersection corresponds to the values of $\\gamma$ and $\\ell$ at which   some  root $k$   crosses the real axis and goes down from the upper to lower complex half-plane. Thus,  a finite-width resonance transforms  to a complex eigenvalue through the spectral singularity (we recall again that  in view of   $\\PT$ symmetry  any root $k\\ne 0$ crosses the real line simultaneously with its counterpart $-\\bar{k}$, i.e.  the corresponding   spectral singularity is self-dual).\nIn order to illustrate this process, in Figure~\\ref{fig:g=3}  we show the evolution of three numerically found complex zeros of function $F(k, \\gamma, \\ell)$  under   the increase of $\\ell$.  In this figure the imaginary part of each complex zero changes sign from positive to negative and then asymptotically approaches zero (remaining negative). In the limit of large $\\ell$,  this behavior agrees  with expansion (\\ref{3.4}) of Lemma~\\ref{lm3.4} , where, for the chosen value of $\\gamma$,   we have $\\sin\\sqrt{2\\gamma}>0$.   Thus the growing distance   $\\ell$ results in a sequence of self-dual spectral singularities and in the increasing number of complex-conjugate eigenvalues in the spectrum.\n\n \\begin{figure}\n \t\\centering\n \t\\includegraphics[width=0.8\\columnwidth]{fig07.eps}\n \t\\caption{(a,b) Real and imaginary parts of three complex zeros of function $F$ for fixed $\\gamma=3$ and increasing $\\ell$. For each curve, the imaginary part is positive for small $\\ell$ and becomes negative for all sufficiently large $\\ell$. \\textcolor{black}{For each three shown eigenvalues, its real and \timaginary parts are of the same colour in both panels.}}\n \t\\label{fig:g=3}\n \\end{figure}\n\n\nThe second group of spectral singularities [blue curves in figure~\\ref{fig:general}(a)] was obtained by means of the continuation from the  next solution in the limit $\\ell=0$, i.e.  from $\\gamma_\\star^{(1)}$ and $k_\\star^{(1)}$ in Table~\\ref{tbl:1}. Again, one of the curves [the leftmost bold curve in figure~\\ref{fig:general}(a)] was obtained through the direct continuation from the limit $\\ell=0$, while other blue curves were generated  using the periodicity of function $F(k, \\gamma, \\ell)$  in $\\ell$ and cannot be continued to the limit $\\ell\\to0$. In the limit $\\ell\\to\\infty$ the blue curves approach the horizontal asymptotes  $\\gamma=2\\pi^2\\approx19.739$ and $\\gamma=9\\pi^2/2\\approx 44.413$. In  the $(\\gamma, \\ell)$-plane  the  gain-and-loss amplitudes corresponding to the   group of blue curves are well separated from  those for the red curves: indeed, all red curves are bounded from above by the asymptote $\\gamma= \\pi^2/2\\approx 4.935$, while all blue curves are bounded from below by $\\gamma_*\\approx 11.561$, see (\\ref{eq:gap}). Thus, the  emergence of new spectral singularities with the increase of $\\ell$ is sensitive to the value of the gain-and-loss amplitude and does not occur for     the gain-and-loss amplitudes lying in the  gap  between the red and blue curves.\n\nIn comparison with the red curves discussed above, the curves from the blue group in figure~\\ref{fig:general}(a) feature more complicated behavior and, in particular, can intersect each other (and also intersect the curves from the next, third group of green curves discussed below). The intersections between the blue curves occur for the gain-and-loss amplitudes in the interval $11.561 \\lessapprox   \\gamma <  9\\pi^2/2\\approx 19.739$, where $\\sin\\sqrt{2\\gamma}$ is negative.  \\textcolor{black}{At first glance,} this might seem to contradict to  the  expansion (\\ref{3.4}) of Lemma~\\ref{lm3.4},  which  suggests that in this case the multitude of complex zeros   accumulate in the upper complex half-plane with the growth of $\\ell$. However, this apparent contradiction is resolved if we trace the behavior of the complex roots more closely.   Indeed, choosing for an example $\\gamma =16$ [see the vertical axis tick in  figure~\\ref{fig:general}(a)] and computing several  complex roots under the increase of $\\ell$, we observe that each considered root first goes down from the upper half-plane to the lower one but then again returns to the upper half-plane, see Figure~\\ref{fig:g=16}(b) and (b$_1$). Thus,  in  this  interval of the gain-and-loss amplitudes the increase of $\\ell$  results  either in the transformation from the resonance to the eigenvalue or to the opposite process,  i.e. to the disappearance of the complex-conjugate pair.  Respectively, the intersection between the two blue curves corresponds to  two coexisting spectral singularities, i.e. to the moment  when one complex-conjugate pair of eigenvalues disappears and another pair (with different $k$) emerges. For sufficiently large $\\ell$ the imaginary part  of each   considered root  remains positive and approaches zero, in accordance with expansion (\\ref{3.4}). Thus, in this interval of the gain-and-loss strengths, the limit $\\ell\\to\\infty$ the spectrum contains only a finite number of complex-conjugate eigenvalues, which correspond to complex zeros $k$ whose behavior is not covered   by expansion  (\\ref{3.4}).\n\n \\begin{figure}\n\t\\centering\n\t\\includegraphics[width=0.99\\columnwidth]{fig08.eps}\n\t\\caption{(a,b) Real and imaginary parts for three   complex zeros of function $F$ for fixed $\\gamma=16$ and increasing $\\ell$. Panel b$_1$ is the magnification of some region of (b) and  shows more clearly that imaginary parts of all   three shown eigenvalues are positive for all sufficiently large  $\\ell$. \\textcolor{black}{For each three shown eigenvalues, its real and \timaginary parts are of the same colour in both panels.}}\n\t\\label{fig:g=16}\n\\end{figure}\n\nThe third  group of spectral singularities [green curves in figure~\\ref{fig:general}(a)] was obtained by means of the continuation from  the solution $\\gamma_\\star^{(2)}$ and $k_\\star^{(2)}$ in Table~\\ref{tbl:1}. In view of the very complicated structure of the overall resulting picture, we only show a section of these curves corresponding to relatively small   values of the gain-and-loss amplitudes $\\gamma$. Quite interestingly, there is no gap between the blue and green groups of curves, which results in the multitude of intersections between blue and green curves, \\textcolor{black}{see Fig.~\\ref{fig:general}(a) and the magnified view in Fig.~\\ref{fig:general}(c)}. These   intersections suggest a possibility of   simultaneous emergence of two complex-conjugate pairs of eigenvalues from two different interior points of the continuous spectra.\n\nConsidering further solutions $\\gamma_\\star^{(n)}$, $k_\\star^{(n)}$, $n=3,4,\\ldots$, in the limit $\\ell=0$ one can construct new groups of spectral singularities  with larger values of $\\gamma$, which are not shown in Figure~\\ref{fig:general}.\n\n\n5.5 General picture of spectral singularities\n\\subsection{General picture of spectral singularities}\n\\label{sec:general}\n\n\\begin{figure}\n\t\\centering\n\t\\includegraphics[width=0.99\\columnwidth]{fig06.eps}\n\t\\caption{(a) Values of the gain-and-loss amplitude $\\gamma$ and distance  $\\ell$, for which   spectral singularities occur. (b) Corresponding  wavenumbers   $k$. \\textcolor{black}{Panel (c) magnifies   the region  $(\\ell, \\gamma)\\in [2,4]\\times[10,50]$ from   (a), and panel  (d) is the  magnification of corresponding curves from panel (b).}}\n\t\\label{fig:general}\n\\end{figure}\n\nLet us now turn to description of the general picture of spectral singularities. In order to construct systematically different solutions, we again start from the limit $\\ell=0$, where the values of $\\gamma$ and $k$ corresponding to spectral singularities are known, see Table~\\ref{tbl:1}. Then we use the periodicity of function $e^{-4ik\\ell}$ in $\\ell$ in order to construct new branches of solutions having no counterparts in the limit $\\ell=0$, cf. equation (\\ref{eq:periodic}). This procedure results in a fairly complicated picture containing a multitude of spectral singularities, some part of which (corresponding to  relatively  small   values of the gain-and-loss) is shown in figure~\\ref{fig:general}(a,b) as the curves on the plane $\\gamma$ {\\it vs} $\\ell$ and $k$ {\\it vs} $\\ell$.\n\n\nLet us describe the structure of the found solutions using the  diagram $\\gamma$ {\\it vs} $\\ell$ in figure~\\ref{fig:general}(a). The multitude of curves shown in this plot can be divided into three groups (plotted with red, blue and green curves) which have been obtained by means of the continuation from three different solutions in the limit $\\ell=0$. There is a well-visible vertical gap between red and blue curves, which corresponds to the ``forbidden'' values of the gain-and-loss amplitudes $\\gamma$ found above in (\\ref{eq:gap}). At the same time,  there is no gap between blue and green curves, which  results in a multitude of intersections between these curves.\nThe first group of spectral singularities,  corresponding to red curves in Figure~\\ref{fig:general}(a),   is obtained through  the continuation from the  spectral singularity  in the limit $\\ell=0$ with the smallest gain-and-loss amplitude, i.e.  from values $\\gamma_\\star^{(0)}$ and $k_\\star^{(0)}$ in Table~\\ref{tbl:1}.  The leftmost (bold)  curve in this group which originates in the limit $\\ell=0$  is the $\\PT$-symmetry breaking threshold which was already shown in figure~\\ref{fig:threshold}(a). For values of $\\gamma$ above this curve, there is always one or more (but finitely many, see Section~\\ref{sec:zeros})   complex-conjugate pairs of eigenvalues in the spectrum.  Several curves situated to the right from the bold red curve are obtained using the fact that if $\\gamma$ and $k$ are solutions in the limit $\\ell=0$, then the same values of $\\gamma$ and $k$ also correspond to a spectral singularity with $\\ell_n = (n\\pi)/(2k)$, $n=1,2,\\ldots$. Once a single solution with a new distance   $\\ell_n$ is obtained, a new branch of solutions can be constructed using numerical continuation in $\\gamma$ or in $\\ell$. In the limit $\\ell \\to \\infty$ all the red curves in figure~\\ref{fig:general}(a) approach  the asymptotic value $\\gamma=0$ or at $\\gamma_*= \\pi^2/2\\approx 4.935$. Notice that, except for the bold line demarcating the $\\PT$-symmetry breaking threshold, none of the red curves can be continued to the limit $\\ell\\to 0$.\n\nThe multitude of red curves  in figure~\\ref{fig:general}(a) demonstrates explicitly how new spectral singularities emerge with the increase of $\\ell$. Indeed, drawing an imaginary horizontal line, say, at $\\gamma=3$ [see the vertical axis  tick  in figure~\\ref{fig:general}(a)], we observe that with the increase of $\\ell$ this line   intersects more and more red curves. Each intersection corresponds to the values of $\\gamma$ and $\\ell$ at which   some  root $k$   crosses the real axis and goes down from the upper to lower complex half-plane. Thus,  a finite-width resonance transforms  to a complex eigenvalue through the spectral singularity (we recall again that  in view of   $\\PT$ symmetry  any root $k\\ne 0$ crosses the real line simultaneously with its counterpart $-\\bar{k}$, i.e.  the corresponding   spectral singularity is self-dual).\nIn order to illustrate this process, in Figure~\\ref{fig:g=3}  we show the evolution of three numerically found complex zeros of function $F(k, \\gamma, \\ell)$  under   the increase of $\\ell$.  In this figure the imaginary part of each complex zero changes sign from positive to negative and then asymptotically approaches zero (remaining negative). In the limit of large $\\ell$,  this behavior agrees  with expansion (\\ref{3.4}) of Lemma~\\ref{lm3.4} , where, for the chosen value of $\\gamma$,   we have $\\sin\\sqrt{2\\gamma}>0$.   Thus the growing distance   $\\ell$ results in a sequence of self-dual spectral singularities and in the increasing number of complex-conjugate eigenvalues in the spectrum.\n\n \\begin{figure}\n \t\\centering\n \t\\includegraphics[width=0.8\\columnwidth]{fig07.eps}\n \t\\caption{(a,b) Real and imaginary parts of three complex zeros of function $F$ for fixed $\\gamma=3$ and increasing $\\ell$. For each curve, the imaginary part is positive for small $\\ell$ and becomes negative for all sufficiently large $\\ell$. \\textcolor{black}{For each three shown eigenvalues, its real and \timaginary parts are of the same colour in both panels.}}\n \t\\label{fig:g=3}\n \\end{figure}\n\n\nThe second group of spectral singularities [blue curves in figure~\\ref{fig:general}(a)] was obtained by means of the continuation from the  next solution in the limit $\\ell=0$, i.e.  from $\\gamma_\\star^{(1)}$ and $k_\\star^{(1)}$ in Table~\\ref{tbl:1}. Again, one of the curves [the leftmost bold curve in figure~\\ref{fig:general}(a)] was obtained through the direct continuation from the limit $\\ell=0$, while other blue curves were generated  using the periodicity of function $F(k, \\gamma, \\ell)$  in $\\ell$ and cannot be continued to the limit $\\ell\\to0$. In the limit $\\ell\\to\\infty$ the blue curves approach the horizontal asymptotes  $\\gamma=2\\pi^2\\approx19.739$ and $\\gamma=9\\pi^2/2\\approx 44.413$. In  the $(\\gamma, \\ell)$-plane  the  gain-and-loss amplitudes corresponding to the   group of blue curves are well separated from  those for the red curves: indeed, all red curves are bounded from above by the asymptote $\\gamma= \\pi^2/2\\approx 4.935$, while all blue curves are bounded from below by $\\gamma_*\\approx 11.561$, see (\\ref{eq:gap}). Thus, the  emergence of new spectral singularities with the increase of $\\ell$ is sensitive to the value of the gain-and-loss amplitude and does not occur for     the gain-and-loss amplitudes lying in the  gap  between the red and blue curves.\n\nIn comparison with the red curves discussed above, the curves from the blue group in figure~\\ref{fig:general}(a) feature more complicated behavior and, in particular, can intersect each other (and also intersect the curves from the next, third group of green curves discussed below). The intersections between the blue curves occur for the gain-and-loss amplitudes in the interval $11.561 \\lessapprox   \\gamma <  9\\pi^2/2\\approx 19.739$, where $\\sin\\sqrt{2\\gamma}$ is negative.  \\textcolor{black}{At first glance,} this might seem to contradict to  the  expansion (\\ref{3.4}) of Lemma~\\ref{lm3.4},  which  suggests that in this case the multitude of complex zeros   accumulate in the upper complex half-plane with the growth of $\\ell$. However, this apparent contradiction is resolved if we trace the behavior of the complex roots more closely.   Indeed, choosing for an example $\\gamma =16$ [see the vertical axis tick in  figure~\\ref{fig:general}(a)] and computing several  complex roots under the increase of $\\ell$, we observe that each considered root first goes down from the upper half-plane to the lower one but then again returns to the upper half-plane, see Figure~\\ref{fig:g=16}(b) and (b$_1$). Thus,  in  this  interval of the gain-and-loss amplitudes the increase of $\\ell$  results  either in the transformation from the resonance to the eigenvalue or to the opposite process,  i.e. to the disappearance of the complex-conjugate pair.  Respectively, the intersection between the two blue curves corresponds to  two coexisting spectral singularities, i.e. to the moment  when one complex-conjugate pair of eigenvalues disappears and another pair (with different $k$) emerges. For sufficiently large $\\ell$ the imaginary part  of each   considered root  remains positive and approaches zero, in accordance with expansion (\\ref{3.4}). Thus, in this interval of the gain-and-loss strengths, the limit $\\ell\\to\\infty$ the spectrum contains only a finite number of complex-conjugate eigenvalues, which correspond to complex zeros $k$ whose behavior is not covered   by expansion  (\\ref{3.4}).\n\n \\begin{figure}\n\t\\centering\n\t\\includegraphics[width=0.99\\columnwidth]{fig08.eps}\n\t\\caption{(a,b) Real and imaginary parts for three   complex zeros of function $F$ for fixed $\\gamma=16$ and increasing $\\ell$. Panel b$_1$ is the magnification of some region of (b) and  shows more clearly that imaginary parts of all   three shown eigenvalues are positive for all sufficiently large  $\\ell$. \\textcolor{black}{For each three shown eigenvalues, its real and \timaginary parts are of the same colour in both panels.}}\n\t\\label{fig:g=16}\n\\end{figure}\n\nThe third  group of spectral singularities [green curves in figure~\\ref{fig:general}(a)] was obtained by means of the continuation from  the solution $\\gamma_\\star^{(2)}$ and $k_\\star^{(2)}$ in Table~\\ref{tbl:1}. In view of the very complicated structure of the overall resulting picture, we only show a section of these curves corresponding to relatively small   values of the gain-and-loss amplitudes $\\gamma$. Quite interestingly, there is no gap between the blue and green groups of curves, which results in the multitude of intersections between blue and green curves, \\textcolor{black}{see Fig.~\\ref{fig:general}(a) and the magnified view in Fig.~\\ref{fig:general}(c)}. These   intersections suggest a possibility of   simultaneous emergence of two complex-conjugate pairs of eigenvalues from two different interior points of the continuous spectra.\n\nConsidering further solutions $\\gamma_\\star^{(n)}$, $k_\\star^{(n)}$, $n=3,4,\\ldots$, in the limit $\\ell=0$ one can construct new groups of spectral singularities  with larger values of $\\gamma$, which are not shown in Figure~\\ref{fig:general}.\n\n\n",
        "label": "fig:g=3",
        "Descriptive_question1": "What is the fixed value of gamma (γ) used in figure_7?",
        "Descriptive_question2": "In figure_7, what color scheme is used for the real and imaginary parts of each eigenvalue?",
        "Reasoning_question1": "How does the imaginary part of the complex zeros in figure_7 change with increasing ℓ, and what does this imply about the system's behavior?",
        "Reasoning_question2": "What trend can be inferred from the transition of the imaginary parts from positive to negative in figure_7 as ℓ increases, and how does this relate to spectral singularities?",
        "Descriptive_answer1": "3",
        "Descriptive_answer2": "same colour",
        "Reasoning_answer1": "The imaginary part of each complex zero is positive for small ℓ but becomes negative for sufficiently large ℓ, as illustrated by the curves in figure_7. This behavior implies that with increasing ℓ, the system transitions from resonances with positive imaginary parts (which correspond to decaying modes) to states with negative imaginary parts (growing or unstable modes). This indicates a spectral transition influenced by the parameter ℓ.",
        "Reasoning_answer2": "The transition of imaginary parts from positive to negative as ℓ increases shows that the complex eigenvalues cross the real axis, signifying the occurrence of spectral singularities at those crossing points. These spectral singularities mark the boundary where resonant states transform into true eigenstates with complex eigenvalues, implying changes in system stability and the emergence or disappearance of modes associated with the gain and loss."
    },
    {
        "paper_id": "1908.06383.json",
        "image_id": "figure_8",
        "image_path": [
            "/home/yz979/scratch/chengye/scimolmo/test-images-tables/dataset_imagestables/1908.06383/images/fig08.eps"
        ],
        "caption": "(a,b) Real and imaginary parts for three   complex zeros of function $F$ for fixed $\\gamma=16$ and increasing $\\ell$. Panel b$_1$ is the magnification of some region of (b) and  shows more clearly that imaginary parts of all   three shown eigenvalues are positive for all sufficiently large  $\\ell$. \\textcolor{black}{For each three shown eigenvalues, its real and \timaginary parts are of the same colour in both panels.}",
        "classify": "Chart",
        "section_info": "5 Spectral singularities\n\\section{Spectral singularities}\n\\label{sec:ss}\n\\subsection{General analytical expressions}\n\nIn this section we study real zeroes of the function $F$ corresponding to spectral singularities, i.e. to  zero-width resonances.\nFor such zeroes, equation (\\ref{2.10}) is a pair of two real equations for one real variable $k$ and two parameters $\\ell$ and $\\g$.\nThanks to the symmetry of the zeroes with respect to the imaginary axis, it is sufficient to find only positive real resonances since the negative ones are located symmetrically with respect to the origin. Similar to the proof of Lemma~\\ref{lm3.6}, for real positive $k$  we make  change (\\ref{4.1})\nand rewrite equation (\\ref{2.10}) in the following form:\n\\begin{align*}\n(1-2u^{-4})\\cos\\b u^{-1}&+(2u^4-1)\\cosh\\b u + 2i \\sqrt{1-u^4}\\left(u^2\\sinh\\b u - u^{-4}\\sin\\b u^{-1}\\right)\n\\\\\n&-e^{-2i\\b\\ell\\sqrt{u^{-2}-u^2}} \\left(\\cosh(\\b u)-\\cos\\b u^{-1}\\right)=0,\\qquad \\b=\\sqrt{2\\g}.\n\\end{align*}\nTaking the real and imaginary part of this equation and multiplying the equation by $u^4$, we obtain:\n\\begin{equation}\\label{4.3}\n\\begin{aligned}\n&(u^4-2)\\cos\\b u^{-1}+u^4(2u^4-1)\\cosh\\b u\n=u^4\\cos\\left(2\\b\\ell\\sqrt{u^{-2}-u^2}\\right)\\left(\\cosh \\b u-\\cos\\b u^{-1}\\right),\n\\\\\n&2\\sqrt{1-u^4}\\Big(u^6\\sinh\\b u-\\sin\\b u^{-1}\\big)\n=-u^4\\sin\\left(2\\b\\ell\\sqrt{u^{-2}-u^2}\\right)\\left(\\cosh \\b u-\\cos\\b u^{-1}\\right).\n\\end{aligned}\n\\end{equation}\nThis is a system of two real equations with three real variables. If we are given $(\\b,\\ell)$ and we try to find $u$, the system is overdetermined and does not necessary have a root. In other words, it is solvable with respect to $u$ only if $(\\b,\\ell)$ are located on some (solvability) curves. In order to avoid working with an overdetermined system, in what follows we regard (\\ref{4.3}) as a system for two unknown variable with one parameter.\n\nTo find the curves in $(\\b,\\ell)$ plane, on which equations (\\ref{4.3}) are solvable with respect to $u$, we shall regard $u$ as a parameter and $(\\b,\\ell)$ as unknown variables. We take the sum of squares of equations (\\ref{4.3}) and divide the result by $(1-u^4)$. We also divide equations (\\ref{4.3}). This leads us to a   pair  of equations:\n\\begin{align}\\label{4.4}\n&u^4(1-u^4) \\cosh\\b u\\cos\\b u^{-1}-2u^6 \\sinh\\b u\\sin\\b u^{-1} +1-u^{12}=0,\n\\\\\n&\\frac{2\\sqrt{1-u^4}(u^6\\sinh\\b u-\\sin\\b u^{-1})}{(2-u^4)\\cos\\b u^{-1}+u^4(1-2u^4)\\cosh\\b u}=\\tan \\Big( 2\\b\\ell\\sqrt{u^{-2}-u^2}\\Big).\\nonumber\n\\end{align}\nThe second equation can be solved explicitly with respect to $\\ell$:\n\\begin{equation}\\label{4.10}\n\\ell=\\frac{1}{2\\b\\sqrt{u^{-2}-u^2}} \\left(\\arctan \\frac{2\\sqrt{1-u^4}(u^6\\sinh\\b u-\\sin\\b u^{-1})}{(2-u^4)\\cos\\b u^{-1}+u^4(1-2u^4)\\cosh\\b u}+\\pi n\\right),\n\\end{equation}\nwhere $n\\in\\mathds{N}$ is an arbitrary natural number. As the next lemma states, to make equations (\\ref{4.4}), (\\ref{4.10}) equivalent to (\\ref{4.3}), we should also assume that\n\\begin{equation}\\label{4.5}\n(-1)^n=\\sign \\big((u^4-2)\\cos\\b u^{-1}+u^4(2u^4-1)\\cosh\\b u\\big).\n\\end{equation}\n\n\\begin{lemma}\\label{lm4.1}\nEquations (\\ref{4.3}) are equivalent to (\\ref{4.4}), (\\ref{4.10}), (\\ref{4.5}).\n\\end{lemma}\n\n\\begin{proof}\nWe rewrite shortly equations (\\ref{4.3}) as\n$A_1=B\\cos\\a$, $ A_2=-B\\sin\\a$,\nwhere $A_1$, $A_2$ are the left hand sides in (\\ref{4.3}), $\\a=2\\b\\ell\\sqrt{u^{-2}-u^2}$ and $B=u^4(\\cosh\\b u-\\cos\\b u^{-1})$. Then equations (\\ref{4.4}), (\\ref{4.10}) become\n$A_1^2+A_2^2=B^2$, $\\a=-\\arctan\\frac{A_2}{A_1}+\\pi n$.\nWe have:\n\\begin{equation*}\nB\\cos \\a=(-1)^n\\cos\\arctan\\frac{A_2}{A_1}=\\frac{(-1)^n}{\\sqrt{1+\\frac{A_2^2}{A_1^2}}} = \\frac{(-1)^n|A_1|}{\\sqrt{A_1^2+A_2^2}}\n\\end{equation*}\nand we get the first equation $A_1=B\\cos\\a$  provided condition (\\ref{4.5}) is satisfied. In the same way we check that the latter condition also ensures the second equation $A_2=-B\\sin\\a$.\n\\end{proof}\n\n\nEquation (\\ref{4.4}) is transcendental, and we can not solve it analytically.\nNevertheless, for each $u\\in(0,1)$, this is an equation only for a single variable $\\b$, not for two as equations (\\ref{4.3}). So, we propose the following algorithm of recovering the aforementioned solvability curves: choose $u\\in(0,1)$, then solve equation (\\ref{4.4}) and recover the sequence of  distances  $\\ell$ by formula (\\ref{4.10}) with different integer $n$. Then the gain-and-loss amplitude $\\gamma$ and the corresponding wavenumber $k$ can be readily recovered from $\\beta$ and  $u$.  In a similar way, one can first fix some value of $\\beta$ (i.e., fix the gain-and-loss strength) and then   solve equation (\\ref{4.4}) with respect to $u$ and   recover   $\\ell$ by (\\ref{4.10}). Equation (\\ref{4.4})  is well-behaved, and for each $\\beta$  all its zeros  $u$ can be easily found numerically.\n\nAlternatively, as explained below  in Section~\\ref{sec:general}, the values  corresponding to  spectral singularities can be found systematically by means of the numerical continuation from the limit $\\ell=0$. However, in this case   equation (\\ref{4.4}) is still useful because it allows one to check that all spectral singularities have been found for the given value of the gain-and-loss $\\gamma$.\n\n\n\\subsection{\nAbsence of   spectral singularities}\n\\label{sec:gap}\n\nFor $u=0$ and $u=1$, the left-hand-side of equation (\\ref{4.4}) is equal respectively to $1$ and $-2\\sinh\\beta\\sin\\beta$. Then a sufficient condition for the existence of a spectral singularity at the given gain-and-loss amplitude $\\gamma$ is $\\sin\\beta=\\sin\\sqrt{2\\gamma}>0$. At the same time, it is also possible to establish sufficient conditions that forbid the existence of   spectral singularities in a certain interval of parameters. In this subsection we prove the existence of two ``forbidden gaps'' for the roots of equation (\\ref{4.4}). The first one exists for all $\\b\\geqslant 0$ and it states that there is no roots in certain interval. The second gap is a certain interval of values of $\\b$, for which equation (\\ref{4.4}) has no zeroes at all.\n\nFor the convenience, by $g(u,\\b)$ we denote the left hand side of equation (\\ref{4.4}). The first ``forbidden gap'' is described in the following lemma.\n\n\\begin{lemma}\\label{lm5.1}\nFor all $\\b\\geqslant 0$, equation (\\ref{4.4}) has no roots in the interval $\\big[0,(1+\\tfrac{\\b}{4})^{-1}\\big)$.\n\\end{lemma}\n\\begin{proof}\nEmploying a standard inequality $a\\cos\\a+b\\sin\\a\\leqslant \\sqrt{a^2+b^2}$, we estimate the first two term in equation (\\ref{4.4}) as\n\\begin{equation*}\nu^4(1-u^4) \\cosh\\b u\\cos\\b u^{-1}-2u^6 \\sinh\\b u\\sin\\b u^{-1} \\leqslant \\sqrt{u^8(1-u^4)^2\\cosh^2 \\b u+4u^{12}\\sinh^2\\b u}.\n\\end{equation*}\nHence, equation (\\ref{4.4}) surely has no roots for values of $u$ satisfying\n\\begin{equation*}\n \\sqrt{u^8(1-u^4)^2\\cosh^2 \\b u+4u^{12}\\sinh^2\\b u} <1-u^{12}.\n\\end{equation*}\nExpressing $\\cosh^2 \\b u$ via $\\sinh^2 \\b u$ and simplifying this inequality, we obtain $ u^4(1+u^4)\\cosh \\b u <1+u^{12}$ and hence,\n\\begin{equation}\\label{3.37}\n\\cosh \\b u-1 <\\frac{1-2u^4+u^8}{u^4},\\qquad \\sqrt{2}\\sinh \\b u\n<u^{-2}-u^2.\n\\end{equation}\nThanks to the estimate $\\sqrt{2}\\sinh 2\\ln u^{-1}\\leqslant u^{-2}-u^2$, $u\\in(0,1]$, the last inequality in (\\ref{3.37}) holds once $\\frac{\\b}{4}<u^{-1}\\ln u^{-1}$. It is easy to confirm that this inequality is true once $u^{-1}>1+\\tfrac{\\b}{4}$.\nThe proof is complete.\n\\end{proof}\n\n\nThe next lemma is   auxiliary  and will be employed in studying the second forbidden zone.\n\n\\begin{lemma}\\label{lm5.2}\nThe function $g(u,\\pi)$ is positive on $[0,1)$.\n\\end{lemma}\n\n\\begin{proof}\nWe have $g(0,\\pi)=1$ and by Lemma~\\ref{lm5.1}, it is positive for $u<(1+\\tfrac{\\pi}{4})^{-1}$. This is why in what follows we consider only the values $u\\geqslant (1+\\tfrac{\\pi}{4})^{-1}$. For such values of $u$ we have\n$\\pi\\leqslant \\pi u^{-1}\\leqslant \\pi (1+\\tfrac{\\pi}{4})<1.79\\pi$.\nAs\n$\\tfrac{3\\pi}{2} \\leqslant \\pi u^{-1}\\leqslant \\pi (1+\\tfrac{\\pi}{4})$,\nthe function $\\sin\\b u^{-1}$ is negative, while $\\cos \\b u^{-1}$ is positive. Hence, for such values of $u$, the function $g(u,\\pi)$ is positive. It remains to consider the values $\\frac{2}{3}<u\\leqslant 1$.\n\nFor such values of $u$ we first observe the following simple estimates:\n\\begin{equation*}\n\\frac{1-u^4}{1-u}\\cos\\pi u^{-1}\\geqslant -4,\\qquad -\\frac{\\sin \\pi u^{-1}}{1-u}=\\frac{\\sin \\pi (u^{-1}-1)}{1-u}\\geqslant \\pi u.\n\\end{equation*}\nThen the function $g$ satisfies the estimate:\n\\begin{equation}\\label{3.12}\n\\frac{g(u,\\pi)}{u^4(1-u)}\\geqslant g_1(u)+g_2(u),\\qquad g_1(u):=-4\\cosh\\pi u +2\\pi u^3 \\sinh \\pi u,\\qquad g_2(u):=\\frac{1-u^{12}}{u^4(1-u)}.\n\\end{equation}\nThe function $g_1(u)$ is monotonically increasing in $u\\in[\\tfrac{2}{3},1]$ since\n\\begin{equation*}\ng_1'(u)=2\\pi^2 u^3\\cosh\\pi u+(6\\pi u^2-4\\pi)\\sinh\\pi u>2\\pi(\\pi u^3+6u^2-2)\\sinh\\pi u>2\\pi\\sinh\\pi u>0.\n\\end{equation*}\nHence,\n$g_1(u)\\geqslant g_1\\left(\\frac{2}{3}\\right)>-9.05$.\nFor the function $g_2$ we have the following representation and estimate:\n\\begin{equation*}\ng_2(u)=1+\\sum\\limits_{j=1}^{4}(u^j+u^{-j})+\\sum\\limits_{j=5}^{8} u^j\\geqslant 9 +\\sum\\limits_{j=5}^{8}\\left(\\frac{2}{3}\\right)^j>9.31.\n\\end{equation*}\nTwo last estimates and (\\ref{3.12}) imply the positivity of the function $g$ for $u\\in[\\tfrac{2}{3},1)$.\n\\end{proof}\n\n\nDenote\n\\begin{align*}\ng_*(u,\\b):=&\\b u^3(1-3u^4)\\cosh\\b u \\sin \\b u^{-1}+\\b u^5 (3-u^4)\\sinh\\b u \\cos \\b u^{-1}\n\\\\\n&-2 u^4(1+u^4) \\cosh\\b u \\cos \\b u^{-1} -6(1+u^{12}).\n\\end{align*}\n\n\nThe next lemma states the existence of the second forbidden zone.\n\n\\begin{lemma}\\label{lm5.3}\nEquation (\\ref{4.4}) has no roots as $\\pi<\\b<\\b_*<5$, where $(u_*, \\b_*)$ is  the root of the system of the equations\n\\begin{equation}\\label{3.13}\ng(u,\\b)=0,\\qquad g_*(u,\\b)=0,\\qquad u\\in[0,1],\\qquad \\pi<\\b<5,\n\\end{equation}\nwith minimal possible $\\b$.\nTheir approximate values are\n\\begin{equation}\\label{3.14}\n\\b_*=4.808438,\\qquad u_*=0.611772.\n\\end{equation}\n\\end{lemma}\n\n\\begin{proof}\nThe function $g(u,\\pi)$ is positive on $[0,1)$ and $g(1,\\pi)=0$, see Figure~\\ref{fig:forbidden}a. As $\\pi<\\b<2\\pi$, we have $g(0,\\b)=1>0$ and $g(1,\\b)=-2\\sin\\b\\sinh\\b>0$. Hence, for $\\b$ close enough to $\\pi$, the function $g(u,\\b)$ is positive for all $u\\in[0,1]$. At the same time, we have $g(0.65,5)<-0.617<0$ and therefore, for $\\b=5$, equation (\\ref{4.4}) possesses at least two roots, one in $(0,0.65)$ and another in $(0.65,1)$. cf. Figure~\\ref{fig:forbidden}c. We also observe that the function $g$ is jointly continuous in $(u,\\b)$. The above facts means that as $\\b$ grows from $\\pi$ to $5$, at some value $\\b=\\b_*$, the graph of the function $g$ is still located in the upper half-plane but touches the $u$-axis at some point $u=u_*$,\nsee Figure~\\ref{fig:forbidden}b. The function $g(u,\\b)$ is positive as $\\pi<\\b<\\b_*$ and $u\\in[0,1]$. Then the point $u=u_*$ is obviously the global minimum of $g$ and hence, $(u_*,\\b_*)$ is a solution to the system of equations $g(u,\\b)=0$, $\\frac{\\p g}{\\p u}(u,\\b)=0$. It is easy to check that $g_*=\\frac{\\p g}{\\p u}-6 g$ and hence, $(u_*,\\b_*)$ solves system (\\ref{3.13}). These roots can be found numerically and this gives (\\ref{3.14}). The proof is complete.\n\\end{proof}\n\n\n\\begin{figure}\n\t\\centering\n\\includegraphics[width=0.99\\columnwidth]{fig03.eps}\n\\caption{Illustration for proof of Lemma~\\ref{lm5.3}. Graphs of the function $g(u, \\beta)$ for $\\b=\\pi$ (a), $\\b=\\b_*\\approx  4.808438$ (b), and $\\b=5$ (c). Notice broken vertical axes in (b) and (c). }\n\\label{fig:forbidden}\n\\end{figure}\n\nReturning from the auxiliary variable $\\beta$ to the gain-and-loss amplitude $\\gamma=\\beta^2/2$, from   Lemma~\\ref{lm5.3}  we deduce  the following important result:\n\\begin{equation}\n\\label{eq:gap}\n\\textrm{there is no spectral singularities for\\quad }  \\frac{\\pi^2}{2} < \\gamma< \\gamma_*\\approx  11.561.\n\\end{equation}\n\n\n\n\\subsection{Creating a  spectral singularity  at a given wavenumber}\n\n\\begin{table}\n\t\\centering\n\t\\begin{tabular}{c|cccc}\n\t\t$n$ & 0 & 1& 2&\\\\\\hline\n\t\t$\\gamma_\\star^{(n)}$ & 2.071&13.307&27.783 &\\\\[2mm]\n\t\t$k_\\star^{(n)}$ & 1.065&4.318 &7.529 &\n\t\\end{tabular}\n\t\\caption{Approximate values of gain-and-loss amplitudes $\\gamma=\\gamma_\\star^{(n)}$  and wavenumbers   $k=k_\\star^{(n)}$, $n=0, 1, \\ldots$,  corresponding to spectral singularities with lowest $\\gamma$ in the limit $\\ell=0$, see Table~I in  \\cite{Mostafazadeh2009}. \\label{tbl:1}}\n\\end{table}\n\n\nFor $\\ell=0$ spectral singularity can only be obtained for some isolated values of the wavenumber $k$ and the gain-and-loss amplitude $\\gamma$ \\cite{Mostafazadeh2009}. Several lowest values of $\\gamma$ corresponding to the spectral singularities  and the associated wavenumbers  $k$ are listed in Table~\\ref{tbl:1}.  An  important advantage of the more general system with nonzero gain-to-loss disctance    $\\ell>0$ consists  in the possibility to create a spectral   singularity   at any wavenumber $k$ given beforehand. Indeed, let us  return back to equations (\\ref{4.4}), (\\ref{4.10}) and    discuss the following   issue: given a point $k$ on the real axis, how to choose $\\b$ and $\\ell$ to have a  resonance at this point? Equations (\\ref{4.4})--(\\ref{4.10}) allow us to answer  easily this question.\n\nWe fix $k>0$ and we find the associated value of $u$ by resolving (\\ref{4.1}):\n\\begin{equation}\\label{4.12}\nu^{-2}-u^2=4k^2\\b^{-2},\\qquad\n\nu=\\b R^{-1}, \\quad \\mbox{where\\ } R = \\sqrt{2k^2+\\sqrt{4k^4+\\b^4}}.\n\\end{equation}\nWe divide equation (\\ref{4.4}) by $u^6$ and substitute then the above formulae and\n\\begin{equation*}\n\\frac{1-u^{12}}{u^6}=\\frac{1-u^4}{u^2}\\frac{1+u^4+u^8}{u^4}=(u^{-2}-u^2)\\big((u^{-2}-u^2)^2+3\n\\big).\n\\end{equation*}\nThis gives the equation:\n\\begin{equation}\\label{4.11}\n\\begin{aligned}\n2\\b^4 k^2&\\cosh(\\b^2R^{-1})  \\cos R\n\n\n - \\b^6 \\sinh(\\b^2R^{-1})  \\sin R\n  +2k^2(16k^4+3\\b^4)=0.\n\\end{aligned}\n\\end{equation}\nAn algorithm for creating a resonance at a prescribed point  $k$ is as follows. Given $k>0$, we first solve equation (\\ref{4.11}) with respect to $\\b$ and we also find $u$ by (\\ref{4.12}). Then needed values of  $\\ell$ are determined by (\\ref{4.10}), (\\ref{4.5}).\n\n\nIn order to illustrate this  algorithm,  we us consider a finite interval of wavenumbers $k\\in (0, k_1]$, where  we set $k_1 = 10$ for the numerics reported on in what follows. We scan   the chosen interval  with a sufficiently small step ($\\Delta k=0.01$) and for each value of $k$ solve equation (\\ref{4.11}) numerically  using the simple dichotomy method. While for each $k$  equation (\\ref{4.11}) might have several roots $\\beta$, in our numerical procedure we always choose the  minimal positive root, i.e., the one which allows to achieve the spectral singularity with given $k$ at the smallest possible value  of the gain-and-loss amplitude $\\gamma=\\gamma_\\textrm{min}$. Next, we choose the minimal positive distance $\\ell_\\textrm{min}$ which satisfies the conditions   (\\ref{4.10}), (\\ref{4.5}) and then we use the periodicity in $\\ell$ to generate the sequence of larger gain-to-loss distances $\\ell_n = \\ell_\\textrm{min} + n\\pi/(2k)$, $n=1,2\\ldots$ [see   (\\ref{eq:periodic})]. The resulting dependencies $\\gamma_\\textrm{min}(k)$ and $\\ell_\\textrm{min}(k)$, $\\ell_n(k)$   are shown in figure~\\ref{fig:min}. The minimal gain-and-loss amplitude $\\g_{min}(k)$  and the minimal gain-to-loss distance $\\ell_{min}$ are   discontinuous,  which means that the small variation  in  the wavenumuber $k$  might require  a significant  change either in   $\\gamma$ or in $\\ell$.  It is especially important that the values of the  distance  $\\ell$ are generically different from zero, which points out explicitly    that the new degree of freedom offered by the nonzero gain-to-loss distance    is   important for the achieving a spectral singularity  at   the given wavenumber $k$.\n\n\n\\begin{figure}\n\t\\centering\n\t\\includegraphics[width=0.8\\columnwidth]{fig04.eps}\n\t\\caption{(a) Minimal value of the gain-and-loss amplitude $\\gamma_\\textrm{min}$  which corresponds to   a spectral singularity with the given value of the wavenumber $k$. (b)  Minimal gain-to-loss distance  $\\ell_\\textrm{min}$  which corresponds to   a spectral singularity with the  $k$ and $\\gamma$ from the  left panel (bold curves) and larger distances $\\ell_n$ obtained using the periodicity in $\\ell$ (thin \\textcolor{black}{dotted} curves).}\n\t\\label{fig:min}\n\\end{figure}\n\n\n\\subsection{$\\PT$-symmetry breaking laser-antilaser threshold}\n\nA particularly important characteristics of any $\\PT$-symmetric structure is the $\\PT$ symmetry breaking threshold, i.e., the amplitude of the gain-and-loss\ncorresponding to the  ``phase transition'' from the purely real to complex spectrum.  The best studied scenario of the phase transition  \nis the collision of two real discrete  eigenvalues at an exceptional point with the subsequent splitting in a complex-conjugate pair. However, in systems with nonempty continuous spectrum,  the phase transition can also occur through   the splitting of a self-dual spectral singularity, which results in a bifurcation of a complex-conjugate pair from an interior point of the continuum \\cite{Yang17,KZ17,Konotop2019}. At the moment corresponding to the formation of the spectral singularity, the system operates in the CPA-laser regime \\cite{Longhi10}. Thus, in such a system, the $\\PT$-symmetry breaking threshold at the same time corresponds to the CPA-laser threshold.\n\nLemma~\\ref{lm3.3} guarantees that the spectrum of our system is real for sufficiently small gain-and-loss amplitudes $\\gamma$. Additionally, according to Lemma~\\ref{lm3.5}, the spectrum  does not have any real discrete eigenvalue. Hence, the $\\PT$-symmetry breaking is expected to occur through the emergence of a self-dual spectral singularity. In order to identify the $\\PT$-symmetry breaking threshold in our system, we start from the limit $\\ell=0$, there the phase transition takes place at  $\\gamma_\\star^{(0)} \\approx 2.072$, see \\cite{Mostafazadeh2009,KZ17} and Table~\\ref{tbl:1}. Thus, the spectrum with $\\ell=0$ is purely real and continuous for $\\gamma \\in [0, \\gamma_\\star]$, while the  increase of the gain-and-loss just above $\\gamma_\\star^{(0)}$ leads to the bifurcation of a complex-conjugate pair from an interior point  of the continuum. The spectral singularity forming at  $\\gamma_\\star^{(0)}$ takes place at wavenumber $k=k_\\star^{(0)} \\approx  1.065$. Respectively, the complex-conjugate pair of eigenvalues bifurcates from $\\lambda_0 =  [k_\\star^{(0)}]^2$. (Notice that the further increase of $\\gamma$ above the next threshold values listed in Table~\\ref{tbl:1} leads to the formation of new spectral singularities and, respectively, to bifurcations of new complex-conjugate pairs in the spectrum.)\n\nNext, we use the numerical continuation in $\\ell$ in order to continue the known solution at $\\ell=0$ to the domain $\\ell>0$.  The obtained dependence of the threshold value  of the gain-and-loss amplitude on distance $\\ell$ is shown in figure~\\ref{fig:threshold}(a), where we observe that the phase transition threshold decreases monotonically with the growth of $\\ell$. This means that introducing an additional space   between the gain and loss, one can decrease the $\\PT$-symmetry breaking  threshold, i.e. achieve the laser-antilaser operation at \\textcolor{black}{lower} gain-and-loss amplitudes than in a waveguide with the adjacent gain and loss.\n\n\n\\begin{figure}\n\t\\centering\n\t\\includegraphics[width=0.85\\columnwidth]{fig05.eps}\n\t\\caption{(a) $\\PT$-symmetry breaking threshold $\\gamma_\\star^{(0)}$ \\textit{vs} the distance between the gain and loss $\\ell$. The spectrum is purely real and continuous for $\\gamma\\leq \\gamma_\\star$, but acquires a pair of complex conjugate eigenvalues  as the gain-and-loss amplitude exceeds the threshold $\\gamma_\\star^{(0)}$. (b) Values of the wavevector $k_\\star^{(0)}$ corresponding to the dependence in (a).}\n\t\\label{fig:threshold}\n\\end{figure}\n\n\\subsection{General picture of spectral singularities}\n\\label{sec:general}\n\n\\begin{figure}\n\t\\centering\n\t\\includegraphics[width=0.99\\columnwidth]{fig06.eps}\n\t\\caption{(a) Values of the gain-and-loss amplitude $\\gamma$ and distance  $\\ell$, for which   spectral singularities occur. (b) Corresponding  wavenumbers   $k$. \\textcolor{black}{Panel (c) magnifies   the region  $(\\ell, \\gamma)\\in [2,4]\\times[10,50]$ from   (a), and panel  (d) is the  magnification of corresponding curves from panel (b).}}\n\t\\label{fig:general}\n\\end{figure}\n\nLet us now turn to description of the general picture of spectral singularities. In order to construct systematically different solutions, we again start from the limit $\\ell=0$, where the values of $\\gamma$ and $k$ corresponding to spectral singularities are known, see Table~\\ref{tbl:1}. Then we use the periodicity of function $e^{-4ik\\ell}$ in $\\ell$ in order to construct new branches of solutions having no counterparts in the limit $\\ell=0$, cf. equation (\\ref{eq:periodic}). This procedure results in a fairly complicated picture containing a multitude of spectral singularities, some part of which (corresponding to  relatively  small   values of the gain-and-loss) is shown in figure~\\ref{fig:general}(a,b) as the curves on the plane $\\gamma$ {\\it vs} $\\ell$ and $k$ {\\it vs} $\\ell$.\n\n\nLet us describe the structure of the found solutions using the  diagram $\\gamma$ {\\it vs} $\\ell$ in figure~\\ref{fig:general}(a). The multitude of curves shown in this plot can be divided into three groups (plotted with red, blue and green curves) which have been obtained by means of the continuation from three different solutions in the limit $\\ell=0$. There is a well-visible vertical gap between red and blue curves, which corresponds to the ``forbidden'' values of the gain-and-loss amplitudes $\\gamma$ found above in (\\ref{eq:gap}). At the same time,  there is no gap between blue and green curves, which  results in a multitude of intersections between these curves.\nThe first group of spectral singularities,  corresponding to red curves in Figure~\\ref{fig:general}(a),   is obtained through  the continuation from the  spectral singularity  in the limit $\\ell=0$ with the smallest gain-and-loss amplitude, i.e.  from values $\\gamma_\\star^{(0)}$ and $k_\\star^{(0)}$ in Table~\\ref{tbl:1}.  The leftmost (bold)  curve in this group which originates in the limit $\\ell=0$  is the $\\PT$-symmetry breaking threshold which was already shown in figure~\\ref{fig:threshold}(a). For values of $\\gamma$ above this curve, there is always one or more (but finitely many, see Section~\\ref{sec:zeros})   complex-conjugate pairs of eigenvalues in the spectrum.  Several curves situated to the right from the bold red curve are obtained using the fact that if $\\gamma$ and $k$ are solutions in the limit $\\ell=0$, then the same values of $\\gamma$ and $k$ also correspond to a spectral singularity with $\\ell_n = (n\\pi)/(2k)$, $n=1,2,\\ldots$. Once a single solution with a new distance   $\\ell_n$ is obtained, a new branch of solutions can be constructed using numerical continuation in $\\gamma$ or in $\\ell$. In the limit $\\ell \\to \\infty$ all the red curves in figure~\\ref{fig:general}(a) approach  the asymptotic value $\\gamma=0$ or at $\\gamma_*= \\pi^2/2\\approx 4.935$. Notice that, except for the bold line demarcating the $\\PT$-symmetry breaking threshold, none of the red curves can be continued to the limit $\\ell\\to 0$.\n\nThe multitude of red curves  in figure~\\ref{fig:general}(a) demonstrates explicitly how new spectral singularities emerge with the increase of $\\ell$. Indeed, drawing an imaginary horizontal line, say, at $\\gamma=3$ [see the vertical axis  tick  in figure~\\ref{fig:general}(a)], we observe that with the increase of $\\ell$ this line   intersects more and more red curves. Each intersection corresponds to the values of $\\gamma$ and $\\ell$ at which   some  root $k$   crosses the real axis and goes down from the upper to lower complex half-plane. Thus,  a finite-width resonance transforms  to a complex eigenvalue through the spectral singularity (we recall again that  in view of   $\\PT$ symmetry  any root $k\\ne 0$ crosses the real line simultaneously with its counterpart $-\\bar{k}$, i.e.  the corresponding   spectral singularity is self-dual).\nIn order to illustrate this process, in Figure~\\ref{fig:g=3}  we show the evolution of three numerically found complex zeros of function $F(k, \\gamma, \\ell)$  under   the increase of $\\ell$.  In this figure the imaginary part of each complex zero changes sign from positive to negative and then asymptotically approaches zero (remaining negative). In the limit of large $\\ell$,  this behavior agrees  with expansion (\\ref{3.4}) of Lemma~\\ref{lm3.4} , where, for the chosen value of $\\gamma$,   we have $\\sin\\sqrt{2\\gamma}>0$.   Thus the growing distance   $\\ell$ results in a sequence of self-dual spectral singularities and in the increasing number of complex-conjugate eigenvalues in the spectrum.\n\n \\begin{figure}\n \t\\centering\n \t\\includegraphics[width=0.8\\columnwidth]{fig07.eps}\n \t\\caption{(a,b) Real and imaginary parts of three complex zeros of function $F$ for fixed $\\gamma=3$ and increasing $\\ell$. For each curve, the imaginary part is positive for small $\\ell$ and becomes negative for all sufficiently large $\\ell$. \\textcolor{black}{For each three shown eigenvalues, its real and \timaginary parts are of the same colour in both panels.}}\n \t\\label{fig:g=3}\n \\end{figure}\n\n\nThe second group of spectral singularities [blue curves in figure~\\ref{fig:general}(a)] was obtained by means of the continuation from the  next solution in the limit $\\ell=0$, i.e.  from $\\gamma_\\star^{(1)}$ and $k_\\star^{(1)}$ in Table~\\ref{tbl:1}. Again, one of the curves [the leftmost bold curve in figure~\\ref{fig:general}(a)] was obtained through the direct continuation from the limit $\\ell=0$, while other blue curves were generated  using the periodicity of function $F(k, \\gamma, \\ell)$  in $\\ell$ and cannot be continued to the limit $\\ell\\to0$. In the limit $\\ell\\to\\infty$ the blue curves approach the horizontal asymptotes  $\\gamma=2\\pi^2\\approx19.739$ and $\\gamma=9\\pi^2/2\\approx 44.413$. In  the $(\\gamma, \\ell)$-plane  the  gain-and-loss amplitudes corresponding to the   group of blue curves are well separated from  those for the red curves: indeed, all red curves are bounded from above by the asymptote $\\gamma= \\pi^2/2\\approx 4.935$, while all blue curves are bounded from below by $\\gamma_*\\approx 11.561$, see (\\ref{eq:gap}). Thus, the  emergence of new spectral singularities with the increase of $\\ell$ is sensitive to the value of the gain-and-loss amplitude and does not occur for     the gain-and-loss amplitudes lying in the  gap  between the red and blue curves.\n\nIn comparison with the red curves discussed above, the curves from the blue group in figure~\\ref{fig:general}(a) feature more complicated behavior and, in particular, can intersect each other (and also intersect the curves from the next, third group of green curves discussed below). The intersections between the blue curves occur for the gain-and-loss amplitudes in the interval $11.561 \\lessapprox   \\gamma <  9\\pi^2/2\\approx 19.739$, where $\\sin\\sqrt{2\\gamma}$ is negative.  \\textcolor{black}{At first glance,} this might seem to contradict to  the  expansion (\\ref{3.4}) of Lemma~\\ref{lm3.4},  which  suggests that in this case the multitude of complex zeros   accumulate in the upper complex half-plane with the growth of $\\ell$. However, this apparent contradiction is resolved if we trace the behavior of the complex roots more closely.   Indeed, choosing for an example $\\gamma =16$ [see the vertical axis tick in  figure~\\ref{fig:general}(a)] and computing several  complex roots under the increase of $\\ell$, we observe that each considered root first goes down from the upper half-plane to the lower one but then again returns to the upper half-plane, see Figure~\\ref{fig:g=16}(b) and (b$_1$). Thus,  in  this  interval of the gain-and-loss amplitudes the increase of $\\ell$  results  either in the transformation from the resonance to the eigenvalue or to the opposite process,  i.e. to the disappearance of the complex-conjugate pair.  Respectively, the intersection between the two blue curves corresponds to  two coexisting spectral singularities, i.e. to the moment  when one complex-conjugate pair of eigenvalues disappears and another pair (with different $k$) emerges. For sufficiently large $\\ell$ the imaginary part  of each   considered root  remains positive and approaches zero, in accordance with expansion (\\ref{3.4}). Thus, in this interval of the gain-and-loss strengths, the limit $\\ell\\to\\infty$ the spectrum contains only a finite number of complex-conjugate eigenvalues, which correspond to complex zeros $k$ whose behavior is not covered   by expansion  (\\ref{3.4}).\n\n \\begin{figure}\n\t\\centering\n\t\\includegraphics[width=0.99\\columnwidth]{fig08.eps}\n\t\\caption{(a,b) Real and imaginary parts for three   complex zeros of function $F$ for fixed $\\gamma=16$ and increasing $\\ell$. Panel b$_1$ is the magnification of some region of (b) and  shows more clearly that imaginary parts of all   three shown eigenvalues are positive for all sufficiently large  $\\ell$. \\textcolor{black}{For each three shown eigenvalues, its real and \timaginary parts are of the same colour in both panels.}}\n\t\\label{fig:g=16}\n\\end{figure}\n\nThe third  group of spectral singularities [green curves in figure~\\ref{fig:general}(a)] was obtained by means of the continuation from  the solution $\\gamma_\\star^{(2)}$ and $k_\\star^{(2)}$ in Table~\\ref{tbl:1}. In view of the very complicated structure of the overall resulting picture, we only show a section of these curves corresponding to relatively small   values of the gain-and-loss amplitudes $\\gamma$. Quite interestingly, there is no gap between the blue and green groups of curves, which results in the multitude of intersections between blue and green curves, \\textcolor{black}{see Fig.~\\ref{fig:general}(a) and the magnified view in Fig.~\\ref{fig:general}(c)}. These   intersections suggest a possibility of   simultaneous emergence of two complex-conjugate pairs of eigenvalues from two different interior points of the continuous spectra.\n\nConsidering further solutions $\\gamma_\\star^{(n)}$, $k_\\star^{(n)}$, $n=3,4,\\ldots$, in the limit $\\ell=0$ one can construct new groups of spectral singularities  with larger values of $\\gamma$, which are not shown in Figure~\\ref{fig:general}.\n\n\n5.5 General picture of spectral singularities\n\\subsection{General picture of spectral singularities}\n\\label{sec:general}\n\n\\begin{figure}\n\t\\centering\n\t\\includegraphics[width=0.99\\columnwidth]{fig06.eps}\n\t\\caption{(a) Values of the gain-and-loss amplitude $\\gamma$ and distance  $\\ell$, for which   spectral singularities occur. (b) Corresponding  wavenumbers   $k$. \\textcolor{black}{Panel (c) magnifies   the region  $(\\ell, \\gamma)\\in [2,4]\\times[10,50]$ from   (a), and panel  (d) is the  magnification of corresponding curves from panel (b).}}\n\t\\label{fig:general}\n\\end{figure}\n\nLet us now turn to description of the general picture of spectral singularities. In order to construct systematically different solutions, we again start from the limit $\\ell=0$, where the values of $\\gamma$ and $k$ corresponding to spectral singularities are known, see Table~\\ref{tbl:1}. Then we use the periodicity of function $e^{-4ik\\ell}$ in $\\ell$ in order to construct new branches of solutions having no counterparts in the limit $\\ell=0$, cf. equation (\\ref{eq:periodic}). This procedure results in a fairly complicated picture containing a multitude of spectral singularities, some part of which (corresponding to  relatively  small   values of the gain-and-loss) is shown in figure~\\ref{fig:general}(a,b) as the curves on the plane $\\gamma$ {\\it vs} $\\ell$ and $k$ {\\it vs} $\\ell$.\n\n\nLet us describe the structure of the found solutions using the  diagram $\\gamma$ {\\it vs} $\\ell$ in figure~\\ref{fig:general}(a). The multitude of curves shown in this plot can be divided into three groups (plotted with red, blue and green curves) which have been obtained by means of the continuation from three different solutions in the limit $\\ell=0$. There is a well-visible vertical gap between red and blue curves, which corresponds to the ``forbidden'' values of the gain-and-loss amplitudes $\\gamma$ found above in (\\ref{eq:gap}). At the same time,  there is no gap between blue and green curves, which  results in a multitude of intersections between these curves.\nThe first group of spectral singularities,  corresponding to red curves in Figure~\\ref{fig:general}(a),   is obtained through  the continuation from the  spectral singularity  in the limit $\\ell=0$ with the smallest gain-and-loss amplitude, i.e.  from values $\\gamma_\\star^{(0)}$ and $k_\\star^{(0)}$ in Table~\\ref{tbl:1}.  The leftmost (bold)  curve in this group which originates in the limit $\\ell=0$  is the $\\PT$-symmetry breaking threshold which was already shown in figure~\\ref{fig:threshold}(a). For values of $\\gamma$ above this curve, there is always one or more (but finitely many, see Section~\\ref{sec:zeros})   complex-conjugate pairs of eigenvalues in the spectrum.  Several curves situated to the right from the bold red curve are obtained using the fact that if $\\gamma$ and $k$ are solutions in the limit $\\ell=0$, then the same values of $\\gamma$ and $k$ also correspond to a spectral singularity with $\\ell_n = (n\\pi)/(2k)$, $n=1,2,\\ldots$. Once a single solution with a new distance   $\\ell_n$ is obtained, a new branch of solutions can be constructed using numerical continuation in $\\gamma$ or in $\\ell$. In the limit $\\ell \\to \\infty$ all the red curves in figure~\\ref{fig:general}(a) approach  the asymptotic value $\\gamma=0$ or at $\\gamma_*= \\pi^2/2\\approx 4.935$. Notice that, except for the bold line demarcating the $\\PT$-symmetry breaking threshold, none of the red curves can be continued to the limit $\\ell\\to 0$.\n\nThe multitude of red curves  in figure~\\ref{fig:general}(a) demonstrates explicitly how new spectral singularities emerge with the increase of $\\ell$. Indeed, drawing an imaginary horizontal line, say, at $\\gamma=3$ [see the vertical axis  tick  in figure~\\ref{fig:general}(a)], we observe that with the increase of $\\ell$ this line   intersects more and more red curves. Each intersection corresponds to the values of $\\gamma$ and $\\ell$ at which   some  root $k$   crosses the real axis and goes down from the upper to lower complex half-plane. Thus,  a finite-width resonance transforms  to a complex eigenvalue through the spectral singularity (we recall again that  in view of   $\\PT$ symmetry  any root $k\\ne 0$ crosses the real line simultaneously with its counterpart $-\\bar{k}$, i.e.  the corresponding   spectral singularity is self-dual).\nIn order to illustrate this process, in Figure~\\ref{fig:g=3}  we show the evolution of three numerically found complex zeros of function $F(k, \\gamma, \\ell)$  under   the increase of $\\ell$.  In this figure the imaginary part of each complex zero changes sign from positive to negative and then asymptotically approaches zero (remaining negative). In the limit of large $\\ell$,  this behavior agrees  with expansion (\\ref{3.4}) of Lemma~\\ref{lm3.4} , where, for the chosen value of $\\gamma$,   we have $\\sin\\sqrt{2\\gamma}>0$.   Thus the growing distance   $\\ell$ results in a sequence of self-dual spectral singularities and in the increasing number of complex-conjugate eigenvalues in the spectrum.\n\n \\begin{figure}\n \t\\centering\n \t\\includegraphics[width=0.8\\columnwidth]{fig07.eps}\n \t\\caption{(a,b) Real and imaginary parts of three complex zeros of function $F$ for fixed $\\gamma=3$ and increasing $\\ell$. For each curve, the imaginary part is positive for small $\\ell$ and becomes negative for all sufficiently large $\\ell$. \\textcolor{black}{For each three shown eigenvalues, its real and \timaginary parts are of the same colour in both panels.}}\n \t\\label{fig:g=3}\n \\end{figure}\n\n\nThe second group of spectral singularities [blue curves in figure~\\ref{fig:general}(a)] was obtained by means of the continuation from the  next solution in the limit $\\ell=0$, i.e.  from $\\gamma_\\star^{(1)}$ and $k_\\star^{(1)}$ in Table~\\ref{tbl:1}. Again, one of the curves [the leftmost bold curve in figure~\\ref{fig:general}(a)] was obtained through the direct continuation from the limit $\\ell=0$, while other blue curves were generated  using the periodicity of function $F(k, \\gamma, \\ell)$  in $\\ell$ and cannot be continued to the limit $\\ell\\to0$. In the limit $\\ell\\to\\infty$ the blue curves approach the horizontal asymptotes  $\\gamma=2\\pi^2\\approx19.739$ and $\\gamma=9\\pi^2/2\\approx 44.413$. In  the $(\\gamma, \\ell)$-plane  the  gain-and-loss amplitudes corresponding to the   group of blue curves are well separated from  those for the red curves: indeed, all red curves are bounded from above by the asymptote $\\gamma= \\pi^2/2\\approx 4.935$, while all blue curves are bounded from below by $\\gamma_*\\approx 11.561$, see (\\ref{eq:gap}). Thus, the  emergence of new spectral singularities with the increase of $\\ell$ is sensitive to the value of the gain-and-loss amplitude and does not occur for     the gain-and-loss amplitudes lying in the  gap  between the red and blue curves.\n\nIn comparison with the red curves discussed above, the curves from the blue group in figure~\\ref{fig:general}(a) feature more complicated behavior and, in particular, can intersect each other (and also intersect the curves from the next, third group of green curves discussed below). The intersections between the blue curves occur for the gain-and-loss amplitudes in the interval $11.561 \\lessapprox   \\gamma <  9\\pi^2/2\\approx 19.739$, where $\\sin\\sqrt{2\\gamma}$ is negative.  \\textcolor{black}{At first glance,} this might seem to contradict to  the  expansion (\\ref{3.4}) of Lemma~\\ref{lm3.4},  which  suggests that in this case the multitude of complex zeros   accumulate in the upper complex half-plane with the growth of $\\ell$. However, this apparent contradiction is resolved if we trace the behavior of the complex roots more closely.   Indeed, choosing for an example $\\gamma =16$ [see the vertical axis tick in  figure~\\ref{fig:general}(a)] and computing several  complex roots under the increase of $\\ell$, we observe that each considered root first goes down from the upper half-plane to the lower one but then again returns to the upper half-plane, see Figure~\\ref{fig:g=16}(b) and (b$_1$). Thus,  in  this  interval of the gain-and-loss amplitudes the increase of $\\ell$  results  either in the transformation from the resonance to the eigenvalue or to the opposite process,  i.e. to the disappearance of the complex-conjugate pair.  Respectively, the intersection between the two blue curves corresponds to  two coexisting spectral singularities, i.e. to the moment  when one complex-conjugate pair of eigenvalues disappears and another pair (with different $k$) emerges. For sufficiently large $\\ell$ the imaginary part  of each   considered root  remains positive and approaches zero, in accordance with expansion (\\ref{3.4}). Thus, in this interval of the gain-and-loss strengths, the limit $\\ell\\to\\infty$ the spectrum contains only a finite number of complex-conjugate eigenvalues, which correspond to complex zeros $k$ whose behavior is not covered   by expansion  (\\ref{3.4}).\n\n \\begin{figure}\n\t\\centering\n\t\\includegraphics[width=0.99\\columnwidth]{fig08.eps}\n\t\\caption{(a,b) Real and imaginary parts for three   complex zeros of function $F$ for fixed $\\gamma=16$ and increasing $\\ell$. Panel b$_1$ is the magnification of some region of (b) and  shows more clearly that imaginary parts of all   three shown eigenvalues are positive for all sufficiently large  $\\ell$. \\textcolor{black}{For each three shown eigenvalues, its real and \timaginary parts are of the same colour in both panels.}}\n\t\\label{fig:g=16}\n\\end{figure}\n\nThe third  group of spectral singularities [green curves in figure~\\ref{fig:general}(a)] was obtained by means of the continuation from  the solution $\\gamma_\\star^{(2)}$ and $k_\\star^{(2)}$ in Table~\\ref{tbl:1}. In view of the very complicated structure of the overall resulting picture, we only show a section of these curves corresponding to relatively small   values of the gain-and-loss amplitudes $\\gamma$. Quite interestingly, there is no gap between the blue and green groups of curves, which results in the multitude of intersections between blue and green curves, \\textcolor{black}{see Fig.~\\ref{fig:general}(a) and the magnified view in Fig.~\\ref{fig:general}(c)}. These   intersections suggest a possibility of   simultaneous emergence of two complex-conjugate pairs of eigenvalues from two different interior points of the continuous spectra.\n\nConsidering further solutions $\\gamma_\\star^{(n)}$, $k_\\star^{(n)}$, $n=3,4,\\ldots$, in the limit $\\ell=0$ one can construct new groups of spectral singularities  with larger values of $\\gamma$, which are not shown in Figure~\\ref{fig:general}.\n\n\n",
        "label": "fig:g=16",
        "Descriptive_question1": "What is the fixed value of gamma shown in figure_8?",
        "Descriptive_question2": "What color coding is used for the eigenvalues in figure_8?",
        "Reasoning_question1": "What trend can be observed in the imaginary parts of the complex zeros in figure_8 as the parameter ell increases?",
        "Reasoning_question2": "How does the behavior of the imaginary parts in figure_8 panel b1 illustrate the impact of large ell values on the eigenvalues?",
        "Descriptive_answer1": "16",
        "Descriptive_answer2": "same colour",
        "Reasoning_answer1": "As the parameter ell increases, the imaginary parts of the complex zeros initially change sign from positive to negative and then eventually become positive again for sufficiently large ell. This indicates that the eigenvalues move through the complex plane in a non-monotonic manner with respect to ell.",
        "Reasoning_answer2": "In panel b1, the magnification clearly shows that the imaginary parts of all three eigenvalues remain positive when ell is sufficiently large. This demonstrates that at large ell values, the eigenvalues stabilize in the upper half of the complex plane, suggesting that large separation supports eigenvalues with positive imaginary parts, consistent with the asymptotic analysis."
    },
    {
        "paper_id": "1511.04027.json",
        "image_id": "figure_2",
        "image_path": [
            "/home/yz979/scratch/chengye/scimolmo/test-images-tables/dataset_imagestables/1511.04027/images/R0SAnew11.jpg"
        ],
        "caption": "Percent change in $\\mathcal{R}_c$ with respect to a $1\\%$ change in the parameter value, for a low value of $f_T$ ($f_T = 0.25$) and two different levels of isolation effectiveness ($r = 0.35$ and $r = 0.95$). The other parameter values are kept as shown in Table \\ref{tab:ParamsDef}.",
        "classify": "Chart",
        "section_info": "3 \\protect\\normalsize Model analysis\n\\section{\\protect\\normalsize Model analysis}\n\n \\subsection{\\protect\\normalsize Basic properties}\nSince model (\\ref{model1}) imitates the dynamics of human populations, all variables and parameters should be non-negative. Thus, following the approach shown in appendix A of [\\ref{lit:HTh}], we show the following result.\n\\begin{theorem}\nThe variables of model (\\ref{model1}) are non-negative for all time.\n\\end{theorem}\n\n\\begin{lemma}\nThe closed set \n\\begin{equation*}\n\\Omega = \\big\\lbrace (S, E_1, E_2, I, J, R)\\in \\mathbb{R}_+^6: \\frac{\\Lambda}{\\mu + q_1\\gamma + q_2\\gamma_r}\\le{S + E_1 + E_2 + I + J + R} \\le \\frac{\\Lambda}{\\mu}  \\big\\rbrace\n\\end{equation*}\nis positively invariant for model (\\ref{model1}) and is absorbing.\n\\end{lemma}\n\\noindent Proof: Equation (\\ref{Neq}) implies that\n\\begin{eqnarray}\n\\frac{dN}{dt} &\\le& \\Lambda - \\mu N, \\label{Neq1}\\\\\n\\frac{dN}{dt} &\\ge& \\Lambda - (\\mu+q_1\\gamma+q_2\\gamma_r) N. \\label{Neq2}\n\\end{eqnarray}\nIt follows from (\\ref{Neq1}) that \n\\begin{equation}\nN(t) \\le  \\frac{\\Lambda}{\\mu} + \\left(N(0) -\\frac{\\Lambda}{\\mu} \\right) e^{- \\mu t}\\label{ineq1}\n\\end{equation}\nand from (\\ref{Neq2}) that \n\\begin{equation}\nN(t)\\ge\\frac{\\Lambda}{\\mu + q_1\\gamma + q_2\\gamma_r} + \\left(N(0) -\\frac{\\Lambda}{\\mu + q_1\\gamma + q_2\\gamma_r} \\right) e^{-(\\mu + q_1\\gamma + q_2\\gamma_r)t}. \\label{ineq2}\n\\end{equation}\nIf we assume $N(0) > \\Lambda/\\mu$, then $dN/dt < 0$ and therefore (based on inequality (\\ref{ineq1})), $N(t)$ decreases steadily until reaching $\\Lambda/\\mu$ when $t$ tends to $\\infty$. Similarly, if we assume $N(0) < \\Lambda/(\\mu + q_1\\gamma + q_2\\gamma_r)$, then $dN/dt > 0$ and therefore  (based on inequality (\\ref{ineq2})), $N(t)$ increases steadily until reaching a maximum at $\\Lambda/(\\mu + q_1\\gamma + q_2\\gamma_r)$ when $t$ tends to $\\infty$. It remains to check the case if $N(0)$ lies in the phase between $\\Lambda/(\\mu + q_1\\gamma + q_2\\gamma_r)$ and $\\Lambda/\\mu$. To this end, both inequalities (\\ref{ineq1}) and (\\ref{ineq2}) are combined together to get \n\\begin{equation*}\n\\frac{\\Lambda}{\\mu + q_1\\gamma + q_2\\gamma_r} + \\left(N(0) -\\frac{\\Lambda}{\\mu + q_1\\gamma + q_2\\gamma_r} \\right) e^{-(\\mu + q_1\\gamma + q_2\\gamma_r)t}\\le N(t) \\le \\frac{\\Lambda}{\\mu} + \\left(N(0) -\\frac{\\Lambda}{\\mu} \\right) e^{- \\mu t}.\n\\end{equation*}\nOn taking the limit when $t$ tends to $\\infty$, we find that $N(t)$ remains within the same phase. Thus, the set $\\Omega$ is positively invariant and absorbing.\n\n \\subsection{\\protect\\normalsize Equilibrium analysis}\n \\subsubsection*{\\protect\\normalsize Ebola-free equilibrium and the control reproduction number $\\mathcal{R}_c$}\nIt is easy to check that model (\\ref{model1}) has the Ebola-free equilibrium \n\\begin{equation}\nE_0 = \\left(\\frac{\\Lambda}{\\mu}, 0, 0, 0, 0, 0\\right)^{\\prime}\n\\end{equation}\nwhere the prime `` ${}^{\\prime}$ '' means vector transpose.\\newline \n\\indent The basic reproduction number,  $\\mathcal{R}_0$,  is a measure of the average number of secondary cases produced by a typical infectious individual during the entire course of infection in a completely susceptible population and in the absence of control interventions [\\ref{lit:  BrauerF},\\ref{lit: AndersonRM}]. On the other hand, the control reproduction number, $\\mathcal{R}_c$, quantifies the potential for infectious disease transmission in the context of a partially susceptible population due to the implementation of control interventions. When $\\mathcal{R}_c > 1$, the infection may spread in the population, and the rate of spread is higher with increasingly high values of $\\mathcal{R}_c$. If $\\mathcal{R}_c < 1$, infection cannot be sustained and is unable to generate an epidemic.  For our model, $\\mathcal{R}_c$  is computed using the next generation matrix approach shown in [\\ref{lit:PVDDJW2002}].  Accordingly, we compute the matrices $\\mathbf{F}$ (for the new infection terms) and $\\mathbf{V}$ (for the\ntransition terms) as\n\n\\begin{eqnarray*}\n  \\mathbf{F} = \\left(\\begin{array}{cccc}\n    0 & 0 & \\beta & (1-r) \\ell \\beta \\\\\n    0 & 0 & 0 & 0 \\\\\n    0 & 0 & 0 & 0 \\\\\n    0 & 0 & 0 & 0 \\\\\n  \\end{array}\\right) \\quad, \\quad\n   \\mathbf{V} = \\left(\\begin{array}{cccc}\n    \\kappa_1 + \\mu & 0 & 0 & 0 \\\\\n    -\\kappa_1 & \\kappa_2 + \\mu  & 0 & 0\\\\\n    0 & -(1-f_T) \\kappa_2 & \\alpha+\\gamma+\\mu & 0\\\\\n    0 & - f_T \\kappa_2  & - \\alpha & \\gamma_r + \\mu\\\\\n  \\end{array}\\right).\n\\end{eqnarray*}\n\\noindent Thus, the control reproduction number is given by\n\\begin{eqnarray}\n\\mathcal{R}_c & = &\\rho(\\mathbf{F}\\mathbf{V}^{-1}) = \\frac{\\kappa_1 \\kappa_2 \\beta[(1-f_T) (\\mu + \\gamma_r) + (1-r)\\ell(\\alpha+f_T(\\gamma+\\mu))]}{(\\kappa_1 + \\mu)(\\kappa_2 + \\mu)(\\alpha+\\gamma+\\mu)(\\gamma_r + \\mu)} \\nonumber\\\\\n& = & \\frac{\\kappa_1\\kappa_2\\beta}{(\\kappa_1 + \\mu)(\\kappa_2 + \\mu)(\\alpha+\\gamma+\\mu)}\\left[1 - f_T + (1-r) \\ell \\left( \\frac{\\alpha}{\\gamma_r+\\mu} + f_T  \\frac{\\gamma + \\mu}{\\gamma_r+\\mu} \\right) \\right]\\nonumber\\\\\n& = & \\mathcal{R}_0\\left[1-\\frac{\\alpha}{(\\alpha+\\gamma+\\mu)}\\right] \\left[1 - f_T + (1-r) \\ell \\left( \\frac{\\alpha}{\\gamma_r+\\mu} + f_T  \\frac{\\gamma + \\mu}{\\gamma_r+\\mu} \\right) \\right]\\label{R0eq}\n\\end{eqnarray}\nwhere $\\rho$ is the spectral radius (dominant eigenvalue in\nmagnitude) of the matrix $\\mathbf{F}\\mathbf{V}^{-1}$ and \n\\begin{equation}\n\\mathcal{R}_0 = \\frac{\\kappa_1\\kappa_2\\beta}{(\\kappa_1 + \\mu)(\\kappa_2 + \\mu)(\\gamma+\\mu)}\n\\end{equation}\nis the basic reproduction number for the model.\n\n\\indent The local stability of the Ebola-free equilibrium, $E_0$, for values of $\\mathcal{R}_c < 1$ is established based on a direct use of Theorem 2 in [\\ref{lit:PVDDJW2002}]. We summarize our result in the following lemma.\n\\begin{lemma}\nThe Ebola-free equilibrium $E_0$ of model (\\ref{model1}) is locally asymptotically stable if and only if $\\mathcal{R}_c < 1$.\n\\end{lemma}\n\n\n\\subsubsection*{\\protect\\normalsize Ebola-endemic equilibrium}\nOn putting the derivatives in the left hand side of (\\ref{model1}) equal zero and solving the resulting algebraic system with respect to the variables $\\bar{S}, \\bar{E}_1, \\bar{E}_2, \\bar{I}, \\bar{J}$, and $\\bar{R}$, we obtain\n \\begin{eqnarray}\n\\bar{S} & = & \\frac{\\Lambda}{\\bar\\lambda + \\mu},\\nonumber\\\\\n\\bar{E}_1 & = & \\frac{\\Lambda}{\\bar\\lambda + \\mu} \\cdot \\frac{\\bar\\lambda}{\\kappa_1 + \\mu},\\nonumber\\\\\n\\bar{E}_2 & = & \\frac{\\kappa_1}{\\kappa_2 + \\mu}\\cdot \\frac{\\Lambda}{\\bar\\lambda + \\mu} \\cdot \\frac{\\bar\\lambda}{\\kappa_1 + \\mu},\\nonumber \\\\\n\\bar{I} & = & \\frac{(1-f_T)\\kappa_2}{\\alpha+\\gamma + \\mu} \\cdot \\frac{\\kappa_1}{\\kappa_2 + \\mu}\\cdot \\frac{\\Lambda}{\\bar\\lambda + \\mu} \\cdot \\frac{\\bar\\lambda}{\\kappa_1 + \\mu},\\label{eqvar} \\\\\n\\bar{J} & = &  \\frac{\\kappa_1}{\\kappa_2 + \\mu}\\cdot \\frac{\\Lambda}{\\bar\\lambda + \\mu} \\cdot \\frac{\\bar\\lambda}{\\kappa_1 + \\mu} \\cdot \\frac{\\kappa_2}{\\gamma_r + \\mu} \\left[f_T + (1-f_T) \\frac{\\alpha}{\\alpha+\\gamma + \\mu} \\right], \\nonumber\\\\\n\\bar{R} & = & \\frac{1}{\\mu}[(1-q_1)\\gamma I + (1-q_2) \\gamma_r J]\\nonumber\n\\end{eqnarray}\nwhere\n\\begin{equation}\n\\bar\\lambda = \\frac{\\beta(I + (1-r)\\ell \\bar{J})}{\\bar{N} - r \\bar{J}}\\label{lambda}\n\\end{equation}\nis the equilibrium force of infection. On substituting from (\\ref{eqvar}) into (\\ref{lambda}) and simplifying (with the assumption that $\\lambda \\ne 0$), we get\n \\begin{equation}\n\\bar\\lambda = \\frac{\\mu(\\mathcal{R}_c - 1)}{1 - Term}\n\\end{equation}\nwhere \n \\begin{equation*}\nTerm = \\frac{\\kappa_1 \\kappa_2 [q_1(1-f_T)\\gamma(\\gamma_r + \\mu) + (r\\mu + q_2\\gamma_r)(f_T(\\gamma + \\mu) + \\alpha)]}{(\\kappa_1 + \\mu)(\\kappa_2 + \\mu)(\\alpha+\\gamma+\\mu)(\\gamma_r + \\mu)}.\n\\end{equation*}\nHence, the Ebola-endemic equilibrium is unique and we show the following lemma.\n \\begin{lemma}\n Model (\\ref{model1}) has a unique endemic equilibrium that exists if and only if $\\mathcal{R}_c > 1$.\n \\end{lemma}\n \n\n\n\n\n\n\n\\subsection{\\protect\\normalsize Normalized sensitivity analysis on $\\mathcal{R}_c$ }\\label{sensitivity}\n\n\n\nIn considering the dynamics of the Ebola system (\\ref{model1}), we conduct normalized sensitivity analysis on $\\mathcal{R}_c$ to determine the impact of parameter perturbations on the transmission dynamics of the system. By computing the normalized sensitivity indices, we consider the percent change in the output with respect to a percent change in the parameter input. Those parameters with the largest magnitude of change impact the compartment model the most; the sign indicates whether the change produces an increase or a decrease on $\\mathcal{R}_c$.\\newline\n\\indent The normalized sensitivity indices for $\\mathcal{R}_c$ are calculated by taking the partial derivative of $\\mathcal{R}_c$ with respect to each parameter and multiply the derivative with the ratio of the parameter to $\\mathcal{R}_c$. This value represents the percent change in $\\mathcal{R}_c$ with respect to a 1\\% change in the parameter value [\\ref{lit: CaswellH}]. \\newline\n \n\\vspace{-5mm}\n\n\\begin{table}[h!]\n\\caption{Percent change in $\\mathcal{R}_c$ with respect to a $1\\%$ change in the parameter value, for a low and a high isolation effectiveness $r$, and a low and a high value of $f_T$, while keeping the other parameter values as presented in Table \\ref{tab:ParamsDef}.} \\label{r0change1}\n\\vspace{-6mm}\n\\begin{center}\n\\begin{tabular}{|c|c|c|c|c|c|c|c|c|}\n\\hline\n&Parameter & $\\beta$ & $r$ & $\\ell$ & $\\gamma_r$ & $\\gamma$ & $\\alpha$ & $f_T$ \\\\\n\\hline\n\n &\\% change & 1\\% & -0.23\\% & 0.423\\%  & -0.423\\% & -0.382\\% & -0.195\\% & -0.119\\% \\\\\n $f_T = 0.25$& for $r = 0.35$ &  &  &    &   &   &   &  \\\\\n\\cline{2-9}\n&\\% change & 1\\% & -1.014\\% & 0.053\\%  & -0.053\\% & -0.445\\% & -0.501\\% & -0.306\\% \\\\\n  & for $r = 0.95$ &  &  &    &   &   &   &  \\\\\n\\hline\n&\\% change & 1\\% & -0.402\\% & 0.747\\%  & -0.747\\% & -0.167\\% & -0.086\\% & -0.471\\% \\\\\n $f_T = 0.75$& for $r = 0.35$ &  &  &    &   &   &   &  \\\\\n\\cline{2-9}\n&\\% change & 1\\% & -3.521\\% & 0.185\\%  & -0.185\\% & -0.383\\% & -0.431\\% & -2.373\\% \\\\\n  & for $r = 0.95$ &  &  &    &   &   &   &  \\\\\n\\hline\n\\end{tabular}\n\\end{center}\n\\end{table}\n\n\n\n\n\\begin{figure}[h!]\n\\begin{center}\n\\includegraphics[scale=0.57]{R0SAnew11.jpg}\n\\caption{Percent change in $\\mathcal{R}_c$ with respect to a $1\\%$ change in the parameter value, for a low value of $f_T$ ($f_T = 0.25$) and two different levels of isolation effectiveness ($r = 0.35$ and $r = 0.95$). The other parameter values are kept as shown in Table \\ref{tab:ParamsDef}.} \\label{graphofR01}\n\\end{center}\n\\end{figure}\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\\indent We use the parameters values from Table \\ref{tab:ParamsDef} to study the sensitivity of $\\mathcal{R}_c$ to each parameter. We compute normalized sensitivity analysis on all parameters, but we just consider the impact of parameters that are the most sensitive: $\\beta, r, \\ell, \\gamma_r, \\gamma, \\alpha$, and $f_T$. The other parameters ($\\mu, \\kappa_1$, and  $\\kappa_2$)  have a very low impact, namely less than $0.001\\%$. The numerical simulations to the sensitivity of $\\mathcal{R}_c$ with respect to each of the most sensitive parameters are given in Table \\ref{r0change1}, for two different levels of isolation effectiveness ($r = 0.35$ and $r = 0.95$) and two values of $f_T$ ($f_T = 0.25$ and $f_T = 0.75$), which is the fraction of pre-symptomatic individuals diagnosed and isolated. The other parameter values are kept as shown in Table \\ref{tab:ParamsDef}.\n\\newline\n\\indent A graphical illustration of the numerical results for the scenario when $f_T = 0.25$ and the two levels of isolation effectiveness ($r = 0.35$ and $r = 0.95$) is given in Figure \\ref{graphofR01}. In the case of high isolation effectiveness ($r = 0.95$), simulations show that both the removal rate, $\\gamma_r$, of isolated individuals and the relative transmissibility parameter $\\ell$ of isolated individuals with respect to infectious individuals are the least sensitive parameters (with $0.053 \\%$ change of $\\mathcal{R}_c$), while the parameter of isolation effectiveness, $r$, is the most sensitive one, where a $1\\%$ increase in $r$ causes a $1.014 \\%$ reduction in the value of $\\mathcal{R}_c$. Also, the rate at which infectious individuals get isolated, $\\alpha$, and the fraction of pre-symptomatic individuals detected and isolated, $f_T$, impact negatively on the level of $\\mathcal{R}_c$, where a $1 \\%$ percent increase in the value of $f_T$ causes approximately a $0.31\\%$ decline in the value of the reproduction number $\\mathcal{R}_c$. Thus, as pre-symptomatic individuals are diagnosed and as isolation is highly effective, the number of available infectious individuals who are capable of transmitting Ebola decreases and therefore, the reproduction number decreases. Also, the removal (by recovery or Ebola-induced death) rate $\\gamma$ of infectious individuals affects negatively on $\\mathcal{R}_c$. Hence, for the case of highly effective isolation, the parameters concerning early diagnosis and isolation have a significant impact on the reproduction number. \n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\\indent This percent impact of the parameters on $\\mathcal{R}_c$ remains so as long as isolation is highly effective. However, if the effectiveness of isolation is low, in the sense that all parameter values are kept the same except the value of the parameter $r$, which is reduced to $0.35$, then we get the results presented in Table \\ref{r0change1} and Figure \\ref{graphofR01}. In this case, both the relative transmissibility $\\ell$ and the removal rate of isolated individuals, $\\gamma_r$, are the second most sensitive parameters, after $\\beta$ which is the most impactful one. Also, $\\ell$ became more sensitive than $r$. The implication is that, when isolation is less effective, there exists the possibility for isolated people to make successful contacts with susceptible individuals and therefore the possibility of causing new infections increases. This causes an increase in the reproduction number. Also, it is noted that the effect of $f_T$ and $\\alpha$ is reduced, which means that diagnosing and isolating infected individuals becomes a weak strategy if the effectiveness of isolation is low.\n \n\n\n\\begin{figure}[h!]\n\\begin{center}\n\\includegraphics[scale=0.585]{R0SAnew12.jpg}\n\\caption{Percent change in $\\mathcal{R}_c$ with respect to a $1 \\%$ change in the parameter value, for a high value of $f_T$ ($f_T = 0.75$) and two different levels of isolation effectiveness ($r = 0.35$ and $r = 0.95$). The other parameter values are kept as shown in Table \\ref{tab:ParamsDef}.} \\label{graphofR02}\n\\end{center}\n\\end{figure}\n\nOn repeating the previous analyses, but this time for a higher value of $f_T$ ($f_T = 0.75$), we obtain the results shown in Table  \\ref{r0change1}, which are also illustrated in Figure \\ref{graphofR02}. In comparison to the scenario when $f_T = 0.25$, the simulations show that increasing the fraction of pre-symptomatic individuals who are diagnosed and isolated,  $f_T$, increases the percent impact of the parameters $r, \\ell, \\gamma_r,$ and $f_T$, and decreases the percent impact of the parameters $\\gamma$ and $\\alpha$, on the value of the control reproduction number $\\mathcal{R}_c$.   \n\n \n\n  \n\n\n\\subsection{\\protect\\normalsize Impact of early detection and isolation on the value of $\\mathcal{R}_c$}\n\n \\begin{figure}[H]\n\\centering\n\\includegraphics[scale=0.55]{R0fT.jpg}\n\\caption{Impact of early detection of pre-symptomatic individuals on the value of $\\mathcal{R}_c$.}\n\\label{fig:R0fT}\n\\end{figure}\n\n\nTo study the impact of early detection of pre-symptomatic individuals and isolation on the reproduction number, we first depict $\\mathcal{R}_c$ as a function of $f_T$, for different levels of isolation effectiveness $r$. Figure \\ref{fig:R0fT} shows that the control reproduction number declines as the proportion, $f_T$, of pre-symptomatic individuals, who get diagnosed and isolated, increases.  Simulations are done using parameter values from Table \\ref{tab:ParamsDef}, but for three different values of $r$. It further shows that the curve corresponding to a low and an intermidate value of isolation effectivenes $r$ (e.g. $r = 0.35$ for the solid curve and $r = 0.65$ for the dashed curve) hits $\\mathcal{R}_c = 1$ at some critical value of $f_T$ (say $f_T^{\\star}$), while for the high value of $r$ ($r = 0.95$), it never hits the critical threshold $\\mathcal{R}_c = 1$, as the curve is totally below the critical threshold. This indicates that for a high effectiveness of isolation, the control reproduction number is less than one and therefore the infection dies out. Analytically, the exact form of $f_T^{\\star}$ is \n\\begin{equation}\nf_T^{\\star} = \\left[ 1 + (1-r)\\ell \\frac{\\alpha}{\\gamma_r + \\mu} - \\frac{1}{\\mathcal{R}_0} \\left(1 + \\frac{\\alpha}{\\gamma+\\mu}\\right) \\right] / \\left[ 1 - \\frac{(1-r)\\ell(\\gamma + \\mu)}{\\gamma_r + \\mu} \\right].  \\label{fTcond1}\n\\end{equation}\nThe critical proportion $f_T^{\\star}$ represents the minimum proportion of pre-symptomatic individuals who are detected and get isolated to ensure an effective control of Ebola. This critical value remains feasible as long as the following inequality holds\n\n\n\n\\begin{equation}\n(1-r)\\ell < \\frac{\\gamma_r + \\mu}{(\\gamma + \\mu)\\mathcal{R}_0}. \\label{fTcond20}\n\\end{equation}\nIf we keep all parameters fixed except $r$, then condition (\\ref{fTcond20}) could be rewritten in a more convenient form\n\n\n\n\\begin{equation}\nr > 1- \\frac{\\gamma_r + \\mu}{\\ell (\\gamma + \\mu) \\mathcal{R}_0}. \\label{fTcond3}\n\\end{equation}\nThis gives the minimum level of effectiveness of isolation required to obtain an isolation and early diagnosis-based control strategy for Ebola tranmission. \n\n\n\n\n \\begin{figure}[H]\n\\centering\n\\includegraphics[scale=0.55]{R0alpha.jpg}\n\\caption{Impact of isolating infectious individuals on the value of $\\mathcal{R}_c$.}\n\\label{fig:R0alpha}\n\\end{figure}\n\nNow, we could also ask a similar question on the role of isolating infectious individuals to contain Ebola transmission. Figure \\ref{fig:R0alpha} shows the impact of changing the rate at which infectious individuals get isolated, $\\alpha$, on $\\mathcal{R}_c$, for the same three different levels of isolation effectivenes, as used above. The analysis shows that it is possible to control the epidemic if and only if $\\alpha > \\alpha^\\star$, where\n\\begin{equation}\n\\alpha^\\star = \\frac{[ (1-f_T)(\\gamma_r + \\mu)(\\gamma+\\mu) + (1-r)\\ell f_T (\\gamma+\\mu)^2]\\mathcal{R}_0 -  (\\gamma_r + \\mu)(\\gamma+\\mu) }{(\\gamma_r + \\mu) - \\ell (1-r) \\mathcal{R}_0(\\gamma + \\mu)}\n\\end{equation}\n\n\n\n\nand with the implementation of condition (\\ref{fTcond20}).\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n3.3 \\protect\\normalsize Normalized sensitivity analysis on $\\mathcal{R}_c$\n\\subsection{\\protect\\normalsize Normalized sensitivity analysis on $\\mathcal{R}_c$ }\\label{sensitivity}\n\n\n\nIn considering the dynamics of the Ebola system (\\ref{model1}), we conduct normalized sensitivity analysis on $\\mathcal{R}_c$ to determine the impact of parameter perturbations on the transmission dynamics of the system. By computing the normalized sensitivity indices, we consider the percent change in the output with respect to a percent change in the parameter input. Those parameters with the largest magnitude of change impact the compartment model the most; the sign indicates whether the change produces an increase or a decrease on $\\mathcal{R}_c$.\\newline\n\\indent The normalized sensitivity indices for $\\mathcal{R}_c$ are calculated by taking the partial derivative of $\\mathcal{R}_c$ with respect to each parameter and multiply the derivative with the ratio of the parameter to $\\mathcal{R}_c$. This value represents the percent change in $\\mathcal{R}_c$ with respect to a 1\\% change in the parameter value [\\ref{lit: CaswellH}]. \\newline\n \n\\vspace{-5mm}\n\n\\begin{table}[h!]\n\\caption{Percent change in $\\mathcal{R}_c$ with respect to a $1\\%$ change in the parameter value, for a low and a high isolation effectiveness $r$, and a low and a high value of $f_T$, while keeping the other parameter values as presented in Table \\ref{tab:ParamsDef}.} \\label{r0change1}\n\\vspace{-6mm}\n\\begin{center}\n\\begin{tabular}{|c|c|c|c|c|c|c|c|c|}\n\\hline\n&Parameter & $\\beta$ & $r$ & $\\ell$ & $\\gamma_r$ & $\\gamma$ & $\\alpha$ & $f_T$ \\\\\n\\hline\n\n &\\% change & 1\\% & -0.23\\% & 0.423\\%  & -0.423\\% & -0.382\\% & -0.195\\% & -0.119\\% \\\\\n $f_T = 0.25$& for $r = 0.35$ &  &  &    &   &   &   &  \\\\\n\\cline{2-9}\n&\\% change & 1\\% & -1.014\\% & 0.053\\%  & -0.053\\% & -0.445\\% & -0.501\\% & -0.306\\% \\\\\n  & for $r = 0.95$ &  &  &    &   &   &   &  \\\\\n\\hline\n&\\% change & 1\\% & -0.402\\% & 0.747\\%  & -0.747\\% & -0.167\\% & -0.086\\% & -0.471\\% \\\\\n $f_T = 0.75$& for $r = 0.35$ &  &  &    &   &   &   &  \\\\\n\\cline{2-9}\n&\\% change & 1\\% & -3.521\\% & 0.185\\%  & -0.185\\% & -0.383\\% & -0.431\\% & -2.373\\% \\\\\n  & for $r = 0.95$ &  &  &    &   &   &   &  \\\\\n\\hline\n\\end{tabular}\n\\end{center}\n\\end{table}\n\n\n\n\n\\begin{figure}[h!]\n\\begin{center}\n\\includegraphics[scale=0.57]{R0SAnew11.jpg}\n\\caption{Percent change in $\\mathcal{R}_c$ with respect to a $1\\%$ change in the parameter value, for a low value of $f_T$ ($f_T = 0.25$) and two different levels of isolation effectiveness ($r = 0.35$ and $r = 0.95$). The other parameter values are kept as shown in Table \\ref{tab:ParamsDef}.} \\label{graphofR01}\n\\end{center}\n\\end{figure}\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\\indent We use the parameters values from Table \\ref{tab:ParamsDef} to study the sensitivity of $\\mathcal{R}_c$ to each parameter. We compute normalized sensitivity analysis on all parameters, but we just consider the impact of parameters that are the most sensitive: $\\beta, r, \\ell, \\gamma_r, \\gamma, \\alpha$, and $f_T$. The other parameters ($\\mu, \\kappa_1$, and  $\\kappa_2$)  have a very low impact, namely less than $0.001\\%$. The numerical simulations to the sensitivity of $\\mathcal{R}_c$ with respect to each of the most sensitive parameters are given in Table \\ref{r0change1}, for two different levels of isolation effectiveness ($r = 0.35$ and $r = 0.95$) and two values of $f_T$ ($f_T = 0.25$ and $f_T = 0.75$), which is the fraction of pre-symptomatic individuals diagnosed and isolated. The other parameter values are kept as shown in Table \\ref{tab:ParamsDef}.\n\\newline\n\\indent A graphical illustration of the numerical results for the scenario when $f_T = 0.25$ and the two levels of isolation effectiveness ($r = 0.35$ and $r = 0.95$) is given in Figure \\ref{graphofR01}. In the case of high isolation effectiveness ($r = 0.95$), simulations show that both the removal rate, $\\gamma_r$, of isolated individuals and the relative transmissibility parameter $\\ell$ of isolated individuals with respect to infectious individuals are the least sensitive parameters (with $0.053 \\%$ change of $\\mathcal{R}_c$), while the parameter of isolation effectiveness, $r$, is the most sensitive one, where a $1\\%$ increase in $r$ causes a $1.014 \\%$ reduction in the value of $\\mathcal{R}_c$. Also, the rate at which infectious individuals get isolated, $\\alpha$, and the fraction of pre-symptomatic individuals detected and isolated, $f_T$, impact negatively on the level of $\\mathcal{R}_c$, where a $1 \\%$ percent increase in the value of $f_T$ causes approximately a $0.31\\%$ decline in the value of the reproduction number $\\mathcal{R}_c$. Thus, as pre-symptomatic individuals are diagnosed and as isolation is highly effective, the number of available infectious individuals who are capable of transmitting Ebola decreases and therefore, the reproduction number decreases. Also, the removal (by recovery or Ebola-induced death) rate $\\gamma$ of infectious individuals affects negatively on $\\mathcal{R}_c$. Hence, for the case of highly effective isolation, the parameters concerning early diagnosis and isolation have a significant impact on the reproduction number. \n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\\indent This percent impact of the parameters on $\\mathcal{R}_c$ remains so as long as isolation is highly effective. However, if the effectiveness of isolation is low, in the sense that all parameter values are kept the same except the value of the parameter $r$, which is reduced to $0.35$, then we get the results presented in Table \\ref{r0change1} and Figure \\ref{graphofR01}. In this case, both the relative transmissibility $\\ell$ and the removal rate of isolated individuals, $\\gamma_r$, are the second most sensitive parameters, after $\\beta$ which is the most impactful one. Also, $\\ell$ became more sensitive than $r$. The implication is that, when isolation is less effective, there exists the possibility for isolated people to make successful contacts with susceptible individuals and therefore the possibility of causing new infections increases. This causes an increase in the reproduction number. Also, it is noted that the effect of $f_T$ and $\\alpha$ is reduced, which means that diagnosing and isolating infected individuals becomes a weak strategy if the effectiveness of isolation is low.\n \n\n\n\\begin{figure}[h!]\n\\begin{center}\n\\includegraphics[scale=0.585]{R0SAnew12.jpg}\n\\caption{Percent change in $\\mathcal{R}_c$ with respect to a $1 \\%$ change in the parameter value, for a high value of $f_T$ ($f_T = 0.75$) and two different levels of isolation effectiveness ($r = 0.35$ and $r = 0.95$). The other parameter values are kept as shown in Table \\ref{tab:ParamsDef}.} \\label{graphofR02}\n\\end{center}\n\\end{figure}\n\nOn repeating the previous analyses, but this time for a higher value of $f_T$ ($f_T = 0.75$), we obtain the results shown in Table  \\ref{r0change1}, which are also illustrated in Figure \\ref{graphofR02}. In comparison to the scenario when $f_T = 0.25$, the simulations show that increasing the fraction of pre-symptomatic individuals who are diagnosed and isolated,  $f_T$, increases the percent impact of the parameters $r, \\ell, \\gamma_r,$ and $f_T$, and decreases the percent impact of the parameters $\\gamma$ and $\\alpha$, on the value of the control reproduction number $\\mathcal{R}_c$.   \n\n \n\n  \n\n\n",
        "label": "graphofR01",
        "Descriptive_question1": "What is the percent change in R_c for a 1% increase in the parameter beta in figure_2?",
        "Descriptive_question2": "What is the value of isolation effectiveness r for high effectiveness in figure_2?",
        "Reasoning_question1": "How does the sensitivity of R_c to the isolation effectiveness parameter r compare between low and high isolation effectiveness in figure_2, and what might this imply about the control of Ebola transmission?",
        "Reasoning_question2": "Why might the parameter beta have a consistently high impact on R_c across different levels of isolation effectiveness in figure_2, and how does this relate to the transmission dynamics of Ebola?",
        "Descriptive_answer1": "1%",
        "Descriptive_answer2": "0.95",
        "Reasoning_answer1": "The sensitivity analysis data in figure_2 shows that at a low isolation effectiveness (r = 0.35), a 1% increase in r leads to approximately a 0.23% decrease in the control reproduction number R_c, whereas at a high isolation effectiveness (r = 0.95), a 1% increase in r results in a larger decrease of about 1.014% in R_c. This indicates that R_c is more sensitive to changes in isolation effectiveness when the baseline isolation effectiveness is already high. This implies that improving isolation measures when they are already effective can significantly reduce Ebola transmission, emphasizing the importance of achieving and maintaining high isolation effectiveness to control the disease spread.",
        "Reasoning_answer2": "The parameter beta represents the transmission rate or effective contact rate between infectious and susceptible individuals. It consistently shows a 1% increase causing a 1% increase in R_c across all scenarios, highlighting its direct proportional effect. This strong and consistent impact reflects that transmission rates are fundamental drivers of the epidemic's potential to spread. Regardless of isolation effectiveness, controlling beta, such as through reducing contacts or improved hygiene, is crucial for reducing Ebola transmission, as it directly affects how many new infections arise from each infectious individual."
    },
    {
        "paper_id": "1511.04027.json",
        "image_id": "figure_3",
        "image_path": [
            "/home/yz979/scratch/chengye/scimolmo/test-images-tables/dataset_imagestables/1511.04027/images/R0SAnew12.jpg"
        ],
        "caption": "Percent change in $\\mathcal{R}_c$ with respect to a $1 \\%$ change in the parameter value, for a high value of $f_T$ ($f_T = 0.75$) and two different levels of isolation effectiveness ($r = 0.35$ and $r = 0.95$). The other parameter values are kept as shown in Table \\ref{tab:ParamsDef}.",
        "classify": "Chart",
        "section_info": "3 \\protect\\normalsize Model analysis\n\\section{\\protect\\normalsize Model analysis}\n\n \\subsection{\\protect\\normalsize Basic properties}\nSince model (\\ref{model1}) imitates the dynamics of human populations, all variables and parameters should be non-negative. Thus, following the approach shown in appendix A of [\\ref{lit:HTh}], we show the following result.\n\\begin{theorem}\nThe variables of model (\\ref{model1}) are non-negative for all time.\n\\end{theorem}\n\n\\begin{lemma}\nThe closed set \n\\begin{equation*}\n\\Omega = \\big\\lbrace (S, E_1, E_2, I, J, R)\\in \\mathbb{R}_+^6: \\frac{\\Lambda}{\\mu + q_1\\gamma + q_2\\gamma_r}\\le{S + E_1 + E_2 + I + J + R} \\le \\frac{\\Lambda}{\\mu}  \\big\\rbrace\n\\end{equation*}\nis positively invariant for model (\\ref{model1}) and is absorbing.\n\\end{lemma}\n\\noindent Proof: Equation (\\ref{Neq}) implies that\n\\begin{eqnarray}\n\\frac{dN}{dt} &\\le& \\Lambda - \\mu N, \\label{Neq1}\\\\\n\\frac{dN}{dt} &\\ge& \\Lambda - (\\mu+q_1\\gamma+q_2\\gamma_r) N. \\label{Neq2}\n\\end{eqnarray}\nIt follows from (\\ref{Neq1}) that \n\\begin{equation}\nN(t) \\le  \\frac{\\Lambda}{\\mu} + \\left(N(0) -\\frac{\\Lambda}{\\mu} \\right) e^{- \\mu t}\\label{ineq1}\n\\end{equation}\nand from (\\ref{Neq2}) that \n\\begin{equation}\nN(t)\\ge\\frac{\\Lambda}{\\mu + q_1\\gamma + q_2\\gamma_r} + \\left(N(0) -\\frac{\\Lambda}{\\mu + q_1\\gamma + q_2\\gamma_r} \\right) e^{-(\\mu + q_1\\gamma + q_2\\gamma_r)t}. \\label{ineq2}\n\\end{equation}\nIf we assume $N(0) > \\Lambda/\\mu$, then $dN/dt < 0$ and therefore (based on inequality (\\ref{ineq1})), $N(t)$ decreases steadily until reaching $\\Lambda/\\mu$ when $t$ tends to $\\infty$. Similarly, if we assume $N(0) < \\Lambda/(\\mu + q_1\\gamma + q_2\\gamma_r)$, then $dN/dt > 0$ and therefore  (based on inequality (\\ref{ineq2})), $N(t)$ increases steadily until reaching a maximum at $\\Lambda/(\\mu + q_1\\gamma + q_2\\gamma_r)$ when $t$ tends to $\\infty$. It remains to check the case if $N(0)$ lies in the phase between $\\Lambda/(\\mu + q_1\\gamma + q_2\\gamma_r)$ and $\\Lambda/\\mu$. To this end, both inequalities (\\ref{ineq1}) and (\\ref{ineq2}) are combined together to get \n\\begin{equation*}\n\\frac{\\Lambda}{\\mu + q_1\\gamma + q_2\\gamma_r} + \\left(N(0) -\\frac{\\Lambda}{\\mu + q_1\\gamma + q_2\\gamma_r} \\right) e^{-(\\mu + q_1\\gamma + q_2\\gamma_r)t}\\le N(t) \\le \\frac{\\Lambda}{\\mu} + \\left(N(0) -\\frac{\\Lambda}{\\mu} \\right) e^{- \\mu t}.\n\\end{equation*}\nOn taking the limit when $t$ tends to $\\infty$, we find that $N(t)$ remains within the same phase. Thus, the set $\\Omega$ is positively invariant and absorbing.\n\n \\subsection{\\protect\\normalsize Equilibrium analysis}\n \\subsubsection*{\\protect\\normalsize Ebola-free equilibrium and the control reproduction number $\\mathcal{R}_c$}\nIt is easy to check that model (\\ref{model1}) has the Ebola-free equilibrium \n\\begin{equation}\nE_0 = \\left(\\frac{\\Lambda}{\\mu}, 0, 0, 0, 0, 0\\right)^{\\prime}\n\\end{equation}\nwhere the prime `` ${}^{\\prime}$ '' means vector transpose.\\newline \n\\indent The basic reproduction number,  $\\mathcal{R}_0$,  is a measure of the average number of secondary cases produced by a typical infectious individual during the entire course of infection in a completely susceptible population and in the absence of control interventions [\\ref{lit:  BrauerF},\\ref{lit: AndersonRM}]. On the other hand, the control reproduction number, $\\mathcal{R}_c$, quantifies the potential for infectious disease transmission in the context of a partially susceptible population due to the implementation of control interventions. When $\\mathcal{R}_c > 1$, the infection may spread in the population, and the rate of spread is higher with increasingly high values of $\\mathcal{R}_c$. If $\\mathcal{R}_c < 1$, infection cannot be sustained and is unable to generate an epidemic.  For our model, $\\mathcal{R}_c$  is computed using the next generation matrix approach shown in [\\ref{lit:PVDDJW2002}].  Accordingly, we compute the matrices $\\mathbf{F}$ (for the new infection terms) and $\\mathbf{V}$ (for the\ntransition terms) as\n\n\\begin{eqnarray*}\n  \\mathbf{F} = \\left(\\begin{array}{cccc}\n    0 & 0 & \\beta & (1-r) \\ell \\beta \\\\\n    0 & 0 & 0 & 0 \\\\\n    0 & 0 & 0 & 0 \\\\\n    0 & 0 & 0 & 0 \\\\\n  \\end{array}\\right) \\quad, \\quad\n   \\mathbf{V} = \\left(\\begin{array}{cccc}\n    \\kappa_1 + \\mu & 0 & 0 & 0 \\\\\n    -\\kappa_1 & \\kappa_2 + \\mu  & 0 & 0\\\\\n    0 & -(1-f_T) \\kappa_2 & \\alpha+\\gamma+\\mu & 0\\\\\n    0 & - f_T \\kappa_2  & - \\alpha & \\gamma_r + \\mu\\\\\n  \\end{array}\\right).\n\\end{eqnarray*}\n\\noindent Thus, the control reproduction number is given by\n\\begin{eqnarray}\n\\mathcal{R}_c & = &\\rho(\\mathbf{F}\\mathbf{V}^{-1}) = \\frac{\\kappa_1 \\kappa_2 \\beta[(1-f_T) (\\mu + \\gamma_r) + (1-r)\\ell(\\alpha+f_T(\\gamma+\\mu))]}{(\\kappa_1 + \\mu)(\\kappa_2 + \\mu)(\\alpha+\\gamma+\\mu)(\\gamma_r + \\mu)} \\nonumber\\\\\n& = & \\frac{\\kappa_1\\kappa_2\\beta}{(\\kappa_1 + \\mu)(\\kappa_2 + \\mu)(\\alpha+\\gamma+\\mu)}\\left[1 - f_T + (1-r) \\ell \\left( \\frac{\\alpha}{\\gamma_r+\\mu} + f_T  \\frac{\\gamma + \\mu}{\\gamma_r+\\mu} \\right) \\right]\\nonumber\\\\\n& = & \\mathcal{R}_0\\left[1-\\frac{\\alpha}{(\\alpha+\\gamma+\\mu)}\\right] \\left[1 - f_T + (1-r) \\ell \\left( \\frac{\\alpha}{\\gamma_r+\\mu} + f_T  \\frac{\\gamma + \\mu}{\\gamma_r+\\mu} \\right) \\right]\\label{R0eq}\n\\end{eqnarray}\nwhere $\\rho$ is the spectral radius (dominant eigenvalue in\nmagnitude) of the matrix $\\mathbf{F}\\mathbf{V}^{-1}$ and \n\\begin{equation}\n\\mathcal{R}_0 = \\frac{\\kappa_1\\kappa_2\\beta}{(\\kappa_1 + \\mu)(\\kappa_2 + \\mu)(\\gamma+\\mu)}\n\\end{equation}\nis the basic reproduction number for the model.\n\n\\indent The local stability of the Ebola-free equilibrium, $E_0$, for values of $\\mathcal{R}_c < 1$ is established based on a direct use of Theorem 2 in [\\ref{lit:PVDDJW2002}]. We summarize our result in the following lemma.\n\\begin{lemma}\nThe Ebola-free equilibrium $E_0$ of model (\\ref{model1}) is locally asymptotically stable if and only if $\\mathcal{R}_c < 1$.\n\\end{lemma}\n\n\n\\subsubsection*{\\protect\\normalsize Ebola-endemic equilibrium}\nOn putting the derivatives in the left hand side of (\\ref{model1}) equal zero and solving the resulting algebraic system with respect to the variables $\\bar{S}, \\bar{E}_1, \\bar{E}_2, \\bar{I}, \\bar{J}$, and $\\bar{R}$, we obtain\n \\begin{eqnarray}\n\\bar{S} & = & \\frac{\\Lambda}{\\bar\\lambda + \\mu},\\nonumber\\\\\n\\bar{E}_1 & = & \\frac{\\Lambda}{\\bar\\lambda + \\mu} \\cdot \\frac{\\bar\\lambda}{\\kappa_1 + \\mu},\\nonumber\\\\\n\\bar{E}_2 & = & \\frac{\\kappa_1}{\\kappa_2 + \\mu}\\cdot \\frac{\\Lambda}{\\bar\\lambda + \\mu} \\cdot \\frac{\\bar\\lambda}{\\kappa_1 + \\mu},\\nonumber \\\\\n\\bar{I} & = & \\frac{(1-f_T)\\kappa_2}{\\alpha+\\gamma + \\mu} \\cdot \\frac{\\kappa_1}{\\kappa_2 + \\mu}\\cdot \\frac{\\Lambda}{\\bar\\lambda + \\mu} \\cdot \\frac{\\bar\\lambda}{\\kappa_1 + \\mu},\\label{eqvar} \\\\\n\\bar{J} & = &  \\frac{\\kappa_1}{\\kappa_2 + \\mu}\\cdot \\frac{\\Lambda}{\\bar\\lambda + \\mu} \\cdot \\frac{\\bar\\lambda}{\\kappa_1 + \\mu} \\cdot \\frac{\\kappa_2}{\\gamma_r + \\mu} \\left[f_T + (1-f_T) \\frac{\\alpha}{\\alpha+\\gamma + \\mu} \\right], \\nonumber\\\\\n\\bar{R} & = & \\frac{1}{\\mu}[(1-q_1)\\gamma I + (1-q_2) \\gamma_r J]\\nonumber\n\\end{eqnarray}\nwhere\n\\begin{equation}\n\\bar\\lambda = \\frac{\\beta(I + (1-r)\\ell \\bar{J})}{\\bar{N} - r \\bar{J}}\\label{lambda}\n\\end{equation}\nis the equilibrium force of infection. On substituting from (\\ref{eqvar}) into (\\ref{lambda}) and simplifying (with the assumption that $\\lambda \\ne 0$), we get\n \\begin{equation}\n\\bar\\lambda = \\frac{\\mu(\\mathcal{R}_c - 1)}{1 - Term}\n\\end{equation}\nwhere \n \\begin{equation*}\nTerm = \\frac{\\kappa_1 \\kappa_2 [q_1(1-f_T)\\gamma(\\gamma_r + \\mu) + (r\\mu + q_2\\gamma_r)(f_T(\\gamma + \\mu) + \\alpha)]}{(\\kappa_1 + \\mu)(\\kappa_2 + \\mu)(\\alpha+\\gamma+\\mu)(\\gamma_r + \\mu)}.\n\\end{equation*}\nHence, the Ebola-endemic equilibrium is unique and we show the following lemma.\n \\begin{lemma}\n Model (\\ref{model1}) has a unique endemic equilibrium that exists if and only if $\\mathcal{R}_c > 1$.\n \\end{lemma}\n \n\n\n\n\n\n\n\\subsection{\\protect\\normalsize Normalized sensitivity analysis on $\\mathcal{R}_c$ }\\label{sensitivity}\n\n\n\nIn considering the dynamics of the Ebola system (\\ref{model1}), we conduct normalized sensitivity analysis on $\\mathcal{R}_c$ to determine the impact of parameter perturbations on the transmission dynamics of the system. By computing the normalized sensitivity indices, we consider the percent change in the output with respect to a percent change in the parameter input. Those parameters with the largest magnitude of change impact the compartment model the most; the sign indicates whether the change produces an increase or a decrease on $\\mathcal{R}_c$.\\newline\n\\indent The normalized sensitivity indices for $\\mathcal{R}_c$ are calculated by taking the partial derivative of $\\mathcal{R}_c$ with respect to each parameter and multiply the derivative with the ratio of the parameter to $\\mathcal{R}_c$. This value represents the percent change in $\\mathcal{R}_c$ with respect to a 1\\% change in the parameter value [\\ref{lit: CaswellH}]. \\newline\n \n\\vspace{-5mm}\n\n\\begin{table}[h!]\n\\caption{Percent change in $\\mathcal{R}_c$ with respect to a $1\\%$ change in the parameter value, for a low and a high isolation effectiveness $r$, and a low and a high value of $f_T$, while keeping the other parameter values as presented in Table \\ref{tab:ParamsDef}.} \\label{r0change1}\n\\vspace{-6mm}\n\\begin{center}\n\\begin{tabular}{|c|c|c|c|c|c|c|c|c|}\n\\hline\n&Parameter & $\\beta$ & $r$ & $\\ell$ & $\\gamma_r$ & $\\gamma$ & $\\alpha$ & $f_T$ \\\\\n\\hline\n\n &\\% change & 1\\% & -0.23\\% & 0.423\\%  & -0.423\\% & -0.382\\% & -0.195\\% & -0.119\\% \\\\\n $f_T = 0.25$& for $r = 0.35$ &  &  &    &   &   &   &  \\\\\n\\cline{2-9}\n&\\% change & 1\\% & -1.014\\% & 0.053\\%  & -0.053\\% & -0.445\\% & -0.501\\% & -0.306\\% \\\\\n  & for $r = 0.95$ &  &  &    &   &   &   &  \\\\\n\\hline\n&\\% change & 1\\% & -0.402\\% & 0.747\\%  & -0.747\\% & -0.167\\% & -0.086\\% & -0.471\\% \\\\\n $f_T = 0.75$& for $r = 0.35$ &  &  &    &   &   &   &  \\\\\n\\cline{2-9}\n&\\% change & 1\\% & -3.521\\% & 0.185\\%  & -0.185\\% & -0.383\\% & -0.431\\% & -2.373\\% \\\\\n  & for $r = 0.95$ &  &  &    &   &   &   &  \\\\\n\\hline\n\\end{tabular}\n\\end{center}\n\\end{table}\n\n\n\n\n\\begin{figure}[h!]\n\\begin{center}\n\\includegraphics[scale=0.57]{R0SAnew11.jpg}\n\\caption{Percent change in $\\mathcal{R}_c$ with respect to a $1\\%$ change in the parameter value, for a low value of $f_T$ ($f_T = 0.25$) and two different levels of isolation effectiveness ($r = 0.35$ and $r = 0.95$). The other parameter values are kept as shown in Table \\ref{tab:ParamsDef}.} \\label{graphofR01}\n\\end{center}\n\\end{figure}\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\\indent We use the parameters values from Table \\ref{tab:ParamsDef} to study the sensitivity of $\\mathcal{R}_c$ to each parameter. We compute normalized sensitivity analysis on all parameters, but we just consider the impact of parameters that are the most sensitive: $\\beta, r, \\ell, \\gamma_r, \\gamma, \\alpha$, and $f_T$. The other parameters ($\\mu, \\kappa_1$, and  $\\kappa_2$)  have a very low impact, namely less than $0.001\\%$. The numerical simulations to the sensitivity of $\\mathcal{R}_c$ with respect to each of the most sensitive parameters are given in Table \\ref{r0change1}, for two different levels of isolation effectiveness ($r = 0.35$ and $r = 0.95$) and two values of $f_T$ ($f_T = 0.25$ and $f_T = 0.75$), which is the fraction of pre-symptomatic individuals diagnosed and isolated. The other parameter values are kept as shown in Table \\ref{tab:ParamsDef}.\n\\newline\n\\indent A graphical illustration of the numerical results for the scenario when $f_T = 0.25$ and the two levels of isolation effectiveness ($r = 0.35$ and $r = 0.95$) is given in Figure \\ref{graphofR01}. In the case of high isolation effectiveness ($r = 0.95$), simulations show that both the removal rate, $\\gamma_r$, of isolated individuals and the relative transmissibility parameter $\\ell$ of isolated individuals with respect to infectious individuals are the least sensitive parameters (with $0.053 \\%$ change of $\\mathcal{R}_c$), while the parameter of isolation effectiveness, $r$, is the most sensitive one, where a $1\\%$ increase in $r$ causes a $1.014 \\%$ reduction in the value of $\\mathcal{R}_c$. Also, the rate at which infectious individuals get isolated, $\\alpha$, and the fraction of pre-symptomatic individuals detected and isolated, $f_T$, impact negatively on the level of $\\mathcal{R}_c$, where a $1 \\%$ percent increase in the value of $f_T$ causes approximately a $0.31\\%$ decline in the value of the reproduction number $\\mathcal{R}_c$. Thus, as pre-symptomatic individuals are diagnosed and as isolation is highly effective, the number of available infectious individuals who are capable of transmitting Ebola decreases and therefore, the reproduction number decreases. Also, the removal (by recovery or Ebola-induced death) rate $\\gamma$ of infectious individuals affects negatively on $\\mathcal{R}_c$. Hence, for the case of highly effective isolation, the parameters concerning early diagnosis and isolation have a significant impact on the reproduction number. \n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\\indent This percent impact of the parameters on $\\mathcal{R}_c$ remains so as long as isolation is highly effective. However, if the effectiveness of isolation is low, in the sense that all parameter values are kept the same except the value of the parameter $r$, which is reduced to $0.35$, then we get the results presented in Table \\ref{r0change1} and Figure \\ref{graphofR01}. In this case, both the relative transmissibility $\\ell$ and the removal rate of isolated individuals, $\\gamma_r$, are the second most sensitive parameters, after $\\beta$ which is the most impactful one. Also, $\\ell$ became more sensitive than $r$. The implication is that, when isolation is less effective, there exists the possibility for isolated people to make successful contacts with susceptible individuals and therefore the possibility of causing new infections increases. This causes an increase in the reproduction number. Also, it is noted that the effect of $f_T$ and $\\alpha$ is reduced, which means that diagnosing and isolating infected individuals becomes a weak strategy if the effectiveness of isolation is low.\n \n\n\n\\begin{figure}[h!]\n\\begin{center}\n\\includegraphics[scale=0.585]{R0SAnew12.jpg}\n\\caption{Percent change in $\\mathcal{R}_c$ with respect to a $1 \\%$ change in the parameter value, for a high value of $f_T$ ($f_T = 0.75$) and two different levels of isolation effectiveness ($r = 0.35$ and $r = 0.95$). The other parameter values are kept as shown in Table \\ref{tab:ParamsDef}.} \\label{graphofR02}\n\\end{center}\n\\end{figure}\n\nOn repeating the previous analyses, but this time for a higher value of $f_T$ ($f_T = 0.75$), we obtain the results shown in Table  \\ref{r0change1}, which are also illustrated in Figure \\ref{graphofR02}. In comparison to the scenario when $f_T = 0.25$, the simulations show that increasing the fraction of pre-symptomatic individuals who are diagnosed and isolated,  $f_T$, increases the percent impact of the parameters $r, \\ell, \\gamma_r,$ and $f_T$, and decreases the percent impact of the parameters $\\gamma$ and $\\alpha$, on the value of the control reproduction number $\\mathcal{R}_c$.   \n\n \n\n  \n\n\n\\subsection{\\protect\\normalsize Impact of early detection and isolation on the value of $\\mathcal{R}_c$}\n\n \\begin{figure}[H]\n\\centering\n\\includegraphics[scale=0.55]{R0fT.jpg}\n\\caption{Impact of early detection of pre-symptomatic individuals on the value of $\\mathcal{R}_c$.}\n\\label{fig:R0fT}\n\\end{figure}\n\n\nTo study the impact of early detection of pre-symptomatic individuals and isolation on the reproduction number, we first depict $\\mathcal{R}_c$ as a function of $f_T$, for different levels of isolation effectiveness $r$. Figure \\ref{fig:R0fT} shows that the control reproduction number declines as the proportion, $f_T$, of pre-symptomatic individuals, who get diagnosed and isolated, increases.  Simulations are done using parameter values from Table \\ref{tab:ParamsDef}, but for three different values of $r$. It further shows that the curve corresponding to a low and an intermidate value of isolation effectivenes $r$ (e.g. $r = 0.35$ for the solid curve and $r = 0.65$ for the dashed curve) hits $\\mathcal{R}_c = 1$ at some critical value of $f_T$ (say $f_T^{\\star}$), while for the high value of $r$ ($r = 0.95$), it never hits the critical threshold $\\mathcal{R}_c = 1$, as the curve is totally below the critical threshold. This indicates that for a high effectiveness of isolation, the control reproduction number is less than one and therefore the infection dies out. Analytically, the exact form of $f_T^{\\star}$ is \n\\begin{equation}\nf_T^{\\star} = \\left[ 1 + (1-r)\\ell \\frac{\\alpha}{\\gamma_r + \\mu} - \\frac{1}{\\mathcal{R}_0} \\left(1 + \\frac{\\alpha}{\\gamma+\\mu}\\right) \\right] / \\left[ 1 - \\frac{(1-r)\\ell(\\gamma + \\mu)}{\\gamma_r + \\mu} \\right].  \\label{fTcond1}\n\\end{equation}\nThe critical proportion $f_T^{\\star}$ represents the minimum proportion of pre-symptomatic individuals who are detected and get isolated to ensure an effective control of Ebola. This critical value remains feasible as long as the following inequality holds\n\n\n\n\\begin{equation}\n(1-r)\\ell < \\frac{\\gamma_r + \\mu}{(\\gamma + \\mu)\\mathcal{R}_0}. \\label{fTcond20}\n\\end{equation}\nIf we keep all parameters fixed except $r$, then condition (\\ref{fTcond20}) could be rewritten in a more convenient form\n\n\n\n\\begin{equation}\nr > 1- \\frac{\\gamma_r + \\mu}{\\ell (\\gamma + \\mu) \\mathcal{R}_0}. \\label{fTcond3}\n\\end{equation}\nThis gives the minimum level of effectiveness of isolation required to obtain an isolation and early diagnosis-based control strategy for Ebola tranmission. \n\n\n\n\n \\begin{figure}[H]\n\\centering\n\\includegraphics[scale=0.55]{R0alpha.jpg}\n\\caption{Impact of isolating infectious individuals on the value of $\\mathcal{R}_c$.}\n\\label{fig:R0alpha}\n\\end{figure}\n\nNow, we could also ask a similar question on the role of isolating infectious individuals to contain Ebola transmission. Figure \\ref{fig:R0alpha} shows the impact of changing the rate at which infectious individuals get isolated, $\\alpha$, on $\\mathcal{R}_c$, for the same three different levels of isolation effectivenes, as used above. The analysis shows that it is possible to control the epidemic if and only if $\\alpha > \\alpha^\\star$, where\n\\begin{equation}\n\\alpha^\\star = \\frac{[ (1-f_T)(\\gamma_r + \\mu)(\\gamma+\\mu) + (1-r)\\ell f_T (\\gamma+\\mu)^2]\\mathcal{R}_0 -  (\\gamma_r + \\mu)(\\gamma+\\mu) }{(\\gamma_r + \\mu) - \\ell (1-r) \\mathcal{R}_0(\\gamma + \\mu)}\n\\end{equation}\n\n\n\n\nand with the implementation of condition (\\ref{fTcond20}).\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n3.3 \\protect\\normalsize Normalized sensitivity analysis on $\\mathcal{R}_c$\n\\subsection{\\protect\\normalsize Normalized sensitivity analysis on $\\mathcal{R}_c$ }\\label{sensitivity}\n\n\n\nIn considering the dynamics of the Ebola system (\\ref{model1}), we conduct normalized sensitivity analysis on $\\mathcal{R}_c$ to determine the impact of parameter perturbations on the transmission dynamics of the system. By computing the normalized sensitivity indices, we consider the percent change in the output with respect to a percent change in the parameter input. Those parameters with the largest magnitude of change impact the compartment model the most; the sign indicates whether the change produces an increase or a decrease on $\\mathcal{R}_c$.\\newline\n\\indent The normalized sensitivity indices for $\\mathcal{R}_c$ are calculated by taking the partial derivative of $\\mathcal{R}_c$ with respect to each parameter and multiply the derivative with the ratio of the parameter to $\\mathcal{R}_c$. This value represents the percent change in $\\mathcal{R}_c$ with respect to a 1\\% change in the parameter value [\\ref{lit: CaswellH}]. \\newline\n \n\\vspace{-5mm}\n\n\\begin{table}[h!]\n\\caption{Percent change in $\\mathcal{R}_c$ with respect to a $1\\%$ change in the parameter value, for a low and a high isolation effectiveness $r$, and a low and a high value of $f_T$, while keeping the other parameter values as presented in Table \\ref{tab:ParamsDef}.} \\label{r0change1}\n\\vspace{-6mm}\n\\begin{center}\n\\begin{tabular}{|c|c|c|c|c|c|c|c|c|}\n\\hline\n&Parameter & $\\beta$ & $r$ & $\\ell$ & $\\gamma_r$ & $\\gamma$ & $\\alpha$ & $f_T$ \\\\\n\\hline\n\n &\\% change & 1\\% & -0.23\\% & 0.423\\%  & -0.423\\% & -0.382\\% & -0.195\\% & -0.119\\% \\\\\n $f_T = 0.25$& for $r = 0.35$ &  &  &    &   &   &   &  \\\\\n\\cline{2-9}\n&\\% change & 1\\% & -1.014\\% & 0.053\\%  & -0.053\\% & -0.445\\% & -0.501\\% & -0.306\\% \\\\\n  & for $r = 0.95$ &  &  &    &   &   &   &  \\\\\n\\hline\n&\\% change & 1\\% & -0.402\\% & 0.747\\%  & -0.747\\% & -0.167\\% & -0.086\\% & -0.471\\% \\\\\n $f_T = 0.75$& for $r = 0.35$ &  &  &    &   &   &   &  \\\\\n\\cline{2-9}\n&\\% change & 1\\% & -3.521\\% & 0.185\\%  & -0.185\\% & -0.383\\% & -0.431\\% & -2.373\\% \\\\\n  & for $r = 0.95$ &  &  &    &   &   &   &  \\\\\n\\hline\n\\end{tabular}\n\\end{center}\n\\end{table}\n\n\n\n\n\\begin{figure}[h!]\n\\begin{center}\n\\includegraphics[scale=0.57]{R0SAnew11.jpg}\n\\caption{Percent change in $\\mathcal{R}_c$ with respect to a $1\\%$ change in the parameter value, for a low value of $f_T$ ($f_T = 0.25$) and two different levels of isolation effectiveness ($r = 0.35$ and $r = 0.95$). The other parameter values are kept as shown in Table \\ref{tab:ParamsDef}.} \\label{graphofR01}\n\\end{center}\n\\end{figure}\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\\indent We use the parameters values from Table \\ref{tab:ParamsDef} to study the sensitivity of $\\mathcal{R}_c$ to each parameter. We compute normalized sensitivity analysis on all parameters, but we just consider the impact of parameters that are the most sensitive: $\\beta, r, \\ell, \\gamma_r, \\gamma, \\alpha$, and $f_T$. The other parameters ($\\mu, \\kappa_1$, and  $\\kappa_2$)  have a very low impact, namely less than $0.001\\%$. The numerical simulations to the sensitivity of $\\mathcal{R}_c$ with respect to each of the most sensitive parameters are given in Table \\ref{r0change1}, for two different levels of isolation effectiveness ($r = 0.35$ and $r = 0.95$) and two values of $f_T$ ($f_T = 0.25$ and $f_T = 0.75$), which is the fraction of pre-symptomatic individuals diagnosed and isolated. The other parameter values are kept as shown in Table \\ref{tab:ParamsDef}.\n\\newline\n\\indent A graphical illustration of the numerical results for the scenario when $f_T = 0.25$ and the two levels of isolation effectiveness ($r = 0.35$ and $r = 0.95$) is given in Figure \\ref{graphofR01}. In the case of high isolation effectiveness ($r = 0.95$), simulations show that both the removal rate, $\\gamma_r$, of isolated individuals and the relative transmissibility parameter $\\ell$ of isolated individuals with respect to infectious individuals are the least sensitive parameters (with $0.053 \\%$ change of $\\mathcal{R}_c$), while the parameter of isolation effectiveness, $r$, is the most sensitive one, where a $1\\%$ increase in $r$ causes a $1.014 \\%$ reduction in the value of $\\mathcal{R}_c$. Also, the rate at which infectious individuals get isolated, $\\alpha$, and the fraction of pre-symptomatic individuals detected and isolated, $f_T$, impact negatively on the level of $\\mathcal{R}_c$, where a $1 \\%$ percent increase in the value of $f_T$ causes approximately a $0.31\\%$ decline in the value of the reproduction number $\\mathcal{R}_c$. Thus, as pre-symptomatic individuals are diagnosed and as isolation is highly effective, the number of available infectious individuals who are capable of transmitting Ebola decreases and therefore, the reproduction number decreases. Also, the removal (by recovery or Ebola-induced death) rate $\\gamma$ of infectious individuals affects negatively on $\\mathcal{R}_c$. Hence, for the case of highly effective isolation, the parameters concerning early diagnosis and isolation have a significant impact on the reproduction number. \n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\\indent This percent impact of the parameters on $\\mathcal{R}_c$ remains so as long as isolation is highly effective. However, if the effectiveness of isolation is low, in the sense that all parameter values are kept the same except the value of the parameter $r$, which is reduced to $0.35$, then we get the results presented in Table \\ref{r0change1} and Figure \\ref{graphofR01}. In this case, both the relative transmissibility $\\ell$ and the removal rate of isolated individuals, $\\gamma_r$, are the second most sensitive parameters, after $\\beta$ which is the most impactful one. Also, $\\ell$ became more sensitive than $r$. The implication is that, when isolation is less effective, there exists the possibility for isolated people to make successful contacts with susceptible individuals and therefore the possibility of causing new infections increases. This causes an increase in the reproduction number. Also, it is noted that the effect of $f_T$ and $\\alpha$ is reduced, which means that diagnosing and isolating infected individuals becomes a weak strategy if the effectiveness of isolation is low.\n \n\n\n\\begin{figure}[h!]\n\\begin{center}\n\\includegraphics[scale=0.585]{R0SAnew12.jpg}\n\\caption{Percent change in $\\mathcal{R}_c$ with respect to a $1 \\%$ change in the parameter value, for a high value of $f_T$ ($f_T = 0.75$) and two different levels of isolation effectiveness ($r = 0.35$ and $r = 0.95$). The other parameter values are kept as shown in Table \\ref{tab:ParamsDef}.} \\label{graphofR02}\n\\end{center}\n\\end{figure}\n\nOn repeating the previous analyses, but this time for a higher value of $f_T$ ($f_T = 0.75$), we obtain the results shown in Table  \\ref{r0change1}, which are also illustrated in Figure \\ref{graphofR02}. In comparison to the scenario when $f_T = 0.25$, the simulations show that increasing the fraction of pre-symptomatic individuals who are diagnosed and isolated,  $f_T$, increases the percent impact of the parameters $r, \\ell, \\gamma_r,$ and $f_T$, and decreases the percent impact of the parameters $\\gamma$ and $\\alpha$, on the value of the control reproduction number $\\mathcal{R}_c$.   \n\n \n\n  \n\n\n",
        "label": "graphofR02",
        "Descriptive_question1": "What is the percent change in Rc for a 1% increase in the parameter beta in figure_3?",
        "Descriptive_question2": "What is the value of f_T used in figure_3?",
        "Reasoning_question1": "How does the effectiveness of isolation (r) impact the sensitivity of Rc to changes in f_T as shown in figure_3?",
        "Reasoning_question2": "Why might the parameter r have a larger negative impact on Rc at higher isolation effectiveness levels in figure_3?",
        "Descriptive_answer1": "1%",
        "Descriptive_answer2": "0.75",
        "Reasoning_answer1": "The sensitivity analysis in figure_3 shows that with a high fraction of pre-symptomatic individuals diagnosed and isolated (f_T = 0.75), the percent change in the control reproduction number Rc with respect to a 1% change in f_T is larger when the isolation effectiveness r is high (e.g., r = 0.95) compared to when r is low (r = 0.35). Specifically, the percent decrease in Rc is -2.373% for high r and -0.471% for low r. This suggests that improving isolation effectiveness amplifies the impact of early detection and isolation (f_T) on reducing Rc, meaning that the effectiveness of isolation modulates how sensitive Rc is to changes in f_T.",
        "Reasoning_answer2": "At higher levels of isolation effectiveness (r close to 1), isolating infectious individuals substantially reduces their capacity to transmit Ebola. Therefore, increasing r further significantly decreases the transmission potential, which reflects as a larger negative sensitivity of Rc to r. Conversely, when isolation is not very effective (lower r), changes in r have a smaller impact because isolated individuals can still transmit the disease. Thus, higher values of r result in a greater negative impact on Rc because improvements in effectiveness directly translate to fewer secondary infections."
    },
    {
        "paper_id": "1511.04027.json",
        "image_id": "figure_4",
        "image_path": [
            "/home/yz979/scratch/chengye/scimolmo/test-images-tables/dataset_imagestables/1511.04027/images/R0fT.jpg"
        ],
        "caption": "Impact of early detection of pre-symptomatic individuals on the value of $\\mathcal{R}_c$.",
        "classify": "Chart",
        "section_info": "3 \\protect\\normalsize Model analysis\n\\section{\\protect\\normalsize Model analysis}\n\n \\subsection{\\protect\\normalsize Basic properties}\nSince model (\\ref{model1}) imitates the dynamics of human populations, all variables and parameters should be non-negative. Thus, following the approach shown in appendix A of [\\ref{lit:HTh}], we show the following result.\n\\begin{theorem}\nThe variables of model (\\ref{model1}) are non-negative for all time.\n\\end{theorem}\n\n\\begin{lemma}\nThe closed set \n\\begin{equation*}\n\\Omega = \\big\\lbrace (S, E_1, E_2, I, J, R)\\in \\mathbb{R}_+^6: \\frac{\\Lambda}{\\mu + q_1\\gamma + q_2\\gamma_r}\\le{S + E_1 + E_2 + I + J + R} \\le \\frac{\\Lambda}{\\mu}  \\big\\rbrace\n\\end{equation*}\nis positively invariant for model (\\ref{model1}) and is absorbing.\n\\end{lemma}\n\\noindent Proof: Equation (\\ref{Neq}) implies that\n\\begin{eqnarray}\n\\frac{dN}{dt} &\\le& \\Lambda - \\mu N, \\label{Neq1}\\\\\n\\frac{dN}{dt} &\\ge& \\Lambda - (\\mu+q_1\\gamma+q_2\\gamma_r) N. \\label{Neq2}\n\\end{eqnarray}\nIt follows from (\\ref{Neq1}) that \n\\begin{equation}\nN(t) \\le  \\frac{\\Lambda}{\\mu} + \\left(N(0) -\\frac{\\Lambda}{\\mu} \\right) e^{- \\mu t}\\label{ineq1}\n\\end{equation}\nand from (\\ref{Neq2}) that \n\\begin{equation}\nN(t)\\ge\\frac{\\Lambda}{\\mu + q_1\\gamma + q_2\\gamma_r} + \\left(N(0) -\\frac{\\Lambda}{\\mu + q_1\\gamma + q_2\\gamma_r} \\right) e^{-(\\mu + q_1\\gamma + q_2\\gamma_r)t}. \\label{ineq2}\n\\end{equation}\nIf we assume $N(0) > \\Lambda/\\mu$, then $dN/dt < 0$ and therefore (based on inequality (\\ref{ineq1})), $N(t)$ decreases steadily until reaching $\\Lambda/\\mu$ when $t$ tends to $\\infty$. Similarly, if we assume $N(0) < \\Lambda/(\\mu + q_1\\gamma + q_2\\gamma_r)$, then $dN/dt > 0$ and therefore  (based on inequality (\\ref{ineq2})), $N(t)$ increases steadily until reaching a maximum at $\\Lambda/(\\mu + q_1\\gamma + q_2\\gamma_r)$ when $t$ tends to $\\infty$. It remains to check the case if $N(0)$ lies in the phase between $\\Lambda/(\\mu + q_1\\gamma + q_2\\gamma_r)$ and $\\Lambda/\\mu$. To this end, both inequalities (\\ref{ineq1}) and (\\ref{ineq2}) are combined together to get \n\\begin{equation*}\n\\frac{\\Lambda}{\\mu + q_1\\gamma + q_2\\gamma_r} + \\left(N(0) -\\frac{\\Lambda}{\\mu + q_1\\gamma + q_2\\gamma_r} \\right) e^{-(\\mu + q_1\\gamma + q_2\\gamma_r)t}\\le N(t) \\le \\frac{\\Lambda}{\\mu} + \\left(N(0) -\\frac{\\Lambda}{\\mu} \\right) e^{- \\mu t}.\n\\end{equation*}\nOn taking the limit when $t$ tends to $\\infty$, we find that $N(t)$ remains within the same phase. Thus, the set $\\Omega$ is positively invariant and absorbing.\n\n \\subsection{\\protect\\normalsize Equilibrium analysis}\n \\subsubsection*{\\protect\\normalsize Ebola-free equilibrium and the control reproduction number $\\mathcal{R}_c$}\nIt is easy to check that model (\\ref{model1}) has the Ebola-free equilibrium \n\\begin{equation}\nE_0 = \\left(\\frac{\\Lambda}{\\mu}, 0, 0, 0, 0, 0\\right)^{\\prime}\n\\end{equation}\nwhere the prime `` ${}^{\\prime}$ '' means vector transpose.\\newline \n\\indent The basic reproduction number,  $\\mathcal{R}_0$,  is a measure of the average number of secondary cases produced by a typical infectious individual during the entire course of infection in a completely susceptible population and in the absence of control interventions [\\ref{lit:  BrauerF},\\ref{lit: AndersonRM}]. On the other hand, the control reproduction number, $\\mathcal{R}_c$, quantifies the potential for infectious disease transmission in the context of a partially susceptible population due to the implementation of control interventions. When $\\mathcal{R}_c > 1$, the infection may spread in the population, and the rate of spread is higher with increasingly high values of $\\mathcal{R}_c$. If $\\mathcal{R}_c < 1$, infection cannot be sustained and is unable to generate an epidemic.  For our model, $\\mathcal{R}_c$  is computed using the next generation matrix approach shown in [\\ref{lit:PVDDJW2002}].  Accordingly, we compute the matrices $\\mathbf{F}$ (for the new infection terms) and $\\mathbf{V}$ (for the\ntransition terms) as\n\n\\begin{eqnarray*}\n  \\mathbf{F} = \\left(\\begin{array}{cccc}\n    0 & 0 & \\beta & (1-r) \\ell \\beta \\\\\n    0 & 0 & 0 & 0 \\\\\n    0 & 0 & 0 & 0 \\\\\n    0 & 0 & 0 & 0 \\\\\n  \\end{array}\\right) \\quad, \\quad\n   \\mathbf{V} = \\left(\\begin{array}{cccc}\n    \\kappa_1 + \\mu & 0 & 0 & 0 \\\\\n    -\\kappa_1 & \\kappa_2 + \\mu  & 0 & 0\\\\\n    0 & -(1-f_T) \\kappa_2 & \\alpha+\\gamma+\\mu & 0\\\\\n    0 & - f_T \\kappa_2  & - \\alpha & \\gamma_r + \\mu\\\\\n  \\end{array}\\right).\n\\end{eqnarray*}\n\\noindent Thus, the control reproduction number is given by\n\\begin{eqnarray}\n\\mathcal{R}_c & = &\\rho(\\mathbf{F}\\mathbf{V}^{-1}) = \\frac{\\kappa_1 \\kappa_2 \\beta[(1-f_T) (\\mu + \\gamma_r) + (1-r)\\ell(\\alpha+f_T(\\gamma+\\mu))]}{(\\kappa_1 + \\mu)(\\kappa_2 + \\mu)(\\alpha+\\gamma+\\mu)(\\gamma_r + \\mu)} \\nonumber\\\\\n& = & \\frac{\\kappa_1\\kappa_2\\beta}{(\\kappa_1 + \\mu)(\\kappa_2 + \\mu)(\\alpha+\\gamma+\\mu)}\\left[1 - f_T + (1-r) \\ell \\left( \\frac{\\alpha}{\\gamma_r+\\mu} + f_T  \\frac{\\gamma + \\mu}{\\gamma_r+\\mu} \\right) \\right]\\nonumber\\\\\n& = & \\mathcal{R}_0\\left[1-\\frac{\\alpha}{(\\alpha+\\gamma+\\mu)}\\right] \\left[1 - f_T + (1-r) \\ell \\left( \\frac{\\alpha}{\\gamma_r+\\mu} + f_T  \\frac{\\gamma + \\mu}{\\gamma_r+\\mu} \\right) \\right]\\label{R0eq}\n\\end{eqnarray}\nwhere $\\rho$ is the spectral radius (dominant eigenvalue in\nmagnitude) of the matrix $\\mathbf{F}\\mathbf{V}^{-1}$ and \n\\begin{equation}\n\\mathcal{R}_0 = \\frac{\\kappa_1\\kappa_2\\beta}{(\\kappa_1 + \\mu)(\\kappa_2 + \\mu)(\\gamma+\\mu)}\n\\end{equation}\nis the basic reproduction number for the model.\n\n\\indent The local stability of the Ebola-free equilibrium, $E_0$, for values of $\\mathcal{R}_c < 1$ is established based on a direct use of Theorem 2 in [\\ref{lit:PVDDJW2002}]. We summarize our result in the following lemma.\n\\begin{lemma}\nThe Ebola-free equilibrium $E_0$ of model (\\ref{model1}) is locally asymptotically stable if and only if $\\mathcal{R}_c < 1$.\n\\end{lemma}\n\n\n\\subsubsection*{\\protect\\normalsize Ebola-endemic equilibrium}\nOn putting the derivatives in the left hand side of (\\ref{model1}) equal zero and solving the resulting algebraic system with respect to the variables $\\bar{S}, \\bar{E}_1, \\bar{E}_2, \\bar{I}, \\bar{J}$, and $\\bar{R}$, we obtain\n \\begin{eqnarray}\n\\bar{S} & = & \\frac{\\Lambda}{\\bar\\lambda + \\mu},\\nonumber\\\\\n\\bar{E}_1 & = & \\frac{\\Lambda}{\\bar\\lambda + \\mu} \\cdot \\frac{\\bar\\lambda}{\\kappa_1 + \\mu},\\nonumber\\\\\n\\bar{E}_2 & = & \\frac{\\kappa_1}{\\kappa_2 + \\mu}\\cdot \\frac{\\Lambda}{\\bar\\lambda + \\mu} \\cdot \\frac{\\bar\\lambda}{\\kappa_1 + \\mu},\\nonumber \\\\\n\\bar{I} & = & \\frac{(1-f_T)\\kappa_2}{\\alpha+\\gamma + \\mu} \\cdot \\frac{\\kappa_1}{\\kappa_2 + \\mu}\\cdot \\frac{\\Lambda}{\\bar\\lambda + \\mu} \\cdot \\frac{\\bar\\lambda}{\\kappa_1 + \\mu},\\label{eqvar} \\\\\n\\bar{J} & = &  \\frac{\\kappa_1}{\\kappa_2 + \\mu}\\cdot \\frac{\\Lambda}{\\bar\\lambda + \\mu} \\cdot \\frac{\\bar\\lambda}{\\kappa_1 + \\mu} \\cdot \\frac{\\kappa_2}{\\gamma_r + \\mu} \\left[f_T + (1-f_T) \\frac{\\alpha}{\\alpha+\\gamma + \\mu} \\right], \\nonumber\\\\\n\\bar{R} & = & \\frac{1}{\\mu}[(1-q_1)\\gamma I + (1-q_2) \\gamma_r J]\\nonumber\n\\end{eqnarray}\nwhere\n\\begin{equation}\n\\bar\\lambda = \\frac{\\beta(I + (1-r)\\ell \\bar{J})}{\\bar{N} - r \\bar{J}}\\label{lambda}\n\\end{equation}\nis the equilibrium force of infection. On substituting from (\\ref{eqvar}) into (\\ref{lambda}) and simplifying (with the assumption that $\\lambda \\ne 0$), we get\n \\begin{equation}\n\\bar\\lambda = \\frac{\\mu(\\mathcal{R}_c - 1)}{1 - Term}\n\\end{equation}\nwhere \n \\begin{equation*}\nTerm = \\frac{\\kappa_1 \\kappa_2 [q_1(1-f_T)\\gamma(\\gamma_r + \\mu) + (r\\mu + q_2\\gamma_r)(f_T(\\gamma + \\mu) + \\alpha)]}{(\\kappa_1 + \\mu)(\\kappa_2 + \\mu)(\\alpha+\\gamma+\\mu)(\\gamma_r + \\mu)}.\n\\end{equation*}\nHence, the Ebola-endemic equilibrium is unique and we show the following lemma.\n \\begin{lemma}\n Model (\\ref{model1}) has a unique endemic equilibrium that exists if and only if $\\mathcal{R}_c > 1$.\n \\end{lemma}\n \n\n\n\n\n\n\n\\subsection{\\protect\\normalsize Normalized sensitivity analysis on $\\mathcal{R}_c$ }\\label{sensitivity}\n\n\n\nIn considering the dynamics of the Ebola system (\\ref{model1}), we conduct normalized sensitivity analysis on $\\mathcal{R}_c$ to determine the impact of parameter perturbations on the transmission dynamics of the system. By computing the normalized sensitivity indices, we consider the percent change in the output with respect to a percent change in the parameter input. Those parameters with the largest magnitude of change impact the compartment model the most; the sign indicates whether the change produces an increase or a decrease on $\\mathcal{R}_c$.\\newline\n\\indent The normalized sensitivity indices for $\\mathcal{R}_c$ are calculated by taking the partial derivative of $\\mathcal{R}_c$ with respect to each parameter and multiply the derivative with the ratio of the parameter to $\\mathcal{R}_c$. This value represents the percent change in $\\mathcal{R}_c$ with respect to a 1\\% change in the parameter value [\\ref{lit: CaswellH}]. \\newline\n \n\\vspace{-5mm}\n\n\\begin{table}[h!]\n\\caption{Percent change in $\\mathcal{R}_c$ with respect to a $1\\%$ change in the parameter value, for a low and a high isolation effectiveness $r$, and a low and a high value of $f_T$, while keeping the other parameter values as presented in Table \\ref{tab:ParamsDef}.} \\label{r0change1}\n\\vspace{-6mm}\n\\begin{center}\n\\begin{tabular}{|c|c|c|c|c|c|c|c|c|}\n\\hline\n&Parameter & $\\beta$ & $r$ & $\\ell$ & $\\gamma_r$ & $\\gamma$ & $\\alpha$ & $f_T$ \\\\\n\\hline\n\n &\\% change & 1\\% & -0.23\\% & 0.423\\%  & -0.423\\% & -0.382\\% & -0.195\\% & -0.119\\% \\\\\n $f_T = 0.25$& for $r = 0.35$ &  &  &    &   &   &   &  \\\\\n\\cline{2-9}\n&\\% change & 1\\% & -1.014\\% & 0.053\\%  & -0.053\\% & -0.445\\% & -0.501\\% & -0.306\\% \\\\\n  & for $r = 0.95$ &  &  &    &   &   &   &  \\\\\n\\hline\n&\\% change & 1\\% & -0.402\\% & 0.747\\%  & -0.747\\% & -0.167\\% & -0.086\\% & -0.471\\% \\\\\n $f_T = 0.75$& for $r = 0.35$ &  &  &    &   &   &   &  \\\\\n\\cline{2-9}\n&\\% change & 1\\% & -3.521\\% & 0.185\\%  & -0.185\\% & -0.383\\% & -0.431\\% & -2.373\\% \\\\\n  & for $r = 0.95$ &  &  &    &   &   &   &  \\\\\n\\hline\n\\end{tabular}\n\\end{center}\n\\end{table}\n\n\n\n\n\\begin{figure}[h!]\n\\begin{center}\n\\includegraphics[scale=0.57]{R0SAnew11.jpg}\n\\caption{Percent change in $\\mathcal{R}_c$ with respect to a $1\\%$ change in the parameter value, for a low value of $f_T$ ($f_T = 0.25$) and two different levels of isolation effectiveness ($r = 0.35$ and $r = 0.95$). The other parameter values are kept as shown in Table \\ref{tab:ParamsDef}.} \\label{graphofR01}\n\\end{center}\n\\end{figure}\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\\indent We use the parameters values from Table \\ref{tab:ParamsDef} to study the sensitivity of $\\mathcal{R}_c$ to each parameter. We compute normalized sensitivity analysis on all parameters, but we just consider the impact of parameters that are the most sensitive: $\\beta, r, \\ell, \\gamma_r, \\gamma, \\alpha$, and $f_T$. The other parameters ($\\mu, \\kappa_1$, and  $\\kappa_2$)  have a very low impact, namely less than $0.001\\%$. The numerical simulations to the sensitivity of $\\mathcal{R}_c$ with respect to each of the most sensitive parameters are given in Table \\ref{r0change1}, for two different levels of isolation effectiveness ($r = 0.35$ and $r = 0.95$) and two values of $f_T$ ($f_T = 0.25$ and $f_T = 0.75$), which is the fraction of pre-symptomatic individuals diagnosed and isolated. The other parameter values are kept as shown in Table \\ref{tab:ParamsDef}.\n\\newline\n\\indent A graphical illustration of the numerical results for the scenario when $f_T = 0.25$ and the two levels of isolation effectiveness ($r = 0.35$ and $r = 0.95$) is given in Figure \\ref{graphofR01}. In the case of high isolation effectiveness ($r = 0.95$), simulations show that both the removal rate, $\\gamma_r$, of isolated individuals and the relative transmissibility parameter $\\ell$ of isolated individuals with respect to infectious individuals are the least sensitive parameters (with $0.053 \\%$ change of $\\mathcal{R}_c$), while the parameter of isolation effectiveness, $r$, is the most sensitive one, where a $1\\%$ increase in $r$ causes a $1.014 \\%$ reduction in the value of $\\mathcal{R}_c$. Also, the rate at which infectious individuals get isolated, $\\alpha$, and the fraction of pre-symptomatic individuals detected and isolated, $f_T$, impact negatively on the level of $\\mathcal{R}_c$, where a $1 \\%$ percent increase in the value of $f_T$ causes approximately a $0.31\\%$ decline in the value of the reproduction number $\\mathcal{R}_c$. Thus, as pre-symptomatic individuals are diagnosed and as isolation is highly effective, the number of available infectious individuals who are capable of transmitting Ebola decreases and therefore, the reproduction number decreases. Also, the removal (by recovery or Ebola-induced death) rate $\\gamma$ of infectious individuals affects negatively on $\\mathcal{R}_c$. Hence, for the case of highly effective isolation, the parameters concerning early diagnosis and isolation have a significant impact on the reproduction number. \n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\\indent This percent impact of the parameters on $\\mathcal{R}_c$ remains so as long as isolation is highly effective. However, if the effectiveness of isolation is low, in the sense that all parameter values are kept the same except the value of the parameter $r$, which is reduced to $0.35$, then we get the results presented in Table \\ref{r0change1} and Figure \\ref{graphofR01}. In this case, both the relative transmissibility $\\ell$ and the removal rate of isolated individuals, $\\gamma_r$, are the second most sensitive parameters, after $\\beta$ which is the most impactful one. Also, $\\ell$ became more sensitive than $r$. The implication is that, when isolation is less effective, there exists the possibility for isolated people to make successful contacts with susceptible individuals and therefore the possibility of causing new infections increases. This causes an increase in the reproduction number. Also, it is noted that the effect of $f_T$ and $\\alpha$ is reduced, which means that diagnosing and isolating infected individuals becomes a weak strategy if the effectiveness of isolation is low.\n \n\n\n\\begin{figure}[h!]\n\\begin{center}\n\\includegraphics[scale=0.585]{R0SAnew12.jpg}\n\\caption{Percent change in $\\mathcal{R}_c$ with respect to a $1 \\%$ change in the parameter value, for a high value of $f_T$ ($f_T = 0.75$) and two different levels of isolation effectiveness ($r = 0.35$ and $r = 0.95$). The other parameter values are kept as shown in Table \\ref{tab:ParamsDef}.} \\label{graphofR02}\n\\end{center}\n\\end{figure}\n\nOn repeating the previous analyses, but this time for a higher value of $f_T$ ($f_T = 0.75$), we obtain the results shown in Table  \\ref{r0change1}, which are also illustrated in Figure \\ref{graphofR02}. In comparison to the scenario when $f_T = 0.25$, the simulations show that increasing the fraction of pre-symptomatic individuals who are diagnosed and isolated,  $f_T$, increases the percent impact of the parameters $r, \\ell, \\gamma_r,$ and $f_T$, and decreases the percent impact of the parameters $\\gamma$ and $\\alpha$, on the value of the control reproduction number $\\mathcal{R}_c$.   \n\n \n\n  \n\n\n\\subsection{\\protect\\normalsize Impact of early detection and isolation on the value of $\\mathcal{R}_c$}\n\n \\begin{figure}[H]\n\\centering\n\\includegraphics[scale=0.55]{R0fT.jpg}\n\\caption{Impact of early detection of pre-symptomatic individuals on the value of $\\mathcal{R}_c$.}\n\\label{fig:R0fT}\n\\end{figure}\n\n\nTo study the impact of early detection of pre-symptomatic individuals and isolation on the reproduction number, we first depict $\\mathcal{R}_c$ as a function of $f_T$, for different levels of isolation effectiveness $r$. Figure \\ref{fig:R0fT} shows that the control reproduction number declines as the proportion, $f_T$, of pre-symptomatic individuals, who get diagnosed and isolated, increases.  Simulations are done using parameter values from Table \\ref{tab:ParamsDef}, but for three different values of $r$. It further shows that the curve corresponding to a low and an intermidate value of isolation effectivenes $r$ (e.g. $r = 0.35$ for the solid curve and $r = 0.65$ for the dashed curve) hits $\\mathcal{R}_c = 1$ at some critical value of $f_T$ (say $f_T^{\\star}$), while for the high value of $r$ ($r = 0.95$), it never hits the critical threshold $\\mathcal{R}_c = 1$, as the curve is totally below the critical threshold. This indicates that for a high effectiveness of isolation, the control reproduction number is less than one and therefore the infection dies out. Analytically, the exact form of $f_T^{\\star}$ is \n\\begin{equation}\nf_T^{\\star} = \\left[ 1 + (1-r)\\ell \\frac{\\alpha}{\\gamma_r + \\mu} - \\frac{1}{\\mathcal{R}_0} \\left(1 + \\frac{\\alpha}{\\gamma+\\mu}\\right) \\right] / \\left[ 1 - \\frac{(1-r)\\ell(\\gamma + \\mu)}{\\gamma_r + \\mu} \\right].  \\label{fTcond1}\n\\end{equation}\nThe critical proportion $f_T^{\\star}$ represents the minimum proportion of pre-symptomatic individuals who are detected and get isolated to ensure an effective control of Ebola. This critical value remains feasible as long as the following inequality holds\n\n\n\n\\begin{equation}\n(1-r)\\ell < \\frac{\\gamma_r + \\mu}{(\\gamma + \\mu)\\mathcal{R}_0}. \\label{fTcond20}\n\\end{equation}\nIf we keep all parameters fixed except $r$, then condition (\\ref{fTcond20}) could be rewritten in a more convenient form\n\n\n\n\\begin{equation}\nr > 1- \\frac{\\gamma_r + \\mu}{\\ell (\\gamma + \\mu) \\mathcal{R}_0}. \\label{fTcond3}\n\\end{equation}\nThis gives the minimum level of effectiveness of isolation required to obtain an isolation and early diagnosis-based control strategy for Ebola tranmission. \n\n\n\n\n \\begin{figure}[H]\n\\centering\n\\includegraphics[scale=0.55]{R0alpha.jpg}\n\\caption{Impact of isolating infectious individuals on the value of $\\mathcal{R}_c$.}\n\\label{fig:R0alpha}\n\\end{figure}\n\nNow, we could also ask a similar question on the role of isolating infectious individuals to contain Ebola transmission. Figure \\ref{fig:R0alpha} shows the impact of changing the rate at which infectious individuals get isolated, $\\alpha$, on $\\mathcal{R}_c$, for the same three different levels of isolation effectivenes, as used above. The analysis shows that it is possible to control the epidemic if and only if $\\alpha > \\alpha^\\star$, where\n\\begin{equation}\n\\alpha^\\star = \\frac{[ (1-f_T)(\\gamma_r + \\mu)(\\gamma+\\mu) + (1-r)\\ell f_T (\\gamma+\\mu)^2]\\mathcal{R}_0 -  (\\gamma_r + \\mu)(\\gamma+\\mu) }{(\\gamma_r + \\mu) - \\ell (1-r) \\mathcal{R}_0(\\gamma + \\mu)}\n\\end{equation}\n\n\n\n\nand with the implementation of condition (\\ref{fTcond20}).\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n3.4 \\protect\\normalsize Impact of early detection and isolation on the value of $\\mathcal{R}_c$\n\\subsection{\\protect\\normalsize Impact of early detection and isolation on the value of $\\mathcal{R}_c$}\n\n \\begin{figure}[H]\n\\centering\n\\includegraphics[scale=0.55]{R0fT.jpg}\n\\caption{Impact of early detection of pre-symptomatic individuals on the value of $\\mathcal{R}_c$.}\n\\label{fig:R0fT}\n\\end{figure}\n\n\nTo study the impact of early detection of pre-symptomatic individuals and isolation on the reproduction number, we first depict $\\mathcal{R}_c$ as a function of $f_T$, for different levels of isolation effectiveness $r$. Figure \\ref{fig:R0fT} shows that the control reproduction number declines as the proportion, $f_T$, of pre-symptomatic individuals, who get diagnosed and isolated, increases.  Simulations are done using parameter values from Table \\ref{tab:ParamsDef}, but for three different values of $r$. It further shows that the curve corresponding to a low and an intermidate value of isolation effectivenes $r$ (e.g. $r = 0.35$ for the solid curve and $r = 0.65$ for the dashed curve) hits $\\mathcal{R}_c = 1$ at some critical value of $f_T$ (say $f_T^{\\star}$), while for the high value of $r$ ($r = 0.95$), it never hits the critical threshold $\\mathcal{R}_c = 1$, as the curve is totally below the critical threshold. This indicates that for a high effectiveness of isolation, the control reproduction number is less than one and therefore the infection dies out. Analytically, the exact form of $f_T^{\\star}$ is \n\\begin{equation}\nf_T^{\\star} = \\left[ 1 + (1-r)\\ell \\frac{\\alpha}{\\gamma_r + \\mu} - \\frac{1}{\\mathcal{R}_0} \\left(1 + \\frac{\\alpha}{\\gamma+\\mu}\\right) \\right] / \\left[ 1 - \\frac{(1-r)\\ell(\\gamma + \\mu)}{\\gamma_r + \\mu} \\right].  \\label{fTcond1}\n\\end{equation}\nThe critical proportion $f_T^{\\star}$ represents the minimum proportion of pre-symptomatic individuals who are detected and get isolated to ensure an effective control of Ebola. This critical value remains feasible as long as the following inequality holds\n\n\n\n\\begin{equation}\n(1-r)\\ell < \\frac{\\gamma_r + \\mu}{(\\gamma + \\mu)\\mathcal{R}_0}. \\label{fTcond20}\n\\end{equation}\nIf we keep all parameters fixed except $r$, then condition (\\ref{fTcond20}) could be rewritten in a more convenient form\n\n\n\n\\begin{equation}\nr > 1- \\frac{\\gamma_r + \\mu}{\\ell (\\gamma + \\mu) \\mathcal{R}_0}. \\label{fTcond3}\n\\end{equation}\nThis gives the minimum level of effectiveness of isolation required to obtain an isolation and early diagnosis-based control strategy for Ebola tranmission. \n\n\n\n\n \\begin{figure}[H]\n\\centering\n\\includegraphics[scale=0.55]{R0alpha.jpg}\n\\caption{Impact of isolating infectious individuals on the value of $\\mathcal{R}_c$.}\n\\label{fig:R0alpha}\n\\end{figure}\n\nNow, we could also ask a similar question on the role of isolating infectious individuals to contain Ebola transmission. Figure \\ref{fig:R0alpha} shows the impact of changing the rate at which infectious individuals get isolated, $\\alpha$, on $\\mathcal{R}_c$, for the same three different levels of isolation effectivenes, as used above. The analysis shows that it is possible to control the epidemic if and only if $\\alpha > \\alpha^\\star$, where\n\\begin{equation}\n\\alpha^\\star = \\frac{[ (1-f_T)(\\gamma_r + \\mu)(\\gamma+\\mu) + (1-r)\\ell f_T (\\gamma+\\mu)^2]\\mathcal{R}_0 -  (\\gamma_r + \\mu)(\\gamma+\\mu) }{(\\gamma_r + \\mu) - \\ell (1-r) \\mathcal{R}_0(\\gamma + \\mu)}\n\\end{equation}\n\n\n\n\nand with the implementation of condition (\\ref{fTcond20}).\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n",
        "label": "fig:R0fT",
        "Descriptive_question1": "What is the value of isolation effectiveness represented by the solid curve in figure_4?",
        "Descriptive_question2": "Which curve in figure_4 corresponds to an isolation effectiveness value of 0.95?",
        "Reasoning_question1": "Why does the control reproduction number decline as the proportion of pre-symptomatic individuals diagnosed and isolated increases in figure_4?",
        "Reasoning_question2": "How does a high isolation effectiveness value impact the likelihood of the infection dying out according to figure_4?",
        "Descriptive_answer1": "0.35",
        "Descriptive_answer2": "dotted",
        "Reasoning_answer1": "The control reproduction number declines as the proportion of pre-symptomatic individuals diagnosed and isolated increases because isolating these individuals early reduces the number of infectious contacts they can make, thereby limiting the spread of the disease. As a greater fraction of pre-symptomatic individuals are detected and isolated promptly, fewer secondary infections occur, lowering the reproduction number.",
        "Reasoning_answer2": "A high isolation effectiveness value (such as 0.95) results in the control reproduction number remaining below the critical threshold of 1 across all values of the proportion of pre-symptomatic individuals diagnosed and isolated. This means that the infection cannot sustain transmission and will eventually die out, indicating that highly effective isolation is a key factor in controlling the epidemic."
    }
]