\section{Wavefunction optimization}
\label{sec:optimization}

Optimizing wavefunction is critical in all kinds of real-space QMC calculations
because it significantly improves both the accuracy and efficiency of computation.
However, it is very difficult to directly adopt deterministic minimization approaches because of the stochastic nature of evaluating quantities with MC.
Thanks to the algorithmic breakthrough during the first decade of this century and the tremendous computer power available,
it is now feasible to optimize tens of thousands of parameters in a wavefunction for a solid or molecule.
QMCPACK has multiple optimizers implemented based on the state-of-the-art linear method.
We are continually improving our optimizers for robustness and friendliness and are trying to provide a single solution.
Because of the large variation of wavefunction types carrying distinct characteristics, using several optimizers might be needed in some cases.
We strongly suggested reading recommendations from the experts who maintain these optimizers.

A typical optimization block looks like the following. It starts with method=``linear" and contains three blocks of parameters.
\begin{lstlisting}[style=QMCPXML]
 <loop max="10">
  <qmc method="linear" move="pbyp" gpu="yes">
    <!-- Specify the VMC options -->
    <parameter name="walkers">              256 </parameter>
    <parameter name="samples">          2867200 </parameter>
    <parameter name="stepsbetweensamples">    1 </parameter>
    <parameter name="substeps">               5 </parameter>
    <parameter name="warmupSteps">            5 </parameter>
    <parameter name="blocks">                70 </parameter>
    <parameter name="timestep">             1.0 </parameter>
    <parameter name="usedrift">              no </parameter>
    <estimator name="LocalEnergy" hdf5="no"/>
    ...
    <!-- Specify the correlated sampling options and define the cost function -->
    <parameter name="minwalkers">            0.3 </parameter>
         <cost name="energy">               0.95 </cost>
         <cost name="unreweightedvariance"> 0.00 </cost>
         <cost name="reweightedvariance">   0.05 </cost>
    ...
    <!-- Specify the optimizer options -->
    <parameter name="MinMethod">    OneShiftOnly </parameter>
    ...
  </qmc>
 </loop>
\end{lstlisting}
\begin{itemize}
\item Loop is helpful to repeatedly execute identical optimization blocks.
\item The first part is highly identical to a regular VMC block.
\item The second part is to specify the correlated sampling options and define the cost function.
\item The last part is used to specify the options of different optimizers, which can be very distinct from one to another.
\end{itemize}

\subsection{VMC run for the optimization}
The VMC calculation for the wavefunction optimization has a strict requirement
that \ixml{samples} or \ixml{samplesperthread} must be specified because of the optimizer needs for the stored \ixml{samples}.
The input parameters of this part are identical to the VMC method.

Recommendations:
\begin{itemize}
\item Run the inclusive VMC calculation correctly and efficiently because this takes a significant amount of time during optimization.
For example, make sure the derived \ixml{steps} per block is 1 and use larger \ixml{substeps} to control the correlation between \ixml{samples}.
\item A reasonable starting wavefunction is necessary. A lot of optimization fails because of a bad wavefunction starting point.
The sign of a bad initial wavefunction includes but is not limited to a very long equilibration time, low acceptance ratio, and huge variance.
The first thing to do after a failed optimization is to check the information provided by the VMC calculation via \texttt{*.scalar.dat files}.
\end{itemize}

\subsection{Correlated sampling and cost function}
After generating the samples with VMC, the derivatives of the wavefunction with respect to the parameters are computed for proposing a new set of parameters by optimizers.
And later, a correlated sampling calculation is performed to quickly evaluate values of the cost function on the old set of parameters and the new set for further decisions.
The input parameters are listed in the following table.
\begin{table}[h]
\begin{center}
\begin{tabularx}{\textwidth}{l l l l l X }
\hline
\multicolumn{6}{l}{\texttt{linear} method} \\
\hline
\multicolumn{2}{l}{parameters}  & \multicolumn{4}{l}{}\\
   &   \bfseries name     & \bfseries datatype & \bfseries values & \bfseries default   & \bfseries description \\
   &   \texttt{nonlocalpp} &  text     & yes, no & no  & include non-local PP energy in the cost function\\
%   &   \texttt{GEVMethod} &  text     & mixed, H2 & mixed  & methods of generalized eigenvalue problem\\
%   &   \texttt{beta} &  real     & any value & 0.0  & a parameter for GEVMethod\\
%   &   \texttt{use\_nonlocalpp\_deriv} &  text     & yes, no & no  & include the derivatives of non-local PP\\
   &   \texttt{minwalkers} &  real     & 0--1   & 0.3 & Lower bound of the effective weight\\
   &   \texttt{maxWeight} &  real     & $>1$   & 1e6 & Maximum weight allowed in reweighting\\
  \hline
\end{tabularx}
\end{center}
\end{table}

Additional information:
\begin{itemize}
\item \ixml{maxWeight}: The default should be good.
\item \ixml{nonlocalpp}: The \texttt{nonlocalpp} contribution to the local energy depends on the wavefunction.
When a new set of parameters is proposed, this contribution needs to be updated if the cost function consists of local energy.
Fortunately, nonlocal contribution is chosen small when making a PP for small locality error.
We can ignore its change and avoid the expensive computational cost.
An implementation issue with GPU code is that a large amount of memory is consumed with this option.
\item \ixml{minwalkers}: This is a \textit{critical} parameter. When the ratio of effective samples to actual number of samples in a reweighting step goes lower than \ixml{minwalkers},
the proposed set of parameters is invalid. % The last set of acceptable parameters is kept.
\end{itemize}

The cost function consists of three components: energy, unreweighted variance, and reweighted variance.
\begin{lstlisting}[style=QMCPXML]
     <cost name="energy">                   0.95 </cost>
     <cost name="unreweightedvariance">     0.00 </cost>
     <cost name="reweightedvariance">       0.05 </cost>
\end{lstlisting}

\subsection{Optimizers}
QMCPACK implements a number of different optimizers each with different priorities for accuracy, convergence, memory usage, and stability.
The optimizers can be switched among  ``OneShiftOnly'' (default), ``adaptive,'' ``descent,'' ``hybrid,'' and ``quartic'' (old) using the following line in the optimization block:
\begin{lstlisting}
<parameter name="MinMethod"> THE METHOD YOU LIKE </parameter>
\end{lstlisting}

\subsection{OneShiftOnly Optimizer}
The OneShiftOnly optimizer targets a fast optimization by moving parameters more aggressively. It works with OpenMP and GPU and can be considered for large systems.
This method relies on the effective weight of correlated sampling rather than the cost function value to justify a new set of parameters.
If the effective weight is larger than \ixml{minwalkers}, the new set is taken whether or not the cost function value decreases.
If a proposed set is rejected, the standard output prints the measured ratio of effective samples to the total number of samples
and adjustment on \ixml{minwalkers} can be made if needed.

\begin{table}[h]
\begin{center}
\begin{tabularx}{\textwidth}{l l l l l X }
\hline
\multicolumn{6}{l}{\texttt{linear} method} \\
\hline
\multicolumn{2}{l}{parameters}  & \multicolumn{4}{l}{}\\
   &   \bfseries name     & \bfseries datatype & \bfseries values & \bfseries default   & \bfseries description \\
   &   \texttt{shift\_i} &  real     & $>0$ & 0.01 & Direct stabilizer added to the Hamiltonian matrix\\
   &   \texttt{shift\_s} &  real     & $>0$ & 1.00 & Initial stabilizer based on the overlap matrix\\
  \hline
\end{tabularx}
\end{center}
\end{table}

Additional information:
\begin{itemize}
\item \ixml{shift_i}: This is the direct term added to the diagonal of the Hamiltonian matrix.
                         It provides more stable but slower optimization with a large value.
\item \ixml{shift_s}: This is the initial value of the stabilizer based on the overlap matrix added to the Hamiltonian matrix.
                         It provides more stable but slower optimization with a large value. The used value is auto-adjusted by the optimizer.
\end{itemize}

Recommendations:
\begin{itemize}
  \item Default \ixml{shift_i}, \ixml{shift_s} should be fine.
  \item For hard cases, increasing \ixml{shift_i} (by a factor of 5 or 10) can significantly stabilize the optimization by reducing the pace towards the optimal parameter set.
  \item If the VMC energy of the last optimization iterations grows significantly, increase \ixml{minwalkers} closer to 1 and make the optimization stable.
  \item If the first iterations of optimization are rejected on a reasonable initial wavefunction,
        lower the \ixml{minwalkers} value based on the measured value printed in the standard output to accept the move.
\end{itemize}

We recommended using this optimizer in two sections with a very small \ixml{minwalkers} in the first and a large value in the second, such as the following.
In the very beginning, parameters are far away from optimal values and large changes are proposed by the optimizer.
Having a small \ixml{minwalkers} makes it much easier to accept these changes.
When the energy gradually converges, we can have a large \ixml{minwalkers} to avoid risky parameter sets.
\begin{lstlisting}[style=QMCPXML]
 <loop max="6">
  <qmc method="linear" move="pbyp" gpu="yes">
    <!-- Specify the VMC options -->
    <parameter name="walkers">                1 </parameter>
    <parameter name="samples">            10000 </parameter>
    <parameter name="stepsbetweensamples">    1 </parameter>
    <parameter name="substeps">               5 </parameter>
    <parameter name="warmupSteps">            5 </parameter>
    <parameter name="blocks">                25 </parameter>
    <parameter name="timestep">             1.0 </parameter>
    <parameter name="usedrift">              no </parameter>
    <estimator name="LocalEnergy" hdf5="no"/>
    <!-- Specify the optimizer options -->
    <parameter name="MinMethod">    OneShiftOnly </parameter>
    <parameter name="minwalkers">           1e-4 </parameter>
  </qmc>
 </loop>
 <loop max="12">
  <qmc method="linear" move="pbyp" gpu="yes">
    <!-- Specify the VMC options -->
    <parameter name="walkers">                1 </parameter>
    <parameter name="samples">            20000 </parameter>
    <parameter name="stepsbetweensamples">    1 </parameter>
    <parameter name="substeps">               5 </parameter>
    <parameter name="warmupSteps">            2 </parameter>
    <parameter name="blocks">                50 </parameter>
    <parameter name="timestep">             1.0 </parameter>
    <parameter name="usedrift">              no </parameter>
    <estimator name="LocalEnergy" hdf5="no"/>
    <!-- Specify the optimizer options -->
    <parameter name="MinMethod">    OneShiftOnly </parameter>
    <parameter name="minwalkers">            0.5 </parameter>
  </qmc>
 </loop>
\end{lstlisting}

For each optimization step, you will see\\
\begin{lstlisting}
The new set of parameters is valid. Updating the trial wave function!
\end{lstlisting}\par
or\par
\begin{lstlisting}
The new set of parameters is not valid. Revert to the old set!
\end{lstlisting}
Occasional rejection is fine. Frequent rejection indicates potential problems, and users should inspect the VMC calculation or change optimization strategy.
To track the progress of optimization, use the command \texttt{qmca -q ev *.scalar.dat} to look at the VMC energy and variance for each optimization step.

\subsection{Adaptive Optimizer}

The default setting of the adaptive optimizer is to construct the linear method Hamiltonian and overlap matrices explicitly and add different shifts to the Hamiltonian matrix
as ``stabilizers.''
The generalized eigenvalue problem is solved for each shift to obtain updates to the wavefunction parameters.
Then a correlated sampling is performed for each shift's updated wavefunction and the initial trial wavefunction
using the middle shift's updated wavefunction as the guiding function.
The cost function for these wavefunctions is compared, and the update corresponding to the best cost function is selected.
In the next iteration, the median magnitude of the stabilizers is set to the magnitude that generated the best update in the current iteration, thus adapting the magnitude of
the stabilizers automatically.

When the trial wavefunction contains more than 10,000 parameters, constructing and storing the linear method matrices could become a memory bottleneck.
To avoid explicit construction of these matrices, the adaptive optimizer implements the block linear method (BLM) approach. \cite{Zhao:2017:blocked_lm}
The BLM tries to find an approximate
solution\:  $\vec{c}_{opt}$ to the standard LM generalized eigenvalue problem by dividing the variable space into a number of blocks
and making intelligent estimates for which directions within those blocks will be most important for constructing\:   $\vec{c}_{opt}$,
which is then obtained by solving a smaller, more memory-efficient
eigenproblem in the basis of these supposedly important block-wise directions.

\begin{table}[h]
\begin{center}
\begin{tabularx}{\textwidth}{l l l l l X }
\hline
\multicolumn{6}{l}{\texttt{linear} method} \\
\hline
\multicolumn{2}{l}{parameters}  & \multicolumn{4}{l}{}\\
   &   \bfseries name     & \bfseries datatype & \bfseries values & \bfseries default   & \bfseries description \\
   %&   \texttt{stepsize} &  real     & 0--1 & 0.25  & Step size for moving parameters\\
   &   \texttt{max\_relative\_change} &  real     & $>0$ & 10.0 & Allowed change in cost function\\
   &   \texttt{max\_param\_change} &  real     & $>0$ & 0.3 & Allowed change in wavefunction parameter\\
   &   \texttt{shift\_i} &  real     & $>0$ & 0.01 & Initial diagonal      stabilizer added to the Hamiltonian matrix\\
   &   \texttt{shift\_s} &  real     & $>0$ & 1.00 & Initial overlap-based stabilizer added to the Hamiltonian matrix\\
   &   \texttt{target\_shift\_i} &  real     & any & -1.0 & Diagonal stabilizer value aimed for during adaptive method (disabled if $\leq$ 0)\\
   &   \texttt{cost\_increase\_tol} &  real     & $\geq 0$ & 0.0 & Tolerance for cost function increases\\
   &   \texttt{chase\_lowest} &  text   & yes, no & yes & Chase the lowest eigenvector in iterative solver\\
   &   \texttt{chase\_closest} &  text   & yes, no & no & Chase the eigenvector closest to initial guess\\
   &   \texttt{block\_lm} &  text   & yes, no & no & Use BLM\\
   &   \texttt{nblocks} &  integer   & $>0$ &  & Number of blocks in BLM\\
   &   \texttt{nolds} &  integer   & $>0$ &  & Number of old update vectors used in BLM\\
   &   \texttt{nkept} &  integer   & $>0$ &  & Number of eigenvectors to keep per block in BLM\\
  \hline
\end{tabularx}
\end{center}
\end{table}

Additional information:
\begin{itemize}
  \item \ixml{shift_i}:  This is the initial coefficient used to scale the diagonal stabilizer.
                            More stable but slower optimization is expected with a large value.
                            The adaptive method will automatically adjust this value after each linear method iteration.
  \item \ixml{shift_s}:  This is the initial coefficient used to scale the overlap-based stabilizer.
                            More stable but slower optimization is expected with a large value.
                            The adaptive method will automatically adjust this value after each linear method iteration.
  \item \ixml{target_shift_i}:  If set greater than zero, the adaptive method will choose the update whose shift\_i value is closest to
                              this target value so long as the associated cost is within cost\_increase\_tol of the lowest cost.
                              Disable this behavior by setting target\_shift\_i to a negative number.
  \item \ixml{cost_increase_tol}:  Tolerance for cost function increases when selecting the best shift.
  \item \ixml{nblocks}:   This is the number of blocks used in BLM. The amount of memory required to store LM matrices decreases
                            as the number of blocks increases. But the error introduced by BLM would increase as the number of blocks increases.
  \item \ixml{nolds}:     In BLM, the interblock correlation is accounted for by including a small number of wavefunction update vectors
                            outside the block. Larger \ixml{nolds} would include more interblock correlation and more accurate results but
                            also higher memory requirements.
  \item \ixml{nkept}:     This is the number of update directions retained from each block in the BLM. If all directions are retained in each block,
                            then the BLM becomes equivalent to the standard LM.  Retaining five or fewer directions per block is often sufficient.
\end{itemize}

Recommendations:
\begin{itemize}
  \item Default \ixml{shift_i}, \ixml{shift_s} should be fine.
  \item When there are fewer than about 5,000 variables being optimized, the traditional LM is preferred because it has a lower overhead than the BLM when the number of variables is small.
  \item Initial experience with the BLM suggests that a few hundred blocks and a handful of \ixml{nolds} and \ixml{nkept}
        often provide a good balance between memory use and accuracy.  In general, using fewer blocks should be more accurate but would require more memory.
\end{itemize}

\begin{lstlisting}[style=QMCPXML]
 <loop max="15">
  <qmc method="linear" move="pbyp">
    <!-- Specify the VMC options -->
    <parameter name="walkers">                1 </parameter>
    <parameter name="samples">            20000 </parameter>
    <parameter name="stepsbetweensamples">    1 </parameter>
    <parameter name="substeps">               5 </parameter>
    <parameter name="warmupSteps">            5 </parameter>
    <parameter name="blocks">                50 </parameter>
    <parameter name="timestep">             1.0 </parameter>
    <parameter name="usedrift">              no </parameter>
    <estimator name="LocalEnergy" hdf5="no"/>
    <!-- Specify the correlated sampling options and define the cost function -->
         <cost name="energy">               1.00 </cost>
         <cost name="unreweightedvariance"> 0.00 </cost>
         <cost name="reweightedvariance">   0.00 </cost>
    <!-- Specify the optimizer options -->
    <parameter name="MinMethod">adaptive</parameter>
    <parameter name="max_relative_cost_change">10.0</parameter>
    <parameter name="shift_i"> 1.00 </parameter>
    <parameter name="shift_s"> 1.00 </parameter>
    <parameter name="max_param_change"> 0.3 </parameter>
    <parameter name="chase_lowest"> yes </parameter>
    <parameter name="chase_closest"> yes </parameter>
    <parameter name="block_lm"> no </parameter>
    <!-- Specify the BLM specific options if needed
      <parameter name="nblocks"> 100 </parameter>
      <parameter name="nolds"> 5 </parameter>
      <parameter name="nkept"> 3 </parameter>
    -->
  </qmc>
 </loop>
\end{lstlisting}
%To activate this optimizer, add ``-D BUILD\_LMYENGINE\_INTERFACE=1'' in the CMake command line.

The adaptive optimizer is also able to optimize individual excited states directly. \cite{Zhao:2016:dir_tar}
In this case, it tries to minimize the following function:
\begin{equation*}
\Omega[\Psi]=\frac{\left<\Psi|\omega-H|\Psi\right>}{\left<\Psi|{\left(\omega-H\right)}^2|\Psi\right>}\:.
\end{equation*}
The global minimum of this function corresponds to the state whose energy lies immediately above the shift parameter $\omega$ in the energy spectrum.
For example, if $\omega$ were placed in between the ground state energy and the first excited state energy and the wavefunction ansatz was capable of a good
description for the first excited state, then the wavefunction would be optimized for the first excited state.
Note that if the ansatz is not capable of a good description of the excited state in question, the optimization could converge to a different
state, as is known to occur in some circumstances for traditional ground state optimizations.
Note also that the ground state can be targeted by this method by choosing $\omega$ to be below the ground state energy, although we should stress that this
is not the same thing as a traditional ground state optimization and will in general give a slightly different wavefunction.
Excited state targeting requires two additional parameters, as shown in the following table.\\

\begin{table}[h]
\begin{center}
\begin{tabularx}{\textwidth}{l l l l l X }
\hline
\multicolumn{6}{l}{Excited state targeting} \\
\hline
\multicolumn{2}{l}{parameters}  & \multicolumn{4}{l}{}\\
   &   \bfseries name     & \bfseries datatype & \bfseries values & \bfseries default   & \bfseries description \\
   %&   \texttt{stepsize} &  real     & 0--1 & 0.25  & Step size for moving parameters\\
   &   \texttt{targetExcited} &  text   & yes, no      & no   & Whether to use the excited state targeting optimization\\
   &   \texttt{omega}         &  real   & real numbers & none & Energy shift used to target different excited states\\
  \hline
\end{tabularx}
\end{center}
\end{table}

Excited state recommendations:
\begin{itemize}
  \item Because of the finite variance in any approximate wavefunction, we recommended setting $\omega=\omega_0-\sigma$, where $\omega_0$ is placed just
        below the energy of the targeted state and $\sigma^2$ is the energy variance.
  \item To obtain an unbiased excitation energy, the ground state should be optimized with the excited state variational principle as well by setting
        \ixml{omega} below the ground state energy.  Note that using the ground state variational principle for the ground state and the excited state variational
        principle for the excited state creates a bias in favor of the ground state.
\end{itemize}

\subsection{Descent Optimizer}
Gradient descent algorithms are an alternative set of optimization methods to the OneShiftOnly and adaptive optimizers based on the linear method.
These methods use only first derivatives to optimize trial wave functions and convergence can be accelerated by retaining a memory of previous derivative values.
Multiple flavors of accelerated descent methods are available. They differ in details such as the schemes for adaptive adjustment of step sizes.\cite{Otis2019}
Descent algorithms avoid the construction of matrices that occurs in the linear method and consequently can be applied to larger sets of
optimizable parameters.
Currently, descent optimization is only available for ground state calculations.
Parameters for descent are shown in the table below.\\

\begin{table}[H]
\begin{center}
\begin{tabularx}{\textwidth}{l l l l l X }
\hline
\multicolumn{6}{l}{\texttt{descent} method} \\
\hline
\multicolumn{2}{l}{parameters}  & \multicolumn{4}{l}{}\\
   &   \bfseries name     & \bfseries datatype & \bfseries values & \bfseries default   & \bfseries description \\
   &   \texttt{flavor} &  text     & \begin{tabular}{@{}c@{}}RMSprop,Random \\ ADAM,AMSGrad\end{tabular} & RMSprop & Particular type of descent method\\
   &   \texttt{Ramp\_eta} &  text     & yes,no & no & Whether to gradually ramp up step sizes\\
   &   \texttt{Ramp\_num} &  integer     & $>0$ & 30 & Number of steps over which to ramp up step size\\
   &   \texttt{TJF\_2Body\_eta} &  real     & $>0$ & 0.01 & Step size for two body Jastrow parameters\\
   &   \texttt{TJF\_1Body\_eta} &  real     & $>0$ & 0.01 & Step size for one body Jastrow parameters\\
   &   \texttt{F\_eta} &  real     & $>0$ & 0.001 & Step size for number counting Jastrow F matrix parameters\\
   &   \texttt{Gauss\_eta} &  real     & $>0$ & 0.001 & Step size for number counting Jastrow gaussian basis parameters\\
   &   \texttt{CI\_eta} &  real     & $>0$ & 0.01 & Step size for CI parameters\\
   &   \texttt{Orb\_eta} &  real     & $>0$ & 0.001 & Step size for orbital parameters\\
  \hline
\end{tabularx}
\end{center}
\end{table}

Additional information and recommendations:
\begin{itemize}

    \item It is generally advantageous to set different step sizes for different types of parameters.
	More nonlinear parameters such as those for number counting Jastrow factors or orbitals typically require
	smaller steps sizes than those for CI coefficients or traditional Jastrow parameters. There are defaults for
	several parameter types and a default of .001 has been chosen for all other parameters.
    \item The ability to gradually ramp up step sizes to their input values is useful for avoiding spikes in the average local energy
	during early iterations of descent optimization. This initial rise in the energy occurs as a memory of past gradients is being
	built up and it may be possible for the energy to recover without ramping if there are enough iterations in the optimization.
    \item The step sizes chosen can have a substantial influence on the quality of the optimization and the final variational energy achieved.
	Larger step sizes may be helpful if there is reason to think the descent optimization is not reaching the minimum energy. There are also
	additional hyperparameters in the descent algorithms with default values.\cite{Otis2019} They seem to have limited influence
	on the effectiveness of the optimization compared to step sizes, but users can adjust them within the source code of the descent engine if they wish.
    \item The sampling effort for individual descent steps can be small compared that for linear method iterations as shown in the example input below. Something in the range of 10,000 to 30,000
	seems sufficient for molecules with tens of electrons. However, descent optimizations may require anywhere from a few hundred to a few thousand iterations.
    \item In cases where a descent optimization struggles to reach the minimum and a linear method optimization is not possible or unsatisfactory, it may be useful to try
	the hybrid optimization approach described in the next subsection.
\end{itemize}

\begin{lstlisting}[style=QMCPXML]

 <loop max="100">
    <qmc method="linear" move="pbyp" checkpoint="-1" gpu="no">

    <!-- VMC inputs -->
     <parameter name="blocks">2000</parameter>
     <parameter name="steps">1</parameter>
     <parameter name="samples">20000</parameter>
     <parameter name="warmupsteps">100</parameter>
     <parameter name="timestep">0.05</parameter>

     <parameter name="MinMethod">descent</parameter>
     <estimator name="LocalEnergy" hdf5="no"/>
     <parameter name="usebuffer">yes</parameter>

     <estimator name="LocalEnergy" hdf5="no"/>

	   <!-- Descent Inputs -->
       <parameter name="flavor">RMSprop</parameter>

       <parameter name="Ramp_eta">no</parameter>
       <parameter name="Ramp_num">30</parameter>

      <parameter name="TJF_2Body_eta">.02</parameter>
       <parameter name="TJF_1Body_eta">.02</parameter>
      <parameter name="F_eta">.001</parameter>
      <parameter name="Gauss_eta">.001</parameter>
      <parameter name="CI_eta">.1</parameter>
      <parameter name="Orb_eta">.0001</parameter>

    </qmc>
</loop>
 \end{lstlisting}

\subsection{Hybrid Optimizer}

Another optimization option is to use a hybrid combination of accelerated descent and blocked linear method.
It provides a means to retain the advantages of both individual methods while scaling to large numbers of parameters beyond the traditional 10,000 parameter limit of the linear method.\cite{Otis2019}
In a hybrid optimization, alternating sections of descent and BLM optimization are used.
Gradient descent is used to identify the previous important directions in parameter space used by the BLM, the number of which is set by the \texttt{nold} input for the BLM.
Over the course of a section of descent, vectors of parameter differences are stored and then passed to the linear method engine after the optimization changes to the BLM.
One motivation for including sections of descent is to counteract noise in linear method updates due to uncertainties in its step direction and allow for a smoother movement to the minimum.
There are two additional parameters used in the hybrid optimization and it requires a slightly different format of input to specify the constituent methods as shown below in the example.

\begin{table}[H]
\begin{center}
\begin{tabularx}{\textwidth}{l l l l l X }
\hline
\multicolumn{6}{l}{\texttt{descent} method} \\
\hline
\multicolumn{2}{l}{parameters}  & \multicolumn{4}{l}{}\\
   &   \bfseries name     & \bfseries datatype & \bfseries values & \bfseries default   & \bfseries description \\
   &   \texttt{num\_updates} &  integer     & $>0$ &  & Number of steps for a method\\
   &   \texttt{Stored\_Vectors} &  integer     & $>0$ & 5 & Number of vectors to transfer to BLM\\
  \hline
\end{tabularx}
\end{center}
\end{table}

\begin{lstlisting}[style=QMCPXML]

  <loop max="203">
  <qmc method="linear" move="pbyp" checkpoint="-1" gpu="no">
   <parameter name="Minmethod"> hybrid </parameter>

   <optimizer num_updates="100">

	<parameter name="blocks">1000</parameter>
       <parameter name="steps">1</parameter>
       <parameter name="samples">20000</parameter>
       <parameter name="warmupsteps">1000</parameter>
       <parameter name="timestep">0.05</parameter>

       <estimator name="LocalEnergy" hdf5="no"/>

       <parameter name="Minmethod"> descent </parameter>
       <parameter name="Stored_Vectors">5</parameter>
       <parameter name="flavor">RMSprop</parameter>
       <parameter name="TJF_2Body_eta">.01</parameter>
       <parameter name="TJF_1Body_eta">.01</parameter>
       <parameter name="CI_eta">.1</parameter>

       <parameter name="Ramp_eta">no</parameter>
       <parameter name="Ramp_num">10</parameter>
   </optimizer>

   <optimizer num_updates="3">

       <parameter name="blocks">2000</parameter>
       <parameter name="steps">1</parameter>
       <parameter name="samples">1000000</parameter>
       <parameter name="warmupsteps">1000</parameter>
       <parameter name="timestep">0.05</parameter>

       <estimator name="LocalEnergy" hdf5="no"/>

       <parameter name="Minmethod"> adaptive </parameter>
       <parameter name="max_relative_cost_change">10.0</parameter>
       <parameter name="max_param_change">3</parameter>
       <parameter name="shift_i">0.01</parameter>
       <parameter name="shift_s">1.00</parameter>

       <parameter name="block_lm">yes</parameter>
       <parameter name="nblocks">2</parameter>
       <parameter name="nolds">5</parameter>
       <parameter name="nkept">5</parameter>

   </optimizer>
  </qmc>
</loop>
 \end{lstlisting}

Additional information and recommendations:
\begin{itemize}

    \item In the example above, the input for \texttt{loop} gives the total number of steps for the full optimization while the inputs for \texttt{num\_updates} specify
	the number of steps in the constituent methods. For this case, the optimization would begin with 100 steps of descent using the parameters in the first \texttt{optimizer} block
	and then switch to the BLM for 3 steps before switching back to descent for the final 100 iterations of the total of 203.
    \item The design of the hybrid method allows for more than two \texttt{optimizer} blocks to be used and the optimization will cycle through the individual methods. However, the effectiveness
	of this in terms of the quality of optimization results is unexplored.
    \item As the descent algorithms are currently only implemented for ground state optimizations, this hybrid combination of them with the BLM is also restricted to the ground state for now.
    \item It can be useful to follow a hybrid optimization with a section of pure descent optimization and take an average energy over the last few hundred iterations as the final variational energy.
	This approach can achieve a lower statistical uncertainty on the energy for less overall sampling effort compared to what a pure linear method optimization would require.
\end{itemize}

\subsection{Quartic Optimizer}
\textit{This is an older optimizer method retained for compatibility. We recommend starting with the newest OneShiftOnly or adaptive optimizers.}
The quartic optimizer fits a quartic polynomial to 7 values of the cost function obtained using reweighting along the chosen direction and determines the optimal move.
This optimizer is very robust but is a bit conservative when accepting new steps, especially when large parameters changes are proposed.
\begin{table}[h]
\begin{center}
\begin{tabularx}{\textwidth}{l l l l l X }
\hline
\multicolumn{6}{l}{\texttt{linear} method} \\
\hline
\multicolumn{2}{l}{parameters}  & \multicolumn{4}{l}{}\\
   &   \bfseries name     & \bfseries datatype & \bfseries values & \bfseries default   & \bfseries description \\
   %&   \texttt{stepsize} &  real     & 0--1 & 0.25  & Step size for moving parameters\\
   &   \texttt{bigchange} &  real     & $>0$ & 50.0  & Largest parameter change allowed\\
   &   \texttt{alloweddifference} &  real     & $>0$ & 1e-4 & Allowed increased in energy\\
   &   \texttt{exp0} &  real     & any value & -16.0 & Initial value for stabilizer\\
   &   \texttt{stabilizerscale} &  real     & $>0$ & 2.0 & Increase in value of \texttt{exp0} between iterations\\
   &   \texttt{nstabilizers} &  integer     & $>0$ & 3 & Number of stabilizers to try\\
   &   \texttt{max\_its} &  integer   & $>0$ & 1 & Number of inner loops with same samples\\
  \hline
\end{tabularx}
\end{center}
\end{table}

Additional information:
\begin{itemize}
\item \ixml{exp0}. This is the initial value for stabilizer (shift to diagonal of H). The actual value of stabilizer is $10^{\textrm{exp0}}$.
\end{itemize}

Recommendations:
\begin{itemize}
  \item{For hard cases (e.g., simultaneous optimization of long MSD and 3-Body J), set \ixml{exp0}
to 0 and do a single inner iteration (max its=1) per sample of configurations.}
\end{itemize}

\begin{lstlisting}[style=QMCPXML]
    <!-- Specify the optimizer options -->
    <parameter name="MinMethod">quartic</parameter>
    <parameter name="exp0">-6</parameter>
    <parameter name="alloweddifference"> 1.0e-4 </parameter>
    <parameter name="nstabilizers"> 1 </parameter>
    <parameter name="bigchange">15.0</parameter>
\end{lstlisting}

\subsection{General recommendations}
Here are a few recommendations to make wavefunction optimization easier.
\begin{itemize}
\item All electron wavefunctions are typically more difficult to optimize than pseudopotential wavefunctions because of the importance of the wavefunction near the nucleus.
\item Two-body Jastrow contributes the largest portion of correlation energy from bare Slater determinants. Consequently, the recommended order for optimizing wavefunction components is two-body, one-body, three-body Jastrow factors and MSD coefficients.
\item For two-body spline Jastrows, always start from a reasonable one. The lack of physically motivated constraints in the functional form at large distances can cause slow convergence if starting from zero.
\item One-body spline Jastrow from old calculations can be a good starting point.
\item Three-body polynomial Jastrow can start from zero. It is beneficial to first optimize one-body and two-body Jastrow factors without adding three-body terms in the calculation and then add the three-body Jastrow and optimize all the three components together.
\end{itemize}
\subsubsection{Optimization of CI coefficients}
When storing a CI wavefunction in HDF5 format, the CI coefficients and the $\alpha$ and $\beta$ components of each CI are not in the XML input file. When optimizing the CI coefficients, they will be stored in HDF5 format.
The optimization header block will have to specify that the new CI coefficients will be saved to HDF5 format. If the tag is not added coefficients will not be saved.
\begin{lstlisting}[style=QMCPXML]
  <qmc method="linear" move="pbyp" gpu="no" hdf5="yes">
\end{lstlisting}

The rest of the optimization block remains the same.

When running the optimization, the new coefficients will be stored in a *.sXXX.opt.h5 file,  where XXX coressponds to the series number. The H5 file contains only the optimized coefficients. The corresponding *.sXXX.opt.xml  will be updated for each optimization block as follow:
\begin{lstlisting}[style=QMCPXML]
<detlist size="1487" type="DETS" nca="0" ncb="0" nea="2" neb="2" nstates="85" cutoff="1e-2" href="../LiH.orbs.h5" opt_coeffs="LiH.s001.opt.h5"/>
\end{lstlisting}

The opt\_coeffs tag will then reference where the new CI coefficients are stored.\\

When restarting the run with the new optimized coeffs, you need to specify the previous hdf5 containing the basis set, orbitals,
and MSD, as well as the new optimized coefficients. The code will read the previous data but will rewrite the coefficients that
were optimized with the values found in the *.sXXX.opt.h5 file. Be careful to keep the pair of optimized CI coefficients and
Jastrow coefficients together to avoid inconsistencies.


\section{The Importance of Inelastic Collisions}
The treatment of the inelastic collisions will not be as detailed as for the elastic case as we will focus on the qualitative implications of allowing inelastic annihilation and creation processes during thermalization. A full treatment of all inelastic effects is nearly impossible and even today it is not yet fully understood which processes are the most relevant ones driving the system towards the equilibrium state. I will try to highlight the most important observations and refer to the respective research papers for further details. \\
\noindent
First of all we observe that the inelastic processes will modify the collision integral on the right hand side of eqn. (\ref{eqn:transport}). Interestingly it can be shown that this modification leaves the relation for the scattering time $t$ unchanged and therefore also the evolution equation for the defining scales $\Lambda$ and $\Lambda_{\mathrm{s}}$. Implications on the condensate formation can be obtained via an extensive numerical study of solutions of the complete transport equation. The elastic contributions to the collision integral provide a source term for the condensate whereas the inelastic contributions may be interpreted as sink term. The task is to understand the balancing of the source and sink contributions which may allow a condensate to exist during most of the thermalization process.\\
\noindent
One could consider for example the effect of the strong longitudinal expansion of matter involved in relativistic heavy-ion collisions, i.\,e. the flattening of an isotropic initial distribution in the direction of the collision axis, by introducing a suitable drift term on the left hand side of the transport equation  or instabilities in the isotropy of $p_z$ and the transversal momentum $p_{\mathrm{T}}$ (cf. \cite{Blaizot2012}). Another important effect is the influence of gluon radiation allowing transport of momenta over a wide range of momenta on a very short time scale. This can be understood as a certain feature of gauge theories. For more details the interested reader is referred to publication \cite{Blaizot2016}.


% !TEX root = ParticleFilter.tex
\section{Method\label{method}}

\subsection{The Agent-Based Model: StationSim}

\textit{StationSim} is a simple agent-based model that has been designed to very loosely represent the behaviour of a crowd of people moving from an entrance on one side of a rectangular environment to an exit on the other side. This is analogous to a train arriving at a train station and passengers moving across the concourse to leave. A number of agents, $N$, which varies in the later experiments, are created when the model starts. They are able to enter the environment (leave their train) at a uniform rate through one of three entrances. They move across the `concourse' and then leave by one of the two exits. The entrances and exits have a set size, such that only a limited number of agents can pass through them in any given iteration. Once all agents have entered the environment and passed through the concourse then the simulation ends. The model environment is illustrated in Figure \ref{fig:StationSim}, with the trajectories of two interacting agents for illustration.
%The model is also outlined in full as per the ODD protocol~\citep{grimm_odd_2010} in Appendix~\ref{odd}.

\begin{figure}[ht]
\centering
\includegraphics[width=0.5\textwidth]{figures/PF_ABM}
\caption{StationSim environment with 3 entrance and 2 exit doors}.\label{fig:StationSim}
\end{figure}

The model has deliberately been designed to be extremely simple and does not attempt to match the behavioural realism offered by more developed crowd models \citep{chen_multiagentbased_2017, helbing_simulating_2000, klugl_largescale_2007, vanderwal_simulating_2017}. The reason for this simplicity is so that: (1) the model can execute relatively quickly; (2) the probabilistic elements in the model are limited (we know precisely from where probabilistic behaviour arises); (3) the model can be described fully using a relatively simple state vector, as discussed in Section~\ref{state_vector}. Importantly, the model is able to capture the emergence of \textit{crowding}. This results because each agent has a different maximum speed that they can travel at. Therefore, when a fast agent approaches a slower one, they attempt to get past by making a random binary choice to move left or right around them. Depending on the agents in the vicinity, this behaviour can start to lead to the formation of crowds. To illustrate this, Figure~\ref{fig:crowding} shows the paths of the agents (\ref{fig:crowding-trails}) and the total agent density (\ref{fig:crowding-density}) during an example simulation.   The degree and location of crowding depends on the random allocation of maximum speeds to agents and their random of direction taken to avoid slower agents; these cannot be estimated \textit{a priori}. Unlike in previous work where the models did not necessarily meet the common criteria that define agent-based models \citep[e.g.]{lloyd_exploring_2016, ward_dynamic_2016} this model respects three of the most important characteristics:

\begin{figure}
    \centering
    \begin{subfigure}[b]{0.48\textwidth}
        \centering
        \includegraphics[width=.95\linewidth]{figures/crowding-trails}
        \caption{Individual trails showing the paths taken by agents}
        \label{fig:crowding-trails}
    \end{subfigure}
    \label{fig:crowding}
    \begin{subfigure}[b]{0.48\textwidth}
        \centering
        \includegraphics[width=.95\linewidth]{figures/crowding-density}
        \caption{The total crowd density over the simulation run.}
        \label{fig:crowding-density}
    \end{subfigure}
    \caption{An example of crowding in the StationSim model}
    \label{fig:crowding}
\end{figure}

\begin{itemize}
	\item individual heterogeneity -- agents have different maximum travel speeds;
	\item agent interactions -- agents are not allowed to occupy the same space and try to move around slower agents who are blocking their path;
	\item emergence -- crowding is an emergent property of the system that arises as a result of the choice of exit that each agent is heading to and their maximum speed.
\end{itemize}

The model code is relatively short and easy to understand. It is written in Python, and is available in its entirety at in the project repository \citep{stationsimgit}.
% \footnote{\url{XXXX Link to repo}}.

\subsection{Data Assimilation - Introduction and Definitions}

DA methods are built on the following assumptions:

\begin{enumerate}
	\item Although they have low uncertainty, observational data are often spatio-temporally sparse. Therefore there are typically insufficient amounts of data to to describe the system in sufficient detail and a data-driven approach would not work.
	\item Models are not sparse; they can represent the target system in great detail and hence fill in the spatio-temporal gaps in observational data by propagating data from observed to unobserved areas \citep{carrassi_data_2018}. For example, some parts of a building might be more heavily observed than others, so a model that assimilated data from the observed areas might be able to estimate the state of the unobserved areas. However, if the underlying systems are complex, a model will rapidly diverge from the real system in the absence of \textit{up to date} data \citep{ward_dynamic_2016}.
	\item The combination a model and up-to-date observational data allow ``all the available information'' to be used to determine the state of the system as accurately as possible \citep{talagrand_use_1991}.
\end{enumerate}

DA algorithms work by running a model forward in time up to the point that some new observational data become available. This is typically called the \textit{predict} step. At this point, the algorithm has an estimate of the current system state and its uncertainty (the prior). The next step, \textit{update},  involves using the new observations, and their uncertainties, to update the current state estimate to create a posterior estimate of the state. As the posterior has combined the best guess of the state from the model \textit{and} the best guess of the state from the observations, it should be a closer estimate of the true system state than that which could be estimated from the observations or the model in isolation.

\subsection{The Particle Filter\label{particle_filter}}

There are many different ways to perform data assimilation, as discussed in Section \ref{da_pf}. Here, a potentially appropriate solution to the data assimilation problem for agent-based models is the particle filter -- also known as a Bayesian bootstrap filter or a sequential Monte Carlo method -- which represents the posterior state using a collection of model samples, called particles \citep{gordon_novel_1993,carpenter_improved_1999,wang_data_2015, carrassi_data_2018}. Figure~\ref{fig:PF_flowchart} illustrates the process of running a particle filter. Note that the `pseudo-truth model' is a single instance of the agent-based model that is used as a proxy for the real system as per the identical twin experimental framework that we have adopted.

The data assimilation `window' determines how often new observations are assimilated into the particle filter. The size of the window is an important factor -- larger windows result in the particles deviating further from the real system state -- but here we fix the window at 100 iterations. The simulation terminates when all agents have left the system.

\begin{figure}[ht]
\centering
\includegraphics[width=0.8\textwidth]{figures/PF_flowchart2}
\caption{Flowchart of data assimilation process using a particle filter.\label{fig:PF_flowchart}}
\end{figure}

\subsubsection{The State Vector and Transition Function\label{state_vector}}

Here, the \textit{state vector}, at a time $t$, contains all the information that a transition function needs to iterate the model forward by one step, including all of the agent ($i = \{ 0, 1, \dots, N \} $) parameters ($\overrightarrow{p_i}$) and variables ($\overrightarrow{v_i}$) as well as global model parameters $\overrightarrow{P}$:
\begin{equation}
  S_t  = \left[ \begin{array}{cccccccc}
\overrightarrow{p_0} & \overrightarrow{v_0} & \overrightarrow{p_1} &  \overrightarrow{v_1} &  \dots &  \overrightarrow{p_N} &  \overrightarrow{v_N} & \overrightarrow{P}
\end{array} \right]
\end{equation}

A similar structure, the \textit{observation vector}, contains all of the observations made from the `real world' (in this case the pseudo-truth model) at a time $t$. Here, the particle filter is only used to estimate the state of the models variables ($\overrightarrow{v_i}$), not any of the parameters ($\overrightarrow{p_i}$ and $\overrightarrow{P}$) (although it is worth noting that parameter estimation is technically feasible and will be experimented with in future work). Also, the current speed of an agent can be calculated from its current location and the locations of the agents surrounding it, so in effect the observation vector only needs to include the positions of the agents with the addition of some Gaussian noise, $\epsilon$:
\begin{equation}
  O_t  = \left[ \begin{array}{ccccccc}
x_0 & y_0 & x_1 & y_1 & \dots & x_n & y_n
\end{array} \right]
\end{equation}

 Therefore in the experiments conducted here, all model parameters are fixed. Hence a further vector is required to map the observations to the state vector that the particles can actually manipulate. We define the partial state vector $S'$ to match the shape of $O$, i.e.:
\begin{equation}
  S'_t  = \left[ \begin{array}{ccccccc}
x_0 & y_0 & x_1 & y_1 & \dots & x_n & y_n
\end{array} \right]
\end{equation}

This has the effect of `pairing' agents in the particles to those in the pseudo-truth data, in a similar approach to that taken by \citep{wang_data_2015}. It is worth noting that, because the particle filter will not be tasked with parameter estimation, then the data assimilation is somewhat simpler than it would be in a real application. For example, one of the agent parameters is used to store the location of the exit out of the environment that the agent is moving towards. As this parameter is set \textit{a priori} for each agent, then the particle filter does not need to estimate where the agents are ultimately going, only where they currently are.

\subsubsection{Observations from the pseudo-truth data}

In a real application, the particle filter would be assimilating data in real time from sensors of the real world. This is not the case here so instead, we take `observations' from the pseudo-truth data (which are, as it happens, generated by the StationSim model). Each particle evolves in time according to the StationSim dynamics and receives new observations at regular intervals. Measurement error (i.e. noise, $\epsilon$) is added to the observations (in the real world, sensors observations will be noisy). Here, observations take the form of the $(x,y)$ locations of all agents. This is analogous to tracking all individuals in a crowd and providing snapshots of their locations at discrete points in time to the particle filter. This `synthetic observation' is probably more detailed than reality, so future work will vary the amount of detail provided to the algorithm. It will firstly reduce the number of agents who are observed (e.g. tracking only \textit{some} people) and then provide only aggregate population counts (which is analogous to using a camera or other sensor to count the number of people at a certain point).

\subsubsection{Particle Weights}

Each particle in the particle filter has a weight associated with it that quantifies how similar a particle is to an observation. The weights are calculated at the end of each data assimilation window (i.e. when observations become available). At the start of the following window the particles then evolve independently from each other \citep{fearnhead_particle_2018}.  The weights are, in effect, the average distance between agents in that particle and the corresponding agents in StationSim (recall that there is a one-to-one mapping between agents in the particles and agents in the truth model). Formally, let $x^n(i,t)$ be the location of the $i$-th agent at time $t$ in the $n$-th particle for $n \in \{1,\dots,N\}$ and let $x(i,t)$ be the location of the $i$-th agent in StationSim for $i \in \{1,\dots,I\}$. The error of the $n$-th particle $\epsilon^n(t)$ at time $t$ is then given by
\begin{equation}
\epsilon^n(t) = \frac{1}{I} \sum_{i=1}^I |x(i,t) - x^n(i,t)|,
\end{equation}
and the particle filter error $\nu(t)$ at time $t$ is given by
\begin{equation}
\label{eqn:particle_error}
\begin{split}
\nu(t) =& \frac{1}{N} \sum_{n=1}^{N} \epsilon^n(t), \\
=& \frac{1}{NI} \sum_{n=1}^{N}\sum_{i=1}^I |x(i,t) - x^n(i,t)|.
\end{split}
\end{equation}

It is worth noting that, because the agent locations are the only data stored in the partial state vector, the particle error is equivalent to the Euclidean distance ($l_2$-norm) between the particle partial state vector $S'_{t}$ and the observation vector $O_t$,
\begin{equation}
 \nu(t) = || S'_{t} - O_t  ||_2.
\end{equation}

Particles with relatively large error are likely to be removed during the sampling procedure (discussed in the following section), whereas those with low error are likely to be duplicated. In addition for their use in resampling, the population of particle weights can be used to gain insight into how well the particle filter is able to represent the `true' system state overall.

\subsubsection{Sampling Procedure\label{particle_sampling}}

Here, a bootstrap filter is implemented which uses systematic resampling \citep{doucet_introduction_2001, douc_comparison_2005, wang_data_2015, long_spatial_2017, carrassi_data_2018}. This begins by taking a random sample $U$ from the uniform distribution on the interval $[0,1/N]$ and then selecting $N$ points $U^i$ for $i \in \{1,\dots,N\}$ on the interval $[0,1]$ such that
\begin{equation}
U^i = (i-1)/N + U.
\end{equation}
Let the particles currently have locations $x_i$. Using the inversion method, we calculate the cumulative sum of the normalised particle weights $w_i$ and define the inverse function $D$ of this cumulative sum, that is:
\begin{equation}
D(u) = i \text{ for } u \in \left(\sum_{j=1}^{i-1}w_j,\sum_{j=1}^{i}w_j\right].
\end{equation}
Finally, the new locations of the resampled particles are given by $x_{D(U^i)}$.

As discussed in Section~\ref{background}, a well-studied issue that particle filters face is that of particle deprivation \citep{snyder_obstacles_2008}, which refers to the problem of particles converging to a single point such that all particles, but one, vanish \citep{kong_sequential_1994}.
% This vastly reduces the size of the state space covered by the population of particles and will make it difficult or impossible to find particles with low error in later windows. Common approaches to resolve the deprivation problem include increasing the number of particles or trying to reduce the dimensionality of the state space. Bespoke approaches also exist; for example \citep{wang_data_2015} develop a technique called `component set resampling' that samples \textit{parts} of particles (i.e. those parts that are working well) rather than whole particles in their entirety.
Here, the problem is addressed in two ways. Firstly by simply using large numbers of particles relative to the size of the state space and, secondly, by diversifying the particles \citep{vadakkepat_improved_2006} -- also known as roughening, jittering, and diffusing \citep{li_fight_2014, shephard_learning_2009, pantrigo_combining_2005}. In each iteration of the model we add Gaussian white noise to the particles' state vector to increase their variance, which increases particle diversity. This encourages a greater variety of particles to be resampled and therefore makes the algorithm more likely to represent the state of the underlying model. This method is a special case of the resample-move method presented in \citep{gilks_following_2001}. The amount of noise to add is a key hyper-parameter -- too little and it has no effect, too much and the state of the particles moves far away from the true state -- as discussed in the following section.


%!TEX root = ../main.tex

\section{Dielectric Laser Acceleration}

In the public sphere, particle accelerators most commonly conjure images of giant facilities for performing particle physics experiments, such as the Large Hadron Collider.
However, in fact, most particle accelerators are used in other applications, such as radiotherapy, X-ray generation, and ion implantation for semiconductor device fabrication \cite{england2014dielectric}.
Conventional radio-frequency (RF) accelerators use a metal or superconducting cavity, driven with microwave radiation, to provide sustained acceleration to charged particles traversing the structure.
However, the amount of achievable acceleration per unit length is fundamentally limited by the material breakdown and damage limit of the cavity.
Therefore, for an accelerator already driven at its damage threshold, the only option to achieve high total energy gains from an accelerator is to make the device longer.
The largest current particle accelerators reach several of kilometers in length, requiring substantial resources to operate and maintain.

Dielectric laser acceleration (DLA) is an emerging method that seeks to revolutionize particle accelerator technology by exploiting the decades of progress in nanofabrication, materials science, and laser technology.
In DLA, instead of using metal structures driven by microwaves, dielectric structures are illuminated with infrared laser light, which creates an electromagnetic field pattern in their vicinity that may accelerate electrically charged particles, such as electrons.
When compared to metal surfaces at microwave frequencies, dielectric materials have very high damage thresholds at short pulse durations and infrared wavelengths \cite{mcneur2016elements, soong2012laser}.
This fact allows DLAs to achieve energy gains per length that are between 10 to 100 times higher than those found in conventional radio frequency (RF) accelerators.

Experimental demonstrations of these acceleration gradients have been made practical in recent years by the availability of robust nanofabrication techniques combined with modern solid state laser systems \cite{dawson2008analysis}.
As a result, the development of DLA may lead to compact particle accelerators that enable new applications.
By providing the potential for generating relativistic electron beams in relatively short length scales, DLA technology is projected to have numerous applications where tabletop accelerators may be useful, including medical imaging, radiation therapy, and X-ray generation \cite{plettner2008microstructure,england2014dielectric}.

Several recently demonstrated candidate DLA structures consist of a planar dielectric structure that is periodic along the particle axis with either an semi-open geometry or a narrow (micron to sub-micron) vacuum gap in which the particles travel \cite{plettner2006proposed, peralta2013demonstration, mcneur2016elements, leedle2015dielectric, chang_silicon_2014, breuer2014dielectric, breuer2014dielectric2, kozak2016dielectric}.
These structures are then side-illuminated by laser pulses.
\fig{DLA_def} shows a schematic of the setup, with a laser pulse incident from the bottom.

\figdef{DLA_definition.pdf}{DLA_def}{
    Diagram outlining the system setup for side-coupled DLA with an arbitrary die,lectric structure $\eps(x,y)$ (green).
    A charged particle moves through the vacuum gap with speed $\beta c_0$.
    The periodicity is set at $\beta \lambda$ where $\lambda$ is the central wavelength of the laser pulse.
}

The laser field may also be treated with a pulse front tilt \cite{hebling1996derivation, akturk2004pulse} to enable group velocity matching over a distance greater than the laser's pulse length.

\section{Adjoint Method for Accelerator}

To achieve high energy gain in a compact size, it is of principle interest to design structures that may produce the largest acceleration gradients possible without exceeding their respective damage thresholds.
Here we will discuss the use of the adjoint method and inverse design to design such a structure, as explained originally in Ref. \citenum{hughes_method_2017}.
To begin, we must first define the optimization figure of merit and design parameters.

\subsection{Mathematical Definition}

We first seek to maximize the \textit{acceleration gradient} of the device, which is defined as the amount of energy gain per unit length achieved by a particle that is phased correctly with the driving field.
For acceleration to occur, the dielectric structure must be designed such that the particle feels an electric field that is largely parallel to its trajectory over many optical periods.
Here we consider framing an optimization problem to maximize the acceleration gradient, which take as our figure of merit (labelled $J$ in previous discussions).
In a general DLA device, the acceleration gradient `$G$' over a time period `$T$' is defined mathematically as
%
\eqdef{Gintro}{
    G = \frac{1}{T}\int_0^{T}{ E_{||}(\vr(t),t)\ dt},
}
%
where $\vr(t)$ is the position of the electron over time and $E_{||}$ signifies the (real) electric field component parallel to the electron trajectory.

Since we assume the structure is invariant in the $\hat{z}$ direction, we may work in two dimensions, examining only the $H_z$, $E_x$ and $E_y$ field components.
While this approximation neglects fringing fields that will be present in any fabricated device, it is a good approximation for the fields experienced by particles traversing the center of the acceleration channel.
For an approximately monochromatic input laser source with angular frequency $\omega$, the electric fields are, in general, of the form
%
\eqdef{}{
    \vE(\vr,t) = \real{\tE(\vr)e^{i\omega t}},
}
where $\tE$ is complex-valued.
In the following chapter we discuss the generalization of this analysis to broadband light.

Let us assume the particle we wish to accelerate is moving on the line $y=0$ with velocity $\vec{v} = \beta c_0 \hat{x}$, where $c_0$ is the speed of light in vacuum and $\beta \leq 1$.
The $x$ position of the particle as a function of time is given by $x(t) = x_0 + \beta c_0 t$, where $x_0$ represents an arbitrary choice of initial starting position.
For normal incidence of the laser (laser propagating in the $+\hat{y}$ direction), phase velocity matching between the particle and the electromagnetic fields is established by introducing a spatial periodicity in our structure of period $\beta \lambda$ along $\hat{x}$ , where $\lambda$ is the laser wavelength.
In the limit of an infinitely long structure (or equivalently, $T \to \infty $) we may rewrite our expression for the gradient in Eq. (\ref{eq:Gintro}) as an integral over one spatial period, given by
%
\eqdef{}{
    G = \frac{1}{\beta\lambda}\real{ e^{-i\phi_0}\int_0^{\beta\lambda}{dx \ }E_x(x,0)e^{i\frac{2\pi}{\beta\lambda}x}}.
}
%
Here the quantity $\phi_0 = \frac{2\pi x_0}{\beta\lambda}$ is representative of the phase of the particle as it enters the spatial period.
In further calculations, we set $\phi_0 = 0$, only examining the acceleration gradients experienced by particles entering the accelerator with this specific phase.
Since we have arbitrary control over our input laser phase, this does not impose any constraint on the acceleration gradient attainable.

To simplify the following derivations, we define the following inner product operation involving the integral over two vector quantities $\va$ and $\vb$ over a single period volume $V'$
%
\eqdef{}{
    \bracket{\va}{\vb} = \int_{V'} dv
    \left(\va \cdot \vb \right) = \int_0^{\beta\lambda}dx\int_{-\infty}^\infty dy \left(\va\cdot\vb\right).
}
%
With this definition, we then may express the gradient simply as
%
\eqdef{G}{
    G = \real{\bracket{\vE}{\veta}},
}
%
where we define the vector field `$\veta$' to signify the position and phase of the moving electron as
\eqdef{}{
    \veta(x,y) = \frac{1}{\beta\lambda}e^{i\frac{2\pi}{\beta\lambda}x}\delta(y)\hat{x}.
}
The physical interpretation of $\veta$ is digrammed in \fig{eta_def}.

\figdef{DLA_eta_definition.png}{eta_def}{
    Definition of the vector field, $\veta$, which defines the position of the electron in the frequency domain.
    The green regions represent domains where we will optimize the material properties using the adjoint method.
    The central gap is constrained to vacuum to allow passage of the electron beam.
    The red arrow signifies the driving laser.
}

Our goal in designing the accelerator for maximum acceleration gradient is to create a permittivity distribution that maximizes $G$ subject to a few constraints.
We assume geometry of the DLA structure is represented by a spatially varying dielectric constant $\eps(x,y)$.
As mentioned, we assume invariance in one coordinate ($\hat{z}$) in keeping with the planar symmetry of most current designs.
However the methodology we present can be extended to include a third dimension.
We consider performing this optimization in a small design region surrounding a small gap defined for the electron to travel through the structure.
Secondly, we assume that the structure has a finite extent along the direction of the incoming laser beam.
We also consider realizing this device through the patterning of a material with permittivity $\eps_\textrm{max}$.
Therefore, the final device should have permittivity of either 1 or $\eps_\textrm{max}$ at all points.

To perform this optimization task, we discretize our entire spatial domain into a rectangular grid, which will be necessary for numerical simulation.
We define our design parameters, $\bfphi$, as the relative permittivity of each grid cell within the design region.
Our problem then becomes finding the permittivity of each cell that will maximize the acceleration gradient, subject to each grid cell having a permittivity value of either 1 or $\eps_\textrm{max}$.

To accomplish this, the most naive approach would involve performing a direct search over the full design space.
For example, one could label each cell within the design region with an identifier `0' or `1' corresponding to `vaccuum' and `material', respectively.
Then, one would generate all possible structures and check their respective acceleration gradients.
However, this method would be far too computationally expensive to perform in practice.
For example, even considering a very small design region consisting of 10 $\times$ 10 = 100 grid cells would result in $2^{100} \approx 10^{30}$ device simulations, which is far too many to realistically perform.
While one may consider more efficient ways of searching through this device space without checking each structure, for example using global optimization approaches such as genetic algorithms \cite{whitley_genetic_1994} or particle swarm optimization \cite{noauthor_particle_nodate}, this problem is still quite computationally expensive and the size of design space becomes exponentially larger as the number of design parameters are increased.

As mentioned, a more effective approach involves performing $\textit{gradient-based optimization}$, in which we search the design space according to the local gradient of the figure of merit with respect to each of the parameters.
For example, we may start with an initially random device, compute how the performance will change with respect to a change in the permittivity of each cell in the design region, and make a small update.
This process may be repeated until convergence on a locally optimal solution.
If the design space contains several local optima, then this whole process may be repeated several times with different initial conditions.

For gradient-based optimization to be useful, one would like an efficient means to compute the gradient of the figure of merit with respect to the design parameters.
For photonic devices, one may achieve this using the \textit{adjoint method}, which allows one to analytically compute the gradient directly from Maxwell's Equations and evaluate the result with only one additional electromagnetic simulation.
This remarkable efficiency is largely responsible for the success of inverse design in photonics.

\subsection{Adjoint Formalism}

The adjoint method is typically introduced for linear optical systems, although, as we will show in a later chapter, it may be extended to nonlinear systems without much additional complication.
In the frequency domain, Maxwell's equations may be written as
%
\eqdef{FDFD_analytical}{
    \dcurl \vE (\vr)\ -\ k_0^2\ \eps_r(\vr)\ \vE(\vr) \equiv A\vE(\vr) = -i\mu_0\omega\vJ(\vr),
}
%
Here, $\vE(\vr)$ and $\vJ(\vr)$ are the electric field and electric current distributions, respectively. $k_0 = \omega/c_0$, $\eps_r$ is the relative permittivity and a non-magnetic material is assumed ($\mu = \mu_0$).
\eq{FDFD_analytical} can be solved with the \textit{finite-difference frequency-domain} (FDFD) method \cite{shin2012choice, taflove2000computational}, where both the structure and the fields are discretized on a finite-difference grid.

More abstractly, we may write \eq{FDFD_analytical} as
%
\eqdef{FDFD_simple}{
    A \bfx = \bfb,
}
%
where $A$ is a sparse, complex symmetric matrix that encodes Maxwell's equations in the presence of the device.
$\bfx$ is a vector containing the electromagnetic fields at each position in the domain, which are the solution to \eq{FDFD_simple} given the vector $\bfb$ describing the electric current source distribution in the domain.
When the number of grid points is small, \eq{FDFD_simple} is typically solved using lower-upper (LU) decomposition techniques.
However, in several cases, it must be solved using iterative methods instead \cite{shin2012choice}.

Our device is described by a set of design variables $\bm{\phi}$, which influence the system matrix, $A = A(\bfphi)$.
Differentiating \eq{FDFD_analytical} with respect to $\bfphi$, and assuming that the current source, $\bfb$, does not depend on $\bfphi$, we may recover the change in the solution with respect to the parameters as
%
\eqdef{dEdgamma}{
    \ddfrac{\bfx}{\bfphi} = -\invA \pfrac{A}{\bfphi} \invA \bfb = -\invA \pfrac{A}{\bfphi}\bfx
}

Now, we consider differentiating an objective function $J = J(\bfx)$ that depends explicitly on the field solution.
By the chain rule, this gives
%
\eqdef{dGdgamma_2}{
    \ddfrac{J}{\bfphi} =
    - \real{ \pfrac{J}{\bfx} \ddfrac{\bfx}{\bfphi} } = 2 \real{ -\pfrac{J}{\bfx} A^{-1} \pfrac{A}{\bfphi} \bfx }.
}
We note that the factor of $2\real{\cdot}$ comes from the fact that $\bfx$ is complex-valued, and therefore in taking the derivative of (real-valued) $J$, one must consider the dependence on both $\bfx$ and $\bfx^*$.

To evaluate \eq{dGdgamma_2}, we define a second simulation with source term $-\pfrac{J}{\bfx}^T$,
%
\eqdef{adjoint_source}{
    A^T\bfx_\aj = A\bfx_\aj = -\pfrac{J}{\bfx}^T,
}
where we have used the fact that $A$ is symmetric.
Then, the field solution, $\bfx_\aj = -A^{-1}\pfrac{J}{\bfx}^T$, can be easily identified in \eq{dGdgamma_2}, which gives the expression
%
\eqdef{final_form_DGdgamma}{
    \ddfrac{G}{\bfphi} = 2 \real{ \bfx_\aj^T \pfrac{A}{\bfphi} \bfx }.
}
%
The only quantity in this expression that depends on the parameter $\bfphi$ is $\pfrac{A}{\bfphi}$.
As we will soon discuss, this quantity will generally be trivial to compute.
On the other hand, the full field calculations of $\bfx$ and $\bfx_\aj$ are computationally expensive, but may be computed once and used for an arbitrarily large set of parameters $\bfphi_i$.
This gives the adjoint method a significant scaling advantage with respect to traditional direct sensitivity methods, such as the numerical, 'finite-difference' differentiation approach, in which each parameter is independently perturbed a small amount and the resulting change in objective function is measured.

Previously, we expressed the acceleration gradient as the inner product of the electric fields $\vE(\vr)$ and our vector field $\veta(\vr)$ as
%
\eqdef{G2}{
    G = \real{\bracket{\vE}{\veta}},
}
%
which we may express in matrix notation as
%
\eqdef{G2}{
    G = \real{\bfeta^T \bfx},
}
%
where $\bfeta$ is a vector representing $\veta$ on the finite difference grid and $\bfx$ is a vector containing the electric fields, as before.

In this notation, the adjoint problem is therefore
%
\eqdef{adjoint_DLA}{
    A^T \bfx_\aj = -\pfrac{G}{\bfx}^T = -\bfeta.
}
%

\subsection{Interpretation of the Adjoint Field as Radiation}

Intuitively, the adjoint source, $-\bfeta$, represents a line of electric current located at the central gap where the accelerator is traversing the structure.
To make this connection to radiation more explicit, let us now consider the fields radiated by a point particle of charge $q$ flowing through our domain at $y = 0$ with velocity $\vec{v} = \beta c_0 \hat{x}$.
In the time domain, we can represent the current density of this particle as
%
\eqdef{point-current-setup}{
    \vJ_\textrm{rad}(x,y;t) = q \beta c_0 \delta(x-x_0-c_0\beta t)\delta(y)\hat{x}.
}
%
To express this in the frequency domain, we take the Fourier transform of $\vJ_\textrm{rad}$ with respect to time, giving
%
\begin{align}
    \vec{J}_{rad}(x,y;\omega) &= q \beta c_0 \delta(y)\hat{x}\int _{-\infty}^{\infty}{dt\ }\exp(i\omega t)\delta(x-x_0-c_0\beta t)\\
    &= q\ \exp\Big(i\frac{\omega\ (x-x_0)}{c_0 \beta}\Big)\delta(y)\hat{x}\\
    &= q\ \exp\Big(i\frac{2\pi}{ \beta\lambda}x\Big)\ \exp(-i\phi_0)\delta(y)\hat{x}.
\label{eq:point-current}
\end{align}
%
Comparing with the source of our adjoint problem, $\vJ_\aj = \frac{-i}{\omega\mu_0}\veta$, we see that
%
\eqdef{J-eta}{
    \vec{J}_{aj} = \frac{-i\ \exp(i\phi_0)}{2\pi q\beta c_0\mu_0}\vJ_\textrm{rad}.
}

This finding shows that the adjoint field solution ($\vE_\aj$) corresponds (up to a complex constant) to the field radiating from a test particle flowing through the accelerator structure.
To put this another way, in order to calculate the acceleration gradient sensitivity with the adjoint method, we must simulate the same structure operating both as an accelerator ($A\vE=-i\omega\mu_0\vJ_\textrm{acc}$) and as a radiator ($A\vE_\aj=-i\omega\mu_0\vJ_\aj$).

It is understood that one way to create acceleration is to run a radiative process in reverse.
Indeed, this is the working principle behind accelerator schemes such as inverse free electron lasers \cite{musumeci2005high, courant1985high}, inverse Cherenkov accelerators \cite{kimura1995laser, fontana1983high}, and inverse Smith-Purcell accelerators \cite{bae1992experimental, mizuno1987experimental}.
Here, we see that this relationship can be expressed in an elegant fashion using the adjoint method.

\subsection{Application to Design Parameters}

To test our adjoint formalism, we examine a simple accelerator geometry composed of two opposing dielectric squares each of relative permittivity $\epsilon$.
We take a single $\phi$ parameter to be the relative permittivity of the entire square region.
Because we only change the region inside the dielectric square, we may identify the $\pfrac{A}{\bfphi}$ operator from \eq{FDFD_analytical} as
%
\eqdef{dAdepsilon}{
    \ddfrac{A}{\epsilon}(\vr) = \Bigg\{
    \begin{array}{ll}
        -2k_0^2 &\textrm{if } \vr \textrm{ in square} \\
        0 &\textrm{otherwise}
    \end{array}.
}

Thus, given the form of the sensitivity of the acceleration gradient from \eq{final_form_DGdgamma}, combined with \eq{dAdepsilon}, the change in acceleration gradient with respect to changing the entire square permittivity is simply given by the integral of the two field solutions over the square region, labeled `$sq$'
%
\eqdef{}{
    \ddfrac{G}{\epsilon_{sq}} = -2k_0^2 \ \real{\int_{sq}{d^2\vr}.
    \vE(\vr) \cdot \vE_\aj(\vr)}.
}

In Fig. \fig{DLA_test} we compare this result with the direct sensitivity calculation where the system is manually changed and simulated again, using a finite difference derivative.
The two methods agree with excellent precision, which confirms that the adjoint formalism is giving the correct results.

\figdef{DLA_test}{DLA_test}{
    Demonstration of adjoint method in calculating accelerator sensitivities.
    (a) The acceleration gradient ($G$) of a square accelerator structure (inset) as a function of the square's relative permittivity.
    We express the acceleration gradient in its dimensionless form, normalized by the electric field amplitude of the incident plane wave $(E_0)$.
    The particle traverses along the dotted line with a velocity of $c_0$ ($\beta = 1$) and a plane wave is incident from the bottom of the structure.
    (b) The sensitivity $\ddfrac{G}{\epsilon}$ of the gradient with respect to changing the square relative permittivity for direct central difference (solid line) $\ddfrac{G}{\epsilon} = \frac{G(\epsilon+\Delta\epsilon)-G(\epsilon-\Delta\epsilon)}{2\Delta\epsilon}$ and using the adjoint method (circles).
    The two calculations agree with excellent precision.
    The dotted line at $\ddfrac{G}{\epsilon}=0$, corresponds to local minima and maxima of $G(\epsilon)$ above.
}

Extending this example to the general case of perturbing the permittivity at an arbitrary position $\vr$, we see that
%
\begin{align}
    \ddfrac{G}{\epsilon}(\vr) &= -k_0^2 \ \real{ \int{d^2\vr'}.
    \vE(\vr') \cdot \vE_\aj(\vr')~
    \delta(\vr-\vr') } \\
    &= -2k_0^2 \ \real{ \vE(\vr) \cdot \vE_\aj(\vr) }.
\end{align}
%
\section{Inverse Design of Accelerator}

\subsection{Maximization of Acceleration Gradient}

With the mathematical form of the adjoint problem discussed, now we focus on the optimization and inverse design of the DLA device.
In our FDFD simulation, we use a grid spacing that corresponds to 200 grid points per free space wavelength in each dimension.
Perfectly matched layers are implemented as absorbing regions on the edges parallel to the electron trajectory, with periodic boundary conditions employed on boundaries perpendicular to the electron trajectory.
A total-field scattered-field \cite{taflove2000computational} formalism is used to create a perfect plane wave input for the acceleration mode.

Since the adjoint method gives us a highly efficient method to calculate $\ddfrac{G}{\epsilon_{i}}$, we use this information in an iterative optimization procedure.
During each iteration, we first calculate $\ddfrac{G}{\epsilon_{i}}$ for all pixels `$i$' within some specified design region.
Then, we update the $\epsilon_i$ of each grid cell as follows
%
\eqdef{}{
    \epsilon_{i} := \epsilon_{i} + \alpha\ddfrac{G}{\epsilon_{i}}.
}
%
Here, $\alpha$ is a step parameter that we can tune.
We need $\alpha$ to be small enough to find local maxima, but large enough to have the optimization run in reasonable amount of time.
This process is repeated until convergence on $G$, as diagrammed in \fig{opt_loop}.

\figdef{DLA_optimization_routine.png}{opt_loop}{Optimization routine for DLA.}

During the course of optimization, the permittivity distribution is considered as a continuous variable, which is not realistic in physical devices.
To address this issue, we employ a permittivity capping scheme during optimization.
We define a maximum permittivity `$\epsilon_m$' corresponding to a material of interest.
During the iterative process, if the relative permittivity of any cell becomes either less than $1$ (vacuum) or greater than $\epsilon_m$, that cell is pushed back into the acceptable range.
It was found that with this capping scheme, the structures converged to binary (each pixel being either vacuum or material with a permittivity of $\epsilon_m$) after a number of iterations without specifying this choice of binary materials as a requirement of the optimization.
Therefore, only minimal post-processing of the structures was required.
Since high index contrast is favorable for maximizing the acceleration gradient, the optimization often favors increasing index contrast within the structure, which leads to each pixel being pushed towards either $1$ or $\epsilon_m$.
This is a possible explanation for why the structures converge to binary distributions under our optimization procedure.

The results of this optimization scheme are shown in Figs. \ref{fig:optimization-results_beta0.5}(b)-\ref{fig:optimization-results_beta0.5}(d) for three different $\epsilon_m$ values corresponding to commonly explored DLA materials.
The design region was taken to be a rectangle fully surrounding but not including the particle gap.
The design region was made smaller for higher index materials, since making it too large led to divergence during the iteration.
We found that a totally vacuum initial structure worked well for these optimizations.
However, initially random values between 1 and $\epsilon_m$ for each pixel within the design region also gave reasonable results.
For materials of Si, Si$_3$N$_4$, and SiO$_2$, the achieved gradients (normalized by the incident field, $E_0$) were 0.90, 0.56, and 0.31 ($E_0$), respectively.
Assuming incident field values consistent with the laser damage thresholds given in Ref. \cite{soong2012laser}, these correspond to acceleration gradients of 0.25, 0.83, and 1.33 $\textrm{GV}\,\textrm{m}^{-1}$, respectively.
Without the dielectric mirrors, these structures give normalized acceleration gradients of 0.28, 0.15, and 0.07 ($E_0$), respectively.
For comparison, the Si dual pillar structures presented in Ref. \cite{leedle2015dielectric} give gradients of around 0.3 $(E_0)$.
Therefore, the adjoint optimized structures show about a three-fold improvement in acceleration gradient over established structures.

\figdef{DLA_Adjoint.png}{optimization-results_beta0.5}{
    Demonstration of the structure optimization for $\beta=0.5$, laser wavelength $\lambda=2\,\mu$m, and a gap size of $400$\,nm.
    A plane wave is incident from the bottom in all cases.
    (a) Acceleration gradient as a function of iteration number for different maximum relative permittivity values, corresponding to those of Si, Si$_3$N$_4$, and SiO$_2$ at the laser wavelength.
    The acceleration gradient is normalized by the electric field amplitude of the incident plane wave $(E_0)$.
    The optimizations converge after about five-hundred iterations.
    (b-d) Final structure permittivity distributions (white = vacuum, black = $\epsilon_m$) corresponding to the three curves in (a).
    Eight periods are shown, corresponding to four laser wavelengths.
    For each (b-d), design region widths on each side of the particle gap were given by $1$, $2$, and $4\,\mu$m for Si, Si$_3$N$_4$, and SiO$_2$, respectively.
}

This optimization scheme seems to favor geometries consisting of a staggered array of field-reversing pillars surrounding the vacuum gap, which is already a popular geometry for DLA.
However, these optimal designs also include reflective mirrors on either side of the pillar array, which suggests that for strictly higher acceleration gradients, it is useful to use dielectric mirrors to resonantly enhance the fields in the gap.

It was observed that for random initial starting permittivity distributions, the same structures as shown in \fig{optimization-results_beta0.5} are generated every time.
Furthermore, as shown in \fig{DLA_sim}, the geometries found using this method are remarkably similar to those recently proposed through human design using physical intuition \cite{ACHIP_meeting3}.
These findings together suggest that the proposed structures may be close to the globally optimal structure for maximizing $G$.

\figdef{DLA_Similarity.png}{DLA_sim}{Similarity between DLA structures designed using adjoint method (left) and those independently proposed using human intuition.}

There are several variants of gradient descent which speed up the optimization in most cases. ‘Momentum’ is one such technique in which the design parameters are updated not only from the gradient information at the current iteration, but also a weighted sum of previous gradients.
Intuitively, this means if an optimization has been trending in a given direction, it will be more likely to continue in this direction, which can improve speed and stability.
Specifically, the momentum term term corresponds to the sensitivity calculated at the last iteration multiplied by a constant, $\alpha' < 1$. For iteration number `$j$' and pixel `$i$' the update is defined as

\eqdef{}{
    \epsilon_i^{ (j+1) } := \epsilon_i^{ (j) } + \alpha\Bigg[ \ddfrac{G}{\epsilon_i}^{ (j) } + \alpha'\ddfrac{G}{\epsilon_i}^{ (j-1) }\Bigg].
}

\subsection{Maximization of Acceleration Factor}

DLAs are often driven with the highest input field possible before damage occurs.
Therefore, another highly relevant quantity to maximize is the \textit{acceleration factor}, given by the acceleration gradient divided by the maximum electric field amplitude in the system.
This quantity will ultimately limit the amount of acceleration gradient we can achieve when running at damage threshold.
Explicitly, the acceleration factor is given by

\eqdef{}{
    f_{A} = \frac{G}{\max\{|\vec{E}|\}}.
}
%
Here, $|\vec{E}|$ is a vector of electric field amplitudes in our system, which are normalized, as in the case of $G$, by the electric field amplitude of the incident plane wave ($E_0$).
The $\max\{\}$ function is designed to pick out the highest value of this vector in either our design or material region, depending on the context.
The \textit{design region} is defined as the total region outside of the particle gap where the permittivity is updated.
The \textit{material region} is defined as any region where the permittivity is equal to $\epsilon_m$.
We would like to use the same basic formalism to maximize $f_{A}$.
However, since the $\max\{\}$ function is not differentiable, this is not possible directly.
Instead we may use a `smooth-max' function to approximate $\max\{\}$ as a weighted sum of vector components
%
\eqdef{}{
    \max\{|\vec{E}|\} \approx \frac{\sum_i |\vec{E}_i| \ \textrm{exp}\big({a|\vec{E}_i|}\big)} {\sum_i \textrm{exp}\big({a|\vec{E}_i|}\big)}.
}
%
Here, the parameter $a \geq 0$ controls the relative strength of the exponential sum terms, for $a = 0$, this function simply gives the average value of the field amplitudes.
By sweeping $a$ and examining the acceleration factors of the resulting optimized structures, we determined that $a = 3$ gave the best improvement in $f_A$.
If $a$ is too large, the calculation may induce floating point overflow or rounding error issues.

Using this smooth-max function, one may calculate $\ddfrac{f_{A}}{\epsilon_i}$ analytically and perform structure optimizations in the same way that was discussed previously.
Two structures with identical parameters but optimized, respectively, for maximum $G$ and $f_A$ are shown in Fig. \ref{fig:optimization-results_fAC}.
On the left, we see that the $G$ maximized structure shows the characteristic dielectric mirrors, giving resonant field enhancement.
On the right is the structure optimized for $f_A$, which has eliminated most of its dielectric mirrors and also introduces interesting pillar shapes.
In Table \ref{tb:table} the main DLA performance quantities of interest are compared between these two structures.
Whereas the acceleration gradient is greatly reduced when maximizing for $f_A$, the $f_A$ value itself is improved by about 25\% or 23\% depending on whether one measures the maximum field in the design region or the material region, respectively.
As a comparison with the geometry from Ref. \cite{leedle2015dielectric}, a SiO$_2$ dual pillar structure with a 400 nm gap was optimized for acceleration gradient, giving an acceleration gradient of 0.025 ($E_0$) with a pillar radius of 670 nm.
This structure was found to have $\max\{|\vec{E}|\}$ in the design region of 1.612 ($E_0$) and $\max\{|\vec{E}|\}$ in the material region of 1.209 ($E_0$), leading to acceleration factors of 0.016 and 0.021, respectively.
Comparing these numbers to those in Table \ref{tb:table}, it is apparent that the optimized structures perform better than the dual pillar structures in terms of both gradient and acceleration factor.

These findings suggest that the inverse design strategy is effective in designing not only resonant, high acceleration gradient structures, but also non-resonant structures that are more damage resistant.
In the future, when more components of DLA are moved on-chip (such as the optical power delivery), it will be important to have control over the resonance characteristics of the DLA structures to prevent damage breakdown at the input facet.
Our technique may be invaluable in designing structures with tailor-made quality factors for this application.

\figdef{DLA_Adjoint_fA}{optimization-results_fAC}{
    Demonstration of the final structures after optimization for (a) maximizing gradient only, (b) maximizing the acceleration factor.
    $\beta = 0.5$, laser wavelength $\lambda=2\,\mu$m, gap size of $400$\,nm.
    $\epsilon_m = 2.1$, corresponding to SiO$_2$.
    In (a), the high gradients are achieved using reflective dielectric mirrors to confine and enhance the fields in the center region.
    In (b), these dielectric mirrors are removed and the pillar structures are augmented.
    The structure in (b) shows a 23\% increase in the acceleration factor in the material region when compared to (a).
}

\begin{table}[ht!]
\centering
\caption{Acceleration factor ($f_A$) before and after maximization.}
\begin{tabular}{lccc}
\hline
Quantity & Value (max $G$) & Value (max $f_A$) & Chang.
\\ \hline
Gradient ($E_0$) & 0.1774 & 0.0970 & -45.32\%\\
$\max\{|\vec{E}|\}$ in design region ($E_0$) & 4.1263 & 1.7940 & -56.52\%\\
$\max\{|\vec{E}|\}$ in material region ($E_0$) & 2.7923 & 1.2385 & -55.84\% \\
$f_A$ in design region & 0.0430 & 0.0541 & +25.81\% \\
$f_A$ in material region & 0.0635 & 0.0783 & +23.31\% \\ \hline
\end{tabular}
\label{tb:table}
\end{table}

\section{Conclusions}

%1. Major findings
We found that the adjoint method is a reliable method for optimizing DLA structures for both maximum acceleration gradient and also acceleration factor.
The optimization algorithm discussed shows good convergence and rarely requires further post-processing of structures to create binary permittivity distributions.
Therefore, it is a simple and effective method for designing DLAs.
%2. What is new and novel
Whereas most structure optimization in this field uses parameter sweeps to search the design space, the efficiency of our method allows us to more intelligently find optimal geometries without shape parameterization.
Furthermore, the structures that we design are fabricable.

%3. How does it compare with literature?
Although no DLA structures have been tested at the proposed wavelength of $2\, \mu$m, both simulations \cite{plettner2006proposed} and experimental results from other wavelengths \cite{leedle2015dielectric} show gradients far below those presented here.
%4. What are the limitations?
We had limited success designing DLA structures in the regime of $\beta > 1 / n$, where $n$ is the refractive index of the DLA structure.  In this regime, the electron is moving faster than the speed of light in the material region, which leads to stronger coupling between the electron beam and incident plane wave.  This fact causes optimizations that are more unstable and do not converge on the same structure for several starting parameters.

In addition to the side-incident geometry explored, this technique is applicable to designing other dielectric-based accelerator structures.
This includes particle-laser co-propagating schemes \cite{cowan2008three} and dielectric wakefield acceleration
\cite{zhang1997stimulated}, among others.
Therefore, we expect that our results may find use in the larger advanced accelerator community.

In this chapter, we have introduced the adjoint variable method as a powerful tool for designing dielectric laser accelerators for high gradient acceleration and high acceleration factor.
We have further shown that the adjoint simulation is sourced by a point charge flowing through the accelerator, which quantifies the reciprocal relationship between an accelerator and a radiator.

Optimization algorithms built on this approach allow us to search a substantially larger design space and generate structures that give acceleration gradients far above those normally used for each material.
Furthermore, the structures designed by adjoint are fundamentally not constrained by shape parameterization, allowing never-before-seen geometries to be generated and tested.
This theme will continue to be explored in other chapters.