\section{Higgs boson processes}
%\label{sec:higgs}

In this section, the set-up of the current ATLAS samples for Higgs boson production in gluon--gluon fusion,
vector-boson fusion and Higgs-strahlung processes is described.

\subsection{$H$ via gluon--gluon fusion}
%\label{sec:higgs-ggH}

\subsubsection*{\texorpdfstring{\POWPY[8]}{Powheg+Pythia8} samples}
%\label{sec:higgs-ggH-samples}

%HGam: https://twiki.cern.ch/twiki/bin/view/Sandbox/RuggeroTurraSandbox
%ZZ: https://twiki.cern.ch/twiki/bin/view/AtlasProtected/HZZllllRun2MCSamplesMC16#mc16
%WW: https://twiki.cern.ch/twiki/bin/view/AtlasProtected/HWWMC
%Hlep:https://twiki.cern.ch/twiki/bin/view/AtlasProtected/HSG4MCrequests2015
%Hbb: https://twiki.cern.ch/twiki/bin/view/AtlasProtected/HbbMC

The descriptions below correspond to the samples in Table~\ref{tab:higgs-ggH-samples}.
\begin{table}[!htbp]
\begin{center}
\caption{\POWHEG Higgs gluon--gluon fusion samples with \POWPY[8] for different Higgs boson decay channels.} \label{tab:higgs-ggH-samples}
\begin{tabular}{ l | l | l}
\hline
DSID   & Decay channel & Additional comment \\
\hline
343981 & $H\to\gamma\gamma$&   \\
345316 & $H\to Z\gamma$ & \\
345060 & $H\to ZZ^{*} \to 4\ell $ & $\ell=e,\mu,\tau$   \\
345324 & $H\to WW^{*} \to 2\ell2\nu $ &  $p_{{\textrm T},\ell1} > 15$~\GeV\  and $p_{{\textrm T},\ell2} > 5$~\GeV   \\
308284 & $H\to ZZ^{*}\to 4\nu$ & $E_\text{T}^\text{miss} > 75$~\GeV\\
345342 & $H\to b \bar{b}$ & \\
345097 & $H\to\mu\mu$      &     \\
345120 & $H\to\tau\tau\to \ell^{+}\ell^{-}$      & $p_{{\textrm T},\ell1} > 13$~\GeV\  and $p_{{\textrm T},\ell2} > 7$~\GeV \\
345121 & $H\to\tau\tau\to h^{+}\ell^{-}$      & $p_{{\textrm T},\ell} > 15$~\GeV\  and $p_{{\textrm T},h} > 20$~\GeV\\
345122 & $H\to\tau\tau\to \ell^{+}h^{-}$      & $p_{{\textrm T},\ell} > 15$~\GeV\  and $p_{{\textrm T},h} > 20$~\GeV\\
345123 & $H\to\tau\tau\to h^{+}h^{-}$      & $p_{{\textrm T},h1} > 30$~\GeV\  and $p_{{\textrm T},h2} > 20$~\GeV \\
345124 & $H\to\tau\tau\to \mu\tau$      &\\ %$p_{\textrm T},h1 > 30$~GeV  and $p_{\textrm T},h2 > 20$~GeV
345125 & $H\to\tau\tau\to e \tau$      & \\ %$p_{\textrm T},h1 > 30$~GeV  and $p_{\textrm T},h2 > 20$~GeV
\hline
\end{tabular}
\end{center}
\end{table}

\paragraph{Short description:}

Higgs boson production via gluon--gluon fusion was simulated at next-to-next-to-leading-order (NNLO) accuracy in
QCD using \POWHEGBOX[v2]~\cite{Hamilton:2013fea,Hamilton:2015nsa,Alioli:2010xd,Nason:2004rx,Frixione:2007vw}.
The simulation achieved NNLO accuracy for arbitrary inclusive $gg\to H$ observables by reweighting the Higgs boson
rapidity spectrum in \textsc{Hj}-\MINLO~\cite{Hamilton:2012np,Campbell:2012am,Hamilton:2012rf} to that of HNNLO~\cite{Catani:2007vq}.
%The transverse momentum spectrum of the Higgs boson obtained with this sample was found to be compatible with the fixed-order HNNLO calculation and the Hres2.3 calculation~\cite{Bozzi:2005wk,deFlorian:2011xf} performing resummation at next-to-next-to- leading-logarithm accuracy matched to a NNLO fixed-order calculation (NNLL+NNLO).
The \PDFforLHC[15nnlo] PDF set~\cite{Butterworth:2015oua} and the \AZNLO tune~\cite{STDM-2012-23}
of \PYTHIA[8]~\cite{Sjostrand:2014zea} were used.

The gluon--gluon fusion prediction from the Monte Carlo samples was normalised to the
next-to-next-to-next-to-leading-order cross-section in QCD plus electroweak corrections
at next-to-leading order
(NLO)~\cite{deFlorian:2016spz,Anastasiou:2016cez,Anastasiou:2015ema,Dulat:2018rbf,Harlander:2009mq,Harlander:2009bw,Harlander:2009my,Pak:2009dg,Actis:2008ug,Actis:2008ts,Bonetti:2018ukf,Bonetti:2018ukf}. The decays of bottom and charm hadrons
were performed by \EVTGEN~\cite{Lange:2001uf}.
The normalisation of all Higgs boson
samples accounts for the decay branching ratio calculated with HDECAY~\cite{Djouadi:1997yw,Spira:1997dg,Djouadi:2006bz}
and \PROPHECY~\cite{Bredenstein:2006ha,Bredenstein:2006rh,Bredenstein:2006nk}.

\paragraph{Long description:}

Higgs boson production via gluon--gluon fusion was simulated at next-to-next-to-leading-order (NNLO) accuracy in QCD using
\POWHEGBOX[v2]~\cite{Hamilton:2013fea,Hamilton:2015nsa,Alioli:2010xd,Nason:2004rx,Frixione:2007vw}. The simulation achieved NNLO
accuracy for arbitrary inclusive $gg\to H$ observables by reweighting the Higgs boson rapidity spectrum
in \textsc{Hj}-\MINLO~\cite{Hamilton:2012np,Campbell:2012am,Hamilton:2012rf} to that of HNNLO~\cite{Catani:2007vq}. The transverse momentum
spectrum of the Higgs boson obtained with this sample was found to be compatible with the fixed-order HNNLO calculation and the
\HRES[2.3] calculation~\cite{Bozzi:2005wk,deFlorian:2011xf} performing resummation at next-to-next-to-leading-logarithm accuracy
matched to a NNLO fixed-order calculation (NNLL+NNLO).
Top- and bottom-quark mass effects were included up to next-to-leading order (NLO).

The renormalisation and factorisation scales were set to half of the Higgs boson mass and the \PDFforLHC[15nnlo] PDF
set~\cite{Butterworth:2015oua} was used. The matrix elements were matched to the parton shower of \PYTHIA[8]~\cite{Sjostrand:2014zea}
which uses the AZNLO tune~\cite{STDM-2012-23}. The decays of bottom and charm hadrons
were performed by \EVTGEN~\cite{Lange:2001uf}.

The QCD scale uncertainties were obtained using nine-point scale variations of the NLO renormalisation and factorisation scales and applying the NNLO reweighting to those variations, including up and down variations of $\muR=\muF$ around the central value for the NNLO part, yielding a total of 27 scale variations.
PDF and \alphas uncertainties were estimated using the \PDFforLHC[15nlo] set of eigenvectors.
The envelope of the resulting 27 scale variations was taken to estimate the QCD scale uncertainty.
Uncertainties were also provided for switching off bottom- and top-quark mass effects.
%From YR4

The prediction from the Monte Carlo samples was normalised to the next-to-next-to-next-to-leading-order
cross-section in QCD in the infinite top-quark mass
limit~\cite{deFlorian:2016spz,Anastasiou:2016cez,Anastasiou:2015ema,Dulat:2018rbf,Aglietti:2004nj} and including exact
corrections for all finite quark-mass effects at NLO in QCD as well as NLO electroweak
effects~\cite{Actis:2008ug,Bonetti:2018ukf}. Additionally, corrections to the inverse of the top-quark mass were taken
into account at NNLO~\cite{Harlander:2009mq,Harlander:2009bw,Harlander:2009my,Pak:2009dg}.
The normalisation of all Higgs boson samples accounts for the decay branching ratio calculated with
HDECAY~\cite{Djouadi:1997yw,Spira:1997dg,Djouadi:2006bz}
and \PROPHECY~\cite{Bredenstein:2006ha,Bredenstein:2006rh,Bredenstein:2006nk}.

\subsection{$H$ via vector-boson fusion}
%\label{sec:higgs-VBF}

\subsubsection*{\texorpdfstring{\POWPY[8]}{Powheg+Pythia8} samples}
%\label{sec:higgs-VBF-samples}
The descriptions below correspond to the samples in Table~\ref{tab:higgs-VBF-samples}.
\begin{table}[!htbp]
\begin{center}
\caption{\POWPY[8] Higgs vector-boson fusion samples for different Higgs boson decay channels.} \label{tab:higgs-VBF-samples}
\begin{tabular}{ l | l | l}
\hline
DSID   & Decay channel & Additional comment \\
\hline
346317 & $H\to$ all  & \\
346214 & $H\to\gamma\gamma$&   \\
345833 & $H\to Z\gamma$ & $Z\to \ell^{+}\ell^{-}$\\
345834 & $H\to \gamma\gamma^{*}$ & $\gamma^{*}\to \ell^{+}\ell^{-}$\\
346228 & $H\to ZZ^{*} \to 4\ell $ & $\ell=e,\mu$   \\
450576 & $H\to ZZ^{*} \to 2\ell2b $ &   \\
345948& $H\to WW^{*} \to 2\ell2\nu $ &  $p_{{\textrm T},\ell1} > 15$~\GeV\  and $p_{{\textrm T},\ell2} > 5$~\GeV   \\
346600& $H\to ZZ^{*}\to 4\nu$ & $\MET>75$~\GeV\\
345949 & $H\to b \bar{b}$ & \\
% & $H\to\mu\mu$      &     \\
346190& $H\to\tau\tau\to \ell^{+}\ell^{-}$      & $p_{{\textrm T},\ell1} > 13$~\GeV\  and $p_{{\textrm T},\ell2} > 7$~\GeV \\
346191 & $H\to\tau\tau\to h^{+}\ell^{-}$      & $p_{{\textrm T},\ell} > 15$~\GeV\  and $p_{{\textrm T},h} > 20$~\GeV\\
346192 & $H\to\tau\tau\to \ell^{+}h^{-}$      & $p_{{\textrm T},\ell} > 15$~\GeV\  and $p_{{\textrm T},h} > 20$~\GeV\\
346193 & $H\to\tau\tau\to h^{+}h^{-}$      & $p_{{\textrm T},h1} > 30$~\GeV\  and $p_{{\textrm T},h2} > 20$~\GeV \\
346194 & $H\to\tau\tau\to e\tau$      & \\%$p_{\textrm T},l1 > 10$~GeV  and $p_{\textrm T},h1 > 20$~GeV \\
346195 & $H\to\tau\tau\to \mu\tau$      & \\%$p_{\textrm T},l1 > 10$~GeV  and $p_{\textrm T},h1 > 20$~GeV \\
\hline
\end{tabular}
\end{center}
\end{table}

\paragraph{Short description:}
Higgs boson production via vector-boson fusion was simulated with
\POWHEGBOX[v2]~\cite{Nason:2009ai,Alioli:2010xd,Nason:2004rx,Frixione:2007vw}
and interfaced with \PYTHIA[8]~\cite{Sjostrand:2014zea} for parton shower and non-perturbative effects,
with parameters set according to the \AZNLO tune~\cite{STDM-2012-23}.
The \POWHEGBOX prediction is accurate to next-to-leading order (NLO) and uses
the \PDFforLHC[15nlo] PDF set~\cite{Butterworth:2015oua}.
%It was tuned to match calculations with effects due to finite heavy-quark masses
%and soft-gluon resummation up to NNLL.
It was normalised to an approximate-NNLO QCD cross-section
with NLO electroweak corrections~\cite{Ciccolini:2007jr,Ciccolini:2007ec,Bolzoni:2010xr}.
The decays of bottom and charm hadrons were performed by \EVTGEN~\cite{Lange:2001uf}.
The normalisation of all Higgs boson samples accounts for the decay branching ratio calculated
with \textsc{HDECAY}~\cite{Djouadi:1997yw,Spira:1997dg,Djouadi:2006bz}
and \PROPHECY~\cite{Bredenstein:2006ha,Bredenstein:2006rh,Bredenstein:2006nk}.

\paragraph{Long description:}
Higgs boson production via vector-boson fusion was simulated with
\POWHEGBOX[v2]~\cite{Nason:2009ai,Alioli:2010xd,Nason:2004rx,Frixione:2007vw}. A factorised approximation, where cross-talk between the
fermion lines is neglected, was used. The implementation is based on the respective NLO QCD calculations for genuine $W/Z$ vector-boson
fusion topologies (VBF approximation). Quark--antiquark annihilation and interference contributions
between $t$- and $u$-channel contributions were disregarded.
%External fermion masses are neglected throughout.  For the choice of renormalisation andfactorisation scales, various options are available.  For this report, fixed scales,μF=μR=MW, areused, and contributions from external bottom and top quarks are entirely disregarded.

The renormalisation and factorisation scales were set to the $W$ boson mass and the \PDFforLHC[15nlo] PDF set~\cite{Butterworth:2015oua}
was used. The matrix elements were matched to the parton shower of \PYTHIA[8]~\cite{Sjostrand:2014zea} which uses the \AZNLO
tune~\cite{STDM-2012-23}. A dipole-recoil strategy was used for the parton shower.The decays of bottom and charm hadrons
were performed by \EVTGEN~\cite{Lange:2001uf}.%, which provides a better agreement than other recoil schemes with the NNLO predictions of proVBFH (not included in the HiggsReference twiki). % cite https://arxiv.org/pdf/1710.00391.pdf ?,

%The samples include weight variations evaluated on-the-fly for the estimation of QCD scale, PDF- and \alphas-induced uncertainties.
The QCD scales \muR and \muF were varied independently by factors of 0.5 and 2.0, both in the matrix element
and in the parton shower.

The prediction from the \POWHEGBOX sample was normalised to the next-to-next-to-leading-order cross-section in QCD
using the VBF approximation~\cite{Ciccolini:2007jr,Ciccolini:2007ec,Bolzoni:2010xr}. Relative
next-to-leading-order electroweak corrections were also taken into account for the $t$- and $u$-channel contribution
considered in the VBF approximation. The normalisation of all Higgs boson samples accounts for the decay branching
ratio calculated with \textsc{HDECAY}~\cite{Djouadi:1997yw,Spira:1997dg,Djouadi:2006bz} and
\PROPHECY~\cite{Bredenstein:2006ha,Bredenstein:2006rh,Bredenstein:2006nk}.

\subsection{$VH$}
%\label{sec:higgs-VH}

This section describes the generation details of nominal samples for Higgs-strahlung production.

\subsubsection*{\texorpdfstring{\POWPY[8]}{Powheg+Pythia8} samples}
%\label{sec:higgs-VH-samples}
The descriptions below correspond to the samples in Tables~\ref{tab:higgs-ZH-samples} to \ref{tab:higgs-ggZH-samples}.

\begin{table}[!htbp]
\begin{center}
\caption{\POWPY[8] samples of Higgs boson production in association with a $Z$ boson for different Higgs boson decay channels}
\label{tab:higgs-ZH-samples}
\begin{tabular}{ l | l | l}
\hline
DSID   & Decay channel & Additional comment \\
\hline
345038 & $H\to ZZ^{*} \to 4\ell$ & $Z\to$ all\\
345319 & $H\to \gamma\gamma$ & $Z\to$ all\\
345322 & $H\to Z\gamma$ & $Z\to$ all\\
345103 & $H\to\mu\mu$ & $Z\to$ all\\
345217 & $H\to\tau\tau$& $Z\to$ all\\
345218 & $H\to e\tau$ &$Z\to$ all\\
345219 & $H\to \mu\tau$&$Z\to$ all\\
345445 &$H\to WW^{*} \to 2\ell2\nu $ & $Z\to$ all \\
345876 & $H\to ee$ &$Z\to$ all \\
345965 &  $H\to \gamma\gamma^{*}$ & $\gamma^{*}\to \ell^{+}\ell^{-}$; $Z\to$ all\\
346310 & $H\to$ all&$Z\to$ all\\
346607 & $H\to ZZ^{*} \to 4\nu $ &$Z\to \ell^{+}\ell^{-}$ \\
345055 & $H\to b\bar{b}$ & $p_{{\textrm T},Z}$ enhancement; $Z\to \ell^{+}\ell^{-}$\\
345111 & $H\to c\bar{c}$ & $p_{{\textrm T},Z}$ enhancement; $Z\to \ell^{+}\ell^{-}$\\
345337 & $H\to WW^{*} \to 2\ell2\nu $ &$p_{{\textrm T},Z}$ enhancement; $Z\to \ell^{+}\ell^{-}$\\
346326 & $H\to\tau\tau$& $p_{{\textrm T},Z}$ enhancement; $Z\to \ell^{+}\ell^{-}$\\
346693 & $H\to ZZ^{*} \to 4\nu $ & $p_{{\textrm T},Z}$ enhancement; $Z\to \ell^{+}\ell^{-}$\\
345056 & $H\to b\bar{b}$ & $p_{{\textrm T},Z}$ enhancement; $Z\to \nu\bar{\nu}$\\
345112 & $H\to c\bar{c}$ & $p_{{\textrm T},Z}$ enhancement; $Z\to \nu\bar{\nu}$\\
345445 & $H\to WW^{*} \to 2\ell2\nu$  & $p_{{\textrm T},Z}$ enhancement; $Z\to \nu\bar{\nu}$\\
\hline
\end{tabular}
\end{center}
\end{table}

\begin{table}[!htbp]
\begin{center}
\caption{ \POWPY[8] samples of Higgs boson production in association with a $W^{+}$ boson for different Higgs boson decay channels}
%\label{tab:higgs-WpH-samples}
\begin{tabular}{ l | l | l}
\hline
DSID   & Decay channel & Additional comment \\
\hline
345039 & $H\to ZZ^{*} \to 4\ell$ & $W^{+}\to$ all\\
345318 & $H\to \gamma\gamma$ & $W^{+}\to$ all\\
345104 & $H\to \mu\mu$ &  $W^{+}\to$ all\\
345212 & $H\to \tau\tau$ &  $W^{+}\to$ all\\
345214 & $H\to e \tau $ & $W^{+}\to$ all\\
345216 & $H\to \mu \tau $ & $W^{+}\to$ all\\
345321 & $H\to Z\gamma$ & $W^{+}\to$ all\\
345325 & $H\to WW^{*} \to 2\ell2\nu $ & $W^{+} \to q\bar{q}$ \\
345877 & $H\to e^{+}e^{-}$ & $W^{+}\to$ all\\
345964 & $H\to \gamma\gamma^{*}$ & $\gamma^{*}\to \ell^{+}\ell^{-}$; $W^{+}\to$ all\\
346311 & $H\to$ all & $W^{+}\to$ all\\
346605 & $H\to ZZ^{*} \to 4\nu $ & $W^{+}\to$ all\\
346699 & $H \to 4\ell$ & interfaced to PROPHECY4F \\
346705 & $H \to 4\ell$ & interfaced to Hto4l \\
345054 & $H\to b\bar{b}$ & $p_{{\textrm T},W}$ enhancement; $W^{+}\to \ell^{+}\nu$ \\
345110 & $H\to c\bar{c}$ & $p_{{\textrm T},W}$ enhancement; $W^{+}\to \ell^{+}\nu$ \\
345327 & $H\to WW^{*} \to 2\ell2\nu$ & $p_{{\textrm T},W}$ enhancement; $W^{+}\to \ell^{+}\nu$ \\
346561 & $H\to WW^{*} \to q\bar{q}\ell\nu$ & $p_{{\textrm T},W}$ enhancement; $W^{+}\to \ell^{+}\nu$ \\
346325 & $H\to \tau\tau$ & $p_{{\textrm T},W}$ enhancement; $W^{+}\to \ell^{+}\nu$ \\
346729 & $H\to ZZ^{*} \to 4\nu$ & $p_{{\textrm T},W}$ enhancement; $W^{+}\to \ell^{+}\nu$ \\
\hline
\end{tabular}
\end{center}
\end{table}

\begin{table}[!htbp]
\begin{center}
\caption{\POWPY[8] samples of Higgs boson production in association with a $W^{-}$ boson for different Higgs boson decay channels}
%\label{tab:higgs-WmH-samples}
\begin{tabular}{ l | l | l}
\hline
DSID   & Decay channel & Additional comment \\
\hline
345040 & $H\to ZZ^{*} \to 4\ell$ & $W^{-}\to$ all\\
345317 & $H\to \gamma\gamma$ & $W^{-}\to$ all\\
345105 & $H\to \mu\mu$ &  $W^{-}\to$ all\\
345211 & $H\to \tau\tau$ &  $W^{-}\to$ all\\
345213 & $H\to e \tau $ & $W^{-}\to$ all\\
345215 & $H\to \mu \tau $ & $W^{-}\to$ all\\
345320 & $H\to Z\gamma$ & $W^{-}\to$ all\\
345333 & $H\to WW^{*} \to 2\ell2\nu $ & $W^{-} \to q\bar{q}$ \\
345878 & $H\to e^{+}e^{-}$ & $W^{-}\to$ all\\
345963 & $H\to \gamma\gamma^{*}$ & $\gamma^{*}\to \ell^{+}\ell^{-}$; $W^{-}\to$ all\\
346312 & $H\to$ all & $W^{-}\to$ all\\
346606 & $H\to ZZ^{*} \to 4\nu $ & $W^{-}\to$ all\\
346700 & $H \to 4\ell$ & interfaced to PROPHECY4F \\
346706 & $H \to 4\ell$ & interfaced to Hto4l \\
345053 & $H\to b\bar{b}$ & $p_{{\textrm T},W}$ enhancement; $W^{-}\to \ell^{-}\bar{\nu}$ \\
345109 & $H\to c\bar{c}$ & $p_{{\textrm T},W}$ enhancement; $W^{-}\to \ell^{-}\bar{\nu}$  \\
345326 & $H\to WW^{*} \to 2\ell2\nu$ & $p_{{\textrm T},W}$ enhancement;  $W^{-}\to \ell^{-}\bar{\nu}$ \\
346560 & $H\to WW^{*} \to q\bar{q}\ell\nu$ & $p_{{\textrm T},W}$ enhancement; $W^{-}\to \ell^{-}\bar{\nu}$ \\
346324 & $H\to \tau\tau$ & $p_{{\textrm T},W}$ enhancement; $W^{-}\to \ell^{-}\bar{\nu}$ \\
346730 & $H\to ZZ^{*} \to 4\nu$ & $p_{{\textrm T},W}$ enhancement; $W^{-}\to \ell^{-}\bar{\nu}$ \\
\hline
\end{tabular}
\end{center}
\end{table}

\begin{table}[!htbp]
\begin{center}
\caption{\POWPY[8] samples of loop-induced Higgs boson production in association with a $Z$ boson for different
Higgs boson decay channels}
\label{tab:higgs-ggZH-samples}
\begin{tabular}{ l | l | l}
\hline
DSID   & Decay channel & Additional comment \\
\hline
345061 & $H\to \gamma\gamma$ & $Z\to$ all \\
345066 & $H\to ZZ^{*} \to 4\ell$ & $Z\to$ all \\
345098 & $H\to \mu\mu$ & $Z\to$ all \\
345596 & $H\to ZZ^{*} \to 4\nu$ & $Z\to$ all \\
346524 & $H\to WW^{*} \to 2\ell2\nu $ & $Z\to$ all \\
346697 & $H\to ZZ^{*} \to 4\nu$ & interfaced to PROPHECY4F \\
346703 & $H\to ZZ^{*} \to 4\nu$ & interfaced to Hto4l \\
345057 & $H\to b\bar{b}$ &  $Z\to \ell^{+}\ell^{-}$\\
345113 &  $H\to c\bar{c}$ & $Z\to \ell^{+}\ell^{-}$\\
345446 & $H\to WW^{*} \to 2\ell2\nu$ &  $Z\to \ell^{+}\ell^{-}$\\
346329 & $H\to \tau\tau$ &  $Z\to \ell^{+}\ell^{-}$\\
346694 & $H\to ZZ^{*} \to 4\nu$ &  $Z\to \ell^{+}\ell^{-}$\\
345058 & $ H\to b\bar{b}$ & $Z\to \nu\bar{\nu}$\\
345114 & $ H\to c\bar{c}$ & $Z\to \nu\bar{\nu}$\\
\hline
\end{tabular}
\end{center}
\end{table}

\paragraph{Short description:}

Higgs boson production in association with a vector boson was simulated using
\POWHEGBOX[v2]~\cite{Nason:2009ai,Alioli:2010xd,Nason:2004rx,Frixione:2007vw} and interfaced with \PYTHIA[8]~\cite{Sjostrand:2014zea} for
parton shower and non-perturbative effects. The \POWHEGBOX prediction is accurate to next-to-leading order for $VH$ boson plus one-jet production.
The loop-induced $gg\to ZH$ process was generated separately at leading order. The \PDFforLHC[15nlo] PDF
set~\cite{Butterworth:2015oua} and the \AZNLO tune~\cite{STDM-2012-23} of \PYTHIA[8]~\cite{Sjostrand:2014zea} were used. The decays of bottom and charm hadrons
were performed by \EVTGEN~\cite{Lange:2001uf}. The Monte Carlo
prediction was normalised to cross-sections calculated at NNLO in QCD with NLO electroweak corrections for $q\bar{q}/qg \to VH$ and at NLO
and next-to-leading-logarithm accuracy in QCD for $gg \to
ZH$~\cite{Ciccolini:2003jy,Brein:2003wg,Brein:2011vx,Altenkamp:2012sx,Denner:2014cla,Brein:2012ne,Harlander:2014wda}. The
normalisation of all Higgs boson samples accounts for the decay branching ratio calculated with
HDECAY~\cite{Djouadi:1997yw,Spira:1997dg,Djouadi:2006bz} and \PROPHECY~\cite{Bredenstein:2006ha,Bredenstein:2006rh,Bredenstein:2006nk}.

\paragraph{Long description:}
Higgs boson production in association with a vector boson was simulated using
\POWHEGBOX[v2]~\cite{Nason:2009ai,Alioli:2010xd,Nason:2004rx,Frixione:2007vw}. The computation was carried out using the
\MINLO~\cite{Luisoni:2013cuh} prescription, which achieves NLO accuracy for the $VH$ and $VH$ boson plus one-jet production
inclusive distributions and dictates the choice of renormalisation and factorisation scales. Virtual amplitudes were constructed
through the interface to the \GOSAM package~\cite{Cullen:2011ac}. The loop-induced $gg\to ZH$ process was generated separately at
leading order with \POWHEGBOX. In all cases, the \PDFforLHC[15nlo] PDF set~\cite{Butterworth:2015oua} was used.

The matrix elements were matched to the parton shower of \PYTHIA[8]~\cite{Sjostrand:2014zea} which uses the \AZNLO
tune~\cite{STDM-2012-23}. The decays of bottom and charm hadrons were performed by \EVTGEN~\cite{Lange:2001uf}.
%The samples include weight variations evaluated on-the-fly for the estimation of QCD scale, PDF- and \alphas-induced uncertainties.
The QCD scales \muR and \muF were varied independently by factors of 0.5 and 2.0 to account for their uncertainties.

The predictions from \POWHEGBOX were normalised to the best available theoretical prediction. The  $q\bar{q}/qg \to VH$ cross-sections
were calculated at NNLO in QCD with NLO electroweak corrections and the $gg \to ZH$ cross-sections were calculated at NLO and
next-to-leading-logarithm accuracy~\cite{Ciccolini:2003jy,Brein:2003wg,Brein:2011vx,Altenkamp:2012sx,Denner:2014cla,Brein:2012ne,Harlander:2014wda}.
The normalisation of the $q\bar{q}\to ZH$ samples was extracted from the subtraction of the latter from the former.
Differential NLO EW corrections were available from the \textsc{HAWK} program~\cite{Denner:2014cla} to be applied to
$q\bar{q}$-initiated $VH$ production as a function of the vector boson's transverse momentum.
The normalisation of all Higgs boson samples accounts for the decay branching ratio calculated with
\textsc{HDECAY}~\cite{Djouadi:1997yw,Spira:1997dg,Djouadi:2006bz} and
\PROPHECY~\cite{Bredenstein:2006ha,Bredenstein:2006rh,Bredenstein:2006nk}.


%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
% This work is licensed under the Creative Commons Attribution 4.0 International %
% License. To view a copy of this license, visit                                 %
% http://creativecommons.org/licenses/by/4.0/.                                   %
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\documentclass[11pt]{article}
\usepackage[cm]{fullpage}
%%AVC PACKAGES
\usepackage{avcgreek}
\usepackage{avcfonts}
\usepackage{avcmath}
\usepackage[numberby=section,skip=9pt plus 2pt minus 5pt]{avcthm}
\usepackage{qcmacros}
\usepackage{goldstone}
%%MACROS FOR THIS DOCUMENT
\numberwithin{equation}{section}
\usepackage[
  margin=1.5cm,
  includefoot,
  footskip=30pt,
  headsep=0.2cm,headheight=1.3cm
]{geometry}
\usepackage{fancyhdr}
\pagestyle{fancy}
\fancyhf{}
\fancyhead[LE,RO]{Quiz 6, Handout 1: Perturbation Theory}
\fancyfoot[CE,CO]{\thepage}
\usepackage{url}
\makeatother
\newcommand{\resolventline}[2][1]{
  \tikz[overlay]{
      \draw[thick,flexdotted] (0,-1ex) to ++(0,#1*4.5ex) node[above,inner sep=1pt] {#2};
  }
}

\begin{document}

\setcounter{section}{5}
\section{Perturbation theory}

\begin{dfn}
\thmtitle{Model Hamiltonian}
The electronic Hamiltonian\footnote{For the sake of brevity I will here refer to $H_\mr{c}$ as ``the electronic Hamiltonian''.  We could also use $H=E_0+H_\mr{c}$, which will simply shift some of the equations by a constant.} can be expressed as the sum of a \textit{zeroth order} or \textit{``model''~Hamiltonian} $H_0$ and a \textit{perturbation} $V_\mr{c}$, known as the \textit{fluctuation potential}.
For well-behaved electronic systems, a common choice for the model Hamiltonian is the diagonal part of the Fock operator.
\begin{align}
\label{eq:diagonal-fock-model-hamiltonian}
  H_0
\equiv
  f_p^p
  \tl{a}^p_p
&&
  V_\mr{c}
\equiv
  f_p^q
  (
    1
  -
    \d_p^q
  )
  \tl{a}^p_q
+
  \tfr{1}{4}
  \ol{g}_{pq}^{rs}
  \tl{a}^{pq}_{rs}
\end{align}
This choice of $H_0$ brings the advantage that its eigenbasis is the standard basis of determinants.
\begin{align}
\label{eq:model-problem}
  H_0
  \F
=
  0\,
  \F
&&
  H_0
  \F_{i_1\cd i_k}^{a_1\cd a_k}
=
  \mc{E}_{i_1\cd i_k}^{a_1\cd a_k}
  \F_{i_1\cd i_k}^{a_1\cd a_k}
&&
  \mc{E}_{q_1\cd q_k}^{p_1\cd p_k}
\equiv
  \sum_{r=1}^k
  f_{p_r}^{p_r}
-
  \sum_{r=1}^k
  f_{q_r}^{q_r}
\end{align}
In general the model Hamiltonian is chosen to make the matrix representation of $H_\mr{c}$ in the model eigenbasis diagonally dominant.\footnote{See \url{https://en.wikipedia.org/wiki/Diagonally_dominant_matrix}.}
Our choice of $H_0$ is appropriate for \textit{weakly correlated systems}, where the reference determinant can be chosen to satisfy $\ip{\F|\Y}\gg \ip{\F_{i_1\cd i_k}^{a_1\cd a_k}|\Y}$ for all substituted determinants.
In this context it is convenient to employ intermediate normalization for the wavefunction, which will be assumed from here on out.
\end{dfn}

\begin{dfn}
\thmtitle{Perturbation theory}
\textit{Perturbation theory} analyzes the polynomial order with which the wavefunction and its observables depend on the fluctuation potential.
For this purpose, we define a continuous series of Hamiltonians
$
  H(\la)
\equiv
  H_0
+
  \la
  V_\mr{c}
$
parametrized by a \textit{strength parameter} $\la$ that smoothly toggles between the model Hamiltonian at $\la=0$ to the exact one at $\la=1$.
The \textit{$m\eth$-order contribution} to a quantity $X$ is then defined as the $m\eth$ coefficient in its Taylor series about $\la=0$, denoted $X\ord{m}$.
In particular, the wavefunction and correlation energy can be expanded as follows.
\begin{align}
\label{eq:series-schrodinger-equation}
  \Y
=
  \sum_{m=0}^\infty
  \Y\ord{m}
&&
  E_\mr{c}
=
  \sum_{m=0}^\infty
  E_\mr{c}\ord{m}
&&
  \Y\ord{m}
\equiv
  \fr{1}{m!}
  \left.
    \pd{^m\Y(\la)}{\la^m}
  \right|_{\la=0}
&&
  E_\mr{c}\ord{m}
\equiv
  \fr{1}{m!}
  \left.
    \pd{^mE(\la)}{\la^m}
  \right|_{\la=0}
&&
  H(\la)
  \Y(\la)
=
  E(\la)
  \Y(\la)
\end{align}
The order(s) at which a term contributes to the wavefunction or energy provides one measure of its relative importance.
\end{dfn}

\begin{rmk}
Projecting the Schr\"odinger equation by $\F$ and using eq~\ref{eq:model-problem}, along with intermediate normalization, implies
\begin{align}
  E_\mr{c}
=
  \ip{\F|V_\mr{c}|\Y}
\hspace{20pt}
\implies
\hspace{20pt}
  E_\mr{c}\ord{m+1}
=
  \ip{\F|V_\mr{c}|\Y\ord{m}}
\end{align}
where the equation on the right follows from generalizing the energy expression to
$
  E(\la)
=
  \ip{\F|\la V_\mr{c}|\Y(\la)}
$.
In words, this says that the $m\eth$-order wavefunction contribution determines the $(m+1)\eth$-order energy contribution.
This immediately identifies the first-order energy as
$
  E_\mr{c}\ord{1}
=
  \ip{\F|V_\mr{c}|\F}
=
  0
$,
since $V_\mr{c}$ consists of $\F$-normal-ordered operators.
\end{rmk}

\begin{dfn}
\thmtitle{Model space projection operator}
The projection onto the reference determinant, $P=\kt{\F}\br{\F}$, is termed the \textit{model space projection operator}.
Its complement is the \textit{orthogonal space projection operator}.\footnote{$1_n\equiv 1|_{\mc{F}_n}$ is the identity on $\mc{F}_n$, which is equivalent to a projection onto this subspace.  For our purposes, this is the identity.}
\begin{align}
\label{eq:orthogonal-space-projection-operator}
  Q
\equiv
  1_n
-
  P
=
  \sum_k
  \pr{
    \tfr{1}{k!}
  }^2
  \sum_{\substack{a_1\cd a_k\\i_1\cd i_k}}
  \kt{\F_{i_1\cd i_k}^{a_1\cd a_k}}
  \br{\F_{i_1\cd i_k}^{a_1\cd a_k}}
\end{align}
Note that $P$ and $Q$ satisfy the following relationships, which are characteristic of complementary projection operators.
\begin{align}
  P
+
  Q
=
  1_n
&&
  P^2
=
  P
&&
  Q^2
=
  Q
&&
  PQ
=
  QP
=
  0
\end{align}
Due to intermediate normalization, we also have that
$
  P\Y
=
  \F
$
and
$
  Q\Y
=
  \Y
-
  \F
$.
\end{dfn}

\begin{samepage}
\begin{dfn}
\thmtitle{Resolvent}
The \textit{resolvent},
$
  R_0
\equiv
  (-H_0)^{-1}Q
$, is the negative\footnote{The annoying sign factor is required for consistency with $R(\zeta)\equiv(\zeta-H_0)^{-1}Q$, which is a more general definition of the resolvent.} inverse of $H_0$ in the orthogonal space.\footnote{Note that this implies $R_0P=0$ and $R_0Q=R_0$.}
\begin{align}
\label{eq:resolvent-spectral-decomposition}
  R_0
  \F
=
  0
  \F
&&
  R_0
  \F_{i_1\cd i_k}^{a_1\cd a_k}
=
  (\mc{E}_{a_1\cd a_k}^{i_1\cd i_k})^{-1}
  \F_{i_1\cd i_k}^{a_1\cd a_k}
&&
  R_0
=
  \sum_k
  \pr{\tfr{1}{k!}}^2
  \sum_{\substack{a_1\cd a_k\\i_1\cd i_k}}
  \fr{
    \kt{\F_{i_1\cd i_k}^{a_1\cd a_k}}
    \br{\F_{i_1\cd i_k}^{a_1\cd a_k}}
  }{
    \mc{E}_{a_1\cd a_k}^{i_1\cd i_k}
  }
\end{align}
The equation on the right is the spectral decomposition of the resolvent.\footnote{This follows from the eigenvalue equations, but you can derive it explicitly by substituting equation~\ref{eq:orthogonal-space-projection-operator} into $R_0=(-H_0)^{-1}Q$.}
Restriction to the orthogonal space is necessary because $H_0$ is singular in the model space, which means that $H_0^{-1}$ does not exist there.
\end{dfn}
\end{samepage}

\begin{samepage}
\begin{rmk}
\thmtitle{A recursive solution to the Schr\"odinger equation}
Operating $R_0$ on $H(\la)\Y(\la)=E(\la)\Y(\la)$ gives\footnote{This follows from $R_0H_0\Y=-Q\Y=-\Y+\F$.}
\begin{align}
\label{eq:lambda-dependent-recursive-series}
  \Y(\la)
=
  \F
+
  R_0
  (
    \la V_\mr{c}
  -
    E(\la)
  )
  \Y(\la)
\end{align}
which provides a recursive equation for $\Y(\la)$ that can be used to solve for wavefunction contributions order by order.
\end{rmk}
\end{samepage}

\begin{ex}
The first two derivatives of equation~\ref{eq:lambda-dependent-recursive-series} are given by
\begin{align*}
  \pd{\Y(\la)}{\la}
=&
  R_0
  \pr{
    V_\mr{c}
  -
    \pd{E(\la)}{\la}
  }
  \Y(\la)
+
  R_0
  (
    \la V_\mr{c}
  -
    E(\la)
  )
  \pd{\Y(\la)}{\la}
\\
  \pd{^2\Y(\la)}{\la^2}
=&
-
  R_0
  \pd{^2E(\la)}{\la^2}
  \Y(\la)
+
  2
  R_0
  \pr{
    V_\mr{c}
  -
    \pd{E(\la)}{\la}
  }
  \pd{\Y(\la)}{\la}
+
  R_0
  (
    \la V_\mr{c}
  -
    E(\la)
  )
  \pd{^2\Y(\la)}{\la^2}
\end{align*}
which can be used to determine the first- and second-order wavefunction contributions.
\begin{align}
\label{eq:first-and-second-order-principal-terms}
  \Y\ord{1}
=
  \left.
  \pd{\Y(\la)}{\la}
  \right|_{\la=0}
=
  R_0
  V_\mr{c}
  \F
&&
  \Y\ord{2}
=
  \left.
  \fr{1}{2}
  \pd{^2\Y(\la)}{\la^2}
  \right|_{\la=0}
=
  R_0
  V_\mr{c}
  \Y\ord{1}
=
  R_0
  V_\mr{c}
  R_0
  V_\mr{c}
  \F
\end{align}
Here we have used $E_\mr{c}\ord{0}=E_\mr{c}\ord{1}=0$ and $R_0\F=0$ to simplify the result.
\end{ex}

\begin{ex}
\label{ex:first-order-wavefunction-expansion-unsimplified}
Plugging in the spectral decomposition for $R_0$ allows us to expand $\Y\ord{1}$ in the determinant basis.
\begin{align}
\label{eq:first-order-wavefunction-expansion-unsimplified}
  \Y\ord{1}
=
  R_0
  V_\mr{c}
  \F
=
  \sum_{\substack{a\\i}}
  \F_i^a
  \fr{\ip{\F_i^a|V_\mr{c}|\F}}{\mc{E}_a^i}
+
  (\tfr{1}{2!})^2
  \sum_{\substack{ab\\ij}}
  \F_{ij}^{ab}
  \fr{\ip{\F_{ij}^{ab}|V_\mr{c}|\F}}{\mc{E}_{ab}^{ij}}
\end{align}
The expansion truncates at double excitations because the maximum excitation level of $V_\mr{c}$ is $+2$.
\end{ex}

\begin{ex}
The numerators in example~\ref{ex:first-order-wavefunction-expansion-unsimplified} are easily evaluated using Slater's rules, which leads to the following.
\begin{align*}
\label{eq:first-order-wavefunction-expansion}
  \Y\ord{1}
=
  \sum_{\substack{a\\i}}
  \F_i^a
  \fr{f_a^i}{\mc{E}_a^i}
+
  (\tfr{1}{2!})^2
  \sum_{\substack{ab\\ij}}
  \F_{ij}^{ab}\,
  \fr{\ol{g}_{ab}^{ij}}{\mc{E}_{ab}^{ij}}
\hspace{20pt}
\implies
\hspace{20pt}
  E_\mr{c}\ord{2}
=
  \ip{\F|V_\mr{c}|\Y\ord{1}}
=
  \sum_{\substack{a\\i}}
  \fr{f_i^af_a^i}{\mc{E}_a^i}
+
  (\tfr{1}{2!})^2
  \sum_{\substack{ab\\ij}}
  \fr{\ol{g}_{ij}^{ab}\,\ol{g}_{ab}^{ij}}{\mc{E}_{ab}^{ij}}
\end{align*}
Note that the singles contribution vanishes for canonical Hartree-Fock references, since $f_a^i=0$.
These extra terms are required for non-canonical orbitals, such as those obtained from restricted open-shell Hartree-Fock (ROHF) theory.
\end{ex}

\begin{dfn}\label{dfn:resolvent-line}
\thmtitle{Resolvent line}
We can generalize our previous definition of the \textit{resolvent line} as follows
\begin{align}
  \,\resolventline[0.7]{}\,
  Y
\equiv
  \sum_k
  \pr{\tfr{1}{k!}}^2
  \sum_{\substack{a_1\cd a_k\\i_1\cd i_k}}
  \fr{y_{a_1\cd a_k}^{i_1\cd i_k}}{\mc{E}_{a_1\cd a_k}^{i_1\cd i_k}}
  \tl{a}^{a_1\cd a_k}_{i_1\cd i_k}
&&
  Y
=
  Y_{n\rightarrow n}
+
  Y_{n\not\rightarrow n}
&&
  Y_{n\rightarrow n}
=
  y_0
+
  \sum_k
  \pr{\tfr{1}{k!}}^2
  \sum_{\substack{p_1\cd p_k\\q_1\cd q_k}}
  y_{p_1\cd p_k}^{q_1\cd q_k}
  \tl{a}^{p_1\cd p_k}_{q_1\cd q_k}
\end{align}
where $Y$ is an arbitrary operator.
The last equation is the Wick expansion of $Y_{n\rightarrow n}$, which denotes the purely particle-number-conserving part\footnote{The component that maps $\mc{F}_n\rightarrow\mc{F}_n$ for all $n$, which can always be written as a linear combination of excitation operators.} of $Y$.
This definition immediately implies
$
  \,\resolventline[0.7]{}\,\,
  \kt{\Y}
=
  R_0
  \kt{\Y}
$
for all $\Y$.\footnote{Since any $\kt{\Y}$ can be written as $Y\kt{\F}$, this follows from applying eq~\ref{eq:resolvent-spectral-decomposition} to each term in the Wick expansion of $Y$ in $R_0Y\kt{\F}$.}
Other expressions are defined by giving resolvent lines priority in the order of operations, with maximum priority given to the rightmost resolvent.
\begin{align}
    Y_1
  \,\resolventline[0.7]{}\,
    Y_2
  \cd
  \,\resolventline[0.7]{}\,
    Y_n
\equiv
    Y_1
  \pr{
  \,\resolventline[0.7]{}\,
    Y_2
  \pr{
  \cd
  \pr{
  \,\resolventline[0.7]{}\,
    Y_n
    \vphantom{Y_Y^Y}
  }\cd}}
&&
\gno{\ol{
    Y_1
  \,\resolventline[0.8]{}\,
    Y_2
  \cd
  \,\resolventline[0.8]{}\,
    Y_n
}}
\equiv
\gno{\ol{
  \,\,
    Y_1
  \pr{
  \,\resolventline[0.7]{}\,
    Y_2
  \pr{
  \cd
  \pr{
  \,\resolventline[0.7]{}\,
    Y_n
    \vphantom{Y_Y^Y}
  }\cd}}
}}
\end{align}
This definition also specifies the interpretation rule for a graphs with resolvent lines, which are formally defined below.
\end{dfn}

\begin{cor}\label{cor:wicks-theorem-for-pt}
\thmtitle{Wick's theorem for perturbation theory}
\thmstatement{
$
  YR_0Y_1\cd R_0 Y_m
  \kt{\F}
=
  \pr{
    \gno{
      Y
      \,\resolventline[0.8]{}\,
      Y_1
      \cd
      \,\resolventline[0.8]{}\,
      Y_m
    }
  +
    \gno{\ol{
      Y
      \,\resolventline[0.8]{}\,
      Y_1
      \cd
      \,\resolventline[0.8]{}\,
      Y_m
    }}
  }
  \kt{\F}
$%
}%
\vspace{5pt}
\thmproof{
  This follows directly from Wick's theorem and definition~\ref{dfn:resolvent-line}.
}
\end{cor}

\begin{dfn}\label{dfn:resolvent-graph}
\thmtitle{Resolvent graph}
A \textit{resolvent graph} represents a normal-ordered product of operators and resolvents.
Graphs with disconnected parts that don't share any resolvent lines are considered products of separate resolvent graphs.
Vertical spaces between resolvent lines in a resolvent graph are termed \textit{levels}, which are numbered from bottom with zero indexing.
Therefore, an operator lies in the $k\eth$ level if there are $k$ resolvent lines below it.
Formally, then, an \textit{$m$-level resolvent graph} $G(\rh,m)\equiv(G,\rh,m)$ associates each operator $o$ in $G$ with a specific level $\rh(o)=\rh_o$ in $\mb{Z}_m=\{0,1,\ld,m-1\}$ through the \textit{level map} $\rh$.\,\footnote{
  Note that an $m$-level resolvent graph contains $m-1$ resolvents.
}
Therefore, each line $l$ in $G$ crosses resolvents
$
  \mr{min}(\rh_{h(l)},\rh_{t(l)}) + 1
$
through
$
  \mr{max}(\rh_{h(l)},\rh_{t(l)})
$.
\end{dfn}

\begin{ex}
In diagram notation, $\Y\ord{1}$ and $E_\mr{c}\ord{2}$ can be expressed as follows.
\begin{align}
  \Y\ord{1}
=
\diagram[bottom]{
  \draw
    (0,-0.5)
      node[circlep] {}
    to
    ++(0.5,0)
      node[ddot] (f1) {};
  \draw[->-]
    (f1)
    to
      ++(-0.25,1);
  \draw[-<-]
    (f1)
    to
      ++(+0.25,1);
  \draw[thick,flexdotted] (0.2,+0.25) to ++(0.6,0);
}
+
\diagram[bottom]{
  \interaction{2}{g}{(0,-0.5)}{ddot}{sawtooth};
  \draw[->-]
    (g1)
    to
      ++(-0.25,1);
  \draw[-<-]
    (g1)
    to
      ++(+0.25,1);
  \draw[->-]
    (g2)
    to
      ++(-0.25,1);
  \draw[-<-]
    (g2)
    to
      ++(+0.25,1);
  \draw[opacity=0] (0.5,-0.5) circle (0.125cm);
  \draw[thick,flexdotted] (-0.3,+0.25) to ++(1.6,0);
}
&&
  E_\mr{c}\ord{2}
=
\diagram{
  %top
  \draw
    (0,+0.5)
      node[circlep] {}
    to
    ++(0.5,0)
      node[ddot] (1f1) {};
  %bottom
  \draw
    (0,-0.5)
      node[circlep] {}
    to
    ++(0.5,0)
      node[ddot] (2f1) {};
  \draw[->-=0.4,bend left ] (2f1) to (1f1);
  \draw[-<-=0.6,bend right] (2f1) to (1f1);
  \draw[thick,flexdotted] (0.2,0) to ++(0.6,0);
}
+
\diagram{
  %top
  \interaction{2}{1g}{(0,+0.5)}{ddot}{sawtooth};
  %bottom
  \interaction{2}{2g}{(0,-0.5)}{ddot}{sawtooth};
  %lines
  \draw[->-=0.4,bend left ] (2g1) to (1g1);
  \draw[-<-=0.6,bend right] (2g1) to (1g1);
  \draw[->-=0.4,bend left ] (2g2) to (1g2);
  \draw[-<-=0.6,bend right] (2g2) to (1g2);
  \draw[thick,flexdotted] (-0.3,0) to ++(1.6,0);
}
\end{align}
\end{ex}

\begin{ex}
The expansion for $\Y\ord{2}$ can be evaluated using \cref{cor:wicks-theorem-for-pt}.
Assuming Brillouin's theorem for simplicity,
\begin{align}
\nonumber
  \Y\ord{2}
=
  R_0
  V_\mr{c}
  R_0
  V_\mr{c}
  \kt{\F}
=&\
\nonumber
\diagram[bottom]{
  \interaction{2}{1g}{(0,-0.5)}{ddot}{sawtooth};
  \node[ddot] (2g2) at (1,0) {};
  \draw[-<-] (1g1) to ++(-0.25,+1);
  \draw[->-=0.25,->-=0.75] (1g1) to node[midway,ddot] (2g1) {} ++(+0.25,+1);
  \draw[sawtooth] (2g1) to (2g2);
  \draw[->-,bend left =45] (1g2) to (2g2);
  \draw[-<-,bend right=45] (1g2) to (2g2);
  \draw[thick,flexdotted] (-0.3,-0.27) to ++(1.6,0);
  \draw[thick,flexdotted] (-0.3,+0.35) to ++(0.7,0);
  \draw[opacity=0] (0.5,-0.5) circle (0.125cm);
}
+
\diagram[bottom]{
  \interaction{2}{1g}{(0,-0.5)}{ddot}{sawtooth};
  \node[ddot] (2g2) at (1,0) {};
  \draw[->-] (1g1) to ++(-0.25,+1);
  \draw[-<-=0.25,-<-=0.75] (1g1) to node[midway,ddot] (2g1) {} ++(+0.25,+1);
  \draw[sawtooth] (2g1) to (2g2);
  \draw[->-,bend left =45] (1g2) to (2g2);
  \draw[-<-,bend right=45] (1g2) to (2g2);
  \draw[thick,flexdotted] (-0.3,-0.27) to ++(1.6,0);
  \draw[thick,flexdotted] (-0.3,+0.35) to ++(0.7,0);
  \draw[opacity=0] (0.5,-0.5) circle (0.125cm);
}
+
\diagram[bottom]{
  \interaction{2}{t}{(0,-0.5)}{ddot}{sawtooth};
  \draw[->-=0.25,->-=0.75] (t1) to node[midway,ddot] (g1) {}
    ++(-0.25,1);
  \draw[-<-=0.7] (t1) to ++(+0.25,1);
  \draw[->-=0.25,->-=0.75] (t2) to node[midway,ddot] (g2) {}
    ++(-0.25,1);
  \draw[-<-=0.7] (t2) to ++(+0.25,1);
  \draw[sawtooth] (g1)--(g2);
  \draw[thick,flexdotted] (-0.3,-0.27) to ++(1.6,0);
  \draw[thick,flexdotted] (-0.4,+0.35) to ++(1.8,0);
  \draw[opacity=0] (0.5,-0.5) circle (0.125cm);
}
+
\diagram[bottom]{
  \interaction{2}{t}{(0,-0.5)}{ddot}{sawtooth};
  \draw[-<-=0.25,-<-=0.75] (t1) to node[midway,ddot] (g1) {}
    ++(-0.25,1);
  \draw[->-=0.7] (t1) to ++(+0.25,1);
  \draw[-<-=0.25,-<-=0.75] (t2) to node[midway,ddot] (g2) {}
    ++(-0.25,1);
  \draw[->-=0.7] (t2) to ++(+0.25,1);
  \draw[sawtooth] (g1)--(g2);
  \draw[thick,flexdotted] (-0.3,-0.27) to ++(1.6,0);
  \draw[thick,flexdotted] (-0.4,+0.35) to ++(1.8,0);
  \draw[opacity=0] (0.5,-0.5) circle (0.125cm);
}
+
\diagram[bottom]{
  \interaction{2}{t}{(0,-0.5)}{ddot}{sawtooth};
  \interaction{2}{g}{(1,+0.0)}{ddot}{sawtooth};
  \draw[->-] (t1) to ++(-0.25,1);
  \draw[-<-] (t1) to ++(+0.25,1);
  \draw[->-,bend left] (t2) to (g1);
  \draw[-<-,bend right] (t2) to (g1);
  \draw[->-] (g2) to ++(-0.25,0.5);
  \draw[-<-] (g2) to ++(+0.25,0.5);
  \draw[thick,flexdotted] (-0.3,-0.27) to ++(1.6,0);
  \draw[thick,flexdotted] (-0.4,+0.35) to ++(2.8,0);
  \draw[opacity=0] (0.5,-0.5) circle (0.125cm);
}
\\&\
\label{eq:second-order-wavefunction-graphical}
+
\diagram[bottom]{
  \interaction{2}{1g}{(0,-0.5)}{ddot}{sawtooth};
  \interaction{2}{2g}{(1.125,0)}{ddot}{sawtooth};
  \draw[-<-] (1g1) to ++(-0.25,1);
  \draw[->-] (1g1) to ++(+0.25,1);
  \draw[-<-] (1g2) to ++(-0.25,1);
  \draw[->-=0.25,->-=0.75] (1g2) to ++(+0.25,1);
  \draw[-<-] (2g2) to ++(-0.25,0.5);
  \draw[->-] (2g2) to ++(+0.25,0.5);
  \draw[thick,flexdotted] (-0.4,-0.27) to ++(2.9,0);
  \draw[thick,flexdotted] (-0.4,+0.35) to ++(2.9,0);
  \draw[opacity=0] (0.5,-0.5) circle (0.125cm);
}
+
\diagram[bottom]{
  \interaction{2}{1g}{(0,-0.5)}{ddot}{sawtooth};
  \interaction{2}{2g}{(1.125,0)}{ddot}{sawtooth};
  \draw[->-] (1g1) to ++(-0.25,1);
  \draw[-<-] (1g1) to ++(+0.25,1);
  \draw[->-] (1g2) to ++(-0.25,1);
  \draw[-<-=0.25,-<-=0.75] (1g2) to ++(+0.25,1);
  \draw[->-] (2g2) to ++(-0.25,0.5);
  \draw[-<-] (2g2) to ++(+0.25,0.5);
  \draw[thick,flexdotted] (-0.4,-0.27) to ++(2.9,0);
  \draw[thick,flexdotted] (-0.4,+0.35) to ++(2.9,0);
  \draw[opacity=0] (0.5,-0.5) circle (0.125cm);
}
+
\diagram[bottom]{
  \interaction{2}{1g}{(0,-0.5)}{ddot}{sawtooth};
  \interaction{2}{2g}{(2,0)}{ddot}{sawtooth};
  \draw[-<-] (1g1) to ++(-0.25,1);
  \draw[->-] (1g1) to ++(+0.25,1);
  \draw[-<-] (1g2) to ++(-0.25,1);
  \draw[->-] (1g2) to ++(+0.25,1);
  \draw[-<-] (2g1) to ++(-0.25,0.5);
  \draw[->-] (2g1) to ++(+0.25,0.5);
  \draw[-<-] (2g2) to ++(-0.25,0.5);
  \draw[->-] (2g2) to ++(+0.25,0.5);
  \draw[thick,flexdotted] (-0.3,-0.27) to ++(1.6,0);
  \draw[thick,flexdotted] (-0.4,+0.35) to ++(3.9,0);
  \draw[opacity=0] (0.5,-0.5) circle (0.125cm);
}
\\
=&\
  \tfr{1}{2}
  \sum_{\substack{abc\\ij}}
  \F_i^a\,
  \fr{
    \ol{g}_{aj}^{bc}
    \ol{g}_{bc}^{ij}
  }{
    \mc{E}_a^i
    \mc{E}_{bc}^{ij}
  }
-
  \tfr{1}{2}
  \sum_{\substack{ab\\ijk}}
  \F_i^a
  \fr{
    \ol{g}_{jk}^{ib}
    \ol{g}_{ab}^{jk}
  }{
    \mc{E}_a^i
    \mc{E}_{ab}^{jk}
  }
+
  \tfr{1}{2^3}
  \sum_{\substack{abcd\\ij}}
  \F_{ij}^{ab}
  \fr{
    \ol{g}_{ab}^{cd}
    \ol{g}_{cd}^{ij}
  }{
    \mc{E}_{ab}^{ij}
    \mc{E}_{cd}^{ij}
  }
+
  \tfr{1}{2^3}
  \sum_{\substack{ab\\ijkl}}
  \F_{ij}^{ab}
  \fr{
    \ol{g}_{kl}^{ij}
    \ol{g}_{ab}^{kl}
  }{
    \mc{E}_{ab}^{ij}
    \mc{E}_{ab}^{kl}
  }
\nonumber
\\&\
+
  \sum_{\substack{abc\\ijk}}
  \F_{ij}^{ab}
  \fr{
    \ol{g}_{ac}^{ik}
    \ol{g}_{kb}^{cj}
  }{
    \mc{E}_{ab}^{ij}
    \mc{E}_{ac}^{ik}
  }
+
  \tfr{1}{2^2}
  \sum_{\substack{abcd\\ijk}}
  \F_{ijk}^{abc}
  \fr{
    \ol{g}_{ad}^{ij}
    \ol{g}_{bc}^{dk}
  }{
    \mc{E}_{abc}^{ijk}
    \mc{E}_{ad}^{ij}
  }
-
  \tfr{1}{2^2}
  \sum_{\substack{abc\\ijkl}}
  \F_{ijk}^{abc}
  \fr{
    \ol{g}_{ab}^{il}
    \ol{g}_{lc}^{jk}
  }{
    \mc{E}_{abc}^{ijk}
    \mc{E}_{ab}^{il}
  }
+
  \tfr{1}{2^4}
  \sum_{\substack{abcd\\ijkl}}
  \F_{ijkl}^{abcd}
  \fr{
    \ol{g}_{ab}^{ij}
    \ol{g}_{cd}^{kl}
  }{
    \mc{E}_{abcd}^{ijkl}
    \mc{E}_{ab}^{ij}
  }
\end{align}
where the operators in the final diagram do not form an equivalent pair because they pass through different resolvent lines.
The third-order contribution to the correlation energy can be evaluated as the complete contractions of $V_\mr{c}R_0V_\mr{c}R_0V_\mr{c}$
\begin{align}
  E_\mr{c}\ord{3}
\,{=}\,
\diagram{
  %top
  \interaction{2}{1g}{(0,+0.5)}{ddot}{sawtooth};
  %bottom
  \interaction{2}{2g}{(0,-0.5)}{ddot}{sawtooth};
  \draw[->-=0.25,->-=0.75, bend left]
    (2g1)
    to
      node[midway,ddot] (g1) {}
    (1g1);
  \draw[-<-=0.65,bend right] (2g1) to (1g1);
  \draw[->-=0.25,->-=0.75, bend left]
    (2g2)
    to
      node[midway,ddot] (g2) {}
    (1g2);
  \draw[-<-=0.65,bend right] (2g2) to (1g2);
  \draw[sawtooth] (g1)--(g2);
  \draw[thick,flexdotted] (-0.3,-0.3) to ++(1.6,0);
  \draw[thick,flexdotted] (-0.3,+0.3) to ++(1.6,0);
  \draw[opacity=0] (0.5,-0.5) circle (0.125cm);
}
{+}
\diagram{
  %top
  \interaction{2}{1g}{(0,+0.5)}{ddot}{sawtooth};
  %bottom
  \interaction{2}{2g}{(0,-0.5)}{ddot}{sawtooth};
  \draw[-<-=0.25,-<-=0.75, bend left]
    (2g1)
    to
      node[midway,ddot] (g1) {}
    (1g1);
  \draw[->-=0.65,bend right] (2g1) to (1g1);
  \draw[-<-=0.25,-<-=0.75, bend left]
    (2g2)
    to
      node[midway,ddot] (g2) {}
    (1g2);
  \draw[->-=0.65,bend right] (2g2) to (1g2);
  \draw[sawtooth] (g1)--(g2);
  \draw[thick,flexdotted] (-0.3,-0.3) to ++(1.6,0);
  \draw[thick,flexdotted] (-0.3,+0.3) to ++(1.6,0);
  \draw[opacity=0] (0.5,-0.5) circle (0.125cm);
}
{+}
\diagram{
  %top
  \draw[sawtooth]
    (0,+0.5)
      node[ddot] (1g1) {}
    to
    ++(2,0)
      node[ddot] (1g2) {};
  %middle
  \interaction{2}{g}{(1,+0.0)}{ddot}{sawtooth};
  %bottom
  \interaction{2}{2g}{(0,-0.5)}{ddot}{sawtooth};
  \draw[->-,bend left ] (2g1) to (1g1);
  \draw[-<-,bend right] (2g1) to (1g1);
  \draw[->-,bend left ] (2g2) to (g1);
  \draw[-<-,bend right] (2g2) to (g1);
  \draw[->-,bend left ] (g2) to (1g2);
  \draw[-<-,bend right] (g2) to (1g2);
  \draw[thick,flexdotted] (-0.3,-0.3) to ++(1.6,0);
  \draw[thick,flexdotted] (-0.3,+0.3) to ++(2.6,0);
  \draw[opacity=0] (0.5,-0.5) circle (0.125cm);
}
{=}
  \tfr{1}{2^3}
  \sum_{\substack{abcd\\ij}}
  \fr{
    \ol{g}_{ij}^{ab}
    \ol{g}_{ab}^{cd}
    \ol{g}_{cd}^{ij}
  }{
    \mc{E}_{ab}^{ij}
    \mc{E}_{cd}^{ij}
  }
{+}
  \tfr{1}{2^3}
  \sum_{\substack{ab\\ijkl}}
  \fr{
    \ol{g}_{ij}^{ab}
    \ol{g}_{kl}^{ij}
    \ol{g}_{ab}^{kl}
  }{
    \mc{E}_{ab}^{ij}
    \mc{E}_{ab}^{kl}
  }
{+}
  \sum_{\substack{abc\\ijk}}
  \fr{
    \ol{g}_{ij}^{ab}
    \ol{g}_{ac}^{ik}
    \ol{g}_{kb}^{cj}
  }{
    \mc{E}_{ab}^{ij}
    \mc{E}_{ac}^{ik}
  }
\end{align}
which is equivalent to contracting the doubles contributions to $\Y\ord{2}$ with $\tfr{1}{4}\ol{g}_{ij}^{ab}\tl{a}^{ij}_{ab}$.
Note that $E_\mr{c}\ord{m+1}$ always only depends on the doubles contribution to $\Y\ord{m}$, but that the doubles coefficients themselves may involve triples, quadruples and higher contributions from wavefunction components of order less than $m$.
\end{ex}

\begin{ex}
Using
${}\ord{m}c_{ab\cd}^{ij\cd}=\ip{\F_{ij\cd}^{ab\cd}|\Y\ord{m}}$, the second order CI coefficients can be determined from eq~\ref{eq:second-order-wavefunction-graphical} by contracting a bare excitation operator with the top of each diagram.
Interpreting these graphs gives the following.
\begin{align*}
  {}\ord{2}c_a^i
&=
  \tfr{1}{2}
  \sum_{\substack{bc\\j}}
  \fr{
    \ol{g}_{aj}^{bc}
    \ol{g}_{bc}^{ij}
  }{
    \mc{E}_a^i
    \mc{E}_{bc}^{ij}
  }
-
  \tfr{1}{2}
  \sum_{\substack{b\\jk}}
  \fr{
    \ol{g}_{jk}^{ib}
    \ol{g}_{ab}^{jk}
  }{
    \mc{E}_a^i
    \mc{E}_{ab}^{jk}
  }
\\
  {}\ord{2}c_{ab}^{ij}
&=
  \tfr{1}{2}
  \sum_{\substack{cd}}
  \fr{
    \ol{g}_{ab}^{cd}
    \ol{g}_{cd}^{ij}
  }{
    \mc{E}_{ab}^{ij}
    \mc{E}_{cd}^{ij}
  }
+
  \tfr{1}{2}
  \sum_{\substack{kl}}
  \fr{
    \ol{g}_{kl}^{ij}
    \ol{g}_{ab}^{kl}
  }{
    \mc{E}_{ab}^{ij}
    \mc{E}_{ab}^{kl}
  }
+
  \op{P}_{(a/b)}^{(i/j)}
  \sum_{\substack{c\\k}}
  \fr{
    \ol{g}_{ac}^{ik}
    \ol{g}_{kb}^{cj}
  }{
    \mc{E}_{ab}^{ij}
    \mc{E}_{ac}^{ik}
  }
\\
  {}\ord{2}c_{abc}^{ijk}
&=
  \op{P}_{(a/bc)}^{(ij/k)}
  \sum_{\substack{d}}
  \fr{
    \ol{g}_{ad}^{ij}
    \ol{g}_{bc}^{dk}
  }{
    \mc{E}_{abc}^{ijk}
    \mc{E}_{ad}^{ij}
  }
-
  \op{P}_{(ab/c)}^{(i/jk)}
  \sum_{\substack{l}}
  \fr{
    \ol{g}_{ab}^{il}
    \ol{g}_{lc}^{jk}
  }{
    \mc{E}_{abc}^{ijk}
    \mc{E}_{ab}^{il}
  }
\\
  {}\ord{2}c_{abcd}^{ijkl}
&=
  \op{P}_{(ab/cd)}^{(ij/kl)}
  \fr{
    \ol{g}_{ab}^{ij}
    \ol{g}_{cd}^{kl}
  }{
    \mc{E}_{abcd}^{ijkl}
    \mc{E}_{ab}^{ij}
  }
\end{align*}
Note that the second order quadruples coefficient is disconnected.
Prop.~\ref{prop:second-order-c4} shows that the second-order quadruples operator is actually a simple product of first-order doubles operators.
This fact was an early motivation for coupled-pair many-electron theory,\footnote{This is the original name for coupled-cluster doubles.} since it justifies approximating
$
  \Y_\mr{CIDQ}
=
  (1+C_2+C_4)\F
$
by
$
  \Y_\mr{CPMET}
=
  (1 + C_2 + \tfr{1}{2}C_2^2)\F
$.
\end{ex}

\begin{prop}
\label{prop:second-order-c4}
\thmstatement{
$
  {}\ord{2} C_4
=
  \tfr{1}{2}
  {}\ord{1} C_2^2
$
}
\thmproof{
This follows from rearranging the resolvent denominator.
\begin{align*}
  \fr{1}{\mc{E}_{abcd}^{ijkl}\mc{E}_{ab}^{ij}}
+
  \fr{1}{\mc{E}_{abcd}^{ijkl}\mc{E}_{cd}^{kl}}
=
  \fr{
    \mc{E}_{cd}^{kl} + \mc{E}_{ab}^{ij}
  }{
    \mc{E}_{abcd}^{ijkl}
    \mc{E}_{ab}^{ij}
    \mc{E}_{cd}^{kl}
  }
=
  \fr{
    1
  }{
    \mc{E}_{ab}^{ij}
    \mc{E}_{cd}^{kl}
  }
\implies
  {}\ord{2}C_4
=
  \pr{\tfr{1}{2}}^4
  \sum_{\substack{abcd\\ijkl}}
  \tl{a}^{abcd}_{ijkl}
  \fr{
    \ol{g}_{ab}^{ij}
    \ol{g}_{cd}^{kl}
  }{
    \mc{E}_{abcd}^{ijkl}
    \mc{E}_{ab}^{ij}
  }
=
  \tfr{1}{2}\cdot
  \pr{\tfr{1}{2}}^4
  \sum_{\substack{abcd\\ijkl}}
  \tl{a}^{abcd}_{ijkl}
  \fr{
    \ol{g}_{ab}^{ij}
    \ol{g}_{cd}^{kl}
  }{
    \mc{E}_{ab}^{ij}
    \mc{E}_{cd}^{kl}
  }
=
  \tfr{1}{2}
  {}\ord{1}C_2^2
\end{align*}
}
\end{prop}

\begin{lem}\label{lem:energy-substitution}
\thmtitle{The Energy Substitution Lemma}
\thmstatement{
  $\Y\ord{m}$ equals the sum of a ``principal term''
  $(R_0V_\mr{c})^m\F$
  plus all possible substitutions of adjacent factors $(R_0V_\mr{c})^{r_i}$ in the principal term by $R_0E_\mr{c}\ord{r_i}$.
  Each term in the sum is weighted by a sign factor $(-)^k$, where $k$ is the number of substitutions.
}
\thmproof{
  See \cref{app:linked-diagram-theorem}.
}
\end{lem}

\begin{ex}
Lemma~\ref{lem:energy-substitution} is consistent with equation~\ref{eq:first-and-second-order-principal-terms} because substitution of the rightmost factors in the principal term leaves a resolvent acting on the reference determinant and because the first-order energy contribution equals zero.
The first non-trivial examples of the energy substitution lemma begin at third order.
\begin{align}
\label{eq:energy-substitution-psi-3}
  \Y\ord{3}
=&\
  R_0V_\mr{c}R_0V_\mr{c}R_0V_\mr{c}\F
-
  R_0E_\mr{c}\ord{2}R_0V_\mr{c}\F
\\
\label{eq:energy-substitution-psi-4}
  \Y\ord{4}
=&\
  R_0V_\mr{c}R_0V_\mr{c}R_0V_\mr{c}R_0V_\mr{c}\F
-
  R_0E_\mr{c}\ord{2}R_0V_\mr{c}R_0V_\mr{c}\F
-
  R_0V_\mr{c}R_0E_\mr{c}\ord{2}R_0V_\mr{c}\F
-
  R_0E_\mr{c}\ord{3}R_0V_\mr{c}\F
\\
\nonumber
  \Y\ord{5}
=&\
  R_0V_\mr{c}R_0V_\mr{c}R_0V_\mr{c}R_0V_\mr{c}R_0V_\mr{c}\F
-
  R_0E_\mr{c}\ord{2}R_0V_\mr{c}R_0V_\mr{c}R_0V_\mr{c}\F
-
  R_0V_\mr{c}R_0E_\mr{c}\ord{2}R_0V_\mr{c}R_0V_\mr{c}\F
\\&\
\nonumber
-
  R_0V_\mr{c}R_0V_\mr{c}R_0E_\mr{c}\ord{2}R_0V_\mr{c}\F
+
  R_0E_\mr{c}\ord{2}R_0E_\mr{c}\ord{2}R_0V_\mr{c}\F
-
  R_0E_\mr{c}\ord{3}R_0V_\mr{c}R_0V_\mr{c}\F
\\&\
\label{eq:energy-substitution-psi-5}
-
  R_0V_\mr{c}R_0E_\mr{c}\ord{3}R_0V_\mr{c}\F
-
  R_0E_\mr{c}\ord{4}R_0V_\mr{c}\F
\end{align}
\end{ex}

\begin{thm}\label{thm:bracketing-theorem}
\thmtitle{The Bracketing Theorem}
\thmstatement{
  $\Y\ord{m}$ equals the principal term plus all possible insertions of nested brackets into the principal term.
  Each term in the sum is weighted by $(-)^k$ where $k$ is the total number of brackets.\footnote{
  The ``brackets'' here are reference expectation values: $\ip{W}\equiv\ip{\F|W|\F}$.
}
}
\thmproof{
  See \cref{app:linked-diagram-theorem}.
}
\end{thm}

\begin{ex}
Equations~\ref{eq:energy-substitution-psi-3} and~\ref{eq:energy-substitution-psi-4} are clearly consistent with \cref{thm:bracketing-theorem}, since $E_\mr{c}\ord{2}{=}\,\ip{V_\mr{c}R_0V_\mr{c}}$ and $E_\mr{c}\ord{3}{=}\,\ip{V_\mr{c}R_0V_\mr{c}R_0V_\mr{c}}$.
\begin{align}
\label{eq:bracketing-psi-3}
  \Y\ord{3}
=&\
  R_0V_\mr{c}R_0V_\mr{c}R_0V_\mr{c}\F
-
  R_0\ip{V_\mr{c}R_0V_\mr{c}}R_0V_\mr{c}\F
\\
  \Y\ord{4}
=&\
  R_0V_\mr{c}R_0V_\mr{c}R_0V_\mr{c}R_0V_\mr{c}\F
-
  R_0\ip{V_\mr{c}R_0V_\mr{c}}R_0V_\mr{c}R_0V_\mr{c}\F
-
  R_0V_\mr{c}R_0\ip{V_\mr{c}R_0V_\mr{c}}R_0V_\mr{c}\F
-
  R_0\ip{V_\mr{c}R_0V_\mr{c}R_0V_\mr{c}}R_0V_\mr{c}\F
\\
\intertext{
The first non-vanishing terms with nested brackets appear at fifth-order
}
\nonumber
  \Y\ord{5}
=&\
  R_0V_\mr{c}R_0V_\mr{c}R_0V_\mr{c}R_0V_\mr{c}R_0V_\mr{c}\F
-
  R_0\ip{V_\mr{c}R_0V_\mr{c}}R_0V_\mr{c}R_0V_\mr{c}R_0V_\mr{c}\F
-
  R_0V_\mr{c}R_0\ip{V_\mr{c}R_0V_\mr{c}}R_0V_\mr{c}R_0V_\mr{c}\F
\\&\
\nonumber
-
  R_0V_\mr{c}R_0V_\mr{c}R_0\ip{V_\mr{c}R_0V_\mr{c}}R_0V_\mr{c}\F
+
  R_0\ip{V_\mr{c}R_0V_\mr{c}}R_0\ip{V_\mr{c}R_0V_\mr{c}}R_0V_\mr{c}\F
-
  R_0\ip{V_\mr{c}R_0V_\mr{c}R_0V_\mr{c}}R_0V_\mr{c}R_0V_\mr{c}\F
\\&\
-
  R_0V_\mr{c}R_0\ip{V_\mr{c}R_0V_\mr{c}R_0V_\mr{c}}R_0V_\mr{c}\F
-
  R_0\ip{V_\mr{c}R_0V_\mr{c}R_0V_\mr{c}R_0V_\mr{c}}R_0V_\mr{c}\F
+
  R_0\ip{V_\mr{c}R_0\ip{V_\mr{c}R_0V_\mr{c}}R_0V_\mr{c}}R_0V_\mr{c}\F
\end{align}
which follows from substituting equation~\ref{eq:bracketing-psi-3} into $E\ord{4}=\ip{\F|V_\mr{c}|\Y\ord{3}}$ in the energy substitution expansion of $\Y\ord{5}$.
\end{ex}

\begin{ex}
Assuming Brillouin's theorem, the simplest non-vanishing term with an inserted bracket appears in $\Y\ord{3}$.
\begin{align*}
  R_0
  \ip{V_\mr{c}R_0V_\mr{c}}
  R_0
  V_\mr{c}
  \F
=
\diagram{
  \interaction{2}{1g}{(0,-0.5)}{ddot}{sawtooth};
  \draw[->-] (1g1) to ++(-0.25,1);
  \draw[-<-] (1g1) to ++(+0.25,1);
  \draw[->-] (1g2) to ++(-0.25,1);
  \draw[-<-] (1g2) to ++(+0.25,1);
  \interaction{2}{2g}{(2,-0.5)}{ddot}{sawtooth};
  \interaction{2}{3g}{(2,+0.5)}{ddot}{sawtooth};
  \draw[->-=0.4,bend left ] (2g1) to (3g1);
  \draw[-<-=0.6,bend right] (2g1) to (3g1);
  \draw[->-=0.4,bend left ] (2g2) to (3g2);
  \draw[-<-=0.6,bend right] (2g2) to (3g2);
  \draw[thick,flexdotted] (-0.3,-0.25) to ++(1.6,0);
  \draw[thick,flexdotted] (-0.4,+0.35) to ++(1.8,0);
  \draw[opacity=0] (0.5,-0.5) circle (0.125cm);
  \draw[thick,flexdotted] (1.7,0) to ++(1.6,0);
  \padborder{5pt};
  \draw[double, thick] (current bounding box.south west)--(current bounding box.south east);
  \node at (0.5,-1) {remainder};
  \node at (2.5,-1) {insertion};
  \node[inner sep=0pt] at (-1.5,0) {
    \begin{tabular}{c}
    level of the\\insertion
    \end{tabular}
  };
  \draw[->] (-2.5,0) to (-0.3,0);
  \node[inner sep=0pt] at (+4,+0.4) {$1\rst$\,level};
  \node[inner sep=0pt] at (+4,-0.3) {$0\eth$\,level};
}
\end{align*}
\end{ex}

\begin{prop}
\thmtitle{Wigner's (2n+1) rule}
\end{prop}

\newpage
\appendix
\section{Proof of the Linked-Diagram Theorem}\label{app:linked-diagram-theorem}

\begin{ntt}\label{ntt:operator-combinations}
Let
``$Y^m$ choose $Z^k$'', denoted ${}^mC_k(Y:Z)$,
refer to a sum over the $m$ choose $k$ permutations of $Y^{m-k}Z^k$,\,\footnote{For example,
$
  {}^4C_2(Y:Z)
=
  Y^2Z^2
+
  YZYZ
+
  YZ^2Y
+
  ZY^2Z
+
  ZYZY
+
  Z^2Y^2
$.
}
where $Y$ and $Z$ are operators that may or may not commute.\,\footnote{
  If they do commute, then ${}^mC_k(Y:Z)={n\choose k}Y^{m-k}Z^k$.
}
This defines a generalization of the binomial theorem.
\begin{align}
\label{eq:generalized-binomial-theorem}
  (
    Y
  +
    Z
  )^m
=
  \sum_{k=0}^m
  {}^mC_k(Y:Z)
\end{align}
Furthermore, let
$
  {}^mC(Y:Z_1,\ld,Z_k)
$
be a sum over permutations of
$
  Y^{m-k}
  Z_1\cd Z_k
$ that preserve the ordering of the $Z_i$'s.\,\footnote{
  For example,
$
  {}^4C(Y:Z_1,Z_2)
=
  Y^2Z_1Z_2
+
  YZ_1YZ_2
+
  YZ_1Z_2Y
+
  Z_1Y^2Z_2
+
  Z_1YZ_2Y
+
  Z_1Z_2Y^2
$.
}
When all of the $Z_i$'s equal $Z$, we can write
$
  {}^mC(Y:Z_1,\ld,Z_k)
=
  {}^mC_k(Y:Z)
$.
\end{ntt}

\begin{prop}
\label{prop:wavefunction-infinite-recursion}
\thmstatement{
$\ds{
  \Y(\la)
=
  \sum_{m=0}^\infty
  \pr{
    R_0
    (\la V_\mr{c} - E(\la))
  }^m
  \F
}$
}
\thmproof{
  This follows by infinite recursion of equation~\ref{eq:lambda-dependent-recursive-series} with the assumption
  $\ds{
  \lim_{m\rightarrow\infty}
  \pr{
    R_0
    (\la V_\mr{c} - E(\la))
  }^m
  \Y(\la)
  =
    0
  }$.
}
\end{prop}

\begin{dfn}\label{dfn:integer-compositions}
\thmtitle{Integer compositions}
The \textit{compositions} of an integer $m$ are the ways of writing $m$ as a sum of positive integers.
The full set of integer compositions of $m$ is given by
$
  \mc{C}(m)
=
  \mc{C}_1(m)
  \cup
  \mc{C}_2(m)
  \cup
  \cd
  \cup
  \mc{C}_m(m)
$
where
$
  \mc{C}_k(m)
=
  \{
    (r_1,\ld,r_k)\in\mb{N}_0^k
  \,|\,
    r_1+\cd+r_k
  =
    m
  \}
$
are the integer compositions of $m$ into $k$ parts.
\end{dfn}

\begin{lem}
\label{lem:energy-substitution-proof}
\thmtitle{The Energy Substitution Lemma}
\thmstatement{
  $\Y\ord{m}$ equals the sum of a ``principal term''
  $(R_0V_\mr{c})^m\F$
  plus all possible substitutions of adjacent factors $(R_0V_\mr{c})^{r_i}$ in the principal term by $R_0E_\mr{c}\ord{r_i}$.
  Each term in the sum is weighted by a sign factor $(-)^k$, where $k$ is the number of substitutions.
}\vspace{5pt}
\thmproof{
Using equation~\ref{eq:generalized-binomial-theorem} and a double sum identity\footnote{
  Reverse double-sum reduction:
  $\ds{
    \sum_{m=0}^\infty
    \sum_{k=0}^m
    t_{m-k,k}
  =
    \sum_{k'=0}^\infty
    \sum_{k=0}^\infty
    t_{k',k}
  }$.
  See
  \url{http://functions.wolfram.com/GeneralIdentities/12/}.
} in the infinite recursion formula for $\Y(\la)$ gives the following.
{\footnotesize
\begin{align*}
  \Y(\la)
=
  \sum_{m=0}^\infty
  (
    R_0
    (
      \la V_\mr{c}
    -
      E(\la)
    )
  )^m
  \F
=
  \sum_{m=0}^\infty
  \sum_{k=0}^m
  \la^{m-k}
  (-)^k\,\,
  {}^mC_k
  (
    R_0V_\mr{c}:
    R_0E(\la)
  )
  \F
=
  \sum_{k'=0}^\infty
  \sum_{k=0}^\infty
  \la^{k'}
  (-)^{k}\,\,
  {}^{k'+k}\hspace{-1pt}C_k
  (
    R_0V_\mr{c}:
    R_0E(\la)
  )
  \F
\end{align*}}%
The $k'=0$ term has no operators separating $\F$ from the resolvent and vanishes.
Taylor expansion of the energies gives
{\footnotesize
\begin{align*}
  \Y(\la)
=&\
  \sum_{k=0}^\infty
  \sum_{k'=1}^\infty
  {\sum_{p_1=1}^\infty}
  \cd
  {\sum_{p_k=1}^\infty}
  \la^{k' + p_1 + \cd + p_k}
  (-)^{k}\,\,
  {}^{k'+k}\hspace{-1pt}C
  (
    R_0V_\mr{c}:
    R_0E_\mr{c}\ord{p_1},\ld,
    R_0E_\mr{c}\ord{p_k}
  )
  \F
\\
=&\
  \sum_{m=1}^\infty
  \sum_{k=0}^{m-1}
  \sum_{(r_1,\ld,r_{k+1})}^{\mc{C}_{k+1}(m)}
  \la^m
  (-)^k\,\,
  {}^{k+r_1}\hspace{-1pt}C
  (
    R_0V_\mr{c}:
    R_0E_\mr{c}\ord{r_2},\ld,
    R_0E_\mr{c}\ord{r_{k+1}}
  )
  \F
\end{align*}}%
where we have grouped powers of $\la$ using a multi-sum reduction.
Writing the inner sums as a sum over $\mc{C}(m)$ we find
\begin{align}
  \Y\ord{m}
=
  \left.
  \fr{1}{m!}
  \pd{^m\Y(\la)}{\la^m}
  \right|_{\la=0}
=
  \sum_{(r_1,\ld,r_{k+1})}^{\mc{C}(m)}
  (-)^k\,\,
  {}^{k+r_1}\hspace{-1pt}C
  (
    R_0V_\mr{c}:
    R_0E_\mr{c}\ord{r_2},\ld,
    R_0E_\mr{c}\ord{r_{k+1}}
  )
  \F
\end{align}
which, given \cref{ntt:operator-combinations} and definition~\ref{dfn:integer-compositions}, is an algebraic statement of the proposition, completing the proof.
}
\end{lem}

\begin{thm}
\thmtitle{The Bracketing Theorem}
\thmstatement{
  $\Y\ord{m}$ equals the principal term plus all possible insertions of nested brackets into the principal term.
  Each term in the sum is weighted by $(-)^k$ where $k$ is the total number of brackets.
}\vspace{5pt}
\thmproof{
  The proposition holds for $m=1$ because
  $\Y\ord{1}=R_0V_\mr{c}\F$ and there are no possible bracketings.
  Assume it holds for $m-1$.
  Then by the energy substitution lemma it also holds for $m$ because $E_\mr{c}\ord{r_i}$
  equals
  $\ip{\F|V_\mr{c}|\Y\ord{r_i}}$
  which, by our inductive assumption, equals
  $\ip{V_\mr{c}(R_0V_\mr{c})^{r_i}}$
  plus all nested bracketings weighted by appropriate sign factors.
}
\end{thm}

\end{document}


\chapter{De-Entanglement by additional Divergences - Case study with Emphatic Speech Synthesis}

section{Problem Motivation and Introduction}
Humans exhibit both coarse as well as fine grained explicit control over how they speak an utterance. This targeted control on speech - often manifested in the form of prosodic constructions - allows us to effectively convey our intent in a conversation. Examples of controlled speech generation include simple prosodic manipulations such as implying specific meaning, highlighting or expressing interest in something as well as various communication strategies such as contradiction, contrast, complaints or grudging admiration\citep{nigel_ward_prosodic_patterns}. Further, such manipulation in prosody has been shown effective in
applications such as  Infant Behavior Programs \citep{parental_prosody_changes_mediate_infant_language_production}, improving language acquisition\citep{prosody_functionwords_acquisition} and promoting rapport\citep{rapport_dialogsystems}. It seems natural to employ generative models of speech\citep{tacotron_original,deepvoice2,clarinet,parrotron} to assist in such scenarios \citep{does_tts_help_comprehension}. However, although there has been tremendous progress in the neural generative models for speech in the context of vocoder fidelity\citep{waveglow,wavenet_original}, the notion of controllability in such models is not yet fully evolved. While there have been works towards models aimed at controlling prosody\citep{ tacotron_hierarchical,tacotron_stylefacoruncovering}, the exerted control is still global or coarse grained in terms of styles of speech\citep{tacotron_prosodycontrol,tacotron_styletokens}, etc. In this work, we propose an approach that allows both global as well as local control over the prosodic variation in the generated speech.

Typically TTS is formulated as a conditional generative modeling problem. In our approach, we propose to instead formulate it as a conditional variational auto-encoder and incorporate automatically derivable information from speech data into the model architecture. This is motivated by the understanding that the utterances themselves do not always contain all the information needed to comprehend the appropriate prosody information. The missing information is either part of background knowledge about the world - implicit to humans but not annotated in the data - or is provided by accompanying context of the utterance. Formulating the task using variational inference allows us to efficiently capture the distribution of prosody thereby avoiding the averaging effect observed in a typical TTS system due to prosody marginalization. To illustrate this, consider an example sentence: `\textit{You do not have a pet shark}'. Most prosodic constructions for this sentence involve sarcasm since it is not commonplace to have sharks as pets - world knowledge. Similarly consider the sentence: `\textit{I dont want to be a nun}'. The linguistic unit subject to realization of prosodic stress in this sentence depends on the context information. Finally, consider the example of a TTS system deployed in a screenreader to assist visually impaired students comprehend math equations. Human voice talent would almost certainly place appropriate prosodic cues that help in comprehension of $x^{(y+z)}$  as opposed to ($x^y + z$). Our formulation allows the model to leverage prosodic information available from the speech signal and capture prosodic distribution.

To accomplish local as well as global prosody control, we incorporate inductive biases into the model architecture in the form of fundamental frequency($F_0$). Specifically, we quantize $F_0$ into multiple bins and constrain the latent space to disentangle these quantized values from acoustics at the level of phonemes. Our model is explained in detail in section \ref{proposed_approach}. During inference, the prosody distribution can be utilized to control and generate variability in the output speech. In short, our contributions from this work are: (1) We present EDITH, a hierarchical model that disentangles prosodic features in the form of $F_0$ enabling explicit global as well as local control. (2) We show that EDITH captures reliable representation of local prosody by generating speech with desired variations at the chosen linguistic level.

section{Emphasis by Disentangling Tonal Heuristics(EDITH)}
\label{proposed_approach}

% This is the main section of the paper. This is where the technical novelty of your paper should be clearly explained and motivated. Whenever possible, take the time to remind people how your proposed model/approaches address the challenges highlighted in the introduction. Also, make sure that the novelty of your new approaches is highlighted (i.e., what part was already done vs. what part is novel). You can divide this into two parts - model description and learning. (1) Model Description: formalize mathematically your proposed model or approach. This should include the mathematical definition of the variables involved in your problem (e.g., input variable x and labels y). You should also include an objective function describing what you are modeling (e.g., P(y|x)). While this section may be relatively short (e.g., half a page), you should be consistent with your notation later in the paper. (2) Learning: Describe how you are learning the parameters of your model. State clearly your loss function and the learning algorithm used in your experiments. If you have enough space, give derivation of your gradient, at least for some of the main model parameters.

%In our approach, we propose to incorporate automatically derivable information from speech data into the model architecture so as to enable to Seq2Seq architectures better model prosody. This is motivated by the understanding that the utterances themselves do not always contain all the information needed to comprehend the appropriate prosody information. The missing information is either part of background knowledge about the world - implicit to humans but not annotated in the data - or is provided by accompanying context of the utterance. Consider an example sentence: `\textit{You do not have a pet shark}'. Most prosodic constructions for this sentence involve sarcasm since it is not commonplace to have sharks as pets - world knowledge. Similarly consider the sentence: `\textit{I dont want to be a nun}'. The linguistic unit subject to realization of prosodic stress in this sentence depends on the context information.  While it is possible to include context information in long form content generation such as audiobooks, there are scenarios where we do not have access to the context or the the available context is ambiguous. Consider the example of a TTS system deployed to assist visually impaired students comprehend math equations. There is not sufficient signal in the text to demarcate between  $x^{(y+z)}$ and $x^y + z$. However, human voice talent would almost certainly place appropriate prosodic cues that help in comprehension in such cases. Therefore, we believe that it is helpful to leverage information from the speech signal and incorporate biases into the model architecture during the training itself to help the model accomplish the desired synthesis.

\begin{figure}[t]
\centering
\includegraphics[scale=0.2]{images/EDITH_arch.png}
\caption{\textit{Architecture of EDITH. Circles denote LSTM cells, rectangles represent vectors and pentagons represent global latent vectors. (Best viewed in color)}}
\label{edith_architecture}
\end{figure}

EDITH learns the joint distribution between pairs of temporal sequences \{\textbf{x}, \textbf{y}\} where \textbf{x} denotes the features and \textbf{y} denotes the acoustic parameters. Let Ti and To denote the lengths of input and output sequences respectively. Input features \textbf{x} consist of both linguistic features denoted as \textbf{$x_{linguistic}^{1:Ti}$} as well as features extracted from the acoustic signal denoted as \textbf{$x_{acoustic}^{1:To}$}. The output features \textbf{y} consist of linear feature representation \textbf{$y_{linear}^{1:To}$} as well as mel features \textbf{$y_{mel}^{1:To}$}. It has to be noted that \textbf{$x_{acoustic}^{1:To}$} = \textbf{$y_{mel}^{1:To}$}. To efficiently model varying prosody and prevent the averaging effect, we incorporate a variational layer. Therefore, EDITH is a conditional variational auto-encoder. During inference, we discard the encoder part of our model. Our model can be summarized by the following set of equations:

%section{EDITH: Model Description}

%We are interested in incorporating inductive biases into the architecture of Seq2Seq TTS enabling explicit control over prosody. Ideally, we desire the priors to push model towards learning and exploiting latent representations that can generate different prosodic events. In addition, we also desire the learnt representations to be generalized so that the approach can be extended to different speakers as well as languages or language combinations such as code mixing.  While there are many other parameters associated with prosody, $F_0$ can be considered one of the most important attributes. Based on this, we have chosen to incorporate priors in terms of $F_0$. To accomplish this, we have extracted fundamental frequency using SPTK toolkit\citep{sptk2009speech} and then quantized it into multiple bins with each bin spanning 25 Hz. In other words, continuous $F_0$ values were mapped to their discrete bin indices, resulting in ordinal $F_0$ values for each utterance. We then incorporated these values into the model architecture. We have investigated two ways of incorporating $F_0$ priors: (1) Explicit labeling by directly adding the ordinal $F_0$ as additional input to the model and (2) Implicit incorporation by adding prediction of ordinal $F_0$ as a secondary task in the model. The predicted $F_0$ is used as conditioning information to the decoder along with the input from soft attention.

%\begin{flalign*}
\begin{equation} \label{eq1}
\begin{split}
encoded ={}& \textbf{H}^{Encoder}(x_{linguistic}, x_{acoustic}) \\
z_{g}, z_{l} ={}& \textbf{VI}(encoded) \\
\hat{y}_{mel} ={}& \textbf{H}^{Decoder}(x_{linguistic},z_{g}, z_{l}) \\
\hat{y}_{linear} ={}& \textbf{H}^{postnet}(\hat{y}_{mel})
\end{split}
\end{equation}
%\end{flalign*}

\iffalse
\begin{equation} \label{eq1}
\centering
\begin{split}
e_{1:T_{input}} & = Encoder(x_{1:T_{input}}) \\
z_{global}, z_{local} = VI(e_{1:T_{input}}) \\
\alpha_{i} & = Attention( e_{1:T_{input}}, d_{i-1}) \\
e^1_i &= \sum_j \alpha_{ij} e_j \\
d_i &= Decoder(e^1_i, s)
\end{split}
\end{equation}
We imagine the generative process as follows:

\begin{itemize}
    \item We sample a global prosody embedding for the utterance.
    \item Conditioned on the sampled embedding and the textual input, we predict local prosody embedding at time $t0$
    \item The predicted prosody embedding is combined with the textual embedding to generate frame level acoustic representation at $t0$.
    \item The process continues.
\end{itemize}

section{Model Components}

\subsubsection{EDITH Encoder}

- clockwork hierarchical LSTM for prosodic features \\
- Vanilla LSTM for linguistic features \\
- Both reset at phone boundaries \\

\subsubsection{EDITH Variational Layer}

- Vector Quantization. \\
- Stop Gradient \\
- Commitment Loss \\
- Normalization of latent vectors

\subsubsection{EDITH Decoder}

- Dot product Attention \\
- 5 frames at once \\
- No masking of outputs to enable better end of sentence prediction.
\fi

Design of our encoder is inpired by the encoder from \citep{chive}. We use clockwork hierarchical LSTM to encode \textbf{$x_{linguistic}^{1:Ti}$} and \textbf{$x_{acoustic}^{1:To}$}. However, our models are clocked at the rate of phones as opposed to syllables. In addition, we do not incorporate any features from word or sentence levels in our encoder to keep the architecture compact. Our variational layer is derived from \citep{vqvae} and is employed  to generate global and local latent variables $z_{g}$, $z_{l}$ respectively. Our decoder is similar to a typical attention based acoustic decoder\citep{tacotron_original} and includes a postnet. While similar in formulation, EDITH has an important difference from \citep{chive} in that our local latent variables follow the rate of input as opposed to output as in \citep{chive}. This allows us to exercise more control over the generated prosodic variations.

% In \citep{chive}, authors employ clockwork hierarchical decoding for this step. We employ a vanilla architecture instead.

\textbf{Optimization and Learning}: $x_{acoustic}$ is passed through phone rate LSTM. This is shown as block 1 in figure \ref{edith_architecture}. $x_{linguistic}$ is passed through phone LSTM. The representations are concatenated and passed through EDITH Encoder. This is shown as block 2. Outputs from the encoder are passed through the variational layer where vector quantization is performed to pick the most suitable global latent prosodic vector. Conditioned on encoder outputs and the global latent prosodic vector, we predict Ti local prosodic vectors corresponding to predicted local prosodic features.
We constrain the local latent variables to correspond to quantized $F_0$ by modeling their prediction as a classification task. These local latent variables thus capture the local variations in prosody while global latent variable is reserved for capturing sentence level variations. Ground truth quantized values for classification are obtained by selecting the maximum bin within the duration of phoneme. This is shown as block 3 in the figure. We then employ dot product attention in our decoder. \textbf{$y_{mel}$} is generated by decoder conditioned on local, global latent variables and the encoded $x_{linguistic}$. A postnet is employed to generate \textbf{$y_{linear}$} conditioned on \textbf{$y_{mel}$}. EDITH is optimized to minimize two \textit{L1} losses one each for \textbf{$y_{mel}$} and \textbf{$y_{linear}$} and one classification loss for local latent variables. Additionally, to train the vector quantization layer, we minimize encoder commitment loss for $z_g$ and vector quantization loss following \cite{vqvae} for both $z_g$ and $z_l$. This can be expressed as below:

\begin{equation} \label{eq1}
\begin{split}
\textit{L} = \lambda_{linear} \sum_{t=0}^{To} \| y_{linear}^t - \hat{y}_{linear}^t \|  \\
+ \lambda_{mel} \sum_{t=0}^{To} \| y_{mel}^t - \hat{y}_{mel}^t \| \\
+ \lambda_{qF_0} \sum_{t=0}^{Ti} Div(qF_0, \hat{qF_0}) + \lambda_{e} L_{e} + L_{VQ}\\
\end{split}
\end{equation}

%The weight for the encoder commitment loss is linearly increased to reach 0.2 at update

\iffalse
\begin{itemize}

    \item $x_{prosodic}$ is passed through phone rate LSTM. This is shown as block 1 in figure \ref{edith_architecture}. Following the approach in \citep{chive}, we reset the state of LSTM after every phoneme boundary.
    \item $x_{linguistic}$ is passed through phone rate LSTM. This is shown as block 2 in figure \ref{edith_architecture}.
    \item The representations are concatenated and passed through EDITH encoder
    \item Outputs from the encoder are passed through the variational layer where vector quantization happens to pick the most suitable global latent prosodic vector. This is shown as block 3 in figure \ref{edith_architecture}.
    \item Conditioned on encoder outputs and the global latent prosodic vector, we predict T local prosodic vectors corresponding to predicted local prosodic features. This is not shown for brevity in the figure.
    \item Global and local latent vectors are then employed in addition to the attention outputs to generate the frame level acoustic features.
    \item Loss Terms : (1) Mel Acoustic feature reconstruction (2) Linear Acoustic feature reconstruction (3) Prosodic Feature Reconstruction (3) Embedding Loss from latent space (4) Encoder commitment loss from latent space

\end{itemize}

\fi

section{Model Interpretation}
This approach can be interpreted as VQVAE\citep{vqvae}. It can also be seen as GST\citep{tacotron_styletokens} based encoding but our approach has two differences:(1) We do not use a different encoder for spectral information and (2) We explicitly constrain the latent classes to correspond to the quantized $F_0$s. We divide the model into individual blocks or modules. Therefore, it can be seen as an extension to Neural Module Networks\cite{neural_module_networks}. In \cite{chive}, authors introduce clockwork hierarchical VAE to predict $F_0$, duration and $C_0$. Our approach of incorporating $F_0$ information at the output of encoder in the form of additional task can be seen similar to this work. However, we use quantized $F_0$s, do not employ clockwork structure in our model and do not explicitly model duration or $C_0$.

section{Experimental Setup}
\label{experimental_setup}

%  Start this section by clearly stating what your main research questions are. Whenever possible, try to design experiments to test specific research hypotheses or questions. If you claimed that components A and B are the main novelties of your model, then you should have ablation experiments that compare your model with and without A (and similarly for B). Your experiments should also include comparison with baseline models (from your midterm report). Furthermore, the experiments should give the reader an understanding of the effect of your model hyper-parameters. For example, plot performance of your model with different regularization factors. Finally, make sure to write the experimental setup method in such a way that a person not familiar with your work would be able to reproduce your model just by reading the paper. Describe the dataset you are using for this project. Describe the input modalities and annotations available in this dataset. Describe your experimental methodology for evaluating the baseline model(s). This should include the way you split your dataset for training, validation and testing. It should also include any details about the hyper-parameters and the evaluation metrics. If space is too short, you can move some of the implementation details in the supplementary material, but the main design decision should be explained in the main paper.

\textbf{Data}: We have used data from LJSpeech dataset\citep{ljspeech_dataset} to build our systems. We have used all of the 13100 sentences. The text was normalized manually to convert non standard forms (for ex. 1993) to written forms (nineteen ninety three).

\iffalse
section{Baseline}

Our acoustic model is based on Tacotron\cite{tacotron_original} Seq2Seq speech synthesis system and is built using PyTorch. We have used phones as the input instead of characters.  We have not performed masking of padded frames as is typically done in Seq2Seq models. We found that not masking helps model better predict end of sentence as mentioned in \cite{tacotron_original}. Since adjacent frames seem to be correlated, our decoder predicts 3 frames per timestep. We have used a batch size of 64 to train the baseline model.
\fi
%Our vocoder is based on WaveNet\cite{wavenet_original}. Speech signal was power normalized and squashed to the range (-1,1). We have used 16 bit mulaw quantization to encode individual samples. Instead of transposed convolutions we have employed linear interpolation to upsample the acoustic frames to match the time resolution of speech samples. To optimize the model, we use discretized Mixture of Logistics loss\cite{salimans2017pixelcnn++,juvela2019waveform} with 12 logistic classes.

\textbf{Baselines}: Our acoustic model is based on Tacotron\cite{tacotron_original} Seq2Seq speech synthesis system is built using PyTorch\citep{pytorch}. We have not performed masking of padded frames as is typically done in Seq2Seq models. We found that not masking helps model better predict end of sentence as mentioned in \cite{tacotron_original}. Since adjacent frames seem to be correlated, our decoder predicts 5 frames per timestep. Our model has three deviations from the original implementation: (1) Phones are used as the input instead of characters. (2) CBHG module in the encoder and postnet has been replaced with with three LSTM layers. (3) We use all the predicted frames at a time step as input to the decoder(as opposed to only the last time step) while predicting the next frames.  We have used a batch size of 64 to train the baseline model. To enable control of prosody, we employ quantized $F_0$ values as additional inputs to this baseline model. For this, we first extract $F_0$ values for the dataset and quantize them into multiple bins each spanning 25 \textit{Hz} without any overlap. These quantized $F_0$ values are embedded and added as additional inputs to the baseline model. In other words, this is a conditional generative model with phones and quantized $F_0$s as inputs. Additionally, we also build a model that uses word level prosodic features extracted using AuToBI\citep{rosenberg2010autobi}. We refer to this system as \textbf{AuToBI}.

\textbf{EDITH Hyperparameters}: The encoders of both $x_{acoustic}$ and $x_{linguistic}$ are realized using bidirectional LSTMs. We have used 256 as the hidden dimensions for both these encoders. Both our global and local latent variables are of 256 dimensions. We employ 10 global latent classes. The network to predict local latent variables is implemented using bidirectional LSTMs that takes 512 dimensional input and outputs 256 dimensional vectors. Encoder weight $\lambda_e$ was linearly increased to 0.2 till 10K timesteps and remained constant after that. For quantization of $F_0$, we have followed the same procedure as in Baseline. 25 Hz was chosen as the size of bin. This effectively resulted in a total of 14 bins and thus 14 local latent classes. After every update step, we normalize the local latent variables by the norm. Since these classes correspond to ordinal data in terms of quantized $F_0$s, we believe that normalizing places the vectors on a unit circle.

\textbf{SubUtterance Models}: Long utterances present in audiobooks  are rich in prosodic variations but also lead to computational overhead in terms of processing speed. Therefore, we have built systems that have access to only part of the utterance by selecting aligned segments of text and acoustics within a full sentence. We note that such an approach is already used for vocoding: Typical vocoders the authors are aware of are trained using aligned chunks of acoustic vectors and corresponding speech samples  as opposed to full utterances. Encouraged by this, we build sub utterance based models for both baseline as well as proposed approach. To distinguish from the full sentence models, we refer to these systems as Sub Utterance Baseline(\textit{SUB}) and Sub Utterance EDITH(\textit{SUE}) while referring to the full sentence models as Full Utterance Baseline(\textit{FUB}) and Full Utterance EDITH(\textit{FUE}) respectively.

%We have built multiple variants to compare the performance of the proposed approach. Extending our baseline, we have built systems that employ quantized F0 in their architectures. Additionally, we have also built models trained with access to only part of the full sentence. We refer to these systems as Full Sentence Quantized F0 (FSQ) and Sub Sentence Quantized F0 (SSQ) respectively. For each of the above systems, we have incorporated the prior information about F0 either at the input or as an additional loss term after the encoder. We refer to these systems with subscripts. For instance, the system employing full sentences for training with quantized F0 as inputs in addition to phoneme embeddings is referred to as $FSQ_i$. Similarly, the system employing sub sentence training but with additional loss term is referred to as $SSQ_l$. In addition to the proposed approaches, we have also built systems that employ AuToBI\cite{rosenberg2010autobi} labels. We have not built a sub sentence variant in this approach and refer to the full sentence variant as AUTOBI. We have utilized 1300 sentences as validation data to tune our hyperparameters.

\begin{table}[t]
\centering
\caption{\textit{Results from Preference and MOS Tests for Emphasis generation. The entries for the preference portion(columns 2 through 6)indicate preference values obtained by the systems in the first column against every other system in the subsequent columns.}}
\small
\setlength\tabcolsep{2pt}
\begin{tabular}{|c|c|c|c|c|c|| c| }
  \hline
 Config &  $FUB$ &  $FUE$ & $SUB$ & $SUE$ & AUToBI & MOS\\
  \hline
  $FUB$ &  -  & 92 & 396 & 363 & \textbf{441} & 4.0\\
  \hline
  $FUE$(ours) &  \textbf{345} & - & \textbf{424}  & \textbf{378} & \textbf{477} & 4.0 \\
  \hline
 $SUB$ &  91 & 86 & - & 235 & \textbf{278} & 3.4 \\
  \hline
 $SUE$(ours) &  64 & 86 &  243 & - & 227 & 3.6 \\
  \hline
 AUToBI &  47 & 19 & 219 & 256 & - & 3.9 \\
  \hline
\end{tabular}
\label{table_results_quantvsautobi}
\end{table}

\textbf{Evaluation}:
Evaluation was performed  in the form of listening tests using \citep{testvox_parlikar}. We have conducted two types of listening tests: (1) Rating the naturalness in  terms of Mean Opinion Score (MOS) on a scale of 1(least natural) to 5(highly natural) and (2) ABX Preference test on Emphasis where the users need to mention their preference towards either of the systems or state that they prefer neither. For the preference evaluation we have manually curated 50 sentences where the meaning was implied based on prosody. Participants were shown the entire sentence and its implication in parenthesis. An example sentence from our testset is `\textit{It looks like a starfish} (but it really is not).' Every system was used to generate this test set\footnote{in the mentioned example, the systems generated just the part '\textit{It looks like a starfish}' and not the part in parenthesis}. For baseline and proposed approaches, the phonemes to be emphasized are rendered with embedding vector corresponding to bin 12 while others are rendered with bin 8 The participants are to mention their preference to the system that faithfully generates prosody in line with the information in parenthesis. We had 25 listeners and each participant rated 20 random sentences giving us a total of 500 ratings per pair of systems.

\begin{figure}[t]
\centering
\includegraphics[scale=0.25]{images/f0_manipulation.png}
\caption{\textit{Plot of Fundamental Frequency($F_0$) trajectories obtained from generated waves using proposed approach \textit{FUE}. Variants of the sentence `John loves Mary' are generated with emphasis on individual words(captialized). The blue trajectory corresponds to $F_0$ when no emphasis was applied to any word. The plot highlights that the proposed approach allows explicit local control at the desired level in the generated speech. We have submitted the generated wavefiles as supplementary material.}}
\label{plot_overview}
% These and other samples are attached in the supplementary material for reference.
\end{figure}

\textbf{Discussion}: The preference evaluation results for the proposed approaches are presented in table \ref{table_results_quantvsautobi}.  We have excluded the \textit{No Preference} values from this table for brevity. However, they can be estimated based on the values in the table. The full utterance based systems seem to outperform sub utternace as well as AuToBI based systems consistently. Within the full sentence systems,  our proposed approach(\textit{FUE}) outperforms the baseline conditional generative model(\textit{FUB}). A sample output generated by conditioning the local latent variables to emphasize individual linguistic units(words) from our approach can be examined in figure \ref{plot_overview}. An informal listening test in the scenarios where full sentence models were not preferred revealed an interesting finding: All these scenarios were when the emphasized word was the first in the sentence. We hypothesize that this might be due to the canonical word order(\textbf{SVO}) in English. One approach to handle this could be to incorporate a suitable weighting to consider this effect and we plan to investigate this further.  The sub utterance based approaches seem to match the performance of AUToBI systems while clearly under performing their full utterance counterparts. Informal listening evaluations revealed that the sub utterance models seem to have repetition of phoneme units within the generated sentence. We attribute this to the errors in alignment and phoneme boundary estimation and plan to investigate approaches to circumvent this behavior in future work.

%Some listeners have expressed that they felt multiple words were focused in the sentence but nevertheless the intended word more or less stood out from the rest except when it was the first word in sentence. We hypothesize that this might be due to the data distribution and plan to investigate this further.

section{Conclusion}

In this case study, we have proposed an approach to obtain local and fine grained control over prosody in neural generative models for speech. For this we quantize fundamental frequency, which is highly correlated with prosody information, into multiple bins. We infer this information employing hierarchical global and local latent variables in the model architecture.  We show that our approach generates appropriate emphasis at word level and significantly outperforms AuToBI in terms of flexibility.