source/setup_modeling.tex

We are concerned in the present work with network models in which the adjacency matrix given by a low-rank signal matrix subject to additive noise.
Models of this form are often termed ``signal-plus-noise'' models in the literature 
\citep[see, e.g.,][]{cape_signal-plus-noise_2019,hao_neurips}.
We thus consider
\begin{equation} \label{eqn:model_low_rank_plus_noise}
    \mA = \mP + \mE ,
\end{equation}
where $\mE$ is a mean-zero noise matrix and $\mP$ is an $n \times n$ symmetric matrix of rank $r$ whose spectral decomposition is given by
\begin{equation} \label{eqn:P-spec-decomp}
    \mP  = \mU_{1:r} \mS_{1:r} \mU_{1:r}^\top ,
\end{equation}
where $\mS_{1:r}$ is a diagonal matrix of non-zero eigenvalues sorted in non-increasing order ($\spop_1 \ge \spop_2 \ge \cdots \ge \spop_r > 0$) and $\mU_{1:r} \in \R^{n \times r}$ is the matrix of corresponding orthonormal eigenvectors. 

In latent variable network models such as the SBM, RDPG and variants thereof discussed in Section~\ref{sec:intro}, $\mP$ is expressible as 
\begin{equation*}
    \mP = \rho_n\latent\latent^\top 
    = \left( \rho_n^{1/2} \latent \right) \left( \rho_n^{1/2} \latent \right)^\top,
\end{equation*}
where the rows of $\latent \in \R^{n \times r}$ encode vertex-level $r$-dimensional latent variables and $\rho_n \in (0, 1]$ can be interpreted as a signal strength parameter.
%By construction, under this model, the entries of $\mP$ are given by the inner products between the rows of $\latent$, $P_{ij} = \rho_n\latent_i^\top\latent_j$, and 
We consider in this work the task of recovering the scaled latent structure $\rho_n^{1/2} \latent$ from the observed $\mA$.
From the spectral decomposition of $\mP$ in Equation~\eqref{eqn:P-spec-decomp}, we may take 
\begin{equation} \label{eq:latent2eigen}
    \rho_n^{1/2} \latent = \mU_{1:r} \mS^{1/2}_{1:r} . 
\end{equation}
An obstacle arises from the observation that for any $\mW \in \bbO_r$, 
\begin{equation} \label{eq:orthogNonID}
    \latent\latent^\top = (\latent\mW)(\latent\mW)^\top .
\end{equation}
That is, any orthogonal rotation of $\latent$ gives rise to the same expected adjacency matrix $\mP$.
Thus, in estimating the scaled latent positions $\rho_n^{1/2} \latent$, we must account for this non-identifiability.
Specifically, we will be content with estimators that recover $\rho_n^{1/2} \latent$ up to orthogonal rotation as in the above display equation.

The expression in Equation~\eqref{eq:latent2eigen} suggests a natural approach to estimating $\rho_n^{1/2} \latent$, namely using the leading eigenvalues and eigenvectors of the observed $\mA$ in place of those of $\mP$.
This motivates the adjacency spectral embedding, first considered in \cite{ASE_spectral_clustering_original}.

\begin{definition}[Adjacency Spectral Embedding] \label{def:ASE}
Given a real, symmetric matrix $\mA$, whose spectral decomposition is given by
\begin{equation*}
   \mA = \mUhat\mShat\mUhat^\top,
\end{equation*}
we define the $d$-dimensional adjacency spectral embedding \citep[ASE;][]{ASE_spectral_clustering_original} of $\mA$ to be
\begin{equation*}
   \mXhat_{1:d} = \mUhat_{1:d}|\mShat|^{1/2}_{1:d} \in \R^d.
\end{equation*}
\end{definition}

The vast majority of recent work on models like the one just described evaluates estimation error via the maximum Euclidean norm between the true structure $\rho_n^{1/2} \latent$ and its estimate \citep[see, e.g.,][]{RDPG_survey,GRDPG_original,levin_central_2019,singular_subspace,levin_recovering_nodate}, after accounting for non-identifiability.
That is, a good estimator $\mXtilde \in \bbO_r$ will be one that achieves small values of
\begin{equation*}
\min_{\mW \in \bbO_r} \twoinfnorm{ \mXtilde \mW - \rho_n^{1/2} \latent } ,
\end{equation*}
where $\twoinfnorm{ \mZ }$ is defined the be the maximum Euclidean norm of the rows of matrix $\mZ$.
Convergence in this $(\tti)$-norm implies uniform recovery of vertex-level structures encoded in the rows of $\latent$ \citep{consistent_dot_product_SBM}.
Such convergence can be guaranteed under a number of latent variable network models, among which we consider the Random Dot Product Graph \citep[RDPG;][]{RDPG_survey,original_RDPG_Young}, a model in which network structure is driven by latent vertex-level geometric structure.

\begin{definition}[Random Dot Product Graph] \label{def:RDPG}
    Let $F$ be a probability distribution on $\R^r$, and let $\latent_1, \latent_2, \dots, \latent_n$ be drawn i.i.d.~according to $F$ and collect these into the rows of $\latent \in \R^{n \times r}$
    and let $(\rho_n)_{n=1}^\infty$ be a sequence with $\rho_n \in [0,1]$ for all $n=1,2,\dots$.
    We say that $\mA$ is distributed according to a {\em weighted RDPG} with latent positions $\latent$ and signal strength $\rho_n$ if, conditional on $\latent$, $\mE = \mA - \rho_n \latent \latent^\top$ has independent entries (up to symmetry).
    When $F$ is such that  $\bf x^\top \bf y \in [0, 1]$ for all $\bf x, \bf y$ in the support of $F$, we say that $\mA$ is distributed according to a {\em binary RDPG} with latent positions $\latent$ and sparsity parameter $\rho_n$ if
    \begin{equation}\label{eqn:conditional_prob}
        \Prob{\mA \mid \latent} = \prod_{i < j}(\rho_n\latent_i^\top\latent_j)^{A_{ij}}(1-\rho_n\latent_i^\top\latent_j)^{1-A_{ij}} .
    \end{equation}
    In either case, we write $(\mA, \latent) \sim \RDPG(F, n)$ with dimension $r$ and parameter $\rho_n$ clear from context, and refer to the rows of $\latent$ as the latent positions of the network.
\end{definition} 

The RDPG extends a number of widely-used network models.
For example, it recovers the SBM as a special case by taking the latent positions to be drawn from a suitably-chosen mixture of point masses.
Similarly, the RDPG recovers the degree-corrected SBM \citep{dcSBM_original} as a special case by taking $\latent_i = c_iY_i$, where $c_i \in (0,1)$ and $Y_i$ is randomly chosen to be one of $r$ distinct vectors $y_1,  y_2, \dots, y_r \in \R^r$.
The RDPG can also be extended and generalized in a number of ways.
For example, one may modify the model so that edge probabilities are given by a kernel function on the latent space $\kappa: \calX\times\calX \rightarrow [0,1]$ \citep{TanSusPri2013}. 
In the RDPG as given in Definition~\ref{def:RDPG}, $\kappa$ is simply the inner product and $\calX = \R^r$. 
The positive semidefinite structure of $\mP = \rho_n \latent \latent^\top$ can also be relaxed \citep{GRDPG_original}.
See \cite{RDPG_survey} for a more in-depth discussion of the RDPG.

\begin{remark} \label{remark:signal-strength-parameter}
The reader may have noticed that the parameter $\rho_n$ in Definition~\ref{def:ASE} is playing two distinct but related roles.
Under the binary RDPG, $\rho_n$ describes sparsity, in the sense familiar to most network researchers: as the number of vertices $n$ grows, a vanishing fraction of the possible $n^2$ edges are actually present in the observed network.
That is, the probability of an edge appearing between two particular nodes shrinks to zero as the number of nodes grows.
As discussed in Definition~\ref{def:RDPG}, we incorporate this into our model through the parameter $\rho_n$, which, for binary networks, we will refer to as the sparsity parameter. 
In the case of the weighted RDPG, $\rho_n$ describes a signal strength: if $\rho_n=1$, the eigenvalues of $\mP$ would grow linearly with $n$ (see Lemma~\ref{lemma:pop_eigenvalues_bound} in Appendix~\ref{apx:spectral}).
Having $\rho_n$ shrink to zero as the network size grows allows us to make the estimation problem harder by having the eigenvalues of $\mP$ grow sublinearly.
The key distinction between these two settings is that in the weighted RDPG, we allow for the variance of the entries of $\mE$ to be specified separately from the signal strength $\rho_n$, while in the case of the binary RDPG, the variance of the edges, being Bernoullis, is inherently tied to the behavior of $\rho_n$.
\end{remark}

\subsection{Model Misspecification}

The consistency of the ASE has been studied extensively in the context where the embedding dimension is correctly specified \citep[i.e., $k = 0$ in our notation above; see][]{RDPG_survey, levin_recovering_nodate, ASE_spectral_clustering_original, GRDPG_original, Levin-Laplacian, Tang-Laplacian, lyzinski_community_2016}.
Of course, in practice, the model rank $r$ is typically unknown and needs to be estimated from observed data.
Methods to estimate the embedding dimension have been widely explored in the literature, as discussed in Section \ref{sec:intro}.
Even with the best embedding dimension estimation techniques, however, it is still possible to misspecify the embedding dimension.
Thus, our goal in the present paper is to characterize the behavior of the ASE when the embedding dimension is incorrectly chosen.
To the best of our knowledge, this is the first work to do so.

Showing consistency of the ASE in the $(\tti)$-norm when the true embedding dimension $r$ is known require that we control the quantity
\begin{equation}\label{eqn:main:norm}
    \min_{\mW \in \bbO_{r}} \twoinfnorm{\mXhat_{1:r} \mW - \rho_n^{1/2}\mX_{1:r} } ,
\end{equation}
where we recall that the minimization over $\mW$ accounts for the non-identifiability of the latent positions.
The quantity described in Equation \eqref{eqn:main:norm}, along with related ones, has been controlled in a range of settings \citep{lyzinski_community_2016, levin_recovering_nodate, levin_central_2019, GRDPG_original, singular_subspace} when the true embedding dimension is known.
These results are contingent on being able to apply the Davis-Kahan $\sin\Theta$ theorem or variants thereof \citep[][]{Davis-Kahan, Samworth, Belkin}, which requires a non-zero eigengap.
When we embed into $d\neq r$ dimensions, these approaches are no longer feasible.
In particular, when the embedding dimension is chosen too large, the ASE includes eigenvectors whose associated population eigenvalues are zero, resulting in a zero eigengap.
Thus, if we are to characterize the behavior of the ASE when the embedding dimension is misspecified, we will need to  deploy different tools.

In investigating the effects of dimension misspecification, we will find it easiest to speak of the true embedding dimension $r$ and the difference $k = d-r$ between the selected dimension and true dimension.
We will then consider the $(r+k)$-dimensional adjacency spectral embedding,
    \begin{equation*}
   \mXhat_{1:r+k} = \mUhat_{1:r+k}|\mShat |^{1/2}_{1:r+k} ,
    \end{equation*}
where $r$ is the rank of $\mP$ and $k=d-r\in \{-r, -r +  1, \dots, n - r \}$ describes the difference between the true model rank and the selected embedding dimension.

In adapting the quantity in Equation \eqref{eqn:main:norm} to the setting where the embedding dimension is incorrectly chosen, we will need the following notation to precisely formulate the estimation error for the ASE. 
When we choose our embedding dimension too small (i.e., $k<0$), we will need to pad our estimate $ \mXhat_{1:r+k}$ with additional columns of zeros so that it can sensibly be compared with the $n$-by-$r$ matrix $\latent_{1:r}$.
Similarly, when the embedding dimension is chosen too large (i.e., $k > 0$), we will need to pad $\latent_{1:r}$ with additional columns of zeros to ensure that our matrix of embeddings and true latent positions are conformable.
Hence, in order to study the setting where $k<0$, we define $\mXhat_{1:r}^\circ \in \R^{n \times r}$ to be a zero-padded version of $\mXhat_{1:r+k}$. 
That is, when $k < 0$, we define
\begin{equation}\label{eq:def:Xcirc}
   \mXhat_{1:r}^\circ :=  \begin{bmatrix}
        \mXhat_{1:r+k} & \mathbf{0}_{r+k+1:r}
    \end{bmatrix} .
\end{equation}
Similarly, when $k> 0$, we define $\mX_{1:r+k}$ as the $n\times (r+k)$ matrix
\begin{equation}\label{eq:def:X_r+k}
    \mX_{1:r+k} :=  \begin{bmatrix}
        \mX_{1:r} & \mathbf{0}_{r+1:r+k}
    \end{bmatrix} .
\end{equation}

In understanding the estimation error of the ASE, in the setting where the embedding dimension is chosen too large ($k>0$), we perform the following decomposition, ignoring the non-identifiability for the time being:
\begin{equation*}
        \begin{aligned}
        \twoinfnorm{\mXhat_{1:r+k} - \rho_n^{1/2}\mX_{1:r+k}} 
         & \le  \twoinfnorm{\mXhat_{1:r} - \rho_n^{1/2}\mX_{1:r}} + \twoinfnorm{\mXhat_{r+1:r+k} - \rho_n^{1/2}\mX_{r+1:r+k} } \\
         & = \underbrace{\twoinfnorm{\mXhat_{1:r} - \rho_n^{1/2}\mX_{1:r}}}_{\text{Controlled via standard techniques}} \hspace{-0.1 cm} + \underbrace{\twoinfnorm{\mXhat_{r+1:r+k} } }_{\text{Goal}} .
        \end{aligned}
\end{equation*}

From this decomposition, we see that there are two terms, one corresponding to the rate that would hold if the embedding dimension were correctly chosen, and the other corresponding to the \enquote{trailing} dimensions. 
We formalize this intuition in the following lemma, which gives results both for the case where the embedding dimension is chosen too large and for the case where it is chosen too small.
A proof is given in Appendix~\ref{apx:estimation-error}.

\begin{lemma}[Misspecified Model Bounds]\label{lemma:decomp}
Suppose that $\mA = \rho_n\latent \latent^\top + \mE$, and denote the $(r+k)$-dimensional ASE of $\mA$ as $\mXhat_{1:r+k} = \mUhat_{1:r+k}|\mShat|^{1/2}_{1:r+k}$.
When $k<0$,
\begin{equation*} 
\min_{\mW \in \bbO_{r + k}} \twoinfnorm{\mXhat_{1:r}^\circ \mW - \rho^{1/2}_n \latent_{1:r} }
\ge \sqrt{ \sum_{j=r+k+1}^r \frac{ \spop_j }{ n } } .
\end{equation*}
On the other hand, when $k > 0$, suppose that there exists a sequence of $\mW^* \in \bbO_r$ such that
\begin{equation} \label{eq:def:etan}
    \twoinfnorm{\mXhat_{1:r}\mW^* - \rho_n^{1/2}\mX_{1:r}} \lesssim \etan,
\end{equation}
for some sequence $(\etan)_{n=1}^\infty$.
Then there exists a sequence of $\mW \in \bbO_{r+k}$ such that
\begin{equation*}
    \twoinfnorm{\mXhat_{1:r+k}\mW - \rho^{1/2}_n\latent_{1:r+k}} 
    \lesssim \etan
    + \norm{\mUhat_{r+1:r+k}}_{\tti}\norm{\mE}^{1/2} .
\end{equation*}
\end{lemma}

The term $\etan$ in Equation~\eqref{eq:def:etan} is meant to capture the estimation rate that would hold if the embedding dimensions were correctly chosen.
As mentioned earlier, this setting has been studied extensively in the literature, and the precise behavior of $\etan$ is known under several different variants of the RDPG corresponding to different modeling choices for $\latent$ and the edge noise $\mE$.
We collect a few representative results below, by way of example.

In the case of binary networks, we present the following theorem adapted from \cite{lyzinski_community_2016}.
\begin{theorem}[\cite{lyzinski_community_2016}, Theorem 5] \label{thm:lyzinski}
     Suppose that $(\mA,\latent) \sim \RDPG(F,n)$, then there exists a sequence of $\mW \in \bbO_{r}$ such that
     \begin{equation*}
        \twoinfnorm{\mXhat_{1:r} \mW- \rho_n^{1/2}\mX_{1:r}} \lesssim \frac{r^{1/2} \log^2 n}{\sqrt{\rho_n n}} 
     \end{equation*}
     with high probability. 
\end{theorem}

For the next example, we introduce the generalized random dot product graph (GRDPG) from \cite{GRDPG_original}, which extends the RDPG from Definition~\ref{def:RDPG} to allow $\mP$ to be indefinite.

\begin{definition}[\cite{GRDPG_original}, Definition 2]
    Denote $\mI_{p,q}$ as a $(p+q) \times (p+q)$ diagonal matrix with $1$ in the first $p$ diagonal entries and $-1$ in the last $q$ diagonal entries. Suppose $\calX \subset \R^r$ with the property that $x^\top \mI_{p,q}y \in [0,1]$ for all $x,y$. Let $F$ be a distribution on $\calX^n$, and $\latent_1, \latent_2, \dots, \latent_n$ be drawn i.i.d according to $F$. Collect these latent positions in the rows of $\latent \in \R^{n\times r}$. Then conditional on $\mX$, if the adjacency matrix $\mA$ is obtained by
    \begin{equation*}
        A_{ij} \stackrel{\text{ind}}{\sim} \Bernoulli(\rho_n\latent_i^\top \mI_{p,q}\latent_j) 
    \end{equation*}
    for all $i > j$. Then we say $(\mA, \latent) \sim \GRDPG(F, n)$ with signature $(p,q)$.
\end{definition}

\begin{theorem}[\cite{GRDPG_original}, Theorem 1]
Suppose that $(\mA, \latent) \sim \GRDPG(F, n)$ with signature $(p,q)$. Further suppose $\rank(\mP)$ is fixed. There exists a universal constant $c > 1$ such that, provided the sparsity factor satisfies $\rho_n n = \omega(\log^{4c} n)$, there exists a sequence of $\mW \in \bbO_{r}$ such that
\begin{equation}\label{thm:grdpg-rate}
    \twoinfnorm{\mXhat_{1:r} \mW - \rho_n^{1/2}\latent_{1:r}} \lesssim \frac{\log^c n}{\sqrt{n}} 
\end{equation}
with high probability. 
\end{theorem}

The following theorem, adapted from \cite{levin_recovering_nodate}, provides a bound for $\etan$ in the weighted network setting, when the entries of $\mE$ obey sub-gamma tail decay \citep[see Chapter 2 in][]{BLM2013}.
\begin{theorem}[\cite{levin_recovering_nodate}, Theorem 6]
    Suppose $\mA = \mP + \mE = \rho_n\latent\latent^\top + \mE$, and suppose that conditional on $\mX \in \R^{n\times r}$, the entries of $\mE$ are independent $(\nu, b)$-sub-gamma random variables, with $ n(\nu + b^2)\log^2 n \ll \spop_r^2$.
    Then there exists a sequence of $\mW \in \bbO_r$ such that
    \begin{equation*}
        \twoinfnorm{\mXhat_{1:r} \mW - \rho_n^{1/2}\latent_{1:r}}
        \lesssim \frac{r(\log n)\sqrt{\nu + b^2}}{\spop_r^{1/2}} 
                + \frac{rn(\log n)^2(\nu + b^2) \spop_1 }{\spop_r^{5/2}}
    \end{equation*}
    with high probability.
\end{theorem}

From Lemma~\ref{lemma:decomp}, we see that showing consistency when the embedding dimension is chosen too large (i.e., $k > 0$) amounts to being able to control $\twoinfnorm{\mXhat_{r+1:r+k}}$.
By Definition~\ref{def:ASE}, the columns of $\mXhat_{r+1:r+k}$ can be expressed as
\begin{equation*}
    \mXhat_{r+1:r+k} 
    = \begin{bmatrix}
    \uhat_{r+1}|\shat_{r+1}|^{1/2} & \uhat_{r+2}|\shat_{r+2}|^{1/2} &\dots 
    & \uhat_{r+k}|\shat_{r+k}|^{1/2}
    \end{bmatrix} \in \R^{n \times (r+k)}.
\end{equation*}
Bounding the eigenvalues as $\shat_{r+k} \le \| \mE \|$, we may deploy a range of tools from the matrix concentration inequalities literature \citep[see, e.g.,][]{Lei_community_consistency, Erdos,Vershynin_2018}, depending on the distribution of the entries of $\mE$.
Thus, the challenge lies in controlling the entries of the eigenvectors associated with these \enquote{trailing} eigenvalues.
Intuitively, the entries of the ASE associated with these eigenvectors must shrink to zero, as these are zero in the true latent positions $\latent$.
Since $\norm{\mE} \lesssim \sqrt{n}$ (ignoring logarithmic factors), the entries of $\mUhat_{r+1:r+k}$ must vanish uniformly.
Many random matrix models produce eigenvectors whose entries are bounded as $C(\log n)^c/\sqrt{n}$ for some $c > 0$.
These eigenvectors are called \textit{delocalized}, and they have been widely explored in the random matrix literature \citep{Erdos, Erdos_book, ErdHos2007SemicircleLO, eigenvectors_survey, Benigni2020OptimalDF, veryshnin_rudelson}. 
To show delocalization for our setting of a real, symmetric matrix $\mA$, we assume that $\mE$ is an $n \times n$ real, symmetric matrix whose entries are mean-zero, conditionally independent random variables given $\mP$ (up to symmetry).
Moreover, we will make the following assumptions on the spectral properties of $\mP$ and the moments of $E_{ij}$, both of which are allowed to depend on $n$:
\begin{enumerate}[label=(\textbf{A\arabic*}),ref=\textbf{A\arabic*}]
    \item{Assumption 1 (Delocalization)}: The eigenvectors associated with non-zero eigenvalues of $\mL$ are delocalized.
    That is, for some $\gamma \ge 0$ with no dependence on $n$.,
    \begin{equation*} \label{assumption_main:delocalization}
        |u_{ij}| \lesssim \frac{(\log n)^{\gamma}}{\sqrt{n}}
    \end{equation*}
    for all $i \in [n]$, and $j \in [r]$ where $v_{ij}$ refers to the $i$-th entry of the $j$-th eigenvector of $\mP$. 
    \label{assumption_main:deloc} 
    \item{Assumption 2}: The signal strength parameter $\rho_n = \omega(1/\sqrt{n})$\label{assumption_main:signal_strength}
    
    \item{Assumption 3}: The non-zero eigenvalues of $\mP$ are distinct. \label{assumption_main:eigen_nondegen}

    \item{Assumption 4}: The non-zero eigenvalues of $\mP$ are of order $\rho_n n$: $\spop_j \asymp \rho_n n$ for $j \in [r]$. \label{assumption_main:eigen_bound}

    \item{Assumption 5}: $r = \rank(\mP) \ll (\log n)^{\zeta}$ for some $\zeta \ge 0$. \label{assumption_main:rank}

    \item{Assumption 6 (Eigengap)}: The eigengap $\Delta := \min_{j \in [r]} \min_{i\ne j} |\spop_i - \spop_j|$ obeys $\Delta = \Omega\left(\rho_n n\right)$. \label{assumption_main:eigengap}

    \item{Assumption 7}: The entries of $\mE$ satisfy the moment conditions \label{assumption_main:moments}
    \begin{equation*} 
        \E{E_{ij}} = 0, \quad \E{|E_{ij}|^2} = \sigma^2, \quad \E{|E_{ij}|^{p}} \le \frac{C_1^p}{\tilde{q}_n^{(p/2-1)}} \quad(p >2)
    \end{equation*}
    where $\tilde{q}_n \in \left[ n^{-1/2} (\log n)^6, 1 \right]$ is a parameter the controls the tail behavior of $\mE$, corresponding to the sparsity parameter in the context of binary networks.
\end{enumerate}

\begin{remark}
We pause to briefly discuss some of these assumptions.
Assumption \ref{assumption_main:deloc} is similar to the incoherence condition often found in spectral methods \citep{spectral_methods_textbook, hao_neurips, Zheng_inference}. 
It is crucial for our main delocalization result to hold. In general, without such conditions, we do not necessarily expect eigenvectors of random matrices to delocalize \citep[see][for further discussion]{eigenvectors_survey}. 
In the case of the RDPG, it can be shown that $\twoinfnorm{\mU} \lesssim 1/\sqrt{n}$ (see Corollary \ref{cor:RDPG_delocalization:binary} below).
Assumption \ref{assumption_main:eigen_nondegen} assumes distinct non-zero eigenvalues in $\mP$.
We expect that this assumption can be relaxed at the expense of increased notational complexity in tracking the eigenspaces corresponding to repeated eigenvalues in the proof of Theorem \ref{theorem:main:deloc}, a task we leave to future work.
Assumptions \ref{assumption_main:signal_strength} and \ref{assumption_main:eigengap} are necessary to use results from spectral methods. 
We suspect delocalization of the eigenvectors of $\mA$ still hold even when these assumptions are relaxed, as suggested by results in \cite{hao_neurips}.
Finally, the moment conditions for $\mE$ in Assumption \ref{assumption_main:moments} allow for sub-exponential decay. 
See Remark 2.5 from \cite{Erdos} for details. 
\end{remark}