%%%%%%%% ICML 2026 EXAMPLE LATEX SUBMISSION FILE %%%%%%%%%%%%%%%%% \documentclass{article} % Recommended, but optional, packages for figures and better typesetting: \usepackage{microtype} \usepackage{graphicx} \usepackage{subcaption} \usepackage{booktabs} % for professional tables \usepackage{comment} % hyperref makes hyperlinks in the resulting PDF. % If your build breaks (sometimes temporarily if a hyperlink spans a page) % please comment out the following usepackage line and replace % \usepackage{icml2026} with \usepackage[nohyperref]{icml2026} above. \usepackage{hyperref} \usepackage{xcolor} % Attempt to make hyperref and algorithmic work together better: \newcommand{\theHalgorithm}{\arabic{algorithm}} % Use the following line for the initial blind version submitted for review: %\usepackage{icml2026} % For preprint, use \usepackage[preprint]{icml2026} % If accepted, instead use the following line for the camera-ready submission: %\usepackage[accepted]{icml2026} \usepackage{amsmath} \usepackage{amssymb} \usepackage{mathtools} \usepackage{amsthm} % if you use cleveref.. \usepackage[capitalize,noabbrev]{cleveref} %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% % THEOREMS %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% \theoremstyle{plain} \newtheorem{theorem}{Theorem}[section] \newtheorem{proposition}[theorem]{Proposition} \newtheorem{lemma}[theorem]{Lemma} \newtheorem{corollary}[theorem]{Corollary} \theoremstyle{definition} \newtheorem{definition}[theorem]{Definition} \newtheorem{assumption}[theorem]{Assumption} \theoremstyle{remark} \newtheorem{remark}[theorem]{Remark} \newcommand{\E}{\mathbb{E}} \newcommand{\PP}{\mathbb{P}} \newcommand{\Var}{\textrm{Var}} \newcommand{\argmax}{\arg\!\max} % Todonotes is useful during development; simply uncomment the next line % and comment out the line below the next line to turn off comments %\usepackage[disable,textsize=tiny]{todonotes} \usepackage[textsize=tiny]{todonotes} % The \icmltitle you define below is probably too long as a header. % Therefore, a short form for the running title is supplied here: \icmltitlerunning{Beyond First-order Asymptotics in Sequential Mean Testing} \begin{document} \twocolumn[ \icmltitle{Beyond First-order Asymptotics in Sequential Mean Testing} % It is OKAY to include author information, even for blind submissions: the % style file will automatically remove it for you unless you've provided % the [accepted] option to the icml2026 package. % List of affiliations: The first argument should be a (short) identifier you % will use later to specify author affiliations Academic affiliations % should list Department, University, City, Region, Country Industry % affiliations should list Company, City, Region, Country % You can specify symbols, otherwise they are numbered in order. Ideally, you % should not use this facility. Affiliations will be numbered in order of % appearance and this is the preferred way. \begin{icmlauthorlist} \icmlauthor{Vikas Deep}{yyy} \icmlauthor{Shubhada Agrawal}{comp} %\icmlauthor{}{sch} %\icmlauthor{}{sch} \end{icmlauthorlist} \icmlaffiliation{yyy}{Department of Information Systems and Analytics at the National University of Singapore} \icmlaffiliation{comp}{Department of Electrical Communication Engineering (ECE) at the Indian Institute of Science, Bengaluru} \icmlcorrespondingauthor{Vikas Deep}{vikas\_deep@nus.edu.sg} \icmlcorrespondingauthor{Shubhada Agrawal}{shubhada@iisc.ac.in} % You may provide any keywords that you find helpful for describing your % paper; these are used to populate the "keywords" metadata in the PDF but % will not be shown in the document \icmlkeywords{Machine Learning, ICML} \vskip 0.3in ] % this must go after the closing bracket ] following \twocolumn[ ... % This command actually creates the footnote in the first column listing the % affiliations and the copyright notice. The command takes one argument, which % is text to display at the start of the footnote. The \icmlEqualContribution % command is standard text for equal contribution. Remove it (just {}) if you % do not need this facility. % Use ONE of the following lines. DO NOT remove the command. % If you have no special notice, KEEP empty braces: \printAffiliationsAndNotice{} % no special notice (required even if empty) % Or, if applicable, use the standard equal contribution text: % \printAffiliationsAndNotice{\icmlEqualContribution} \begin{abstract} We revisit the problem of sequentially testing the mean of bounded distributions in a level-$\alpha$ power-one framework. We study a $\mathrm{KL_{inf}}$-based sequential test that is known to attain the information-theoretic lower bound on the expected stopping time with exact constants as $\alpha \to 0$. Going beyond first-order asymptotics, we establish a central limit theorem (CLT) for the stopping time of this test. Our analysis proceeds in two steps. First, we prove a novel CLT for the $\mathrm{KL_{inf}}$ statistic itself, characterizing its fluctuations around its deterministic limit. We then leverage this result to show that the stopping time, centered appropriately and scaled by $\sqrt{\log(1/\alpha)}$, converges in distribution to a Gaussian limit with an explicit variance. This yields a second-order characterization of an asymptotically optimal sequential test for bounded distributions. Finally, we present numerical experiments that corroborate our theoretical findings. \end{abstract} % Acknowledgements should only appear in the accepted version. %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% \section{Introduction} \label{sec_introduction} Sequential mean testing is a fundamental primitive underlying online experimentation, adaptive monitoring, and sequential learning. Informally, with streaming data, we want to decide as quickly as possible whether the mean is at a target level \(m_o\), while controlling false alarms at level \(\alpha\in(0,1)\). This problem arises in applications such as A/B testing and adaptive monitoring, where samples are costly, decisions are made online, and the data distribution is rarely parametric. Motivated by this, we consider the setting of the sequential mean testing problem where the underlying data distribution is non-parametric, with support bounded in $[0,1]$. Specifically, we observe an i.i.d.\ sequence $X_1, X_2, \dots$ with a common probability distribution $p$ supported on $[0,1]$ and mean $m(p) := \mathbb E_p[X]$. Given $m_o \in (0,1)$, we study the classical sequential mean testing problem \[ H_0 : m(p) = m_o \qquad\text{versus}\qquad H_1 : m(p) \neq m_o \] in an $\alpha$-correct power-one framework. %based on sequentially observed data. Given $\alpha > 0$, an \emph{$\alpha$-correct (or level $\alpha$) power-one sequential test} (initially studied by \citet{darling1967iterated} for parametric models) controls type~I error at level $\alpha$, while attaining power one under the alternative. Such tests are specified by a stopping time $\tau_\alpha$, where stopping corresponds to rejecting $H_0$. Formally, if $\mathcal{P}$ and $\mathcal{Q}$ denote the sets of distributions in $H_0$ and $H_1$, then $\tau_\alpha$ satisfies \[ \sup_{p \in \mathcal{P}}~ p(\tau_\alpha < \infty) \le \alpha, \quad \text{and} \quad \inf_{q \in \mathcal{Q}}~ q(\tau_\alpha < \infty) = 1. \] Recently, \citet{agrawal2025stopping} developed a general theory of $\alpha$-correct power-one sequential tests for composite hypotheses, establishing tight information-theoretic lower bounds and matching upper bounds on the \emph{expected stopping time} $\mathbb{E}_q[\tau_\alpha]$ in two different asymptotic regimes. In the small-error regime $\alpha\downarrow 0$, for the bounded nonparametric mean testing problem, their lower bounds and optimal tests are in terms of a certain $\mathrm{KL}$-projection function, termed $\mathrm{KL}_{\inf}$ (defined in Section \ref{sec:bounded_model}). Moreover, these tests satisfy the first-order optimality: $$ \lim\limits_{\alpha\rightarrow 0} \frac{\mathbb{E}_q[\tau_\alpha]}{\log(1/\alpha)} = \frac{1}{\mathrm{KL}_{\inf}(q,m_o)}, $$ for every fixed alternative $q$ with $m(q) \neq m_o$. However, the expected stopping time alone does not tell the full story: sequential procedures can exhibit substantial run-to-run variability, and in practice one often cares about predictability guarantees, such as the probability that the test terminates by a given deadline. For example, in sequential monitoring of safety metrics (e.g., extreme losses or excess risk in financial systems), a procedure with a small $\mathbb{E}[\tau_\alpha]$ may still be unsatisfactory if it has a non-negligible probability of stopping very late. Such delayed detection can incur high costs in safety-critical applications. These considerations motivate a second-order, distributional analysis of the stopping time of sequential tests. In this work, \emph{we establish a Central Limit Theorem (CLT) for the stopping time} of an optimal sequential test for the nonparametric mean-testing problem. The CLT for $\tau_\alpha$ quantifies typical fluctuations around its deterministic growth rate, yielding explicit variance and a Gaussian approximation that refines first-order asymptotic optimality into a more complete characterization of stopping-time uncertainty. CLTs for stopping times are well understood in the classical setting of first-passage times for random walks \citep{gut2009stopped,asmussen2003applied}. We refer the reader to Section \ref{sec:related_work} for a detailed literature review. These tools underpin stopping-time CLTs for parametric likelihood-ratio procedures, which are known to yield asymptotically optimal $\alpha$-correct, power-one tests for sequential mean testing in parametric models \citep{robbins1974expected}. This approach works for the parametric models because the log-likelihood ratio statistic in these settings can be expressed as a sum of i.i.d.\ log-likelihood increments, thereby inducing a fixed random-walk structure. However, this fails in the non-parametric settings. The nonparametric $\mathrm{KL}_{\inf}$ statistic is defined through an optimization over distributions (equivalently, via a dual optimization problem), and the resulting process cannot be reduced to a fixed random walk in a straightforward manner. As a result, classical first-passage CLTs for random walks do not directly apply, and existing techniques do not appear to readily yield distributional limit theorems for stopping times for the optimal $\mathrm{KL}_{\inf}$-based power-one tests. Further, to the best of our knowledge, no stopping-time CLT is known for asymptotically optimal nonparametric power-one sequential tests. %In this work, we leverage the CLT for $\mathrm{KL}_{\inf}$ to derive a CLT for the associated stopping time. The main contributions of this work are as follows. \begin{enumerate} \item We establish a CLT for an appropriate centering and scaling of the empirical $\mathrm{KL}_{\inf}$-statistic, characterizing its fluctuations around its deterministic limit under bounded nonparametric models (Theorem~\ref{lem:kl_inf_clt}). \item Building on the above, we derive the first stopping-time CLT for an asymptotically optimal nonparametric power-one sequential test based on $\mathrm{KL}_{\inf}$ statistic, obtaining an explicit Gaussian limit for the properly normalized stopping time (Theorem~\ref{thm:clt_bd}). \item As an application, we construct an asymptotically valid confidence interval for the stopping time when $\alpha$ is small, using only a \emph{single simulation run} (Proposition~\ref{lem:ci-tau}). \item We validate the theory through simulations (Section~\ref{sec:numerics}) as well as experiments on real-world DSSAT crop-yield dataset (\url{https://dssat.net}). \end{enumerate} Our key idea is to leverage the fact that the dual representation of $\mathrm{KL}_{\inf}$ is a one-dimensional convex optimization problem, which rewrites the statistic as the maximum of a concave objective indexed by a single scalar parameter. We show that the empirical maximizer computed from the data converges to the maximizer of the dual problem for the true (unknown) distribution at \(O_p(1/{\sqrt{n}})\) rate. Crucially, once the maximizer is stable, the optimization itself does not contribute to the leading-order fluctuations, leading to the Gaussian limit in Theorem~\ref{lem:kl_inf_clt}. %\textcolor{red}{reviewer comment is to merge the parafgraph above and the two above one} %Another key consideration in proving the above result is that the maximizer of this dual problem under the true unknown distribution can lie either in the interior of the optimization domain of the parameter, or at its boundary (see Section \ref{sec:results}). Both cases require separate handling, involving delicate convergence arguments for the empirical maximizer around the true maximizer of the dual problem. To pass from CLT for $\mathrm{KL}_{\inf}$-statistic to the stopping-time CLT, we use the fact that the stopping time grows on the order of \(\log(1/\alpha)\). Around that scale, the accumulated statistic behaves like a deterministic linear drift plus the fluctuation term identified above. A standard boundary-crossing inversion argument then converts fluctuations of the accumulated statistic into fluctuations of the time at which the boundary is crossed, giving a Gaussian limit for the properly rescaled stopping time with an explicit variance constant inherited from the first step (Theorem \ref{thm:clt_bd}). %To the best of our knowledge, this is the first stopping-time CLT for an asymptotically optimal nonparametric power-one sequential testing procedure. A key technical step in our approach is justifying the CLT for the empirical-$\mathrm{KL}_{\inf}$ statistic at the stopping time \(\tau_\alpha\). This corresponds to verifying an Anscombe-type condition (see \citet{anscombe1952large} and \citet[Chapter 1]{gut2009stopped}), ensuring that the fixed-\(n\) CLT transfers to a CLT at the stopping time $\tau_\alpha$. Standard proofs for this typically rely on i.i.d.\ increments and Kolmogorov's maximal inequality \citep[Chapter 1]{gut2009stopped}. However, since the empirical-\(\mathrm{KL}_{\inf}\) statistic does not directly yield an i.i.d.\ increments formulation, a Taylor expansion decomposes the statistic into a sum of a leading partial-sum term with i.i.d.\ increments, and a smaller remainder term (see Lemma \ref{lemm:anscombe}). %As an application of our stopping-time CLT, we construct a consistent estimator for the asymptotic variance in the Gaussian limit of the properly rescaled stopping time. This allows us to form asymptotically valid confidence intervals for the stopping time when $\alpha$ is small, using only a single simulation run. Unlike standard simulation-based confidence intervals, which typically require many independent replications, Proposition~\ref{lem:ci-tau} provides an asymptotically valid confidence interval from the same run that produces the stopping time. Its validity follows directly from the CLT for the scaled stopping time established in Theorem~\ref{thm:clt_bd}. %Finally, we corroborate our theoretical findings via numerical simulations. The first set of simulations illustrates the CLT for \(\mathrm{KL}_{\inf}\), while the second set validates the CLT for the properly rescaled stopping time. We find numerically that the distribution of the properly rescaled stopping time is close to its Gaussian limit only for very small values of $\alpha$ (Proposition \ref{lem:ci-tau}). We further validate the rescaled stopping-time CLT on a real-world DSSAT crop-yield dataset (\url{https://dssat.net}), observing close agreement with the Gaussian limit in this setting as well. %We now conclude this section with a brief roadmap for the rest of the paper. \noindent{\bf Paper organization.} Section~\ref{sec:related_work} reviews related work on sequential testing and stopping-time limit theorems. Section~\ref{sec:setup} introduces the bounded nonparametric mean testing model, defines $\mathrm{KL}_{\inf}$ and its dual representation, and describes the $\mathrm{KL}_{\inf}$-based stopping rule. Section~\ref{sec:results} presents our main results, including a CLT for the empirical $\mathrm{KL}_{\inf}$ statistic and the resulting stopping time $\tau_\alpha$, with proofs deferred to the appendices. Section \ref{sec:application} contains the results related to application of stopping time CLT to construct confidence interval for stopping time. Section~\ref{sec:numerics} reports numerical experiments that support the theoretical findings, and we conclude with a discussion of future directions. \section{Related Work} \label{sec:related_work} Our results connect two classical strands of work: (i) \emph{sequential testing} for mean and, (ii) distributional limit theorems for \emph{stopping times} (in particular, first-passage times). We briefly review the most relevant literature in each area below. \noindent{\bf Sequential testing. } Sequential testing originated in Wald’s framework with the sequential probability ratio test (SPRT), which is optimal for simple-vs-simple parametric hypotheses: among all tests satisfying given error constraints, it minimizes the expected number of samples \citep{wald1992sequential,wald1948optimum}. Further, power-one and $\alpha$-correct sequential tests were first studied by \citet{darling1967iterated}. In classical parametric settings, the optimality of sequential tests is characterized by a certain Kullback-Leibler (KL) divergence; see early optimality results and lower bounds in sequential design and testing \citep{siegmund2013sequential,chernoff1992sequential}. In nonparametric settings (eg, mean testing with bounded support), the relevant information quantity becomes a KL-projection onto the null set, often denoted as a $\mathrm{KL}_{\inf}$. This KL-projection function has previously appeared in the lower bounds in the multi-armed bandit literature, dating back to works of \citet{lai1985asymptotically,burnetas1996optimal}, and bandit algorithms designed using an empirical version of it have been shown to be optimal in a wide variety of non-parametric bandit settings \citep{honda2010asymptotically,honda2015non,agrawal2020optimal,pmlr-v134-agrawal21a,agrawal2021optimal, jourdan2022top}. Furthermore, this KL-projection function has previously appeared in the construction of asymptotically optimal confidence intervals for the mean of a distribution and the average treatment effect in A/B testing \citep{deep2024asymptotically, deep2025asymptotic}. Recently, \citet{agrawal2025stopping} develop a general theory of power-one, level-$\alpha$ sequential tests for composite hypotheses. However, they focus on the expected stopping time. In contrast, the focus of this work is to understand other properties of these optimized tests. %In particular, this framework encompasses nonparametric sequential mean testing as a special case and shows that $\mathrm{KL}{\inf}$-driven tests can match information-theoretic lower bounds on $\mathbb{E}[\tau\alpha]$ with sharp constants as $\alpha \downarrow 0$. \noindent{\bf Stopping-time limit theorems and first-passage CLTs. } Distributional limit theorems for stopping times are classical in applied probability and sequential analysis, with a large literature devoted to first-passage times of drifted random walks and their refinements via renewal theory. These results yield both laws of large numbers and central limit theorems for stopping times \citep{gut2009stopped,asmussen2003applied}. In parametric settings, log-likelihood ratio statistics (and their mixture or generalized variants) evolve additively, allowing first-passage CLTs to be applied directly to obtain distributional approximations for stopping times. Another independent work, \citet{mukhopadhyay2020asymptotic} provides sufficient conditions for establishing a CLT for a stopping time defined through a test statistic, assuming that the test statistic itself already satisfies a CLT. Establishing such a CLT for our test statistic is itself one of the main technical contributions of this work. Furthermore, our stopping rule does not directly fit into their framework, preventing a black-box application of their results. %In contrast, the nonparametric empirical-$\mathrm{KL}_{\inf}$ statistic that drives our procedure is defined through an optimization over distributions, or equivalently via a dual representation, and the %resulting process does not admit a random-walk structure trivially. As a consequence, classical first-passage CLTs do not apply. Our main contribution addresses this gap by first establishing a novel CLT for the empirical-$\mathrm{KL}_{\inf}$ statistic itself and %then transferring these fluctuations to the stopping time through a boundary-crossing argument, yielding a second-order characterization for an asymptotically optimal nonparametric sequential test. %Interestingly, \citet{mukhopadhyay2020asymptotic} also develops a methodology for transferring the CLT result of a test statistic evaluated at the stopping time to the CLT of the stopping time. %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% \section{Preliminaries: Setup and Background} \label{sec:setup} We now formally describe our setup. Let $\mathcal{B}$ denote the set of all probability measures supported on $[0,1]$. For any $ p \in\mathcal{B}$, write $m(p):=\mathbb{E}_p[X]$ for its mean. We observe an i.i.d.\ sequence $X_1,X_2,\ldots\in[0,1]$ with common law $p\in\mathcal{B}$, and define the natural filtration $\mathcal{F}_n:=\sigma(X_1,\ldots,X_n)$. We fix $m_o\in(0,1)$ and an error tolerance level $\alpha\in(0,1)$. Throughout, $\mathrm{KL}(q,p)$ denotes the Kullback-Leibler divergence (with the usual convention $\mathrm{KL}(q,p)=+\infty$ if $q\not\ll p$). We use the notation $\mathbb{P}_p(\cdot)$ to denote the probability of the input event under the measure on an infinite length sequence $X_1, X_2, X_3, \ldots$, generated i.i.d. from $p$. Similarly, we now use $\mathbb{E}_p[\cdot]$ to denote the expectation under this joint measure. We consider the following mean testing problem in this work, \begin{equation} \label{eq:setup-hypotheses} \begin{aligned} H_0 &:~ p \in \mathcal{P}^{\rm bd} := \{p\in\mathcal{B} : m(p) = m_o\}, \\ H_1 &:~ p \in \mathcal{Q}^{\rm bd} := \{p\in\mathcal{B} : m(p) \neq m_o\}. \end{aligned} \end{equation} \subsection{$\alpha$-correct Power-one Sequential Tests} A sequential test is specified by a stopping time $\tau_\alpha$ (with respect to $(\mathcal{F}_n)_{n\ge 1}$) taking values in $\mathbb{N}\cup\{\infty\}$, with the convention that the procedure rejects $H_0$ at time $\tau_\alpha$ if $\tau_\alpha<\infty$; otherwise, the test never rejects. %(equivalently, accepts $H_0$ in the limit). \begin{definition}[$\alpha$-correct power-one sequential test] \label{def:delta-correct} For $\alpha \in (0,1)$, a stopping time $\tau_\alpha$ is called an \emph{$\alpha$-correct power-one sequential test} for \eqref{eq:setup-hypotheses} if \begin{align*} \sup_{p \in \mathcal{P}^{\rm bd}} p(\tau_\alpha < \infty) &\le \alpha, \\ \inf_{q \in \mathcal{Q}^{\rm bd}} q(\tau_\alpha < \infty) &= 1. \end{align*} \end{definition} To formally introduce the known asymptotically optimal sequential test, we first introduce the \(\mathrm{KL}_{\inf}\) function, which will be used in the construction of the test. \subsection{KL Projection Function: $\mathrm{KL}_{\inf}$}\label{sec:bounded_model} $\mathrm{KL}_{\inf}$ for bounded distributions is a well-known object in the literature \citep{honda2010asymptotically,honda2015non}. For a given $q \in \mathcal{B}$ and $m_o\in (0,1)$, \[ \mathrm{KL}_{\inf}(q,m_o) \triangleq \begin{cases} \mathrm{KL}_{\inf}^{+}(q,m_o), & m_o \ge m(q),\\ \mathrm{KL}_{\inf}^{-}(q,m_o), & m_o < m(q), \end{cases} \] with \begin{align*} \mathrm{KL}_{\inf}^{+}(q,m_o) &:=\inf_{p\in\mathcal{B}:~m(p)\ge m_o}\ \mathrm{KL}(q,p),\\ \mathrm{KL}_{\inf}^{-}(q,m_o) &:=\inf_{p\in\mathcal{B}:~m(p)\le m_o}\ \mathrm{KL}(q,p). \end{align*} Note that $\mathrm{KL}_{\inf}(q, \cdot)$ is an increasing function on $[m(q), 1]$ and decreasing on $[0, m(q)]$. \paragraph{Dual representation.} While the (primal) definition of $\mathrm{KL}_{\inf}$ involves an optimization over the space of probability measures, which can be computationally inconvenient, $\mathrm{KL}_{\inf}$ is, by now, a well-understood object. In particular, it has a dual formulation that provides a tractable and explicit characterization that is especially convenient for algorithmic and analytical purposes. In our proofs, we will move between the primal and dual, exploiting properties of both the problems. We therefore recall below the dual representations: \begin{align} \label{eq:kl_inf} \mathrm{KL}_{\inf}^{+}(q,m_o) &= \sup_{\lambda\in \left[0, \frac{1}{1-m_o} \right]}\! \ \mathbb{E}_q\left[\log\bigl(1-\lambda(X-m_o)\bigr)\right], \\[0.5ex] \nonumber \mathrm{KL}_{\inf}^{-}(q,m_o) &= \sup_{\lambda\in\left[-\frac{1}{m_o}, 0\right]} \ \mathbb{E}_q\left[\log\bigl(1-\lambda(X-m_o)\bigr)\right]. \end{align} \paragraph{Dual maximizers.} Recall from \citet{honda2010asymptotically} that there exists a unique dual maximizer in~\eqref{eq:kl_inf}. We denote by $\lambda^\star(q)$ the value of $\lambda$ that achieves the supremum in the definition of $\mathrm{KL}_{\inf}(q, m_0)$. If $m_o \ge m(q)$, then $\lambda^\ast(q)$ denotes the value of $\lambda$ that achieves the supremum in the definition of $\mathrm{KL}_{\inf}^{+}(q,m_o)$. Specifically, for \(m_o \geq m(q)\), \[ \lambda^\star(q)\in\arg\!\max_{\lambda\in\left[0,\frac{1}{1-m_o}\right]}\ \mathbb{E}_q\!\left[\log\bigl(1-\lambda(X-m_o)\bigr)\right]. \] Similarly, if $m_o < m(q)$, then $\lambda^\ast(q)$ denotes the corresponding maximizer in $\mathrm{KL}_{\inf}^{-}(q,m_o)$, and one can symmetrically define the maximizer for this case. Whenever it is clear from the context, we will drop the dependence of this maximizer on $q$, and instead refer to it as $\lambda^\ast$. \subsection{$\mathrm{KL}_{\inf}$- based Optimal Sequential Test} \label{sec:moderate_dev_bd} Now, we describe the optimal sequential test using the empirical-$\mathrm{KL}_{\inf}$ statistic, which is well known in the literature. Later, we will prove the CLT for the stopping time of this test. For $x\in [0,1]$ let $\delta_x$ denote a unit point mass at $x$. For $n\ge 1$, let \[ \hat q_n := \frac{1}{n}\sum_{i=1}^n \delta_{X_i} \] denote the empirical distribution. Define \begin{align} \label{eq:stopping_rule_bd} \tau_{\alpha} &= \inf\Bigl\{n\in\mathbb{N}:\ n\,\mathrm{KL}_{\inf}(\hat q_n,m_o)\ge \beta(n,\alpha)\Bigr\}, %\beta(n,\alpha) &= 1+ \log\left( \frac{2(1+n)}{\alpha}\right), \end{align} where $\beta(n,\alpha) = 1+ \log( \frac{2(1+n)}{\alpha})$. Using the martingale construction proposed in \citet[Lemma F.1]{agrawal2021optimal}, it is easy to see that $\tau_\alpha$ is an $\alpha$-correct stopping rule. Furthermore, for any $q \in \mathcal{Q}^{\rm bd}$, \begin{align*} \lim_{\alpha\to 0}\frac{\mathbb{E}_q[\tau_{\alpha}]}{\log(1/\alpha)} = \frac{1}{\mathrm{KL}_{\inf}(q,m_o)} . \end{align*} The above equality (lower bound + upper bound) was implicitly proven in the multi-armed bandit literature \citep{agrawal2020optimal,jourdan2022top}, and later, made explicit in \citet{agrawal2025stopping}. It is worth noting that the asymptotically optimal stopping rule $\tau_\alpha$ defined above tracks the empirical analogue \(n\,\mathrm{KL}_{\inf}(\hat q_n,m_o)\), which behaves like an accumulated evidences against \(H_0\), and compares it to a boundary $\beta(n,\alpha)$ to ensure \(\alpha\)-level type-I error control. Our goal in this paper is to study finer distributional properties of the associated stopping time $\tau_\alpha$. %In particular, in the following section, we aim to establish a CLT for $\tau_\alpha$ as $\alpha \downarrow 0$. \section{Main Results} \label{sec:results} In this section, we will establish a CLT for $\tau_\alpha$ as $\alpha \downarrow 0$. Towards this, we will first establish a CLT for the empirical \(\mathrm{KL}_{\inf}(\hat q_n,m_o)\) statistic, which is the key technical contribution of the paper and is of independent interest. For $I_{m_o} := [-1/m_o, 1/(1-m_o)]$, $\lambda \in I_{m_o}$, and $x\in [0,1]$, define \[ \ell(\lambda,x) := \log(1-\lambda(x-m_o)). \] Then, $(1-\lambda(x-m_o)) \ge 0$. Further, it is $0$ only if either $(\lambda, x) = (1/(1-m_o), 1)$ or $(\lambda, x) = (-1/m_o, 0)$. This observation will be useful in the proofs of our results. \citet{honda2010asymptotically} show that $\mathrm{KL}_{\inf}(\hat{q}_n, m_o) \to \mathrm{KL}_{\inf}(q,m_o)$ as $n\to \infty$ almost surely (as $\hat{q}_n \Rightarrow q$ almost surely, and $\mathrm{KL}_{\inf}(\cdot, m_o)$ is continuous in the weak topology). Thus, we will center the statistic around $\mathrm{KL}_{\inf}(q,m_o)$ in our CLT. We now state a minor technical condition. It excludes only distributions with sufficiently heavy mass near the boundary point $1$, while still allowing distributions that place positive probability mass at $1$. In Appendix~\ref{sec_ass}, we show that this assumption is mild and holds for several commonly studied bounded-support distributions, including Bernoulli distributions, the uniform distribution, and most Beta distributions. \begin{assumption} \label{ass_1} For $q\in\mathcal{B}$ and $m_o\in(0,1)$ with $m_o\neq m(q)$, assume the following: if $m_o > m(q)$ and $\mathbb{E}_q\!\left[(1-m_o)/(1-X)\right]=1$, then $\mathbb{E}_q\!\left[1/(1-X)^2\right]<\infty$; and if $m_o < m(q)$ and $\mathbb{E}_q\!\left[m_o/X\right]=1$, then $\mathbb{E}_q\!\left[1/X^2\right]<\infty$. \end{assumption} \begin{theorem} \label{lem:kl_inf_clt} Fix $q\in \mathcal{Q}^{\rm bd}$ and $m_o\in(0,1)$. Let $\hat q_n=\tfrac1n\sum_{i=1}^n \delta_{X_i}$ be the empirical distribution of i.i.d.\ $X_i\sim q$. Then, under Assumption \ref{ass_1}, \begin{align*} \sqrt{n}&\Big( \mathrm{KL}_{\inf}(\hat q_n,m_o) -\mathrm{KL}_{\inf}(q,m_o) \Big)\\ &\qquad \xRightarrow{d}\ \mathcal{N}\!\big(0,\sigma^2(q,m_o)\big), \end{align*} where \[ \sigma^2(q,m_o) =\operatorname{Var}_q\!\big(\ell(\lambda^\star,X)\big) <\infty . \] \end{theorem} As mentioned earlier, the main difficulty in analyzing the CLT for $\mathrm{KL}_{\inf}$ statistic is that it is defined via an optimization problem, which necessitates a case-by-case treatment depending on whether the dual maximizer lies in the interior or on the boundary. We present a proof sketch of this theorem in Section~\ref{sec:proof_sketch}, and defer the complete proof to Appendix~\ref{app:proof_lem:kl_inf_clt}. Next, to get the centering in the CLT for $\tau_\alpha$, we need to find its almost sure limit. To this end, we first prove a result which states that ${\tau_\alpha}/{\log(1/\alpha)}$ converges to ${1}/{\mathrm{KL}_{\inf}(q,m_o)}$ almost surely when alternate hypothesis is true. Next result formalizes this. \begin{lemma} \label{lemma_almost_sure} Fix $q\in\mathcal{Q}^{\rm bd}$. \begin{equation} \mathbb{P}_q \left(\lim_{\alpha\to 0}\frac{\tau_{\alpha}}{\log(1/\alpha)} = \frac{1}{\mathrm{KL}_{\inf}(q,m_o)} \right) = 1. \end{equation} \end{lemma} We next obtain the CLT for $\tau_\alpha$ using the above two results. \begin{theorem} \label{thm:clt_bd} Fix $q\in \mathcal{Q}^{\rm bd}$ and $m_o\in(0,1)$. Then, under Assumption \ref{ass_1}, the following holds, as $\alpha\downarrow 0$: \begin{align*} \sqrt{\log(1/\alpha)} &\left( \frac{\tau_{\alpha}}{\log(1/\alpha)}-\frac{1}{\mathrm{KL}_{\inf}(q,m_o)} \right)\\ &\qquad\xRightarrow{d}\mathcal{N}\!\big(0,\ \sigma^2_{\rm bd}(q,m_o)\big), \end{align*} where, \[ \sigma^2_{\rm bd}(q,m_o) = \frac{\operatorname{Var}_q(\ell(\lambda^\star,X))}{(\mathrm{KL}_{\inf}(q,m_o))^3}. \] \end{theorem} A complete and rigorous proof of this theorem is deferred to the Appendix. We give the intuition and key steps of the proof now. From Theorem~\ref{lem:kl_inf_clt}, we have CLT for $\sqrt{n}( \mathrm{KL}_{\inf}(\hat q_n,m_o) -\mathrm{KL}_{\inf}(q,m_o))$. To prove CLT for $\tau_\alpha$, we will next develop a CLT for the same statistic at the stopping time $\tau_\alpha$. This corresponds to proving that the following converges to a Gaussian limit as $\alpha\downarrow 0$: $$\sqrt{\tau_\alpha}\Big( \mathrm{KL}_{\inf}(\hat q_{\tau_\alpha},m_o) -\mathrm{KL}_{\inf}(q,m_o) \Big).$$ As is standard in literature, the proof relies on Anscombe's theorem \citep[Chapter 1]{gut2009stopped}. We verify Anscombe's condition in Lemma~\ref{lemm:anscombe}. This corresponds to showing that the process is uniformly continuous in probability. However, the usual argument, which combines i.i.d.\ increments with Kolmogorov's maximal inequality, does not apply here, since our statistic is defined through an optimization: $$\mathrm{KL}_{\inf}(\hat q_n,m_o) = \sup_{\lambda\in\bigl[0,\frac{1}{1-m_o}\bigr]} \mathbb{E}_{\hat q_n}\left[\log\!\bigl(1-\lambda(X-m_o)\bigr)\right],$$ when $m_o>m(\hat q_n)$. However, observe that, \begin{align*} \sqrt{n}\big( \mathrm{KL}_{\inf}(\hat q_n,m_o) -\mathrm{KL}_{\inf}(q,m_o) \big) = (T_{1,n}+T_{2,n}). \end{align*} where $T_{1,n}$ and $T_{2,n}$ are defined later in \eqref{eq_t1} and \eqref{eq_t2}, respectively. It is worth noting that \(T_{2,n}\) can be handled using Kolmogorov's maximal inequality to show uniform continuity in probability, since it is a partial-sum term with i.i.d.\ increments. For \(T_{1,n}\), we perform a Taylor expansion and exploit the first-order optimality conditions of the dual optimization problem to obtain the desired result. Once we have a CLT for \(\sqrt{\tau_\alpha}\big(\mathrm{KL}_{\inf}(\hat q_{\tau_\alpha},m_o)-\mathrm{KL}_{\inf}(q,m_o)\big)\), rest of the steps are standard in literature: we invert the relation via a delta-method argument to obtain a CLT for the properly rescaled stopping time. We conclude this section with a proof sketch for Theorem~\ref{lem:kl_inf_clt}. But before that, we introduce some notation. Define \[ \Phi(q,m_o) :=\mathbb{E}_q\!\left[\frac{1-m_o}{1-X}\right], \] where by convention, we let $\Phi(q,m_o)=+\infty$ if $q(\{1\})>0$. Next, for $\lambda \in I_{m_o}$, $x\in [0,1]$, and $$g(\lambda,x):=\dfrac{x-m_o}{1-\lambda(x-m_o)},$$ define $$\Psi(\lambda,q):=\mathbb{E}_q\!\bigl[g(\lambda,X)\bigr],$$ which denotes the negative derivative of $\mathbb{E}_q[\ell(\lambda, X)]$ with respect to $\lambda$. Then $(\lambda^\star, q)$ satisfy exactly one of the following cases. \begin{itemize} \item \textbf{Case 1:} If $\ \Phi(q,m_o)<1$, then the supremum in the dual problem is attained at the boundary, that is, $\ \lambda^\star=\bar\lambda:={1}/({1-m_o})$. \item \textbf{Case 2:} If $\ \Phi(q,m_o)>1$, then the maximizer $\ \lambda^\star<\bar\lambda:= {1}/({1-m_o})$ is the unique maximizer which lies in the interior of the dual feasible region, that is, it is the unique $\lambda$ that satisfies $\ \Psi(\lambda,q)=0$. \item \textbf{Case 3:} If $\ \Phi(q,m_o)=1$, then $\ \lambda^\star=\bar\lambda:= {1}/({1-m_o})$, and $\ \Psi(\lambda^\star,q)=0$. \end{itemize} In most of our proofs, we handle these cases separately. \subsection{Proof Sketch of Theorem~\ref{lem:kl_inf_clt}.}\label{sec:proof_sketch} Let $\lambda^\star_n:=\lambda^\star(\hat q_n)$ be the dual maximizer. Results for the two case --- $m_o>m(q)$ and $m_o0$. Further, \[ \Psi(\lambda, q) = \mathbb{E}_{q}\left[g(\lambda, X)\right] := \mathbb{E}_{q}\left[ \frac{X-m_o}{1-\lambda(X-m_o)} \right],\] which denotes the negative derivative of $\mathbb{E}_q[\ell(\lambda, X)]$ with respect to $\lambda$. Then, one of the following three cases occurs. \begin{itemize} \item \textbf{Case 1:} If $\ \Phi(q,m_o)<1$, then the supremum in the dual problem is attained at the boundry, that is, $\ \lambda^\star=\bar\lambda =\tfrac{1}{1-m_o}$. \item \textbf{Case 2:} If $\ \Phi(q,m_o)>1$, then the maximizer $\ \lambda^\star<\bar\lambda:= \tfrac{1}{1-m_o}$ is the unique maximizer which lies in the interior of the dual feasible region, that is, it is the unique $\lambda$ that satisfies $\ \Psi(\lambda,q)=0$. \item \textbf{Case 3:} If $\ \Phi(q,m_o)=1$, then $\ \lambda^\star=\bar\lambda = \tfrac{1}{1-m_o}$, and $\ \Psi(\lambda^\star,q)=0$. \end{itemize} \begin{remark} Under Case 2, the choice of $q$ and $m_o$ can lead to $\lambda^* = 0$. In this scenario, $\mathrm{KL}_{\inf}(q,m_o) = 0$, which holds if and only if $m(q) = m_o$ (see, for example, \citet[Appendix F]{jourdan2022top}) . However, all of the results in this paper assume that $q \in \mathcal{Q}^{\rm bd}$, which implies that $m(q) \neq m_o$. Hence, without loss of generality, we assume that under this case, when $\lambda^* < \frac{1}{1-m_o}$, it also holds that $\lambda^* > 0$. \end{remark} %\textcolor{red}{add some discussion on concavtiy of klinf etc} Our proofs often treat these cases separately. Finally, before presenting our proofs, we introduce certain notation which will be used in this Appendix. \paragraph{Notation} \begin{itemize} \item Let $g'(c,x)= \frac{\partial g(\lambda, x)}{\partial \lambda}|_{\lambda = c}$ and ${\Psi}'(c, q):=\left.\frac{\partial}{\partial \lambda}\Psi(\lambda,q)\right|_{\lambda=c}$. \item Convergence notation: $\xrightarrow[]{\rm a.s.}$ almost surely, $\xrightarrow{p}$ in probability, $\xRightarrow{d}$ in distribution. \item Stochastic order notation: $X_n = O_p(1)$ means $\{X_n\}$ is bounded in probability, i.e., for every $\epsilon > 0$ there exists $M < \infty$ such that $\sup_n \Pr(|X_n| > M) < \epsilon$. More generally, $X_n = O_p(a_n)$ means $X_n/a_n = O_p(1)$. We write $X_n = o_p(1)$ if $X_n \xrightarrow{p} 0$, and $X_n = o_p(a_n)$ if $X_n/a_n \xrightarrow{p} 0$. \end{itemize} \subsection{Proof of Theorem \ref{lem:kl_inf_clt}.}\label{app:proof_lem:kl_inf_clt} As mentioned earlier, we prove the theorem for the case when $m_o>m(q)$. The other case (when $m_o 0$ and infinitely differentiable for all $\lambda \in [0,\tfrac{1}{1-m_o}]$). In Case 2, we have from Lemma \ref{sup_lemma_case_2_combined} that on each sample path (for all realizations of $X_i \in [0,1]$), the dual optimizer $\lambda^\star_n$ lies in the interior for sufficiently large $n$. In particular, $ \lambda^\star_n \in [\lambda^\star - \eta, \lambda^\star + \eta]$ for sufficiently large $n$, where $\eta > 0$ is chosen such that $\lambda^\star + \eta < \frac{1}{1-m_o}$ and $\lambda^\star - \eta > 0$. Hence, $\ell(\lambda, X_i) > 0$ for all $i$ and for all $\lambda$ in this neighborhood containing $\lambda^\star$. Hence, using Taylor expansion of $Q_n(\cdot)$ around $\lambda^\star$ for large $n$ yields $$Q_n(\lambda^\star_n ) = \underbrace{Q_n(\lambda^\star)}_{=0} + (\lambda^\star_n-\lambda^\star) \frac{\partial Q_n(\lambda) }{\partial \lambda}\bigg|_{\lambda= c(X_1,X_2,\ldots X_n, \lambda^\star_n)}, $$ for some $c(X_1,\dots, X_n, \lambda^\star_n)$. For ease of notation, we let $c_n := c(X_1,X_2,\ldots X_n, \lambda^\star_n)$. Now, observe that $T_{1,n} = {Q_n(\lambda^\star_n ) }/{\sqrt{n}}$. Hence, \[ T_{1,n} = -\sqrt{n} (\lambda^\star_n-\lambda^\star) \cdot A_n, \qquad A_n:=\frac1n\sum_{i=1}^n g(c_n,X_i), \] for some (possibly random) $c_n$ between $\lambda^\star_n$ and $\lambda^\star$. Now, from Lemma \ref{sup_lemma_case_2_combined}, we have the following \[ \lambda^\star_n \xrightarrow[]{\rm a.s.} \lambda^\star \quad\text{and}\quad \sqrt{n}\,(\lambda^\star-\lambda^\star_n)=O_p(1), \] for cases 2 and 3, respectively. Further, Lemma \ref{sup_lemma_A_n} gives that $A_n \to 0$ almost surely. Hence, it follows that $T_{1,n}\xrightarrow{p}\ 0$ under Case 2 and Case 3. \noindent{\bf Step 3: Conclusion.} In both the cases, Case 2 and Case 3, $T_{1,n}\xrightarrow{p}\ 0$, while $T_{2,n}\xRightarrow{d} \mathcal{N}(0,\sigma^2(q,m_o))$. Thus, by Slutsky's theorem, \[ \sqrt{n}\Big(\mathrm{KL}_{\inf}(\hat{q}_n,m_o)-\mathrm{KL}_{\inf}(q,m_o)\Big) \ \xRightarrow{d}\ \mathcal{N}\big(0,\sigma^2(q,m_o)\big). \] This completes the proof. \end{proof} \subsection{Proof of Lemma \ref{lemma_almost_sure}} We prove the lemma for the case \(m_o > m(q)\); the proof for the case \(m_o < m(q)\) follows analogously. First we show that \begin{equation}\label{sup_eq_vik_1} \lim_{\alpha\to 0}\tau_\alpha = \infty \qquad \text{almost surely.} \end{equation} Recall the stopping time \[ \tau_\alpha =\inf\Bigl\{n\ge 1:\ n\,\mathrm{KL}_{\inf}(\hat q_n,m_o)\ \ge\ \beta(n,\alpha)\Bigr\}. \] Using \citet[Lemma~14]{honda2010asymptotically}, we have the uniform bound \begin{equation}\label{eq:klinf_bd} \mathrm{KL}_{\inf}(\hat q_n,m_o)\ \le\ \frac{1}{1-m_o}, \qquad \text{for all } n\ge 1 . \end{equation} Consequently, \begin{equation}\label{eq:nklinf_bd} n\,\mathrm{KL}_{\inf}(\hat q_n,m_o) \ \le\ \frac{n}{1-m_o}, \qquad \text{for all } n\ge 1 . \end{equation} Fix an arbitrary integer \(N\ge 1\). For all \(1\le n\le N\), \[ n\,\mathrm{KL}_{\inf}(\hat q_n,m_o) \ \le\ \frac{N}{1-m_o}. \] On the other hand, since \(\beta(n,\alpha)=O(\log(n/\alpha))\), we have \[ \lim_{\alpha \downarrow 0}\beta(n,\alpha)\ \to \infty \qquad \text{for each fixed } n . \] Hence, there exists \(\alpha_N\in(0,1)\) such that \begin{equation}\label{eq:beta_dom} \min_{1\le n\le N}\beta(n,\alpha)\ >\ \frac{N}{1-m_o}, \qquad \text{for all } \alpha\in(0,\alpha_N). \end{equation} Combining \eqref{eq:nklinf_bd} and \eqref{eq:beta_dom}, we obtain that for all \(\alpha\in(0,\alpha_N)\) and all \(1\le n\le N\), \[ n\,\mathrm{KL}_{\inf}(\hat q_n,m_o)\ <\ \beta(n,\alpha). \] Therefore, using the definition of $\tau_\alpha$, it follows that \[ \tau_\alpha > N \qquad \text{for all } \alpha\in(0,\alpha_N). \] Since \(N\ge 1\) was arbitrary, it follows that \[ \lim_{\alpha\to 0}\tau_\alpha=\infty \quad \text{almost surely}. \] Now we prove the statement of the lemma. Since $q\in\mathcal{Q}^{\rm bd}$, and the fact that this test has power one, following holds for any fixed $\alpha \in (0,1)$: $$\mathbb{P}_{q}(\tau_\alpha <\infty) = 1 \textrm{ almost surely.}$$ Using the above fact and the definition of $\tau_\alpha$, we get, \begin{equation} \label{sup_eq_vik_2} \frac{\beta(\tau_\alpha-1,\alpha)}{\log(1/\alpha)\,\mathrm{KL}_{\inf}(\hat q_{\tau_\alpha-1},m_o)} + \frac{1}{\log(1/\alpha)}\geq \frac{\tau_\alpha}{\log(1/\alpha)} \ \ge\ \frac{\beta(\tau_\alpha,\alpha)}{\log(1/\alpha)\,\mathrm{KL}_{\inf}(\hat q_{\tau_\alpha},m_o)}. \end{equation} First, we show that following claim holds: \noindent\emph{Claim : $\tau_\alpha=O(\log(1/\alpha))$ almost surely.} \noindent {Proof of the claim.} Fix $\epsilon\in\bigl(0,\mathrm{KL}_{\inf}(q,m_o)\bigr)$ and define the event \[ \Omega_\epsilon :=\Bigl\{\exists\,N_\epsilon(\omega)\ \text{s.t.}\ \mathrm{KL}_{\inf}(\hat q_n,m_o)\ \ge\ \mathrm{KL}_{\inf}(q,m_o)-\epsilon,\ \forall n\ge N_\epsilon(\omega)\Bigr\}. \] Further, fix $\delta\in\bigl(0,\mathrm{KL}_{\inf}(q,m_o)-\epsilon\bigr)$. Since $\log x=o(x)$ as $x\to\infty$, there exists $M_\delta<\infty$ such that \begin{equation} \label{extra} \log x \le \delta x,\qquad \forall\, x\ge M_\delta. \end{equation} Using Lemma \ref{lem:mom_conv_LnL}, we get $\mathrm{KL}_{\inf}(\hat q_n,m_o)\xrightarrow[]{a.s.} \mathrm{KL}_{\inf}(q,m_o)$. Hence, it follows that, we have $\mathbb P_q(\Omega_\epsilon)=1$. Fix $\omega\in\Omega_\epsilon$. Since $\tau_\alpha(\omega)\to\infty$ as $\alpha\downarrow 0$ (see \ \eqref{sup_eq_vik_1}), there exists $\alpha_0(\omega)\in(0,1)$ such that $\tau_\alpha(\omega)-1\ge \max{ \{N_\epsilon(\omega), M_\delta -1 \}}$ for all $\alpha\in(0,\alpha_0(\omega))$. Using the definition of $\tau_\alpha$, we get, \[ \big(\tau_\alpha(\omega)-1\big)\,\mathrm{KL}_{\inf}\big(\hat q_{\tau_\alpha(\omega)-1},m_o\big) \ <\ \beta(\tau_\alpha(\omega)-1,\alpha) \ =\ 1+\log\Bigl(\frac{2\tau_\alpha(\omega)}{\alpha}\Bigr). \] On $\Omega_\epsilon$ and for $\alpha<\alpha_0(\omega)$ we also have $\mathrm{KL}_{\inf}(\hat q_{\tau_\alpha(\omega)-1},m_o)\ge \mathrm{KL}_{\inf}(q,m_o)-\epsilon$, hence \begin{equation}\label{eq:tau_upper_intermediate_correct} \big(\tau_\alpha(\omega)-1\big)\big(\mathrm{KL}_{\inf}(q,m_o)-\epsilon\big) \ <\ 1+\log\Bigl(\frac{2}{\alpha}\Bigr)+\log\big(\tau_\alpha(\omega)\big). \end{equation} Using \eqref{extra} and definition of $\alpha_0(\omega)$, we get, \[ \big(\tau_\alpha(\omega)-1\big)\big(\mathrm{KL}_{\inf}(q,m_o)-\epsilon\big) \ <\ 1+\delta+\log\Bigl(\frac{2}{\alpha}\Bigr)+\delta\big(\tau_\alpha(\omega)-1\big). \] Rearranging gives \[ \big(\mathrm{KL}_{\inf}(q,m_o)-\epsilon-\delta\big)\big(\tau_\alpha(\omega)-1\big) \ <\ 1+\delta+\log\Bigl(\frac{2}{\alpha}\Bigr). \] Choose $\delta:=\tfrac{1}{2}\big(\mathrm{KL}_{\inf}(q,m_o)-\epsilon\big)$, so that $\mathrm{KL}_{\inf}(q,m_o)-\epsilon-\delta=\tfrac{1}{2}(\mathrm{KL}_{\inf}(q,m_o)-\epsilon)$. Then, for all $\alpha \in (0, \alpha_0(\omega))$ (depending on $\omega$), \[ \tau_\alpha(\omega) \ \le\ 1+\frac{2}{\mathrm{KL}_{\inf}(q,m_o)-\epsilon}\Bigl(1+\delta+\log\!\Bigl(\frac{2}{\alpha}\Bigr)\Bigr). \] Since $1+\delta$ is a finite constant and $\log(2/\alpha)=\log(1/\alpha)+\log 2$, this proves $\tau_\alpha=O(\log(1/\alpha))$ almost surely. Using the claim, \eqref{sup_eq_vik_1}, \eqref{sup_eq_vik_2} and the fact that $\mathrm{KL}_{\inf}(\hat q_{n},m_o) \to \mathrm{KL}_{\inf}(q,m_o)$ almost surely (see Lemma \ref{lem:mom_conv_LnL}), we get the desired result. This completes the proof. \iffalse We now show how the claim, \eqref{sup_eq_vik_1}, \eqref{sup_eq_vik_2}, and the almost sure convergence $\mathrm{KL}_{\inf}(\hat q_n, m_o)\to\mathrm{KL}_{\inf}(q,m_o)$ together yield the desired result \[ \frac{\tau_\alpha}{\log(1/\alpha)}\ \xrightarrow{\alpha\to 0}\ \frac{1}{\mathrm{KL}_{\inf}(q,m_o)} \qquad\text{almost surely.} \] \medskip \noindent\textbf{Step 1: Simplifying $\beta(n,\alpha)/\log(1/\alpha)$.} With $\beta(n,\alpha)=1+\log\!\bigl(\tfrac{2(1+n)}{\alpha}\bigr) =1+\log(2(1+n))+\log(1/\alpha)$, we have \[ \frac{\beta(n,\alpha)}{\log(1/\alpha)} \ =\ 1+\frac{1+\log(2(1+n))}{\log(1/\alpha)}. \] By the claim, $\tau_\alpha=O(\log(1/\alpha))$ a.s., so $\log(1+\tau_\alpha)=O(\log\log(1/\alpha))=o(\log(1/\alpha))$. Therefore, \begin{equation}\label{eq:beta_ratio_limit} \frac{\beta(\tau_\alpha,\alpha)}{\log(1/\alpha)}\ \longrightarrow\ 1 \qquad\text{a.s. as }\alpha\to 0, \end{equation} and the same holds with $\tau_\alpha-1$ in place of $\tau_\alpha$. \medskip \noindent\textbf{Step 2: Almost sure convergence of $\mathrm{KL}_{\inf}(\hat q_{\tau_\alpha},m_o)$.} Since $\mathrm{KL}_{\inf}(\hat q_n,m_o)\to\mathrm{KL}_{\inf}(q,m_o)$ almost surely and $\tau_\alpha\to\infty$ a.s.\ (cf.\ \eqref{sup_eq_vik_1}), by composition, \begin{equation}\label{eq:klinf_tau_limit} \mathrm{KL}_{\inf}(\hat q_{\tau_\alpha},m_o)\ \longrightarrow\ \mathrm{KL}_{\inf}(q,m_o) \qquad\text{a.s. as }\alpha\to 0, \end{equation} and likewise with $\tau_\alpha-1$. \medskip \noindent\textbf{Step 3: Squeeze argument via \eqref{sup_eq_vik_2}.} Dividing \eqref{sup_eq_vik_2} through by $\log(1/\alpha)$ gives \[ \frac{\beta(\tau_\alpha-1,\alpha)}{\log(1/\alpha)\,\mathrm{KL}_{\inf}(\hat q_{\tau_\alpha-1},m_o)} +\frac{1}{\log(1/\alpha)} \ \ge\ \frac{\tau_\alpha}{\log(1/\alpha)} \ \ge\ \frac{\beta(\tau_\alpha,\alpha)}{\log(1/\alpha)\,\mathrm{KL}_{\inf}(\hat q_{\tau_\alpha},m_o)}. \] Using \eqref{eq:beta_ratio_limit} and \eqref{eq:klinf_tau_limit}, the right-hand side satisfies \[ \frac{\beta(\tau_\alpha,\alpha)}{\log(1/\alpha)\,\mathrm{KL}_{\inf}(\hat q_{\tau_\alpha},m_o)} \ \longrightarrow\ \frac{1}{\mathrm{KL}_{\inf}(q,m_o)} \qquad\text{a.s.} \] For the left-hand side, the same reasoning gives \[ \frac{\beta(\tau_\alpha-1,\alpha)}{\log(1/\alpha)\,\mathrm{KL}_{\inf}(\hat q_{\tau_\alpha-1},m_o)} \ \longrightarrow\ \frac{1}{\mathrm{KL}_{\inf}(q,m_o)} \qquad\text{a.s.,} \] and $\tfrac{1}{\log(1/\alpha)}\to 0$, so the entire left-hand side also converges to $\tfrac{1}{\mathrm{KL}_{\inf}(q,m_o)}$ almost surely. By the squeeze theorem, \[ \frac{\tau_\alpha}{\log(1/\alpha)}\ \longrightarrow\ \frac{1}{\mathrm{KL}_{\inf}(q,m_o)} \qquad\text{almost surely as }\alpha\to 0. \] This completes the proof. \qed \fi \hfill$\Box$ \subsection{Proof of Theorem \ref{thm:clt_bd}} Fix $q\in\mathcal{Q}^{\rm bd}$. Recall that \[ \tau_\alpha =\inf\left\{ n:\ \mathrm{KL}_{\inf}(\hat q_n,m_o)\ge \frac{\beta(n,\alpha)}{n} \right\}. \] By definition of $\tau_\alpha$, \begin{equation}\label{eq:tau_basic_ineq_final} \mathrm{KL}_{\inf}(\hat q_{\tau_\alpha},m_o)\ \ge\ \frac{\beta(\tau_\alpha,\alpha)}{\tau_\alpha}, \qquad \mathrm{KL}_{\inf}(\hat q_{\tau_\alpha-1},m_o)\ <\ \frac{\beta(\tau_\alpha-1,\alpha)}{\tau_\alpha-1}. \end{equation} Equivalently, \begin{equation}\label{eq:tau_sandwich_final} \frac{\beta(\tau_\alpha,\alpha)} {\mathrm{KL}_{\inf}(\hat q_{\tau_\alpha},m_o)} \ \le\ \tau_\alpha \ <\ 1+\frac{\beta(\tau_\alpha-1,\alpha)} {\mathrm{KL}_{\inf}(\hat q_{\tau_\alpha-1},m_o)}. \end{equation} By Theorem~\ref{lem:kl_inf_clt}, we have \begin{equation}\label{eq:klinf_clt_det_final} \sqrt{n}\Big( \mathrm{KL}_{\inf}(\hat q_n,m_o)-\mathrm{KL}_{\inf}(q,m_o) \Big) \xRightarrow{d} \mathcal{N}\!\bigl(0,\sigma^2(q,m_o)\bigr). \end{equation} Let $b_\alpha=\log(1/\alpha)$. Since $\tau_\alpha/b_\alpha\to 1/\mathrm{KL}_{\inf}(q,m_o)$ almost surely by Lemma \ref{lemma_almost_sure}, and using Lemma \ref{lemm:anscombe}, we get that Anscombe's condition is satisfied and hence it follows that, using Anscombe's theorem, we have, \begin{equation}\label{eq:klinf_clt_tau_final} \sqrt{\tau_\alpha}\Big( \mathrm{KL}_{\inf}(\hat q_{\tau_\alpha},m_o)-\mathrm{KL}_{\inf}(q,m_o) \Big) \xRightarrow{d} \mathcal{N}\!\bigl(0,\sigma^2(q,m_o)\bigr). \end{equation} Using a similar argument as above for $\tau_\alpha -1$ instead of $\tau_\alpha$, we get, \begin{equation}\label{eq:klinf_clt_tau_final_2} \sqrt{\tau_\alpha -1 } \Big( \mathrm{KL}_{\inf}(\hat q_{\tau_\alpha-1},m_o)-\mathrm{KL}_{\inf}(q,m_o) \Big) \xRightarrow{d} \mathcal{N}\!\bigl(0,\sigma^2(q,m_o)\bigr). \end{equation} It is worth noting that the CLT for a test statistic evaluated at a random stopping time does not, in general, imply the CLT for it evaluated at a finite number of steps before that stopping time. However, in our setting, this follows trivially by applying Anscombe's theorem to both $\tau_\alpha$ and $\tau_\alpha-1$. Using Delta method, we get \[ \sqrt{\tau_\alpha}\left( \frac{1}{\mathrm{KL}_{\inf}(\hat q_{\tau_\alpha},m_o)} -\frac{1}{\mathrm{KL}_{\inf}(q,m_o)} \right) \xRightarrow{d} \mathcal{N}\!\left( 0,\ \frac{\sigma^2(q,m_o)}{\mathrm{KL}_{\inf}(q,m_o)^4} \right). \] \[ \sqrt{\tau_\alpha -1}\left( \frac{1}{\mathrm{KL}_{\inf}(\hat q_{\tau_\alpha -1},m_o)} -\frac{1}{\mathrm{KL}_{\inf}(q,m_o)} \right) \xRightarrow{d} \mathcal{N}\!\left( 0,\ \frac{\sigma^2(q,m_o)}{\mathrm{KL}_{\inf}(q,m_o)^4} \right). \] Also, by Lemma~\ref{lemma_almost_sure}, \[ \sqrt{\frac{b_\alpha}{\tau_\alpha}} \xrightarrow{p} \sqrt{\mathrm{KL}_{\inf}(q,m_o)}. \] Therefore, by Slutsky's theorem, we get, \begin{equation}\label{eq:reciprocal_scaled_final} \begin{gathered} \sqrt{b_\alpha}\left( \frac{1}{\mathrm{KL}_{\inf}(\hat q_{\tau_\alpha},m_o)} -\frac{1}{\mathrm{KL}_{\inf}(q,m_o)} \right) \xRightarrow{d} \mathcal{N}\!\left( 0,\ \frac{\sigma^2(q,m_o)}{\mathrm{KL}_{\inf}(q,m_o)^3} \right), \\ \sqrt{b_\alpha}\left( \frac{1}{\mathrm{KL}_{\inf}(\hat q_{\tau_\alpha-1},m_o)} -\frac{1}{\mathrm{KL}_{\inf}(q,m_o)} \right) \xRightarrow{d} \mathcal{N}\!\left( 0,\ \frac{\sigma^2(q,m_o)}{\mathrm{KL}_{\inf}(q,m_o)^3} \right). \end{gathered} \end{equation} It follows that \eqref{eq:tau_sandwich_final} can be re-written as, \[ \sqrt{b_\alpha}\, \frac{\beta(\tau_\alpha,\alpha)}{b_\alpha} \left( \frac{1}{\mathrm{KL}_{\inf}(\hat q_{\tau_\alpha},m_o)} - \frac{1}{\mathrm{KL}_{\inf}(q,m_o)} \right) \ \le\ \sqrt{b_\alpha} \left( \frac{\tau_\alpha}{b_\alpha} - \frac{1}{\mathrm{KL}_{\inf}(q,m_o)} \right) \] \[ <\ \frac{1}{\sqrt{b_\alpha}} + \sqrt{b_\alpha}\, \frac{\beta(\tau_\alpha-1,\alpha)}{b_\alpha} \left( \frac{1}{\mathrm{KL}_{\inf}(\hat q_{\tau_\alpha-1},m_o)} - \frac{1}{\mathrm{KL}_{\inf}(q,m_o)} \right). \] Using \eqref{eq:reciprocal_scaled_final}, the definition of $\beta(n,\alpha)$ and Lemma \ref{lemma_almost_sure}, we get the desired result. \hfill$\Box$ \subsection{Proof of Proposition \ref{lem:ci-tau}} We first prove the proposition for the case $m_o>m(q)$ (the case $m_o m(q)\); the proof for the case \(m_o < m(q)\) follows analogously. To show this, we show that the moment-generating function for the random variable $\ell(\lambda^\star, X)$, when $X\sim q$, is finite in a neighborhood containing $0$, and hence, all its moments are finite. In particular, we show the following for both $\theta = 1 $ and $\theta = -1$, $$\mathbb{E}_q[e^{\theta \ell(\lambda^\star, X)}] = \mathbb{E}_q[e^{\theta \log(1-\lambda^\star(X-m_o))}] < \infty.$$ %It follows that the above claim and using the property of moment generating function of $\log(1-\lambda^\star(X-m))$, we get the desired result. For $\theta = 1$, we have $$\mathbb{E}_q[e^{\theta \log(1-\lambda^\star(X-m_o))}] =(1- \lambda^\star(m(q)-m_o)). $$ Since $m_o > m(q)$ and $\lambda^\star\in [0, \frac{1}{1-m_o}]$, the term on the right hand side in the above equation is finite. %$$\mathbb{E}_q[e^{\theta \log(1-\lambda^\star(X-m_o))}] = \log(1- \lambda^\star(m(q)-m_o)) < \infty. $$ Finally, for $\theta = -1$, we have $$\mathbb{E}_q[e^{\theta \log(1-\lambda^\star(X-m_o))}] = \mathbb{E}_q \left[ \frac{1}{1-\lambda^\star(X-m_o)}\right] \le 1, $$ %Finally, recall that %$$ \mathbb{E}_q \left[ \frac{1}{1-\lambda^\star(X-m_o)}\right] \leq 1$$ where the last inequality follows since the middle expression in the set of inequalities above corresponds to the mass that the primal optimizer in $\mathrm{KL}_{\rm inf}(q,m_o)$ puts on the support of distribution $q$ (see \cite{agrawal2022bandits}). \end{proof} \begin{lemma} \label{lem:uniform_bounds_l_g_gprime} Let \(\mathcal R\) be either of the following sets: \[ \mathcal R = I_\epsilon\times[0,1], \qquad I_\epsilon:=[\lambda^\star-\epsilon,\lambda^\star+\epsilon] \subset(0,\bar\lambda), \quad \lambda^\star\in(0,\bar\lambda), \] for some \(\epsilon>0\), or \[ \mathcal R = [0,\bar\lambda]\times[0,1-\kappa], \qquad \kappa\in(0,1),\quad 1-\kappa>m_o . \] Then \(\ell(\lambda,x)\), \(g(\lambda,x)\), and \(g'(\lambda,x)\) are uniformly bounded on \(\mathcal R\). Moreover, they are uniformly Lipschitz in \(\lambda\) on \(\mathcal R\). \end{lemma} \begin{proof} First suppose \( \mathcal R=I_\epsilon\times[0,1]\), where \(\lambda^\star\in(0,\bar\lambda)\) and \(I_\epsilon=[\lambda^\star-\epsilon,\lambda^\star+\epsilon] \subset(0,\bar\lambda)\). Let, \[ m_\epsilon:=1-(\lambda^\star+\epsilon)(1-m_o)>0 . \] For all \((\lambda,x)\in \mathcal R\), \[ 1-\lambda(x-m_o)\ge \begin{cases} 1, & x\le m_o,\\[2pt] m_\epsilon, & x\ge m_o. \end{cases} \] Hence \(\ell\) is uniformly bounded on \(\mathcal R\). Moreover, \[ |g(\lambda,x)| \le B_\epsilon := \max\left\{m_o,\frac{1-m_o}{m_\epsilon}\right\}<\infty, \] and \[ 0\le g'(\lambda,x) \le L_\epsilon := \max\left\{m_o^2,\frac{(1-m_o)^2}{m_\epsilon^2}\right\}<\infty. \] Thus \(g\) and \(g'\) are uniformly bounded on \(\mathcal R\). Finally, \[ \frac{\partial \ell(\lambda,x)}{\partial \lambda} =-g(\lambda,x),\qquad \frac{\partial g(\lambda,x)}{\partial \lambda} =g'(\lambda,x), \] and \[ \left|\frac{\partial g'(\lambda,x)}{\partial \lambda}\right| = \frac{2|x-m_o|^3}{(1-\lambda(x-m_o))^3} \le \max\left\{2m_o^3,\frac{2(1-m_o)^3}{m_\epsilon^3}\right\}<\infty . \] The mean value theorem therefore gives the uniform Lipschitz property of \(\ell\), \(g\), and \(g'\) in \(\lambda\) on \(\mathcal R\). Next consider \( \mathcal R = [0,\bar\lambda]\times[0,1-\kappa]\). For all \((\lambda,x)\in\mathcal R\), \[ 1-\lambda(x-m_o)\ge \begin{cases} 1, & x\le m_o,\\[2pt] \dfrac{\kappa}{1-m_o}, & x\in[m_o,1-\kappa]. \end{cases} \] Hence \(\ell\) is uniformly bounded on \(\mathcal R\). Also, \[ |g(\lambda,x)| \le \max\left\{ m_o,\frac{(1-\kappa-m_o)(1-m_o)}{\kappa} \right\}<\infty, \] and \[ 0\le g'(\lambda,x) \le K_\kappa := \max\left\{ m_o^2, \frac{(1-\kappa-m_o)^2}{\bigl(\kappa/(1-m_o)\bigr)^2} \right\}<\infty . \] Thus \(g\) and \(g'\) are uniformly bounded on \(\mathcal R\). Finally, \[ \frac{\partial \ell(\lambda,x)}{\partial \lambda} =-g(\lambda,x),\qquad \frac{\partial g(\lambda,x)}{\partial \lambda} =g'(\lambda,x), \] and \[ \left|\frac{\partial g'(\lambda,x)}{\partial \lambda}\right| = \frac{2|x-m_o|^3}{(1-\lambda(x-m_o))^3} \le \max\left\{ 2m_o^3, \frac{2(1-\kappa-m_o)^3}{\bigl(\kappa/(1-m_o)\bigr)^3} \right\}<\infty . \] The mean value theorem gives the uniform Lipschitz property of \(\ell\), \(g\), and \(g'\) in \(\lambda\) on \(\mathcal R\). \end{proof} \begin{lemma}\label{lem:mom_conv_LnL} Let $q \in \mathcal{Q}^{\rm bd}$. For $k\in\{1,2\}$, \begin{equation}\label{eq:mom_goal_LnL} \frac1n\sum_{i=1}^n \ell(\lambda^\star_n,X_i)^k \;\xrightarrow[]{\rm a.s.}\; \mathbb{E}_q\!\big[\ell(\lambda^\star,X)^k\big] <\infty. \end{equation} In particular, taking $k=1$ gives $\mathrm{KL}_{\inf}(\hat q_n,m_o)\xrightarrow[]{\rm a.s.}\mathrm{KL}_{\inf}(q,m_o)$. \end{lemma} \begin{proof} We prove the lemma only for the case $m_o>m(q)$ (the case $m_o0$ with $I_\epsilon =[\lambda^\star-\epsilon,\lambda^\star+\epsilon]\subset(0,\bar\lambda)$. First observe that, using Lemma \ref{lem:uniform_bounds_l_g_gprime}, $\ell$ is bounded on $I_\epsilon \times[0,1]$ by some $C_\epsilon<\infty$. By Lemma \ref{sup_lemma_case_2_combined}, $\lambda^\star_n\in I_\epsilon$ eventually almost surely. On that full-measure event, using Lemma \ref{sup_lemma_case_2_combined} and Lemma \ref{lem:uniform_bounds_l_g_gprime}, we get, \[ A_{n,1} \le \frac1n\sum_{i=1}^n\big|\ell(\lambda^\star_n,X_i)-\ell(\lambda^\star,X_i)\big| \le B_\epsilon |\lambda^\star_n-\lambda^\star| \xrightarrow[]{\rm a.s.} 0, \] and, using $|u^2-v^2|\le(|u|+|v|)|u-v|$ together with boundedness of $\ell$, \[ A_{n,2}\;\le\;2C_\epsilon B_\epsilon\,|\lambda^\star_n-\lambda^\star|\;\xrightarrow[]{\rm a.s.}\;0. \] \noindent{\bf Step 3: $A_{n,k}\xrightarrow[]{\rm a.s.}0$ when $\lambda^\star=\bar\lambda$.} Fix $\kappa \in(0,1)$ such that $1-\kappa > m_o$ (since $m_o \in (0,1)$) and split $A_{n,k}\le A_{n,k}^{(1)}(\kappa)+A_{n,k}^{(2)}(\kappa)$, where \[ A_{n,k}^{(1)}(\kappa):=\frac1n\sum_{i=1}^n\big|\ell(\lambda^\star_n,X_i)^k-\ell(\bar\lambda,X_i)^k\big| \mathbf 1_{\{X_i\le 1-\kappa\}}, \] \[ A_{n,k}^{(2)}(\kappa):=\frac1n\sum_{i=1}^n\big|\ell(\lambda^\star_n,X_i)^k-\ell(\bar\lambda,X_i)^k\big| \mathbf 1_{\{X_i>1-\kappa\}}. \] \emph{Bounding $A^{(1)}_{n,k}(\kappa)$.} For $(\lambda,x)\in [0,\bar\lambda]\times[0,1-\kappa]$, using Lemma \ref{lem:uniform_bounds_l_g_gprime}, we get that $\ell$ is uniformly bounded and uniformly Lipschitz in $\lambda$ over $x \in [0, 1-\kappa]$, with constants depending only on $\kappa$ and $m_o$. We also know that $\lambda^\star_n\to\bar\lambda$ a.s. (see Lemma \ref{sup_lemma_case_2_combined}). Hence, using arguments similar to Step 2, for each fixed $\kappa$ and for $k\in\{1,2\}$, we get \[ A_{n,k}^{(1)}(\kappa)\;\xrightarrow[]{\rm a.s.}\;0. \] \emph{Bounding $A^{(2)}_{n,k}(\kappa)$.} For each $x\in[0,1)$, observe that, $\frac{\partial}{\partial \lambda}\ell(\lambda,x) = -\frac{x-m_o}{1-\lambda(x-m_o)}.$ Hence \(\lambda\mapsto\ell(\lambda,x)\) is increasing if \(xm_o\), and identically zero if \(x=m_o\). Since \(\ell(0,x)=0\), it follows that \(\ell(\lambda,x)\) lies between \(0\) and \(\ell(\bar\lambda,x)\) for all \(\lambda\in[0,\bar\lambda]\). Therefore \[ |\ell(\lambda,x)|\le|\ell(\bar\lambda,x)|, \qquad \lambda\in[0,\bar\lambda]. \] Therefore for $k\in \{1,2\}$, \[ \big|\ell(\lambda^\star_n,x)^k-\ell(\bar\lambda,x)^k\big| \le |\ell(\lambda^\star_n,x)|^k+|\ell(\bar\lambda,x)|^k \le 2|\ell(\bar\lambda,x)|^k, \] whence \[ A_{n,k}^{(2)}(\kappa) \le \frac{2}{n}\sum_{i=1}^n|\ell(\bar\lambda,X_i)|^k \mathbf 1_{\{X_i>1-\kappa\}}. \] By the SLLN, for each fixed $\kappa$, \[ \frac1n\sum_{i=1}^n|\ell(\bar\lambda,X_i)|^k \mathbf 1_{\{X_i>1-\kappa\}} \xrightarrow[]{\rm a.s.} g_k(\kappa):=\mathbb{E}_q\Big[|\ell(\bar\lambda,X)|^k \mathbf 1_{\{X>1-\kappa\}}\Big]. \] \emph{Conclusion.} Fix any sequence $\kappa_m\downarrow 0$ as $m\to \infty$. Since the countable union of null sets is null, there exists a full-measure event $\Omega_0$ on which, simultaneously for every $m\ge 1$, \[ A_{n,k}^{(1)}(\kappa_m)\to 0 \quad\text{and}\quad \frac1n\sum_{i=1}^n|\ell(\bar\lambda,X_i)|^k \mathbf 1_{\{X_i>1-\kappa_m\}}\to g_k(\kappa_m). \] On $\Omega_0$, for every $m$, \[ \limsup_{n\to\infty} A_{n,k} \le\limsup_{n\to\infty} A_{n,k}^{(1)}(\kappa_m)+\limsup_{n\to\infty} A_{n,k}^{(2)}(\kappa_m) \le 2 g_k(\kappa_m). \] Since $\mathbf 1_{\{X>1-\kappa_m\}}\downarrow 0$ pointwise and $|\ell(\bar\lambda, X)|^k$ is integrable (Lemma~\ref{lem:finiteMGF}), dominated convergence gives $g_k(\kappa_m)\downarrow 0$. Letting $m\to\infty$ yields $\limsup_n A_{n,k}=0$ on $\Omega_0$, i.e.\ $A_{n,k}\xrightarrow[]{\rm a.s.}0$. Combining Steps 1, 2, and 3, gives \eqref{eq:mom_goal_LnL}, completing the proof. \end{proof} \begin{lemma}\label{sup_lemma_case_2_combined}\label{sup_lemma_case_3_combined} Assume one of the following holds: \begin{enumerate} \item[(i)] $\Phi(q,m_o)>1$. \item[(ii)] $\Phi(q,m_o)=1$ and $\mathbb{E}_q\left[{1}/{(1-X)^2}\right] < \infty$. \end{enumerate} Then $\lambda^\star_n \xrightarrow[]{\rm a.s.} \lambda^\star$. Moreover, in case (i), \[ \sqrt{n}\big(\lambda^\star_n-\lambda^\star\big) \xRightarrow{d}\ \mathcal{N}\left(0, \frac{\operatorname{Var}_q\big(g(\lambda^\star,X)\big)}{\Psi'(\lambda^\star,q)^2}\right), \] while in case (ii), \[ \sqrt{n} \big(\lambda^\star-\lambda^\star_n\big) \xRightarrow{d} \frac{Z_+}{\Psi'(\lambda^\star,q)},\] where $Z\sim\mathcal{N}(0,\ \operatorname{Var}_q(g(\lambda^\star,X)))$ and $Z_+:=\max\{Z,0\}$. %\[Z\sim\mathcal{N}\big(0,\ \operatorname{Var}_q\big(g(\lambda^\star,X)\big)\big), \quad \text{ and }\quad Z_+:=\max\{Z,0\}.\] \end{lemma} \begin{proof} First we prove the result for the case when $\Phi(q,m_o)>1$. \noindent \textbf{Part 1: $\Phi(q,m_o)>1$}. We proceed in three steps. \medskip \noindent \textbf{Step 1: Almost sure consistency of $\lambda_n^\star$.} Recall when $\Phi(q,m_o)>1$, then there exists a unique $\lambda^\star\in(0,\bar\lambda)$ such that, $\Psi(\lambda^\star,q)= 0.$ Pick $\epsilon>0$ with $I_\epsilon =[\lambda^\star-\epsilon,\lambda^\star+\epsilon]\subset(0,\bar\lambda)$. Using Lemma \ref{lem:uniform_bounds_l_g_gprime}, we get, \[ |g(\lambda,x)-g(\lambda',x)|\le L_\epsilon\,|\lambda-\lambda'|\quad\text{for all }\lambda,\lambda'\in I_\epsilon. \] Now, fix $\eta>0$ and take a finite $\eta/(4L_\epsilon)$-net $\{\lambda_j\}_{j=1}^J$ of $I_\epsilon$. Since $g(\lambda_j,\cdot)$ is bounded, the SLLN gives $\Psi(\lambda_j,\hat q_n)\to \Psi(\lambda_j,q)$ a.s.\ for each $j$; thus \[ \max_{j\le J}|\Psi(\lambda_j,\hat q_n)-\Psi(\lambda_j, q)|\le \eta/2\quad\text{for all large $n$ a.s.} \] For any $\lambda\in I_\epsilon$, pick $j$ with $|\lambda-\lambda_j|\le \eta/(4L_\epsilon)$; then \begin{align*} \big|\Psi(\lambda,\hat q_n)-\Psi(\lambda,q)\big| &\le \int |g(\lambda,x)-g(\lambda_j,x)|\, d\hat q_n(x) + \big|\Psi(\lambda_j,\hat q_n)-\Psi(\lambda_j,q)\big| + \int |g(\lambda,x)-g(\lambda_j,x)|\, dq(x) \\ &\le L_\epsilon |\lambda-\lambda_j| + \frac{\eta}{2} + L_\epsilon |\lambda-\lambda_j| \\ &\le 2 L_\epsilon \, |\lambda-\lambda_j| + \frac{\eta}{2} \\ &\le 2 L_\epsilon \cdot \frac{\eta}{4L_\epsilon} + \frac{\eta}{2} \\ &= \eta. \end{align*} Hence, \begin{equation} \label{ulln1} \sup_{\lambda\in I_\epsilon} \big|\Psi(\lambda,\hat q_n)-\Psi(\lambda,q)\big| \xrightarrow[]{\rm a.s.}0. \tag{ULLN-1} \end{equation} Using Lemma \ref{lem:uniform_bounds_l_g_gprime}, we know that, $g(\lambda, x)$ and $g'(\lambda, x)$ are bounded on $I_\epsilon\times[0,1]$, Hence, by dominated convergence theorem, we get, \[ \Psi'(\lambda,q)=E_q[g'(\lambda,X)], \quad \lambda\in I_\epsilon. \] Using the similar arguments as above, we obtain the uniform LLN \[ \sup_{\lambda\in I_\epsilon} \big|\Psi'(\lambda,\hat q_n)-\Psi'(\lambda,q)\big| \xrightarrow[]{\rm a.s.}0. \tag{ULLN-2} \] Since \[ \Psi'(\lambda,q) =\mathbb{E}_q\!\left[\frac{(X-m_o)^2}{(1-\lambda(X-m_o))^2}\right] > 0 \quad\text{on }I_\epsilon, \] the map $\Psi(\cdot,q)$ is strictly increasing on $I_\epsilon$ and we know that $\Psi(\lambda^\star,q) = 0$. Thus there exists $\upsilon_\epsilon \in(0,\epsilon)$ such that \[ \Psi(\lambda^\star- \upsilon_\epsilon,q)<0<\Psi(\lambda^\star+\upsilon_\epsilon,q). \] By (ULLN-1), for a given sample path, for all large $n$, \[ \Psi(\lambda^\star-\upsilon_\epsilon,\hat q_n)<0<\Psi(\lambda^\star+\upsilon_\epsilon,\hat q_n). \] By (ULLN-2), for a given sample path, for all large $n$, there exists $c_\epsilon>0$ such that \[ \inf_{\lambda\in I_\epsilon}\Psi'(\lambda,\hat q_n)\ge c_\epsilon>0, \] so $\Psi(\cdot,\hat q_n)$ is eventually strictly increasing on $I_\epsilon$ on each sample path. For the case $\Phi(q,m_o)>1$, on each sample path, for all large $n$, we also have $\Phi(\hat{q}_n,m_o)>1$ by the SLLN, so $\Psi(\lambda^\star_n,\hat q_n)=0$ for all large $n$. The intermediate value theorem then yields, on each sample path, for all large $n$, \[ \lambda^\star_n\in(\lambda^\star-\upsilon_\epsilon,\lambda^\star+\upsilon_\epsilon) \quad\text{with}\quad \Psi(\lambda^\star_n,\hat q_n)=0. \] Letting $\epsilon \downarrow 0$ yields $\upsilon_\epsilon \downarrow 0$. Hence, it follows that, \[ \lambda^\star_n\xrightarrow[]{\rm a.s.}\lambda^\star. \] \noindent \textbf{Step 2: CLT for $\lambda^*_n$.} We now prove the second part of the Lemma. Fix $\epsilon>0$ small. Since $\lambda\mapsto\Psi(\lambda,\hat q_n)$ is continuously differentiable on $I_\epsilon$, and $\lambda^\star_n\xrightarrow[]{\rm a.s.}\lambda^\star$, it follows that $\lambda^\star_n\in I_\epsilon$ eventually almost surely. Applying the Mean Value Theorem to $\lambda\mapsto\Psi(\lambda,\hat q_n)$ along a fixed sample path, for all sufficiently large $n$ there exists \[ c_n \in \big( \min\{\lambda^{\star},\lambda^{\star}_{n}\},\ \max\{\lambda^{\star},\lambda^{\star}_{n}\} \big) \] such that \[ \Psi(\lambda^{\star}_{n},\hat q_n) = \Psi(\lambda^{\star},\hat q_n) + (\lambda^{\star}_{n}-\lambda^{\star})\Psi'(c_n,\hat q_n). \] Rearranging and multiplying by $\sqrt{n}$ gives \begin{equation}\label{eq:one-step} \sqrt{n}\big(\lambda^{\star}_{n}-\lambda^{\star}\big) = -\,\frac{\sqrt{n}\big(\Psi(\lambda^{\star},\hat q_n)-\Psi(\lambda^{\star},q)\big)} {\Psi'(c_n,\hat q_n)} + \frac{\sqrt{n}\,\Psi(\lambda^{\star}_{n},\hat q_n)} {\Psi'(c_n,\hat q_n)}, \qquad \text{since }\Psi(\lambda^{\star},q)=0. \end{equation} \medskip \emph{First term in \eqref{eq:one-step}.} Using Lemma \ref{lem:uniform_bounds_l_g_gprime}, for all $(\lambda,x)\in I_\epsilon\times[0,1]$, $|g(\lambda,x)| \leq B_\epsilon <\infty$. Since, we know that by definition $\lambda^\star \in I_\epsilon$, hence it following using classical CLT, \begin{equation}\label{eq:one-step_4} \sqrt{n}\big(\Psi(\lambda^{\star},\hat q_n)-\Psi(\lambda^{\star},q)\big) = \sqrt{n}\Big(\frac{1}{n}\sum_{i=1}^n g(\lambda^{\star},X_i) -\mathbb{E}_q[g(\lambda^{\star},X)]\Big) \ \xRightarrow{d}\ \mathcal{N}(0,\operatorname{Var}_q(g(\lambda^{\star},X))), \end{equation} where $\operatorname{Var}_q(g(\lambda^{\star},X)) < \infty$. \medskip \emph{Denominator in \eqref{eq:one-step}.} We show that \[ \Psi'(c_n,\hat q_n) \ \xrightarrow[]{\rm a.s.}\ \Psi'(\lambda^{\star},q)\in(0,\infty). \] Decompose: \[ \Psi'(c_n,\hat q_n)-\Psi'(\lambda^\star,q) = \big(\Psi'(c_n,\hat q_n)-\Psi'(c_n,q)\big) + \big(\Psi'(c_n,q)-\Psi'(\lambda^\star,q)\big). \] For the first term, the uniform LLN (ULLN-2) implies \[ \sup_{c\in I_\epsilon} \left| \Psi'(c,\hat q_n)-\Psi'(c,q) \right| \ \xrightarrow[]{\rm a.s.}\ 0. \] Since $c_n\to\lambda^{\star}$ a.s., this term converges to $0$ a.s. For the second term, note that \[ g'(\lambda,x)=\frac{(x-m_o)^2}{(1-\lambda(x-m_o))^2} \] is continuous in $\lambda$ and uniformly bounded by $L_\epsilon$ on $I_\epsilon\times[0,1]$. Thus, by dominated convergence, \[ \Psi'(c_n,q)\to \Psi'(\lambda^{\star},q) \quad\text{a.s.} \] Therefore, \begin{equation}\label{eq:one-step_3} \Psi'(c_n,\hat q_n) \ \xrightarrow[]{\rm a.s.}\ \Psi'(\lambda^{\star},q). \end{equation} %Using Appendix \ref{appen_kl} \textcolor{red}{[[KL property]]}, we get that $\Psi'(\lambda^{\star},q)\in(0,\infty)$. Using the definition of $\Psi(\lambda^{\star},q)$, we get that $\Psi'(\lambda^{\star},q)\in(0,\infty)$. \medskip \emph{Second term in \eqref{eq:one-step}.} Recall from step 1 of this proof, on each sample path, for large $n$, $\Phi(\hat q_n,m_o)>1$, hence \[ \Psi(\lambda^{\star}_{n},\hat q_n)=0. \] Therefore, by \eqref{eq:one-step_3}, \begin{equation}\label{eq:one-step_2} \frac{\sqrt{n}\,\Psi(\lambda^{\star}_{n},\hat q_n)} {\Psi'(c_n,\hat q_n)} \ \longrightarrow\ 0 \quad\text{a.s.} \end{equation} \medskip \noindent \textbf{Step 3: Conclusion.} Combining \eqref{eq:one-step_4}, \eqref{eq:one-step_3}, and \eqref{eq:one-step_2} and applying Slutsky’s theorem yields \[ \sqrt{n}\big(\lambda^{\star}_{n}-\lambda^{\star}\big) \ \xRightarrow{d}\ \mathcal{N}\!\left(0,\frac{\operatorname{Var}_q(g(\lambda^{\star},X))}{\Psi'(\lambda^{\star},q)^2}\right). \] This completes the proof under the case when $\Phi(q,m_o)>1$. \bigskip Now, we prove the results for the case when $\Phi(q,m_o)=1$. \noindent \textbf{Part 2: $\Phi(q,m_o)=1$}. We proceed in five steps. \medskip \noindent \textbf{Step 1: Almost sure consistency of $\lambda_n^\star$.} Fix any $\epsilon\in(0,\lambda^\star)$. Define \[ L(\lambda):=\mathbb{E}_q[\ell(\lambda,X)]. \] Observe that $L$ is continuous and increasing on $[0,\bar\lambda]$ and has a unique maximizer at $\lambda^\star$ (hence $L(\lambda^\star)>L(\lambda^\star-\epsilon)$), there exists $\eta>0$ such that \begin{equation} \label{eq:suppp} L(\lambda^\star)-L(\lambda^\star-\epsilon)\ge 4\eta>0. \end{equation} Moreover, on the compact interval $[0,\lambda^\star-\epsilon]$, the function $\ell(\lambda,x)$ is bounded and continuously differentiable in $\lambda$ for every $x\in[0,1]$. Hence, by the same arguments used to establish \ref{ulln1}, we get, \[ \sup_{\lambda\in[0,\lambda^\star-\epsilon]} \Big|\mathbb{E}_{\hat q_n}[\ell(\lambda,X)]-L(\lambda)\Big| \xrightarrow{\mathrm{a.s.}}0. \] In addition, using SLLN we have, \[ \mathbb{E}_{\hat q_n}[\ell(\lambda^\star,X)]\xrightarrow{\mathrm{a.s.}} L(\lambda^\star). \] Hence, for all sufficiently large $n$ (a.s.), \begin{align*} \mathbb{E}_{\hat q_n}[\ell(\lambda^\star,X)] &\ge L(\lambda^\star)-\eta,\\ \sup_{\lambda\in[0,\lambda^\star-\epsilon]}\mathbb{E}_{\hat q_n}[\ell(\lambda,X)] &\le \sup_{\lambda\in[0,\lambda^\star-\epsilon]}L(\lambda)+\eta = L(\lambda^\star -\epsilon)+\eta, \end{align*} where the last equality uses that $L$ is increasing on $[0,\lambda^\star]$. Combining the last two displays with \eqref{eq:suppp} yields, for all sufficiently large $n$ (a.s.), \[ \mathbb{E}_{\hat q_n}[\ell(\lambda^\star,X)] \;\ge\; \sup_{\lambda\in[0,\lambda^\star-\epsilon]}\mathbb{E}_{\hat q_n}[\ell(\lambda,X)] \;\ge\; \mathbb{E}_{\hat q_n}[\ell(\lambda,X)] \qquad \forall\,\lambda\in[0,\lambda^\star-\epsilon]. \] Hence, it follows that any maximizer $\lambda_n^\star$ must satisfy $\lambda_n^\star>\lambda^\star-\epsilon$. As $\epsilon\downarrow 0$ and always $\lambda_n^\star\le \lambda^\star$, this gives $\lambda_n^\star\xrightarrow[]{\rm a.s.}\lambda^\star$. \medskip \noindent\textbf{Step 2: Taylor series expansion.} Consider the following function: \[ N(X_1,X_2,\ldots X_n, \lambda^\star_n ) = \sum_{i=1}^n \big(g(\lambda^\star_n,X_i)-g(\lambda^\star,X_i)\big). \] Since $\Phi(q,m_o)=1$, it follows that $q\{1\}=0$, and hence we may restrict the analysis to the measure-one event $\{X_i\in[0,1)\ \forall\, i=1,2,3,\ldots\}$. Using the definition of $g(\lambda,X)$, $N(X_1,X_2,\ldots X_n, \lambda^\star_n )$ is continuously differentiable in $\lambda^\star_n$ for any values of $X_1,X_2,\ldots X_n \in [0,1)$ and any value of $\lambda^\star_n \in \left[0, \frac{1}{1-m_o}\right]$. Hence, using Taylor expansion of $N(X_1,X_2,\ldots X_n, \lambda^\star_n )$ in $\lambda^\star_n$ around $\lambda^\star$ yields \[ N(X_1,X_2,\ldots X_n, \lambda^\star_n ) = (\lambda^\star_n-\lambda^\star) \frac{\partial N(X_1,X_2,\ldots X_n, \lambda ) }{\partial \lambda}\bigg|_{\lambda= c(X_1,X_2,\ldots X_n, \lambda^\star_n)}. \] For ease of notation, let $c_n = c(X_1,X_2,\ldots X_n, \lambda^\star_n)$. Observe that \[ \Psi(\lambda^\star_n,\hat q_n) - \Psi(\lambda^\star,\hat q_n) = \frac{N(X_1,X_2,\ldots X_n, \lambda^\star_n ) }{n}. \] Hence, it follows that following holds: \[ \Psi(\lambda^\star_n,\hat q_n) - \Psi(\lambda^\star,\hat q_n) = \Psi'(c_n,\hat q_n)\,(\lambda^\star_n- \lambda^\star), \] for some $c_n$ between $\lambda^\star_n$ and $\lambda^\star$. Observe that, for a given $X_1,X_2,\ldots X_n \in [0,1)$, $\mathbb{E}_{\hat q_n}[\ell(\lambda,X)]$ is continuously differentiable in $\lambda$ for $\lambda \in \left[0, \frac{1}{1-m_o}\right]$. Since $\lambda^\star_n$ maximizes $\mathbb{E}_{\hat q_n}[\ell(\lambda,X)]$ on $(0,\lambda^\star]$ and this objective is concave in $\lambda$, we have the usual first-order optimality conditions: \begin{itemize} \item If $\lambda^\star_n\in(0,\lambda^\star)$, then \[ \frac{\partial}{\partial\lambda}\mathbb{E}_{\hat q_n}[\ell(\lambda,X)]\Big|_{\lambda=\lambda^\star_n} =-\Psi(\lambda^\star_n,\hat q_n)=0, \] and monotonicity in $\lambda$ gives $\Psi(\lambda^\star,\hat q_n)\ge 0$. \item If $\lambda^\star_n=\lambda^\star$, we have, \[ \frac{\partial}{\partial\lambda}\mathbb{E}_{\hat q_n}[\ell(\lambda,X)]\Big|_{\lambda=\lambda^\star} \ge 0, \quad\text{i.e. }-\Psi(\lambda^\star,\hat q_n)\ge 0 \ \implies \Psi(\lambda^\star,\hat q_n)\le 0. \] \end{itemize} Thus, when $\lambda^\star_n\in(0,\lambda^\star)$, \[ \lambda^\star-\lambda^\star_n = \frac{\Psi(\lambda^\star,\hat q_n)}{\Psi'(c_n,\hat q_n)}, \qquad \Psi(\lambda^\star,\hat q_n)\ge 0, \] whereas when $\lambda^\star_n=\lambda^\star$, we trivially have \[ \lambda^\star-\lambda^\star_n=0 \qquad\text{and}\qquad \Psi(\lambda^\star,\hat q_n)\le 0 . \] Hence, following holds: \begin{equation}\label{eq:bdry-onesided-final-psi} \lambda^\star-\lambda^\star_n = \frac{\big[\Psi(\lambda^\star,\hat q_n)\big]_+}{\Psi'(c_n,\hat q_n)}, \qquad [x]_+:=\max\{x,0\}. \end{equation} \medskip \noindent\textbf{Step 3: Convergence of the denominator in \eqref{eq:bdry-onesided-final-psi}.} Fix $\kappa\in(0,1)$ and decompose \begin{align*} \Psi'(c_n,\hat q_n)- \mathbb{E}_q[g'(\lambda^\star,X)] &=\Big(\mathbb{E}_{\hat q_n}[g'(c_n,X)]-\mathbb{E}_q[g'(c_n,X)]\Big) +\Big(\mathbb{E}_q[g'(c_n,X)]-\mathbb{E}_q[g'(\lambda^\star,X)]\Big)\\ &=:A_n+B_n. \end{align*} \smallskip \noindent\textbf{Step 3.1: $B_n\to 0$ a.s.} Since $c_n\to\lambda^\star$ a.s. and for each $x<1$ the map $\lambda\mapsto g'(\lambda,x)$ is continuous, we have $g'(c_n,x)\to g'(\lambda^\star,x)$ pointwise for all $x<1$. Moreover, for any $\lambda \in [0, \frac{1}{1-m_o}]$ and $x\in[0,1)$, \[ 0\le g'(\lambda,x)=\frac{(x-m_o)^2}{(1-\lambda(x-m_o))^2} \le (1-m_o)^2\frac{1}{(1-x)^2}. \] The function $(1-m_o)^2(1-x)^{-2}$ is integrable under $\mathbb{E}_q[(1-X)^{-2}]<\infty$. Hence, by dominated convergence, \[ B_n=\mathbb{E}_q[g'(c_n,X)-g'(\lambda^\star,X)]\xrightarrow[]{\rm a.s.}0. \] \smallskip \noindent\textbf{Step 3.2: $A_n\to 0$ a.s.} Write $A_n=A_{n,\kappa}^{(1)}+A_{n,\kappa}^{(2)}$, where \[ A_{n,\kappa}^{(1)} :=\mathbb{E}_{\hat q_n}[g'(c_n,X)\mathbf 1_{\{X\le 1-\kappa\}}] -\mathbb{E}_q[g'(c_n,X)\mathbf 1_{\{X\le 1-\kappa\}}], \] \[ A_{n,\kappa}^{(2)} :=\mathbb{E}_{\hat q_n}[g'(c_n,X)\mathbf 1_{\{X> 1-\kappa\}}] -\mathbb{E}_q[g'(c_n,X)\mathbf 1_{\{X> 1-\kappa\}}]. \] \emph{(i) Handling $A_{n,\kappa}^{(1)}$.} Using Lemma \ref{lem:uniform_bounds_l_g_gprime}, we have, \( \{g'(\lambda,\cdot)\mathbf 1_{\{\,\cdot\le 1-\kappa\}}:\lambda\in[0,\lambda^\star]\} \) is uniformly bounded and Lipschitz in $\lambda$. Hence, using arguments similar to show \ref{ulln1}, we get, \[ \sup_{\lambda\in[0,\lambda^\star]} \Big| \mathbb{E}_{\hat q_n}[g'(\lambda,X)\mathbf 1_{\{X\le 1-\kappa\}}] -\mathbb{E}_q[g'(\lambda,X)\mathbf 1_{\{X\le 1-\kappa\}}] \Big| \xrightarrow[]{\rm a.s.}0, \] hence $A_{n,\kappa}^{(1)}\to 0$ a.s. \emph{(ii) Handling $A_{n,\kappa}^{(2)}$.} For all $\lambda\in [0,\lambda^\star]$ and $x\in[0,1)$, \[ 0\le g'(\lambda,x)\mathbf 1_{\{x>1-\kappa\}} \le (1-m_o)^2\frac{1}{(1-x)^2}\mathbf 1_{\{x>1-\kappa\}} =:(1-m_o)^2\,H_\kappa(x). \] By the SLLN, \[ \mathbb{E}_{\hat q_n}[H_\kappa(X)] \xrightarrow[]{\rm a.s.}\mathbb{E}_q[H_\kappa(X)]. \] Therefore, \[ \limsup_{n\to\infty}|A_{n,\kappa}^{(2)}| \le 2(1-m_o)^2\,\mathbb{E}_q[H_\kappa(X)] \quad\text{a.s.} \] Since $H_\kappa(X)\downarrow 0$ a.s. as $\kappa\downarrow 0$ and $0\le H_\kappa(X)\le (1-X)^{-2}$ with $\mathbb{E}_q[(1-X)^{-2}]<\infty$, dominated convergence yields $\mathbb{E}_q[H_\kappa(X)]\to 0$. Letting $\kappa\downarrow 0$ gives $A_n\to 0$ a.s. \smallskip Combining Steps 3.1 and 3.2 yields \[ \Psi'(c_n,\hat q_n) =\mathbb{E}_{\hat q_n}[g'(c_n,X)] \;\xrightarrow[]{\rm a.s.}\; \mathbb{E}_q[g'(\lambda^\star,X)]. \] Applying the same arguments as in Step 3.1 of this proof, via the dominated convergence theorem, we get $\mathbb{E}_q[g'(\lambda^\star,X)] = \Psi'(\lambda^\star,q)$. \medskip \noindent\textbf{Step 4: CLT for the numerator in \eqref{eq:bdry-onesided-final-psi}.} Define \[ \alpha_n:=\Psi(\lambda^\star,\hat q_n)-\Psi(\lambda^\star,q) . \] Note that $\Psi(\lambda^\star,q)=0$, since $\Phi(q,m_o)=1$. Using, \[ g(\lambda^\star,x)=(1-m_o)\frac{x-m_o}{1-x}, \qquad g(\lambda^\star,x)^2\le (1-m_o)^2(1-x)^{-2}, \] the assumption $\mathbb{E}_q[(1-X)^{-2}]<\infty$ implies $\mathbb{E}_q[g(\lambda^\star,X)^2]<\infty$. Thus, by the classical CLT, \[ \sqrt{n}\,\alpha_n =\sqrt{n}\Big(\frac1n\sum_{i=1}^n g(\lambda^\star,X_i)-\mathbb{E}_q[g(\lambda^\star,X)]\Big) \ \xRightarrow{d}\ \mathcal{N}\!\big(0,\ \operatorname{Var}_q\!\big(g(\lambda^\star,X)\big)\big), \] with $ \operatorname{Var}_q\!\big(g(\lambda^\star,X)\big) < \infty.$ \medskip \noindent\textbf{Step 5: Conclusion.} From \eqref{eq:bdry-onesided-final-psi}, \[ \sqrt{n}(\lambda^\star-\lambda^\star_n) =\frac{\big[\sqrt{n} \Psi(\lambda^\star,\hat q_n)\big]_+}{\Psi'(c_n,\hat q_n)} =\frac{\big[\sqrt{n} \alpha_n\big]_+}{\Psi'(c_n,\hat q_n)}. \] By Step~3, $\Psi'(c_n,\hat q_n)\to \Psi'( \lambda^\star,q)\in(0,\infty)$ a.s., and by Step~4, $\sqrt{n} \alpha_n\xRightarrow{d} \mathcal{N}\big(0, \operatorname{Var}_q \big(g(\lambda^\star,X)\big)\big)$. By Slutsky’s theorem and continuity of $z\mapsto z_+$, \[ \sqrt{n}(\lambda^\star-\lambda^\star_n) \ \xRightarrow{d} \frac{Z_+}{ \Psi'( \lambda^\star,q)}, \] where $Z \sim \mathcal{N} \big(0, \operatorname{Var}_q \big(g(\lambda^\star,X)\big)\big)$. This completes the proof. \qed \end{proof} \begin{lemma} \label{sup_lemma_A_n} When $\Phi(q,m_o) \geq 1$, for $c_n\in [0,{1}/({1-m_o})]$ with $c_n\to\lambda^\star$ almost surely, the following holds: \[ A_n := \frac{1}{n}\sum_{i=1}^n g(c_n,X_i) \xrightarrow[]{\rm a.s.} 0. \] \end{lemma} \begin{proof} Observe that, \[ A_n = \underbrace{\frac{1}{n}\sum_{i=1}^n g(\lambda^\star,X_i)}_{\text{Term 1}} + \underbrace{\frac{1}{n}\sum_{i=1}^n\Bigl(g(c_n,X_i)-g(\lambda^\star,X_i)\Bigr)}_{\text{Term 2}}. \] Below, we show that each of the two terms above converge almost surely to $0$, giving the desired convergence result. \noindent{\bf Analyzing Term 1.} By the SLLN, we have \begin{equation}\label{eq:Term1Bound} \frac{1}{n}\sum_{i=1}^n g(\lambda^\star,X_i)\xrightarrow[]{\rm a.s.} \Psi(\lambda^\star, q) = 0, \end{equation} where the last equality follows from the definition of $\lambda^*$ when $\Phi(q,m_o) \ge 1$ holds. \noindent{\bf Analyzing Term 2.} We treat the two cases, $\Phi(q,m_o) > 1$ and $\Phi(q,m_o) = 1$, separately. Consider the probability $1$ set \[\Omega_0:=\left\{\lim_{n\to\infty} c_n=\lambda^\star\right\}.\] \noindent\emph{Case (A): $\Phi(q,m_o) > 1$. } In this case, we have, $\lambda^\star< \bar\lambda $. Fix any $\epsilon>0$ with $I_\epsilon=[\lambda^\star-\epsilon,\ \lambda^\star+\epsilon]\subset(0,\bar\lambda)$. Next, fix $\omega\in\Omega_0$. There exists $N(\omega)$ such that $c_n(\omega)\in I_\epsilon$ for all $n\ge N(\omega)$. For such $n$, using Lemma \ref{lem:uniform_bounds_l_g_gprime}, \[ \big|g(c_n,X_i)-g(\lambda^\star,X_i)\big| \le L_\epsilon |c_n-\lambda^\star|\quad\text{for each }i. \] Hence, \[ \left|\frac{1}{n}\sum_{i=1}^n\Big(g(c_n,X_i)-g(\lambda^\star,X_i)\Big)\right| \le L_\epsilon |c_n-\lambda^\star| \xrightarrow[n\to\infty]{} 0 \quad\text{a.s.} \] \medskip \emph{Case (B): $\Phi(q,m_o) = 1$.} Recall that in this case, $\lambda^\star={1}/({1-m_o})$ and $q(\{1\}) = 0$. Fix $\kappa \in(0,1)$ such that $1-\kappa > m_o$ (since $m_o \in (0,1)$), and split \begin{align*} \frac{1}{n}\sum_{i=1}^n &\Bigl(g(c_n,X_i)-g(\lambda^\star,X_i)\Bigr)\\ &= \underbrace{\frac{1}{n}\sum_{i=1}^n \mathbf{1}_{\{X_i\le 1-\kappa\}} \Bigl(g(c_n,X_i)-g(\lambda^\star,X_i)\Bigr)}_{\text{Term A}} + \underbrace{\frac{1}{n}\sum_{i=1}^n \mathbf{1}_{\{X_i>1-\kappa\}} \Bigl(g(c_n,X_i)-g(\lambda^\star,X_i)\Bigr)}_{\text{Term B}}. \end{align*} Using Lemma \ref{lem:uniform_bounds_l_g_gprime}, we get, \[ |g(\lambda,x)-g(\lambda',x)| \le K_\kappa\,|\lambda-\lambda'| \quad\text{for all }\lambda,\lambda'\in\Bigl[0,\frac{1}{1-m_o}\Bigr], \text{ and } x\in[0,1-\kappa]. \] Hence \begin{equation}\label{eq:TermABound} \left|\frac{1}{n}\sum_{i=1}^n \mathbf{1}_{\{X_i\le 1-\kappa\}}\Bigl(g(c_n,X_i)-g(\lambda^\star,X_i)\Bigr)\right| \le K_\kappa\,|c_n-\lambda^\star| \xrightarrow[n\to\infty]{\text{a.s.}} 0. \end{equation} Next, since $q(\{1\}) = 0$, we have $\{1>X>1-\kappa\} = \{1\geq X>1-\kappa\}$ almost surely, where $X \sim q$. Now, for any $c\in [0,{1}/({1-m_o})]$ and $ m_o < 1-\kappa < x<1$, \[ |g(c,x)|=\frac{|x-m_o|}{1-c(x-m_o)} \le \frac{|x-m_o|}{1-\frac{1}{1-m_o}(x-m_o)} = \frac{(1-m_o)\,|x-m_o|}{1-x}. \] Hence, we have \begin{align} \left|\frac{1}{n}\sum_{i=1}^n \mathbf{1}_{\{X_i>1-\kappa\}} \Bigl(g(c_n,X_i)-g(\lambda^\star,X_i)\Bigr)\right| &\le \frac{1}{n}\sum_{i=1}^n \mathbf{1}_{\{1>X_i>1-\kappa\}} \Bigl(|g(c_n,X_i)|+|g(\lambda^\star,X_i)|\Bigr) \notag\\ &\le \frac{2(1-m_o)}{n}\sum_{i=1}^n \mathbf{1}_{\{1>X_i>1-\kappa\}} \frac{|X_i-m_o|}{1-X_i}. \end{align} Define \[ Y_\kappa(X) :=\frac{|X-m_o|}{1-X}\mathbf 1_{\{1>X>1-\kappa\}} . \] From the bound above, \[ \left|\frac{1}{n}\sum_{i=1}^n \mathbf{1}_{\{X_i>1-\kappa\}} \Bigl(g(c_n,X_i)-g(\lambda^\star,X_i)\Bigr)\right| \le \frac{2(1-m_o)}{n}\sum_{i=1}^n Y_\kappa(X_i). \] Observe that, \[ 0\le Y_\kappa(X)\le \frac{(1-m_o)}{(1-X)}. \] Further, ${(1-m_o)}/{(1-X)}$ is integrable since we have assumed that $\mathbb E_q[{1}/{(1-X)^2}] < \infty$. Hence, by SLLN, \[ \frac{1}{n}\sum_{i=1}^n Y_\kappa(X_i) \xrightarrow[n\to\infty]{a.s.}\mathbb E_q[Y_\kappa(X)]. \] Moreover, $Y_\kappa(X)\downarrow 0$ a.s. as $\kappa\downarrow 0$ hence by dominated convergence, \[ \mathbb E_q[Y_\kappa(X)]\to 0. \] Therefore, \begin{equation}\label{eq:TermBBound} \lim_{\kappa\downarrow 0}\lim_{n\to\infty} \left|\frac{1}{n}\sum_{i=1}^n \mathbf{1}_{\{X_i>1-\kappa\}} \Bigl(g(c_n,X_i)-g(\lambda^\star,X_i)\Bigr)\right| =0 \quad\text{a.s.} \end{equation} Combining the bounds from~\eqref{eq:TermABound} and~\eqref{eq:TermBBound}, we get \begin{equation}\label{eq:Term2Bound} \frac{1}{n}\sum_{i=1}^n\Big(g(c_n,X_i)-g(\lambda^\star,X_i)\Big)\to 0 \quad\text{a.s.} \end{equation} Combining~\eqref{eq:Term1Bound} and~\eqref{eq:Term2Bound}, we get the desired result, completing the proof. \end{proof} \begin{lemma}\label{lemm:anscombe} Let $L_n:=\mathrm{KL}_{\inf}(\hat q_n,m_o)$ and $ L:=\mathrm{KL}_{\inf}(q,m_o)$. Define $$Y_n:=\sqrt{n}(L_n-L)=\sqrt{n}(\mathrm{KL}_{\inf}(\hat q_n,m_o)-\mathrm{KL}_{\inf}(q,m_o)).$$ Then $\{Y_n\}$ satisfies the Anscombe condition, that is, for every $\epsilon>0$ and $\eta>0$, there exist $\delta\in(0,1)$ and $n_0 \ge 1$ such that for all $n\ge n_0$, \[ \PP\left(\max_{\substack{k\in\mathbb N:\\ |k-n|\le n\delta}}|Y_k- Y_n|>\epsilon\right)<\eta. \] \end{lemma} \begin{proof} As earlier, we only prove the result for the case $m_o>m(q)$. The other case ($m_o0$ and $\eta>0$. %Write $\Delta_n:=L_n-L$, so $Y_n=\sqrt{n}\Delta_n$. The proof proceeds in 7 steps, detailed below. \noindent\textbf{Step 1.} For any $\delta\in (0,1)$ and integer $k$ with $|k-n|\le n\delta$, \[ Y_k-Y_n%=\sqrt{k}\Delta_k-\sqrt{n}\Delta_n %=\sqrt{n}\left(\sqrt{k/n}\,\Delta_k-\Delta_n\right) =\sqrt{n}\left((L_k-L_n) + \left(\sqrt{k/n}-1 \right) (L_k-L)\right). \] Therefore, \begin{align} \max_{|k-n|\le n\delta}|Y_k-Y_n| &\le \sqrt{n}\max_{|k-n|\le n\delta}|L_k-L_n| +\sqrt{n}\left(\max_{|k-n|\le n\delta}|\sqrt{k/n}-1|\right)\left(\max_{|k-n|\le n\delta}|L_k-L|\right)\notag\\ &\le \sqrt{n}\max_{|k-n|\le n\delta}|L_k-L_n| +\sqrt{n}\delta \left(\max_{|k-n|\le n\delta}|L_k-L|\right), \label{eq:absorb} \end{align} where the last inequality follows since $|k-n|\le n\delta$ implies $k/n\in[1-\delta,1+\delta]$, and we have \begin{equation} \max_{|k-n|\le n\delta}\big|\sqrt{k/n}-1\big| = \max\{1-\sqrt{1-\delta},\sqrt{1+\delta}-1\} = 1-\sqrt{1-\delta} \le \delta.\notag \end{equation} Define \[ M_{n,\delta}:=\max_{|k-n|\le n\delta}\sqrt{n}\,|L_k-L|. \] Using \eqref{eq:absorb} along with a union bound, we get that for any $\delta\in (0,1)$, the following inequality holds: \begin{equation}\label{eq:absorb2} \PP\left(\max_{|k-n|\le n\delta}|Y_k-Y_n|>\epsilon\right) \le \PP\left(\sqrt{n}\max_{|k-n|\le n\delta}|L_k-L_n|>\epsilon/2\right) + \PP\left(M_{n,\delta}>\frac{\epsilon}{2\delta}\right). \end{equation} Thus it suffices to prove that there exists $\delta\in (0,1)$ and $n_0$ such that for all $n\ge n_0$ the following hold: \[\label{A} \PP\left(\sqrt{n}\max_{|k-n|\le n\delta}|L_k-L_n|>\frac{\epsilon}{2} \right) < \eta/2, \tag{A} \] and \[\label{B}\PP\left(M_{n,\delta}>\frac{\epsilon}{2\delta}\right) \le \eta/2.\tag{B} \] We prove existence of $\delta$ and $n_0$ such that (\ref{A}) and (\ref{B}) hold, below. \noindent\textbf{Step 2: three-term decomposition of $L_k-L_n$.} Add and subtract $\ell(\lambda^\star,\cdot)$. \begin{align} L_k-L_n &= \underbrace{\frac1k\sum_{i=1}^k\big(\ell(\lambda_k^\star,X_i)-\ell(\lambda^\star,X_i)\big)}_{=:U_k} - \underbrace{\frac1n\sum_{i=1}^n\big(\ell(\lambda_n^\star,X_i)-\ell(\lambda^\star,X_i)\big)}_{=:U_n} \notag\\ &\quad+ \underbrace{\left(\frac1k\sum_{i=1}^k \ell(\lambda^\star,X_i)-\frac1n\sum_{i=1}^n \ell(\lambda^\star,X_i)\right)}_{=:V_{k,n}}. \label{eq:3term} \end{align} Hence, \begin{equation}\label{eq:3term_max} \sqrt{n}\max_{|k-n|\le n\delta}|L_k-L_n| \le \underbrace{\sqrt{n}\max_{|k-n|\le n\delta}|U_k|}_{\text{Term 1}} + \underbrace{\sqrt{n}|U_n|}_{\text{Term 2}} + \underbrace{\sqrt{n}\max_{|k-n|\le n\delta}|V_{k,n}|}_{\text{Term 3}}. \end{equation} \medskip \noindent\textbf{Step 3: control Term 3}. Let \[ Z_i:=\ell(\lambda^\star,X_i)-\E[\ell(\lambda^\star,X)],\qquad S_t:=\sum_{i=1}^t Z_i. \] Then $\E[Z_i]=0$. In addition, $\Var(Z_1)<\infty$ (Lemma~\ref{lem:finiteMGF}). Since \[ \frac1t\sum_{i=1}^t\ell(\lambda^\star,X_i)-\E[\ell(\lambda^\star,X)]=\frac{S_t}{t}, \] we have \[ V_{k,n}=\frac{S_k}{k}-\frac{S_n}{n} =\frac{S_k-S_n}{k}+S_n\Big(\frac1k-\frac1n\Big). \] For $|k-n|\le n\delta$, we have $k\ge (1-\delta)n$ and $|n-k|\le n\delta$, hence \[ \sqrt{n}|V_{k,n}| \le \frac{\sqrt{n}}{k}|S_k-S_n|+\sqrt{n}|S_n|\frac{|n-k|}{kn} \le \frac{1}{(1-\delta)\sqrt{n}}|S_k-S_n|+\frac{\delta}{(1-\delta)\sqrt{n}}|S_n|. \] Therefore, with $a:=\epsilon/6$, \begin{align} \PP\!\left(\sqrt{n}\max_{|k-n|\le n\delta}|V_{k,n}|>a\right) &\le \PP\!\left(\max_{|k-n|\le n\delta}|S_k-S_n|>\frac{a}{2}(1-\delta)\sqrt{n}\right) \notag\\ &\quad+ \PP\!\left(|S_n|>\frac{a}{2}\frac{1-\delta}{\delta}\sqrt{n}\right). \label{eq:Vsplit} \end{align} Let $\sigma_Z^2:=\Var(Z_1)$. For the increment term, set $N:=\lfloor n\delta\rfloor$ and note that \[ \max_{|k-n|\le N}|S_k-S_n| = \max\left\{ \max_{1\le j\le N}\Big|\sum_{i=n+1}^{n+j}Z_i\Big|, \ \max_{1\le j\le N}\Big|\sum_{i=n-j+1}^{n}Z_i\Big| \right\}. \] Hence, for any $u>0$, \[ \PP\!\left(\max_{|k-n|\le N}|S_k-S_n|>u\right) \le \PP\!\left(\max_{1\le j\le N}\Big|\sum_{i=n+1}^{n+j}Z_i\Big|>u\right) + \PP\!\left(\max_{1\le j\le N}\Big|\sum_{i=n-j+1}^{n}Z_i\Big|>u\right). \] By stationarity, each term has the same bound as $\PP\!\left(\max_{1\le j\le N}\big|\sum_{i=1}^{j}Z_i\big|>u\right)$. By Kolmogorov's maximal inequality \citep[Theorem 22.4]{billingsley2017probability}, for any $u>0$, \[ \PP\!\left(\max_{1\le j\le N}\Big|\sum_{i=1}^{j}Z_i\Big|\ge u\right)\le \frac{N\sigma_Z^2}{u^2}. \] Taking $u=\frac{a}{2}(1-\delta)\sqrt{n}$ yields \begin{equation}\label{eq:Vinc} \PP\!\left(\max_{|k-n|\le n\delta}|S_k-S_n|>\frac{a}{2}(1-\delta)\sqrt{n}\right) \le 2\cdot \frac{(n\delta)\sigma_Z^2}{(\frac{a}{2}(1-\delta)\sqrt{n})^2} = \frac{8\sigma_Z^2}{a^2(1-\delta)^2}\,\delta. \end{equation} For the second term in \eqref{eq:Vsplit}, using Chebyshev inequality, \begin{equation}\label{eq:Vsn} \PP\!\left(|S_n|>\frac{a}{2}\frac{1-\delta}{\delta}\sqrt{n}\right) \le \frac{n\sigma_Z^2}{(\frac{a}{2}\frac{1-\delta}{\delta}\sqrt{n})^2} = \frac{4\sigma_Z^2}{a^2(1-\delta)^2}\,\delta^2. \end{equation} Combining \eqref{eq:Vsplit}--\eqref{eq:Vinc}--\eqref{eq:Vsn} yields the explicit bound \begin{equation}\label{eq:Vbound_final} \PP\!\left(\sqrt{n}\max_{|k-n|\le n\delta}|V_{k,n}|>\epsilon/6\right) \le C_V\,\delta+C_V'\,\delta^2, \end{equation} for constants $C_V,C_V'$ depending only on $\sigma_Z^2$ and $\epsilon$ (and not on $n$). \medskip \noindent\textbf{Step 4: case analysis for Terms 1 and 2 .} \smallskip \noindent \textbf{Case 1: $\Phi(q,m_o)<1$.} In this case, $\lambda^\star=\bar\lambda$. By the SLLN, \[ \Phi(\hat q_n,m_o)=\frac1n\sum_{i=1}^n \frac{1-m_o}{1-X_i}\ \xrightarrow{\mathrm{a.s.}}\ \Phi(q,m_o)<1. \] Hence, on each sample path there exists $N_0(\omega)$ such that for all $t\ge N_0(\omega)$, $\Phi(\hat q_t,m_o)<1$, which implies $\lambda_t^\star=\bar\lambda=\lambda^\star$ for all $t\ge N_0(\omega)$. Therefore, it follows that $ \sqrt{n}\max_{|k-n|\le n\delta}|U_k| \xrightarrow{p} 0 \qquad\text{and}\qquad \sqrt{n}|U_n| \xrightarrow{p} 0. $ \medskip \noindent\textbf{Case 2 and 3: $\Phi(q,m_o)\geq 1$.} Observe that this $U_n$ is same as $T_{1,n}$ defined in the proof of Theorem \ref{lem:kl_inf_clt}. Hence using Taylor series it follows that, we have \[ \sqrt{n}|U_n|=\sqrt{n}\,|\lambda_n^\star-\lambda^\star|\,|A_n|, \qquad \sqrt{n}\max_{|k-n|\le n\delta}|U_k| \le \frac{1}{\sqrt{1-\delta}}\Big(\max_{|k-n|\le n\delta}\sqrt{k}\,|\lambda_k^\star-\lambda^\star|\Big)\Big(\max_{|k-n|\le n\delta}|A_k|\Big), \] because $k\ge (1-\delta)n$ implies $\sqrt{n}\le \sqrt{k}/\sqrt{1-\delta}$. Here, \[ A_n:=\frac1n\sum_{i=1}^n g(c_n,X_i), \] for some $c_n$ between $\lambda^\star_n$ and $\lambda^\star$. Thus, to control Terms 1--2 we will show (in each case): \begin{equation}\label{eq:Ato0_goal} \max_{|k-n|\le n\delta}|A_k|\to 0\ \text{(a.s.)}\quad\text{and}\quad \max_{|k-n|\le n\delta}\sqrt{k}\,|\lambda_k^\star-\lambda^\star|=O_p(1). \end{equation} %------------------------------------------------------------ \smallskip \noindent \textbf{Case 2: $\Phi(q,m_o)>1$.} In this case, $\lambda^\star<\bar\lambda$ and it is the unique solution of $\Psi(\lambda^\star,q)=0$. For Term 2, we have \[ \sqrt{n}|U_n|=\sqrt{n}\,|\lambda_n^\star-\lambda^\star|\,|A_n|. \] By Lemma \ref{sup_lemma_case_2_combined} amd Lemma \ref{sup_lemma_A_n}, $\sqrt{n}(\lambda_n^\star-\lambda^\star)=O_p(1)$ and, $A_n\to 0$ a.s., hence $\sqrt{n}|U_n|\to 0$ in probability. For Term 1, it is enough to show: (i) $\max_{|k-n|\le n\delta}|A_k|\to 0$ a.s., and (ii) $\max_{|k-n|\le n\delta}\sqrt{k}|\lambda_k^\star-\lambda^\star|=O_p(1)$. Property (i) follows from $A_k\to 0$ a.s.\ and the fact that $\sup_{k\ge (1-\delta)n}|A_k|\to0$ a.s. For (ii), using \eqref{eq:one-step}, we have \begin{equation} \sqrt{k}\big(\lambda^{\star}_{k}-\lambda^{\star}\big) = -\,\frac{\sqrt{k}\big(\Psi(\lambda^{\star},\hat q_k)-\Psi(\lambda^{\star},q)\big)} {\Psi'(c_k,\hat q_k)} + \frac{\sqrt{k}\,\Psi(\lambda^{\star}_{k},\hat q_k)} {\Psi'(c_k,\hat q_k)}, \qquad \text{since }\Psi(\lambda^{\star},q)=0, \end{equation} for some $c_k$ between $\lambda^\star_k$ and $\lambda^\star$. Recall that under this case, for all sufficiently large $k$, $\Psi(\lambda^{\star}_{k},\hat q_k)=0$, and that $\Psi'(c_k,\hat q_k) \to \Psi'(\lambda^\star,q) \in (0,\infty)$ almost surely (see Proof of Lemma \ref{sup_lemma_case_2_combined}). First, this implies that $\Psi'(c_k,\hat q_k)$ is bounded below by a positive constant $1/C$ for all sufficiently large $k$ on each sample path. Consequently, we obtain \[ \sqrt{k}\,|\lambda_k^\star-\lambda^\star| \le C\,\left|\frac{1}{\sqrt{k}}\sum_{i=1}^k\Big(g(\lambda^\star,X_i)-\E[g(\lambda^\star,X)]\Big)\right|. \] Let $\widetilde Z_i:=g(\lambda^\star,X_i)-\E[g(\lambda^\star,X)]$ and $\widetilde S_t:=\sum_{i=1}^t \widetilde Z_i$. Then $\E[\widetilde Z_i]=0$ and $\Var(\widetilde Z_1)<\infty$ (see Proof of Lemma \ref{sup_lemma_case_2_combined}). By Kolmogorov's maximal inequality, for any $M>0$ and any $n\ge1$, \[ \PP\!\left(\max_{t\le 2n}|\widetilde S_t|>M\sqrt{n}\right) \le \frac{\Var(\widetilde S_{2n})}{M^2 n} = \frac{2n\,\Var(\widetilde Z_1)}{M^2 n} = \frac{2\,\Var(\widetilde Z_1)}{M^2}. \] For all $k$ with $|k-n|\le n\delta$ we have $k\ge (1-\delta)n$ and $k\le 2n$, hence \[ \max_{|k-n|\le n\delta}\frac{|\widetilde S_k|}{\sqrt{k}} \le \frac{1}{\sqrt{(1-\delta)n}}\max_{t\le 2n}|\widetilde S_t|. \] Therefore, for any $M>0$ and any $n\ge1$, \[ \PP\!\left( \max_{|k-n|\le n\delta}\frac{|\widetilde S_k|}{\sqrt{k}} > M \right) \le \PP\!\left( \max_{t\le 2n}|\widetilde S_t| > M\sqrt{(1-\delta)n} \right) \le \frac{2\,\Var(\widetilde Z_1)}{(1-\delta)M^2}. \] Combining this with the display above yields, for any $M>0$, \[ \PP\!\left( \max_{|k-n|\le n\delta}\sqrt{k}\,|\lambda_k^\star-\lambda^\star| > C M \right) \le \PP\!\left( \max_{|k-n|\le n\delta}\frac{|\widetilde S_k|}{\sqrt{k}} > M \right) \le \frac{2\,\Var(\widetilde Z_1)}{(1-\delta)M^2}. \] In particular, for any $\epsilon>0$, choosing $M=\sqrt{\frac{2\,\Var(\widetilde Z_1)}{(1-\delta)\epsilon}}$ gives for large $n$, \[ \PP\!\left( \max_{|k-n|\le n\delta}\sqrt{k}\,|\lambda_k^\star-\lambda^\star| > C M \right)\le \epsilon, \] which implies $\max_{|k-n|\le n\delta}\sqrt{k}|\lambda_k^\star-\lambda^\star|=O_p(1)$, proving $\sqrt{n}\max_{|k-n|\le n\delta}|U_k|$ $\to0$ in probability. \smallskip \noindent \textbf{Case 3: $\Phi(q,m_o)=1$.} In this case, $\lambda^\star=\bar\lambda$ and $\Psi(\lambda^\star,q)=0$. Using Lemma \ref{sup_lemma_case_3_combined}, we also have, \begin{equation}\label{eq:case3_onesided} \lambda^\star-\lambda_k^\star=\frac{[\Psi(\lambda^\star,\hat q_k)- \Psi(\lambda^\star,q)]_+}{\Psi'(c_k,\hat q_k)}, \qquad [x]_+:=\max\{x,0\}, \end{equation} and $\Psi'(c_k,\hat q_k)\to \E[g'(\lambda^\star,X)] \in (0,\infty)$ a.s. Now, recall Term 2, \[ \sqrt{n}|U_n|=\sqrt{n}\,|\lambda_n^\star-\lambda^\star|\,|A_n|. \] By Lemma \ref{sup_lemma_case_3_combined} and Lemma \ref{sup_lemma_A_n}, we have $\sqrt{n}|\lambda_n^\star-\lambda^\star|=O_p(1)$, and $A_n\to0$ a.s., hence $\sqrt{n}|U_n|\to0$ in probability. Term 1: We already know that $A_k\to0$ a.s., hence $\max_{|k-n|\le n\delta}|A_k|\to0$ a.s. It remains to show $\max_{|k-n|\le n\delta}\sqrt{k}|\lambda_k^\star-\lambda^\star|=O_p(1)$. From \eqref{eq:case3_onesided} for all large $k$, using proofs similar to case 2, we have \[ \sqrt{k}|\lambda_k^\star-\lambda^\star| \le C\, \sqrt{k}\big|\Psi(\lambda^\star,\hat q_k) - \Psi(\lambda^\star,q)\big| = C\,\left|\frac{1}{\sqrt{k}}\sum_{i=1}^k g(\lambda^\star,X_i) - \E[g(\lambda^\star,X)]\right|. \] Similar to the proof of case 2, using Kolmogorov inequality, we get that Term 1, i.e., $ \sqrt{n}\max_{|k-n|\le n\delta}|U_k| \to0$ in probability. %------------------------------------------------------------ \medskip \noindent\textbf{Step 5: Handling term (A).} By \eqref{eq:3term_max} and the union bound, \begin{align*} \PP\!\left(\sqrt{n}\max_{|k-n|\le n\delta}|L_k-L_n|>\epsilon/2\right) &\le \PP\!\left(\sqrt{n}\max_{|k-n|\le n\delta}|U_k|>\epsilon/6\right) +\PP\!\left(\sqrt{n}|U_n|>\epsilon/6\right)\\ &\quad+ \PP\!\left(\sqrt{n}\max_{|k-n|\le n\delta}|V_{k,n}|>\epsilon/6\right). \end{align*} Recall that from step 4, we have $\sqrt{n}\,|U_n|\to 0$ in probability and $\sqrt{n}\max_{|k-n|\le n\delta}|U_k|\to 0$ in probability. Therefore, for each fixed $\delta\in(0,1)$ and each $\epsilon>0$, \[ \lim_{n\to\infty}\PP\!\left(\sqrt{n}|U_n|>\epsilon/6\right)=0, \qquad \lim_{n\to\infty}\PP\!\left(\sqrt{n}\max_{|k-n|\le n\delta}|U_k|>\epsilon/6\right)=0. \] For the third probability, \eqref{eq:Vbound_final} yields, for every $\delta\in(0,1)$ and every $n\ge1$, \[ \PP\!\left(\sqrt{n}\max_{|k-n|\le n\delta}|V_{k,n}|>\epsilon/6\right)\le C_V\delta+C_V'\delta^2. \] Now fix $\epsilon>0$ and $\eta>0$, and choose $\delta\in(0,1)$ such that \[ C_V\delta+C_V'\delta^2<\eta/4. \] With this choice of $\delta$, by the limits above there exists $n_0=n_0(\delta,\epsilon,\eta)$ such that for all $n\ge n_0$, \[ \PP\!\left(\sqrt{n}\max_{|k-n|\le n\delta}|U_k|>\epsilon/6\right)<\eta/8, \qquad \PP\!\left(\sqrt{n}|U_n|>\epsilon/6\right)<\eta/8. \] Combining these bounds, we get that for all $n\ge n_0$, \[ \PP\!\left(\sqrt{n}\max_{|k-n|\le n\delta}|L_k-L_n|>\epsilon/2\right) < \eta/8+\eta/8+\eta/4 =\eta/2, \] which proves (A). %------------------------------------------------------------ \medskip \noindent\textbf{Step 6: Handling term (B).} Fix $\epsilon>0$ and $\eta>0$. Let $\delta\in(0,1)$ be arbitrary (to be chosen later). Recall \[ M_{n,\delta}:=\max_{|k-n|\le n\delta}\sqrt{n}\,|L_k-L|. \] For $|k-n|\le n\delta$ we have $k\in[(1-\delta)n,(1+\delta)n]\subset[1,2n]$ for all large $n$. Moreover, \[ \sqrt{n}|L_k-L| \le \sqrt{n}|U_k| + \sqrt{n}\left|\frac1k\sum_{i=1}^k \ell(\lambda^\star,X_i)-\E[\ell(\lambda^\star,X)]\right| = \sqrt{n}|U_k|+\sqrt{n}\left|\frac{S_k}{k}\right|, \] where $S_k:=\sum_{i=1}^k Z_i$ and $Z_i:=\ell(\lambda^\star,X_i)-\E[\ell(\lambda^\star,X)]$ with $\E[Z_i]=0$ and $\Var(Z_1)=:\sigma_Z^2<\infty$. Hence, \[ M_{n,\delta} \le \underbrace{\max_{|k-n|\le n\delta}\sqrt{n}|U_k|}_{=:M^{(U)}_{n,\delta}} + \underbrace{\max_{|k-n|\le n\delta}\sqrt{n}\left|\frac{S_k}{k}\right|}_{=:M^{(S)}_{n,\delta}}. \] Therefore, \begin{equation}\label{eq:B_split_prob} \PP\!\left(M_{n,\delta}>\frac{\epsilon}{2\delta}\right) \le \PP\!\left(M^{(U)}_{n,\delta}>\frac{\epsilon}{4\delta}\right) + \PP\!\left(M^{(S)}_{n,\delta}>\frac{\epsilon}{4\delta}\right). \end{equation} For $k \in \mathbb{Z}^{+}$ such that $|k-n|\le n\delta$ we have $k\ge(1-\delta)n$, so for all large $n$, \[ \sqrt{n}\left|\frac{S_k}{k}\right| = \frac{n}{k}\left|\frac{S_k}{\sqrt{n}}\right| \le \frac{1}{1-\delta}\left|\frac{S_k}{\sqrt{n}}\right| \le \frac{1}{1-\delta}\max_{t\le 2n}\left|\frac{S_t}{\sqrt{n}}\right|. \] Hence, for any $a>0$ and all large $n$, \[ \PP\!\left(M^{(S)}_{n,\delta}>a\right) \le \PP\!\left( \max_{t\le 2n}|S_t|>a(1-\delta)\sqrt{n} \right). \] By Kolmogorov's maximal inequality, for any $x>0$ and any $n\ge1$, \[ \PP\!\left(\max_{t\le 2n}|S_t|>x\right)\le \frac{\Var(S_{2n})}{x^2} = \frac{2n\sigma_Z^2}{x^2}. \] Applying this with $x=a(1-\delta)\sqrt{n}$ yields, for all large $n$, \[ \PP\!\left(M^{(S)}_{n,\delta}>a\right) \le \frac{2\sigma_Z^2}{a^2(1-\delta)^2}. \] Now take $a=\epsilon/(4\delta)$. Then for all large $n$, \begin{equation}\label{eq:B_S_bound} \PP\!\left(M^{(S)}_{n,\delta}>\frac{\epsilon}{4\delta}\right) \le \frac{2\sigma_Z^2}{(1-\delta)^2}\cdot \frac{16\delta^2}{\epsilon^2} = \frac{32\sigma_Z^2}{\epsilon^2}\cdot \frac{\delta^2}{(1-\delta)^2}. \end{equation} Choose $\delta_0\in(0,1/2)$ such that for all $\delta\in(0,\delta_0)$, \[ \frac{32\sigma_Z^2}{\epsilon^2}\cdot \frac{\delta^2}{(1-\delta)^2} \le \frac{\eta}{4}. \] Then, for any such $\delta$, \eqref{eq:B_S_bound} gives for all large $n$, \begin{equation}\label{eq:B_S_eta} \PP\!\left(M^{(S)}_{n,\delta}>\frac{\epsilon}{4\delta}\right)\le \frac{\eta}{4}. \end{equation} \medskip In Case 1, $M^{(U)}_{n,\delta}=0$ for all large $n$ a.s., so \[ \PP\!\left(M^{(U)}_{n,\delta}>\frac{\epsilon}{4\delta}\right)=0 \quad\text{for all large $n$}. \] In Cases 2 and 3, we have $M^{(U)}_{n,\delta}\to0$ in probability (Step~4). Hence, for the fixed $\delta\in(0,\delta_0)$ chosen above, there exists $N(\delta,\epsilon,\eta)$ such that for all $n\ge N(\delta,\epsilon,\eta)$, \begin{equation}\label{eq:B_U_eta} \PP\!\left(M^{(U)}_{n,\delta}>\frac{\epsilon}{4\delta}\right)\le \frac{\eta}{4}. \end{equation} \medskip \noindent\emph{Combine.} Combining \eqref{eq:B_split_prob}, \eqref{eq:B_S_eta}, and \eqref{eq:B_U_eta}, for any $\delta\in(0,\delta_0)$ and all $n\ge N(\delta,\epsilon,\eta)$, \[ \PP\!\left(M_{n,\delta}>\frac{\epsilon}{2\delta}\right) \le \frac{\eta}{4}+\frac{\eta}{4} = \frac{\eta}{2}. \] This proves (B). \medskip \noindent\textbf{Step 7: Conclusion.} Fix $\epsilon>0$ and $\eta>0$. First, by (A) (proved in Step~4), choose $\delta_A\in(0,1)$ and $n_A$ such that for all $n\ge n_A$, \[ \PP\!\left(\sqrt{n}\max_{|k-n|\le n\delta_A}|L_k-L_n|>\epsilon/2\right)<\eta/2. \] Second, by (B) (proved in Step~5), choose $\delta_0\in(0,1/2)$ sufficiently small so that for some $n_B$ and all $n\ge n_B$, \[ \PP\!\left(M_{n,\delta_0}>\frac{\epsilon}{2\delta_0}\right)<\eta/2. \] Now set \[ \delta:=\min\{\delta_A,\delta_0\}. \] Since shrinking $\delta$ can only decrease the maxima over the set $\{|k-n|\le n\delta\}$, the two bounds above remain valid (possibly with the same or smaller probabilities) when $\delta_A$ and $\delta_0$ are replaced by $\delta$. Hence, letting \[ n_1:=\max\{n_A,n_B\}, \] we have for all $n\ge n_1$, \[ \PP\!\left(\sqrt{n}\max_{|k-n|\le n\delta}|L_k-L_n|>\epsilon/2\right)<\eta/2, \qquad \PP\!\left(M_{n,\delta}>\frac{\epsilon}{2\delta}\right)<\eta/2. \] Therefore, using \eqref{eq:absorb2} and the union bound, for all $n\ge n_1$, \[ \PP\!\left(\max_{|k-n|\le n\delta}|Y_k-Y_n|>\epsilon\right) \le \PP\!\left(\sqrt{n}\max_{|k-n|\le n\delta}|L_k-L_n|>\epsilon/2\right) + \PP\!\left(M_{n,\delta}>\frac{\epsilon}{2\delta}\right) <\eta. \] This completes the proof. \end{proof} \section{Discussion on Assumption \ref{ass_1}} \label{sec_ass} We now show that Assumption~\ref{ass_1} is mild, by demonstrating that it holds across several commonly studied distribution families. In the discussion below, we focus on the case \(m_o>m(q)\); the case \(m_o 0$, then $\mathbb{E}_q\!\left[\frac{1-m_o}{1-X}\right] =\infty$ for every $m_o\in(0,1)$. Hence, Assumption~\ref{ass_1} is vacuously satisfied for such distributions, and in particular, for the entire Bernoulli family. \medskip \noindent\textbf{Uniform distribution.} We note that $\mathrm{Uniform}[-1,1]$ is not supported on $[0,1]$; however, all the results in the paper can be adapted to the setting of distributions supported in $[a,b]$ for any $a < b$, via a rescaling argument. For simplicity of presentation, we consider $q = \mathrm{Uniform}[0,1]$ here. For this, \[ \mathbb{E}_q\!\left[\frac{1-m_o}{1-X}\right] \;=\; (1-m_o)\int_0^1 \frac{\mathrm{d}x}{1-x} \;=\; \infty. \] Hence, Assumption~\ref{ass_1} is vacuously satisfied for $\mathrm{Uniform}[0,1]$. The same argument applies for the \textbf{Truncated Normal} distribution on $[0,1]$. \medskip \noindent\textbf{Beta family.} For $q=\mathrm{Beta}(a,b)$ with $a,b>0$, we have $q(\{1\})=0$ and \[ \mathbb{E}_q\!\left[\frac{1-m_o}{1-X}\right] \;=\; (1-m_o)\,\frac{a+b-1}{b-1}, \qquad b>1, \] and $\infty$ for $b\le 1$, so Assumption~\ref{ass_1} holds vacuously for $b\le 1$. For $b>2$, $\mathbb{E}_q[1/(1-X)^2]<\infty$, so Assumption~\ref{ass_1} holds trivially as well. It remains to consider $b\in(1,2]$, where $\mathbb{E}_q\!\left[\frac{1-m_o}{1-X}\right]=1$ if and only if \[ m_o = m_o^\star(a,b) := \frac{a}{a+b-1}, \] and at this $m_o$ one has $\mathbb{E}_q[1/(1-X)^2]=\infty$. Hence Assumption~\ref{ass_1} is \textbf{violated for exactly one value of $m_o$}, namely $m_o^\star(a,b)$. In summary, for a given $\mathrm{Beta}(a,b)$ distribution, Assumption~\ref{ass_1} always holds except for a unique critical value of $m_o$ in the case \(m_o>m(q)\). The case \(m_o x) \approx 2\Phi(-\sqrt{x})$ for large $x$, this quantile is approximately $2\log(1/\alpha)$ for small $\alpha$, yielding $\beta(n,\alpha) \approx \log(1/\alpha)$ as the natural leading term. However, this $\chi^2$-based argument is inherently asymptotic in $n$ and therefore does not, by itself, guarantee type-I error control at any finite sample size or finite $\alpha$. The adjusted threshold $\beta(n, \alpha) = 1 + \log(\frac{2(1+n)}{\alpha})$ precisely compensates for this limitation. First, the additional terms correct for the asymptotic approximation and ensure that the above bound holds non-asymptotically, i.e., for all finite $\alpha$ and all sample sizes. Second, this correction also yields a \emph{time-uniform} (in $n$) guarantee, meaning that the type-I error control holds simultaneously over all (possibly random) stopping times. Empirically, we observe that the $\chi^2(1)$ approximation becomes accurate already at practically relevant levels such as $\alpha = 0.05$ and $\alpha = 0.01$, which further supports $\log(1/\alpha)$ as the dominant term governing the choice of $\beta(n,\alpha)$. \end{document} %%%%% \begin{assumption} \label{ass_1} For $q\in\mathcal{B}$ and $m_o\in(0,1)$ with $m_o\neq m(q)$, if $\mathbb{E}_q \left[({1-m_o})/({1-X})\right] =1$, then $\mathbb{E}_q\!\left[1/(1-X)^2\right] < \infty$. \end{assumption} \section{Discussion on Assumption \ref{ass_1}} \label{sec_ass} We now show that Assumption~1 is mild, by demonstrating that it holds across several commonly studied distribution families. \medskip \noindent\textbf{Distributions with an atom at $1$ (e.g., Bernoulli).} If $q(\{1\}) > 0$, then $\mathbb{E}_q\!\left[\frac{1-m_o}{1-X}\right] =\infty$ for every $m_o\in(0,1)$. Hence, Assumption 1 is vacuously satisfied for such distributions, and in particular, for the entire Bernoulli family. \medskip \noindent\textbf{Uniform distribution.} We note that $\mathrm{Uniform}[-1,1]$ is not supported on $[0,1]$, however all the results in the paper can be adapted to the setting of distributions support in $[a,b]$ for any $a < b$, via a rescaling argument. For simplicity of presentation, we consider $q = \mathrm{Uniform}[0,1]$ here. For this, \[ \mathbb{E}_q\!\left[\frac{1-m_o}{1-X}\right] \;=\; (1-m_o)\int_0^1 \frac{\mathrm{d}x}{1-x} \;=\; \infty. \] Hence, Assumption 1 is vacuously satisfied even for $\mathrm{Uniform}[-1,1]$. The same argument applies for the \textbf{Truncated Normal} distribution on $[0,1]$. \medskip \noindent\textbf{Beta family.} For $q=\mathrm{Beta}(a,b)$ with $a,b>0$, we have $q(\{1\})=0$ and \[ \mathbb{E}_q\!\left[\frac{1-m_o}{1-X}\right] \;=\; (1-m_o)\,\frac{a+b-1}{b-1}, \qquad b>1, \] and $\infty$ for $b\le 1$, so Assumption~1 holds vacuously for $b\le 1$. For $b>2$, $\mathbb{E}_q[1/(1-X)^2]<\infty$, so Assumption~1 holds trivially as well. It remains to consider $b\in(1,2]$, where $\mathbb{E}_q\!\left[\frac{1-m_o}{1-X}\right]=1$ if and only if \[ m_o = m_o^\star(a,b) := \frac{a}{a+b-1}, \] and at this $m_o$ one has $\mathbb{E}_q[1/(1-X)^2]=\infty$. Hence Assumption~1 is \textbf{violated for exactly one value of $m_o$}, namely $m_o^\star(a,b)$. In summary, for a given $\mathrm{Beta}(a,b)$ distribution, Assumption~1 always holds except for a unique critical value of $m_o$. Thus, this assumption holds for several commonly studied families of distributions with bounded support. Hence, we believe Assumption 1 is a mild technical condition. %%%%------------------------------------------------------------ \begin{comment} For any $\lambda\in(0,\bar\lambda]$, \[ \mathbb{E}_q[\ell(\lambda,X)]=\mathbb{E}_q\Big[\log\big(1-\lambda(X-m_o)\big)\Big], \qquad \mathbb{E}_{\hat q_n}[\ell(\lambda,X)] =\frac1n\sum_{i=1}^n\ell(\lambda,X_i). \] By definition, $\lambda^\star$ maximizes $\lambda\mapsto\mathbb{E}_q[\ell(\lambda,X)]$ on $(0,\bar\lambda]$, and $\lambda^\star_n$ maximizes $\lambda\mapsto\mathbb{E}_{\hat q_n}[\ell(\lambda,X)]$ on $(0,\bar\lambda]$. Differentiating $\ell(\lambda,x)$ in $\lambda$ gives \[ \frac{\partial}{\partial\lambda} \ell(\lambda,x) =-\frac{x-m_o}{1-\lambda(x-m_o)} =-g(\lambda,x), \] hence \[ \frac{\partial}{\partial\lambda}\mathbb{E}_q[\ell(\lambda,X)]=-\,\Psi(\lambda,q),\qquad \frac{\partial}{\partial\lambda}\mathbb{E}_{\hat q_n}[\ell(\lambda,X)]=-\,\Psi(\lambda,\hat q_n). \] Moreover, \[ g'(\lambda,x) =\frac{(x-m_o)^2}{\bigl(1-\lambda(x-m_o)\bigr)^2}\ge 0, \] so $\Psi(\lambda,q)$ is strictly increasing in $\lambda$ on $(0,\bar\lambda)$ (because $q$ is not a point mass and $m(q)\neq m_o$). When $\Phi(q,m_o)=1$, then it follows that, \[ \lambda^\star=\bar\lambda,\quad\text{and}\quad \Psi(\lambda^\star,q)=0,\ \ \Psi(\lambda,q)<0\ \text{for all }\lambda\in(0,\lambda^\star). \] Recall, when $\Phi(q,m_o)=1$, then we have, \[ \lambda^\star=\bar\lambda,\quad\text{and}\quad \Psi(\lambda^\star,q)=0. \] First, we show that, $\lambda^\star_n\to\lambda^\star$ almost surely. Fix $\epsilon\in(0,\lambda^\star)$ and let \[ J_\epsilon:=[\lambda^\star-\epsilon,\lambda^\star-\epsilon/2]\subset(0,\lambda^\star). \] On $J_\epsilon\times[0,1]$, $1-\lambda(x-m_o)$ is uniformly bounded away from $0$, so $g(\lambda,\cdot)$ and $g'(\lambda,\cdot)$ are uniformly bounded and Lipschitz in $\lambda$. A finite-net plus SLLN argument gives the uniform LLN \[ \sup_{\lambda\in J_\epsilon}\big|\Psi(\lambda,\hat q_n)-\Psi(\lambda,q)\big|\xrightarrow[]{\rm a.s.}0. \tag{ULLN-3} \] [[need to show that]] $\Psi(\cdot,q)$ is strictly increasing and continuous on $(0,\lambda^\star)$ with $\Psi(\lambda^\star,q)=0$ and $\Psi(\lambda,q)<0$ for $\lambda<\lambda^\star$, we have \[ c_\epsilon:=\max_{\lambda\in J_\epsilon}\Psi(\lambda,q)<0. \] By ULLN-3, for all large $n$ (a.s.), \[ \Psi(\lambda,\hat q_n)\le \tfrac12 c_\epsilon<0\quad\text{for all }\lambda\in J_\epsilon, \] so the derivative of $\lambda\mapsto\mathbb{E}_{\hat q_n}[\ell(\lambda,X)]$ is \[ \frac{\partial}{\partial\lambda}\mathbb{E}_{\hat q_n}[\ell(\lambda,X)] =-\Psi(\lambda,\hat q_n)>0 \quad\text{on }J_\epsilon \] for all large $n$. Thus $\lambda\mapsto\mathbb{E}_{\hat q_n}[\ell(\lambda,X)]$ is strictly increasing on $J_\epsilon$ eventually a.s. Now fix any small $\alpha >0$. Since $\lambda\mapsto\mathbb{E}_q[\ell(\lambda,X)]$ is continuous and strictly increasing on $[0,\lambda^\star]$ with unique maximizer at $\lambda^\star$, there exists $\eta>0$ such that \[ \mathbb{E}_q[\ell(\lambda^\star,X)]-\mathbb{E}_q[\ell(\lambda^\star-\alpha,X)]\ge 2\eta>0. \] On the compact interval $[0,\lambda^\star-\alpha]$, each $\ell(\lambda,\cdot)$ is bounded and continuous, and the family $\{\ell(\lambda,\cdot):\lambda\in[0,\lambda^\star-\alpha]\}$ is equicontinuous. A standard uniform LLN then implies \[ \sup_{\lambda\in[0,\lambda^\star-\alpha]}\big|\mathbb{E}_{\hat q_n}[\ell(\lambda,X)]-\mathbb{E}_q[\ell(\lambda,X)]\big|\xrightarrow[]{\rm a.s.}0. \] In particular, for all large $n$ (a.s.), \[ \mathbb{E}_{\hat q_n}[\ell(\lambda^\star,X)]-\mathbb{E}_{\hat q_n}[\ell(\lambda^\star-\alpha,X)] \ge \big(\mathbb{E}_q[\ell(\lambda^\star,X)]-\eta\big)-\big(\mathbb{E}_q[\ell(\lambda^\star-\alpha,X)]+\eta\big) = \mathbb{E}_q[\ell(\lambda^\star,X)]-\mathbb{E}_q[\ell(\lambda^\star-\alpha,X)]-2\eta \ge 0. \] Hence, for large $n$, \[ \mathbb{E}_{\hat q_n}[\ell(\lambda^\star,X)]\ge \mathbb{E}_{\hat q_n}[\ell(\lambda,X)] \quad\text{for all }\lambda\in[0,\lambda^\star-\alpha]. \] Combining this with the fact that $\lambda\mapsto\mathbb{E}_{\hat q_n}[\ell(\lambda,X)]$ is strictly increasing on $J_\epsilon$, we obtain: for all large $n$ (a.s.), any maximizer $\lambda^\star_n$ of $\lambda\mapsto\mathbb{E}_{\hat q_n}[\ell(\lambda,X)]$ on $(0,\lambda^\star]$ must lie in $[\lambda^\star-\alpha,\lambda^\star]$. Since $\alpha>0$ is arbitrary, this yields \[ \lambda^\star_n\to\lambda^\star\quad\text{a.s.} \] \end{comment} %%%%%% \begin{comment} \begin{verbatim} alpha = 1e-04 Empirical 95% CI for tau: [6.0, 50.0] Single-path 95% CI for tau: [2.0, 41.0] alpha = 1e-08 Empirical 95% CI for tau: [22.0, 84.0] Single-path 95% CI for tau: [11.9, 49.0] \end{verbatim} \end{comment}