
\section{Introduction}
\looseness=-1
When machine learning predictions affect user outcomes, deployed models can induce shifts in the data distribution. These shifts may result from strategic user behavior---where individuals try to secure favorable outcomes such as loan approval or college admission \citep{pmlr-v130-bechavod21a, JMLR:v24:22-0131, pmlr-v202-wang23ap}---or from self-fulfilling prophecies, for example in economic forecasts, recommendation systems, or predictive policing \citep{morgenstern1928wirtschaftsprognose, pmlr-v81-ensign18a,Ursu2015}. Such distribution shifts can undermine predictive performance over time, amplifying bias and reducing model quality \citep{pmlr-v202-taori23a,pmlr-v235-pan24d}. Performative learning \citep{perdomo_performative} addresses this feedback loop by parameterizing the data distribution with the same parameter as the model. This allows optimization to account not only for the training loss but also for the steering of the data distribution.

Unfortunately, while optimizing model parameters is a classical problem in machine learning, estimating the performative effect on the distribution is generally infeasible, as the distribution is unknown to the learner. Several algorithms have been proposed to approximate this effect \citep{miller_outside, izzo2021learn, NEURIPS2024_7de66547}, typically by assuming that it depends on a small number of parameters in a sufficiently simple way that can be learned across the first few deployments. However, these methods are limited to relatively toy examples in small dimensions and may be impractical in high-dimensional settings. In particular, many approaches require numerous repeated deployments, alternating between loss minimization and distribution steering. Yet in practice, deployment often happens only once after full training. This makes repeated risk minimization (RRM) \citep{perdomo_performative}---where one trains until convergence before deployment, and the number of deployments is small---the default in many applications, even though it remains largely unaddressed by existing mitigation methods. This motivates a shift away from exact estimation of performative effects toward the study of principled choices of loss functions and models, and in particular regularization is a natural and tractable candidate. 

In this work, we study how regularization mitigates performative effects in repeated retraining. Unlike estimation-based methods, regularization does not depend on a precise characterization of the distribution shift, avoids their limitations, and introduces little computational overhead. Prior work suggests its potential benefits: \citet{perdomo_performative} proved that retraining converges to an optimal solution under assumptions tied to the strong convexity of the loss, which ridge regularization can enforce; more recently, \citet{NEURIPS2024_7de66547} showed that, in classification, the performative optimum can be interpreted as a regularized version of the non-performative problem, with numerical evidence that ridge penalties perform well in small-dimensional classification tasks. However, these results, as most of the performative literature, do not cover the high-dimensional regime. In particular, regularization may encourage reliance on spurious features \citep{bombari2025spuriouscorrelationshighdimensional} in higher dimension, especially when such features are reinforced by performativity.

To better understand these tradeoffs, we study the role of ridge regularization in linear regression in the presence of performativity and spurious features. We consider both \emph{(i)} the population regime, with enough data to recover exactly the unknown vector of regression coefficients at each deployment, and \emph{(ii)} the over-parameterized regime, where the number of data samples is a fixed fraction of the number of parameters. This last setting, though simple, captures behaviors relevant to deep learning, such as double descent \citep{doi:10.1073/pnas.1903070116,hastie2022surprises}, benign overfitting \citep{Bartlett_2020} and adversarial robustness \citep{fawzi2016analysisclassifiersrobustnessadversarial, ribeiro2023regularization}. In performative learning, it also brings the additional advantage that parameters and data live in the same space, simplifying the encoding of performative effects. The theoretical framework we develop enables us to provide strong evidence for the effectiveness of regularization under performativity, and to show how regularization should be scaled.
More precisely, our contributions are summarized below.
\begin{enumerate}[leftmargin=1em]
\item In the population regime, we characterize how the risk depends on magnitude and direction of the performative effect, as well as on spurious features (Theorem \ref{thm:pop}). We find that the optimal regularization is proportional to the strength of the performative effect and it mitigates the performance loss due to performativity: zero excess risk is achieved with identity covariance and constant entries of the performative vector, while the risk remains significant in the presence of a complex covariance structure and highly variable entries of the performative vector (Corollary \ref{cor:pop}).
\item In the proportional regime with random data, we establish a deterministic equivalent of the performative fixed point, depending only on population covariance and regularization (Theorem \ref{thm:over}). The analysis of this deterministic equivalent then unveils a remarkable phenomenology: for small noise variance, the optimal regularization moves in the same direction as the performative effect on predictive features, while it moves in the opposite direction as the performative effect on spurious features; remarkably, the optimally-regularized risk improves in the presence of a performative effect that reinforces existing trends. 
\item  We illustrate these behaviors on both synthetic data and real-world datasets (Housing, LSAC), showing empirically that our findings provide valuable insights beyond the assumptions in the theory %
and extend to other regularizers.
\end{enumerate}

\vspace{-.5em}
\section{Related work}
\vspace{-.5em}

\paragraph{Performative learning.} Performative learning was introduced by \citet{perdomo_performative}, who showed that retraining converges under assumptions including strong convexity. Subsequent works demonstrated that retraining can enable adaptation over time \citep{li2022, Drusvyatskiy2023, brown2022performative, wang2023constrainedoptimizationdecisiondependentdistributions} but can also fail dramatically \citep{miller_outside, izzo2021learn, NEURIPS2024_7de66547}. Several works evaluate performance on the initial distribution rather than the induced one \citep{demirel2024adjustingpretrainedbackbonesperformativity, Tsoy2025OnTI,pmlr-v202-taori23a}. Label shift is standard in domain adaptation \citep{label2, Cai2021label} and can encode performative effects, such as placebo effects or traffic prediction in \citep{nikolalabel, hardt2023performative}. The role of model choice has been studied in the related setting of collective action \citep{pmlr-v235-ben-dov24a}. Improved performance due to performativity was observed by \citet{pmlr-v130-bechavod21a}, and our work provides further evidence supporting this claim. Finally, performative learning effects tend to be harder to learn in high-dimensional settings, as noted by \citet{pmlr-v162-jagadeesan22a, bracale2025learningdistributionmapreverse}. Our work is the first to use tools from high-dimensional statistics to study performative learning, to our knowledge.

\vspace{-.7em}

\paragraph{High-dimensional regression and role of ridge regularization.} The high-dimensional setting where numbers of features and samples %
scale proportionally was considered by a rich line of work: the test error of ridgeless and ridge regression is characterized by \citet{hastie2022surprises, wu2020optimal,richards2021asymptotics,tsigler2023benign}; %
max-margin classification is studied by   \citet{montanari2019generalization,deng2022model}, 
model compression by \citet{chang2021provable}, distribution shift by \citet{patil2024optimal,mallinar2024minimumnorm}, transfer learning by  \citet{yang2020precise,song2024generalization} and learning from surrogate data by  \citet{kolossovtowards,jain2024scaling,rezaei2025high}.The role of ridge regularization has also been studied. \citet{hastie2022surprises} optimally tune the ridge penalty, while \citet{richards2021asymptotics} give conditions for the optimality of ridgeless interpolation. The sign of the optimal ridge penalty was studied for the standard in-distribution regression setup \citep{wu2020optimal, tsigler2023benign}, as well as out-of-distribution \citep{patil2024optimal}: these works give conditions under which the optimal ridge is negative, associating the phenomenon of negative optimal regularization to over-parameterization. Our paper shows that such a phenomenon occurs also in the population setting, due to performative effects.
The distribution of the empirical risk minimizer was established by  \citet{han2023distribution}. Leveraging this characterization, spurious correlations were studied by \citet{bombari2025spuriouscorrelationshighdimensional} and weak-to-strong generalization by \citet{ildizhigh}. We will also build on these tools to analyze the risk of repeated risk minimization.


\vspace{-.3em}
\section{Preliminaries and problem setup}
\vspace{-.3em}

In this section, we introduce our performative regression setting. We consider a sequence of model deployments $(\theta_k)_{k\ge 0}$, and let $\D(\theta)$ be the dataset generated in reaction to the deployment of $\theta$. At each deployment, $n$ new samples are collected, and the model is fully retrained. %
This setting, known as repeated risk minimization (RRM) \citep{perdomo_performative}, reflects real-world scenarios where deployments are costly and thus limited in number. It also aligns with the fact that convergence to the fixed point is fast and requires only a few iterations to reach equilibrium in practice.
We encode the performative effect as a shift in the label, where each feature’s contribution varies depending on an additional linear term in the model parameter. %

\begin{assumption}[Regression performative model]
For $\theta \in \R^p$, samples from $\D(\theta)$ are taken i.i.d.\ with features $x$ having zero mean and covariance $\Sigma$ %
drawn independently of $\theta$ and with the label $y$ given by
\vspace{-.3em}
\begin{equation}\label{eq:data}    
y = x^\top \thetapop + x^\top D \theta + w, \quad w \sim \N(0, \sigma^2).
\end{equation}
We assume $p = 2d$, $(\thetapop)^{\top} = (a^\top, 0)$ with $a$ having \revised{zero mean and} covariance $I_d/d$, and $D = \di(b, c)$ where $b, c \in \R^d$ with $\|b\|_{\infty}, \|c\|_{\infty} < 1$.
\label{assum:model}
\end{assumption}

This model generalizes the one-dimensional setting (Example 2.2) in \citet{perdomo_performative}, where labels follow a binomial distribution with parameter $\tfrac{1}{2} + x \theta^*  + x \bar{b} \theta$, for $\theta^* \in (0, \tfrac{1}{2})$ and $\bar{b} < \tfrac{1}{2}-\mu$. Focusing on label shifts is natural in regression: it keeps the feature distribution centered and unchanged across deployments (this can be enforced via pre-processing) despite performative effects, and it can encode scenario such as placebo effects \citep{nikolalabel}. We do not cover feature shifts. However, when a feature shift affects all data points in a regression task, replacing $x$ by $x+\theta$ can be absorbed by centering the data, and prior work provides numerical evidence that regularization remains beneficial when feature shifts affect only one class in binary classification~\cite{NEURIPS2024_7de66547}.  %

The performative term $x^\top D \theta$ enforces coordinate-wise effects. This is %
consistent with %
previous works \citep{NEURIPS2024_7de66547,izzo2021learn, hardt2023performative} and also %
close to the model 
$y = x^\top \thetapop + \mu^{\top} \theta + w$ studied by \citet{miller_outside}, where the performative effect does not depend on $x$ but only on a fixed vector $\mu$. Assuming linearity in $\theta$ is reasonable, as performative effects are expected to be moderate to avoid iterations to diverge. We specify in the rest of the paper when the fact that $D$ is diagonal is needed. Intuitively, diagonal coefficients can be interpreted directly as the modifications made by a strategic agent, depending on how the feature is used and the cost of modifying it. Most existing methods impose explicit constraints on the performative effect \citep{miller_outside}, and our setting is no more restrictive: we only require $\|b\|_\infty, \|c\|_\infty < 1$. %
We set the second half of $\thetapop$ to zero to represent spurious features, and $c$ captures the corresponding performative effect. %
This %
enables us to express correlations between predictive and spurious features via %
the block structure of the covariance
\vspace{-.3em}
\begin{equation*}\label{eq:block}
\Sigma=\begin{bmatrix}\Sigma_1&\Sigma_{12}\\ \Sigma_{12} &\Sigma_2\end{bmatrix},  
\end{equation*}
where $\Sigma_1$ denotes the covariance for the predictive part, $\Sigma_2$ the covariance for the spurious part, and $\Sigma_{12}$ the covariance between the two blocks. 
Under this setting, RRM corresponds to solving %
\begin{equation}
    \hspace{-.2em}\theta_{k} = \arg\min_{\theta\in\mathbb{R}^p}\left\{\frac{1}{2n}\sum_{i=1}^n \ell(x_i^{\scriptscriptstyle(k-1)}, y_i^{\scriptscriptstyle(k-1)}; \theta) \hspace{-.2em}+\hspace{-.2em}\frac{\lambda}{2}\|\theta\|_2^2\right\},
    \label{eq:ERM}
\end{equation}
where $\{(x_i^{(k-1)},y_i^{(k-1)})\}_{i=1}^n\stackrel{\mathrm{i.i.d.}}{\sim}\mathcal{D}(\theta_{k-1})$ and $\ell$ is the squared loss. This defines a recurring sequence in both population and over-parameterized regimes. In the population case, the sequence converges in parameter space to a fixed vector $\thetapop$  (\Cref{sec:pop}), while in the over-parameterized case the vector varies at each iteration but the excess risk still converges deterministically (\Cref{sec:over}).

We evaluate the test risk when the final model is deployed on the untouched distribution $\D(\theta = 0)$. %
Testing on $\D(\theta = 0)$ is particularly relevant for long-term fairness, as it prevents bias amplification over time \citep{pmlr-v81-ensign18a, pmlr-v202-taori23a} or steering the distribution toward undesirable regimes that decrease the risk by collapsing the data distribution to make it easier to predict (e.g., reducing entropy or producing a single possible label in a classification task) \citep{Tsoy2025OnTI,demirel2024adjustingpretrainedbackbonesperformativity}. This choice also enables testing whether regularization increases reliance on spurious features \citep{bombari2025spuriouscorrelationshighdimensional}, which one wants to avoid. 
We thus aim to minimize the following excess risk:
\vspace{-.3em}
\begin{equation}
\begin{aligned}
    \mathcal{R}(\Sigma, \theta, \thetapop) := &\E_{\D(\theta = 0)}\left[ (y - x^\top \theta)^2]\right] - \sigma^2 \\=&
    \|\Sigma^{1/2}(\theta-\thetapop)\|_2^2 , 
\end{aligned}
    \label{eq:risk}
\end{equation}
where %
we have subtracted the Bayes risk $\sigma^2$. %
In \Cref{sec:pop}, we analyze this risk in the population setting, where enough data is available to exactly recover the parameter vector and  the optimal solution is $\thetapop$ (as suggested by the notation). %
In \Cref{sec:over}, we then focus on the %
over-parameterized regime where $p>n$. %

We note that, in the population setting, testing on $\D(\theta)$ gives zero excess risk, thus trivializing the problem. We provide results for testing on $\D(\theta)$ in the over-parameterized setting at the end of Section \ref{sec:over}. 

\vspace{-.5em}
\section{Analysis in the population setting}
\label{sec:pop}
\vspace{-.5em}

In this section, we tackle the population regime where there are enough samples from $\D(\theta_k)$ at each deployment to compute exactly the next regressor, as would typically happen in a low-dimensional setting. The sequence $(\theta_k)_k$ is thus deterministically defined by
\vspace{-.3em}
\begin{equation}\label{eq:recpop}
\begin{aligned}
    \theta_k &= (\Sigma +\lambda I_p)^{-1} \E_{(x,y)\sim \D(\theta_{k-1})}[xy]\\ &=  (\Sigma +\lambda I_p)^{-1} (\Sigma \thetapop + \Sigma D \theta_{k-1}),
\end{aligned}
\end{equation}
where in the second equality we plug back the definition of the current data distribution in (\ref{eq:data}).

\begin{figure*}[h]
    \centering
    \begin{subfigure}{0.32\textwidth}
        \includegraphics[width=\linewidth]{fig/Population/risk_vs_lambda_bars_d100_n500000_rho0_b-0.1_c0_noise0.1.pdf}
        \caption{\(\bar b = -0.1\)}
        \label{fig:expandtheo}
    \end{subfigure}
    \hfill
    \begin{subfigure}{0.32\textwidth}
        \includegraphics[width=\linewidth]{fig/Population/risk_vs_lambda_bars_d100_n500000_rho0_b0_c0_noise0.1.pdf}
        \caption{\(\bar b = 0\)}
        \label{fig:covariance}
    \end{subfigure}
    \hfill
    \begin{subfigure}{0.32\textwidth}
        \includegraphics[width=\linewidth]{fig/Population/risk_vs_lambda_bars_d100_n500000_rho0_b0.2_c0_noise0.1.pdf}
        \caption{\(\bar b = 0.2\)}
        \label{fig:c}
    \end{subfigure}
\vspace{-.3em}
    \caption{Excess risk at the performative fixed point $\theta^{\infty}$ in (\ref{eq:fppop}), as a function of ridge regularization $\lambda$, for $d = 100$, $\Sigma=I_p$, entries of $b$ uniform in \(\left[\min\{0, 2 \bar b\}, \max\{0, 2 \bar b\} \right]\), $c=0$ and $\sigma = 0.1$. Empirical values (blue dots) are computed from 20 i.i.d.\ trials on $a$ and 5 i.i.d.\ trials on $b$, with error band at 1 standard deviation. Theoretical predictions (blue dashed curves) are from (\ref{eq:fppopavg}) and match perfectly empirical values. First-order approximations (orange dashed curves) are given by $\widetilde{\mathcal R}_{\rm pop}(D, \lambda, \Sigma)$ in (\ref{eq:popapex}) and still provide a good match when $\lambda$ is near-optimal. The green vertical line is the optimal regularization obtained by numerically optimizing the excess risk of $\theta^\infty$ ($\lambda^*_{\rm emp}$), the red one is the first-order approximation ($\lambda_{\rm pop}^*$ from (\ref{eq:optpopap1})) and the violet one the second-order approximation ($\lambda^*_{{\rm pop}, 2}$ minimizing (\ref{eq:pophigh})).}
    \label{fig:pop1}
\end{figure*}


\vspace{-.5em}

\paragraph{Excess risk at the performative fixed point.}
By unrolling (\ref{eq:recpop}), \revised{for any arbitrary (possibly non-diagonal) matrix $D$}, we have that the sequence $(\theta_k)_k$ converges at an exponential rate to the fixed point 
\vspace{-.3em}
\begin{equation}\label{eq:fppop}
\theta^{\infty} = (I_p +\lambda \Sigma^{-1} - D)^{-1}\thetapop .    
\end{equation}
The formal statement, including an explicit convergence rate, is deferred to Lemma \ref{lemma:cr} in Appendix \ref{app:pop}. %
By inserting (\ref{eq:fppop}) into (\ref{eq:risk}) and taking the expectation with respect to $\thetapop$, we have
\vspace{-.3em}
\begin{equation}\label{eq:fppopavg}
\begin{split}
\E_{\thetapop}\mathcal{R}(\Sigma, \theta^{\infty}, \thetapop) ={}&\E_{\thetapop}[(\thetapop)^\top A^\top \Sigma A \thetapop] \\&{}=\frac{1}{d} \tr\left[(A^\top \Sigma A)_{1}\right],
\end{split}
\end{equation} 
where we define $A := (\Sigma + \lambda I_p- \Sigma D )^{-1} \Sigma - I_p$, use $(\thetapop)^{\top} = (a^\top, 0)$ with $a$ having \revised{zero mean and} covariance $I_d/d$ and, given a $p\times p$ matrix $M$, denote by $(M)_1$ its top-left $d\times d$ block. %
This leads to the following approximation for the excess risk, proved in Appendix~\ref{app:pop}. 
\begin{theorem}[Excess risk -- population]\label{thm:pop}
    Let $F = D - \lambda \Sigma^{-1}$. Then, %
    we have %
\vspace{-.3em}
\begin{equation}\label{eq:popapex}
    \begin{split}
\E_{\thetapop}\mathcal{R}(\Sigma, \theta^{\infty}, \thetapop) =
\widetilde{\mathcal R}_{\rm pop}(D, \lambda, \Sigma)  + O(\|F\|_{\mathrm{op}}^2),\\
\widetilde{\mathcal R}_{\rm pop}(D, \lambda, \Sigma)  :=  \frac{1}{d}\tr[\di(b^2)\Sigma_1]  -  2 \lambda \bar{b}  +  \frac{1}{d}\lambda^2 \tr(S_1),
    \end{split}
\end{equation}
where $\|\cdot\|_{\mathrm{op}}$ denotes the operator norm, $\bar{b} := \frac{1}{d}\tr[\di(b)] =
\frac{1}{d}\sum_{i=1}^d b_i$, $b^2:=[b_1^2, \ldots, b_d^2]\in\mathbb R^d$ and $S_1 = (\Sigma_1 - \Sigma_{12}\Sigma_2^{-1} \Sigma_{21})^{-1}$ is the Schur complement of $\Sigma$.
\end{theorem}

The matrix $F = D - \lambda \Sigma^{-1}$ naturally appears in the computation, and its norm can be bounded explicitly as a function of $b$, $c$, $\lambda$ and $\Sigma$ by applying  Weyl's inequality (see Lemma \ref{lemma:weyl} in Appendix \ref{app:pop}) and ensuring that the approximation $O(\|F\|_{\mathrm{op}}^2)$ is tighter than  $O(\max(\|b\|_{\infty}, \|c\|_{\infty}, \lambda)^2)$, since $b$ and $c$ can be partially canceled by $\lambda$. In fact, this cancellation occurs when $\lambda$ is near-optimal, resulting in an accurate approximation, see Figure \ref{fig:pop1}.



\vspace{-.5em}



\begin{figure*}[tbh]
    \centering
    \begin{subfigure}{0.32\textwidth}
        \includegraphics[width=\linewidth]{fig/Population/lambda_vs_b_by_rho_d100_n500000_rhos_all_noise0.1.pdf}
        \caption{Spurious correlations}
        \label{fig:spurious}
    \end{subfigure}
    \hfill
    \begin{subfigure}{0.32\textwidth}
        \includegraphics[width=\linewidth]{fig/Population/opt_risk_vs_b_by_bstd_d100_n500000_bst_ds_all_noise0.1.pdf}
        \caption{Variance of entries of $b$}
        \label{fig:variance}
    \end{subfigure}
    \hfill
    \begin{subfigure}{0.32\textwidth}
        \includegraphics[width=\linewidth]{fig/Population/lambda_vs_b_by_c_d100_n500000_cs_all_noise0.1.pdf}
        \caption{Spurious performative effect}
        \label{fig:spurperfo}
    \end{subfigure}
       \vspace{-.3em} \caption{Optimal regularization and risk for the performative fixed point $\theta^{\infty}$ in (\ref{eq:fppop}), with $d = 100$, $\Sigma_1=\Sigma_2=I_d$, $\Sigma_{12}=\rho I_d$. Values are computed from 20 i.i.d.\ trials on $a$ and 5 i.i.d.\ trials on $b$, with error band at 1 standard deviation. (a) Optimal regularization as a function of $\bar b$ for $\rho\in \{0, 0.4, 0.8\}$. The entries of $b$ are uniform in \(\left[\min\{0, 2 \bar b\}, \max\{0, 2 \bar b\} \right]\) and $c = 0$. (b) Optimal risk as a function of $\bar b$. Different curves correspond to
       different variances $\sigma_b^2$ of the entries of $b$, which  are uniform in \(\left[\bar b  - \sigma_b \sqrt{3}, \bar b + \sigma_b \sqrt{3} \right]\) for $\sigma_b\in\{0, 0.2, 0.4\}$. We pick $c=0$ and $\rho=0$. We note that, when 
       $\rho=0$, ${\mathcal R}_{\rm pop}^*(D, \Sigma)$ equals the empirical variance of the entries of $b$ and, as such, it does not depend on $\bar b$.  (c) Optimal regularization as a function of $\bar b$ for $\bar{c}\in \{-0.3, 0, 0.3\}$. The entries of $b$ are uniform in \(\left[\min\{0, 2 \bar b\}, \max\{0, 2 \bar b\} \right]\), the entries of $c$ are uniform in \(\left[\min\{0, 2 \bar c\}, \max\{0, 2 \bar c\} \right]\),  and $\rho = 0.5$.}
    \vspace{-1em}
    \label{fig:pop2}
\end{figure*}


\paragraph{Optimal regularization and optimally regularized risk.}

Leveraging the risk expression for small $F$ of Theorem \ref{thm:pop}, 
we next study the behavior of the optimal regularization and of the corresponding optimal risk. 
Formally, define
\vspace{-.3em}
\begin{equation}\label{eq:optpopdef}
\begin{split}
    \lambda^*_{\rm pop}(D, \Sigma):=\arg\min_{\lambda\in\mathbb R}\widetilde{\mathcal R}_{\rm pop}(D, \lambda, \Sigma),\\
        {\mathcal R}_{\rm pop}^*(D, \Sigma):=\min_{\lambda\in\mathbb R}\widetilde{\mathcal R}_{\rm pop}(D, \lambda, \Sigma).
\end{split}
\end{equation}
From (\ref{eq:popapex}), we note that $\widetilde{\mathcal R}_{\rm pop}(D, \lambda, \Sigma)$ is quadratic in $\lambda$, so the minimization in (\ref{eq:optpopdef}) can be solved explicitly, leading to the expressions below. %

\begin{corollary}[Optimal regularization -- population]\label{cor:pop}
    In the setting described above, we have %
\vspace{-.3em}
    \begin{equation}
        \begin{split}
    \label{eq:optpopap1}     
    \lambda^*_{\rm pop}(D, \Sigma) ={}& \frac{\bar{b}d}{\tr(S_1)}, \\  {\mathcal R}_{\rm pop}^*(D, \Sigma)={}&\frac{1}{d}\tr(\di(b^2)\Sigma_1) - \frac{\bar{b}^2d}{\tr(S_1)} .
    \end{split}
    \end{equation}
\end{corollary}
These formulas call for several comments. First, the optimal regularization $\lambda^*_{\rm pop}(D, \Sigma)$ is proportional to the strength of the performative effect $\bar{b}$, see Figure  \ref{fig:spurious} (and also the location of the minima in Figure \ref{fig:pop1}). %
The fact that $\lambda^*_{\rm pop}(D, \Sigma)$ grows with $\bar b$ captures an effect common in practice: when performativity reinforces existing trends%
—corresponding to ``rich-get-richer'' phenomena, such as a feature becoming more important over successive deployments—the optimal regularizer increases and helps to limit this effect. Conversely, when the performative effect already mitigates the influence of some feature, the optimal solution calls for less regularization. In the population case, this corresponds to negative regularization (see Figure \ref{fig:expandtheo}) which, although less common, has also been studied in the literature \citep{wu2020optimal,tsigler2023benign,patil2024optimal}. Spurious correlations tend to reduce the optimal regularization, although their effect is mild, see Figure \ref{fig:spurious}. 

Second, the optimal risk ${\mathcal R}_{\rm pop}^*(D, \Sigma)$ is always positive and, thus, worse than in the non-performative scenario, where it is zero. More specifically, zero excess risk can only be reached with a non-zero $b$ if $\Sigma = I_p$ and $b$ is aligned with the all-1 vector, see the blue line in Figure \ref{fig:variance}. Intuitively, as regularization impacts all features equally, it better compensates performativity in this uniform case. If the variance in the entries of $b$ grows, then ${\mathcal R}_{\rm pop}^*(D, \Sigma)$ increases, see Figure \ref{fig:variance}. %
We finally note that ${\tilde{\mathcal R}}_{\rm pop}(D, \lambda, \Sigma)$ does not depend on $c$ and, in fact, the effect of $c$ is only visible at higher order, as seen in this formula, proven in Appendix \ref{app:pop}:
\begin{equation}\label{eq:pophigh}    
\begin{aligned}
&\E_{\thetapop}\mathcal{R}(\Sigma, \theta^{\infty}, \thetapop) =\frac{1}{d} \biggl(\hspace{-0.15em} -2\lambda^3\tr \bigl[\left( \Sigma^{-2}\right)_1\bigr] \hspace{-0.15em}+\hspace{-0.15em} \lambda^2 \bigl(\tr\!\left[S_1\right]\\& + 6 \tr \bigl[\di(b) S_1\bigr]\bigr) - \lambda \Bigl(2\tr\!\bigl[\di(b)\Sigma_1\di(b)S_1\bigr] \\&
         + 2\tr\!\bigl[\di(b)\Sigma_{12}\di(c)S_{21}\bigr]
       + 2 d \bar b + 4\tr[\di(b^2)] \Bigr)\\
    & + \tr\left[\di(b^2)\Sigma_1\right] + 2\tr\left[\di(b^3) \Sigma_1\right]\biggr) + O(\|F\|_{\mathrm{op}}^4),
\end{aligned}
\end{equation}
with $S_{21}^\top = -(\Sigma_1 - \Sigma_{12}\Sigma_2^{-1} \Sigma_{21})^{-1}\Sigma_{12} \Sigma_2^{-1}$. %
Figure \ref{fig:pop1} illustrates that the minimizer of this second-order approximation ($\lambda^*_{{\rm pop}, 2}$) is close to the minimizer obtained numerically ($\lambda^*_{\rm emp}$). While $c$ tends to steer the optimal regularizer in the opposite direction (less regularization in the case of a self-reinforcing performative effect), it does so only through the cross term $2\tr\!\left[\di(b)\Sigma_{12}\di(c)S_{21}\right]$, which also depends on $b$. \Cref{fig:spurperfo} shows that $c$ moves the optimal regularization in a direction opposite to its sign, but its effect remains rather limited. 



\section{Analysis in the over-parameterized setting}
\label{sec:over}



Next, we consider the case where $n, p$ are both large and scale proportionally, with $p/n=\kappa>1$. All \emph{constants} (e.g., $R, M$) are intended to be positive values independent of $n, p$. For mathematical convenience, we opt for a different normalization w.r.t.\ (\ref{eq:ERM}), and the estimator $\theta_{k}$ is given by
\begin{equation}    
\theta_{k} = \arg\min_{\theta\in\mathbb{R}^p}\left\{\frac{1}{2p}\sum_{i=1}^n \ell\left(x_i^{(k-1)}, y_i^{(k-1)}; \theta\right) +\frac{\lambda}{2}\|\theta\|_2^2\right\}.
\end{equation}
Solving for $\theta_{k}$ yields
\begin{equation}\label{eq:thetak}    
\theta_{k} = \frac{1}{p} \left(\frac{1}{p} X^{\scriptscriptstyle(k-1)  \top} X^{\scriptscriptstyle(k-1)} + \lambda I_p\right)^{-1} \hspace{-1em} X^{\scriptscriptstyle(k-1) \top} y^{\scriptscriptstyle(k-1)},
\end{equation}
where $X^{(k-1)}=[x_1^{(k-1)}, \ldots, x_n^{(k-1)}]\in \mathbb R^{n\times p}$ and $y^{(k-1)}=[y_1^{(k-1)}, \ldots, y_n^{(k-1)}]\in\mathbb R^n$. Note that, when $\theta_k$ is given by \eqref{eq:thetak}, the risk $\mathcal{R}(\Sigma, \theta_k, \thetapop)$ as defined in \eqref{eq:risk} is a random quantity since the data $\{X^{(\ell)}\}_{\ell=1}^{k-1}$ and the noise contained in the labels $\{y^{(\ell)}\}_{\ell=1}^{k-1}$ are random. This makes it challenging to characterize optimal ridge penalty and optimally-tuned risk. To address the challenge, we first establish a \emph{deterministic} equivalent of the risk at the performative fixed point. We next optimize such deterministic equivalent and study the effect of performativity on the optimal regularization.  

\paragraph{Deterministic equivalent of the performative fixed point.} First, note that, if we regard the performative effect as small and aim at characterizing its effect on the fixed point up to the leading (first) order, it suffices to do two iterations of the recursion in \eqref{eq:thetak}. In fact, the labels $y^{(k-1)}$ are linear in $D\theta_{k-1}$, so we expect that, after two iterations, the performative fixed point is reached up to fluctuations of order $O(\|D\|_{\rm op}^2)$. Now, the risk after two iterations is still a random quantity, so we apply techniques from \cite{han2023distribution,ildizhigh} to derive a %
deterministic equivalent. The formal statement is below and the proof is deferred to Appendix \ref{app:pf}.
\begin{theorem}[Excess risk -- over-parameterized]\label{thm:over}
\revised{Let Assumption \ref{assum:model} hold with $x \sim \N(0, \Sigma)$.} Let $R>0$ be a constant s.t.\ $\|\thetapop\|_2, \|\theta_0\|_2\le R$. Assume that $\kappa, \sigma, \lambda\in (1/M, M)$ and $\|\Sigma\|_{\mathrm{op}},\ \|\Sigma^{-1}\|_{\mathrm{op}} \le M$ for some constant $M>1$. Then, there exists a constant $C=C\left(M, R\right)$ such that for any $\delta \in (0,1/2]$, with probability at least $1-Cpe^{-p\delta^{4}/C}$, %
\begin{equation}    
\left|\mathcal{R}(\Sigma, \theta_{2}, \thetapop)-\fixedriskeq\left(\Sigma, \thetapop, D, \lambda\right)\right|\le \delta+O(\|D\|_{\mathrm{op}}^2),
\end{equation}
where
\begin{equation}   \label{eq:defdet} 
\resizebox{.49\textwidth}{!}{$\displaystyle
\begin{aligned}
&\fixedriskeq\left(\Sigma, \thetapop, D, \lambda\right)
= 
\tau
\langle \thetapop, \Xi\left(\tau I_p
-2
\Xi
\Sigma^2 D
\right)\Sigma \Xi
\thetapop\rangle
\\
&
\hspace{-.6em}+\hspace{-.2em} \kappa
\tr\left[\Sigma^{2} \Xi^{2}\right]
\hspace{-.2em}\frac{
\sigma^{2}
\hspace{-.2em}+\hspace{-.2em} 
\tau^{2}
\langle \thetapop,\Xi 
\left(I_p + 2 \Xi
\Sigma D\right)\Sigma \Xi \thetapop\rangle
}{
p - \kappa
\tr\left[\Sigma^{2}\Xi^{2}\right]
},
\end{aligned}$}
\end{equation}
with $\Xi = (\Sigma + \tau I_p)^{-1}$ %
and $\tau$ is the unique solution of %
\begin{equation}   \label{eq:tau} 
\kappa^{-1} - \frac{\lambda}{\tau} = \frac1p \tr\left[(\Sigma + \tau I_p)^{-1}\Sigma\right].
\end{equation}
\end{theorem}


In words, Theorem \ref{thm:over} shows that the risk $\mathcal{R}(\Sigma, \theta_2, \thetapop)$ is well approximated by the quantity \revised{$\fixedriskeq\left(\Sigma, \thetapop, D, \lambda\right)$} defined in \eqref{eq:defdet}. We highlight that this quantity does not depend on the initialization $\theta_0$: up a fluctuation of order $O(\|D\|_{\mathrm{op}}^2)$, \emph{the risk has reached a fixed point} after two iterations. While the data (and, consequently, $\mathcal{R}(\Sigma, \theta_2, \thetapop)$) are random, $\fixedriskeq\left(\Sigma, \thetapop, D, \lambda\right)$ provides a \emph{deterministic equivalent} that depends only on the population covariance $\Sigma$, the ground-truth vector $\thetapop$, the matrix $D$ capturing the performative effect and the regularization $\lambda$.

We note that the result of Theorem \ref{thm:over} holds for a general matrix $D$ with bounded operator norm---not necessarily a diagonal $D$ as in Assumption \ref{assum:model}.
The assumptions ($\|\thetapop\|_2, \|\theta_0\|_2\le R$, $\kappa, \sigma, \lambda\in (1/M, M)$, $\|\Sigma\|_{\mathrm{op}},\ \|\Sigma^{-1}\|_{\mathrm{op}} \le M$) are all standard in the related literature \citep{han2023distribution,ildizhigh}. We could handle the ridgeless case $\lambda=0$ in a similar way to \cite{han2023distribution,ildizhigh}. However, this requires changing some details and we have opted to avoid the notation clutter, since our focus is on the effect of regularization. 
The assumption on the features $x$ being Gaussian can be also relaxed. In fact, the results of \cite{han2023distribution} (see Theorem 2.4 therein) hold for $\Sigma^{-1/2}x$ having independent, zero mean, unit variance and uniformly subgaussian entries. We prove a formal extension of Theorem \ref{thm:over} to sub-Gaussian data in Appendix \ref{app:extension}. 


\begin{figure*}[t]
    \centering
    \begin{subfigure}{0.32\textwidth}
        \includegraphics[width=\linewidth]{fig/prop/riskfinal.pdf}
        \caption{$\kappa=1.1$, $\sigma=0.2$, $\rho=0$}
        \label{fig:propa}
    \end{subfigure}
    \hfill
    \begin{subfigure}{0.32\textwidth}
        \includegraphics[width=\linewidth]{fig/prop/risknoisynew.pdf}
                \caption{$\kappa=1.1$, $\sigma=0.7$, $\rho=0$}
        \label{fig:propb}
    \end{subfigure}
    \hfill
    \begin{subfigure}{0.32\textwidth}
        \includegraphics[width=\linewidth]{fig/prop/riskcovfinal.pdf}
        \caption{$\kappa=2$, $\sigma=0.5$, $\rho=0.5$}
        \label{fig:propc}
    \end{subfigure}
       \vspace{-.3em} \caption{Excess risk as a function of ridge regularization $\lambda$ with Gaussian data, for $n = 4000$, $\Sigma_1=\Sigma_2=I_d$, $\Sigma_{12}=\rho I_d$, entries of $b$ equal to $\bar b$, and entries of $c$ equal to $\bar c$.  Values are computed from 20 i.i.d.\ trials, with error band at 1 standard deviation. We perform $5$ steps of RRM to approximate the fixed point, as in the simulation setup of Section \ref{sec:num}. (a) In the low-noise regime ($\sigma=0.2$), taking $\bar b=0.2$ instead of $\bar b=0$ \emph{increases the optimal regularization} and \emph{reduces the optimal risk}. We set $\bar c=0$ to emphasize the dependence on $\bar b$. (b) In the large-noise regime ($\sigma=0.7$), taking $\bar b=0.2$ instead of $\bar b=0$ \emph{reduces both optimal regularization and optimal risk}. As in (a), we set $\bar c=0$. (c) Taking $\bar c=0.2$ instead of $\bar c=0$ \emph{reduces the optimal risk}, although the impact of $\bar c$ is less pronounced. We set $\bar b=0$ to emphasize the dependence on $\bar c$.
       }
    \vspace{-1em}
    \label{fig:prop}
\end{figure*}

\paragraph{Optimal regularization and optimally regularized risk.} Leveraging the characterization of Theorem \ref{thm:over}, we optimize the ridge regularization. We focus on the case $\Sigma=\begin{bmatrix}I_d&\rho I_d\\ \rho I_d&I_d\end{bmatrix}$ for small $\rho$ \revised{and require $D$ to be diagonal as in Assumption \ref{assum:model}}. While simplified, this setting captures the performative effect of both predictive and spurious features\revised{, which are mixed via the covariance matrix $\Sigma$}, leading to an interesting phenomenology. Lemma \ref{lemma:explicit} in Appendix \ref{app:pfequiv} computes $\mathbb E_{\thetapop}\fixedriskeq\left(\Sigma, \thetapop, D, \lambda\right)$, as well as the following expansion in $\rho$:
\vspace{-.3em}\begin{equation}\label{eq:expansion}
\begin{split}    
&\mathbb E_{\thetapop}\fixedriskeq\left(\Sigma, \thetapop, D, \lambda\right)=\widetilde{\mathcal R}(D, \lambda, \rho)+ O(\bar b\rho^2+ \rho^4),\\&
 \widetilde{\mathcal R}(D, \lambda, \rho):=\mathcal R_0(\lambda, \rho) + \bar b  A_1(\lambda) %
 + \bar c \rho^2 A_2(\lambda),
\end{split}
\end{equation}
with $\bar b=\tr[\di(b)]/d, \bar c=\tr[\di(c)]/d$. %
Explicit expressions for $\mathcal R_0(\lambda, \rho), A_1(\lambda)$ and $A_2(\lambda)$ are given in \eqref{eq:explexpr} in Appendix \ref{app:pfequiv}. We note that $\mathbb E_{\thetapop}\fixedriskeq\left(\Sigma, \thetapop, D, \lambda\right)$ is even in $\rho$, hence the odd powers of $\rho$ are absent from \eqref{eq:expansion}. Now, we define  optimal regularization and %
risk as 
\vspace{-.3em}\begin{equation}
\begin{split}
    \lambdaeqs(D, \rho):=\arg\min_{\lambda\ge 0}\widetilde{\mathcal R}(D, \lambda, \rho),\\
        \fixedriskeqs(D, \rho):=\min_{\lambda\ge 0}\widetilde{\mathcal R}(D, \lambda, \rho).
\end{split}
\end{equation}
Our goal is to characterize the performative effect on $\lambdaeqs(D, \lambda, \rho), \fixedriskeqs(D, \lambda, \rho)$ and, to do so, we compare these quantities to their values when $D=0$, defined as
\vspace{-.3em}\begin{equation}
    \lambdaeqsz(\rho):=\arg\min_{\lambda\ge 0}\mathcal R_0(\lambda, \rho),
        \fixedriskeqs(\rho)=\min_{\lambda\ge 0}\mathcal R_0(\lambda, \rho).
\end{equation}
This is formalized by the result below whose proof is deferred to Appendix \ref{app:pfequiv}.
\begin{theorem}[Optimal regularization -- over-parameterized]\label{thm:equiv}
In the setting described above, we have 
\vspace{-.3em}\begin{align}
     \begin{aligned}\lambdaeqs(D, \rho)&{}= \lambdaeqsz(\rho)+\bar b (B_1(\sigma, \kappa)+O(\rho^2)) \\&+\bar c \rho^2( C_1(\sigma, \kappa)+O(\rho^2))+O(\bar b^2+\bar c^2)
     ,\label{eq:thmequivl}\end{aligned}\\
    \begin{aligned}\fixedriskeqs(D, \rho)&{}= \fixedriskeqs(\rho)+\bar b (B_2(\sigma, \kappa)+O(\rho^2))\\&+\bar c \rho^2( C_2(\sigma, \kappa)+O(\rho^2))+O(\bar b^2+\bar c^2),
    \end{aligned}\label{eq:thmequivR}
\end{align}
where the functions $B_1(\sigma, \kappa), B_2(\sigma, \kappa), C_1(\sigma, \kappa), C_2(\sigma, \kappa)$ depend only on $\sigma, \kappa$ and they are explicitly given in  \eqref{eq:lfor}-\eqref{eq:Rfor}. Furthermore, these functions satisfy
\vspace{-.3em}\begin{align}
    B_1(\sigma, \kappa)&\ge 0 \quad \text{for } 0\le \sigma\le \sigma_{B_1}(\kappa), \,\,\kappa>1,\label{eq:relations1a}\\ B_1(\sigma, \kappa)&\le 0 \quad \text{for }  \sigma> \sigma_{B_1}(\kappa), \,\,\kappa>1,\label{eq:relations1b}\\
C_1(\kappa,\sigma)&\le 0 \quad \text{for } \sigma\ge 0, \kappa\ge 2,\label{eq:relations2}\\
B_2(\kappa,\sigma)&\le 0 \quad \text{for } \sigma\ge 0, \kappa> 1,\label{eq:relations3}\\
C_2(\kappa,\sigma)&\le 0 \quad \text{for } \sigma\ge 0, \kappa> 1,\label{eq:relations4}
\end{align}
with $\sigma^2_{B_1}(\kappa)=1/2-7\kappa^{-1}/18 +O(\kappa^{-2})$.
\end{theorem}



In words, \eqref{eq:thmequivl} gives a quantitative comparison between optimal regularization with performative effect ($\lambdaeqs(D, \rho)$) and without it ($\lambdaeqsz(\rho)$). Similarly, \eqref{eq:thmequivR} compares optimally-regularized risks $\fixedriskeqs(D, \rho)$ and $\fixedriskeqs(\rho)$ respectively with and without performativity. The study of the signs of the  auxiliary functions $B_1(\sigma, \kappa), B_2(\sigma, \kappa), C_1(\sigma, \kappa), C_2(\sigma, \kappa)$ leads to the considerations below:
\begin{itemize}[leftmargin=1em]
    \item \Cref{eq:relations1a,eq:relations1b} imply that \emph{(i)} if the noise variance $\sigma^2$ is small, then the optimal regularization moves in the same direction as the performative effect on the predictive features; and \emph{(ii)}  if the noise variance is large, the effect is reversed and the optimal regularization moves in the opposite direction to the performative effect. From a Bayesian perspective, informally, this acts as the noise level controlling the model's confidence: if the noise variance increases, the model moves back towards its "prior", and thus the shift in the regularization due to performativity decreases. This is illustrated in Figures \ref{fig:propa} and \ref{fig:propb}.      
    \item \Cref{eq:relations2} implies that, when $\kappa\ge 2$, the optimal regularization moves in the opposite direction to the performative effect on the spurious features. This effect is however significantly attenuated by the factor $\rho^2$ multiplying $\bar c$ in \eqref{eq:thmequivl} and, as such, it is hardly noticeable both with Gaussian data (as considered in this section) and in real-world settings (as considered in Section \ref{sec:num}). %
    \item \Cref{eq:relations3} implies that, when performativity reinforces existing trends ($\bar b>0$), the optimally-regularized risk improves in the presence of a performative effect on the predictive features. This occurs regardless of the size of the noise variance, and it is illustrated in Figures \ref{fig:propa} and \ref{fig:propb}. Instead, when performativity dampens existing trends ($\bar b<0$), the effect is reversed and the optimal risk worsens.
    \item Finally, \Cref{eq:relations4} implies that the dependence of the optimally-regularized risk on the performative effect on the spurious features is analogous: the optimal risk decreases when $\bar c>0$, and increases when $\bar c<0$. However, as for $\lambdaeqs(D, \rho)$, the impact of performativity on $\fixedriskeqs(D, \rho)$ is less pronounced for the spurious features, due to the factor $\rho^2$ multiplying $\bar c$ in \eqref{eq:thmequivR}. This is illustrated in Figure \ref{fig:propc}.
\end{itemize}


\paragraph{\revised{Evaluating the model on the shifted distribution.}} By following similar steps, we can also provide an analysis in the over-parameterized setting of the model tested on the shifted distribution (i.e., on the same distribution used to train it). We defer the details to Appendix \ref{app:test-bis} and summarize the main results below. Theorem \ref{thm:over2} is the equivalent of Theorem \ref{thm:over}, and it provides a deterministic equivalent of the performative fixed point. Lemma \ref{lemma:explicit-bis} specializes the risk expression from Theorem \ref{thm:over2} to a covariance of the form $\Sigma=\begin{bmatrix}I_d&\rho I_d\\ \rho I_d&I_d\end{bmatrix}$, and it is the equivalent of Lemma \ref{lemma:explicit} (which gives \eqref{eq:expansion}). Next, we provide expressions for the optimal $\tau$ in Lemma \ref{lemma:taustar-bis}, for the optimal regularization parameter in \eqref{eq:lambdastar-bis}-\eqref{eq:lfor-bis}, and for the optimal risk in Corollary \ref{cor:risk-bis}. Lemma \ref{lemma:B1-bis} shows that the optimal regularization moves in the same direction as the performative effect on the predictive features, as it is the case for testing over $\mathcal D(\theta=0)$ provided that the noise variance is  enough. Finally, Lemmas \ref{lemma:B2-bis} and \ref{lemma:C2-bis} show that the optimally regularized risk worsens in the presence of a performative effect (either on the predictive or on the spurious features) that reinforces existing trends ($\bar b, \bar c>0$). This is in contrast with the behavior of the optimally regularized risk tested over $\mathcal D(\theta=0)$, which instead decreases when either $\bar b>0$ or $\bar c>0$. 

\vspace{-.5em}


\section{Numerical experiments}\label{sec:num}

\vspace{-.5em}

\begin{figure*}[h!tb]
    \centering
    \begin{subfigure}{0.32\textwidth}
        \includegraphics[width=\linewidth]{fig/real/housingbis.pdf}
        \caption{Housing ($n=4000$)}
        \label{fig:housing}
    \end{subfigure}
    \hfill
    \begin{subfigure}{0.32\textwidth}
        \includegraphics[width=\linewidth]{fig/real/lsacmanysamplesbis.pdf}
        \caption{LSAC ($n=4000$)}
        \label{fig:LSACmany}
    \end{subfigure}
    \hfill
    \begin{subfigure}{0.32\textwidth}
        \includegraphics[width=\linewidth]{fig/real/lsacfewsamplesbis.pdf}
        \caption{LSAC ($n=100$)}
        \label{fig:LSACfew}
    \end{subfigure}
       \vspace{-.3em} \caption{Excess risk as a function of ridge regularization $\lambda$ in real-world datasets (Housing, LSAC). Different curves (in different colors) correspond to different values of $\bar{b} \in \{0, 0.05, 0.1, 0.15, 0.2\}$, and we connect with a red dashed line the optima of the risk for various choices of $\bar b$. %
       The plots in (a)-(b) use $n=4000$ data points at each training step, which corresponds to the population setting ($n \gg d$); the plot in (c) uses $n=100$, a value closer to the number of features $d=22$.}
    \vspace{-1em}
    \label{fig:real}
\end{figure*}

In this section, we test the effect of regularization and performative shifts on real data. Since no dataset currently provides a real performative shift, the shift must be encoded synthetically. In practice, we take a real-world dataset, randomly split the samples across time steps, train a model on one split, compute the parameter $\theta$, and then shift the samples of the next split according to the theoretical model. This methodology follows previous work \citep{perdomo_performative, hardt2023performative, zezulka_performativity_2023}. These experiments allow us to test whether the theory remains predictive when \emph{(i)} the data is not i.i.d.\ random and $\thetapop$ is fixed by the task, when \emph{(ii)} the true relationship between the feature and the target is likely non-linear, and when \emph{(iii)} the regularization is not the ridge penalty. The code to reproduce the experiments is available at \url{https://github.com/totilas/regularization-vs-perf}.

We consider two datasets. First, we use the Housing dataset,\footnote{\url{https://www.openml.org/d/823}} where the goal is to predict house prices from housing features and local demographics. We follow the methodology of \citet{NEURIPS2024_7de66547} to choose performative features. The dataset has $8$ features and $20{,}640$ datapoints, which we split into five folds: four for training and one for test. Four training steps suffice experimentally to reach the fixed point, which is consistent with the theory, where the first-order effect stabilizes after only two iterations.  
Second, we use the Law School Admission Council (LSAC) dataset,\footnote{\url{https://storage.googleapis.com/lawschool_dataset/bar_pass_prediction.csv}} where the default task is to predict bar passage from demographic features and previous grades. We change the target to GPA to maintain a regression task, and randomly choose features affected by performativity. After dropping redundant columns or those too correlated with GPA, the dataset has $22$ features and $20{,}427$ samples, which again we split in five folds. We report detailed pre-processing, parameters and data covariance in Appendix \ref{app:real}.

When $n=4000$, both for the Housing (Figure \ref{fig:housing}) and the LSAC (Figure \ref{fig:LSACmany}) dataset, we note that \emph{(i)} the optimal regularizer increases proportionally to $\bar{b}$, and \emph{(ii)} the optimally-regularized risk becomes worse as $\bar{b}$ grows. This can be attributed to the fact that $n\gg d$, and it is consistent with our theoretical results in the population setting (Corollary \ref{cor:pop}). In contrast, when training with very few samples on LSAC (\Cref{fig:LSACfew}), the behavior of the regularized risk follows the predictions of Theorem \ref{thm:over} for the proportional setting in the large-noise regime: as $\bar{b}$ grows, the optimal regularizer gets smaller and the risk improves. 
Note that, even if Figures \ref{fig:LSACmany} and \ref{fig:LSACfew} consider the same dataset, the ranges of excess risk and regularizer are not the same due to the different sample sizes. We did not find numerical evidence for the role of $c$, suggesting that its effect may be dominated by data noise, consistent with our theoretical findings on the limited impact of spurious features.  


In Appendix \ref{sec:fifig} we showcase the results of the same experiments when the ridge regularization is replaced with \emph{(i)} dropout (\Cref{fig:dropout}), \emph{(ii)} Lasso regularization (\Cref{fig:lasso}), and \emph{(iii)} elastic net (\Cref{fig:elasticnet}). This demonstrates that the relationship between regularization and performative strength persists beyond ridge regression.

To further assess whether our findings extend beyond linear models, we also run an experiment with neural networks. We follow the strategic classification setting of~\citet{pmlr-v206-mofakhami23a}, using the \texttt{GiveMeSomeCredit} dataset through the \texttt{whynot} package. In this setting, the strength of the performative effect is controlled by a parameter $\delta$: when a data point would receive a negative classification under the previous model, it may strategically modify its manipulable features by copying features from another data point, with probability depending on $\delta$. We build on the publicly available code of~\citet{pmlr-v206-mofakhami23a}, use the largest neural network reported there, and keep the preprocessing, learning rate, and other hyperparameters unchanged. For each value of $\delta$, we run the dynamics until convergence for several values of the $\ell_2$ regularization parameter $\lambda$, sharing randomness across values of $\delta$ and $\lambda$. Each setting is run several times, and we report the final test accuracy with standard deviations. As shown in \Cref{fig:nn-reg}, the same qualitative behavior appears: $\ell_2$ regularization mitigates the sharp drop in accuracy induced by the performative shift, and the optimal amount of regularization increases with the strength of the performative effect.

\begin{figure}[thb]
    \centering
    \includegraphics[width=\columnwidth]{fig/metric_vs_lambda_final_acc.pdf}
    \caption{Final test accuracy of a neural network under performative shifts of increasing strength $\delta$, as a function of the $\ell_2$ regularization parameter $\lambda$. Regularization mitigates the loss in accuracy caused by the performative shift, and the best regularization level increases with $\delta$.}
    \label{fig:nn-reg}
\end{figure}

\vspace{-.5em}
\section{Conclusions}
\vspace{-.5em}

\looseness-1In this work, we demonstrate that regularization and performative effect are strongly related, as one can partially cancel out the other. In the population regime, the excess risk is worsened by performative effects. However, optimal ridge regularization mitigates this issue, especially when the data is isotropic and the entries of the vector modeling performativity have little variability. %
In the proportional regime, we provide a deterministic equivalent of the performative fixed point for random data. This in turn unveils a remarkable phenomenology: in contrast with the population setting, the optimal risk improves when performativity reinforces existing trends; furthermore, the optimal regularization follows the direction of the performative effect on the predictive features when the noise is small, while it goes in the opposite direction when the noise is large. Although the theoretical results focus on random data and a linear target model, our experiments on real-world data follow the theoretical predictions, suggesting their generality. Overall, these findings indicate that regularization could help in a wider range of scenarios, which we leave for future work. Beyond studying more complex data or models, interesting directions include the impact of other forms of regularization, such as early stopping or pruning, to mitigate performative effects.
