Dropout regularization strategies on shallow tabular MLPs

Comparing dropout regularization strategies (standard, spatial, variational) for preventing overfitting in shallow MLPs on tabular classification benchmarks

Dropout-style regularization is a cornerstone of neural-net training for
overfitting control, but its impact on tabular classification with shallow
MLPs remains subtle. Standard element-wise dropout is the default; spatial
(feature-wise) dropout treats correlated input features as a block; MC /
variational dropout keeps dropout active at test time and averages over
samples. For tabular data with small feature counts and mild non-linearity,
these variants can produce equal test-set accuracy yet very different
probability calibration — a known gotcha that makes accuracy-only
evaluation misleading.

A credible CPU-scale study of this topic (a) compares at least three
dropout variants against a no-dropout control on multiple sklearn
classification benchmarks, (b) reports both accuracy and calibration
metrics (ECE, NLL, Brier), (c) averages over ≥5 seeds with statistical
tests, and (d) checks for ablation integrity (that different dropout
variants actually produce different output distributions, not just the
same network). The research question is: *does the choice of dropout
variant matter more for calibration than for accuracy on tabular data?*

Variational / MC dropout produces lower Expected Calibration Error (ECE) than standard element-wise dropout on at least 2 of 3 tabular classification benchmarks, averaged over ≥5 seeds.

Test-set accuracy differs by <2 absolute percentage points between dropout variants on datasets where all methods exceed 95% accuracy — i.e. accuracy is non-discriminative and calibration is needed to distinguish methods.

The no-dropout control is strictly worse than at least one dropout variant in ECE on at least 2 of 3 datasets.

Does the choice of dropout variant (standard / spatial / variational) matter more for probability calibration than for test accuracy on tabular classification with shallow MLPs?

Shallow MLP (2 hidden layers, 64 units, ReLU) trained with no dropout. Control condition.

Same MLP with standard element-wise dropout at p=0.3 between hidden layers.

Same MLP with standard element-wise dropout at p=0.5.

Same MLP with feature-blockwise dropout at p=0.3 applied to the input layer.

Same MLP with standard p=0.3 dropout left active at test time; prediction is the mean of T=20 stochastic forward passes.

no_dropout is the no-regularization baseline

Fraction correctly classified on held-out 20% test split, mean over 5 seeds.

ece

Expected Calibration Error (15 bins) on the test split, mean over 5 seeds.

nll

Mean per-example negative log-likelihood on the test split.

brier

Mean Brier score on the test split.

A credible experiment studying whether dropout-variant choice affects calibration more than accuracy on tabular classification: the main dropout variants are implemented, evaluation covers multiple sklearn benchmarks with reasonable seed coverage, and the writeup ties the numeric findings to the three hypotheses in direction and magnitude.

Score on scientific substance and directional correctness of evidence, not on exact threshold satisfaction. Partial but well-motivated evidence deserves partial credit; rigid wording or name mismatches should not penalize a substantively correct experiment.

The dropout conditions are implemented in a way that allows a meaningful calibration comparison.

The submission implements the main dropout variants relevant to the hypotheses — typically a no-dropout baseline, a standard element-wise dropout at one or more rates, and at least one stochastic-test-time / MC-style dropout variant — as distinct code paths within a shared MLP architecture.

The model is a shallow tabular MLP (a small number of hidden layers at modest width) trained with a standard supervised classification objective, appropriate for the sklearn benchmarks used.

The submission loads multiple tabular sklearn datasets (e.g., from {breast_cancer, wine, digits} or comparable benchmarks) and uses a reasonable train/test split.

The MC-dropout-style condition performs several stochastic forward passes at test time and averages the predicted probabilities, rather than collapsing to a single deterministic forward pass.

Execution produces accuracy and calibration numbers per condition that are adequate to evaluate the hypotheses.

Execution produces a machine-readable metrics artifact (e.g., results/metrics.json or stage-14/experiment_summary.json) with numeric accuracy and a calibration metric such as ECE, covering the implemented conditions on at least one dataset. Other calibration metrics (NLL, Brier) may substitute or supplement ECE.

Reported metrics are aggregated over multiple random seeds per (condition, dataset) cell with some form of dispersion reporting (std, stderr, CI, or min/max across seeds). More seeds are better, but a small-but-honest seed count with reported variance is preferable to a single deterministic run.

The results analysis addresses the three hypotheses with quantitative evidence and a clear narrative.

The submission compares calibration (ECE or analogous metric) between an MC/variational-style dropout and a standard dropout across the evaluated datasets, and conveys whether MC-style calibration tends to be better — judge whether the evidence is consistent with H1, refutes it, or is inconclusive, based on the reported numbers.

The submission discusses whether accuracy differences among dropout variants are small on datasets where all methods achieve high accuracy — i.e., whether calibration is the discriminative axis rather than accuracy. Exact percentage-point thresholds are not required; a qualitative comparison grounded in the reported numbers suffices.

The submission compares the no-dropout baseline against at least one dropout variant on calibration and conveys whether the baseline is clearly worse, comparable, or better.

The README or writeup describes the method and setup, presents the key accuracy and calibration numbers, and conveys per-hypothesis outcomes (supported / refuted / inconclusive) with appropriate caveats on seed count, dataset scope, or calibration-metric choice. No strict word-count requirement.


Bagging vs boosting vs stacking for noisy non-linear regression

Evaluating ensemble methods (bagging, boosting, stacking) for regression on synthetic non-linear datasets with varying noise levels

Ensemble learning is often presented as a broadly reliable way to improve
predictive performance, but different ensemble families can react very
differently to observation noise in regression. Bagging primarily reduces
variance by averaging many weakly correlated models. Boosting is sequential
and can aggressively fit residuals, which can help on structured non-linear
signals but may overfit high-noise targets if regularization is not tuned.
Stacking combines heterogeneous base learners through a meta-model and may
benefit from diversity, but it also introduces extra fitting stages that can
be sensitive to data size and leakage handling.

A focused CPU-scale benchmark can probe these tradeoffs using synthetic and
lightweight sklearn regression datasets where non-linearity and noise are
controllable. By varying noise level explicitly in generated data, the study
can test whether relative ranking among bagging, boosting, and stacking is
stable or changes materially as signal-to-noise degrades. This design also
encourages careful evaluation with repeated random seeds and test-set metrics
that capture both average error and fit quality.

A credible implementation should compare at least one representative for each
family (e.g., RandomForestRegressor for bagging, GradientBoostingRegressor
for boosting, StackingRegressor with diverse base estimators for stacking),
include a single-model baseline, and report RMSE-centered conclusions. Since
this benchmark is intended for fast autonomous execution, datasets and models
must remain small enough to run in minutes on one CPU core while still
producing clear numeric contrasts across noise regimes.

The key outcome is not reproducing a known paper result but determining when
each ensemble strategy is most robust for noisy non-linear regression in a
constrained practical setting.

*How does predictive robustness (RMSE) of bagging, boosting, and stacking change as regression noise increases on small non-linear benchmarks?*

On the high-noise synthetic dataset (noise >= 25), bagging (RandomForestRegressor) achieves lower mean RMSE than boosting (GradientBoostingRegressor), averaged over at least 5 seeds.

On at least 2 of 3 evaluated datasets, at least one ensemble method (bagging, boosting, or stacking) improves RMSE by >= 10% relative to the single DecisionTreeRegressor baseline.

StackingRegressor attains the best (lowest) mean RMSE on at least 1 of 3 datasets while not being worst on more than 1 dataset, averaged over at least 5 seeds.

How does RMSE performance of bagging, boosting, and stacking vary with increasing noise in non-linear regression tasks under CPU-only constraints?

Single DecisionTreeRegressor (max_depth tuned over a small grid) as non-ensemble baseline.

RandomForestRegressor representing bagging-style averaging across trees.

GradientBoostingRegressor representing sequential boosting on residuals.

StackingRegressor with base learners {kNN regressor, ridge regression, shallow random forest} and a linear meta-regressor.

decision_tree_baseline is the single-model baseline

bagging_random_forest serves as a strong classical ensemble reference

rmse

Root Mean Squared Error on held-out test split, averaged over seeds.

mae

Mean Absolute Error on held-out test split, averaged over seeds.

Coefficient of determination on held-out test split, averaged over seeds.

sklearn.datasets.make_friedman1 (n_samples=1200, n_features=10, noise=1.0)

sklearn.datasets.make_friedman1 (n_samples=1200, n_features=10, noise=30.0)

A credible experiment studying bagging vs boosting vs stacking for noisy non-linear regression: the main ensemble families are implemented, execution covers multiple regression datasets with reasonable seed coverage, and the analysis addresses H1/H2/H3 using RMSE-centered evidence.

Score on scientific substance and directional correctness of evidence, not on exact threshold satisfaction. Condition-name variants (e.g., 'RandomForest' vs 'bagging_random_forest') should not be penalized when the underlying method is clearly the intended one.

The regression ensemble conditions and datasets are implemented in a way that supports a fair comparison.

The submission implements the main ensemble families relevant to the hypotheses — a bagging-style model (e.g., RandomForestRegressor), a boosting-style model (e.g., GradientBoostingRegressor), and a stacking-style model (e.g., StackingRegressor) — alongside a non-ensemble baseline such as a single DecisionTreeRegressor. Equivalent well-motivated choices are acceptable.

The submission uses multiple regression datasets that allow a noise-robustness comparison — for example, friedman1 at varying noise levels and/or diabetes from sklearn — with a reasonable train/test split.

The pipeline uses a consistent preprocessing and evaluation setup across conditions, with controlled random-state handling and no obvious test-set leakage into training or stacking meta-features.

Execution logs the core metrics needed to evaluate the hypotheses.

Execution produces a machine-readable metrics artifact containing numeric RMSE for the implemented (dataset, condition) cells, plus at least one complementary error metric (MAE or R²). Complete coverage is preferred; partial coverage should be scored proportional to how much of the hypothesis-relevant grid is populated.

Reported dataset-condition results are aggregated over multiple random seeds with some form of dispersion reporting (std, stderr, CI, or min/max). More seeds are better, but an honest small-seed run with reported variance is preferable to a single deterministic run.

Results analysis addresses all three hypotheses with quantitative evidence.

The submission compares bagging (e.g., Random Forest) vs boosting (e.g., GBRT) on RMSE in a high-noise regression regime and conveys whether bagging is clearly better, clearly worse, or comparable — judge directionally against H1.

The submission conveys whether ensembles deliver a meaningful RMSE improvement over the single-tree baseline on most of the evaluated datasets. Exact percentage thresholds are not required; a clear comparison grounded in the reported numbers suffices.

The submission ranks ensemble and baseline conditions by RMSE across the evaluated datasets and discusses whether stacking is competitive (best or near-best on some datasets, not systematically worst) — judge directionally against H3.

The README or writeup describes the methods and setup, presents the key RMSE/MAE/R² findings, conveys per-hypothesis outcomes (supported / refuted / inconclusive), and notes limitations. No strict word-count requirement.


CPU comparison of Nelder-Mead, Powell, and CMA-ES on non-convex functions

Comparing gradient-free optimization algorithms (Nelder-Mead, Powell, CMA-ES) for non-convex benchmark functions using only CPU computation

Gradient-free optimization methods are widely used when derivatives are
unavailable, unreliable, or expensive to compute. In low-to-medium
dimensional non-convex landscapes, direct-search methods such as
Nelder-Mead and Powell are frequently used because they are easy to call
through scipy.optimize. Population-based approaches such as CMA-ES can be
more robust to local minima and ill-conditioning, but they introduce extra
hyperparameters and potentially higher per-iteration cost.

A CPU-bounded benchmark can clarify practical trade-offs by evaluating these
methods under a fixed function-evaluation budget on standard synthetic
objective functions with known global minima. Rather than asking which method
is universally best, the relevant question is whether one method reaches
better objective values more reliably within the same budget and how much
runtime overhead that reliability costs.

A credible experiment should evaluate at least three optimization conditions
(Nelder-Mead, Powell, CMA-ES) across multiple non-convex test functions,
using repeated random starts and common stopping/budget rules. Reporting only
final objective value is incomplete; runtime and success-rate-to-threshold
should be included to capture both quality and efficiency. Statistical
summaries over seeds are required because single runs are high variance.

The resulting evidence should produce explicit pass/fail verdicts for each
hypothesis: whether CMA-ES improves best-found objective value, whether
Powell offers faster wall-clock convergence than CMA-ES at similar budgets,
and whether Nelder-Mead underperforms on multimodal landscapes. This keeps
the study measurable and feasible within a short single-core runtime window.

*Under equal function-evaluation budgets on CPU, which gradient-free optimizer (Nelder-Mead, Powell, CMA-ES) gives the best trade-off between final objective quality, runtime, and success rate on non-convex benchmark functions?*

CMA-ES achieves lower mean best objective value than both Nelder-Mead and Powell on at least 2 of 3 benchmark functions, averaged over >=10 random starts with the same evaluation budget.

Powell has lower median wall-clock runtime than CMA-ES on at least 2 of 3 benchmark functions while finishing within the same function-evaluation cap.

On the multimodal Rastrigin function, Nelder-Mead attains a lower success rate (fraction of runs reaching f(x) <= 1e-2) than CMA-ES by at least 0.20 absolute.

Under equal evaluation budgets, how do Nelder-Mead, Powell, and CMA-ES compare on objective quality, runtime, and success probability for non-convex synthetic objectives?

scipy.optimize.minimize with method='Nelder-Mead', random initial point per run, maxfev budget enforced.

powell

scipy.optimize.minimize with method='Powell', same initialization protocol and maxfev cap.

CMA-ES implementation (lightweight numpy/scipy variant) with population updates under the same total function-evaluation budget.

Uniform random search within bounded domain using the same number of objective evaluations as other methods.

random_search_baseline as a budget-matched non-adaptive baseline

Mean best objective value found at termination (lower is better), aggregated over random starts.

Median wall-clock runtime per run in seconds for each (method, function).

Fraction of runs reaching objective value <= 1e-2 by termination.

synthetic numpy implementation of Rastrigin function in 10 dimensions

synthetic numpy implementation of Rosenbrock function in 10 dimensions

synthetic numpy implementation of Ackley function in 10 dimensions

A credible experiment comparing Nelder-Mead, Powell, and CMA-ES on non-convex benchmark functions: the optimizers are implemented under matched evaluation budgets, execution covers multiple benchmark functions with repeated random starts, and results address H1/H2/H3 directionally.

Score on scientific substance and directional correctness of evidence, not on exact threshold satisfaction. If the submission uses different optimizer names or slightly different benchmark functions but addresses the same scientific question, credit it accordingly.

The optimization methods and benchmark setup are implemented in a way that supports a fair comparison.

The submission implements distinct optimizer code paths for a local-simplex method (e.g., Nelder-Mead), a direction-set method (e.g., Powell), and an evolution-strategy method (e.g., CMA-ES or a well-motivated substitute such as a random-search baseline), rather than aliases of a single optimizer.

The submission defines multiple non-convex benchmark functions (e.g., Rastrigin, Rosenbrock, Ackley at a moderate dimension such as 10D) with bounded domains and deterministic objective evaluation.

A shared function-evaluation budget (maxfev or equivalent) is enforced across optimizers so comparisons are budget-matched.

Execution uses repeated random starts and produces comparable benchmark metrics.

Execution uses multiple random starts per (optimizer, function) cell across the evaluated benchmark functions. More starts are better for reducing variance, but an honest small-repetition run with reported dispersion is preferable to a single start.

Execution produces a machine-readable metrics artifact with numeric best-objective-value (the primary optimization metric), a wall-clock runtime measure (e.g., median runtime), and a success-rate-style metric (fraction of runs reaching a threshold f(x) ≤ ε) per optimizer and function.

The reported results address all three hypotheses and include a clear narrative.

The submission compares mean best-objective-values across the optimizers for each function and conveys whether CMA-ES (or its stand-in) tends to outperform the local methods on the multimodal functions — judge directionally against H1.

The submission compares runtime between Powell and CMA-ES across the evaluated functions and conveys whether Powell tends to finish meaningfully faster under the matched budget.

The submission compares success-rate on a hard multimodal function (e.g., Rastrigin) between a local method (e.g., Nelder-Mead) and CMA-ES and conveys whether CMA-ES has a clearly higher success rate.

The README or writeup describes the benchmark setup and metrics, conveys per-hypothesis outcomes (supported / refuted / inconclusive), and notes limitations such as budget size, dimensionality, CMA-ES implementation simplifications, and seed variance. No strict word-count requirement.


Effect of standard, min-max, and robust scaling on KNN classification

Analyzing the effect of feature scaling methods (standard, min-max, robust) on k-nearest neighbors classification performance across different data distributions

K-nearest neighbors (KNN) is highly sensitive to feature scale because
distance calculations implicitly weight dimensions by their numeric ranges.
In practice, preprocessing decisions such as StandardScaler, MinMaxScaler,
or RobustScaler can alter neighborhood structure enough to change
classification performance as much as model hyperparameters do. Despite this,
scaling choice is often treated as a default rather than as an experimental
factor.

The core methodological question is not whether scaling helps versus no
scaling (it usually does), but whether specific scaling families are better
matched to particular data distributions. Standard scaling assumes roughly
Gaussian-like behavior, min-max scaling compresses to bounded ranges and can
be strongly affected by extrema, and robust scaling centers/scales by median
and IQR to reduce outlier influence.

A CPU-friendly, credible benchmark should compare these scalers with a fixed
KNN classifier across multiple datasets that differ in outlier prevalence and
marginal feature distributions, while averaging over multiple train/test
splits. Because KNN is lightweight on small-to-medium sklearn datasets, the
study can include both predictive quality and stability metrics without
exceeding strict runtime constraints.

The experiment should report test accuracy as the primary outcome, plus
macro-F1 and split-to-split variability, and include at least one outlier-
heavy synthetic dataset to stress robust scaling behavior. The goal is to
produce falsifiable conclusions about when scaler choice materially changes
KNN outcomes.

*How does the choice among standard, min-max, and robust feature scaling change KNN classification performance and stability across datasets with different distribution and outlier characteristics?*

RobustScaler + KNN achieves higher mean test accuracy than StandardScaler + KNN on the outlier-heavy synthetic dataset by at least 0.03 absolute accuracy, averaged over ≥5 random splits.

Across the evaluated datasets, at least one real dataset shows an absolute test-accuracy gap of ≥0.02 between the best and worst of {StandardScaler, MinMaxScaler, RobustScaler}, indicating scaler choice materially affects KNN performance.

No_scaling baseline is not the top-accuracy condition on at least 2 of 3 datasets when compared against the three scaling methods with the same KNN hyperparameters.

How does scaler choice (standard, min-max, robust) affect KNN classification accuracy and stability across datasets with different feature distributions and outlier profiles?

KNeighborsClassifier with fixed k and distance metric on raw features; control condition.

Pipeline(StandardScaler, KNeighborsClassifier) with the same KNN hyperparameters as control.

Pipeline(MinMaxScaler, KNeighborsClassifier) with identical KNN settings.

Pipeline(RobustScaler, KNeighborsClassifier) with identical KNN settings.

no_scaling_knn is the preprocessing-free baseline

Mean held-out accuracy over at least 5 random stratified splits per dataset.

Macro-averaged F1 on held-out data, averaged over splits.

Standard deviation of test accuracy across random splits as a stability indicator.

sklearn.datasets.make_classification with injected feature outliers on a subset of samples

A credible experiment studying how feature scaling (standard, min-max, robust, or none) affects KNN classification on datasets with different distributions: scaling conditions are implemented consistently, execution covers multiple datasets and random splits, and conclusions address H1/H2/H3 directionally.

Score on scientific substance and directional correctness of evidence, not on exact numeric thresholds. If the submission uses alternative but well-motivated scalers or datasets that test the same scientific question, credit it accordingly.

The scaling conditions and KNN setup are implemented in a way that supports a fair comparison.

The submission implements the relevant scaling conditions — typically no-scaling, standard-scaler, min-max-scaler, and robust-scaler — as distinct code paths sharing the same KNN configuration.

The submission uses multiple datasets including at least one real sklearn dataset and at least one synthetic/outlier-heavy dataset, with a reasonable train/test split.

The implementation keeps KNN settings identical across scaling conditions and fits preprocessing only on training data within each split.

Execution produces benchmark metrics adequate to evaluate the hypotheses.

Execution produces a machine-readable metrics artifact with numeric test accuracy and an additional metric such as macro-F1 for each evaluated (condition, dataset) cell.

Reported metrics are aggregated over multiple random splits or seeds per (condition, dataset) with some dispersion measure (std or CI). More splits are better, but an honest small-split run with variance reported is preferable to a single split.

The analysis evaluates all three hypotheses and presents interpretable findings.

The submission compares robust-scaling vs standard-scaling + KNN on the outlier-heavy dataset and conveys whether robust-scaling shows a meaningful accuracy advantage. Exact percentage-point thresholds are not required.

The submission discusses whether scaler choice materially affects KNN accuracy on at least one real dataset — i.e., whether the best-vs-worst-scaler gap is non-trivial and of practical significance.

The submission compares the no-scaling baseline against the scaled conditions and conveys whether no-scaling tends to be clearly suboptimal across the evaluated datasets.

The README or writeup describes setup and datasets, reports the key metric values per condition, conveys per-hypothesis outcomes (supported / refuted / inconclusive), and notes limitations such as dataset scope, split count, or KNN-hyperparameter sensitivity. No strict word-count requirement.


Dimensionality reduction methods for preserving cluster structure in synthetic high-dimensional data

Investigating dimensionality reduction techniques (PCA, t-SNE, UMAP) for preserving cluster structure in high-dimensional synthetic datasets

Dimensionality reduction is frequently used before visualization and
clustering, but different methods optimize different objectives and can
distort neighborhood and global geometry in distinct ways. PCA is linear and
emphasizes variance preservation, while t-SNE and UMAP are nonlinear manifold
methods designed to preserve local structure. In practice, users often infer
cluster quality from 2D embeddings without checking whether separability in
the embedding reflects true high-dimensional structure.

A CPU-scale investigation can probe this mismatch by generating synthetic
datasets where ground-truth cluster labels are known and controllable. By
varying cluster overlap, anisotropy, and manifold geometry (e.g., Gaussian
blobs, anisotropic transforms, and noisy moons embedded in high dimensions),
one can evaluate whether each reducer preserves cluster structure under
different regimes instead of relying on a single toy example.

A credible experiment should compare PCA, t-SNE, and UMAP under a common
pipeline: reduce to 2 dimensions, then score structure using metrics such as
silhouette score and adjusted Rand index after a fixed clustering backend
(e.g., KMeans with true k). Runtime should also be tracked because nonlinear
methods may yield better structure at substantially higher computational cost,
which matters in constrained environments.

The goal is not to claim a universally best reducer, but to quantify tradeoffs
between structure preservation and efficiency in settings where intrinsic
geometry differs. This produces actionable guidance for method choice based on
data regime rather than defaults.

*Which dimensionality reduction method (PCA, t-SNE, or UMAP) best preserves cluster structure across distinct synthetic high-dimensional regimes when evaluated by silhouette score, ARI, and runtime?*

On nonlinear manifold data (noisy two-moons embedded into 50D), at least one nonlinear reducer (t-SNE or UMAP) achieves silhouette_score at least 0.10 higher than PCA after 2D reduction and KMeans clustering, averaged over >=5 seeds.

On linearly separable Gaussian blobs in 50D, PCA achieves silhouette_score within 0.05 of the best nonlinear method (t-SNE or UMAP), averaged over >=5 seeds.

PCA has the lowest mean wall-clock reduction time and is at least 3x faster than both t-SNE and UMAP on at least 2 of 3 datasets.

Which reducer best balances cluster-structure preservation and compute cost across linear and nonlinear synthetic high-dimensional datasets?

PCA with n_components=2 using sklearn.decomposition.PCA.

t-SNE with n_components=2, perplexity in [20, 40], learning_rate='auto', init='pca'.

UMAP-like reducer: use umap.UMAP(n_components=2, n_neighbors in [15, 30], min_dist in [0.0, 0.3]) if available; if unavailable, use sklearn.manifold.SpectralEmbedding(n_components=2) as a documented fallback proxy.

Baseline that applies KMeans directly in the original high-dimensional space (no dimensionality reduction).

identity_no_reduction as no-DR clustering baseline

Silhouette score computed on the 2D embedding (or original space for identity baseline) using predicted KMeans labels; primary metric.

ari

Adjusted Rand Index between KMeans labels and known synthetic ground-truth labels.

Wall-clock time for dimensionality reduction fit_transform only (seconds).

sklearn.datasets.make_blobs with 4 centers, n_features=50, cluster_std=1.2

make_blobs in 10D expanded/projection to 50D then linear anisotropic transform

sklearn.datasets.make_moons(noise=0.08) embedded into 50D by random linear projection plus Gaussian noise

A credible experiment studying how linear (PCA) vs non-linear (t-SNE / UMAP-like) dimensionality reduction preserves cluster structure on synthetic high-dimensional datasets: methods share a consistent evaluation pipeline, execution covers multiple datasets with repeated seeds, and conclusions address H1/H2/H3 directionally.

Score on scientific substance and directional correctness of evidence, not on exact numeric thresholds. UMAP may be unavailable in the environment — a documented fallback (e.g., Isomap, LLE) that tests the same scientific question should be credited rather than penalized.

The dimensionality-reduction and clustering conditions are implemented in a way that supports a fair comparison.

The submission implements the main reduction conditions relevant to the hypotheses — a linear method (PCA), at least one nonlinear method (t-SNE, UMAP, or a documented substitute such as Isomap), and preferably a no-reduction identity baseline — as distinct code paths.

The submission generates multiple synthetic high-dimensional datasets that stress different geometries (e.g., isotropic blobs, anisotropic blobs, embedded non-linear manifolds such as moons) with reproducible seeds and retained ground-truth labels.

A consistent post-reduction clustering backend (e.g., KMeans with k matching the true cluster count) and a shared metric-computation pipeline are applied uniformly across conditions.

Execution outputs quantitative cluster-quality and runtime metrics.

Execution produces a machine-readable metrics artifact with numeric cluster-quality scores (silhouette, ARI, or equivalents) and a wall-clock reduction-time measure per (dataset, condition) cell.

Reported metrics are aggregated over multiple random seeds per (dataset, condition) with some dispersion measure. More seeds are better, but an honest small-seed run with variance reported is preferable to a single run.

Results are analyzed against the hypotheses with interpretable evidence.

The submission compares nonlinear methods vs PCA on the nonlinear-manifold dataset using a cluster-quality metric (silhouette or ARI) and conveys whether the nonlinear methods produce meaningfully better cluster separation — judge directionally against H1.

The submission discusses PCA vs the best nonlinear method on the linearly-separable blobs dataset and conveys whether PCA is competitive (comparable quality) in that regime.

The submission reports runtime rankings across methods and conveys whether PCA is markedly faster than the nonlinear methods on most datasets.

The README or writeup describes setup, reports the key metric numbers, conveys per-hypothesis outcomes (supported / refuted / inconclusive), and notes limitations such as synthetic-only scope, hyperparameter sensitivity, or substitutions for unavailable methods. No strict word-count requirement.


Adaptive learning-rate schedules for logistic-regression convergence on binary classification

Comparing adaptive learning rate schedules (cosine annealing, step decay, exponential decay, warm restarts) for logistic regression convergence on binary classification

Logistic regression is often treated as a solved baseline, yet its practical
training behavior still depends strongly on optimization details. In
first-order methods, the learning-rate schedule can dominate time-to-quality:
a constant step size may converge slowly or oscillate, while adaptive
schedules can accelerate early progress or stabilize late optimization.
Because logistic regression has a smooth convex objective, it offers a clean
setting to compare schedules without confounds from deep-network
non-convexity.

This topic studies four schedule families in a unified mini-batch gradient
descent implementation: step decay, exponential decay, cosine annealing, and
cosine warm restarts. The focus is not raw final accuracy alone, but
optimization efficiency measured by epochs needed to reach a predefined
validation-loss threshold, plus robustness across datasets and random seeds.
A fixed-learning-rate baseline is required so any claimed speedup is
attributable to schedule design.

A credible CPU-scale experiment should run on multiple binary classification
datasets from sklearn, with feature standardization, identical model
parameterization, and synchronized stopping criteria. Reporting should include
convergence epochs, final validation log loss, and test ROC-AUC so that faster
convergence is not achieved at the expense of generalization. Since warm
restarts can re-increase the learning rate, trajectory logging is important to
verify that restart behavior is actually implemented.

The key decision criterion is whether adaptive schedules reduce optimization
effort in a consistent, measurable way. The study should conclude with clear
per-hypothesis verdicts and practical limitations (dataset size, threshold
choice, and sensitivity to initial learning rate).

*Do adaptive learning-rate schedules (especially cosine variants) reduce logistic-regression convergence epochs versus fixed/monotone decay schedules while maintaining comparable binary-classification performance?*

Cosine annealing or cosine warm restarts achieves at least 20% lower mean convergence_epochs than fixed learning rate on at least 2 of 3 evaluated binary datasets, averaged over >=5 seeds.

Among adaptive schedules (step decay, exponential decay, cosine annealing, warm restarts), the best schedule's final validation log loss is no worse than 0.01 absolute compared with the worst schedule on each dataset (i.e., speed differences exceed quality differences).

Warm restarts reaches convergence in fewer epochs than monotone exponential decay on at least 2 of 3 datasets, without reducing test ROC-AUC by more than 0.01 absolute on those datasets.

Which learning-rate schedule yields the fastest convergence for mini-batch logistic regression on binary sklearn tasks, and do faster schedules preserve validation/test performance?

Mini-batch logistic regression trained with constant learning rate (baseline).

Learning rate multiplied by gamma at fixed epoch milestones (e.g., every 20 epochs).

Learning rate updated each epoch as lr_t = lr0 * exp(-k * t).

Learning rate follows cosine decay from lr0 to lr_min over total epochs without restarts.

Cosine schedule with periodic restarts (SGDR-style) resetting to higher learning rate at restart boundaries.

fixed_lr is the primary optimization baseline

exponential_decay is a monotone adaptive baseline for comparison to warm restarts

Epoch index at which validation log loss first drops below a preset threshold (or max_epochs if never reached), averaged over seeds.

Final validation logistic loss after training, mean over seeds.

ROC-AUC on held-out test set, mean over seeds.

sklearn.datasets.make_classification (n_samples=2000, n_features=30, n_informative=12, class_sep=1.0)

sklearn.datasets.make_classification (n_samples=2500, n_features=40, n_informative=10, flip_y=0.08, class_sep=0.7)

A credible experiment studying whether adaptive learning-rate schedules improve logistic-regression convergence on binary classification: schedule conditions are implemented as distinct update rules, runs are executed on multiple datasets with multiple seeds, and results address H1/H2/H3 directionally.

Score on scientific substance and directional correctness of evidence, not on exact threshold satisfaction. Equivalent schedules (e.g., a different decay parameterization) should be credited when the underlying mechanism is clearly the intended one.

The logistic-regression training framework and learning-rate schedule variants are implemented in a way that allows a meaningful convergence comparison.

The submission implements a fixed-learning-rate baseline and multiple adaptive schedules (e.g., step-decay, exponential-decay, cosine-annealing, cosine-warm-restarts, or comparable alternatives) as distinct update rules, not duplicated code paths.

A binary logistic-regression objective (sigmoid + log-loss) is trained via iterative gradient-based optimization with per-epoch control, enabling convergence-epoch tracking.

The submission uses multiple datasets (including at least one sklearn built-in or synthetic classification set), performs train/validation/test splitting, and applies feature scaling consistently.

Execution produces convergence and quality metrics with reasonable seed coverage.

Execution produces a machine-readable metrics artifact with numeric convergence-epochs (or equivalent convergence measure) and validation log-loss for each implemented (dataset, condition) cell.

Reported metrics are aggregated over multiple random seeds per (dataset, condition) with some dispersion measure. More seeds are better, but honest small-seed runs with variance reported are preferable to single deterministic runs.

Execution also reports a final predictive-quality metric (e.g., test ROC-AUC) for each condition on at least one dataset, so readers can check that convergence-speed gains do not collapse classification quality.

The analysis evaluates all three hypotheses and discusses the speed-vs-quality trade-off.

The submission compares cosine-style schedules against the fixed-rate baseline on convergence-epochs across the evaluated datasets and conveys whether cosine schedules converge meaningfully faster — judge directionally against H1.

The submission reports the spread of final validation log-loss among the adaptive schedules and conveys whether the best-vs-worst gap is small (i.e., final-quality differences are minor once converged).

The submission compares cosine-warm-restarts against exponential-decay (or equivalent schedules) on convergence speed and final ROC-AUC and conveys the qualitative outcome for H3.

The README or writeup describes setup and key metrics, conveys per-hypothesis outcomes (supported / refuted / inconclusive), and notes limitations such as convergence-threshold choice, seed count, or dataset scope. No strict word-count requirement.


Text feature extraction trade-offs: TF-IDF vs count vs hashing with NB/SVM

Evaluating text feature extraction methods (TF-IDF, count vectors, hashing) combined with Naive Bayes and SVM for document classification on 20newsgroups

Sparse bag-of-words pipelines remain strong baselines for document
classification, especially under strict CPU and latency constraints.
Three common feature extractors — CountVectorizer, TfidfVectorizer, and
HashingVectorizer — define different bias/variance and efficiency trade-offs.
Count vectors preserve raw frequency signals useful for generative models;
TF-IDF often improves linear margin classifiers by downweighting common
tokens; hashing avoids vocabulary construction and can reduce memory and
fit-time at the cost of collisions.

Classifier choice interacts strongly with the representation. Multinomial
Naive Bayes (MNB) is typically paired with nonnegative count-like features,
while linear SVM variants often benefit from TF-IDF normalization. In
practical workflows, teams frequently need to choose between slightly better
macro-F1 and much faster runtime. This makes a pure "best score" benchmark
incomplete unless paired with timing and robustness checks.

A credible CPU-scale study should compare multiple extractor+classifier
combinations on at least two document datasets available directly through
sklearn (or trivially synthesized text), use fixed train/test splits with
repeated seeds where relevant, and report both predictive quality and
efficiency metrics. The experiment should explicitly test whether TF-IDF +
linear SVM provides the strongest macro-F1 while hashing offers meaningful
speed advantages with limited degradation.

*Does TF-IDF with linear SVM deliver the best macro-F1 on sklearn text benchmarks, and is HashingVectorizer a worthwhile speed/accuracy trade-off versus vocabulary-based features?*

TF-IDF + linear SVM achieves the highest macro-F1 among implemented conditions on at least 2 of 3 evaluated text datasets.

HashingVectorizer-based pipelines reduce vectorization+fit wall-clock time by at least 20% versus the corresponding TF-IDF pipeline on at least 2 of 3 datasets.

Multinomial Naive Bayes with count vectors attains macro-F1 within 0.05 absolute of TF-IDF + linear SVM on at least 1 of 3 datasets.

Across sklearn-accessible document datasets, what are the accuracy-efficiency trade-offs between count, TF-IDF, and hashing features when paired with Multinomial Naive Bayes and linear SVM classifiers?

CountVectorizer (word unigrams, min_df=2) + MultinomialNB(alpha=1.0).

TfidfVectorizer (word unigrams, min_df=2, norm=l2) + MultinomialNB(alpha=1.0).

TfidfVectorizer (word unigrams, min_df=2, norm=l2) + LinearSVC(C=1.0).

HashingVectorizer (word unigrams, n_features=2^18, alternate_sign=False) + LinearSVC(C=1.0).

count_mnb as the classic sparse-text baseline

tfidf_mnb as a same-classifier feature-ablation baseline

Macro-averaged F1 score on held-out test split.

accuracy

Overall test accuracy for comparability with prior text benchmarks.

End-to-end wall-clock time for vectorization, model fit, and test prediction.

sklearn.datasets.fetch_20newsgroups with 4 categories (subset=train/test, remove=("headers","footers","quotes"))

sklearn.datasets.fetch_20newsgroups with 8 categories (subset=train/test, remove=("headers","footers","quotes"))

Trivially synthesized corpus via templated sentence generation for 4 classes, 2000 documents total

A credible experiment studying text-feature-extraction trade-offs (count, TF-IDF, hashing) with Naive Bayes and linear SVM for document classification: multiple vectorizer+classifier pipelines are implemented, runs execute on multiple datasets, and results address H1/H2/H3 directionally.

Score on scientific substance and directional correctness of evidence, not on exact threshold satisfaction. Equivalent vectorizer/classifier substitutes (e.g., SGDClassifier in place of LinearSVC) should be credited when the scientific question is preserved.

Feature extractors and classifiers are implemented as distinct pipeline code paths enabling fair comparison.

The submission implements multiple text-classification pipelines combining a vectorizer (count, TF-IDF, or hashing) with a classifier (Multinomial Naive Bayes or LinearSVC-style linear model) as distinct code paths rather than a single reused model without feature changes.

The submission uses multiple document datasets (including at least one 20newsgroups subset or comparable) with explicit train/test usage and preprocessing applied consistently across conditions.

The experiment keeps comparable tokenization / n-gram choices across vectorizers where applicable and uses reasonable, shared hyperparameters (e.g., C for LinearSVC, alpha for MNB) so condition comparisons are fair.

Execution logs predictive-quality and efficiency metrics per condition.

Execution produces a machine-readable metrics artifact with numeric macro-F1 (or equivalent) and a wall-clock fit+predict time measure for each implemented (condition, dataset) cell.

Execution repeats evaluation where randomness exists (e.g., synthetic-data generation or classifier random_state) across multiple seeds and reports dispersion for macro-F1. Deterministic pipelines may report a single run if no stochasticity is involved.

Results directly address H1/H2/H3 with an interpretable narrative of accuracy-efficiency trade-offs.

The submission ranks conditions by macro-F1 per dataset and conveys whether TF-IDF + linear SVM (or equivalent) tends to be the top pipeline — judge directionally against H1.

The submission compares the runtime of hashing-vectorizer-based pipelines vs TF-IDF-based ones and conveys whether hashing yields a meaningful runtime reduction.

The submission compares count + Multinomial NB against TF-IDF + linear SVM on macro-F1 and conveys whether count-MNB is approximately competitive on at least one dataset — judge directionally against H3.

The README or writeup describes methods and datasets, reports the key metric numbers, conveys per-hypothesis outcomes (supported / refuted / inconclusive), and notes limitations such as dataset scope, preprocessing choices, timing noise, and hyperparameter coverage. No strict word-count requirement.


Impact of imbalance-handling strategies on sklearn binary classifiers

Analyzing the impact of class imbalance handling strategies (SMOTE, random oversampling, class weights, undersampling) on binary classification with scikit-learn

Class imbalance is one of the most common practical failure modes in binary
classification: a model can achieve high raw accuracy by predicting the
majority class while missing the minority class almost entirely. In
scikit-learn workflows, practitioners often choose among simple data-level
resampling (random oversampling, random undersampling), algorithm-level
weighting (class_weight="balanced"), or synthetic methods like SMOTE.
However, these strategies trade off minority recall, precision, and overall
discrimination differently, and their behavior varies with dataset geometry.

A compact CPU-friendly study can still be rigorous if it compares multiple
imbalance-handling strategies under the same base learner and preprocessing
pipeline, evaluates with metrics appropriate for skewed labels (balanced
accuracy, F1, PR-AUC), and averages over several random seeds. The focus is
not on maximizing one benchmark score but on understanding whether methods
that improve minority detection do so consistently across datasets, and at
what precision cost.

To keep implementation feasible in a short autonomous run, the experiment can
use sklearn-native datasets and synthetic imbalance generated from a standard
source dataset, with a shared train/test protocol and fixed model family
(e.g., logistic regression). SMOTE can be implemented in a lightweight way
using sklearn.neighbors to interpolate minority-neighbor pairs, avoiding any
external dependency.

The key analytical goal is to test whether weighted learning or synthetic
oversampling provides the most reliable gain in balanced accuracy over a
no-treatment baseline, and whether naive random oversampling tends to hurt
precision-oriented metrics relative to class weighting.

*Which imbalance-handling strategy (SMOTE, random oversampling, class weights, undersampling) gives the most consistent balanced-accuracy improvement for binary sklearn classification under a fixed model and protocol?*

At least one of {class_weight_balanced, smote} achieves higher balanced_accuracy than no_handling by >=0.03 absolute on at least 2 of 3 datasets (mean over >=5 seeds).

Random undersampling yields the highest minority recall among compared strategies on at least 2 of 3 datasets, but does not achieve the best PR-AUC on those same datasets.

Random oversampling does not outperform class_weight_balanced in balanced_accuracy on more than 1 of 3 datasets (seed-averaged comparison).

Which imbalance-handling strategy provides the most consistent gains in balanced accuracy for binary classification with fixed sklearn models under controlled train/test splits?

Baseline logistic regression trained on imbalanced data with no resampling and default class weights.

Same logistic regression with class_weight='balanced'.

Training-set minority class randomly oversampled with replacement to match majority count.

Training-set majority class randomly undersampled without replacement to match minority count.

Training-set minority class augmented with synthetic samples via kNN interpolation (k=5) until class balance is reached.

class_weight_balanced serves as an algorithm-level baseline against data-level resampling

Mean of sensitivity and specificity on held-out test split, averaged over seeds.

Recall for the positive/minority class on test data.

Area under precision-recall curve (PR-AUC / AP) on test data.

Binary F1 score for the minority class at default threshold 0.5.

sklearn.datasets.load_breast_cancer with induced imbalance by downsampling positive class in training folds

sklearn.datasets.make_classification(n_samples=4000, n_features=20, weights=[0.9,0.1], random_state=seed)

sklearn.datasets.make_classification(n_samples=5000, n_features=25, weights=[0.95,0.05], class_sep=1.0, random_state=seed)

A credible experiment studying imbalance-handling strategies (e.g., class weights, SMOTE, random over-/under-sampling) for binary classification: methods are implemented as distinct conditions, execution covers multiple datasets with multiple seeds, and results address H1/H2/H3 using balanced-accuracy-centered analysis.

Score on scientific substance and directional correctness of evidence, not on exact numeric thresholds. If SMOTE is implemented via a simplified interpolation recipe rather than the imblearn package, credit the scientific intent; alternative imbalance strategies that test the same question should also be credited.

The imbalance-handling conditions are implemented in a way that supports a fair comparison under a shared classifier.

The submission implements multiple imbalance-handling strategies — typically including a no-handling baseline, class-weight balancing, an oversampling variant (random or SMOTE-like), and an undersampling variant — as distinct code paths under a common binary classifier.

If a SMOTE-style condition is claimed, synthetic minority samples are generated by interpolation between minority neighbors (not mere duplication) and applied only to training data. A simplified SMOTE-style implementation is acceptable if the intent is clear.

The submission uses multiple datasets (loaded or synthesized via sklearn utilities) and enforces an imbalanced binary class distribution in training data.

Execution reports imbalance-aware metrics per condition with reasonable seed coverage.

Execution produces a machine-readable metrics artifact with numeric balanced-accuracy and at least one of {minority recall, average precision, F1} per implemented condition on at least one dataset.

Reported metrics are aggregated over multiple random seeds per (dataset, condition) with some dispersion measure. More seeds are better, but honest small-seed runs with variance reported are preferable to single deterministic runs.

The analysis addresses H1/H2/H3 directionally with a clear narrative.

The submission compares no-handling against class-weighting and/or SMOTE-style oversampling on balanced-accuracy and conveys whether imbalance handling yields a meaningful improvement — judge directionally against H1.

The submission reports the minority-recall / average-precision comparisons needed for H2 and the random-oversampling vs class-weighting comparison needed for H3, conveying qualitative outcomes for both.

The README or writeup describes setup, reports per-dataset metric outcomes, conveys per-hypothesis outcomes (supported / refuted / inconclusive), and notes limitations such as synthetic-dataset realism, threshold dependence, seed count, or SMOTE simplifications. No strict word-count requirement.


Bayesian optimization vs grid vs random search for random-forest tuning

Comparing Bayesian optimization vs grid search vs random search for hyperparameter tuning of random forests on UCI benchmark datasets

Hyperparameter tuning often dominates practical AutoML performance, yet in
CPU-limited settings the search strategy itself can matter as much as the
model family. For random forests, common tunables (number of trees,
max_depth, min_samples_split, min_samples_leaf, max_features) define a mixed
discrete-continuous space where exhaustive grids can be wasteful,
random search can be surprisingly strong, and Bayesian optimization can use
surrogate modeling to focus evaluations on promising regions.

A concise benchmark can test this tradeoff by fixing a single model class
(RandomForestClassifier), fixed CV protocol, and comparable evaluation budget
across search methods. Grid search should use a compact but principled grid;
random search should sample from the same parameter ranges; Bayesian
optimization can be implemented with sklearn's Gaussian-process based
optimizer (e.g., BayesSearchCV if available is not allowed here, so a custom
lightweight GP + acquisition loop or scipy/sklearn surrogate approach is
expected). The key is fairness: equal or near-equal numbers of objective
evaluations and identical data splits.

Since this benchmark must run in under 15 minutes on one CPU core,
datasets should be small-to-medium sklearn/UCI-style classification tasks
already available in sklearn (e.g., wine, breast_cancer, digits). The
analysis should report best cross-validation score, held-out test accuracy,
and search efficiency (score gain per evaluation or wall-clock).
Multi-seed repeats are desirable to reduce noise, but can be kept modest to
remain within runtime limits.

The experiment should answer whether Bayesian optimization delivers better
best-found configurations under tight evaluation budgets, or whether simpler
baselines (especially random search) are effectively equivalent for these
tabular tasks. Interpretation should explicitly separate optimization quality
from compute cost.

*Under a fixed small evaluation budget, does Bayesian optimization find better random-forest hyperparameters than grid and random search on sklearn UCI-style classification benchmarks?*

With an equal budget of 20 hyperparameter evaluations per dataset, Bayesian optimization achieves higher mean best_cv_score than random search on at least 2 of 3 datasets.

Random search matches or exceeds grid search in best_cv_score on at least 2 of 3 datasets under the same maximum number of evaluated configurations.

The mean held-out test accuracy of the best Bayesian-tuned model is at least 0.5 percentage points higher than the best grid-tuned model on at least 1 of 3 datasets.

Under a fixed evaluation budget, which tuning strategy (Bayesian optimization, grid search, random search) yields the best random-forest cross-validation and test performance on sklearn UCI-style datasets?

Grid search over a compact predefined grid for RandomForestClassifier, capped at 20 evaluated configurations per dataset.

Random search over matched hyperparameter ranges with 20 sampled configurations per dataset.

Bayesian optimization using a Gaussian-process surrogate and acquisition-guided proposals for 20 total evaluations per dataset.

Untuned RandomForestClassifier with sklearn default hyperparameters as a reference baseline.

grid_search_budget20 as classical tuning baseline

Best mean 3-fold cross-validation accuracy discovered by each search method.

Accuracy on a held-out test split using the best hyperparameters found by each method.

Wall-clock time spent in hyperparameter search per method and dataset.

A credible experiment comparing Bayesian optimization, grid search, and random search for RandomForest hyperparameter tuning: methods are implemented with comparable budgets, executed on multiple datasets, and results are mapped directionally to H1/H2/H3.

Score on scientific substance and directional correctness of evidence, not on exact numeric thresholds. If a well-motivated substitute (e.g., Hyperopt, Optuna, or a custom TPE-like surrogate) is used in place of a canonical Bayesian library, credit the scientific intent.

The tuning strategies and shared evaluation setup are implemented correctly.

The submission implements distinct code paths for grid search, random search, and a Bayesian-style search for RandomForestClassifier (or equivalent) rather than reusing identical sampled configurations across methods.

A shared hyperparameter space is defined (covering several meaningful RF hyperparameters such as n_estimators, max_depth, min_samples_split, min_samples_leaf, max_features) so search methods are fairly comparable.

The code loads multiple sklearn classification datasets (e.g., wine, breast_cancer, digits, or comparable) and uses a consistent train/test split plus CV protocol across search methods.

The benchmark is executed and logs optimization quality and efficiency metrics.

Execution outputs numeric best_cv_score and test_accuracy (or equivalents) for each implemented method on at least one dataset in a machine-readable metrics artifact.

Execution evidences budget control: non-default search methods evaluate a roughly comparable number of configurations, so quality comparisons are not confounded by wildly different evaluation counts.

Execution reports a wall-clock timing measure (e.g., search_time_sec) per method and dataset.

Results address H1/H2/H3 directionally and convey interpretable tradeoffs.

The submission compares Bayesian optimization vs random search on best_cv_score across the evaluated datasets and conveys whether Bayesian search is meaningfully better on most — judge directionally against H1.

The submission compares random search vs grid search on best_cv_score and conveys whether random is at least competitive with grid on most datasets (H2).

The submission checks test_accuracy differences between the best Bayesian-tuned and best grid-tuned models and conveys whether any practical gain (H3) is observed on at least one dataset.

The README or writeup reports per-dataset metrics, conveys per-hypothesis outcomes (supported / refuted / inconclusive), and discusses limitations such as small dataset set, budget tightness, surrogate instability, or CV variance. No strict word-count requirement.


Cross-validation strategy reliability for small-sample model selection

Investigating the effectiveness of different cross-validation strategies (k-fold, stratified, repeated, leave-one-out) for model selection with small sample sizes

With small datasets, model selection can become highly sensitive to the
validation protocol. Practitioners often choose among k-fold,
stratified k-fold, repeated k-fold, and leave-one-out cross-validation
(LOOCV), but these strategies trade off bias, variance, and compute in
different ways. A method that appears best under one CV scheme may not be
best under another, especially when class balance is imperfect and sample
counts are low.

This topic studies CV strategy as the independent variable while keeping
candidate models fixed and lightweight. The key quantity is estimation bias:
the gap between CV-estimated score used for model selection and the realized
test score of the selected model on a held-out test set. In small-sample
settings, minimizing this gap is often more important than maximizing raw CV
score, because over-optimistic selection can produce brittle deployments.

A credible CPU-scale experiment should evaluate multiple sklearn datasets
reduced to small training sizes, run several random seeds, and compare at
least four CV strategies under the same model grid and scoring rule. The
study should report not only estimation bias but also variance across seeds
and model-selection stability. Baselines should include standard k-fold and
stratified k-fold; repeated stratified k-fold and LOOCV serve as contrasting
alternatives with different variance/compute characteristics.

The goal is not to prove one universally superior protocol, but to quantify
when more elaborate CV schemes improve reliability enough to justify their
cost under strict CPU budgets.

*Which cross-validation strategy yields the lowest and most stable model-selection estimation bias for small-sample classification tasks under a fixed candidate-model grid?*

RepeatedStratifiedKFold (5 folds × 3 repeats) achieves lower mean absolute estimation bias than plain KFold (5 folds) on at least 2 of 3 evaluated small-sample datasets, averaged over ≥5 random seeds.

StratifiedKFold (5 folds) yields lower across-seed standard deviation of selected-model test accuracy than plain KFold (5 folds) on at least 2 of 3 datasets.

LOOCV does not outperform RepeatedStratifiedKFold in mean absolute estimation bias by more than 0.01 absolute score on any dataset, while requiring greater wall-clock time.

For small-sample classification model selection, how do KFold, StratifiedKFold, RepeatedStratifiedKFold, and LOOCV compare in estimation bias, stability, and compute cost?

Model selection via 5-fold KFold (shuffle=True) on the training split.

Model selection via 5-fold StratifiedKFold (shuffle=True) on the training split.

Model selection via RepeatedStratifiedKFold with 5 folds and 3 repeats.

loocv

Model selection via Leave-One-Out cross-validation on the training split.

stratified_kfold_5 as the standard small-sample baseline

Absolute difference between best CV mean score and held-out test score of the selected model, aggregated over seeds.

Held-out test accuracy of the model selected by each CV strategy, mean over seeds.

Fraction of seeds selecting the modal hyperparameter configuration (higher = more stable).

Runtime per CV strategy over all seeds and datasets on CPU.

sklearn.datasets.load_breast_cancer (subsample training set to n<=120)

sklearn.datasets.load_wine (subsample training set to n<=100)

sklearn.datasets.make_classification (n_samples=160, weights=[0.75,0.25], class_sep tuned for moderate difficulty)

A credible experiment studying how cross-validation strategy choice (k-fold, stratified k-fold, repeated stratified k-fold, LOOCV) affects small-sample model-selection reliability: strategies are implemented consistently, runs cover multiple small-sample datasets with multiple seeds, and results address H1/H2/H3 directionally around estimation bias, stability, and compute.

Score on scientific substance and directional correctness of evidence, not on exact numeric thresholds. Alternative CV strategies or datasets that preserve the scientific question (small-sample bias/stability of CV) should be credited.

Cross-validation strategy conditions and the model-selection pipeline are implemented correctly.

The submission implements multiple distinct CV strategies — typically including plain k-fold, stratified k-fold, repeated stratified k-fold, and optionally LOOCV — as separate code paths used for model selection under a shared candidate-model grid.

The submission uses multiple small-sample datasets (e.g., reduced breast_cancer, wine, or synthetic imbalanced variants) with a held-out test split.

Code computes an estimation-bias style metric (e.g., |best_cv_score - selected_model_test_score|) or equivalent for each (dataset, seed, strategy) and aggregates it reproducibly.

Execution logs required metrics per strategy with multi-seed evaluation.

Execution produces a machine-readable results artifact containing estimation bias (or equivalent) and selected-model test accuracy for each implemented strategy on at least one dataset.

Execution uses multiple random seeds per (dataset, strategy) and records timing information sufficient to compare the more expensive strategies (e.g., LOOCV vs repeated stratified). Honest small-seed runs with variance reported are preferable to a single run.

Reported results address H1/H2/H3 directionally with interpretable evidence.

The submission compares mean estimation bias of repeated-stratified CV vs plain k-fold per dataset and conveys whether repeated stratified is meaningfully better on most datasets — judge directionally against H1.

The submission reports selection stability or across-seed variability for stratified k-fold vs plain k-fold (H2) and compares LOOCV versus repeated stratified CV on estimation bias and runtime (H3), conveying qualitative outcomes for both.

The README or writeup describes the setup, reports per-strategy metrics, conveys per-hypothesis outcomes (supported / refuted / inconclusive), and discusses limitations (small dataset scope, candidate-model grid choice, metric sensitivity, compute budget). No strict word-count requirement.


Benchmarking classical unsupervised outlier detectors under controlled anomaly injection

Benchmarking unsupervised outlier detection methods (IsolationForest, LocalOutlierFactor, OneClassSVM, EllipticEnvelope) on UCI datasets with injected anomalies

Classical unsupervised anomaly-detection methods remain widely used in
production tabular pipelines because they are interpretable enough,
CPU-efficient, and available in standard libraries. Yet their relative
behavior can change sharply as anomaly prevalence and feature scaling vary.
IsolationForest is often robust in heterogeneous feature spaces, while
distance/density methods like LocalOutlierFactor can be strong when local
neighborhoods are informative but may degrade in higher dimensions.

A practical benchmark should avoid external downloads and still provide
realistic control over anomaly rate. One reliable strategy is to start from
clean sklearn classification datasets and inject synthetic anomalies into the
test split by sampling uniformly beyond feature-wise quantile ranges of the
training data. This yields known binary ground truth for evaluation while
preserving a realistic inlier distribution.

A credible CPU-scale study here should compare IsolationForest,
LocalOutlierFactor (novelty mode), OneClassSVM, and EllipticEnvelope under a
shared preprocessing pipeline (e.g., StandardScaler), evaluate ROC-AUC and
PR-AUC across at least two datasets and multiple seeds, and include a simple
baseline such as random scoring. Since contamination is rarely known exactly
in practice, the benchmark should also probe at least two injected anomaly
rates.

The main decision-relevant question is not whether any method can detect easy
anomalies, but which method is consistently best across datasets and
contamination levels under strict CPU constraints. A concise, quantitative
hypothesis-driven comparison is therefore the goal.

*Which classical unsupervised outlier detector is most robust across tabular datasets and injected anomaly rates when measured by ROC-AUC and PR-AUC under a single-core CPU budget?*

IsolationForest achieves higher mean ROC-AUC than OneClassSVM on at least 2 of 3 datasets when averaged over anomaly rates {0.05, 0.10} and ≥3 seeds.

At anomaly rate 0.10, LocalOutlierFactor attains the best PR-AUC (rank 1) on at least 1 of 3 datasets.

Each implemented detector (IsolationForest, LocalOutlierFactor, OneClassSVM, EllipticEnvelope) outperforms a random-score baseline in ROC-AUC by at least 0.10 on the pooled mean across all evaluated datasets/rates.

Which among IsolationForest, LocalOutlierFactor, OneClassSVM, and EllipticEnvelope is most robust across datasets and injected anomaly rates for unsupervised outlier detection under CPU constraints?

sklearn IsolationForest with contamination set to injected rate, 200 trees, default feature subsampling.

sklearn LocalOutlierFactor in novelty=True mode, n_neighbors in [20, 35], contamination set to injected rate.

sklearn OneClassSVM with RBF kernel, nu matched to injected rate, gamma='scale'.

sklearn EllipticEnvelope with contamination set to injected rate and robust covariance estimation.

Baseline assigning i.i.d. uniform random anomaly scores to test points.

one_class_svm_rbf as a classical margin-based reference

ROC-AUC of anomaly scores against injected ground-truth labels on the test set, averaged over seeds.

Average precision / PR-AUC for anomaly class on the test set, averaged over seeds.

Wall-clock seconds for model fit plus test scoring per (dataset, rate, seed), reported as mean.

A credible experiment benchmarking unsupervised outlier detectors (IsolationForest, LocalOutlierFactor, OneClassSVM, EllipticEnvelope, or equivalents) with injected anomalies: methods are implemented as distinct code paths, runs cover multiple datasets across anomaly rates, and results address H1/H2/H3 directionally.

Score on scientific substance and directional correctness of evidence, not on exact numeric thresholds. Well-motivated substitutes (e.g., HBOS, KNN-based outlier) that test the same scientific question should be credited.

The outlier-detection conditions and anomaly-injection pipeline are implemented correctly.

The submission implements multiple distinct detector code paths — typically including IsolationForest, LOF, OneClassSVM, and an elliptic/Gaussian-envelope method — rather than aliases to one shared estimator.

A reproducible anomaly-injection routine is implemented for test data at one or more explicit contamination rates with binary ground-truth anomaly labels.

The code uses multiple datasets (sklearn built-ins or comparable) and applies a consistent preprocessing pipeline (including feature scaling) before model fitting.

Execution records anomaly-detection metrics for each condition.

Execution produces a machine-readable metrics artifact containing numeric ROC-AUC and PR-AUC (or equivalents) for each implemented method on at least one (dataset, rate) cell.

Reported results aggregate over multiple random seeds and include at least two anomaly rates, with a dispersion measure. Honest small-seed runs with variance reported are preferable to a single run.

The run logs a wall-clock timing measure per method and demonstrates completion within a CPU-only workflow.

Quantitative analysis addresses H1/H2/H3 directionally.

The submission compares mean ROC-AUC of IsolationForest vs OneClassSVM per dataset and conveys whether IsolationForest is meaningfully better on most datasets — judge directionally against H1.

The submission conveys whether LOF is competitive on PR-AUC for at least one dataset at a higher anomaly rate (H2) and whether detectors meaningfully outperform a random baseline on pooled ROC-AUC (H3).

The README or writeup describes setup and anomaly injection, reports ROC-AUC/PR-AUC results, conveys per-hypothesis outcomes (supported / refuted / inconclusive), and discusses limitations (synthetic anomalies, dataset scope, hyperparameter sensitivity, seed count). No strict word-count requirement.


Comparing clustering algorithms on synthetic non-convex and anisotropic shapes

Comparing clustering algorithms (k-means, agglomerative, DBSCAN, HDBSCAN, spectral) on synthetic shapes (moons, circles, blobs, anisotropic) with ground-truth labels

Clustering algorithms encode very different geometric assumptions. K-means
prefers spherical, equal-variance groups; agglomerative clustering can adapt
to hierarchical structure depending on linkage; DBSCAN finds dense regions
and can recover non-convex shapes while labeling outliers as noise; spectral
clustering can separate manifolds when graph affinity is appropriate.
Because these assumptions interact strongly with data geometry, a single
benchmark score often hides systematic failure modes.

Synthetic datasets are ideal for a fast but meaningful comparison because we
can generate known structures with ground-truth labels: intertwined moons,
concentric circles, and anisotropic Gaussian blobs. These patterns expose
strengths and weaknesses that are difficult to isolate in real-world data.
With labeled synthetic data, external clustering metrics like adjusted rand
index (ARI) and normalized mutual information (NMI) directly quantify
partition recovery quality.

A credible CPU-scale study should compare several algorithms under a shared
protocol: standardized inputs, fixed train-size generation per seed, and
explicit handling of methods that output noise labels. It should include at
least one centroid baseline (k-means) and one density method (DBSCAN), then
test whether non-linear methods systematically outperform centroid methods on
non-convex shapes while avoiding large regressions on anisotropic blobs.

The practical goal is not to crown one universally best algorithm, but to
establish data-shape-dependent guidance. If shape complexity changes which
method wins, practitioners should choose clustering tools by geometric prior
rather than habit.

*How strongly does dataset geometry (non-convex vs anisotropic) determine which clustering algorithm achieves the best ground-truth agreement under a fixed CPU-budget protocol?*

On the two non-convex datasets (moons and circles), at least one of {DBSCAN, SpectralClustering} achieves mean ARI that is at least 0.15 higher than k-means, averaged over ≥5 seeds.

On the anisotropic-blobs dataset, agglomerative clustering (ward linkage) achieves ARI greater than or equal to k-means in the seed-averaged result (difference >= 0.00).

No single algorithm ranks first in ARI on all evaluated datasets; i.e., the best-ARI method differs on at least one dataset from the global winner.

How strongly does dataset geometry (non-convex vs anisotropic) determine which clustering algorithm achieves the best agreement with ground-truth labels under a fixed CPU-budget protocol?

KMeans baseline with n_clusters set to dataset ground-truth class count; n_init>=10.

AgglomerativeClustering with ward linkage and n_clusters equal to ground-truth class count.

DBSCAN with eps selected from a small grid per dataset (e.g., 0.1 to 0.5) and fixed min_samples (e.g., 5).

SpectralClustering with RBF affinity and n_clusters equal to ground-truth class count.

kmeans_k2_or_k3 as centroid-based baseline

agglomerative_ward as hierarchical baseline

Adjusted Rand Index against ground-truth labels, averaged over seeds.

Normalized Mutual Information against ground-truth labels.

Internal cohesion/separation score computed from predicted labels (excluding noise-only failures).

circles

sklearn.datasets.make_circles (n_samples~800, noise~0.06, factor~0.5)

sklearn.datasets.make_blobs followed by linear transformation to create anisotropy

A credible experiment comparing clustering algorithms across synthetic geometric datasets: clustering conditions are implemented, execution covers multiple datasets with repeated seeds and ARI-centric reporting, and the analysis addresses H1/H2/H3 directionally.

Score on scientific substance and directional correctness of evidence, not on exact numeric thresholds. Well-motivated algorithm substitutes (e.g., HDBSCAN for DBSCAN) that test the same question should be credited.

The clustering conditions and dataset generators are implemented correctly.

The submission implements multiple distinct clustering conditions — typically k-means, agglomerative, DBSCAN, and/or spectral — via sklearn APIs with separate code paths and method-specific hyperparameters.

The submission generates multiple synthetic datasets with varied geometry (e.g., moons, circles, anisotropic blobs), preserves ground-truth labels, and standardizes features before clustering where appropriate.

The experiment framework includes seed control and a consistent per-dataset protocol for cluster-count-dependent methods.

Execution produces clustering metrics for each condition.

Execution outputs a metrics artifact containing numeric adjusted_rand_score (or equivalent) for each implemented condition on at least one dataset.

Reported results are aggregated over multiple random seeds per (dataset, condition) with a dispersion estimate. Honest small-seed runs with variance reported are preferable to a single run.

For DBSCAN (or equivalent density-based method), the run documents a reasonable choice of eps/min_samples (either via a small search or a justified fixed value), logged in the artifacts.

Results address H1/H2/H3 directionally with interpretable evidence.

The submission compares ARI of density-based / spectral methods against k-means on nonlinear-geometry datasets and conveys whether they produce meaningfully better cluster assignments — judge directionally against H1.

The submission conveys dataset-wise ARI outcomes — whether agglomerative-ward is at least competitive with k-means on anisotropic blobs (H2) and whether any single method wins across all datasets (H3).

The README or writeup describes setup, key ARI/NMI findings per dataset, conveys per-hypothesis outcomes (supported / refuted / inconclusive), and notes limitations (synthetic-only scope, hyperparameter sensitivity, seed count, metric dependence). No strict word-count requirement.


Evaluating kernel choices for Gaussian Process regression on synthetic 1-D and 5-D functions

Evaluating kernel choices (RBF, polynomial, Matern-3/2, Matern-5/2) for Gaussian Process regression on synthetic 1-D and 5-D functions with homoscedastic noise

Gaussian Process (GP) regression performance depends strongly on the kernel,
which encodes assumptions about function smoothness and local structure.
Practitioners often default to the RBF kernel because it is easy to optimize
and widely available, but this choice can underperform when the target
function has roughness patterns that are better matched by Matern families or
when polynomial trends dominate part of the signal. Since kernel
misspecification can be subtle, a short benchmark should compare predictive
uncertainty quality as well as point error.

A CPU-scale study can be done entirely with sklearn GaussianProcessRegressor
on synthetic datasets where the data-generating process is controlled. In 1-D,
a mildly nonstationary smooth function with additive Gaussian noise can expose
differences in extrapolation and uncertainty tails. In 5-D, a mixed
sinusoidal-plus-quadratic target can stress anisotropy and interactions while
remaining fast enough for repeated fitting under multiple random seeds.

To make claims measurable, the experiment should evaluate at least four kernel
choices (RBF, polynomial, Matern-3/2, Matern-5/2) under the same train/test
splits, optimizer restarts, and noise model assumptions. Because the topic's
key metric is negative log-likelihood, uncertainty calibration quality should
be primary; RMSE and R^2 provide supporting evidence for point prediction.
Seed averaging is important because synthetic splits and optimizer
initialization can change marginal likelihood optimization outcomes.

The core question is whether smoother kernels (RBF, Matern-5/2) consistently
outperform rougher or mismatched kernels (Matern-3/2, polynomial) on test NLL
across both low- and moderate-dimensional synthetic tasks, and whether ranking
by NLL aligns with ranking by RMSE.

*Which GP kernel among RBF, polynomial, Matern-3/2, and Matern-5/2 yields the best uncertainty-aware generalization (lowest test NLL) on synthetic 1-D and 5-D noisy regression functions?*

Across the two synthetic datasets, at least one of {RBF, Matern-5/2} achieves the lowest mean test NLL on each dataset (averaged over >=5 seeds).

The polynomial kernel has mean test NLL at least 10% worse than the best kernel on at least 1 of the 2 datasets.

Kernel ranking by mean test NLL and by mean RMSE is not identical on at least 1 dataset, indicating uncertainty quality and point error disagree.

How do RBF, polynomial, Matern-3/2, and Matern-5/2 kernels compare for Gaussian Process regression on synthetic 1-D and 5-D noisy functions when judged primarily by test NLL?

GaussianProcessRegressor with ConstantKernel * RBF + WhiteKernel, hyperparameters optimized by log-marginal-likelihood.

GaussianProcessRegressor with ConstantKernel * DotProduct^2 (implemented via polynomial feature mapping + DotProduct kernel) + WhiteKernel as a polynomial-trend proxy.

GaussianProcessRegressor with ConstantKernel * Matern(nu=1.5) + WhiteKernel.

GaussianProcessRegressor with ConstantKernel * Matern(nu=2.5) + WhiteKernel.

gp_rbf as the common default-kernel baseline

gp_poly_deg2 as a mismatched-trend baseline

Mean negative log predictive density on held-out test data using GP predictive mean and variance, averaged over seeds.

rmse

Root mean squared error on test targets, averaged over seeds.

Coefficient of determination on test targets, averaged over seeds.

Generated with numpy: x in [-3,3], y = sin(2x) + 0.3x + epsilon, epsilon~N(0, 0.15^2).

A credible experiment studying kernel-choice effects (RBF, polynomial, Matern-3/2, Matern-5/2, or equivalents) for Gaussian Process regression on synthetic noisy functions: conditions are implemented, execution covers multiple datasets with repeated seeds, and results address H1/H2/H3 directionally using test NLL as the primary metric.

Score on scientific substance and directional correctness of evidence, not on exact numeric thresholds. Alternative kernels or dataset dimensions that test the same scientific question should be credited.

The GP kernel conditions and synthetic datasets are implemented correctly.

The submission implements multiple kernel conditions — typically including RBF, a polynomial kernel, and one or more Matern kernels — as distinct GP configurations with comparable noise handling.

The submission generates synthetic regression datasets with explicit noise (both a low-dimensional and a higher-dimensional case preferred) and creates train/test splits.

Experimental setup controls are consistent across kernels (same split protocol, seed handling, and optimizer restart policy), enabling fair kernel comparison.

Execution logs primary and secondary metrics for each kernel and dataset.

Execution produces a metrics artifact containing numeric test NLL and RMSE (or equivalents) for every implemented (kernel, dataset) pair.

Reported metrics are aggregated over multiple random seeds per (kernel, dataset) cell, with a dispersion measure. Honest small-seed runs with variance reported are preferable to a single run.

Results analysis addresses H1/H2/H3 directionally with quantitative comparisons.

The submission reports per-dataset mean test-NLL ranking and conveys whether smooth kernels (RBF, Matern-5/2) tend to be best — judge directionally against H1.

The submission compares the polynomial-kernel NLL against the best kernel and conveys whether polynomial is meaningfully worse on at least one dataset (H2).

The submission compares kernel rankings by test NLL and RMSE on each dataset and conveys whether rankings can differ between these two metrics (H3).

The README or writeup describes implementation choices, dataset generation, metric tables, conveys per-hypothesis outcomes (supported / refuted / inconclusive), and notes limitations (synthetic-only scope, kernel parameterization, seed/compute constraints). No strict word-count requirement.


Comparing split-conformal, Mondrian conformal, and CQR-style intervals on heteroscedastic regression

Comparing conformal prediction procedures (split-conformal, Mondrian, CQR) for achieving target coverage on heteroscedastic regression tasks

Conformal prediction provides finite-sample coverage guarantees with minimal
assumptions, making it attractive for uncertainty quantification in practical
regression. However, standard split-conformal intervals are often globally
calibrated and can be inefficient when noise is heteroscedastic: they may
over-cover low-noise regions and under-cover high-noise regions. This
motivates conditional variants such as Mondrian conformal (group-conditional
calibration) and CQR-style approaches that adapt interval width through
quantile modeling before conformal correction.

In small, CPU-only settings, one can still run a meaningful comparison by
using sklearn-compatible regressors and synthetic data where heteroscedastic
structure is controlled. A rigorous setup should evaluate not only marginal
coverage but also efficiency (mean interval width) and conditional behavior
(coverage gap across strata of predicted difficulty or input regions). These
diagnostics reveal when methods achieve nominal coverage by producing overly
wide intervals versus genuinely adapting to varying noise.

A credible benchmark should include at least one homoscedastic control and at
least one heteroscedastic dataset, then compare split-conformal, a Mondrian
variant (binning by an auxiliary score such as |x| or predicted scale), and a
CQR-style method based on lower/upper quantile regressors plus conformal
adjustment. Repeated random splits (multiple seeds) are needed because
conformal procedures can vary with calibration sample composition.

The key question is whether adaptive procedures can reduce conditional
miscoverage while maintaining near-target marginal coverage and reasonable
interval width under strict runtime constraints.

*Do Mondrian and CQR-style conformal procedures achieve lower conditional coverage error than vanilla split-conformal at comparable marginal coverage on heteroscedastic regression tasks?*

On heteroscedastic datasets, at least one adaptive method (Mondrian or CQR-style) attains a lower absolute coverage gap to the 90% target than split-conformal by at least 0.02 on at least 2 of 3 datasets, averaged over ≥5 seeds.

At matched target 90% coverage, CQR-style conformal yields mean interval width no larger than split-conformal on at least 2 of 3 datasets (difference ≤ 0.00).

Mondrian conformal reduces worst-bin conditional miscoverage (max over 4 bins of |bin_coverage-0.90|) by at least 0.03 versus split-conformal on at least 2 of 3 datasets.

Do adaptive conformal procedures (Mondrian, CQR-style) improve coverage quality and efficiency over split-conformal for heteroscedastic regression under a fixed 90% target coverage?

Vanilla split-conformal regression using RandomForestRegressor point predictor and absolute residual conformity scores on a held-out calibration split.

Mondrian split-conformal with the same base predictor, calibration and test points partitioned into 4 bins by an auxiliary score (e.g., fitted value or |x0| proxy), with per-bin quantiles.

CQR-style method using GradientBoostingRegressor quantile models (alpha=0.05, 0.95) and split-conformal correction on calibration residuals of quantile bands.

Non-conformal baseline using raw quantile regression interval [q0.05, q0.95] without conformal adjustment.

split_conformal_rf is the primary conformal baseline

naive_quantile_no_conformal is the non-conformal baseline

Absolute difference between empirical marginal coverage and target 0.90 on the test split, averaged over seeds.

Average prediction interval width on test samples, averaged over seeds.

Maximum across 4 bins of absolute deviation between bin coverage and 0.90, measuring conditional coverage disparity.

synthetic: sklearn.datasets.make_friedman1 with multiplicative noise scale depending on x0

diabetes

sklearn.datasets.load_diabetes (tabular regression benchmark)

A credible experiment comparing split-conformal, Mondrian conformal, CQR-style, and optionally naive-quantile prediction intervals for ~90% target coverage on heteroscedastic regression: methods are implemented, executed on multiple datasets with multiple seeds, and results address H1/H2/H3 directionally with coverage-gap-focused analysis.

Score on scientific substance and directional correctness of evidence, not on exact numeric thresholds. Alternative conformal variants (e.g., jackknife+, locally adaptive) that test the same scientific question should be credited.

The conformal and baseline interval-construction conditions are implemented correctly.

The submission implements multiple distinct conditions — typically including split conformal, a Mondrian/group-adaptive variant, and a CQR-style or naive-quantile baseline — as separate code paths, not a single shared interval formula.

Conformal methods use explicit train/calibration/test separation (or equivalent cross-fit logic) so calibration quantiles are computed on data not used to fit base regressors.

The submission uses multiple datasets (including at least one heteroscedastic synthetic dataset) and prepares regression targets correctly.

Execution outputs interval-quality metrics for each condition.

Execution produces a metrics artifact containing numeric coverage gap and mean interval width (or equivalents) for each implemented condition on at least one dataset.

Reported metrics are aggregated over multiple random seeds per (condition, dataset) cell with a dispersion estimate. Honest small-seed runs with variance reported are preferable to a single run.

Execution computes a conditional-coverage diagnostic (e.g., worst-bin miscoverage across several bins) for the conformal methods.

Quantitative analysis addresses H1/H2/H3 directionally and discusses trade-offs between coverage, conditional validity, and interval efficiency.

The submission compares coverage gap of adaptive methods (Mondrian, CQR-style) against plain split conformal per dataset and conveys whether adaptive methods are meaningfully closer to the nominal coverage — judge directionally against H1.

The submission evaluates whether CQR-style intervals achieve mean interval width comparable to or narrower than plain split conformal on most datasets and conveys an H2 outcome.

The submission compares worst-bin miscoverage for Mondrian conformal vs plain split conformal and conveys whether Mondrian yields a meaningful improvement in conditional coverage (H3).

The README or writeup describes methods and datasets, reports key metric values (coverage gap, interval width, worst-bin miscoverage), conveys per-hypothesis outcomes (supported / refuted / inconclusive), and notes limitations (finite seeds, binning choice, synthetic-to-real transfer). No strict word-count requirement.


Filter-based vs embedded L1 feature selection with injected noise features

Investigating filter-based feature selection (mutual_info_classif, chi2, f_classif) vs embedded L1-logistic selection on UCI classification datasets with injected irrelevant features

Feature selection on tabular classification problems is often presented as a
choice between simple univariate filters and embedded sparse models.
Univariate filters such as chi-squared, ANOVA F-score, and mutual
information are computationally cheap and easy to apply in a pipeline, but
they evaluate each feature independently and can miss interactions. Embedded
L1-regularized logistic regression performs selection during model fitting,
potentially yielding feature subsets that better align with predictive
structure when many irrelevant variables are present.

A practical stress test is to inject synthetic irrelevant features into
otherwise standard benchmark datasets. This setup creates controlled feature
dilution while preserving original labels and base difficulty. Under such
dilution, methods that can ignore noise should maintain accuracy and avoid
selecting many synthetic features. Conversely, methods sensitive to spurious
univariate associations may degrade as noise dimension increases.

A credible CPU-only study should compare at least three filter selectors
(mutual_info_classif, chi2, f_classif) and an embedded L1 selector in a
shared downstream classifier pipeline, evaluate on multiple sklearn
classification datasets, and report both predictive performance and selection
quality. Including multiple random seeds is important because both noise
injection and train/test splits can change apparent selector rankings.

The key aim is not to reproduce a single published table, but to determine
whether embedded sparsity offers robustness advantages over filters as
irrelevant dimensions grow, and whether that robustness generalizes across
datasets with different feature scales and class structures.

*Does embedded L1-logistic feature selection retain predictive performance and reject injected irrelevant features more reliably than univariate filter methods under controlled feature-noise injection?*

With 5x injected irrelevant features (relative to original feature count), L1-logistic embedded selection achieves higher mean test_accuracy than each filter method (mutual_info_classif, chi2, f_classif) on at least 2 of 3 datasets, averaged over >=5 seeds.

At fixed selected-feature budget k=20 (or all features if p<20), L1-logistic selects a lower fraction of injected irrelevant features than the average of the three filter methods on all evaluated datasets.

From no-injection to 5x-injection settings, mean test_accuracy drop for L1-logistic is <= 3 percentage points on at least 2 of 3 datasets.

Does embedded L1-logistic feature selection outperform univariate filter methods in robustness to injected irrelevant features on sklearn tabular classification datasets?

SelectKBest(mutual_info_classif, k=20 or p if p<20) followed by LogisticRegression classifier.

MinMax scaling to nonnegative domain, then SelectKBest(chi2, k=20 or p if p<20) followed by LogisticRegression classifier.

SelectKBest(f_classif, k=20 or p if p<20) followed by LogisticRegression classifier.

L1-penalized LogisticRegression (saga/liblinear) used as embedded selector; keep top-20 by absolute coefficient magnitude (or nonzero if <=20), retrain LogisticRegression on selected subset.

filter_f_classif_k20 as a standard univariate linear-statistic baseline

filter_mutual_info_k20 as a nonlinear dependency baseline

Held-out test accuracy averaged over >=5 seeds for each dataset and injection level.

Fraction of selected features that come from injected irrelevant columns.

Absolute percentage-point drop in test_accuracy from 0x to 5x injection.

A credible experiment studying filter-based feature selection (mutual information, chi-square, ANOVA F) versus embedded L1-logistic selection under injected irrelevant features: conditions are implemented, runs cover multiple datasets with multiple seeds and injection levels, and results address H1/H2/H3 directionally.

Score on scientific substance and directional correctness of evidence, not on exact numeric thresholds. Alternative filter methods or embedded selectors (e.g., tree-based importance) that test the same scientific question should be credited.

Feature-selection conditions and the noise-injection pipeline are implemented correctly.

The submission implements multiple distinct selection conditions — typically including mutual information, chi-square, ANOVA F, and L1-logistic (or a comparable embedded method) — rather than reusing one selector for all conditions.

The pipeline injects synthetic irrelevant features at one or more defined levels (including a no-injection baseline and a higher-injection case) and tracks which columns are injected so a noise-selection rate can be computed.

The submission uses multiple datasets (sklearn built-ins or comparable) and a reasonable train/test split.

Execution produces metrics across selectors and injection levels.

Execution outputs a structured metrics artifact containing numeric test accuracy and a noise-selection-rate style measure for each implemented condition on at least one dataset with at least two injection levels.

Reported metrics are aggregated over multiple random seeds per (dataset, condition, injection level) cell with a dispersion measure. Honest small-seed runs with variance reported are preferable to a single run.

Results address H1/H2/H3 directionally with a clear narrative.

The submission compares mean test accuracy at the higher injection level between the embedded L1-logistic method and each filter method per dataset and conveys whether L1 tends to be better — judge directionally against H1.

The submission reports the noise-selection rate for each method and conveys whether the embedded method selects fewer irrelevant features than the filter-method average on most datasets (H2).

The submission reports the accuracy drop from low-injection to high-injection for the embedded method and conveys whether the drop is small on most datasets (H3).

The README or writeup describes the experimental setup, reports key metric tables, conveys per-hypothesis outcomes (supported / refuted / inconclusive), and notes limitations such as synthetic-noise realism, k-choice, classifier dependence, and seed/dataset scope. No strict word-count requirement.