index.html

<!doctype html>
<html lang="en">
<head>
  <meta charset="utf-8">
  <meta name="viewport" content="width=device-width, initial-scale=1">
  <!-- Bumped with every data refresh, together with the version history at the bottom. -->
  <meta name="selectarena-version" content="0.2.1">
  <title>SelectArena Leaderboard</title>
  <meta name="description" content="Leaderboard of feature selection methods on real-world tabular data.">
  <link rel="icon" type="image/svg+xml" href="assets/logo.svg">
  <link rel="stylesheet" href="assets/style.css">
</head>
<body>
<div class="page">
  <header class="header">
    <div class="brand">
      <img class="logo" src="assets/logo.svg" alt="" width="120" height="84">
      <div>
        <h1><span class="mark">Select</span>Arena</h1>
        <p class="subtitle">A modern benchmark for feature selection on real-world tabular data</p>
      </div>
    </div>
    <ul class="facts" id="facts"></ul>
    <nav class="links">
      <a href="https://arxiv.org/abs/2610.13644" target="_blank" rel="noopener">Paper</a>
      <a href="https://github.com/schaeferbasti/SelectArena" target="_blank" rel="noopener">Code</a>
      <a href="https://github.com/autogluon/tabarena" target="_blank" rel="noopener">Built on TabArena</a>
    </nav>
  </header>

  <p class="intro">
    Each feature selection method keeps a fixed budget of a dataset's features; a downstream model is
    then trained on the selected features only. Methods compete on the same dataset split, downstream
    model and budget, and the leaderboard ranks them by <strong>Elo</strong>. Pick a task type,
    downstream model or budget to see the leaderboard of that slice.
  </p>

  <div class="tabs" role="tablist" aria-label="Leaderboard sections">
    <button role="tab" id="tab-leaderboard" aria-controls="panel-leaderboard" aria-selected="true" data-tab="leaderboard">Leaderboard</button>
    <button role="tab" id="tab-datasets" aria-controls="panel-datasets" aria-selected="false" data-tab="datasets">Per dataset</button>
    <button role="tab" id="tab-robustness" aria-controls="panel-robustness" aria-selected="false" data-tab="robustness">Stability &amp; validity</button>
    <button role="tab" id="tab-about" aria-controls="panel-about" aria-selected="false" data-tab="about">About</button>
  </div>

  <!-- Leaderboard -->
  <section role="tabpanel" id="panel-leaderboard" aria-labelledby="tab-leaderboard">
    <div id="no-results" class="notice" hidden>
      <strong>No results published yet.</strong> The leaderboard fills in once the first benchmark
      results are published from the
      <a href="https://github.com/schaeferbasti/SelectArena" target="_blank" rel="noopener">SelectArena repository</a>.
    </div>
    <div id="leaderboard-content" hidden>
      <div class="controls">
        <div class="control"><span class="control-label" id="label-tasks">Task type</span>
          <div class="chips" id="chips-tasks" role="group" aria-labelledby="label-tasks"></div></div>
        <div class="control"><span class="control-label" id="label-model">Downstream model</span>
          <div class="chips" id="chips-model" role="group" aria-labelledby="label-model"></div></div>
        <div class="control"><span class="control-label" id="label-budget">Feature budget</span>
          <div class="chips" id="chips-budget" role="group" aria-labelledby="label-budget"></div></div>
      </div>
      <p class="summary" id="summary" aria-live="polite"></p>
      <div id="subset-missing" class="notice" hidden>
        No leaderboard for this selection: fewer than two methods have results.
      </div>
      <div id="subset-content">
        <iframe class="frame sa-frame" id="frame-figure" title="Elo rating with stability and validity" style="height:480px"></iframe>
        <p class="caption">
          <strong>Elo rating of each feature selection method, from worst to best.</strong> Elo
          summarizes how often a method beats another on the same dataset, budget and downstream model
          (lower downstream error wins), with Random anchored at 1000 (dashed line). Error bars are 95%
          intervals from cluster-bootstrap resamples of the datasets. The left bar of each method is
          shaded by <span class="stab">stability</span> (how little the selected subset changes across
          bootstrap resamples of the data), the right bar by <span class="val">validity</span> (how well
          injected noise features are rejected); darker is better for both. Hatched bars have no
          stability or validity result yet.
        </p>
        <iframe class="frame sa-frame" id="frame-table" title="Leaderboard table" style="height:640px"></iframe>
      </div>
    </div>
  </section>

  <!-- Per dataset -->
  <section role="tabpanel" id="panel-datasets" aria-labelledby="tab-datasets" hidden>
    <p class="prose">Mean rank of each method on each dataset, over splits, downstream models and budgets (1 is best).</p>
    <div id="datasets-missing" class="notice" hidden>No per-dataset results published yet.</div>
    <iframe class="frame sa-frame" id="frame-datasets" title="Mean rank per dataset" style="height:900px"></iframe>
  </section>

  <!-- Stability & validity -->
  <section role="tabpanel" id="panel-robustness" aria-labelledby="tab-robustness" hidden>
    <div class="prose">
      <p>These metrics come from runs of the selectors alone, without a downstream model.</p>
      <ul>
        <li><strong>Stability</strong> (Nogueira, Sechidis &amp; Brown, JMLR 2018): agreement of the
          selected features across bootstrap resamples of the data, corrected for chance; 1 means
          identical selections, 0 the agreement of random selections.</li>
        <li><strong>Validity</strong>: share of the budget filled with original features after synthetic
          noise features are added (the noise level is the number of noise features per original one).
          The leaderboard and the figure use the validity at noise level <span id="validity-noise">1.0</span>.</li>
      </ul>
    </div>
    <h2>Stability</h2>
    <div id="stability-missing" class="notice" hidden>No stability results published yet.</div>
    <iframe class="frame sa-frame" id="frame-stability" title="Stability per method" style="height:640px"></iframe>
    <h2>Validity</h2>
    <div id="validity-missing" class="notice" hidden>No validity results published yet.</div>
    <div id="validity-tables"></div>
  </section>

  <!-- About -->
  <section role="tabpanel" id="panel-about" aria-labelledby="tab-about" hidden>
    <div class="prose">
      <h2>How the leaderboard is computed</h2>
      <p>Methods are compared within a <em>comparison group</em>: one dataset split, one downstream
        model configuration and one feature budget.</p>
      <ul>
        <li><strong>Elo</strong>: Bradley-Terry rating fit on all pairwise matches. Within each dataset,
          every combination of budget and downstream model (errors averaged over splits) is a setting in
          which every pair of methods plays a match, won by the lower error; ties are skipped. Random is
          anchored at 1000; a gap of 150 points means about 70% win probability. The 95% interval comes
          from cluster-bootstrap resamples of the datasets.</li>
        <li><strong>Avg. rank</strong>: mean rank of the method's downstream error within the groups (1 is best).</li>
        <li><strong>Win rate</strong>: share of pairwise comparisons won against the other methods (ties count half).</li>
        <li><strong>Improvability</strong>: mean relative gap <code>(error − best error) / error</code> to
          the best method of the group (0 is best).</li>
        <li><strong>Error vs. all features</strong>: mean ratio of the error to the same model trained on
          all features. Below 1 means the selection improved the model.</li>
        <li><strong>Stability</strong> and <strong>Validity</strong>: see the <em>Stability &amp; validity</em> tab.</li>
        <li><strong>Time</strong>: median wall-clock time of the selection. <strong>Fallback</strong>:
          share of runs in which the method ran out of time and random features filled the budget.</li>
      </ul>
      <p><strong>Budgets.</strong> Budget <em>i</em> of <em>b</em> keeps <code>ceil(d · i / (b + 1))</code>
        of the <code>d</code> features when <code>d ≤ 100</code>, and <code>ceil(d ^ (i / (b + 1)))</code> otherwise.</p>
      <p><strong>Method types.</strong> The Type column follows the paper's functional taxonomy
        (Appendix A). <em>Search</em> methods evaluate many feature subsets with a model-dependent
        utility: forward search (SFS), backward search (SBE, MarkovBlanket) and leave-one-out (LOCO).
        <em>Non-search</em> methods avoid repeated subset evaluations and rely on a proxy instead:
        correlation (F-Test), distance ((R)ReliefF, LaplacianScore), information theory (MI, mRMR,
        GainRatio), trees (CART, RFImportance) and regularization (Lasso, ElasticNet). Random is the
        baseline.</p>

      <h2>Adding a method</h2>
      <p>Implement the method as described in the
        <a href="https://github.com/schaeferbasti/SelectArena" target="_blank" rel="noopener">SelectArena repository</a>
        (one <code>AbstractFeatureSelector</code> subclass) and open a pull request or an issue there.</p>

      <h2>Data</h2>
      <p>Everything shown here is plain files in this Space's <code>data/</code> folder:
        <a href="data/meta.json" target="_blank">meta.json</a> lists every published leaderboard, and each
        one is a CSV, e.g. the
        <a href="data/subsets/tasks_all/models_all/budgets_all/website_leaderboard.csv" target="_blank">overall leaderboard</a>.
        Every table and figure also has its own CSV / SVG / PNG download button.</p>
    </div>
  </section>

  <footer>
    <details>
      <summary>Citation</summary>
<pre>@article{schaefer2026selectarena,
  title   = {SelectArena: A Modern Benchmark for Feature Selection on Real-world Tabular Data},
  author  = {Sch{\"a}fer, Bastian and Matus, Dominika and Purucker, Lennart and Knauer, Ricardo and
             Rodner, Erik and Hutter, Frank and Sokolov, Anton and Liefooghe, Arnaud},
  journal = {arXiv preprint arXiv:2610.13644},
  year    = {2026}
}</pre>
    </details>
    <details>
      <summary>Version history (current: <span id="current-version"></span>)</summary>
      <!-- version-history: newest entry first; the top entry's version must equal selectarena-version above. -->
      <ul class="history" id="version-history">
        <li><strong>2026/10/14 · v0.2.1</strong>: New logo; datasets named for results that use
          TabArena task ids.</li>
        <li><strong>2026/10/14 · v0.2.0</strong>: Type column follows the paper's taxonomy (search /
          non-search subgroups); datasets are shown by name in the per-dataset table; one validity
          table per noise level.</li>
        <li><strong>2026/10/13 · v0.1.0</strong>: Static leaderboard site (free Hugging Face static Space):
          one leaderboard per task type, downstream model and budget, interactive main figure (Elo with
          stability and validity), per-dataset ranks and stability / validity tables.</li>
      </ul>
      <!-- /version-history -->
    </details>
  </footer>
</div>
<script src="assets/app.js"></script>
</body>
</html>