Spaces:
Running
Running
New LB
#3
by LennartPurucker - opened
This view is limited to 50 files because it contains too many changes. See the raw diff here.
- .gitignore +0 -2
- README.md +3 -11
- data/full-imputed-cls/leaderboard.tex +53 -0
- data/full-imputed-cls/tabarena_leaderboard.csv +46 -0
- data/full-imputed-reg/leaderboard.tex +52 -0
- data/full-imputed-reg/tabarena_leaderboard.csv +45 -0
- data/{imputation_no/splits_all/tasks_multiclass/datasets_medium/tuning-impact-elo.png.zip β full-imputed-reg/time_plot.png.zip} +2 -2
- data/full-imputed/leaderboard.tex +53 -0
- data/full-imputed/tabarena_leaderboard.csv +46 -0
- data/imputation_no/splits_all/tasks_all/datasets_all/n_datasets_51 +0 -0
- data/imputation_no/splits_all/tasks_all/datasets_all/pareto_front_improvability_vs_time_infer.png.zip +0 -3
- data/imputation_no/splits_all/tasks_all/datasets_all/pareto_n_configs_imp.png.zip +0 -3
- data/imputation_no/splits_all/tasks_all/datasets_all/tuning-impact-elo.png.zip +0 -3
- data/imputation_no/splits_all/tasks_all/datasets_all/website_leaderboard.csv +0 -60
- data/imputation_no/splits_all/tasks_all/datasets_all/winrate_matrix.png.zip +0 -3
- data/imputation_no/splits_all/tasks_all/datasets_medium/n_datasets_15 +0 -0
- data/imputation_no/splits_all/tasks_all/datasets_medium/pareto_front_improvability_vs_time_infer.png.zip +0 -3
- data/imputation_no/splits_all/tasks_all/datasets_medium/pareto_n_configs_imp.png.zip +0 -3
- data/imputation_no/splits_all/tasks_all/datasets_medium/tuning-impact-elo.png.zip +0 -3
- data/imputation_no/splits_all/tasks_all/datasets_medium/website_leaderboard.csv +0 -60
- data/imputation_no/splits_all/tasks_all/datasets_medium/winrate_matrix.png.zip +0 -3
- data/imputation_no/splits_all/tasks_all/datasets_small/n_datasets_36 +0 -0
- data/imputation_no/splits_all/tasks_all/datasets_small/pareto_front_improvability_vs_time_infer.png.zip +0 -3
- data/imputation_no/splits_all/tasks_all/datasets_small/pareto_n_configs_imp.png.zip +0 -3
- data/imputation_no/splits_all/tasks_all/datasets_small/tuning-impact-elo.png.zip +0 -3
- data/imputation_no/splits_all/tasks_all/datasets_small/website_leaderboard.csv +0 -60
- data/imputation_no/splits_all/tasks_all/datasets_small/winrate_matrix.png.zip +0 -3
- data/imputation_no/splits_all/tasks_binary/datasets_all/n_datasets_30 +0 -0
- data/imputation_no/splits_all/tasks_binary/datasets_all/pareto_front_improvability_vs_time_infer.png.zip +0 -3
- data/imputation_no/splits_all/tasks_binary/datasets_all/pareto_n_configs_imp.png.zip +0 -3
- data/imputation_no/splits_all/tasks_binary/datasets_all/tuning-impact-elo.png.zip +0 -3
- data/imputation_no/splits_all/tasks_binary/datasets_all/website_leaderboard.csv +0 -62
- data/imputation_no/splits_all/tasks_binary/datasets_all/winrate_matrix.png.zip +0 -3
- data/imputation_no/splits_all/tasks_binary/datasets_medium/n_datasets_9 +0 -0
- data/imputation_no/splits_all/tasks_binary/datasets_medium/pareto_front_improvability_vs_time_infer.png.zip +0 -3
- data/imputation_no/splits_all/tasks_binary/datasets_medium/pareto_n_configs_imp.png.zip +0 -3
- data/imputation_no/splits_all/tasks_binary/datasets_medium/tuning-impact-elo.png.zip +0 -3
- data/imputation_no/splits_all/tasks_binary/datasets_medium/website_leaderboard.csv +0 -63
- data/imputation_no/splits_all/tasks_binary/datasets_medium/winrate_matrix.png.zip +0 -3
- data/imputation_no/splits_all/tasks_binary/datasets_small/n_datasets_21 +0 -0
- data/imputation_no/splits_all/tasks_binary/datasets_small/pareto_front_improvability_vs_time_infer.png.zip +0 -3
- data/imputation_no/splits_all/tasks_binary/datasets_small/pareto_n_configs_imp.png.zip +0 -3
- data/imputation_no/splits_all/tasks_binary/datasets_small/tuning-impact-elo.png.zip +0 -3
- data/imputation_no/splits_all/tasks_binary/datasets_small/website_leaderboard.csv +0 -62
- data/imputation_no/splits_all/tasks_binary/datasets_small/winrate_matrix.png.zip +0 -3
- data/imputation_no/splits_all/tasks_classification/datasets_all/n_datasets_38 +0 -0
- data/imputation_no/splits_all/tasks_classification/datasets_all/pareto_front_improvability_vs_time_infer.png.zip +0 -3
- data/imputation_no/splits_all/tasks_classification/datasets_all/pareto_n_configs_imp.png.zip +0 -3
- data/imputation_no/splits_all/tasks_classification/datasets_all/tuning-impact-elo.png.zip +0 -3
- data/imputation_no/splits_all/tasks_classification/datasets_all/website_leaderboard.csv +0 -62
.gitignore
CHANGED
|
@@ -8,5 +8,3 @@ wheels/
|
|
| 8 |
|
| 9 |
# Virtual environments
|
| 10 |
.venv
|
| 11 |
-
|
| 12 |
-
*.png
|
|
|
|
| 8 |
|
| 9 |
# Virtual environments
|
| 10 |
.venv
|
|
|
|
|
|
README.md
CHANGED
|
@@ -8,7 +8,7 @@ app_file: main.py
|
|
| 8 |
pinned: true
|
| 9 |
license: apache-2.0
|
| 10 |
short_description: 'TabArena'
|
| 11 |
-
sdk_version: 5.
|
| 12 |
---
|
| 13 |
|
| 14 |
# TabArena Leaderboard Code
|
|
@@ -18,19 +18,11 @@ The leaderboard is hosted on a HuggingFace space.
|
|
| 18 |
|
| 19 |
Reference:
|
| 20 |
* Website: https://tabarena.ai
|
| 21 |
-
* Paper:
|
| 22 |
* TabArena Codebase: https://tabarena.ai/code
|
| 23 |
|
| 24 |
# Install LB Code for Development
|
| 25 |
|
| 26 |
```bash
|
| 27 |
pip install -e ".[dev]"
|
| 28 |
-
|
| 29 |
-
uv pip install -r pyproject.toml
|
| 30 |
-
```
|
| 31 |
-
|
| 32 |
-
# Current Steps to get results:
|
| 33 |
-
1. Run https://github.com/autogluon/tabarena/blob/main/examples/plots/run_generate_website_artifacts.py
|
| 34 |
-
2. Delete the current `data` folder contents in this repo to remove old results.
|
| 35 |
-
3. Unzip the generated `clean_website_artifacts.zip` zip file into the `data` folder in this repo.
|
| 36 |
-
4. Test the LB locally, otherwise you are done :)
|
|
|
|
| 8 |
pinned: true
|
| 9 |
license: apache-2.0
|
| 10 |
short_description: 'TabArena'
|
| 11 |
+
sdk_version: 5.33.2
|
| 12 |
---
|
| 13 |
|
| 14 |
# TabArena Leaderboard Code
|
|
|
|
| 18 |
|
| 19 |
Reference:
|
| 20 |
* Website: https://tabarena.ai
|
| 21 |
+
* Paper: TBA
|
| 22 |
* TabArena Codebase: https://tabarena.ai/code
|
| 23 |
|
| 24 |
# Install LB Code for Development
|
| 25 |
|
| 26 |
```bash
|
| 27 |
pip install -e ".[dev]"
|
| 28 |
+
```
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
data/full-imputed-cls/leaderboard.tex
ADDED
|
@@ -0,0 +1,53 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
\begin{tabular}{llcccccrr}
|
| 2 |
+
\toprule
|
| 3 |
+
\textbf{Model} & \textbf{Elo ($\uparrow$)} & \textbf{Norm.} & \textbf{Avg.} & \textbf{Harm.} & \textbf{\#wins ($\uparrow$)} & \textbf{Improva-} & \textbf{Train time} & \textbf{Predict time} \\
|
| 4 |
+
& & \textbf{score ($\uparrow$)} & \textbf{rank ($\downarrow$)} & \textbf{mean} & & \textbf{bility ($\downarrow$)} & \textbf{per 1K [s]} & \textbf{per 1K [s]} \\
|
| 5 |
+
& & & & \textbf{rank ($\downarrow$)} & & & & \\
|
| 6 |
+
\midrule
|
| 7 |
+
TabM (T+E) & \textcolor{gold}{\textbf{1574${}_{-28,+28}$}} & \textcolor{bronze}{\textbf{0.505}} & \textcolor{gold}{\textbf{9.6}} & 4.5 & 3 & \textcolor{bronze}{\textbf{8.2\%}} & 2466.21 & 1.50 \\
|
| 8 |
+
AutoGluon 1.3 (4h) & \textcolor{silver}{\textbf{1573${}_{-34,+30}$}} & \textcolor{gold}{\textbf{0.577}} & \textcolor{silver}{\textbf{9.7}} & \textcolor{silver}{\textbf{3.3}} & \textcolor{silver}{\textbf{6}} & \textcolor{gold}{\textbf{6.9\%}} & 1322.72 & 2.36 \\
|
| 9 |
+
RealMLP (T+E) & \textcolor{bronze}{\textbf{1552${}_{-30,+32}$}} & 0.470 & \textcolor{bronze}{\textbf{10.5}} & 7.4 & 0 & \textcolor{bronze}{\textbf{8.2\%}} & 6519.69 & 10.84 \\
|
| 10 |
+
LightGBM (T+E) & 1537${}_{-30,+26}$ & 0.421 & 11.1 & 6.4 & 1 & 9.8\% & 382.05 & 1.49 \\
|
| 11 |
+
TabICL (D) & 1530${}_{-31,+23}$ & \textcolor{silver}{\textbf{0.512}} & 11.4 & \textcolor{bronze}{\textbf{3.5}} & \textcolor{silver}{\textbf{6}} & \textcolor{silver}{\textbf{7.9\%}} & 8.68 & 1.74 \\
|
| 12 |
+
TabM (T) & 1487${}_{-30,+29}$ & 0.420 & 13.2 & 6.4 & 1 & 9.3\% & 2466.21 & 0.18 \\
|
| 13 |
+
CatBoost (T+E) & 1479${}_{-26,+24}$ & 0.395 & 13.6 & 8.7 & 0 & 9.1\% & 1372.94 & 0.56 \\
|
| 14 |
+
CatBoost (T) & 1469${}_{-27,+25}$ & 0.379 & 13.9 & 7.0 & 1 & 9.3\% & 1372.94 & 0.07 \\
|
| 15 |
+
LightGBM (T) & 1463${}_{-29,+30}$ & 0.336 & 14.3 & 12.1 & 0 & 10.6\% & 382.05 & 0.25 \\
|
| 16 |
+
CatBoost (D) & 1454${}_{-28,+23}$ & 0.362 & 14.7 & 7.1 & 1 & 10.3\% & 5.72 & 0.08 \\
|
| 17 |
+
TabPFNv2 (T+E) & 1454${}_{-31,+25}$ & 0.497 & 14.7 & \textcolor{gold}{\textbf{3.1}} & \textcolor{gold}{\textbf{8}} & 9.7\% & 3008.22 & 20.85 \\
|
| 18 |
+
XGBoost (T+E) & 1452${}_{-28,+31}$ & 0.332 & 14.8 & 9.3 & 0 & 10.7\% & 685.87 & 1.45 \\
|
| 19 |
+
ModernNCA (T) & 1416${}_{-34,+24}$ & 0.289 & 16.5 & 9.7 & 1 & 10.5\% & 4879.89 & 0.52 \\
|
| 20 |
+
XGBoost (T) & 1412${}_{-26,+28}$ & 0.279 & 16.6 & 13.3 & 0 & 11.1\% & 685.87 & 0.21 \\
|
| 21 |
+
ModernNCA (T+E) & 1410${}_{-28,+27}$ & 0.382 & 16.7 & 7.5 & 0 & 10.4\% & 4879.89 & 8.74 \\
|
| 22 |
+
TabPFNv2 (T) & 1388${}_{-30,+33}$ & 0.385 & 17.8 & 5.2 & 1 & 12.1\% & 3008.22 & 0.51 \\
|
| 23 |
+
TabM (D) & 1375${}_{-26,+30}$ & 0.280 & 18.3 & 11.8 & 0 & 12.6\% & 10.21 & 0.14 \\
|
| 24 |
+
TabPFNv2 (D) & 1366${}_{-31,+25}$ & 0.354 & 18.8 & 4.8 & 4 & 13.0\% & 3.37 & 0.32 \\
|
| 25 |
+
TorchMLP (T+E) & 1364${}_{-28,+29}$ & 0.233 & 19.1 & 14.8 & 0 & 11.6\% & 2389.22 & 2.16 \\
|
| 26 |
+
RealMLP (T) & 1357${}_{-27,+29}$ & 0.188 & 19.4 & 16.7 & 0 & 12.0\% & 6519.69 & 0.53 \\
|
| 27 |
+
EBM (T+E) & 1356${}_{-28,+26}$ & 0.188 & 19.3 & 13.5 & 0 & 14.9\% & 914.23 & 0.22 \\
|
| 28 |
+
FastaiMLP (T+E) & 1323${}_{-29,+23}$ & 0.203 & 21.0 & 12.4 & 0 & 14.6\% & 618.90 & 4.77 \\
|
| 29 |
+
ModernNCA (D) & 1306${}_{-28,+26}$ & 0.140 & 21.8 & 12.3 & 1 & 14.7\% & 14.78 & 0.35 \\
|
| 30 |
+
EBM (T) & 1295${}_{-30,+26}$ & 0.130 & 22.4 & 17.9 & 0 & 15.6\% & 914.23 & 0.03 \\
|
| 31 |
+
EBM (D) & 1265${}_{-36,+29}$ & 0.143 & 23.9 & 11.4 & 1 & 16.6\% & 4.31 & 0.05 \\
|
| 32 |
+
RealMLP (D) & 1253${}_{-27,+23}$ & 0.088 & 24.5 & 21.0 & 0 & 14.6\% & 21.83 & 0.90 \\
|
| 33 |
+
XGBoost (D) & 1253${}_{-31,+29}$ & 0.116 & 24.5 & 19.3 & 0 & 14.1\% & 1.77 & 0.12 \\
|
| 34 |
+
ExtraTrees (T+E) & 1252${}_{-29,+24}$ & 0.110 & 24.6 & 16.8 & 0 & 15.8\% & 189.76 & 0.74 \\
|
| 35 |
+
TabDPT (D) & 1239${}_{-35,+28}$ & 0.180 & 25.1 & 8.1 & 2 & 15.5\% & 22.61 & 8.55 \\
|
| 36 |
+
TorchMLP (T) & 1237${}_{-26,+26}$ & 0.106 & 25.1 & 21.7 & 0 & 14.1\% & 2389.22 & 0.15 \\
|
| 37 |
+
FastaiMLP (T) & 1220${}_{-29,+28}$ & 0.093 & 26.1 & 20.7 & 0 & 16.6\% & 618.90 & 0.30 \\
|
| 38 |
+
RandomForest (T+E) & 1214${}_{-24,+25}$ & 0.105 & 26.3 & 14.4 & 0 & 16.6\% & 323.74 & 0.74 \\
|
| 39 |
+
LightGBM (D) & 1198${}_{-28,+29}$ & 0.087 & 27.0 & 23.6 & 0 & 15.4\% & 1.79 & 0.12 \\
|
| 40 |
+
ExtraTrees (T) & 1196${}_{-33,+24}$ & 0.080 & 27.1 & 17.2 & 0 & 17.2\% & 189.76 & 0.08 \\
|
| 41 |
+
RandomForest (T) & 1159${}_{-33,+33}$ & 0.078 & 28.8 & 16.5 & 0 & 17.8\% & 323.74 & 0.08 \\
|
| 42 |
+
TorchMLP (D) & 1082${}_{-30,+22}$ & 0.023 & 32.0 & 29.3 & 0 & 19.3\% & 6.83 & 0.15 \\
|
| 43 |
+
FastaiMLP (D) & 1053${}_{-31,+29}$ & 0.031 & 33.1 & 29.9 & 0 & 22.1\% & 2.91 & 0.37 \\
|
| 44 |
+
RandomForest (D) & 1000${}_{-0,+0}$ & 0.013 & 34.8 & 32.9 & 0 & 24.1\% & 0.38 & 0.04 \\
|
| 45 |
+
Linear (T+E) & 998${}_{-37,+26}$ & 0.042 & 34.9 & 25.5 & 0 & 29.2\% & 51.79 & 0.22 \\
|
| 46 |
+
Linear (T) & 962${}_{-28,+24}$ & 0.027 & 36.1 & 30.6 & 0 & 30.1\% & 51.79 & 0.08 \\
|
| 47 |
+
Linear (D) & 951${}_{-30,+28}$ & 0.019 & 36.3 & 27.8 & 0 & 31.1\% & 1.61 & 0.10 \\
|
| 48 |
+
ExtraTrees (D) & 915${}_{-31,+32}$ & 0.010 & 37.3 & 34.3 & 0 & 26.7\% & 0.25 & 0.04 \\
|
| 49 |
+
KNN (T+E) & 688${}_{-46,+39}$ & 0.000 & 41.7 & 41.4 & 0 & 48.5\% & 2.97 & 0.19 \\
|
| 50 |
+
KNN (T) & 605${}_{-45,+48}$ & 0.000 & 42.7 & 42.5 & 0 & 50.3\% & 2.97 & 0.04 \\
|
| 51 |
+
KNN (D) & 462${}_{-71,+70}$ & 0.000 & 43.9 & 43.7 & 0 & 58.7\% & 0.07 & 0.02 \\
|
| 52 |
+
\bottomrule
|
| 53 |
+
\end{tabular}
|
data/full-imputed-cls/tabarena_leaderboard.csv
ADDED
|
@@ -0,0 +1,46 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
method,time_train_s,time_infer_s,time_train_s_per_1K,time_infer_s_per_1K,normalized-error,normalized-error-task,imputed,champ_delta,loss_rescaled,time_train_s_rescaled,time_infer_s_rescaled,rank,median_metric_error,median_time_train_s,median_time_infer_s,median_time_train_s_per_1K,median_time_infer_s_per_1K,median_normalized-error,median_normalized-error-task,median_imputed,median_champ_delta,median_loss_rescaled,median_time_train_s_rescaled,median_time_infer_s_rescaled,median_rank,rank=1_count,rank=2_count,rank=3_count,rank>3_count,elo,elo+,elo-,winrate,mrr
|
| 2 |
+
TABM_GPU (tuned + ensemble),34341.376551290836,9.60910518720136,4223.58680397795,3.4317121842487635,0.4949870968434646,0.5299923578456494,0.0,0.08239957044412496,0.05800528924519605,45864.929770826835,134.55326674312073,9.644736842105264,0.17047,8245.070318195554,2.6588550435172187,2466.2108648716858,1.5022465694891438,0.4807674997729695,0.5356016471758118,0.0,0.0374695276183053,0.02990990162448854,39661.48674821285,121.43882105497326,9.0,3,2,2,31,1574.1,27.7,27.1,0.8035287081339713,0.22390813053432343
|
| 3 |
+
AutoGluon 1.3 (4h),7918.9967654708535,21.9962719316371,2834.6060407147975,3.0873029091200266,0.4234797507066772,0.42851773721350084,0.0,0.06891082208685434,0.03901042356832644,31011.170971980166,263.893057970457,9.710526315789474,0.159155,7086.632828738954,3.4718479580349393,1322.7174946761893,2.35500290690449,0.33567162503552783,0.4181224156574922,0.0,0.03226297629046343,0.018303111934730745,23372.022224152715,142.27612475951975,6.0,6,3,1,28,1573.4,30.0,33.4,0.8020334928229665,0.3009017525052929
|
| 4 |
+
REALMLP (tuned + ensemble),88520.97457650169,55.83305884453288,9129.519936677754,18.43403910125304,0.5303101151258073,0.5509587531175434,0.0,0.08216963958005759,0.04917872511787768,141789.21603781544,866.2933837817693,10.460526315789474,0.16568,30350.410282479395,23.161008212301468,6519.687377737515,10.838853332215592,0.4880190327087218,0.5530787180349611,0.0,0.046844798741680516,0.023927651830585747,114204.24024987879,769.6999972028425,9.25,0,1,1,36,1552.0,31.2,29.1,0.784988038277512,0.1359044319221565
|
| 5 |
+
GBM (tuned + ensemble),2957.8209862183407,11.98416593290909,759.1477152793035,2.5417865978850473,0.579228860288798,0.5992528140715053,0.0,0.09787395108553403,0.0587719228886129,8568.794143617388,184.93302208876983,11.131578947368421,0.16698000000000002,1552.8742877377404,3.4757837878333198,382.05361557599804,1.4876036641335277,0.6139242992830947,0.6320945623165324,0.0,0.050681941850522993,0.0219954442643497,7514.342598912193,103.35126359750211,10.0,1,1,2,34,1536.6,25.3,29.4,0.7697368421052632,0.15602996440606
|
| 6 |
+
TABICL_GPU (default),107.96023476381747,19.280086713506464,9.625347263659664,2.230965763361112,0.4883393558984766,0.5463468792496827,0.05263157894736842,0.0794222885606876,0.05664734932761992,168.76491644336255,230.09998232516136,11.394736842105264,0.17207,25.732762111557854,3.449363695250617,8.684246340890724,1.7433667301105085,0.49313937962268345,0.5681798960380293,0.0,0.03858333399786745,0.018532235328146802,137.81489160855577,127.65482709424812,10.0,6,4,1,27,1529.5,22.7,30.1,0.763755980861244,0.2848797584062183
|
| 7 |
+
TABM_GPU (tuned),34341.376551290836,1.0683778852050068,4223.58680397795,0.37888409513825755,0.580460857439686,0.5850736457317778,0.0,0.0927595492056652,0.07202518163358282,45864.929770826835,13.890136033418576,13.223684210526315,0.172585,8245.070318195554,0.2723346948623657,2466.2108648716858,0.17557376557934842,0.5594681994427791,0.6408535524589001,0.0,0.04800849593484757,0.03209545871697756,39661.48674821285,10.546435553124088,14.0,1,3,2,32,1487.1,28.6,29.5,0.722188995215311,0.15718600055013962
|
| 8 |
+
CAT (tuned + ensemble),16504.07579820414,2.581779239609925,3201.0265707590415,0.8915752484646539,0.6052324431308997,0.6079016429921736,0.0,0.09115569260254941,0.05171280995480023,27193.868813721343,44.08651163500922,13.552631578947368,0.16040500000000002,5046.900271190538,1.2630292971928916,1372.9411122807264,0.5562989902961428,0.6047135908900858,0.6538224316024461,0.0,0.05787378892595674,0.023358345198772973,20541.46486167592,37.76547447057217,13.0,0,1,3,34,1479.0,23.4,25.1,0.7147129186602871,0.11519163508706867
|
| 9 |
+
CAT (tuned),16504.07579820414,0.42638187025025576,3201.0265707590415,0.12024042770987867,0.6205816792872123,0.6195401164987226,0.0,0.09330164070819684,0.05138375883564522,27193.868813721343,6.229177098348047,13.881578947368421,0.161705,5046.900271190538,0.12877851062350804,1372.9411122807264,0.07385371803542701,0.6259023413364113,0.6684527369967126,0.0,0.05645008601114826,0.027822328565894415,20541.46486167592,4.9725379405984995,14.0,1,2,1,34,1469.3,24.3,26.2,0.7072368421052632,0.14306281114130998
|
| 10 |
+
GBM (tuned),2957.8209862183407,1.875632255676894,759.1477152793035,0.5049369881097493,0.664414327281209,0.6568405029860219,0.0,0.10551185264608158,0.06848969063218771,8568.794143617388,32.37091249654915,14.302631578947368,0.16877999999999999,1552.8742877377404,0.5045170254177518,382.05361557599804,0.2538713244201605,0.7037360500796572,0.6730961643801268,0.0,0.05207128139882822,0.031112468640158004,7514.342598912193,15.175591971121808,12.5,0,0,0,38,1463.4,29.5,28.3,0.6976674641148325,0.08286433502159408
|
| 11 |
+
TABPFNV2_GPU (tuned + ensemble),10002.759252123178,101.4355180454533,2653.4255131981217,44.749282859570734,0.5025912169508552,0.5552745352373752,0.3157894736842105,0.09695040281680861,0.08124534064551475,49565.00539849654,3444.4876051657507,14.671052631578947,0.17261500000000002,2077.023049354553,12.760541562239329,3008.2157047151595,20.848616639963154,0.508383993403513,0.5777919166792829,0.0,0.04061436251124223,0.031106953519191773,28624.579287895787,825.5109643363919,8.5,8,3,2,25,1453.8,24.8,30.9,0.6892942583732058,0.3228293379761294
|
| 12 |
+
CAT (default),220.2627424415789,0.2710267032099049,109.9732905896213,0.13662992734425208,0.6377213723455996,0.653981819624179,0.0,0.10281875743582664,0.05525161599592243,404.9759898295662,6.476751807794606,14.68421052631579,0.16373,18.264583627382912,0.17750852637820774,5.723546572951673,0.0761539571798428,0.6679727149434583,0.668034897235229,0.0,0.05481736123368691,0.025874497523709752,103.69107151573829,5.120402547941957,16.0,1,3,1,33,1454.1,22.2,27.3,0.6889952153110048,0.14085028107005457
|
| 13 |
+
XGB (tuned + ensemble),5957.374200563333,6.456417350253166,1167.526219278486,2.7907218103162132,0.6678696607918362,0.667447365405449,0.0,0.1067022658124541,0.0693895201734448,10832.944520166355,137.58836217438702,14.763157894736842,0.165745,1680.0658507664998,2.2809726662105985,685.86510540535,1.4547593315263065,0.7199373388598738,0.7259505715046674,0.0,0.0603611320440825,0.027448205694685396,8251.297786111467,74.47019952683146,13.0,0,1,1,36,1452.0,30.3,27.2,0.687200956937799,0.1077066830171761
|
| 14 |
+
MNCA_GPU (tuned),57356.284085895306,20.170870465214488,5990.817791505437,2.0485901061394993,0.7110105277591534,0.6595940582554153,0.0,0.1050159299928894,0.0724200510289198,80456.91211763977,129.4389431731015,16.526315789473685,0.17054999999999998,14186.536935488384,0.6020842525694106,4879.890404506269,0.5247194359730172,0.7789380151687756,0.6741481297678236,0.0,0.06755655006003625,0.04344719805270737,66956.02864547497,27.735396922747526,16.5,1,0,0,37,1415.9,23.5,33.3,0.6471291866028708,0.10295238153020833
|
| 15 |
+
XGB (tuned),5957.374200563333,1.301870340352867,1167.526219278486,0.6678561539347552,0.7214356698637235,0.7066756243519319,0.0,0.11080653156459744,0.07384263953337422,10832.944520166355,28.47499330573137,16.63157894736842,0.16848000000000002,1680.0658507664998,0.3827125522825453,685.86510540535,0.2050912539994952,0.7533869940641466,0.7531012522750911,0.0,0.0694652487234973,0.03410738014755424,8251.297786111467,11.461364611937853,15.5,0,0,0,38,1412.1,27.2,25.8,0.6447368421052632,0.07527320270823155
|
| 16 |
+
MNCA_GPU (tuned + ensemble),57356.284085895306,531.7593351699455,5990.817791505437,50.07821547982156,0.6178778787330054,0.6105886884096879,0.0,0.10389701155615068,0.07981152020070442,80456.91211763977,3301.6213661597712,16.68421052631579,0.183035,14186.536935488384,14.17156207561493,4879.890404506269,8.743516387788919,0.6004419642795275,0.5800963441226324,0.0,0.06497462948315735,0.039747452446492706,66956.02864547497,548.1975258046007,12.5,0,2,4,32,1410.4,26.7,27.6,0.6435406698564593,0.13300821802463408
|
| 17 |
+
TABPFNV2_GPU (tuned),10002.759252123178,3.410806728176206,2653.4255131981217,1.555769686886014,0.6152128339751168,0.6368249005546318,0.3157894736842105,0.12130685228554555,0.0956967324044558,49565.00539849654,114.00749994252256,17.842105263157894,0.1868,2077.023049354553,0.5060818235079447,3008.2157047151595,0.5144277113236544,0.6861393226426085,0.651413564581804,0.0,0.08561151707807774,0.042273687584978896,28624.579287895787,25.570859321617846,13.0,1,8,1,28,1387.8,32.2,29.9,0.6172248803827751,0.19384505458173865
|
| 18 |
+
TABM_GPU (default),150.0762373017289,1.2507152029645372,19.886819500646006,0.46487430096802723,0.7200913651969231,0.7318128476710409,0.0,0.1257035538283091,0.09205633408961793,189.61145131944258,14.042337078320946,18.32894736842105,0.17246,31.126562476158142,0.20260944763819377,10.213381764059356,0.1381032773929915,0.8315398727681542,0.7950654471852272,0.0,0.06154427249600947,0.03281090889681732,144.96049255349743,10.968725907290855,18.0,0,0,1,37,1375.0,29.4,25.1,0.6061602870813397,0.08448030842994195
|
| 19 |
+
TABPFNV2_GPU (default),11.400364582092442,0.8958970822786029,4.227300497658887,0.4575723059305235,0.646401200043383,0.6913643236884485,0.3157894736842105,0.13032850434642715,0.10434014574898658,53.73712299744585,29.471362460513575,18.842105263157894,0.1886,7.994279013739691,0.2908047080039978,3.368600991426515,0.3152861168047789,0.777395370386663,0.7325492470622512,0.0,0.07929605277196083,0.04313507441518306,41.34560285308973,17.75894630310718,17.0,4,1,4,29,1366.1,24.9,30.7,0.5944976076555024,0.21051957029694982
|
| 20 |
+
NN_TORCH (tuned + ensemble),24331.947126566527,16.31368946276213,3050.9102763481856,4.325404104362444,0.7674791714524483,0.751965702834341,0.0,0.11637811814114107,0.08020239923609776,56038.718047349466,227.24142067178357,19.05263157894737,0.17071999999999998,9097.789536105262,5.056680162747702,2389.2199648500327,2.157502904371376,0.8880043848397714,0.8073663594549582,0.0,0.06782479524801643,0.04484831234958845,44480.90343297912,173.17543399472862,19.5,0,0,0,38,1364.3,29.0,27.1,0.5897129186602871,0.06737424255513398
|
| 21 |
+
EBM (tuned + ensemble),36729.440274559965,1.3371900389766136,6141.104384884199,0.5036823041953499,0.811594312476448,0.8073262691422789,0.0,0.14906209202998957,0.10957472519228896,25271.27145534202,19.518642702178116,19.342105263157894,0.17122500000000002,2366.879786974854,0.4240463972091675,914.2329798556116,0.21634762578811195,0.9034396658321221,0.8509428943624828,0.0,0.0823298574360965,0.040258063765572304,15273.913117491418,11.254406005139426,19.0,0,0,1,37,1356.1,25.2,27.9,0.5831339712918661,0.07392904855395822
|
| 22 |
+
REALMLP (tuned),88520.97457650169,2.4448067613512454,9129.519936677754,0.9659323148202512,0.8116906333552832,0.7529096930561373,0.0,0.11967014747000536,0.08502935951276701,141789.21603781544,39.671058690327705,19.355263157894736,0.17194500000000001,30350.410282479395,0.9911958641476102,6519.687377737515,0.5341468926001405,0.8954815082143113,0.7906338583190218,0.0,0.076222073390521,0.04561771439992753,114204.24024987879,35.28258688191838,18.75,0,0,0,38,1356.9,28.2,26.5,0.5828349282296651,0.059857762488610834
|
| 23 |
+
FASTAI (tuned + ensemble),7309.51755415473,18.692008190266574,1376.1098802486467,8.473426897342513,0.7967114176056229,0.7935375376704039,0.0,0.14576367767903883,0.08697133573600783,18964.092381812123,455.5897128961598,21.013157894736842,0.178125,3087.37076303694,11.787012616793314,618.8953909329178,4.7655686359255345,0.9993616733332265,0.8596726175256504,0.0,0.08731644523471122,0.054465543302125864,15284.817189242676,443.8905026950689,22.75,0,1,0,37,1323.2,22.6,28.2,0.5451555023923444,0.08034444085096946
|
| 24 |
+
MNCA_GPU (default),304.22019695963775,10.484658345144394,17.60828380729721,1.3061868722894623,0.8600476423581978,0.8133618407225838,0.0,0.14684370064110588,0.09623286753523605,254.62493914649235,74.63976386946672,21.789473684210527,0.18519,31.500762327512106,0.5732622504234314,14.777266169000828,0.34634581634079226,1.0,0.893695279008777,0.0,0.09230605494417649,0.051103226670839136,209.09978531409226,23.91921150117286,23.0,1,0,0,37,1306.1,25.7,27.4,0.527511961722488,0.08099601234817519
|
| 25 |
+
EBM (tuned),36729.440274559965,0.18373215324000308,6141.104384884199,0.08212434505580339,0.8701293455239967,0.8449855604868557,0.0,0.15617886229027844,0.11711713769734972,25271.27145534202,2.5019643980329747,22.42105263157895,0.17203000000000002,2366.879786974854,0.0449512971772088,914.2329798556116,0.02528246646038782,1.0,0.8943570495572675,0.0,0.08835084225135731,0.04420635043937865,15273.913117491418,1.252412448907763,23.5,0,0,0,38,1295.1,25.2,29.9,0.5131578947368421,0.05576760659451273
|
| 26 |
+
EBM (default),119.68166406294058,0.19009479611937763,11.429209024387047,0.10185762188607354,0.8572475476046651,0.8539911315322448,0.0,0.16604276757956396,0.12380006768435853,109.6134925105603,3.4110931424674527,23.894736842105264,0.17447000000000001,9.92618230978648,0.06371633741590713,4.31382445805991,0.0475851422516083,1.0,0.9282824912353611,0.0,0.09554880640873625,0.03755874966541475,60.70178540099199,2.7834768100426253,24.0,1,0,2,35,1264.9,28.3,35.7,0.4796650717703349,0.08753724330105657
|
| 27 |
+
REALMLP (default),302.86446626785903,3.0018129969200893,26.277581916467653,2.856467141391241,0.9122374081508348,0.85760098469969,0.0,0.14572889637854225,0.10212034328423684,472.82883048215837,115.85189921010567,24.5,0.17367,103.08245442973242,2.582352219687568,21.83278809042843,0.8957992336206358,1.0,0.8896440117833417,0.0,0.11770994919600164,0.06845697585035651,372.59928806765174,52.85349774079455,25.5,0,0,0,38,1253.4,23.0,26.4,0.4659090909090909,0.04751011701630104
|
| 28 |
+
XGB (default),13.116732126927516,0.5742131232518202,3.202512268619222,0.2981186068489448,0.8843514041095932,0.8510762104833869,0.0,0.14066462097580287,0.11671766984062094,31.46292861336276,13.4756369742995,24.5,0.17317,5.653352538744608,0.30113152662913,1.771208861779989,0.11707781619763814,1.0,0.9410060237993583,0.0,0.09869044613168698,0.06054796407572259,28.26053761224749,9.127662964815336,24.0,0,0,0,38,1253.3,28.1,30.7,0.4659090909090909,0.05191659220532364
|
| 29 |
+
XT (tuned + ensemble),1317.4209560674533,2.9519454171085915,472.65138083581655,1.3657584923642982,0.889559570547466,0.8556261516301992,0.0,0.1579271337121436,0.11679174830217959,4571.615940832833,75.60244638381447,24.57894736842105,0.17667500000000003,756.8230986197789,1.8136235740449693,189.76252609436062,0.7431041876698922,1.0,0.93225990863037,0.0,0.09299076955407681,0.0685642717298475,2805.66154207989,66.39288968996527,27.5,0,0,1,37,1251.6,23.8,28.6,0.46411483253588515,0.05948664377868326
|
| 30 |
+
TABDPT_GPU (default),171.71139350780967,66.09824930987163,27.724576482795502,22.626481529214185,0.8200159004748209,0.8098229820925923,0.0,0.1548400388440821,0.1161870388203104,481.10896045076544,1338.9171702872993,25.105263157894736,0.190385,97.80311637454562,28.07416233751509,22.609050986069803,8.552450841932743,1.0,0.9583506424916117,0.0,0.1092338236572048,0.046097964205681456,400.67828468381333,1123.8959746745188,30.0,2,0,3,33,1238.6,27.7,34.6,0.45215311004784686,0.12327697073991685
|
| 31 |
+
NN_TORCH (tuned),24331.947126566527,0.8716282456241854,3050.9102763481856,0.24396545036982029,0.8935160561232315,0.8422311335793125,0.0,0.14120278296028227,0.10508427239733291,56038.718047349466,12.138164397560297,25.144736842105264,0.17446499999999998,9097.789536105262,0.29952494303385413,2389.2199648500327,0.15177921475257505,1.0,0.9030726729865997,0.0,0.10379368194908523,0.06014400941853005,44480.90343297912,9.196372470124006,25.5,0,0,0,38,1237.2,25.5,25.3,0.451255980861244,0.046101906732064955
|
| 32 |
+
FASTAI (tuned),7309.51755415473,1.0496307073977955,1376.1098802486467,0.623937267000547,0.9066743837597047,0.8621224292232853,0.0,0.16578900249006634,0.10852452399743645,18964.092381812123,32.00078545496663,26.06578947368421,0.18023499999999998,3087.37076303694,0.8054822285970051,618.8953909329178,0.2978802219128553,1.0,0.9088593404454739,0.0,0.09789639604232409,0.0649858160039401,15284.817189242676,26.536834640421894,26.0,0,0,0,38,1220.2,27.5,28.1,0.43032296650717705,0.04823155314220717
|
| 33 |
+
RF (tuned + ensemble),2309.3465478268977,2.3587986137434753,541.3031953907538,1.2662572218585502,0.8949088487058612,0.8653921072161802,0.0,0.16581458532483917,0.12803665978914544,5371.163113535875,67.79453318661523,26.31578947368421,0.177925,871.1966819789675,1.9029027620951335,323.74369638605225,0.7428875097152683,1.0,0.976443274983263,0.0,0.10359622790389916,0.07479078376008455,4278.677975908691,61.67862848692378,28.5,0,1,1,36,1213.5,24.8,23.2,0.4246411483253589,0.06923937840225046
|
| 34 |
+
GBM (default),7.9087888545460165,0.5753568479889317,2.951996847158713,0.17011749256975625,0.9125347540697407,0.8867856543564054,0.0,0.15377361970482045,0.11510537200611927,31.85523047906877,10.81395814590769,27.026315789473685,0.172975,5.532836645179325,0.2585195038053725,1.7913477923414471,0.12049981156984965,1.0,0.953002789600119,0.0,0.11068698010353623,0.0625734900521432,25.045220341015206,6.549914554959342,28.0,0,0,0,38,1197.8,28.8,27.4,0.4084928229665072,0.04242553702308852
|
| 35 |
+
XT (tuned),1317.4209560674533,0.3043035832762021,472.65138083581655,0.1722314796858175,0.9201818360376706,0.887453690669917,0.0,0.1716073337665082,0.12727956433250776,4571.615940832833,8.349379059762423,27.11842105263158,0.17796,756.8230986197789,0.18769407272338867,189.76252609436062,0.07878183958882805,1.0,0.9660073423997149,0.0,0.10587649481110034,0.07275368947647169,2805.66154207989,8.013491457037578,30.0,0,1,0,37,1195.6,24.0,32.1,0.4063995215311005,0.05828850461349728
|
| 36 |
+
RF (tuned),2309.3465478268977,0.23789871352457861,541.3031953907538,0.15647241398955916,0.9224816586070888,0.8914900089794889,0.0,0.17793267481613345,0.14008678334835084,5371.163113535875,7.241889916329925,28.763157894736842,0.178915,871.1966819789675,0.17230602105458576,323.74369638605225,0.07643497412773152,1.0,0.9949804738019044,0.0,0.11788865245982966,0.07710393358859187,4278.677975908691,6.216263662191708,31.5,0,1,1,36,1159.2,32.8,32.1,0.36901913875598086,0.060732505395188326
|
| 37 |
+
NN_TORCH (default),48.66164081361559,0.6504810580733227,11.536659167937149,0.237589986723434,0.9768277299582552,0.9516276223313249,0.0,0.19345935921063773,0.14725824727620437,155.18773864824473,10.878820110101595,31.986842105263158,0.180355,26.916632894674937,0.2675716214709811,6.83469910157457,0.14703020953097523,1.0,0.9983912483912485,0.0,0.142508632176782,0.09058242754644441,137.05069981706868,8.619821917518276,33.0,0,0,0,38,1081.6,21.3,29.3,0.29575358851674644,0.0341695156824269
|
| 38 |
+
FASTAI (default),31.12571889106293,1.103970385504048,5.0919225272079345,0.5139922583887925,0.9692956872023926,0.9418353795989344,0.0,0.2205946249171078,0.17081144202844967,74.47424680202033,27.97257070113272,33.078947368421055,0.19183499999999998,12.713354892200893,0.7982388072543674,2.9120182447539116,0.36810695156439827,1.0,1.0,0.0,0.16584224786083723,0.11423976777713259,60.1261932941261,24.301698162325565,36.0,0,0,0,38,1052.8,28.1,30.9,0.27093301435406697,0.03346355889150736
|
| 39 |
+
RF (default),3.928520040972191,0.1435067986187182,0.8021093004373405,0.07125612411976835,0.9873793306417248,0.9675347593198196,0.0,0.24146620137116087,0.23100094778843142,6.102631035159317,3.7416299644570024,34.80263157894737,0.21025,1.1966572999954224,0.08589340580834282,0.3813090053938437,0.03721195658349352,1.0,1.0,0.0,0.17586669886220047,0.11692775501298655,5.480722222590385,3.4579154094346745,36.5,0,0,0,38,1000.0,0.0,0.0,0.23175837320574164,0.03044058622007152
|
| 40 |
+
LR (tuned + ensemble),310.9206490230839,1.856038474618343,112.43523004573552,0.6244822981818133,0.9583388751756515,0.9525027008333006,0.0,0.2917619246531289,0.2639449201736458,1090.6543235756506,24.154836057241496,34.94736842105263,0.20569500000000002,172.99803659651013,0.33067578077316284,51.78500762114817,0.22385943240534312,1.0,1.0,0.0,0.23092004636837282,0.1418450851884247,696.2608974821628,13.25530093456624,38.0,0,0,1,37,998.1,25.5,37.0,0.2284688995215311,0.039205008884686546
|
| 41 |
+
LR (tuned),310.9206490230839,0.5155129476597434,112.43523004573552,0.17331679222553462,0.9727265210610188,0.9599648989348959,0.0,0.30105734113761295,0.27391129557139166,1090.6543235756506,6.903044838007482,36.05263157894737,0.20751,172.99803659651013,0.12874411212073433,51.78500762114817,0.07805190196778514,1.0,1.0,0.0,0.23702586233900474,0.14460038903245942,696.2608974821628,4.422167155622043,38.5,0,0,0,38,961.6,23.9,27.7,0.20334928229665072,0.0327143779465347
|
| 42 |
+
LR (default),7.59960079534709,0.5285763780973111,2.68894957171299,0.18897855365485863,0.9811233759976523,0.9657118387068965,0.0,0.3109312407898901,0.298004116613438,27.747857838408066,7.876763349331487,36.3421052631579,0.21230500000000002,5.359276652336121,0.13779839674631755,1.6116061178401027,0.09774404154712064,1.0,1.0,0.0,0.23702577885376547,0.1541261802297627,18.023625361427616,4.735441569338942,40.0,0,0,1,37,950.6,27.1,29.7,0.19677033492822968,0.03600631416753817
|
| 43 |
+
XT (default),2.7359117016457675,0.18087529669031066,0.7550887156747417,0.07422230753863952,0.9896376739572407,0.9751584322493536,0.0,0.26658722117284267,0.26095559827613696,5.173741811459425,4.243951698357253,37.26315789473684,0.21292,1.01790091726515,0.09051434199015299,0.24605929188859293,0.04072451222400395,1.0,1.0,0.0,0.18424442460299273,0.13780186094528035,4.456264068369281,3.7424721126192138,39.0,0,0,0,38,914.9,32.0,30.5,0.17583732057416268,0.0291505121116296
|
| 44 |
+
KNN (tuned + ensemble),167.0455302492917,11.66904854404996,12.260943976874097,0.77695539036087,1.0,0.9962934098751769,0.13157894736842105,0.48462823969745605,0.5970228537293082,72.48328913141673,77.9475913060204,41.671052631578945,0.318405,9.821025305324131,0.2367298404375712,2.969713103492417,0.18997109296167614,1.0,1.0,0.0,0.42625485653324524,0.6596342220080722,56.9019152818522,12.487074071232104,43.0,0,0,0,38,688.4,38.7,45.5,0.0756578947368421,0.024178278159192945
|
| 45 |
+
KNN (tuned),167.0455302492917,1.8054817143936601,12.260943976874097,0.13106223823409818,1.0,0.9976590626673897,0.13157894736842105,0.5033620273790689,0.6430012930336472,72.48328913141673,12.511864612142434,42.671052631578945,0.322975,9.821025305324131,0.0851174063152737,2.969713103492417,0.040417757021976156,1.0,1.0,0.0,0.4549174904387682,0.7006933550921264,56.9019152818522,2.342964973116946,44.0,0,0,0,38,605.3,48.0,44.4,0.05293062200956938,0.02351625016097909
|
| 46 |
+
KNN (default),1.7449495283483762,0.22627578220869365,0.489346568018936,0.038714559202156204,1.0,1.0,0.13157894736842105,0.5871637410556744,0.939313827722613,1.0055419249185589,2.3649007838074403,43.921052631578945,0.382765,0.27595198154449463,0.036337282922532826,0.07126887487893994,0.021006283652748647,1.0,1.0,0.0,0.5463314318406584,1.0,1.0,1.2165713596834893,45.0,0,0,0,38,462.1,69.7,70.6,0.02452153110047847,0.022868529201343697
|
data/full-imputed-reg/leaderboard.tex
ADDED
|
@@ -0,0 +1,52 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
\begin{tabular}{llcccccrr}
|
| 2 |
+
\toprule
|
| 3 |
+
\textbf{Model} & \textbf{Elo ($\uparrow$)} & \textbf{Norm.} & \textbf{Avg.} & \textbf{Harm.} & \textbf{\#wins ($\uparrow$)} & \textbf{Improva-} & \textbf{Train time} & \textbf{Predict time} \\
|
| 4 |
+
& & \textbf{score ($\uparrow$)} & \textbf{rank ($\downarrow$)} & \textbf{mean} & & \textbf{bility ($\downarrow$)} & \textbf{per 1K [s]} & \textbf{per 1K [s]} \\
|
| 5 |
+
& & & & \textbf{rank ($\downarrow$)} & & & & \\
|
| 6 |
+
\midrule
|
| 7 |
+
AutoGluon 1.3 (4h) & \textcolor{gold}{\textbf{1795${}_{-55,+67}$}} & \textcolor{gold}{\textbf{0.740}} & \textcolor{gold}{\textbf{4.6}} & \textcolor{silver}{\textbf{2.5}} & 2 & \textcolor{silver}{\textbf{2.7\%}} & 1625.74 & 6.76 \\
|
| 8 |
+
RealMLP (T+E) & \textcolor{silver}{\textbf{1766${}_{-64,+61}$}} & \textcolor{silver}{\textbf{0.732}} & \textcolor{silver}{\textbf{5.3}} & 3.4 & 0 & \textcolor{gold}{\textbf{2.0\%}} & 7141.94 & 9.66 \\
|
| 9 |
+
ModernNCA (T+E) & \textcolor{bronze}{\textbf{1632${}_{-52,+51}$}} & 0.625 & \textcolor{bronze}{\textbf{8.3}} & \textcolor{bronze}{\textbf{2.8}} & \textcolor{silver}{\textbf{3}} & 3.8\% & 3779.52 & 7.69 \\
|
| 10 |
+
TabDPT (D) & 1620${}_{-62,+53}$ & \textcolor{bronze}{\textbf{0.651}} & 8.8 & \textcolor{gold}{\textbf{2.3}} & \textcolor{gold}{\textbf{5}} & \textcolor{bronze}{\textbf{2.9\%}} & 22.53 & 8.55 \\
|
| 11 |
+
CatBoost (T+E) & 1616${}_{-51,+59}$ & 0.547 & 8.9 & 7.2 & 0 & 4.5\% & 3552.96 & 0.97 \\
|
| 12 |
+
LightGBM (T+E) & 1609${}_{-53,+58}$ & 0.542 & 9.2 & 7.1 & 0 & 5.0\% & 700.15 & 9.32 \\
|
| 13 |
+
CatBoost (T) & 1568${}_{-46,+64}$ & 0.524 & 10.3 & 6.9 & 0 & 4.6\% & 3552.96 & 0.10 \\
|
| 14 |
+
TabM (T+E) & 1562${}_{-64,+54}$ & 0.494 & 10.5 & 6.5 & 0 & 3.3\% & 4158.29 & 1.41 \\
|
| 15 |
+
XGBoost (T+E) & 1496${}_{-51,+43}$ & 0.447 & 12.8 & 12.3 & 0 & 5.5\% & 834.93 & 2.61 \\
|
| 16 |
+
LightGBM (T) & 1490${}_{-50,+55}$ & 0.445 & 12.9 & 10.8 & 0 & 5.6\% & 700.15 & 0.97 \\
|
| 17 |
+
XGBoost (T) & 1474${}_{-46,+46}$ & 0.414 & 13.7 & 13.1 & 0 & 5.6\% & 834.93 & 0.39 \\
|
| 18 |
+
ModernNCA (T) & 1446${}_{-49,+52}$ & 0.360 & 14.6 & 7.2 & 0 & 5.9\% & 3779.52 & 0.40 \\
|
| 19 |
+
TabM (T) & 1440${}_{-54,+39}$ & 0.392 & 15.0 & 11.6 & 0 & 4.3\% & 4158.29 & 0.17 \\
|
| 20 |
+
CatBoost (D) & 1439${}_{-57,+48}$ & 0.400 & 15.0 & 11.5 & 0 & 6.2\% & 10.89 & 0.09 \\
|
| 21 |
+
RealMLP (T) & 1405${}_{-53,+53}$ & 0.350 & 16.4 & 13.9 & 0 & 4.6\% & 7141.94 & 0.39 \\
|
| 22 |
+
TabPFNv2 (T+E) & 1381${}_{-47,+55}$ & 0.414 & 17.2 & 3.1 & \textcolor{silver}{\textbf{3}} & 5.1\% & 4223.87 & 27.54 \\
|
| 23 |
+
ModernNCA (D) & 1340${}_{-54,+50}$ & 0.216 & 18.8 & 14.1 & 0 & 7.4\% & 15.50 & 0.30 \\
|
| 24 |
+
TabM (D) & 1329${}_{-48,+53}$ & 0.300 & 19.2 & 15.6 & 0 & 6.0\% & 13.32 & 0.13 \\
|
| 25 |
+
TorchMLP (T+E) & 1306${}_{-57,+46}$ & 0.182 & 20.2 & 14.6 & 0 & 7.6\% & 4608.59 & 1.23 \\
|
| 26 |
+
TabPFNv2 (T) & 1300${}_{-55,+53}$ & 0.287 & 20.6 & 8.4 & 0 & 6.2\% & 4223.87 & 0.45 \\
|
| 27 |
+
RealMLP (D) & 1284${}_{-48,+47}$ & 0.152 & 21.3 & 18.4 & 0 & 7.0\% & 21.86 & 0.84 \\
|
| 28 |
+
ExtraTrees (T+E) & 1272${}_{-60,+53}$ & 0.162 & 21.5 & 13.6 & 0 & 10.0\% & 158.22 & 0.84 \\
|
| 29 |
+
LightGBM (D) & 1264${}_{-54,+53}$ & 0.091 & 21.9 & 21.3 & 0 & 8.1\% & 2.11 & 0.27 \\
|
| 30 |
+
ExtraTrees (T) & 1255${}_{-53,+42}$ & 0.136 & 22.3 & 16.9 & 0 & 10.3\% & 158.22 & 0.15 \\
|
| 31 |
+
TabPFNv2 (D) & 1231${}_{-61,+59}$ & 0.238 & 23.3 & 11.5 & 0 & 7.6\% & 2.80 & 0.31 \\
|
| 32 |
+
TorchMLP (T) & 1230${}_{-47,+46}$ & 0.131 & 23.3 & 20.1 & 0 & 8.4\% & 4608.59 & 0.10 \\
|
| 33 |
+
XGBoost (D) & 1215${}_{-46,+42}$ & 0.114 & 24.1 & 21.5 & 0 & 8.8\% & 2.24 & 0.24 \\
|
| 34 |
+
RandomForest (T+E) & 1203${}_{-48,+46}$ & 0.076 & 24.5 & 22.4 & 0 & 10.9\% & 515.73 & 0.77 \\
|
| 35 |
+
RandomForest (T) & 1153${}_{-58,+49}$ & 0.055 & 26.2 & 24.5 & 0 & 11.4\% & 515.73 & 0.12 \\
|
| 36 |
+
EBM (T+E) & 1136${}_{-61,+46}$ & 0.171 & 27.0 & 13.4 & 0 & 13.7\% & 1890.68 & 0.13 \\
|
| 37 |
+
ExtraTrees (D) & 1100${}_{-63,+56}$ & 0.069 & 28.1 & 25.0 & 0 & 12.2\% & 0.47 & 0.06 \\
|
| 38 |
+
EBM (T) & 1092${}_{-57,+50}$ & 0.150 & 28.5 & 17.0 & 0 & 14.2\% & 1890.68 & 0.01 \\
|
| 39 |
+
FastaiMLP (T+E) & 1040${}_{-50,+62}$ & 0.024 & 30.0 & 28.0 & 0 & 12.2\% & 540.06 & 2.67 \\
|
| 40 |
+
TorchMLP (D) & 1038${}_{-63,+58}$ & 0.017 & 30.2 & 28.2 & 0 & 11.9\% & 20.48 & 0.08 \\
|
| 41 |
+
EBM (D) & 1034${}_{-67,+44}$ & 0.109 & 30.6 & 27.7 & 0 & 15.1\% & 6.33 & 0.04 \\
|
| 42 |
+
RandomForest (D) & 1000${}_{-0,+0}$ & 0.000 & 31.3 & 30.7 & 0 & 12.9\% & 0.53 & 0.06 \\
|
| 43 |
+
FastaiMLP (T) & 992${}_{-65,+49}$ & 0.014 & 31.7 & 30.4 & 0 & 12.7\% & 540.06 & 0.32 \\
|
| 44 |
+
FastaiMLP (D) & 858${}_{-70,+53}$ & 0.000 & 34.9 & 34.3 & 0 & 17.1\% & 2.60 & 0.39 \\
|
| 45 |
+
KNN (T+E) & 532${}_{-96,+78}$ & 0.000 & 39.8 & 39.6 & 0 & 36.1\% & 2.43 & 0.14 \\
|
| 46 |
+
Linear (T+E) & 489${}_{-95,+53}$ & 0.000 & 40.3 & 40.2 & 0 & 35.4\% & 45.74 & 0.11 \\
|
| 47 |
+
KNN (T) & 442${}_{-135,+68}$ & 0.000 & 40.8 & 40.6 & 0 & 36.8\% & 2.43 & 0.03 \\
|
| 48 |
+
Linear (T) & 421${}_{-95,+82}$ & 0.000 & 40.9 & 40.8 & 0 & 35.6\% & 45.74 & 0.05 \\
|
| 49 |
+
Linear (D) & 290${}_{-113,+87}$ & 0.000 & 42.2 & 42.1 & 0 & 38.1\% & 1.19 & 0.09 \\
|
| 50 |
+
KNN (D) & 239${}_{-146,+87}$ & 0.000 & 42.6 & 42.4 & 0 & 40.8\% & 0.04 & 0.02 \\
|
| 51 |
+
\bottomrule
|
| 52 |
+
\end{tabular}
|
data/full-imputed-reg/tabarena_leaderboard.csv
ADDED
|
@@ -0,0 +1,45 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
method,time_train_s,time_infer_s,time_train_s_per_1K,time_infer_s_per_1K,normalized-error,normalized-error-task,imputed,champ_delta,loss_rescaled,time_train_s_rescaled,time_infer_s_rescaled,rank,median_metric_error,median_time_train_s,median_time_infer_s,median_time_train_s_per_1K,median_time_infer_s_per_1K,median_normalized-error,median_normalized-error-task,median_imputed,median_champ_delta,median_loss_rescaled,median_time_train_s_rescaled,median_time_infer_s_rescaled,median_rank,rank=1_count,rank=2_count,rank=3_count,rank>3_count,elo,elo+,elo-,winrate,mrr
|
| 2 |
+
AutoGluon 1.3 (4h),10053.09853225386,65.70725388180496,3138.716102354025,8.201870690838728,0.2604881581536672,0.2971194348020468,0.0,0.026995880876839205,0.03762065178367739,79227.35072832079,1344.7158520024777,4.615384615384615,4.16484,12938.261539538702,10.17258334159851,1625.738447135909,6.759745919437983,0.23036881663552905,0.30366624015393634,0.0,0.015883109174547605,0.016643068102528722,85166.92529682687,517.5224332581762,4.0,2,3,1,7,1795.1,67.0,54.8,0.9159212880143113,0.3928904428904429
|
| 3 |
+
REALMLP (tuned + ensemble),66543.44573790843,35.91296550086421,7325.429690273165,14.32081689268905,0.2681319612182075,0.3168526714039812,0.0,0.019775498611648384,0.029999838203468132,290863.52338623884,790.182850123545,5.3076923076923075,4.13253,35612.93265602324,25.25520912806193,7141.940159399529,9.664030993978182,0.21463277228840488,0.3314660386375113,0.0,0.015498759976369625,0.017118693739991864,262171.8658686582,808.3809230744637,3.0,0,4,3,6,1765.6,60.7,63.8,0.8998211091234347,0.2966880341880342
|
| 4 |
+
MNCA_GPU (tuned + ensemble),31759.101002019288,48.99773550746787,6166.048475961011,17.643273328787647,0.37486519261010337,0.3801843857676789,0.0,0.038093035353910405,0.03729025388197337,186080.1526868885,677.6872983460363,8.307692307692308,4.48697,16310.556293937894,10.227302259869045,3779.5248398651206,7.690422738079533,0.33564273912263787,0.35100871313945403,0.0,0.019057076597787104,0.02404483039332963,198038.2195884601,507.5335424076248,7.0,3,1,1,8,1632.5,51.0,51.8,0.8300536672629696,0.35811118705855544
|
| 5 |
+
TABDPT_GPU (default),150.02111775732448,54.38835432835114,28.728808967170515,25.042434382939593,0.34889596641909676,0.3868397999855826,0.0,0.0290119218252697,0.03451422020936026,859.9949957462201,1916.4746059233426,8.846153846153847,4.26747,137.4807067182329,29.239719518025716,22.52690614988171,8.547073882575031,0.2832974905649298,0.3551487188815256,0.0,0.024988993973347418,0.022011234994563023,904.9754515353668,1821.0928942426492,8.0,5,0,0,8,1620.5,52.9,61.7,0.817531305903399,0.44424741924741923
|
| 6 |
+
CAT (tuned + ensemble),30302.612278386874,4.803950041991014,6846.603031359736,1.23549623448159,0.4533481852287783,0.4926936982252365,0.0,0.04475906131711523,0.05586568278238831,150000.99966111325,116.73562230823504,8.923076923076923,4.20864,22090.5574801498,2.685193909539117,3552.958864906998,0.9657383741190037,0.4197753110205541,0.5095036442102125,0.0,0.0214910315460225,0.0360010253274791,164350.82166574517,80.04598787510639,9.0,0,0,1,12,1615.9,58.2,51.0,0.815742397137746,0.13981452645420256
|
| 7 |
+
GBM (tuned + ensemble),3469.6354441180188,51.50608464469258,806.792675338677,8.57889161766087,0.4578704418351117,0.4915767713552377,0.0,0.049628721858481745,0.06009436123338137,20529.34803212188,1135.938430886033,9.153846153846153,4.21165,3055.419878217909,17.84196005927192,700.1537746143185,9.321818212785747,0.41469312312474516,0.4239687696189567,0.0,0.026696516466175324,0.03428340330572247,21786.570035668443,556.5446710812895,7.0,0,0,0,13,1608.9,57.6,52.5,0.8103756708407871,0.14022351917088757
|
| 8 |
+
CAT (tuned),30302.612278386874,0.5550543483505901,6846.603031359736,0.16909210735792113,0.47626653515508005,0.5114892609915386,0.0,0.04596154469633722,0.059493084445231575,150000.99966111325,13.40589862804556,10.26923076923077,4.23064,22090.5574801498,0.3789627022213406,3552.958864906998,0.10450043094654878,0.5098664416231588,0.5042459184199164,0.0,0.026006470182449948,0.03741223775007456,164350.82166574517,10.277501137463553,10.0,0,1,1,11,1568.5,63.6,45.8,0.7844364937388193,0.14579928618390156
|
| 9 |
+
TABM_GPU (tuned + ensemble),42663.03407314721,6.401903104782104,7337.2216133773,2.4450322937588758,0.5064842509501634,0.5097062474762206,0.0,0.03285461585754071,0.07923986245243698,180683.6213219068,120.23686774247648,10.538461538461538,4.1458,20820.912871148852,3.0152715841929116,4158.291053548661,1.4096720886484275,0.41808698686448353,0.44811012371201403,0.0,0.031023728849073895,0.02730307401826167,166291.5239595113,107.26839105814781,6.0,0,0,1,12,1562.3,53.2,63.1,0.778175313059034,0.15363693425074498
|
| 10 |
+
XGB (tuned + ensemble),6384.208144447945,12.513759183476113,1409.0815132938305,4.086239880001324,0.5528907467847085,0.5842672042727117,0.0,0.05501928538394466,0.06767162618009932,26398.72664659111,353.2622484982949,12.846153846153847,4.22219,2596.930354913076,6.618133616447449,834.9300717202715,2.614265349176195,0.6061614351314211,0.632068655632026,0.0,0.03390767360912417,0.047976586381100285,26370.57574541831,164.97865636299682,13.0,0,0,0,13,1495.9,43.0,50.2,0.7245080500894454,0.0815581414563315
|
| 11 |
+
GBM (tuned),3469.6354441180188,7.385669826034806,806.792675338677,1.273710690539978,0.5553229125473071,0.5751848987218714,0.0,0.0557293790854645,0.07273694432227554,20529.34803212188,170.85098129275468,12.923076923076923,4.23482,3055.419878217909,2.690451833936903,700.1537746143185,0.9682498776019389,0.5552965190223861,0.5297118411020986,0.0,0.027972044011269404,0.03922298758674819,21786.570035668443,130.77892053638826,12.0,0,0,0,13,1490.1,54.7,49.3,0.7227191413237924,0.09231764967059085
|
| 12 |
+
XGB (tuned),6384.208144447945,2.5463058025409016,1409.0815132938305,0.7958717570786976,0.5861202672596529,0.6108958009331424,0.0,0.056497224579359025,0.07175664097406265,26398.72664659111,74.35901041305158,13.73076923076923,4.23559,2596.930354913076,1.7797584003872342,834.9300717202715,0.3883258596259137,0.6620816436761664,0.6284024475165967,0.0,0.03317270939304562,0.04927060675426248,26370.57574541831,37.05707269936294,12.0,0,0,0,13,1474.5,45.4,45.1,0.7039355992844365,0.07642316833493304
|
| 13 |
+
MNCA_GPU (tuned),31759.101002019288,2.180037997319148,6166.048475961011,0.7699041848491119,0.640014197538291,0.6006804484966409,0.0,0.05893528628182787,0.07225555979752735,186080.1526868885,32.66971509741366,14.615384615384615,4.74895,16310.556293937894,0.48738079600863987,3779.5248398651206,0.3958729871859153,0.6568741279318248,0.634131414678689,0.0,0.027829209062692706,0.05013010527584701,198038.2195884601,22.639832589759934,16.0,0,1,2,10,1446.4,51.4,48.7,0.6833631484794276,0.13858206746510898
|
| 14 |
+
TABM_GPU (tuned),42663.03407314721,0.6790118826760185,7337.2216133773,0.2826346699151617,0.6078346969283082,0.6089177873032084,0.0,0.04291459744083105,0.1000977395685804,180683.6213219068,13.142332198971314,15.0,4.26932,20820.912871148852,0.3543446593814426,4158.291053548661,0.16724776809008424,0.5483643325463751,0.5720139858507244,0.0,0.039608740417227284,0.035365730806074164,166291.5239595113,12.524825096296247,13.0,0,0,0,13,1440.3,38.9,53.5,0.6744186046511628,0.08607273216026531
|
| 15 |
+
CAT (default),101.69209800418625,0.27084102752881173,23.09400681118272,0.13269262448628016,0.5995482740998302,0.6304472090224245,0.0,0.06246023871361243,0.07708318676535167,496.3567653671193,9.26273127760114,15.038461538461538,4.21395,62.699359814325966,0.2736650307973226,10.889876924735699,0.09199146861016888,0.5823029883647647,0.640700383892448,0.0,0.030438989353365198,0.03712451320809096,412.72250349454265,8.72239577968297,13.0,0,0,0,13,1438.7,47.7,56.4,0.6735241502683363,0.08685217342752263
|
| 16 |
+
REALMLP (tuned),66543.44573790843,1.7411092819311682,7325.429690273165,0.8126671252939139,0.6503917944927049,0.6335765314526475,0.0,0.04638057342567833,0.08360703671071543,290863.52338623884,43.982520909683025,16.384615384615383,4.34383,35612.93265602324,1.0202796989017062,7141.940159399529,0.394499832888234,0.6897823031948702,0.6773094419226776,0.0,0.042257970846457216,0.05482988743609004,262171.8658686582,39.332347707117826,17.0,0,0,0,13,1404.9,52.4,52.4,0.6422182468694096,0.07190526635498172
|
| 17 |
+
TABPFNV2_GPU (tuned + ensemble),7396.527001981857,78.03043969614893,3418.770967200307,56.25472027093109,0.5864638170450713,0.6417803537075669,0.46153846153846156,0.05066211692688948,0.09308967259297021,101895.60372403836,4530.894138806997,17.192307692307693,4.11819,3945.7989165067675,13.798940539360046,4223.8673583405725,27.542795487744606,0.7230870148989033,0.7944604017471419,0.0,0.03408450486195502,0.03467029575770307,99491.49864758716,2157.1863969834576,19.0,3,1,1,8,1380.7,54.4,46.1,0.6234347048300537,0.31937392494449274
|
| 18 |
+
MNCA_GPU (default),99.58075828980176,2.0463109099966847,18.34482305470451,0.7039566701063034,0.7844574247003018,0.7513144031861464,0.0,0.07422761878566567,0.09797180456330291,547.7971597202185,27.012858819060174,18.76923076923077,4.94033,49.107323222690155,0.41290783882141113,15.50085128260229,0.29868905742963153,0.9390070942659281,0.8904304199346637,0.0,0.03934956674595991,0.09658469357099869,536.5510136544884,15.897990578305649,18.0,0,0,0,13,1340.3,49.6,53.2,0.5867620751341681,0.07094113282642321
|
| 19 |
+
TABM_GPU (default),109.08048485323914,0.5565733836247371,30.11061011517182,0.2424694191265854,0.6999854926260701,0.720056022702389,0.0,0.060357587291533545,0.131331635834194,580.6229196238922,9.300928058329742,19.23076923076923,4.27071,65.0270922978719,0.1736939483218723,13.315938751381204,0.13061717308512005,0.6917071760987948,0.6600858527247827,0.0,0.05131334419415079,0.05222355107697133,354.8698900002396,7.016702417594013,17.0,0,0,0,13,1329.0,52.5,47.2,0.5760286225402504,0.06404901956439267
|
| 20 |
+
NN_TORCH (tuned + ensemble),44960.17932087947,4.03415755573501,5714.412959426929,1.3298489659738517,0.8184873120659005,0.8128616776400956,0.0,0.07610055505274997,0.12013496922800454,220430.49479235077,97.5953017172201,20.23076923076923,4.65351,15497.21247045199,2.8239229255252414,4608.594420268999,1.2325370779691025,0.9269135314055867,0.8896068156264655,0.0,0.05961664423383828,0.08340398612928321,177697.9133637009,96.53385472639289,21.0,0,0,0,13,1306.3,46.0,56.5,0.552772808586762,0.06851055964209914
|
| 21 |
+
TABPFNV2_GPU (tuned),7396.527001981857,3.794137261464045,3418.770967200307,1.977808999126841,0.7134155719592534,0.7418050587973032,0.46153846153846156,0.06180041784216738,0.11851504520949105,101895.60372403836,193.9605838613745,20.576923076923077,4.17059,3945.7989165067675,0.8012150287628174,4223.8673583405725,0.45159866325593434,1.0,1.0,0.0,0.05688219535982175,0.06558436878663254,99491.49864758716,19.472884837488856,28.0,0,1,1,11,1299.8,52.5,54.5,0.5447227191413238,0.11949963987283921
|
| 22 |
+
REALMLP (default),188.56736958373307,2.8279882822281275,22.387772498457988,3.1534644940370367,0.8482043138660138,0.8308100952567204,0.0,0.07049246793231062,0.12882128240536025,856.0818546821283,166.2876065138886,21.307692307692307,4.68787,113.99487728542752,2.6188460985819497,21.860990319181454,0.8382976743420607,0.985771610248003,0.9237062317944604,0.0,0.056187515565281676,0.10042346984448043,594.293460461532,57.629925208930345,22.0,0,0,0,13,1283.5,46.1,47.1,0.5277280858676208,0.054416123092593686
|
| 23 |
+
XT (tuned + ensemble),1206.5194327423715,3.367576224375994,442.36567464742467,1.069316176291616,0.8375900418733949,0.8513977817862853,0.0,0.10048529388089425,0.120775550104976,8220.916376158188,88.72398178178156,21.53846153846154,5.04794,766.4287914435068,3.576531834072537,158.22496863160976,0.8436571643025992,1.0,0.9970859984141835,0.0,0.04988373521321621,0.12269401261109883,5379.1926370125775,93.77443700336585,25.0,0,0,0,13,1272.5,52.4,59.1,0.5223613595706619,0.07374831493706267
|
| 24 |
+
GBM (default),10.696644908750159,2.245801542559241,3.3219193518156818,0.4939224454467883,0.9087857894621156,0.8834968815958729,0.0,0.08060606825955649,0.11976658053442996,80.53396018410795,58.38494586351267,21.923076923076923,4.4838,7.6057972113291425,0.9335102770063612,2.1107352135741744,0.2745991643955073,0.9712364739383097,0.9121375216694869,0.0,0.04937573951369445,0.10636020418552539,91.35379089849596,24.071571166022103,21.0,0,0,0,13,1263.8,53.0,53.2,0.5134168157423972,0.04684267482014358
|
| 25 |
+
XT (tuned),1206.5194327423715,0.45855456478575357,442.36567464742467,0.18106308268317445,0.8639666647767655,0.8723069971625098,0.0,0.10266445862489976,0.12783943156365074,8220.916376158188,14.733646953552363,22.346153846153847,5.07281,766.4287914435068,0.34996385044521755,158.22496863160976,0.15116311924152026,1.0,0.9903008544022194,0.0,0.050434912392149145,0.12806468799723258,5379.1926370125775,14.85253877204164,24.0,0,0,0,13,1254.9,41.8,52.3,0.5035778175313059,0.05933488091441373
|
| 26 |
+
TABPFNV2_GPU (default),13.930354648573786,0.6212240172247602,5.213536664770012,0.4399683707018391,0.7617874026169423,0.7938464861305419,0.46153846153846156,0.07567956002207855,0.13811994843145634,115.71517682132266,30.565681397453538,23.26923076923077,4.25916,9.373317972819011,0.4424108028411865,2.803336175829957,0.3130195506083406,1.0,1.0,0.0,0.06281203604096552,0.07601826824112161,120.93804022420875,33.21203399192692,28.5,0,0,0,13,1231.2,58.6,60.3,0.4821109123434705,0.08671869218804537
|
| 27 |
+
NN_TORCH (tuned),44960.17932087947,0.2904610134597517,5714.412959426929,0.10578639607158029,0.8688481869538994,0.8668830387716895,0.0,0.0842812846349075,0.1388441315022607,220430.49479235077,7.008617614539204,23.307692307692307,4.7148,15497.21247045199,0.18749599986606175,4608.594420268999,0.09690652350320204,1.0,0.9449347108759663,0.0,0.0730528979706524,0.0987961550603162,177697.9133637009,6.6162587674624795,23.0,0,0,0,13,1230.2,46.0,47.0,0.481216457960644,0.0496801955102473
|
| 28 |
+
XGB (default),12.109628748486186,0.9366908014330091,3.1242101566662273,0.3058622971724495,0.8862480764613536,0.8643492470670666,0.0,0.08752819812522358,0.12572826983269936,67.76669386312491,28.981192639888032,24.115384615384617,4.75458,7.79438853263855,0.5301833947499593,2.2441525977447814,0.24247013095584213,1.0,0.9587276133688395,0.0,0.053268364359151055,0.105372572651016,75.78126190262087,16.70456778360137,24.0,0,0,0,13,1214.6,41.8,46.0,0.462432915921288,0.046556052745046386
|
| 29 |
+
RF (tuned + ensemble),2056.9684221919783,3.432567118171953,500.6828145780351,1.0841887922659657,0.9244962574709313,0.9210959637362763,0.0,0.10851512257257524,0.13041932100317657,10863.776799780928,90.24173062908258,24.46153846153846,5.10626,1088.11842862765,1.444333102968004,515.7302180242054,0.7709478321252661,1.0,0.9973408733427616,0.0,0.06747831829737316,0.13942046203027075,11909.387526812274,91.13733276245972,25.0,0,0,0,13,1202.6,45.8,47.1,0.4543828264758497,0.044566348818597695
|
| 30 |
+
RF (tuned),2056.9684221919783,0.3944415463341607,500.6828145780351,0.1540690468576434,0.9446007082199963,0.9408578430892015,0.0,0.11360740147863871,0.13890562008885568,10863.776799780928,12.055960583687282,26.23076923076923,5.18858,1088.11842862765,0.3136819733513726,515.7302180242054,0.12356183047078627,1.0,1.0,0.0,0.07199386349783587,0.14203697859305436,11909.387526812274,11.406268840053338,28.0,0,0,0,13,1152.7,48.4,57.1,0.41323792486583183,0.040812401020999874
|
| 31 |
+
EBM (tuned + ensemble),26125.740811508127,1.1628193525167612,3543.4601988059676,0.44619286429284205,0.8289294744561572,0.8515101676633261,0.0,0.1367713941762849,0.19454116603903474,102743.72801916403,13.991542249573758,27.0,4.37076,6865.337549757957,0.3269915845659044,1890.6770917738593,0.13287644820933492,1.0,1.0,0.0,0.1248421766855935,0.1374516591403558,100996.51953892264,10.735277156769923,33.0,0,1,0,12,1136.0,45.1,60.4,0.3953488372093023,0.07486975066160587
|
| 32 |
+
XT (default),4.038539225015885,0.28929371426248146,0.7743502790637721,0.08386602222003156,0.9314402600373994,0.9226381137036738,0.0,0.12238733986993501,0.17918453000563647,12.035409694284047,7.5346994725559195,28.115384615384617,5.13889,2.058894846174452,0.22108591927422416,0.46766450701756074,0.055025941487738636,1.0,1.0,0.0,0.08970042626125052,0.15512507020900368,11.279879616649893,6.868103658105229,31.0,0,0,0,13,1100.4,55.5,62.8,0.3694096601073345,0.04000377617175779
|
| 33 |
+
EBM (tuned),26125.740811508127,0.09807351307991222,3543.4601988059676,0.04003457596727012,0.8498896422587312,0.8690004985226352,0.0,0.14218261386610534,0.20871356711851735,102743.72801916403,1.2806753432569877,28.53846153846154,4.42885,6865.337549757957,0.03976681497361925,1890.6770917738593,0.012767925630469206,1.0,1.0,0.0,0.1292889595673019,0.16421526338237233,100996.51953892264,1.0,34.0,0,0,1,12,1091.5,49.5,56.7,0.3595706618962433,0.05876238598299918
|
| 34 |
+
FASTAI (tuned + ensemble),4642.340904501768,8.77000401672135,1248.4159649024482,5.455805855994076,0.9763387226193126,0.966122715762154,0.0,0.12202126425715551,0.19741406156575694,32660.757986061028,336.5557366220283,30.0,5.27232,3620.151651991738,7.070411682128906,540.0550122797715,2.672383567926809,1.0,1.0,0.0,0.1022000155980195,0.1756471698456369,33252.590479353305,324.2617231497696,31.0,0,0,0,13,1040.0,61.4,49.3,0.32558139534883723,0.0357112186773915
|
| 35 |
+
NN_TORCH (default),151.43337847534409,0.22200924249795764,23.930992166336175,0.08705592871966639,0.9827422496757343,0.9678685866022871,0.0,0.1193977421999948,0.19547728530185762,831.2534637345914,6.296769612194,30.23076923076923,4.81722,65.27966655625238,0.14955372280544704,20.47535029226034,0.07966387341594139,1.0,1.0,0.0,0.14050249681884142,0.13614151881375058,546.0088939544379,6.398331846186531,32.0,0,0,0,13,1038.3,57.6,62.3,0.32021466905187834,0.03550445228984282
|
| 36 |
+
EBM (default),61.24195487458481,0.11774356426336827,10.057660454061297,0.07098698960857609,0.8908598695337496,0.8984005515464777,0.0,0.1505943663504528,0.22072991542366915,260.56038663650384,2.782386904835081,30.615384615384617,4.44221,19.539602756500244,0.04366175333658854,6.327684720357259,0.039228283502797535,1.0,1.0,0.0,0.13598656576261503,0.17428198310764084,214.4635432482436,1.7877726532004268,35.0,0,0,0,13,1034.2,43.3,66.2,0.3112701252236136,0.036159887762040425
|
| 37 |
+
RF (default),9.267833071284823,0.31600932585887426,1.1524694810969494,0.08818027662752802,1.0,0.9870278022164788,0.0,0.12896216114203193,0.1901035125951938,27.793203837891944,7.952067403429296,31.346153846153847,5.26146,5.924832847383287,0.26005201869540745,0.5271703851240811,0.062155184511682476,1.0,1.0,0.0,0.08142906388988314,0.17217694118713336,20.351375021697393,6.812325017295935,31.5,0,0,0,13,1000.0,0.0,0.0,0.29427549194991054,0.03259951679594785
|
| 38 |
+
FASTAI (tuned),4642.340904501768,0.9693611326380673,1248.4159649024482,0.5658202849858516,0.9859539042371194,0.9812210227156647,0.0,0.1268703134805104,0.21643571616372462,32660.757986061028,38.043706128121705,31.692307692307693,5.18701,3620.151651991738,1.0039918687608507,540.0550122797715,0.32495714500374384,1.0,1.0,0.0,0.11787324661401322,0.18136473255372101,33252.590479353305,36.96325797937462,33.0,0,0,0,13,991.5,48.2,64.3,0.28622540250447226,0.03287029132617368
|
| 39 |
+
FASTAI (default),18.837261917856004,0.970951892168094,4.540025666114834,0.4814105255967321,1.0,0.9959412543360979,0.0,0.1713566965946914,0.27575692774796806,123.96432879696866,34.056295479841545,34.92307692307692,6.36878,15.734567880630493,1.036571078830295,2.6035035917474465,0.39168673048638547,1.0,1.0,0.0,0.11912950408368406,0.27762932440609267,138.38888737159849,36.35774230804583,36.0,0,0,0,13,858.2,52.8,69.4,0.2110912343470483,0.029119709509087932
|
| 40 |
+
KNN (tuned + ensemble),20.494465350493407,2.240403470422468,2.7608828307729607,0.3822937952790146,1.0,0.9953083923448788,0.07692307692307693,0.3613185363694843,0.6538216817153736,74.30220244876604,25.830539833366643,39.76923076923077,8.1617,12.562016169230143,0.2777775393591987,2.427842858706766,0.14225338857865483,1.0,1.0,0.0,0.41352430110915517,0.7463425706140449,61.12257192545374,15.449077396683121,41.0,0,0,0,13,532.3,77.3,95.3,0.09838998211091235,0.025256773577856426
|
| 41 |
+
LR (tuned + ensemble),260.53517347665934,0.6450919024964683,90.82426667518703,0.24319307436598073,1.0,1.0,0.0,0.35422421997100256,0.6560326250854102,2016.752248654558,10.333316070921164,40.30769230769231,8.15038,155.9093332555559,0.2263851695590549,45.737119844257606,0.10677772758861735,1.0,1.0,0.0,0.3837007856022804,0.7329121008806585,1735.7243863195374,7.455644136363483,40.0,0,0,0,13,488.7,52.5,94.2,0.08586762075134168,0.024850461752159034
|
| 42 |
+
KNN (tuned),20.494465350493407,0.2574708763350788,2.7608828307729607,0.05410989827741383,1.0,0.9988376690148505,0.07692307692307693,0.36756573770309675,0.6864360755861012,74.30220244876604,3.3744578525576268,40.76923076923077,8.25638,12.562016169230143,0.09626038869222005,2.427842858706766,0.028318042556444805,1.0,1.0,0.0,0.42023780750084094,0.7514125894823619,61.12257192545374,2.1032822899149863,42.0,0,0,0,13,441.9,67.3,134.2,0.07513416815742398,0.024635899785160127
|
| 43 |
+
LR (tuned),260.53517347665934,0.13893676680377404,90.82426667518703,0.0633906225067962,1.0,1.0,0.0,0.3558634149624985,0.6630785837896581,2016.752248654558,3.076869056672641,40.92307692307692,8.15927,155.9093332555559,0.05223793453640408,45.737119844257606,0.05009355954825878,1.0,1.0,0.0,0.38692302721983185,0.7430806596590566,1735.7243863195374,2.8910146707817423,41.0,0,0,0,13,420.6,81.6,94.5,0.07155635062611806,0.02449625775032643
|
| 44 |
+
LR (default),4.903454369968838,0.19863318235446245,2.033074367074023,0.07563726553282422,1.0,1.0,0.0,0.3808091655367071,0.7702257899846487,45.803264898726965,4.508173919929611,42.15384615384615,8.22758,5.286295996771918,0.09712121221754286,1.191311693685635,0.0859815087197251,1.0,1.0,0.0,0.38694683448707623,0.8770019722050186,34.79737790723628,4.100500531123858,42.0,0,0,0,13,290.1,86.6,112.6,0.04293381037567084,0.023757512031433232
|
| 45 |
+
KNN (default),2.0401845610039864,0.07292802802517882,0.5185761600539422,0.029646415007926124,1.0,0.999847626563869,0.07692307692307693,0.4076491058824559,0.8492569035691879,1.1215157323318734,1.57971988992164,42.61538461538461,8.55152,0.15191650390625,0.04789047771030002,0.03752343922831045,0.021942545900661376,1.0,1.0,0.0,0.4573857803481276,1.0,1.0,1.5317182357480712,44.0,0,0,0,13,238.6,87.0,145.9,0.03220035778175313,0.023578620171993768
|
data/{imputation_no/splits_all/tasks_multiclass/datasets_medium/tuning-impact-elo.png.zip β full-imputed-reg/time_plot.png.zip}
RENAMED
|
@@ -1,3 +1,3 @@
|
|
| 1 |
version https://git-lfs.github.com/spec/v1
|
| 2 |
-
oid sha256:
|
| 3 |
-
size
|
|
|
|
| 1 |
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:da71d8796986e598f595ef583a35349dd78b721b6d2ad86658f6df61f9e4e254
|
| 3 |
+
size 84316
|
data/full-imputed/leaderboard.tex
ADDED
|
@@ -0,0 +1,53 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
\begin{tabular}{llcccccrr}
|
| 2 |
+
\toprule
|
| 3 |
+
\textbf{Model} & \textbf{Elo ($\uparrow$)} & \textbf{Norm.} & \textbf{Avg.} & \textbf{Harm.} & \textbf{\#wins ($\uparrow$)} & \textbf{Improva-} & \textbf{Train time} & \textbf{Predict time} \\
|
| 4 |
+
& & \textbf{score ($\uparrow$)} & \textbf{rank ($\downarrow$)} & \textbf{mean} & & \textbf{bility ($\downarrow$)} & \textbf{per 1K [s]} & \textbf{per 1K [s]} \\
|
| 5 |
+
& & & & \textbf{rank ($\downarrow$)} & & & & \\
|
| 6 |
+
\midrule
|
| 7 |
+
AutoGluon 1.3 (4h) & \textcolor{gold}{\textbf{1590${}_{-28,+24}$}} & \textcolor{gold}{\textbf{0.618}} & \textcolor{gold}{\textbf{8.4}} & \textcolor{gold}{\textbf{3.1}} & \textcolor{silver}{\textbf{8}} & \textcolor{gold}{\textbf{5.8\%}} & 1408.78 & 3.34 \\
|
| 8 |
+
RealMLP (T+E) & \textcolor{silver}{\textbf{1565${}_{-23,+28}$}} & \textcolor{silver}{\textbf{0.537}} & \textcolor{silver}{\textbf{9.1}} & 5.7 & 0 & \textcolor{silver}{\textbf{6.6\%}} & 6564.71 & 10.26 \\
|
| 9 |
+
TabM (T+E) & \textcolor{bronze}{\textbf{1544${}_{-22,+27}$}} & \textcolor{bronze}{\textbf{0.502}} & \textcolor{bronze}{\textbf{9.9}} & 4.9 & 3 & \textcolor{bronze}{\textbf{7.0\%}} & 3285.87 & 1.47 \\
|
| 10 |
+
LightGBM (T+E) & 1526${}_{-24,+24}$ & 0.452 & 10.6 & 6.6 & 1 & 8.6\% & 416.98 & 2.64 \\
|
| 11 |
+
CatBoost (T+E) & 1484${}_{-26,+19}$ & 0.433 & 12.4 & 8.2 & 0 & 7.9\% & 1658.41 & 0.65 \\
|
| 12 |
+
CatBoost (T) & 1470${}_{-23,+18}$ & 0.416 & 13.0 & 7.0 & 1 & 8.1\% & 1658.41 & 0.08 \\
|
| 13 |
+
TabM (T) & 1453${}_{-27,+20}$ & 0.413 & 13.7 & 7.2 & 1 & 8.0\% & 3285.87 & 0.17 \\
|
| 14 |
+
LightGBM (T) & 1448${}_{-27,+23}$ & 0.363 & 14.0 & 11.7 & 0 & 9.3\% & 416.98 & 0.33 \\
|
| 15 |
+
XGBoost (T+E) & 1439${}_{-20,+21}$ & 0.361 & 14.3 & 9.9 & 0 & 9.4\% & 693.49 & 1.69 \\
|
| 16 |
+
ModernNCA (T+E) & 1434${}_{-27,+24}$ & 0.444 & 14.5 & 5.3 & 3 & 8.7\% & 4621.67 & 8.15 \\
|
| 17 |
+
CatBoost (D) & 1428${}_{-20,+27}$ & 0.372 & 14.8 & 7.9 & 1 & 9.3\% & 6.83 & 0.08 \\
|
| 18 |
+
TabPFNv2 (T+E) & 1415${}_{-26,+27}$ & 0.476 & 15.4 & \textcolor{gold}{\textbf{3.1}} & \textcolor{gold}{\textbf{11}} & 8.5\% & 3030.15 & 21.44 \\
|
| 19 |
+
XGBoost (T) & 1405${}_{-23,+22}$ & 0.313 & 15.9 & 13.2 & 0 & 9.7\% & 693.49 & 0.31 \\
|
| 20 |
+
ModernNCA (T) & 1402${}_{-21,+18}$ & 0.307 & 16.0 & 8.9 & 1 & 9.3\% & 4621.67 & 0.47 \\
|
| 21 |
+
TabICL (D) & 1390${}_{-24,+24}$ & 0.381 & 16.6 & \textcolor{bronze}{\textbf{4.5}} & 6 & 9.2\% & 6.63 & 1.48 \\
|
| 22 |
+
TabPFNv2 (T) & 1348${}_{-24,+24}$ & 0.360 & 18.6 & 5.7 & 1 & 10.6\% & 3030.15 & 0.46 \\
|
| 23 |
+
RealMLP (T) & 1348${}_{-21,+20}$ & 0.229 & 18.6 & 15.9 & 0 & 10.1\% & 6564.71 & 0.49 \\
|
| 24 |
+
TabM (D) & 1347${}_{-19,+20}$ & 0.285 & 18.6 & 12.6 & 0 & 10.9\% & 10.49 & 0.13 \\
|
| 25 |
+
TorchMLP (T+E) & 1332${}_{-21,+25}$ & 0.220 & 19.4 & 14.8 & 0 & 10.6\% & 2874.67 & 1.95 \\
|
| 26 |
+
TabPFNv2 (D) & 1321${}_{-23,+21}$ & 0.324 & 20.0 & 5.6 & 4 & 11.6\% & 3.36 & 0.31 \\
|
| 27 |
+
TabDPT (D) & 1300${}_{-28,+25}$ & 0.300 & 21.0 & 4.9 & \textcolor{bronze}{\textbf{7}} & 12.3\% & 22.53 & 8.55 \\
|
| 28 |
+
ModernNCA (D) & 1299${}_{-23,+25}$ & 0.159 & 21.0 & 12.8 & 1 & 12.8\% & 14.87 & 0.31 \\
|
| 29 |
+
EBM (T+E) & 1292${}_{-25,+24}$ & 0.184 & 21.4 & 13.5 & 0 & 14.6\% & 1331.68 & 0.20 \\
|
| 30 |
+
FastaiMLP (T+E) & 1250${}_{-19,+24}$ & 0.158 & 23.5 & 14.5 & 0 & 14.0\% & 593.24 & 4.47 \\
|
| 31 |
+
RealMLP (D) & 1244${}_{-25,+21}$ & 0.104 & 23.7 & 20.3 & 0 & 12.7\% & 21.86 & 0.84 \\
|
| 32 |
+
ExtraTrees (T+E) & 1243${}_{-21,+22}$ & 0.124 & 23.8 & 15.8 & 0 & 14.3\% & 183.02 & 0.76 \\
|
| 33 |
+
EBM (T) & 1236${}_{-21,+18}$ & 0.135 & 24.1 & 17.7 & 0 & 15.3\% & 1331.68 & 0.02 \\
|
| 34 |
+
XGBoost (D) & 1232${}_{-24,+19}$ & 0.115 & 24.4 & 19.8 & 0 & 12.7\% & 1.94 & 0.12 \\
|
| 35 |
+
TorchMLP (T) & 1223${}_{-21,+25}$ & 0.113 & 24.8 & 21.3 & 0 & 12.7\% & 2874.67 & 0.13 \\
|
| 36 |
+
EBM (D) & 1202${}_{-29,+22}$ & 0.134 & 25.7 & 13.5 & 1 & 16.2\% & 4.67 & 0.04 \\
|
| 37 |
+
LightGBM (D) & 1202${}_{-23,+23}$ & 0.088 & 25.7 & 23.0 & 0 & 13.5\% & 1.96 & 0.14 \\
|
| 38 |
+
RandomForest (T+E) & 1201${}_{-24,+21}$ & 0.098 & 25.8 & 15.9 & 0 & 15.1\% & 373.18 & 0.77 \\
|
| 39 |
+
ExtraTrees (T) & 1200${}_{-28,+18}$ & 0.094 & 25.9 & 17.1 & 0 & 15.4\% & 183.02 & 0.09 \\
|
| 40 |
+
FastaiMLP (T) & 1161${}_{-19,+24}$ & 0.073 & 27.7 & 22.6 & 0 & 15.6\% & 593.24 & 0.31 \\
|
| 41 |
+
RandomForest (T) & 1152${}_{-25,+19}$ & 0.072 & 28.1 & 18.0 & 0 & 16.2\% & 373.18 & 0.09 \\
|
| 42 |
+
TorchMLP (D) & 1066${}_{-21,+24}$ & 0.022 & 31.6 & 29.0 & 0 & 17.5\% & 9.99 & 0.13 \\
|
| 43 |
+
FastaiMLP (D) & 1011${}_{-23,+23}$ & 0.023 & 33.7 & 31.0 & 0 & 20.8\% & 2.86 & 0.37 \\
|
| 44 |
+
RandomForest (D) & 1000${}_{-0,+0}$ & 0.009 & 34.0 & 32.4 & 0 & 21.3\% & 0.43 & 0.05 \\
|
| 45 |
+
ExtraTrees (D) & 972${}_{-28,+21}$ & 0.025 & 35.0 & 31.4 & 0 & 23.0\% & 0.25 & 0.05 \\
|
| 46 |
+
Linear (T+E) & 917${}_{-21,+31}$ & 0.031 & 36.6 & 28.3 & 0 & 30.8\% & 47.49 & 0.17 \\
|
| 47 |
+
Linear (T) & 882${}_{-30,+32}$ & 0.020 & 37.5 & 32.8 & 0 & 31.5\% & 47.49 & 0.07 \\
|
| 48 |
+
Linear (D) & 862${}_{-30,+23}$ & 0.014 & 38.1 & 30.5 & 0 & 32.9\% & 1.52 & 0.09 \\
|
| 49 |
+
KNN (T+E) & 685${}_{-32,+27}$ & 0.000 & 41.4 & 41.1 & 0 & 45.3\% & 2.74 & 0.18 \\
|
| 50 |
+
KNN (T) & 608${}_{-48,+32}$ & 0.000 & 42.4 & 42.3 & 0 & 46.9\% & 2.74 & 0.04 \\
|
| 51 |
+
KNN (D) & 459${}_{-45,+42}$ & 0.000 & 43.8 & 43.6 & 0 & 54.1\% & 0.05 & 0.02 \\
|
| 52 |
+
\bottomrule
|
| 53 |
+
\end{tabular}
|
data/full-imputed/tabarena_leaderboard.csv
ADDED
|
@@ -0,0 +1,46 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
method,time_train_s,time_infer_s,time_train_s_per_1K,time_infer_s_per_1K,normalized-error,normalized-error-task,imputed,champ_delta,loss_rescaled,time_train_s_rescaled,time_infer_s_rescaled,rank,median_metric_error,median_time_train_s,median_time_infer_s,median_time_train_s_per_1K,median_time_infer_s_per_1K,median_normalized-error,median_normalized-error-task,median_imputed,median_champ_delta,median_loss_rescaled,median_time_train_s_rescaled,median_time_infer_s_rescaled,median_rank,rank=1_count,rank=2_count,rank=3_count,rank>3_count,elo,elo+,elo-,winrate,mrr
|
| 2 |
+
AutoGluon 1.3 (4h),8462.98349033711,33.138286938542635,2912.1242917208747,4.391016265244401,0.381932874173557,0.3938004850803849,0.0,0.05822662138626224,0.038656168015376684,43301.56973340033,539.3969074295995,8.411764705882353,0.2055,7367.614226023356,3.9490213659074573,1408.7828331379249,3.337414261487274,0.32575782629688715,0.34354944358388295,0.0,0.022224886416280176,0.016643068102528722,33121.993698706065,185.27028732280235,5.0,8,6,2,35,1589.7,23.3,27.4,0.8315508021390374,0.32434985005444883
|
| 3 |
+
REALMLP (tuned + ensemble),82918.85938235045,50.75538799222518,8669.653795437369,17.385570695148495,0.4634803896199485,0.48980490794993603,0.0,0.0662652507057572,0.04428998923773407,179788.549283492,846.8926595159473,9.147058823529411,0.20859,30826.14291326205,23.329282654656303,6564.713231881598,10.263652729721208,0.4233707478718706,0.4394625063276673,0.0,0.03156540515236528,0.021508501844188344,136595.8709190495,808.3809230744637,7.0,0,5,4,42,1564.6,27.5,23.0,0.8148395721925134,0.17688848740169394
|
| 4 |
+
TABM_GPU (tuned + ensemble),36462.58337058756,8.791583087761158,5017.258422060137,3.1802055454964386,0.49791774396870153,0.5234045251267629,0.0,0.06977046437303486,0.06341802359213981,80230.4785975727,130.9039885664859,9.872549019607844,0.20535,8420.993191123009,2.7141049438052707,3285.8688373170553,1.4723486146222118,0.4651893659350262,0.5081840173827434,0.0,0.03250086419089859,0.027567784172117953,47614.825535817596,111.53030159160512,8.0,3,2,3,43,1543.9,26.8,21.9,0.7983511586452763,0.20599586481497986
|
| 5 |
+
GBM (tuned + ensemble),3088.2834950947295,22.058380506500956,771.2925090199282,4.080656504886727,0.5482943614672702,0.5697587102764388,0.0,0.08557614755706973,0.059109015015710746,11617.562781863631,427.3461655076801,10.627450980392156,0.2112,1667.667911251386,3.7860276963975696,416.9832926671224,2.6387318875879693,0.5661942244497019,0.5831585230842022,0.0,0.04023364809387353,0.02358455413295459,9501.568897853978,118.5632719261967,10.0,1,1,2,47,1526.2,23.7,23.4,0.7811942959001783,0.1520008705225847
|
| 6 |
+
CAT (tuned + ensemble),20021.3498029566,3.148214934334516,4130.291158755297,0.9792413821552454,0.5665168479793786,0.5762538702782276,0.0,0.07932910031410541,0.05277138538144033,58497.64726501732,62.60491239485109,12.372549019607844,0.21079,7127.110804247856,1.3727677133348253,1658.412974283854,0.65278892715772,0.5695869531891277,0.5907800269434179,0.0,0.04606752490069621,0.02774139238223889,22349.13952253535,41.43902270876451,11.0,0,1,4,46,1484.5,18.5,25.4,0.7415329768270945,0.12146805837673025
|
| 7 |
+
CAT (tuned),20021.3498029566,0.45918073721700764,4130.291158755297,0.13269281663977184,0.5837954660770609,0.5897551898062613,0.0,0.08123455741105615,0.05345084183416724,58497.64726501732,8.058537488270943,12.96078431372549,0.21123,7127.110804247856,0.16668947537740073,1658.412974283854,0.08101450844552308,0.5845247966032667,0.6225038615712537,0.0,0.05045951030687079,0.030933776248240515,22349.13952253535,5.582269457200397,12.5,1,3,2,45,1470.5,17.3,22.3,0.7281639928698752,0.1437603439953039
|
| 8 |
+
TABM_GPU (tuned),36462.58337058756,0.9691277276976176,5017.258422060137,0.3543499279245272,0.5874385027995309,0.5895097011198668,0.0,0.08005397326560945,0.07918093169544495,80230.4785975727,13.69951936973594,13.696078431372548,0.21042,8420.993191123009,0.2791590425703261,3285.8688373170553,0.1728818165133111,0.559085035183601,0.6013449948327668,0.0,0.041022605221466724,0.035365730806074164,47614.825535817596,11.564361688735561,14.0,1,3,2,45,1452.9,20.0,26.7,0.7114527629233511,0.1390380052974814
|
| 9 |
+
GBM (tuned),3088.2834950947295,3.280151636356362,771.2925090199282,0.7008989122586311,0.6366067117608026,0.6336828329141213,0.0,0.09282220252278703,0.0695723239257395,11617.562781863631,67.66975356224859,13.950980392156863,0.21181,1667.667911251386,0.5984517203436958,416.9832926671224,0.33384935590955944,0.6296869528283883,0.6590584882044693,0.0,0.05048061255226499,0.03622173132751954,9501.568897853978,17.561755180687065,12.0,0,0,0,51,1447.5,22.7,26.1,0.7056595365418895,0.08527400346153444
|
| 10 |
+
XGB (tuned + ensemble),6066.175009788823,8.000445660682546,1229.0991373608288,3.120951906510457,0.6385613101625683,0.6434959772946812,0.0,0.09352817276204972,0.06895162562612145,14800.692905333448,192.56405868832437,14.27450980392157,0.21458,2256.9276883072325,3.0598979949951173,693.4907982506384,1.6904262577061315,0.671301017368657,0.6581535714975986,0.0,0.05552089944600691,0.03200108485206017,11769.825378159274,82.2877578838671,13.0,0,1,1,49,1438.7,20.3,19.4,0.698306595365419,0.1010413685016667
|
| 11 |
+
MNCA_GPU (tuned + ensemble),50831.51192726024,408.7024568246081,6035.4844365627405,41.810485127597225,0.5559334685448146,0.5505192494459418,0.0,0.08712344899479532,0.06897276604102788,107380.48324313454,2632.7754273052897,14.549019607843137,0.21294,14486.050127214856,13.240725604693095,4621.665633563503,8.148513113458952,0.5384753932635177,0.5315179526728905,0.0,0.035276378418488075,0.031199094919591976,89084.88026764008,537.8274695033359,11.0,3,3,5,40,1433.8,23.6,26.7,0.6920677361853832,0.1903874062097513
|
| 12 |
+
CAT (default),190.03885268302784,0.2709793741147243,87.82759080296049,0.13562630112555335,0.6279909747535407,0.646125708691512,0.0,0.0925312918791838,0.06081652619205146,428.26912869207973,7.186903437353133,14.794117647058824,0.21142,28.268621895048355,0.1854714552561442,6.827020885706225,0.08026752106160412,0.6534673037587888,0.6439696146756235,0.0,0.04582469430261593,0.029179417473348953,113.47556087857672,6.139565341592893,16.0,1,3,1,46,1428.5,26.2,19.6,0.6864973262032086,0.12705337781476866
|
| 13 |
+
TABPFNV2_GPU (tuned + ensemble),9338.42554130284,95.46951768190513,2848.5135701006398,47.682041415407696,0.5239705071709494,0.5769312699511674,0.35294117647058826,0.08515142798212334,0.08426448408310144,62904.17752069347,3721.414760799794,15.372549019607844,0.22709,2098.550690642993,13.798940539360046,3030.145221648074,21.443889733771147,0.5915448073500406,0.6069332103645363,0.0,0.03866296841083827,0.03250100481138207,29840.56658077224,861.272449686595,11.0,11,4,3,33,1414.7,26.3,25.1,0.6733511586452763,0.3218844858197347
|
| 14 |
+
XGB (tuned),6066.175009788823,1.6190793797341305,1229.0991373608288,0.7004875821871327,0.6869435084156271,0.6795624628612148,0.0,0.09696298272522294,0.07331091441041244,14800.692905333448,40.17091923504828,15.892156862745098,0.21499,2256.9276883072325,0.41774741808573407,693.4907982506384,0.3083513292273418,0.7193303230280006,0.7149553857561783,0.0,0.06100086263844462,0.03462012868860662,11769.825378159274,12.136706817415504,15.0,0,0,0,51,1404.9,21.4,22.2,0.6615418894830659,0.07556633120131231
|
| 15 |
+
MNCA_GPU (tuned),50831.51192726024,15.584971993005873,6035.4844365627405,1.7226505575752833,0.692913423977365,0.6430754651403978,0.0,0.09326988355673647,0.07237812189150603,107380.48324313454,104.77227719302421,16.03921568627451,0.21328,14486.050127214856,0.585451708899604,4621.665633563503,0.4747242314576746,0.7553848478357736,0.6366009339265687,0.0,0.06113391902932641,0.0493534926522164,89084.88026764008,26.6476237825924,16.0,1,1,2,47,1402.5,17.6,20.6,0.6581996434937611,0.1120344583371438
|
| 16 |
+
TABICL_GPU (default),82.80334805787777,14.446106202929627,7.4655941033985815,1.6847655412525513,0.6187626573361198,0.6590285685589051,0.29411764705882354,0.09205009921867735,0.09066558702327601,132.83095048510535,173.47404322746496,16.607843137254903,0.21659,20.0619904200236,1.316457470258077,6.625686376434735,1.479660415649414,0.6419318555001297,0.737430199336512,0.0,0.05183434289459943,0.0307061823959099,110.37274820175764,91.25701004379168,14.0,6,4,1,40,1389.8,23.7,23.2,0.6452762923351159,0.22043663312484155
|
| 17 |
+
REALMLP (tuned),82918.85938235045,2.265432894048088,8669.653795437369,0.9268647174900084,0.770575243056979,0.7198426366318216,0.0,0.10098849134105925,0.08466680664165582,179788.549283492,40.77005886388886,18.61764705882353,0.21508,30826.14291326205,1.0006839964124892,6564.713231881598,0.487512478277248,0.866614722551564,0.7557285234401128,0.0,0.06823335132903552,0.05320262249246514,136595.8709190495,36.173930978956705,18.0,0,0,0,51,1347.8,19.9,20.3,0.5995989304812834,0.06290892887580088
|
| 18 |
+
TABPFNV2_GPU (tuned),9338.42554130284,3.508518432739773,2848.5135701006398,1.6633483351042642,0.6402449044416614,0.6630497140643348,0.35294117647058826,0.10613854546664524,0.10151316508024909,62904.17752069347,134.3876978041907,18.61764705882353,0.22751,2098.550690642993,0.5097733656565349,3030.145221648074,0.46198440414964803,0.7603547647708915,0.7104310097848833,0.0,0.07667977096051137,0.04654872982993791,29840.56658077224,22.48627830890164,15.5,1,9,2,39,1348.2,23.9,23.8,0.5995989304812834,0.1748060544077359
|
| 19 |
+
TABM_GPU (default),139.62633961878052,1.0737770921524312,22.4928837749369,0.40818286049864,0.7149663388553331,0.7265875725216725,0.0,0.10904673882873887,0.10206768551470595,289.28104127940037,12.833742622244758,18.61764705882353,0.21188,40.332407061258955,0.18133597903781468,10.492230631952996,0.13221126395693922,0.8197432069948619,0.7908796299405498,0.0,0.05780353607735633,0.03944837820705307,186.77830789084513,10.724711035853641,17.0,0,0,1,50,1346.8,19.5,18.3,0.5995989304812834,0.07920722957248505
|
| 20 |
+
NN_TORCH (tuned + ensemble),29590.123960411,13.183612702147375,3729.842332819238,3.5618312259496654,0.7804812465107793,0.7650139226387963,0.0,0.1061112883342963,0.09038128962619164,97942.50427646744,194.194370742189,19.392156862745097,0.21479,10848.660208092795,3.946354971991645,2874.6743506773596,1.9516254299583915,0.910350140134773,0.8406524459297455,0.0,0.06624137678637454,0.057854463852261596,61441.834728019065,144.9992483814283,20.0,0,0,0,51,1332.3,24.5,20.5,0.5819964349376114,0.06762812359455618
|
| 21 |
+
TABPFNV2_GPU (default),12.045264010803372,0.825882379421741,4.478694030451919,0.4530850283232118,0.6758133693268394,0.7167075967625335,0.35294117647058826,0.11639838128335792,0.11295068368765535,69.53545044274777,29.75030650326298,20.04901960784314,0.22887,9.117408725950453,0.42119165261586505,3.3572004182270923,0.3130195506083406,0.956340758739766,0.8514943913344442,0.0,0.07592598037026033,0.052795487060598015,52.331266670190345,18.512009051191257,21.0,4,1,4,42,1320.6,20.6,22.2,0.5670677361853832,0.1788864671189465
|
| 22 |
+
TABDPT_GPU (default),166.18249968905855,63.11337411850366,27.980557312145997,23.24231264879125,0.6999265055194402,0.699617586999111,0.0,0.12276620509418873,0.09536847721359762,577.6877537613715,1486.1376930964866,20.96078431372549,0.22562,99.10453534126282,28.39870807859633,22.52690614988171,8.550738306685618,0.9903591350487536,0.8691048403402053,0.0,0.05029899916003,0.0433680924463882,528.528670151851,1255.434427440434,22.0,7,0,3,41,1300.5,24.7,27.1,0.5463458110516934,0.2050929674182998
|
| 23 |
+
MNCA_GPU (default),252.05720278889527,8.333707038145958,17.796029105655933,1.1526772129094414,0.840779547661087,0.7959093866791253,0.0,0.12833371899167995,0.09667612599337075,329.3551130182264,62.49957238602975,21.03921568627451,0.21885,36.0034454398685,0.5316168732113309,14.869495839530392,0.30768591310277943,1.0,0.8857694762766525,0.0,0.07424254289688792,0.06351531911154724,236.99495784925895,20.43419277465042,23.0,1,0,0,50,1299.1,24.9,22.8,0.5445632798573975,0.07840885625256006
|
| 24 |
+
EBM (tuned + ensemble),34026.536489860475,1.2927426090946903,5478.959788432886,0.48902813323980865,0.8160130792555895,0.8175846459536137,0.0,0.1459291690476727,0.13123283756498888,45019.152540237825,18.109773959357398,21.431372549019606,0.21615,2925.6548613442314,0.400875727335612,1331.6775166450918,0.19908260374566636,0.958308811651104,0.8761716756909942,0.0,0.08565806627599937,0.05590445639895017,17751.99098903195,11.13922354039944,21.0,0,1,1,49,1292.5,23.9,24.9,0.535650623885918,0.07406174849391879
|
| 25 |
+
FASTAI (tuned + ensemble),6629.648996400055,16.16286987151975,1343.5604508466745,7.704229376998794,0.8424987698640145,0.8369589067092069,0.0,0.13971168994404898,0.11512340310437526,22455.399300542234,425.24771894393024,23.46078431372549,0.21679,3182.895098288854,10.795994228786892,593.237788402893,4.466873745216533,1.0,0.9552259937574975,0.0,0.0932055660765102,0.07243348401502496,19851.125229101002,409.94392325616803,25.0,0,1,0,50,1249.7,24.0,19.0,0.48952762923351156,0.06883106383498735
|
| 26 |
+
REALMLP (default),273.729912211121,2.9575047363123343,25.2860618687397,2.9321723489284044,0.8959152468625471,0.8483293899440195,0.0,0.1265509832451891,0.10892646502099379,570.5207778272487,128.70805989538366,23.745098039215687,0.21516,107.08700556225247,2.6188460985819497,21.860990319181454,0.8382976743420607,1.0,0.8905283644114734,0.0,0.09824845633171009,0.07444845646092337,497.27886689729087,54.5386788723178,24.0,0,0,0,51,1244.4,20.7,24.5,0.483065953654189,0.049206352618688116
|
| 27 |
+
XT (tuned + ensemble),1289.1519403179227,3.0578905248434194,464.9314949446578,1.2901947647379284,0.8763124357874087,0.8533692619097849,0.0,0.14328509610809964,0.11780722719308848,5501.829777288316,78.94715148525705,23.823529411764707,0.21814,763.5855970117781,1.8364481396145291,183.01944048073585,0.761281055543471,1.0,0.9521925032545069,0.0,0.08603990026311015,0.07137952858629182,3537.1622158448404,68.20231667792302,27.0,0,0,1,50,1243.2,21.4,20.4,0.48128342245989303,0.06310725111805937
|
| 28 |
+
EBM (tuned),34026.536489860475,0.16189759790507796,5478.959788432886,0.07139558038617727,0.8649702054759879,0.8501629933938847,0.0,0.15261119112333235,0.14046524715764735,45019.152540237825,2.1906554232861546,24.11764705882353,0.21772,2925.6548613442314,0.0440410852432251,1331.6775166450918,0.022730636596679687,1.0,0.9362089365721089,0.0,0.09714090003058773,0.06554952824985644,17751.99098903195,1.2223545537568268,24.0,0,0,1,50,1236.3,17.4,20.5,0.47459893048128343,0.056429653896796474
|
| 29 |
+
XGB (default),12.860019501050314,0.6666093941607507,3.1825529067488505,0.3000924886961127,0.8848348696110224,0.853007192078055,0.0,0.12712004260212578,0.11901448944644484,40.716829559380564,17.42803351650834,24.441176470588236,0.21667,5.82192047437032,0.3283502260843913,1.9409389396340444,0.12262328807300621,1.0,0.9508124068469325,0.0,0.09769803191981652,0.0669685923578734,34.48133232859512,10.22021081972956,24.0,0,0,0,51,1232.4,18.8,23.5,0.4672459893048128,0.05049212098034601
|
| 30 |
+
NN_TORCH (tuned),29590.123960411,0.7234875786018787,3729.842332819238,0.20874333848987678,0.8872281679035978,0.8461065690351314,0.0,0.1266933814263632,0.11368972667898117,97942.50427646744,10.830632864633353,24.754901960784313,0.21784,10848.660208092795,0.2631075382232666,2874.6743506773596,0.131112832826372,1.0,0.9115838155711167,0.0,0.1030878662833985,0.06896829387646289,61441.834728019065,8.236695976463633,25.0,0,0,0,51,1223.2,25.0,20.4,0.46011586452762926,0.04694203272118342
|
| 31 |
+
EBM (default),104.78526760316363,0.17165232525412033,11.079598604500092,0.09398863718788793,0.8658153943709024,0.8643480238945523,0.0,0.1621049398152807,0.1485076759316338,148.0901517975655,3.2508346897376326,25.745098039215687,0.21677,11.465454594294231,0.05977429548899333,4.6738599788414605,0.03961948198354664,1.0,0.9628721496233147,0.0,0.10271933543623024,0.06419214710978799,75.98904610132907,2.400460311914132,26.0,1,0,2,48,1202.5,21.1,28.4,0.4376114081996435,0.07434503258700696
|
| 32 |
+
GBM (default),8.619418829147072,1.0011564760166576,3.0462908189340183,0.25265600996978405,0.9115791356403461,0.8832381096058513,0.0,0.13512306737563554,0.11629352319961023,44.26353412937288,22.939896191571705,25.745098039215687,0.21706,5.971681065029568,0.2858244842953152,1.9600326879612946,0.14173548842042513,1.0,0.9463131041198098,0.0,0.09851412306889507,0.07460814115278391,32.72231234173676,8.612355874421967,26.0,0,0,0,51,1202.3,22.3,22.7,0.4376114081996435,0.04352354270813194
|
| 33 |
+
RF (tuned + ensemble),2245.014868743487,2.632504310950734,530.9489806737863,1.2198476221584795,0.9024507372146047,0.8782930186177115,0.0,0.15120883991739936,0.12864400480448673,6771.2411119904955,73.51636782881279,25.84313725490196,0.21966,886.9249708387587,1.8479143513573542,373.17861356387994,0.7709478321252661,1.0,0.9768978814786328,0.0,0.0916564450840821,0.08516072052322858,5833.628398157948,63.708677480395814,28.0,0,1,1,49,1200.7,20.1,23.2,0.4353832442067736,0.06295017478288799
|
| 34 |
+
XT (tuned),1289.1519403179227,0.3436224609158917,464.9314949446578,0.1744826726067124,0.9058524786574399,0.8823009390085037,0.0,0.15403365971080407,0.127422275587505,5501.829777288316,9.97674146406182,25.92156862745098,0.21915,763.5855970117781,0.19129647148980033,183.01944048073585,0.09120693679998855,1.0,0.970701256454361,0.0,0.10343347628405763,0.07994728986866971,3537.1622158448404,8.867381424714122,29.0,0,1,0,50,1199.9,17.5,28.0,0.4336007130124777,0.05853966620405146
|
| 35 |
+
FASTAI (tuned),6629.648996400055,1.0291698354002177,1343.5604508466745,0.6091231343301344,0.9268828889794378,0.8920727938696232,0.0,0.1558685523503756,0.1360312984711962,22455.399300542234,33.54113778341792,27.65686274509804,0.2173,3182.895098288854,0.8112125396728516,593.237788402893,0.306391541190021,1.0,0.9746050760858544,0.0,0.0988046136347035,0.08330206169020851,19851.125229101002,31.29559841059079,28.0,0,0,0,51,1160.6,23.5,18.9,0.39416221033868093,0.04418776652877106
|
| 36 |
+
RF (tuned),2245.014868743487,0.2778017885544721,530.9489806737863,0.15585979099514927,0.9281198477241044,0.9030193429154391,0.0,0.1615360365144191,0.13978570251749914,6771.2411119904955,8.469005968793565,28.137254901960784,0.22016,886.9249708387587,0.17402595943874782,373.17861356387994,0.08526202343425164,1.0,0.9971929103870871,0.0,0.10456209496862212,0.09283342230804408,5833.628398157948,7.94881742200239,31.0,0,1,1,49,1151.9,18.9,24.1,0.38324420677361853,0.055631086241924914
|
| 37 |
+
NN_TORCH (default),74.85835825680128,0.5412627521423472,14.695998951842784,0.19921856017345402,0.978335352631338,0.9550533922310487,0.0,0.17458090781576796,0.15954937461607677,327.51821759182326,9.71084645377221,31.637254901960784,0.22759,34.192402362823486,0.22645958264668783,9.990997226772679,0.1258046787872722,1.0,1.0,0.0,0.14050249681884142,0.10372091439545034,204.15320945265722,7.844934898940183,33.0,0,0,0,51,1065.7,23.7,20.8,0.303698752228164,0.03443490177911753
|
| 38 |
+
FASTAI (default),27.993367113578813,1.070063710732138,4.951242935164595,0.5056871108143457,0.9771222767390375,0.9555114040073619,0.0,0.2080437804427664,0.19756225211381712,87.08936574190912,29.523324076097715,33.745098039215684,0.24127,12.973222759034899,0.8480017715030246,2.8561126039251374,0.37317813888351864,1.0,1.0,0.0,0.15801035292114685,0.14150143443329027,77.36570184770021,28.70383139894662,36.0,0,0,0,51,1010.9,22.5,22.1,0.2557932263814617,0.03221137350683143
|
| 39 |
+
RF (default),5.289521401640116,0.18747803105248345,0.8914167974682212,0.07557012377860906,0.9905963640075596,0.9728548321405758,0.0,0.2127887009205986,0.22057611136662575,11.63160057311077,4.814878723410724,34.049019607843135,0.2484,1.2628253036075168,0.08775801128811306,0.43425701731008426,0.05354175385774865,1.0,1.0,0.0,0.15453679020731148,0.13517433236964935,7.010079512208331,3.9076874559468244,35.5,0,0,0,51,1000.0,0.0,0.0,0.24888591800356505,0.03085450482928122
|
| 40 |
+
XT (default),3.067954011524425,0.20851136391458946,0.7599985259503769,0.07668050932017083,0.9748030390364969,0.9607261977937244,0.0,0.22983038868386624,0.2401119926385584,6.922794409042171,5.082769758447109,34.990196078431374,0.24428,1.029017792807685,0.0932659043206109,0.2473449527431567,0.04980480471851431,1.0,1.0,0.0,0.17836788960429084,0.15384123117868195,5.707021421143385,4.456198846093199,38.0,0,0,0,51,972.1,20.7,27.8,0.22749554367201427,0.03187662324031601
|
| 41 |
+
LR (tuned + ensemble),298.0772925112502,1.5473658189794337,106.92655310814473,0.5272909274052285,0.968958377581858,0.9646098555228514,0.0,0.3076836862047438,0.36388884495507595,1326.7185005565484,20.631703511708864,36.568627450980394,0.25172,171.24826147821216,0.28876688745286727,47.49214683366935,0.1676693138737952,1.0,1.0,0.0,0.24584842785173022,0.2585604322609332,1064.131515124883,12.175213508812917,40.0,0,0,1,50,917.3,30.7,20.1,0.19162210338680927,0.03539216425693013
|
| 42 |
+
LR (tuned),298.0772925112502,0.41952294077488855,106.92655310814473,0.14529639602271893,0.9796785843199748,0.970169924696589,0.0,0.31502751681846614,0.3731108004113419,1326.7185005565484,5.927745129039777,37.549019607843135,0.25255,171.24826147821216,0.10993631680806477,47.49214683366935,0.0665447885938415,1.0,1.0,0.0,0.2610498572359117,0.2575122541077485,1064.131515124883,4.067640041721783,41.0,0,0,0,51,881.6,31.6,29.4,0.16934046345811052,0.03046989425942536
|
| 43 |
+
LR (default),6.912347784956555,0.4444732105550163,2.521765696020704,0.1600876370747322,0.9859350644688389,0.9744519582521974,0.0,0.3287432608233925,0.4183743470806094,32.35021650084229,7.018103298699637,38.07843137254902,0.2546,5.298751910527547,0.12218634287516277,1.5162047581506444,0.0887949061495271,1.0,1.0,0.0,0.26947668808304803,0.29385559917332205,22.991652687961256,4.725399415979996,41.5,0,0,1,50,862.4,22.1,29.5,0.1573083778966132,0.03274333914572857
|
| 44 |
+
KNN (tuned + ensemble),129.6893764515588,9.265668427242952,9.839359763162042,0.6763553759282402,1.0,0.9960130371949129,0.11764705882352941,0.45319635453542406,0.6115009863532073,72.94693370250576,64.66285269534394,41.431372549019606,0.34658,10.182021194034153,0.27287014325459796,2.7415773839237785,0.17676389939136214,1.0,1.0,0.0,0.41352430110915517,0.6662116649574654,57.25701094840839,13.90517184260326,43.0,0,0,0,51,685.3,26.5,31.2,0.08110516934046345,0.02430200505969366
|
| 45 |
+
KNN (tuned),129.6893764515588,1.4108907164571591,9.839359763162042,0.11144693589219826,1.0,0.9979197340731149,0.11764705882352941,0.4687472868734289,0.6540729042725081,72.94693370250576,10.182721712640424,42.431372549019606,0.35012,10.182021194034153,0.08784447775946723,2.7415773839237785,0.03624739765555662,1.0,1.0,0.0,0.4456786012073394,0.740394718900499,57.25701094840839,2.2791618782574123,44.0,0,0,0,51,607.8,31.1,47.9,0.05837789661319073,0.023658093377554222
|
| 46 |
+
KNN (default),1.8202055170645122,0.18718713898544476,0.4967972483415847,0.03640307146637206,1.0,0.9999597757876026,0.11764705882352941,0.5414051085605402,0.9163581411737007,1.035103875827835,2.1647566343855695,43.833333333333336,0.40859,0.22567404641045463,0.036888705359564886,0.050692503527237594,0.021942545900661376,1.0,1.0,0.0,0.5109534123207524,1.0,1.0,1.2564234767690339,45.0,0,0,0,51,459.1,41.7,44.9,0.026515151515151516,0.022918450740172708
|
data/imputation_no/splits_all/tasks_all/datasets_all/n_datasets_51
DELETED
|
File without changes
|
data/imputation_no/splits_all/tasks_all/datasets_all/pareto_front_improvability_vs_time_infer.png.zip
DELETED
|
@@ -1,3 +0,0 @@
|
|
| 1 |
-
version https://git-lfs.github.com/spec/v1
|
| 2 |
-
oid sha256:e071ec499e04cea0e9973fdf0227d5b12acbf8717c8a231d86d2f5a088d68846
|
| 3 |
-
size 488333
|
|
|
|
|
|
|
|
|
|
|
|
data/imputation_no/splits_all/tasks_all/datasets_all/pareto_n_configs_imp.png.zip
DELETED
|
@@ -1,3 +0,0 @@
|
|
| 1 |
-
version https://git-lfs.github.com/spec/v1
|
| 2 |
-
oid sha256:87df03b1368bd17592376d101f7369d23ad20d8673d6c5882dbe70043fe48ca5
|
| 3 |
-
size 424170
|
|
|
|
|
|
|
|
|
|
|
|
data/imputation_no/splits_all/tasks_all/datasets_all/tuning-impact-elo.png.zip
DELETED
|
@@ -1,3 +0,0 @@
|
|
| 1 |
-
version https://git-lfs.github.com/spec/v1
|
| 2 |
-
oid sha256:adf8b956073f8e691b2e0ba397b0f342c9a0d6e6d066527f2a8d7397adb5f448
|
| 3 |
-
size 98390
|
|
|
|
|
|
|
|
|
|
|
|
data/imputation_no/splits_all/tasks_all/datasets_all/website_leaderboard.csv
DELETED
|
@@ -1,60 +0,0 @@
|
|
| 1 |
-
#,Type,TypeName,Model,Elo [β¬οΈ],Elo 95% CI,Score [β¬οΈ],Rank [β¬οΈ],Harmonic Rank [β¬οΈ],Improvability (%) [β¬οΈ],Median Train Time (s/1K) [β¬οΈ],Median Predict Time (s/1K) [β¬οΈ],Verified,Imputed (%) [β¬οΈ],Imputed,Hardware
|
| 2 |
-
0,π,Reference Pipeline,"[AutoGluon 1.5 (extreme, 4h)](https://arxiv.org/abs/2003.06505)",1700.0,+89/-74,0.795,7.04,2.85,4.972,289.07,4.031,βοΈ,0.0,False,GPU
|
| 3 |
-
1,π§ β‘,Foundation Model,[TabPFN-2.6 (default)](https://arxiv.org/abs/2511.08667),1626.0,+82/-53,0.707,9.48,4.68,8.033,5.48,0.555,βοΈ,0.0,False,GPU
|
| 4 |
-
2,π§ β‘,Foundation Model,[RealTabPFN-2.5 (tuned + ensembled)](https://arxiv.org/abs/2511.08667),1604.0,+82/-62,0.67,10.34,3.95,7.666,2040.22,8.92,βοΈ,0.0,False,GPU
|
| 5 |
-
3,π§ β‘,Foundation Model,[TabICLv2 (default)](https://arxiv.org/abs/2602.11139),1598.0,+80/-66,0.718,10.55,3.03,6.995,4.02,0.381,βοΈ,0.0,False,GPU
|
| 6 |
-
4,π§ β‘,Foundation Model,[RealTabPFN-2.5 (tuned)](https://arxiv.org/abs/2511.08667),1559.0,+71/-55,0.617,12.21,5.16,8.438,2040.22,1.218,βοΈ,0.0,False,GPU
|
| 7 |
-
5,π§ β‘,Foundation Model,[RealTabPFN-2.5 (default)](https://arxiv.org/abs/2511.08667),1528.0,+67/-48,0.582,13.65,6.9,8.957,5.81,0.645,βοΈ,0.0,False,GPU
|
| 8 |
-
6,π,Reference Pipeline,"[AutoGluon 1.4 (best, 4h)](https://arxiv.org/abs/2003.06505)",1516.0,+60/-46,0.502,14.25,6.19,10.45,1735.72,2.559,βοΈ,0.0,False,CPU
|
| 9 |
-
7,π§ π,Neural Network,[RealMLP (tuned + ensembled)](https://arxiv.org/abs/2407.04491),1514.0,+58/-45,0.505,14.32,8.8,10.564,2950.72,11.988,βοΈ,0.0,False,GPU
|
| 10 |
-
8,π§ β‘,Foundation Model,[TabDPT (tuned + ensembled)](https://arxiv.org/abs/2410.18164),1460.0,+63/-54,0.472,17.17,5.92,11.016,4907.64,286.652,βοΈ,0.0,False,GPU
|
| 11 |
-
9,π§ π,Neural Network,[TabM (tuned + ensembled)](https://arxiv.org/abs/2410.24210),1448.0,+57/-43,0.403,17.8,9.79,11.971,3285.87,1.472,βοΈ,0.0,False,GPU
|
| 12 |
-
10,π³,Tree-based,[LightGBM (tuned + ensembled)](https://papers.nips.cc/paper_files/paper/2017/hash/6449f44a102fde848669bdd9eb6b76fa-Abstract.html),1435.0,+36/-30,0.349,18.57,13.16,12.986,416.98,2.639,βοΈ,0.0,False,CPU
|
| 13 |
-
11,π§ π,Neural Network,[RealMLP (tuned)](https://arxiv.org/abs/2407.04491),1430.0,+47/-46,0.372,18.84,10.57,11.953,2950.72,0.663,βοΈ,0.0,False,GPU
|
| 14 |
-
12,π³,Tree-based,[CatBoost (tuned + ensembled)](https://arxiv.org/abs/1706.09516),1418.0,+40/-41,0.341,19.54,11.92,12.603,1658.41,0.653,βοΈ,0.0,False,CPU
|
| 15 |
-
13,π³,Tree-based,[CatBoost (tuned)](https://arxiv.org/abs/1706.09516),1407.0,+40/-43,0.328,20.19,11.35,12.8,1658.41,0.081,βοΈ,0.0,False,CPU
|
| 16 |
-
14,π§ β‘,Foundation Model,[TabDPT (tuned)](https://arxiv.org/abs/2410.18164),1403.0,+60/-55,0.388,20.41,7.91,12.264,4907.64,39.96,βοΈ,0.0,False,GPU
|
| 17 |
-
15,π§ π,Neural Network,[TabM (tuned)](https://arxiv.org/abs/2410.24210),1391.0,+53/-43,0.335,21.15,11.0,12.846,3285.87,0.173,βοΈ,0.0,False,GPU
|
| 18 |
-
16,π§ π,Neural Network,[ModernNCA (tuned + ensembled)](https://arxiv.org/abs/2407.03257),1390.0,+82/-54,0.363,21.19,9.61,13.263,4621.67,8.149,βοΈ,0.0,False,GPU
|
| 19 |
-
17,π³,Tree-based,[LightGBM (tuned)](https://papers.nips.cc/paper_files/paper/2017/hash/6449f44a102fde848669bdd9eb6b76fa-Abstract.html),1387.0,+33/-27,0.27,21.37,15.69,13.662,416.98,0.334,βοΈ,0.0,False,CPU
|
| 20 |
-
18,π³,Tree-based,[XGBoost (tuned + ensembled)](https://arxiv.org/abs/1603.02754),1376.0,+34/-34,0.277,22.04,14.73,13.755,693.49,1.69,βοΈ,0.0,False,CPU
|
| 21 |
-
19,π³,Tree-based,[CatBoost (default)](https://arxiv.org/abs/1706.09516),1368.0,+41/-42,0.291,22.52,13.14,13.561,6.83,0.08,βοΈ,0.0,False,CPU
|
| 22 |
-
20,π§ π,Neural Network,[ModernNCA (tuned)](https://arxiv.org/abs/2407.03257),1357.0,+40/-37,0.246,23.24,12.89,13.784,4621.67,0.475,βοΈ,0.0,False,GPU
|
| 23 |
-
21,π³,Tree-based,[XGBoost (tuned)](https://arxiv.org/abs/1603.02754),1352.0,+33/-34,0.234,23.52,17.78,14.07,693.49,0.308,βοΈ,0.0,False,CPU
|
| 24 |
-
22,β,Other,[xRFM (tuned + ensembled)](https://arxiv.org/abs/2508.10053),1350.0,+51/-40,0.258,23.62,15.05,14.164,846.88,2.552,βοΈ,0.0,False,GPU
|
| 25 |
-
23,π§ β‘,Foundation Model,[TabDPT (default)](https://arxiv.org/abs/2410.18164),1324.0,+68/-56,0.304,25.26,9.86,14.638,47.62,43.739,βοΈ,0.0,False,GPU
|
| 26 |
-
24,β,Other,[xRFM (tuned)](https://arxiv.org/abs/2508.10053),1302.0,+45/-38,0.191,26.67,17.3,15.395,846.88,0.13,βοΈ,0.0,False,GPU
|
| 27 |
-
25,π§ π,Neural Network,[TabM (default)](https://arxiv.org/abs/2410.24210),1297.0,+49/-43,0.226,27.0,17.03,15.092,10.49,0.132,βοΈ,0.0,False,GPU
|
| 28 |
-
26,π§ π,Neural Network,[TorchMLP (tuned + ensembled)](https://arxiv.org/abs/2003.06505),1289.0,+49/-46,0.165,27.54,19.74,14.967,2874.67,1.952,βοΈ,0.0,False,CPU
|
| 29 |
-
27,π§ β‘,Foundation Model,[SAP-RPT-OSS (default)](https://arxiv.org/abs/2506.10707),1284.0,+57/-54,0.25,27.81,11.02,16.098,14.11,2.075,β,0.0,False,GPU
|
| 30 |
-
28,π³,Tree-based,[EBM (tuned + ensembled)](https://www.cs.cornell.edu/~yinlou/papers/lou-kdd13.pdf),1272.0,+42/-39,0.157,28.59,20.18,16.795,2929.85,0.422,βοΈ,0.0,False,CPU
|
| 31 |
-
29,π§ π,Neural Network,[ModernNCA (default)](https://arxiv.org/abs/2407.03257),1248.0,+44/-39,0.125,30.13,19.25,17.117,14.87,0.308,βοΈ,0.0,False,GPU
|
| 32 |
-
30,π³,Tree-based,[EBM (tuned)](https://www.cs.cornell.edu/~yinlou/papers/lou-kdd13.pdf),1236.0,+44/-41,0.113,30.94,22.27,17.463,2929.85,0.051,βοΈ,0.0,False,CPU
|
| 33 |
-
31,π§ π,Neural Network,[RealMLP (default)](https://arxiv.org/abs/2407.04491),1232.0,+38/-37,0.088,31.15,23.51,16.482,10.06,1.689,βοΈ,0.0,False,GPU
|
| 34 |
-
32,π³,Tree-based,[XGBoost (default)](https://arxiv.org/abs/1603.02754),1214.0,+39/-37,0.085,32.33,23.75,16.872,1.94,0.123,βοΈ,0.0,False,CPU
|
| 35 |
-
33,π§ π,Neural Network,[TorchMLP (tuned)](https://arxiv.org/abs/2003.06505),1213.0,+42/-45,0.092,32.41,24.23,16.753,2874.67,0.131,βοΈ,0.0,False,CPU
|
| 36 |
-
34,π§ π,Neural Network,[FastaiMLP (tuned + ensembled)](https://arxiv.org/abs/2003.06505),1208.0,+49/-57,0.13,32.7,19.96,17.625,593.24,4.467,βοΈ,0.0,False,CPU
|
| 37 |
-
35,π³,Tree-based,[EBM (default)](https://www.cs.cornell.edu/~yinlou/papers/lou-kdd13.pdf),1202.0,+44/-48,0.103,33.06,23.28,18.493,7.32,0.05,βοΈ,0.0,False,CPU
|
| 38 |
-
36,π³,Tree-based,[ExtraTrees (tuned + ensembled)](https://link.springer.com/article/10.1007/s10994-006-6226-1),1201.0,+47/-42,0.096,33.15,22.75,18.148,183.02,0.761,βοΈ,0.0,False,CPU
|
| 39 |
-
37,π³,Tree-based,[LightGBM (default)](https://papers.nips.cc/paper_files/paper/2017/hash/6449f44a102fde848669bdd9eb6b76fa-Abstract.html),1188.0,+33/-29,0.057,33.99,29.19,17.439,1.96,0.142,βοΈ,0.0,False,CPU
|
| 40 |
-
38,π³,Tree-based,[RandomForest (tuned + ensembled)](https://link.springer.com/article/10.1023/A:1010933404324),1172.0,+50/-45,0.078,34.95,23.48,19.09,373.18,0.771,βοΈ,0.0,False,CPU
|
| 41 |
-
39,π³,Tree-based,[ExtraTrees (tuned)](https://link.springer.com/article/10.1007/s10994-006-6226-1),1166.0,+45/-46,0.073,35.34,23.07,19.081,183.02,0.091,βοΈ,0.0,False,CPU
|
| 42 |
-
40,π§ π,Neural Network,[FastaiMLP (tuned)](https://arxiv.org/abs/2003.06505),1146.0,+50/-54,0.063,36.61,24.71,19.081,593.24,0.306,βοΈ,0.0,False,CPU
|
| 43 |
-
41,π³,Tree-based,[RandomForest (tuned)](https://link.springer.com/article/10.1023/A:1010933404324),1137.0,+48/-42,0.056,37.12,24.01,19.835,373.18,0.085,βοΈ,0.0,False,CPU
|
| 44 |
-
42,π§ β‘,Foundation Model,[TabSTAR (tuned)](https://arxiv.org/abs/2505.18125),1096.0,+76/-83,0.131,39.6,13.44,24.048,28729.74,4.27,βοΈ,0.0,False,GPU
|
| 45 |
-
43,π§ β‘,Foundation Model,[TabSTAR (tuned + ensembled)](https://arxiv.org/abs/2505.18125),1094.0,+80/-84,0.129,39.69,13.18,24.069,28729.74,18.819,βοΈ,0.0,False,GPU
|
| 46 |
-
44,π³,Tree-based,[PerpetualBooster (tuned + ensembled)](https://perpetual-ml.com/),1086.0,+46/-42,0.036,40.13,30.45,24.63,185.3,0.6,βοΈ,0.0,False,CPU
|
| 47 |
-
45,π§ π,Neural Network,[TorchMLP (default)](https://arxiv.org/abs/2003.06505),1074.0,+34/-40,0.017,40.86,32.65,20.928,9.99,0.126,βοΈ,0.0,False,CPU
|
| 48 |
-
46,π³,Tree-based,[PerpetualBooster (tuned)](https://perpetual-ml.com/),1047.0,+48/-46,0.019,42.34,31.9,26.023,185.3,0.258,βοΈ,0.0,False,CPU
|
| 49 |
-
47,β,Other,[xRFM (default)](https://arxiv.org/abs/2508.10053),1038.0,+63/-65,0.036,42.79,35.5,24.63,3.23,0.919,βοΈ,0.0,False,GPU
|
| 50 |
-
48,π§ π,Neural Network,[FastaiMLP (default)](https://arxiv.org/abs/2003.06505),1006.0,+52/-56,0.017,44.49,33.22,23.85,2.86,0.373,βοΈ,0.0,False,CPU
|
| 51 |
-
49,π³,Tree-based,[RandomForest (default)](https://link.springer.com/article/10.1023/A:1010933404324),1000.0,+45/-44,0.007,44.77,40.57,24.655,0.43,0.054,βοΈ,0.0,False,CPU
|
| 52 |
-
50,π,Baseline,[KNN (tuned + ensembled)](https://scikit-learn.org/stable/modules/neighbors.html),993.0,+59/-73,0.02,45.11,32.69,26.381,129.01,1.802,βοΈ,0.0,False,CPU
|
| 53 |
-
51,π§ β‘,Foundation Model,[TabSTAR (default)](https://arxiv.org/abs/2505.18125),987.0,+90/-102,0.079,45.4,21.19,29.102,384.69,5.351,βοΈ,0.0,False,GPU
|
| 54 |
-
52,π³,Tree-based,[ExtraTrees (default)](https://link.springer.com/article/10.1007/s10994-006-6226-1),980.0,+55/-62,0.012,45.77,41.09,26.069,0.25,0.05,βοΈ,0.0,False,CPU
|
| 55 |
-
53,π,Baseline,[Linear (tuned + ensembled)](https://scikit-learn.org/stable/modules/linear_model.html),960.0,+64/-99,0.032,46.71,34.75,32.362,237.58,0.424,βοΈ,0.0,False,CPU
|
| 56 |
-
54,π,Baseline,[Linear (tuned)](https://scikit-learn.org/stable/modules/linear_model.html),933.0,+70/-99,0.022,47.87,36.37,32.933,237.58,0.085,βοΈ,0.0,False,CPU
|
| 57 |
-
55,π³,Tree-based,[PerpetualBooster (default)](https://perpetual-ml.com/),932.0,+44/-62,0.001,47.93,36.2,30.184,27.32,0.029,βοΈ,0.0,False,CPU
|
| 58 |
-
56,π,Baseline,[KNN (tuned)](https://scikit-learn.org/stable/modules/neighbors.html),886.0,+61/-95,0.01,49.78,39.21,31.533,129.01,0.18,βοΈ,0.0,False,CPU
|
| 59 |
-
57,π,Baseline,[Linear (default)](https://scikit-learn.org/stable/modules/linear_model.html),855.0,+70/-112,0.014,50.92,40.14,35.591,1.19,0.12,βοΈ,0.0,False,CPU
|
| 60 |
-
58,π,Baseline,[KNN (default)](https://scikit-learn.org/stable/modules/neighbors.html),648.0,+77/-93,0.0,56.13,51.15,45.014,0.19,0.036,βοΈ,0.0,False,CPU
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
data/imputation_no/splits_all/tasks_all/datasets_all/winrate_matrix.png.zip
DELETED
|
@@ -1,3 +0,0 @@
|
|
| 1 |
-
version https://git-lfs.github.com/spec/v1
|
| 2 |
-
oid sha256:f3b0c91997f970373ae5756882ea749d97e111c187869654a99a6573b843c726
|
| 3 |
-
size 453130
|
|
|
|
|
|
|
|
|
|
|
|
data/imputation_no/splits_all/tasks_all/datasets_medium/n_datasets_15
DELETED
|
File without changes
|
data/imputation_no/splits_all/tasks_all/datasets_medium/pareto_front_improvability_vs_time_infer.png.zip
DELETED
|
@@ -1,3 +0,0 @@
|
|
| 1 |
-
version https://git-lfs.github.com/spec/v1
|
| 2 |
-
oid sha256:b05ef2155137bf842c784f6b4b8c985ec880cd076bff8639fbb5a2431505e171
|
| 3 |
-
size 486104
|
|
|
|
|
|
|
|
|
|
|
|
data/imputation_no/splits_all/tasks_all/datasets_medium/pareto_n_configs_imp.png.zip
DELETED
|
@@ -1,3 +0,0 @@
|
|
| 1 |
-
version https://git-lfs.github.com/spec/v1
|
| 2 |
-
oid sha256:e2de5ac5d0a900b2e5bb5c52c46d48d8f9848463bfb0c37b40620ef75039ca89
|
| 3 |
-
size 434896
|
|
|
|
|
|
|
|
|
|
|
|
data/imputation_no/splits_all/tasks_all/datasets_medium/tuning-impact-elo.png.zip
DELETED
|
@@ -1,3 +0,0 @@
|
|
| 1 |
-
version https://git-lfs.github.com/spec/v1
|
| 2 |
-
oid sha256:8e5af83454f207caec1b4849a9d4e456b5b325ef52ce0c26276dfa52fa647641
|
| 3 |
-
size 100299
|
|
|
|
|
|
|
|
|
|
|
|
data/imputation_no/splits_all/tasks_all/datasets_medium/website_leaderboard.csv
DELETED
|
@@ -1,60 +0,0 @@
|
|
| 1 |
-
#,Type,TypeName,Model,Elo [β¬οΈ],Elo 95% CI,Score [β¬οΈ],Rank [β¬οΈ],Harmonic Rank [β¬οΈ],Improvability (%) [β¬οΈ],Median Train Time (s/1K) [β¬οΈ],Median Predict Time (s/1K) [β¬οΈ],Verified,Imputed (%) [β¬οΈ],Imputed,Hardware
|
| 2 |
-
0,π,Reference Pipeline,"[AutoGluon 1.5 (extreme, 4h)](https://arxiv.org/abs/2003.06505)",1932.0,+95/-58,0.857,4.36,2.28,2.354,191.18,2.207,βοΈ,0.0,False,GPU
|
| 3 |
-
1,π,Reference Pipeline,"[AutoGluon 1.4 (best, 4h)](https://arxiv.org/abs/2003.06505)",1794.0,+95/-46,0.717,7.59,3.24,3.572,432.35,4.085,βοΈ,0.0,False,CPU
|
| 4 |
-
2,π§ β‘,Foundation Model,[TabPFN-2.6 (default)](https://arxiv.org/abs/2511.08667),1749.0,+125/-76,0.649,8.98,5.63,5.387,2.76,0.701,βοΈ,0.0,False,GPU
|
| 5 |
-
3,π§ β‘,Foundation Model,[TabICLv2 (default)](https://arxiv.org/abs/2602.11139),1715.0,+214/-112,0.66,10.16,2.55,4.309,0.76,0.139,βοΈ,0.0,False,GPU
|
| 6 |
-
4,π§ β‘,Foundation Model,[RealTabPFN-2.5 (tuned + ensembled)](https://arxiv.org/abs/2511.08667),1671.0,+156/-117,0.579,11.81,4.87,6.235,735.58,11.736,βοΈ,0.0,False,GPU
|
| 7 |
-
5,π§ π,Neural Network,[RealMLP (tuned + ensembled)](https://arxiv.org/abs/2407.04491),1647.0,+98/-92,0.529,12.81,8.83,6.546,1719.82,1.675,βοΈ,0.0,False,GPU
|
| 8 |
-
6,π§ π,Neural Network,[ModernNCA (tuned + ensembled)](https://arxiv.org/abs/2407.03257),1631.0,+116/-114,0.529,13.49,8.39,6.662,2526.28,6.013,βοΈ,0.0,False,GPU
|
| 9 |
-
7,π³,Tree-based,[CatBoost (tuned + ensembled)](https://arxiv.org/abs/1706.09516),1623.0,+86/-62,0.49,13.83,7.59,6.452,777.59,0.25,βοΈ,0.0,False,CPU
|
| 10 |
-
8,π§ β‘,Foundation Model,[RealTabPFN-2.5 (tuned)](https://arxiv.org/abs/2511.08667),1618.0,+137/-108,0.527,14.06,6.91,6.936,735.58,1.386,βοΈ,0.0,False,GPU
|
| 11 |
-
9,π³,Tree-based,[CatBoost (tuned)](https://arxiv.org/abs/1706.09516),1613.0,+93/-61,0.49,14.26,6.61,6.653,777.59,0.052,βοΈ,0.0,False,CPU
|
| 12 |
-
10,π³,Tree-based,[LightGBM (tuned + ensembled)](https://papers.nips.cc/paper_files/paper/2017/hash/6449f44a102fde848669bdd9eb6b76fa-Abstract.html),1602.0,+69/-50,0.435,14.76,10.3,8.214,131.56,2.639,βοΈ,0.0,False,CPU
|
| 13 |
-
11,π³,Tree-based,[CatBoost (default)](https://arxiv.org/abs/1706.09516),1571.0,+105/-99,0.47,16.17,8.09,6.869,3.24,0.031,βοΈ,0.0,False,CPU
|
| 14 |
-
12,π³,Tree-based,[XGBoost (tuned + ensembled)](https://arxiv.org/abs/1603.02754),1562.0,+91/-56,0.393,16.61,9.94,8.346,282.13,0.563,βοΈ,0.0,False,CPU
|
| 15 |
-
13,π§ π,Neural Network,[RealMLP (tuned)](https://arxiv.org/abs/2407.04491),1551.0,+110/-86,0.397,17.16,10.32,7.814,1719.82,0.081,βοΈ,0.0,False,GPU
|
| 16 |
-
14,π§ β‘,Foundation Model,[RealTabPFN-2.5 (default)](https://arxiv.org/abs/2511.08667),1543.0,+147/-94,0.429,17.55,10.42,7.777,1.88,0.645,βοΈ,0.0,False,GPU
|
| 17 |
-
15,π§ π,Neural Network,[TabM (tuned + ensembled)](https://arxiv.org/abs/2410.24210),1540.0,+153/-91,0.395,17.7,8.36,8.274,1993.14,0.619,βοΈ,0.0,False,GPU
|
| 18 |
-
16,π§ β‘,Foundation Model,[TabDPT (tuned + ensembled)](https://arxiv.org/abs/2410.18164),1533.0,+144/-126,0.432,18.01,5.14,7.892,4786.55,444.544,βοΈ,0.0,False,GPU
|
| 19 |
-
17,π³,Tree-based,[LightGBM (tuned)](https://papers.nips.cc/paper_files/paper/2017/hash/6449f44a102fde848669bdd9eb6b76fa-Abstract.html),1514.0,+81/-63,0.313,18.99,13.44,9.409,131.56,0.132,βοΈ,0.0,False,CPU
|
| 20 |
-
18,π³,Tree-based,[XGBoost (tuned)](https://arxiv.org/abs/1603.02754),1512.0,+68/-50,0.324,19.07,14.83,8.908,282.13,0.066,βοΈ,0.0,False,CPU
|
| 21 |
-
19,π§ π,Neural Network,[TabM (tuned)](https://arxiv.org/abs/2410.24210),1490.0,+160/-93,0.335,20.19,9.35,9.007,1993.14,0.061,βοΈ,0.0,False,GPU
|
| 22 |
-
20,π§ π,Neural Network,[ModernNCA (tuned)](https://arxiv.org/abs/2407.03257),1479.0,+98/-84,0.311,20.73,15.41,8.968,2526.28,0.349,βοΈ,0.0,False,GPU
|
| 23 |
-
21,β,Other,[xRFM (tuned + ensembled)](https://arxiv.org/abs/2508.10053),1436.0,+113/-112,0.262,23.03,15.17,10.476,566.74,0.951,βοΈ,0.0,False,GPU
|
| 24 |
-
22,π§ β‘,Foundation Model,[TabDPT (tuned)](https://arxiv.org/abs/2410.18164),1409.0,+122/-124,0.275,24.46,9.98,10.42,4786.55,42.641,βοΈ,0.0,False,GPU
|
| 25 |
-
23,π§ π,Neural Network,[TorchMLP (tuned + ensembled)](https://arxiv.org/abs/2003.06505),1386.0,+74/-68,0.176,25.7,19.95,10.226,1770.56,1.069,βοΈ,0.0,False,CPU
|
| 26 |
-
24,β,Other,[xRFM (tuned)](https://arxiv.org/abs/2508.10053),1384.0,+99/-103,0.198,25.79,20.46,12.128,566.74,0.06,βοΈ,0.0,False,GPU
|
| 27 |
-
25,π³,Tree-based,[XGBoost (default)](https://arxiv.org/abs/1603.02754),1374.0,+98/-112,0.191,26.36,15.9,10.756,0.49,0.053,βοΈ,0.0,False,CPU
|
| 28 |
-
26,π§ β‘,Foundation Model,[TabDPT (default)](https://arxiv.org/abs/2410.18164),1332.0,+133/-143,0.222,28.61,12.07,13.196,46.62,43.739,βοΈ,0.0,False,GPU
|
| 29 |
-
27,π§ π,Neural Network,[TabM (default)](https://arxiv.org/abs/2410.24210),1330.0,+128/-102,0.166,28.76,19.62,11.857,5.16,0.074,βοΈ,0.0,False,GPU
|
| 30 |
-
28,π³,Tree-based,[EBM (tuned + ensembled)](https://www.cs.cornell.edu/~yinlou/papers/lou-kdd13.pdf),1328.0,+122/-130,0.192,28.83,20.49,13.755,1517.65,0.134,βοΈ,0.0,False,CPU
|
| 31 |
-
29,π§ π,Neural Network,[ModernNCA (default)](https://arxiv.org/abs/2407.03257),1302.0,+77/-79,0.11,30.29,26.46,12.273,8.06,0.25,βοΈ,0.0,False,GPU
|
| 32 |
-
30,π§ π,Neural Network,[RealMLP (default)](https://arxiv.org/abs/2407.04491),1279.0,+81/-69,0.064,31.5,28.42,12.947,6.75,0.235,βοΈ,0.0,False,GPU
|
| 33 |
-
31,π§ π,Neural Network,[TorchMLP (tuned)](https://arxiv.org/abs/2003.06505),1278.0,+77/-74,0.072,31.6,27.08,11.947,1770.56,0.051,βοΈ,0.0,False,CPU
|
| 34 |
-
32,π³,Tree-based,[EBM (tuned)](https://www.cs.cornell.edu/~yinlou/papers/lou-kdd13.pdf),1267.0,+110/-140,0.128,32.19,25.38,14.642,1517.65,0.015,βοΈ,0.0,False,CPU
|
| 35 |
-
33,π³,Tree-based,[LightGBM (default)](https://papers.nips.cc/paper_files/paper/2017/hash/6449f44a102fde848669bdd9eb6b76fa-Abstract.html),1262.0,+60/-68,0.058,32.44,30.18,12.659,0.29,0.045,βοΈ,0.0,False,CPU
|
| 36 |
-
34,π³,Tree-based,[RandomForest (tuned + ensembled)](https://link.springer.com/article/10.1023/A:1010933404324),1249.0,+85/-83,0.042,33.15,27.74,13.167,66.0,0.341,βοΈ,0.0,False,CPU
|
| 37 |
-
35,π³,Tree-based,[EBM (default)](https://www.cs.cornell.edu/~yinlou/papers/lou-kdd13.pdf),1218.0,+102/-135,0.099,34.81,29.72,16.189,3.12,0.016,βοΈ,0.0,False,CPU
|
| 38 |
-
36,π³,Tree-based,[RandomForest (tuned)](https://link.springer.com/article/10.1023/A:1010933404324),1217.0,+91/-80,0.046,34.85,29.25,13.841,66.0,0.024,βοΈ,0.0,False,CPU
|
| 39 |
-
37,π³,Tree-based,[ExtraTrees (tuned + ensembled)](https://link.springer.com/article/10.1007/s10994-006-6226-1),1206.0,+75/-96,0.028,35.4,32.86,13.309,36.48,0.329,βοΈ,0.0,False,CPU
|
| 40 |
-
38,π§ π,Neural Network,[FastaiMLP (tuned + ensembled)](https://arxiv.org/abs/2003.06505),1204.0,+121/-140,0.083,35.52,27.63,15.115,351.94,1.658,βοΈ,0.0,False,CPU
|
| 41 |
-
39,π§ β‘,Foundation Model,[SAP-RPT-OSS (default)](https://arxiv.org/abs/2506.10707),1202.0,+87/-94,0.046,35.63,30.77,15.154,11.44,10.416,β,0.0,False,GPU
|
| 42 |
-
40,π³,Tree-based,[ExtraTrees (tuned)](https://link.springer.com/article/10.1007/s10994-006-6226-1),1161.0,+68/-75,0.014,37.74,35.83,14.467,36.48,0.037,βοΈ,0.0,False,CPU
|
| 43 |
-
41,π§ π,Neural Network,[TorchMLP (default)](https://arxiv.org/abs/2003.06505),1122.0,+68/-74,0.004,39.71,37.95,16.236,2.18,0.039,βοΈ,0.0,False,CPU
|
| 44 |
-
42,π§ π,Neural Network,[FastaiMLP (tuned)](https://arxiv.org/abs/2003.06505),1098.0,+90/-101,0.024,40.86,36.07,17.304,351.94,0.114,βοΈ,0.0,False,CPU
|
| 45 |
-
43,π³,Tree-based,[PerpetualBooster (tuned + ensembled)](https://perpetual-ml.com/),1094.0,+105/-121,0.024,41.04,35.31,19.604,20.26,0.515,βοΈ,0.0,False,CPU
|
| 46 |
-
44,π³,Tree-based,[PerpetualBooster (tuned)](https://perpetual-ml.com/),1044.0,+95/-115,0.01,43.4,39.56,20.76,20.26,0.2,βοΈ,0.0,False,CPU
|
| 47 |
-
45,β,Other,[xRFM (default)](https://arxiv.org/abs/2508.10053),1034.0,+154/-192,0.023,43.81,37.67,24.374,2.49,0.145,βοΈ,0.0,False,GPU
|
| 48 |
-
46,π§ π,Neural Network,[FastaiMLP (default)](https://arxiv.org/abs/2003.06505),1021.0,+139/-166,0.027,44.41,39.24,20.959,1.51,0.111,βοΈ,0.0,False,CPU
|
| 49 |
-
47,π³,Tree-based,[RandomForest (default)](https://link.springer.com/article/10.1023/A:1010933404324),1000.0,+56/-92,0.0,45.31,44.21,19.395,0.2,0.021,βοΈ,0.0,False,CPU
|
| 50 |
-
48,π,Baseline,[KNN (tuned + ensembled)](https://scikit-learn.org/stable/modules/neighbors.html),985.0,+102/-124,0.007,45.95,40.09,22.93,150.3,10.071,βοΈ,0.0,False,CPU
|
| 51 |
-
49,π§ β‘,Foundation Model,[TabSTAR (tuned)](https://arxiv.org/abs/2505.18125),983.0,+59/-98,0.0,46.04,44.38,24.3,3381.73,1.724,βοΈ,0.0,False,GPU
|
| 52 |
-
50,π§ β‘,Foundation Model,[TabSTAR (tuned + ensembled)](https://arxiv.org/abs/2505.18125),982.0,+60/-92,0.0,46.07,44.53,24.278,3381.73,5.283,βοΈ,0.0,False,GPU
|
| 53 |
-
51,π³,Tree-based,[ExtraTrees (default)](https://link.springer.com/article/10.1007/s10994-006-6226-1),940.0,+107/-128,0.004,47.74,45.79,22.49,0.11,0.028,βοΈ,0.0,False,CPU
|
| 54 |
-
52,π,Baseline,[Linear (tuned + ensembled)](https://scikit-learn.org/stable/modules/linear_model.html),926.0,+157/-216,0.007,48.25,43.68,31.744,43.64,0.081,βοΈ,0.0,False,CPU
|
| 55 |
-
53,π,Baseline,[Linear (tuned)](https://scikit-learn.org/stable/modules/linear_model.html),882.0,+159/-230,0.004,49.84,46.21,32.653,43.64,0.018,βοΈ,0.0,False,CPU
|
| 56 |
-
54,π³,Tree-based,[PerpetualBooster (default)](https://perpetual-ml.com/),881.0,+99/-160,0.004,49.86,47.98,27.991,1.5,0.015,βοΈ,0.0,False,CPU
|
| 57 |
-
55,π§ β‘,Foundation Model,[TabSTAR (default)](https://arxiv.org/abs/2505.18125),841.0,+50/-95,0.0,51.15,50.53,29.403,110.78,1.964,βοΈ,0.0,False,GPU
|
| 58 |
-
56,π,Baseline,[KNN (tuned)](https://scikit-learn.org/stable/modules/neighbors.html),808.0,+56/-95,0.0,52.13,51.83,28.803,150.3,0.482,βοΈ,0.0,False,CPU
|
| 59 |
-
57,π,Baseline,[Linear (default)](https://scikit-learn.org/stable/modules/linear_model.html),787.0,+135/-221,0.0,52.7,50.9,34.725,0.19,0.019,βοΈ,0.0,False,CPU
|
| 60 |
-
58,π,Baseline,[KNN (default)](https://scikit-learn.org/stable/modules/neighbors.html),580.0,+86/-145,0.0,56.78,56.63,42.573,0.1,0.072,βοΈ,0.0,False,CPU
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
data/imputation_no/splits_all/tasks_all/datasets_medium/winrate_matrix.png.zip
DELETED
|
@@ -1,3 +0,0 @@
|
|
| 1 |
-
version https://git-lfs.github.com/spec/v1
|
| 2 |
-
oid sha256:7a1709586877e7a38ad7b604d2fe69f92b154dbad04f33288628d4fb29eba092
|
| 3 |
-
size 427492
|
|
|
|
|
|
|
|
|
|
|
|
data/imputation_no/splits_all/tasks_all/datasets_small/n_datasets_36
DELETED
|
File without changes
|
data/imputation_no/splits_all/tasks_all/datasets_small/pareto_front_improvability_vs_time_infer.png.zip
DELETED
|
@@ -1,3 +0,0 @@
|
|
| 1 |
-
version https://git-lfs.github.com/spec/v1
|
| 2 |
-
oid sha256:2173bab31bbab009f3f6fdba522121d6e8eb8d383034c4723c85c234de7d27f1
|
| 3 |
-
size 476539
|
|
|
|
|
|
|
|
|
|
|
|
data/imputation_no/splits_all/tasks_all/datasets_small/pareto_n_configs_imp.png.zip
DELETED
|
@@ -1,3 +0,0 @@
|
|
| 1 |
-
version https://git-lfs.github.com/spec/v1
|
| 2 |
-
oid sha256:6713b59a3421e90c28f960343be48d46c5cea99ce5f01d44177deffdbef754ac
|
| 3 |
-
size 419177
|
|
|
|
|
|
|
|
|
|
|
|
data/imputation_no/splits_all/tasks_all/datasets_small/tuning-impact-elo.png.zip
DELETED
|
@@ -1,3 +0,0 @@
|
|
| 1 |
-
version https://git-lfs.github.com/spec/v1
|
| 2 |
-
oid sha256:4821e4d0f9c0f3c5d9da45ea77bd3aba2771c84f19dd76224d14680072268532
|
| 3 |
-
size 106380
|
|
|
|
|
|
|
|
|
|
|
|
data/imputation_no/splits_all/tasks_all/datasets_small/website_leaderboard.csv
DELETED
|
@@ -1,60 +0,0 @@
|
|
| 1 |
-
#,Type,TypeName,Model,Elo [β¬οΈ],Elo 95% CI,Score [β¬οΈ],Rank [β¬οΈ],Harmonic Rank [β¬οΈ],Improvability (%) [β¬οΈ],Median Train Time (s/1K) [β¬οΈ],Median Predict Time (s/1K) [β¬οΈ],Verified,Imputed (%) [β¬οΈ],Imputed,Hardware
|
| 2 |
-
0,π,Reference Pipeline,"[AutoGluon 1.5 (extreme, 4h)](https://arxiv.org/abs/2003.06505)",1642.0,+87/-57,0.769,8.16,3.19,6.063,346.57,6.557,βοΈ,0.0,False,GPU
|
| 3 |
-
1,π§ β‘,Foundation Model,[TabPFN-2.6 (default)](https://arxiv.org/abs/2511.08667),1599.0,+77/-48,0.731,9.69,4.37,9.136,7.03,0.548,βοΈ,0.0,False,GPU
|
| 4 |
-
2,π§ β‘,Foundation Model,[RealTabPFN-2.5 (tuned + ensembled)](https://arxiv.org/abs/2511.08667),1598.0,+96/-63,0.708,9.73,3.66,8.263,2289.05,8.049,βοΈ,0.0,False,GPU
|
| 5 |
-
3,π§ β‘,Foundation Model,[TabICLv2 (default)](https://arxiv.org/abs/2602.11139),1573.0,+109/-84,0.743,10.72,3.29,8.114,7.06,0.671,βοΈ,0.0,False,GPU
|
| 6 |
-
4,π§ β‘,Foundation Model,[RealTabPFN-2.5 (tuned)](https://arxiv.org/abs/2511.08667),1556.0,+73/-58,0.655,11.44,4.67,9.065,2289.05,1.143,βοΈ,0.0,False,GPU
|
| 7 |
-
5,π§ β‘,Foundation Model,[RealTabPFN-2.5 (default)](https://arxiv.org/abs/2511.08667),1543.0,+88/-54,0.645,12.03,6.05,9.448,6.76,0.636,βοΈ,0.0,False,GPU
|
| 8 |
-
6,π§ π,Neural Network,[RealMLP (tuned + ensembled)](https://arxiv.org/abs/2407.04491),1482.0,+64/-47,0.494,14.95,8.79,12.238,3770.75,21.902,βοΈ,0.0,False,GPU
|
| 9 |
-
7,π§ β‘,Foundation Model,[TabDPT (tuned + ensembled)](https://arxiv.org/abs/2410.18164),1447.0,+81/-59,0.489,16.81,6.32,12.318,5119.36,218.711,βοΈ,0.0,False,GPU
|
| 10 |
-
8,π,Reference Pipeline,"[AutoGluon 1.4 (best, 4h)](https://arxiv.org/abs/2003.06505)",1444.0,+53/-49,0.413,17.03,9.96,13.315,3716.07,2.547,βοΈ,0.0,False,CPU
|
| 11 |
-
9,π§ π,Neural Network,[TabM (tuned + ensembled)](https://arxiv.org/abs/2410.24210),1429.0,+58/-51,0.407,17.84,10.54,13.511,3553.12,1.741,βοΈ,0.0,False,GPU
|
| 12 |
-
10,π§ β‘,Foundation Model,[TabDPT (tuned)](https://arxiv.org/abs/2410.18164),1414.0,+73/-58,0.435,18.73,7.29,13.032,5119.36,28.351,βοΈ,0.0,False,GPU
|
| 13 |
-
11,π§ π,Neural Network,[RealMLP (tuned)](https://arxiv.org/abs/2407.04491),1400.0,+54/-40,0.362,19.54,10.68,13.678,3770.75,1.778,βοΈ,0.0,False,GPU
|
| 14 |
-
12,π³,Tree-based,[LightGBM (tuned + ensembled)](https://papers.nips.cc/paper_files/paper/2017/hash/6449f44a102fde848669bdd9eb6b76fa-Abstract.html),1390.0,+36/-31,0.313,20.16,14.88,14.975,892.41,2.574,βοΈ,0.0,False,CPU
|
| 15 |
-
13,π§ π,Neural Network,[TabM (tuned)](https://arxiv.org/abs/2410.24210),1367.0,+56/-51,0.334,21.54,11.87,14.445,3553.12,0.241,βοΈ,0.0,False,GPU
|
| 16 |
-
14,π³,Tree-based,[CatBoost (tuned + ensembled)](https://arxiv.org/abs/1706.09516),1361.0,+46/-41,0.278,21.92,15.62,15.166,2476.51,0.814,βοΈ,0.0,False,CPU
|
| 17 |
-
15,π³,Tree-based,[LightGBM (tuned)](https://papers.nips.cc/paper_files/paper/2017/hash/6449f44a102fde848669bdd9eb6b76fa-Abstract.html),1354.0,+36/-29,0.252,22.36,16.87,15.434,892.41,0.346,βοΈ,0.0,False,CPU
|
| 18 |
-
16,π³,Tree-based,[CatBoost (tuned)](https://arxiv.org/abs/1706.09516),1349.0,+48/-35,0.26,22.66,16.17,15.361,2476.51,0.101,βοΈ,0.0,False,CPU
|
| 19 |
-
17,π§ β‘,Foundation Model,[TabDPT (default)](https://arxiv.org/abs/2410.18164),1330.0,+74/-66,0.338,23.87,9.16,15.239,50.32,43.709,βοΈ,0.0,False,GPU
|
| 20 |
-
18,β,Other,[xRFM (tuned + ensembled)](https://arxiv.org/abs/2508.10053),1330.0,+46/-41,0.256,23.87,15.0,15.7,1126.71,2.941,βοΈ,0.0,False,GPU
|
| 21 |
-
19,π§ π,Neural Network,[ModernNCA (tuned)](https://arxiv.org/abs/2407.03257),1324.0,+42/-40,0.219,24.29,12.06,15.791,6521.96,0.506,βοΈ,0.0,False,GPU
|
| 22 |
-
20,π³,Tree-based,[XGBoost (tuned + ensembled)](https://arxiv.org/abs/1603.02754),1324.0,+34/-36,0.229,24.3,18.43,16.009,884.18,2.368,βοΈ,0.0,False,CPU
|
| 23 |
-
21,π§ π,Neural Network,[ModernNCA (tuned + ensembled)](https://arxiv.org/abs/2407.03257),1322.0,+65/-56,0.294,24.4,10.23,16.013,6521.96,8.396,βοΈ,0.0,False,GPU
|
| 24 |
-
22,π§ β‘,Foundation Model,[SAP-RPT-OSS (default)](https://arxiv.org/abs/2506.10707),1320.0,+71/-57,0.335,24.55,8.7,16.492,16.22,1.278,β,0.0,False,GPU
|
| 25 |
-
23,π³,Tree-based,[CatBoost (default)](https://arxiv.org/abs/1706.09516),1310.0,+36/-35,0.217,25.17,17.76,16.35,9.64,0.128,βοΈ,0.0,False,CPU
|
| 26 |
-
24,π³,Tree-based,[XGBoost (tuned)](https://arxiv.org/abs/1603.02754),1307.0,+31/-38,0.197,25.38,19.38,16.22,884.18,0.393,βοΈ,0.0,False,CPU
|
| 27 |
-
25,π§ π,Neural Network,[TabM (default)](https://arxiv.org/abs/2410.24210),1294.0,+55/-46,0.252,26.26,16.14,16.44,13.18,0.17,βοΈ,0.0,False,GPU
|
| 28 |
-
26,β,Other,[xRFM (tuned)](https://arxiv.org/abs/2508.10053),1282.0,+40/-45,0.188,27.03,16.25,16.756,1126.71,0.163,βοΈ,0.0,False,GPU
|
| 29 |
-
27,π§ π,Neural Network,[TorchMLP (tuned + ensembled)](https://arxiv.org/abs/2003.06505),1263.0,+44/-50,0.161,28.3,19.66,16.943,3865.95,2.158,βοΈ,0.0,False,CPU
|
| 30 |
-
28,π³,Tree-based,[EBM (tuned + ensembled)](https://www.cs.cornell.edu/~yinlou/papers/lou-kdd13.pdf),1260.0,+42/-45,0.143,28.48,20.06,18.062,3351.28,0.47,βοΈ,0.0,False,CPU
|
| 31 |
-
29,π§ π,Neural Network,[ModernNCA (default)](https://arxiv.org/abs/2407.03257),1236.0,+48/-52,0.132,30.06,17.29,19.136,16.27,0.321,βοΈ,0.0,False,GPU
|
| 32 |
-
30,π³,Tree-based,[EBM (tuned)](https://www.cs.cornell.edu/~yinlou/papers/lou-kdd13.pdf),1231.0,+47/-44,0.106,30.42,21.19,18.638,3351.28,0.056,βοΈ,0.0,False,CPU
|
| 33 |
-
31,π§ π,Neural Network,[RealMLP (default)](https://arxiv.org/abs/2407.04491),1222.0,+37/-42,0.098,31.0,21.94,17.954,15.69,4.691,βοΈ,0.0,False,GPU
|
| 34 |
-
32,π§ π,Neural Network,[FastaiMLP (tuned + ensembled)](https://arxiv.org/abs/2003.06505),1214.0,+60/-61,0.15,31.52,17.89,18.671,1678.92,8.069,βοΈ,0.0,False,CPU
|
| 35 |
-
33,π³,Tree-based,[ExtraTrees (tuned + ensembled)](https://link.springer.com/article/10.1007/s10994-006-6226-1),1204.0,+65/-64,0.125,32.21,20.17,20.165,499.14,1.466,βοΈ,0.0,False,CPU
|
| 36 |
-
34,π³,Tree-based,[EBM (default)](https://www.cs.cornell.edu/~yinlou/papers/lou-kdd13.pdf),1202.0,+48/-50,0.105,32.33,21.36,19.452,9.08,0.057,βοΈ,0.0,False,CPU
|
| 37 |
-
35,π§ π,Neural Network,[TorchMLP (tuned)](https://arxiv.org/abs/2003.06505),1196.0,+40/-47,0.1,32.74,23.21,18.755,3865.95,0.148,βοΈ,0.0,False,CPU
|
| 38 |
-
36,π³,Tree-based,[ExtraTrees (tuned)](https://link.springer.com/article/10.1007/s10994-006-6226-1),1172.0,+61/-61,0.097,34.33,20.09,21.004,499.14,0.207,βοΈ,0.0,False,CPU
|
| 39 |
-
37,π³,Tree-based,[LightGBM (default)](https://papers.nips.cc/paper_files/paper/2017/hash/6449f44a102fde848669bdd9eb6b76fa-Abstract.html),1167.0,+40/-41,0.057,34.64,28.8,19.43,3.61,0.17,βοΈ,0.0,False,CPU
|
| 40 |
-
38,π³,Tree-based,[XGBoost (default)](https://arxiv.org/abs/1603.02754),1164.0,+31/-37,0.041,34.82,29.92,19.42,3.29,0.253,βοΈ,0.0,False,CPU
|
| 41 |
-
39,π§ π,Neural Network,[FastaiMLP (tuned)](https://arxiv.org/abs/2003.06505),1164.0,+49/-51,0.079,34.83,21.84,19.822,1678.92,0.826,βοΈ,0.0,False,CPU
|
| 42 |
-
40,π³,Tree-based,[RandomForest (tuned + ensembled)](https://link.springer.com/article/10.1023/A:1010933404324),1151.0,+56/-67,0.093,35.7,22.07,21.558,590.04,1.461,βοΈ,0.0,False,CPU
|
| 43 |
-
41,π§ β‘,Foundation Model,[TabSTAR (tuned)](https://arxiv.org/abs/2505.18125),1131.0,+75/-77,0.186,36.92,10.41,23.943,40848.87,5.609,βοΈ,0.0,False,GPU
|
| 44 |
-
42,π§ β‘,Foundation Model,[TabSTAR (tuned + ensembled)](https://arxiv.org/abs/2505.18125),1130.0,+75/-78,0.183,37.04,10.19,23.982,40848.87,25.959,βοΈ,0.0,False,GPU
|
| 45 |
-
43,π³,Tree-based,[RandomForest (tuned)](https://link.springer.com/article/10.1023/A:1010933404324),1113.0,+49/-65,0.06,38.06,22.34,22.333,590.04,0.15,βοΈ,0.0,False,CPU
|
| 46 |
-
44,π³,Tree-based,[PerpetualBooster (tuned + ensembled)](https://perpetual-ml.com/),1086.0,+44/-57,0.041,39.75,28.79,26.724,341.86,0.895,βοΈ,0.0,False,CPU
|
| 47 |
-
45,π§ π,Neural Network,[TorchMLP (default)](https://arxiv.org/abs/2003.06505),1058.0,+45/-47,0.022,41.33,30.86,22.883,13.69,0.155,βοΈ,0.0,False,CPU
|
| 48 |
-
46,π³,Tree-based,[PerpetualBooster (tuned)](https://perpetual-ml.com/),1049.0,+47/-54,0.022,41.91,29.52,28.216,341.86,0.313,βοΈ,0.0,False,CPU
|
| 49 |
-
47,β,Other,[xRFM (default)](https://arxiv.org/abs/2508.10053),1040.0,+59/-82,0.042,42.36,34.67,24.737,5.44,1.098,βοΈ,0.0,False,GPU
|
| 50 |
-
48,π§ β‘,Foundation Model,[TabSTAR (default)](https://arxiv.org/abs/2505.18125),1029.0,+80/-114,0.112,43.0,17.06,28.977,471.66,7.221,βοΈ,0.0,False,GPU
|
| 51 |
-
49,π§ π,Neural Network,[FastaiMLP (default)](https://arxiv.org/abs/2003.06505),1000.0,+46/-59,0.013,44.52,31.22,25.054,5.48,0.685,βοΈ,0.0,False,CPU
|
| 52 |
-
50,π³,Tree-based,[RandomForest (default)](https://link.springer.com/article/10.1023/A:1010933404324),1000.0,+58/-69,0.01,44.55,39.23,26.847,0.51,0.089,βοΈ,0.0,False,CPU
|
| 53 |
-
51,π,Baseline,[KNN (tuned + ensembled)](https://scikit-learn.org/stable/modules/neighbors.html),996.0,+66/-90,0.026,44.77,30.35,27.818,111.84,1.215,βοΈ,0.0,False,CPU
|
| 54 |
-
52,π³,Tree-based,[ExtraTrees (default)](https://link.springer.com/article/10.1007/s10994-006-6226-1),992.0,+63/-68,0.015,44.94,39.41,27.559,0.47,0.092,βοΈ,0.0,False,CPU
|
| 55 |
-
53,π,Baseline,[Linear (tuned + ensembled)](https://scikit-learn.org/stable/modules/linear_model.html),970.0,+83/-95,0.042,46.06,32.03,32.619,520.95,0.647,βοΈ,0.0,False,CPU
|
| 56 |
-
54,π,Baseline,[Linear (tuned)](https://scikit-learn.org/stable/modules/linear_model.html),949.0,+82/-103,0.03,47.04,33.41,33.05,520.95,0.113,βοΈ,0.0,False,CPU
|
| 57 |
-
55,π³,Tree-based,[PerpetualBooster (default)](https://perpetual-ml.com/),948.0,+39/-55,0.0,47.12,32.84,31.097,39.08,0.041,βοΈ,0.0,False,CPU
|
| 58 |
-
56,π,Baseline,[KNN (tuned)](https://scikit-learn.org/stable/modules/neighbors.html),909.0,+68/-86,0.014,48.8,35.6,32.671,111.84,0.111,βοΈ,0.0,False,CPU
|
| 59 |
-
57,π,Baseline,[Linear (default)](https://scikit-learn.org/stable/modules/linear_model.html),875.0,+97/-116,0.019,50.17,36.89,35.951,2.59,0.138,βοΈ,0.0,False,CPU
|
| 60 |
-
58,π,Baseline,[KNN (default)](https://scikit-learn.org/stable/modules/neighbors.html),666.0,+99/-128,0.0,55.86,49.16,46.031,0.37,0.033,βοΈ,0.0,False,CPU
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
data/imputation_no/splits_all/tasks_all/datasets_small/winrate_matrix.png.zip
DELETED
|
@@ -1,3 +0,0 @@
|
|
| 1 |
-
version https://git-lfs.github.com/spec/v1
|
| 2 |
-
oid sha256:e7469631a34d8b36ef3458837d406b534c534acd0ea088256610d70d23ed1f39
|
| 3 |
-
size 454390
|
|
|
|
|
|
|
|
|
|
|
|
data/imputation_no/splits_all/tasks_binary/datasets_all/n_datasets_30
DELETED
|
File without changes
|
data/imputation_no/splits_all/tasks_binary/datasets_all/pareto_front_improvability_vs_time_infer.png.zip
DELETED
|
@@ -1,3 +0,0 @@
|
|
| 1 |
-
version https://git-lfs.github.com/spec/v1
|
| 2 |
-
oid sha256:9ae1fd05c8128272d4a0460d25853f5a33aa9c6d6a0b678b4f129620c26bdace
|
| 3 |
-
size 500306
|
|
|
|
|
|
|
|
|
|
|
|
data/imputation_no/splits_all/tasks_binary/datasets_all/pareto_n_configs_imp.png.zip
DELETED
|
@@ -1,3 +0,0 @@
|
|
| 1 |
-
version https://git-lfs.github.com/spec/v1
|
| 2 |
-
oid sha256:7a7435153c939e3b37c316fcc033d8ef33dc975b0a0df929d6c546c151719fb4
|
| 3 |
-
size 419358
|
|
|
|
|
|
|
|
|
|
|
|
data/imputation_no/splits_all/tasks_binary/datasets_all/tuning-impact-elo.png.zip
DELETED
|
@@ -1,3 +0,0 @@
|
|
| 1 |
-
version https://git-lfs.github.com/spec/v1
|
| 2 |
-
oid sha256:7a74f6b25564735a3019aa9d5d2fd71ef0542526ebf5b1811297c9cb557eba77
|
| 3 |
-
size 108703
|
|
|
|
|
|
|
|
|
|
|
|
data/imputation_no/splits_all/tasks_binary/datasets_all/website_leaderboard.csv
DELETED
|
@@ -1,62 +0,0 @@
|
|
| 1 |
-
#,Type,TypeName,Model,Elo [β¬οΈ],Elo 95% CI,Score [β¬οΈ],Rank [β¬οΈ],Harmonic Rank [β¬οΈ],Improvability (%) [β¬οΈ],Median Train Time (s/1K) [β¬οΈ],Median Predict Time (s/1K) [β¬οΈ],Verified,Imputed (%) [β¬οΈ],Imputed,Hardware
|
| 2 |
-
0,π,Reference Pipeline,"[AutoGluon 1.5 (extreme, 4h)](https://arxiv.org/abs/2003.06505)",1685.0,+123/-92,0.777,7.55,2.76,5.022,272.41,4.562,βοΈ,0.0,False,GPU
|
| 3 |
-
1,π§ β‘,Foundation Model,[TabICLv2 (default)](https://arxiv.org/abs/2602.11139),1605.0,+123/-67,0.711,10.44,3.1,7.359,3.01,0.337,βοΈ,0.0,False,GPU
|
| 4 |
-
2,π§ β‘,Foundation Model,[TabPFN-2.6 (default)](https://arxiv.org/abs/2511.08667),1603.0,+80/-60,0.681,10.55,4.87,9.332,4.82,0.464,βοΈ,0.0,False,GPU
|
| 5 |
-
3,π§ β‘,Foundation Model,[RealTabPFN-2.5 (tuned + ensembled)](https://arxiv.org/abs/2511.08667),1555.0,+112/-58,0.595,12.67,4.95,10.009,1923.88,9.103,βοΈ,0.0,False,GPU
|
| 6 |
-
4,π§ β‘,Foundation Model,[RealTabPFN-2.5 (default)](https://arxiv.org/abs/2511.08667),1536.0,+85/-57,0.586,13.57,6.78,10.48,4.91,0.694,βοΈ,0.0,False,GPU
|
| 7 |
-
5,π§ β‘,Foundation Model,[RealTabPFN-2.5 (tuned)](https://arxiv.org/abs/2511.08667),1525.0,+93/-56,0.557,14.16,5.79,10.69,1923.88,1.329,βοΈ,0.0,False,GPU
|
| 8 |
-
6,π,Reference Pipeline,"[AutoGluon 1.4 (best, 4h)](https://arxiv.org/abs/2003.06505)",1504.0,+77/-69,0.503,15.26,5.6,11.459,1152.95,1.784,βοΈ,0.0,False,CPU
|
| 9 |
-
7,π§ π,Neural Network,[RealMLP (tuned + ensembled)](https://arxiv.org/abs/2407.04491),1489.0,+50/-54,0.455,16.05,10.66,12.456,2775.44,8.849,βοΈ,0.0,False,GPU
|
| 10 |
-
8,π§ π,Neural Network,[TabM (tuned + ensembled)](https://arxiv.org/abs/2410.24210),1466.0,+82/-56,0.416,17.37,8.52,13.167,2229.7,1.387,βοΈ,0.0,False,GPU
|
| 11 |
-
9,π³,Tree-based,[LightGBM (tuned + ensembled)](https://papers.nips.cc/paper_files/paper/2017/hash/6449f44a102fde848669bdd9eb6b76fa-Abstract.html),1431.0,+54/-40,0.338,19.44,12.88,14.373,328.96,1.102,βοΈ,0.0,False,CPU
|
| 12 |
-
10,π§ β‘,Foundation Model,[TabDPT (tuned + ensembled)](https://arxiv.org/abs/2410.18164),1412.0,+70/-59,0.37,20.62,8.18,12.195,4982.66,358.753,βοΈ,0.0,False,GPU
|
| 13 |
-
11,π§ π,Neural Network,[TabM (tuned)](https://arxiv.org/abs/2410.24210),1410.0,+79/-60,0.352,20.76,9.81,14.026,2229.7,0.133,βοΈ,0.0,False,GPU
|
| 14 |
-
12,π³,Tree-based,[CatBoost (tuned + ensembled)](https://arxiv.org/abs/1706.09516),1404.0,+66/-53,0.329,21.15,11.53,14.171,1055.6,0.482,βοΈ,0.0,False,CPU
|
| 15 |
-
13,π³,Tree-based,[CatBoost (tuned)](https://arxiv.org/abs/1706.09516),1400.0,+62/-48,0.33,21.45,10.73,14.282,1055.6,0.063,βοΈ,0.0,False,CPU
|
| 16 |
-
14,π³,Tree-based,[CatBoost (default)](https://arxiv.org/abs/1706.09516),1398.0,+56/-53,0.345,21.52,10.94,14.164,4.27,0.064,βοΈ,0.0,False,CPU
|
| 17 |
-
15,π§ π,Neural Network,[RealMLP (tuned)](https://arxiv.org/abs/2407.04491),1396.0,+50/-50,0.299,21.67,11.74,14.127,2775.44,0.415,βοΈ,0.0,False,GPU
|
| 18 |
-
16,π³,Tree-based,[LightGBM (tuned)](https://papers.nips.cc/paper_files/paper/2017/hash/6449f44a102fde848669bdd9eb6b76fa-Abstract.html),1380.0,+47/-30,0.246,22.69,15.77,15.206,328.96,0.126,βοΈ,0.0,False,CPU
|
| 19 |
-
17,π³,Tree-based,[XGBoost (tuned + ensembled)](https://arxiv.org/abs/1603.02754),1379.0,+64/-51,0.283,22.77,13.01,15.251,472.99,0.621,βοΈ,0.0,False,CPU
|
| 20 |
-
18,π§ β‘,Foundation Model,[BetaTabPFN (default)](https://arxiv.org/abs/2502.02527),1377.0,+64/-54,0.308,22.92,12.37,16.226,250.36,1.708,β,0.0,False,GPU
|
| 21 |
-
19,π§ π,Neural Network,[ModernNCA (tuned)](https://arxiv.org/abs/2407.03257),1361.0,+54/-58,0.244,23.95,12.55,15.152,3383.91,0.51,βοΈ,0.0,False,GPU
|
| 22 |
-
20,π§ π,Neural Network,[ModernNCA (tuned + ensembled)](https://arxiv.org/abs/2407.03257),1357.0,+94/-94,0.346,24.24,10.97,15.235,3383.91,8.396,βοΈ,0.0,False,GPU
|
| 23 |
-
21,π§ β‘,Foundation Model,[TabDPT (tuned)](https://arxiv.org/abs/2410.18164),1349.0,+62/-53,0.268,24.77,9.54,13.746,4982.66,42.199,βοΈ,0.0,False,GPU
|
| 24 |
-
22,π³,Tree-based,[XGBoost (tuned)](https://arxiv.org/abs/1603.02754),1346.0,+47/-42,0.223,24.97,17.35,15.749,472.99,0.13,βοΈ,0.0,False,CPU
|
| 25 |
-
23,β,Other,[xRFM (tuned + ensembled)](https://arxiv.org/abs/2508.10053),1325.0,+57/-57,0.199,26.41,16.58,15.913,642.33,2.767,βοΈ,0.0,False,GPU
|
| 26 |
-
24,π§ π,Neural Network,[TorchMLP (tuned + ensembled)](https://arxiv.org/abs/2003.06505),1316.0,+46/-42,0.19,26.99,18.86,15.579,2241.65,2.415,βοΈ,0.0,False,CPU
|
| 27 |
-
25,π§ π,Neural Network,[TabM (default)](https://arxiv.org/abs/2410.24210),1311.0,+66/-59,0.231,27.33,15.91,16.346,8.72,0.126,βοΈ,0.0,False,GPU
|
| 28 |
-
26,π³,Tree-based,[EBM (tuned + ensembled)](https://www.cs.cornell.edu/~yinlou/papers/lou-kdd13.pdf),1308.0,+50/-44,0.155,27.52,19.29,17.519,1560.02,0.404,βοΈ,0.0,False,CPU
|
| 29 |
-
27,β,Other,[xRFM (tuned)](https://arxiv.org/abs/2508.10053),1299.0,+49/-59,0.165,28.18,17.0,17.102,642.33,0.146,βοΈ,0.0,False,GPU
|
| 30 |
-
28,π§ β‘,Foundation Model,[SAP-RPT-OSS (default)](https://arxiv.org/abs/2506.10707),1292.0,+64/-60,0.211,28.64,13.1,18.162,12.86,3.167,β,0.0,False,GPU
|
| 31 |
-
29,π§ β‘,Foundation Model,[TabDPT (default)](https://arxiv.org/abs/2410.18164),1277.0,+63/-73,0.206,29.68,13.02,16.675,49.67,43.824,βοΈ,0.0,False,GPU
|
| 32 |
-
30,π³,Tree-based,[EBM (tuned)](https://www.cs.cornell.edu/~yinlou/papers/lou-kdd13.pdf),1274.0,+47/-44,0.101,29.88,21.56,18.223,1560.02,0.052,βοΈ,0.0,False,CPU
|
| 33 |
-
31,π§ π,Neural Network,[ModernNCA (default)](https://arxiv.org/abs/2407.03257),1266.0,+47/-44,0.148,30.44,17.21,18.05,13.5,0.311,βοΈ,0.0,False,GPU
|
| 34 |
-
32,π§ π,Neural Network,[RealMLP (default)](https://arxiv.org/abs/2407.04491),1266.0,+41/-46,0.113,30.48,21.64,16.919,9.33,1.116,βοΈ,0.0,False,GPU
|
| 35 |
-
33,π§ π,Neural Network,[FastaiMLP (tuned + ensembled)](https://arxiv.org/abs/2003.06505),1265.0,+57/-64,0.143,30.49,19.4,17.247,559.38,4.08,βοΈ,0.0,False,CPU
|
| 36 |
-
34,π³,Tree-based,[EBM (default)](https://www.cs.cornell.edu/~yinlou/papers/lou-kdd13.pdf),1248.0,+48/-51,0.106,31.69,21.3,19.115,4.98,0.049,βοΈ,0.0,False,CPU
|
| 37 |
-
35,π³,Tree-based,[XGBoost (default)](https://arxiv.org/abs/1603.02754),1231.0,+62/-53,0.113,32.83,21.38,18.347,1.24,0.11,βοΈ,0.0,False,CPU
|
| 38 |
-
36,π§ π,Neural Network,[TorchMLP (tuned)](https://arxiv.org/abs/2003.06505),1229.0,+43/-38,0.086,33.02,24.16,17.69,2241.65,0.136,βοΈ,0.0,False,CPU
|
| 39 |
-
37,π§ π,Neural Network,[FastaiMLP (tuned)](https://arxiv.org/abs/2003.06505),1192.0,+55/-63,0.064,35.53,23.03,19.222,559.38,0.276,βοΈ,0.0,False,CPU
|
| 40 |
-
38,π³,Tree-based,[ExtraTrees (tuned + ensembled)](https://link.springer.com/article/10.1007/s10994-006-6226-1),1184.0,+50/-54,0.047,36.03,27.31,19.614,125.38,0.674,βοΈ,0.0,False,CPU
|
| 41 |
-
39,π³,Tree-based,[LightGBM (default)](https://papers.nips.cc/paper_files/paper/2017/hash/6449f44a102fde848669bdd9eb6b76fa-Abstract.html),1184.0,+48/-37,0.056,36.05,30.62,19.067,0.81,0.093,βοΈ,0.0,False,CPU
|
| 42 |
-
40,π§ β‘,Foundation Model,[TabSTAR (tuned)](https://arxiv.org/abs/2505.18125),1168.0,+73/-74,0.171,37.14,10.46,22.701,25373.34,3.667,βοΈ,0.0,False,GPU
|
| 43 |
-
41,π§ β‘,Foundation Model,[TabSTAR (tuned + ensembled)](https://arxiv.org/abs/2505.18125),1160.0,+72/-80,0.165,37.63,10.4,22.836,25373.34,14.078,βοΈ,0.0,False,GPU
|
| 44 |
-
42,π³,Tree-based,[RandomForest (tuned + ensembled)](https://link.springer.com/article/10.1023/A:1010933404324),1156.0,+48/-59,0.035,37.95,27.55,21.052,169.75,0.603,βοΈ,0.0,False,CPU
|
| 45 |
-
43,π³,Tree-based,[ExtraTrees (tuned)](https://link.springer.com/article/10.1007/s10994-006-6226-1),1148.0,+49/-52,0.029,38.46,29.78,20.729,125.38,0.069,βοΈ,0.0,False,CPU
|
| 46 |
-
44,π³,Tree-based,[RandomForest (tuned)](https://link.springer.com/article/10.1023/A:1010933404324),1122.0,+41/-55,0.024,40.07,30.9,21.742,169.75,0.059,βοΈ,0.0,False,CPU
|
| 47 |
-
45,π§ π,Neural Network,[TorchMLP (default)](https://arxiv.org/abs/2003.06505),1099.0,+56/-44,0.024,41.55,31.34,21.71,4.95,0.091,βοΈ,0.0,False,CPU
|
| 48 |
-
46,π³,Tree-based,[PerpetualBooster (tuned + ensembled)](https://perpetual-ml.com/),1098.0,+58/-74,0.047,41.59,28.28,23.287,117.83,0.773,βοΈ,0.0,False,CPU
|
| 49 |
-
47,π§ π,Neural Network,[FastaiMLP (default)](https://arxiv.org/abs/2003.06505),1063.0,+63/-81,0.027,43.7,28.95,23.032,2.56,0.267,βοΈ,0.0,False,CPU
|
| 50 |
-
48,π³,Tree-based,[PerpetualBooster (tuned)](https://perpetual-ml.com/),1062.0,+59/-74,0.028,43.73,30.88,24.692,117.83,0.29,βοΈ,0.0,False,CPU
|
| 51 |
-
49,π§ β‘,Foundation Model,[TabSTAR (default)](https://arxiv.org/abs/2505.18125),1060.0,+86/-103,0.11,43.87,17.17,27.666,391.1,4.052,βοΈ,0.0,False,GPU
|
| 52 |
-
50,π,Baseline,[Linear (tuned + ensembled)](https://scikit-learn.org/stable/modules/linear_model.html),1050.0,+84/-100,0.055,44.47,29.7,28.28,162.68,0.405,βοΈ,0.0,False,CPU
|
| 53 |
-
51,π§ β‘,Foundation Model,[TabFlex (default)](https://arxiv.org/abs/2506.05584),1043.0,+101/-107,0.084,44.83,27.52,27.118,0.85,0.129,β,0.0,False,GPU
|
| 54 |
-
52,π,Baseline,[Linear (tuned)](https://scikit-learn.org/stable/modules/linear_model.html),1024.0,+84/-112,0.038,45.89,31.3,28.935,162.68,0.054,βοΈ,0.0,False,CPU
|
| 55 |
-
53,π,Baseline,[KNN (tuned + ensembled)](https://scikit-learn.org/stable/modules/neighbors.html),1024.0,+59/-92,0.017,45.91,33.65,25.608,139.84,3.917,βοΈ,0.0,False,CPU
|
| 56 |
-
54,π³,Tree-based,[RandomForest (default)](https://link.springer.com/article/10.1023/A:1010933404324),1000.0,+62/-73,0.014,47.15,41.31,25.873,0.25,0.029,βοΈ,0.0,False,CPU
|
| 57 |
-
55,β,Other,[xRFM (default)](https://arxiv.org/abs/2508.10053),985.0,+73/-100,0.026,47.92,40.21,28.374,2.8,0.564,βοΈ,0.0,False,GPU
|
| 58 |
-
56,π,Baseline,[Linear (default)](https://scikit-learn.org/stable/modules/linear_model.html),966.0,+83/-107,0.024,48.84,34.55,30.816,0.84,0.094,βοΈ,0.0,False,CPU
|
| 59 |
-
57,π³,Tree-based,[ExtraTrees (default)](https://link.springer.com/article/10.1007/s10994-006-6226-1),959.0,+76/-94,0.009,49.14,44.03,27.217,0.2,0.037,βοΈ,0.0,False,CPU
|
| 60 |
-
58,π³,Tree-based,[PerpetualBooster (default)](https://perpetual-ml.com/),946.0,+56/-80,0.0,49.77,36.57,29.067,15.79,0.029,βοΈ,0.0,False,CPU
|
| 61 |
-
59,π,Baseline,[KNN (tuned)](https://scikit-learn.org/stable/modules/neighbors.html),920.0,+71/-126,0.003,50.89,38.05,30.607,139.84,0.193,βοΈ,0.0,False,CPU
|
| 62 |
-
60,π,Baseline,[KNN (default)](https://scikit-learn.org/stable/modules/neighbors.html),620.0,+102/-166,0.0,58.76,54.33,44.885,0.15,0.023,βοΈ,0.0,False,CPU
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
data/imputation_no/splits_all/tasks_binary/datasets_all/winrate_matrix.png.zip
DELETED
|
@@ -1,3 +0,0 @@
|
|
| 1 |
-
version https://git-lfs.github.com/spec/v1
|
| 2 |
-
oid sha256:e803d3d2d713d053f8486d77802556e46fa4e84b04a3ef81cb0860c0020cd241
|
| 3 |
-
size 506137
|
|
|
|
|
|
|
|
|
|
|
|
data/imputation_no/splits_all/tasks_binary/datasets_medium/n_datasets_9
DELETED
|
File without changes
|
data/imputation_no/splits_all/tasks_binary/datasets_medium/pareto_front_improvability_vs_time_infer.png.zip
DELETED
|
@@ -1,3 +0,0 @@
|
|
| 1 |
-
version https://git-lfs.github.com/spec/v1
|
| 2 |
-
oid sha256:bdfc756f5d102608f1aa2820c38f529f8d13a7ec6164bb6701dcb15931c592ed
|
| 3 |
-
size 519423
|
|
|
|
|
|
|
|
|
|
|
|
data/imputation_no/splits_all/tasks_binary/datasets_medium/pareto_n_configs_imp.png.zip
DELETED
|
@@ -1,3 +0,0 @@
|
|
| 1 |
-
version https://git-lfs.github.com/spec/v1
|
| 2 |
-
oid sha256:812e56ce81094bcc8d86cf5cb91413eaa593cd6adadd17246b83e8ae5bf51888
|
| 3 |
-
size 425042
|
|
|
|
|
|
|
|
|
|
|
|
data/imputation_no/splits_all/tasks_binary/datasets_medium/tuning-impact-elo.png.zip
DELETED
|
@@ -1,3 +0,0 @@
|
|
| 1 |
-
version https://git-lfs.github.com/spec/v1
|
| 2 |
-
oid sha256:4f2ec458a283febb87c79fea82d87ca56f254da25280b7694da00150613c2b64
|
| 3 |
-
size 107190
|
|
|
|
|
|
|
|
|
|
|
|
data/imputation_no/splits_all/tasks_binary/datasets_medium/website_leaderboard.csv
DELETED
|
@@ -1,63 +0,0 @@
|
|
| 1 |
-
#,Type,TypeName,Model,Elo [β¬οΈ],Elo 95% CI,Score [β¬οΈ],Rank [β¬οΈ],Harmonic Rank [β¬οΈ],Improvability (%) [β¬οΈ],Median Train Time (s/1K) [β¬οΈ],Median Predict Time (s/1K) [β¬οΈ],Verified,Imputed (%) [β¬οΈ],Imputed,Hardware
|
| 2 |
-
0,π,Reference Pipeline,"[AutoGluon 1.5 (extreme, 4h)](https://arxiv.org/abs/2003.06505)",2095.0,+165/-59,0.905,3.51,2.21,2.432,128.64,2.023,βοΈ,0.0,False,GPU
|
| 3 |
-
1,π,Reference Pipeline,"[AutoGluon 1.4 (best, 4h)](https://arxiv.org/abs/2003.06505)",1913.0,+190/-74,0.787,7.23,2.58,3.423,325.91,2.156,βοΈ,0.0,False,CPU
|
| 4 |
-
2,π§ β‘,Foundation Model,[TabPFN-2.6 (default)](https://arxiv.org/abs/2511.08667),1804.0,+141/-89,0.621,10.81,6.02,6.94,2.76,0.886,βοΈ,0.0,False,GPU
|
| 5 |
-
3,π§ β‘,Foundation Model,[TabICLv2 (default)](https://arxiv.org/abs/2602.11139),1784.0,+187/-121,0.603,11.58,3.0,5.237,0.84,0.202,βοΈ,0.0,False,GPU
|
| 6 |
-
4,π³,Tree-based,[CatBoost (tuned + ensembled)](https://arxiv.org/abs/1706.09516),1770.0,+154/-87,0.608,12.19,6.52,5.606,619.15,0.198,βοΈ,0.0,False,CPU
|
| 7 |
-
5,π³,Tree-based,[CatBoost (tuned)](https://arxiv.org/abs/1706.09516),1764.0,+160/-96,0.613,12.41,5.46,5.907,619.15,0.016,βοΈ,0.0,False,CPU
|
| 8 |
-
6,π§ π,Neural Network,[RealMLP (tuned + ensembled)](https://arxiv.org/abs/2407.04491),1761.0,+119/-43,0.553,12.56,9.14,6.321,1633.8,2.053,βοΈ,0.0,False,GPU
|
| 9 |
-
7,π³,Tree-based,[CatBoost (default)](https://arxiv.org/abs/1706.09516),1749.0,+143/-81,0.604,13.04,6.16,5.697,2.21,0.025,βοΈ,0.0,False,CPU
|
| 10 |
-
8,π³,Tree-based,[XGBoost (tuned + ensembled)](https://arxiv.org/abs/1603.02754),1727.0,+160/-94,0.505,14.0,7.73,8.4,110.13,0.386,βοΈ,0.0,False,CPU
|
| 11 |
-
9,π³,Tree-based,[LightGBM (tuned + ensembled)](https://papers.nips.cc/paper_files/paper/2017/hash/6449f44a102fde848669bdd9eb6b76fa-Abstract.html),1709.0,+159/-78,0.493,14.84,9.81,8.801,86.85,0.668,βοΈ,0.0,False,CPU
|
| 12 |
-
10,π§ π,Neural Network,[TabM (tuned + ensembled)](https://arxiv.org/abs/2410.24210),1691.0,+206/-125,0.473,15.68,6.27,8.164,804.65,0.71,βοΈ,0.0,False,GPU
|
| 13 |
-
11,π§ π,Neural Network,[ModernNCA (tuned + ensembled)](https://arxiv.org/abs/2407.03257),1682.0,+148/-136,0.498,16.1,8.96,7.855,2139.17,6.41,βοΈ,0.0,False,GPU
|
| 14 |
-
12,π§ β‘,Foundation Model,[RealTabPFN-2.5 (tuned + ensembled)](https://arxiv.org/abs/2511.08667),1662.0,+198/-115,0.443,17.06,8.17,9.222,842.24,15.119,βοΈ,0.0,False,GPU
|
| 15 |
-
13,π§ π,Neural Network,[RealMLP (tuned)](https://arxiv.org/abs/2407.04491),1662.0,+140/-65,0.412,17.1,10.14,7.81,1633.8,0.088,βοΈ,0.0,False,GPU
|
| 16 |
-
14,π³,Tree-based,[XGBoost (tuned)](https://arxiv.org/abs/1603.02754),1650.0,+123/-79,0.403,17.69,12.95,9.231,110.13,0.061,βοΈ,0.0,False,CPU
|
| 17 |
-
15,π§ π,Neural Network,[TabM (tuned)](https://arxiv.org/abs/2410.24210),1636.0,+204/-148,0.429,18.37,6.9,8.836,804.65,0.063,βοΈ,0.0,False,GPU
|
| 18 |
-
16,π§ β‘,Foundation Model,[RealTabPFN-2.5 (tuned)](https://arxiv.org/abs/2511.08667),1615.0,+195/-113,0.394,19.46,9.38,9.857,842.24,1.465,βοΈ,0.0,False,GPU
|
| 19 |
-
17,π§ β‘,Foundation Model,[RealTabPFN-2.5 (default)](https://arxiv.org/abs/2511.08667),1590.0,+158/-118,0.371,20.79,11.49,9.518,2.3,0.896,βοΈ,0.0,False,GPU
|
| 20 |
-
18,π³,Tree-based,[LightGBM (tuned)](https://papers.nips.cc/paper_files/paper/2017/hash/6449f44a102fde848669bdd9eb6b76fa-Abstract.html),1588.0,+150/-86,0.323,20.89,13.55,10.578,86.85,0.078,βοΈ,0.0,False,CPU
|
| 21 |
-
19,π§ β‘,Foundation Model,[TabICL (default)](https://arxiv.org/abs/2502.05564),1573.0,+171/-157,0.361,21.74,10.72,10.177,8.53,3.578,βοΈ,0.0,False,GPU
|
| 22 |
-
20,π§ π,Neural Network,[ModernNCA (tuned)](https://arxiv.org/abs/2407.03257),1560.0,+121/-76,0.266,22.43,15.29,10.591,2139.17,0.349,βοΈ,0.0,False,GPU
|
| 23 |
-
21,π§ β‘,Foundation Model,[TabDPT (tuned + ensembled)](https://arxiv.org/abs/2410.18164),1540.0,+173/-150,0.314,23.56,10.5,9.911,4860.88,475.791,βοΈ,0.0,False,GPU
|
| 24 |
-
22,π§ π,Neural Network,[TorchMLP (tuned + ensembled)](https://arxiv.org/abs/2003.06505),1537.0,+101/-108,0.24,23.68,17.28,9.926,1156.69,1.558,βοΈ,0.0,False,CPU
|
| 25 |
-
23,β,Other,[xRFM (tuned + ensembled)](https://arxiv.org/abs/2508.10053),1494.0,+162/-120,0.209,26.09,16.43,11.475,572.82,1.935,βοΈ,0.0,False,GPU
|
| 26 |
-
24,π³,Tree-based,[EBM (tuned + ensembled)](https://www.cs.cornell.edu/~yinlou/papers/lou-kdd13.pdf),1487.0,+134/-115,0.212,26.49,20.13,13.72,270.2,0.192,βοΈ,0.0,False,CPU
|
| 27 |
-
25,π³,Tree-based,[XGBoost (default)](https://arxiv.org/abs/1603.02754),1472.0,+133/-135,0.213,27.35,14.95,11.694,0.49,0.053,βοΈ,0.0,False,CPU
|
| 28 |
-
26,β,Other,[xRFM (tuned)](https://arxiv.org/abs/2508.10053),1464.0,+152/-127,0.182,27.85,20.57,13.423,572.82,0.109,βοΈ,0.0,False,GPU
|
| 29 |
-
27,π§ π,Neural Network,[TabM (default)](https://arxiv.org/abs/2410.24210),1460.0,+157/-170,0.229,28.05,16.53,11.377,5.5,0.074,βοΈ,0.0,False,GPU
|
| 30 |
-
28,π§ π,Neural Network,[RealMLP (default)](https://arxiv.org/abs/2407.04491),1441.0,+82/-52,0.11,29.14,26.15,11.644,6.41,0.247,βοΈ,0.0,False,GPU
|
| 31 |
-
29,π§ π,Neural Network,[FastaiMLP (tuned + ensembled)](https://arxiv.org/abs/2003.06505),1434.0,+121/-145,0.143,29.56,22.46,12.192,371.59,2.386,βοΈ,0.0,False,CPU
|
| 32 |
-
30,π§ οΏ½οΏ½,Foundation Model,[BetaTabPFN (default)](https://arxiv.org/abs/2502.02527),1426.0,+116/-139,0.135,30.02,23.46,14.853,96.61,0.754,β,0.0,False,GPU
|
| 33 |
-
31,π³,Tree-based,[EBM (tuned)](https://www.cs.cornell.edu/~yinlou/papers/lou-kdd13.pdf),1413.0,+142/-106,0.112,30.77,26.55,14.845,270.2,0.015,βοΈ,0.0,False,CPU
|
| 34 |
-
32,π§ π,Neural Network,[TorchMLP (tuned)](https://arxiv.org/abs/2003.06505),1390.0,+98/-117,0.081,32.07,25.92,12.235,1156.69,0.054,βοΈ,0.0,False,CPU
|
| 35 |
-
33,π§ β‘,Foundation Model,[TabDPT (tuned)](https://arxiv.org/abs/2410.18164),1374.0,+136/-141,0.102,32.99,21.41,13.494,4860.88,42.641,βοΈ,0.0,False,GPU
|
| 36 |
-
34,π³,Tree-based,[EBM (default)](https://www.cs.cornell.edu/~yinlou/papers/lou-kdd13.pdf),1368.0,+142/-120,0.091,33.36,28.93,16.624,1.07,0.018,βοΈ,0.0,False,CPU
|
| 37 |
-
35,π³,Tree-based,[LightGBM (default)](https://papers.nips.cc/paper_files/paper/2017/hash/6449f44a102fde848669bdd9eb6b76fa-Abstract.html),1355.0,+92/-123,0.083,34.1,30.86,13.259,0.29,0.033,βοΈ,0.0,False,CPU
|
| 38 |
-
36,π³,Tree-based,[RandomForest (tuned + ensembled)](https://link.springer.com/article/10.1023/A:1010933404324),1326.0,+60/-67,0.007,35.72,33.59,14.864,42.95,0.226,βοΈ,0.0,False,CPU
|
| 39 |
-
37,π§ π,Neural Network,[ModernNCA (default)](https://arxiv.org/abs/2407.03257),1320.0,+90/-80,0.027,36.05,32.83,14.764,8.34,0.25,βοΈ,0.0,False,GPU
|
| 40 |
-
38,π§ β‘,Foundation Model,[SAP-RPT-OSS (default)](https://arxiv.org/abs/2506.10707),1308.0,+79/-111,0.013,36.68,33.2,16.834,16.41,16.24,β,0.0,False,GPU
|
| 41 |
-
39,π³,Tree-based,[RandomForest (tuned)](https://link.springer.com/article/10.1023/A:1010933404324),1288.0,+76/-78,0.017,37.78,34.01,15.783,42.95,0.021,βοΈ,0.0,False,CPU
|
| 42 |
-
40,π³,Tree-based,[ExtraTrees (tuned + ensembled)](https://link.springer.com/article/10.1007/s10994-006-6226-1),1283.0,+68/-95,0.002,38.06,35.41,14.209,40.36,0.318,βοΈ,0.0,False,CPU
|
| 43 |
-
41,π§ β‘,Foundation Model,[TabDPT (default)](https://arxiv.org/abs/2410.18164),1282.0,+181/-185,0.065,38.08,25.8,17.722,47.03,45.876,βοΈ,0.0,False,GPU
|
| 44 |
-
42,π§ π,Neural Network,[FastaiMLP (tuned)](https://arxiv.org/abs/2003.06505),1282.0,+94/-124,0.041,38.14,32.24,15.655,371.59,0.102,βοΈ,0.0,False,CPU
|
| 45 |
-
43,π§ π,Neural Network,[TorchMLP (default)](https://arxiv.org/abs/2003.06505),1234.0,+68/-94,0.0,40.62,38.79,15.854,1.86,0.046,βοΈ,0.0,False,CPU
|
| 46 |
-
44,π³,Tree-based,[ExtraTrees (tuned)](https://link.springer.com/article/10.1007/s10994-006-6226-1),1224.0,+66/-81,0.0,41.16,39.25,15.818,40.36,0.028,βοΈ,0.0,False,CPU
|
| 47 |
-
45,π§ π,Neural Network,[FastaiMLP (default)](https://arxiv.org/abs/2003.06505),1210.0,+133/-205,0.047,41.86,35.34,18.624,1.52,0.115,βοΈ,0.0,False,CPU
|
| 48 |
-
46,π,Baseline,[Linear (tuned + ensembled)](https://scikit-learn.org/stable/modules/linear_model.html),1148.0,+179/-229,0.017,44.88,39.29,24.71,43.64,0.085,βοΈ,0.0,False,CPU
|
| 49 |
-
47,π,Baseline,[KNN (tuned + ensembled)](https://scikit-learn.org/stable/modules/neighbors.html),1135.0,+104/-178,0.012,45.47,37.02,21.414,513.06,78.02,βοΈ,0.0,False,CPU
|
| 50 |
-
48,π§ β‘,Foundation Model,[TabSTAR (tuned)](https://arxiv.org/abs/2505.18125),1135.0,+101/-121,0.0,45.49,43.6,24.018,3265.66,1.948,βοΈ,0.0,False,GPU
|
| 51 |
-
49,π§ β‘,Foundation Model,[TabSTAR (tuned + ensembled)](https://arxiv.org/abs/2505.18125),1125.0,+98/-130,0.0,45.96,44.13,24.103,3265.66,5.38,βοΈ,0.0,False,GPU
|
| 52 |
-
50,π,Baseline,[Linear (tuned)](https://scikit-learn.org/stable/modules/linear_model.html),1104.0,+186/-216,0.008,46.87,42.26,25.652,43.64,0.022,βοΈ,0.0,False,CPU
|
| 53 |
-
51,π³,Tree-based,[PerpetualBooster (tuned + ensembled)](https://perpetual-ml.com/),1056.0,+134/-223,0.002,48.93,41.93,22.623,20.26,0.641,βοΈ,0.0,False,CPU
|
| 54 |
-
52,π³,Tree-based,[PerpetualBooster (tuned)](https://perpetual-ml.com/),1016.0,+132/-242,0.0,50.49,46.02,23.796,20.26,0.258,βοΈ,0.0,False,CPU
|
| 55 |
-
53,π³,Tree-based,[RandomForest (default)](https://link.springer.com/article/10.1023/A:1010933404324),1000.0,+95/-217,0.0,51.09,50.07,22.929,0.12,0.014,βοΈ,0.0,False,CPU
|
| 56 |
-
54,π,Baseline,[Linear (default)](https://scikit-learn.org/stable/modules/linear_model.html),988.0,+137/-244,0.0,51.52,49.21,27.323,0.22,0.024,βοΈ,0.0,False,CPU
|
| 57 |
-
55,β,Other,[xRFM (default)](https://arxiv.org/abs/2508.10053),974.0,+189/-390,0.025,52.01,44.1,29.18,2.62,0.186,βοΈ,0.0,False,GPU
|
| 58 |
-
56,π§ β‘,Foundation Model,[TabSTAR (default)](https://arxiv.org/abs/2505.18125),954.0,+73/-132,0.0,52.7,51.87,29.853,101.07,2.021,βοΈ,0.0,False,GPU
|
| 59 |
-
57,π³,Tree-based,[ExtraTrees (default)](https://link.springer.com/article/10.1007/s10994-006-6226-1),934.0,+103/-238,0.0,53.35,52.69,24.795,0.1,0.021,βοΈ,0.0,False,CPU
|
| 60 |
-
58,π§ β‘,Foundation Model,[TabFlex (default)](https://arxiv.org/abs/2506.05584),920.0,+89/-168,0.0,53.79,52.68,30.981,0.24,0.036,β,0.0,False,GPU
|
| 61 |
-
59,π,Baseline,[KNN (tuned)](https://scikit-learn.org/stable/modules/neighbors.html),894.0,+58/-123,0.0,54.57,54.28,28.309,513.06,2.568,βοΈ,0.0,False,CPU
|
| 62 |
-
60,π³,Tree-based,[PerpetualBooster (default)](https://perpetual-ml.com/),820.0,+157/-291,0.0,56.51,55.44,32.967,1.21,0.03,βοΈ,0.0,False,CPU
|
| 63 |
-
61,π,Baseline,[KNN (default)](https://scikit-learn.org/stable/modules/neighbors.html),546.0,+108/-356,0.0,60.68,60.63,45.843,0.11,0.087,βοΈ,0.0,False,CPU
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
data/imputation_no/splits_all/tasks_binary/datasets_medium/winrate_matrix.png.zip
DELETED
|
@@ -1,3 +0,0 @@
|
|
| 1 |
-
version https://git-lfs.github.com/spec/v1
|
| 2 |
-
oid sha256:7e49f2642a251f8e9251e4c7dd666bd070bef27cd60e1fe8a1f2b745621a3f6a
|
| 3 |
-
size 522665
|
|
|
|
|
|
|
|
|
|
|
|
data/imputation_no/splits_all/tasks_binary/datasets_small/n_datasets_21
DELETED
|
File without changes
|
data/imputation_no/splits_all/tasks_binary/datasets_small/pareto_front_improvability_vs_time_infer.png.zip
DELETED
|
@@ -1,3 +0,0 @@
|
|
| 1 |
-
version https://git-lfs.github.com/spec/v1
|
| 2 |
-
oid sha256:68dfb1a9032f033870b58d22ae8ecf013f11b7f0c879e488bd97dfcdac62edce
|
| 3 |
-
size 528126
|
|
|
|
|
|
|
|
|
|
|
|
data/imputation_no/splits_all/tasks_binary/datasets_small/pareto_n_configs_imp.png.zip
DELETED
|
@@ -1,3 +0,0 @@
|
|
| 1 |
-
version https://git-lfs.github.com/spec/v1
|
| 2 |
-
oid sha256:8b9be27d52c2fbcef74b7f3ed96d8f7e10f9b2872be6ea37e50689282738c4b4
|
| 3 |
-
size 411577
|
|
|
|
|
|
|
|
|
|
|
|
data/imputation_no/splits_all/tasks_binary/datasets_small/tuning-impact-elo.png.zip
DELETED
|
@@ -1,3 +0,0 @@
|
|
| 1 |
-
version https://git-lfs.github.com/spec/v1
|
| 2 |
-
oid sha256:41df38f0472d91fc6600a02a286b2db9522f9e985593ff2d745e118918e12513
|
| 3 |
-
size 106749
|
|
|
|
|
|
|
|
|
|
|
|
data/imputation_no/splits_all/tasks_binary/datasets_small/website_leaderboard.csv
DELETED
|
@@ -1,62 +0,0 @@
|
|
| 1 |
-
#,Type,TypeName,Model,Elo [β¬οΈ],Elo 95% CI,Score [β¬οΈ],Rank [β¬οΈ],Harmonic Rank [β¬οΈ],Improvability (%) [β¬οΈ],Median Train Time (s/1K) [β¬οΈ],Median Predict Time (s/1K) [β¬οΈ],Verified,Imputed (%) [β¬οΈ],Imputed,Hardware
|
| 2 |
-
0,π,Reference Pipeline,"[AutoGluon 1.5 (extreme, 4h)](https://arxiv.org/abs/2003.06505)",1595.0,+127/-92,0.722,9.33,3.11,6.133,368.12,6.655,βοΈ,0.0,False,GPU
|
| 3 |
-
1,π§ β‘,Foundation Model,[TabICLv2 (default)](https://arxiv.org/abs/2602.11139),1577.0,+146/-87,0.757,10.02,3.15,8.27,5.02,0.381,βοΈ,0.0,False,GPU
|
| 4 |
-
2,π§ β‘,Foundation Model,[TabPFN-2.6 (default)](https://arxiv.org/abs/2511.08667),1564.0,+112/-68,0.706,10.54,4.52,10.358,5.48,0.433,βοΈ,0.0,False,GPU
|
| 5 |
-
3,π§ β‘,Foundation Model,[RealTabPFN-2.5 (default)](https://arxiv.org/abs/2511.08667),1561.0,+98/-48,0.676,10.66,5.79,10.893,5.96,0.554,βοΈ,0.0,False,GPU
|
| 6 |
-
4,π§ β‘,Foundation Model,[RealTabPFN-2.5 (tuned + ensembled)](https://arxiv.org/abs/2511.08667),1555.0,+137/-67,0.659,10.93,4.25,10.348,2052.28,8.252,βοΈ,0.0,False,GPU
|
| 7 |
-
5,π§ β‘,Foundation Model,[RealTabPFN-2.5 (tuned)](https://arxiv.org/abs/2511.08667),1530.0,+125/-57,0.626,12.05,4.99,11.048,2052.28,1.266,βοΈ,0.0,False,GPU
|
| 8 |
-
6,π§ π,Neural Network,[RealMLP (tuned + ensembled)](https://arxiv.org/abs/2407.04491),1423.0,+72/-46,0.412,17.7,11.57,15.086,2950.72,15.595,βοΈ,0.0,False,GPU
|
| 9 |
-
7,π§ π,Neural Network,[TabM (tuned + ensembled)](https://arxiv.org/abs/2410.24210),1414.0,+81/-54,0.391,18.26,10.13,15.312,2457.54,1.658,βοΈ,0.0,False,GPU
|
| 10 |
-
8,π,Reference Pipeline,"[AutoGluon 1.4 (best, 4h)](https://arxiv.org/abs/2003.06505)",1405.0,+88/-65,0.381,18.82,11.44,14.904,1921.63,1.718,βοΈ,0.0,False,CPU
|
| 11 |
-
9,π§ β‘,Foundation Model,[TabDPT (tuned + ensembled)](https://arxiv.org/abs/2410.18164),1392.0,+78/-69,0.392,19.62,7.52,13.174,5351.68,268.579,βοΈ,0.0,False,GPU
|
| 12 |
-
10,π§ β‘,Foundation Model,[BetaTabPFN (default)](https://arxiv.org/abs/2502.02527),1383.0,+82/-63,0.382,20.16,10.34,16.816,317.91,1.983,β,0.0,False,GPU
|
| 13 |
-
11,π³,Tree-based,[LightGBM (tuned + ensembled)](https://papers.nips.cc/paper_files/paper/2017/hash/6449f44a102fde848669bdd9eb6b76fa-Abstract.html),1361.0,+48/-38,0.271,21.58,15.12,16.763,512.57,1.18,βοΈ,0.0,False,CPU
|
| 14 |
-
12,π§ β‘,Foundation Model,[TabDPT (tuned)](https://arxiv.org/abs/2410.18164),1361.0,+83/-71,0.338,21.6,7.76,13.855,5351.68,37.621,βοΈ,0.0,False,GPU
|
| 15 |
-
13,π§ π,Neural Network,[TabM (tuned)](https://arxiv.org/abs/2410.24210),1355.0,+87/-59,0.319,21.98,12.1,16.252,2457.54,0.178,βοΈ,0.0,False,GPU
|
| 16 |
-
14,π³,Tree-based,[LightGBM (tuned)](https://papers.nips.cc/paper_files/paper/2017/hash/6449f44a102fde848669bdd9eb6b76fa-Abstract.html),1330.0,+42/-35,0.212,23.69,17.15,17.19,512.57,0.243,βοΈ,0.0,False,CPU
|
| 17 |
-
15,π§ π,Neural Network,[RealMLP (tuned)](https://arxiv.org/abs/2407.04491),1328.0,+75/-59,0.248,23.83,12.66,16.835,2950.72,1.225,βοΈ,0.0,False,GPU
|
| 18 |
-
16,π§ π,Neural Network,[ModernNCA (tuned)](https://arxiv.org/abs/2407.03257),1314.0,+86/-64,0.234,24.79,11.78,17.108,4806.04,0.512,βοΈ,0.0,False,GPU
|
| 19 |
-
17,π³,Tree-based,[CatBoost (tuned + ensembled)](https://arxiv.org/abs/1706.09516),1309.0,+50/-44,0.208,25.13,17.59,17.843,1096.5,0.584,βοΈ,0.0,False,CPU
|
| 20 |
-
18,π³,Tree-based,[CatBoost (default)](https://arxiv.org/abs/1706.09516),1306.0,+63/-45,0.233,25.28,16.65,17.794,4.81,0.08,βοΈ,0.0,False,CPU
|
| 21 |
-
19,π³,Tree-based,[CatBoost (tuned)](https://arxiv.org/abs/1706.09516),1304.0,+52/-44,0.208,25.45,18.45,17.872,1096.5,0.074,βοΈ,0.0,False,CPU
|
| 22 |
-
20,π§ β‘,Foundation Model,[SAP-RPT-OSS (default)](https://arxiv.org/abs/2506.10707),1303.0,+71/-74,0.296,25.51,10.43,18.732,9.05,1.763,β,0.0,False,GPU
|
| 23 |
-
21,π§ β‘,Foundation Model,[TabDPT (default)](https://arxiv.org/abs/2410.18164),1289.0,+86/-75,0.264,26.47,10.79,16.228,51.83,43.615,βοΈ,0.0,False,GPU
|
| 24 |
-
22,π³,Tree-based,[XGBoost (tuned + ensembled)](https://arxiv.org/abs/1603.02754),1286.0,+45/-42,0.187,26.72,18.94,18.188,693.49,1.398,βοΈ,0.0,False,CPU
|
| 25 |
-
23,β,Other,[xRFM (tuned + ensembled)](https://arxiv.org/abs/2508.10053),1285.0,+48/-65,0.194,26.81,16.74,17.817,711.84,2.983,βοΈ,0.0,False,GPU
|
| 26 |
-
24,π§ π,Neural Network,[TabM (default)](https://arxiv.org/abs/2410.24210),1278.0,+63/-58,0.231,27.27,15.74,18.477,10.08,0.152,βοΈ,0.0,False,GPU
|
| 27 |
-
25,π§ π,Neural Network,[ModernNCA (tuned + ensembled)](https://arxiv.org/abs/2407.03257),1269.0,+108/-105,0.28,27.9,12.36,18.4,4806.04,8.461,βοΈ,0.0,False,GPU
|
| 28 |
-
26,π³,Tree-based,[EBM (tuned + ensembled)](https://www.cs.cornell.edu/~yinlou/papers/lou-kdd13.pdf),1264.0,+44/-45,0.129,28.25,19.09,19.148,1634.54,0.422,βοΈ,0.0,False,CPU
|
| 29 |
-
27,π³,Tree-based,[XGBoost (tuned)](https://arxiv.org/abs/1603.02754),1263.0,+40/-33,0.144,28.31,20.52,18.543,693.49,0.194,βοΈ,0.0,False,CPU
|
| 30 |
-
28,π§ π,Neural Network,[ModernNCA (default)](https://arxiv.org/abs/2407.03257),1262.0,+58/-60,0.2,28.4,14.37,19.459,14.37,0.334,βοΈ,0.0,False,GPU
|
| 31 |
-
29,β,Other,[xRFM (tuned)](https://arxiv.org/abs/2508.10053),1259.0,+50/-60,0.157,28.61,15.92,18.68,711.84,0.162,βοΈ,0.0,False,GPU
|
| 32 |
-
30,π§ π,Neural Network,[TorchMLP (tuned + ensembled)](https://arxiv.org/abs/2003.06505),1258.0,+79/-54,0.166,28.66,19.89,18.003,2665.36,2.619,βοΈ,0.0,False,CPU
|
| 33 |
-
31,π³,Tree-based,[EBM (tuned)](https://www.cs.cornell.edu/~yinlou/papers/lou-kdd13.pdf),1242.0,+46/-46,0.096,29.81,20.1,19.672,1634.54,0.054,βοΈ,0.0,False,CPU
|
| 34 |
-
32,π§ π,Neural Network,[FastaiMLP (tuned + ensembled)](https://arxiv.org/abs/2003.06505),1223.0,+62/-75,0.141,31.17,18.45,19.415,644.55,5.974,βοΈ,0.0,False,CPU
|
| 35 |
-
33,π³,Tree-based,[EBM (default)](https://www.cs.cornell.edu/~yinlou/papers/lou-kdd13.pdf),1222.0,+48/-51,0.111,31.29,19.29,20.184,7.32,0.054,βοΈ,0.0,False,CPU
|
| 36 |
-
34,π§ π,Neural Network,[RealMLP (default)](https://arxiv.org/abs/2407.04491),1221.0,+51/-46,0.115,31.33,20.3,19.181,10.89,2.249,βοΈ,0.0,False,GPU
|
| 37 |
-
35,π§ π,Neural Network,[TorchMLP (tuned)](https://arxiv.org/abs/2003.06505),1188.0,+67/-48,0.089,33.71,23.68,20.029,2665.36,0.143,βοΈ,0.0,False,CPU
|
| 38 |
-
36,π§ β‘,Foundation Model,[TabSTAR (tuned)](https://arxiv.org/abs/2505.18125),1184.0,+116/-91,0.245,33.97,7.9,22.137,34059.64,3.91,βοΈ,0.0,False,GPU
|
| 39 |
-
37,π§ β‘,Foundation Model,[TabSTAR (tuned + ensembled)](https://arxiv.org/abs/2505.18125),1177.0,+118/-90,0.236,34.47,7.85,22.294,34059.64,18.819,βοΈ,0.0,False,GPU
|
| 40 |
-
38,π§ π,Neural Network,[FastaiMLP (tuned)](https://arxiv.org/abs/2003.06505),1174.0,+61/-79,0.074,34.7,20.61,20.753,644.55,0.398,βοΈ,0.0,False,CPU
|
| 41 |
-
39,π³,Tree-based,[XGBoost (default)](https://arxiv.org/abs/1603.02754),1163.0,+56/-54,0.071,35.48,28.47,21.199,1.94,0.121,βοΈ,0.0,False,CPU
|
| 42 |
-
40,π³,Tree-based,[ExtraTrees (tuned + ensembled)](https://link.springer.com/article/10.1007/s10994-006-6226-1),1162.0,+59/-73,0.067,35.5,25.04,21.931,203.15,0.968,βοΈ,0.0,False,CPU
|
| 43 |
-
41,π³,Tree-based,[LightGBM (default)](https://papers.nips.cc/paper_files/paper/2017/hash/6449f44a102fde848669bdd9eb6b76fa-Abstract.html),1137.0,+44/-51,0.042,37.26,30.93,21.558,2.44,0.121,βοΈ,0.0,False,CPU
|
| 44 |
-
42,π³,Tree-based,[ExtraTrees (tuned)](https://link.springer.com/article/10.1007/s10994-006-6226-1),1131.0,+55/-76,0.041,37.65,27.16,22.836,203.15,0.091,βοΈ,0.0,False,CPU
|
| 45 |
-
43,π³,Tree-based,[PerpetualBooster (tuned + ensembled)](https://perpetual-ml.com/),1113.0,+58/-70,0.066,38.86,25.07,23.572,218.34,0.904,βοΈ,0.0,False,CPU
|
| 46 |
-
44,π³,Tree-based,[RandomForest (tuned + ensembled)](https://link.springer.com/article/10.1023/A:1010933404324),1108.0,+55/-84,0.048,39.24,25.78,23.706,484.22,1.025,βοΈ,0.0,False,CPU
|
| 47 |
-
45,π§ β‘,Foundation Model,[TabSTAR (default)](https://arxiv.org/abs/2505.18125),1088.0,+126/-107,0.158,40.51,13.37,26.73,460.02,6.641,βοΈ,0.0,False,GPU
|
| 48 |
-
46,π³,Tree-based,[PerpetualBooster (tuned)](https://perpetual-ml.com/),1077.0,+53/-76,0.041,41.26,27.25,25.076,218.34,0.322,βοΈ,0.0,False,CPU
|
| 49 |
-
47,π³,Tree-based,[RandomForest (tuned)](https://link.springer.com/article/10.1023/A:1010933404324),1075.0,+46/-83,0.026,41.39,30.08,24.298,484.22,0.086,βοΈ,0.0,False,CPU
|
| 50 |
-
48,π§ β‘,Foundation Model,[TabFlex (default)](https://arxiv.org/abs/2506.05584),1075.0,+104/-126,0.119,41.4,22.93,25.463,1.3,0.15,β,0.0,False,GPU
|
| 51 |
-
49,π§ π,Neural Network,[TorchMLP (default)](https://arxiv.org/abs/2003.06505),1061.0,+62/-79,0.034,42.29,29.13,24.221,9.39,0.13,βοΈ,0.0,False,CPU
|
| 52 |
-
50,π,Baseline,[Linear (tuned + ensembled)](https://scikit-learn.org/stable/modules/linear_model.html),1022.0,+92/-130,0.072,44.61,26.99,29.811,278.97,0.873,βοΈ,0.0,False,CPU
|
| 53 |
-
51,π§ π,Neural Network,[FastaiMLP (default)](https://arxiv.org/abs/2003.06505),1019.0,+72/-83,0.018,44.82,27.03,24.922,2.97,0.388,βοΈ,0.0,False,CPU
|
| 54 |
-
52,π,Baseline,[Linear (tuned)](https://scikit-learn.org/stable/modules/linear_model.html),1002.0,+97/-128,0.051,45.79,28.26,30.343,278.97,0.103,βοΈ,0.0,False,CPU
|
| 55 |
-
53,π³,Tree-based,[RandomForest (default)](https://link.springer.com/article/10.1023/A:1010933404324),1000.0,+55/-114,0.02,45.89,38.69,27.136,0.42,0.041,βοΈ,0.0,False,CPU
|
| 56 |
-
54,π,Baseline,[KNN (tuned + ensembled)](https://scikit-learn.org/stable/modules/neighbors.html),990.0,+78/-112,0.019,46.46,32.59,27.406,90.71,1.623,βοΈ,0.0,False,CPU
|
| 57 |
-
55,β,Other,[xRFM (default)](https://arxiv.org/abs/2508.10053),988.0,+81/-106,0.026,46.57,38.99,28.03,3.71,0.919,βοΈ,0.0,False,GPU
|
| 58 |
-
56,π³,Tree-based,[PerpetualBooster (default)](https://perpetual-ml.com/),974.0,+58/-64,0.0,47.31,32.07,27.396,30.82,0.029,βοΈ,0.0,False,CPU
|
| 59 |
-
57,π³,Tree-based,[ExtraTrees (default)](https://link.springer.com/article/10.1007/s10994-006-6226-1),965.0,+75/-126,0.013,47.77,41.41,28.255,0.33,0.043,βοΈ,0.0,False,CPU
|
| 60 |
-
58,π,Baseline,[Linear (default)](https://scikit-learn.org/stable/modules/linear_model.html),959.0,+91/-158,0.033,48.09,30.78,32.314,1.48,0.139,βοΈ,0.0,False,CPU
|
| 61 |
-
59,π,Baseline,[KNN (tuned)](https://scikit-learn.org/stable/modules/neighbors.html),925.0,+81/-137,0.004,49.74,33.89,31.593,90.71,0.125,βοΈ,0.0,False,CPU
|
| 62 |
-
60,π,Baseline,[KNN (default)](https://scikit-learn.org/stable/modules/neighbors.html),632.0,+105/-188,0.0,58.37,52.33,44.475,0.22,0.02,βοΈ,0.0,False,CPU
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
data/imputation_no/splits_all/tasks_binary/datasets_small/winrate_matrix.png.zip
DELETED
|
@@ -1,3 +0,0 @@
|
|
| 1 |
-
version https://git-lfs.github.com/spec/v1
|
| 2 |
-
oid sha256:5dda2f7ef5f28af31438e6852224fa9cdcdad52b47dcd18aad5ec21ea95c14eb
|
| 3 |
-
size 515812
|
|
|
|
|
|
|
|
|
|
|
|
data/imputation_no/splits_all/tasks_classification/datasets_all/n_datasets_38
DELETED
|
File without changes
|
data/imputation_no/splits_all/tasks_classification/datasets_all/pareto_front_improvability_vs_time_infer.png.zip
DELETED
|
@@ -1,3 +0,0 @@
|
|
| 1 |
-
version https://git-lfs.github.com/spec/v1
|
| 2 |
-
oid sha256:d184c708808d08e20439e8a9a37d90bc19e93b2abb6ddcf0f1664315f8cda370
|
| 3 |
-
size 481149
|
|
|
|
|
|
|
|
|
|
|
|
data/imputation_no/splits_all/tasks_classification/datasets_all/pareto_n_configs_imp.png.zip
DELETED
|
@@ -1,3 +0,0 @@
|
|
| 1 |
-
version https://git-lfs.github.com/spec/v1
|
| 2 |
-
oid sha256:02a727dc80bc1db190559fa3dc1b358fd62fe55cf8b33f59c6d3708c83d383de
|
| 3 |
-
size 429101
|
|
|
|
|
|
|
|
|
|
|
|
data/imputation_no/splits_all/tasks_classification/datasets_all/tuning-impact-elo.png.zip
DELETED
|
@@ -1,3 +0,0 @@
|
|
| 1 |
-
version https://git-lfs.github.com/spec/v1
|
| 2 |
-
oid sha256:eb6460eadb73d4e2bbfe610f1437ded0a50876c40b752da9640962fa3404d652
|
| 3 |
-
size 101629
|
|
|
|
|
|
|
|
|
|
|
|
data/imputation_no/splits_all/tasks_classification/datasets_all/website_leaderboard.csv
DELETED
|
@@ -1,62 +0,0 @@
|
|
| 1 |
-
#,Type,TypeName,Model,Elo [β¬οΈ],Elo 95% CI,Score [β¬οΈ],Rank [β¬οΈ],Harmonic Rank [β¬οΈ],Improvability (%) [β¬οΈ],Median Train Time (s/1K) [β¬οΈ],Median Predict Time (s/1K) [β¬οΈ],Verified,Imputed (%) [β¬οΈ],Imputed,Hardware
|
| 2 |
-
0,π,Reference Pipeline,"[AutoGluon 1.5 (extreme, 4h)](https://arxiv.org/abs/2003.06505)",1689.0,+107/-85,0.791,7.43,2.82,5.82,267.31,3.983,βοΈ,0.0,False,GPU
|
| 3 |
-
1,π§ β‘,Foundation Model,[TabPFN-2.6 (default)](https://arxiv.org/abs/2511.08667),1604.0,+70/-69,0.69,10.48,5.02,9.903,5.17,0.539,βοΈ,0.0,False,GPU
|
| 4 |
-
2,π§ β‘,Foundation Model,[TabICLv2 (default)](https://arxiv.org/abs/2602.11139),1589.0,+94/-76,0.701,11.12,3.28,8.535,4.15,0.412,βοΈ,0.0,False,GPU
|
| 5 |
-
3,π§ β‘,Foundation Model,[RealTabPFN-2.5 (tuned + ensembled)](https://arxiv.org/abs/2511.08667),1578.0,+78/-76,0.644,11.6,4.22,9.634,2046.25,8.976,βοΈ,0.0,False,GPU
|
| 6 |
-
4,π§ β‘,Foundation Model,[RealTabPFN-2.5 (tuned)](https://arxiv.org/abs/2511.08667),1553.0,+74/-68,0.614,12.73,5.14,10.411,2046.25,1.329,βοΈ,0.0,False,GPU
|
| 7 |
-
5,π§ β‘,Foundation Model,[RealTabPFN-2.5 (default)](https://arxiv.org/abs/2511.08667),1540.0,+68/-62,0.603,13.38,6.41,10.59,5.76,0.794,βοΈ,0.0,False,GPU
|
| 8 |
-
6,π,Reference Pipeline,"[AutoGluon 1.4 (best, 4h)](https://arxiv.org/abs/2003.06505)",1502.0,+74/-57,0.496,15.31,5.8,12.294,1684.49,1.97,βοΈ,0.0,False,CPU
|
| 9 |
-
7,π§ π,Neural Network,[RealMLP (tuned + ensembled)](https://arxiv.org/abs/2407.04491),1488.0,+62/-45,0.467,16.08,9.95,12.94,2879.46,12.49,βοΈ,0.0,False,GPU
|
| 10 |
-
8,π§ π,Neural Network,[TabM (tuned + ensembled)](https://arxiv.org/abs/2410.24210),1462.0,+71/-50,0.422,17.55,8.91,14.175,2466.21,1.502,βοΈ,0.0,False,GPU
|
| 11 |
-
9,π³,Tree-based,[LightGBM (tuned + ensembled)](https://papers.nips.cc/paper_files/paper/2017/hash/6449f44a102fde848669bdd9eb6b76fa-Abstract.html),1431.0,+52/-36,0.352,19.37,13.19,15.0,382.05,1.488,βοΈ,0.0,False,CPU
|
| 12 |
-
10,π§ π,Neural Network,[TabM (tuned)](https://arxiv.org/abs/2410.24210),1409.0,+65/-60,0.36,20.71,9.99,15.022,2466.21,0.176,βοΈ,0.0,False,GPU
|
| 13 |
-
11,π§ β‘,Foundation Model,[TabDPT (tuned + ensembled)](https://arxiv.org/abs/2410.18164),1409.0,+80/-55,0.383,20.72,8.44,13.819,4940.61,307.751,βοΈ,0.0,False,GPU
|
| 14 |
-
12,π§ π,Neural Network,[RealMLP (tuned)](https://arxiv.org/abs/2407.04491),1408.0,+55/-48,0.336,20.76,11.27,14.469,2879.46,0.598,βοΈ,0.0,False,GPU
|
| 15 |
-
13,π³,Tree-based,[CatBoost (tuned + ensembled)](https://arxiv.org/abs/1706.09516),1408.0,+56/-47,0.341,20.8,12.29,14.643,1372.94,0.556,βοΈ,0.0,False,CPU
|
| 16 |
-
14,π³,Tree-based,[CatBoost (tuned)](https://arxiv.org/abs/1706.09516),1400.0,+56/-45,0.332,21.33,11.65,14.866,1372.94,0.074,βοΈ,0.0,False,CPU
|
| 17 |
-
15,π³,Tree-based,[LightGBM (tuned)](https://papers.nips.cc/paper_files/paper/2017/hash/6449f44a102fde848669bdd9eb6b76fa-Abstract.html),1388.0,+42/-35,0.275,22.05,15.73,15.704,382.05,0.254,βοΈ,0.0,False,CPU
|
| 18 |
-
16,π³,Tree-based,[XGBoost (tuned + ensembled)](https://arxiv.org/abs/1603.02754),1378.0,+52/-48,0.288,22.71,13.8,15.849,685.87,1.455,βοΈ,0.0,False,CPU
|
| 19 |
-
17,π³,Tree-based,[CatBoost (default)](https://arxiv.org/abs/1706.09516),1377.0,+47/-44,0.315,22.74,12.36,15.347,5.72,0.076,βοΈ,0.0,False,CPU
|
| 20 |
-
18,π§ β‘,Foundation Model,[BetaTabPFN (default)](https://arxiv.org/abs/2502.02527),1370.0,+50/-49,0.314,23.21,13.22,17.241,282.72,1.886,β,0.0,False,GPU
|
| 21 |
-
19,π§ π,Neural Network,[ModernNCA (tuned)](https://arxiv.org/abs/2407.03257),1362.0,+57/-40,0.247,23.75,12.54,15.784,4879.89,0.525,βοΈ,0.0,False,GPU
|
| 22 |
-
20,π§ π,Neural Network,[ModernNCA (tuned + ensembled)](https://arxiv.org/abs/2407.03257),1360.0,+87/-81,0.33,23.91,11.77,15.764,4879.89,8.744,βοΈ,0.0,False,GPU
|
| 23 |
-
21,π³,Tree-based,[XGBoost (tuned)](https://arxiv.org/abs/1603.02754),1353.0,+48/-42,0.241,24.37,17.42,16.22,685.87,0.205,βοΈ,0.0,False,CPU
|
| 24 |
-
22,π§ β‘,Foundation Model,[TabDPT (tuned)](https://arxiv.org/abs/2410.18164),1349.0,+65/-57,0.286,24.62,9.81,15.339,4940.61,41.606,βοΈ,0.0,False,GPU
|
| 25 |
-
23,β,Other,[xRFM (tuned + ensembled)](https://arxiv.org/abs/2508.10053),1329.0,+44/-47,0.221,25.94,16.35,16.912,934.1,3.063,βοΈ,0.0,False,GPU
|
| 26 |
-
24,π³,Tree-based,[EBM (tuned + ensembled)](https://www.cs.cornell.edu/~yinlou/papers/lou-kdd13.pdf),1313.0,+39/-34,0.175,26.98,19.27,18.003,2686.11,0.47,βοΈ,0.0,False,CPU
|
| 27 |
-
25,π§ π,Neural Network,[TabM (default)](https://arxiv.org/abs/2410.24210),1312.0,+57/-50,0.243,27.09,16.01,17.45,10.21,0.138,βοΈ,0.0,False,GPU
|
| 28 |
-
26,π§ π,Neural Network,[TorchMLP (tuned + ensembled)](https://arxiv.org/abs/2003.06505),1310.0,+44/-41,0.185,27.21,19.31,16.776,2389.22,2.158,βοΈ,0.0,False,CPU
|
| 29 |
-
27,π§ β‘,Foundation Model,[SAP-RPT-OSS (default)](https://arxiv.org/abs/2506.10707),1297.0,+62/-73,0.254,28.09,10.25,18.354,13.7,2.033,β,0.0,False,GPU
|
| 30 |
-
28,β,Other,[xRFM (tuned)](https://arxiv.org/abs/2508.10053),1290.0,+52/-52,0.168,28.6,17.08,18.334,934.1,0.169,βοΈ,0.0,False,GPU
|
| 31 |
-
29,π³,Tree-based,[EBM (tuned)](https://www.cs.cornell.edu/~yinlou/papers/lou-kdd13.pdf),1277.0,+40/-39,0.121,29.43,21.47,18.716,2686.11,0.054,βοΈ,0.0,False,CPU
|
| 32 |
-
30,π§ οΏ½οΏ½,Foundation Model,[TabDPT (default)](https://arxiv.org/abs/2410.18164),1269.0,+61/-56,0.208,30.0,12.29,18.204,49.21,43.824,βοΈ,0.0,False,GPU
|
| 33 |
-
31,π§ π,Neural Network,[FastaiMLP (tuned + ensembled)](https://arxiv.org/abs/2003.06505),1260.0,+60/-62,0.166,30.61,17.63,18.823,618.9,4.766,βοΈ,0.0,False,CPU
|
| 34 |
-
32,π³,Tree-based,[EBM (default)](https://www.cs.cornell.edu/~yinlou/papers/lou-kdd13.pdf),1250.0,+41/-43,0.119,31.27,21.67,19.779,6.86,0.057,βοΈ,0.0,False,CPU
|
| 35 |
-
33,π§ π,Neural Network,[ModernNCA (default)](https://arxiv.org/abs/2407.03257),1249.0,+46/-44,0.126,31.39,18.75,19.733,14.78,0.346,βοΈ,0.0,False,GPU
|
| 36 |
-
34,π§ π,Neural Network,[RealMLP (default)](https://arxiv.org/abs/2407.04491),1241.0,+38/-34,0.089,31.91,23.47,18.973,10.47,1.707,βοΈ,0.0,False,GPU
|
| 37 |
-
35,π³,Tree-based,[XGBoost (default)](https://arxiv.org/abs/1603.02754),1230.0,+47/-50,0.106,32.63,22.55,18.957,1.77,0.117,βοΈ,0.0,False,CPU
|
| 38 |
-
36,π§ π,Neural Network,[TorchMLP (tuned)](https://arxiv.org/abs/2003.06505),1228.0,+45/-36,0.098,32.8,23.87,18.902,2389.22,0.152,βοΈ,0.0,False,CPU
|
| 39 |
-
37,π³,Tree-based,[ExtraTrees (tuned + ensembled)](https://link.springer.com/article/10.1007/s10994-006-6226-1),1207.0,+53/-52,0.094,34.18,22.52,20.247,189.76,0.743,βοΈ,0.0,False,CPU
|
| 40 |
-
38,π§ π,Neural Network,[FastaiMLP (tuned)](https://arxiv.org/abs/2003.06505),1191.0,+53/-58,0.08,35.25,22.21,20.612,618.9,0.298,βοΈ,0.0,False,CPU
|
| 41 |
-
39,π³,Tree-based,[LightGBM (default)](https://papers.nips.cc/paper_files/paper/2017/hash/6449f44a102fde848669bdd9eb6b76fa-Abstract.html),1190.0,+48/-40,0.075,35.34,29.31,19.944,1.79,0.12,βοΈ,0.0,False,CPU
|
| 42 |
-
40,π³,Tree-based,[RandomForest (tuned + ensembled)](https://link.springer.com/article/10.1023/A:1010933404324),1185.0,+61/-57,0.091,35.7,21.7,21.247,323.74,0.743,βοΈ,0.0,False,CPU
|
| 43 |
-
41,π³,Tree-based,[ExtraTrees (tuned)](https://link.springer.com/article/10.1007/s10994-006-6226-1),1170.0,+55/-52,0.068,36.7,23.15,21.427,189.76,0.079,βοΈ,0.0,False,CPU
|
| 44 |
-
42,π³,Tree-based,[RandomForest (tuned)](https://link.springer.com/article/10.1023/A:1010933404324),1149.0,+58/-57,0.069,38.04,22.02,22.081,323.74,0.076,βοΈ,0.0,False,CPU
|
| 45 |
-
43,π§ β‘,Foundation Model,[TabSTAR (tuned)](https://arxiv.org/abs/2505.18125),1136.0,+70/-81,0.135,38.88,12.43,24.546,29479.47,4.09,βοΈ,0.0,False,GPU
|
| 46 |
-
44,π§ β‘,Foundation Model,[TabSTAR (tuned + ensembled)](https://arxiv.org/abs/2505.18125),1130.0,+69/-84,0.131,39.23,12.33,24.631,29479.47,18.318,βοΈ,0.0,False,GPU
|
| 47 |
-
45,π§ π,Neural Network,[TorchMLP (default)](https://arxiv.org/abs/2003.06505),1090.0,+41/-50,0.019,41.72,31.77,23.336,6.83,0.147,βοΈ,0.0,False,CPU
|
| 48 |
-
46,π³,Tree-based,[PerpetualBooster (tuned + ensembled)](https://perpetual-ml.com/),1062.0,+55/-60,0.037,43.36,30.58,28.957,187.0,0.955,βοΈ,0.0,False,CPU
|
| 49 |
-
47,π§ π,Neural Network,[FastaiMLP (default)](https://arxiv.org/abs/2003.06505),1040.0,+62/-66,0.024,44.58,31.32,25.558,2.91,0.368,βοΈ,0.0,False,CPU
|
| 50 |
-
48,π,Baseline,[Linear (tuned + ensembled)](https://scikit-learn.org/stable/modules/linear_model.html),1035.0,+85/-84,0.043,44.91,31.87,30.889,252.58,0.528,βοΈ,0.0,False,CPU
|
| 51 |
-
49,π³,Tree-based,[PerpetualBooster (tuned)](https://perpetual-ml.com/),1025.0,+57/-60,0.022,45.43,31.58,30.455,187.0,0.338,βοΈ,0.0,False,CPU
|
| 52 |
-
50,π§ β‘,Foundation Model,[TabSTAR (default)](https://arxiv.org/abs/2505.18125),1024.0,+78/-102,0.087,45.52,19.93,30.312,439.44,5.886,βοΈ,0.0,False,GPU
|
| 53 |
-
51,π,Baseline,[KNN (tuned + ensembled)](https://scikit-learn.org/stable/modules/neighbors.html),1017.0,+68/-92,0.027,45.85,31.01,28.592,139.84,3.615,βοΈ,0.0,False,CPU
|
| 54 |
-
52,π§ β‘,Foundation Model,[TabFlex (default)](https://arxiv.org/abs/2506.05584),1012.0,+80/-94,0.066,46.13,30.44,29.832,1.11,0.189,β,0.0,False,GPU
|
| 55 |
-
53,π,Baseline,[Linear (tuned)](https://scikit-learn.org/stable/modules/linear_model.html),1008.0,+84/-84,0.03,46.35,33.47,31.597,252.58,0.089,βοΈ,0.0,False,CPU
|
| 56 |
-
54,β,Other,[xRFM (default)](https://arxiv.org/abs/2508.10053),1006.0,+69/-87,0.032,46.46,38.94,28.769,3.61,0.939,βοΈ,0.0,False,GPU
|
| 57 |
-
55,π³,Tree-based,[RandomForest (default)](https://link.springer.com/article/10.1023/A:1010933404324),1000.0,+57/-58,0.011,46.75,41.31,28.033,0.38,0.037,βοΈ,0.0,False,CPU
|
| 58 |
-
56,π³,Tree-based,[ExtraTrees (default)](https://link.springer.com/article/10.1007/s10994-006-6226-1),950.0,+72/-71,0.007,49.18,44.43,30.143,0.25,0.041,βοΈ,0.0,False,CPU
|
| 59 |
-
57,π,Baseline,[Linear (default)](https://scikit-learn.org/stable/modules/linear_model.html),930.0,+87/-98,0.019,50.05,37.38,34.335,1.29,0.131,βοΈ,0.0,False,CPU
|
| 60 |
-
58,π³,Tree-based,[PerpetualBooster (default)](https://perpetual-ml.com/),921.0,+57/-75,0.0,50.43,34.93,34.834,26.06,0.041,βοΈ,0.0,False,CPU
|
| 61 |
-
59,π,Baseline,[KNN (tuned)](https://scikit-learn.org/stable/modules/neighbors.html),902.0,+87/-105,0.012,51.23,38.39,34.68,139.84,0.193,βοΈ,0.0,False,CPU
|
| 62 |
-
60,π,Baseline,[KNN (default)](https://scikit-learn.org/stable/modules/neighbors.html),582.0,+99/-106,0.0,59.08,55.47,50.328,0.18,0.029,βοΈ,0.0,False,CPU
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|