Spaces:
Running
Running
Taha Aksu
commited on
Commit
·
b6bb7c3
1
Parent(s):
a88a326
Change repro code availability for most deep learning and statistical models to No
Browse files- results/DLinear/config.json +1 -1
- results/N-BEATS/config.json +1 -1
- results/PatchTST/config.json +1 -1
- results/auto_arima/config.json +1 -1
- results/auto_ets/config.json +1 -1
- results/auto_theta/config.json +2 -2
- results/crossformer/config.json +1 -1
- results/deepar/config.json +1 -1
- results/tft/config.json +1 -1
- results/tide/config.json +1 -1
- src/about.py +3 -0
results/DLinear/config.json
CHANGED
|
@@ -4,5 +4,5 @@
|
|
| 4 |
"model_dtype": "float32",
|
| 5 |
"org": "The Chinese University of Hong Kong",
|
| 6 |
"testdata_leakage": "No",
|
| 7 |
-
"replication_code_available": "
|
| 8 |
}
|
|
|
|
| 4 |
"model_dtype": "float32",
|
| 5 |
"org": "The Chinese University of Hong Kong",
|
| 6 |
"testdata_leakage": "No",
|
| 7 |
+
"replication_code_available": "No"
|
| 8 |
}
|
results/N-BEATS/config.json
CHANGED
|
@@ -4,5 +4,5 @@
|
|
| 4 |
"model_dtype": "float32",
|
| 5 |
"org": "ServiceNow",
|
| 6 |
"testdata_leakage": "No",
|
| 7 |
-
"replication_code_available": "
|
| 8 |
}
|
|
|
|
| 4 |
"model_dtype": "float32",
|
| 5 |
"org": "ServiceNow",
|
| 6 |
"testdata_leakage": "No",
|
| 7 |
+
"replication_code_available": "No"
|
| 8 |
}
|
results/PatchTST/config.json
CHANGED
|
@@ -4,5 +4,5 @@
|
|
| 4 |
"model_dtype": "float32",
|
| 5 |
"org": "Princeton University",
|
| 6 |
"testdata_leakage": "No",
|
| 7 |
-
"replication_code_available": "
|
| 8 |
}
|
|
|
|
| 4 |
"model_dtype": "float32",
|
| 5 |
"org": "Princeton University",
|
| 6 |
"testdata_leakage": "No",
|
| 7 |
+
"replication_code_available": "No"
|
| 8 |
}
|
results/auto_arima/config.json
CHANGED
|
@@ -3,5 +3,5 @@
|
|
| 3 |
"model_type": "statistical",
|
| 4 |
"model_dtype": "float32",
|
| 5 |
"testdata_leakage": "No",
|
| 6 |
-
"replication_code_available": "
|
| 7 |
}
|
|
|
|
| 3 |
"model_type": "statistical",
|
| 4 |
"model_dtype": "float32",
|
| 5 |
"testdata_leakage": "No",
|
| 6 |
+
"replication_code_available": "No"
|
| 7 |
}
|
results/auto_ets/config.json
CHANGED
|
@@ -3,5 +3,5 @@
|
|
| 3 |
"model_type": "statistical",
|
| 4 |
"model_dtype": "float32",
|
| 5 |
"testdata_leakage": "No",
|
| 6 |
-
"replication_code_available": "
|
| 7 |
}
|
|
|
|
| 3 |
"model_type": "statistical",
|
| 4 |
"model_dtype": "float32",
|
| 5 |
"testdata_leakage": "No",
|
| 6 |
+
"replication_code_available": "No"
|
| 7 |
}
|
results/auto_theta/config.json
CHANGED
|
@@ -3,5 +3,5 @@
|
|
| 3 |
"model_type": "statistical",
|
| 4 |
"model_dtype": "float32",
|
| 5 |
"testdata_leakage": "No",
|
| 6 |
-
"replication_code_available": "
|
| 7 |
-
}
|
|
|
|
| 3 |
"model_type": "statistical",
|
| 4 |
"model_dtype": "float32",
|
| 5 |
"testdata_leakage": "No",
|
| 6 |
+
"replication_code_available": "No"
|
| 7 |
+
}
|
results/crossformer/config.json
CHANGED
|
@@ -4,5 +4,5 @@
|
|
| 4 |
"model_dtype": "float32",
|
| 5 |
"org": "Shanghai Jiao Tong University",
|
| 6 |
"testdata_leakage": "No",
|
| 7 |
-
"replication_code_available": "
|
| 8 |
}
|
|
|
|
| 4 |
"model_dtype": "float32",
|
| 5 |
"org": "Shanghai Jiao Tong University",
|
| 6 |
"testdata_leakage": "No",
|
| 7 |
+
"replication_code_available": "No"
|
| 8 |
}
|
results/deepar/config.json
CHANGED
|
@@ -4,5 +4,5 @@
|
|
| 4 |
"model_dtype": "float32",
|
| 5 |
"org": "Amazon Research",
|
| 6 |
"testdata_leakage": "No",
|
| 7 |
-
"replication_code_available": "
|
| 8 |
}
|
|
|
|
| 4 |
"model_dtype": "float32",
|
| 5 |
"org": "Amazon Research",
|
| 6 |
"testdata_leakage": "No",
|
| 7 |
+
"replication_code_available": "No"
|
| 8 |
}
|
results/tft/config.json
CHANGED
|
@@ -4,5 +4,5 @@
|
|
| 4 |
"model_dtype": "float32",
|
| 5 |
"org": "Google Research",
|
| 6 |
"testdata_leakage": "No",
|
| 7 |
-
"replication_code_available": "
|
| 8 |
}
|
|
|
|
| 4 |
"model_dtype": "float32",
|
| 5 |
"org": "Google Research",
|
| 6 |
"testdata_leakage": "No",
|
| 7 |
+
"replication_code_available": "No"
|
| 8 |
}
|
results/tide/config.json
CHANGED
|
@@ -4,5 +4,5 @@
|
|
| 4 |
"model_dtype": "float32",
|
| 5 |
"org": "Google Research",
|
| 6 |
"testdata_leakage": "No",
|
| 7 |
-
"replication_code_available": "
|
| 8 |
}
|
|
|
|
| 4 |
"model_dtype": "float32",
|
| 5 |
"org": "Google Research",
|
| 6 |
"testdata_leakage": "No",
|
| 7 |
+
"replication_code_available": "No"
|
| 8 |
}
|
src/about.py
CHANGED
|
@@ -44,6 +44,9 @@ points, spanning seven domains, 10 frequencies, multivariate inputs, and predict
|
|
| 44 |
LLM_BENCHMARKS_TEXT = f"""
|
| 45 |
## Update Log
|
| 46 |
|
|
|
|
|
|
|
|
|
|
| 47 |
### 2025-08-25
|
| 48 |
- Added new model type: Zero-shot to distinguish between foundation model submissions that don't use training data of GIFT-Eval. Now models tagged with zero-shot indicate that the model is not trained on the GIFT-Eval training data. Test data leakage is still separately tracked with the TestData Leakage column. For a model be tagged as `zero-shot`, it must both not have test data leakage and not use any training split from GIFT-Eval.
|
| 49 |
|
|
|
|
| 44 |
LLM_BENCHMARKS_TEXT = f"""
|
| 45 |
## Update Log
|
| 46 |
|
| 47 |
+
### 2025-10-17
|
| 48 |
+
- Added new column: Repro. Code to indicate whether the model's evaluation code is made available. This column is a binary indicator specifying whether the model's evaluation code is made available to the public by the submission author. The preferable way to share the evaluation code is to share a notebook in the GIFT-Eval github repository (as many previous submissions have done), but a standalone repo for the evaluation code is also acceptable as long as it is accessible to the public and the link is provided in the config.json file.
|
| 49 |
+
|
| 50 |
### 2025-08-25
|
| 51 |
- Added new model type: Zero-shot to distinguish between foundation model submissions that don't use training data of GIFT-Eval. Now models tagged with zero-shot indicate that the model is not trained on the GIFT-Eval training data. Test data leakage is still separately tracked with the TestData Leakage column. For a model be tagged as `zero-shot`, it must both not have test data leakage and not use any training split from GIFT-Eval.
|
| 52 |
|