Update README.md
Browse files
README.md
CHANGED
|
@@ -1,10 +1,9 @@
|
|
| 1 |
# TDDBench: A Benchmark for Training data detection
|
| 2 |
|
| 3 |
-
We have uploaded the datasets and target models used by TDDBench on
|
| 4 |
|
| 5 |
To load an evaluation dataset, you can use the following code:
|
| 6 |
```python
|
| 7 |
-
|
| 8 |
# Load dataset
|
| 9 |
from datasets import load_dataset
|
| 10 |
dataset_name = "student"
|
|
@@ -14,7 +13,6 @@ dataset = load_dataset(dataset_path)["train"]
|
|
| 14 |
|
| 15 |
To load a target model, you can use the following code:
|
| 16 |
```python
|
| 17 |
-
|
| 18 |
from transformers import AutoConfig, AutoModel
|
| 19 |
from hfmodel import MLPConfig, MLPHFModel, WRNConfig, WRNHFModel
|
| 20 |
|
|
@@ -38,7 +36,6 @@ tdd_label = np.array(config.tdd_label)
|
|
| 38 |
The [demo.ipynb](https://github.com/zzh9568/TDDBench/blob/main/demo.ipynb) file in our [release code](https://github.com/zzh9568/TDDBench) hub offers a straightforward example of how to download the target model and dataset from Hugging Face, along with instructions for recording the output loss of the model for both training and non-training data.
|
| 39 |
|
| 40 |
### References
|
| 41 |
-
|
| 42 |
```python
|
| 43 |
@article{zhu2024tddbench,
|
| 44 |
title={TDDBench: A Benchmark for Training data detection},
|
|
|
|
| 1 |
# TDDBench: A Benchmark for Training data detection
|
| 2 |
|
| 3 |
+
We have uploaded the datasets and target models used by TDDBench on Huggingface to facilitate a quick evaluation of the Training Data Detection algorithm. This includes 12 datasets and 60 target models, with plans to upload more data and target models in the future.
|
| 4 |
|
| 5 |
To load an evaluation dataset, you can use the following code:
|
| 6 |
```python
|
|
|
|
| 7 |
# Load dataset
|
| 8 |
from datasets import load_dataset
|
| 9 |
dataset_name = "student"
|
|
|
|
| 13 |
|
| 14 |
To load a target model, you can use the following code:
|
| 15 |
```python
|
|
|
|
| 16 |
from transformers import AutoConfig, AutoModel
|
| 17 |
from hfmodel import MLPConfig, MLPHFModel, WRNConfig, WRNHFModel
|
| 18 |
|
|
|
|
| 36 |
The [demo.ipynb](https://github.com/zzh9568/TDDBench/blob/main/demo.ipynb) file in our [release code](https://github.com/zzh9568/TDDBench) hub offers a straightforward example of how to download the target model and dataset from Hugging Face, along with instructions for recording the output loss of the model for both training and non-training data.
|
| 37 |
|
| 38 |
### References
|
|
|
|
| 39 |
```python
|
| 40 |
@article{zhu2024tddbench,
|
| 41 |
title={TDDBench: A Benchmark for Training data detection},
|