## What are you reporting:
- [ ] Evaluation dataset(s) found in a pre-trained model. (e.g. FLAN T5 has been trained on ANLI)
**Evaluation dataset(s)**: openai_humaneval
**Contaminated model(s)**: gpt-3.5-turbo-1106, gpt-3.5-turbo-0613
**Contaminated split(s)**: 41.47%, 23.79%
## Briefly describe your method to detect data contamination
- [ ] Model-based approach
#### Model-based approaches
The cited paper highlights how ChatGPT, when tested with the HumanEval dataset, shows high contamination levels. This is evident from the high Average Peak and Leak Ratios, especially compared to the clean CodeForces2305 dataset where ChatGPT's performance drops. The TED method proves effective in identifying and mitigating these contamination issues. The values can be verified from Table 5 of the cited paper.
## Citation
Is there a paper that reports the data contamination or describes the method used to detect data contamination?
URL: `https://arxiv.org/pdf/2402.…
Add Reports Based on "Llemma: An Open Language Model For Mathematics" (#23)
- Add Reports Based on "Llemma: An Open Language Model For Mathematics" (c50904f0ee3707576647934fba99b878f0ce563d)
- Add PR number + Postprocessing (582a8ca7bb4a6bdd2b05cfe3c1b2d62df251c916)
Co-authored-by: Wei-Lin (William) Chen <wlchen@users.noreply.huggingface.co>
- Add Aquila model series which have gsm8k test set contamination (9cf7873ad5a119cc9111a21dd88b38c74911937f)
- Merge remote-tracking branch 'origin/main' into pr/21 (446d97b84641db8fd0ff52b17dad4f9224ab28f0)
- Add PR number + postprocessing (ecb29a1b2b34353c6a1d2fa7bf037237ecb91107)
Co-authored-by: Bhavish Pahwa <bpHigh@users.noreply.huggingface.co>
- GPT-3.5 Spider contamination based on https://arxiv.org/pdf/2402.08100 (9588afb347cd07a7645852f6c9b966c5222aa76e)
- convert arxiv pdf link to abs (f6e12a010342efd037641bd2cb947fca4b1a5374)
- Add percentage of validation data contaminated by including db ids on which gpt-3.5 achieves more than 75% DC-accuracy (2a68192028574cf246c6af3b1d053cae7fd82acc)
- Add PR number (326e3ca343d14319e6f0c0930fdd06b7507aaacb)
Co-authored-by: Bhavish Pahwa <bpHigh@users.noreply.huggingface.co>
- Superglue/RealNews Contamination based on "Noise-Robust De-Duplication at Scale" (ec46e45d761b5d6620031f28ccf4b581e65614bf)
- Merge branch 'main' of https://huggingface.co/spaces/CONDA-Workshop/Data-Contamination-Report into pr/15 (854619bb3c2f72243cbc255d23bb13f03240e7ae)
Co-authored-by: Emily Silcock <emilys@users.noreply.huggingface.co>
Mistral 7B Arc Easy Contamination based on "Proving Test Set Contamination in Black Box Language Models" (#14)
- Mistral 7B Arc Easy Contamination based on "Proving Test Set Contamination in Black Box Language Models" (c36b27e9f9aa53a35f3d20eda2f4dcd4af826441)
- Changing name to match HF id (b151ec135996a305afce6f866026b5afcc096b6a)
Co-authored-by: Ameya Prabhu <AmeyaPrabhu@users.noreply.huggingface.co>
Added Contamination Evidence from GPT4 Tech Report using String matching on GPT-4 (#11)
GPT-3.5Turbo HumanEval Contamination based on "Generalization or Memorization: Data Contamination and Trustworthy Evaluation for Large Language Models" (#16)
- GPT-3.5Turbo HumanEval Contamination based on "Generalization or Memorization: Data Contamination and Trustworthy Evaluation for Large Language Models" (bad32343b38cfc431017fcd4c4fb1f13edb266b3)
- small fixes (d7810fa0909e40b3b422e4b09d4600df29295671)
Co-authored-by: Kate Solonko <jupyter31@users.noreply.huggingface.co>
Added Contamination Evidence on MMLU of ChatGPT/GPT4 from "Investigating data contamination in modern benchmarks for large language models" (#10)
- Add model-based results for MedNLI, RadNLI for GPT-3.5 and GPT-4 (2a036b2f599b01e80c284e70284212c6477145f6)
- Add PR link (ee21990995aed9ca66afaa00541ead58248ddbb2)
- Replace name with HF dataset (8ff4d82c089e3e2f2582ecf8d6f75014f4b733e7)
Co-authored-by: Jenny Chim <j-chim@users.noreply.huggingface.co>
Add data from "An Open-Source Data Contamination Report for Large Language Models" (#5)
- Add data from "An Open-Source Data Contamination Report for Large Language Models" (6169ce28edf11369ba1957b2266cba0bb4da2024)
- Update contamination_report.csv (4831ab13739e53bf664a1b82467a6a2d4794ed7e)
- Fix merge error (2e138568a5af505805315c084367b0c4c8bcab27)
Co-authored-by: Vishaal Udandarao <vishaal27@users.noreply.huggingface.co>
- Add data from "Documenting Large Webtext Corpora: A Case Study on the Colossal Clean Crawled Corpus" (ad06fdcd383dcbf2bd4b407332cdd6e7440c47af)
Co-authored-by: Vishaal Udandarao <vishaal27@users.noreply.huggingface.co>
