diff --git a/granite-4.0-h-tiny/base_attn/epochs/epoch_001/adapter/README.md b/granite-4.0-h-tiny/base_attn/epochs/epoch_001/adapter/README.md
deleted file mode 100644
index 4c79e7cadd1f7126ea2bc554117a0efbf5b341ab..0000000000000000000000000000000000000000
--- a/granite-4.0-h-tiny/base_attn/epochs/epoch_001/adapter/README.md
+++ /dev/null
@@ -1,207 +0,0 @@
----
-base_model: ibm-granite/granite-4.0-h-tiny-base
-library_name: peft
-pipeline_tag: text-generation
-tags:
-- base_model:adapter:ibm-granite/granite-4.0-h-tiny-base
-- lora
-- transformers
----
-
-# Model Card for Model ID
-
-<!-- Provide a quick summary of what the model is/does. -->
-
-
-
-## Model Details
-
-### Model Description
-
-<!-- Provide a longer summary of what this model is. -->
-
-
-
-- **Developed by:** [More Information Needed]
-- **Funded by [optional]:** [More Information Needed]
-- **Shared by [optional]:** [More Information Needed]
-- **Model type:** [More Information Needed]
-- **Language(s) (NLP):** [More Information Needed]
-- **License:** [More Information Needed]
-- **Finetuned from model [optional]:** [More Information Needed]
-
-### Model Sources [optional]
-
-<!-- Provide the basic links for the model. -->
-
-- **Repository:** [More Information Needed]
-- **Paper [optional]:** [More Information Needed]
-- **Demo [optional]:** [More Information Needed]
-
-## Uses
-
-<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
-
-### Direct Use
-
-<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
-
-[More Information Needed]
-
-### Downstream Use [optional]
-
-<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
-
-[More Information Needed]
-
-### Out-of-Scope Use
-
-<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
-
-[More Information Needed]
-
-## Bias, Risks, and Limitations
-
-<!-- This section is meant to convey both technical and sociotechnical limitations. -->
-
-[More Information Needed]
-
-### Recommendations
-
-<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
-
-Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
-
-## How to Get Started with the Model
-
-Use the code below to get started with the model.
-
-[More Information Needed]
-
-## Training Details
-
-### Training Data
-
-<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
-
-[More Information Needed]
-
-### Training Procedure
-
-<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
-
-#### Preprocessing [optional]
-
-[More Information Needed]
-
-
-#### Training Hyperparameters
-
-- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
-
-#### Speeds, Sizes, Times [optional]
-
-<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
-
-[More Information Needed]
-
-## Evaluation
-
-<!-- This section describes the evaluation protocols and provides the results. -->
-
-### Testing Data, Factors & Metrics
-
-#### Testing Data
-
-<!-- This should link to a Dataset Card if possible. -->
-
-[More Information Needed]
-
-#### Factors
-
-<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
-
-[More Information Needed]
-
-#### Metrics
-
-<!-- These are the evaluation metrics being used, ideally with a description of why. -->
-
-[More Information Needed]
-
-### Results
-
-[More Information Needed]
-
-#### Summary
-
-
-
-## Model Examination [optional]
-
-<!-- Relevant interpretability work for the model goes here -->
-
-[More Information Needed]
-
-## Environmental Impact
-
-<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
-
-Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
-
-- **Hardware Type:** [More Information Needed]
-- **Hours used:** [More Information Needed]
-- **Cloud Provider:** [More Information Needed]
-- **Compute Region:** [More Information Needed]
-- **Carbon Emitted:** [More Information Needed]
-
-## Technical Specifications [optional]
-
-### Model Architecture and Objective
-
-[More Information Needed]
-
-### Compute Infrastructure
-
-[More Information Needed]
-
-#### Hardware
-
-[More Information Needed]
-
-#### Software
-
-[More Information Needed]
-
-## Citation [optional]
-
-<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
-
-**BibTeX:**
-
-[More Information Needed]
-
-**APA:**
-
-[More Information Needed]
-
-## Glossary [optional]
-
-<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
-
-[More Information Needed]
-
-## More Information [optional]
-
-[More Information Needed]
-
-## Model Card Authors [optional]
-
-[More Information Needed]
-
-## Model Card Contact
-
-[More Information Needed]
-### Framework versions
-
-- PEFT 0.18.1
\ No newline at end of file
diff --git a/granite-4.0-h-tiny/base_attn/epochs/epoch_001/adapter/adapter_config.json b/granite-4.0-h-tiny/base_attn/epochs/epoch_001/adapter/adapter_config.json
deleted file mode 100644
index 89c71efc5afb392c91d308508241e6b3d235224e..0000000000000000000000000000000000000000
--- a/granite-4.0-h-tiny/base_attn/epochs/epoch_001/adapter/adapter_config.json
+++ /dev/null
@@ -1,46 +0,0 @@
-{
-  "alora_invocation_tokens": null,
-  "alpha_pattern": {},
-  "arrow_config": null,
-  "auto_mapping": null,
-  "base_model_name_or_path": "ibm-granite/granite-4.0-h-tiny-base",
-  "bias": "none",
-  "corda_config": null,
-  "ensure_weight_tying": true,
-  "eva_config": null,
-  "exclude_modules": null,
-  "fan_in_fan_out": false,
-  "inference_mode": true,
-  "init_lora_weights": true,
-  "layer_replication": null,
-  "layers_pattern": null,
-  "layers_to_transform": null,
-  "loftq_config": {},
-  "lora_alpha": 32,
-  "lora_bias": false,
-  "lora_dropout": 0.05,
-  "megatron_config": null,
-  "megatron_core": "megatron.core",
-  "modules_to_save": [
-    "lm_head",
-    "embed_tokens"
-  ],
-  "peft_type": "LORA",
-  "peft_version": "0.18.1",
-  "qalora_group_size": 16,
-  "r": 16,
-  "rank_pattern": {},
-  "revision": null,
-  "target_modules": [
-    "q_proj",
-    "o_proj",
-    "v_proj",
-    "k_proj"
-  ],
-  "target_parameters": null,
-  "task_type": "CAUSAL_LM",
-  "trainable_token_indices": null,
-  "use_dora": false,
-  "use_qalora": false,
-  "use_rslora": false
-}
\ No newline at end of file
diff --git a/granite-4.0-h-tiny/base_attn/epochs/epoch_001/audit_results.json b/granite-4.0-h-tiny/base_attn/epochs/epoch_001/audit_results.json
deleted file mode 100644
index 970bcb25185c058e2238fc80c20185185d8b4c4c..0000000000000000000000000000000000000000
--- a/granite-4.0-h-tiny/base_attn/epochs/epoch_001/audit_results.json
+++ /dev/null
@@ -1,137 +0,0 @@
-{
-  "delta": 1e-05,
-  "num_canaries": 500,
-  "num_members": 250,
-  "paper_guess_fraction": 0.2,
-  "paper_guess_steps": 20,
-  "loss": {
-    "auc": 0.762512,
-    "empirical_epsilon": {
-      "0.05": 3.4791953936219215,
-      "0.01": 3.023197554051876
-    },
-    "empirical_epsilon_details": {
-      "0.05": {
-        "epsilon": 3.4791953936219215,
-        "num_guesses": 100,
-        "correct_guesses": 100,
-        "candidate_num_guesses": [
-          5,
-          10,
-          15,
-          20,
-          25,
-          30,
-          35,
-          40,
-          45,
-          50,
-          55,
-          60,
-          65,
-          70,
-          75,
-          80,
-          85,
-          90,
-          95,
-          100
-        ],
-        "direction": "lower"
-      },
-      "0.01": {
-        "epsilon": 3.023197554051876,
-        "num_guesses": 100,
-        "correct_guesses": 100,
-        "candidate_num_guesses": [
-          5,
-          10,
-          15,
-          20,
-          25,
-          30,
-          35,
-          40,
-          45,
-          50,
-          55,
-          60,
-          65,
-          70,
-          75,
-          80,
-          85,
-          90,
-          95,
-          100
-        ],
-        "direction": "lower"
-      }
-    }
-  },
-  "embedding": {
-    "auc": 0.75,
-    "empirical_epsilon": {
-      "0.05": 3.4791953936219215,
-      "0.01": 3.023197554051876
-    },
-    "empirical_epsilon_details": {
-      "0.05": {
-        "epsilon": 3.4791953936219215,
-        "num_guesses": 100,
-        "correct_guesses": 100,
-        "candidate_num_guesses": [
-          5,
-          10,
-          15,
-          20,
-          25,
-          30,
-          35,
-          40,
-          45,
-          50,
-          55,
-          60,
-          65,
-          70,
-          75,
-          80,
-          85,
-          90,
-          95,
-          100
-        ],
-        "direction": "lower"
-      },
-      "0.01": {
-        "epsilon": 3.023197554051876,
-        "num_guesses": 100,
-        "correct_guesses": 100,
-        "candidate_num_guesses": [
-          5,
-          10,
-          15,
-          20,
-          25,
-          30,
-          35,
-          40,
-          45,
-          50,
-          55,
-          60,
-          65,
-          70,
-          75,
-          80,
-          85,
-          90,
-          95,
-          100
-        ],
-        "direction": "lower"
-      }
-    }
-  }
-}
\ No newline at end of file
diff --git a/granite-4.0-h-tiny/base_attn/epochs/epoch_002/adapter/README.md b/granite-4.0-h-tiny/base_attn/epochs/epoch_002/adapter/README.md
deleted file mode 100644
index 4c79e7cadd1f7126ea2bc554117a0efbf5b341ab..0000000000000000000000000000000000000000
--- a/granite-4.0-h-tiny/base_attn/epochs/epoch_002/adapter/README.md
+++ /dev/null
@@ -1,207 +0,0 @@
----
-base_model: ibm-granite/granite-4.0-h-tiny-base
-library_name: peft
-pipeline_tag: text-generation
-tags:
-- base_model:adapter:ibm-granite/granite-4.0-h-tiny-base
-- lora
-- transformers
----
-
-# Model Card for Model ID
-
-<!-- Provide a quick summary of what the model is/does. -->
-
-
-
-## Model Details
-
-### Model Description
-
-<!-- Provide a longer summary of what this model is. -->
-
-
-
-- **Developed by:** [More Information Needed]
-- **Funded by [optional]:** [More Information Needed]
-- **Shared by [optional]:** [More Information Needed]
-- **Model type:** [More Information Needed]
-- **Language(s) (NLP):** [More Information Needed]
-- **License:** [More Information Needed]
-- **Finetuned from model [optional]:** [More Information Needed]
-
-### Model Sources [optional]
-
-<!-- Provide the basic links for the model. -->
-
-- **Repository:** [More Information Needed]
-- **Paper [optional]:** [More Information Needed]
-- **Demo [optional]:** [More Information Needed]
-
-## Uses
-
-<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
-
-### Direct Use
-
-<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
-
-[More Information Needed]
-
-### Downstream Use [optional]
-
-<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
-
-[More Information Needed]
-
-### Out-of-Scope Use
-
-<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
-
-[More Information Needed]
-
-## Bias, Risks, and Limitations
-
-<!-- This section is meant to convey both technical and sociotechnical limitations. -->
-
-[More Information Needed]
-
-### Recommendations
-
-<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
-
-Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
-
-## How to Get Started with the Model
-
-Use the code below to get started with the model.
-
-[More Information Needed]
-
-## Training Details
-
-### Training Data
-
-<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
-
-[More Information Needed]
-
-### Training Procedure
-
-<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
-
-#### Preprocessing [optional]
-
-[More Information Needed]
-
-
-#### Training Hyperparameters
-
-- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
-
-#### Speeds, Sizes, Times [optional]
-
-<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
-
-[More Information Needed]
-
-## Evaluation
-
-<!-- This section describes the evaluation protocols and provides the results. -->
-
-### Testing Data, Factors & Metrics
-
-#### Testing Data
-
-<!-- This should link to a Dataset Card if possible. -->
-
-[More Information Needed]
-
-#### Factors
-
-<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
-
-[More Information Needed]
-
-#### Metrics
-
-<!-- These are the evaluation metrics being used, ideally with a description of why. -->
-
-[More Information Needed]
-
-### Results
-
-[More Information Needed]
-
-#### Summary
-
-
-
-## Model Examination [optional]
-
-<!-- Relevant interpretability work for the model goes here -->
-
-[More Information Needed]
-
-## Environmental Impact
-
-<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
-
-Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
-
-- **Hardware Type:** [More Information Needed]
-- **Hours used:** [More Information Needed]
-- **Cloud Provider:** [More Information Needed]
-- **Compute Region:** [More Information Needed]
-- **Carbon Emitted:** [More Information Needed]
-
-## Technical Specifications [optional]
-
-### Model Architecture and Objective
-
-[More Information Needed]
-
-### Compute Infrastructure
-
-[More Information Needed]
-
-#### Hardware
-
-[More Information Needed]
-
-#### Software
-
-[More Information Needed]
-
-## Citation [optional]
-
-<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
-
-**BibTeX:**
-
-[More Information Needed]
-
-**APA:**
-
-[More Information Needed]
-
-## Glossary [optional]
-
-<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
-
-[More Information Needed]
-
-## More Information [optional]
-
-[More Information Needed]
-
-## Model Card Authors [optional]
-
-[More Information Needed]
-
-## Model Card Contact
-
-[More Information Needed]
-### Framework versions
-
-- PEFT 0.18.1
\ No newline at end of file
diff --git a/granite-4.0-h-tiny/base_attn/epochs/epoch_002/adapter/adapter_config.json b/granite-4.0-h-tiny/base_attn/epochs/epoch_002/adapter/adapter_config.json
deleted file mode 100644
index 89c71efc5afb392c91d308508241e6b3d235224e..0000000000000000000000000000000000000000
--- a/granite-4.0-h-tiny/base_attn/epochs/epoch_002/adapter/adapter_config.json
+++ /dev/null
@@ -1,46 +0,0 @@
-{
-  "alora_invocation_tokens": null,
-  "alpha_pattern": {},
-  "arrow_config": null,
-  "auto_mapping": null,
-  "base_model_name_or_path": "ibm-granite/granite-4.0-h-tiny-base",
-  "bias": "none",
-  "corda_config": null,
-  "ensure_weight_tying": true,
-  "eva_config": null,
-  "exclude_modules": null,
-  "fan_in_fan_out": false,
-  "inference_mode": true,
-  "init_lora_weights": true,
-  "layer_replication": null,
-  "layers_pattern": null,
-  "layers_to_transform": null,
-  "loftq_config": {},
-  "lora_alpha": 32,
-  "lora_bias": false,
-  "lora_dropout": 0.05,
-  "megatron_config": null,
-  "megatron_core": "megatron.core",
-  "modules_to_save": [
-    "lm_head",
-    "embed_tokens"
-  ],
-  "peft_type": "LORA",
-  "peft_version": "0.18.1",
-  "qalora_group_size": 16,
-  "r": 16,
-  "rank_pattern": {},
-  "revision": null,
-  "target_modules": [
-    "q_proj",
-    "o_proj",
-    "v_proj",
-    "k_proj"
-  ],
-  "target_parameters": null,
-  "task_type": "CAUSAL_LM",
-  "trainable_token_indices": null,
-  "use_dora": false,
-  "use_qalora": false,
-  "use_rslora": false
-}
\ No newline at end of file
diff --git a/granite-4.0-h-tiny/base_attn/epochs/epoch_002/audit_results.json b/granite-4.0-h-tiny/base_attn/epochs/epoch_002/audit_results.json
deleted file mode 100644
index 55dacd6fb62962215ae2be46df8c313d61552b9d..0000000000000000000000000000000000000000
--- a/granite-4.0-h-tiny/base_attn/epochs/epoch_002/audit_results.json
+++ /dev/null
@@ -1,137 +0,0 @@
-{
-  "delta": 1e-05,
-  "num_canaries": 500,
-  "num_members": 250,
-  "paper_guess_fraction": 0.2,
-  "paper_guess_steps": 20,
-  "loss": {
-    "auc": 0.88128,
-    "empirical_epsilon": {
-      "0.05": 3.4791953936219215,
-      "0.01": 3.023197554051876
-    },
-    "empirical_epsilon_details": {
-      "0.05": {
-        "epsilon": 3.4791953936219215,
-        "num_guesses": 100,
-        "correct_guesses": 100,
-        "candidate_num_guesses": [
-          5,
-          10,
-          15,
-          20,
-          25,
-          30,
-          35,
-          40,
-          45,
-          50,
-          55,
-          60,
-          65,
-          70,
-          75,
-          80,
-          85,
-          90,
-          95,
-          100
-        ],
-        "direction": "lower"
-      },
-      "0.01": {
-        "epsilon": 3.023197554051876,
-        "num_guesses": 100,
-        "correct_guesses": 100,
-        "candidate_num_guesses": [
-          5,
-          10,
-          15,
-          20,
-          25,
-          30,
-          35,
-          40,
-          45,
-          50,
-          55,
-          60,
-          65,
-          70,
-          75,
-          80,
-          85,
-          90,
-          95,
-          100
-        ],
-        "direction": "lower"
-      }
-    }
-  },
-  "embedding": {
-    "auc": 0.81,
-    "empirical_epsilon": {
-      "0.05": 3.4791953936219215,
-      "0.01": 3.023197554051876
-    },
-    "empirical_epsilon_details": {
-      "0.05": {
-        "epsilon": 3.4791953936219215,
-        "num_guesses": 100,
-        "correct_guesses": 100,
-        "candidate_num_guesses": [
-          5,
-          10,
-          15,
-          20,
-          25,
-          30,
-          35,
-          40,
-          45,
-          50,
-          55,
-          60,
-          65,
-          70,
-          75,
-          80,
-          85,
-          90,
-          95,
-          100
-        ],
-        "direction": "lower"
-      },
-      "0.01": {
-        "epsilon": 3.023197554051876,
-        "num_guesses": 100,
-        "correct_guesses": 100,
-        "candidate_num_guesses": [
-          5,
-          10,
-          15,
-          20,
-          25,
-          30,
-          35,
-          40,
-          45,
-          50,
-          55,
-          60,
-          65,
-          70,
-          75,
-          80,
-          85,
-          90,
-          95,
-          100
-        ],
-        "direction": "lower"
-      }
-    }
-  }
-}
\ No newline at end of file
diff --git a/granite-4.0-h-tiny/base_attn/epochs/epoch_003/adapter/README.md b/granite-4.0-h-tiny/base_attn/epochs/epoch_003/adapter/README.md
deleted file mode 100644
index 4c79e7cadd1f7126ea2bc554117a0efbf5b341ab..0000000000000000000000000000000000000000
--- a/granite-4.0-h-tiny/base_attn/epochs/epoch_003/adapter/README.md
+++ /dev/null
@@ -1,207 +0,0 @@
----
-base_model: ibm-granite/granite-4.0-h-tiny-base
-library_name: peft
-pipeline_tag: text-generation
-tags:
-- base_model:adapter:ibm-granite/granite-4.0-h-tiny-base
-- lora
-- transformers
----
-
-# Model Card for Model ID
-
-<!-- Provide a quick summary of what the model is/does. -->
-
-
-
-## Model Details
-
-### Model Description
-
-<!-- Provide a longer summary of what this model is. -->
-
-
-
-- **Developed by:** [More Information Needed]
-- **Funded by [optional]:** [More Information Needed]
-- **Shared by [optional]:** [More Information Needed]
-- **Model type:** [More Information Needed]
-- **Language(s) (NLP):** [More Information Needed]
-- **License:** [More Information Needed]
-- **Finetuned from model [optional]:** [More Information Needed]
-
-### Model Sources [optional]
-
-<!-- Provide the basic links for the model. -->
-
-- **Repository:** [More Information Needed]
-- **Paper [optional]:** [More Information Needed]
-- **Demo [optional]:** [More Information Needed]
-
-## Uses
-
-<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
-
-### Direct Use
-
-<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
-
-[More Information Needed]
-
-### Downstream Use [optional]
-
-<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
-
-[More Information Needed]
-
-### Out-of-Scope Use
-
-<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
-
-[More Information Needed]
-
-## Bias, Risks, and Limitations
-
-<!-- This section is meant to convey both technical and sociotechnical limitations. -->
-
-[More Information Needed]
-
-### Recommendations
-
-<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
-
-Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
-
-## How to Get Started with the Model
-
-Use the code below to get started with the model.
-
-[More Information Needed]
-
-## Training Details
-
-### Training Data
-
-<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
-
-[More Information Needed]
-
-### Training Procedure
-
-<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
-
-#### Preprocessing [optional]
-
-[More Information Needed]
-
-
-#### Training Hyperparameters
-
-- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
-
-#### Speeds, Sizes, Times [optional]
-
-<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
-
-[More Information Needed]
-
-## Evaluation
-
-<!-- This section describes the evaluation protocols and provides the results. -->
-
-### Testing Data, Factors & Metrics
-
-#### Testing Data
-
-<!-- This should link to a Dataset Card if possible. -->
-
-[More Information Needed]
-
-#### Factors
-
-<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
-
-[More Information Needed]
-
-#### Metrics
-
-<!-- These are the evaluation metrics being used, ideally with a description of why. -->
-
-[More Information Needed]
-
-### Results
-
-[More Information Needed]
-
-#### Summary
-
-
-
-## Model Examination [optional]
-
-<!-- Relevant interpretability work for the model goes here -->
-
-[More Information Needed]
-
-## Environmental Impact
-
-<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
-
-Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
-
-- **Hardware Type:** [More Information Needed]
-- **Hours used:** [More Information Needed]
-- **Cloud Provider:** [More Information Needed]
-- **Compute Region:** [More Information Needed]
-- **Carbon Emitted:** [More Information Needed]
-
-## Technical Specifications [optional]
-
-### Model Architecture and Objective
-
-[More Information Needed]
-
-### Compute Infrastructure
-
-[More Information Needed]
-
-#### Hardware
-
-[More Information Needed]
-
-#### Software
-
-[More Information Needed]
-
-## Citation [optional]
-
-<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
-
-**BibTeX:**
-
-[More Information Needed]
-
-**APA:**
-
-[More Information Needed]
-
-## Glossary [optional]
-
-<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
-
-[More Information Needed]
-
-## More Information [optional]
-
-[More Information Needed]
-
-## Model Card Authors [optional]
-
-[More Information Needed]
-
-## Model Card Contact
-
-[More Information Needed]
-### Framework versions
-
-- PEFT 0.18.1
\ No newline at end of file
diff --git a/granite-4.0-h-tiny/base_attn/epochs/epoch_003/adapter/adapter_config.json b/granite-4.0-h-tiny/base_attn/epochs/epoch_003/adapter/adapter_config.json
deleted file mode 100644
index 89c71efc5afb392c91d308508241e6b3d235224e..0000000000000000000000000000000000000000
--- a/granite-4.0-h-tiny/base_attn/epochs/epoch_003/adapter/adapter_config.json
+++ /dev/null
@@ -1,46 +0,0 @@
-{
-  "alora_invocation_tokens": null,
-  "alpha_pattern": {},
-  "arrow_config": null,
-  "auto_mapping": null,
-  "base_model_name_or_path": "ibm-granite/granite-4.0-h-tiny-base",
-  "bias": "none",
-  "corda_config": null,
-  "ensure_weight_tying": true,
-  "eva_config": null,
-  "exclude_modules": null,
-  "fan_in_fan_out": false,
-  "inference_mode": true,
-  "init_lora_weights": true,
-  "layer_replication": null,
-  "layers_pattern": null,
-  "layers_to_transform": null,
-  "loftq_config": {},
-  "lora_alpha": 32,
-  "lora_bias": false,
-  "lora_dropout": 0.05,
-  "megatron_config": null,
-  "megatron_core": "megatron.core",
-  "modules_to_save": [
-    "lm_head",
-    "embed_tokens"
-  ],
-  "peft_type": "LORA",
-  "peft_version": "0.18.1",
-  "qalora_group_size": 16,
-  "r": 16,
-  "rank_pattern": {},
-  "revision": null,
-  "target_modules": [
-    "q_proj",
-    "o_proj",
-    "v_proj",
-    "k_proj"
-  ],
-  "target_parameters": null,
-  "task_type": "CAUSAL_LM",
-  "trainable_token_indices": null,
-  "use_dora": false,
-  "use_qalora": false,
-  "use_rslora": false
-}
\ No newline at end of file
diff --git a/granite-4.0-h-tiny/base_attn/epochs/epoch_003/audit_results.json b/granite-4.0-h-tiny/base_attn/epochs/epoch_003/audit_results.json
deleted file mode 100644
index e040b2c4a4d85e1fe1c819426cc6e053f64fc3a1..0000000000000000000000000000000000000000
--- a/granite-4.0-h-tiny/base_attn/epochs/epoch_003/audit_results.json
+++ /dev/null
@@ -1,137 +0,0 @@
-{
-  "delta": 1e-05,
-  "num_canaries": 500,
-  "num_members": 250,
-  "paper_guess_fraction": 0.2,
-  "paper_guess_steps": 20,
-  "loss": {
-    "auc": 0.924608,
-    "empirical_epsilon": {
-      "0.05": 3.4791953936219215,
-      "0.01": 3.023197554051876
-    },
-    "empirical_epsilon_details": {
-      "0.05": {
-        "epsilon": 3.4791953936219215,
-        "num_guesses": 100,
-        "correct_guesses": 100,
-        "candidate_num_guesses": [
-          5,
-          10,
-          15,
-          20,
-          25,
-          30,
-          35,
-          40,
-          45,
-          50,
-          55,
-          60,
-          65,
-          70,
-          75,
-          80,
-          85,
-          90,
-          95,
-          100
-        ],
-        "direction": "lower"
-      },
-      "0.01": {
-        "epsilon": 3.023197554051876,
-        "num_guesses": 100,
-        "correct_guesses": 100,
-        "candidate_num_guesses": [
-          5,
-          10,
-          15,
-          20,
-          25,
-          30,
-          35,
-          40,
-          45,
-          50,
-          55,
-          60,
-          65,
-          70,
-          75,
-          80,
-          85,
-          90,
-          95,
-          100
-        ],
-        "direction": "lower"
-      }
-    }
-  },
-  "embedding": {
-    "auc": 0.756,
-    "empirical_epsilon": {
-      "0.05": 3.4791953936219215,
-      "0.01": 3.023197554051876
-    },
-    "empirical_epsilon_details": {
-      "0.05": {
-        "epsilon": 3.4791953936219215,
-        "num_guesses": 100,
-        "correct_guesses": 100,
-        "candidate_num_guesses": [
-          5,
-          10,
-          15,
-          20,
-          25,
-          30,
-          35,
-          40,
-          45,
-          50,
-          55,
-          60,
-          65,
-          70,
-          75,
-          80,
-          85,
-          90,
-          95,
-          100
-        ],
-        "direction": "lower"
-      },
-      "0.01": {
-        "epsilon": 3.023197554051876,
-        "num_guesses": 100,
-        "correct_guesses": 100,
-        "candidate_num_guesses": [
-          5,
-          10,
-          15,
-          20,
-          25,
-          30,
-          35,
-          40,
-          45,
-          50,
-          55,
-          60,
-          65,
-          70,
-          75,
-          80,
-          85,
-          90,
-          95,
-          100
-        ],
-        "direction": "lower"
-      }
-    }
-  }
-}
\ No newline at end of file
diff --git a/granite-4.0-h-tiny/dp3_attn/epochs/epoch_001/adapter/README.md b/granite-4.0-h-tiny/dp3_attn/epochs/epoch_001/adapter/README.md
deleted file mode 100644
index 4c79e7cadd1f7126ea2bc554117a0efbf5b341ab..0000000000000000000000000000000000000000
--- a/granite-4.0-h-tiny/dp3_attn/epochs/epoch_001/adapter/README.md
+++ /dev/null
@@ -1,207 +0,0 @@
----
-base_model: ibm-granite/granite-4.0-h-tiny-base
-library_name: peft
-pipeline_tag: text-generation
-tags:
-- base_model:adapter:ibm-granite/granite-4.0-h-tiny-base
-- lora
-- transformers
----
-
-# Model Card for Model ID
-
-<!-- Provide a quick summary of what the model is/does. -->
-
-
-
-## Model Details
-
-### Model Description
-
-<!-- Provide a longer summary of what this model is. -->
-
-
-
-- **Developed by:** [More Information Needed]
-- **Funded by [optional]:** [More Information Needed]
-- **Shared by [optional]:** [More Information Needed]
-- **Model type:** [More Information Needed]
-- **Language(s) (NLP):** [More Information Needed]
-- **License:** [More Information Needed]
-- **Finetuned from model [optional]:** [More Information Needed]
-
-### Model Sources [optional]
-
-<!-- Provide the basic links for the model. -->
-
-- **Repository:** [More Information Needed]
-- **Paper [optional]:** [More Information Needed]
-- **Demo [optional]:** [More Information Needed]
-
-## Uses
-
-<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
-
-### Direct Use
-
-<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
-
-[More Information Needed]
-
-### Downstream Use [optional]
-
-<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
-
-[More Information Needed]
-
-### Out-of-Scope Use
-
-<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
-
-[More Information Needed]
-
-## Bias, Risks, and Limitations
-
-<!-- This section is meant to convey both technical and sociotechnical limitations. -->
-
-[More Information Needed]
-
-### Recommendations
-
-<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
-
-Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
-
-## How to Get Started with the Model
-
-Use the code below to get started with the model.
-
-[More Information Needed]
-
-## Training Details
-
-### Training Data
-
-<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
-
-[More Information Needed]
-
-### Training Procedure
-
-<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
-
-#### Preprocessing [optional]
-
-[More Information Needed]
-
-
-#### Training Hyperparameters
-
-- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
-
-#### Speeds, Sizes, Times [optional]
-
-<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
-
-[More Information Needed]
-
-## Evaluation
-
-<!-- This section describes the evaluation protocols and provides the results. -->
-
-### Testing Data, Factors & Metrics
-
-#### Testing Data
-
-<!-- This should link to a Dataset Card if possible. -->
-
-[More Information Needed]
-
-#### Factors
-
-<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
-
-[More Information Needed]
-
-#### Metrics
-
-<!-- These are the evaluation metrics being used, ideally with a description of why. -->
-
-[More Information Needed]
-
-### Results
-
-[More Information Needed]
-
-#### Summary
-
-
-
-## Model Examination [optional]
-
-<!-- Relevant interpretability work for the model goes here -->
-
-[More Information Needed]
-
-## Environmental Impact
-
-<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
-
-Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
-
-- **Hardware Type:** [More Information Needed]
-- **Hours used:** [More Information Needed]
-- **Cloud Provider:** [More Information Needed]
-- **Compute Region:** [More Information Needed]
-- **Carbon Emitted:** [More Information Needed]
-
-## Technical Specifications [optional]
-
-### Model Architecture and Objective
-
-[More Information Needed]
-
-### Compute Infrastructure
-
-[More Information Needed]
-
-#### Hardware
-
-[More Information Needed]
-
-#### Software
-
-[More Information Needed]
-
-## Citation [optional]
-
-<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
-
-**BibTeX:**
-
-[More Information Needed]
-
-**APA:**
-
-[More Information Needed]
-
-## Glossary [optional]
-
-<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
-
-[More Information Needed]
-
-## More Information [optional]
-
-[More Information Needed]
-
-## Model Card Authors [optional]
-
-[More Information Needed]
-
-## Model Card Contact
-
-[More Information Needed]
-### Framework versions
-
-- PEFT 0.18.1
\ No newline at end of file
diff --git a/granite-4.0-h-tiny/dp3_attn/epochs/epoch_001/adapter/adapter_config.json b/granite-4.0-h-tiny/dp3_attn/epochs/epoch_001/adapter/adapter_config.json
deleted file mode 100644
index 4a26cfc6fc851cabaca33dc7dae19ef8ef3bd2aa..0000000000000000000000000000000000000000
--- a/granite-4.0-h-tiny/dp3_attn/epochs/epoch_001/adapter/adapter_config.json
+++ /dev/null
@@ -1,46 +0,0 @@
-{
-  "alora_invocation_tokens": null,
-  "alpha_pattern": {},
-  "arrow_config": null,
-  "auto_mapping": null,
-  "base_model_name_or_path": "ibm-granite/granite-4.0-h-tiny-base",
-  "bias": "none",
-  "corda_config": null,
-  "ensure_weight_tying": true,
-  "eva_config": null,
-  "exclude_modules": null,
-  "fan_in_fan_out": false,
-  "inference_mode": true,
-  "init_lora_weights": true,
-  "layer_replication": null,
-  "layers_pattern": null,
-  "layers_to_transform": null,
-  "loftq_config": {},
-  "lora_alpha": 32,
-  "lora_bias": false,
-  "lora_dropout": 0.05,
-  "megatron_config": null,
-  "megatron_core": "megatron.core",
-  "modules_to_save": [
-    "lm_head",
-    "embed_tokens"
-  ],
-  "peft_type": "LORA",
-  "peft_version": "0.18.1",
-  "qalora_group_size": 16,
-  "r": 16,
-  "rank_pattern": {},
-  "revision": null,
-  "target_modules": [
-    "v_proj",
-    "o_proj",
-    "k_proj",
-    "q_proj"
-  ],
-  "target_parameters": null,
-  "task_type": "CAUSAL_LM",
-  "trainable_token_indices": null,
-  "use_dora": false,
-  "use_qalora": false,
-  "use_rslora": false
-}
\ No newline at end of file
diff --git a/granite-4.0-h-tiny/dp3_attn/epochs/epoch_001/audit_results.json b/granite-4.0-h-tiny/dp3_attn/epochs/epoch_001/audit_results.json
deleted file mode 100644
index 556ea8364851df20fc3032e00d2fbe1a96c0f803..0000000000000000000000000000000000000000
--- a/granite-4.0-h-tiny/dp3_attn/epochs/epoch_001/audit_results.json
+++ /dev/null
@@ -1,137 +0,0 @@
-{
-  "delta": 1e-05,
-  "num_canaries": 500,
-  "num_members": 250,
-  "paper_guess_fraction": 0.2,
-  "paper_guess_steps": 20,
-  "loss": {
-    "auc": 0.530984,
-    "empirical_epsilon": {
-      "0.05": 0.22670934721827507,
-      "0.01": 0.015452107414603233
-    },
-    "empirical_epsilon_details": {
-      "0.05": {
-        "epsilon": 0.22670934721827507,
-        "num_guesses": 45,
-        "correct_guesses": 31,
-        "candidate_num_guesses": [
-          5,
-          10,
-          15,
-          20,
-          25,
-          30,
-          35,
-          40,
-          45,
-          50,
-          55,
-          60,
-          65,
-          70,
-          75,
-          80,
-          85,
-          90,
-          95,
-          100
-        ],
-        "direction": "lower"
-      },
-      "0.01": {
-        "epsilon": 0.015452107414603233,
-        "num_guesses": 45,
-        "correct_guesses": 31,
-        "candidate_num_guesses": [
-          5,
-          10,
-          15,
-          20,
-          25,
-          30,
-          35,
-          40,
-          45,
-          50,
-          55,
-          60,
-          65,
-          70,
-          75,
-          80,
-          85,
-          90,
-          95,
-          100
-        ],
-        "direction": "lower"
-      }
-    }
-  },
-  "embedding": {
-    "auc": 0.549064,
-    "empirical_epsilon": {
-      "0.05": 0.0,
-      "0.01": 0.0
-    },
-    "empirical_epsilon_details": {
-      "0.05": {
-        "epsilon": 0.0,
-        "num_guesses": 0,
-        "correct_guesses": 0,
-        "candidate_num_guesses": [
-          5,
-          10,
-          15,
-          20,
-          25,
-          30,
-          35,
-          40,
-          45,
-          50,
-          55,
-          60,
-          65,
-          70,
-          75,
-          80,
-          85,
-          90,
-          95,
-          100
-        ],
-        "direction": "lower"
-      },
-      "0.01": {
-        "epsilon": 0.0,
-        "num_guesses": 0,
-        "correct_guesses": 0,
-        "candidate_num_guesses": [
-          5,
-          10,
-          15,
-          20,
-          25,
-          30,
-          35,
-          40,
-          45,
-          50,
-          55,
-          60,
-          65,
-          70,
-          75,
-          80,
-          85,
-          90,
-          95,
-          100
-        ],
-        "direction": "lower"
-      }
-    }
-  }
-}
\ No newline at end of file
diff --git a/granite-4.0-h-tiny/dp3_attn/epochs/epoch_002/adapter/README.md b/granite-4.0-h-tiny/dp3_attn/epochs/epoch_002/adapter/README.md
deleted file mode 100644
index 4c79e7cadd1f7126ea2bc554117a0efbf5b341ab..0000000000000000000000000000000000000000
--- a/granite-4.0-h-tiny/dp3_attn/epochs/epoch_002/adapter/README.md
+++ /dev/null
@@ -1,207 +0,0 @@
----
-base_model: ibm-granite/granite-4.0-h-tiny-base
-library_name: peft
-pipeline_tag: text-generation
-tags:
-- base_model:adapter:ibm-granite/granite-4.0-h-tiny-base
-- lora
-- transformers
----
-
-# Model Card for Model ID
-
-<!-- Provide a quick summary of what the model is/does. -->
-
-
-
-## Model Details
-
-### Model Description
-
-<!-- Provide a longer summary of what this model is. -->
-
-
-
-- **Developed by:** [More Information Needed]
-- **Funded by [optional]:** [More Information Needed]
-- **Shared by [optional]:** [More Information Needed]
-- **Model type:** [More Information Needed]
-- **Language(s) (NLP):** [More Information Needed]
-- **License:** [More Information Needed]
-- **Finetuned from model [optional]:** [More Information Needed]
-
-### Model Sources [optional]
-
-<!-- Provide the basic links for the model. -->
-
-- **Repository:** [More Information Needed]
-- **Paper [optional]:** [More Information Needed]
-- **Demo [optional]:** [More Information Needed]
-
-## Uses
-
-<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
-
-### Direct Use
-
-<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
-
-[More Information Needed]
-
-### Downstream Use [optional]
-
-<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
-
-[More Information Needed]
-
-### Out-of-Scope Use
-
-<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
-
-[More Information Needed]
-
-## Bias, Risks, and Limitations
-
-<!-- This section is meant to convey both technical and sociotechnical limitations. -->
-
-[More Information Needed]
-
-### Recommendations
-
-<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
-
-Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
-
-## How to Get Started with the Model
-
-Use the code below to get started with the model.
-
-[More Information Needed]
-
-## Training Details
-
-### Training Data
-
-<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
-
-[More Information Needed]
-
-### Training Procedure
-
-<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
-
-#### Preprocessing [optional]
-
-[More Information Needed]
-
-
-#### Training Hyperparameters
-
-- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
-
-#### Speeds, Sizes, Times [optional]
-
-<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
-
-[More Information Needed]
-
-## Evaluation
-
-<!-- This section describes the evaluation protocols and provides the results. -->
-
-### Testing Data, Factors & Metrics
-
-#### Testing Data
-
-<!-- This should link to a Dataset Card if possible. -->
-
-[More Information Needed]
-
-#### Factors
-
-<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
-
-[More Information Needed]
-
-#### Metrics
-
-<!-- These are the evaluation metrics being used, ideally with a description of why. -->
-
-[More Information Needed]
-
-### Results
-
-[More Information Needed]
-
-#### Summary
-
-
-
-## Model Examination [optional]
-
-<!-- Relevant interpretability work for the model goes here -->
-
-[More Information Needed]
-
-## Environmental Impact
-
-<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
-
-Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
-
-- **Hardware Type:** [More Information Needed]
-- **Hours used:** [More Information Needed]
-- **Cloud Provider:** [More Information Needed]
-- **Compute Region:** [More Information Needed]
-- **Carbon Emitted:** [More Information Needed]
-
-## Technical Specifications [optional]
-
-### Model Architecture and Objective
-
-[More Information Needed]
-
-### Compute Infrastructure
-
-[More Information Needed]
-
-#### Hardware
-
-[More Information Needed]
-
-#### Software
-
-[More Information Needed]
-
-## Citation [optional]
-
-<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
-
-**BibTeX:**
-
-[More Information Needed]
-
-**APA:**
-
-[More Information Needed]
-
-## Glossary [optional]
-
-<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
-
-[More Information Needed]
-
-## More Information [optional]
-
-[More Information Needed]
-
-## Model Card Authors [optional]
-
-[More Information Needed]
-
-## Model Card Contact
-
-[More Information Needed]
-### Framework versions
-
-- PEFT 0.18.1
\ No newline at end of file
diff --git a/granite-4.0-h-tiny/dp3_attn/epochs/epoch_002/adapter/adapter_config.json b/granite-4.0-h-tiny/dp3_attn/epochs/epoch_002/adapter/adapter_config.json
deleted file mode 100644
index 4a26cfc6fc851cabaca33dc7dae19ef8ef3bd2aa..0000000000000000000000000000000000000000
--- a/granite-4.0-h-tiny/dp3_attn/epochs/epoch_002/adapter/adapter_config.json
+++ /dev/null
@@ -1,46 +0,0 @@
-{
-  "alora_invocation_tokens": null,
-  "alpha_pattern": {},
-  "arrow_config": null,
-  "auto_mapping": null,
-  "base_model_name_or_path": "ibm-granite/granite-4.0-h-tiny-base",
-  "bias": "none",
-  "corda_config": null,
-  "ensure_weight_tying": true,
-  "eva_config": null,
-  "exclude_modules": null,
-  "fan_in_fan_out": false,
-  "inference_mode": true,
-  "init_lora_weights": true,
-  "layer_replication": null,
-  "layers_pattern": null,
-  "layers_to_transform": null,
-  "loftq_config": {},
-  "lora_alpha": 32,
-  "lora_bias": false,
-  "lora_dropout": 0.05,
-  "megatron_config": null,
-  "megatron_core": "megatron.core",
-  "modules_to_save": [
-    "lm_head",
-    "embed_tokens"
-  ],
-  "peft_type": "LORA",
-  "peft_version": "0.18.1",
-  "qalora_group_size": 16,
-  "r": 16,
-  "rank_pattern": {},
-  "revision": null,
-  "target_modules": [
-    "v_proj",
-    "o_proj",
-    "k_proj",
-    "q_proj"
-  ],
-  "target_parameters": null,
-  "task_type": "CAUSAL_LM",
-  "trainable_token_indices": null,
-  "use_dora": false,
-  "use_qalora": false,
-  "use_rslora": false
-}
\ No newline at end of file
diff --git a/granite-4.0-h-tiny/dp3_attn/epochs/epoch_002/audit_results.json b/granite-4.0-h-tiny/dp3_attn/epochs/epoch_002/audit_results.json
deleted file mode 100644
index ae5adbace361bbefbb5612f21b7add1eff8866cc..0000000000000000000000000000000000000000
--- a/granite-4.0-h-tiny/dp3_attn/epochs/epoch_002/audit_results.json
+++ /dev/null
@@ -1,137 +0,0 @@
-{
-  "delta": 1e-05,
-  "num_canaries": 500,
-  "num_members": 250,
-  "paper_guess_fraction": 0.2,
-  "paper_guess_steps": 20,
-  "loss": {
-    "auc": 0.548176,
-    "empirical_epsilon": {
-      "0.05": 0.1475986074656248,
-      "0.01": 0.0
-    },
-    "empirical_epsilon_details": {
-      "0.05": {
-        "epsilon": 0.1475986074656248,
-        "num_guesses": 65,
-        "correct_guesses": 42,
-        "candidate_num_guesses": [
-          5,
-          10,
-          15,
-          20,
-          25,
-          30,
-          35,
-          40,
-          45,
-          50,
-          55,
-          60,
-          65,
-          70,
-          75,
-          80,
-          85,
-          90,
-          95,
-          100
-        ],
-        "direction": "lower"
-      },
-      "0.01": {
-        "epsilon": 0.0,
-        "num_guesses": 0,
-        "correct_guesses": 0,
-        "candidate_num_guesses": [
-          5,
-          10,
-          15,
-          20,
-          25,
-          30,
-          35,
-          40,
-          45,
-          50,
-          55,
-          60,
-          65,
-          70,
-          75,
-          80,
-          85,
-          90,
-          95,
-          100
-        ],
-        "direction": "lower"
-      }
-    }
-  },
-  "embedding": {
-    "auc": 0.52796,
-    "empirical_epsilon": {
-      "0.05": 0.0,
-      "0.01": 0.0
-    },
-    "empirical_epsilon_details": {
-      "0.05": {
-        "epsilon": 0.0,
-        "num_guesses": 0,
-        "correct_guesses": 0,
-        "candidate_num_guesses": [
-          5,
-          10,
-          15,
-          20,
-          25,
-          30,
-          35,
-          40,
-          45,
-          50,
-          55,
-          60,
-          65,
-          70,
-          75,
-          80,
-          85,
-          90,
-          95,
-          100
-        ],
-        "direction": "lower"
-      },
-      "0.01": {
-        "epsilon": 0.0,
-        "num_guesses": 0,
-        "correct_guesses": 0,
-        "candidate_num_guesses": [
-          5,
-          10,
-          15,
-          20,
-          25,
-          30,
-          35,
-          40,
-          45,
-          50,
-          55,
-          60,
-          65,
-          70,
-          75,
-          80,
-          85,
-          90,
-          95,
-          100
-        ],
-        "direction": "lower"
-      }
-    }
-  }
-}
\ No newline at end of file
diff --git a/granite-4.0-h-tiny/dp3_attn/epochs/epoch_003/adapter/README.md b/granite-4.0-h-tiny/dp3_attn/epochs/epoch_003/adapter/README.md
deleted file mode 100644
index 4c79e7cadd1f7126ea2bc554117a0efbf5b341ab..0000000000000000000000000000000000000000
--- a/granite-4.0-h-tiny/dp3_attn/epochs/epoch_003/adapter/README.md
+++ /dev/null
@@ -1,207 +0,0 @@
----
-base_model: ibm-granite/granite-4.0-h-tiny-base
-library_name: peft
-pipeline_tag: text-generation
-tags:
-- base_model:adapter:ibm-granite/granite-4.0-h-tiny-base
-- lora
-- transformers
----
-
-# Model Card for Model ID
-
-<!-- Provide a quick summary of what the model is/does. -->
-
-
-
-## Model Details
-
-### Model Description
-
-<!-- Provide a longer summary of what this model is. -->
-
-
-
-- **Developed by:** [More Information Needed]
-- **Funded by [optional]:** [More Information Needed]
-- **Shared by [optional]:** [More Information Needed]
-- **Model type:** [More Information Needed]
-- **Language(s) (NLP):** [More Information Needed]
-- **License:** [More Information Needed]
-- **Finetuned from model [optional]:** [More Information Needed]
-
-### Model Sources [optional]
-
-<!-- Provide the basic links for the model. -->
-
-- **Repository:** [More Information Needed]
-- **Paper [optional]:** [More Information Needed]
-- **Demo [optional]:** [More Information Needed]
-
-## Uses
-
-<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
-
-### Direct Use
-
-<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
-
-[More Information Needed]
-
-### Downstream Use [optional]
-
-<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
-
-[More Information Needed]
-
-### Out-of-Scope Use
-
-<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
-
-[More Information Needed]
-
-## Bias, Risks, and Limitations
-
-<!-- This section is meant to convey both technical and sociotechnical limitations. -->
-
-[More Information Needed]
-
-### Recommendations
-
-<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
-
-Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
-
-## How to Get Started with the Model
-
-Use the code below to get started with the model.
-
-[More Information Needed]
-
-## Training Details
-
-### Training Data
-
-<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
-
-[More Information Needed]
-
-### Training Procedure
-
-<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
-
-#### Preprocessing [optional]
-
-[More Information Needed]
-
-
-#### Training Hyperparameters
-
-- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
-
-#### Speeds, Sizes, Times [optional]
-
-<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
-
-[More Information Needed]
-
-## Evaluation
-
-<!-- This section describes the evaluation protocols and provides the results. -->
-
-### Testing Data, Factors & Metrics
-
-#### Testing Data
-
-<!-- This should link to a Dataset Card if possible. -->
-
-[More Information Needed]
-
-#### Factors
-
-<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
-
-[More Information Needed]
-
-#### Metrics
-
-<!-- These are the evaluation metrics being used, ideally with a description of why. -->
-
-[More Information Needed]
-
-### Results
-
-[More Information Needed]
-
-#### Summary
-
-
-
-## Model Examination [optional]
-
-<!-- Relevant interpretability work for the model goes here -->
-
-[More Information Needed]
-
-## Environmental Impact
-
-<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
-
-Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
-
-- **Hardware Type:** [More Information Needed]
-- **Hours used:** [More Information Needed]
-- **Cloud Provider:** [More Information Needed]
-- **Compute Region:** [More Information Needed]
-- **Carbon Emitted:** [More Information Needed]
-
-## Technical Specifications [optional]
-
-### Model Architecture and Objective
-
-[More Information Needed]
-
-### Compute Infrastructure
-
-[More Information Needed]
-
-#### Hardware
-
-[More Information Needed]
-
-#### Software
-
-[More Information Needed]
-
-## Citation [optional]
-
-<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
-
-**BibTeX:**
-
-[More Information Needed]
-
-**APA:**
-
-[More Information Needed]
-
-## Glossary [optional]
-
-<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
-
-[More Information Needed]
-
-## More Information [optional]
-
-[More Information Needed]
-
-## Model Card Authors [optional]
-
-[More Information Needed]
-
-## Model Card Contact
-
-[More Information Needed]
-### Framework versions
-
-- PEFT 0.18.1
\ No newline at end of file
diff --git a/granite-4.0-h-tiny/dp3_attn/epochs/epoch_003/adapter/adapter_config.json b/granite-4.0-h-tiny/dp3_attn/epochs/epoch_003/adapter/adapter_config.json
deleted file mode 100644
index 4a26cfc6fc851cabaca33dc7dae19ef8ef3bd2aa..0000000000000000000000000000000000000000
--- a/granite-4.0-h-tiny/dp3_attn/epochs/epoch_003/adapter/adapter_config.json
+++ /dev/null
@@ -1,46 +0,0 @@
-{
-  "alora_invocation_tokens": null,
-  "alpha_pattern": {},
-  "arrow_config": null,
-  "auto_mapping": null,
-  "base_model_name_or_path": "ibm-granite/granite-4.0-h-tiny-base",
-  "bias": "none",
-  "corda_config": null,
-  "ensure_weight_tying": true,
-  "eva_config": null,
-  "exclude_modules": null,
-  "fan_in_fan_out": false,
-  "inference_mode": true,
-  "init_lora_weights": true,
-  "layer_replication": null,
-  "layers_pattern": null,
-  "layers_to_transform": null,
-  "loftq_config": {},
-  "lora_alpha": 32,
-  "lora_bias": false,
-  "lora_dropout": 0.05,
-  "megatron_config": null,
-  "megatron_core": "megatron.core",
-  "modules_to_save": [
-    "lm_head",
-    "embed_tokens"
-  ],
-  "peft_type": "LORA",
-  "peft_version": "0.18.1",
-  "qalora_group_size": 16,
-  "r": 16,
-  "rank_pattern": {},
-  "revision": null,
-  "target_modules": [
-    "v_proj",
-    "o_proj",
-    "k_proj",
-    "q_proj"
-  ],
-  "target_parameters": null,
-  "task_type": "CAUSAL_LM",
-  "trainable_token_indices": null,
-  "use_dora": false,
-  "use_qalora": false,
-  "use_rslora": false
-}
\ No newline at end of file
diff --git a/granite-4.0-h-tiny/dp3_attn/epochs/epoch_003/audit_results.json b/granite-4.0-h-tiny/dp3_attn/epochs/epoch_003/audit_results.json
deleted file mode 100644
index d1451f21ff6400fdd51a885cb244a3e88bd7daf1..0000000000000000000000000000000000000000
--- a/granite-4.0-h-tiny/dp3_attn/epochs/epoch_003/audit_results.json
+++ /dev/null
@@ -1,137 +0,0 @@
-{
-  "delta": 1e-05,
-  "num_canaries": 500,
-  "num_members": 250,
-  "paper_guess_fraction": 0.2,
-  "paper_guess_steps": 20,
-  "loss": {
-    "auc": 0.551136,
-    "empirical_epsilon": {
-      "0.05": 0.13380606938153505,
-      "0.01": 0.0
-    },
-    "empirical_epsilon_details": {
-      "0.05": {
-        "epsilon": 0.13380606938153505,
-        "num_guesses": 45,
-        "correct_guesses": 30,
-        "candidate_num_guesses": [
-          5,
-          10,
-          15,
-          20,
-          25,
-          30,
-          35,
-          40,
-          45,
-          50,
-          55,
-          60,
-          65,
-          70,
-          75,
-          80,
-          85,
-          90,
-          95,
-          100
-        ],
-        "direction": "lower"
-      },
-      "0.01": {
-        "epsilon": 0.0,
-        "num_guesses": 0,
-        "correct_guesses": 0,
-        "candidate_num_guesses": [
-          5,
-          10,
-          15,
-          20,
-          25,
-          30,
-          35,
-          40,
-          45,
-          50,
-          55,
-          60,
-          65,
-          70,
-          75,
-          80,
-          85,
-          90,
-          95,
-          100
-        ],
-        "direction": "lower"
-      }
-    }
-  },
-  "embedding": {
-    "auc": 0.527048,
-    "empirical_epsilon": {
-      "0.05": 0.0,
-      "0.01": 0.0
-    },
-    "empirical_epsilon_details": {
-      "0.05": {
-        "epsilon": 0.0,
-        "num_guesses": 0,
-        "correct_guesses": 0,
-        "candidate_num_guesses": [
-          5,
-          10,
-          15,
-          20,
-          25,
-          30,
-          35,
-          40,
-          45,
-          50,
-          55,
-          60,
-          65,
-          70,
-          75,
-          80,
-          85,
-          90,
-          95,
-          100
-        ],
-        "direction": "lower"
-      },
-      "0.01": {
-        "epsilon": 0.0,
-        "num_guesses": 0,
-        "correct_guesses": 0,
-        "candidate_num_guesses": [
-          5,
-          10,
-          15,
-          20,
-          25,
-          30,
-          35,
-          40,
-          45,
-          50,
-          55,
-          60,
-          65,
-          70,
-          75,
-          80,
-          85,
-          90,
-          95,
-          100
-        ],
-        "direction": "lower"
-      }
-    }
-  }
-}
\ No newline at end of file
diff --git a/granite-4.0-h-tiny/dp8_attn/epochs/epoch_001/adapter/README.md b/granite-4.0-h-tiny/dp8_attn/epochs/epoch_001/adapter/README.md
deleted file mode 100644
index 4c79e7cadd1f7126ea2bc554117a0efbf5b341ab..0000000000000000000000000000000000000000
--- a/granite-4.0-h-tiny/dp8_attn/epochs/epoch_001/adapter/README.md
+++ /dev/null
@@ -1,207 +0,0 @@
----
-base_model: ibm-granite/granite-4.0-h-tiny-base
-library_name: peft
-pipeline_tag: text-generation
-tags:
-- base_model:adapter:ibm-granite/granite-4.0-h-tiny-base
-- lora
-- transformers
----
-
-# Model Card for Model ID
-
-<!-- Provide a quick summary of what the model is/does. -->
-
-
-
-## Model Details
-
-### Model Description
-
-<!-- Provide a longer summary of what this model is. -->
-
-
-
-- **Developed by:** [More Information Needed]
-- **Funded by [optional]:** [More Information Needed]
-- **Shared by [optional]:** [More Information Needed]
-- **Model type:** [More Information Needed]
-- **Language(s) (NLP):** [More Information Needed]
-- **License:** [More Information Needed]
-- **Finetuned from model [optional]:** [More Information Needed]
-
-### Model Sources [optional]
-
-<!-- Provide the basic links for the model. -->
-
-- **Repository:** [More Information Needed]
-- **Paper [optional]:** [More Information Needed]
-- **Demo [optional]:** [More Information Needed]
-
-## Uses
-
-<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
-
-### Direct Use
-
-<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
-
-[More Information Needed]
-
-### Downstream Use [optional]
-
-<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
-
-[More Information Needed]
-
-### Out-of-Scope Use
-
-<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
-
-[More Information Needed]
-
-## Bias, Risks, and Limitations
-
-<!-- This section is meant to convey both technical and sociotechnical limitations. -->
-
-[More Information Needed]
-
-### Recommendations
-
-<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
-
-Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
-
-## How to Get Started with the Model
-
-Use the code below to get started with the model.
-
-[More Information Needed]
-
-## Training Details
-
-### Training Data
-
-<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
-
-[More Information Needed]
-
-### Training Procedure
-
-<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
-
-#### Preprocessing [optional]
-
-[More Information Needed]
-
-
-#### Training Hyperparameters
-
-- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
-
-#### Speeds, Sizes, Times [optional]
-
-<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
-
-[More Information Needed]
-
-## Evaluation
-
-<!-- This section describes the evaluation protocols and provides the results. -->
-
-### Testing Data, Factors & Metrics
-
-#### Testing Data
-
-<!-- This should link to a Dataset Card if possible. -->
-
-[More Information Needed]
-
-#### Factors
-
-<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
-
-[More Information Needed]
-
-#### Metrics
-
-<!-- These are the evaluation metrics being used, ideally with a description of why. -->
-
-[More Information Needed]
-
-### Results
-
-[More Information Needed]
-
-#### Summary
-
-
-
-## Model Examination [optional]
-
-<!-- Relevant interpretability work for the model goes here -->
-
-[More Information Needed]
-
-## Environmental Impact
-
-<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
-
-Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
-
-- **Hardware Type:** [More Information Needed]
-- **Hours used:** [More Information Needed]
-- **Cloud Provider:** [More Information Needed]
-- **Compute Region:** [More Information Needed]
-- **Carbon Emitted:** [More Information Needed]
-
-## Technical Specifications [optional]
-
-### Model Architecture and Objective
-
-[More Information Needed]
-
-### Compute Infrastructure
-
-[More Information Needed]
-
-#### Hardware
-
-[More Information Needed]
-
-#### Software
-
-[More Information Needed]
-
-## Citation [optional]
-
-<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
-
-**BibTeX:**
-
-[More Information Needed]
-
-**APA:**
-
-[More Information Needed]
-
-## Glossary [optional]
-
-<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
-
-[More Information Needed]
-
-## More Information [optional]
-
-[More Information Needed]
-
-## Model Card Authors [optional]
-
-[More Information Needed]
-
-## Model Card Contact
-
-[More Information Needed]
-### Framework versions
-
-- PEFT 0.18.1
\ No newline at end of file
diff --git a/granite-4.0-h-tiny/dp8_attn/epochs/epoch_001/adapter/adapter_config.json b/granite-4.0-h-tiny/dp8_attn/epochs/epoch_001/adapter/adapter_config.json
deleted file mode 100644
index 490093735f19e66c8428d2ac498c6050a2ab623c..0000000000000000000000000000000000000000
--- a/granite-4.0-h-tiny/dp8_attn/epochs/epoch_001/adapter/adapter_config.json
+++ /dev/null
@@ -1,46 +0,0 @@
-{
-  "alora_invocation_tokens": null,
-  "alpha_pattern": {},
-  "arrow_config": null,
-  "auto_mapping": null,
-  "base_model_name_or_path": "ibm-granite/granite-4.0-h-tiny-base",
-  "bias": "none",
-  "corda_config": null,
-  "ensure_weight_tying": true,
-  "eva_config": null,
-  "exclude_modules": null,
-  "fan_in_fan_out": false,
-  "inference_mode": true,
-  "init_lora_weights": true,
-  "layer_replication": null,
-  "layers_pattern": null,
-  "layers_to_transform": null,
-  "loftq_config": {},
-  "lora_alpha": 32,
-  "lora_bias": false,
-  "lora_dropout": 0.05,
-  "megatron_config": null,
-  "megatron_core": "megatron.core",
-  "modules_to_save": [
-    "lm_head",
-    "embed_tokens"
-  ],
-  "peft_type": "LORA",
-  "peft_version": "0.18.1",
-  "qalora_group_size": 16,
-  "r": 16,
-  "rank_pattern": {},
-  "revision": null,
-  "target_modules": [
-    "q_proj",
-    "v_proj",
-    "o_proj",
-    "k_proj"
-  ],
-  "target_parameters": null,
-  "task_type": "CAUSAL_LM",
-  "trainable_token_indices": null,
-  "use_dora": false,
-  "use_qalora": false,
-  "use_rslora": false
-}
\ No newline at end of file
diff --git a/granite-4.0-h-tiny/dp8_attn/epochs/epoch_001/audit_results.json b/granite-4.0-h-tiny/dp8_attn/epochs/epoch_001/audit_results.json
deleted file mode 100644
index cfa1c2286ac7b6d92679addf741689f232eae039..0000000000000000000000000000000000000000
--- a/granite-4.0-h-tiny/dp8_attn/epochs/epoch_001/audit_results.json
+++ /dev/null
@@ -1,137 +0,0 @@
-{
-  "delta": 1e-05,
-  "num_canaries": 500,
-  "num_members": 250,
-  "paper_guess_fraction": 0.2,
-  "paper_guess_steps": 20,
-  "loss": {
-    "auc": 0.50788,
-    "empirical_epsilon": {
-      "0.05": 0.0,
-      "0.01": 0.0
-    },
-    "empirical_epsilon_details": {
-      "0.05": {
-        "epsilon": 0.0,
-        "num_guesses": 0,
-        "correct_guesses": 0,
-        "candidate_num_guesses": [
-          5,
-          10,
-          15,
-          20,
-          25,
-          30,
-          35,
-          40,
-          45,
-          50,
-          55,
-          60,
-          65,
-          70,
-          75,
-          80,
-          85,
-          90,
-          95,
-          100
-        ],
-        "direction": "lower"
-      },
-      "0.01": {
-        "epsilon": 0.0,
-        "num_guesses": 0,
-        "correct_guesses": 0,
-        "candidate_num_guesses": [
-          5,
-          10,
-          15,
-          20,
-          25,
-          30,
-          35,
-          40,
-          45,
-          50,
-          55,
-          60,
-          65,
-          70,
-          75,
-          80,
-          85,
-          90,
-          95,
-          100
-        ],
-        "direction": "lower"
-      }
-    }
-  },
-  "embedding": {
-    "auc": 0.527256,
-    "empirical_epsilon": {
-      "0.05": 0.0,
-      "0.01": 0.0
-    },
-    "empirical_epsilon_details": {
-      "0.05": {
-        "epsilon": 0.0,
-        "num_guesses": 0,
-        "correct_guesses": 0,
-        "candidate_num_guesses": [
-          5,
-          10,
-          15,
-          20,
-          25,
-          30,
-          35,
-          40,
-          45,
-          50,
-          55,
-          60,
-          65,
-          70,
-          75,
-          80,
-          85,
-          90,
-          95,
-          100
-        ],
-        "direction": "lower"
-      },
-      "0.01": {
-        "epsilon": 0.0,
-        "num_guesses": 0,
-        "correct_guesses": 0,
-        "candidate_num_guesses": [
-          5,
-          10,
-          15,
-          20,
-          25,
-          30,
-          35,
-          40,
-          45,
-          50,
-          55,
-          60,
-          65,
-          70,
-          75,
-          80,
-          85,
-          90,
-          95,
-          100
-        ],
-        "direction": "lower"
-      }
-    }
-  }
-}
\ No newline at end of file
diff --git a/granite-4.0-h-tiny/dp8_attn/epochs/epoch_002/adapter/README.md b/granite-4.0-h-tiny/dp8_attn/epochs/epoch_002/adapter/README.md
deleted file mode 100644
index 4c79e7cadd1f7126ea2bc554117a0efbf5b341ab..0000000000000000000000000000000000000000
--- a/granite-4.0-h-tiny/dp8_attn/epochs/epoch_002/adapter/README.md
+++ /dev/null
@@ -1,207 +0,0 @@
----
-base_model: ibm-granite/granite-4.0-h-tiny-base
-library_name: peft
-pipeline_tag: text-generation
-tags:
-- base_model:adapter:ibm-granite/granite-4.0-h-tiny-base
-- lora
-- transformers
----
-
-# Model Card for Model ID
-
-<!-- Provide a quick summary of what the model is/does. -->
-
-
-
-## Model Details
-
-### Model Description
-
-<!-- Provide a longer summary of what this model is. -->
-
-
-
-- **Developed by:** [More Information Needed]
-- **Funded by [optional]:** [More Information Needed]
-- **Shared by [optional]:** [More Information Needed]
-- **Model type:** [More Information Needed]
-- **Language(s) (NLP):** [More Information Needed]
-- **License:** [More Information Needed]
-- **Finetuned from model [optional]:** [More Information Needed]
-
-### Model Sources [optional]
-
-<!-- Provide the basic links for the model. -->
-
-- **Repository:** [More Information Needed]
-- **Paper [optional]:** [More Information Needed]
-- **Demo [optional]:** [More Information Needed]
-
-## Uses
-
-<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
-
-### Direct Use
-
-<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
-
-[More Information Needed]
-
-### Downstream Use [optional]
-
-<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
-
-[More Information Needed]
-
-### Out-of-Scope Use
-
-<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
-
-[More Information Needed]
-
-## Bias, Risks, and Limitations
-
-<!-- This section is meant to convey both technical and sociotechnical limitations. -->
-
-[More Information Needed]
-
-### Recommendations
-
-<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
-
-Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
-
-## How to Get Started with the Model
-
-Use the code below to get started with the model.
-
-[More Information Needed]
-
-## Training Details
-
-### Training Data
-
-<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
-
-[More Information Needed]
-
-### Training Procedure
-
-<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
-
-#### Preprocessing [optional]
-
-[More Information Needed]
-
-
-#### Training Hyperparameters
-
-- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
-
-#### Speeds, Sizes, Times [optional]
-
-<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
-
-[More Information Needed]
-
-## Evaluation
-
-<!-- This section describes the evaluation protocols and provides the results. -->
-
-### Testing Data, Factors & Metrics
-
-#### Testing Data
-
-<!-- This should link to a Dataset Card if possible. -->
-
-[More Information Needed]
-
-#### Factors
-
-<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
-
-[More Information Needed]
-
-#### Metrics
-
-<!-- These are the evaluation metrics being used, ideally with a description of why. -->
-
-[More Information Needed]
-
-### Results
-
-[More Information Needed]
-
-#### Summary
-
-
-
-## Model Examination [optional]
-
-<!-- Relevant interpretability work for the model goes here -->
-
-[More Information Needed]
-
-## Environmental Impact
-
-<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
-
-Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
-
-- **Hardware Type:** [More Information Needed]
-- **Hours used:** [More Information Needed]
-- **Cloud Provider:** [More Information Needed]
-- **Compute Region:** [More Information Needed]
-- **Carbon Emitted:** [More Information Needed]
-
-## Technical Specifications [optional]
-
-### Model Architecture and Objective
-
-[More Information Needed]
-
-### Compute Infrastructure
-
-[More Information Needed]
-
-#### Hardware
-
-[More Information Needed]
-
-#### Software
-
-[More Information Needed]
-
-## Citation [optional]
-
-<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
-
-**BibTeX:**
-
-[More Information Needed]
-
-**APA:**
-
-[More Information Needed]
-
-## Glossary [optional]
-
-<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
-
-[More Information Needed]
-
-## More Information [optional]
-
-[More Information Needed]
-
-## Model Card Authors [optional]
-
-[More Information Needed]
-
-## Model Card Contact
-
-[More Information Needed]
-### Framework versions
-
-- PEFT 0.18.1
\ No newline at end of file
diff --git a/granite-4.0-h-tiny/dp8_attn/epochs/epoch_002/adapter/adapter_config.json b/granite-4.0-h-tiny/dp8_attn/epochs/epoch_002/adapter/adapter_config.json
deleted file mode 100644
index 490093735f19e66c8428d2ac498c6050a2ab623c..0000000000000000000000000000000000000000
--- a/granite-4.0-h-tiny/dp8_attn/epochs/epoch_002/adapter/adapter_config.json
+++ /dev/null
@@ -1,46 +0,0 @@
-{
-  "alora_invocation_tokens": null,
-  "alpha_pattern": {},
-  "arrow_config": null,
-  "auto_mapping": null,
-  "base_model_name_or_path": "ibm-granite/granite-4.0-h-tiny-base",
-  "bias": "none",
-  "corda_config": null,
-  "ensure_weight_tying": true,
-  "eva_config": null,
-  "exclude_modules": null,
-  "fan_in_fan_out": false,
-  "inference_mode": true,
-  "init_lora_weights": true,
-  "layer_replication": null,
-  "layers_pattern": null,
-  "layers_to_transform": null,
-  "loftq_config": {},
-  "lora_alpha": 32,
-  "lora_bias": false,
-  "lora_dropout": 0.05,
-  "megatron_config": null,
-  "megatron_core": "megatron.core",
-  "modules_to_save": [
-    "lm_head",
-    "embed_tokens"
-  ],
-  "peft_type": "LORA",
-  "peft_version": "0.18.1",
-  "qalora_group_size": 16,
-  "r": 16,
-  "rank_pattern": {},
-  "revision": null,
-  "target_modules": [
-    "q_proj",
-    "v_proj",
-    "o_proj",
-    "k_proj"
-  ],
-  "target_parameters": null,
-  "task_type": "CAUSAL_LM",
-  "trainable_token_indices": null,
-  "use_dora": false,
-  "use_qalora": false,
-  "use_rslora": false
-}
\ No newline at end of file
diff --git a/granite-4.0-h-tiny/dp8_attn/epochs/epoch_002/audit_results.json b/granite-4.0-h-tiny/dp8_attn/epochs/epoch_002/audit_results.json
deleted file mode 100644
index 35532ecbd5fd2bd4edbfc797440d652feabe5cd0..0000000000000000000000000000000000000000
--- a/granite-4.0-h-tiny/dp8_attn/epochs/epoch_002/audit_results.json
+++ /dev/null
@@ -1,137 +0,0 @@
-{
-  "delta": 1e-05,
-  "num_canaries": 500,
-  "num_members": 250,
-  "paper_guess_fraction": 0.2,
-  "paper_guess_steps": 20,
-  "loss": {
-    "auc": 0.509568,
-    "empirical_epsilon": {
-      "0.05": 0.0,
-      "0.01": 0.0
-    },
-    "empirical_epsilon_details": {
-      "0.05": {
-        "epsilon": 0.0,
-        "num_guesses": 0,
-        "correct_guesses": 0,
-        "candidate_num_guesses": [
-          5,
-          10,
-          15,
-          20,
-          25,
-          30,
-          35,
-          40,
-          45,
-          50,
-          55,
-          60,
-          65,
-          70,
-          75,
-          80,
-          85,
-          90,
-          95,
-          100
-        ],
-        "direction": "lower"
-      },
-      "0.01": {
-        "epsilon": 0.0,
-        "num_guesses": 0,
-        "correct_guesses": 0,
-        "candidate_num_guesses": [
-          5,
-          10,
-          15,
-          20,
-          25,
-          30,
-          35,
-          40,
-          45,
-          50,
-          55,
-          60,
-          65,
-          70,
-          75,
-          80,
-          85,
-          90,
-          95,
-          100
-        ],
-        "direction": "lower"
-      }
-    }
-  },
-  "embedding": {
-    "auc": 0.517992,
-    "empirical_epsilon": {
-      "0.05": 0.0,
-      "0.01": 0.0
-    },
-    "empirical_epsilon_details": {
-      "0.05": {
-        "epsilon": 0.0,
-        "num_guesses": 0,
-        "correct_guesses": 0,
-        "candidate_num_guesses": [
-          5,
-          10,
-          15,
-          20,
-          25,
-          30,
-          35,
-          40,
-          45,
-          50,
-          55,
-          60,
-          65,
-          70,
-          75,
-          80,
-          85,
-          90,
-          95,
-          100
-        ],
-        "direction": "lower"
-      },
-      "0.01": {
-        "epsilon": 0.0,
-        "num_guesses": 0,
-        "correct_guesses": 0,
-        "candidate_num_guesses": [
-          5,
-          10,
-          15,
-          20,
-          25,
-          30,
-          35,
-          40,
-          45,
-          50,
-          55,
-          60,
-          65,
-          70,
-          75,
-          80,
-          85,
-          90,
-          95,
-          100
-        ],
-        "direction": "lower"
-      }
-    }
-  }
-}
\ No newline at end of file
diff --git a/granite-4.0-h-tiny/dp8_attn/epochs/epoch_003/adapter/README.md b/granite-4.0-h-tiny/dp8_attn/epochs/epoch_003/adapter/README.md
deleted file mode 100644
index 4c79e7cadd1f7126ea2bc554117a0efbf5b341ab..0000000000000000000000000000000000000000
--- a/granite-4.0-h-tiny/dp8_attn/epochs/epoch_003/adapter/README.md
+++ /dev/null
@@ -1,207 +0,0 @@
----
-base_model: ibm-granite/granite-4.0-h-tiny-base
-library_name: peft
-pipeline_tag: text-generation
-tags:
-- base_model:adapter:ibm-granite/granite-4.0-h-tiny-base
-- lora
-- transformers
----
-
-# Model Card for Model ID
-
-<!-- Provide a quick summary of what the model is/does. -->
-
-
-
-## Model Details
-
-### Model Description
-
-<!-- Provide a longer summary of what this model is. -->
-
-
-
-- **Developed by:** [More Information Needed]
-- **Funded by [optional]:** [More Information Needed]
-- **Shared by [optional]:** [More Information Needed]
-- **Model type:** [More Information Needed]
-- **Language(s) (NLP):** [More Information Needed]
-- **License:** [More Information Needed]
-- **Finetuned from model [optional]:** [More Information Needed]
-
-### Model Sources [optional]
-
-<!-- Provide the basic links for the model. -->
-
-- **Repository:** [More Information Needed]
-- **Paper [optional]:** [More Information Needed]
-- **Demo [optional]:** [More Information Needed]
-
-## Uses
-
-<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
-
-### Direct Use
-
-<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
-
-[More Information Needed]
-
-### Downstream Use [optional]
-
-<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
-
-[More Information Needed]
-
-### Out-of-Scope Use
-
-<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
-
-[More Information Needed]
-
-## Bias, Risks, and Limitations
-
-<!-- This section is meant to convey both technical and sociotechnical limitations. -->
-
-[More Information Needed]
-
-### Recommendations
-
-<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
-
-Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
-
-## How to Get Started with the Model
-
-Use the code below to get started with the model.
-
-[More Information Needed]
-
-## Training Details
-
-### Training Data
-
-<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
-
-[More Information Needed]
-
-### Training Procedure
-
-<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
-
-#### Preprocessing [optional]
-
-[More Information Needed]
-
-
-#### Training Hyperparameters
-
-- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
-
-#### Speeds, Sizes, Times [optional]
-
-<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
-
-[More Information Needed]
-
-## Evaluation
-
-<!-- This section describes the evaluation protocols and provides the results. -->
-
-### Testing Data, Factors & Metrics
-
-#### Testing Data
-
-<!-- This should link to a Dataset Card if possible. -->
-
-[More Information Needed]
-
-#### Factors
-
-<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
-
-[More Information Needed]
-
-#### Metrics
-
-<!-- These are the evaluation metrics being used, ideally with a description of why. -->
-
-[More Information Needed]
-
-### Results
-
-[More Information Needed]
-
-#### Summary
-
-
-
-## Model Examination [optional]
-
-<!-- Relevant interpretability work for the model goes here -->
-
-[More Information Needed]
-
-## Environmental Impact
-
-<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
-
-Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
-
-- **Hardware Type:** [More Information Needed]
-- **Hours used:** [More Information Needed]
-- **Cloud Provider:** [More Information Needed]
-- **Compute Region:** [More Information Needed]
-- **Carbon Emitted:** [More Information Needed]
-
-## Technical Specifications [optional]
-
-### Model Architecture and Objective
-
-[More Information Needed]
-
-### Compute Infrastructure
-
-[More Information Needed]
-
-#### Hardware
-
-[More Information Needed]
-
-#### Software
-
-[More Information Needed]
-
-## Citation [optional]
-
-<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
-
-**BibTeX:**
-
-[More Information Needed]
-
-**APA:**
-
-[More Information Needed]
-
-## Glossary [optional]
-
-<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
-
-[More Information Needed]
-
-## More Information [optional]
-
-[More Information Needed]
-
-## Model Card Authors [optional]
-
-[More Information Needed]
-
-## Model Card Contact
-
-[More Information Needed]
-### Framework versions
-
-- PEFT 0.18.1
\ No newline at end of file
diff --git a/granite-4.0-h-tiny/dp8_attn/epochs/epoch_003/adapter/adapter_config.json b/granite-4.0-h-tiny/dp8_attn/epochs/epoch_003/adapter/adapter_config.json
deleted file mode 100644
index 490093735f19e66c8428d2ac498c6050a2ab623c..0000000000000000000000000000000000000000
--- a/granite-4.0-h-tiny/dp8_attn/epochs/epoch_003/adapter/adapter_config.json
+++ /dev/null
@@ -1,46 +0,0 @@
-{
-  "alora_invocation_tokens": null,
-  "alpha_pattern": {},
-  "arrow_config": null,
-  "auto_mapping": null,
-  "base_model_name_or_path": "ibm-granite/granite-4.0-h-tiny-base",
-  "bias": "none",
-  "corda_config": null,
-  "ensure_weight_tying": true,
-  "eva_config": null,
-  "exclude_modules": null,
-  "fan_in_fan_out": false,
-  "inference_mode": true,
-  "init_lora_weights": true,
-  "layer_replication": null,
-  "layers_pattern": null,
-  "layers_to_transform": null,
-  "loftq_config": {},
-  "lora_alpha": 32,
-  "lora_bias": false,
-  "lora_dropout": 0.05,
-  "megatron_config": null,
-  "megatron_core": "megatron.core",
-  "modules_to_save": [
-    "lm_head",
-    "embed_tokens"
-  ],
-  "peft_type": "LORA",
-  "peft_version": "0.18.1",
-  "qalora_group_size": 16,
-  "r": 16,
-  "rank_pattern": {},
-  "revision": null,
-  "target_modules": [
-    "q_proj",
-    "v_proj",
-    "o_proj",
-    "k_proj"
-  ],
-  "target_parameters": null,
-  "task_type": "CAUSAL_LM",
-  "trainable_token_indices": null,
-  "use_dora": false,
-  "use_qalora": false,
-  "use_rslora": false
-}
\ No newline at end of file
diff --git a/granite-4.0-h-tiny/dp8_attn/epochs/epoch_003/audit_results.json b/granite-4.0-h-tiny/dp8_attn/epochs/epoch_003/audit_results.json
deleted file mode 100644
index 907940fec689a335271e7117f6591bda8b7f1020..0000000000000000000000000000000000000000
--- a/granite-4.0-h-tiny/dp8_attn/epochs/epoch_003/audit_results.json
+++ /dev/null
@@ -1,137 +0,0 @@
-{
-  "delta": 1e-05,
-  "num_canaries": 500,
-  "num_members": 250,
-  "paper_guess_fraction": 0.2,
-  "paper_guess_steps": 20,
-  "loss": {
-    "auc": 0.508808,
-    "empirical_epsilon": {
-      "0.05": 0.0,
-      "0.01": 0.0
-    },
-    "empirical_epsilon_details": {
-      "0.05": {
-        "epsilon": 0.0,
-        "num_guesses": 0,
-        "correct_guesses": 0,
-        "candidate_num_guesses": [
-          5,
-          10,
-          15,
-          20,
-          25,
-          30,
-          35,
-          40,
-          45,
-          50,
-          55,
-          60,
-          65,
-          70,
-          75,
-          80,
-          85,
-          90,
-          95,
-          100
-        ],
-        "direction": "lower"
-      },
-      "0.01": {
-        "epsilon": 0.0,
-        "num_guesses": 0,
-        "correct_guesses": 0,
-        "candidate_num_guesses": [
-          5,
-          10,
-          15,
-          20,
-          25,
-          30,
-          35,
-          40,
-          45,
-          50,
-          55,
-          60,
-          65,
-          70,
-          75,
-          80,
-          85,
-          90,
-          95,
-          100
-        ],
-        "direction": "lower"
-      }
-    }
-  },
-  "embedding": {
-    "auc": 0.521224,
-    "empirical_epsilon": {
-      "0.05": 0.0,
-      "0.01": 0.0
-    },
-    "empirical_epsilon_details": {
-      "0.05": {
-        "epsilon": 0.0,
-        "num_guesses": 0,
-        "correct_guesses": 0,
-        "candidate_num_guesses": [
-          5,
-          10,
-          15,
-          20,
-          25,
-          30,
-          35,
-          40,
-          45,
-          50,
-          55,
-          60,
-          65,
-          70,
-          75,
-          80,
-          85,
-          90,
-          95,
-          100
-        ],
-        "direction": "lower"
-      },
-      "0.01": {
-        "epsilon": 0.0,
-        "num_guesses": 0,
-        "correct_guesses": 0,
-        "candidate_num_guesses": [
-          5,
-          10,
-          15,
-          20,
-          25,
-          30,
-          35,
-          40,
-          45,
-          50,
-          55,
-          60,
-          65,
-          70,
-          75,
-          80,
-          85,
-          90,
-          95,
-          100
-        ],
-        "direction": "lower"
-      }
-    }
-  }
-}
\ No newline at end of file
diff --git a/llama3-8b/base/epochs/epoch_001/adapter/README.md b/llama3-8b/base/epochs/epoch_001/adapter/README.md
deleted file mode 100644
index d99a36f688caa7f483f250cdbf16c2feb9635fef..0000000000000000000000000000000000000000
--- a/llama3-8b/base/epochs/epoch_001/adapter/README.md
+++ /dev/null
@@ -1,207 +0,0 @@
----
-base_model: meta-llama/Meta-Llama-3-8B
-library_name: peft
-pipeline_tag: text-generation
-tags:
-- base_model:adapter:meta-llama/Meta-Llama-3-8B
-- lora
-- transformers
----
-
-# Model Card for Model ID
-
-<!-- Provide a quick summary of what the model is/does. -->
-
-
-
-## Model Details
-
-### Model Description
-
-<!-- Provide a longer summary of what this model is. -->
-
-
-
-- **Developed by:** [More Information Needed]
-- **Funded by [optional]:** [More Information Needed]
-- **Shared by [optional]:** [More Information Needed]
-- **Model type:** [More Information Needed]
-- **Language(s) (NLP):** [More Information Needed]
-- **License:** [More Information Needed]
-- **Finetuned from model [optional]:** [More Information Needed]
-
-### Model Sources [optional]
-
-<!-- Provide the basic links for the model. -->
-
-- **Repository:** [More Information Needed]
-- **Paper [optional]:** [More Information Needed]
-- **Demo [optional]:** [More Information Needed]
-
-## Uses
-
-<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
-
-### Direct Use
-
-<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
-
-[More Information Needed]
-
-### Downstream Use [optional]
-
-<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
-
-[More Information Needed]
-
-### Out-of-Scope Use
-
-<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
-
-[More Information Needed]
-
-## Bias, Risks, and Limitations
-
-<!-- This section is meant to convey both technical and sociotechnical limitations. -->
-
-[More Information Needed]
-
-### Recommendations
-
-<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
-
-Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
-
-## How to Get Started with the Model
-
-Use the code below to get started with the model.
-
-[More Information Needed]
-
-## Training Details
-
-### Training Data
-
-<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
-
-[More Information Needed]
-
-### Training Procedure
-
-<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
-
-#### Preprocessing [optional]
-
-[More Information Needed]
-
-
-#### Training Hyperparameters
-
-- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
-
-#### Speeds, Sizes, Times [optional]
-
-<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
-
-[More Information Needed]
-
-## Evaluation
-
-<!-- This section describes the evaluation protocols and provides the results. -->
-
-### Testing Data, Factors & Metrics
-
-#### Testing Data
-
-<!-- This should link to a Dataset Card if possible. -->
-
-[More Information Needed]
-
-#### Factors
-
-<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
-
-[More Information Needed]
-
-#### Metrics
-
-<!-- These are the evaluation metrics being used, ideally with a description of why. -->
-
-[More Information Needed]
-
-### Results
-
-[More Information Needed]
-
-#### Summary
-
-
-
-## Model Examination [optional]
-
-<!-- Relevant interpretability work for the model goes here -->
-
-[More Information Needed]
-
-## Environmental Impact
-
-<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
-
-Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
-
-- **Hardware Type:** [More Information Needed]
-- **Hours used:** [More Information Needed]
-- **Cloud Provider:** [More Information Needed]
-- **Compute Region:** [More Information Needed]
-- **Carbon Emitted:** [More Information Needed]
-
-## Technical Specifications [optional]
-
-### Model Architecture and Objective
-
-[More Information Needed]
-
-### Compute Infrastructure
-
-[More Information Needed]
-
-#### Hardware
-
-[More Information Needed]
-
-#### Software
-
-[More Information Needed]
-
-## Citation [optional]
-
-<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
-
-**BibTeX:**
-
-[More Information Needed]
-
-**APA:**
-
-[More Information Needed]
-
-## Glossary [optional]
-
-<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
-
-[More Information Needed]
-
-## More Information [optional]
-
-[More Information Needed]
-
-## Model Card Authors [optional]
-
-[More Information Needed]
-
-## Model Card Contact
-
-[More Information Needed]
-### Framework versions
-
-- PEFT 0.18.1
\ No newline at end of file
diff --git a/llama3-8b/base/epochs/epoch_001/adapter/adapter_config.json b/llama3-8b/base/epochs/epoch_001/adapter/adapter_config.json
deleted file mode 100644
index 142654567bf521f9f62f008f7d9e1b6f2e164b7a..0000000000000000000000000000000000000000
--- a/llama3-8b/base/epochs/epoch_001/adapter/adapter_config.json
+++ /dev/null
@@ -1,46 +0,0 @@
-{
-  "alora_invocation_tokens": null,
-  "alpha_pattern": {},
-  "arrow_config": null,
-  "auto_mapping": null,
-  "base_model_name_or_path": "meta-llama/Meta-Llama-3-8B",
-  "bias": "none",
-  "corda_config": null,
-  "ensure_weight_tying": true,
-  "eva_config": null,
-  "exclude_modules": null,
-  "fan_in_fan_out": false,
-  "inference_mode": true,
-  "init_lora_weights": true,
-  "layer_replication": null,
-  "layers_pattern": null,
-  "layers_to_transform": null,
-  "loftq_config": {},
-  "lora_alpha": 32,
-  "lora_bias": false,
-  "lora_dropout": 0.05,
-  "megatron_config": null,
-  "megatron_core": "megatron.core",
-  "modules_to_save": [
-    "lm_head",
-    "embed_tokens"
-  ],
-  "peft_type": "LORA",
-  "peft_version": "0.18.1",
-  "qalora_group_size": 16,
-  "r": 16,
-  "rank_pattern": {},
-  "revision": null,
-  "target_modules": [
-    "q_proj",
-    "v_proj",
-    "o_proj",
-    "k_proj"
-  ],
-  "target_parameters": null,
-  "task_type": "CAUSAL_LM",
-  "trainable_token_indices": null,
-  "use_dora": false,
-  "use_qalora": false,
-  "use_rslora": false
-}
\ No newline at end of file
diff --git a/llama3-8b/base/epochs/epoch_001/audit_results.json b/llama3-8b/base/epochs/epoch_001/audit_results.json
deleted file mode 100644
index 71003b3519abb18421e95f67e05d5d3662250c76..0000000000000000000000000000000000000000
--- a/llama3-8b/base/epochs/epoch_001/audit_results.json
+++ /dev/null
@@ -1,137 +0,0 @@
-{
-  "delta": 1e-05,
-  "num_canaries": 500,
-  "num_members": 250,
-  "paper_guess_fraction": 0.2,
-  "paper_guess_steps": 20,
-  "loss": {
-    "auc": 0.85872,
-    "empirical_epsilon": {
-      "0.05": 3.4791953936219215,
-      "0.01": 3.023197554051876
-    },
-    "empirical_epsilon_details": {
-      "0.05": {
-        "epsilon": 3.4791953936219215,
-        "num_guesses": 100,
-        "correct_guesses": 100,
-        "candidate_num_guesses": [
-          5,
-          10,
-          15,
-          20,
-          25,
-          30,
-          35,
-          40,
-          45,
-          50,
-          55,
-          60,
-          65,
-          70,
-          75,
-          80,
-          85,
-          90,
-          95,
-          100
-        ],
-        "direction": "lower"
-      },
-      "0.01": {
-        "epsilon": 3.023197554051876,
-        "num_guesses": 100,
-        "correct_guesses": 100,
-        "candidate_num_guesses": [
-          5,
-          10,
-          15,
-          20,
-          25,
-          30,
-          35,
-          40,
-          45,
-          50,
-          55,
-          60,
-          65,
-          70,
-          75,
-          80,
-          85,
-          90,
-          95,
-          100
-        ],
-        "direction": "lower"
-      }
-    }
-  },
-  "embedding": {
-    "auc": 0.676,
-    "empirical_epsilon": {
-      "0.05": 3.4791953936219215,
-      "0.01": 3.023197554051876
-    },
-    "empirical_epsilon_details": {
-      "0.05": {
-        "epsilon": 3.4791953936219215,
-        "num_guesses": 100,
-        "correct_guesses": 100,
-        "candidate_num_guesses": [
-          5,
-          10,
-          15,
-          20,
-          25,
-          30,
-          35,
-          40,
-          45,
-          50,
-          55,
-          60,
-          65,
-          70,
-          75,
-          80,
-          85,
-          90,
-          95,
-          100
-        ],
-        "direction": "lower"
-      },
-      "0.01": {
-        "epsilon": 3.023197554051876,
-        "num_guesses": 100,
-        "correct_guesses": 100,
-        "candidate_num_guesses": [
-          5,
-          10,
-          15,
-          20,
-          25,
-          30,
-          35,
-          40,
-          45,
-          50,
-          55,
-          60,
-          65,
-          70,
-          75,
-          80,
-          85,
-          90,
-          95,
-          100
-        ],
-        "direction": "lower"
-      }
-    }
-  }
-}
\ No newline at end of file
diff --git a/llama3-8b/base/epochs/epoch_002/adapter/README.md b/llama3-8b/base/epochs/epoch_002/adapter/README.md
deleted file mode 100644
index d99a36f688caa7f483f250cdbf16c2feb9635fef..0000000000000000000000000000000000000000
--- a/llama3-8b/base/epochs/epoch_002/adapter/README.md
+++ /dev/null
@@ -1,207 +0,0 @@
----
-base_model: meta-llama/Meta-Llama-3-8B
-library_name: peft
-pipeline_tag: text-generation
-tags:
-- base_model:adapter:meta-llama/Meta-Llama-3-8B
-- lora
-- transformers
----
-
-# Model Card for Model ID
-
-<!-- Provide a quick summary of what the model is/does. -->
-
-
-
-## Model Details
-
-### Model Description
-
-<!-- Provide a longer summary of what this model is. -->
-
-
-
-- **Developed by:** [More Information Needed]
-- **Funded by [optional]:** [More Information Needed]
-- **Shared by [optional]:** [More Information Needed]
-- **Model type:** [More Information Needed]
-- **Language(s) (NLP):** [More Information Needed]
-- **License:** [More Information Needed]
-- **Finetuned from model [optional]:** [More Information Needed]
-
-### Model Sources [optional]
-
-<!-- Provide the basic links for the model. -->
-
-- **Repository:** [More Information Needed]
-- **Paper [optional]:** [More Information Needed]
-- **Demo [optional]:** [More Information Needed]
-
-## Uses
-
-<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
-
-### Direct Use
-
-<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
-
-[More Information Needed]
-
-### Downstream Use [optional]
-
-<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
-
-[More Information Needed]
-
-### Out-of-Scope Use
-
-<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
-
-[More Information Needed]
-
-## Bias, Risks, and Limitations
-
-<!-- This section is meant to convey both technical and sociotechnical limitations. -->
-
-[More Information Needed]
-
-### Recommendations
-
-<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
-
-Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
-
-## How to Get Started with the Model
-
-Use the code below to get started with the model.
-
-[More Information Needed]
-
-## Training Details
-
-### Training Data
-
-<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
-
-[More Information Needed]
-
-### Training Procedure
-
-<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
-
-#### Preprocessing [optional]
-
-[More Information Needed]
-
-
-#### Training Hyperparameters
-
-- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
-
-#### Speeds, Sizes, Times [optional]
-
-<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
-
-[More Information Needed]
-
-## Evaluation
-
-<!-- This section describes the evaluation protocols and provides the results. -->
-
-### Testing Data, Factors & Metrics
-
-#### Testing Data
-
-<!-- This should link to a Dataset Card if possible. -->
-
-[More Information Needed]
-
-#### Factors
-
-<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
-
-[More Information Needed]
-
-#### Metrics
-
-<!-- These are the evaluation metrics being used, ideally with a description of why. -->
-
-[More Information Needed]
-
-### Results
-
-[More Information Needed]
-
-#### Summary
-
-
-
-## Model Examination [optional]
-
-<!-- Relevant interpretability work for the model goes here -->
-
-[More Information Needed]
-
-## Environmental Impact
-
-<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
-
-Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
-
-- **Hardware Type:** [More Information Needed]
-- **Hours used:** [More Information Needed]
-- **Cloud Provider:** [More Information Needed]
-- **Compute Region:** [More Information Needed]
-- **Carbon Emitted:** [More Information Needed]
-
-## Technical Specifications [optional]
-
-### Model Architecture and Objective
-
-[More Information Needed]
-
-### Compute Infrastructure
-
-[More Information Needed]
-
-#### Hardware
-
-[More Information Needed]
-
-#### Software
-
-[More Information Needed]
-
-## Citation [optional]
-
-<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
-
-**BibTeX:**
-
-[More Information Needed]
-
-**APA:**
-
-[More Information Needed]
-
-## Glossary [optional]
-
-<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
-
-[More Information Needed]
-
-## More Information [optional]
-
-[More Information Needed]
-
-## Model Card Authors [optional]
-
-[More Information Needed]
-
-## Model Card Contact
-
-[More Information Needed]
-### Framework versions
-
-- PEFT 0.18.1
\ No newline at end of file
diff --git a/llama3-8b/base/epochs/epoch_002/adapter/adapter_config.json b/llama3-8b/base/epochs/epoch_002/adapter/adapter_config.json
deleted file mode 100644
index 142654567bf521f9f62f008f7d9e1b6f2e164b7a..0000000000000000000000000000000000000000
--- a/llama3-8b/base/epochs/epoch_002/adapter/adapter_config.json
+++ /dev/null
@@ -1,46 +0,0 @@
-{
-  "alora_invocation_tokens": null,
-  "alpha_pattern": {},
-  "arrow_config": null,
-  "auto_mapping": null,
-  "base_model_name_or_path": "meta-llama/Meta-Llama-3-8B",
-  "bias": "none",
-  "corda_config": null,
-  "ensure_weight_tying": true,
-  "eva_config": null,
-  "exclude_modules": null,
-  "fan_in_fan_out": false,
-  "inference_mode": true,
-  "init_lora_weights": true,
-  "layer_replication": null,
-  "layers_pattern": null,
-  "layers_to_transform": null,
-  "loftq_config": {},
-  "lora_alpha": 32,
-  "lora_bias": false,
-  "lora_dropout": 0.05,
-  "megatron_config": null,
-  "megatron_core": "megatron.core",
-  "modules_to_save": [
-    "lm_head",
-    "embed_tokens"
-  ],
-  "peft_type": "LORA",
-  "peft_version": "0.18.1",
-  "qalora_group_size": 16,
-  "r": 16,
-  "rank_pattern": {},
-  "revision": null,
-  "target_modules": [
-    "q_proj",
-    "v_proj",
-    "o_proj",
-    "k_proj"
-  ],
-  "target_parameters": null,
-  "task_type": "CAUSAL_LM",
-  "trainable_token_indices": null,
-  "use_dora": false,
-  "use_qalora": false,
-  "use_rslora": false
-}
\ No newline at end of file
diff --git a/llama3-8b/base/epochs/epoch_002/audit_results.json b/llama3-8b/base/epochs/epoch_002/audit_results.json
deleted file mode 100644
index 7c95a3c12163e93edeb4088bacc4ce1637dcb54e..0000000000000000000000000000000000000000
--- a/llama3-8b/base/epochs/epoch_002/audit_results.json
+++ /dev/null
@@ -1,137 +0,0 @@
-{
-  "delta": 1e-05,
-  "num_canaries": 500,
-  "num_members": 250,
-  "paper_guess_fraction": 0.2,
-  "paper_guess_steps": 20,
-  "loss": {
-    "auc": 0.811752,
-    "empirical_epsilon": {
-      "0.05": 3.4791953936219215,
-      "0.01": 3.023197554051876
-    },
-    "empirical_epsilon_details": {
-      "0.05": {
-        "epsilon": 3.4791953936219215,
-        "num_guesses": 100,
-        "correct_guesses": 100,
-        "candidate_num_guesses": [
-          5,
-          10,
-          15,
-          20,
-          25,
-          30,
-          35,
-          40,
-          45,
-          50,
-          55,
-          60,
-          65,
-          70,
-          75,
-          80,
-          85,
-          90,
-          95,
-          100
-        ],
-        "direction": "lower"
-      },
-      "0.01": {
-        "epsilon": 3.023197554051876,
-        "num_guesses": 100,
-        "correct_guesses": 100,
-        "candidate_num_guesses": [
-          5,
-          10,
-          15,
-          20,
-          25,
-          30,
-          35,
-          40,
-          45,
-          50,
-          55,
-          60,
-          65,
-          70,
-          75,
-          80,
-          85,
-          90,
-          95,
-          100
-        ],
-        "direction": "lower"
-      }
-    }
-  },
-  "embedding": {
-    "auc": 0.5,
-    "empirical_epsilon": {
-      "0.05": 3.4791953936219215,
-      "0.01": 3.023197554051876
-    },
-    "empirical_epsilon_details": {
-      "0.05": {
-        "epsilon": 3.4791953936219215,
-        "num_guesses": 100,
-        "correct_guesses": 100,
-        "candidate_num_guesses": [
-          5,
-          10,
-          15,
-          20,
-          25,
-          30,
-          35,
-          40,
-          45,
-          50,
-          55,
-          60,
-          65,
-          70,
-          75,
-          80,
-          85,
-          90,
-          95,
-          100
-        ],
-        "direction": "lower"
-      },
-      "0.01": {
-        "epsilon": 3.023197554051876,
-        "num_guesses": 100,
-        "correct_guesses": 100,
-        "candidate_num_guesses": [
-          5,
-          10,
-          15,
-          20,
-          25,
-          30,
-          35,
-          40,
-          45,
-          50,
-          55,
-          60,
-          65,
-          70,
-          75,
-          80,
-          85,
-          90,
-          95,
-          100
-        ],
-        "direction": "lower"
-      }
-    }
-  }
-}
\ No newline at end of file
diff --git a/llama3-8b/base/epochs/epoch_003/adapter/README.md b/llama3-8b/base/epochs/epoch_003/adapter/README.md
deleted file mode 100644
index d99a36f688caa7f483f250cdbf16c2feb9635fef..0000000000000000000000000000000000000000
--- a/llama3-8b/base/epochs/epoch_003/adapter/README.md
+++ /dev/null
@@ -1,207 +0,0 @@
----
-base_model: meta-llama/Meta-Llama-3-8B
-library_name: peft
-pipeline_tag: text-generation
-tags:
-- base_model:adapter:meta-llama/Meta-Llama-3-8B
-- lora
-- transformers
----
-
-# Model Card for Model ID
-
-<!-- Provide a quick summary of what the model is/does. -->
-
-
-
-## Model Details
-
-### Model Description
-
-<!-- Provide a longer summary of what this model is. -->
-
-
-
-- **Developed by:** [More Information Needed]
-- **Funded by [optional]:** [More Information Needed]
-- **Shared by [optional]:** [More Information Needed]
-- **Model type:** [More Information Needed]
-- **Language(s) (NLP):** [More Information Needed]
-- **License:** [More Information Needed]
-- **Finetuned from model [optional]:** [More Information Needed]
-
-### Model Sources [optional]
-
-<!-- Provide the basic links for the model. -->
-
-- **Repository:** [More Information Needed]
-- **Paper [optional]:** [More Information Needed]
-- **Demo [optional]:** [More Information Needed]
-
-## Uses
-
-<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
-
-### Direct Use
-
-<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
-
-[More Information Needed]
-
-### Downstream Use [optional]
-
-<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
-
-[More Information Needed]
-
-### Out-of-Scope Use
-
-<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
-
-[More Information Needed]
-
-## Bias, Risks, and Limitations
-
-<!-- This section is meant to convey both technical and sociotechnical limitations. -->
-
-[More Information Needed]
-
-### Recommendations
-
-<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
-
-Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
-
-## How to Get Started with the Model
-
-Use the code below to get started with the model.
-
-[More Information Needed]
-
-## Training Details
-
-### Training Data
-
-<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
-
-[More Information Needed]
-
-### Training Procedure
-
-<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
-
-#### Preprocessing [optional]
-
-[More Information Needed]
-
-
-#### Training Hyperparameters
-
-- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
-
-#### Speeds, Sizes, Times [optional]
-
-<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
-
-[More Information Needed]
-
-## Evaluation
-
-<!-- This section describes the evaluation protocols and provides the results. -->
-
-### Testing Data, Factors & Metrics
-
-#### Testing Data
-
-<!-- This should link to a Dataset Card if possible. -->
-
-[More Information Needed]
-
-#### Factors
-
-<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
-
-[More Information Needed]
-
-#### Metrics
-
-<!-- These are the evaluation metrics being used, ideally with a description of why. -->
-
-[More Information Needed]
-
-### Results
-
-[More Information Needed]
-
-#### Summary
-
-
-
-## Model Examination [optional]
-
-<!-- Relevant interpretability work for the model goes here -->
-
-[More Information Needed]
-
-## Environmental Impact
-
-<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
-
-Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
-
-- **Hardware Type:** [More Information Needed]
-- **Hours used:** [More Information Needed]
-- **Cloud Provider:** [More Information Needed]
-- **Compute Region:** [More Information Needed]
-- **Carbon Emitted:** [More Information Needed]
-
-## Technical Specifications [optional]
-
-### Model Architecture and Objective
-
-[More Information Needed]
-
-### Compute Infrastructure
-
-[More Information Needed]
-
-#### Hardware
-
-[More Information Needed]
-
-#### Software
-
-[More Information Needed]
-
-## Citation [optional]
-
-<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
-
-**BibTeX:**
-
-[More Information Needed]
-
-**APA:**
-
-[More Information Needed]
-
-## Glossary [optional]
-
-<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
-
-[More Information Needed]
-
-## More Information [optional]
-
-[More Information Needed]
-
-## Model Card Authors [optional]
-
-[More Information Needed]
-
-## Model Card Contact
-
-[More Information Needed]
-### Framework versions
-
-- PEFT 0.18.1
\ No newline at end of file
diff --git a/llama3-8b/base/epochs/epoch_003/adapter/adapter_config.json b/llama3-8b/base/epochs/epoch_003/adapter/adapter_config.json
deleted file mode 100644
index 142654567bf521f9f62f008f7d9e1b6f2e164b7a..0000000000000000000000000000000000000000
--- a/llama3-8b/base/epochs/epoch_003/adapter/adapter_config.json
+++ /dev/null
@@ -1,46 +0,0 @@
-{
-  "alora_invocation_tokens": null,
-  "alpha_pattern": {},
-  "arrow_config": null,
-  "auto_mapping": null,
-  "base_model_name_or_path": "meta-llama/Meta-Llama-3-8B",
-  "bias": "none",
-  "corda_config": null,
-  "ensure_weight_tying": true,
-  "eva_config": null,
-  "exclude_modules": null,
-  "fan_in_fan_out": false,
-  "inference_mode": true,
-  "init_lora_weights": true,
-  "layer_replication": null,
-  "layers_pattern": null,
-  "layers_to_transform": null,
-  "loftq_config": {},
-  "lora_alpha": 32,
-  "lora_bias": false,
-  "lora_dropout": 0.05,
-  "megatron_config": null,
-  "megatron_core": "megatron.core",
-  "modules_to_save": [
-    "lm_head",
-    "embed_tokens"
-  ],
-  "peft_type": "LORA",
-  "peft_version": "0.18.1",
-  "qalora_group_size": 16,
-  "r": 16,
-  "rank_pattern": {},
-  "revision": null,
-  "target_modules": [
-    "q_proj",
-    "v_proj",
-    "o_proj",
-    "k_proj"
-  ],
-  "target_parameters": null,
-  "task_type": "CAUSAL_LM",
-  "trainable_token_indices": null,
-  "use_dora": false,
-  "use_qalora": false,
-  "use_rslora": false
-}
\ No newline at end of file
diff --git a/llama3-8b/base/epochs/epoch_003/audit_results.json b/llama3-8b/base/epochs/epoch_003/audit_results.json
deleted file mode 100644
index b57056e116161520077981a4927d872ebb0c150d..0000000000000000000000000000000000000000
--- a/llama3-8b/base/epochs/epoch_003/audit_results.json
+++ /dev/null
@@ -1,137 +0,0 @@
-{
-  "delta": 1e-05,
-  "num_canaries": 500,
-  "num_members": 250,
-  "paper_guess_fraction": 0.2,
-  "paper_guess_steps": 20,
-  "loss": {
-    "auc": 0.999392,
-    "empirical_epsilon": {
-      "0.05": 3.4791953936219215,
-      "0.01": 3.023197554051876
-    },
-    "empirical_epsilon_details": {
-      "0.05": {
-        "epsilon": 3.4791953936219215,
-        "num_guesses": 100,
-        "correct_guesses": 100,
-        "candidate_num_guesses": [
-          5,
-          10,
-          15,
-          20,
-          25,
-          30,
-          35,
-          40,
-          45,
-          50,
-          55,
-          60,
-          65,
-          70,
-          75,
-          80,
-          85,
-          90,
-          95,
-          100
-        ],
-        "direction": "lower"
-      },
-      "0.01": {
-        "epsilon": 3.023197554051876,
-        "num_guesses": 100,
-        "correct_guesses": 100,
-        "candidate_num_guesses": [
-          5,
-          10,
-          15,
-          20,
-          25,
-          30,
-          35,
-          40,
-          45,
-          50,
-          55,
-          60,
-          65,
-          70,
-          75,
-          80,
-          85,
-          90,
-          95,
-          100
-        ],
-        "direction": "lower"
-      }
-    }
-  },
-  "embedding": {
-    "auc": 0.558,
-    "empirical_epsilon": {
-      "0.05": 3.4791953936219215,
-      "0.01": 3.023197554051876
-    },
-    "empirical_epsilon_details": {
-      "0.05": {
-        "epsilon": 3.4791953936219215,
-        "num_guesses": 100,
-        "correct_guesses": 100,
-        "candidate_num_guesses": [
-          5,
-          10,
-          15,
-          20,
-          25,
-          30,
-          35,
-          40,
-          45,
-          50,
-          55,
-          60,
-          65,
-          70,
-          75,
-          80,
-          85,
-          90,
-          95,
-          100
-        ],
-        "direction": "lower"
-      },
-      "0.01": {
-        "epsilon": 3.023197554051876,
-        "num_guesses": 100,
-        "correct_guesses": 100,
-        "candidate_num_guesses": [
-          5,
-          10,
-          15,
-          20,
-          25,
-          30,
-          35,
-          40,
-          45,
-          50,
-          55,
-          60,
-          65,
-          70,
-          75,
-          80,
-          85,
-          90,
-          95,
-          100
-        ],
-        "direction": "lower"
-      }
-    }
-  }
-}
\ No newline at end of file
diff --git a/llama3-8b/base/epochs/epoch_004/adapter/README.md b/llama3-8b/base/epochs/epoch_004/adapter/README.md
deleted file mode 100644
index d99a36f688caa7f483f250cdbf16c2feb9635fef..0000000000000000000000000000000000000000
--- a/llama3-8b/base/epochs/epoch_004/adapter/README.md
+++ /dev/null
@@ -1,207 +0,0 @@
----
-base_model: meta-llama/Meta-Llama-3-8B
-library_name: peft
-pipeline_tag: text-generation
-tags:
-- base_model:adapter:meta-llama/Meta-Llama-3-8B
-- lora
-- transformers
----
-
-# Model Card for Model ID
-
-<!-- Provide a quick summary of what the model is/does. -->
-
-
-
-## Model Details
-
-### Model Description
-
-<!-- Provide a longer summary of what this model is. -->
-
-
-
-- **Developed by:** [More Information Needed]
-- **Funded by [optional]:** [More Information Needed]
-- **Shared by [optional]:** [More Information Needed]
-- **Model type:** [More Information Needed]
-- **Language(s) (NLP):** [More Information Needed]
-- **License:** [More Information Needed]
-- **Finetuned from model [optional]:** [More Information Needed]
-
-### Model Sources [optional]
-
-<!-- Provide the basic links for the model. -->
-
-- **Repository:** [More Information Needed]
-- **Paper [optional]:** [More Information Needed]
-- **Demo [optional]:** [More Information Needed]
-
-## Uses
-
-<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
-
-### Direct Use
-
-<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
-
-[More Information Needed]
-
-### Downstream Use [optional]
-
-<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
-
-[More Information Needed]
-
-### Out-of-Scope Use
-
-<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
-
-[More Information Needed]
-
-## Bias, Risks, and Limitations
-
-<!-- This section is meant to convey both technical and sociotechnical limitations. -->
-
-[More Information Needed]
-
-### Recommendations
-
-<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
-
-Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
-
-## How to Get Started with the Model
-
-Use the code below to get started with the model.
-
-[More Information Needed]
-
-## Training Details
-
-### Training Data
-
-<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
-
-[More Information Needed]
-
-### Training Procedure
-
-<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
-
-#### Preprocessing [optional]
-
-[More Information Needed]
-
-
-#### Training Hyperparameters
-
-- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
-
-#### Speeds, Sizes, Times [optional]
-
-<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
-
-[More Information Needed]
-
-## Evaluation
-
-<!-- This section describes the evaluation protocols and provides the results. -->
-
-### Testing Data, Factors & Metrics
-
-#### Testing Data
-
-<!-- This should link to a Dataset Card if possible. -->
-
-[More Information Needed]
-
-#### Factors
-
-<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
-
-[More Information Needed]
-
-#### Metrics
-
-<!-- These are the evaluation metrics being used, ideally with a description of why. -->
-
-[More Information Needed]
-
-### Results
-
-[More Information Needed]
-
-#### Summary
-
-
-
-## Model Examination [optional]
-
-<!-- Relevant interpretability work for the model goes here -->
-
-[More Information Needed]
-
-## Environmental Impact
-
-<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
-
-Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
-
-- **Hardware Type:** [More Information Needed]
-- **Hours used:** [More Information Needed]
-- **Cloud Provider:** [More Information Needed]
-- **Compute Region:** [More Information Needed]
-- **Carbon Emitted:** [More Information Needed]
-
-## Technical Specifications [optional]
-
-### Model Architecture and Objective
-
-[More Information Needed]
-
-### Compute Infrastructure
-
-[More Information Needed]
-
-#### Hardware
-
-[More Information Needed]
-
-#### Software
-
-[More Information Needed]
-
-## Citation [optional]
-
-<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
-
-**BibTeX:**
-
-[More Information Needed]
-
-**APA:**
-
-[More Information Needed]
-
-## Glossary [optional]
-
-<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
-
-[More Information Needed]
-
-## More Information [optional]
-
-[More Information Needed]
-
-## Model Card Authors [optional]
-
-[More Information Needed]
-
-## Model Card Contact
-
-[More Information Needed]
-### Framework versions
-
-- PEFT 0.18.1
\ No newline at end of file
diff --git a/llama3-8b/base/epochs/epoch_004/adapter/adapter_config.json b/llama3-8b/base/epochs/epoch_004/adapter/adapter_config.json
deleted file mode 100644
index 142654567bf521f9f62f008f7d9e1b6f2e164b7a..0000000000000000000000000000000000000000
--- a/llama3-8b/base/epochs/epoch_004/adapter/adapter_config.json
+++ /dev/null
@@ -1,46 +0,0 @@
-{
-  "alora_invocation_tokens": null,
-  "alpha_pattern": {},
-  "arrow_config": null,
-  "auto_mapping": null,
-  "base_model_name_or_path": "meta-llama/Meta-Llama-3-8B",
-  "bias": "none",
-  "corda_config": null,
-  "ensure_weight_tying": true,
-  "eva_config": null,
-  "exclude_modules": null,
-  "fan_in_fan_out": false,
-  "inference_mode": true,
-  "init_lora_weights": true,
-  "layer_replication": null,
-  "layers_pattern": null,
-  "layers_to_transform": null,
-  "loftq_config": {},
-  "lora_alpha": 32,
-  "lora_bias": false,
-  "lora_dropout": 0.05,
-  "megatron_config": null,
-  "megatron_core": "megatron.core",
-  "modules_to_save": [
-    "lm_head",
-    "embed_tokens"
-  ],
-  "peft_type": "LORA",
-  "peft_version": "0.18.1",
-  "qalora_group_size": 16,
-  "r": 16,
-  "rank_pattern": {},
-  "revision": null,
-  "target_modules": [
-    "q_proj",
-    "v_proj",
-    "o_proj",
-    "k_proj"
-  ],
-  "target_parameters": null,
-  "task_type": "CAUSAL_LM",
-  "trainable_token_indices": null,
-  "use_dora": false,
-  "use_qalora": false,
-  "use_rslora": false
-}
\ No newline at end of file
diff --git a/llama3-8b/base/epochs/epoch_004/audit_results.json b/llama3-8b/base/epochs/epoch_004/audit_results.json
deleted file mode 100644
index ab8236002d7e93c190a3b25c7568ffacc3a9e1c6..0000000000000000000000000000000000000000
--- a/llama3-8b/base/epochs/epoch_004/audit_results.json
+++ /dev/null
@@ -1,137 +0,0 @@
-{
-  "delta": 1e-05,
-  "num_canaries": 500,
-  "num_members": 250,
-  "paper_guess_fraction": 0.2,
-  "paper_guess_steps": 20,
-  "loss": {
-    "auc": 1.0,
-    "empirical_epsilon": {
-      "0.05": 3.4791953936219215,
-      "0.01": 3.023197554051876
-    },
-    "empirical_epsilon_details": {
-      "0.05": {
-        "epsilon": 3.4791953936219215,
-        "num_guesses": 100,
-        "correct_guesses": 100,
-        "candidate_num_guesses": [
-          5,
-          10,
-          15,
-          20,
-          25,
-          30,
-          35,
-          40,
-          45,
-          50,
-          55,
-          60,
-          65,
-          70,
-          75,
-          80,
-          85,
-          90,
-          95,
-          100
-        ],
-        "direction": "lower"
-      },
-      "0.01": {
-        "epsilon": 3.023197554051876,
-        "num_guesses": 100,
-        "correct_guesses": 100,
-        "candidate_num_guesses": [
-          5,
-          10,
-          15,
-          20,
-          25,
-          30,
-          35,
-          40,
-          45,
-          50,
-          55,
-          60,
-          65,
-          70,
-          75,
-          80,
-          85,
-          90,
-          95,
-          100
-        ],
-        "direction": "lower"
-      }
-    }
-  },
-  "embedding": {
-    "auc": 0.558,
-    "empirical_epsilon": {
-      "0.05": 3.4791953936219215,
-      "0.01": 3.023197554051876
-    },
-    "empirical_epsilon_details": {
-      "0.05": {
-        "epsilon": 3.4791953936219215,
-        "num_guesses": 100,
-        "correct_guesses": 100,
-        "candidate_num_guesses": [
-          5,
-          10,
-          15,
-          20,
-          25,
-          30,
-          35,
-          40,
-          45,
-          50,
-          55,
-          60,
-          65,
-          70,
-          75,
-          80,
-          85,
-          90,
-          95,
-          100
-        ],
-        "direction": "lower"
-      },
-      "0.01": {
-        "epsilon": 3.023197554051876,
-        "num_guesses": 100,
-        "correct_guesses": 100,
-        "candidate_num_guesses": [
-          5,
-          10,
-          15,
-          20,
-          25,
-          30,
-          35,
-          40,
-          45,
-          50,
-          55,
-          60,
-          65,
-          70,
-          75,
-          80,
-          85,
-          90,
-          95,
-          100
-        ],
-        "direction": "lower"
-      }
-    }
-  }
-}
\ No newline at end of file
diff --git a/llama3-8b/base/epochs/epoch_005/adapter/README.md b/llama3-8b/base/epochs/epoch_005/adapter/README.md
deleted file mode 100644
index d99a36f688caa7f483f250cdbf16c2feb9635fef..0000000000000000000000000000000000000000
--- a/llama3-8b/base/epochs/epoch_005/adapter/README.md
+++ /dev/null
@@ -1,207 +0,0 @@
----
-base_model: meta-llama/Meta-Llama-3-8B
-library_name: peft
-pipeline_tag: text-generation
-tags:
-- base_model:adapter:meta-llama/Meta-Llama-3-8B
-- lora
-- transformers
----
-
-# Model Card for Model ID
-
-<!-- Provide a quick summary of what the model is/does. -->
-
-
-
-## Model Details
-
-### Model Description
-
-<!-- Provide a longer summary of what this model is. -->
-
-
-
-- **Developed by:** [More Information Needed]
-- **Funded by [optional]:** [More Information Needed]
-- **Shared by [optional]:** [More Information Needed]
-- **Model type:** [More Information Needed]
-- **Language(s) (NLP):** [More Information Needed]
-- **License:** [More Information Needed]
-- **Finetuned from model [optional]:** [More Information Needed]
-
-### Model Sources [optional]
-
-<!-- Provide the basic links for the model. -->
-
-- **Repository:** [More Information Needed]
-- **Paper [optional]:** [More Information Needed]
-- **Demo [optional]:** [More Information Needed]
-
-## Uses
-
-<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
-
-### Direct Use
-
-<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
-
-[More Information Needed]
-
-### Downstream Use [optional]
-
-<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
-
-[More Information Needed]
-
-### Out-of-Scope Use
-
-<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
-
-[More Information Needed]
-
-## Bias, Risks, and Limitations
-
-<!-- This section is meant to convey both technical and sociotechnical limitations. -->
-
-[More Information Needed]
-
-### Recommendations
-
-<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
-
-Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
-
-## How to Get Started with the Model
-
-Use the code below to get started with the model.
-
-[More Information Needed]
-
-## Training Details
-
-### Training Data
-
-<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
-
-[More Information Needed]
-
-### Training Procedure
-
-<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
-
-#### Preprocessing [optional]
-
-[More Information Needed]
-
-
-#### Training Hyperparameters
-
-- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
-
-#### Speeds, Sizes, Times [optional]
-
-<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
-
-[More Information Needed]
-
-## Evaluation
-
-<!-- This section describes the evaluation protocols and provides the results. -->
-
-### Testing Data, Factors & Metrics
-
-#### Testing Data
-
-<!-- This should link to a Dataset Card if possible. -->
-
-[More Information Needed]
-
-#### Factors
-
-<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
-
-[More Information Needed]
-
-#### Metrics
-
-<!-- These are the evaluation metrics being used, ideally with a description of why. -->
-
-[More Information Needed]
-
-### Results
-
-[More Information Needed]
-
-#### Summary
-
-
-
-## Model Examination [optional]
-
-<!-- Relevant interpretability work for the model goes here -->
-
-[More Information Needed]
-
-## Environmental Impact
-
-<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
-
-Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
-
-- **Hardware Type:** [More Information Needed]
-- **Hours used:** [More Information Needed]
-- **Cloud Provider:** [More Information Needed]
-- **Compute Region:** [More Information Needed]
-- **Carbon Emitted:** [More Information Needed]
-
-## Technical Specifications [optional]
-
-### Model Architecture and Objective
-
-[More Information Needed]
-
-### Compute Infrastructure
-
-[More Information Needed]
-
-#### Hardware
-
-[More Information Needed]
-
-#### Software
-
-[More Information Needed]
-
-## Citation [optional]
-
-<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
-
-**BibTeX:**
-
-[More Information Needed]
-
-**APA:**
-
-[More Information Needed]
-
-## Glossary [optional]
-
-<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
-
-[More Information Needed]
-
-## More Information [optional]
-
-[More Information Needed]
-
-## Model Card Authors [optional]
-
-[More Information Needed]
-
-## Model Card Contact
-
-[More Information Needed]
-### Framework versions
-
-- PEFT 0.18.1
\ No newline at end of file
diff --git a/llama3-8b/base/epochs/epoch_005/adapter/adapter_config.json b/llama3-8b/base/epochs/epoch_005/adapter/adapter_config.json
deleted file mode 100644
index 142654567bf521f9f62f008f7d9e1b6f2e164b7a..0000000000000000000000000000000000000000
--- a/llama3-8b/base/epochs/epoch_005/adapter/adapter_config.json
+++ /dev/null
@@ -1,46 +0,0 @@
-{
-  "alora_invocation_tokens": null,
-  "alpha_pattern": {},
-  "arrow_config": null,
-  "auto_mapping": null,
-  "base_model_name_or_path": "meta-llama/Meta-Llama-3-8B",
-  "bias": "none",
-  "corda_config": null,
-  "ensure_weight_tying": true,
-  "eva_config": null,
-  "exclude_modules": null,
-  "fan_in_fan_out": false,
-  "inference_mode": true,
-  "init_lora_weights": true,
-  "layer_replication": null,
-  "layers_pattern": null,
-  "layers_to_transform": null,
-  "loftq_config": {},
-  "lora_alpha": 32,
-  "lora_bias": false,
-  "lora_dropout": 0.05,
-  "megatron_config": null,
-  "megatron_core": "megatron.core",
-  "modules_to_save": [
-    "lm_head",
-    "embed_tokens"
-  ],
-  "peft_type": "LORA",
-  "peft_version": "0.18.1",
-  "qalora_group_size": 16,
-  "r": 16,
-  "rank_pattern": {},
-  "revision": null,
-  "target_modules": [
-    "q_proj",
-    "v_proj",
-    "o_proj",
-    "k_proj"
-  ],
-  "target_parameters": null,
-  "task_type": "CAUSAL_LM",
-  "trainable_token_indices": null,
-  "use_dora": false,
-  "use_qalora": false,
-  "use_rslora": false
-}
\ No newline at end of file
diff --git a/llama3-8b/base/epochs/epoch_005/audit_results.json b/llama3-8b/base/epochs/epoch_005/audit_results.json
deleted file mode 100644
index ab8236002d7e93c190a3b25c7568ffacc3a9e1c6..0000000000000000000000000000000000000000
--- a/llama3-8b/base/epochs/epoch_005/audit_results.json
+++ /dev/null
@@ -1,137 +0,0 @@
-{
-  "delta": 1e-05,
-  "num_canaries": 500,
-  "num_members": 250,
-  "paper_guess_fraction": 0.2,
-  "paper_guess_steps": 20,
-  "loss": {
-    "auc": 1.0,
-    "empirical_epsilon": {
-      "0.05": 3.4791953936219215,
-      "0.01": 3.023197554051876
-    },
-    "empirical_epsilon_details": {
-      "0.05": {
-        "epsilon": 3.4791953936219215,
-        "num_guesses": 100,
-        "correct_guesses": 100,
-        "candidate_num_guesses": [
-          5,
-          10,
-          15,
-          20,
-          25,
-          30,
-          35,
-          40,
-          45,
-          50,
-          55,
-          60,
-          65,
-          70,
-          75,
-          80,
-          85,
-          90,
-          95,
-          100
-        ],
-        "direction": "lower"
-      },
-      "0.01": {
-        "epsilon": 3.023197554051876,
-        "num_guesses": 100,
-        "correct_guesses": 100,
-        "candidate_num_guesses": [
-          5,
-          10,
-          15,
-          20,
-          25,
-          30,
-          35,
-          40,
-          45,
-          50,
-          55,
-          60,
-          65,
-          70,
-          75,
-          80,
-          85,
-          90,
-          95,
-          100
-        ],
-        "direction": "lower"
-      }
-    }
-  },
-  "embedding": {
-    "auc": 0.558,
-    "empirical_epsilon": {
-      "0.05": 3.4791953936219215,
-      "0.01": 3.023197554051876
-    },
-    "empirical_epsilon_details": {
-      "0.05": {
-        "epsilon": 3.4791953936219215,
-        "num_guesses": 100,
-        "correct_guesses": 100,
-        "candidate_num_guesses": [
-          5,
-          10,
-          15,
-          20,
-          25,
-          30,
-          35,
-          40,
-          45,
-          50,
-          55,
-          60,
-          65,
-          70,
-          75,
-          80,
-          85,
-          90,
-          95,
-          100
-        ],
-        "direction": "lower"
-      },
-      "0.01": {
-        "epsilon": 3.023197554051876,
-        "num_guesses": 100,
-        "correct_guesses": 100,
-        "candidate_num_guesses": [
-          5,
-          10,
-          15,
-          20,
-          25,
-          30,
-          35,
-          40,
-          45,
-          50,
-          55,
-          60,
-          65,
-          70,
-          75,
-          80,
-          85,
-          90,
-          95,
-          100
-        ],
-        "direction": "lower"
-      }
-    }
-  }
-}
\ No newline at end of file
diff --git a/llama3-8b/dp3/epochs/epoch_001/adapter/README.md b/llama3-8b/dp3/epochs/epoch_001/adapter/README.md
deleted file mode 100644
index d99a36f688caa7f483f250cdbf16c2feb9635fef..0000000000000000000000000000000000000000
--- a/llama3-8b/dp3/epochs/epoch_001/adapter/README.md
+++ /dev/null
@@ -1,207 +0,0 @@
----
-base_model: meta-llama/Meta-Llama-3-8B
-library_name: peft
-pipeline_tag: text-generation
-tags:
-- base_model:adapter:meta-llama/Meta-Llama-3-8B
-- lora
-- transformers
----
-
-# Model Card for Model ID
-
-<!-- Provide a quick summary of what the model is/does. -->
-
-
-
-## Model Details
-
-### Model Description
-
-<!-- Provide a longer summary of what this model is. -->
-
-
-
-- **Developed by:** [More Information Needed]
-- **Funded by [optional]:** [More Information Needed]
-- **Shared by [optional]:** [More Information Needed]
-- **Model type:** [More Information Needed]
-- **Language(s) (NLP):** [More Information Needed]
-- **License:** [More Information Needed]
-- **Finetuned from model [optional]:** [More Information Needed]
-
-### Model Sources [optional]
-
-<!-- Provide the basic links for the model. -->
-
-- **Repository:** [More Information Needed]
-- **Paper [optional]:** [More Information Needed]
-- **Demo [optional]:** [More Information Needed]
-
-## Uses
-
-<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
-
-### Direct Use
-
-<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
-
-[More Information Needed]
-
-### Downstream Use [optional]
-
-<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
-
-[More Information Needed]
-
-### Out-of-Scope Use
-
-<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
-
-[More Information Needed]
-
-## Bias, Risks, and Limitations
-
-<!-- This section is meant to convey both technical and sociotechnical limitations. -->
-
-[More Information Needed]
-
-### Recommendations
-
-<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
-
-Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
-
-## How to Get Started with the Model
-
-Use the code below to get started with the model.
-
-[More Information Needed]
-
-## Training Details
-
-### Training Data
-
-<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
-
-[More Information Needed]
-
-### Training Procedure
-
-<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
-
-#### Preprocessing [optional]
-
-[More Information Needed]
-
-
-#### Training Hyperparameters
-
-- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
-
-#### Speeds, Sizes, Times [optional]
-
-<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
-
-[More Information Needed]
-
-## Evaluation
-
-<!-- This section describes the evaluation protocols and provides the results. -->
-
-### Testing Data, Factors & Metrics
-
-#### Testing Data
-
-<!-- This should link to a Dataset Card if possible. -->
-
-[More Information Needed]
-
-#### Factors
-
-<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
-
-[More Information Needed]
-
-#### Metrics
-
-<!-- These are the evaluation metrics being used, ideally with a description of why. -->
-
-[More Information Needed]
-
-### Results
-
-[More Information Needed]
-
-#### Summary
-
-
-
-## Model Examination [optional]
-
-<!-- Relevant interpretability work for the model goes here -->
-
-[More Information Needed]
-
-## Environmental Impact
-
-<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
-
-Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
-
-- **Hardware Type:** [More Information Needed]
-- **Hours used:** [More Information Needed]
-- **Cloud Provider:** [More Information Needed]
-- **Compute Region:** [More Information Needed]
-- **Carbon Emitted:** [More Information Needed]
-
-## Technical Specifications [optional]
-
-### Model Architecture and Objective
-
-[More Information Needed]
-
-### Compute Infrastructure
-
-[More Information Needed]
-
-#### Hardware
-
-[More Information Needed]
-
-#### Software
-
-[More Information Needed]
-
-## Citation [optional]
-
-<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
-
-**BibTeX:**
-
-[More Information Needed]
-
-**APA:**
-
-[More Information Needed]
-
-## Glossary [optional]
-
-<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
-
-[More Information Needed]
-
-## More Information [optional]
-
-[More Information Needed]
-
-## Model Card Authors [optional]
-
-[More Information Needed]
-
-## Model Card Contact
-
-[More Information Needed]
-### Framework versions
-
-- PEFT 0.18.1
\ No newline at end of file
diff --git a/llama3-8b/dp3/epochs/epoch_001/adapter/adapter_config.json b/llama3-8b/dp3/epochs/epoch_001/adapter/adapter_config.json
deleted file mode 100644
index 87681ceb84a7a9bd25a46addd37669c4837d5d84..0000000000000000000000000000000000000000
--- a/llama3-8b/dp3/epochs/epoch_001/adapter/adapter_config.json
+++ /dev/null
@@ -1,46 +0,0 @@
-{
-  "alora_invocation_tokens": null,
-  "alpha_pattern": {},
-  "arrow_config": null,
-  "auto_mapping": null,
-  "base_model_name_or_path": "meta-llama/Meta-Llama-3-8B",
-  "bias": "none",
-  "corda_config": null,
-  "ensure_weight_tying": true,
-  "eva_config": null,
-  "exclude_modules": null,
-  "fan_in_fan_out": false,
-  "inference_mode": true,
-  "init_lora_weights": true,
-  "layer_replication": null,
-  "layers_pattern": null,
-  "layers_to_transform": null,
-  "loftq_config": {},
-  "lora_alpha": 32,
-  "lora_bias": false,
-  "lora_dropout": 0.05,
-  "megatron_config": null,
-  "megatron_core": "megatron.core",
-  "modules_to_save": [
-    "lm_head",
-    "embed_tokens"
-  ],
-  "peft_type": "LORA",
-  "peft_version": "0.18.1",
-  "qalora_group_size": 16,
-  "r": 16,
-  "rank_pattern": {},
-  "revision": null,
-  "target_modules": [
-    "o_proj",
-    "k_proj",
-    "v_proj",
-    "q_proj"
-  ],
-  "target_parameters": null,
-  "task_type": "CAUSAL_LM",
-  "trainable_token_indices": null,
-  "use_dora": false,
-  "use_qalora": false,
-  "use_rslora": false
-}
\ No newline at end of file
diff --git a/llama3-8b/dp3/epochs/epoch_001/audit_results.json b/llama3-8b/dp3/epochs/epoch_001/audit_results.json
deleted file mode 100644
index 2f04420f14d8081c96209b4c6351c687bdc9fb7f..0000000000000000000000000000000000000000
--- a/llama3-8b/dp3/epochs/epoch_001/audit_results.json
+++ /dev/null
@@ -1,137 +0,0 @@
-{
-  "delta": 1e-05,
-  "num_canaries": 500,
-  "num_members": 250,
-  "paper_guess_fraction": 0.2,
-  "paper_guess_steps": 20,
-  "loss": {
-    "auc": 0.506352,
-    "empirical_epsilon": {
-      "0.05": 0.0,
-      "0.01": 0.0
-    },
-    "empirical_epsilon_details": {
-      "0.05": {
-        "epsilon": 0.0,
-        "num_guesses": 0,
-        "correct_guesses": 0,
-        "candidate_num_guesses": [
-          5,
-          10,
-          15,
-          20,
-          25,
-          30,
-          35,
-          40,
-          45,
-          50,
-          55,
-          60,
-          65,
-          70,
-          75,
-          80,
-          85,
-          90,
-          95,
-          100
-        ],
-        "direction": "lower"
-      },
-      "0.01": {
-        "epsilon": 0.0,
-        "num_guesses": 0,
-        "correct_guesses": 0,
-        "candidate_num_guesses": [
-          5,
-          10,
-          15,
-          20,
-          25,
-          30,
-          35,
-          40,
-          45,
-          50,
-          55,
-          60,
-          65,
-          70,
-          75,
-          80,
-          85,
-          90,
-          95,
-          100
-        ],
-        "direction": "lower"
-      }
-    }
-  },
-  "embedding": {
-    "auc": 0.503592,
-    "empirical_epsilon": {
-      "0.05": 0.0,
-      "0.01": 0.0
-    },
-    "empirical_epsilon_details": {
-      "0.05": {
-        "epsilon": 0.0,
-        "num_guesses": 0,
-        "correct_guesses": 0,
-        "candidate_num_guesses": [
-          5,
-          10,
-          15,
-          20,
-          25,
-          30,
-          35,
-          40,
-          45,
-          50,
-          55,
-          60,
-          65,
-          70,
-          75,
-          80,
-          85,
-          90,
-          95,
-          100
-        ],
-        "direction": "lower"
-      },
-      "0.01": {
-        "epsilon": 0.0,
-        "num_guesses": 0,
-        "correct_guesses": 0,
-        "candidate_num_guesses": [
-          5,
-          10,
-          15,
-          20,
-          25,
-          30,
-          35,
-          40,
-          45,
-          50,
-          55,
-          60,
-          65,
-          70,
-          75,
-          80,
-          85,
-          90,
-          95,
-          100
-        ],
-        "direction": "lower"
-      }
-    }
-  }
-}
\ No newline at end of file
diff --git a/llama3-8b/dp3/epochs/epoch_002/adapter/README.md b/llama3-8b/dp3/epochs/epoch_002/adapter/README.md
deleted file mode 100644
index d99a36f688caa7f483f250cdbf16c2feb9635fef..0000000000000000000000000000000000000000
--- a/llama3-8b/dp3/epochs/epoch_002/adapter/README.md
+++ /dev/null
@@ -1,207 +0,0 @@
----
-base_model: meta-llama/Meta-Llama-3-8B
-library_name: peft
-pipeline_tag: text-generation
-tags:
-- base_model:adapter:meta-llama/Meta-Llama-3-8B
-- lora
-- transformers
----
-
-# Model Card for Model ID
-
-<!-- Provide a quick summary of what the model is/does. -->
-
-
-
-## Model Details
-
-### Model Description
-
-<!-- Provide a longer summary of what this model is. -->
-
-
-
-- **Developed by:** [More Information Needed]
-- **Funded by [optional]:** [More Information Needed]
-- **Shared by [optional]:** [More Information Needed]
-- **Model type:** [More Information Needed]
-- **Language(s) (NLP):** [More Information Needed]
-- **License:** [More Information Needed]
-- **Finetuned from model [optional]:** [More Information Needed]
-
-### Model Sources [optional]
-
-<!-- Provide the basic links for the model. -->
-
-- **Repository:** [More Information Needed]
-- **Paper [optional]:** [More Information Needed]
-- **Demo [optional]:** [More Information Needed]
-
-## Uses
-
-<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
-
-### Direct Use
-
-<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
-
-[More Information Needed]
-
-### Downstream Use [optional]
-
-<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
-
-[More Information Needed]
-
-### Out-of-Scope Use
-
-<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
-
-[More Information Needed]
-
-## Bias, Risks, and Limitations
-
-<!-- This section is meant to convey both technical and sociotechnical limitations. -->
-
-[More Information Needed]
-
-### Recommendations
-
-<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
-
-Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
-
-## How to Get Started with the Model
-
-Use the code below to get started with the model.
-
-[More Information Needed]
-
-## Training Details
-
-### Training Data
-
-<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
-
-[More Information Needed]
-
-### Training Procedure
-
-<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
-
-#### Preprocessing [optional]
-
-[More Information Needed]
-
-
-#### Training Hyperparameters
-
-- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
-
-#### Speeds, Sizes, Times [optional]
-
-<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
-
-[More Information Needed]
-
-## Evaluation
-
-<!-- This section describes the evaluation protocols and provides the results. -->
-
-### Testing Data, Factors & Metrics
-
-#### Testing Data
-
-<!-- This should link to a Dataset Card if possible. -->
-
-[More Information Needed]
-
-#### Factors
-
-<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
-
-[More Information Needed]
-
-#### Metrics
-
-<!-- These are the evaluation metrics being used, ideally with a description of why. -->
-
-[More Information Needed]
-
-### Results
-
-[More Information Needed]
-
-#### Summary
-
-
-
-## Model Examination [optional]
-
-<!-- Relevant interpretability work for the model goes here -->
-
-[More Information Needed]
-
-## Environmental Impact
-
-<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
-
-Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
-
-- **Hardware Type:** [More Information Needed]
-- **Hours used:** [More Information Needed]
-- **Cloud Provider:** [More Information Needed]
-- **Compute Region:** [More Information Needed]
-- **Carbon Emitted:** [More Information Needed]
-
-## Technical Specifications [optional]
-
-### Model Architecture and Objective
-
-[More Information Needed]
-
-### Compute Infrastructure
-
-[More Information Needed]
-
-#### Hardware
-
-[More Information Needed]
-
-#### Software
-
-[More Information Needed]
-
-## Citation [optional]
-
-<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
-
-**BibTeX:**
-
-[More Information Needed]
-
-**APA:**
-
-[More Information Needed]
-
-## Glossary [optional]
-
-<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
-
-[More Information Needed]
-
-## More Information [optional]
-
-[More Information Needed]
-
-## Model Card Authors [optional]
-
-[More Information Needed]
-
-## Model Card Contact
-
-[More Information Needed]
-### Framework versions
-
-- PEFT 0.18.1
\ No newline at end of file
diff --git a/llama3-8b/dp3/epochs/epoch_002/adapter/adapter_config.json b/llama3-8b/dp3/epochs/epoch_002/adapter/adapter_config.json
deleted file mode 100644
index 87681ceb84a7a9bd25a46addd37669c4837d5d84..0000000000000000000000000000000000000000
--- a/llama3-8b/dp3/epochs/epoch_002/adapter/adapter_config.json
+++ /dev/null
@@ -1,46 +0,0 @@
-{
-  "alora_invocation_tokens": null,
-  "alpha_pattern": {},
-  "arrow_config": null,
-  "auto_mapping": null,
-  "base_model_name_or_path": "meta-llama/Meta-Llama-3-8B",
-  "bias": "none",
-  "corda_config": null,
-  "ensure_weight_tying": true,
-  "eva_config": null,
-  "exclude_modules": null,
-  "fan_in_fan_out": false,
-  "inference_mode": true,
-  "init_lora_weights": true,
-  "layer_replication": null,
-  "layers_pattern": null,
-  "layers_to_transform": null,
-  "loftq_config": {},
-  "lora_alpha": 32,
-  "lora_bias": false,
-  "lora_dropout": 0.05,
-  "megatron_config": null,
-  "megatron_core": "megatron.core",
-  "modules_to_save": [
-    "lm_head",
-    "embed_tokens"
-  ],
-  "peft_type": "LORA",
-  "peft_version": "0.18.1",
-  "qalora_group_size": 16,
-  "r": 16,
-  "rank_pattern": {},
-  "revision": null,
-  "target_modules": [
-    "o_proj",
-    "k_proj",
-    "v_proj",
-    "q_proj"
-  ],
-  "target_parameters": null,
-  "task_type": "CAUSAL_LM",
-  "trainable_token_indices": null,
-  "use_dora": false,
-  "use_qalora": false,
-  "use_rslora": false
-}
\ No newline at end of file
diff --git a/llama3-8b/dp3/epochs/epoch_002/audit_results.json b/llama3-8b/dp3/epochs/epoch_002/audit_results.json
deleted file mode 100644
index ffc9cf3631e7025b1722633b9400c24d96c4d21f..0000000000000000000000000000000000000000
--- a/llama3-8b/dp3/epochs/epoch_002/audit_results.json
+++ /dev/null
@@ -1,137 +0,0 @@
-{
-  "delta": 1e-05,
-  "num_canaries": 500,
-  "num_members": 250,
-  "paper_guess_fraction": 0.2,
-  "paper_guess_steps": 20,
-  "loss": {
-    "auc": 0.519304,
-    "empirical_epsilon": {
-      "0.05": 0.09290625154972076,
-      "0.01": 0.0
-    },
-    "empirical_epsilon_details": {
-      "0.05": {
-        "epsilon": 0.09290625154972076,
-        "num_guesses": 70,
-        "correct_guesses": 44,
-        "candidate_num_guesses": [
-          5,
-          10,
-          15,
-          20,
-          25,
-          30,
-          35,
-          40,
-          45,
-          50,
-          55,
-          60,
-          65,
-          70,
-          75,
-          80,
-          85,
-          90,
-          95,
-          100
-        ],
-        "direction": "higher"
-      },
-      "0.01": {
-        "epsilon": 0.0,
-        "num_guesses": 0,
-        "correct_guesses": 0,
-        "candidate_num_guesses": [
-          5,
-          10,
-          15,
-          20,
-          25,
-          30,
-          35,
-          40,
-          45,
-          50,
-          55,
-          60,
-          65,
-          70,
-          75,
-          80,
-          85,
-          90,
-          95,
-          100
-        ],
-        "direction": "lower"
-      }
-    }
-  },
-  "embedding": {
-    "auc": 0.518168,
-    "empirical_epsilon": {
-      "0.05": 0.0,
-      "0.01": 0.0
-    },
-    "empirical_epsilon_details": {
-      "0.05": {
-        "epsilon": 0.0,
-        "num_guesses": 0,
-        "correct_guesses": 0,
-        "candidate_num_guesses": [
-          5,
-          10,
-          15,
-          20,
-          25,
-          30,
-          35,
-          40,
-          45,
-          50,
-          55,
-          60,
-          65,
-          70,
-          75,
-          80,
-          85,
-          90,
-          95,
-          100
-        ],
-        "direction": "lower"
-      },
-      "0.01": {
-        "epsilon": 0.0,
-        "num_guesses": 0,
-        "correct_guesses": 0,
-        "candidate_num_guesses": [
-          5,
-          10,
-          15,
-          20,
-          25,
-          30,
-          35,
-          40,
-          45,
-          50,
-          55,
-          60,
-          65,
-          70,
-          75,
-          80,
-          85,
-          90,
-          95,
-          100
-        ],
-        "direction": "lower"
-      }
-    }
-  }
-}
\ No newline at end of file
diff --git a/llama3-8b/dp3/epochs/epoch_003/adapter/README.md b/llama3-8b/dp3/epochs/epoch_003/adapter/README.md
deleted file mode 100644
index d99a36f688caa7f483f250cdbf16c2feb9635fef..0000000000000000000000000000000000000000
--- a/llama3-8b/dp3/epochs/epoch_003/adapter/README.md
+++ /dev/null
@@ -1,207 +0,0 @@
----
-base_model: meta-llama/Meta-Llama-3-8B
-library_name: peft
-pipeline_tag: text-generation
-tags:
-- base_model:adapter:meta-llama/Meta-Llama-3-8B
-- lora
-- transformers
----
-
-# Model Card for Model ID
-
-<!-- Provide a quick summary of what the model is/does. -->
-
-
-
-## Model Details
-
-### Model Description
-
-<!-- Provide a longer summary of what this model is. -->
-
-
-
-- **Developed by:** [More Information Needed]
-- **Funded by [optional]:** [More Information Needed]
-- **Shared by [optional]:** [More Information Needed]
-- **Model type:** [More Information Needed]
-- **Language(s) (NLP):** [More Information Needed]
-- **License:** [More Information Needed]
-- **Finetuned from model [optional]:** [More Information Needed]
-
-### Model Sources [optional]
-
-<!-- Provide the basic links for the model. -->
-
-- **Repository:** [More Information Needed]
-- **Paper [optional]:** [More Information Needed]
-- **Demo [optional]:** [More Information Needed]
-
-## Uses
-
-<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
-
-### Direct Use
-
-<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
-
-[More Information Needed]
-
-### Downstream Use [optional]
-
-<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
-
-[More Information Needed]
-
-### Out-of-Scope Use
-
-<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
-
-[More Information Needed]
-
-## Bias, Risks, and Limitations
-
-<!-- This section is meant to convey both technical and sociotechnical limitations. -->
-
-[More Information Needed]
-
-### Recommendations
-
-<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
-
-Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
-
-## How to Get Started with the Model
-
-Use the code below to get started with the model.
-
-[More Information Needed]
-
-## Training Details
-
-### Training Data
-
-<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
-
-[More Information Needed]
-
-### Training Procedure
-
-<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
-
-#### Preprocessing [optional]
-
-[More Information Needed]
-
-
-#### Training Hyperparameters
-
-- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
-
-#### Speeds, Sizes, Times [optional]
-
-<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
-
-[More Information Needed]
-
-## Evaluation
-
-<!-- This section describes the evaluation protocols and provides the results. -->
-
-### Testing Data, Factors & Metrics
-
-#### Testing Data
-
-<!-- This should link to a Dataset Card if possible. -->
-
-[More Information Needed]
-
-#### Factors
-
-<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
-
-[More Information Needed]
-
-#### Metrics
-
-<!-- These are the evaluation metrics being used, ideally with a description of why. -->
-
-[More Information Needed]
-
-### Results
-
-[More Information Needed]
-
-#### Summary
-
-
-
-## Model Examination [optional]
-
-<!-- Relevant interpretability work for the model goes here -->
-
-[More Information Needed]
-
-## Environmental Impact
-
-<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
-
-Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
-
-- **Hardware Type:** [More Information Needed]
-- **Hours used:** [More Information Needed]
-- **Cloud Provider:** [More Information Needed]
-- **Compute Region:** [More Information Needed]
-- **Carbon Emitted:** [More Information Needed]
-
-## Technical Specifications [optional]
-
-### Model Architecture and Objective
-
-[More Information Needed]
-
-### Compute Infrastructure
-
-[More Information Needed]
-
-#### Hardware
-
-[More Information Needed]
-
-#### Software
-
-[More Information Needed]
-
-## Citation [optional]
-
-<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
-
-**BibTeX:**
-
-[More Information Needed]
-
-**APA:**
-
-[More Information Needed]
-
-## Glossary [optional]
-
-<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
-
-[More Information Needed]
-
-## More Information [optional]
-
-[More Information Needed]
-
-## Model Card Authors [optional]
-
-[More Information Needed]
-
-## Model Card Contact
-
-[More Information Needed]
-### Framework versions
-
-- PEFT 0.18.1
\ No newline at end of file
diff --git a/llama3-8b/dp3/epochs/epoch_003/adapter/adapter_config.json b/llama3-8b/dp3/epochs/epoch_003/adapter/adapter_config.json
deleted file mode 100644
index 87681ceb84a7a9bd25a46addd37669c4837d5d84..0000000000000000000000000000000000000000
--- a/llama3-8b/dp3/epochs/epoch_003/adapter/adapter_config.json
+++ /dev/null
@@ -1,46 +0,0 @@
-{
-  "alora_invocation_tokens": null,
-  "alpha_pattern": {},
-  "arrow_config": null,
-  "auto_mapping": null,
-  "base_model_name_or_path": "meta-llama/Meta-Llama-3-8B",
-  "bias": "none",
-  "corda_config": null,
-  "ensure_weight_tying": true,
-  "eva_config": null,
-  "exclude_modules": null,
-  "fan_in_fan_out": false,
-  "inference_mode": true,
-  "init_lora_weights": true,
-  "layer_replication": null,
-  "layers_pattern": null,
-  "layers_to_transform": null,
-  "loftq_config": {},
-  "lora_alpha": 32,
-  "lora_bias": false,
-  "lora_dropout": 0.05,
-  "megatron_config": null,
-  "megatron_core": "megatron.core",
-  "modules_to_save": [
-    "lm_head",
-    "embed_tokens"
-  ],
-  "peft_type": "LORA",
-  "peft_version": "0.18.1",
-  "qalora_group_size": 16,
-  "r": 16,
-  "rank_pattern": {},
-  "revision": null,
-  "target_modules": [
-    "o_proj",
-    "k_proj",
-    "v_proj",
-    "q_proj"
-  ],
-  "target_parameters": null,
-  "task_type": "CAUSAL_LM",
-  "trainable_token_indices": null,
-  "use_dora": false,
-  "use_qalora": false,
-  "use_rslora": false
-}
\ No newline at end of file
diff --git a/llama3-8b/dp3/epochs/epoch_003/audit_results.json b/llama3-8b/dp3/epochs/epoch_003/audit_results.json
deleted file mode 100644
index 2ecb733b2ed0cb3855c6137ee97f8f1faa36c4ab..0000000000000000000000000000000000000000
--- a/llama3-8b/dp3/epochs/epoch_003/audit_results.json
+++ /dev/null
@@ -1,137 +0,0 @@
-{
-  "delta": 1e-05,
-  "num_canaries": 500,
-  "num_members": 250,
-  "paper_guess_fraction": 0.2,
-  "paper_guess_steps": 20,
-  "loss": {
-    "auc": 0.507536,
-    "empirical_epsilon": {
-      "0.05": 0.14375991560518742,
-      "0.01": 0.0
-    },
-    "empirical_epsilon_details": {
-      "0.05": {
-        "epsilon": 0.14375991560518742,
-        "num_guesses": 60,
-        "correct_guesses": 39,
-        "candidate_num_guesses": [
-          5,
-          10,
-          15,
-          20,
-          25,
-          30,
-          35,
-          40,
-          45,
-          50,
-          55,
-          60,
-          65,
-          70,
-          75,
-          80,
-          85,
-          90,
-          95,
-          100
-        ],
-        "direction": "higher"
-      },
-      "0.01": {
-        "epsilon": 0.0,
-        "num_guesses": 0,
-        "correct_guesses": 0,
-        "candidate_num_guesses": [
-          5,
-          10,
-          15,
-          20,
-          25,
-          30,
-          35,
-          40,
-          45,
-          50,
-          55,
-          60,
-          65,
-          70,
-          75,
-          80,
-          85,
-          90,
-          95,
-          100
-        ],
-        "direction": "lower"
-      }
-    }
-  },
-  "embedding": {
-    "auc": 0.500928,
-    "empirical_epsilon": {
-      "0.05": 0.0,
-      "0.01": 0.0
-    },
-    "empirical_epsilon_details": {
-      "0.05": {
-        "epsilon": 0.0,
-        "num_guesses": 0,
-        "correct_guesses": 0,
-        "candidate_num_guesses": [
-          5,
-          10,
-          15,
-          20,
-          25,
-          30,
-          35,
-          40,
-          45,
-          50,
-          55,
-          60,
-          65,
-          70,
-          75,
-          80,
-          85,
-          90,
-          95,
-          100
-        ],
-        "direction": "lower"
-      },
-      "0.01": {
-        "epsilon": 0.0,
-        "num_guesses": 0,
-        "correct_guesses": 0,
-        "candidate_num_guesses": [
-          5,
-          10,
-          15,
-          20,
-          25,
-          30,
-          35,
-          40,
-          45,
-          50,
-          55,
-          60,
-          65,
-          70,
-          75,
-          80,
-          85,
-          90,
-          95,
-          100
-        ],
-        "direction": "lower"
-      }
-    }
-  }
-}
\ No newline at end of file
diff --git a/llama3-8b/dp3/epochs/epoch_004/adapter/README.md b/llama3-8b/dp3/epochs/epoch_004/adapter/README.md
deleted file mode 100644
index d99a36f688caa7f483f250cdbf16c2feb9635fef..0000000000000000000000000000000000000000
--- a/llama3-8b/dp3/epochs/epoch_004/adapter/README.md
+++ /dev/null
@@ -1,207 +0,0 @@
----
-base_model: meta-llama/Meta-Llama-3-8B
-library_name: peft
-pipeline_tag: text-generation
-tags:
-- base_model:adapter:meta-llama/Meta-Llama-3-8B
-- lora
-- transformers
----
-
-# Model Card for Model ID
-
-<!-- Provide a quick summary of what the model is/does. -->
-
-
-
-## Model Details
-
-### Model Description
-
-<!-- Provide a longer summary of what this model is. -->
-
-
-
-- **Developed by:** [More Information Needed]
-- **Funded by [optional]:** [More Information Needed]
-- **Shared by [optional]:** [More Information Needed]
-- **Model type:** [More Information Needed]
-- **Language(s) (NLP):** [More Information Needed]
-- **License:** [More Information Needed]
-- **Finetuned from model [optional]:** [More Information Needed]
-
-### Model Sources [optional]
-
-<!-- Provide the basic links for the model. -->
-
-- **Repository:** [More Information Needed]
-- **Paper [optional]:** [More Information Needed]
-- **Demo [optional]:** [More Information Needed]
-
-## Uses
-
-<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
-
-### Direct Use
-
-<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
-
-[More Information Needed]
-
-### Downstream Use [optional]
-
-<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
-
-[More Information Needed]
-
-### Out-of-Scope Use
-
-<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
-
-[More Information Needed]
-
-## Bias, Risks, and Limitations
-
-<!-- This section is meant to convey both technical and sociotechnical limitations. -->
-
-[More Information Needed]
-
-### Recommendations
-
-<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
-
-Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
-
-## How to Get Started with the Model
-
-Use the code below to get started with the model.
-
-[More Information Needed]
-
-## Training Details
-
-### Training Data
-
-<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
-
-[More Information Needed]
-
-### Training Procedure
-
-<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
-
-#### Preprocessing [optional]
-
-[More Information Needed]
-
-
-#### Training Hyperparameters
-
-- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
-
-#### Speeds, Sizes, Times [optional]
-
-<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
-
-[More Information Needed]
-
-## Evaluation
-
-<!-- This section describes the evaluation protocols and provides the results. -->
-
-### Testing Data, Factors & Metrics
-
-#### Testing Data
-
-<!-- This should link to a Dataset Card if possible. -->
-
-[More Information Needed]
-
-#### Factors
-
-<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
-
-[More Information Needed]
-
-#### Metrics
-
-<!-- These are the evaluation metrics being used, ideally with a description of why. -->
-
-[More Information Needed]
-
-### Results
-
-[More Information Needed]
-
-#### Summary
-
-
-
-## Model Examination [optional]
-
-<!-- Relevant interpretability work for the model goes here -->
-
-[More Information Needed]
-
-## Environmental Impact
-
-<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
-
-Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
-
-- **Hardware Type:** [More Information Needed]
-- **Hours used:** [More Information Needed]
-- **Cloud Provider:** [More Information Needed]
-- **Compute Region:** [More Information Needed]
-- **Carbon Emitted:** [More Information Needed]
-
-## Technical Specifications [optional]
-
-### Model Architecture and Objective
-
-[More Information Needed]
-
-### Compute Infrastructure
-
-[More Information Needed]
-
-#### Hardware
-
-[More Information Needed]
-
-#### Software
-
-[More Information Needed]
-
-## Citation [optional]
-
-<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
-
-**BibTeX:**
-
-[More Information Needed]
-
-**APA:**
-
-[More Information Needed]
-
-## Glossary [optional]
-
-<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
-
-[More Information Needed]
-
-## More Information [optional]
-
-[More Information Needed]
-
-## Model Card Authors [optional]
-
-[More Information Needed]
-
-## Model Card Contact
-
-[More Information Needed]
-### Framework versions
-
-- PEFT 0.18.1
\ No newline at end of file
diff --git a/llama3-8b/dp3/epochs/epoch_004/adapter/adapter_config.json b/llama3-8b/dp3/epochs/epoch_004/adapter/adapter_config.json
deleted file mode 100644
index 87681ceb84a7a9bd25a46addd37669c4837d5d84..0000000000000000000000000000000000000000
--- a/llama3-8b/dp3/epochs/epoch_004/adapter/adapter_config.json
+++ /dev/null
@@ -1,46 +0,0 @@
-{
-  "alora_invocation_tokens": null,
-  "alpha_pattern": {},
-  "arrow_config": null,
-  "auto_mapping": null,
-  "base_model_name_or_path": "meta-llama/Meta-Llama-3-8B",
-  "bias": "none",
-  "corda_config": null,
-  "ensure_weight_tying": true,
-  "eva_config": null,
-  "exclude_modules": null,
-  "fan_in_fan_out": false,
-  "inference_mode": true,
-  "init_lora_weights": true,
-  "layer_replication": null,
-  "layers_pattern": null,
-  "layers_to_transform": null,
-  "loftq_config": {},
-  "lora_alpha": 32,
-  "lora_bias": false,
-  "lora_dropout": 0.05,
-  "megatron_config": null,
-  "megatron_core": "megatron.core",
-  "modules_to_save": [
-    "lm_head",
-    "embed_tokens"
-  ],
-  "peft_type": "LORA",
-  "peft_version": "0.18.1",
-  "qalora_group_size": 16,
-  "r": 16,
-  "rank_pattern": {},
-  "revision": null,
-  "target_modules": [
-    "o_proj",
-    "k_proj",
-    "v_proj",
-    "q_proj"
-  ],
-  "target_parameters": null,
-  "task_type": "CAUSAL_LM",
-  "trainable_token_indices": null,
-  "use_dora": false,
-  "use_qalora": false,
-  "use_rslora": false
-}
\ No newline at end of file
diff --git a/llama3-8b/dp3/epochs/epoch_004/audit_results.json b/llama3-8b/dp3/epochs/epoch_004/audit_results.json
deleted file mode 100644
index 05a668b27439be17d2c30c232fdea1ee2af8c336..0000000000000000000000000000000000000000
--- a/llama3-8b/dp3/epochs/epoch_004/audit_results.json
+++ /dev/null
@@ -1,137 +0,0 @@
-{
-  "delta": 1e-05,
-  "num_canaries": 500,
-  "num_members": 250,
-  "paper_guess_fraction": 0.2,
-  "paper_guess_steps": 20,
-  "loss": {
-    "auc": 0.509088,
-    "empirical_epsilon": {
-      "0.05": 0.17315445840358734,
-      "0.01": 0.0
-    },
-    "empirical_epsilon_details": {
-      "0.05": {
-        "epsilon": 0.17315445840358734,
-        "num_guesses": 5,
-        "correct_guesses": 5,
-        "candidate_num_guesses": [
-          5,
-          10,
-          15,
-          20,
-          25,
-          30,
-          35,
-          40,
-          45,
-          50,
-          55,
-          60,
-          65,
-          70,
-          75,
-          80,
-          85,
-          90,
-          95,
-          100
-        ],
-        "direction": "higher"
-      },
-      "0.01": {
-        "epsilon": 0.0,
-        "num_guesses": 0,
-        "correct_guesses": 0,
-        "candidate_num_guesses": [
-          5,
-          10,
-          15,
-          20,
-          25,
-          30,
-          35,
-          40,
-          45,
-          50,
-          55,
-          60,
-          65,
-          70,
-          75,
-          80,
-          85,
-          90,
-          95,
-          100
-        ],
-        "direction": "lower"
-      }
-    }
-  },
-  "embedding": {
-    "auc": 0.505304,
-    "empirical_epsilon": {
-      "0.05": 0.0,
-      "0.01": 0.0
-    },
-    "empirical_epsilon_details": {
-      "0.05": {
-        "epsilon": 0.0,
-        "num_guesses": 0,
-        "correct_guesses": 0,
-        "candidate_num_guesses": [
-          5,
-          10,
-          15,
-          20,
-          25,
-          30,
-          35,
-          40,
-          45,
-          50,
-          55,
-          60,
-          65,
-          70,
-          75,
-          80,
-          85,
-          90,
-          95,
-          100
-        ],
-        "direction": "lower"
-      },
-      "0.01": {
-        "epsilon": 0.0,
-        "num_guesses": 0,
-        "correct_guesses": 0,
-        "candidate_num_guesses": [
-          5,
-          10,
-          15,
-          20,
-          25,
-          30,
-          35,
-          40,
-          45,
-          50,
-          55,
-          60,
-          65,
-          70,
-          75,
-          80,
-          85,
-          90,
-          95,
-          100
-        ],
-        "direction": "lower"
-      }
-    }
-  }
-}
\ No newline at end of file
diff --git a/llama3-8b/dp3/epochs/epoch_005/adapter/README.md b/llama3-8b/dp3/epochs/epoch_005/adapter/README.md
deleted file mode 100644
index d99a36f688caa7f483f250cdbf16c2feb9635fef..0000000000000000000000000000000000000000
--- a/llama3-8b/dp3/epochs/epoch_005/adapter/README.md
+++ /dev/null
@@ -1,207 +0,0 @@
----
-base_model: meta-llama/Meta-Llama-3-8B
-library_name: peft
-pipeline_tag: text-generation
-tags:
-- base_model:adapter:meta-llama/Meta-Llama-3-8B
-- lora
-- transformers
----
-
-# Model Card for Model ID
-
-<!-- Provide a quick summary of what the model is/does. -->
-
-
-
-## Model Details
-
-### Model Description
-
-<!-- Provide a longer summary of what this model is. -->
-
-
-
-- **Developed by:** [More Information Needed]
-- **Funded by [optional]:** [More Information Needed]
-- **Shared by [optional]:** [More Information Needed]
-- **Model type:** [More Information Needed]
-- **Language(s) (NLP):** [More Information Needed]
-- **License:** [More Information Needed]
-- **Finetuned from model [optional]:** [More Information Needed]
-
-### Model Sources [optional]
-
-<!-- Provide the basic links for the model. -->
-
-- **Repository:** [More Information Needed]
-- **Paper [optional]:** [More Information Needed]
-- **Demo [optional]:** [More Information Needed]
-
-## Uses
-
-<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
-
-### Direct Use
-
-<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
-
-[More Information Needed]
-
-### Downstream Use [optional]
-
-<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
-
-[More Information Needed]
-
-### Out-of-Scope Use
-
-<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
-
-[More Information Needed]
-
-## Bias, Risks, and Limitations
-
-<!-- This section is meant to convey both technical and sociotechnical limitations. -->
-
-[More Information Needed]
-
-### Recommendations
-
-<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
-
-Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
-
-## How to Get Started with the Model
-
-Use the code below to get started with the model.
-
-[More Information Needed]
-
-## Training Details
-
-### Training Data
-
-<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
-
-[More Information Needed]
-
-### Training Procedure
-
-<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
-
-#### Preprocessing [optional]
-
-[More Information Needed]
-
-
-#### Training Hyperparameters
-
-- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
-
-#### Speeds, Sizes, Times [optional]
-
-<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
-
-[More Information Needed]
-
-## Evaluation
-
-<!-- This section describes the evaluation protocols and provides the results. -->
-
-### Testing Data, Factors & Metrics
-
-#### Testing Data
-
-<!-- This should link to a Dataset Card if possible. -->
-
-[More Information Needed]
-
-#### Factors
-
-<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
-
-[More Information Needed]
-
-#### Metrics
-
-<!-- These are the evaluation metrics being used, ideally with a description of why. -->
-
-[More Information Needed]
-
-### Results
-
-[More Information Needed]
-
-#### Summary
-
-
-
-## Model Examination [optional]
-
-<!-- Relevant interpretability work for the model goes here -->
-
-[More Information Needed]
-
-## Environmental Impact
-
-<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
-
-Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
-
-- **Hardware Type:** [More Information Needed]
-- **Hours used:** [More Information Needed]
-- **Cloud Provider:** [More Information Needed]
-- **Compute Region:** [More Information Needed]
-- **Carbon Emitted:** [More Information Needed]
-
-## Technical Specifications [optional]
-
-### Model Architecture and Objective
-
-[More Information Needed]
-
-### Compute Infrastructure
-
-[More Information Needed]
-
-#### Hardware
-
-[More Information Needed]
-
-#### Software
-
-[More Information Needed]
-
-## Citation [optional]
-
-<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
-
-**BibTeX:**
-
-[More Information Needed]
-
-**APA:**
-
-[More Information Needed]
-
-## Glossary [optional]
-
-<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
-
-[More Information Needed]
-
-## More Information [optional]
-
-[More Information Needed]
-
-## Model Card Authors [optional]
-
-[More Information Needed]
-
-## Model Card Contact
-
-[More Information Needed]
-### Framework versions
-
-- PEFT 0.18.1
\ No newline at end of file
diff --git a/llama3-8b/dp3/epochs/epoch_005/adapter/adapter_config.json b/llama3-8b/dp3/epochs/epoch_005/adapter/adapter_config.json
deleted file mode 100644
index 87681ceb84a7a9bd25a46addd37669c4837d5d84..0000000000000000000000000000000000000000
--- a/llama3-8b/dp3/epochs/epoch_005/adapter/adapter_config.json
+++ /dev/null
@@ -1,46 +0,0 @@
-{
-  "alora_invocation_tokens": null,
-  "alpha_pattern": {},
-  "arrow_config": null,
-  "auto_mapping": null,
-  "base_model_name_or_path": "meta-llama/Meta-Llama-3-8B",
-  "bias": "none",
-  "corda_config": null,
-  "ensure_weight_tying": true,
-  "eva_config": null,
-  "exclude_modules": null,
-  "fan_in_fan_out": false,
-  "inference_mode": true,
-  "init_lora_weights": true,
-  "layer_replication": null,
-  "layers_pattern": null,
-  "layers_to_transform": null,
-  "loftq_config": {},
-  "lora_alpha": 32,
-  "lora_bias": false,
-  "lora_dropout": 0.05,
-  "megatron_config": null,
-  "megatron_core": "megatron.core",
-  "modules_to_save": [
-    "lm_head",
-    "embed_tokens"
-  ],
-  "peft_type": "LORA",
-  "peft_version": "0.18.1",
-  "qalora_group_size": 16,
-  "r": 16,
-  "rank_pattern": {},
-  "revision": null,
-  "target_modules": [
-    "o_proj",
-    "k_proj",
-    "v_proj",
-    "q_proj"
-  ],
-  "target_parameters": null,
-  "task_type": "CAUSAL_LM",
-  "trainable_token_indices": null,
-  "use_dora": false,
-  "use_qalora": false,
-  "use_rslora": false
-}
\ No newline at end of file
diff --git a/llama3-8b/dp3/epochs/epoch_005/audit_results.json b/llama3-8b/dp3/epochs/epoch_005/audit_results.json
deleted file mode 100644
index 77ccee7992ff636075dfe3baa7120ec35370069a..0000000000000000000000000000000000000000
--- a/llama3-8b/dp3/epochs/epoch_005/audit_results.json
+++ /dev/null
@@ -1,137 +0,0 @@
-{
-  "delta": 1e-05,
-  "num_canaries": 500,
-  "num_members": 250,
-  "paper_guess_fraction": 0.2,
-  "paper_guess_steps": 20,
-  "loss": {
-    "auc": 0.500744,
-    "empirical_epsilon": {
-      "0.05": 0.0,
-      "0.01": 0.0
-    },
-    "empirical_epsilon_details": {
-      "0.05": {
-        "epsilon": 0.0,
-        "num_guesses": 0,
-        "correct_guesses": 0,
-        "candidate_num_guesses": [
-          5,
-          10,
-          15,
-          20,
-          25,
-          30,
-          35,
-          40,
-          45,
-          50,
-          55,
-          60,
-          65,
-          70,
-          75,
-          80,
-          85,
-          90,
-          95,
-          100
-        ],
-        "direction": "lower"
-      },
-      "0.01": {
-        "epsilon": 0.0,
-        "num_guesses": 0,
-        "correct_guesses": 0,
-        "candidate_num_guesses": [
-          5,
-          10,
-          15,
-          20,
-          25,
-          30,
-          35,
-          40,
-          45,
-          50,
-          55,
-          60,
-          65,
-          70,
-          75,
-          80,
-          85,
-          90,
-          95,
-          100
-        ],
-        "direction": "lower"
-      }
-    }
-  },
-  "embedding": {
-    "auc": 0.503384,
-    "empirical_epsilon": {
-      "0.05": 0.0,
-      "0.01": 0.0
-    },
-    "empirical_epsilon_details": {
-      "0.05": {
-        "epsilon": 0.0,
-        "num_guesses": 0,
-        "correct_guesses": 0,
-        "candidate_num_guesses": [
-          5,
-          10,
-          15,
-          20,
-          25,
-          30,
-          35,
-          40,
-          45,
-          50,
-          55,
-          60,
-          65,
-          70,
-          75,
-          80,
-          85,
-          90,
-          95,
-          100
-        ],
-        "direction": "lower"
-      },
-      "0.01": {
-        "epsilon": 0.0,
-        "num_guesses": 0,
-        "correct_guesses": 0,
-        "candidate_num_guesses": [
-          5,
-          10,
-          15,
-          20,
-          25,
-          30,
-          35,
-          40,
-          45,
-          50,
-          55,
-          60,
-          65,
-          70,
-          75,
-          80,
-          85,
-          90,
-          95,
-          100
-        ],
-        "direction": "lower"
-      }
-    }
-  }
-}
\ No newline at end of file
diff --git a/llama3-8b/dp8/epochs/epoch_001/adapter/README.md b/llama3-8b/dp8/epochs/epoch_001/adapter/README.md
deleted file mode 100644
index d99a36f688caa7f483f250cdbf16c2feb9635fef..0000000000000000000000000000000000000000
--- a/llama3-8b/dp8/epochs/epoch_001/adapter/README.md
+++ /dev/null
@@ -1,207 +0,0 @@
----
-base_model: meta-llama/Meta-Llama-3-8B
-library_name: peft
-pipeline_tag: text-generation
-tags:
-- base_model:adapter:meta-llama/Meta-Llama-3-8B
-- lora
-- transformers
----
-
-# Model Card for Model ID
-
-<!-- Provide a quick summary of what the model is/does. -->
-
-
-
-## Model Details
-
-### Model Description
-
-<!-- Provide a longer summary of what this model is. -->
-
-
-
-- **Developed by:** [More Information Needed]
-- **Funded by [optional]:** [More Information Needed]
-- **Shared by [optional]:** [More Information Needed]
-- **Model type:** [More Information Needed]
-- **Language(s) (NLP):** [More Information Needed]
-- **License:** [More Information Needed]
-- **Finetuned from model [optional]:** [More Information Needed]
-
-### Model Sources [optional]
-
-<!-- Provide the basic links for the model. -->
-
-- **Repository:** [More Information Needed]
-- **Paper [optional]:** [More Information Needed]
-- **Demo [optional]:** [More Information Needed]
-
-## Uses
-
-<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
-
-### Direct Use
-
-<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
-
-[More Information Needed]
-
-### Downstream Use [optional]
-
-<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
-
-[More Information Needed]
-
-### Out-of-Scope Use
-
-<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
-
-[More Information Needed]
-
-## Bias, Risks, and Limitations
-
-<!-- This section is meant to convey both technical and sociotechnical limitations. -->
-
-[More Information Needed]
-
-### Recommendations
-
-<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
-
-Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
-
-## How to Get Started with the Model
-
-Use the code below to get started with the model.
-
-[More Information Needed]
-
-## Training Details
-
-### Training Data
-
-<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
-
-[More Information Needed]
-
-### Training Procedure
-
-<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
-
-#### Preprocessing [optional]
-
-[More Information Needed]
-
-
-#### Training Hyperparameters
-
-- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
-
-#### Speeds, Sizes, Times [optional]
-
-<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
-
-[More Information Needed]
-
-## Evaluation
-
-<!-- This section describes the evaluation protocols and provides the results. -->
-
-### Testing Data, Factors & Metrics
-
-#### Testing Data
-
-<!-- This should link to a Dataset Card if possible. -->
-
-[More Information Needed]
-
-#### Factors
-
-<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
-
-[More Information Needed]
-
-#### Metrics
-
-<!-- These are the evaluation metrics being used, ideally with a description of why. -->
-
-[More Information Needed]
-
-### Results
-
-[More Information Needed]
-
-#### Summary
-
-
-
-## Model Examination [optional]
-
-<!-- Relevant interpretability work for the model goes here -->
-
-[More Information Needed]
-
-## Environmental Impact
-
-<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
-
-Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
-
-- **Hardware Type:** [More Information Needed]
-- **Hours used:** [More Information Needed]
-- **Cloud Provider:** [More Information Needed]
-- **Compute Region:** [More Information Needed]
-- **Carbon Emitted:** [More Information Needed]
-
-## Technical Specifications [optional]
-
-### Model Architecture and Objective
-
-[More Information Needed]
-
-### Compute Infrastructure
-
-[More Information Needed]
-
-#### Hardware
-
-[More Information Needed]
-
-#### Software
-
-[More Information Needed]
-
-## Citation [optional]
-
-<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
-
-**BibTeX:**
-
-[More Information Needed]
-
-**APA:**
-
-[More Information Needed]
-
-## Glossary [optional]
-
-<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
-
-[More Information Needed]
-
-## More Information [optional]
-
-[More Information Needed]
-
-## Model Card Authors [optional]
-
-[More Information Needed]
-
-## Model Card Contact
-
-[More Information Needed]
-### Framework versions
-
-- PEFT 0.18.1
\ No newline at end of file
diff --git a/llama3-8b/dp8/epochs/epoch_001/adapter/adapter_config.json b/llama3-8b/dp8/epochs/epoch_001/adapter/adapter_config.json
deleted file mode 100644
index 5c6876cc26c4df0a214acdca29dbc44c3a4e6217..0000000000000000000000000000000000000000
--- a/llama3-8b/dp8/epochs/epoch_001/adapter/adapter_config.json
+++ /dev/null
@@ -1,46 +0,0 @@
-{
-  "alora_invocation_tokens": null,
-  "alpha_pattern": {},
-  "arrow_config": null,
-  "auto_mapping": null,
-  "base_model_name_or_path": "meta-llama/Meta-Llama-3-8B",
-  "bias": "none",
-  "corda_config": null,
-  "ensure_weight_tying": true,
-  "eva_config": null,
-  "exclude_modules": null,
-  "fan_in_fan_out": false,
-  "inference_mode": true,
-  "init_lora_weights": true,
-  "layer_replication": null,
-  "layers_pattern": null,
-  "layers_to_transform": null,
-  "loftq_config": {},
-  "lora_alpha": 32,
-  "lora_bias": false,
-  "lora_dropout": 0.05,
-  "megatron_config": null,
-  "megatron_core": "megatron.core",
-  "modules_to_save": [
-    "lm_head",
-    "embed_tokens"
-  ],
-  "peft_type": "LORA",
-  "peft_version": "0.18.1",
-  "qalora_group_size": 16,
-  "r": 16,
-  "rank_pattern": {},
-  "revision": null,
-  "target_modules": [
-    "q_proj",
-    "k_proj",
-    "v_proj",
-    "o_proj"
-  ],
-  "target_parameters": null,
-  "task_type": "CAUSAL_LM",
-  "trainable_token_indices": null,
-  "use_dora": false,
-  "use_qalora": false,
-  "use_rslora": false
-}
\ No newline at end of file
diff --git a/llama3-8b/dp8/epochs/epoch_001/audit_results.json b/llama3-8b/dp8/epochs/epoch_001/audit_results.json
deleted file mode 100644
index 437080ae238c3e09842c8dcb8ebcdb8fb37995cf..0000000000000000000000000000000000000000
--- a/llama3-8b/dp8/epochs/epoch_001/audit_results.json
+++ /dev/null
@@ -1,137 +0,0 @@
-{
-  "delta": 1e-05,
-  "num_canaries": 500,
-  "num_members": 250,
-  "paper_guess_fraction": 0.2,
-  "paper_guess_steps": 20,
-  "loss": {
-    "auc": 0.502168,
-    "empirical_epsilon": {
-      "0.05": 0.17315445840358734,
-      "0.01": 0.0
-    },
-    "empirical_epsilon_details": {
-      "0.05": {
-        "epsilon": 0.17315445840358734,
-        "num_guesses": 5,
-        "correct_guesses": 5,
-        "candidate_num_guesses": [
-          5,
-          10,
-          15,
-          20,
-          25,
-          30,
-          35,
-          40,
-          45,
-          50,
-          55,
-          60,
-          65,
-          70,
-          75,
-          80,
-          85,
-          90,
-          95,
-          100
-        ],
-        "direction": "higher"
-      },
-      "0.01": {
-        "epsilon": 0.0,
-        "num_guesses": 0,
-        "correct_guesses": 0,
-        "candidate_num_guesses": [
-          5,
-          10,
-          15,
-          20,
-          25,
-          30,
-          35,
-          40,
-          45,
-          50,
-          55,
-          60,
-          65,
-          70,
-          75,
-          80,
-          85,
-          90,
-          95,
-          100
-        ],
-        "direction": "lower"
-      }
-    }
-  },
-  "embedding": {
-    "auc": 0.501072,
-    "empirical_epsilon": {
-      "0.05": 0.0,
-      "0.01": 0.0
-    },
-    "empirical_epsilon_details": {
-      "0.05": {
-        "epsilon": 0.0,
-        "num_guesses": 0,
-        "correct_guesses": 0,
-        "candidate_num_guesses": [
-          5,
-          10,
-          15,
-          20,
-          25,
-          30,
-          35,
-          40,
-          45,
-          50,
-          55,
-          60,
-          65,
-          70,
-          75,
-          80,
-          85,
-          90,
-          95,
-          100
-        ],
-        "direction": "lower"
-      },
-      "0.01": {
-        "epsilon": 0.0,
-        "num_guesses": 0,
-        "correct_guesses": 0,
-        "candidate_num_guesses": [
-          5,
-          10,
-          15,
-          20,
-          25,
-          30,
-          35,
-          40,
-          45,
-          50,
-          55,
-          60,
-          65,
-          70,
-          75,
-          80,
-          85,
-          90,
-          95,
-          100
-        ],
-        "direction": "lower"
-      }
-    }
-  }
-}
\ No newline at end of file
diff --git a/llama3-8b/dp8/epochs/epoch_002/adapter/README.md b/llama3-8b/dp8/epochs/epoch_002/adapter/README.md
deleted file mode 100644
index d99a36f688caa7f483f250cdbf16c2feb9635fef..0000000000000000000000000000000000000000
--- a/llama3-8b/dp8/epochs/epoch_002/adapter/README.md
+++ /dev/null
@@ -1,207 +0,0 @@
----
-base_model: meta-llama/Meta-Llama-3-8B
-library_name: peft
-pipeline_tag: text-generation
-tags:
-- base_model:adapter:meta-llama/Meta-Llama-3-8B
-- lora
-- transformers
----
-
-# Model Card for Model ID
-
-<!-- Provide a quick summary of what the model is/does. -->
-
-
-
-## Model Details
-
-### Model Description
-
-<!-- Provide a longer summary of what this model is. -->
-
-
-
-- **Developed by:** [More Information Needed]
-- **Funded by [optional]:** [More Information Needed]
-- **Shared by [optional]:** [More Information Needed]
-- **Model type:** [More Information Needed]
-- **Language(s) (NLP):** [More Information Needed]
-- **License:** [More Information Needed]
-- **Finetuned from model [optional]:** [More Information Needed]
-
-### Model Sources [optional]
-
-<!-- Provide the basic links for the model. -->
-
-- **Repository:** [More Information Needed]
-- **Paper [optional]:** [More Information Needed]
-- **Demo [optional]:** [More Information Needed]
-
-## Uses
-
-<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
-
-### Direct Use
-
-<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
-
-[More Information Needed]
-
-### Downstream Use [optional]
-
-<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
-
-[More Information Needed]
-
-### Out-of-Scope Use
-
-<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
-
-[More Information Needed]
-
-## Bias, Risks, and Limitations
-
-<!-- This section is meant to convey both technical and sociotechnical limitations. -->
-
-[More Information Needed]
-
-### Recommendations
-
-<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
-
-Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
-
-## How to Get Started with the Model
-
-Use the code below to get started with the model.
-
-[More Information Needed]
-
-## Training Details
-
-### Training Data
-
-<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
-
-[More Information Needed]
-
-### Training Procedure
-
-<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
-
-#### Preprocessing [optional]
-
-[More Information Needed]
-
-
-#### Training Hyperparameters
-
-- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
-
-#### Speeds, Sizes, Times [optional]
-
-<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
-
-[More Information Needed]
-
-## Evaluation
-
-<!-- This section describes the evaluation protocols and provides the results. -->
-
-### Testing Data, Factors & Metrics
-
-#### Testing Data
-
-<!-- This should link to a Dataset Card if possible. -->
-
-[More Information Needed]
-
-#### Factors
-
-<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
-
-[More Information Needed]
-
-#### Metrics
-
-<!-- These are the evaluation metrics being used, ideally with a description of why. -->
-
-[More Information Needed]
-
-### Results
-
-[More Information Needed]
-
-#### Summary
-
-
-
-## Model Examination [optional]
-
-<!-- Relevant interpretability work for the model goes here -->
-
-[More Information Needed]
-
-## Environmental Impact
-
-<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
-
-Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
-
-- **Hardware Type:** [More Information Needed]
-- **Hours used:** [More Information Needed]
-- **Cloud Provider:** [More Information Needed]
-- **Compute Region:** [More Information Needed]
-- **Carbon Emitted:** [More Information Needed]
-
-## Technical Specifications [optional]
-
-### Model Architecture and Objective
-
-[More Information Needed]
-
-### Compute Infrastructure
-
-[More Information Needed]
-
-#### Hardware
-
-[More Information Needed]
-
-#### Software
-
-[More Information Needed]
-
-## Citation [optional]
-
-<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
-
-**BibTeX:**
-
-[More Information Needed]
-
-**APA:**
-
-[More Information Needed]
-
-## Glossary [optional]
-
-<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
-
-[More Information Needed]
-
-## More Information [optional]
-
-[More Information Needed]
-
-## Model Card Authors [optional]
-
-[More Information Needed]
-
-## Model Card Contact
-
-[More Information Needed]
-### Framework versions
-
-- PEFT 0.18.1
\ No newline at end of file
diff --git a/llama3-8b/dp8/epochs/epoch_002/adapter/adapter_config.json b/llama3-8b/dp8/epochs/epoch_002/adapter/adapter_config.json
deleted file mode 100644
index 5c6876cc26c4df0a214acdca29dbc44c3a4e6217..0000000000000000000000000000000000000000
--- a/llama3-8b/dp8/epochs/epoch_002/adapter/adapter_config.json
+++ /dev/null
@@ -1,46 +0,0 @@
-{
-  "alora_invocation_tokens": null,
-  "alpha_pattern": {},
-  "arrow_config": null,
-  "auto_mapping": null,
-  "base_model_name_or_path": "meta-llama/Meta-Llama-3-8B",
-  "bias": "none",
-  "corda_config": null,
-  "ensure_weight_tying": true,
-  "eva_config": null,
-  "exclude_modules": null,
-  "fan_in_fan_out": false,
-  "inference_mode": true,
-  "init_lora_weights": true,
-  "layer_replication": null,
-  "layers_pattern": null,
-  "layers_to_transform": null,
-  "loftq_config": {},
-  "lora_alpha": 32,
-  "lora_bias": false,
-  "lora_dropout": 0.05,
-  "megatron_config": null,
-  "megatron_core": "megatron.core",
-  "modules_to_save": [
-    "lm_head",
-    "embed_tokens"
-  ],
-  "peft_type": "LORA",
-  "peft_version": "0.18.1",
-  "qalora_group_size": 16,
-  "r": 16,
-  "rank_pattern": {},
-  "revision": null,
-  "target_modules": [
-    "q_proj",
-    "k_proj",
-    "v_proj",
-    "o_proj"
-  ],
-  "target_parameters": null,
-  "task_type": "CAUSAL_LM",
-  "trainable_token_indices": null,
-  "use_dora": false,
-  "use_qalora": false,
-  "use_rslora": false
-}
\ No newline at end of file
diff --git a/llama3-8b/dp8/epochs/epoch_002/audit_results.json b/llama3-8b/dp8/epochs/epoch_002/audit_results.json
deleted file mode 100644
index 95d11a2a0ac19fd43d80ac22aaa13b1dcb87ee2d..0000000000000000000000000000000000000000
--- a/llama3-8b/dp8/epochs/epoch_002/audit_results.json
+++ /dev/null
@@ -1,137 +0,0 @@
-{
-  "delta": 1e-05,
-  "num_canaries": 500,
-  "num_members": 250,
-  "paper_guess_fraction": 0.2,
-  "paper_guess_steps": 20,
-  "loss": {
-    "auc": 0.502312,
-    "empirical_epsilon": {
-      "0.05": 0.0,
-      "0.01": 0.0
-    },
-    "empirical_epsilon_details": {
-      "0.05": {
-        "epsilon": 0.0,
-        "num_guesses": 0,
-        "correct_guesses": 0,
-        "candidate_num_guesses": [
-          5,
-          10,
-          15,
-          20,
-          25,
-          30,
-          35,
-          40,
-          45,
-          50,
-          55,
-          60,
-          65,
-          70,
-          75,
-          80,
-          85,
-          90,
-          95,
-          100
-        ],
-        "direction": "lower"
-      },
-      "0.01": {
-        "epsilon": 0.0,
-        "num_guesses": 0,
-        "correct_guesses": 0,
-        "candidate_num_guesses": [
-          5,
-          10,
-          15,
-          20,
-          25,
-          30,
-          35,
-          40,
-          45,
-          50,
-          55,
-          60,
-          65,
-          70,
-          75,
-          80,
-          85,
-          90,
-          95,
-          100
-        ],
-        "direction": "lower"
-      }
-    }
-  },
-  "embedding": {
-    "auc": 0.519992,
-    "empirical_epsilon": {
-      "0.05": 0.0,
-      "0.01": 0.0
-    },
-    "empirical_epsilon_details": {
-      "0.05": {
-        "epsilon": 0.0,
-        "num_guesses": 0,
-        "correct_guesses": 0,
-        "candidate_num_guesses": [
-          5,
-          10,
-          15,
-          20,
-          25,
-          30,
-          35,
-          40,
-          45,
-          50,
-          55,
-          60,
-          65,
-          70,
-          75,
-          80,
-          85,
-          90,
-          95,
-          100
-        ],
-        "direction": "lower"
-      },
-      "0.01": {
-        "epsilon": 0.0,
-        "num_guesses": 0,
-        "correct_guesses": 0,
-        "candidate_num_guesses": [
-          5,
-          10,
-          15,
-          20,
-          25,
-          30,
-          35,
-          40,
-          45,
-          50,
-          55,
-          60,
-          65,
-          70,
-          75,
-          80,
-          85,
-          90,
-          95,
-          100
-        ],
-        "direction": "lower"
-      }
-    }
-  }
-}
\ No newline at end of file
diff --git a/llama3-8b/dp8/epochs/epoch_003/adapter/README.md b/llama3-8b/dp8/epochs/epoch_003/adapter/README.md
deleted file mode 100644
index d99a36f688caa7f483f250cdbf16c2feb9635fef..0000000000000000000000000000000000000000
--- a/llama3-8b/dp8/epochs/epoch_003/adapter/README.md
+++ /dev/null
@@ -1,207 +0,0 @@
----
-base_model: meta-llama/Meta-Llama-3-8B
-library_name: peft
-pipeline_tag: text-generation
-tags:
-- base_model:adapter:meta-llama/Meta-Llama-3-8B
-- lora
-- transformers
----
-
-# Model Card for Model ID
-
-<!-- Provide a quick summary of what the model is/does. -->
-
-
-
-## Model Details
-
-### Model Description
-
-<!-- Provide a longer summary of what this model is. -->
-
-
-
-- **Developed by:** [More Information Needed]
-- **Funded by [optional]:** [More Information Needed]
-- **Shared by [optional]:** [More Information Needed]
-- **Model type:** [More Information Needed]
-- **Language(s) (NLP):** [More Information Needed]
-- **License:** [More Information Needed]
-- **Finetuned from model [optional]:** [More Information Needed]
-
-### Model Sources [optional]
-
-<!-- Provide the basic links for the model. -->
-
-- **Repository:** [More Information Needed]
-- **Paper [optional]:** [More Information Needed]
-- **Demo [optional]:** [More Information Needed]
-
-## Uses
-
-<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
-
-### Direct Use
-
-<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
-
-[More Information Needed]
-
-### Downstream Use [optional]
-
-<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
-
-[More Information Needed]
-
-### Out-of-Scope Use
-
-<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
-
-[More Information Needed]
-
-## Bias, Risks, and Limitations
-
-<!-- This section is meant to convey both technical and sociotechnical limitations. -->
-
-[More Information Needed]
-
-### Recommendations
-
-<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
-
-Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
-
-## How to Get Started with the Model
-
-Use the code below to get started with the model.
-
-[More Information Needed]
-
-## Training Details
-
-### Training Data
-
-<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
-
-[More Information Needed]
-
-### Training Procedure
-
-<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
-
-#### Preprocessing [optional]
-
-[More Information Needed]
-
-
-#### Training Hyperparameters
-
-- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
-
-#### Speeds, Sizes, Times [optional]
-
-<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
-
-[More Information Needed]
-
-## Evaluation
-
-<!-- This section describes the evaluation protocols and provides the results. -->
-
-### Testing Data, Factors & Metrics
-
-#### Testing Data
-
-<!-- This should link to a Dataset Card if possible. -->
-
-[More Information Needed]
-
-#### Factors
-
-<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
-
-[More Information Needed]
-
-#### Metrics
-
-<!-- These are the evaluation metrics being used, ideally with a description of why. -->
-
-[More Information Needed]
-
-### Results
-
-[More Information Needed]
-
-#### Summary
-
-
-
-## Model Examination [optional]
-
-<!-- Relevant interpretability work for the model goes here -->
-
-[More Information Needed]
-
-## Environmental Impact
-
-<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
-
-Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
-
-- **Hardware Type:** [More Information Needed]
-- **Hours used:** [More Information Needed]
-- **Cloud Provider:** [More Information Needed]
-- **Compute Region:** [More Information Needed]
-- **Carbon Emitted:** [More Information Needed]
-
-## Technical Specifications [optional]
-
-### Model Architecture and Objective
-
-[More Information Needed]
-
-### Compute Infrastructure
-
-[More Information Needed]
-
-#### Hardware
-
-[More Information Needed]
-
-#### Software
-
-[More Information Needed]
-
-## Citation [optional]
-
-<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
-
-**BibTeX:**
-
-[More Information Needed]
-
-**APA:**
-
-[More Information Needed]
-
-## Glossary [optional]
-
-<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
-
-[More Information Needed]
-
-## More Information [optional]
-
-[More Information Needed]
-
-## Model Card Authors [optional]
-
-[More Information Needed]
-
-## Model Card Contact
-
-[More Information Needed]
-### Framework versions
-
-- PEFT 0.18.1
\ No newline at end of file
diff --git a/llama3-8b/dp8/epochs/epoch_003/adapter/adapter_config.json b/llama3-8b/dp8/epochs/epoch_003/adapter/adapter_config.json
deleted file mode 100644
index 5c6876cc26c4df0a214acdca29dbc44c3a4e6217..0000000000000000000000000000000000000000
--- a/llama3-8b/dp8/epochs/epoch_003/adapter/adapter_config.json
+++ /dev/null
@@ -1,46 +0,0 @@
-{
-  "alora_invocation_tokens": null,
-  "alpha_pattern": {},
-  "arrow_config": null,
-  "auto_mapping": null,
-  "base_model_name_or_path": "meta-llama/Meta-Llama-3-8B",
-  "bias": "none",
-  "corda_config": null,
-  "ensure_weight_tying": true,
-  "eva_config": null,
-  "exclude_modules": null,
-  "fan_in_fan_out": false,
-  "inference_mode": true,
-  "init_lora_weights": true,
-  "layer_replication": null,
-  "layers_pattern": null,
-  "layers_to_transform": null,
-  "loftq_config": {},
-  "lora_alpha": 32,
-  "lora_bias": false,
-  "lora_dropout": 0.05,
-  "megatron_config": null,
-  "megatron_core": "megatron.core",
-  "modules_to_save": [
-    "lm_head",
-    "embed_tokens"
-  ],
-  "peft_type": "LORA",
-  "peft_version": "0.18.1",
-  "qalora_group_size": 16,
-  "r": 16,
-  "rank_pattern": {},
-  "revision": null,
-  "target_modules": [
-    "q_proj",
-    "k_proj",
-    "v_proj",
-    "o_proj"
-  ],
-  "target_parameters": null,
-  "task_type": "CAUSAL_LM",
-  "trainable_token_indices": null,
-  "use_dora": false,
-  "use_qalora": false,
-  "use_rslora": false
-}
\ No newline at end of file
diff --git a/llama3-8b/dp8/epochs/epoch_003/audit_results.json b/llama3-8b/dp8/epochs/epoch_003/audit_results.json
deleted file mode 100644
index beb93ff00e81d5304f0bcf453e5576afe4e4e5f0..0000000000000000000000000000000000000000
--- a/llama3-8b/dp8/epochs/epoch_003/audit_results.json
+++ /dev/null
@@ -1,137 +0,0 @@
-{
-  "delta": 1e-05,
-  "num_canaries": 500,
-  "num_members": 250,
-  "paper_guess_fraction": 0.2,
-  "paper_guess_steps": 20,
-  "loss": {
-    "auc": 0.520936,
-    "empirical_epsilon": {
-      "0.05": 0.23387643229216337,
-      "0.01": 0.0
-    },
-    "empirical_epsilon_details": {
-      "0.05": {
-        "epsilon": 0.23387643229216337,
-        "num_guesses": 15,
-        "correct_guesses": 12,
-        "candidate_num_guesses": [
-          5,
-          10,
-          15,
-          20,
-          25,
-          30,
-          35,
-          40,
-          45,
-          50,
-          55,
-          60,
-          65,
-          70,
-          75,
-          80,
-          85,
-          90,
-          95,
-          100
-        ],
-        "direction": "higher"
-      },
-      "0.01": {
-        "epsilon": 0.0,
-        "num_guesses": 0,
-        "correct_guesses": 0,
-        "candidate_num_guesses": [
-          5,
-          10,
-          15,
-          20,
-          25,
-          30,
-          35,
-          40,
-          45,
-          50,
-          55,
-          60,
-          65,
-          70,
-          75,
-          80,
-          85,
-          90,
-          95,
-          100
-        ],
-        "direction": "lower"
-      }
-    }
-  },
-  "embedding": {
-    "auc": 0.502528,
-    "empirical_epsilon": {
-      "0.05": 0.0,
-      "0.01": 0.0
-    },
-    "empirical_epsilon_details": {
-      "0.05": {
-        "epsilon": 0.0,
-        "num_guesses": 0,
-        "correct_guesses": 0,
-        "candidate_num_guesses": [
-          5,
-          10,
-          15,
-          20,
-          25,
-          30,
-          35,
-          40,
-          45,
-          50,
-          55,
-          60,
-          65,
-          70,
-          75,
-          80,
-          85,
-          90,
-          95,
-          100
-        ],
-        "direction": "lower"
-      },
-      "0.01": {
-        "epsilon": 0.0,
-        "num_guesses": 0,
-        "correct_guesses": 0,
-        "candidate_num_guesses": [
-          5,
-          10,
-          15,
-          20,
-          25,
-          30,
-          35,
-          40,
-          45,
-          50,
-          55,
-          60,
-          65,
-          70,
-          75,
-          80,
-          85,
-          90,
-          95,
-          100
-        ],
-        "direction": "lower"
-      }
-    }
-  }
-}
\ No newline at end of file
diff --git a/llama3-8b/dp8/epochs/epoch_004/adapter/README.md b/llama3-8b/dp8/epochs/epoch_004/adapter/README.md
deleted file mode 100644
index d99a36f688caa7f483f250cdbf16c2feb9635fef..0000000000000000000000000000000000000000
--- a/llama3-8b/dp8/epochs/epoch_004/adapter/README.md
+++ /dev/null
@@ -1,207 +0,0 @@
----
-base_model: meta-llama/Meta-Llama-3-8B
-library_name: peft
-pipeline_tag: text-generation
-tags:
-- base_model:adapter:meta-llama/Meta-Llama-3-8B
-- lora
-- transformers
----
-
-# Model Card for Model ID
-
-<!-- Provide a quick summary of what the model is/does. -->
-
-
-
-## Model Details
-
-### Model Description
-
-<!-- Provide a longer summary of what this model is. -->
-
-
-
-- **Developed by:** [More Information Needed]
-- **Funded by [optional]:** [More Information Needed]
-- **Shared by [optional]:** [More Information Needed]
-- **Model type:** [More Information Needed]
-- **Language(s) (NLP):** [More Information Needed]
-- **License:** [More Information Needed]
-- **Finetuned from model [optional]:** [More Information Needed]
-
-### Model Sources [optional]
-
-<!-- Provide the basic links for the model. -->
-
-- **Repository:** [More Information Needed]
-- **Paper [optional]:** [More Information Needed]
-- **Demo [optional]:** [More Information Needed]
-
-## Uses
-
-<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
-
-### Direct Use
-
-<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
-
-[More Information Needed]
-
-### Downstream Use [optional]
-
-<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
-
-[More Information Needed]
-
-### Out-of-Scope Use
-
-<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
-
-[More Information Needed]
-
-## Bias, Risks, and Limitations
-
-<!-- This section is meant to convey both technical and sociotechnical limitations. -->
-
-[More Information Needed]
-
-### Recommendations
-
-<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
-
-Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
-
-## How to Get Started with the Model
-
-Use the code below to get started with the model.
-
-[More Information Needed]
-
-## Training Details
-
-### Training Data
-
-<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
-
-[More Information Needed]
-
-### Training Procedure
-
-<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
-
-#### Preprocessing [optional]
-
-[More Information Needed]
-
-
-#### Training Hyperparameters
-
-- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
-
-#### Speeds, Sizes, Times [optional]
-
-<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
-
-[More Information Needed]
-
-## Evaluation
-
-<!-- This section describes the evaluation protocols and provides the results. -->
-
-### Testing Data, Factors & Metrics
-
-#### Testing Data
-
-<!-- This should link to a Dataset Card if possible. -->
-
-[More Information Needed]
-
-#### Factors
-
-<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
-
-[More Information Needed]
-
-#### Metrics
-
-<!-- These are the evaluation metrics being used, ideally with a description of why. -->
-
-[More Information Needed]
-
-### Results
-
-[More Information Needed]
-
-#### Summary
-
-
-
-## Model Examination [optional]
-
-<!-- Relevant interpretability work for the model goes here -->
-
-[More Information Needed]
-
-## Environmental Impact
-
-<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
-
-Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
-
-- **Hardware Type:** [More Information Needed]
-- **Hours used:** [More Information Needed]
-- **Cloud Provider:** [More Information Needed]
-- **Compute Region:** [More Information Needed]
-- **Carbon Emitted:** [More Information Needed]
-
-## Technical Specifications [optional]
-
-### Model Architecture and Objective
-
-[More Information Needed]
-
-### Compute Infrastructure
-
-[More Information Needed]
-
-#### Hardware
-
-[More Information Needed]
-
-#### Software
-
-[More Information Needed]
-
-## Citation [optional]
-
-<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
-
-**BibTeX:**
-
-[More Information Needed]
-
-**APA:**
-
-[More Information Needed]
-
-## Glossary [optional]
-
-<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
-
-[More Information Needed]
-
-## More Information [optional]
-
-[More Information Needed]
-
-## Model Card Authors [optional]
-
-[More Information Needed]
-
-## Model Card Contact
-
-[More Information Needed]
-### Framework versions
-
-- PEFT 0.18.1
\ No newline at end of file
diff --git a/llama3-8b/dp8/epochs/epoch_004/adapter/adapter_config.json b/llama3-8b/dp8/epochs/epoch_004/adapter/adapter_config.json
deleted file mode 100644
index 5c6876cc26c4df0a214acdca29dbc44c3a4e6217..0000000000000000000000000000000000000000
--- a/llama3-8b/dp8/epochs/epoch_004/adapter/adapter_config.json
+++ /dev/null
@@ -1,46 +0,0 @@
-{
-  "alora_invocation_tokens": null,
-  "alpha_pattern": {},
-  "arrow_config": null,
-  "auto_mapping": null,
-  "base_model_name_or_path": "meta-llama/Meta-Llama-3-8B",
-  "bias": "none",
-  "corda_config": null,
-  "ensure_weight_tying": true,
-  "eva_config": null,
-  "exclude_modules": null,
-  "fan_in_fan_out": false,
-  "inference_mode": true,
-  "init_lora_weights": true,
-  "layer_replication": null,
-  "layers_pattern": null,
-  "layers_to_transform": null,
-  "loftq_config": {},
-  "lora_alpha": 32,
-  "lora_bias": false,
-  "lora_dropout": 0.05,
-  "megatron_config": null,
-  "megatron_core": "megatron.core",
-  "modules_to_save": [
-    "lm_head",
-    "embed_tokens"
-  ],
-  "peft_type": "LORA",
-  "peft_version": "0.18.1",
-  "qalora_group_size": 16,
-  "r": 16,
-  "rank_pattern": {},
-  "revision": null,
-  "target_modules": [
-    "q_proj",
-    "k_proj",
-    "v_proj",
-    "o_proj"
-  ],
-  "target_parameters": null,
-  "task_type": "CAUSAL_LM",
-  "trainable_token_indices": null,
-  "use_dora": false,
-  "use_qalora": false,
-  "use_rslora": false
-}
\ No newline at end of file
diff --git a/llama3-8b/dp8/epochs/epoch_004/audit_results.json b/llama3-8b/dp8/epochs/epoch_004/audit_results.json
deleted file mode 100644
index 5f05fe58b44be9395539a39aad0b3b4bf7443a96..0000000000000000000000000000000000000000
--- a/llama3-8b/dp8/epochs/epoch_004/audit_results.json
+++ /dev/null
@@ -1,137 +0,0 @@
-{
-  "delta": 1e-05,
-  "num_canaries": 500,
-  "num_members": 250,
-  "paper_guess_fraction": 0.2,
-  "paper_guess_steps": 20,
-  "loss": {
-    "auc": 0.529328,
-    "empirical_epsilon": {
-      "0.05": 0.0,
-      "0.01": 0.0
-    },
-    "empirical_epsilon_details": {
-      "0.05": {
-        "epsilon": 0.0,
-        "num_guesses": 0,
-        "correct_guesses": 0,
-        "candidate_num_guesses": [
-          5,
-          10,
-          15,
-          20,
-          25,
-          30,
-          35,
-          40,
-          45,
-          50,
-          55,
-          60,
-          65,
-          70,
-          75,
-          80,
-          85,
-          90,
-          95,
-          100
-        ],
-        "direction": "lower"
-      },
-      "0.01": {
-        "epsilon": 0.0,
-        "num_guesses": 0,
-        "correct_guesses": 0,
-        "candidate_num_guesses": [
-          5,
-          10,
-          15,
-          20,
-          25,
-          30,
-          35,
-          40,
-          45,
-          50,
-          55,
-          60,
-          65,
-          70,
-          75,
-          80,
-          85,
-          90,
-          95,
-          100
-        ],
-        "direction": "lower"
-      }
-    }
-  },
-  "embedding": {
-    "auc": 0.504168,
-    "empirical_epsilon": {
-      "0.05": 0.0,
-      "0.01": 0.0
-    },
-    "empirical_epsilon_details": {
-      "0.05": {
-        "epsilon": 0.0,
-        "num_guesses": 0,
-        "correct_guesses": 0,
-        "candidate_num_guesses": [
-          5,
-          10,
-          15,
-          20,
-          25,
-          30,
-          35,
-          40,
-          45,
-          50,
-          55,
-          60,
-          65,
-          70,
-          75,
-          80,
-          85,
-          90,
-          95,
-          100
-        ],
-        "direction": "lower"
-      },
-      "0.01": {
-        "epsilon": 0.0,
-        "num_guesses": 0,
-        "correct_guesses": 0,
-        "candidate_num_guesses": [
-          5,
-          10,
-          15,
-          20,
-          25,
-          30,
-          35,
-          40,
-          45,
-          50,
-          55,
-          60,
-          65,
-          70,
-          75,
-          80,
-          85,
-          90,
-          95,
-          100
-        ],
-        "direction": "lower"
-      }
-    }
-  }
-}
\ No newline at end of file
diff --git a/llama3-8b/dp8/epochs/epoch_005/adapter/README.md b/llama3-8b/dp8/epochs/epoch_005/adapter/README.md
deleted file mode 100644
index d99a36f688caa7f483f250cdbf16c2feb9635fef..0000000000000000000000000000000000000000
--- a/llama3-8b/dp8/epochs/epoch_005/adapter/README.md
+++ /dev/null
@@ -1,207 +0,0 @@
----
-base_model: meta-llama/Meta-Llama-3-8B
-library_name: peft
-pipeline_tag: text-generation
-tags:
-- base_model:adapter:meta-llama/Meta-Llama-3-8B
-- lora
-- transformers
----
-
-# Model Card for Model ID
-
-<!-- Provide a quick summary of what the model is/does. -->
-
-
-
-## Model Details
-
-### Model Description
-
-<!-- Provide a longer summary of what this model is. -->
-
-
-
-- **Developed by:** [More Information Needed]
-- **Funded by [optional]:** [More Information Needed]
-- **Shared by [optional]:** [More Information Needed]
-- **Model type:** [More Information Needed]
-- **Language(s) (NLP):** [More Information Needed]
-- **License:** [More Information Needed]
-- **Finetuned from model [optional]:** [More Information Needed]
-
-### Model Sources [optional]
-
-<!-- Provide the basic links for the model. -->
-
-- **Repository:** [More Information Needed]
-- **Paper [optional]:** [More Information Needed]
-- **Demo [optional]:** [More Information Needed]
-
-## Uses
-
-<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
-
-### Direct Use
-
-<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
-
-[More Information Needed]
-
-### Downstream Use [optional]
-
-<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
-
-[More Information Needed]
-
-### Out-of-Scope Use
-
-<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
-
-[More Information Needed]
-
-## Bias, Risks, and Limitations
-
-<!-- This section is meant to convey both technical and sociotechnical limitations. -->
-
-[More Information Needed]
-
-### Recommendations
-
-<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
-
-Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
-
-## How to Get Started with the Model
-
-Use the code below to get started with the model.
-
-[More Information Needed]
-
-## Training Details
-
-### Training Data
-
-<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
-
-[More Information Needed]
-
-### Training Procedure
-
-<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
-
-#### Preprocessing [optional]
-
-[More Information Needed]
-
-
-#### Training Hyperparameters
-
-- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
-
-#### Speeds, Sizes, Times [optional]
-
-<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
-
-[More Information Needed]
-
-## Evaluation
-
-<!-- This section describes the evaluation protocols and provides the results. -->
-
-### Testing Data, Factors & Metrics
-
-#### Testing Data
-
-<!-- This should link to a Dataset Card if possible. -->
-
-[More Information Needed]
-
-#### Factors
-
-<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
-
-[More Information Needed]
-
-#### Metrics
-
-<!-- These are the evaluation metrics being used, ideally with a description of why. -->
-
-[More Information Needed]
-
-### Results
-
-[More Information Needed]
-
-#### Summary
-
-
-
-## Model Examination [optional]
-
-<!-- Relevant interpretability work for the model goes here -->
-
-[More Information Needed]
-
-## Environmental Impact
-
-<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
-
-Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
-
-- **Hardware Type:** [More Information Needed]
-- **Hours used:** [More Information Needed]
-- **Cloud Provider:** [More Information Needed]
-- **Compute Region:** [More Information Needed]
-- **Carbon Emitted:** [More Information Needed]
-
-## Technical Specifications [optional]
-
-### Model Architecture and Objective
-
-[More Information Needed]
-
-### Compute Infrastructure
-
-[More Information Needed]
-
-#### Hardware
-
-[More Information Needed]
-
-#### Software
-
-[More Information Needed]
-
-## Citation [optional]
-
-<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
-
-**BibTeX:**
-
-[More Information Needed]
-
-**APA:**
-
-[More Information Needed]
-
-## Glossary [optional]
-
-<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
-
-[More Information Needed]
-
-## More Information [optional]
-
-[More Information Needed]
-
-## Model Card Authors [optional]
-
-[More Information Needed]
-
-## Model Card Contact
-
-[More Information Needed]
-### Framework versions
-
-- PEFT 0.18.1
\ No newline at end of file
diff --git a/llama3-8b/dp8/epochs/epoch_005/adapter/adapter_config.json b/llama3-8b/dp8/epochs/epoch_005/adapter/adapter_config.json
deleted file mode 100644
index 5c6876cc26c4df0a214acdca29dbc44c3a4e6217..0000000000000000000000000000000000000000
--- a/llama3-8b/dp8/epochs/epoch_005/adapter/adapter_config.json
+++ /dev/null
@@ -1,46 +0,0 @@
-{
-  "alora_invocation_tokens": null,
-  "alpha_pattern": {},
-  "arrow_config": null,
-  "auto_mapping": null,
-  "base_model_name_or_path": "meta-llama/Meta-Llama-3-8B",
-  "bias": "none",
-  "corda_config": null,
-  "ensure_weight_tying": true,
-  "eva_config": null,
-  "exclude_modules": null,
-  "fan_in_fan_out": false,
-  "inference_mode": true,
-  "init_lora_weights": true,
-  "layer_replication": null,
-  "layers_pattern": null,
-  "layers_to_transform": null,
-  "loftq_config": {},
-  "lora_alpha": 32,
-  "lora_bias": false,
-  "lora_dropout": 0.05,
-  "megatron_config": null,
-  "megatron_core": "megatron.core",
-  "modules_to_save": [
-    "lm_head",
-    "embed_tokens"
-  ],
-  "peft_type": "LORA",
-  "peft_version": "0.18.1",
-  "qalora_group_size": 16,
-  "r": 16,
-  "rank_pattern": {},
-  "revision": null,
-  "target_modules": [
-    "q_proj",
-    "k_proj",
-    "v_proj",
-    "o_proj"
-  ],
-  "target_parameters": null,
-  "task_type": "CAUSAL_LM",
-  "trainable_token_indices": null,
-  "use_dora": false,
-  "use_qalora": false,
-  "use_rslora": false
-}
\ No newline at end of file
diff --git a/llama3-8b/dp8/epochs/epoch_005/audit_results.json b/llama3-8b/dp8/epochs/epoch_005/audit_results.json
deleted file mode 100644
index fa8a54f630fee5f797363f0d10303d610a14b4b9..0000000000000000000000000000000000000000
--- a/llama3-8b/dp8/epochs/epoch_005/audit_results.json
+++ /dev/null
@@ -1,137 +0,0 @@
-{
-  "delta": 1e-05,
-  "num_canaries": 500,
-  "num_members": 250,
-  "paper_guess_fraction": 0.2,
-  "paper_guess_steps": 20,
-  "loss": {
-    "auc": 0.533056,
-    "empirical_epsilon": {
-      "0.05": 0.0,
-      "0.01": 0.0
-    },
-    "empirical_epsilon_details": {
-      "0.05": {
-        "epsilon": 0.0,
-        "num_guesses": 0,
-        "correct_guesses": 0,
-        "candidate_num_guesses": [
-          5,
-          10,
-          15,
-          20,
-          25,
-          30,
-          35,
-          40,
-          45,
-          50,
-          55,
-          60,
-          65,
-          70,
-          75,
-          80,
-          85,
-          90,
-          95,
-          100
-        ],
-        "direction": "lower"
-      },
-      "0.01": {
-        "epsilon": 0.0,
-        "num_guesses": 0,
-        "correct_guesses": 0,
-        "candidate_num_guesses": [
-          5,
-          10,
-          15,
-          20,
-          25,
-          30,
-          35,
-          40,
-          45,
-          50,
-          55,
-          60,
-          65,
-          70,
-          75,
-          80,
-          85,
-          90,
-          95,
-          100
-        ],
-        "direction": "lower"
-      }
-    }
-  },
-  "embedding": {
-    "auc": 0.500312,
-    "empirical_epsilon": {
-      "0.05": 0.0,
-      "0.01": 0.0
-    },
-    "empirical_epsilon_details": {
-      "0.05": {
-        "epsilon": 0.0,
-        "num_guesses": 0,
-        "correct_guesses": 0,
-        "candidate_num_guesses": [
-          5,
-          10,
-          15,
-          20,
-          25,
-          30,
-          35,
-          40,
-          45,
-          50,
-          55,
-          60,
-          65,
-          70,
-          75,
-          80,
-          85,
-          90,
-          95,
-          100
-        ],
-        "direction": "lower"
-      },
-      "0.01": {
-        "epsilon": 0.0,
-        "num_guesses": 0,
-        "correct_guesses": 0,
-        "candidate_num_guesses": [
-          5,
-          10,
-          15,
-          20,
-          25,
-          30,
-          35,
-          40,
-          45,
-          50,
-          55,
-          60,
-          65,
-          70,
-          75,
-          80,
-          85,
-          90,
-          95,
-          100
-        ],
-        "direction": "lower"
-      }
-    }
-  }
-}
\ No newline at end of file
diff --git a/llama3-8b/dp8_v2/epochs/epoch_001/adapter/README.md b/llama3-8b/dp8_v2/epochs/epoch_001/adapter/README.md
deleted file mode 100644
index d99a36f688caa7f483f250cdbf16c2feb9635fef..0000000000000000000000000000000000000000
--- a/llama3-8b/dp8_v2/epochs/epoch_001/adapter/README.md
+++ /dev/null
@@ -1,207 +0,0 @@
----
-base_model: meta-llama/Meta-Llama-3-8B
-library_name: peft
-pipeline_tag: text-generation
-tags:
-- base_model:adapter:meta-llama/Meta-Llama-3-8B
-- lora
-- transformers
----
-
-# Model Card for Model ID
-
-<!-- Provide a quick summary of what the model is/does. -->
-
-
-
-## Model Details
-
-### Model Description
-
-<!-- Provide a longer summary of what this model is. -->
-
-
-
-- **Developed by:** [More Information Needed]
-- **Funded by [optional]:** [More Information Needed]
-- **Shared by [optional]:** [More Information Needed]
-- **Model type:** [More Information Needed]
-- **Language(s) (NLP):** [More Information Needed]
-- **License:** [More Information Needed]
-- **Finetuned from model [optional]:** [More Information Needed]
-
-### Model Sources [optional]
-
-<!-- Provide the basic links for the model. -->
-
-- **Repository:** [More Information Needed]
-- **Paper [optional]:** [More Information Needed]
-- **Demo [optional]:** [More Information Needed]
-
-## Uses
-
-<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
-
-### Direct Use
-
-<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
-
-[More Information Needed]
-
-### Downstream Use [optional]
-
-<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
-
-[More Information Needed]
-
-### Out-of-Scope Use
-
-<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
-
-[More Information Needed]
-
-## Bias, Risks, and Limitations
-
-<!-- This section is meant to convey both technical and sociotechnical limitations. -->
-
-[More Information Needed]
-
-### Recommendations
-
-<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
-
-Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
-
-## How to Get Started with the Model
-
-Use the code below to get started with the model.
-
-[More Information Needed]
-
-## Training Details
-
-### Training Data
-
-<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
-
-[More Information Needed]
-
-### Training Procedure
-
-<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
-
-#### Preprocessing [optional]
-
-[More Information Needed]
-
-
-#### Training Hyperparameters
-
-- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
-
-#### Speeds, Sizes, Times [optional]
-
-<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
-
-[More Information Needed]
-
-## Evaluation
-
-<!-- This section describes the evaluation protocols and provides the results. -->
-
-### Testing Data, Factors & Metrics
-
-#### Testing Data
-
-<!-- This should link to a Dataset Card if possible. -->
-
-[More Information Needed]
-
-#### Factors
-
-<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
-
-[More Information Needed]
-
-#### Metrics
-
-<!-- These are the evaluation metrics being used, ideally with a description of why. -->
-
-[More Information Needed]
-
-### Results
-
-[More Information Needed]
-
-#### Summary
-
-
-
-## Model Examination [optional]
-
-<!-- Relevant interpretability work for the model goes here -->
-
-[More Information Needed]
-
-## Environmental Impact
-
-<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
-
-Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
-
-- **Hardware Type:** [More Information Needed]
-- **Hours used:** [More Information Needed]
-- **Cloud Provider:** [More Information Needed]
-- **Compute Region:** [More Information Needed]
-- **Carbon Emitted:** [More Information Needed]
-
-## Technical Specifications [optional]
-
-### Model Architecture and Objective
-
-[More Information Needed]
-
-### Compute Infrastructure
-
-[More Information Needed]
-
-#### Hardware
-
-[More Information Needed]
-
-#### Software
-
-[More Information Needed]
-
-## Citation [optional]
-
-<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
-
-**BibTeX:**
-
-[More Information Needed]
-
-**APA:**
-
-[More Information Needed]
-
-## Glossary [optional]
-
-<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
-
-[More Information Needed]
-
-## More Information [optional]
-
-[More Information Needed]
-
-## Model Card Authors [optional]
-
-[More Information Needed]
-
-## Model Card Contact
-
-[More Information Needed]
-### Framework versions
-
-- PEFT 0.18.1
\ No newline at end of file
diff --git a/llama3-8b/dp8_v2/epochs/epoch_001/adapter/adapter_config.json b/llama3-8b/dp8_v2/epochs/epoch_001/adapter/adapter_config.json
deleted file mode 100644
index 0327c3c5d5310b6725dd50830481112b6dd34980..0000000000000000000000000000000000000000
--- a/llama3-8b/dp8_v2/epochs/epoch_001/adapter/adapter_config.json
+++ /dev/null
@@ -1,46 +0,0 @@
-{
-  "alora_invocation_tokens": null,
-  "alpha_pattern": {},
-  "arrow_config": null,
-  "auto_mapping": null,
-  "base_model_name_or_path": "meta-llama/Meta-Llama-3-8B",
-  "bias": "none",
-  "corda_config": null,
-  "ensure_weight_tying": true,
-  "eva_config": null,
-  "exclude_modules": null,
-  "fan_in_fan_out": false,
-  "inference_mode": true,
-  "init_lora_weights": true,
-  "layer_replication": null,
-  "layers_pattern": null,
-  "layers_to_transform": null,
-  "loftq_config": {},
-  "lora_alpha": 32,
-  "lora_bias": false,
-  "lora_dropout": 0.05,
-  "megatron_config": null,
-  "megatron_core": "megatron.core",
-  "modules_to_save": [
-    "lm_head",
-    "embed_tokens"
-  ],
-  "peft_type": "LORA",
-  "peft_version": "0.18.1",
-  "qalora_group_size": 16,
-  "r": 16,
-  "rank_pattern": {},
-  "revision": null,
-  "target_modules": [
-    "k_proj",
-    "q_proj",
-    "o_proj",
-    "v_proj"
-  ],
-  "target_parameters": null,
-  "task_type": "CAUSAL_LM",
-  "trainable_token_indices": null,
-  "use_dora": false,
-  "use_qalora": false,
-  "use_rslora": false
-}
\ No newline at end of file
diff --git a/llama3-8b/dp8_v2/epochs/epoch_001/audit_results.json b/llama3-8b/dp8_v2/epochs/epoch_001/audit_results.json
deleted file mode 100644
index d8bc209e92d11dd00f918364e5f75287c417f13b..0000000000000000000000000000000000000000
--- a/llama3-8b/dp8_v2/epochs/epoch_001/audit_results.json
+++ /dev/null
@@ -1,137 +0,0 @@
-{
-  "delta": 1e-05,
-  "num_canaries": 500,
-  "num_members": 250,
-  "paper_guess_fraction": 0.2,
-  "paper_guess_steps": 20,
-  "loss": {
-    "auc": 0.502928,
-    "empirical_epsilon": {
-      "0.05": 0.0,
-      "0.01": 0.0
-    },
-    "empirical_epsilon_details": {
-      "0.05": {
-        "epsilon": 0.0,
-        "num_guesses": 0,
-        "correct_guesses": 0,
-        "candidate_num_guesses": [
-          5,
-          10,
-          15,
-          20,
-          25,
-          30,
-          35,
-          40,
-          45,
-          50,
-          55,
-          60,
-          65,
-          70,
-          75,
-          80,
-          85,
-          90,
-          95,
-          100
-        ],
-        "direction": "lower"
-      },
-      "0.01": {
-        "epsilon": 0.0,
-        "num_guesses": 0,
-        "correct_guesses": 0,
-        "candidate_num_guesses": [
-          5,
-          10,
-          15,
-          20,
-          25,
-          30,
-          35,
-          40,
-          45,
-          50,
-          55,
-          60,
-          65,
-          70,
-          75,
-          80,
-          85,
-          90,
-          95,
-          100
-        ],
-        "direction": "lower"
-      }
-    }
-  },
-  "embedding": {
-    "auc": 0.5058,
-    "empirical_epsilon": {
-      "0.05": 0.0,
-      "0.01": 0.0
-    },
-    "empirical_epsilon_details": {
-      "0.05": {
-        "epsilon": 0.0,
-        "num_guesses": 0,
-        "correct_guesses": 0,
-        "candidate_num_guesses": [
-          5,
-          10,
-          15,
-          20,
-          25,
-          30,
-          35,
-          40,
-          45,
-          50,
-          55,
-          60,
-          65,
-          70,
-          75,
-          80,
-          85,
-          90,
-          95,
-          100
-        ],
-        "direction": "lower"
-      },
-      "0.01": {
-        "epsilon": 0.0,
-        "num_guesses": 0,
-        "correct_guesses": 0,
-        "candidate_num_guesses": [
-          5,
-          10,
-          15,
-          20,
-          25,
-          30,
-          35,
-          40,
-          45,
-          50,
-          55,
-          60,
-          65,
-          70,
-          75,
-          80,
-          85,
-          90,
-          95,
-          100
-        ],
-        "direction": "lower"
-      }
-    }
-  }
-}
\ No newline at end of file
diff --git a/llama3-8b/dp8_v2/epochs/epoch_002/adapter/README.md b/llama3-8b/dp8_v2/epochs/epoch_002/adapter/README.md
deleted file mode 100644
index d99a36f688caa7f483f250cdbf16c2feb9635fef..0000000000000000000000000000000000000000
--- a/llama3-8b/dp8_v2/epochs/epoch_002/adapter/README.md
+++ /dev/null
@@ -1,207 +0,0 @@
----
-base_model: meta-llama/Meta-Llama-3-8B
-library_name: peft
-pipeline_tag: text-generation
-tags:
-- base_model:adapter:meta-llama/Meta-Llama-3-8B
-- lora
-- transformers
----
-
-# Model Card for Model ID
-
-<!-- Provide a quick summary of what the model is/does. -->
-
-
-
-## Model Details
-
-### Model Description
-
-<!-- Provide a longer summary of what this model is. -->
-
-
-
-- **Developed by:** [More Information Needed]
-- **Funded by [optional]:** [More Information Needed]
-- **Shared by [optional]:** [More Information Needed]
-- **Model type:** [More Information Needed]
-- **Language(s) (NLP):** [More Information Needed]
-- **License:** [More Information Needed]
-- **Finetuned from model [optional]:** [More Information Needed]
-
-### Model Sources [optional]
-
-<!-- Provide the basic links for the model. -->
-
-- **Repository:** [More Information Needed]
-- **Paper [optional]:** [More Information Needed]
-- **Demo [optional]:** [More Information Needed]
-
-## Uses
-
-<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
-
-### Direct Use
-
-<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
-
-[More Information Needed]
-
-### Downstream Use [optional]
-
-<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
-
-[More Information Needed]
-
-### Out-of-Scope Use
-
-<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
-
-[More Information Needed]
-
-## Bias, Risks, and Limitations
-
-<!-- This section is meant to convey both technical and sociotechnical limitations. -->
-
-[More Information Needed]
-
-### Recommendations
-
-<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
-
-Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
-
-## How to Get Started with the Model
-
-Use the code below to get started with the model.
-
-[More Information Needed]
-
-## Training Details
-
-### Training Data
-
-<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
-
-[More Information Needed]
-
-### Training Procedure
-
-<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
-
-#### Preprocessing [optional]
-
-[More Information Needed]
-
-
-#### Training Hyperparameters
-
-- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
-
-#### Speeds, Sizes, Times [optional]
-
-<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
-
-[More Information Needed]
-
-## Evaluation
-
-<!-- This section describes the evaluation protocols and provides the results. -->
-
-### Testing Data, Factors & Metrics
-
-#### Testing Data
-
-<!-- This should link to a Dataset Card if possible. -->
-
-[More Information Needed]
-
-#### Factors
-
-<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
-
-[More Information Needed]
-
-#### Metrics
-
-<!-- These are the evaluation metrics being used, ideally with a description of why. -->
-
-[More Information Needed]
-
-### Results
-
-[More Information Needed]
-
-#### Summary
-
-
-
-## Model Examination [optional]
-
-<!-- Relevant interpretability work for the model goes here -->
-
-[More Information Needed]
-
-## Environmental Impact
-
-<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
-
-Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
-
-- **Hardware Type:** [More Information Needed]
-- **Hours used:** [More Information Needed]
-- **Cloud Provider:** [More Information Needed]
-- **Compute Region:** [More Information Needed]
-- **Carbon Emitted:** [More Information Needed]
-
-## Technical Specifications [optional]
-
-### Model Architecture and Objective
-
-[More Information Needed]
-
-### Compute Infrastructure
-
-[More Information Needed]
-
-#### Hardware
-
-[More Information Needed]
-
-#### Software
-
-[More Information Needed]
-
-## Citation [optional]
-
-<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
-
-**BibTeX:**
-
-[More Information Needed]
-
-**APA:**
-
-[More Information Needed]
-
-## Glossary [optional]
-
-<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
-
-[More Information Needed]
-
-## More Information [optional]
-
-[More Information Needed]
-
-## Model Card Authors [optional]
-
-[More Information Needed]
-
-## Model Card Contact
-
-[More Information Needed]
-### Framework versions
-
-- PEFT 0.18.1
\ No newline at end of file
diff --git a/llama3-8b/dp8_v2/epochs/epoch_002/adapter/adapter_config.json b/llama3-8b/dp8_v2/epochs/epoch_002/adapter/adapter_config.json
deleted file mode 100644
index 0327c3c5d5310b6725dd50830481112b6dd34980..0000000000000000000000000000000000000000
--- a/llama3-8b/dp8_v2/epochs/epoch_002/adapter/adapter_config.json
+++ /dev/null
@@ -1,46 +0,0 @@
-{
-  "alora_invocation_tokens": null,
-  "alpha_pattern": {},
-  "arrow_config": null,
-  "auto_mapping": null,
-  "base_model_name_or_path": "meta-llama/Meta-Llama-3-8B",
-  "bias": "none",
-  "corda_config": null,
-  "ensure_weight_tying": true,
-  "eva_config": null,
-  "exclude_modules": null,
-  "fan_in_fan_out": false,
-  "inference_mode": true,
-  "init_lora_weights": true,
-  "layer_replication": null,
-  "layers_pattern": null,
-  "layers_to_transform": null,
-  "loftq_config": {},
-  "lora_alpha": 32,
-  "lora_bias": false,
-  "lora_dropout": 0.05,
-  "megatron_config": null,
-  "megatron_core": "megatron.core",
-  "modules_to_save": [
-    "lm_head",
-    "embed_tokens"
-  ],
-  "peft_type": "LORA",
-  "peft_version": "0.18.1",
-  "qalora_group_size": 16,
-  "r": 16,
-  "rank_pattern": {},
-  "revision": null,
-  "target_modules": [
-    "k_proj",
-    "q_proj",
-    "o_proj",
-    "v_proj"
-  ],
-  "target_parameters": null,
-  "task_type": "CAUSAL_LM",
-  "trainable_token_indices": null,
-  "use_dora": false,
-  "use_qalora": false,
-  "use_rslora": false
-}
\ No newline at end of file
diff --git a/llama3-8b/dp8_v2/epochs/epoch_002/audit_results.json b/llama3-8b/dp8_v2/epochs/epoch_002/audit_results.json
deleted file mode 100644
index e6f9a9249755baac461bd9bb1231ffd717751406..0000000000000000000000000000000000000000
--- a/llama3-8b/dp8_v2/epochs/epoch_002/audit_results.json
+++ /dev/null
@@ -1,137 +0,0 @@
-{
-  "delta": 1e-05,
-  "num_canaries": 500,
-  "num_members": 250,
-  "paper_guess_fraction": 0.2,
-  "paper_guess_steps": 20,
-  "loss": {
-    "auc": 0.515488,
-    "empirical_epsilon": {
-      "0.05": 0.0,
-      "0.01": 0.0
-    },
-    "empirical_epsilon_details": {
-      "0.05": {
-        "epsilon": 0.0,
-        "num_guesses": 0,
-        "correct_guesses": 0,
-        "candidate_num_guesses": [
-          5,
-          10,
-          15,
-          20,
-          25,
-          30,
-          35,
-          40,
-          45,
-          50,
-          55,
-          60,
-          65,
-          70,
-          75,
-          80,
-          85,
-          90,
-          95,
-          100
-        ],
-        "direction": "lower"
-      },
-      "0.01": {
-        "epsilon": 0.0,
-        "num_guesses": 0,
-        "correct_guesses": 0,
-        "candidate_num_guesses": [
-          5,
-          10,
-          15,
-          20,
-          25,
-          30,
-          35,
-          40,
-          45,
-          50,
-          55,
-          60,
-          65,
-          70,
-          75,
-          80,
-          85,
-          90,
-          95,
-          100
-        ],
-        "direction": "lower"
-      }
-    }
-  },
-  "embedding": {
-    "auc": 0.508552,
-    "empirical_epsilon": {
-      "0.05": 0.0,
-      "0.01": 0.0
-    },
-    "empirical_epsilon_details": {
-      "0.05": {
-        "epsilon": 0.0,
-        "num_guesses": 0,
-        "correct_guesses": 0,
-        "candidate_num_guesses": [
-          5,
-          10,
-          15,
-          20,
-          25,
-          30,
-          35,
-          40,
-          45,
-          50,
-          55,
-          60,
-          65,
-          70,
-          75,
-          80,
-          85,
-          90,
-          95,
-          100
-        ],
-        "direction": "lower"
-      },
-      "0.01": {
-        "epsilon": 0.0,
-        "num_guesses": 0,
-        "correct_guesses": 0,
-        "candidate_num_guesses": [
-          5,
-          10,
-          15,
-          20,
-          25,
-          30,
-          35,
-          40,
-          45,
-          50,
-          55,
-          60,
-          65,
-          70,
-          75,
-          80,
-          85,
-          90,
-          95,
-          100
-        ],
-        "direction": "lower"
-      }
-    }
-  }
-}
\ No newline at end of file
diff --git a/llama3-8b/dp8_v2/epochs/epoch_003/adapter/README.md b/llama3-8b/dp8_v2/epochs/epoch_003/adapter/README.md
deleted file mode 100644
index d99a36f688caa7f483f250cdbf16c2feb9635fef..0000000000000000000000000000000000000000
--- a/llama3-8b/dp8_v2/epochs/epoch_003/adapter/README.md
+++ /dev/null
@@ -1,207 +0,0 @@
----
-base_model: meta-llama/Meta-Llama-3-8B
-library_name: peft
-pipeline_tag: text-generation
-tags:
-- base_model:adapter:meta-llama/Meta-Llama-3-8B
-- lora
-- transformers
----
-
-# Model Card for Model ID
-
-<!-- Provide a quick summary of what the model is/does. -->
-
-
-
-## Model Details
-
-### Model Description
-
-<!-- Provide a longer summary of what this model is. -->
-
-
-
-- **Developed by:** [More Information Needed]
-- **Funded by [optional]:** [More Information Needed]
-- **Shared by [optional]:** [More Information Needed]
-- **Model type:** [More Information Needed]
-- **Language(s) (NLP):** [More Information Needed]
-- **License:** [More Information Needed]
-- **Finetuned from model [optional]:** [More Information Needed]
-
-### Model Sources [optional]
-
-<!-- Provide the basic links for the model. -->
-
-- **Repository:** [More Information Needed]
-- **Paper [optional]:** [More Information Needed]
-- **Demo [optional]:** [More Information Needed]
-
-## Uses
-
-<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
-
-### Direct Use
-
-<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
-
-[More Information Needed]
-
-### Downstream Use [optional]
-
-<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
-
-[More Information Needed]
-
-### Out-of-Scope Use
-
-<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
-
-[More Information Needed]
-
-## Bias, Risks, and Limitations
-
-<!-- This section is meant to convey both technical and sociotechnical limitations. -->
-
-[More Information Needed]
-
-### Recommendations
-
-<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
-
-Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
-
-## How to Get Started with the Model
-
-Use the code below to get started with the model.
-
-[More Information Needed]
-
-## Training Details
-
-### Training Data
-
-<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
-
-[More Information Needed]
-
-### Training Procedure
-
-<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
-
-#### Preprocessing [optional]
-
-[More Information Needed]
-
-
-#### Training Hyperparameters
-
-- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
-
-#### Speeds, Sizes, Times [optional]
-
-<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
-
-[More Information Needed]
-
-## Evaluation
-
-<!-- This section describes the evaluation protocols and provides the results. -->
-
-### Testing Data, Factors & Metrics
-
-#### Testing Data
-
-<!-- This should link to a Dataset Card if possible. -->
-
-[More Information Needed]
-
-#### Factors
-
-<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
-
-[More Information Needed]
-
-#### Metrics
-
-<!-- These are the evaluation metrics being used, ideally with a description of why. -->
-
-[More Information Needed]
-
-### Results
-
-[More Information Needed]
-
-#### Summary
-
-
-
-## Model Examination [optional]
-
-<!-- Relevant interpretability work for the model goes here -->
-
-[More Information Needed]
-
-## Environmental Impact
-
-<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
-
-Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
-
-- **Hardware Type:** [More Information Needed]
-- **Hours used:** [More Information Needed]
-- **Cloud Provider:** [More Information Needed]
-- **Compute Region:** [More Information Needed]
-- **Carbon Emitted:** [More Information Needed]
-
-## Technical Specifications [optional]
-
-### Model Architecture and Objective
-
-[More Information Needed]
-
-### Compute Infrastructure
-
-[More Information Needed]
-
-#### Hardware
-
-[More Information Needed]
-
-#### Software
-
-[More Information Needed]
-
-## Citation [optional]
-
-<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
-
-**BibTeX:**
-
-[More Information Needed]
-
-**APA:**
-
-[More Information Needed]
-
-## Glossary [optional]
-
-<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
-
-[More Information Needed]
-
-## More Information [optional]
-
-[More Information Needed]
-
-## Model Card Authors [optional]
-
-[More Information Needed]
-
-## Model Card Contact
-
-[More Information Needed]
-### Framework versions
-
-- PEFT 0.18.1
\ No newline at end of file
diff --git a/llama3-8b/dp8_v2/epochs/epoch_003/adapter/adapter_config.json b/llama3-8b/dp8_v2/epochs/epoch_003/adapter/adapter_config.json
deleted file mode 100644
index 0327c3c5d5310b6725dd50830481112b6dd34980..0000000000000000000000000000000000000000
--- a/llama3-8b/dp8_v2/epochs/epoch_003/adapter/adapter_config.json
+++ /dev/null
@@ -1,46 +0,0 @@
-{
-  "alora_invocation_tokens": null,
-  "alpha_pattern": {},
-  "arrow_config": null,
-  "auto_mapping": null,
-  "base_model_name_or_path": "meta-llama/Meta-Llama-3-8B",
-  "bias": "none",
-  "corda_config": null,
-  "ensure_weight_tying": true,
-  "eva_config": null,
-  "exclude_modules": null,
-  "fan_in_fan_out": false,
-  "inference_mode": true,
-  "init_lora_weights": true,
-  "layer_replication": null,
-  "layers_pattern": null,
-  "layers_to_transform": null,
-  "loftq_config": {},
-  "lora_alpha": 32,
-  "lora_bias": false,
-  "lora_dropout": 0.05,
-  "megatron_config": null,
-  "megatron_core": "megatron.core",
-  "modules_to_save": [
-    "lm_head",
-    "embed_tokens"
-  ],
-  "peft_type": "LORA",
-  "peft_version": "0.18.1",
-  "qalora_group_size": 16,
-  "r": 16,
-  "rank_pattern": {},
-  "revision": null,
-  "target_modules": [
-    "k_proj",
-    "q_proj",
-    "o_proj",
-    "v_proj"
-  ],
-  "target_parameters": null,
-  "task_type": "CAUSAL_LM",
-  "trainable_token_indices": null,
-  "use_dora": false,
-  "use_qalora": false,
-  "use_rslora": false
-}
\ No newline at end of file
diff --git a/llama3-8b/dp8_v2/epochs/epoch_003/audit_results.json b/llama3-8b/dp8_v2/epochs/epoch_003/audit_results.json
deleted file mode 100644
index 6cc4954580b6d3eb373b013a06c1ec08e5ccf2ae..0000000000000000000000000000000000000000
--- a/llama3-8b/dp8_v2/epochs/epoch_003/audit_results.json
+++ /dev/null
@@ -1,137 +0,0 @@
-{
-  "delta": 1e-05,
-  "num_canaries": 500,
-  "num_members": 250,
-  "paper_guess_fraction": 0.2,
-  "paper_guess_steps": 20,
-  "loss": {
-    "auc": 0.515,
-    "empirical_epsilon": {
-      "0.05": 0.0,
-      "0.01": 0.0
-    },
-    "empirical_epsilon_details": {
-      "0.05": {
-        "epsilon": 0.0,
-        "num_guesses": 0,
-        "correct_guesses": 0,
-        "candidate_num_guesses": [
-          5,
-          10,
-          15,
-          20,
-          25,
-          30,
-          35,
-          40,
-          45,
-          50,
-          55,
-          60,
-          65,
-          70,
-          75,
-          80,
-          85,
-          90,
-          95,
-          100
-        ],
-        "direction": "lower"
-      },
-      "0.01": {
-        "epsilon": 0.0,
-        "num_guesses": 0,
-        "correct_guesses": 0,
-        "candidate_num_guesses": [
-          5,
-          10,
-          15,
-          20,
-          25,
-          30,
-          35,
-          40,
-          45,
-          50,
-          55,
-          60,
-          65,
-          70,
-          75,
-          80,
-          85,
-          90,
-          95,
-          100
-        ],
-        "direction": "lower"
-      }
-    }
-  },
-  "embedding": {
-    "auc": 0.515624,
-    "empirical_epsilon": {
-      "0.05": 0.0,
-      "0.01": 0.0
-    },
-    "empirical_epsilon_details": {
-      "0.05": {
-        "epsilon": 0.0,
-        "num_guesses": 0,
-        "correct_guesses": 0,
-        "candidate_num_guesses": [
-          5,
-          10,
-          15,
-          20,
-          25,
-          30,
-          35,
-          40,
-          45,
-          50,
-          55,
-          60,
-          65,
-          70,
-          75,
-          80,
-          85,
-          90,
-          95,
-          100
-        ],
-        "direction": "lower"
-      },
-      "0.01": {
-        "epsilon": 0.0,
-        "num_guesses": 0,
-        "correct_guesses": 0,
-        "candidate_num_guesses": [
-          5,
-          10,
-          15,
-          20,
-          25,
-          30,
-          35,
-          40,
-          45,
-          50,
-          55,
-          60,
-          65,
-          70,
-          75,
-          80,
-          85,
-          90,
-          95,
-          100
-        ],
-        "direction": "lower"
-      }
-    }
-  }
-}
\ No newline at end of file
diff --git a/llama3-8b/dp8_v2/epochs/epoch_004/adapter/README.md b/llama3-8b/dp8_v2/epochs/epoch_004/adapter/README.md
deleted file mode 100644
index d99a36f688caa7f483f250cdbf16c2feb9635fef..0000000000000000000000000000000000000000
--- a/llama3-8b/dp8_v2/epochs/epoch_004/adapter/README.md
+++ /dev/null
@@ -1,207 +0,0 @@
----
-base_model: meta-llama/Meta-Llama-3-8B
-library_name: peft
-pipeline_tag: text-generation
-tags:
-- base_model:adapter:meta-llama/Meta-Llama-3-8B
-- lora
-- transformers
----
-
-# Model Card for Model ID
-
-<!-- Provide a quick summary of what the model is/does. -->
-
-
-
-## Model Details
-
-### Model Description
-
-<!-- Provide a longer summary of what this model is. -->
-
-
-
-- **Developed by:** [More Information Needed]
-- **Funded by [optional]:** [More Information Needed]
-- **Shared by [optional]:** [More Information Needed]
-- **Model type:** [More Information Needed]
-- **Language(s) (NLP):** [More Information Needed]
-- **License:** [More Information Needed]
-- **Finetuned from model [optional]:** [More Information Needed]
-
-### Model Sources [optional]
-
-<!-- Provide the basic links for the model. -->
-
-- **Repository:** [More Information Needed]
-- **Paper [optional]:** [More Information Needed]
-- **Demo [optional]:** [More Information Needed]
-
-## Uses
-
-<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
-
-### Direct Use
-
-<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
-
-[More Information Needed]
-
-### Downstream Use [optional]
-
-<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
-
-[More Information Needed]
-
-### Out-of-Scope Use
-
-<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
-
-[More Information Needed]
-
-## Bias, Risks, and Limitations
-
-<!-- This section is meant to convey both technical and sociotechnical limitations. -->
-
-[More Information Needed]
-
-### Recommendations
-
-<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
-
-Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
-
-## How to Get Started with the Model
-
-Use the code below to get started with the model.
-
-[More Information Needed]
-
-## Training Details
-
-### Training Data
-
-<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
-
-[More Information Needed]
-
-### Training Procedure
-
-<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
-
-#### Preprocessing [optional]
-
-[More Information Needed]
-
-
-#### Training Hyperparameters
-
-- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
-
-#### Speeds, Sizes, Times [optional]
-
-<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
-
-[More Information Needed]
-
-## Evaluation
-
-<!-- This section describes the evaluation protocols and provides the results. -->
-
-### Testing Data, Factors & Metrics
-
-#### Testing Data
-
-<!-- This should link to a Dataset Card if possible. -->
-
-[More Information Needed]
-
-#### Factors
-
-<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
-
-[More Information Needed]
-
-#### Metrics
-
-<!-- These are the evaluation metrics being used, ideally with a description of why. -->
-
-[More Information Needed]
-
-### Results
-
-[More Information Needed]
-
-#### Summary
-
-
-
-## Model Examination [optional]
-
-<!-- Relevant interpretability work for the model goes here -->
-
-[More Information Needed]
-
-## Environmental Impact
-
-<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
-
-Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
-
-- **Hardware Type:** [More Information Needed]
-- **Hours used:** [More Information Needed]
-- **Cloud Provider:** [More Information Needed]
-- **Compute Region:** [More Information Needed]
-- **Carbon Emitted:** [More Information Needed]
-
-## Technical Specifications [optional]
-
-### Model Architecture and Objective
-
-[More Information Needed]
-
-### Compute Infrastructure
-
-[More Information Needed]
-
-#### Hardware
-
-[More Information Needed]
-
-#### Software
-
-[More Information Needed]
-
-## Citation [optional]
-
-<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
-
-**BibTeX:**
-
-[More Information Needed]
-
-**APA:**
-
-[More Information Needed]
-
-## Glossary [optional]
-
-<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
-
-[More Information Needed]
-
-## More Information [optional]
-
-[More Information Needed]
-
-## Model Card Authors [optional]
-
-[More Information Needed]
-
-## Model Card Contact
-
-[More Information Needed]
-### Framework versions
-
-- PEFT 0.18.1
\ No newline at end of file
diff --git a/llama3-8b/dp8_v2/epochs/epoch_004/adapter/adapter_config.json b/llama3-8b/dp8_v2/epochs/epoch_004/adapter/adapter_config.json
deleted file mode 100644
index 0327c3c5d5310b6725dd50830481112b6dd34980..0000000000000000000000000000000000000000
--- a/llama3-8b/dp8_v2/epochs/epoch_004/adapter/adapter_config.json
+++ /dev/null
@@ -1,46 +0,0 @@
-{
-  "alora_invocation_tokens": null,
-  "alpha_pattern": {},
-  "arrow_config": null,
-  "auto_mapping": null,
-  "base_model_name_or_path": "meta-llama/Meta-Llama-3-8B",
-  "bias": "none",
-  "corda_config": null,
-  "ensure_weight_tying": true,
-  "eva_config": null,
-  "exclude_modules": null,
-  "fan_in_fan_out": false,
-  "inference_mode": true,
-  "init_lora_weights": true,
-  "layer_replication": null,
-  "layers_pattern": null,
-  "layers_to_transform": null,
-  "loftq_config": {},
-  "lora_alpha": 32,
-  "lora_bias": false,
-  "lora_dropout": 0.05,
-  "megatron_config": null,
-  "megatron_core": "megatron.core",
-  "modules_to_save": [
-    "lm_head",
-    "embed_tokens"
-  ],
-  "peft_type": "LORA",
-  "peft_version": "0.18.1",
-  "qalora_group_size": 16,
-  "r": 16,
-  "rank_pattern": {},
-  "revision": null,
-  "target_modules": [
-    "k_proj",
-    "q_proj",
-    "o_proj",
-    "v_proj"
-  ],
-  "target_parameters": null,
-  "task_type": "CAUSAL_LM",
-  "trainable_token_indices": null,
-  "use_dora": false,
-  "use_qalora": false,
-  "use_rslora": false
-}
\ No newline at end of file
diff --git a/llama3-8b/dp8_v2/epochs/epoch_004/audit_results.json b/llama3-8b/dp8_v2/epochs/epoch_004/audit_results.json
deleted file mode 100644
index 723e0a6c5b18f28b3d854bc6d8386c68efde1b2d..0000000000000000000000000000000000000000
--- a/llama3-8b/dp8_v2/epochs/epoch_004/audit_results.json
+++ /dev/null
@@ -1,137 +0,0 @@
-{
-  "delta": 1e-05,
-  "num_canaries": 500,
-  "num_members": 250,
-  "paper_guess_fraction": 0.2,
-  "paper_guess_steps": 20,
-  "loss": {
-    "auc": 0.511312,
-    "empirical_epsilon": {
-      "0.05": 0.3197584319859743,
-      "0.01": 0.030605491250753403
-    },
-    "empirical_epsilon_details": {
-      "0.05": {
-        "epsilon": 0.3197584319859743,
-        "num_guesses": 25,
-        "correct_guesses": 19,
-        "candidate_num_guesses": [
-          5,
-          10,
-          15,
-          20,
-          25,
-          30,
-          35,
-          40,
-          45,
-          50,
-          55,
-          60,
-          65,
-          70,
-          75,
-          80,
-          85,
-          90,
-          95,
-          100
-        ],
-        "direction": "lower"
-      },
-      "0.01": {
-        "epsilon": 0.030605491250753403,
-        "num_guesses": 25,
-        "correct_guesses": 19,
-        "candidate_num_guesses": [
-          5,
-          10,
-          15,
-          20,
-          25,
-          30,
-          35,
-          40,
-          45,
-          50,
-          55,
-          60,
-          65,
-          70,
-          75,
-          80,
-          85,
-          90,
-          95,
-          100
-        ],
-        "direction": "lower"
-      }
-    }
-  },
-  "embedding": {
-    "auc": 0.50536,
-    "empirical_epsilon": {
-      "0.05": 0.0,
-      "0.01": 0.0
-    },
-    "empirical_epsilon_details": {
-      "0.05": {
-        "epsilon": 0.0,
-        "num_guesses": 0,
-        "correct_guesses": 0,
-        "candidate_num_guesses": [
-          5,
-          10,
-          15,
-          20,
-          25,
-          30,
-          35,
-          40,
-          45,
-          50,
-          55,
-          60,
-          65,
-          70,
-          75,
-          80,
-          85,
-          90,
-          95,
-          100
-        ],
-        "direction": "lower"
-      },
-      "0.01": {
-        "epsilon": 0.0,
-        "num_guesses": 0,
-        "correct_guesses": 0,
-        "candidate_num_guesses": [
-          5,
-          10,
-          15,
-          20,
-          25,
-          30,
-          35,
-          40,
-          45,
-          50,
-          55,
-          60,
-          65,
-          70,
-          75,
-          80,
-          85,
-          90,
-          95,
-          100
-        ],
-        "direction": "lower"
-      }
-    }
-  }
-}
\ No newline at end of file
diff --git a/llama3-8b/dp8_v2/epochs/epoch_005/adapter/README.md b/llama3-8b/dp8_v2/epochs/epoch_005/adapter/README.md
deleted file mode 100644
index d99a36f688caa7f483f250cdbf16c2feb9635fef..0000000000000000000000000000000000000000
--- a/llama3-8b/dp8_v2/epochs/epoch_005/adapter/README.md
+++ /dev/null
@@ -1,207 +0,0 @@
----
-base_model: meta-llama/Meta-Llama-3-8B
-library_name: peft
-pipeline_tag: text-generation
-tags:
-- base_model:adapter:meta-llama/Meta-Llama-3-8B
-- lora
-- transformers
----
-
-# Model Card for Model ID
-
-<!-- Provide a quick summary of what the model is/does. -->
-
-
-
-## Model Details
-
-### Model Description
-
-<!-- Provide a longer summary of what this model is. -->
-
-
-
-- **Developed by:** [More Information Needed]
-- **Funded by [optional]:** [More Information Needed]
-- **Shared by [optional]:** [More Information Needed]
-- **Model type:** [More Information Needed]
-- **Language(s) (NLP):** [More Information Needed]
-- **License:** [More Information Needed]
-- **Finetuned from model [optional]:** [More Information Needed]
-
-### Model Sources [optional]
-
-<!-- Provide the basic links for the model. -->
-
-- **Repository:** [More Information Needed]
-- **Paper [optional]:** [More Information Needed]
-- **Demo [optional]:** [More Information Needed]
-
-## Uses
-
-<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
-
-### Direct Use
-
-<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
-
-[More Information Needed]
-
-### Downstream Use [optional]
-
-<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
-
-[More Information Needed]
-
-### Out-of-Scope Use
-
-<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
-
-[More Information Needed]
-
-## Bias, Risks, and Limitations
-
-<!-- This section is meant to convey both technical and sociotechnical limitations. -->
-
-[More Information Needed]
-
-### Recommendations
-
-<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
-
-Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
-
-## How to Get Started with the Model
-
-Use the code below to get started with the model.
-
-[More Information Needed]
-
-## Training Details
-
-### Training Data
-
-<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
-
-[More Information Needed]
-
-### Training Procedure
-
-<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
-
-#### Preprocessing [optional]
-
-[More Information Needed]
-
-
-#### Training Hyperparameters
-
-- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
-
-#### Speeds, Sizes, Times [optional]
-
-<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
-
-[More Information Needed]
-
-## Evaluation
-
-<!-- This section describes the evaluation protocols and provides the results. -->
-
-### Testing Data, Factors & Metrics
-
-#### Testing Data
-
-<!-- This should link to a Dataset Card if possible. -->
-
-[More Information Needed]
-
-#### Factors
-
-<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
-
-[More Information Needed]
-
-#### Metrics
-
-<!-- These are the evaluation metrics being used, ideally with a description of why. -->
-
-[More Information Needed]
-
-### Results
-
-[More Information Needed]
-
-#### Summary
-
-
-
-## Model Examination [optional]
-
-<!-- Relevant interpretability work for the model goes here -->
-
-[More Information Needed]
-
-## Environmental Impact
-
-<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
-
-Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
-
-- **Hardware Type:** [More Information Needed]
-- **Hours used:** [More Information Needed]
-- **Cloud Provider:** [More Information Needed]
-- **Compute Region:** [More Information Needed]
-- **Carbon Emitted:** [More Information Needed]
-
-## Technical Specifications [optional]
-
-### Model Architecture and Objective
-
-[More Information Needed]
-
-### Compute Infrastructure
-
-[More Information Needed]
-
-#### Hardware
-
-[More Information Needed]
-
-#### Software
-
-[More Information Needed]
-
-## Citation [optional]
-
-<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
-
-**BibTeX:**
-
-[More Information Needed]
-
-**APA:**
-
-[More Information Needed]
-
-## Glossary [optional]
-
-<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
-
-[More Information Needed]
-
-## More Information [optional]
-
-[More Information Needed]
-
-## Model Card Authors [optional]
-
-[More Information Needed]
-
-## Model Card Contact
-
-[More Information Needed]
-### Framework versions
-
-- PEFT 0.18.1
\ No newline at end of file
diff --git a/llama3-8b/dp8_v2/epochs/epoch_005/adapter/adapter_config.json b/llama3-8b/dp8_v2/epochs/epoch_005/adapter/adapter_config.json
deleted file mode 100644
index 0327c3c5d5310b6725dd50830481112b6dd34980..0000000000000000000000000000000000000000
--- a/llama3-8b/dp8_v2/epochs/epoch_005/adapter/adapter_config.json
+++ /dev/null
@@ -1,46 +0,0 @@
-{
-  "alora_invocation_tokens": null,
-  "alpha_pattern": {},
-  "arrow_config": null,
-  "auto_mapping": null,
-  "base_model_name_or_path": "meta-llama/Meta-Llama-3-8B",
-  "bias": "none",
-  "corda_config": null,
-  "ensure_weight_tying": true,
-  "eva_config": null,
-  "exclude_modules": null,
-  "fan_in_fan_out": false,
-  "inference_mode": true,
-  "init_lora_weights": true,
-  "layer_replication": null,
-  "layers_pattern": null,
-  "layers_to_transform": null,
-  "loftq_config": {},
-  "lora_alpha": 32,
-  "lora_bias": false,
-  "lora_dropout": 0.05,
-  "megatron_config": null,
-  "megatron_core": "megatron.core",
-  "modules_to_save": [
-    "lm_head",
-    "embed_tokens"
-  ],
-  "peft_type": "LORA",
-  "peft_version": "0.18.1",
-  "qalora_group_size": 16,
-  "r": 16,
-  "rank_pattern": {},
-  "revision": null,
-  "target_modules": [
-    "k_proj",
-    "q_proj",
-    "o_proj",
-    "v_proj"
-  ],
-  "target_parameters": null,
-  "task_type": "CAUSAL_LM",
-  "trainable_token_indices": null,
-  "use_dora": false,
-  "use_qalora": false,
-  "use_rslora": false
-}
\ No newline at end of file
diff --git a/llama3-8b/dp8_v2/epochs/epoch_005/audit_results.json b/llama3-8b/dp8_v2/epochs/epoch_005/audit_results.json
deleted file mode 100644
index 792780f7d8966f285fbe75637df31a25cde5e49c..0000000000000000000000000000000000000000
--- a/llama3-8b/dp8_v2/epochs/epoch_005/audit_results.json
+++ /dev/null
@@ -1,137 +0,0 @@
-{
-  "delta": 1e-05,
-  "num_canaries": 500,
-  "num_members": 250,
-  "paper_guess_fraction": 0.2,
-  "paper_guess_steps": 20,
-  "loss": {
-    "auc": 0.526808,
-    "empirical_epsilon": {
-      "0.05": 0.0,
-      "0.01": 0.0
-    },
-    "empirical_epsilon_details": {
-      "0.05": {
-        "epsilon": 0.0,
-        "num_guesses": 0,
-        "correct_guesses": 0,
-        "candidate_num_guesses": [
-          5,
-          10,
-          15,
-          20,
-          25,
-          30,
-          35,
-          40,
-          45,
-          50,
-          55,
-          60,
-          65,
-          70,
-          75,
-          80,
-          85,
-          90,
-          95,
-          100
-        ],
-        "direction": "lower"
-      },
-      "0.01": {
-        "epsilon": 0.0,
-        "num_guesses": 0,
-        "correct_guesses": 0,
-        "candidate_num_guesses": [
-          5,
-          10,
-          15,
-          20,
-          25,
-          30,
-          35,
-          40,
-          45,
-          50,
-          55,
-          60,
-          65,
-          70,
-          75,
-          80,
-          85,
-          90,
-          95,
-          100
-        ],
-        "direction": "lower"
-      }
-    }
-  },
-  "embedding": {
-    "auc": 0.501096,
-    "empirical_epsilon": {
-      "0.05": 0.0,
-      "0.01": 0.0
-    },
-    "empirical_epsilon_details": {
-      "0.05": {
-        "epsilon": 0.0,
-        "num_guesses": 0,
-        "correct_guesses": 0,
-        "candidate_num_guesses": [
-          5,
-          10,
-          15,
-          20,
-          25,
-          30,
-          35,
-          40,
-          45,
-          50,
-          55,
-          60,
-          65,
-          70,
-          75,
-          80,
-          85,
-          90,
-          95,
-          100
-        ],
-        "direction": "lower"
-      },
-      "0.01": {
-        "epsilon": 0.0,
-        "num_guesses": 0,
-        "correct_guesses": 0,
-        "candidate_num_guesses": [
-          5,
-          10,
-          15,
-          20,
-          25,
-          30,
-          35,
-          40,
-          45,
-          50,
-          55,
-          60,
-          65,
-          70,
-          75,
-          80,
-          85,
-          90,
-          95,
-          100
-        ],
-        "direction": "lower"
-      }
-    }
-  }
-}
\ No newline at end of file
diff --git a/llama3-8b/dp8_v2/epochs/epoch_006/adapter/README.md b/llama3-8b/dp8_v2/epochs/epoch_006/adapter/README.md
deleted file mode 100644
index d99a36f688caa7f483f250cdbf16c2feb9635fef..0000000000000000000000000000000000000000
--- a/llama3-8b/dp8_v2/epochs/epoch_006/adapter/README.md
+++ /dev/null
@@ -1,207 +0,0 @@
----
-base_model: meta-llama/Meta-Llama-3-8B
-library_name: peft
-pipeline_tag: text-generation
-tags:
-- base_model:adapter:meta-llama/Meta-Llama-3-8B
-- lora
-- transformers
----
-
-# Model Card for Model ID
-
-<!-- Provide a quick summary of what the model is/does. -->
-
-
-
-## Model Details
-
-### Model Description
-
-<!-- Provide a longer summary of what this model is. -->
-
-
-
-- **Developed by:** [More Information Needed]
-- **Funded by [optional]:** [More Information Needed]
-- **Shared by [optional]:** [More Information Needed]
-- **Model type:** [More Information Needed]
-- **Language(s) (NLP):** [More Information Needed]
-- **License:** [More Information Needed]
-- **Finetuned from model [optional]:** [More Information Needed]
-
-### Model Sources [optional]
-
-<!-- Provide the basic links for the model. -->
-
-- **Repository:** [More Information Needed]
-- **Paper [optional]:** [More Information Needed]
-- **Demo [optional]:** [More Information Needed]
-
-## Uses
-
-<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
-
-### Direct Use
-
-<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
-
-[More Information Needed]
-
-### Downstream Use [optional]
-
-<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
-
-[More Information Needed]
-
-### Out-of-Scope Use
-
-<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
-
-[More Information Needed]
-
-## Bias, Risks, and Limitations
-
-<!-- This section is meant to convey both technical and sociotechnical limitations. -->
-
-[More Information Needed]
-
-### Recommendations
-
-<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
-
-Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
-
-## How to Get Started with the Model
-
-Use the code below to get started with the model.
-
-[More Information Needed]
-
-## Training Details
-
-### Training Data
-
-<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
-
-[More Information Needed]
-
-### Training Procedure
-
-<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
-
-#### Preprocessing [optional]
-
-[More Information Needed]
-
-
-#### Training Hyperparameters
-
-- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
-
-#### Speeds, Sizes, Times [optional]
-
-<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
-
-[More Information Needed]
-
-## Evaluation
-
-<!-- This section describes the evaluation protocols and provides the results. -->
-
-### Testing Data, Factors & Metrics
-
-#### Testing Data
-
-<!-- This should link to a Dataset Card if possible. -->
-
-[More Information Needed]
-
-#### Factors
-
-<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
-
-[More Information Needed]
-
-#### Metrics
-
-<!-- These are the evaluation metrics being used, ideally with a description of why. -->
-
-[More Information Needed]
-
-### Results
-
-[More Information Needed]
-
-#### Summary
-
-
-
-## Model Examination [optional]
-
-<!-- Relevant interpretability work for the model goes here -->
-
-[More Information Needed]
-
-## Environmental Impact
-
-<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
-
-Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
-
-- **Hardware Type:** [More Information Needed]
-- **Hours used:** [More Information Needed]
-- **Cloud Provider:** [More Information Needed]
-- **Compute Region:** [More Information Needed]
-- **Carbon Emitted:** [More Information Needed]
-
-## Technical Specifications [optional]
-
-### Model Architecture and Objective
-
-[More Information Needed]
-
-### Compute Infrastructure
-
-[More Information Needed]
-
-#### Hardware
-
-[More Information Needed]
-
-#### Software
-
-[More Information Needed]
-
-## Citation [optional]
-
-<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
-
-**BibTeX:**
-
-[More Information Needed]
-
-**APA:**
-
-[More Information Needed]
-
-## Glossary [optional]
-
-<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
-
-[More Information Needed]
-
-## More Information [optional]
-
-[More Information Needed]
-
-## Model Card Authors [optional]
-
-[More Information Needed]
-
-## Model Card Contact
-
-[More Information Needed]
-### Framework versions
-
-- PEFT 0.18.1
\ No newline at end of file
diff --git a/llama3-8b/dp8_v2/epochs/epoch_006/adapter/adapter_config.json b/llama3-8b/dp8_v2/epochs/epoch_006/adapter/adapter_config.json
deleted file mode 100644
index 0327c3c5d5310b6725dd50830481112b6dd34980..0000000000000000000000000000000000000000
--- a/llama3-8b/dp8_v2/epochs/epoch_006/adapter/adapter_config.json
+++ /dev/null
@@ -1,46 +0,0 @@
-{
-  "alora_invocation_tokens": null,
-  "alpha_pattern": {},
-  "arrow_config": null,
-  "auto_mapping": null,
-  "base_model_name_or_path": "meta-llama/Meta-Llama-3-8B",
-  "bias": "none",
-  "corda_config": null,
-  "ensure_weight_tying": true,
-  "eva_config": null,
-  "exclude_modules": null,
-  "fan_in_fan_out": false,
-  "inference_mode": true,
-  "init_lora_weights": true,
-  "layer_replication": null,
-  "layers_pattern": null,
-  "layers_to_transform": null,
-  "loftq_config": {},
-  "lora_alpha": 32,
-  "lora_bias": false,
-  "lora_dropout": 0.05,
-  "megatron_config": null,
-  "megatron_core": "megatron.core",
-  "modules_to_save": [
-    "lm_head",
-    "embed_tokens"
-  ],
-  "peft_type": "LORA",
-  "peft_version": "0.18.1",
-  "qalora_group_size": 16,
-  "r": 16,
-  "rank_pattern": {},
-  "revision": null,
-  "target_modules": [
-    "k_proj",
-    "q_proj",
-    "o_proj",
-    "v_proj"
-  ],
-  "target_parameters": null,
-  "task_type": "CAUSAL_LM",
-  "trainable_token_indices": null,
-  "use_dora": false,
-  "use_qalora": false,
-  "use_rslora": false
-}
\ No newline at end of file
diff --git a/llama3-8b/dp8_v2/epochs/epoch_006/audit_results.json b/llama3-8b/dp8_v2/epochs/epoch_006/audit_results.json
deleted file mode 100644
index a7416cfc38c1a32c7a40a8ac8851b87163ce949a..0000000000000000000000000000000000000000
--- a/llama3-8b/dp8_v2/epochs/epoch_006/audit_results.json
+++ /dev/null
@@ -1,137 +0,0 @@
-{
-  "delta": 1e-05,
-  "num_canaries": 500,
-  "num_members": 250,
-  "paper_guess_fraction": 0.2,
-  "paper_guess_steps": 20,
-  "loss": {
-    "auc": 0.53232,
-    "empirical_epsilon": {
-      "0.05": 0.0,
-      "0.01": 0.0
-    },
-    "empirical_epsilon_details": {
-      "0.05": {
-        "epsilon": 0.0,
-        "num_guesses": 0,
-        "correct_guesses": 0,
-        "candidate_num_guesses": [
-          5,
-          10,
-          15,
-          20,
-          25,
-          30,
-          35,
-          40,
-          45,
-          50,
-          55,
-          60,
-          65,
-          70,
-          75,
-          80,
-          85,
-          90,
-          95,
-          100
-        ],
-        "direction": "lower"
-      },
-      "0.01": {
-        "epsilon": 0.0,
-        "num_guesses": 0,
-        "correct_guesses": 0,
-        "candidate_num_guesses": [
-          5,
-          10,
-          15,
-          20,
-          25,
-          30,
-          35,
-          40,
-          45,
-          50,
-          55,
-          60,
-          65,
-          70,
-          75,
-          80,
-          85,
-          90,
-          95,
-          100
-        ],
-        "direction": "lower"
-      }
-    }
-  },
-  "embedding": {
-    "auc": 0.50044,
-    "empirical_epsilon": {
-      "0.05": 0.0,
-      "0.01": 0.0
-    },
-    "empirical_epsilon_details": {
-      "0.05": {
-        "epsilon": 0.0,
-        "num_guesses": 0,
-        "correct_guesses": 0,
-        "candidate_num_guesses": [
-          5,
-          10,
-          15,
-          20,
-          25,
-          30,
-          35,
-          40,
-          45,
-          50,
-          55,
-          60,
-          65,
-          70,
-          75,
-          80,
-          85,
-          90,
-          95,
-          100
-        ],
-        "direction": "lower"
-      },
-      "0.01": {
-        "epsilon": 0.0,
-        "num_guesses": 0,
-        "correct_guesses": 0,
-        "candidate_num_guesses": [
-          5,
-          10,
-          15,
-          20,
-          25,
-          30,
-          35,
-          40,
-          45,
-          50,
-          55,
-          60,
-          65,
-          70,
-          75,
-          80,
-          85,
-          90,
-          95,
-          100
-        ],
-        "direction": "lower"
-      }
-    }
-  }
-}
\ No newline at end of file
diff --git a/llama3-8b/dp8_v2/epochs/epoch_007/adapter/README.md b/llama3-8b/dp8_v2/epochs/epoch_007/adapter/README.md
deleted file mode 100644
index d99a36f688caa7f483f250cdbf16c2feb9635fef..0000000000000000000000000000000000000000
--- a/llama3-8b/dp8_v2/epochs/epoch_007/adapter/README.md
+++ /dev/null
@@ -1,207 +0,0 @@
----
-base_model: meta-llama/Meta-Llama-3-8B
-library_name: peft
-pipeline_tag: text-generation
-tags:
-- base_model:adapter:meta-llama/Meta-Llama-3-8B
-- lora
-- transformers
----
-
-# Model Card for Model ID
-
-<!-- Provide a quick summary of what the model is/does. -->
-
-
-
-## Model Details
-
-### Model Description
-
-<!-- Provide a longer summary of what this model is. -->
-
-
-
-- **Developed by:** [More Information Needed]
-- **Funded by [optional]:** [More Information Needed]
-- **Shared by [optional]:** [More Information Needed]
-- **Model type:** [More Information Needed]
-- **Language(s) (NLP):** [More Information Needed]
-- **License:** [More Information Needed]
-- **Finetuned from model [optional]:** [More Information Needed]
-
-### Model Sources [optional]
-
-<!-- Provide the basic links for the model. -->
-
-- **Repository:** [More Information Needed]
-- **Paper [optional]:** [More Information Needed]
-- **Demo [optional]:** [More Information Needed]
-
-## Uses
-
-<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
-
-### Direct Use
-
-<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
-
-[More Information Needed]
-
-### Downstream Use [optional]
-
-<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
-
-[More Information Needed]
-
-### Out-of-Scope Use
-
-<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
-
-[More Information Needed]
-
-## Bias, Risks, and Limitations
-
-<!-- This section is meant to convey both technical and sociotechnical limitations. -->
-
-[More Information Needed]
-
-### Recommendations
-
-<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
-
-Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
-
-## How to Get Started with the Model
-
-Use the code below to get started with the model.
-
-[More Information Needed]
-
-## Training Details
-
-### Training Data
-
-<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
-
-[More Information Needed]
-
-### Training Procedure
-
-<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
-
-#### Preprocessing [optional]
-
-[More Information Needed]
-
-
-#### Training Hyperparameters
-
-- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
-
-#### Speeds, Sizes, Times [optional]
-
-<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
-
-[More Information Needed]
-
-## Evaluation
-
-<!-- This section describes the evaluation protocols and provides the results. -->
-
-### Testing Data, Factors & Metrics
-
-#### Testing Data
-
-<!-- This should link to a Dataset Card if possible. -->
-
-[More Information Needed]
-
-#### Factors
-
-<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
-
-[More Information Needed]
-
-#### Metrics
-
-<!-- These are the evaluation metrics being used, ideally with a description of why. -->
-
-[More Information Needed]
-
-### Results
-
-[More Information Needed]
-
-#### Summary
-
-
-
-## Model Examination [optional]
-
-<!-- Relevant interpretability work for the model goes here -->
-
-[More Information Needed]
-
-## Environmental Impact
-
-<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
-
-Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
-
-- **Hardware Type:** [More Information Needed]
-- **Hours used:** [More Information Needed]
-- **Cloud Provider:** [More Information Needed]
-- **Compute Region:** [More Information Needed]
-- **Carbon Emitted:** [More Information Needed]
-
-## Technical Specifications [optional]
-
-### Model Architecture and Objective
-
-[More Information Needed]
-
-### Compute Infrastructure
-
-[More Information Needed]
-
-#### Hardware
-
-[More Information Needed]
-
-#### Software
-
-[More Information Needed]
-
-## Citation [optional]
-
-<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
-
-**BibTeX:**
-
-[More Information Needed]
-
-**APA:**
-
-[More Information Needed]
-
-## Glossary [optional]
-
-<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
-
-[More Information Needed]
-
-## More Information [optional]
-
-[More Information Needed]
-
-## Model Card Authors [optional]
-
-[More Information Needed]
-
-## Model Card Contact
-
-[More Information Needed]
-### Framework versions
-
-- PEFT 0.18.1
\ No newline at end of file
diff --git a/llama3-8b/dp8_v2/epochs/epoch_007/adapter/adapter_config.json b/llama3-8b/dp8_v2/epochs/epoch_007/adapter/adapter_config.json
deleted file mode 100644
index 0327c3c5d5310b6725dd50830481112b6dd34980..0000000000000000000000000000000000000000
--- a/llama3-8b/dp8_v2/epochs/epoch_007/adapter/adapter_config.json
+++ /dev/null
@@ -1,46 +0,0 @@
-{
-  "alora_invocation_tokens": null,
-  "alpha_pattern": {},
-  "arrow_config": null,
-  "auto_mapping": null,
-  "base_model_name_or_path": "meta-llama/Meta-Llama-3-8B",
-  "bias": "none",
-  "corda_config": null,
-  "ensure_weight_tying": true,
-  "eva_config": null,
-  "exclude_modules": null,
-  "fan_in_fan_out": false,
-  "inference_mode": true,
-  "init_lora_weights": true,
-  "layer_replication": null,
-  "layers_pattern": null,
-  "layers_to_transform": null,
-  "loftq_config": {},
-  "lora_alpha": 32,
-  "lora_bias": false,
-  "lora_dropout": 0.05,
-  "megatron_config": null,
-  "megatron_core": "megatron.core",
-  "modules_to_save": [
-    "lm_head",
-    "embed_tokens"
-  ],
-  "peft_type": "LORA",
-  "peft_version": "0.18.1",
-  "qalora_group_size": 16,
-  "r": 16,
-  "rank_pattern": {},
-  "revision": null,
-  "target_modules": [
-    "k_proj",
-    "q_proj",
-    "o_proj",
-    "v_proj"
-  ],
-  "target_parameters": null,
-  "task_type": "CAUSAL_LM",
-  "trainable_token_indices": null,
-  "use_dora": false,
-  "use_qalora": false,
-  "use_rslora": false
-}
\ No newline at end of file
diff --git a/llama3-8b/dp8_v2/epochs/epoch_007/audit_results.json b/llama3-8b/dp8_v2/epochs/epoch_007/audit_results.json
deleted file mode 100644
index 02687c49ad52f4bb091c8bd4a786f1dcc6e760ef..0000000000000000000000000000000000000000
--- a/llama3-8b/dp8_v2/epochs/epoch_007/audit_results.json
+++ /dev/null
@@ -1,137 +0,0 @@
-{
-  "delta": 1e-05,
-  "num_canaries": 500,
-  "num_members": 250,
-  "paper_guess_fraction": 0.2,
-  "paper_guess_steps": 20,
-  "loss": {
-    "auc": 0.535992,
-    "empirical_epsilon": {
-      "0.05": 0.0,
-      "0.01": 0.0
-    },
-    "empirical_epsilon_details": {
-      "0.05": {
-        "epsilon": 0.0,
-        "num_guesses": 0,
-        "correct_guesses": 0,
-        "candidate_num_guesses": [
-          5,
-          10,
-          15,
-          20,
-          25,
-          30,
-          35,
-          40,
-          45,
-          50,
-          55,
-          60,
-          65,
-          70,
-          75,
-          80,
-          85,
-          90,
-          95,
-          100
-        ],
-        "direction": "lower"
-      },
-      "0.01": {
-        "epsilon": 0.0,
-        "num_guesses": 0,
-        "correct_guesses": 0,
-        "candidate_num_guesses": [
-          5,
-          10,
-          15,
-          20,
-          25,
-          30,
-          35,
-          40,
-          45,
-          50,
-          55,
-          60,
-          65,
-          70,
-          75,
-          80,
-          85,
-          90,
-          95,
-          100
-        ],
-        "direction": "lower"
-      }
-    }
-  },
-  "embedding": {
-    "auc": 0.505688,
-    "empirical_epsilon": {
-      "0.05": 0.0,
-      "0.01": 0.0
-    },
-    "empirical_epsilon_details": {
-      "0.05": {
-        "epsilon": 0.0,
-        "num_guesses": 0,
-        "correct_guesses": 0,
-        "candidate_num_guesses": [
-          5,
-          10,
-          15,
-          20,
-          25,
-          30,
-          35,
-          40,
-          45,
-          50,
-          55,
-          60,
-          65,
-          70,
-          75,
-          80,
-          85,
-          90,
-          95,
-          100
-        ],
-        "direction": "lower"
-      },
-      "0.01": {
-        "epsilon": 0.0,
-        "num_guesses": 0,
-        "correct_guesses": 0,
-        "candidate_num_guesses": [
-          5,
-          10,
-          15,
-          20,
-          25,
-          30,
-          35,
-          40,
-          45,
-          50,
-          55,
-          60,
-          65,
-          70,
-          75,
-          80,
-          85,
-          90,
-          95,
-          100
-        ],
-        "direction": "lower"
-      }
-    }
-  }
-}
\ No newline at end of file
diff --git a/llama3-8b/dp8_v2/epochs/epoch_008/adapter/README.md b/llama3-8b/dp8_v2/epochs/epoch_008/adapter/README.md
deleted file mode 100644
index d99a36f688caa7f483f250cdbf16c2feb9635fef..0000000000000000000000000000000000000000
--- a/llama3-8b/dp8_v2/epochs/epoch_008/adapter/README.md
+++ /dev/null
@@ -1,207 +0,0 @@
----
-base_model: meta-llama/Meta-Llama-3-8B
-library_name: peft
-pipeline_tag: text-generation
-tags:
-- base_model:adapter:meta-llama/Meta-Llama-3-8B
-- lora
-- transformers
----
-
-# Model Card for Model ID
-
-<!-- Provide a quick summary of what the model is/does. -->
-
-
-
-## Model Details
-
-### Model Description
-
-<!-- Provide a longer summary of what this model is. -->
-
-
-
-- **Developed by:** [More Information Needed]
-- **Funded by [optional]:** [More Information Needed]
-- **Shared by [optional]:** [More Information Needed]
-- **Model type:** [More Information Needed]
-- **Language(s) (NLP):** [More Information Needed]
-- **License:** [More Information Needed]
-- **Finetuned from model [optional]:** [More Information Needed]
-
-### Model Sources [optional]
-
-<!-- Provide the basic links for the model. -->
-
-- **Repository:** [More Information Needed]
-- **Paper [optional]:** [More Information Needed]
-- **Demo [optional]:** [More Information Needed]
-
-## Uses
-
-<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
-
-### Direct Use
-
-<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
-
-[More Information Needed]
-
-### Downstream Use [optional]
-
-<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
-
-[More Information Needed]
-
-### Out-of-Scope Use
-
-<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
-
-[More Information Needed]
-
-## Bias, Risks, and Limitations
-
-<!-- This section is meant to convey both technical and sociotechnical limitations. -->
-
-[More Information Needed]
-
-### Recommendations
-
-<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
-
-Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
-
-## How to Get Started with the Model
-
-Use the code below to get started with the model.
-
-[More Information Needed]
-
-## Training Details
-
-### Training Data
-
-<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
-
-[More Information Needed]
-
-### Training Procedure
-
-<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
-
-#### Preprocessing [optional]
-
-[More Information Needed]
-
-
-#### Training Hyperparameters
-
-- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
-
-#### Speeds, Sizes, Times [optional]
-
-<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
-
-[More Information Needed]
-
-## Evaluation
-
-<!-- This section describes the evaluation protocols and provides the results. -->
-
-### Testing Data, Factors & Metrics
-
-#### Testing Data
-
-<!-- This should link to a Dataset Card if possible. -->
-
-[More Information Needed]
-
-#### Factors
-
-<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
-
-[More Information Needed]
-
-#### Metrics
-
-<!-- These are the evaluation metrics being used, ideally with a description of why. -->
-
-[More Information Needed]
-
-### Results
-
-[More Information Needed]
-
-#### Summary
-
-
-
-## Model Examination [optional]
-
-<!-- Relevant interpretability work for the model goes here -->
-
-[More Information Needed]
-
-## Environmental Impact
-
-<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
-
-Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
-
-- **Hardware Type:** [More Information Needed]
-- **Hours used:** [More Information Needed]
-- **Cloud Provider:** [More Information Needed]
-- **Compute Region:** [More Information Needed]
-- **Carbon Emitted:** [More Information Needed]
-
-## Technical Specifications [optional]
-
-### Model Architecture and Objective
-
-[More Information Needed]
-
-### Compute Infrastructure
-
-[More Information Needed]
-
-#### Hardware
-
-[More Information Needed]
-
-#### Software
-
-[More Information Needed]
-
-## Citation [optional]
-
-<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
-
-**BibTeX:**
-
-[More Information Needed]
-
-**APA:**
-
-[More Information Needed]
-
-## Glossary [optional]
-
-<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
-
-[More Information Needed]
-
-## More Information [optional]
-
-[More Information Needed]
-
-## Model Card Authors [optional]
-
-[More Information Needed]
-
-## Model Card Contact
-
-[More Information Needed]
-### Framework versions
-
-- PEFT 0.18.1
\ No newline at end of file
diff --git a/llama3-8b/dp8_v2/epochs/epoch_008/adapter/adapter_config.json b/llama3-8b/dp8_v2/epochs/epoch_008/adapter/adapter_config.json
deleted file mode 100644
index 0327c3c5d5310b6725dd50830481112b6dd34980..0000000000000000000000000000000000000000
--- a/llama3-8b/dp8_v2/epochs/epoch_008/adapter/adapter_config.json
+++ /dev/null
@@ -1,46 +0,0 @@
-{
-  "alora_invocation_tokens": null,
-  "alpha_pattern": {},
-  "arrow_config": null,
-  "auto_mapping": null,
-  "base_model_name_or_path": "meta-llama/Meta-Llama-3-8B",
-  "bias": "none",
-  "corda_config": null,
-  "ensure_weight_tying": true,
-  "eva_config": null,
-  "exclude_modules": null,
-  "fan_in_fan_out": false,
-  "inference_mode": true,
-  "init_lora_weights": true,
-  "layer_replication": null,
-  "layers_pattern": null,
-  "layers_to_transform": null,
-  "loftq_config": {},
-  "lora_alpha": 32,
-  "lora_bias": false,
-  "lora_dropout": 0.05,
-  "megatron_config": null,
-  "megatron_core": "megatron.core",
-  "modules_to_save": [
-    "lm_head",
-    "embed_tokens"
-  ],
-  "peft_type": "LORA",
-  "peft_version": "0.18.1",
-  "qalora_group_size": 16,
-  "r": 16,
-  "rank_pattern": {},
-  "revision": null,
-  "target_modules": [
-    "k_proj",
-    "q_proj",
-    "o_proj",
-    "v_proj"
-  ],
-  "target_parameters": null,
-  "task_type": "CAUSAL_LM",
-  "trainable_token_indices": null,
-  "use_dora": false,
-  "use_qalora": false,
-  "use_rslora": false
-}
\ No newline at end of file
diff --git a/llama3-8b/dp8_v2/epochs/epoch_008/audit_results.json b/llama3-8b/dp8_v2/epochs/epoch_008/audit_results.json
deleted file mode 100644
index d61b83b1c70dba23f64e7bf30cfffae117e92b8a..0000000000000000000000000000000000000000
--- a/llama3-8b/dp8_v2/epochs/epoch_008/audit_results.json
+++ /dev/null
@@ -1,137 +0,0 @@
-{
-  "delta": 1e-05,
-  "num_canaries": 500,
-  "num_members": 250,
-  "paper_guess_fraction": 0.2,
-  "paper_guess_steps": 20,
-  "loss": {
-    "auc": 0.540616,
-    "empirical_epsilon": {
-      "0.05": 0.0,
-      "0.01": 0.0
-    },
-    "empirical_epsilon_details": {
-      "0.05": {
-        "epsilon": 0.0,
-        "num_guesses": 0,
-        "correct_guesses": 0,
-        "candidate_num_guesses": [
-          5,
-          10,
-          15,
-          20,
-          25,
-          30,
-          35,
-          40,
-          45,
-          50,
-          55,
-          60,
-          65,
-          70,
-          75,
-          80,
-          85,
-          90,
-          95,
-          100
-        ],
-        "direction": "lower"
-      },
-      "0.01": {
-        "epsilon": 0.0,
-        "num_guesses": 0,
-        "correct_guesses": 0,
-        "candidate_num_guesses": [
-          5,
-          10,
-          15,
-          20,
-          25,
-          30,
-          35,
-          40,
-          45,
-          50,
-          55,
-          60,
-          65,
-          70,
-          75,
-          80,
-          85,
-          90,
-          95,
-          100
-        ],
-        "direction": "lower"
-      }
-    }
-  },
-  "embedding": {
-    "auc": 0.506472,
-    "empirical_epsilon": {
-      "0.05": 0.0,
-      "0.01": 0.0
-    },
-    "empirical_epsilon_details": {
-      "0.05": {
-        "epsilon": 0.0,
-        "num_guesses": 0,
-        "correct_guesses": 0,
-        "candidate_num_guesses": [
-          5,
-          10,
-          15,
-          20,
-          25,
-          30,
-          35,
-          40,
-          45,
-          50,
-          55,
-          60,
-          65,
-          70,
-          75,
-          80,
-          85,
-          90,
-          95,
-          100
-        ],
-        "direction": "lower"
-      },
-      "0.01": {
-        "epsilon": 0.0,
-        "num_guesses": 0,
-        "correct_guesses": 0,
-        "candidate_num_guesses": [
-          5,
-          10,
-          15,
-          20,
-          25,
-          30,
-          35,
-          40,
-          45,
-          50,
-          55,
-          60,
-          65,
-          70,
-          75,
-          80,
-          85,
-          90,
-          95,
-          100
-        ],
-        "direction": "lower"
-      }
-    }
-  }
-}
\ No newline at end of file
diff --git a/llama3.1-8b/dp3/epochs/epoch_001/adapter/README.md b/llama3.1-8b/dp3/epochs/epoch_001/adapter/README.md
deleted file mode 100644
index 2206b79a428a68b43cf68e2323a8aa92987452d5..0000000000000000000000000000000000000000
--- a/llama3.1-8b/dp3/epochs/epoch_001/adapter/README.md
+++ /dev/null
@@ -1,207 +0,0 @@
----
-base_model: meta-llama/Llama-3.1-8B
-library_name: peft
-pipeline_tag: text-generation
-tags:
-- base_model:adapter:meta-llama/Llama-3.1-8B
-- lora
-- transformers
----
-
-# Model Card for Model ID
-
-<!-- Provide a quick summary of what the model is/does. -->
-
-
-
-## Model Details
-
-### Model Description
-
-<!-- Provide a longer summary of what this model is. -->
-
-
-
-- **Developed by:** [More Information Needed]
-- **Funded by [optional]:** [More Information Needed]
-- **Shared by [optional]:** [More Information Needed]
-- **Model type:** [More Information Needed]
-- **Language(s) (NLP):** [More Information Needed]
-- **License:** [More Information Needed]
-- **Finetuned from model [optional]:** [More Information Needed]
-
-### Model Sources [optional]
-
-<!-- Provide the basic links for the model. -->
-
-- **Repository:** [More Information Needed]
-- **Paper [optional]:** [More Information Needed]
-- **Demo [optional]:** [More Information Needed]
-
-## Uses
-
-<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
-
-### Direct Use
-
-<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
-
-[More Information Needed]
-
-### Downstream Use [optional]
-
-<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
-
-[More Information Needed]
-
-### Out-of-Scope Use
-
-<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
-
-[More Information Needed]
-
-## Bias, Risks, and Limitations
-
-<!-- This section is meant to convey both technical and sociotechnical limitations. -->
-
-[More Information Needed]
-
-### Recommendations
-
-<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
-
-Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
-
-## How to Get Started with the Model
-
-Use the code below to get started with the model.
-
-[More Information Needed]
-
-## Training Details
-
-### Training Data
-
-<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
-
-[More Information Needed]
-
-### Training Procedure
-
-<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
-
-#### Preprocessing [optional]
-
-[More Information Needed]
-
-
-#### Training Hyperparameters
-
-- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
-
-#### Speeds, Sizes, Times [optional]
-
-<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
-
-[More Information Needed]
-
-## Evaluation
-
-<!-- This section describes the evaluation protocols and provides the results. -->
-
-### Testing Data, Factors & Metrics
-
-#### Testing Data
-
-<!-- This should link to a Dataset Card if possible. -->
-
-[More Information Needed]
-
-#### Factors
-
-<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
-
-[More Information Needed]
-
-#### Metrics
-
-<!-- These are the evaluation metrics being used, ideally with a description of why. -->
-
-[More Information Needed]
-
-### Results
-
-[More Information Needed]
-
-#### Summary
-
-
-
-## Model Examination [optional]
-
-<!-- Relevant interpretability work for the model goes here -->
-
-[More Information Needed]
-
-## Environmental Impact
-
-<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
-
-Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
-
-- **Hardware Type:** [More Information Needed]
-- **Hours used:** [More Information Needed]
-- **Cloud Provider:** [More Information Needed]
-- **Compute Region:** [More Information Needed]
-- **Carbon Emitted:** [More Information Needed]
-
-## Technical Specifications [optional]
-
-### Model Architecture and Objective
-
-[More Information Needed]
-
-### Compute Infrastructure
-
-[More Information Needed]
-
-#### Hardware
-
-[More Information Needed]
-
-#### Software
-
-[More Information Needed]
-
-## Citation [optional]
-
-<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
-
-**BibTeX:**
-
-[More Information Needed]
-
-**APA:**
-
-[More Information Needed]
-
-## Glossary [optional]
-
-<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
-
-[More Information Needed]
-
-## More Information [optional]
-
-[More Information Needed]
-
-## Model Card Authors [optional]
-
-[More Information Needed]
-
-## Model Card Contact
-
-[More Information Needed]
-### Framework versions
-
-- PEFT 0.18.1
\ No newline at end of file
diff --git a/llama3.1-8b/dp3/epochs/epoch_001/adapter/adapter_config.json b/llama3.1-8b/dp3/epochs/epoch_001/adapter/adapter_config.json
deleted file mode 100644
index 96bd552cbece6fa5ad82ab449ff727b4ceaae15a..0000000000000000000000000000000000000000
--- a/llama3.1-8b/dp3/epochs/epoch_001/adapter/adapter_config.json
+++ /dev/null
@@ -1,46 +0,0 @@
-{
-  "alora_invocation_tokens": null,
-  "alpha_pattern": {},
-  "arrow_config": null,
-  "auto_mapping": null,
-  "base_model_name_or_path": "meta-llama/Llama-3.1-8B",
-  "bias": "none",
-  "corda_config": null,
-  "ensure_weight_tying": true,
-  "eva_config": null,
-  "exclude_modules": null,
-  "fan_in_fan_out": false,
-  "inference_mode": true,
-  "init_lora_weights": true,
-  "layer_replication": null,
-  "layers_pattern": null,
-  "layers_to_transform": null,
-  "loftq_config": {},
-  "lora_alpha": 32,
-  "lora_bias": false,
-  "lora_dropout": 0.05,
-  "megatron_config": null,
-  "megatron_core": "megatron.core",
-  "modules_to_save": [
-    "lm_head",
-    "embed_tokens"
-  ],
-  "peft_type": "LORA",
-  "peft_version": "0.18.1",
-  "qalora_group_size": 16,
-  "r": 16,
-  "rank_pattern": {},
-  "revision": null,
-  "target_modules": [
-    "q_proj",
-    "k_proj",
-    "o_proj",
-    "v_proj"
-  ],
-  "target_parameters": null,
-  "task_type": "CAUSAL_LM",
-  "trainable_token_indices": null,
-  "use_dora": false,
-  "use_qalora": false,
-  "use_rslora": false
-}
\ No newline at end of file
diff --git a/llama3.1-8b/dp3/epochs/epoch_001/audit_results.json b/llama3.1-8b/dp3/epochs/epoch_001/audit_results.json
deleted file mode 100644
index b438a736c405eebbb913ca7a1996d3892b7b7ef8..0000000000000000000000000000000000000000
--- a/llama3.1-8b/dp3/epochs/epoch_001/audit_results.json
+++ /dev/null
@@ -1,137 +0,0 @@
-{
-  "delta": 1e-05,
-  "num_canaries": 500,
-  "num_members": 250,
-  "paper_guess_fraction": 0.2,
-  "paper_guess_steps": 20,
-  "loss": {
-    "auc": 0.53272,
-    "empirical_epsilon": {
-      "0.05": 0.019017613492906094,
-      "0.01": 0.0
-    },
-    "empirical_epsilon_details": {
-      "0.05": {
-        "epsilon": 0.019017613492906094,
-        "num_guesses": 85,
-        "correct_guesses": 51,
-        "candidate_num_guesses": [
-          5,
-          10,
-          15,
-          20,
-          25,
-          30,
-          35,
-          40,
-          45,
-          50,
-          55,
-          60,
-          65,
-          70,
-          75,
-          80,
-          85,
-          90,
-          95,
-          100
-        ],
-        "direction": "higher"
-      },
-      "0.01": {
-        "epsilon": 0.0,
-        "num_guesses": 0,
-        "correct_guesses": 0,
-        "candidate_num_guesses": [
-          5,
-          10,
-          15,
-          20,
-          25,
-          30,
-          35,
-          40,
-          45,
-          50,
-          55,
-          60,
-          65,
-          70,
-          75,
-          80,
-          85,
-          90,
-          95,
-          100
-        ],
-        "direction": "lower"
-      }
-    }
-  },
-  "embedding": {
-    "auc": 0.503592,
-    "empirical_epsilon": {
-      "0.05": 0.0,
-      "0.01": 0.0
-    },
-    "empirical_epsilon_details": {
-      "0.05": {
-        "epsilon": 0.0,
-        "num_guesses": 0,
-        "correct_guesses": 0,
-        "candidate_num_guesses": [
-          5,
-          10,
-          15,
-          20,
-          25,
-          30,
-          35,
-          40,
-          45,
-          50,
-          55,
-          60,
-          65,
-          70,
-          75,
-          80,
-          85,
-          90,
-          95,
-          100
-        ],
-        "direction": "lower"
-      },
-      "0.01": {
-        "epsilon": 0.0,
-        "num_guesses": 0,
-        "correct_guesses": 0,
-        "candidate_num_guesses": [
-          5,
-          10,
-          15,
-          20,
-          25,
-          30,
-          35,
-          40,
-          45,
-          50,
-          55,
-          60,
-          65,
-          70,
-          75,
-          80,
-          85,
-          90,
-          95,
-          100
-        ],
-        "direction": "lower"
-      }
-    }
-  }
-}
\ No newline at end of file
diff --git a/llama3.1-8b/dp3/epochs/epoch_002/adapter/README.md b/llama3.1-8b/dp3/epochs/epoch_002/adapter/README.md
deleted file mode 100644
index 2206b79a428a68b43cf68e2323a8aa92987452d5..0000000000000000000000000000000000000000
--- a/llama3.1-8b/dp3/epochs/epoch_002/adapter/README.md
+++ /dev/null
@@ -1,207 +0,0 @@
----
-base_model: meta-llama/Llama-3.1-8B
-library_name: peft
-pipeline_tag: text-generation
-tags:
-- base_model:adapter:meta-llama/Llama-3.1-8B
-- lora
-- transformers
----
-
-# Model Card for Model ID
-
-<!-- Provide a quick summary of what the model is/does. -->
-
-
-
-## Model Details
-
-### Model Description
-
-<!-- Provide a longer summary of what this model is. -->
-
-
-
-- **Developed by:** [More Information Needed]
-- **Funded by [optional]:** [More Information Needed]
-- **Shared by [optional]:** [More Information Needed]
-- **Model type:** [More Information Needed]
-- **Language(s) (NLP):** [More Information Needed]
-- **License:** [More Information Needed]
-- **Finetuned from model [optional]:** [More Information Needed]
-
-### Model Sources [optional]
-
-<!-- Provide the basic links for the model. -->
-
-- **Repository:** [More Information Needed]
-- **Paper [optional]:** [More Information Needed]
-- **Demo [optional]:** [More Information Needed]
-
-## Uses
-
-<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
-
-### Direct Use
-
-<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
-
-[More Information Needed]
-
-### Downstream Use [optional]
-
-<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
-
-[More Information Needed]
-
-### Out-of-Scope Use
-
-<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
-
-[More Information Needed]
-
-## Bias, Risks, and Limitations
-
-<!-- This section is meant to convey both technical and sociotechnical limitations. -->
-
-[More Information Needed]
-
-### Recommendations
-
-<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
-
-Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
-
-## How to Get Started with the Model
-
-Use the code below to get started with the model.
-
-[More Information Needed]
-
-## Training Details
-
-### Training Data
-
-<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
-
-[More Information Needed]
-
-### Training Procedure
-
-<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
-
-#### Preprocessing [optional]
-
-[More Information Needed]
-
-
-#### Training Hyperparameters
-
-- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
-
-#### Speeds, Sizes, Times [optional]
-
-<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
-
-[More Information Needed]
-
-## Evaluation
-
-<!-- This section describes the evaluation protocols and provides the results. -->
-
-### Testing Data, Factors & Metrics
-
-#### Testing Data
-
-<!-- This should link to a Dataset Card if possible. -->
-
-[More Information Needed]
-
-#### Factors
-
-<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
-
-[More Information Needed]
-
-#### Metrics
-
-<!-- These are the evaluation metrics being used, ideally with a description of why. -->
-
-[More Information Needed]
-
-### Results
-
-[More Information Needed]
-
-#### Summary
-
-
-
-## Model Examination [optional]
-
-<!-- Relevant interpretability work for the model goes here -->
-
-[More Information Needed]
-
-## Environmental Impact
-
-<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
-
-Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
-
-- **Hardware Type:** [More Information Needed]
-- **Hours used:** [More Information Needed]
-- **Cloud Provider:** [More Information Needed]
-- **Compute Region:** [More Information Needed]
-- **Carbon Emitted:** [More Information Needed]
-
-## Technical Specifications [optional]
-
-### Model Architecture and Objective
-
-[More Information Needed]
-
-### Compute Infrastructure
-
-[More Information Needed]
-
-#### Hardware
-
-[More Information Needed]
-
-#### Software
-
-[More Information Needed]
-
-## Citation [optional]
-
-<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
-
-**BibTeX:**
-
-[More Information Needed]
-
-**APA:**
-
-[More Information Needed]
-
-## Glossary [optional]
-
-<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
-
-[More Information Needed]
-
-## More Information [optional]
-
-[More Information Needed]
-
-## Model Card Authors [optional]
-
-[More Information Needed]
-
-## Model Card Contact
-
-[More Information Needed]
-### Framework versions
-
-- PEFT 0.18.1
\ No newline at end of file
diff --git a/llama3.1-8b/dp3/epochs/epoch_002/adapter/adapter_config.json b/llama3.1-8b/dp3/epochs/epoch_002/adapter/adapter_config.json
deleted file mode 100644
index 96bd552cbece6fa5ad82ab449ff727b4ceaae15a..0000000000000000000000000000000000000000
--- a/llama3.1-8b/dp3/epochs/epoch_002/adapter/adapter_config.json
+++ /dev/null
@@ -1,46 +0,0 @@
-{
-  "alora_invocation_tokens": null,
-  "alpha_pattern": {},
-  "arrow_config": null,
-  "auto_mapping": null,
-  "base_model_name_or_path": "meta-llama/Llama-3.1-8B",
-  "bias": "none",
-  "corda_config": null,
-  "ensure_weight_tying": true,
-  "eva_config": null,
-  "exclude_modules": null,
-  "fan_in_fan_out": false,
-  "inference_mode": true,
-  "init_lora_weights": true,
-  "layer_replication": null,
-  "layers_pattern": null,
-  "layers_to_transform": null,
-  "loftq_config": {},
-  "lora_alpha": 32,
-  "lora_bias": false,
-  "lora_dropout": 0.05,
-  "megatron_config": null,
-  "megatron_core": "megatron.core",
-  "modules_to_save": [
-    "lm_head",
-    "embed_tokens"
-  ],
-  "peft_type": "LORA",
-  "peft_version": "0.18.1",
-  "qalora_group_size": 16,
-  "r": 16,
-  "rank_pattern": {},
-  "revision": null,
-  "target_modules": [
-    "q_proj",
-    "k_proj",
-    "o_proj",
-    "v_proj"
-  ],
-  "target_parameters": null,
-  "task_type": "CAUSAL_LM",
-  "trainable_token_indices": null,
-  "use_dora": false,
-  "use_qalora": false,
-  "use_rslora": false
-}
\ No newline at end of file
diff --git a/llama3.1-8b/dp3/epochs/epoch_002/audit_results.json b/llama3.1-8b/dp3/epochs/epoch_002/audit_results.json
deleted file mode 100644
index 224a2a3e0e9e8f9a664383d92bcae7908a7c4f81..0000000000000000000000000000000000000000
--- a/llama3.1-8b/dp3/epochs/epoch_002/audit_results.json
+++ /dev/null
@@ -1,137 +0,0 @@
-{
-  "delta": 1e-05,
-  "num_canaries": 500,
-  "num_members": 250,
-  "paper_guess_fraction": 0.2,
-  "paper_guess_steps": 20,
-  "loss": {
-    "auc": 0.508152,
-    "empirical_epsilon": {
-      "0.05": 0.0,
-      "0.01": 0.0
-    },
-    "empirical_epsilon_details": {
-      "0.05": {
-        "epsilon": 0.0,
-        "num_guesses": 0,
-        "correct_guesses": 0,
-        "candidate_num_guesses": [
-          5,
-          10,
-          15,
-          20,
-          25,
-          30,
-          35,
-          40,
-          45,
-          50,
-          55,
-          60,
-          65,
-          70,
-          75,
-          80,
-          85,
-          90,
-          95,
-          100
-        ],
-        "direction": "lower"
-      },
-      "0.01": {
-        "epsilon": 0.0,
-        "num_guesses": 0,
-        "correct_guesses": 0,
-        "candidate_num_guesses": [
-          5,
-          10,
-          15,
-          20,
-          25,
-          30,
-          35,
-          40,
-          45,
-          50,
-          55,
-          60,
-          65,
-          70,
-          75,
-          80,
-          85,
-          90,
-          95,
-          100
-        ],
-        "direction": "lower"
-      }
-    }
-  },
-  "embedding": {
-    "auc": 0.518088,
-    "empirical_epsilon": {
-      "0.05": 0.0,
-      "0.01": 0.0
-    },
-    "empirical_epsilon_details": {
-      "0.05": {
-        "epsilon": 0.0,
-        "num_guesses": 0,
-        "correct_guesses": 0,
-        "candidate_num_guesses": [
-          5,
-          10,
-          15,
-          20,
-          25,
-          30,
-          35,
-          40,
-          45,
-          50,
-          55,
-          60,
-          65,
-          70,
-          75,
-          80,
-          85,
-          90,
-          95,
-          100
-        ],
-        "direction": "lower"
-      },
-      "0.01": {
-        "epsilon": 0.0,
-        "num_guesses": 0,
-        "correct_guesses": 0,
-        "candidate_num_guesses": [
-          5,
-          10,
-          15,
-          20,
-          25,
-          30,
-          35,
-          40,
-          45,
-          50,
-          55,
-          60,
-          65,
-          70,
-          75,
-          80,
-          85,
-          90,
-          95,
-          100
-        ],
-        "direction": "lower"
-      }
-    }
-  }
-}
\ No newline at end of file
diff --git a/llama3.1-8b/dp3/epochs/epoch_003/adapter/README.md b/llama3.1-8b/dp3/epochs/epoch_003/adapter/README.md
deleted file mode 100644
index 2206b79a428a68b43cf68e2323a8aa92987452d5..0000000000000000000000000000000000000000
--- a/llama3.1-8b/dp3/epochs/epoch_003/adapter/README.md
+++ /dev/null
@@ -1,207 +0,0 @@
----
-base_model: meta-llama/Llama-3.1-8B
-library_name: peft
-pipeline_tag: text-generation
-tags:
-- base_model:adapter:meta-llama/Llama-3.1-8B
-- lora
-- transformers
----
-
-# Model Card for Model ID
-
-<!-- Provide a quick summary of what the model is/does. -->
-
-
-
-## Model Details
-
-### Model Description
-
-<!-- Provide a longer summary of what this model is. -->
-
-
-
-- **Developed by:** [More Information Needed]
-- **Funded by [optional]:** [More Information Needed]
-- **Shared by [optional]:** [More Information Needed]
-- **Model type:** [More Information Needed]
-- **Language(s) (NLP):** [More Information Needed]
-- **License:** [More Information Needed]
-- **Finetuned from model [optional]:** [More Information Needed]
-
-### Model Sources [optional]
-
-<!-- Provide the basic links for the model. -->
-
-- **Repository:** [More Information Needed]
-- **Paper [optional]:** [More Information Needed]
-- **Demo [optional]:** [More Information Needed]
-
-## Uses
-
-<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
-
-### Direct Use
-
-<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
-
-[More Information Needed]
-
-### Downstream Use [optional]
-
-<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
-
-[More Information Needed]
-
-### Out-of-Scope Use
-
-<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
-
-[More Information Needed]
-
-## Bias, Risks, and Limitations
-
-<!-- This section is meant to convey both technical and sociotechnical limitations. -->
-
-[More Information Needed]
-
-### Recommendations
-
-<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
-
-Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
-
-## How to Get Started with the Model
-
-Use the code below to get started with the model.
-
-[More Information Needed]
-
-## Training Details
-
-### Training Data
-
-<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
-
-[More Information Needed]
-
-### Training Procedure
-
-<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
-
-#### Preprocessing [optional]
-
-[More Information Needed]
-
-
-#### Training Hyperparameters
-
-- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
-
-#### Speeds, Sizes, Times [optional]
-
-<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
-
-[More Information Needed]
-
-## Evaluation
-
-<!-- This section describes the evaluation protocols and provides the results. -->
-
-### Testing Data, Factors & Metrics
-
-#### Testing Data
-
-<!-- This should link to a Dataset Card if possible. -->
-
-[More Information Needed]
-
-#### Factors
-
-<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
-
-[More Information Needed]
-
-#### Metrics
-
-<!-- These are the evaluation metrics being used, ideally with a description of why. -->
-
-[More Information Needed]
-
-### Results
-
-[More Information Needed]
-
-#### Summary
-
-
-
-## Model Examination [optional]
-
-<!-- Relevant interpretability work for the model goes here -->
-
-[More Information Needed]
-
-## Environmental Impact
-
-<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
-
-Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
-
-- **Hardware Type:** [More Information Needed]
-- **Hours used:** [More Information Needed]
-- **Cloud Provider:** [More Information Needed]
-- **Compute Region:** [More Information Needed]
-- **Carbon Emitted:** [More Information Needed]
-
-## Technical Specifications [optional]
-
-### Model Architecture and Objective
-
-[More Information Needed]
-
-### Compute Infrastructure
-
-[More Information Needed]
-
-#### Hardware
-
-[More Information Needed]
-
-#### Software
-
-[More Information Needed]
-
-## Citation [optional]
-
-<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
-
-**BibTeX:**
-
-[More Information Needed]
-
-**APA:**
-
-[More Information Needed]
-
-## Glossary [optional]
-
-<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
-
-[More Information Needed]
-
-## More Information [optional]
-
-[More Information Needed]
-
-## Model Card Authors [optional]
-
-[More Information Needed]
-
-## Model Card Contact
-
-[More Information Needed]
-### Framework versions
-
-- PEFT 0.18.1
\ No newline at end of file
diff --git a/llama3.1-8b/dp3/epochs/epoch_003/adapter/adapter_config.json b/llama3.1-8b/dp3/epochs/epoch_003/adapter/adapter_config.json
deleted file mode 100644
index 96bd552cbece6fa5ad82ab449ff727b4ceaae15a..0000000000000000000000000000000000000000
--- a/llama3.1-8b/dp3/epochs/epoch_003/adapter/adapter_config.json
+++ /dev/null
@@ -1,46 +0,0 @@
-{
-  "alora_invocation_tokens": null,
-  "alpha_pattern": {},
-  "arrow_config": null,
-  "auto_mapping": null,
-  "base_model_name_or_path": "meta-llama/Llama-3.1-8B",
-  "bias": "none",
-  "corda_config": null,
-  "ensure_weight_tying": true,
-  "eva_config": null,
-  "exclude_modules": null,
-  "fan_in_fan_out": false,
-  "inference_mode": true,
-  "init_lora_weights": true,
-  "layer_replication": null,
-  "layers_pattern": null,
-  "layers_to_transform": null,
-  "loftq_config": {},
-  "lora_alpha": 32,
-  "lora_bias": false,
-  "lora_dropout": 0.05,
-  "megatron_config": null,
-  "megatron_core": "megatron.core",
-  "modules_to_save": [
-    "lm_head",
-    "embed_tokens"
-  ],
-  "peft_type": "LORA",
-  "peft_version": "0.18.1",
-  "qalora_group_size": 16,
-  "r": 16,
-  "rank_pattern": {},
-  "revision": null,
-  "target_modules": [
-    "q_proj",
-    "k_proj",
-    "o_proj",
-    "v_proj"
-  ],
-  "target_parameters": null,
-  "task_type": "CAUSAL_LM",
-  "trainable_token_indices": null,
-  "use_dora": false,
-  "use_qalora": false,
-  "use_rslora": false
-}
\ No newline at end of file
diff --git a/llama3.1-8b/dp3/epochs/epoch_003/audit_results.json b/llama3.1-8b/dp3/epochs/epoch_003/audit_results.json
deleted file mode 100644
index 759a6dce0472ee3d36db89b5e6c64898b2a67a8d..0000000000000000000000000000000000000000
--- a/llama3.1-8b/dp3/epochs/epoch_003/audit_results.json
+++ /dev/null
@@ -1,137 +0,0 @@
-{
-  "delta": 1e-05,
-  "num_canaries": 500,
-  "num_members": 250,
-  "paper_guess_fraction": 0.2,
-  "paper_guess_steps": 20,
-  "loss": {
-    "auc": 0.500184,
-    "empirical_epsilon": {
-      "0.05": 0.0,
-      "0.01": 0.0
-    },
-    "empirical_epsilon_details": {
-      "0.05": {
-        "epsilon": 0.0,
-        "num_guesses": 0,
-        "correct_guesses": 0,
-        "candidate_num_guesses": [
-          5,
-          10,
-          15,
-          20,
-          25,
-          30,
-          35,
-          40,
-          45,
-          50,
-          55,
-          60,
-          65,
-          70,
-          75,
-          80,
-          85,
-          90,
-          95,
-          100
-        ],
-        "direction": "lower"
-      },
-      "0.01": {
-        "epsilon": 0.0,
-        "num_guesses": 0,
-        "correct_guesses": 0,
-        "candidate_num_guesses": [
-          5,
-          10,
-          15,
-          20,
-          25,
-          30,
-          35,
-          40,
-          45,
-          50,
-          55,
-          60,
-          65,
-          70,
-          75,
-          80,
-          85,
-          90,
-          95,
-          100
-        ],
-        "direction": "lower"
-      }
-    }
-  },
-  "embedding": {
-    "auc": 0.500432,
-    "empirical_epsilon": {
-      "0.05": 0.0,
-      "0.01": 0.0
-    },
-    "empirical_epsilon_details": {
-      "0.05": {
-        "epsilon": 0.0,
-        "num_guesses": 0,
-        "correct_guesses": 0,
-        "candidate_num_guesses": [
-          5,
-          10,
-          15,
-          20,
-          25,
-          30,
-          35,
-          40,
-          45,
-          50,
-          55,
-          60,
-          65,
-          70,
-          75,
-          80,
-          85,
-          90,
-          95,
-          100
-        ],
-        "direction": "lower"
-      },
-      "0.01": {
-        "epsilon": 0.0,
-        "num_guesses": 0,
-        "correct_guesses": 0,
-        "candidate_num_guesses": [
-          5,
-          10,
-          15,
-          20,
-          25,
-          30,
-          35,
-          40,
-          45,
-          50,
-          55,
-          60,
-          65,
-          70,
-          75,
-          80,
-          85,
-          90,
-          95,
-          100
-        ],
-        "direction": "lower"
-      }
-    }
-  }
-}
\ No newline at end of file
diff --git a/llama3.1-8b/dp3/epochs/epoch_004/adapter/README.md b/llama3.1-8b/dp3/epochs/epoch_004/adapter/README.md
deleted file mode 100644
index 2206b79a428a68b43cf68e2323a8aa92987452d5..0000000000000000000000000000000000000000
--- a/llama3.1-8b/dp3/epochs/epoch_004/adapter/README.md
+++ /dev/null
@@ -1,207 +0,0 @@
----
-base_model: meta-llama/Llama-3.1-8B
-library_name: peft
-pipeline_tag: text-generation
-tags:
-- base_model:adapter:meta-llama/Llama-3.1-8B
-- lora
-- transformers
----
-
-# Model Card for Model ID
-
-<!-- Provide a quick summary of what the model is/does. -->
-
-
-
-## Model Details
-
-### Model Description
-
-<!-- Provide a longer summary of what this model is. -->
-
-
-
-- **Developed by:** [More Information Needed]
-- **Funded by [optional]:** [More Information Needed]
-- **Shared by [optional]:** [More Information Needed]
-- **Model type:** [More Information Needed]
-- **Language(s) (NLP):** [More Information Needed]
-- **License:** [More Information Needed]
-- **Finetuned from model [optional]:** [More Information Needed]
-
-### Model Sources [optional]
-
-<!-- Provide the basic links for the model. -->
-
-- **Repository:** [More Information Needed]
-- **Paper [optional]:** [More Information Needed]
-- **Demo [optional]:** [More Information Needed]
-
-## Uses
-
-<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
-
-### Direct Use
-
-<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
-
-[More Information Needed]
-
-### Downstream Use [optional]
-
-<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
-
-[More Information Needed]
-
-### Out-of-Scope Use
-
-<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
-
-[More Information Needed]
-
-## Bias, Risks, and Limitations
-
-<!-- This section is meant to convey both technical and sociotechnical limitations. -->
-
-[More Information Needed]
-
-### Recommendations
-
-<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
-
-Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
-
-## How to Get Started with the Model
-
-Use the code below to get started with the model.
-
-[More Information Needed]
-
-## Training Details
-
-### Training Data
-
-<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
-
-[More Information Needed]
-
-### Training Procedure
-
-<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
-
-#### Preprocessing [optional]
-
-[More Information Needed]
-
-
-#### Training Hyperparameters
-
-- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
-
-#### Speeds, Sizes, Times [optional]
-
-<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
-
-[More Information Needed]
-
-## Evaluation
-
-<!-- This section describes the evaluation protocols and provides the results. -->
-
-### Testing Data, Factors & Metrics
-
-#### Testing Data
-
-<!-- This should link to a Dataset Card if possible. -->
-
-[More Information Needed]
-
-#### Factors
-
-<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
-
-[More Information Needed]
-
-#### Metrics
-
-<!-- These are the evaluation metrics being used, ideally with a description of why. -->
-
-[More Information Needed]
-
-### Results
-
-[More Information Needed]
-
-#### Summary
-
-
-
-## Model Examination [optional]
-
-<!-- Relevant interpretability work for the model goes here -->
-
-[More Information Needed]
-
-## Environmental Impact
-
-<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
-
-Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
-
-- **Hardware Type:** [More Information Needed]
-- **Hours used:** [More Information Needed]
-- **Cloud Provider:** [More Information Needed]
-- **Compute Region:** [More Information Needed]
-- **Carbon Emitted:** [More Information Needed]
-
-## Technical Specifications [optional]
-
-### Model Architecture and Objective
-
-[More Information Needed]
-
-### Compute Infrastructure
-
-[More Information Needed]
-
-#### Hardware
-
-[More Information Needed]
-
-#### Software
-
-[More Information Needed]
-
-## Citation [optional]
-
-<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
-
-**BibTeX:**
-
-[More Information Needed]
-
-**APA:**
-
-[More Information Needed]
-
-## Glossary [optional]
-
-<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
-
-[More Information Needed]
-
-## More Information [optional]
-
-[More Information Needed]
-
-## Model Card Authors [optional]
-
-[More Information Needed]
-
-## Model Card Contact
-
-[More Information Needed]
-### Framework versions
-
-- PEFT 0.18.1
\ No newline at end of file
diff --git a/llama3.1-8b/dp3/epochs/epoch_004/adapter/adapter_config.json b/llama3.1-8b/dp3/epochs/epoch_004/adapter/adapter_config.json
deleted file mode 100644
index 96bd552cbece6fa5ad82ab449ff727b4ceaae15a..0000000000000000000000000000000000000000
--- a/llama3.1-8b/dp3/epochs/epoch_004/adapter/adapter_config.json
+++ /dev/null
@@ -1,46 +0,0 @@
-{
-  "alora_invocation_tokens": null,
-  "alpha_pattern": {},
-  "arrow_config": null,
-  "auto_mapping": null,
-  "base_model_name_or_path": "meta-llama/Llama-3.1-8B",
-  "bias": "none",
-  "corda_config": null,
-  "ensure_weight_tying": true,
-  "eva_config": null,
-  "exclude_modules": null,
-  "fan_in_fan_out": false,
-  "inference_mode": true,
-  "init_lora_weights": true,
-  "layer_replication": null,
-  "layers_pattern": null,
-  "layers_to_transform": null,
-  "loftq_config": {},
-  "lora_alpha": 32,
-  "lora_bias": false,
-  "lora_dropout": 0.05,
-  "megatron_config": null,
-  "megatron_core": "megatron.core",
-  "modules_to_save": [
-    "lm_head",
-    "embed_tokens"
-  ],
-  "peft_type": "LORA",
-  "peft_version": "0.18.1",
-  "qalora_group_size": 16,
-  "r": 16,
-  "rank_pattern": {},
-  "revision": null,
-  "target_modules": [
-    "q_proj",
-    "k_proj",
-    "o_proj",
-    "v_proj"
-  ],
-  "target_parameters": null,
-  "task_type": "CAUSAL_LM",
-  "trainable_token_indices": null,
-  "use_dora": false,
-  "use_qalora": false,
-  "use_rslora": false
-}
\ No newline at end of file
diff --git a/llama3.1-8b/dp3/epochs/epoch_004/audit_results.json b/llama3.1-8b/dp3/epochs/epoch_004/audit_results.json
deleted file mode 100644
index 5a2e124abb592117dbe4e787dd4681b27b3fbbbc..0000000000000000000000000000000000000000
--- a/llama3.1-8b/dp3/epochs/epoch_004/audit_results.json
+++ /dev/null
@@ -1,137 +0,0 @@
-{
-  "delta": 1e-05,
-  "num_canaries": 500,
-  "num_members": 250,
-  "paper_guess_fraction": 0.2,
-  "paper_guess_steps": 20,
-  "loss": {
-    "auc": 0.501216,
-    "empirical_epsilon": {
-      "0.05": 0.0,
-      "0.01": 0.0
-    },
-    "empirical_epsilon_details": {
-      "0.05": {
-        "epsilon": 0.0,
-        "num_guesses": 0,
-        "correct_guesses": 0,
-        "candidate_num_guesses": [
-          5,
-          10,
-          15,
-          20,
-          25,
-          30,
-          35,
-          40,
-          45,
-          50,
-          55,
-          60,
-          65,
-          70,
-          75,
-          80,
-          85,
-          90,
-          95,
-          100
-        ],
-        "direction": "lower"
-      },
-      "0.01": {
-        "epsilon": 0.0,
-        "num_guesses": 0,
-        "correct_guesses": 0,
-        "candidate_num_guesses": [
-          5,
-          10,
-          15,
-          20,
-          25,
-          30,
-          35,
-          40,
-          45,
-          50,
-          55,
-          60,
-          65,
-          70,
-          75,
-          80,
-          85,
-          90,
-          95,
-          100
-        ],
-        "direction": "lower"
-      }
-    }
-  },
-  "embedding": {
-    "auc": 0.505104,
-    "empirical_epsilon": {
-      "0.05": 0.0,
-      "0.01": 0.0
-    },
-    "empirical_epsilon_details": {
-      "0.05": {
-        "epsilon": 0.0,
-        "num_guesses": 0,
-        "correct_guesses": 0,
-        "candidate_num_guesses": [
-          5,
-          10,
-          15,
-          20,
-          25,
-          30,
-          35,
-          40,
-          45,
-          50,
-          55,
-          60,
-          65,
-          70,
-          75,
-          80,
-          85,
-          90,
-          95,
-          100
-        ],
-        "direction": "lower"
-      },
-      "0.01": {
-        "epsilon": 0.0,
-        "num_guesses": 0,
-        "correct_guesses": 0,
-        "candidate_num_guesses": [
-          5,
-          10,
-          15,
-          20,
-          25,
-          30,
-          35,
-          40,
-          45,
-          50,
-          55,
-          60,
-          65,
-          70,
-          75,
-          80,
-          85,
-          90,
-          95,
-          100
-        ],
-        "direction": "lower"
-      }
-    }
-  }
-}
\ No newline at end of file
diff --git a/llama3.1-8b/dp3/epochs/epoch_005/adapter/README.md b/llama3.1-8b/dp3/epochs/epoch_005/adapter/README.md
deleted file mode 100644
index 2206b79a428a68b43cf68e2323a8aa92987452d5..0000000000000000000000000000000000000000
--- a/llama3.1-8b/dp3/epochs/epoch_005/adapter/README.md
+++ /dev/null
@@ -1,207 +0,0 @@
----
-base_model: meta-llama/Llama-3.1-8B
-library_name: peft
-pipeline_tag: text-generation
-tags:
-- base_model:adapter:meta-llama/Llama-3.1-8B
-- lora
-- transformers
----
-
-# Model Card for Model ID
-
-<!-- Provide a quick summary of what the model is/does. -->
-
-
-
-## Model Details
-
-### Model Description
-
-<!-- Provide a longer summary of what this model is. -->
-
-
-
-- **Developed by:** [More Information Needed]
-- **Funded by [optional]:** [More Information Needed]
-- **Shared by [optional]:** [More Information Needed]
-- **Model type:** [More Information Needed]
-- **Language(s) (NLP):** [More Information Needed]
-- **License:** [More Information Needed]
-- **Finetuned from model [optional]:** [More Information Needed]
-
-### Model Sources [optional]
-
-<!-- Provide the basic links for the model. -->
-
-- **Repository:** [More Information Needed]
-- **Paper [optional]:** [More Information Needed]
-- **Demo [optional]:** [More Information Needed]
-
-## Uses
-
-<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
-
-### Direct Use
-
-<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
-
-[More Information Needed]
-
-### Downstream Use [optional]
-
-<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
-
-[More Information Needed]
-
-### Out-of-Scope Use
-
-<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
-
-[More Information Needed]
-
-## Bias, Risks, and Limitations
-
-<!-- This section is meant to convey both technical and sociotechnical limitations. -->
-
-[More Information Needed]
-
-### Recommendations
-
-<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
-
-Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
-
-## How to Get Started with the Model
-
-Use the code below to get started with the model.
-
-[More Information Needed]
-
-## Training Details
-
-### Training Data
-
-<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
-
-[More Information Needed]
-
-### Training Procedure
-
-<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
-
-#### Preprocessing [optional]
-
-[More Information Needed]
-
-
-#### Training Hyperparameters
-
-- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
-
-#### Speeds, Sizes, Times [optional]
-
-<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
-
-[More Information Needed]
-
-## Evaluation
-
-<!-- This section describes the evaluation protocols and provides the results. -->
-
-### Testing Data, Factors & Metrics
-
-#### Testing Data
-
-<!-- This should link to a Dataset Card if possible. -->
-
-[More Information Needed]
-
-#### Factors
-
-<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
-
-[More Information Needed]
-
-#### Metrics
-
-<!-- These are the evaluation metrics being used, ideally with a description of why. -->
-
-[More Information Needed]
-
-### Results
-
-[More Information Needed]
-
-#### Summary
-
-
-
-## Model Examination [optional]
-
-<!-- Relevant interpretability work for the model goes here -->
-
-[More Information Needed]
-
-## Environmental Impact
-
-<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
-
-Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
-
-- **Hardware Type:** [More Information Needed]
-- **Hours used:** [More Information Needed]
-- **Cloud Provider:** [More Information Needed]
-- **Compute Region:** [More Information Needed]
-- **Carbon Emitted:** [More Information Needed]
-
-## Technical Specifications [optional]
-
-### Model Architecture and Objective
-
-[More Information Needed]
-
-### Compute Infrastructure
-
-[More Information Needed]
-
-#### Hardware
-
-[More Information Needed]
-
-#### Software
-
-[More Information Needed]
-
-## Citation [optional]
-
-<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
-
-**BibTeX:**
-
-[More Information Needed]
-
-**APA:**
-
-[More Information Needed]
-
-## Glossary [optional]
-
-<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
-
-[More Information Needed]
-
-## More Information [optional]
-
-[More Information Needed]
-
-## Model Card Authors [optional]
-
-[More Information Needed]
-
-## Model Card Contact
-
-[More Information Needed]
-### Framework versions
-
-- PEFT 0.18.1
\ No newline at end of file
diff --git a/llama3.1-8b/dp3/epochs/epoch_005/adapter/adapter_config.json b/llama3.1-8b/dp3/epochs/epoch_005/adapter/adapter_config.json
deleted file mode 100644
index 96bd552cbece6fa5ad82ab449ff727b4ceaae15a..0000000000000000000000000000000000000000
--- a/llama3.1-8b/dp3/epochs/epoch_005/adapter/adapter_config.json
+++ /dev/null
@@ -1,46 +0,0 @@
-{
-  "alora_invocation_tokens": null,
-  "alpha_pattern": {},
-  "arrow_config": null,
-  "auto_mapping": null,
-  "base_model_name_or_path": "meta-llama/Llama-3.1-8B",
-  "bias": "none",
-  "corda_config": null,
-  "ensure_weight_tying": true,
-  "eva_config": null,
-  "exclude_modules": null,
-  "fan_in_fan_out": false,
-  "inference_mode": true,
-  "init_lora_weights": true,
-  "layer_replication": null,
-  "layers_pattern": null,
-  "layers_to_transform": null,
-  "loftq_config": {},
-  "lora_alpha": 32,
-  "lora_bias": false,
-  "lora_dropout": 0.05,
-  "megatron_config": null,
-  "megatron_core": "megatron.core",
-  "modules_to_save": [
-    "lm_head",
-    "embed_tokens"
-  ],
-  "peft_type": "LORA",
-  "peft_version": "0.18.1",
-  "qalora_group_size": 16,
-  "r": 16,
-  "rank_pattern": {},
-  "revision": null,
-  "target_modules": [
-    "q_proj",
-    "k_proj",
-    "o_proj",
-    "v_proj"
-  ],
-  "target_parameters": null,
-  "task_type": "CAUSAL_LM",
-  "trainable_token_indices": null,
-  "use_dora": false,
-  "use_qalora": false,
-  "use_rslora": false
-}
\ No newline at end of file
diff --git a/llama3.1-8b/dp3/epochs/epoch_005/audit_results.json b/llama3.1-8b/dp3/epochs/epoch_005/audit_results.json
deleted file mode 100644
index 608339a61b2a970f67d5727695ac2335b517b020..0000000000000000000000000000000000000000
--- a/llama3.1-8b/dp3/epochs/epoch_005/audit_results.json
+++ /dev/null
@@ -1,137 +0,0 @@
-{
-  "delta": 1e-05,
-  "num_canaries": 500,
-  "num_members": 250,
-  "paper_guess_fraction": 0.2,
-  "paper_guess_steps": 20,
-  "loss": {
-    "auc": 0.505432,
-    "empirical_epsilon": {
-      "0.05": 0.0,
-      "0.01": 0.0
-    },
-    "empirical_epsilon_details": {
-      "0.05": {
-        "epsilon": 0.0,
-        "num_guesses": 0,
-        "correct_guesses": 0,
-        "candidate_num_guesses": [
-          5,
-          10,
-          15,
-          20,
-          25,
-          30,
-          35,
-          40,
-          45,
-          50,
-          55,
-          60,
-          65,
-          70,
-          75,
-          80,
-          85,
-          90,
-          95,
-          100
-        ],
-        "direction": "lower"
-      },
-      "0.01": {
-        "epsilon": 0.0,
-        "num_guesses": 0,
-        "correct_guesses": 0,
-        "candidate_num_guesses": [
-          5,
-          10,
-          15,
-          20,
-          25,
-          30,
-          35,
-          40,
-          45,
-          50,
-          55,
-          60,
-          65,
-          70,
-          75,
-          80,
-          85,
-          90,
-          95,
-          100
-        ],
-        "direction": "lower"
-      }
-    }
-  },
-  "embedding": {
-    "auc": 0.50372,
-    "empirical_epsilon": {
-      "0.05": 0.0,
-      "0.01": 0.0
-    },
-    "empirical_epsilon_details": {
-      "0.05": {
-        "epsilon": 0.0,
-        "num_guesses": 0,
-        "correct_guesses": 0,
-        "candidate_num_guesses": [
-          5,
-          10,
-          15,
-          20,
-          25,
-          30,
-          35,
-          40,
-          45,
-          50,
-          55,
-          60,
-          65,
-          70,
-          75,
-          80,
-          85,
-          90,
-          95,
-          100
-        ],
-        "direction": "lower"
-      },
-      "0.01": {
-        "epsilon": 0.0,
-        "num_guesses": 0,
-        "correct_guesses": 0,
-        "candidate_num_guesses": [
-          5,
-          10,
-          15,
-          20,
-          25,
-          30,
-          35,
-          40,
-          45,
-          50,
-          55,
-          60,
-          65,
-          70,
-          75,
-          80,
-          85,
-          90,
-          95,
-          100
-        ],
-        "direction": "lower"
-      }
-    }
-  }
-}
\ No newline at end of file
diff --git a/llama3.1-8b/dp8/epochs/epoch_001/adapter/README.md b/llama3.1-8b/dp8/epochs/epoch_001/adapter/README.md
deleted file mode 100644
index 2206b79a428a68b43cf68e2323a8aa92987452d5..0000000000000000000000000000000000000000
--- a/llama3.1-8b/dp8/epochs/epoch_001/adapter/README.md
+++ /dev/null
@@ -1,207 +0,0 @@
----
-base_model: meta-llama/Llama-3.1-8B
-library_name: peft
-pipeline_tag: text-generation
-tags:
-- base_model:adapter:meta-llama/Llama-3.1-8B
-- lora
-- transformers
----
-
-# Model Card for Model ID
-
-<!-- Provide a quick summary of what the model is/does. -->
-
-
-
-## Model Details
-
-### Model Description
-
-<!-- Provide a longer summary of what this model is. -->
-
-
-
-- **Developed by:** [More Information Needed]
-- **Funded by [optional]:** [More Information Needed]
-- **Shared by [optional]:** [More Information Needed]
-- **Model type:** [More Information Needed]
-- **Language(s) (NLP):** [More Information Needed]
-- **License:** [More Information Needed]
-- **Finetuned from model [optional]:** [More Information Needed]
-
-### Model Sources [optional]
-
-<!-- Provide the basic links for the model. -->
-
-- **Repository:** [More Information Needed]
-- **Paper [optional]:** [More Information Needed]
-- **Demo [optional]:** [More Information Needed]
-
-## Uses
-
-<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
-
-### Direct Use
-
-<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
-
-[More Information Needed]
-
-### Downstream Use [optional]
-
-<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
-
-[More Information Needed]
-
-### Out-of-Scope Use
-
-<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
-
-[More Information Needed]
-
-## Bias, Risks, and Limitations
-
-<!-- This section is meant to convey both technical and sociotechnical limitations. -->
-
-[More Information Needed]
-
-### Recommendations
-
-<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
-
-Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
-
-## How to Get Started with the Model
-
-Use the code below to get started with the model.
-
-[More Information Needed]
-
-## Training Details
-
-### Training Data
-
-<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
-
-[More Information Needed]
-
-### Training Procedure
-
-<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
-
-#### Preprocessing [optional]
-
-[More Information Needed]
-
-
-#### Training Hyperparameters
-
-- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
-
-#### Speeds, Sizes, Times [optional]
-
-<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
-
-[More Information Needed]
-
-## Evaluation
-
-<!-- This section describes the evaluation protocols and provides the results. -->
-
-### Testing Data, Factors & Metrics
-
-#### Testing Data
-
-<!-- This should link to a Dataset Card if possible. -->
-
-[More Information Needed]
-
-#### Factors
-
-<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
-
-[More Information Needed]
-
-#### Metrics
-
-<!-- These are the evaluation metrics being used, ideally with a description of why. -->
-
-[More Information Needed]
-
-### Results
-
-[More Information Needed]
-
-#### Summary
-
-
-
-## Model Examination [optional]
-
-<!-- Relevant interpretability work for the model goes here -->
-
-[More Information Needed]
-
-## Environmental Impact
-
-<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
-
-Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
-
-- **Hardware Type:** [More Information Needed]
-- **Hours used:** [More Information Needed]
-- **Cloud Provider:** [More Information Needed]
-- **Compute Region:** [More Information Needed]
-- **Carbon Emitted:** [More Information Needed]
-
-## Technical Specifications [optional]
-
-### Model Architecture and Objective
-
-[More Information Needed]
-
-### Compute Infrastructure
-
-[More Information Needed]
-
-#### Hardware
-
-[More Information Needed]
-
-#### Software
-
-[More Information Needed]
-
-## Citation [optional]
-
-<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
-
-**BibTeX:**
-
-[More Information Needed]
-
-**APA:**
-
-[More Information Needed]
-
-## Glossary [optional]
-
-<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
-
-[More Information Needed]
-
-## More Information [optional]
-
-[More Information Needed]
-
-## Model Card Authors [optional]
-
-[More Information Needed]
-
-## Model Card Contact
-
-[More Information Needed]
-### Framework versions
-
-- PEFT 0.18.1
\ No newline at end of file
diff --git a/llama3.1-8b/dp8/epochs/epoch_001/adapter/adapter_config.json b/llama3.1-8b/dp8/epochs/epoch_001/adapter/adapter_config.json
deleted file mode 100644
index 30ca9a2f23596ff73d4b32b6460c0df56ac61937..0000000000000000000000000000000000000000
--- a/llama3.1-8b/dp8/epochs/epoch_001/adapter/adapter_config.json
+++ /dev/null
@@ -1,46 +0,0 @@
-{
-  "alora_invocation_tokens": null,
-  "alpha_pattern": {},
-  "arrow_config": null,
-  "auto_mapping": null,
-  "base_model_name_or_path": "meta-llama/Llama-3.1-8B",
-  "bias": "none",
-  "corda_config": null,
-  "ensure_weight_tying": true,
-  "eva_config": null,
-  "exclude_modules": null,
-  "fan_in_fan_out": false,
-  "inference_mode": true,
-  "init_lora_weights": true,
-  "layer_replication": null,
-  "layers_pattern": null,
-  "layers_to_transform": null,
-  "loftq_config": {},
-  "lora_alpha": 32,
-  "lora_bias": false,
-  "lora_dropout": 0.05,
-  "megatron_config": null,
-  "megatron_core": "megatron.core",
-  "modules_to_save": [
-    "lm_head",
-    "embed_tokens"
-  ],
-  "peft_type": "LORA",
-  "peft_version": "0.18.1",
-  "qalora_group_size": 16,
-  "r": 16,
-  "rank_pattern": {},
-  "revision": null,
-  "target_modules": [
-    "v_proj",
-    "o_proj",
-    "q_proj",
-    "k_proj"
-  ],
-  "target_parameters": null,
-  "task_type": "CAUSAL_LM",
-  "trainable_token_indices": null,
-  "use_dora": false,
-  "use_qalora": false,
-  "use_rslora": false
-}
\ No newline at end of file
diff --git a/llama3.1-8b/dp8/epochs/epoch_001/audit_results.json b/llama3.1-8b/dp8/epochs/epoch_001/audit_results.json
deleted file mode 100644
index abf1eda4eb147205b7d68ce9ea77331a153b7dc3..0000000000000000000000000000000000000000
--- a/llama3.1-8b/dp8/epochs/epoch_001/audit_results.json
+++ /dev/null
@@ -1,137 +0,0 @@
-{
-  "delta": 1e-05,
-  "num_canaries": 500,
-  "num_members": 250,
-  "paper_guess_fraction": 0.2,
-  "paper_guess_steps": 20,
-  "loss": {
-    "auc": 0.52804,
-    "empirical_epsilon": {
-      "0.05": 0.0,
-      "0.01": 0.0
-    },
-    "empirical_epsilon_details": {
-      "0.05": {
-        "epsilon": 0.0,
-        "num_guesses": 0,
-        "correct_guesses": 0,
-        "candidate_num_guesses": [
-          5,
-          10,
-          15,
-          20,
-          25,
-          30,
-          35,
-          40,
-          45,
-          50,
-          55,
-          60,
-          65,
-          70,
-          75,
-          80,
-          85,
-          90,
-          95,
-          100
-        ],
-        "direction": "lower"
-      },
-      "0.01": {
-        "epsilon": 0.0,
-        "num_guesses": 0,
-        "correct_guesses": 0,
-        "candidate_num_guesses": [
-          5,
-          10,
-          15,
-          20,
-          25,
-          30,
-          35,
-          40,
-          45,
-          50,
-          55,
-          60,
-          65,
-          70,
-          75,
-          80,
-          85,
-          90,
-          95,
-          100
-        ],
-        "direction": "lower"
-      }
-    }
-  },
-  "embedding": {
-    "auc": 0.5012,
-    "empirical_epsilon": {
-      "0.05": 0.0,
-      "0.01": 0.0
-    },
-    "empirical_epsilon_details": {
-      "0.05": {
-        "epsilon": 0.0,
-        "num_guesses": 0,
-        "correct_guesses": 0,
-        "candidate_num_guesses": [
-          5,
-          10,
-          15,
-          20,
-          25,
-          30,
-          35,
-          40,
-          45,
-          50,
-          55,
-          60,
-          65,
-          70,
-          75,
-          80,
-          85,
-          90,
-          95,
-          100
-        ],
-        "direction": "lower"
-      },
-      "0.01": {
-        "epsilon": 0.0,
-        "num_guesses": 0,
-        "correct_guesses": 0,
-        "candidate_num_guesses": [
-          5,
-          10,
-          15,
-          20,
-          25,
-          30,
-          35,
-          40,
-          45,
-          50,
-          55,
-          60,
-          65,
-          70,
-          75,
-          80,
-          85,
-          90,
-          95,
-          100
-        ],
-        "direction": "lower"
-      }
-    }
-  }
-}
\ No newline at end of file
diff --git a/llama3.1-8b/dp8/epochs/epoch_002/adapter/README.md b/llama3.1-8b/dp8/epochs/epoch_002/adapter/README.md
deleted file mode 100644
index 2206b79a428a68b43cf68e2323a8aa92987452d5..0000000000000000000000000000000000000000
--- a/llama3.1-8b/dp8/epochs/epoch_002/adapter/README.md
+++ /dev/null
@@ -1,207 +0,0 @@
----
-base_model: meta-llama/Llama-3.1-8B
-library_name: peft
-pipeline_tag: text-generation
-tags:
-- base_model:adapter:meta-llama/Llama-3.1-8B
-- lora
-- transformers
----
-
-# Model Card for Model ID
-
-<!-- Provide a quick summary of what the model is/does. -->
-
-
-
-## Model Details
-
-### Model Description
-
-<!-- Provide a longer summary of what this model is. -->
-
-
-
-- **Developed by:** [More Information Needed]
-- **Funded by [optional]:** [More Information Needed]
-- **Shared by [optional]:** [More Information Needed]
-- **Model type:** [More Information Needed]
-- **Language(s) (NLP):** [More Information Needed]
-- **License:** [More Information Needed]
-- **Finetuned from model [optional]:** [More Information Needed]
-
-### Model Sources [optional]
-
-<!-- Provide the basic links for the model. -->
-
-- **Repository:** [More Information Needed]
-- **Paper [optional]:** [More Information Needed]
-- **Demo [optional]:** [More Information Needed]
-
-## Uses
-
-<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
-
-### Direct Use
-
-<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
-
-[More Information Needed]
-
-### Downstream Use [optional]
-
-<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
-
-[More Information Needed]
-
-### Out-of-Scope Use
-
-<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
-
-[More Information Needed]
-
-## Bias, Risks, and Limitations
-
-<!-- This section is meant to convey both technical and sociotechnical limitations. -->
-
-[More Information Needed]
-
-### Recommendations
-
-<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
-
-Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
-
-## How to Get Started with the Model
-
-Use the code below to get started with the model.
-
-[More Information Needed]
-
-## Training Details
-
-### Training Data
-
-<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
-
-[More Information Needed]
-
-### Training Procedure
-
-<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
-
-#### Preprocessing [optional]
-
-[More Information Needed]
-
-
-#### Training Hyperparameters
-
-- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
-
-#### Speeds, Sizes, Times [optional]
-
-<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
-
-[More Information Needed]
-
-## Evaluation
-
-<!-- This section describes the evaluation protocols and provides the results. -->
-
-### Testing Data, Factors & Metrics
-
-#### Testing Data
-
-<!-- This should link to a Dataset Card if possible. -->
-
-[More Information Needed]
-
-#### Factors
-
-<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
-
-[More Information Needed]
-
-#### Metrics
-
-<!-- These are the evaluation metrics being used, ideally with a description of why. -->
-
-[More Information Needed]
-
-### Results
-
-[More Information Needed]
-
-#### Summary
-
-
-
-## Model Examination [optional]
-
-<!-- Relevant interpretability work for the model goes here -->
-
-[More Information Needed]
-
-## Environmental Impact
-
-<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
-
-Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
-
-- **Hardware Type:** [More Information Needed]
-- **Hours used:** [More Information Needed]
-- **Cloud Provider:** [More Information Needed]
-- **Compute Region:** [More Information Needed]
-- **Carbon Emitted:** [More Information Needed]
-
-## Technical Specifications [optional]
-
-### Model Architecture and Objective
-
-[More Information Needed]
-
-### Compute Infrastructure
-
-[More Information Needed]
-
-#### Hardware
-
-[More Information Needed]
-
-#### Software
-
-[More Information Needed]
-
-## Citation [optional]
-
-<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
-
-**BibTeX:**
-
-[More Information Needed]
-
-**APA:**
-
-[More Information Needed]
-
-## Glossary [optional]
-
-<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
-
-[More Information Needed]
-
-## More Information [optional]
-
-[More Information Needed]
-
-## Model Card Authors [optional]
-
-[More Information Needed]
-
-## Model Card Contact
-
-[More Information Needed]
-### Framework versions
-
-- PEFT 0.18.1
\ No newline at end of file
diff --git a/llama3.1-8b/dp8/epochs/epoch_002/adapter/adapter_config.json b/llama3.1-8b/dp8/epochs/epoch_002/adapter/adapter_config.json
deleted file mode 100644
index 30ca9a2f23596ff73d4b32b6460c0df56ac61937..0000000000000000000000000000000000000000
--- a/llama3.1-8b/dp8/epochs/epoch_002/adapter/adapter_config.json
+++ /dev/null
@@ -1,46 +0,0 @@
-{
-  "alora_invocation_tokens": null,
-  "alpha_pattern": {},
-  "arrow_config": null,
-  "auto_mapping": null,
-  "base_model_name_or_path": "meta-llama/Llama-3.1-8B",
-  "bias": "none",
-  "corda_config": null,
-  "ensure_weight_tying": true,
-  "eva_config": null,
-  "exclude_modules": null,
-  "fan_in_fan_out": false,
-  "inference_mode": true,
-  "init_lora_weights": true,
-  "layer_replication": null,
-  "layers_pattern": null,
-  "layers_to_transform": null,
-  "loftq_config": {},
-  "lora_alpha": 32,
-  "lora_bias": false,
-  "lora_dropout": 0.05,
-  "megatron_config": null,
-  "megatron_core": "megatron.core",
-  "modules_to_save": [
-    "lm_head",
-    "embed_tokens"
-  ],
-  "peft_type": "LORA",
-  "peft_version": "0.18.1",
-  "qalora_group_size": 16,
-  "r": 16,
-  "rank_pattern": {},
-  "revision": null,
-  "target_modules": [
-    "v_proj",
-    "o_proj",
-    "q_proj",
-    "k_proj"
-  ],
-  "target_parameters": null,
-  "task_type": "CAUSAL_LM",
-  "trainable_token_indices": null,
-  "use_dora": false,
-  "use_qalora": false,
-  "use_rslora": false
-}
\ No newline at end of file
diff --git a/llama3.1-8b/dp8/epochs/epoch_002/audit_results.json b/llama3.1-8b/dp8/epochs/epoch_002/audit_results.json
deleted file mode 100644
index 1226720394a1d8fa83fcf6d1b13b71af9496f759..0000000000000000000000000000000000000000
--- a/llama3.1-8b/dp8/epochs/epoch_002/audit_results.json
+++ /dev/null
@@ -1,137 +0,0 @@
-{
-  "delta": 1e-05,
-  "num_canaries": 500,
-  "num_members": 250,
-  "paper_guess_fraction": 0.2,
-  "paper_guess_steps": 20,
-  "loss": {
-    "auc": 0.527752,
-    "empirical_epsilon": {
-      "0.05": 0.08366204984486103,
-      "0.01": 0.0
-    },
-    "empirical_epsilon_details": {
-      "0.05": {
-        "epsilon": 0.08366204984486103,
-        "num_guesses": 95,
-        "correct_guesses": 58,
-        "candidate_num_guesses": [
-          5,
-          10,
-          15,
-          20,
-          25,
-          30,
-          35,
-          40,
-          45,
-          50,
-          55,
-          60,
-          65,
-          70,
-          75,
-          80,
-          85,
-          90,
-          95,
-          100
-        ],
-        "direction": "lower"
-      },
-      "0.01": {
-        "epsilon": 0.0,
-        "num_guesses": 0,
-        "correct_guesses": 0,
-        "candidate_num_guesses": [
-          5,
-          10,
-          15,
-          20,
-          25,
-          30,
-          35,
-          40,
-          45,
-          50,
-          55,
-          60,
-          65,
-          70,
-          75,
-          80,
-          85,
-          90,
-          95,
-          100
-        ],
-        "direction": "lower"
-      }
-    }
-  },
-  "embedding": {
-    "auc": 0.52024,
-    "empirical_epsilon": {
-      "0.05": 0.0,
-      "0.01": 0.0
-    },
-    "empirical_epsilon_details": {
-      "0.05": {
-        "epsilon": 0.0,
-        "num_guesses": 0,
-        "correct_guesses": 0,
-        "candidate_num_guesses": [
-          5,
-          10,
-          15,
-          20,
-          25,
-          30,
-          35,
-          40,
-          45,
-          50,
-          55,
-          60,
-          65,
-          70,
-          75,
-          80,
-          85,
-          90,
-          95,
-          100
-        ],
-        "direction": "lower"
-      },
-      "0.01": {
-        "epsilon": 0.0,
-        "num_guesses": 0,
-        "correct_guesses": 0,
-        "candidate_num_guesses": [
-          5,
-          10,
-          15,
-          20,
-          25,
-          30,
-          35,
-          40,
-          45,
-          50,
-          55,
-          60,
-          65,
-          70,
-          75,
-          80,
-          85,
-          90,
-          95,
-          100
-        ],
-        "direction": "lower"
-      }
-    }
-  }
-}
\ No newline at end of file
diff --git a/llama3.1-8b/dp8/epochs/epoch_003/adapter/README.md b/llama3.1-8b/dp8/epochs/epoch_003/adapter/README.md
deleted file mode 100644
index 2206b79a428a68b43cf68e2323a8aa92987452d5..0000000000000000000000000000000000000000
--- a/llama3.1-8b/dp8/epochs/epoch_003/adapter/README.md
+++ /dev/null
@@ -1,207 +0,0 @@
----
-base_model: meta-llama/Llama-3.1-8B
-library_name: peft
-pipeline_tag: text-generation
-tags:
-- base_model:adapter:meta-llama/Llama-3.1-8B
-- lora
-- transformers
----
-
-# Model Card for Model ID
-
-<!-- Provide a quick summary of what the model is/does. -->
-
-
-
-## Model Details
-
-### Model Description
-
-<!-- Provide a longer summary of what this model is. -->
-
-
-
-- **Developed by:** [More Information Needed]
-- **Funded by [optional]:** [More Information Needed]
-- **Shared by [optional]:** [More Information Needed]
-- **Model type:** [More Information Needed]
-- **Language(s) (NLP):** [More Information Needed]
-- **License:** [More Information Needed]
-- **Finetuned from model [optional]:** [More Information Needed]
-
-### Model Sources [optional]
-
-<!-- Provide the basic links for the model. -->
-
-- **Repository:** [More Information Needed]
-- **Paper [optional]:** [More Information Needed]
-- **Demo [optional]:** [More Information Needed]
-
-## Uses
-
-<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
-
-### Direct Use
-
-<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
-
-[More Information Needed]
-
-### Downstream Use [optional]
-
-<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
-
-[More Information Needed]
-
-### Out-of-Scope Use
-
-<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
-
-[More Information Needed]
-
-## Bias, Risks, and Limitations
-
-<!-- This section is meant to convey both technical and sociotechnical limitations. -->
-
-[More Information Needed]
-
-### Recommendations
-
-<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
-
-Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
-
-## How to Get Started with the Model
-
-Use the code below to get started with the model.
-
-[More Information Needed]
-
-## Training Details
-
-### Training Data
-
-<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
-
-[More Information Needed]
-
-### Training Procedure
-
-<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
-
-#### Preprocessing [optional]
-
-[More Information Needed]
-
-
-#### Training Hyperparameters
-
-- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
-
-#### Speeds, Sizes, Times [optional]
-
-<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
-
-[More Information Needed]
-
-## Evaluation
-
-<!-- This section describes the evaluation protocols and provides the results. -->
-
-### Testing Data, Factors & Metrics
-
-#### Testing Data
-
-<!-- This should link to a Dataset Card if possible. -->
-
-[More Information Needed]
-
-#### Factors
-
-<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
-
-[More Information Needed]
-
-#### Metrics
-
-<!-- These are the evaluation metrics being used, ideally with a description of why. -->
-
-[More Information Needed]
-
-### Results
-
-[More Information Needed]
-
-#### Summary
-
-
-
-## Model Examination [optional]
-
-<!-- Relevant interpretability work for the model goes here -->
-
-[More Information Needed]
-
-## Environmental Impact
-
-<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
-
-Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
-
-- **Hardware Type:** [More Information Needed]
-- **Hours used:** [More Information Needed]
-- **Cloud Provider:** [More Information Needed]
-- **Compute Region:** [More Information Needed]
-- **Carbon Emitted:** [More Information Needed]
-
-## Technical Specifications [optional]
-
-### Model Architecture and Objective
-
-[More Information Needed]
-
-### Compute Infrastructure
-
-[More Information Needed]
-
-#### Hardware
-
-[More Information Needed]
-
-#### Software
-
-[More Information Needed]
-
-## Citation [optional]
-
-<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
-
-**BibTeX:**
-
-[More Information Needed]
-
-**APA:**
-
-[More Information Needed]
-
-## Glossary [optional]
-
-<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
-
-[More Information Needed]
-
-## More Information [optional]
-
-[More Information Needed]
-
-## Model Card Authors [optional]
-
-[More Information Needed]
-
-## Model Card Contact
-
-[More Information Needed]
-### Framework versions
-
-- PEFT 0.18.1
\ No newline at end of file
diff --git a/llama3.1-8b/dp8/epochs/epoch_003/adapter/adapter_config.json b/llama3.1-8b/dp8/epochs/epoch_003/adapter/adapter_config.json
deleted file mode 100644
index 30ca9a2f23596ff73d4b32b6460c0df56ac61937..0000000000000000000000000000000000000000
--- a/llama3.1-8b/dp8/epochs/epoch_003/adapter/adapter_config.json
+++ /dev/null
@@ -1,46 +0,0 @@
-{
-  "alora_invocation_tokens": null,
-  "alpha_pattern": {},
-  "arrow_config": null,
-  "auto_mapping": null,
-  "base_model_name_or_path": "meta-llama/Llama-3.1-8B",
-  "bias": "none",
-  "corda_config": null,
-  "ensure_weight_tying": true,
-  "eva_config": null,
-  "exclude_modules": null,
-  "fan_in_fan_out": false,
-  "inference_mode": true,
-  "init_lora_weights": true,
-  "layer_replication": null,
-  "layers_pattern": null,
-  "layers_to_transform": null,
-  "loftq_config": {},
-  "lora_alpha": 32,
-  "lora_bias": false,
-  "lora_dropout": 0.05,
-  "megatron_config": null,
-  "megatron_core": "megatron.core",
-  "modules_to_save": [
-    "lm_head",
-    "embed_tokens"
-  ],
-  "peft_type": "LORA",
-  "peft_version": "0.18.1",
-  "qalora_group_size": 16,
-  "r": 16,
-  "rank_pattern": {},
-  "revision": null,
-  "target_modules": [
-    "v_proj",
-    "o_proj",
-    "q_proj",
-    "k_proj"
-  ],
-  "target_parameters": null,
-  "task_type": "CAUSAL_LM",
-  "trainable_token_indices": null,
-  "use_dora": false,
-  "use_qalora": false,
-  "use_rslora": false
-}
\ No newline at end of file
diff --git a/llama3.1-8b/dp8/epochs/epoch_003/audit_results.json b/llama3.1-8b/dp8/epochs/epoch_003/audit_results.json
deleted file mode 100644
index 29ccb9c9e57684b9142fe9cf2fe16cc42f36b924..0000000000000000000000000000000000000000
--- a/llama3.1-8b/dp8/epochs/epoch_003/audit_results.json
+++ /dev/null
@@ -1,137 +0,0 @@
-{
-  "delta": 1e-05,
-  "num_canaries": 500,
-  "num_members": 250,
-  "paper_guess_fraction": 0.2,
-  "paper_guess_steps": 20,
-  "loss": {
-    "auc": 0.525376,
-    "empirical_epsilon": {
-      "0.05": 0.0,
-      "0.01": 0.0
-    },
-    "empirical_epsilon_details": {
-      "0.05": {
-        "epsilon": 0.0,
-        "num_guesses": 0,
-        "correct_guesses": 0,
-        "candidate_num_guesses": [
-          5,
-          10,
-          15,
-          20,
-          25,
-          30,
-          35,
-          40,
-          45,
-          50,
-          55,
-          60,
-          65,
-          70,
-          75,
-          80,
-          85,
-          90,
-          95,
-          100
-        ],
-        "direction": "lower"
-      },
-      "0.01": {
-        "epsilon": 0.0,
-        "num_guesses": 0,
-        "correct_guesses": 0,
-        "candidate_num_guesses": [
-          5,
-          10,
-          15,
-          20,
-          25,
-          30,
-          35,
-          40,
-          45,
-          50,
-          55,
-          60,
-          65,
-          70,
-          75,
-          80,
-          85,
-          90,
-          95,
-          100
-        ],
-        "direction": "lower"
-      }
-    }
-  },
-  "embedding": {
-    "auc": 0.5028,
-    "empirical_epsilon": {
-      "0.05": 0.0,
-      "0.01": 0.0
-    },
-    "empirical_epsilon_details": {
-      "0.05": {
-        "epsilon": 0.0,
-        "num_guesses": 0,
-        "correct_guesses": 0,
-        "candidate_num_guesses": [
-          5,
-          10,
-          15,
-          20,
-          25,
-          30,
-          35,
-          40,
-          45,
-          50,
-          55,
-          60,
-          65,
-          70,
-          75,
-          80,
-          85,
-          90,
-          95,
-          100
-        ],
-        "direction": "lower"
-      },
-      "0.01": {
-        "epsilon": 0.0,
-        "num_guesses": 0,
-        "correct_guesses": 0,
-        "candidate_num_guesses": [
-          5,
-          10,
-          15,
-          20,
-          25,
-          30,
-          35,
-          40,
-          45,
-          50,
-          55,
-          60,
-          65,
-          70,
-          75,
-          80,
-          85,
-          90,
-          95,
-          100
-        ],
-        "direction": "lower"
-      }
-    }
-  }
-}
\ No newline at end of file
diff --git a/llama3.1-8b/dp8/epochs/epoch_004/adapter/README.md b/llama3.1-8b/dp8/epochs/epoch_004/adapter/README.md
deleted file mode 100644
index 2206b79a428a68b43cf68e2323a8aa92987452d5..0000000000000000000000000000000000000000
--- a/llama3.1-8b/dp8/epochs/epoch_004/adapter/README.md
+++ /dev/null
@@ -1,207 +0,0 @@
----
-base_model: meta-llama/Llama-3.1-8B
-library_name: peft
-pipeline_tag: text-generation
-tags:
-- base_model:adapter:meta-llama/Llama-3.1-8B
-- lora
-- transformers
----
-
-# Model Card for Model ID
-
-<!-- Provide a quick summary of what the model is/does. -->
-
-
-
-## Model Details
-
-### Model Description
-
-<!-- Provide a longer summary of what this model is. -->
-
-
-
-- **Developed by:** [More Information Needed]
-- **Funded by [optional]:** [More Information Needed]
-- **Shared by [optional]:** [More Information Needed]
-- **Model type:** [More Information Needed]
-- **Language(s) (NLP):** [More Information Needed]
-- **License:** [More Information Needed]
-- **Finetuned from model [optional]:** [More Information Needed]
-
-### Model Sources [optional]
-
-<!-- Provide the basic links for the model. -->
-
-- **Repository:** [More Information Needed]
-- **Paper [optional]:** [More Information Needed]
-- **Demo [optional]:** [More Information Needed]
-
-## Uses
-
-<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
-
-### Direct Use
-
-<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
-
-[More Information Needed]
-
-### Downstream Use [optional]
-
-<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
-
-[More Information Needed]
-
-### Out-of-Scope Use
-
-<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
-
-[More Information Needed]
-
-## Bias, Risks, and Limitations
-
-<!-- This section is meant to convey both technical and sociotechnical limitations. -->
-
-[More Information Needed]
-
-### Recommendations
-
-<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
-
-Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
-
-## How to Get Started with the Model
-
-Use the code below to get started with the model.
-
-[More Information Needed]
-
-## Training Details
-
-### Training Data
-
-<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
-
-[More Information Needed]
-
-### Training Procedure
-
-<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
-
-#### Preprocessing [optional]
-
-[More Information Needed]
-
-
-#### Training Hyperparameters
-
-- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
-
-#### Speeds, Sizes, Times [optional]
-
-<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
-
-[More Information Needed]
-
-## Evaluation
-
-<!-- This section describes the evaluation protocols and provides the results. -->
-
-### Testing Data, Factors & Metrics
-
-#### Testing Data
-
-<!-- This should link to a Dataset Card if possible. -->
-
-[More Information Needed]
-
-#### Factors
-
-<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
-
-[More Information Needed]
-
-#### Metrics
-
-<!-- These are the evaluation metrics being used, ideally with a description of why. -->
-
-[More Information Needed]
-
-### Results
-
-[More Information Needed]
-
-#### Summary
-
-
-
-## Model Examination [optional]
-
-<!-- Relevant interpretability work for the model goes here -->
-
-[More Information Needed]
-
-## Environmental Impact
-
-<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
-
-Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
-
-- **Hardware Type:** [More Information Needed]
-- **Hours used:** [More Information Needed]
-- **Cloud Provider:** [More Information Needed]
-- **Compute Region:** [More Information Needed]
-- **Carbon Emitted:** [More Information Needed]
-
-## Technical Specifications [optional]
-
-### Model Architecture and Objective
-
-[More Information Needed]
-
-### Compute Infrastructure
-
-[More Information Needed]
-
-#### Hardware
-
-[More Information Needed]
-
-#### Software
-
-[More Information Needed]
-
-## Citation [optional]
-
-<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
-
-**BibTeX:**
-
-[More Information Needed]
-
-**APA:**
-
-[More Information Needed]
-
-## Glossary [optional]
-
-<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
-
-[More Information Needed]
-
-## More Information [optional]
-
-[More Information Needed]
-
-## Model Card Authors [optional]
-
-[More Information Needed]
-
-## Model Card Contact
-
-[More Information Needed]
-### Framework versions
-
-- PEFT 0.18.1
\ No newline at end of file
diff --git a/llama3.1-8b/dp8/epochs/epoch_004/adapter/adapter_config.json b/llama3.1-8b/dp8/epochs/epoch_004/adapter/adapter_config.json
deleted file mode 100644
index 30ca9a2f23596ff73d4b32b6460c0df56ac61937..0000000000000000000000000000000000000000
--- a/llama3.1-8b/dp8/epochs/epoch_004/adapter/adapter_config.json
+++ /dev/null
@@ -1,46 +0,0 @@
-{
-  "alora_invocation_tokens": null,
-  "alpha_pattern": {},
-  "arrow_config": null,
-  "auto_mapping": null,
-  "base_model_name_or_path": "meta-llama/Llama-3.1-8B",
-  "bias": "none",
-  "corda_config": null,
-  "ensure_weight_tying": true,
-  "eva_config": null,
-  "exclude_modules": null,
-  "fan_in_fan_out": false,
-  "inference_mode": true,
-  "init_lora_weights": true,
-  "layer_replication": null,
-  "layers_pattern": null,
-  "layers_to_transform": null,
-  "loftq_config": {},
-  "lora_alpha": 32,
-  "lora_bias": false,
-  "lora_dropout": 0.05,
-  "megatron_config": null,
-  "megatron_core": "megatron.core",
-  "modules_to_save": [
-    "lm_head",
-    "embed_tokens"
-  ],
-  "peft_type": "LORA",
-  "peft_version": "0.18.1",
-  "qalora_group_size": 16,
-  "r": 16,
-  "rank_pattern": {},
-  "revision": null,
-  "target_modules": [
-    "v_proj",
-    "o_proj",
-    "q_proj",
-    "k_proj"
-  ],
-  "target_parameters": null,
-  "task_type": "CAUSAL_LM",
-  "trainable_token_indices": null,
-  "use_dora": false,
-  "use_qalora": false,
-  "use_rslora": false
-}
\ No newline at end of file
diff --git a/llama3.1-8b/dp8/epochs/epoch_004/audit_results.json b/llama3.1-8b/dp8/epochs/epoch_004/audit_results.json
deleted file mode 100644
index b2f54d85833b9b06ea430b2cd8ec71f48c2bf7d5..0000000000000000000000000000000000000000
--- a/llama3.1-8b/dp8/epochs/epoch_004/audit_results.json
+++ /dev/null
@@ -1,137 +0,0 @@
-{
-  "delta": 1e-05,
-  "num_canaries": 500,
-  "num_members": 250,
-  "paper_guess_fraction": 0.2,
-  "paper_guess_steps": 20,
-  "loss": {
-    "auc": 0.522528,
-    "empirical_epsilon": {
-      "0.05": 0.172086076810956,
-      "0.01": 0.0
-    },
-    "empirical_epsilon_details": {
-      "0.05": {
-        "epsilon": 0.172086076810956,
-        "num_guesses": 20,
-        "correct_guesses": 15,
-        "candidate_num_guesses": [
-          5,
-          10,
-          15,
-          20,
-          25,
-          30,
-          35,
-          40,
-          45,
-          50,
-          55,
-          60,
-          65,
-          70,
-          75,
-          80,
-          85,
-          90,
-          95,
-          100
-        ],
-        "direction": "lower"
-      },
-      "0.01": {
-        "epsilon": 0.0,
-        "num_guesses": 0,
-        "correct_guesses": 0,
-        "candidate_num_guesses": [
-          5,
-          10,
-          15,
-          20,
-          25,
-          30,
-          35,
-          40,
-          45,
-          50,
-          55,
-          60,
-          65,
-          70,
-          75,
-          80,
-          85,
-          90,
-          95,
-          100
-        ],
-        "direction": "lower"
-      }
-    }
-  },
-  "embedding": {
-    "auc": 0.504168,
-    "empirical_epsilon": {
-      "0.05": 0.0,
-      "0.01": 0.0
-    },
-    "empirical_epsilon_details": {
-      "0.05": {
-        "epsilon": 0.0,
-        "num_guesses": 0,
-        "correct_guesses": 0,
-        "candidate_num_guesses": [
-          5,
-          10,
-          15,
-          20,
-          25,
-          30,
-          35,
-          40,
-          45,
-          50,
-          55,
-          60,
-          65,
-          70,
-          75,
-          80,
-          85,
-          90,
-          95,
-          100
-        ],
-        "direction": "lower"
-      },
-      "0.01": {
-        "epsilon": 0.0,
-        "num_guesses": 0,
-        "correct_guesses": 0,
-        "candidate_num_guesses": [
-          5,
-          10,
-          15,
-          20,
-          25,
-          30,
-          35,
-          40,
-          45,
-          50,
-          55,
-          60,
-          65,
-          70,
-          75,
-          80,
-          85,
-          90,
-          95,
-          100
-        ],
-        "direction": "lower"
-      }
-    }
-  }
-}
\ No newline at end of file
diff --git a/llama3.1-8b/dp8/epochs/epoch_005/adapter/README.md b/llama3.1-8b/dp8/epochs/epoch_005/adapter/README.md
deleted file mode 100644
index 2206b79a428a68b43cf68e2323a8aa92987452d5..0000000000000000000000000000000000000000
--- a/llama3.1-8b/dp8/epochs/epoch_005/adapter/README.md
+++ /dev/null
@@ -1,207 +0,0 @@
----
-base_model: meta-llama/Llama-3.1-8B
-library_name: peft
-pipeline_tag: text-generation
-tags:
-- base_model:adapter:meta-llama/Llama-3.1-8B
-- lora
-- transformers
----
-
-# Model Card for Model ID
-
-<!-- Provide a quick summary of what the model is/does. -->
-
-
-
-## Model Details
-
-### Model Description
-
-<!-- Provide a longer summary of what this model is. -->
-
-
-
-- **Developed by:** [More Information Needed]
-- **Funded by [optional]:** [More Information Needed]
-- **Shared by [optional]:** [More Information Needed]
-- **Model type:** [More Information Needed]
-- **Language(s) (NLP):** [More Information Needed]
-- **License:** [More Information Needed]
-- **Finetuned from model [optional]:** [More Information Needed]
-
-### Model Sources [optional]
-
-<!-- Provide the basic links for the model. -->
-
-- **Repository:** [More Information Needed]
-- **Paper [optional]:** [More Information Needed]
-- **Demo [optional]:** [More Information Needed]
-
-## Uses
-
-<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
-
-### Direct Use
-
-<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
-
-[More Information Needed]
-
-### Downstream Use [optional]
-
-<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
-
-[More Information Needed]
-
-### Out-of-Scope Use
-
-<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
-
-[More Information Needed]
-
-## Bias, Risks, and Limitations
-
-<!-- This section is meant to convey both technical and sociotechnical limitations. -->
-
-[More Information Needed]
-
-### Recommendations
-
-<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
-
-Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
-
-## How to Get Started with the Model
-
-Use the code below to get started with the model.
-
-[More Information Needed]
-
-## Training Details
-
-### Training Data
-
-<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
-
-[More Information Needed]
-
-### Training Procedure
-
-<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
-
-#### Preprocessing [optional]
-
-[More Information Needed]
-
-
-#### Training Hyperparameters
-
-- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
-
-#### Speeds, Sizes, Times [optional]
-
-<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
-
-[More Information Needed]
-
-## Evaluation
-
-<!-- This section describes the evaluation protocols and provides the results. -->
-
-### Testing Data, Factors & Metrics
-
-#### Testing Data
-
-<!-- This should link to a Dataset Card if possible. -->
-
-[More Information Needed]
-
-#### Factors
-
-<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
-
-[More Information Needed]
-
-#### Metrics
-
-<!-- These are the evaluation metrics being used, ideally with a description of why. -->
-
-[More Information Needed]
-
-### Results
-
-[More Information Needed]
-
-#### Summary
-
-
-
-## Model Examination [optional]
-
-<!-- Relevant interpretability work for the model goes here -->
-
-[More Information Needed]
-
-## Environmental Impact
-
-<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
-
-Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
-
-- **Hardware Type:** [More Information Needed]
-- **Hours used:** [More Information Needed]
-- **Cloud Provider:** [More Information Needed]
-- **Compute Region:** [More Information Needed]
-- **Carbon Emitted:** [More Information Needed]
-
-## Technical Specifications [optional]
-
-### Model Architecture and Objective
-
-[More Information Needed]
-
-### Compute Infrastructure
-
-[More Information Needed]
-
-#### Hardware
-
-[More Information Needed]
-
-#### Software
-
-[More Information Needed]
-
-## Citation [optional]
-
-<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
-
-**BibTeX:**
-
-[More Information Needed]
-
-**APA:**
-
-[More Information Needed]
-
-## Glossary [optional]
-
-<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
-
-[More Information Needed]
-
-## More Information [optional]
-
-[More Information Needed]
-
-## Model Card Authors [optional]
-
-[More Information Needed]
-
-## Model Card Contact
-
-[More Information Needed]
-### Framework versions
-
-- PEFT 0.18.1
\ No newline at end of file
diff --git a/llama3.1-8b/dp8/epochs/epoch_005/adapter/adapter_config.json b/llama3.1-8b/dp8/epochs/epoch_005/adapter/adapter_config.json
deleted file mode 100644
index 30ca9a2f23596ff73d4b32b6460c0df56ac61937..0000000000000000000000000000000000000000
--- a/llama3.1-8b/dp8/epochs/epoch_005/adapter/adapter_config.json
+++ /dev/null
@@ -1,46 +0,0 @@
-{
-  "alora_invocation_tokens": null,
-  "alpha_pattern": {},
-  "arrow_config": null,
-  "auto_mapping": null,
-  "base_model_name_or_path": "meta-llama/Llama-3.1-8B",
-  "bias": "none",
-  "corda_config": null,
-  "ensure_weight_tying": true,
-  "eva_config": null,
-  "exclude_modules": null,
-  "fan_in_fan_out": false,
-  "inference_mode": true,
-  "init_lora_weights": true,
-  "layer_replication": null,
-  "layers_pattern": null,
-  "layers_to_transform": null,
-  "loftq_config": {},
-  "lora_alpha": 32,
-  "lora_bias": false,
-  "lora_dropout": 0.05,
-  "megatron_config": null,
-  "megatron_core": "megatron.core",
-  "modules_to_save": [
-    "lm_head",
-    "embed_tokens"
-  ],
-  "peft_type": "LORA",
-  "peft_version": "0.18.1",
-  "qalora_group_size": 16,
-  "r": 16,
-  "rank_pattern": {},
-  "revision": null,
-  "target_modules": [
-    "v_proj",
-    "o_proj",
-    "q_proj",
-    "k_proj"
-  ],
-  "target_parameters": null,
-  "task_type": "CAUSAL_LM",
-  "trainable_token_indices": null,
-  "use_dora": false,
-  "use_qalora": false,
-  "use_rslora": false
-}
\ No newline at end of file
diff --git a/llama3.1-8b/dp8/epochs/epoch_005/audit_results.json b/llama3.1-8b/dp8/epochs/epoch_005/audit_results.json
deleted file mode 100644
index 88e11174a97f8dbfbbaab5c877d47bc6561dcf30..0000000000000000000000000000000000000000
--- a/llama3.1-8b/dp8/epochs/epoch_005/audit_results.json
+++ /dev/null
@@ -1,137 +0,0 @@
-{
-  "delta": 1e-05,
-  "num_canaries": 500,
-  "num_members": 250,
-  "paper_guess_fraction": 0.2,
-  "paper_guess_steps": 20,
-  "loss": {
-    "auc": 0.523296,
-    "empirical_epsilon": {
-      "0.05": 0.054578399285674095,
-      "0.01": 0.0
-    },
-    "empirical_epsilon_details": {
-      "0.05": {
-        "epsilon": 0.054578399285674095,
-        "num_guesses": 50,
-        "correct_guesses": 32,
-        "candidate_num_guesses": [
-          5,
-          10,
-          15,
-          20,
-          25,
-          30,
-          35,
-          40,
-          45,
-          50,
-          55,
-          60,
-          65,
-          70,
-          75,
-          80,
-          85,
-          90,
-          95,
-          100
-        ],
-        "direction": "lower"
-      },
-      "0.01": {
-        "epsilon": 0.0,
-        "num_guesses": 0,
-        "correct_guesses": 0,
-        "candidate_num_guesses": [
-          5,
-          10,
-          15,
-          20,
-          25,
-          30,
-          35,
-          40,
-          45,
-          50,
-          55,
-          60,
-          65,
-          70,
-          75,
-          80,
-          85,
-          90,
-          95,
-          100
-        ],
-        "direction": "lower"
-      }
-    }
-  },
-  "embedding": {
-    "auc": 0.500064,
-    "empirical_epsilon": {
-      "0.05": 0.0,
-      "0.01": 0.0
-    },
-    "empirical_epsilon_details": {
-      "0.05": {
-        "epsilon": 0.0,
-        "num_guesses": 0,
-        "correct_guesses": 0,
-        "candidate_num_guesses": [
-          5,
-          10,
-          15,
-          20,
-          25,
-          30,
-          35,
-          40,
-          45,
-          50,
-          55,
-          60,
-          65,
-          70,
-          75,
-          80,
-          85,
-          90,
-          95,
-          100
-        ],
-        "direction": "lower"
-      },
-      "0.01": {
-        "epsilon": 0.0,
-        "num_guesses": 0,
-        "correct_guesses": 0,
-        "candidate_num_guesses": [
-          5,
-          10,
-          15,
-          20,
-          25,
-          30,
-          35,
-          40,
-          45,
-          50,
-          55,
-          60,
-          65,
-          70,
-          75,
-          80,
-          85,
-          90,
-          95,
-          100
-        ],
-        "direction": "lower"
-      }
-    }
-  }
-}
\ No newline at end of file
diff --git a/llama3.2-3b/base/epochs/epoch_001/adapter/README.md b/llama3.2-3b/base/epochs/epoch_001/adapter/README.md
deleted file mode 100644
index 9eb98a277210fc09f54d9dc34ff6339fae8f8173..0000000000000000000000000000000000000000
--- a/llama3.2-3b/base/epochs/epoch_001/adapter/README.md
+++ /dev/null
@@ -1,207 +0,0 @@
----
-base_model: meta-llama/Llama-3.2-3B
-library_name: peft
-pipeline_tag: text-generation
-tags:
-- base_model:adapter:meta-llama/Llama-3.2-3B
-- lora
-- transformers
----
-
-# Model Card for Model ID
-
-<!-- Provide a quick summary of what the model is/does. -->
-
-
-
-## Model Details
-
-### Model Description
-
-<!-- Provide a longer summary of what this model is. -->
-
-
-
-- **Developed by:** [More Information Needed]
-- **Funded by [optional]:** [More Information Needed]
-- **Shared by [optional]:** [More Information Needed]
-- **Model type:** [More Information Needed]
-- **Language(s) (NLP):** [More Information Needed]
-- **License:** [More Information Needed]
-- **Finetuned from model [optional]:** [More Information Needed]
-
-### Model Sources [optional]
-
-<!-- Provide the basic links for the model. -->
-
-- **Repository:** [More Information Needed]
-- **Paper [optional]:** [More Information Needed]
-- **Demo [optional]:** [More Information Needed]
-
-## Uses
-
-<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
-
-### Direct Use
-
-<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
-
-[More Information Needed]
-
-### Downstream Use [optional]
-
-<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
-
-[More Information Needed]
-
-### Out-of-Scope Use
-
-<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
-
-[More Information Needed]
-
-## Bias, Risks, and Limitations
-
-<!-- This section is meant to convey both technical and sociotechnical limitations. -->
-
-[More Information Needed]
-
-### Recommendations
-
-<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
-
-Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
-
-## How to Get Started with the Model
-
-Use the code below to get started with the model.
-
-[More Information Needed]
-
-## Training Details
-
-### Training Data
-
-<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
-
-[More Information Needed]
-
-### Training Procedure
-
-<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
-
-#### Preprocessing [optional]
-
-[More Information Needed]
-
-
-#### Training Hyperparameters
-
-- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
-
-#### Speeds, Sizes, Times [optional]
-
-<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
-
-[More Information Needed]
-
-## Evaluation
-
-<!-- This section describes the evaluation protocols and provides the results. -->
-
-### Testing Data, Factors & Metrics
-
-#### Testing Data
-
-<!-- This should link to a Dataset Card if possible. -->
-
-[More Information Needed]
-
-#### Factors
-
-<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
-
-[More Information Needed]
-
-#### Metrics
-
-<!-- These are the evaluation metrics being used, ideally with a description of why. -->
-
-[More Information Needed]
-
-### Results
-
-[More Information Needed]
-
-#### Summary
-
-
-
-## Model Examination [optional]
-
-<!-- Relevant interpretability work for the model goes here -->
-
-[More Information Needed]
-
-## Environmental Impact
-
-<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
-
-Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
-
-- **Hardware Type:** [More Information Needed]
-- **Hours used:** [More Information Needed]
-- **Cloud Provider:** [More Information Needed]
-- **Compute Region:** [More Information Needed]
-- **Carbon Emitted:** [More Information Needed]
-
-## Technical Specifications [optional]
-
-### Model Architecture and Objective
-
-[More Information Needed]
-
-### Compute Infrastructure
-
-[More Information Needed]
-
-#### Hardware
-
-[More Information Needed]
-
-#### Software
-
-[More Information Needed]
-
-## Citation [optional]
-
-<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
-
-**BibTeX:**
-
-[More Information Needed]
-
-**APA:**
-
-[More Information Needed]
-
-## Glossary [optional]
-
-<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
-
-[More Information Needed]
-
-## More Information [optional]
-
-[More Information Needed]
-
-## Model Card Authors [optional]
-
-[More Information Needed]
-
-## Model Card Contact
-
-[More Information Needed]
-### Framework versions
-
-- PEFT 0.18.1
\ No newline at end of file
diff --git a/llama3.2-3b/base/epochs/epoch_001/adapter/adapter_config.json b/llama3.2-3b/base/epochs/epoch_001/adapter/adapter_config.json
deleted file mode 100644
index ba8205f7c631e95368516e1477226c892ce0aa6a..0000000000000000000000000000000000000000
--- a/llama3.2-3b/base/epochs/epoch_001/adapter/adapter_config.json
+++ /dev/null
@@ -1,46 +0,0 @@
-{
-  "alora_invocation_tokens": null,
-  "alpha_pattern": {},
-  "arrow_config": null,
-  "auto_mapping": null,
-  "base_model_name_or_path": "meta-llama/Llama-3.2-3B",
-  "bias": "none",
-  "corda_config": null,
-  "ensure_weight_tying": true,
-  "eva_config": null,
-  "exclude_modules": null,
-  "fan_in_fan_out": false,
-  "inference_mode": true,
-  "init_lora_weights": true,
-  "layer_replication": null,
-  "layers_pattern": null,
-  "layers_to_transform": null,
-  "loftq_config": {},
-  "lora_alpha": 32,
-  "lora_bias": false,
-  "lora_dropout": 0.05,
-  "megatron_config": null,
-  "megatron_core": "megatron.core",
-  "modules_to_save": [
-    "lm_head",
-    "embed_tokens"
-  ],
-  "peft_type": "LORA",
-  "peft_version": "0.18.1",
-  "qalora_group_size": 16,
-  "r": 16,
-  "rank_pattern": {},
-  "revision": null,
-  "target_modules": [
-    "q_proj",
-    "o_proj",
-    "k_proj",
-    "v_proj"
-  ],
-  "target_parameters": null,
-  "task_type": "CAUSAL_LM",
-  "trainable_token_indices": null,
-  "use_dora": false,
-  "use_qalora": false,
-  "use_rslora": false
-}
\ No newline at end of file
diff --git a/llama3.2-3b/base/epochs/epoch_001/audit_results.json b/llama3.2-3b/base/epochs/epoch_001/audit_results.json
deleted file mode 100644
index f7937a4a18b5ff2fae1103bf8c97b51464b763f4..0000000000000000000000000000000000000000
--- a/llama3.2-3b/base/epochs/epoch_001/audit_results.json
+++ /dev/null
@@ -1,137 +0,0 @@
-{
-  "delta": 1e-05,
-  "num_canaries": 500,
-  "num_members": 250,
-  "paper_guess_fraction": 0.2,
-  "paper_guess_steps": 20,
-  "loss": {
-    "auc": 0.636296,
-    "empirical_epsilon": {
-      "0.05": 3.4791953936219215,
-      "0.01": 3.023197554051876
-    },
-    "empirical_epsilon_details": {
-      "0.05": {
-        "epsilon": 3.4791953936219215,
-        "num_guesses": 100,
-        "correct_guesses": 100,
-        "candidate_num_guesses": [
-          5,
-          10,
-          15,
-          20,
-          25,
-          30,
-          35,
-          40,
-          45,
-          50,
-          55,
-          60,
-          65,
-          70,
-          75,
-          80,
-          85,
-          90,
-          95,
-          100
-        ],
-        "direction": "lower"
-      },
-      "0.01": {
-        "epsilon": 3.023197554051876,
-        "num_guesses": 100,
-        "correct_guesses": 100,
-        "candidate_num_guesses": [
-          5,
-          10,
-          15,
-          20,
-          25,
-          30,
-          35,
-          40,
-          45,
-          50,
-          55,
-          60,
-          65,
-          70,
-          75,
-          80,
-          85,
-          90,
-          95,
-          100
-        ],
-        "direction": "lower"
-      }
-    }
-  },
-  "embedding": {
-    "auc": 0.548,
-    "empirical_epsilon": {
-      "0.05": 3.4791953936219215,
-      "0.01": 3.023197554051876
-    },
-    "empirical_epsilon_details": {
-      "0.05": {
-        "epsilon": 3.4791953936219215,
-        "num_guesses": 100,
-        "correct_guesses": 100,
-        "candidate_num_guesses": [
-          5,
-          10,
-          15,
-          20,
-          25,
-          30,
-          35,
-          40,
-          45,
-          50,
-          55,
-          60,
-          65,
-          70,
-          75,
-          80,
-          85,
-          90,
-          95,
-          100
-        ],
-        "direction": "lower"
-      },
-      "0.01": {
-        "epsilon": 3.023197554051876,
-        "num_guesses": 100,
-        "correct_guesses": 100,
-        "candidate_num_guesses": [
-          5,
-          10,
-          15,
-          20,
-          25,
-          30,
-          35,
-          40,
-          45,
-          50,
-          55,
-          60,
-          65,
-          70,
-          75,
-          80,
-          85,
-          90,
-          95,
-          100
-        ],
-        "direction": "lower"
-      }
-    }
-  }
-}
\ No newline at end of file
diff --git a/llama3.2-3b/base/epochs/epoch_002/adapter/README.md b/llama3.2-3b/base/epochs/epoch_002/adapter/README.md
deleted file mode 100644
index 9eb98a277210fc09f54d9dc34ff6339fae8f8173..0000000000000000000000000000000000000000
--- a/llama3.2-3b/base/epochs/epoch_002/adapter/README.md
+++ /dev/null
@@ -1,207 +0,0 @@
----
-base_model: meta-llama/Llama-3.2-3B
-library_name: peft
-pipeline_tag: text-generation
-tags:
-- base_model:adapter:meta-llama/Llama-3.2-3B
-- lora
-- transformers
----
-
-# Model Card for Model ID
-
-<!-- Provide a quick summary of what the model is/does. -->
-
-
-
-## Model Details
-
-### Model Description
-
-<!-- Provide a longer summary of what this model is. -->
-
-
-
-- **Developed by:** [More Information Needed]
-- **Funded by [optional]:** [More Information Needed]
-- **Shared by [optional]:** [More Information Needed]
-- **Model type:** [More Information Needed]
-- **Language(s) (NLP):** [More Information Needed]
-- **License:** [More Information Needed]
-- **Finetuned from model [optional]:** [More Information Needed]
-
-### Model Sources [optional]
-
-<!-- Provide the basic links for the model. -->
-
-- **Repository:** [More Information Needed]
-- **Paper [optional]:** [More Information Needed]
-- **Demo [optional]:** [More Information Needed]
-
-## Uses
-
-<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
-
-### Direct Use
-
-<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
-
-[More Information Needed]
-
-### Downstream Use [optional]
-
-<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
-
-[More Information Needed]
-
-### Out-of-Scope Use
-
-<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
-
-[More Information Needed]
-
-## Bias, Risks, and Limitations
-
-<!-- This section is meant to convey both technical and sociotechnical limitations. -->
-
-[More Information Needed]
-
-### Recommendations
-
-<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
-
-Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
-
-## How to Get Started with the Model
-
-Use the code below to get started with the model.
-
-[More Information Needed]
-
-## Training Details
-
-### Training Data
-
-<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
-
-[More Information Needed]
-
-### Training Procedure
-
-<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
-
-#### Preprocessing [optional]
-
-[More Information Needed]
-
-
-#### Training Hyperparameters
-
-- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
-
-#### Speeds, Sizes, Times [optional]
-
-<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
-
-[More Information Needed]
-
-## Evaluation
-
-<!-- This section describes the evaluation protocols and provides the results. -->
-
-### Testing Data, Factors & Metrics
-
-#### Testing Data
-
-<!-- This should link to a Dataset Card if possible. -->
-
-[More Information Needed]
-
-#### Factors
-
-<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
-
-[More Information Needed]
-
-#### Metrics
-
-<!-- These are the evaluation metrics being used, ideally with a description of why. -->
-
-[More Information Needed]
-
-### Results
-
-[More Information Needed]
-
-#### Summary
-
-
-
-## Model Examination [optional]
-
-<!-- Relevant interpretability work for the model goes here -->
-
-[More Information Needed]
-
-## Environmental Impact
-
-<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
-
-Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
-
-- **Hardware Type:** [More Information Needed]
-- **Hours used:** [More Information Needed]
-- **Cloud Provider:** [More Information Needed]
-- **Compute Region:** [More Information Needed]
-- **Carbon Emitted:** [More Information Needed]
-
-## Technical Specifications [optional]
-
-### Model Architecture and Objective
-
-[More Information Needed]
-
-### Compute Infrastructure
-
-[More Information Needed]
-
-#### Hardware
-
-[More Information Needed]
-
-#### Software
-
-[More Information Needed]
-
-## Citation [optional]
-
-<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
-
-**BibTeX:**
-
-[More Information Needed]
-
-**APA:**
-
-[More Information Needed]
-
-## Glossary [optional]
-
-<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
-
-[More Information Needed]
-
-## More Information [optional]
-
-[More Information Needed]
-
-## Model Card Authors [optional]
-
-[More Information Needed]
-
-## Model Card Contact
-
-[More Information Needed]
-### Framework versions
-
-- PEFT 0.18.1
\ No newline at end of file
diff --git a/llama3.2-3b/base/epochs/epoch_002/adapter/adapter_config.json b/llama3.2-3b/base/epochs/epoch_002/adapter/adapter_config.json
deleted file mode 100644
index ba8205f7c631e95368516e1477226c892ce0aa6a..0000000000000000000000000000000000000000
--- a/llama3.2-3b/base/epochs/epoch_002/adapter/adapter_config.json
+++ /dev/null
@@ -1,46 +0,0 @@
-{
-  "alora_invocation_tokens": null,
-  "alpha_pattern": {},
-  "arrow_config": null,
-  "auto_mapping": null,
-  "base_model_name_or_path": "meta-llama/Llama-3.2-3B",
-  "bias": "none",
-  "corda_config": null,
-  "ensure_weight_tying": true,
-  "eva_config": null,
-  "exclude_modules": null,
-  "fan_in_fan_out": false,
-  "inference_mode": true,
-  "init_lora_weights": true,
-  "layer_replication": null,
-  "layers_pattern": null,
-  "layers_to_transform": null,
-  "loftq_config": {},
-  "lora_alpha": 32,
-  "lora_bias": false,
-  "lora_dropout": 0.05,
-  "megatron_config": null,
-  "megatron_core": "megatron.core",
-  "modules_to_save": [
-    "lm_head",
-    "embed_tokens"
-  ],
-  "peft_type": "LORA",
-  "peft_version": "0.18.1",
-  "qalora_group_size": 16,
-  "r": 16,
-  "rank_pattern": {},
-  "revision": null,
-  "target_modules": [
-    "q_proj",
-    "o_proj",
-    "k_proj",
-    "v_proj"
-  ],
-  "target_parameters": null,
-  "task_type": "CAUSAL_LM",
-  "trainable_token_indices": null,
-  "use_dora": false,
-  "use_qalora": false,
-  "use_rslora": false
-}
\ No newline at end of file
diff --git a/llama3.2-3b/base/epochs/epoch_002/audit_results.json b/llama3.2-3b/base/epochs/epoch_002/audit_results.json
deleted file mode 100644
index 12ad671ce3373921f9bcc8eb41e7850a4275824f..0000000000000000000000000000000000000000
--- a/llama3.2-3b/base/epochs/epoch_002/audit_results.json
+++ /dev/null
@@ -1,137 +0,0 @@
-{
-  "delta": 1e-05,
-  "num_canaries": 500,
-  "num_members": 250,
-  "paper_guess_fraction": 0.2,
-  "paper_guess_steps": 20,
-  "loss": {
-    "auc": 0.964016,
-    "empirical_epsilon": {
-      "0.05": 3.4791953936219215,
-      "0.01": 3.023197554051876
-    },
-    "empirical_epsilon_details": {
-      "0.05": {
-        "epsilon": 3.4791953936219215,
-        "num_guesses": 100,
-        "correct_guesses": 100,
-        "candidate_num_guesses": [
-          5,
-          10,
-          15,
-          20,
-          25,
-          30,
-          35,
-          40,
-          45,
-          50,
-          55,
-          60,
-          65,
-          70,
-          75,
-          80,
-          85,
-          90,
-          95,
-          100
-        ],
-        "direction": "lower"
-      },
-      "0.01": {
-        "epsilon": 3.023197554051876,
-        "num_guesses": 100,
-        "correct_guesses": 100,
-        "candidate_num_guesses": [
-          5,
-          10,
-          15,
-          20,
-          25,
-          30,
-          35,
-          40,
-          45,
-          50,
-          55,
-          60,
-          65,
-          70,
-          75,
-          80,
-          85,
-          90,
-          95,
-          100
-        ],
-        "direction": "lower"
-      }
-    }
-  },
-  "embedding": {
-    "auc": 0.628,
-    "empirical_epsilon": {
-      "0.05": 3.4791953936219215,
-      "0.01": 3.023197554051876
-    },
-    "empirical_epsilon_details": {
-      "0.05": {
-        "epsilon": 3.4791953936219215,
-        "num_guesses": 100,
-        "correct_guesses": 100,
-        "candidate_num_guesses": [
-          5,
-          10,
-          15,
-          20,
-          25,
-          30,
-          35,
-          40,
-          45,
-          50,
-          55,
-          60,
-          65,
-          70,
-          75,
-          80,
-          85,
-          90,
-          95,
-          100
-        ],
-        "direction": "higher"
-      },
-      "0.01": {
-        "epsilon": 3.023197554051876,
-        "num_guesses": 100,
-        "correct_guesses": 100,
-        "candidate_num_guesses": [
-          5,
-          10,
-          15,
-          20,
-          25,
-          30,
-          35,
-          40,
-          45,
-          50,
-          55,
-          60,
-          65,
-          70,
-          75,
-          80,
-          85,
-          90,
-          95,
-          100
-        ],
-        "direction": "higher"
-      }
-    }
-  }
-}
\ No newline at end of file
diff --git a/llama3.2-3b/base/epochs/epoch_003/adapter/README.md b/llama3.2-3b/base/epochs/epoch_003/adapter/README.md
deleted file mode 100644
index 9eb98a277210fc09f54d9dc34ff6339fae8f8173..0000000000000000000000000000000000000000
--- a/llama3.2-3b/base/epochs/epoch_003/adapter/README.md
+++ /dev/null
@@ -1,207 +0,0 @@
----
-base_model: meta-llama/Llama-3.2-3B
-library_name: peft
-pipeline_tag: text-generation
-tags:
-- base_model:adapter:meta-llama/Llama-3.2-3B
-- lora
-- transformers
----
-
-# Model Card for Model ID
-
-<!-- Provide a quick summary of what the model is/does. -->
-
-
-
-## Model Details
-
-### Model Description
-
-<!-- Provide a longer summary of what this model is. -->
-
-
-
-- **Developed by:** [More Information Needed]
-- **Funded by [optional]:** [More Information Needed]
-- **Shared by [optional]:** [More Information Needed]
-- **Model type:** [More Information Needed]
-- **Language(s) (NLP):** [More Information Needed]
-- **License:** [More Information Needed]
-- **Finetuned from model [optional]:** [More Information Needed]
-
-### Model Sources [optional]
-
-<!-- Provide the basic links for the model. -->
-
-- **Repository:** [More Information Needed]
-- **Paper [optional]:** [More Information Needed]
-- **Demo [optional]:** [More Information Needed]
-
-## Uses
-
-<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
-
-### Direct Use
-
-<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
-
-[More Information Needed]
-
-### Downstream Use [optional]
-
-<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
-
-[More Information Needed]
-
-### Out-of-Scope Use
-
-<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
-
-[More Information Needed]
-
-## Bias, Risks, and Limitations
-
-<!-- This section is meant to convey both technical and sociotechnical limitations. -->
-
-[More Information Needed]
-
-### Recommendations
-
-<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
-
-Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
-
-## How to Get Started with the Model
-
-Use the code below to get started with the model.
-
-[More Information Needed]
-
-## Training Details
-
-### Training Data
-
-<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
-
-[More Information Needed]
-
-### Training Procedure
-
-<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
-
-#### Preprocessing [optional]
-
-[More Information Needed]
-
-
-#### Training Hyperparameters
-
-- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
-
-#### Speeds, Sizes, Times [optional]
-
-<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
-
-[More Information Needed]
-
-## Evaluation
-
-<!-- This section describes the evaluation protocols and provides the results. -->
-
-### Testing Data, Factors & Metrics
-
-#### Testing Data
-
-<!-- This should link to a Dataset Card if possible. -->
-
-[More Information Needed]
-
-#### Factors
-
-<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
-
-[More Information Needed]
-
-#### Metrics
-
-<!-- These are the evaluation metrics being used, ideally with a description of why. -->
-
-[More Information Needed]
-
-### Results
-
-[More Information Needed]
-
-#### Summary
-
-
-
-## Model Examination [optional]
-
-<!-- Relevant interpretability work for the model goes here -->
-
-[More Information Needed]
-
-## Environmental Impact
-
-<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
-
-Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
-
-- **Hardware Type:** [More Information Needed]
-- **Hours used:** [More Information Needed]
-- **Cloud Provider:** [More Information Needed]
-- **Compute Region:** [More Information Needed]
-- **Carbon Emitted:** [More Information Needed]
-
-## Technical Specifications [optional]
-
-### Model Architecture and Objective
-
-[More Information Needed]
-
-### Compute Infrastructure
-
-[More Information Needed]
-
-#### Hardware
-
-[More Information Needed]
-
-#### Software
-
-[More Information Needed]
-
-## Citation [optional]
-
-<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
-
-**BibTeX:**
-
-[More Information Needed]
-
-**APA:**
-
-[More Information Needed]
-
-## Glossary [optional]
-
-<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
-
-[More Information Needed]
-
-## More Information [optional]
-
-[More Information Needed]
-
-## Model Card Authors [optional]
-
-[More Information Needed]
-
-## Model Card Contact
-
-[More Information Needed]
-### Framework versions
-
-- PEFT 0.18.1
\ No newline at end of file
diff --git a/llama3.2-3b/base/epochs/epoch_003/adapter/adapter_config.json b/llama3.2-3b/base/epochs/epoch_003/adapter/adapter_config.json
deleted file mode 100644
index ba8205f7c631e95368516e1477226c892ce0aa6a..0000000000000000000000000000000000000000
--- a/llama3.2-3b/base/epochs/epoch_003/adapter/adapter_config.json
+++ /dev/null
@@ -1,46 +0,0 @@
-{
-  "alora_invocation_tokens": null,
-  "alpha_pattern": {},
-  "arrow_config": null,
-  "auto_mapping": null,
-  "base_model_name_or_path": "meta-llama/Llama-3.2-3B",
-  "bias": "none",
-  "corda_config": null,
-  "ensure_weight_tying": true,
-  "eva_config": null,
-  "exclude_modules": null,
-  "fan_in_fan_out": false,
-  "inference_mode": true,
-  "init_lora_weights": true,
-  "layer_replication": null,
-  "layers_pattern": null,
-  "layers_to_transform": null,
-  "loftq_config": {},
-  "lora_alpha": 32,
-  "lora_bias": false,
-  "lora_dropout": 0.05,
-  "megatron_config": null,
-  "megatron_core": "megatron.core",
-  "modules_to_save": [
-    "lm_head",
-    "embed_tokens"
-  ],
-  "peft_type": "LORA",
-  "peft_version": "0.18.1",
-  "qalora_group_size": 16,
-  "r": 16,
-  "rank_pattern": {},
-  "revision": null,
-  "target_modules": [
-    "q_proj",
-    "o_proj",
-    "k_proj",
-    "v_proj"
-  ],
-  "target_parameters": null,
-  "task_type": "CAUSAL_LM",
-  "trainable_token_indices": null,
-  "use_dora": false,
-  "use_qalora": false,
-  "use_rslora": false
-}
\ No newline at end of file
diff --git a/llama3.2-3b/base/epochs/epoch_003/audit_results.json b/llama3.2-3b/base/epochs/epoch_003/audit_results.json
deleted file mode 100644
index f914da7f383aef8a2a8eb3eb0e6b8d66a35199ec..0000000000000000000000000000000000000000
--- a/llama3.2-3b/base/epochs/epoch_003/audit_results.json
+++ /dev/null
@@ -1,137 +0,0 @@
-{
-  "delta": 1e-05,
-  "num_canaries": 500,
-  "num_members": 250,
-  "paper_guess_fraction": 0.2,
-  "paper_guess_steps": 20,
-  "loss": {
-    "auc": 0.995232,
-    "empirical_epsilon": {
-      "0.05": 3.4791953936219215,
-      "0.01": 3.023197554051876
-    },
-    "empirical_epsilon_details": {
-      "0.05": {
-        "epsilon": 3.4791953936219215,
-        "num_guesses": 100,
-        "correct_guesses": 100,
-        "candidate_num_guesses": [
-          5,
-          10,
-          15,
-          20,
-          25,
-          30,
-          35,
-          40,
-          45,
-          50,
-          55,
-          60,
-          65,
-          70,
-          75,
-          80,
-          85,
-          90,
-          95,
-          100
-        ],
-        "direction": "lower"
-      },
-      "0.01": {
-        "epsilon": 3.023197554051876,
-        "num_guesses": 100,
-        "correct_guesses": 100,
-        "candidate_num_guesses": [
-          5,
-          10,
-          15,
-          20,
-          25,
-          30,
-          35,
-          40,
-          45,
-          50,
-          55,
-          60,
-          65,
-          70,
-          75,
-          80,
-          85,
-          90,
-          95,
-          100
-        ],
-        "direction": "lower"
-      }
-    }
-  },
-  "embedding": {
-    "auc": 0.66,
-    "empirical_epsilon": {
-      "0.05": 3.4791953936219215,
-      "0.01": 3.023197554051876
-    },
-    "empirical_epsilon_details": {
-      "0.05": {
-        "epsilon": 3.4791953936219215,
-        "num_guesses": 100,
-        "correct_guesses": 100,
-        "candidate_num_guesses": [
-          5,
-          10,
-          15,
-          20,
-          25,
-          30,
-          35,
-          40,
-          45,
-          50,
-          55,
-          60,
-          65,
-          70,
-          75,
-          80,
-          85,
-          90,
-          95,
-          100
-        ],
-        "direction": "higher"
-      },
-      "0.01": {
-        "epsilon": 3.023197554051876,
-        "num_guesses": 100,
-        "correct_guesses": 100,
-        "candidate_num_guesses": [
-          5,
-          10,
-          15,
-          20,
-          25,
-          30,
-          35,
-          40,
-          45,
-          50,
-          55,
-          60,
-          65,
-          70,
-          75,
-          80,
-          85,
-          90,
-          95,
-          100
-        ],
-        "direction": "higher"
-      }
-    }
-  }
-}
\ No newline at end of file
diff --git a/llama3.2-3b/base/epochs/epoch_004/adapter/README.md b/llama3.2-3b/base/epochs/epoch_004/adapter/README.md
deleted file mode 100644
index 9eb98a277210fc09f54d9dc34ff6339fae8f8173..0000000000000000000000000000000000000000
--- a/llama3.2-3b/base/epochs/epoch_004/adapter/README.md
+++ /dev/null
@@ -1,207 +0,0 @@
----
-base_model: meta-llama/Llama-3.2-3B
-library_name: peft
-pipeline_tag: text-generation
-tags:
-- base_model:adapter:meta-llama/Llama-3.2-3B
-- lora
-- transformers
----
-
-# Model Card for Model ID
-
-<!-- Provide a quick summary of what the model is/does. -->
-
-
-
-## Model Details
-
-### Model Description
-
-<!-- Provide a longer summary of what this model is. -->
-
-
-
-- **Developed by:** [More Information Needed]
-- **Funded by [optional]:** [More Information Needed]
-- **Shared by [optional]:** [More Information Needed]
-- **Model type:** [More Information Needed]
-- **Language(s) (NLP):** [More Information Needed]
-- **License:** [More Information Needed]
-- **Finetuned from model [optional]:** [More Information Needed]
-
-### Model Sources [optional]
-
-<!-- Provide the basic links for the model. -->
-
-- **Repository:** [More Information Needed]
-- **Paper [optional]:** [More Information Needed]
-- **Demo [optional]:** [More Information Needed]
-
-## Uses
-
-<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
-
-### Direct Use
-
-<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
-
-[More Information Needed]
-
-### Downstream Use [optional]
-
-<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
-
-[More Information Needed]
-
-### Out-of-Scope Use
-
-<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
-
-[More Information Needed]
-
-## Bias, Risks, and Limitations
-
-<!-- This section is meant to convey both technical and sociotechnical limitations. -->
-
-[More Information Needed]
-
-### Recommendations
-
-<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
-
-Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
-
-## How to Get Started with the Model
-
-Use the code below to get started with the model.
-
-[More Information Needed]
-
-## Training Details
-
-### Training Data
-
-<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
-
-[More Information Needed]
-
-### Training Procedure
-
-<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
-
-#### Preprocessing [optional]
-
-[More Information Needed]
-
-
-#### Training Hyperparameters
-
-- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
-
-#### Speeds, Sizes, Times [optional]
-
-<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
-
-[More Information Needed]
-
-## Evaluation
-
-<!-- This section describes the evaluation protocols and provides the results. -->
-
-### Testing Data, Factors & Metrics
-
-#### Testing Data
-
-<!-- This should link to a Dataset Card if possible. -->
-
-[More Information Needed]
-
-#### Factors
-
-<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
-
-[More Information Needed]
-
-#### Metrics
-
-<!-- These are the evaluation metrics being used, ideally with a description of why. -->
-
-[More Information Needed]
-
-### Results
-
-[More Information Needed]
-
-#### Summary
-
-
-
-## Model Examination [optional]
-
-<!-- Relevant interpretability work for the model goes here -->
-
-[More Information Needed]
-
-## Environmental Impact
-
-<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
-
-Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
-
-- **Hardware Type:** [More Information Needed]
-- **Hours used:** [More Information Needed]
-- **Cloud Provider:** [More Information Needed]
-- **Compute Region:** [More Information Needed]
-- **Carbon Emitted:** [More Information Needed]
-
-## Technical Specifications [optional]
-
-### Model Architecture and Objective
-
-[More Information Needed]
-
-### Compute Infrastructure
-
-[More Information Needed]
-
-#### Hardware
-
-[More Information Needed]
-
-#### Software
-
-[More Information Needed]
-
-## Citation [optional]
-
-<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
-
-**BibTeX:**
-
-[More Information Needed]
-
-**APA:**
-
-[More Information Needed]
-
-## Glossary [optional]
-
-<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
-
-[More Information Needed]
-
-## More Information [optional]
-
-[More Information Needed]
-
-## Model Card Authors [optional]
-
-[More Information Needed]
-
-## Model Card Contact
-
-[More Information Needed]
-### Framework versions
-
-- PEFT 0.18.1
\ No newline at end of file
diff --git a/llama3.2-3b/base/epochs/epoch_004/adapter/adapter_config.json b/llama3.2-3b/base/epochs/epoch_004/adapter/adapter_config.json
deleted file mode 100644
index ba8205f7c631e95368516e1477226c892ce0aa6a..0000000000000000000000000000000000000000
--- a/llama3.2-3b/base/epochs/epoch_004/adapter/adapter_config.json
+++ /dev/null
@@ -1,46 +0,0 @@
-{
-  "alora_invocation_tokens": null,
-  "alpha_pattern": {},
-  "arrow_config": null,
-  "auto_mapping": null,
-  "base_model_name_or_path": "meta-llama/Llama-3.2-3B",
-  "bias": "none",
-  "corda_config": null,
-  "ensure_weight_tying": true,
-  "eva_config": null,
-  "exclude_modules": null,
-  "fan_in_fan_out": false,
-  "inference_mode": true,
-  "init_lora_weights": true,
-  "layer_replication": null,
-  "layers_pattern": null,
-  "layers_to_transform": null,
-  "loftq_config": {},
-  "lora_alpha": 32,
-  "lora_bias": false,
-  "lora_dropout": 0.05,
-  "megatron_config": null,
-  "megatron_core": "megatron.core",
-  "modules_to_save": [
-    "lm_head",
-    "embed_tokens"
-  ],
-  "peft_type": "LORA",
-  "peft_version": "0.18.1",
-  "qalora_group_size": 16,
-  "r": 16,
-  "rank_pattern": {},
-  "revision": null,
-  "target_modules": [
-    "q_proj",
-    "o_proj",
-    "k_proj",
-    "v_proj"
-  ],
-  "target_parameters": null,
-  "task_type": "CAUSAL_LM",
-  "trainable_token_indices": null,
-  "use_dora": false,
-  "use_qalora": false,
-  "use_rslora": false
-}
\ No newline at end of file
diff --git a/llama3.2-3b/base/epochs/epoch_004/audit_results.json b/llama3.2-3b/base/epochs/epoch_004/audit_results.json
deleted file mode 100644
index 723a8d06238f740d86fbf34e9a613631191930e6..0000000000000000000000000000000000000000
--- a/llama3.2-3b/base/epochs/epoch_004/audit_results.json
+++ /dev/null
@@ -1,137 +0,0 @@
-{
-  "delta": 1e-05,
-  "num_canaries": 500,
-  "num_members": 250,
-  "paper_guess_fraction": 0.2,
-  "paper_guess_steps": 20,
-  "loss": {
-    "auc": 1.0,
-    "empirical_epsilon": {
-      "0.05": 3.4791953936219215,
-      "0.01": 3.023197554051876
-    },
-    "empirical_epsilon_details": {
-      "0.05": {
-        "epsilon": 3.4791953936219215,
-        "num_guesses": 100,
-        "correct_guesses": 100,
-        "candidate_num_guesses": [
-          5,
-          10,
-          15,
-          20,
-          25,
-          30,
-          35,
-          40,
-          45,
-          50,
-          55,
-          60,
-          65,
-          70,
-          75,
-          80,
-          85,
-          90,
-          95,
-          100
-        ],
-        "direction": "lower"
-      },
-      "0.01": {
-        "epsilon": 3.023197554051876,
-        "num_guesses": 100,
-        "correct_guesses": 100,
-        "candidate_num_guesses": [
-          5,
-          10,
-          15,
-          20,
-          25,
-          30,
-          35,
-          40,
-          45,
-          50,
-          55,
-          60,
-          65,
-          70,
-          75,
-          80,
-          85,
-          90,
-          95,
-          100
-        ],
-        "direction": "lower"
-      }
-    }
-  },
-  "embedding": {
-    "auc": 0.668,
-    "empirical_epsilon": {
-      "0.05": 3.4791953936219215,
-      "0.01": 3.023197554051876
-    },
-    "empirical_epsilon_details": {
-      "0.05": {
-        "epsilon": 3.4791953936219215,
-        "num_guesses": 100,
-        "correct_guesses": 100,
-        "candidate_num_guesses": [
-          5,
-          10,
-          15,
-          20,
-          25,
-          30,
-          35,
-          40,
-          45,
-          50,
-          55,
-          60,
-          65,
-          70,
-          75,
-          80,
-          85,
-          90,
-          95,
-          100
-        ],
-        "direction": "higher"
-      },
-      "0.01": {
-        "epsilon": 3.023197554051876,
-        "num_guesses": 100,
-        "correct_guesses": 100,
-        "candidate_num_guesses": [
-          5,
-          10,
-          15,
-          20,
-          25,
-          30,
-          35,
-          40,
-          45,
-          50,
-          55,
-          60,
-          65,
-          70,
-          75,
-          80,
-          85,
-          90,
-          95,
-          100
-        ],
-        "direction": "higher"
-      }
-    }
-  }
-}
\ No newline at end of file
diff --git a/llama3.2-3b/base/epochs/epoch_005/adapter/README.md b/llama3.2-3b/base/epochs/epoch_005/adapter/README.md
deleted file mode 100644
index 9eb98a277210fc09f54d9dc34ff6339fae8f8173..0000000000000000000000000000000000000000
--- a/llama3.2-3b/base/epochs/epoch_005/adapter/README.md
+++ /dev/null
@@ -1,207 +0,0 @@
----
-base_model: meta-llama/Llama-3.2-3B
-library_name: peft
-pipeline_tag: text-generation
-tags:
-- base_model:adapter:meta-llama/Llama-3.2-3B
-- lora
-- transformers
----
-
-# Model Card for Model ID
-
-<!-- Provide a quick summary of what the model is/does. -->
-
-
-
-## Model Details
-
-### Model Description
-
-<!-- Provide a longer summary of what this model is. -->
-
-
-
-- **Developed by:** [More Information Needed]
-- **Funded by [optional]:** [More Information Needed]
-- **Shared by [optional]:** [More Information Needed]
-- **Model type:** [More Information Needed]
-- **Language(s) (NLP):** [More Information Needed]
-- **License:** [More Information Needed]
-- **Finetuned from model [optional]:** [More Information Needed]
-
-### Model Sources [optional]
-
-<!-- Provide the basic links for the model. -->
-
-- **Repository:** [More Information Needed]
-- **Paper [optional]:** [More Information Needed]
-- **Demo [optional]:** [More Information Needed]
-
-## Uses
-
-<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
-
-### Direct Use
-
-<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
-
-[More Information Needed]
-
-### Downstream Use [optional]
-
-<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
-
-[More Information Needed]
-
-### Out-of-Scope Use
-
-<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
-
-[More Information Needed]
-
-## Bias, Risks, and Limitations
-
-<!-- This section is meant to convey both technical and sociotechnical limitations. -->
-
-[More Information Needed]
-
-### Recommendations
-
-<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
-
-Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
-
-## How to Get Started with the Model
-
-Use the code below to get started with the model.
-
-[More Information Needed]
-
-## Training Details
-
-### Training Data
-
-<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
-
-[More Information Needed]
-
-### Training Procedure
-
-<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
-
-#### Preprocessing [optional]
-
-[More Information Needed]
-
-
-#### Training Hyperparameters
-
-- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
-
-#### Speeds, Sizes, Times [optional]
-
-<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
-
-[More Information Needed]
-
-## Evaluation
-
-<!-- This section describes the evaluation protocols and provides the results. -->
-
-### Testing Data, Factors & Metrics
-
-#### Testing Data
-
-<!-- This should link to a Dataset Card if possible. -->
-
-[More Information Needed]
-
-#### Factors
-
-<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
-
-[More Information Needed]
-
-#### Metrics
-
-<!-- These are the evaluation metrics being used, ideally with a description of why. -->
-
-[More Information Needed]
-
-### Results
-
-[More Information Needed]
-
-#### Summary
-
-
-
-## Model Examination [optional]
-
-<!-- Relevant interpretability work for the model goes here -->
-
-[More Information Needed]
-
-## Environmental Impact
-
-<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
-
-Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
-
-- **Hardware Type:** [More Information Needed]
-- **Hours used:** [More Information Needed]
-- **Cloud Provider:** [More Information Needed]
-- **Compute Region:** [More Information Needed]
-- **Carbon Emitted:** [More Information Needed]
-
-## Technical Specifications [optional]
-
-### Model Architecture and Objective
-
-[More Information Needed]
-
-### Compute Infrastructure
-
-[More Information Needed]
-
-#### Hardware
-
-[More Information Needed]
-
-#### Software
-
-[More Information Needed]
-
-## Citation [optional]
-
-<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
-
-**BibTeX:**
-
-[More Information Needed]
-
-**APA:**
-
-[More Information Needed]
-
-## Glossary [optional]
-
-<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
-
-[More Information Needed]
-
-## More Information [optional]
-
-[More Information Needed]
-
-## Model Card Authors [optional]
-
-[More Information Needed]
-
-## Model Card Contact
-
-[More Information Needed]
-### Framework versions
-
-- PEFT 0.18.1
\ No newline at end of file
diff --git a/llama3.2-3b/base/epochs/epoch_005/adapter/adapter_config.json b/llama3.2-3b/base/epochs/epoch_005/adapter/adapter_config.json
deleted file mode 100644
index ba8205f7c631e95368516e1477226c892ce0aa6a..0000000000000000000000000000000000000000
--- a/llama3.2-3b/base/epochs/epoch_005/adapter/adapter_config.json
+++ /dev/null
@@ -1,46 +0,0 @@
-{
-  "alora_invocation_tokens": null,
-  "alpha_pattern": {},
-  "arrow_config": null,
-  "auto_mapping": null,
-  "base_model_name_or_path": "meta-llama/Llama-3.2-3B",
-  "bias": "none",
-  "corda_config": null,
-  "ensure_weight_tying": true,
-  "eva_config": null,
-  "exclude_modules": null,
-  "fan_in_fan_out": false,
-  "inference_mode": true,
-  "init_lora_weights": true,
-  "layer_replication": null,
-  "layers_pattern": null,
-  "layers_to_transform": null,
-  "loftq_config": {},
-  "lora_alpha": 32,
-  "lora_bias": false,
-  "lora_dropout": 0.05,
-  "megatron_config": null,
-  "megatron_core": "megatron.core",
-  "modules_to_save": [
-    "lm_head",
-    "embed_tokens"
-  ],
-  "peft_type": "LORA",
-  "peft_version": "0.18.1",
-  "qalora_group_size": 16,
-  "r": 16,
-  "rank_pattern": {},
-  "revision": null,
-  "target_modules": [
-    "q_proj",
-    "o_proj",
-    "k_proj",
-    "v_proj"
-  ],
-  "target_parameters": null,
-  "task_type": "CAUSAL_LM",
-  "trainable_token_indices": null,
-  "use_dora": false,
-  "use_qalora": false,
-  "use_rslora": false
-}
\ No newline at end of file
diff --git a/llama3.2-3b/base/epochs/epoch_005/audit_results.json b/llama3.2-3b/base/epochs/epoch_005/audit_results.json
deleted file mode 100644
index 723a8d06238f740d86fbf34e9a613631191930e6..0000000000000000000000000000000000000000
--- a/llama3.2-3b/base/epochs/epoch_005/audit_results.json
+++ /dev/null
@@ -1,137 +0,0 @@
-{
-  "delta": 1e-05,
-  "num_canaries": 500,
-  "num_members": 250,
-  "paper_guess_fraction": 0.2,
-  "paper_guess_steps": 20,
-  "loss": {
-    "auc": 1.0,
-    "empirical_epsilon": {
-      "0.05": 3.4791953936219215,
-      "0.01": 3.023197554051876
-    },
-    "empirical_epsilon_details": {
-      "0.05": {
-        "epsilon": 3.4791953936219215,
-        "num_guesses": 100,
-        "correct_guesses": 100,
-        "candidate_num_guesses": [
-          5,
-          10,
-          15,
-          20,
-          25,
-          30,
-          35,
-          40,
-          45,
-          50,
-          55,
-          60,
-          65,
-          70,
-          75,
-          80,
-          85,
-          90,
-          95,
-          100
-        ],
-        "direction": "lower"
-      },
-      "0.01": {
-        "epsilon": 3.023197554051876,
-        "num_guesses": 100,
-        "correct_guesses": 100,
-        "candidate_num_guesses": [
-          5,
-          10,
-          15,
-          20,
-          25,
-          30,
-          35,
-          40,
-          45,
-          50,
-          55,
-          60,
-          65,
-          70,
-          75,
-          80,
-          85,
-          90,
-          95,
-          100
-        ],
-        "direction": "lower"
-      }
-    }
-  },
-  "embedding": {
-    "auc": 0.668,
-    "empirical_epsilon": {
-      "0.05": 3.4791953936219215,
-      "0.01": 3.023197554051876
-    },
-    "empirical_epsilon_details": {
-      "0.05": {
-        "epsilon": 3.4791953936219215,
-        "num_guesses": 100,
-        "correct_guesses": 100,
-        "candidate_num_guesses": [
-          5,
-          10,
-          15,
-          20,
-          25,
-          30,
-          35,
-          40,
-          45,
-          50,
-          55,
-          60,
-          65,
-          70,
-          75,
-          80,
-          85,
-          90,
-          95,
-          100
-        ],
-        "direction": "higher"
-      },
-      "0.01": {
-        "epsilon": 3.023197554051876,
-        "num_guesses": 100,
-        "correct_guesses": 100,
-        "candidate_num_guesses": [
-          5,
-          10,
-          15,
-          20,
-          25,
-          30,
-          35,
-          40,
-          45,
-          50,
-          55,
-          60,
-          65,
-          70,
-          75,
-          80,
-          85,
-          90,
-          95,
-          100
-        ],
-        "direction": "higher"
-      }
-    }
-  }
-}
\ No newline at end of file
diff --git a/llama3.2-3b/dp3/epochs/epoch_001/adapter/README.md b/llama3.2-3b/dp3/epochs/epoch_001/adapter/README.md
deleted file mode 100644
index 9eb98a277210fc09f54d9dc34ff6339fae8f8173..0000000000000000000000000000000000000000
--- a/llama3.2-3b/dp3/epochs/epoch_001/adapter/README.md
+++ /dev/null
@@ -1,207 +0,0 @@
----
-base_model: meta-llama/Llama-3.2-3B
-library_name: peft
-pipeline_tag: text-generation
-tags:
-- base_model:adapter:meta-llama/Llama-3.2-3B
-- lora
-- transformers
----
-
-# Model Card for Model ID
-
-<!-- Provide a quick summary of what the model is/does. -->
-
-
-
-## Model Details
-
-### Model Description
-
-<!-- Provide a longer summary of what this model is. -->
-
-
-
-- **Developed by:** [More Information Needed]
-- **Funded by [optional]:** [More Information Needed]
-- **Shared by [optional]:** [More Information Needed]
-- **Model type:** [More Information Needed]
-- **Language(s) (NLP):** [More Information Needed]
-- **License:** [More Information Needed]
-- **Finetuned from model [optional]:** [More Information Needed]
-
-### Model Sources [optional]
-
-<!-- Provide the basic links for the model. -->
-
-- **Repository:** [More Information Needed]
-- **Paper [optional]:** [More Information Needed]
-- **Demo [optional]:** [More Information Needed]
-
-## Uses
-
-<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
-
-### Direct Use
-
-<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
-
-[More Information Needed]
-
-### Downstream Use [optional]
-
-<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
-
-[More Information Needed]
-
-### Out-of-Scope Use
-
-<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
-
-[More Information Needed]
-
-## Bias, Risks, and Limitations
-
-<!-- This section is meant to convey both technical and sociotechnical limitations. -->
-
-[More Information Needed]
-
-### Recommendations
-
-<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
-
-Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
-
-## How to Get Started with the Model
-
-Use the code below to get started with the model.
-
-[More Information Needed]
-
-## Training Details
-
-### Training Data
-
-<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
-
-[More Information Needed]
-
-### Training Procedure
-
-<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
-
-#### Preprocessing [optional]
-
-[More Information Needed]
-
-
-#### Training Hyperparameters
-
-- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
-
-#### Speeds, Sizes, Times [optional]
-
-<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
-
-[More Information Needed]
-
-## Evaluation
-
-<!-- This section describes the evaluation protocols and provides the results. -->
-
-### Testing Data, Factors & Metrics
-
-#### Testing Data
-
-<!-- This should link to a Dataset Card if possible. -->
-
-[More Information Needed]
-
-#### Factors
-
-<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
-
-[More Information Needed]
-
-#### Metrics
-
-<!-- These are the evaluation metrics being used, ideally with a description of why. -->
-
-[More Information Needed]
-
-### Results
-
-[More Information Needed]
-
-#### Summary
-
-
-
-## Model Examination [optional]
-
-<!-- Relevant interpretability work for the model goes here -->
-
-[More Information Needed]
-
-## Environmental Impact
-
-<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
-
-Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
-
-- **Hardware Type:** [More Information Needed]
-- **Hours used:** [More Information Needed]
-- **Cloud Provider:** [More Information Needed]
-- **Compute Region:** [More Information Needed]
-- **Carbon Emitted:** [More Information Needed]
-
-## Technical Specifications [optional]
-
-### Model Architecture and Objective
-
-[More Information Needed]
-
-### Compute Infrastructure
-
-[More Information Needed]
-
-#### Hardware
-
-[More Information Needed]
-
-#### Software
-
-[More Information Needed]
-
-## Citation [optional]
-
-<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
-
-**BibTeX:**
-
-[More Information Needed]
-
-**APA:**
-
-[More Information Needed]
-
-## Glossary [optional]
-
-<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
-
-[More Information Needed]
-
-## More Information [optional]
-
-[More Information Needed]
-
-## Model Card Authors [optional]
-
-[More Information Needed]
-
-## Model Card Contact
-
-[More Information Needed]
-### Framework versions
-
-- PEFT 0.18.1
\ No newline at end of file
diff --git a/llama3.2-3b/dp3/epochs/epoch_001/adapter/adapter_config.json b/llama3.2-3b/dp3/epochs/epoch_001/adapter/adapter_config.json
deleted file mode 100644
index 55bf7fd558c6e4f911f6aab5463d56dda6db1134..0000000000000000000000000000000000000000
--- a/llama3.2-3b/dp3/epochs/epoch_001/adapter/adapter_config.json
+++ /dev/null
@@ -1,46 +0,0 @@
-{
-  "alora_invocation_tokens": null,
-  "alpha_pattern": {},
-  "arrow_config": null,
-  "auto_mapping": null,
-  "base_model_name_or_path": "meta-llama/Llama-3.2-3B",
-  "bias": "none",
-  "corda_config": null,
-  "ensure_weight_tying": true,
-  "eva_config": null,
-  "exclude_modules": null,
-  "fan_in_fan_out": false,
-  "inference_mode": true,
-  "init_lora_weights": true,
-  "layer_replication": null,
-  "layers_pattern": null,
-  "layers_to_transform": null,
-  "loftq_config": {},
-  "lora_alpha": 32,
-  "lora_bias": false,
-  "lora_dropout": 0.05,
-  "megatron_config": null,
-  "megatron_core": "megatron.core",
-  "modules_to_save": [
-    "lm_head",
-    "embed_tokens"
-  ],
-  "peft_type": "LORA",
-  "peft_version": "0.18.1",
-  "qalora_group_size": 16,
-  "r": 16,
-  "rank_pattern": {},
-  "revision": null,
-  "target_modules": [
-    "v_proj",
-    "k_proj",
-    "o_proj",
-    "q_proj"
-  ],
-  "target_parameters": null,
-  "task_type": "CAUSAL_LM",
-  "trainable_token_indices": null,
-  "use_dora": false,
-  "use_qalora": false,
-  "use_rslora": false
-}
\ No newline at end of file
diff --git a/llama3.2-3b/dp3/epochs/epoch_001/adapter/adapter_model.safetensors b/llama3.2-3b/dp3/epochs/epoch_001/adapter/adapter_model.safetensors
deleted file mode 100644
index 9bf379ee34e159e9cd1205866013520aa6f36999..0000000000000000000000000000000000000000
--- a/llama3.2-3b/dp3/epochs/epoch_001/adapter/adapter_model.safetensors
+++ /dev/null
@@ -1,3 +0,0 @@
-version https://git-lfs.github.com/spec/v1
-oid sha256:695dbc03dcfac44f61ba1cc7a213d5c593f6ac4b890bc3e0e292327a98aab997
-size 4783192176
diff --git a/llama3.2-3b/dp3/epochs/epoch_001/audit_results.json b/llama3.2-3b/dp3/epochs/epoch_001/audit_results.json
deleted file mode 100644
index 015be755e6a821e8f4709a5706918c9c8a0b863b..0000000000000000000000000000000000000000
--- a/llama3.2-3b/dp3/epochs/epoch_001/audit_results.json
+++ /dev/null
@@ -1,137 +0,0 @@
-{
-  "delta": 1e-05,
-  "num_canaries": 500,
-  "num_members": 250,
-  "paper_guess_fraction": 0.2,
-  "paper_guess_steps": 20,
-  "loss": {
-    "auc": 0.521064,
-    "empirical_epsilon": {
-      "0.05": 0.0,
-      "0.01": 0.0
-    },
-    "empirical_epsilon_details": {
-      "0.05": {
-        "epsilon": 0.0,
-        "num_guesses": 0,
-        "correct_guesses": 0,
-        "candidate_num_guesses": [
-          5,
-          10,
-          15,
-          20,
-          25,
-          30,
-          35,
-          40,
-          45,
-          50,
-          55,
-          60,
-          65,
-          70,
-          75,
-          80,
-          85,
-          90,
-          95,
-          100
-        ],
-        "direction": "lower"
-      },
-      "0.01": {
-        "epsilon": 0.0,
-        "num_guesses": 0,
-        "correct_guesses": 0,
-        "candidate_num_guesses": [
-          5,
-          10,
-          15,
-          20,
-          25,
-          30,
-          35,
-          40,
-          45,
-          50,
-          55,
-          60,
-          65,
-          70,
-          75,
-          80,
-          85,
-          90,
-          95,
-          100
-        ],
-        "direction": "lower"
-      }
-    }
-  },
-  "embedding": {
-    "auc": 0.516072,
-    "empirical_epsilon": {
-      "0.05": 0.0,
-      "0.01": 0.0
-    },
-    "empirical_epsilon_details": {
-      "0.05": {
-        "epsilon": 0.0,
-        "num_guesses": 0,
-        "correct_guesses": 0,
-        "candidate_num_guesses": [
-          5,
-          10,
-          15,
-          20,
-          25,
-          30,
-          35,
-          40,
-          45,
-          50,
-          55,
-          60,
-          65,
-          70,
-          75,
-          80,
-          85,
-          90,
-          95,
-          100
-        ],
-        "direction": "lower"
-      },
-      "0.01": {
-        "epsilon": 0.0,
-        "num_guesses": 0,
-        "correct_guesses": 0,
-        "candidate_num_guesses": [
-          5,
-          10,
-          15,
-          20,
-          25,
-          30,
-          35,
-          40,
-          45,
-          50,
-          55,
-          60,
-          65,
-          70,
-          75,
-          80,
-          85,
-          90,
-          95,
-          100
-        ],
-        "direction": "lower"
-      }
-    }
-  }
-}
\ No newline at end of file
diff --git a/llama3.2-3b/dp3/epochs/epoch_002/adapter/README.md b/llama3.2-3b/dp3/epochs/epoch_002/adapter/README.md
deleted file mode 100644
index 9eb98a277210fc09f54d9dc34ff6339fae8f8173..0000000000000000000000000000000000000000
--- a/llama3.2-3b/dp3/epochs/epoch_002/adapter/README.md
+++ /dev/null
@@ -1,207 +0,0 @@
----
-base_model: meta-llama/Llama-3.2-3B
-library_name: peft
-pipeline_tag: text-generation
-tags:
-- base_model:adapter:meta-llama/Llama-3.2-3B
-- lora
-- transformers
----
-
-# Model Card for Model ID
-
-<!-- Provide a quick summary of what the model is/does. -->
-
-
-
-## Model Details
-
-### Model Description
-
-<!-- Provide a longer summary of what this model is. -->
-
-
-
-- **Developed by:** [More Information Needed]
-- **Funded by [optional]:** [More Information Needed]
-- **Shared by [optional]:** [More Information Needed]
-- **Model type:** [More Information Needed]
-- **Language(s) (NLP):** [More Information Needed]
-- **License:** [More Information Needed]
-- **Finetuned from model [optional]:** [More Information Needed]
-
-### Model Sources [optional]
-
-<!-- Provide the basic links for the model. -->
-
-- **Repository:** [More Information Needed]
-- **Paper [optional]:** [More Information Needed]
-- **Demo [optional]:** [More Information Needed]
-
-## Uses
-
-<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
-
-### Direct Use
-
-<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
-
-[More Information Needed]
-
-### Downstream Use [optional]
-
-<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
-
-[More Information Needed]
-
-### Out-of-Scope Use
-
-<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
-
-[More Information Needed]
-
-## Bias, Risks, and Limitations
-
-<!-- This section is meant to convey both technical and sociotechnical limitations. -->
-
-[More Information Needed]
-
-### Recommendations
-
-<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
-
-Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
-
-## How to Get Started with the Model
-
-Use the code below to get started with the model.
-
-[More Information Needed]
-
-## Training Details
-
-### Training Data
-
-<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
-
-[More Information Needed]
-
-### Training Procedure
-
-<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
-
-#### Preprocessing [optional]
-
-[More Information Needed]
-
-
-#### Training Hyperparameters
-
-- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
-
-#### Speeds, Sizes, Times [optional]
-
-<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
-
-[More Information Needed]
-
-## Evaluation
-
-<!-- This section describes the evaluation protocols and provides the results. -->
-
-### Testing Data, Factors & Metrics
-
-#### Testing Data
-
-<!-- This should link to a Dataset Card if possible. -->
-
-[More Information Needed]
-
-#### Factors
-
-<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
-
-[More Information Needed]
-
-#### Metrics
-
-<!-- These are the evaluation metrics being used, ideally with a description of why. -->
-
-[More Information Needed]
-
-### Results
-
-[More Information Needed]
-
-#### Summary
-
-
-
-## Model Examination [optional]
-
-<!-- Relevant interpretability work for the model goes here -->
-
-[More Information Needed]
-
-## Environmental Impact
-
-<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
-
-Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
-
-- **Hardware Type:** [More Information Needed]
-- **Hours used:** [More Information Needed]
-- **Cloud Provider:** [More Information Needed]
-- **Compute Region:** [More Information Needed]
-- **Carbon Emitted:** [More Information Needed]
-
-## Technical Specifications [optional]
-
-### Model Architecture and Objective
-
-[More Information Needed]
-
-### Compute Infrastructure
-
-[More Information Needed]
-
-#### Hardware
-
-[More Information Needed]
-
-#### Software
-
-[More Information Needed]
-
-## Citation [optional]
-
-<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
-
-**BibTeX:**
-
-[More Information Needed]
-
-**APA:**
-
-[More Information Needed]
-
-## Glossary [optional]
-
-<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
-
-[More Information Needed]
-
-## More Information [optional]
-
-[More Information Needed]
-
-## Model Card Authors [optional]
-
-[More Information Needed]
-
-## Model Card Contact
-
-[More Information Needed]
-### Framework versions
-
-- PEFT 0.18.1
\ No newline at end of file
diff --git a/llama3.2-3b/dp3/epochs/epoch_002/adapter/adapter_config.json b/llama3.2-3b/dp3/epochs/epoch_002/adapter/adapter_config.json
deleted file mode 100644
index 55bf7fd558c6e4f911f6aab5463d56dda6db1134..0000000000000000000000000000000000000000
--- a/llama3.2-3b/dp3/epochs/epoch_002/adapter/adapter_config.json
+++ /dev/null
@@ -1,46 +0,0 @@
-{
-  "alora_invocation_tokens": null,
-  "alpha_pattern": {},
-  "arrow_config": null,
-  "auto_mapping": null,
-  "base_model_name_or_path": "meta-llama/Llama-3.2-3B",
-  "bias": "none",
-  "corda_config": null,
-  "ensure_weight_tying": true,
-  "eva_config": null,
-  "exclude_modules": null,
-  "fan_in_fan_out": false,
-  "inference_mode": true,
-  "init_lora_weights": true,
-  "layer_replication": null,
-  "layers_pattern": null,
-  "layers_to_transform": null,
-  "loftq_config": {},
-  "lora_alpha": 32,
-  "lora_bias": false,
-  "lora_dropout": 0.05,
-  "megatron_config": null,
-  "megatron_core": "megatron.core",
-  "modules_to_save": [
-    "lm_head",
-    "embed_tokens"
-  ],
-  "peft_type": "LORA",
-  "peft_version": "0.18.1",
-  "qalora_group_size": 16,
-  "r": 16,
-  "rank_pattern": {},
-  "revision": null,
-  "target_modules": [
-    "v_proj",
-    "k_proj",
-    "o_proj",
-    "q_proj"
-  ],
-  "target_parameters": null,
-  "task_type": "CAUSAL_LM",
-  "trainable_token_indices": null,
-  "use_dora": false,
-  "use_qalora": false,
-  "use_rslora": false
-}
\ No newline at end of file
diff --git a/llama3.2-3b/dp3/epochs/epoch_002/adapter/adapter_model.safetensors b/llama3.2-3b/dp3/epochs/epoch_002/adapter/adapter_model.safetensors
deleted file mode 100644
index 2cd016f9aa2da9f6230f334116690aa9e8e7f574..0000000000000000000000000000000000000000
--- a/llama3.2-3b/dp3/epochs/epoch_002/adapter/adapter_model.safetensors
+++ /dev/null
@@ -1,3 +0,0 @@
-version https://git-lfs.github.com/spec/v1
-oid sha256:4b4aba3b24b61bc02a88932f7d2fdd8608a589f87eda5d2b5b849adaeb70062c
-size 4783192176
diff --git a/llama3.2-3b/dp3/epochs/epoch_002/audit_results.json b/llama3.2-3b/dp3/epochs/epoch_002/audit_results.json
deleted file mode 100644
index 80fa2a71c3f6b605e7f145337d9f8dd89c92c5c9..0000000000000000000000000000000000000000
--- a/llama3.2-3b/dp3/epochs/epoch_002/audit_results.json
+++ /dev/null
@@ -1,137 +0,0 @@
-{
-  "delta": 1e-05,
-  "num_canaries": 500,
-  "num_members": 250,
-  "paper_guess_fraction": 0.2,
-  "paper_guess_steps": 20,
-  "loss": {
-    "auc": 0.540584,
-    "empirical_epsilon": {
-      "0.05": 0.0,
-      "0.01": 0.0
-    },
-    "empirical_epsilon_details": {
-      "0.05": {
-        "epsilon": 0.0,
-        "num_guesses": 0,
-        "correct_guesses": 0,
-        "candidate_num_guesses": [
-          5,
-          10,
-          15,
-          20,
-          25,
-          30,
-          35,
-          40,
-          45,
-          50,
-          55,
-          60,
-          65,
-          70,
-          75,
-          80,
-          85,
-          90,
-          95,
-          100
-        ],
-        "direction": "lower"
-      },
-      "0.01": {
-        "epsilon": 0.0,
-        "num_guesses": 0,
-        "correct_guesses": 0,
-        "candidate_num_guesses": [
-          5,
-          10,
-          15,
-          20,
-          25,
-          30,
-          35,
-          40,
-          45,
-          50,
-          55,
-          60,
-          65,
-          70,
-          75,
-          80,
-          85,
-          90,
-          95,
-          100
-        ],
-        "direction": "lower"
-      }
-    }
-  },
-  "embedding": {
-    "auc": 0.509968,
-    "empirical_epsilon": {
-      "0.05": 0.0,
-      "0.01": 0.0
-    },
-    "empirical_epsilon_details": {
-      "0.05": {
-        "epsilon": 0.0,
-        "num_guesses": 0,
-        "correct_guesses": 0,
-        "candidate_num_guesses": [
-          5,
-          10,
-          15,
-          20,
-          25,
-          30,
-          35,
-          40,
-          45,
-          50,
-          55,
-          60,
-          65,
-          70,
-          75,
-          80,
-          85,
-          90,
-          95,
-          100
-        ],
-        "direction": "lower"
-      },
-      "0.01": {
-        "epsilon": 0.0,
-        "num_guesses": 0,
-        "correct_guesses": 0,
-        "candidate_num_guesses": [
-          5,
-          10,
-          15,
-          20,
-          25,
-          30,
-          35,
-          40,
-          45,
-          50,
-          55,
-          60,
-          65,
-          70,
-          75,
-          80,
-          85,
-          90,
-          95,
-          100
-        ],
-        "direction": "lower"
-      }
-    }
-  }
-}
\ No newline at end of file
diff --git a/llama3.2-3b/dp3/epochs/epoch_003/adapter/README.md b/llama3.2-3b/dp3/epochs/epoch_003/adapter/README.md
deleted file mode 100644
index 9eb98a277210fc09f54d9dc34ff6339fae8f8173..0000000000000000000000000000000000000000
--- a/llama3.2-3b/dp3/epochs/epoch_003/adapter/README.md
+++ /dev/null
@@ -1,207 +0,0 @@
----
-base_model: meta-llama/Llama-3.2-3B
-library_name: peft
-pipeline_tag: text-generation
-tags:
-- base_model:adapter:meta-llama/Llama-3.2-3B
-- lora
-- transformers
----
-
-# Model Card for Model ID
-
-<!-- Provide a quick summary of what the model is/does. -->
-
-
-
-## Model Details
-
-### Model Description
-
-<!-- Provide a longer summary of what this model is. -->
-
-
-
-- **Developed by:** [More Information Needed]
-- **Funded by [optional]:** [More Information Needed]
-- **Shared by [optional]:** [More Information Needed]
-- **Model type:** [More Information Needed]
-- **Language(s) (NLP):** [More Information Needed]
-- **License:** [More Information Needed]
-- **Finetuned from model [optional]:** [More Information Needed]
-
-### Model Sources [optional]
-
-<!-- Provide the basic links for the model. -->
-
-- **Repository:** [More Information Needed]
-- **Paper [optional]:** [More Information Needed]
-- **Demo [optional]:** [More Information Needed]
-
-## Uses
-
-<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
-
-### Direct Use
-
-<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
-
-[More Information Needed]
-
-### Downstream Use [optional]
-
-<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
-
-[More Information Needed]
-
-### Out-of-Scope Use
-
-<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
-
-[More Information Needed]
-
-## Bias, Risks, and Limitations
-
-<!-- This section is meant to convey both technical and sociotechnical limitations. -->
-
-[More Information Needed]
-
-### Recommendations
-
-<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
-
-Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
-
-## How to Get Started with the Model
-
-Use the code below to get started with the model.
-
-[More Information Needed]
-
-## Training Details
-
-### Training Data
-
-<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
-
-[More Information Needed]
-
-### Training Procedure
-
-<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
-
-#### Preprocessing [optional]
-
-[More Information Needed]
-
-
-#### Training Hyperparameters
-
-- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
-
-#### Speeds, Sizes, Times [optional]
-
-<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
-
-[More Information Needed]
-
-## Evaluation
-
-<!-- This section describes the evaluation protocols and provides the results. -->
-
-### Testing Data, Factors & Metrics
-
-#### Testing Data
-
-<!-- This should link to a Dataset Card if possible. -->
-
-[More Information Needed]
-
-#### Factors
-
-<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
-
-[More Information Needed]
-
-#### Metrics
-
-<!-- These are the evaluation metrics being used, ideally with a description of why. -->
-
-[More Information Needed]
-
-### Results
-
-[More Information Needed]
-
-#### Summary
-
-
-
-## Model Examination [optional]
-
-<!-- Relevant interpretability work for the model goes here -->
-
-[More Information Needed]
-
-## Environmental Impact
-
-<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
-
-Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
-
-- **Hardware Type:** [More Information Needed]
-- **Hours used:** [More Information Needed]
-- **Cloud Provider:** [More Information Needed]
-- **Compute Region:** [More Information Needed]
-- **Carbon Emitted:** [More Information Needed]
-
-## Technical Specifications [optional]
-
-### Model Architecture and Objective
-
-[More Information Needed]
-
-### Compute Infrastructure
-
-[More Information Needed]
-
-#### Hardware
-
-[More Information Needed]
-
-#### Software
-
-[More Information Needed]
-
-## Citation [optional]
-
-<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
-
-**BibTeX:**
-
-[More Information Needed]
-
-**APA:**
-
-[More Information Needed]
-
-## Glossary [optional]
-
-<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
-
-[More Information Needed]
-
-## More Information [optional]
-
-[More Information Needed]
-
-## Model Card Authors [optional]
-
-[More Information Needed]
-
-## Model Card Contact
-
-[More Information Needed]
-### Framework versions
-
-- PEFT 0.18.1
\ No newline at end of file
diff --git a/llama3.2-3b/dp3/epochs/epoch_003/adapter/adapter_config.json b/llama3.2-3b/dp3/epochs/epoch_003/adapter/adapter_config.json
deleted file mode 100644
index 55bf7fd558c6e4f911f6aab5463d56dda6db1134..0000000000000000000000000000000000000000
--- a/llama3.2-3b/dp3/epochs/epoch_003/adapter/adapter_config.json
+++ /dev/null
@@ -1,46 +0,0 @@
-{
-  "alora_invocation_tokens": null,
-  "alpha_pattern": {},
-  "arrow_config": null,
-  "auto_mapping": null,
-  "base_model_name_or_path": "meta-llama/Llama-3.2-3B",
-  "bias": "none",
-  "corda_config": null,
-  "ensure_weight_tying": true,
-  "eva_config": null,
-  "exclude_modules": null,
-  "fan_in_fan_out": false,
-  "inference_mode": true,
-  "init_lora_weights": true,
-  "layer_replication": null,
-  "layers_pattern": null,
-  "layers_to_transform": null,
-  "loftq_config": {},
-  "lora_alpha": 32,
-  "lora_bias": false,
-  "lora_dropout": 0.05,
-  "megatron_config": null,
-  "megatron_core": "megatron.core",
-  "modules_to_save": [
-    "lm_head",
-    "embed_tokens"
-  ],
-  "peft_type": "LORA",
-  "peft_version": "0.18.1",
-  "qalora_group_size": 16,
-  "r": 16,
-  "rank_pattern": {},
-  "revision": null,
-  "target_modules": [
-    "v_proj",
-    "k_proj",
-    "o_proj",
-    "q_proj"
-  ],
-  "target_parameters": null,
-  "task_type": "CAUSAL_LM",
-  "trainable_token_indices": null,
-  "use_dora": false,
-  "use_qalora": false,
-  "use_rslora": false
-}
\ No newline at end of file
diff --git a/llama3.2-3b/dp3/epochs/epoch_003/adapter/adapter_model.safetensors b/llama3.2-3b/dp3/epochs/epoch_003/adapter/adapter_model.safetensors
deleted file mode 100644
index 20bbaef1effdfb4b6b6826207584c87be0a753ed..0000000000000000000000000000000000000000
--- a/llama3.2-3b/dp3/epochs/epoch_003/adapter/adapter_model.safetensors
+++ /dev/null
@@ -1,3 +0,0 @@
-version https://git-lfs.github.com/spec/v1
-oid sha256:053fde9af0bb45dfc5addc33a31a58824bae1b15d1a205d7bb832c9d21e1d834
-size 4783192176
diff --git a/llama3.2-3b/dp3/epochs/epoch_003/audit_results.json b/llama3.2-3b/dp3/epochs/epoch_003/audit_results.json
deleted file mode 100644
index 3f2cdc4c3e6622d3960dcb9fbafc9a04f0f6eab6..0000000000000000000000000000000000000000
--- a/llama3.2-3b/dp3/epochs/epoch_003/audit_results.json
+++ /dev/null
@@ -1,137 +0,0 @@
-{
-  "delta": 1e-05,
-  "num_canaries": 500,
-  "num_members": 250,
-  "paper_guess_fraction": 0.2,
-  "paper_guess_steps": 20,
-  "loss": {
-    "auc": 0.5514,
-    "empirical_epsilon": {
-      "0.05": 0.0,
-      "0.01": 0.0
-    },
-    "empirical_epsilon_details": {
-      "0.05": {
-        "epsilon": 0.0,
-        "num_guesses": 0,
-        "correct_guesses": 0,
-        "candidate_num_guesses": [
-          5,
-          10,
-          15,
-          20,
-          25,
-          30,
-          35,
-          40,
-          45,
-          50,
-          55,
-          60,
-          65,
-          70,
-          75,
-          80,
-          85,
-          90,
-          95,
-          100
-        ],
-        "direction": "lower"
-      },
-      "0.01": {
-        "epsilon": 0.0,
-        "num_guesses": 0,
-        "correct_guesses": 0,
-        "candidate_num_guesses": [
-          5,
-          10,
-          15,
-          20,
-          25,
-          30,
-          35,
-          40,
-          45,
-          50,
-          55,
-          60,
-          65,
-          70,
-          75,
-          80,
-          85,
-          90,
-          95,
-          100
-        ],
-        "direction": "lower"
-      }
-    }
-  },
-  "embedding": {
-    "auc": 0.501088,
-    "empirical_epsilon": {
-      "0.05": 0.0,
-      "0.01": 0.0
-    },
-    "empirical_epsilon_details": {
-      "0.05": {
-        "epsilon": 0.0,
-        "num_guesses": 0,
-        "correct_guesses": 0,
-        "candidate_num_guesses": [
-          5,
-          10,
-          15,
-          20,
-          25,
-          30,
-          35,
-          40,
-          45,
-          50,
-          55,
-          60,
-          65,
-          70,
-          75,
-          80,
-          85,
-          90,
-          95,
-          100
-        ],
-        "direction": "lower"
-      },
-      "0.01": {
-        "epsilon": 0.0,
-        "num_guesses": 0,
-        "correct_guesses": 0,
-        "candidate_num_guesses": [
-          5,
-          10,
-          15,
-          20,
-          25,
-          30,
-          35,
-          40,
-          45,
-          50,
-          55,
-          60,
-          65,
-          70,
-          75,
-          80,
-          85,
-          90,
-          95,
-          100
-        ],
-        "direction": "lower"
-      }
-    }
-  }
-}
\ No newline at end of file
diff --git a/llama3.2-3b/dp3/epochs/epoch_004/adapter/README.md b/llama3.2-3b/dp3/epochs/epoch_004/adapter/README.md
deleted file mode 100644
index 9eb98a277210fc09f54d9dc34ff6339fae8f8173..0000000000000000000000000000000000000000
--- a/llama3.2-3b/dp3/epochs/epoch_004/adapter/README.md
+++ /dev/null
@@ -1,207 +0,0 @@
----
-base_model: meta-llama/Llama-3.2-3B
-library_name: peft
-pipeline_tag: text-generation
-tags:
-- base_model:adapter:meta-llama/Llama-3.2-3B
-- lora
-- transformers
----
-
-# Model Card for Model ID
-
-<!-- Provide a quick summary of what the model is/does. -->
-
-
-
-## Model Details
-
-### Model Description
-
-<!-- Provide a longer summary of what this model is. -->
-
-
-
-- **Developed by:** [More Information Needed]
-- **Funded by [optional]:** [More Information Needed]
-- **Shared by [optional]:** [More Information Needed]
-- **Model type:** [More Information Needed]
-- **Language(s) (NLP):** [More Information Needed]
-- **License:** [More Information Needed]
-- **Finetuned from model [optional]:** [More Information Needed]
-
-### Model Sources [optional]
-
-<!-- Provide the basic links for the model. -->
-
-- **Repository:** [More Information Needed]
-- **Paper [optional]:** [More Information Needed]
-- **Demo [optional]:** [More Information Needed]
-
-## Uses
-
-<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
-
-### Direct Use
-
-<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
-
-[More Information Needed]
-
-### Downstream Use [optional]
-
-<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
-
-[More Information Needed]
-
-### Out-of-Scope Use
-
-<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
-
-[More Information Needed]
-
-## Bias, Risks, and Limitations
-
-<!-- This section is meant to convey both technical and sociotechnical limitations. -->
-
-[More Information Needed]
-
-### Recommendations
-
-<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
-
-Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
-
-## How to Get Started with the Model
-
-Use the code below to get started with the model.
-
-[More Information Needed]
-
-## Training Details
-
-### Training Data
-
-<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
-
-[More Information Needed]
-
-### Training Procedure
-
-<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
-
-#### Preprocessing [optional]
-
-[More Information Needed]
-
-
-#### Training Hyperparameters
-
-- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
-
-#### Speeds, Sizes, Times [optional]
-
-<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
-
-[More Information Needed]
-
-## Evaluation
-
-<!-- This section describes the evaluation protocols and provides the results. -->
-
-### Testing Data, Factors & Metrics
-
-#### Testing Data
-
-<!-- This should link to a Dataset Card if possible. -->
-
-[More Information Needed]
-
-#### Factors
-
-<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
-
-[More Information Needed]
-
-#### Metrics
-
-<!-- These are the evaluation metrics being used, ideally with a description of why. -->
-
-[More Information Needed]
-
-### Results
-
-[More Information Needed]
-
-#### Summary
-
-
-
-## Model Examination [optional]
-
-<!-- Relevant interpretability work for the model goes here -->
-
-[More Information Needed]
-
-## Environmental Impact
-
-<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
-
-Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
-
-- **Hardware Type:** [More Information Needed]
-- **Hours used:** [More Information Needed]
-- **Cloud Provider:** [More Information Needed]
-- **Compute Region:** [More Information Needed]
-- **Carbon Emitted:** [More Information Needed]
-
-## Technical Specifications [optional]
-
-### Model Architecture and Objective
-
-[More Information Needed]
-
-### Compute Infrastructure
-
-[More Information Needed]
-
-#### Hardware
-
-[More Information Needed]
-
-#### Software
-
-[More Information Needed]
-
-## Citation [optional]
-
-<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
-
-**BibTeX:**
-
-[More Information Needed]
-
-**APA:**
-
-[More Information Needed]
-
-## Glossary [optional]
-
-<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
-
-[More Information Needed]
-
-## More Information [optional]
-
-[More Information Needed]
-
-## Model Card Authors [optional]
-
-[More Information Needed]
-
-## Model Card Contact
-
-[More Information Needed]
-### Framework versions
-
-- PEFT 0.18.1
\ No newline at end of file
diff --git a/llama3.2-3b/dp3/epochs/epoch_004/adapter/adapter_config.json b/llama3.2-3b/dp3/epochs/epoch_004/adapter/adapter_config.json
deleted file mode 100644
index 55bf7fd558c6e4f911f6aab5463d56dda6db1134..0000000000000000000000000000000000000000
--- a/llama3.2-3b/dp3/epochs/epoch_004/adapter/adapter_config.json
+++ /dev/null
@@ -1,46 +0,0 @@
-{
-  "alora_invocation_tokens": null,
-  "alpha_pattern": {},
-  "arrow_config": null,
-  "auto_mapping": null,
-  "base_model_name_or_path": "meta-llama/Llama-3.2-3B",
-  "bias": "none",
-  "corda_config": null,
-  "ensure_weight_tying": true,
-  "eva_config": null,
-  "exclude_modules": null,
-  "fan_in_fan_out": false,
-  "inference_mode": true,
-  "init_lora_weights": true,
-  "layer_replication": null,
-  "layers_pattern": null,
-  "layers_to_transform": null,
-  "loftq_config": {},
-  "lora_alpha": 32,
-  "lora_bias": false,
-  "lora_dropout": 0.05,
-  "megatron_config": null,
-  "megatron_core": "megatron.core",
-  "modules_to_save": [
-    "lm_head",
-    "embed_tokens"
-  ],
-  "peft_type": "LORA",
-  "peft_version": "0.18.1",
-  "qalora_group_size": 16,
-  "r": 16,
-  "rank_pattern": {},
-  "revision": null,
-  "target_modules": [
-    "v_proj",
-    "k_proj",
-    "o_proj",
-    "q_proj"
-  ],
-  "target_parameters": null,
-  "task_type": "CAUSAL_LM",
-  "trainable_token_indices": null,
-  "use_dora": false,
-  "use_qalora": false,
-  "use_rslora": false
-}
\ No newline at end of file
diff --git a/llama3.2-3b/dp3/epochs/epoch_004/audit_results.json b/llama3.2-3b/dp3/epochs/epoch_004/audit_results.json
deleted file mode 100644
index 50c16928e00a83975448261cd513a8110c136d92..0000000000000000000000000000000000000000
--- a/llama3.2-3b/dp3/epochs/epoch_004/audit_results.json
+++ /dev/null
@@ -1,137 +0,0 @@
-{
-  "delta": 1e-05,
-  "num_canaries": 500,
-  "num_members": 250,
-  "paper_guess_fraction": 0.2,
-  "paper_guess_steps": 20,
-  "loss": {
-    "auc": 0.5616,
-    "empirical_epsilon": {
-      "0.05": 0.0,
-      "0.01": 0.0
-    },
-    "empirical_epsilon_details": {
-      "0.05": {
-        "epsilon": 0.0,
-        "num_guesses": 0,
-        "correct_guesses": 0,
-        "candidate_num_guesses": [
-          5,
-          10,
-          15,
-          20,
-          25,
-          30,
-          35,
-          40,
-          45,
-          50,
-          55,
-          60,
-          65,
-          70,
-          75,
-          80,
-          85,
-          90,
-          95,
-          100
-        ],
-        "direction": "lower"
-      },
-      "0.01": {
-        "epsilon": 0.0,
-        "num_guesses": 0,
-        "correct_guesses": 0,
-        "candidate_num_guesses": [
-          5,
-          10,
-          15,
-          20,
-          25,
-          30,
-          35,
-          40,
-          45,
-          50,
-          55,
-          60,
-          65,
-          70,
-          75,
-          80,
-          85,
-          90,
-          95,
-          100
-        ],
-        "direction": "lower"
-      }
-    }
-  },
-  "embedding": {
-    "auc": 0.507376,
-    "empirical_epsilon": {
-      "0.05": 0.0,
-      "0.01": 0.0
-    },
-    "empirical_epsilon_details": {
-      "0.05": {
-        "epsilon": 0.0,
-        "num_guesses": 0,
-        "correct_guesses": 0,
-        "candidate_num_guesses": [
-          5,
-          10,
-          15,
-          20,
-          25,
-          30,
-          35,
-          40,
-          45,
-          50,
-          55,
-          60,
-          65,
-          70,
-          75,
-          80,
-          85,
-          90,
-          95,
-          100
-        ],
-        "direction": "lower"
-      },
-      "0.01": {
-        "epsilon": 0.0,
-        "num_guesses": 0,
-        "correct_guesses": 0,
-        "candidate_num_guesses": [
-          5,
-          10,
-          15,
-          20,
-          25,
-          30,
-          35,
-          40,
-          45,
-          50,
-          55,
-          60,
-          65,
-          70,
-          75,
-          80,
-          85,
-          90,
-          95,
-          100
-        ],
-        "direction": "lower"
-      }
-    }
-  }
-}
\ No newline at end of file
diff --git a/llama3.2-3b/dp3/epochs/epoch_005/adapter/README.md b/llama3.2-3b/dp3/epochs/epoch_005/adapter/README.md
deleted file mode 100644
index 9eb98a277210fc09f54d9dc34ff6339fae8f8173..0000000000000000000000000000000000000000
--- a/llama3.2-3b/dp3/epochs/epoch_005/adapter/README.md
+++ /dev/null
@@ -1,207 +0,0 @@
----
-base_model: meta-llama/Llama-3.2-3B
-library_name: peft
-pipeline_tag: text-generation
-tags:
-- base_model:adapter:meta-llama/Llama-3.2-3B
-- lora
-- transformers
----
-
-# Model Card for Model ID
-
-<!-- Provide a quick summary of what the model is/does. -->
-
-
-
-## Model Details
-
-### Model Description
-
-<!-- Provide a longer summary of what this model is. -->
-
-
-
-- **Developed by:** [More Information Needed]
-- **Funded by [optional]:** [More Information Needed]
-- **Shared by [optional]:** [More Information Needed]
-- **Model type:** [More Information Needed]
-- **Language(s) (NLP):** [More Information Needed]
-- **License:** [More Information Needed]
-- **Finetuned from model [optional]:** [More Information Needed]
-
-### Model Sources [optional]
-
-<!-- Provide the basic links for the model. -->
-
-- **Repository:** [More Information Needed]
-- **Paper [optional]:** [More Information Needed]
-- **Demo [optional]:** [More Information Needed]
-
-## Uses
-
-<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
-
-### Direct Use
-
-<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
-
-[More Information Needed]
-
-### Downstream Use [optional]
-
-<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
-
-[More Information Needed]
-
-### Out-of-Scope Use
-
-<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
-
-[More Information Needed]
-
-## Bias, Risks, and Limitations
-
-<!-- This section is meant to convey both technical and sociotechnical limitations. -->
-
-[More Information Needed]
-
-### Recommendations
-
-<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
-
-Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
-
-## How to Get Started with the Model
-
-Use the code below to get started with the model.
-
-[More Information Needed]
-
-## Training Details
-
-### Training Data
-
-<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
-
-[More Information Needed]
-
-### Training Procedure
-
-<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
-
-#### Preprocessing [optional]
-
-[More Information Needed]
-
-
-#### Training Hyperparameters
-
-- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
-
-#### Speeds, Sizes, Times [optional]
-
-<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
-
-[More Information Needed]
-
-## Evaluation
-
-<!-- This section describes the evaluation protocols and provides the results. -->
-
-### Testing Data, Factors & Metrics
-
-#### Testing Data
-
-<!-- This should link to a Dataset Card if possible. -->
-
-[More Information Needed]
-
-#### Factors
-
-<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
-
-[More Information Needed]
-
-#### Metrics
-
-<!-- These are the evaluation metrics being used, ideally with a description of why. -->
-
-[More Information Needed]
-
-### Results
-
-[More Information Needed]
-
-#### Summary
-
-
-
-## Model Examination [optional]
-
-<!-- Relevant interpretability work for the model goes here -->
-
-[More Information Needed]
-
-## Environmental Impact
-
-<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
-
-Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
-
-- **Hardware Type:** [More Information Needed]
-- **Hours used:** [More Information Needed]
-- **Cloud Provider:** [More Information Needed]
-- **Compute Region:** [More Information Needed]
-- **Carbon Emitted:** [More Information Needed]
-
-## Technical Specifications [optional]
-
-### Model Architecture and Objective
-
-[More Information Needed]
-
-### Compute Infrastructure
-
-[More Information Needed]
-
-#### Hardware
-
-[More Information Needed]
-
-#### Software
-
-[More Information Needed]
-
-## Citation [optional]
-
-<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
-
-**BibTeX:**
-
-[More Information Needed]
-
-**APA:**
-
-[More Information Needed]
-
-## Glossary [optional]
-
-<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
-
-[More Information Needed]
-
-## More Information [optional]
-
-[More Information Needed]
-
-## Model Card Authors [optional]
-
-[More Information Needed]
-
-## Model Card Contact
-
-[More Information Needed]
-### Framework versions
-
-- PEFT 0.18.1
\ No newline at end of file
diff --git a/llama3.2-3b/dp3/epochs/epoch_005/adapter/adapter_config.json b/llama3.2-3b/dp3/epochs/epoch_005/adapter/adapter_config.json
deleted file mode 100644
index 55bf7fd558c6e4f911f6aab5463d56dda6db1134..0000000000000000000000000000000000000000
--- a/llama3.2-3b/dp3/epochs/epoch_005/adapter/adapter_config.json
+++ /dev/null
@@ -1,46 +0,0 @@
-{
-  "alora_invocation_tokens": null,
-  "alpha_pattern": {},
-  "arrow_config": null,
-  "auto_mapping": null,
-  "base_model_name_or_path": "meta-llama/Llama-3.2-3B",
-  "bias": "none",
-  "corda_config": null,
-  "ensure_weight_tying": true,
-  "eva_config": null,
-  "exclude_modules": null,
-  "fan_in_fan_out": false,
-  "inference_mode": true,
-  "init_lora_weights": true,
-  "layer_replication": null,
-  "layers_pattern": null,
-  "layers_to_transform": null,
-  "loftq_config": {},
-  "lora_alpha": 32,
-  "lora_bias": false,
-  "lora_dropout": 0.05,
-  "megatron_config": null,
-  "megatron_core": "megatron.core",
-  "modules_to_save": [
-    "lm_head",
-    "embed_tokens"
-  ],
-  "peft_type": "LORA",
-  "peft_version": "0.18.1",
-  "qalora_group_size": 16,
-  "r": 16,
-  "rank_pattern": {},
-  "revision": null,
-  "target_modules": [
-    "v_proj",
-    "k_proj",
-    "o_proj",
-    "q_proj"
-  ],
-  "target_parameters": null,
-  "task_type": "CAUSAL_LM",
-  "trainable_token_indices": null,
-  "use_dora": false,
-  "use_qalora": false,
-  "use_rslora": false
-}
\ No newline at end of file
diff --git a/llama3.2-3b/dp3/epochs/epoch_005/adapter/adapter_model.safetensors b/llama3.2-3b/dp3/epochs/epoch_005/adapter/adapter_model.safetensors
deleted file mode 100644
index c24928424e01d1fc9dd19bcdbe2c15e542087026..0000000000000000000000000000000000000000
--- a/llama3.2-3b/dp3/epochs/epoch_005/adapter/adapter_model.safetensors
+++ /dev/null
@@ -1,3 +0,0 @@
-version https://git-lfs.github.com/spec/v1
-oid sha256:da217e3f13e0ea9d2cd26d83216741f05817ab085970ab9c18cabf6f6756dc4c
-size 4783192176
diff --git a/llama3.2-3b/dp3/epochs/epoch_005/audit_results.json b/llama3.2-3b/dp3/epochs/epoch_005/audit_results.json
deleted file mode 100644
index c7edd62279a598d9b7fc72b3b17f2baf2184b976..0000000000000000000000000000000000000000
--- a/llama3.2-3b/dp3/epochs/epoch_005/audit_results.json
+++ /dev/null
@@ -1,137 +0,0 @@
-{
-  "delta": 1e-05,
-  "num_canaries": 500,
-  "num_members": 250,
-  "paper_guess_fraction": 0.2,
-  "paper_guess_steps": 20,
-  "loss": {
-    "auc": 0.5616,
-    "empirical_epsilon": {
-      "0.05": 0.0,
-      "0.01": 0.0
-    },
-    "empirical_epsilon_details": {
-      "0.05": {
-        "epsilon": 0.0,
-        "num_guesses": 0,
-        "correct_guesses": 0,
-        "candidate_num_guesses": [
-          5,
-          10,
-          15,
-          20,
-          25,
-          30,
-          35,
-          40,
-          45,
-          50,
-          55,
-          60,
-          65,
-          70,
-          75,
-          80,
-          85,
-          90,
-          95,
-          100
-        ],
-        "direction": "lower"
-      },
-      "0.01": {
-        "epsilon": 0.0,
-        "num_guesses": 0,
-        "correct_guesses": 0,
-        "candidate_num_guesses": [
-          5,
-          10,
-          15,
-          20,
-          25,
-          30,
-          35,
-          40,
-          45,
-          50,
-          55,
-          60,
-          65,
-          70,
-          75,
-          80,
-          85,
-          90,
-          95,
-          100
-        ],
-        "direction": "lower"
-      }
-    }
-  },
-  "embedding": {
-    "auc": 0.508464,
-    "empirical_epsilon": {
-      "0.05": 0.0,
-      "0.01": 0.0
-    },
-    "empirical_epsilon_details": {
-      "0.05": {
-        "epsilon": 0.0,
-        "num_guesses": 0,
-        "correct_guesses": 0,
-        "candidate_num_guesses": [
-          5,
-          10,
-          15,
-          20,
-          25,
-          30,
-          35,
-          40,
-          45,
-          50,
-          55,
-          60,
-          65,
-          70,
-          75,
-          80,
-          85,
-          90,
-          95,
-          100
-        ],
-        "direction": "lower"
-      },
-      "0.01": {
-        "epsilon": 0.0,
-        "num_guesses": 0,
-        "correct_guesses": 0,
-        "candidate_num_guesses": [
-          5,
-          10,
-          15,
-          20,
-          25,
-          30,
-          35,
-          40,
-          45,
-          50,
-          55,
-          60,
-          65,
-          70,
-          75,
-          80,
-          85,
-          90,
-          95,
-          100
-        ],
-        "direction": "lower"
-      }
-    }
-  }
-}
\ No newline at end of file
diff --git a/llama3.2-3b/dp8/epochs/epoch_001/adapter/README.md b/llama3.2-3b/dp8/epochs/epoch_001/adapter/README.md
deleted file mode 100644
index 9eb98a277210fc09f54d9dc34ff6339fae8f8173..0000000000000000000000000000000000000000
--- a/llama3.2-3b/dp8/epochs/epoch_001/adapter/README.md
+++ /dev/null
@@ -1,207 +0,0 @@
----
-base_model: meta-llama/Llama-3.2-3B
-library_name: peft
-pipeline_tag: text-generation
-tags:
-- base_model:adapter:meta-llama/Llama-3.2-3B
-- lora
-- transformers
----
-
-# Model Card for Model ID
-
-<!-- Provide a quick summary of what the model is/does. -->
-
-
-
-## Model Details
-
-### Model Description
-
-<!-- Provide a longer summary of what this model is. -->
-
-
-
-- **Developed by:** [More Information Needed]
-- **Funded by [optional]:** [More Information Needed]
-- **Shared by [optional]:** [More Information Needed]
-- **Model type:** [More Information Needed]
-- **Language(s) (NLP):** [More Information Needed]
-- **License:** [More Information Needed]
-- **Finetuned from model [optional]:** [More Information Needed]
-
-### Model Sources [optional]
-
-<!-- Provide the basic links for the model. -->
-
-- **Repository:** [More Information Needed]
-- **Paper [optional]:** [More Information Needed]
-- **Demo [optional]:** [More Information Needed]
-
-## Uses
-
-<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
-
-### Direct Use
-
-<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
-
-[More Information Needed]
-
-### Downstream Use [optional]
-
-<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
-
-[More Information Needed]
-
-### Out-of-Scope Use
-
-<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
-
-[More Information Needed]
-
-## Bias, Risks, and Limitations
-
-<!-- This section is meant to convey both technical and sociotechnical limitations. -->
-
-[More Information Needed]
-
-### Recommendations
-
-<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
-
-Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
-
-## How to Get Started with the Model
-
-Use the code below to get started with the model.
-
-[More Information Needed]
-
-## Training Details
-
-### Training Data
-
-<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
-
-[More Information Needed]
-
-### Training Procedure
-
-<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
-
-#### Preprocessing [optional]
-
-[More Information Needed]
-
-
-#### Training Hyperparameters
-
-- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
-
-#### Speeds, Sizes, Times [optional]
-
-<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
-
-[More Information Needed]
-
-## Evaluation
-
-<!-- This section describes the evaluation protocols and provides the results. -->
-
-### Testing Data, Factors & Metrics
-
-#### Testing Data
-
-<!-- This should link to a Dataset Card if possible. -->
-
-[More Information Needed]
-
-#### Factors
-
-<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
-
-[More Information Needed]
-
-#### Metrics
-
-<!-- These are the evaluation metrics being used, ideally with a description of why. -->
-
-[More Information Needed]
-
-### Results
-
-[More Information Needed]
-
-#### Summary
-
-
-
-## Model Examination [optional]
-
-<!-- Relevant interpretability work for the model goes here -->
-
-[More Information Needed]
-
-## Environmental Impact
-
-<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
-
-Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
-
-- **Hardware Type:** [More Information Needed]
-- **Hours used:** [More Information Needed]
-- **Cloud Provider:** [More Information Needed]
-- **Compute Region:** [More Information Needed]
-- **Carbon Emitted:** [More Information Needed]
-
-## Technical Specifications [optional]
-
-### Model Architecture and Objective
-
-[More Information Needed]
-
-### Compute Infrastructure
-
-[More Information Needed]
-
-#### Hardware
-
-[More Information Needed]
-
-#### Software
-
-[More Information Needed]
-
-## Citation [optional]
-
-<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
-
-**BibTeX:**
-
-[More Information Needed]
-
-**APA:**
-
-[More Information Needed]
-
-## Glossary [optional]
-
-<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
-
-[More Information Needed]
-
-## More Information [optional]
-
-[More Information Needed]
-
-## Model Card Authors [optional]
-
-[More Information Needed]
-
-## Model Card Contact
-
-[More Information Needed]
-### Framework versions
-
-- PEFT 0.18.1
\ No newline at end of file
diff --git a/llama3.2-3b/dp8/epochs/epoch_001/adapter/adapter_config.json b/llama3.2-3b/dp8/epochs/epoch_001/adapter/adapter_config.json
deleted file mode 100644
index b2c5738d2da61000eaae23fc654a9d8ff3107d5b..0000000000000000000000000000000000000000
--- a/llama3.2-3b/dp8/epochs/epoch_001/adapter/adapter_config.json
+++ /dev/null
@@ -1,46 +0,0 @@
-{
-  "alora_invocation_tokens": null,
-  "alpha_pattern": {},
-  "arrow_config": null,
-  "auto_mapping": null,
-  "base_model_name_or_path": "meta-llama/Llama-3.2-3B",
-  "bias": "none",
-  "corda_config": null,
-  "ensure_weight_tying": true,
-  "eva_config": null,
-  "exclude_modules": null,
-  "fan_in_fan_out": false,
-  "inference_mode": true,
-  "init_lora_weights": true,
-  "layer_replication": null,
-  "layers_pattern": null,
-  "layers_to_transform": null,
-  "loftq_config": {},
-  "lora_alpha": 32,
-  "lora_bias": false,
-  "lora_dropout": 0.05,
-  "megatron_config": null,
-  "megatron_core": "megatron.core",
-  "modules_to_save": [
-    "lm_head",
-    "embed_tokens"
-  ],
-  "peft_type": "LORA",
-  "peft_version": "0.18.1",
-  "qalora_group_size": 16,
-  "r": 16,
-  "rank_pattern": {},
-  "revision": null,
-  "target_modules": [
-    "k_proj",
-    "q_proj",
-    "v_proj",
-    "o_proj"
-  ],
-  "target_parameters": null,
-  "task_type": "CAUSAL_LM",
-  "trainable_token_indices": null,
-  "use_dora": false,
-  "use_qalora": false,
-  "use_rslora": false
-}
\ No newline at end of file
diff --git a/llama3.2-3b/dp8/epochs/epoch_001/adapter/adapter_model.safetensors b/llama3.2-3b/dp8/epochs/epoch_001/adapter/adapter_model.safetensors
deleted file mode 100644
index 560d79e91d798fab1ab45e267406761c72396eca..0000000000000000000000000000000000000000
--- a/llama3.2-3b/dp8/epochs/epoch_001/adapter/adapter_model.safetensors
+++ /dev/null
@@ -1,3 +0,0 @@
-version https://git-lfs.github.com/spec/v1
-oid sha256:ea077e94c161a09f3445a52e03d8a553278a3349dc619f8bd2e96cb86b9e024e
-size 4783192176
diff --git a/llama3.2-3b/dp8/epochs/epoch_001/audit_results.json b/llama3.2-3b/dp8/epochs/epoch_001/audit_results.json
deleted file mode 100644
index 81bb78b9e014bcede2534b30084f292efc028d5a..0000000000000000000000000000000000000000
--- a/llama3.2-3b/dp8/epochs/epoch_001/audit_results.json
+++ /dev/null
@@ -1,137 +0,0 @@
-{
-  "delta": 1e-05,
-  "num_canaries": 500,
-  "num_members": 250,
-  "paper_guess_fraction": 0.2,
-  "paper_guess_steps": 20,
-  "loss": {
-    "auc": 0.527072,
-    "empirical_epsilon": {
-      "0.05": 0.0,
-      "0.01": 0.0
-    },
-    "empirical_epsilon_details": {
-      "0.05": {
-        "epsilon": 0.0,
-        "num_guesses": 0,
-        "correct_guesses": 0,
-        "candidate_num_guesses": [
-          5,
-          10,
-          15,
-          20,
-          25,
-          30,
-          35,
-          40,
-          45,
-          50,
-          55,
-          60,
-          65,
-          70,
-          75,
-          80,
-          85,
-          90,
-          95,
-          100
-        ],
-        "direction": "lower"
-      },
-      "0.01": {
-        "epsilon": 0.0,
-        "num_guesses": 0,
-        "correct_guesses": 0,
-        "candidate_num_guesses": [
-          5,
-          10,
-          15,
-          20,
-          25,
-          30,
-          35,
-          40,
-          45,
-          50,
-          55,
-          60,
-          65,
-          70,
-          75,
-          80,
-          85,
-          90,
-          95,
-          100
-        ],
-        "direction": "lower"
-      }
-    }
-  },
-  "embedding": {
-    "auc": 0.514232,
-    "empirical_epsilon": {
-      "0.05": 0.0,
-      "0.01": 0.0
-    },
-    "empirical_epsilon_details": {
-      "0.05": {
-        "epsilon": 0.0,
-        "num_guesses": 0,
-        "correct_guesses": 0,
-        "candidate_num_guesses": [
-          5,
-          10,
-          15,
-          20,
-          25,
-          30,
-          35,
-          40,
-          45,
-          50,
-          55,
-          60,
-          65,
-          70,
-          75,
-          80,
-          85,
-          90,
-          95,
-          100
-        ],
-        "direction": "lower"
-      },
-      "0.01": {
-        "epsilon": 0.0,
-        "num_guesses": 0,
-        "correct_guesses": 0,
-        "candidate_num_guesses": [
-          5,
-          10,
-          15,
-          20,
-          25,
-          30,
-          35,
-          40,
-          45,
-          50,
-          55,
-          60,
-          65,
-          70,
-          75,
-          80,
-          85,
-          90,
-          95,
-          100
-        ],
-        "direction": "lower"
-      }
-    }
-  }
-}
\ No newline at end of file
diff --git a/llama3.2-3b/dp8/epochs/epoch_002/adapter/README.md b/llama3.2-3b/dp8/epochs/epoch_002/adapter/README.md
deleted file mode 100644
index 9eb98a277210fc09f54d9dc34ff6339fae8f8173..0000000000000000000000000000000000000000
--- a/llama3.2-3b/dp8/epochs/epoch_002/adapter/README.md
+++ /dev/null
@@ -1,207 +0,0 @@
----
-base_model: meta-llama/Llama-3.2-3B
-library_name: peft
-pipeline_tag: text-generation
-tags:
-- base_model:adapter:meta-llama/Llama-3.2-3B
-- lora
-- transformers
----
-
-# Model Card for Model ID
-
-<!-- Provide a quick summary of what the model is/does. -->
-
-
-
-## Model Details
-
-### Model Description
-
-<!-- Provide a longer summary of what this model is. -->
-
-
-
-- **Developed by:** [More Information Needed]
-- **Funded by [optional]:** [More Information Needed]
-- **Shared by [optional]:** [More Information Needed]
-- **Model type:** [More Information Needed]
-- **Language(s) (NLP):** [More Information Needed]
-- **License:** [More Information Needed]
-- **Finetuned from model [optional]:** [More Information Needed]
-
-### Model Sources [optional]
-
-<!-- Provide the basic links for the model. -->
-
-- **Repository:** [More Information Needed]
-- **Paper [optional]:** [More Information Needed]
-- **Demo [optional]:** [More Information Needed]
-
-## Uses
-
-<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
-
-### Direct Use
-
-<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
-
-[More Information Needed]
-
-### Downstream Use [optional]
-
-<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
-
-[More Information Needed]
-
-### Out-of-Scope Use
-
-<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
-
-[More Information Needed]
-
-## Bias, Risks, and Limitations
-
-<!-- This section is meant to convey both technical and sociotechnical limitations. -->
-
-[More Information Needed]
-
-### Recommendations
-
-<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
-
-Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
-
-## How to Get Started with the Model
-
-Use the code below to get started with the model.
-
-[More Information Needed]
-
-## Training Details
-
-### Training Data
-
-<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
-
-[More Information Needed]
-
-### Training Procedure
-
-<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
-
-#### Preprocessing [optional]
-
-[More Information Needed]
-
-
-#### Training Hyperparameters
-
-- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
-
-#### Speeds, Sizes, Times [optional]
-
-<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
-
-[More Information Needed]
-
-## Evaluation
-
-<!-- This section describes the evaluation protocols and provides the results. -->
-
-### Testing Data, Factors & Metrics
-
-#### Testing Data
-
-<!-- This should link to a Dataset Card if possible. -->
-
-[More Information Needed]
-
-#### Factors
-
-<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
-
-[More Information Needed]
-
-#### Metrics
-
-<!-- These are the evaluation metrics being used, ideally with a description of why. -->
-
-[More Information Needed]
-
-### Results
-
-[More Information Needed]
-
-#### Summary
-
-
-
-## Model Examination [optional]
-
-<!-- Relevant interpretability work for the model goes here -->
-
-[More Information Needed]
-
-## Environmental Impact
-
-<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
-
-Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
-
-- **Hardware Type:** [More Information Needed]
-- **Hours used:** [More Information Needed]
-- **Cloud Provider:** [More Information Needed]
-- **Compute Region:** [More Information Needed]
-- **Carbon Emitted:** [More Information Needed]
-
-## Technical Specifications [optional]
-
-### Model Architecture and Objective
-
-[More Information Needed]
-
-### Compute Infrastructure
-
-[More Information Needed]
-
-#### Hardware
-
-[More Information Needed]
-
-#### Software
-
-[More Information Needed]
-
-## Citation [optional]
-
-<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
-
-**BibTeX:**
-
-[More Information Needed]
-
-**APA:**
-
-[More Information Needed]
-
-## Glossary [optional]
-
-<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
-
-[More Information Needed]
-
-## More Information [optional]
-
-[More Information Needed]
-
-## Model Card Authors [optional]
-
-[More Information Needed]
-
-## Model Card Contact
-
-[More Information Needed]
-### Framework versions
-
-- PEFT 0.18.1
\ No newline at end of file
diff --git a/llama3.2-3b/dp8/epochs/epoch_002/adapter/adapter_config.json b/llama3.2-3b/dp8/epochs/epoch_002/adapter/adapter_config.json
deleted file mode 100644
index b2c5738d2da61000eaae23fc654a9d8ff3107d5b..0000000000000000000000000000000000000000
--- a/llama3.2-3b/dp8/epochs/epoch_002/adapter/adapter_config.json
+++ /dev/null
@@ -1,46 +0,0 @@
-{
-  "alora_invocation_tokens": null,
-  "alpha_pattern": {},
-  "arrow_config": null,
-  "auto_mapping": null,
-  "base_model_name_or_path": "meta-llama/Llama-3.2-3B",
-  "bias": "none",
-  "corda_config": null,
-  "ensure_weight_tying": true,
-  "eva_config": null,
-  "exclude_modules": null,
-  "fan_in_fan_out": false,
-  "inference_mode": true,
-  "init_lora_weights": true,
-  "layer_replication": null,
-  "layers_pattern": null,
-  "layers_to_transform": null,
-  "loftq_config": {},
-  "lora_alpha": 32,
-  "lora_bias": false,
-  "lora_dropout": 0.05,
-  "megatron_config": null,
-  "megatron_core": "megatron.core",
-  "modules_to_save": [
-    "lm_head",
-    "embed_tokens"
-  ],
-  "peft_type": "LORA",
-  "peft_version": "0.18.1",
-  "qalora_group_size": 16,
-  "r": 16,
-  "rank_pattern": {},
-  "revision": null,
-  "target_modules": [
-    "k_proj",
-    "q_proj",
-    "v_proj",
-    "o_proj"
-  ],
-  "target_parameters": null,
-  "task_type": "CAUSAL_LM",
-  "trainable_token_indices": null,
-  "use_dora": false,
-  "use_qalora": false,
-  "use_rslora": false
-}
\ No newline at end of file
diff --git a/llama3.2-3b/dp8/epochs/epoch_002/audit_results.json b/llama3.2-3b/dp8/epochs/epoch_002/audit_results.json
deleted file mode 100644
index d2aa5e6e78264f31acec80dfd84c60394420d389..0000000000000000000000000000000000000000
--- a/llama3.2-3b/dp8/epochs/epoch_002/audit_results.json
+++ /dev/null
@@ -1,137 +0,0 @@
-{
-  "delta": 1e-05,
-  "num_canaries": 500,
-  "num_members": 250,
-  "paper_guess_fraction": 0.2,
-  "paper_guess_steps": 20,
-  "loss": {
-    "auc": 0.56424,
-    "empirical_epsilon": {
-      "0.05": 0.0,
-      "0.01": 0.0
-    },
-    "empirical_epsilon_details": {
-      "0.05": {
-        "epsilon": 0.0,
-        "num_guesses": 0,
-        "correct_guesses": 0,
-        "candidate_num_guesses": [
-          5,
-          10,
-          15,
-          20,
-          25,
-          30,
-          35,
-          40,
-          45,
-          50,
-          55,
-          60,
-          65,
-          70,
-          75,
-          80,
-          85,
-          90,
-          95,
-          100
-        ],
-        "direction": "lower"
-      },
-      "0.01": {
-        "epsilon": 0.0,
-        "num_guesses": 0,
-        "correct_guesses": 0,
-        "candidate_num_guesses": [
-          5,
-          10,
-          15,
-          20,
-          25,
-          30,
-          35,
-          40,
-          45,
-          50,
-          55,
-          60,
-          65,
-          70,
-          75,
-          80,
-          85,
-          90,
-          95,
-          100
-        ],
-        "direction": "lower"
-      }
-    }
-  },
-  "embedding": {
-    "auc": 0.507904,
-    "empirical_epsilon": {
-      "0.05": 0.0,
-      "0.01": 0.0
-    },
-    "empirical_epsilon_details": {
-      "0.05": {
-        "epsilon": 0.0,
-        "num_guesses": 0,
-        "correct_guesses": 0,
-        "candidate_num_guesses": [
-          5,
-          10,
-          15,
-          20,
-          25,
-          30,
-          35,
-          40,
-          45,
-          50,
-          55,
-          60,
-          65,
-          70,
-          75,
-          80,
-          85,
-          90,
-          95,
-          100
-        ],
-        "direction": "lower"
-      },
-      "0.01": {
-        "epsilon": 0.0,
-        "num_guesses": 0,
-        "correct_guesses": 0,
-        "candidate_num_guesses": [
-          5,
-          10,
-          15,
-          20,
-          25,
-          30,
-          35,
-          40,
-          45,
-          50,
-          55,
-          60,
-          65,
-          70,
-          75,
-          80,
-          85,
-          90,
-          95,
-          100
-        ],
-        "direction": "lower"
-      }
-    }
-  }
-}
\ No newline at end of file
diff --git a/llama3.2-3b/dp8/epochs/epoch_003/adapter/README.md b/llama3.2-3b/dp8/epochs/epoch_003/adapter/README.md
deleted file mode 100644
index 9eb98a277210fc09f54d9dc34ff6339fae8f8173..0000000000000000000000000000000000000000
--- a/llama3.2-3b/dp8/epochs/epoch_003/adapter/README.md
+++ /dev/null
@@ -1,207 +0,0 @@
----
-base_model: meta-llama/Llama-3.2-3B
-library_name: peft
-pipeline_tag: text-generation
-tags:
-- base_model:adapter:meta-llama/Llama-3.2-3B
-- lora
-- transformers
----
-
-# Model Card for Model ID
-
-<!-- Provide a quick summary of what the model is/does. -->
-
-
-
-## Model Details
-
-### Model Description
-
-<!-- Provide a longer summary of what this model is. -->
-
-
-
-- **Developed by:** [More Information Needed]
-- **Funded by [optional]:** [More Information Needed]
-- **Shared by [optional]:** [More Information Needed]
-- **Model type:** [More Information Needed]
-- **Language(s) (NLP):** [More Information Needed]
-- **License:** [More Information Needed]
-- **Finetuned from model [optional]:** [More Information Needed]
-
-### Model Sources [optional]
-
-<!-- Provide the basic links for the model. -->
-
-- **Repository:** [More Information Needed]
-- **Paper [optional]:** [More Information Needed]
-- **Demo [optional]:** [More Information Needed]
-
-## Uses
-
-<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
-
-### Direct Use
-
-<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
-
-[More Information Needed]
-
-### Downstream Use [optional]
-
-<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
-
-[More Information Needed]
-
-### Out-of-Scope Use
-
-<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
-
-[More Information Needed]
-
-## Bias, Risks, and Limitations
-
-<!-- This section is meant to convey both technical and sociotechnical limitations. -->
-
-[More Information Needed]
-
-### Recommendations
-
-<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
-
-Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
-
-## How to Get Started with the Model
-
-Use the code below to get started with the model.
-
-[More Information Needed]
-
-## Training Details
-
-### Training Data
-
-<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
-
-[More Information Needed]
-
-### Training Procedure
-
-<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
-
-#### Preprocessing [optional]
-
-[More Information Needed]
-
-
-#### Training Hyperparameters
-
-- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
-
-#### Speeds, Sizes, Times [optional]
-
-<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
-
-[More Information Needed]
-
-## Evaluation
-
-<!-- This section describes the evaluation protocols and provides the results. -->
-
-### Testing Data, Factors & Metrics
-
-#### Testing Data
-
-<!-- This should link to a Dataset Card if possible. -->
-
-[More Information Needed]
-
-#### Factors
-
-<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
-
-[More Information Needed]
-
-#### Metrics
-
-<!-- These are the evaluation metrics being used, ideally with a description of why. -->
-
-[More Information Needed]
-
-### Results
-
-[More Information Needed]
-
-#### Summary
-
-
-
-## Model Examination [optional]
-
-<!-- Relevant interpretability work for the model goes here -->
-
-[More Information Needed]
-
-## Environmental Impact
-
-<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
-
-Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
-
-- **Hardware Type:** [More Information Needed]
-- **Hours used:** [More Information Needed]
-- **Cloud Provider:** [More Information Needed]
-- **Compute Region:** [More Information Needed]
-- **Carbon Emitted:** [More Information Needed]
-
-## Technical Specifications [optional]
-
-### Model Architecture and Objective
-
-[More Information Needed]
-
-### Compute Infrastructure
-
-[More Information Needed]
-
-#### Hardware
-
-[More Information Needed]
-
-#### Software
-
-[More Information Needed]
-
-## Citation [optional]
-
-<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
-
-**BibTeX:**
-
-[More Information Needed]
-
-**APA:**
-
-[More Information Needed]
-
-## Glossary [optional]
-
-<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
-
-[More Information Needed]
-
-## More Information [optional]
-
-[More Information Needed]
-
-## Model Card Authors [optional]
-
-[More Information Needed]
-
-## Model Card Contact
-
-[More Information Needed]
-### Framework versions
-
-- PEFT 0.18.1
\ No newline at end of file
diff --git a/llama3.2-3b/dp8/epochs/epoch_003/adapter/adapter_config.json b/llama3.2-3b/dp8/epochs/epoch_003/adapter/adapter_config.json
deleted file mode 100644
index b2c5738d2da61000eaae23fc654a9d8ff3107d5b..0000000000000000000000000000000000000000
--- a/llama3.2-3b/dp8/epochs/epoch_003/adapter/adapter_config.json
+++ /dev/null
@@ -1,46 +0,0 @@
-{
-  "alora_invocation_tokens": null,
-  "alpha_pattern": {},
-  "arrow_config": null,
-  "auto_mapping": null,
-  "base_model_name_or_path": "meta-llama/Llama-3.2-3B",
-  "bias": "none",
-  "corda_config": null,
-  "ensure_weight_tying": true,
-  "eva_config": null,
-  "exclude_modules": null,
-  "fan_in_fan_out": false,
-  "inference_mode": true,
-  "init_lora_weights": true,
-  "layer_replication": null,
-  "layers_pattern": null,
-  "layers_to_transform": null,
-  "loftq_config": {},
-  "lora_alpha": 32,
-  "lora_bias": false,
-  "lora_dropout": 0.05,
-  "megatron_config": null,
-  "megatron_core": "megatron.core",
-  "modules_to_save": [
-    "lm_head",
-    "embed_tokens"
-  ],
-  "peft_type": "LORA",
-  "peft_version": "0.18.1",
-  "qalora_group_size": 16,
-  "r": 16,
-  "rank_pattern": {},
-  "revision": null,
-  "target_modules": [
-    "k_proj",
-    "q_proj",
-    "v_proj",
-    "o_proj"
-  ],
-  "target_parameters": null,
-  "task_type": "CAUSAL_LM",
-  "trainable_token_indices": null,
-  "use_dora": false,
-  "use_qalora": false,
-  "use_rslora": false
-}
\ No newline at end of file
diff --git a/llama3.2-3b/dp8/epochs/epoch_003/audit_results.json b/llama3.2-3b/dp8/epochs/epoch_003/audit_results.json
deleted file mode 100644
index 64562d53973aa3abad2d0ba20aa8036f29147669..0000000000000000000000000000000000000000
--- a/llama3.2-3b/dp8/epochs/epoch_003/audit_results.json
+++ /dev/null
@@ -1,137 +0,0 @@
-{
-  "delta": 1e-05,
-  "num_canaries": 500,
-  "num_members": 250,
-  "paper_guess_fraction": 0.2,
-  "paper_guess_steps": 20,
-  "loss": {
-    "auc": 0.593872,
-    "empirical_epsilon": {
-      "0.05": 0.12070101220160723,
-      "0.01": 0.0
-    },
-    "empirical_epsilon_details": {
-      "0.05": {
-        "epsilon": 0.12070101220160723,
-        "num_guesses": 90,
-        "correct_guesses": 56,
-        "candidate_num_guesses": [
-          5,
-          10,
-          15,
-          20,
-          25,
-          30,
-          35,
-          40,
-          45,
-          50,
-          55,
-          60,
-          65,
-          70,
-          75,
-          80,
-          85,
-          90,
-          95,
-          100
-        ],
-        "direction": "lower"
-      },
-      "0.01": {
-        "epsilon": 0.0,
-        "num_guesses": 0,
-        "correct_guesses": 0,
-        "candidate_num_guesses": [
-          5,
-          10,
-          15,
-          20,
-          25,
-          30,
-          35,
-          40,
-          45,
-          50,
-          55,
-          60,
-          65,
-          70,
-          75,
-          80,
-          85,
-          90,
-          95,
-          100
-        ],
-        "direction": "lower"
-      }
-    }
-  },
-  "embedding": {
-    "auc": 0.503536,
-    "empirical_epsilon": {
-      "0.05": 0.0,
-      "0.01": 0.0
-    },
-    "empirical_epsilon_details": {
-      "0.05": {
-        "epsilon": 0.0,
-        "num_guesses": 0,
-        "correct_guesses": 0,
-        "candidate_num_guesses": [
-          5,
-          10,
-          15,
-          20,
-          25,
-          30,
-          35,
-          40,
-          45,
-          50,
-          55,
-          60,
-          65,
-          70,
-          75,
-          80,
-          85,
-          90,
-          95,
-          100
-        ],
-        "direction": "lower"
-      },
-      "0.01": {
-        "epsilon": 0.0,
-        "num_guesses": 0,
-        "correct_guesses": 0,
-        "candidate_num_guesses": [
-          5,
-          10,
-          15,
-          20,
-          25,
-          30,
-          35,
-          40,
-          45,
-          50,
-          55,
-          60,
-          65,
-          70,
-          75,
-          80,
-          85,
-          90,
-          95,
-          100
-        ],
-        "direction": "lower"
-      }
-    }
-  }
-}
\ No newline at end of file
diff --git a/llama3.2-3b/dp8/epochs/epoch_004/adapter/README.md b/llama3.2-3b/dp8/epochs/epoch_004/adapter/README.md
deleted file mode 100644
index 9eb98a277210fc09f54d9dc34ff6339fae8f8173..0000000000000000000000000000000000000000
--- a/llama3.2-3b/dp8/epochs/epoch_004/adapter/README.md
+++ /dev/null
@@ -1,207 +0,0 @@
----
-base_model: meta-llama/Llama-3.2-3B
-library_name: peft
-pipeline_tag: text-generation
-tags:
-- base_model:adapter:meta-llama/Llama-3.2-3B
-- lora
-- transformers
----
-
-# Model Card for Model ID
-
-<!-- Provide a quick summary of what the model is/does. -->
-
-
-
-## Model Details
-
-### Model Description
-
-<!-- Provide a longer summary of what this model is. -->
-
-
-
-- **Developed by:** [More Information Needed]
-- **Funded by [optional]:** [More Information Needed]
-- **Shared by [optional]:** [More Information Needed]
-- **Model type:** [More Information Needed]
-- **Language(s) (NLP):** [More Information Needed]
-- **License:** [More Information Needed]
-- **Finetuned from model [optional]:** [More Information Needed]
-
-### Model Sources [optional]
-
-<!-- Provide the basic links for the model. -->
-
-- **Repository:** [More Information Needed]
-- **Paper [optional]:** [More Information Needed]
-- **Demo [optional]:** [More Information Needed]
-
-## Uses
-
-<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
-
-### Direct Use
-
-<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
-
-[More Information Needed]
-
-### Downstream Use [optional]
-
-<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
-
-[More Information Needed]
-
-### Out-of-Scope Use
-
-<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
-
-[More Information Needed]
-
-## Bias, Risks, and Limitations
-
-<!-- This section is meant to convey both technical and sociotechnical limitations. -->
-
-[More Information Needed]
-
-### Recommendations
-
-<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
-
-Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
-
-## How to Get Started with the Model
-
-Use the code below to get started with the model.
-
-[More Information Needed]
-
-## Training Details
-
-### Training Data
-
-<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
-
-[More Information Needed]
-
-### Training Procedure
-
-<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
-
-#### Preprocessing [optional]
-
-[More Information Needed]
-
-
-#### Training Hyperparameters
-
-- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
-
-#### Speeds, Sizes, Times [optional]
-
-<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
-
-[More Information Needed]
-
-## Evaluation
-
-<!-- This section describes the evaluation protocols and provides the results. -->
-
-### Testing Data, Factors & Metrics
-
-#### Testing Data
-
-<!-- This should link to a Dataset Card if possible. -->
-
-[More Information Needed]
-
-#### Factors
-
-<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
-
-[More Information Needed]
-
-#### Metrics
-
-<!-- These are the evaluation metrics being used, ideally with a description of why. -->
-
-[More Information Needed]
-
-### Results
-
-[More Information Needed]
-
-#### Summary
-
-
-
-## Model Examination [optional]
-
-<!-- Relevant interpretability work for the model goes here -->
-
-[More Information Needed]
-
-## Environmental Impact
-
-<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
-
-Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
-
-- **Hardware Type:** [More Information Needed]
-- **Hours used:** [More Information Needed]
-- **Cloud Provider:** [More Information Needed]
-- **Compute Region:** [More Information Needed]
-- **Carbon Emitted:** [More Information Needed]
-
-## Technical Specifications [optional]
-
-### Model Architecture and Objective
-
-[More Information Needed]
-
-### Compute Infrastructure
-
-[More Information Needed]
-
-#### Hardware
-
-[More Information Needed]
-
-#### Software
-
-[More Information Needed]
-
-## Citation [optional]
-
-<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
-
-**BibTeX:**
-
-[More Information Needed]
-
-**APA:**
-
-[More Information Needed]
-
-## Glossary [optional]
-
-<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
-
-[More Information Needed]
-
-## More Information [optional]
-
-[More Information Needed]
-
-## Model Card Authors [optional]
-
-[More Information Needed]
-
-## Model Card Contact
-
-[More Information Needed]
-### Framework versions
-
-- PEFT 0.18.1
\ No newline at end of file
diff --git a/llama3.2-3b/dp8/epochs/epoch_004/adapter/adapter_config.json b/llama3.2-3b/dp8/epochs/epoch_004/adapter/adapter_config.json
deleted file mode 100644
index b2c5738d2da61000eaae23fc654a9d8ff3107d5b..0000000000000000000000000000000000000000
--- a/llama3.2-3b/dp8/epochs/epoch_004/adapter/adapter_config.json
+++ /dev/null
@@ -1,46 +0,0 @@
-{
-  "alora_invocation_tokens": null,
-  "alpha_pattern": {},
-  "arrow_config": null,
-  "auto_mapping": null,
-  "base_model_name_or_path": "meta-llama/Llama-3.2-3B",
-  "bias": "none",
-  "corda_config": null,
-  "ensure_weight_tying": true,
-  "eva_config": null,
-  "exclude_modules": null,
-  "fan_in_fan_out": false,
-  "inference_mode": true,
-  "init_lora_weights": true,
-  "layer_replication": null,
-  "layers_pattern": null,
-  "layers_to_transform": null,
-  "loftq_config": {},
-  "lora_alpha": 32,
-  "lora_bias": false,
-  "lora_dropout": 0.05,
-  "megatron_config": null,
-  "megatron_core": "megatron.core",
-  "modules_to_save": [
-    "lm_head",
-    "embed_tokens"
-  ],
-  "peft_type": "LORA",
-  "peft_version": "0.18.1",
-  "qalora_group_size": 16,
-  "r": 16,
-  "rank_pattern": {},
-  "revision": null,
-  "target_modules": [
-    "k_proj",
-    "q_proj",
-    "v_proj",
-    "o_proj"
-  ],
-  "target_parameters": null,
-  "task_type": "CAUSAL_LM",
-  "trainable_token_indices": null,
-  "use_dora": false,
-  "use_qalora": false,
-  "use_rslora": false
-}
\ No newline at end of file
diff --git a/llama3.2-3b/dp8/epochs/epoch_004/audit_results.json b/llama3.2-3b/dp8/epochs/epoch_004/audit_results.json
deleted file mode 100644
index fd41d1c9274f3760c37b41625d40cd450ae7d58d..0000000000000000000000000000000000000000
--- a/llama3.2-3b/dp8/epochs/epoch_004/audit_results.json
+++ /dev/null
@@ -1,137 +0,0 @@
-{
-  "delta": 1e-05,
-  "num_canaries": 500,
-  "num_members": 250,
-  "paper_guess_fraction": 0.2,
-  "paper_guess_steps": 20,
-  "loss": {
-    "auc": 0.60444,
-    "empirical_epsilon": {
-      "0.05": 0.09125839080661535,
-      "0.01": 0.0
-    },
-    "empirical_epsilon_details": {
-      "0.05": {
-        "epsilon": 0.09125839080661535,
-        "num_guesses": 100,
-        "correct_guesses": 61,
-        "candidate_num_guesses": [
-          5,
-          10,
-          15,
-          20,
-          25,
-          30,
-          35,
-          40,
-          45,
-          50,
-          55,
-          60,
-          65,
-          70,
-          75,
-          80,
-          85,
-          90,
-          95,
-          100
-        ],
-        "direction": "lower"
-      },
-      "0.01": {
-        "epsilon": 0.0,
-        "num_guesses": 0,
-        "correct_guesses": 0,
-        "candidate_num_guesses": [
-          5,
-          10,
-          15,
-          20,
-          25,
-          30,
-          35,
-          40,
-          45,
-          50,
-          55,
-          60,
-          65,
-          70,
-          75,
-          80,
-          85,
-          90,
-          95,
-          100
-        ],
-        "direction": "lower"
-      }
-    }
-  },
-  "embedding": {
-    "auc": 0.509304,
-    "empirical_epsilon": {
-      "0.05": 0.0,
-      "0.01": 0.0
-    },
-    "empirical_epsilon_details": {
-      "0.05": {
-        "epsilon": 0.0,
-        "num_guesses": 0,
-        "correct_guesses": 0,
-        "candidate_num_guesses": [
-          5,
-          10,
-          15,
-          20,
-          25,
-          30,
-          35,
-          40,
-          45,
-          50,
-          55,
-          60,
-          65,
-          70,
-          75,
-          80,
-          85,
-          90,
-          95,
-          100
-        ],
-        "direction": "lower"
-      },
-      "0.01": {
-        "epsilon": 0.0,
-        "num_guesses": 0,
-        "correct_guesses": 0,
-        "candidate_num_guesses": [
-          5,
-          10,
-          15,
-          20,
-          25,
-          30,
-          35,
-          40,
-          45,
-          50,
-          55,
-          60,
-          65,
-          70,
-          75,
-          80,
-          85,
-          90,
-          95,
-          100
-        ],
-        "direction": "lower"
-      }
-    }
-  }
-}
\ No newline at end of file
diff --git a/llama3.2-3b/dp8/epochs/epoch_005/adapter/README.md b/llama3.2-3b/dp8/epochs/epoch_005/adapter/README.md
deleted file mode 100644
index 9eb98a277210fc09f54d9dc34ff6339fae8f8173..0000000000000000000000000000000000000000
--- a/llama3.2-3b/dp8/epochs/epoch_005/adapter/README.md
+++ /dev/null
@@ -1,207 +0,0 @@
----
-base_model: meta-llama/Llama-3.2-3B
-library_name: peft
-pipeline_tag: text-generation
-tags:
-- base_model:adapter:meta-llama/Llama-3.2-3B
-- lora
-- transformers
----
-
-# Model Card for Model ID
-
-<!-- Provide a quick summary of what the model is/does. -->
-
-
-
-## Model Details
-
-### Model Description
-
-<!-- Provide a longer summary of what this model is. -->
-
-
-
-- **Developed by:** [More Information Needed]
-- **Funded by [optional]:** [More Information Needed]
-- **Shared by [optional]:** [More Information Needed]
-- **Model type:** [More Information Needed]
-- **Language(s) (NLP):** [More Information Needed]
-- **License:** [More Information Needed]
-- **Finetuned from model [optional]:** [More Information Needed]
-
-### Model Sources [optional]
-
-<!-- Provide the basic links for the model. -->
-
-- **Repository:** [More Information Needed]
-- **Paper [optional]:** [More Information Needed]
-- **Demo [optional]:** [More Information Needed]
-
-## Uses
-
-<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
-
-### Direct Use
-
-<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
-
-[More Information Needed]
-
-### Downstream Use [optional]
-
-<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
-
-[More Information Needed]
-
-### Out-of-Scope Use
-
-<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
-
-[More Information Needed]
-
-## Bias, Risks, and Limitations
-
-<!-- This section is meant to convey both technical and sociotechnical limitations. -->
-
-[More Information Needed]
-
-### Recommendations
-
-<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
-
-Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
-
-## How to Get Started with the Model
-
-Use the code below to get started with the model.
-
-[More Information Needed]
-
-## Training Details
-
-### Training Data
-
-<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
-
-[More Information Needed]
-
-### Training Procedure
-
-<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
-
-#### Preprocessing [optional]
-
-[More Information Needed]
-
-
-#### Training Hyperparameters
-
-- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
-
-#### Speeds, Sizes, Times [optional]
-
-<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
-
-[More Information Needed]
-
-## Evaluation
-
-<!-- This section describes the evaluation protocols and provides the results. -->
-
-### Testing Data, Factors & Metrics
-
-#### Testing Data
-
-<!-- This should link to a Dataset Card if possible. -->
-
-[More Information Needed]
-
-#### Factors
-
-<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
-
-[More Information Needed]
-
-#### Metrics
-
-<!-- These are the evaluation metrics being used, ideally with a description of why. -->
-
-[More Information Needed]
-
-### Results
-
-[More Information Needed]
-
-#### Summary
-
-
-
-## Model Examination [optional]
-
-<!-- Relevant interpretability work for the model goes here -->
-
-[More Information Needed]
-
-## Environmental Impact
-
-<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
-
-Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
-
-- **Hardware Type:** [More Information Needed]
-- **Hours used:** [More Information Needed]
-- **Cloud Provider:** [More Information Needed]
-- **Compute Region:** [More Information Needed]
-- **Carbon Emitted:** [More Information Needed]
-
-## Technical Specifications [optional]
-
-### Model Architecture and Objective
-
-[More Information Needed]
-
-### Compute Infrastructure
-
-[More Information Needed]
-
-#### Hardware
-
-[More Information Needed]
-
-#### Software
-
-[More Information Needed]
-
-## Citation [optional]
-
-<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
-
-**BibTeX:**
-
-[More Information Needed]
-
-**APA:**
-
-[More Information Needed]
-
-## Glossary [optional]
-
-<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
-
-[More Information Needed]
-
-## More Information [optional]
-
-[More Information Needed]
-
-## Model Card Authors [optional]
-
-[More Information Needed]
-
-## Model Card Contact
-
-[More Information Needed]
-### Framework versions
-
-- PEFT 0.18.1
\ No newline at end of file
diff --git a/llama3.2-3b/dp8/epochs/epoch_005/adapter/adapter_config.json b/llama3.2-3b/dp8/epochs/epoch_005/adapter/adapter_config.json
deleted file mode 100644
index b2c5738d2da61000eaae23fc654a9d8ff3107d5b..0000000000000000000000000000000000000000
--- a/llama3.2-3b/dp8/epochs/epoch_005/adapter/adapter_config.json
+++ /dev/null
@@ -1,46 +0,0 @@
-{
-  "alora_invocation_tokens": null,
-  "alpha_pattern": {},
-  "arrow_config": null,
-  "auto_mapping": null,
-  "base_model_name_or_path": "meta-llama/Llama-3.2-3B",
-  "bias": "none",
-  "corda_config": null,
-  "ensure_weight_tying": true,
-  "eva_config": null,
-  "exclude_modules": null,
-  "fan_in_fan_out": false,
-  "inference_mode": true,
-  "init_lora_weights": true,
-  "layer_replication": null,
-  "layers_pattern": null,
-  "layers_to_transform": null,
-  "loftq_config": {},
-  "lora_alpha": 32,
-  "lora_bias": false,
-  "lora_dropout": 0.05,
-  "megatron_config": null,
-  "megatron_core": "megatron.core",
-  "modules_to_save": [
-    "lm_head",
-    "embed_tokens"
-  ],
-  "peft_type": "LORA",
-  "peft_version": "0.18.1",
-  "qalora_group_size": 16,
-  "r": 16,
-  "rank_pattern": {},
-  "revision": null,
-  "target_modules": [
-    "k_proj",
-    "q_proj",
-    "v_proj",
-    "o_proj"
-  ],
-  "target_parameters": null,
-  "task_type": "CAUSAL_LM",
-  "trainable_token_indices": null,
-  "use_dora": false,
-  "use_qalora": false,
-  "use_rslora": false
-}
\ No newline at end of file
diff --git a/llama3.2-3b/dp8/epochs/epoch_005/adapter/adapter_model.safetensors b/llama3.2-3b/dp8/epochs/epoch_005/adapter/adapter_model.safetensors
deleted file mode 100644
index 2e9c110f3ca6276c94f5e42b8b82b4ee7f66a444..0000000000000000000000000000000000000000
--- a/llama3.2-3b/dp8/epochs/epoch_005/adapter/adapter_model.safetensors
+++ /dev/null
@@ -1,3 +0,0 @@
-version https://git-lfs.github.com/spec/v1
-oid sha256:b4ae06a74d75bfaf6b118d0e2b53875fd9ed8c240e30436eb8327a8883a68b2d
-size 4783192176
diff --git a/llama3.2-3b/dp8/epochs/epoch_005/audit_results.json b/llama3.2-3b/dp8/epochs/epoch_005/audit_results.json
deleted file mode 100644
index f8efa3912df4a1a3dc70e61830766e16187b9f73..0000000000000000000000000000000000000000
--- a/llama3.2-3b/dp8/epochs/epoch_005/audit_results.json
+++ /dev/null
@@ -1,137 +0,0 @@
-{
-  "delta": 1e-05,
-  "num_canaries": 500,
-  "num_members": 250,
-  "paper_guess_fraction": 0.2,
-  "paper_guess_steps": 20,
-  "loss": {
-    "auc": 0.604632,
-    "empirical_epsilon": {
-      "0.05": 0.09125839080661535,
-      "0.01": 0.0
-    },
-    "empirical_epsilon_details": {
-      "0.05": {
-        "epsilon": 0.09125839080661535,
-        "num_guesses": 100,
-        "correct_guesses": 61,
-        "candidate_num_guesses": [
-          5,
-          10,
-          15,
-          20,
-          25,
-          30,
-          35,
-          40,
-          45,
-          50,
-          55,
-          60,
-          65,
-          70,
-          75,
-          80,
-          85,
-          90,
-          95,
-          100
-        ],
-        "direction": "lower"
-      },
-      "0.01": {
-        "epsilon": 0.0,
-        "num_guesses": 0,
-        "correct_guesses": 0,
-        "candidate_num_guesses": [
-          5,
-          10,
-          15,
-          20,
-          25,
-          30,
-          35,
-          40,
-          45,
-          50,
-          55,
-          60,
-          65,
-          70,
-          75,
-          80,
-          85,
-          90,
-          95,
-          100
-        ],
-        "direction": "lower"
-      }
-    }
-  },
-  "embedding": {
-    "auc": 0.508896,
-    "empirical_epsilon": {
-      "0.05": 0.0,
-      "0.01": 0.0
-    },
-    "empirical_epsilon_details": {
-      "0.05": {
-        "epsilon": 0.0,
-        "num_guesses": 0,
-        "correct_guesses": 0,
-        "candidate_num_guesses": [
-          5,
-          10,
-          15,
-          20,
-          25,
-          30,
-          35,
-          40,
-          45,
-          50,
-          55,
-          60,
-          65,
-          70,
-          75,
-          80,
-          85,
-          90,
-          95,
-          100
-        ],
-        "direction": "lower"
-      },
-      "0.01": {
-        "epsilon": 0.0,
-        "num_guesses": 0,
-        "correct_guesses": 0,
-        "candidate_num_guesses": [
-          5,
-          10,
-          15,
-          20,
-          25,
-          30,
-          35,
-          40,
-          45,
-          50,
-          55,
-          60,
-          65,
-          70,
-          75,
-          80,
-          85,
-          90,
-          95,
-          100
-        ],
-        "direction": "lower"
-      }
-    }
-  }
-}
\ No newline at end of file
diff --git a/qwen1.5-moe-a2.7b/base_attn_v2/epochs/epoch_001/adapter/README.md b/qwen1.5-moe-a2.7b/base_attn_v2/epochs/epoch_001/adapter/README.md
deleted file mode 100644
index 3b9e588251d937f16d19c9a0dcc3c6c5e3404063..0000000000000000000000000000000000000000
--- a/qwen1.5-moe-a2.7b/base_attn_v2/epochs/epoch_001/adapter/README.md
+++ /dev/null
@@ -1,207 +0,0 @@
----
-base_model: Qwen/Qwen1.5-MoE-A2.7B
-library_name: peft
-pipeline_tag: text-generation
-tags:
-- base_model:adapter:Qwen/Qwen1.5-MoE-A2.7B
-- lora
-- transformers
----
-
-# Model Card for Model ID
-
-<!-- Provide a quick summary of what the model is/does. -->
-
-
-
-## Model Details
-
-### Model Description
-
-<!-- Provide a longer summary of what this model is. -->
-
-
-
-- **Developed by:** [More Information Needed]
-- **Funded by [optional]:** [More Information Needed]
-- **Shared by [optional]:** [More Information Needed]
-- **Model type:** [More Information Needed]
-- **Language(s) (NLP):** [More Information Needed]
-- **License:** [More Information Needed]
-- **Finetuned from model [optional]:** [More Information Needed]
-
-### Model Sources [optional]
-
-<!-- Provide the basic links for the model. -->
-
-- **Repository:** [More Information Needed]
-- **Paper [optional]:** [More Information Needed]
-- **Demo [optional]:** [More Information Needed]
-
-## Uses
-
-<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
-
-### Direct Use
-
-<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
-
-[More Information Needed]
-
-### Downstream Use [optional]
-
-<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
-
-[More Information Needed]
-
-### Out-of-Scope Use
-
-<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
-
-[More Information Needed]
-
-## Bias, Risks, and Limitations
-
-<!-- This section is meant to convey both technical and sociotechnical limitations. -->
-
-[More Information Needed]
-
-### Recommendations
-
-<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
-
-Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
-
-## How to Get Started with the Model
-
-Use the code below to get started with the model.
-
-[More Information Needed]
-
-## Training Details
-
-### Training Data
-
-<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
-
-[More Information Needed]
-
-### Training Procedure
-
-<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
-
-#### Preprocessing [optional]
-
-[More Information Needed]
-
-
-#### Training Hyperparameters
-
-- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
-
-#### Speeds, Sizes, Times [optional]
-
-<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
-
-[More Information Needed]
-
-## Evaluation
-
-<!-- This section describes the evaluation protocols and provides the results. -->
-
-### Testing Data, Factors & Metrics
-
-#### Testing Data
-
-<!-- This should link to a Dataset Card if possible. -->
-
-[More Information Needed]
-
-#### Factors
-
-<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
-
-[More Information Needed]
-
-#### Metrics
-
-<!-- These are the evaluation metrics being used, ideally with a description of why. -->
-
-[More Information Needed]
-
-### Results
-
-[More Information Needed]
-
-#### Summary
-
-
-
-## Model Examination [optional]
-
-<!-- Relevant interpretability work for the model goes here -->
-
-[More Information Needed]
-
-## Environmental Impact
-
-<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
-
-Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
-
-- **Hardware Type:** [More Information Needed]
-- **Hours used:** [More Information Needed]
-- **Cloud Provider:** [More Information Needed]
-- **Compute Region:** [More Information Needed]
-- **Carbon Emitted:** [More Information Needed]
-
-## Technical Specifications [optional]
-
-### Model Architecture and Objective
-
-[More Information Needed]
-
-### Compute Infrastructure
-
-[More Information Needed]
-
-#### Hardware
-
-[More Information Needed]
-
-#### Software
-
-[More Information Needed]
-
-## Citation [optional]
-
-<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
-
-**BibTeX:**
-
-[More Information Needed]
-
-**APA:**
-
-[More Information Needed]
-
-## Glossary [optional]
-
-<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
-
-[More Information Needed]
-
-## More Information [optional]
-
-[More Information Needed]
-
-## Model Card Authors [optional]
-
-[More Information Needed]
-
-## Model Card Contact
-
-[More Information Needed]
-### Framework versions
-
-- PEFT 0.18.1
\ No newline at end of file
diff --git a/qwen1.5-moe-a2.7b/base_attn_v2/epochs/epoch_001/adapter/adapter_config.json b/qwen1.5-moe-a2.7b/base_attn_v2/epochs/epoch_001/adapter/adapter_config.json
deleted file mode 100644
index 37d244d234d1335c4acbc3c5f8577e0b83b28ed7..0000000000000000000000000000000000000000
--- a/qwen1.5-moe-a2.7b/base_attn_v2/epochs/epoch_001/adapter/adapter_config.json
+++ /dev/null
@@ -1,46 +0,0 @@
-{
-  "alora_invocation_tokens": null,
-  "alpha_pattern": {},
-  "arrow_config": null,
-  "auto_mapping": null,
-  "base_model_name_or_path": "Qwen/Qwen1.5-MoE-A2.7B",
-  "bias": "none",
-  "corda_config": null,
-  "ensure_weight_tying": true,
-  "eva_config": null,
-  "exclude_modules": null,
-  "fan_in_fan_out": false,
-  "inference_mode": true,
-  "init_lora_weights": true,
-  "layer_replication": null,
-  "layers_pattern": null,
-  "layers_to_transform": null,
-  "loftq_config": {},
-  "lora_alpha": 32,
-  "lora_bias": false,
-  "lora_dropout": 0.05,
-  "megatron_config": null,
-  "megatron_core": "megatron.core",
-  "modules_to_save": [
-    "lm_head",
-    "embed_tokens"
-  ],
-  "peft_type": "LORA",
-  "peft_version": "0.18.1",
-  "qalora_group_size": 16,
-  "r": 16,
-  "rank_pattern": {},
-  "revision": null,
-  "target_modules": [
-    "q_proj",
-    "o_proj",
-    "k_proj",
-    "v_proj"
-  ],
-  "target_parameters": null,
-  "task_type": "CAUSAL_LM",
-  "trainable_token_indices": null,
-  "use_dora": false,
-  "use_qalora": false,
-  "use_rslora": false
-}
\ No newline at end of file
diff --git a/qwen1.5-moe-a2.7b/base_attn_v2/epochs/epoch_001/audit_results.json b/qwen1.5-moe-a2.7b/base_attn_v2/epochs/epoch_001/audit_results.json
deleted file mode 100644
index bee71cc461a6184f53ae73fb3d3ca1d550bfd590..0000000000000000000000000000000000000000
--- a/qwen1.5-moe-a2.7b/base_attn_v2/epochs/epoch_001/audit_results.json
+++ /dev/null
@@ -1,137 +0,0 @@
-{
-  "delta": 1e-05,
-  "num_canaries": 500,
-  "num_members": 250,
-  "paper_guess_fraction": 0.2,
-  "paper_guess_steps": 20,
-  "loss": {
-    "auc": 0.8726,
-    "empirical_epsilon": {
-      "0.05": 3.4791953936219215,
-      "0.01": 3.023197554051876
-    },
-    "empirical_epsilon_details": {
-      "0.05": {
-        "epsilon": 3.4791953936219215,
-        "num_guesses": 100,
-        "correct_guesses": 100,
-        "candidate_num_guesses": [
-          5,
-          10,
-          15,
-          20,
-          25,
-          30,
-          35,
-          40,
-          45,
-          50,
-          55,
-          60,
-          65,
-          70,
-          75,
-          80,
-          85,
-          90,
-          95,
-          100
-        ],
-        "direction": "lower"
-      },
-      "0.01": {
-        "epsilon": 3.023197554051876,
-        "num_guesses": 100,
-        "correct_guesses": 100,
-        "candidate_num_guesses": [
-          5,
-          10,
-          15,
-          20,
-          25,
-          30,
-          35,
-          40,
-          45,
-          50,
-          55,
-          60,
-          65,
-          70,
-          75,
-          80,
-          85,
-          90,
-          95,
-          100
-        ],
-        "direction": "lower"
-      }
-    }
-  },
-  "embedding": {
-    "auc": 0.686,
-    "empirical_epsilon": {
-      "0.05": 3.4791953936219215,
-      "0.01": 3.023197554051876
-    },
-    "empirical_epsilon_details": {
-      "0.05": {
-        "epsilon": 3.4791953936219215,
-        "num_guesses": 100,
-        "correct_guesses": 100,
-        "candidate_num_guesses": [
-          5,
-          10,
-          15,
-          20,
-          25,
-          30,
-          35,
-          40,
-          45,
-          50,
-          55,
-          60,
-          65,
-          70,
-          75,
-          80,
-          85,
-          90,
-          95,
-          100
-        ],
-        "direction": "lower"
-      },
-      "0.01": {
-        "epsilon": 3.023197554051876,
-        "num_guesses": 100,
-        "correct_guesses": 100,
-        "candidate_num_guesses": [
-          5,
-          10,
-          15,
-          20,
-          25,
-          30,
-          35,
-          40,
-          45,
-          50,
-          55,
-          60,
-          65,
-          70,
-          75,
-          80,
-          85,
-          90,
-          95,
-          100
-        ],
-        "direction": "lower"
-      }
-    }
-  }
-}
\ No newline at end of file
diff --git a/qwen1.5-moe-a2.7b/base_attn_v2/epochs/epoch_002/adapter/README.md b/qwen1.5-moe-a2.7b/base_attn_v2/epochs/epoch_002/adapter/README.md
deleted file mode 100644
index 3b9e588251d937f16d19c9a0dcc3c6c5e3404063..0000000000000000000000000000000000000000
--- a/qwen1.5-moe-a2.7b/base_attn_v2/epochs/epoch_002/adapter/README.md
+++ /dev/null
@@ -1,207 +0,0 @@
----
-base_model: Qwen/Qwen1.5-MoE-A2.7B
-library_name: peft
-pipeline_tag: text-generation
-tags:
-- base_model:adapter:Qwen/Qwen1.5-MoE-A2.7B
-- lora
-- transformers
----
-
-# Model Card for Model ID
-
-<!-- Provide a quick summary of what the model is/does. -->
-
-
-
-## Model Details
-
-### Model Description
-
-<!-- Provide a longer summary of what this model is. -->
-
-
-
-- **Developed by:** [More Information Needed]
-- **Funded by [optional]:** [More Information Needed]
-- **Shared by [optional]:** [More Information Needed]
-- **Model type:** [More Information Needed]
-- **Language(s) (NLP):** [More Information Needed]
-- **License:** [More Information Needed]
-- **Finetuned from model [optional]:** [More Information Needed]
-
-### Model Sources [optional]
-
-<!-- Provide the basic links for the model. -->
-
-- **Repository:** [More Information Needed]
-- **Paper [optional]:** [More Information Needed]
-- **Demo [optional]:** [More Information Needed]
-
-## Uses
-
-<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
-
-### Direct Use
-
-<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
-
-[More Information Needed]
-
-### Downstream Use [optional]
-
-<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
-
-[More Information Needed]
-
-### Out-of-Scope Use
-
-<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
-
-[More Information Needed]
-
-## Bias, Risks, and Limitations
-
-<!-- This section is meant to convey both technical and sociotechnical limitations. -->
-
-[More Information Needed]
-
-### Recommendations
-
-<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
-
-Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
-
-## How to Get Started with the Model
-
-Use the code below to get started with the model.
-
-[More Information Needed]
-
-## Training Details
-
-### Training Data
-
-<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
-
-[More Information Needed]
-
-### Training Procedure
-
-<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
-
-#### Preprocessing [optional]
-
-[More Information Needed]
-
-
-#### Training Hyperparameters
-
-- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
-
-#### Speeds, Sizes, Times [optional]
-
-<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
-
-[More Information Needed]
-
-## Evaluation
-
-<!-- This section describes the evaluation protocols and provides the results. -->
-
-### Testing Data, Factors & Metrics
-
-#### Testing Data
-
-<!-- This should link to a Dataset Card if possible. -->
-
-[More Information Needed]
-
-#### Factors
-
-<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
-
-[More Information Needed]
-
-#### Metrics
-
-<!-- These are the evaluation metrics being used, ideally with a description of why. -->
-
-[More Information Needed]
-
-### Results
-
-[More Information Needed]
-
-#### Summary
-
-
-
-## Model Examination [optional]
-
-<!-- Relevant interpretability work for the model goes here -->
-
-[More Information Needed]
-
-## Environmental Impact
-
-<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
-
-Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
-
-- **Hardware Type:** [More Information Needed]
-- **Hours used:** [More Information Needed]
-- **Cloud Provider:** [More Information Needed]
-- **Compute Region:** [More Information Needed]
-- **Carbon Emitted:** [More Information Needed]
-
-## Technical Specifications [optional]
-
-### Model Architecture and Objective
-
-[More Information Needed]
-
-### Compute Infrastructure
-
-[More Information Needed]
-
-#### Hardware
-
-[More Information Needed]
-
-#### Software
-
-[More Information Needed]
-
-## Citation [optional]
-
-<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
-
-**BibTeX:**
-
-[More Information Needed]
-
-**APA:**
-
-[More Information Needed]
-
-## Glossary [optional]
-
-<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
-
-[More Information Needed]
-
-## More Information [optional]
-
-[More Information Needed]
-
-## Model Card Authors [optional]
-
-[More Information Needed]
-
-## Model Card Contact
-
-[More Information Needed]
-### Framework versions
-
-- PEFT 0.18.1
\ No newline at end of file
diff --git a/qwen1.5-moe-a2.7b/base_attn_v2/epochs/epoch_002/adapter/adapter_config.json b/qwen1.5-moe-a2.7b/base_attn_v2/epochs/epoch_002/adapter/adapter_config.json
deleted file mode 100644
index 37d244d234d1335c4acbc3c5f8577e0b83b28ed7..0000000000000000000000000000000000000000
--- a/qwen1.5-moe-a2.7b/base_attn_v2/epochs/epoch_002/adapter/adapter_config.json
+++ /dev/null
@@ -1,46 +0,0 @@
-{
-  "alora_invocation_tokens": null,
-  "alpha_pattern": {},
-  "arrow_config": null,
-  "auto_mapping": null,
-  "base_model_name_or_path": "Qwen/Qwen1.5-MoE-A2.7B",
-  "bias": "none",
-  "corda_config": null,
-  "ensure_weight_tying": true,
-  "eva_config": null,
-  "exclude_modules": null,
-  "fan_in_fan_out": false,
-  "inference_mode": true,
-  "init_lora_weights": true,
-  "layer_replication": null,
-  "layers_pattern": null,
-  "layers_to_transform": null,
-  "loftq_config": {},
-  "lora_alpha": 32,
-  "lora_bias": false,
-  "lora_dropout": 0.05,
-  "megatron_config": null,
-  "megatron_core": "megatron.core",
-  "modules_to_save": [
-    "lm_head",
-    "embed_tokens"
-  ],
-  "peft_type": "LORA",
-  "peft_version": "0.18.1",
-  "qalora_group_size": 16,
-  "r": 16,
-  "rank_pattern": {},
-  "revision": null,
-  "target_modules": [
-    "q_proj",
-    "o_proj",
-    "k_proj",
-    "v_proj"
-  ],
-  "target_parameters": null,
-  "task_type": "CAUSAL_LM",
-  "trainable_token_indices": null,
-  "use_dora": false,
-  "use_qalora": false,
-  "use_rslora": false
-}
\ No newline at end of file
diff --git a/qwen1.5-moe-a2.7b/base_attn_v2/epochs/epoch_002/audit_results.json b/qwen1.5-moe-a2.7b/base_attn_v2/epochs/epoch_002/audit_results.json
deleted file mode 100644
index 0d38dea2a51184c12e17f310f32fe3a1bfd30bd1..0000000000000000000000000000000000000000
--- a/qwen1.5-moe-a2.7b/base_attn_v2/epochs/epoch_002/audit_results.json
+++ /dev/null
@@ -1,137 +0,0 @@
-{
-  "delta": 1e-05,
-  "num_canaries": 500,
-  "num_members": 250,
-  "paper_guess_fraction": 0.2,
-  "paper_guess_steps": 20,
-  "loss": {
-    "auc": 0.9734,
-    "empirical_epsilon": {
-      "0.05": 3.4791953936219215,
-      "0.01": 3.023197554051876
-    },
-    "empirical_epsilon_details": {
-      "0.05": {
-        "epsilon": 3.4791953936219215,
-        "num_guesses": 100,
-        "correct_guesses": 100,
-        "candidate_num_guesses": [
-          5,
-          10,
-          15,
-          20,
-          25,
-          30,
-          35,
-          40,
-          45,
-          50,
-          55,
-          60,
-          65,
-          70,
-          75,
-          80,
-          85,
-          90,
-          95,
-          100
-        ],
-        "direction": "lower"
-      },
-      "0.01": {
-        "epsilon": 3.023197554051876,
-        "num_guesses": 100,
-        "correct_guesses": 100,
-        "candidate_num_guesses": [
-          5,
-          10,
-          15,
-          20,
-          25,
-          30,
-          35,
-          40,
-          45,
-          50,
-          55,
-          60,
-          65,
-          70,
-          75,
-          80,
-          85,
-          90,
-          95,
-          100
-        ],
-        "direction": "lower"
-      }
-    }
-  },
-  "embedding": {
-    "auc": 0.536,
-    "empirical_epsilon": {
-      "0.05": 3.4791953936219215,
-      "0.01": 3.023197554051876
-    },
-    "empirical_epsilon_details": {
-      "0.05": {
-        "epsilon": 3.4791953936219215,
-        "num_guesses": 100,
-        "correct_guesses": 100,
-        "candidate_num_guesses": [
-          5,
-          10,
-          15,
-          20,
-          25,
-          30,
-          35,
-          40,
-          45,
-          50,
-          55,
-          60,
-          65,
-          70,
-          75,
-          80,
-          85,
-          90,
-          95,
-          100
-        ],
-        "direction": "lower"
-      },
-      "0.01": {
-        "epsilon": 3.023197554051876,
-        "num_guesses": 100,
-        "correct_guesses": 100,
-        "candidate_num_guesses": [
-          5,
-          10,
-          15,
-          20,
-          25,
-          30,
-          35,
-          40,
-          45,
-          50,
-          55,
-          60,
-          65,
-          70,
-          75,
-          80,
-          85,
-          90,
-          95,
-          100
-        ],
-        "direction": "lower"
-      }
-    }
-  }
-}
\ No newline at end of file
diff --git a/qwen1.5-moe-a2.7b/base_attn_v2/epochs/epoch_003/adapter/README.md b/qwen1.5-moe-a2.7b/base_attn_v2/epochs/epoch_003/adapter/README.md
deleted file mode 100644
index 3b9e588251d937f16d19c9a0dcc3c6c5e3404063..0000000000000000000000000000000000000000
--- a/qwen1.5-moe-a2.7b/base_attn_v2/epochs/epoch_003/adapter/README.md
+++ /dev/null
@@ -1,207 +0,0 @@
----
-base_model: Qwen/Qwen1.5-MoE-A2.7B
-library_name: peft
-pipeline_tag: text-generation
-tags:
-- base_model:adapter:Qwen/Qwen1.5-MoE-A2.7B
-- lora
-- transformers
----
-
-# Model Card for Model ID
-
-<!-- Provide a quick summary of what the model is/does. -->
-
-
-
-## Model Details
-
-### Model Description
-
-<!-- Provide a longer summary of what this model is. -->
-
-
-
-- **Developed by:** [More Information Needed]
-- **Funded by [optional]:** [More Information Needed]
-- **Shared by [optional]:** [More Information Needed]
-- **Model type:** [More Information Needed]
-- **Language(s) (NLP):** [More Information Needed]
-- **License:** [More Information Needed]
-- **Finetuned from model [optional]:** [More Information Needed]
-
-### Model Sources [optional]
-
-<!-- Provide the basic links for the model. -->
-
-- **Repository:** [More Information Needed]
-- **Paper [optional]:** [More Information Needed]
-- **Demo [optional]:** [More Information Needed]
-
-## Uses
-
-<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
-
-### Direct Use
-
-<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
-
-[More Information Needed]
-
-### Downstream Use [optional]
-
-<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
-
-[More Information Needed]
-
-### Out-of-Scope Use
-
-<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
-
-[More Information Needed]
-
-## Bias, Risks, and Limitations
-
-<!-- This section is meant to convey both technical and sociotechnical limitations. -->
-
-[More Information Needed]
-
-### Recommendations
-
-<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
-
-Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
-
-## How to Get Started with the Model
-
-Use the code below to get started with the model.
-
-[More Information Needed]
-
-## Training Details
-
-### Training Data
-
-<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
-
-[More Information Needed]
-
-### Training Procedure
-
-<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
-
-#### Preprocessing [optional]
-
-[More Information Needed]
-
-
-#### Training Hyperparameters
-
-- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
-
-#### Speeds, Sizes, Times [optional]
-
-<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
-
-[More Information Needed]
-
-## Evaluation
-
-<!-- This section describes the evaluation protocols and provides the results. -->
-
-### Testing Data, Factors & Metrics
-
-#### Testing Data
-
-<!-- This should link to a Dataset Card if possible. -->
-
-[More Information Needed]
-
-#### Factors
-
-<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
-
-[More Information Needed]
-
-#### Metrics
-
-<!-- These are the evaluation metrics being used, ideally with a description of why. -->
-
-[More Information Needed]
-
-### Results
-
-[More Information Needed]
-
-#### Summary
-
-
-
-## Model Examination [optional]
-
-<!-- Relevant interpretability work for the model goes here -->
-
-[More Information Needed]
-
-## Environmental Impact
-
-<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
-
-Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
-
-- **Hardware Type:** [More Information Needed]
-- **Hours used:** [More Information Needed]
-- **Cloud Provider:** [More Information Needed]
-- **Compute Region:** [More Information Needed]
-- **Carbon Emitted:** [More Information Needed]
-
-## Technical Specifications [optional]
-
-### Model Architecture and Objective
-
-[More Information Needed]
-
-### Compute Infrastructure
-
-[More Information Needed]
-
-#### Hardware
-
-[More Information Needed]
-
-#### Software
-
-[More Information Needed]
-
-## Citation [optional]
-
-<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
-
-**BibTeX:**
-
-[More Information Needed]
-
-**APA:**
-
-[More Information Needed]
-
-## Glossary [optional]
-
-<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
-
-[More Information Needed]
-
-## More Information [optional]
-
-[More Information Needed]
-
-## Model Card Authors [optional]
-
-[More Information Needed]
-
-## Model Card Contact
-
-[More Information Needed]
-### Framework versions
-
-- PEFT 0.18.1
\ No newline at end of file
diff --git a/qwen1.5-moe-a2.7b/base_attn_v2/epochs/epoch_003/adapter/adapter_config.json b/qwen1.5-moe-a2.7b/base_attn_v2/epochs/epoch_003/adapter/adapter_config.json
deleted file mode 100644
index 37d244d234d1335c4acbc3c5f8577e0b83b28ed7..0000000000000000000000000000000000000000
--- a/qwen1.5-moe-a2.7b/base_attn_v2/epochs/epoch_003/adapter/adapter_config.json
+++ /dev/null
@@ -1,46 +0,0 @@
-{
-  "alora_invocation_tokens": null,
-  "alpha_pattern": {},
-  "arrow_config": null,
-  "auto_mapping": null,
-  "base_model_name_or_path": "Qwen/Qwen1.5-MoE-A2.7B",
-  "bias": "none",
-  "corda_config": null,
-  "ensure_weight_tying": true,
-  "eva_config": null,
-  "exclude_modules": null,
-  "fan_in_fan_out": false,
-  "inference_mode": true,
-  "init_lora_weights": true,
-  "layer_replication": null,
-  "layers_pattern": null,
-  "layers_to_transform": null,
-  "loftq_config": {},
-  "lora_alpha": 32,
-  "lora_bias": false,
-  "lora_dropout": 0.05,
-  "megatron_config": null,
-  "megatron_core": "megatron.core",
-  "modules_to_save": [
-    "lm_head",
-    "embed_tokens"
-  ],
-  "peft_type": "LORA",
-  "peft_version": "0.18.1",
-  "qalora_group_size": 16,
-  "r": 16,
-  "rank_pattern": {},
-  "revision": null,
-  "target_modules": [
-    "q_proj",
-    "o_proj",
-    "k_proj",
-    "v_proj"
-  ],
-  "target_parameters": null,
-  "task_type": "CAUSAL_LM",
-  "trainable_token_indices": null,
-  "use_dora": false,
-  "use_qalora": false,
-  "use_rslora": false
-}
\ No newline at end of file
diff --git a/qwen1.5-moe-a2.7b/base_attn_v2/epochs/epoch_003/audit_results.json b/qwen1.5-moe-a2.7b/base_attn_v2/epochs/epoch_003/audit_results.json
deleted file mode 100644
index bde9c2de16ae36466cba3620c0fba1427f08f514..0000000000000000000000000000000000000000
--- a/qwen1.5-moe-a2.7b/base_attn_v2/epochs/epoch_003/audit_results.json
+++ /dev/null
@@ -1,137 +0,0 @@
-{
-  "delta": 1e-05,
-  "num_canaries": 500,
-  "num_members": 250,
-  "paper_guess_fraction": 0.2,
-  "paper_guess_steps": 20,
-  "loss": {
-    "auc": 0.999256,
-    "empirical_epsilon": {
-      "0.05": 3.4791953936219215,
-      "0.01": 3.023197554051876
-    },
-    "empirical_epsilon_details": {
-      "0.05": {
-        "epsilon": 3.4791953936219215,
-        "num_guesses": 100,
-        "correct_guesses": 100,
-        "candidate_num_guesses": [
-          5,
-          10,
-          15,
-          20,
-          25,
-          30,
-          35,
-          40,
-          45,
-          50,
-          55,
-          60,
-          65,
-          70,
-          75,
-          80,
-          85,
-          90,
-          95,
-          100
-        ],
-        "direction": "lower"
-      },
-      "0.01": {
-        "epsilon": 3.023197554051876,
-        "num_guesses": 100,
-        "correct_guesses": 100,
-        "candidate_num_guesses": [
-          5,
-          10,
-          15,
-          20,
-          25,
-          30,
-          35,
-          40,
-          45,
-          50,
-          55,
-          60,
-          65,
-          70,
-          75,
-          80,
-          85,
-          90,
-          95,
-          100
-        ],
-        "direction": "lower"
-      }
-    }
-  },
-  "embedding": {
-    "auc": 0.606,
-    "empirical_epsilon": {
-      "0.05": 3.4791953936219215,
-      "0.01": 3.023197554051876
-    },
-    "empirical_epsilon_details": {
-      "0.05": {
-        "epsilon": 3.4791953936219215,
-        "num_guesses": 100,
-        "correct_guesses": 100,
-        "candidate_num_guesses": [
-          5,
-          10,
-          15,
-          20,
-          25,
-          30,
-          35,
-          40,
-          45,
-          50,
-          55,
-          60,
-          65,
-          70,
-          75,
-          80,
-          85,
-          90,
-          95,
-          100
-        ],
-        "direction": "higher"
-      },
-      "0.01": {
-        "epsilon": 3.023197554051876,
-        "num_guesses": 100,
-        "correct_guesses": 100,
-        "candidate_num_guesses": [
-          5,
-          10,
-          15,
-          20,
-          25,
-          30,
-          35,
-          40,
-          45,
-          50,
-          55,
-          60,
-          65,
-          70,
-          75,
-          80,
-          85,
-          90,
-          95,
-          100
-        ],
-        "direction": "higher"
-      }
-    }
-  }
-}
\ No newline at end of file
diff --git a/qwen1.5-moe-a2.7b/base_attn_v2/epochs/epoch_004/adapter/README.md b/qwen1.5-moe-a2.7b/base_attn_v2/epochs/epoch_004/adapter/README.md
deleted file mode 100644
index 3b9e588251d937f16d19c9a0dcc3c6c5e3404063..0000000000000000000000000000000000000000
--- a/qwen1.5-moe-a2.7b/base_attn_v2/epochs/epoch_004/adapter/README.md
+++ /dev/null
@@ -1,207 +0,0 @@
----
-base_model: Qwen/Qwen1.5-MoE-A2.7B
-library_name: peft
-pipeline_tag: text-generation
-tags:
-- base_model:adapter:Qwen/Qwen1.5-MoE-A2.7B
-- lora
-- transformers
----
-
-# Model Card for Model ID
-
-<!-- Provide a quick summary of what the model is/does. -->
-
-
-
-## Model Details
-
-### Model Description
-
-<!-- Provide a longer summary of what this model is. -->
-
-
-
-- **Developed by:** [More Information Needed]
-- **Funded by [optional]:** [More Information Needed]
-- **Shared by [optional]:** [More Information Needed]
-- **Model type:** [More Information Needed]
-- **Language(s) (NLP):** [More Information Needed]
-- **License:** [More Information Needed]
-- **Finetuned from model [optional]:** [More Information Needed]
-
-### Model Sources [optional]
-
-<!-- Provide the basic links for the model. -->
-
-- **Repository:** [More Information Needed]
-- **Paper [optional]:** [More Information Needed]
-- **Demo [optional]:** [More Information Needed]
-
-## Uses
-
-<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
-
-### Direct Use
-
-<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
-
-[More Information Needed]
-
-### Downstream Use [optional]
-
-<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
-
-[More Information Needed]
-
-### Out-of-Scope Use
-
-<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
-
-[More Information Needed]
-
-## Bias, Risks, and Limitations
-
-<!-- This section is meant to convey both technical and sociotechnical limitations. -->
-
-[More Information Needed]
-
-### Recommendations
-
-<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
-
-Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
-
-## How to Get Started with the Model
-
-Use the code below to get started with the model.
-
-[More Information Needed]
-
-## Training Details
-
-### Training Data
-
-<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
-
-[More Information Needed]
-
-### Training Procedure
-
-<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
-
-#### Preprocessing [optional]
-
-[More Information Needed]
-
-
-#### Training Hyperparameters
-
-- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
-
-#### Speeds, Sizes, Times [optional]
-
-<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
-
-[More Information Needed]
-
-## Evaluation
-
-<!-- This section describes the evaluation protocols and provides the results. -->
-
-### Testing Data, Factors & Metrics
-
-#### Testing Data
-
-<!-- This should link to a Dataset Card if possible. -->
-
-[More Information Needed]
-
-#### Factors
-
-<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
-
-[More Information Needed]
-
-#### Metrics
-
-<!-- These are the evaluation metrics being used, ideally with a description of why. -->
-
-[More Information Needed]
-
-### Results
-
-[More Information Needed]
-
-#### Summary
-
-
-
-## Model Examination [optional]
-
-<!-- Relevant interpretability work for the model goes here -->
-
-[More Information Needed]
-
-## Environmental Impact
-
-<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
-
-Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
-
-- **Hardware Type:** [More Information Needed]
-- **Hours used:** [More Information Needed]
-- **Cloud Provider:** [More Information Needed]
-- **Compute Region:** [More Information Needed]
-- **Carbon Emitted:** [More Information Needed]
-
-## Technical Specifications [optional]
-
-### Model Architecture and Objective
-
-[More Information Needed]
-
-### Compute Infrastructure
-
-[More Information Needed]
-
-#### Hardware
-
-[More Information Needed]
-
-#### Software
-
-[More Information Needed]
-
-## Citation [optional]
-
-<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
-
-**BibTeX:**
-
-[More Information Needed]
-
-**APA:**
-
-[More Information Needed]
-
-## Glossary [optional]
-
-<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
-
-[More Information Needed]
-
-## More Information [optional]
-
-[More Information Needed]
-
-## Model Card Authors [optional]
-
-[More Information Needed]
-
-## Model Card Contact
-
-[More Information Needed]
-### Framework versions
-
-- PEFT 0.18.1
\ No newline at end of file
diff --git a/qwen1.5-moe-a2.7b/base_attn_v2/epochs/epoch_004/adapter/adapter_config.json b/qwen1.5-moe-a2.7b/base_attn_v2/epochs/epoch_004/adapter/adapter_config.json
deleted file mode 100644
index 37d244d234d1335c4acbc3c5f8577e0b83b28ed7..0000000000000000000000000000000000000000
--- a/qwen1.5-moe-a2.7b/base_attn_v2/epochs/epoch_004/adapter/adapter_config.json
+++ /dev/null
@@ -1,46 +0,0 @@
-{
-  "alora_invocation_tokens": null,
-  "alpha_pattern": {},
-  "arrow_config": null,
-  "auto_mapping": null,
-  "base_model_name_or_path": "Qwen/Qwen1.5-MoE-A2.7B",
-  "bias": "none",
-  "corda_config": null,
-  "ensure_weight_tying": true,
-  "eva_config": null,
-  "exclude_modules": null,
-  "fan_in_fan_out": false,
-  "inference_mode": true,
-  "init_lora_weights": true,
-  "layer_replication": null,
-  "layers_pattern": null,
-  "layers_to_transform": null,
-  "loftq_config": {},
-  "lora_alpha": 32,
-  "lora_bias": false,
-  "lora_dropout": 0.05,
-  "megatron_config": null,
-  "megatron_core": "megatron.core",
-  "modules_to_save": [
-    "lm_head",
-    "embed_tokens"
-  ],
-  "peft_type": "LORA",
-  "peft_version": "0.18.1",
-  "qalora_group_size": 16,
-  "r": 16,
-  "rank_pattern": {},
-  "revision": null,
-  "target_modules": [
-    "q_proj",
-    "o_proj",
-    "k_proj",
-    "v_proj"
-  ],
-  "target_parameters": null,
-  "task_type": "CAUSAL_LM",
-  "trainable_token_indices": null,
-  "use_dora": false,
-  "use_qalora": false,
-  "use_rslora": false
-}
\ No newline at end of file
diff --git a/qwen1.5-moe-a2.7b/base_attn_v2/epochs/epoch_004/audit_results.json b/qwen1.5-moe-a2.7b/base_attn_v2/epochs/epoch_004/audit_results.json
deleted file mode 100644
index 9184ddad3b356fd9a47ccd00baf9ef7b8b4df4bb..0000000000000000000000000000000000000000
--- a/qwen1.5-moe-a2.7b/base_attn_v2/epochs/epoch_004/audit_results.json
+++ /dev/null
@@ -1,137 +0,0 @@
-{
-  "delta": 1e-05,
-  "num_canaries": 500,
-  "num_members": 250,
-  "paper_guess_fraction": 0.2,
-  "paper_guess_steps": 20,
-  "loss": {
-    "auc": 0.99956,
-    "empirical_epsilon": {
-      "0.05": 3.4791953936219215,
-      "0.01": 3.023197554051876
-    },
-    "empirical_epsilon_details": {
-      "0.05": {
-        "epsilon": 3.4791953936219215,
-        "num_guesses": 100,
-        "correct_guesses": 100,
-        "candidate_num_guesses": [
-          5,
-          10,
-          15,
-          20,
-          25,
-          30,
-          35,
-          40,
-          45,
-          50,
-          55,
-          60,
-          65,
-          70,
-          75,
-          80,
-          85,
-          90,
-          95,
-          100
-        ],
-        "direction": "lower"
-      },
-      "0.01": {
-        "epsilon": 3.023197554051876,
-        "num_guesses": 100,
-        "correct_guesses": 100,
-        "candidate_num_guesses": [
-          5,
-          10,
-          15,
-          20,
-          25,
-          30,
-          35,
-          40,
-          45,
-          50,
-          55,
-          60,
-          65,
-          70,
-          75,
-          80,
-          85,
-          90,
-          95,
-          100
-        ],
-        "direction": "lower"
-      }
-    }
-  },
-  "embedding": {
-    "auc": 0.61,
-    "empirical_epsilon": {
-      "0.05": 3.4791953936219215,
-      "0.01": 3.023197554051876
-    },
-    "empirical_epsilon_details": {
-      "0.05": {
-        "epsilon": 3.4791953936219215,
-        "num_guesses": 100,
-        "correct_guesses": 100,
-        "candidate_num_guesses": [
-          5,
-          10,
-          15,
-          20,
-          25,
-          30,
-          35,
-          40,
-          45,
-          50,
-          55,
-          60,
-          65,
-          70,
-          75,
-          80,
-          85,
-          90,
-          95,
-          100
-        ],
-        "direction": "higher"
-      },
-      "0.01": {
-        "epsilon": 3.023197554051876,
-        "num_guesses": 100,
-        "correct_guesses": 100,
-        "candidate_num_guesses": [
-          5,
-          10,
-          15,
-          20,
-          25,
-          30,
-          35,
-          40,
-          45,
-          50,
-          55,
-          60,
-          65,
-          70,
-          75,
-          80,
-          85,
-          90,
-          95,
-          100
-        ],
-        "direction": "higher"
-      }
-    }
-  }
-}
\ No newline at end of file
diff --git a/qwen1.5-moe-a2.7b/base_attn_v2/epochs/epoch_005/adapter/README.md b/qwen1.5-moe-a2.7b/base_attn_v2/epochs/epoch_005/adapter/README.md
deleted file mode 100644
index 3b9e588251d937f16d19c9a0dcc3c6c5e3404063..0000000000000000000000000000000000000000
--- a/qwen1.5-moe-a2.7b/base_attn_v2/epochs/epoch_005/adapter/README.md
+++ /dev/null
@@ -1,207 +0,0 @@
----
-base_model: Qwen/Qwen1.5-MoE-A2.7B
-library_name: peft
-pipeline_tag: text-generation
-tags:
-- base_model:adapter:Qwen/Qwen1.5-MoE-A2.7B
-- lora
-- transformers
----
-
-# Model Card for Model ID
-
-<!-- Provide a quick summary of what the model is/does. -->
-
-
-
-## Model Details
-
-### Model Description
-
-<!-- Provide a longer summary of what this model is. -->
-
-
-
-- **Developed by:** [More Information Needed]
-- **Funded by [optional]:** [More Information Needed]
-- **Shared by [optional]:** [More Information Needed]
-- **Model type:** [More Information Needed]
-- **Language(s) (NLP):** [More Information Needed]
-- **License:** [More Information Needed]
-- **Finetuned from model [optional]:** [More Information Needed]
-
-### Model Sources [optional]
-
-<!-- Provide the basic links for the model. -->
-
-- **Repository:** [More Information Needed]
-- **Paper [optional]:** [More Information Needed]
-- **Demo [optional]:** [More Information Needed]
-
-## Uses
-
-<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
-
-### Direct Use
-
-<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
-
-[More Information Needed]
-
-### Downstream Use [optional]
-
-<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
-
-[More Information Needed]
-
-### Out-of-Scope Use
-
-<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
-
-[More Information Needed]
-
-## Bias, Risks, and Limitations
-
-<!-- This section is meant to convey both technical and sociotechnical limitations. -->
-
-[More Information Needed]
-
-### Recommendations
-
-<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
-
-Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
-
-## How to Get Started with the Model
-
-Use the code below to get started with the model.
-
-[More Information Needed]
-
-## Training Details
-
-### Training Data
-
-<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
-
-[More Information Needed]
-
-### Training Procedure
-
-<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
-
-#### Preprocessing [optional]
-
-[More Information Needed]
-
-
-#### Training Hyperparameters
-
-- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
-
-#### Speeds, Sizes, Times [optional]
-
-<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
-
-[More Information Needed]
-
-## Evaluation
-
-<!-- This section describes the evaluation protocols and provides the results. -->
-
-### Testing Data, Factors & Metrics
-
-#### Testing Data
-
-<!-- This should link to a Dataset Card if possible. -->
-
-[More Information Needed]
-
-#### Factors
-
-<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
-
-[More Information Needed]
-
-#### Metrics
-
-<!-- These are the evaluation metrics being used, ideally with a description of why. -->
-
-[More Information Needed]
-
-### Results
-
-[More Information Needed]
-
-#### Summary
-
-
-
-## Model Examination [optional]
-
-<!-- Relevant interpretability work for the model goes here -->
-
-[More Information Needed]
-
-## Environmental Impact
-
-<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
-
-Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
-
-- **Hardware Type:** [More Information Needed]
-- **Hours used:** [More Information Needed]
-- **Cloud Provider:** [More Information Needed]
-- **Compute Region:** [More Information Needed]
-- **Carbon Emitted:** [More Information Needed]
-
-## Technical Specifications [optional]
-
-### Model Architecture and Objective
-
-[More Information Needed]
-
-### Compute Infrastructure
-
-[More Information Needed]
-
-#### Hardware
-
-[More Information Needed]
-
-#### Software
-
-[More Information Needed]
-
-## Citation [optional]
-
-<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
-
-**BibTeX:**
-
-[More Information Needed]
-
-**APA:**
-
-[More Information Needed]
-
-## Glossary [optional]
-
-<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
-
-[More Information Needed]
-
-## More Information [optional]
-
-[More Information Needed]
-
-## Model Card Authors [optional]
-
-[More Information Needed]
-
-## Model Card Contact
-
-[More Information Needed]
-### Framework versions
-
-- PEFT 0.18.1
\ No newline at end of file
diff --git a/qwen1.5-moe-a2.7b/base_attn_v2/epochs/epoch_005/adapter/adapter_config.json b/qwen1.5-moe-a2.7b/base_attn_v2/epochs/epoch_005/adapter/adapter_config.json
deleted file mode 100644
index 37d244d234d1335c4acbc3c5f8577e0b83b28ed7..0000000000000000000000000000000000000000
--- a/qwen1.5-moe-a2.7b/base_attn_v2/epochs/epoch_005/adapter/adapter_config.json
+++ /dev/null
@@ -1,46 +0,0 @@
-{
-  "alora_invocation_tokens": null,
-  "alpha_pattern": {},
-  "arrow_config": null,
-  "auto_mapping": null,
-  "base_model_name_or_path": "Qwen/Qwen1.5-MoE-A2.7B",
-  "bias": "none",
-  "corda_config": null,
-  "ensure_weight_tying": true,
-  "eva_config": null,
-  "exclude_modules": null,
-  "fan_in_fan_out": false,
-  "inference_mode": true,
-  "init_lora_weights": true,
-  "layer_replication": null,
-  "layers_pattern": null,
-  "layers_to_transform": null,
-  "loftq_config": {},
-  "lora_alpha": 32,
-  "lora_bias": false,
-  "lora_dropout": 0.05,
-  "megatron_config": null,
-  "megatron_core": "megatron.core",
-  "modules_to_save": [
-    "lm_head",
-    "embed_tokens"
-  ],
-  "peft_type": "LORA",
-  "peft_version": "0.18.1",
-  "qalora_group_size": 16,
-  "r": 16,
-  "rank_pattern": {},
-  "revision": null,
-  "target_modules": [
-    "q_proj",
-    "o_proj",
-    "k_proj",
-    "v_proj"
-  ],
-  "target_parameters": null,
-  "task_type": "CAUSAL_LM",
-  "trainable_token_indices": null,
-  "use_dora": false,
-  "use_qalora": false,
-  "use_rslora": false
-}
\ No newline at end of file
diff --git a/qwen1.5-moe-a2.7b/base_attn_v2/epochs/epoch_005/audit_results.json b/qwen1.5-moe-a2.7b/base_attn_v2/epochs/epoch_005/audit_results.json
deleted file mode 100644
index 683efe18da5d2a612b572e9c9a10ede0f95728a8..0000000000000000000000000000000000000000
--- a/qwen1.5-moe-a2.7b/base_attn_v2/epochs/epoch_005/audit_results.json
+++ /dev/null
@@ -1,137 +0,0 @@
-{
-  "delta": 1e-05,
-  "num_canaries": 500,
-  "num_members": 250,
-  "paper_guess_fraction": 0.2,
-  "paper_guess_steps": 20,
-  "loss": {
-    "auc": 1.0,
-    "empirical_epsilon": {
-      "0.05": 3.4791953936219215,
-      "0.01": 3.023197554051876
-    },
-    "empirical_epsilon_details": {
-      "0.05": {
-        "epsilon": 3.4791953936219215,
-        "num_guesses": 100,
-        "correct_guesses": 100,
-        "candidate_num_guesses": [
-          5,
-          10,
-          15,
-          20,
-          25,
-          30,
-          35,
-          40,
-          45,
-          50,
-          55,
-          60,
-          65,
-          70,
-          75,
-          80,
-          85,
-          90,
-          95,
-          100
-        ],
-        "direction": "lower"
-      },
-      "0.01": {
-        "epsilon": 3.023197554051876,
-        "num_guesses": 100,
-        "correct_guesses": 100,
-        "candidate_num_guesses": [
-          5,
-          10,
-          15,
-          20,
-          25,
-          30,
-          35,
-          40,
-          45,
-          50,
-          55,
-          60,
-          65,
-          70,
-          75,
-          80,
-          85,
-          90,
-          95,
-          100
-        ],
-        "direction": "lower"
-      }
-    }
-  },
-  "embedding": {
-    "auc": 0.612,
-    "empirical_epsilon": {
-      "0.05": 3.4791953936219215,
-      "0.01": 3.023197554051876
-    },
-    "empirical_epsilon_details": {
-      "0.05": {
-        "epsilon": 3.4791953936219215,
-        "num_guesses": 100,
-        "correct_guesses": 100,
-        "candidate_num_guesses": [
-          5,
-          10,
-          15,
-          20,
-          25,
-          30,
-          35,
-          40,
-          45,
-          50,
-          55,
-          60,
-          65,
-          70,
-          75,
-          80,
-          85,
-          90,
-          95,
-          100
-        ],
-        "direction": "higher"
-      },
-      "0.01": {
-        "epsilon": 3.023197554051876,
-        "num_guesses": 100,
-        "correct_guesses": 100,
-        "candidate_num_guesses": [
-          5,
-          10,
-          15,
-          20,
-          25,
-          30,
-          35,
-          40,
-          45,
-          50,
-          55,
-          60,
-          65,
-          70,
-          75,
-          80,
-          85,
-          90,
-          95,
-          100
-        ],
-        "direction": "higher"
-      }
-    }
-  }
-}
\ No newline at end of file
diff --git a/qwen1.5-moe-a2.7b/dp3_attn/epochs/epoch_001/adapter/README.md b/qwen1.5-moe-a2.7b/dp3_attn/epochs/epoch_001/adapter/README.md
deleted file mode 100644
index 3b9e588251d937f16d19c9a0dcc3c6c5e3404063..0000000000000000000000000000000000000000
--- a/qwen1.5-moe-a2.7b/dp3_attn/epochs/epoch_001/adapter/README.md
+++ /dev/null
@@ -1,207 +0,0 @@
----
-base_model: Qwen/Qwen1.5-MoE-A2.7B
-library_name: peft
-pipeline_tag: text-generation
-tags:
-- base_model:adapter:Qwen/Qwen1.5-MoE-A2.7B
-- lora
-- transformers
----
-
-# Model Card for Model ID
-
-<!-- Provide a quick summary of what the model is/does. -->
-
-
-
-## Model Details
-
-### Model Description
-
-<!-- Provide a longer summary of what this model is. -->
-
-
-
-- **Developed by:** [More Information Needed]
-- **Funded by [optional]:** [More Information Needed]
-- **Shared by [optional]:** [More Information Needed]
-- **Model type:** [More Information Needed]
-- **Language(s) (NLP):** [More Information Needed]
-- **License:** [More Information Needed]
-- **Finetuned from model [optional]:** [More Information Needed]
-
-### Model Sources [optional]
-
-<!-- Provide the basic links for the model. -->
-
-- **Repository:** [More Information Needed]
-- **Paper [optional]:** [More Information Needed]
-- **Demo [optional]:** [More Information Needed]
-
-## Uses
-
-<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
-
-### Direct Use
-
-<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
-
-[More Information Needed]
-
-### Downstream Use [optional]
-
-<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
-
-[More Information Needed]
-
-### Out-of-Scope Use
-
-<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
-
-[More Information Needed]
-
-## Bias, Risks, and Limitations
-
-<!-- This section is meant to convey both technical and sociotechnical limitations. -->
-
-[More Information Needed]
-
-### Recommendations
-
-<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
-
-Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
-
-## How to Get Started with the Model
-
-Use the code below to get started with the model.
-
-[More Information Needed]
-
-## Training Details
-
-### Training Data
-
-<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
-
-[More Information Needed]
-
-### Training Procedure
-
-<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
-
-#### Preprocessing [optional]
-
-[More Information Needed]
-
-
-#### Training Hyperparameters
-
-- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
-
-#### Speeds, Sizes, Times [optional]
-
-<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
-
-[More Information Needed]
-
-## Evaluation
-
-<!-- This section describes the evaluation protocols and provides the results. -->
-
-### Testing Data, Factors & Metrics
-
-#### Testing Data
-
-<!-- This should link to a Dataset Card if possible. -->
-
-[More Information Needed]
-
-#### Factors
-
-<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
-
-[More Information Needed]
-
-#### Metrics
-
-<!-- These are the evaluation metrics being used, ideally with a description of why. -->
-
-[More Information Needed]
-
-### Results
-
-[More Information Needed]
-
-#### Summary
-
-
-
-## Model Examination [optional]
-
-<!-- Relevant interpretability work for the model goes here -->
-
-[More Information Needed]
-
-## Environmental Impact
-
-<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
-
-Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
-
-- **Hardware Type:** [More Information Needed]
-- **Hours used:** [More Information Needed]
-- **Cloud Provider:** [More Information Needed]
-- **Compute Region:** [More Information Needed]
-- **Carbon Emitted:** [More Information Needed]
-
-## Technical Specifications [optional]
-
-### Model Architecture and Objective
-
-[More Information Needed]
-
-### Compute Infrastructure
-
-[More Information Needed]
-
-#### Hardware
-
-[More Information Needed]
-
-#### Software
-
-[More Information Needed]
-
-## Citation [optional]
-
-<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
-
-**BibTeX:**
-
-[More Information Needed]
-
-**APA:**
-
-[More Information Needed]
-
-## Glossary [optional]
-
-<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
-
-[More Information Needed]
-
-## More Information [optional]
-
-[More Information Needed]
-
-## Model Card Authors [optional]
-
-[More Information Needed]
-
-## Model Card Contact
-
-[More Information Needed]
-### Framework versions
-
-- PEFT 0.18.1
\ No newline at end of file
diff --git a/qwen1.5-moe-a2.7b/dp3_attn/epochs/epoch_001/adapter/adapter_config.json b/qwen1.5-moe-a2.7b/dp3_attn/epochs/epoch_001/adapter/adapter_config.json
deleted file mode 100644
index 169302d5b6f9cbf086e1b5f9ebbfe5dd2cdacc6e..0000000000000000000000000000000000000000
--- a/qwen1.5-moe-a2.7b/dp3_attn/epochs/epoch_001/adapter/adapter_config.json
+++ /dev/null
@@ -1,46 +0,0 @@
-{
-  "alora_invocation_tokens": null,
-  "alpha_pattern": {},
-  "arrow_config": null,
-  "auto_mapping": null,
-  "base_model_name_or_path": "Qwen/Qwen1.5-MoE-A2.7B",
-  "bias": "none",
-  "corda_config": null,
-  "ensure_weight_tying": true,
-  "eva_config": null,
-  "exclude_modules": null,
-  "fan_in_fan_out": false,
-  "inference_mode": true,
-  "init_lora_weights": true,
-  "layer_replication": null,
-  "layers_pattern": null,
-  "layers_to_transform": null,
-  "loftq_config": {},
-  "lora_alpha": 32,
-  "lora_bias": false,
-  "lora_dropout": 0.05,
-  "megatron_config": null,
-  "megatron_core": "megatron.core",
-  "modules_to_save": [
-    "lm_head",
-    "embed_tokens"
-  ],
-  "peft_type": "LORA",
-  "peft_version": "0.18.1",
-  "qalora_group_size": 16,
-  "r": 16,
-  "rank_pattern": {},
-  "revision": null,
-  "target_modules": [
-    "k_proj",
-    "v_proj",
-    "q_proj",
-    "o_proj"
-  ],
-  "target_parameters": null,
-  "task_type": "CAUSAL_LM",
-  "trainable_token_indices": null,
-  "use_dora": false,
-  "use_qalora": false,
-  "use_rslora": false
-}
\ No newline at end of file
diff --git a/qwen1.5-moe-a2.7b/dp3_attn/epochs/epoch_001/audit_results.json b/qwen1.5-moe-a2.7b/dp3_attn/epochs/epoch_001/audit_results.json
deleted file mode 100644
index 004880a4c0ef331c44e1c5045b7cb8cb297dd77e..0000000000000000000000000000000000000000
--- a/qwen1.5-moe-a2.7b/dp3_attn/epochs/epoch_001/audit_results.json
+++ /dev/null
@@ -1,137 +0,0 @@
-{
-  "delta": 1e-05,
-  "num_canaries": 500,
-  "num_members": 250,
-  "paper_guess_fraction": 0.2,
-  "paper_guess_steps": 20,
-  "loss": {
-    "auc": 0.515872,
-    "empirical_epsilon": {
-      "0.05": 0.0,
-      "0.01": 0.0
-    },
-    "empirical_epsilon_details": {
-      "0.05": {
-        "epsilon": 0.0,
-        "num_guesses": 0,
-        "correct_guesses": 0,
-        "candidate_num_guesses": [
-          5,
-          10,
-          15,
-          20,
-          25,
-          30,
-          35,
-          40,
-          45,
-          50,
-          55,
-          60,
-          65,
-          70,
-          75,
-          80,
-          85,
-          90,
-          95,
-          100
-        ],
-        "direction": "lower"
-      },
-      "0.01": {
-        "epsilon": 0.0,
-        "num_guesses": 0,
-        "correct_guesses": 0,
-        "candidate_num_guesses": [
-          5,
-          10,
-          15,
-          20,
-          25,
-          30,
-          35,
-          40,
-          45,
-          50,
-          55,
-          60,
-          65,
-          70,
-          75,
-          80,
-          85,
-          90,
-          95,
-          100
-        ],
-        "direction": "lower"
-      }
-    }
-  },
-  "embedding": {
-    "auc": 0.534056,
-    "empirical_epsilon": {
-      "0.05": 0.0,
-      "0.01": 0.0
-    },
-    "empirical_epsilon_details": {
-      "0.05": {
-        "epsilon": 0.0,
-        "num_guesses": 0,
-        "correct_guesses": 0,
-        "candidate_num_guesses": [
-          5,
-          10,
-          15,
-          20,
-          25,
-          30,
-          35,
-          40,
-          45,
-          50,
-          55,
-          60,
-          65,
-          70,
-          75,
-          80,
-          85,
-          90,
-          95,
-          100
-        ],
-        "direction": "lower"
-      },
-      "0.01": {
-        "epsilon": 0.0,
-        "num_guesses": 0,
-        "correct_guesses": 0,
-        "candidate_num_guesses": [
-          5,
-          10,
-          15,
-          20,
-          25,
-          30,
-          35,
-          40,
-          45,
-          50,
-          55,
-          60,
-          65,
-          70,
-          75,
-          80,
-          85,
-          90,
-          95,
-          100
-        ],
-        "direction": "lower"
-      }
-    }
-  }
-}
\ No newline at end of file
diff --git a/qwen1.5-moe-a2.7b/dp3_attn/epochs/epoch_002/adapter/README.md b/qwen1.5-moe-a2.7b/dp3_attn/epochs/epoch_002/adapter/README.md
deleted file mode 100644
index 3b9e588251d937f16d19c9a0dcc3c6c5e3404063..0000000000000000000000000000000000000000
--- a/qwen1.5-moe-a2.7b/dp3_attn/epochs/epoch_002/adapter/README.md
+++ /dev/null
@@ -1,207 +0,0 @@
----
-base_model: Qwen/Qwen1.5-MoE-A2.7B
-library_name: peft
-pipeline_tag: text-generation
-tags:
-- base_model:adapter:Qwen/Qwen1.5-MoE-A2.7B
-- lora
-- transformers
----
-
-# Model Card for Model ID
-
-<!-- Provide a quick summary of what the model is/does. -->
-
-
-
-## Model Details
-
-### Model Description
-
-<!-- Provide a longer summary of what this model is. -->
-
-
-
-- **Developed by:** [More Information Needed]
-- **Funded by [optional]:** [More Information Needed]
-- **Shared by [optional]:** [More Information Needed]
-- **Model type:** [More Information Needed]
-- **Language(s) (NLP):** [More Information Needed]
-- **License:** [More Information Needed]
-- **Finetuned from model [optional]:** [More Information Needed]
-
-### Model Sources [optional]
-
-<!-- Provide the basic links for the model. -->
-
-- **Repository:** [More Information Needed]
-- **Paper [optional]:** [More Information Needed]
-- **Demo [optional]:** [More Information Needed]
-
-## Uses
-
-<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
-
-### Direct Use
-
-<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
-
-[More Information Needed]
-
-### Downstream Use [optional]
-
-<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
-
-[More Information Needed]
-
-### Out-of-Scope Use
-
-<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
-
-[More Information Needed]
-
-## Bias, Risks, and Limitations
-
-<!-- This section is meant to convey both technical and sociotechnical limitations. -->
-
-[More Information Needed]
-
-### Recommendations
-
-<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
-
-Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
-
-## How to Get Started with the Model
-
-Use the code below to get started with the model.
-
-[More Information Needed]
-
-## Training Details
-
-### Training Data
-
-<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
-
-[More Information Needed]
-
-### Training Procedure
-
-<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
-
-#### Preprocessing [optional]
-
-[More Information Needed]
-
-
-#### Training Hyperparameters
-
-- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
-
-#### Speeds, Sizes, Times [optional]
-
-<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
-
-[More Information Needed]
-
-## Evaluation
-
-<!-- This section describes the evaluation protocols and provides the results. -->
-
-### Testing Data, Factors & Metrics
-
-#### Testing Data
-
-<!-- This should link to a Dataset Card if possible. -->
-
-[More Information Needed]
-
-#### Factors
-
-<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
-
-[More Information Needed]
-
-#### Metrics
-
-<!-- These are the evaluation metrics being used, ideally with a description of why. -->
-
-[More Information Needed]
-
-### Results
-
-[More Information Needed]
-
-#### Summary
-
-
-
-## Model Examination [optional]
-
-<!-- Relevant interpretability work for the model goes here -->
-
-[More Information Needed]
-
-## Environmental Impact
-
-<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
-
-Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
-
-- **Hardware Type:** [More Information Needed]
-- **Hours used:** [More Information Needed]
-- **Cloud Provider:** [More Information Needed]
-- **Compute Region:** [More Information Needed]
-- **Carbon Emitted:** [More Information Needed]
-
-## Technical Specifications [optional]
-
-### Model Architecture and Objective
-
-[More Information Needed]
-
-### Compute Infrastructure
-
-[More Information Needed]
-
-#### Hardware
-
-[More Information Needed]
-
-#### Software
-
-[More Information Needed]
-
-## Citation [optional]
-
-<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
-
-**BibTeX:**
-
-[More Information Needed]
-
-**APA:**
-
-[More Information Needed]
-
-## Glossary [optional]
-
-<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
-
-[More Information Needed]
-
-## More Information [optional]
-
-[More Information Needed]
-
-## Model Card Authors [optional]
-
-[More Information Needed]
-
-## Model Card Contact
-
-[More Information Needed]
-### Framework versions
-
-- PEFT 0.18.1
\ No newline at end of file
diff --git a/qwen1.5-moe-a2.7b/dp3_attn/epochs/epoch_002/adapter/adapter_config.json b/qwen1.5-moe-a2.7b/dp3_attn/epochs/epoch_002/adapter/adapter_config.json
deleted file mode 100644
index 169302d5b6f9cbf086e1b5f9ebbfe5dd2cdacc6e..0000000000000000000000000000000000000000
--- a/qwen1.5-moe-a2.7b/dp3_attn/epochs/epoch_002/adapter/adapter_config.json
+++ /dev/null
@@ -1,46 +0,0 @@
-{
-  "alora_invocation_tokens": null,
-  "alpha_pattern": {},
-  "arrow_config": null,
-  "auto_mapping": null,
-  "base_model_name_or_path": "Qwen/Qwen1.5-MoE-A2.7B",
-  "bias": "none",
-  "corda_config": null,
-  "ensure_weight_tying": true,
-  "eva_config": null,
-  "exclude_modules": null,
-  "fan_in_fan_out": false,
-  "inference_mode": true,
-  "init_lora_weights": true,
-  "layer_replication": null,
-  "layers_pattern": null,
-  "layers_to_transform": null,
-  "loftq_config": {},
-  "lora_alpha": 32,
-  "lora_bias": false,
-  "lora_dropout": 0.05,
-  "megatron_config": null,
-  "megatron_core": "megatron.core",
-  "modules_to_save": [
-    "lm_head",
-    "embed_tokens"
-  ],
-  "peft_type": "LORA",
-  "peft_version": "0.18.1",
-  "qalora_group_size": 16,
-  "r": 16,
-  "rank_pattern": {},
-  "revision": null,
-  "target_modules": [
-    "k_proj",
-    "v_proj",
-    "q_proj",
-    "o_proj"
-  ],
-  "target_parameters": null,
-  "task_type": "CAUSAL_LM",
-  "trainable_token_indices": null,
-  "use_dora": false,
-  "use_qalora": false,
-  "use_rslora": false
-}
\ No newline at end of file
diff --git a/qwen1.5-moe-a2.7b/dp3_attn/epochs/epoch_002/audit_results.json b/qwen1.5-moe-a2.7b/dp3_attn/epochs/epoch_002/audit_results.json
deleted file mode 100644
index 236733a277520b63146b02208f59e9fd87fe5faa..0000000000000000000000000000000000000000
--- a/qwen1.5-moe-a2.7b/dp3_attn/epochs/epoch_002/audit_results.json
+++ /dev/null
@@ -1,137 +0,0 @@
-{
-  "delta": 1e-05,
-  "num_canaries": 500,
-  "num_members": 250,
-  "paper_guess_fraction": 0.2,
-  "paper_guess_steps": 20,
-  "loss": {
-    "auc": 0.545624,
-    "empirical_epsilon": {
-      "0.05": 0.0,
-      "0.01": 0.0
-    },
-    "empirical_epsilon_details": {
-      "0.05": {
-        "epsilon": 0.0,
-        "num_guesses": 0,
-        "correct_guesses": 0,
-        "candidate_num_guesses": [
-          5,
-          10,
-          15,
-          20,
-          25,
-          30,
-          35,
-          40,
-          45,
-          50,
-          55,
-          60,
-          65,
-          70,
-          75,
-          80,
-          85,
-          90,
-          95,
-          100
-        ],
-        "direction": "lower"
-      },
-      "0.01": {
-        "epsilon": 0.0,
-        "num_guesses": 0,
-        "correct_guesses": 0,
-        "candidate_num_guesses": [
-          5,
-          10,
-          15,
-          20,
-          25,
-          30,
-          35,
-          40,
-          45,
-          50,
-          55,
-          60,
-          65,
-          70,
-          75,
-          80,
-          85,
-          90,
-          95,
-          100
-        ],
-        "direction": "lower"
-      }
-    }
-  },
-  "embedding": {
-    "auc": 0.531072,
-    "empirical_epsilon": {
-      "0.05": 0.17315445840358734,
-      "0.01": 0.0
-    },
-    "empirical_epsilon_details": {
-      "0.05": {
-        "epsilon": 0.17315445840358734,
-        "num_guesses": 5,
-        "correct_guesses": 5,
-        "candidate_num_guesses": [
-          5,
-          10,
-          15,
-          20,
-          25,
-          30,
-          35,
-          40,
-          45,
-          50,
-          55,
-          60,
-          65,
-          70,
-          75,
-          80,
-          85,
-          90,
-          95,
-          100
-        ],
-        "direction": "higher"
-      },
-      "0.01": {
-        "epsilon": 0.0,
-        "num_guesses": 0,
-        "correct_guesses": 0,
-        "candidate_num_guesses": [
-          5,
-          10,
-          15,
-          20,
-          25,
-          30,
-          35,
-          40,
-          45,
-          50,
-          55,
-          60,
-          65,
-          70,
-          75,
-          80,
-          85,
-          90,
-          95,
-          100
-        ],
-        "direction": "lower"
-      }
-    }
-  }
-}
\ No newline at end of file
diff --git a/qwen1.5-moe-a2.7b/dp3_attn/epochs/epoch_003/adapter/README.md b/qwen1.5-moe-a2.7b/dp3_attn/epochs/epoch_003/adapter/README.md
deleted file mode 100644
index 3b9e588251d937f16d19c9a0dcc3c6c5e3404063..0000000000000000000000000000000000000000
--- a/qwen1.5-moe-a2.7b/dp3_attn/epochs/epoch_003/adapter/README.md
+++ /dev/null
@@ -1,207 +0,0 @@
----
-base_model: Qwen/Qwen1.5-MoE-A2.7B
-library_name: peft
-pipeline_tag: text-generation
-tags:
-- base_model:adapter:Qwen/Qwen1.5-MoE-A2.7B
-- lora
-- transformers
----
-
-# Model Card for Model ID
-
-<!-- Provide a quick summary of what the model is/does. -->
-
-
-
-## Model Details
-
-### Model Description
-
-<!-- Provide a longer summary of what this model is. -->
-
-
-
-- **Developed by:** [More Information Needed]
-- **Funded by [optional]:** [More Information Needed]
-- **Shared by [optional]:** [More Information Needed]
-- **Model type:** [More Information Needed]
-- **Language(s) (NLP):** [More Information Needed]
-- **License:** [More Information Needed]
-- **Finetuned from model [optional]:** [More Information Needed]
-
-### Model Sources [optional]
-
-<!-- Provide the basic links for the model. -->
-
-- **Repository:** [More Information Needed]
-- **Paper [optional]:** [More Information Needed]
-- **Demo [optional]:** [More Information Needed]
-
-## Uses
-
-<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
-
-### Direct Use
-
-<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
-
-[More Information Needed]
-
-### Downstream Use [optional]
-
-<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
-
-[More Information Needed]
-
-### Out-of-Scope Use
-
-<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
-
-[More Information Needed]
-
-## Bias, Risks, and Limitations
-
-<!-- This section is meant to convey both technical and sociotechnical limitations. -->
-
-[More Information Needed]
-
-### Recommendations
-
-<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
-
-Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
-
-## How to Get Started with the Model
-
-Use the code below to get started with the model.
-
-[More Information Needed]
-
-## Training Details
-
-### Training Data
-
-<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
-
-[More Information Needed]
-
-### Training Procedure
-
-<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
-
-#### Preprocessing [optional]
-
-[More Information Needed]
-
-
-#### Training Hyperparameters
-
-- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
-
-#### Speeds, Sizes, Times [optional]
-
-<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
-
-[More Information Needed]
-
-## Evaluation
-
-<!-- This section describes the evaluation protocols and provides the results. -->
-
-### Testing Data, Factors & Metrics
-
-#### Testing Data
-
-<!-- This should link to a Dataset Card if possible. -->
-
-[More Information Needed]
-
-#### Factors
-
-<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
-
-[More Information Needed]
-
-#### Metrics
-
-<!-- These are the evaluation metrics being used, ideally with a description of why. -->
-
-[More Information Needed]
-
-### Results
-
-[More Information Needed]
-
-#### Summary
-
-
-
-## Model Examination [optional]
-
-<!-- Relevant interpretability work for the model goes here -->
-
-[More Information Needed]
-
-## Environmental Impact
-
-<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
-
-Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
-
-- **Hardware Type:** [More Information Needed]
-- **Hours used:** [More Information Needed]
-- **Cloud Provider:** [More Information Needed]
-- **Compute Region:** [More Information Needed]
-- **Carbon Emitted:** [More Information Needed]
-
-## Technical Specifications [optional]
-
-### Model Architecture and Objective
-
-[More Information Needed]
-
-### Compute Infrastructure
-
-[More Information Needed]
-
-#### Hardware
-
-[More Information Needed]
-
-#### Software
-
-[More Information Needed]
-
-## Citation [optional]
-
-<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
-
-**BibTeX:**
-
-[More Information Needed]
-
-**APA:**
-
-[More Information Needed]
-
-## Glossary [optional]
-
-<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
-
-[More Information Needed]
-
-## More Information [optional]
-
-[More Information Needed]
-
-## Model Card Authors [optional]
-
-[More Information Needed]
-
-## Model Card Contact
-
-[More Information Needed]
-### Framework versions
-
-- PEFT 0.18.1
\ No newline at end of file
diff --git a/qwen1.5-moe-a2.7b/dp3_attn/epochs/epoch_003/adapter/adapter_config.json b/qwen1.5-moe-a2.7b/dp3_attn/epochs/epoch_003/adapter/adapter_config.json
deleted file mode 100644
index 169302d5b6f9cbf086e1b5f9ebbfe5dd2cdacc6e..0000000000000000000000000000000000000000
--- a/qwen1.5-moe-a2.7b/dp3_attn/epochs/epoch_003/adapter/adapter_config.json
+++ /dev/null
@@ -1,46 +0,0 @@
-{
-  "alora_invocation_tokens": null,
-  "alpha_pattern": {},
-  "arrow_config": null,
-  "auto_mapping": null,
-  "base_model_name_or_path": "Qwen/Qwen1.5-MoE-A2.7B",
-  "bias": "none",
-  "corda_config": null,
-  "ensure_weight_tying": true,
-  "eva_config": null,
-  "exclude_modules": null,
-  "fan_in_fan_out": false,
-  "inference_mode": true,
-  "init_lora_weights": true,
-  "layer_replication": null,
-  "layers_pattern": null,
-  "layers_to_transform": null,
-  "loftq_config": {},
-  "lora_alpha": 32,
-  "lora_bias": false,
-  "lora_dropout": 0.05,
-  "megatron_config": null,
-  "megatron_core": "megatron.core",
-  "modules_to_save": [
-    "lm_head",
-    "embed_tokens"
-  ],
-  "peft_type": "LORA",
-  "peft_version": "0.18.1",
-  "qalora_group_size": 16,
-  "r": 16,
-  "rank_pattern": {},
-  "revision": null,
-  "target_modules": [
-    "k_proj",
-    "v_proj",
-    "q_proj",
-    "o_proj"
-  ],
-  "target_parameters": null,
-  "task_type": "CAUSAL_LM",
-  "trainable_token_indices": null,
-  "use_dora": false,
-  "use_qalora": false,
-  "use_rslora": false
-}
\ No newline at end of file
diff --git a/qwen1.5-moe-a2.7b/dp3_attn/epochs/epoch_003/audit_results.json b/qwen1.5-moe-a2.7b/dp3_attn/epochs/epoch_003/audit_results.json
deleted file mode 100644
index 9717a35951c2c45cdc7e33b6b2e3be56e24664bd..0000000000000000000000000000000000000000
--- a/qwen1.5-moe-a2.7b/dp3_attn/epochs/epoch_003/audit_results.json
+++ /dev/null
@@ -1,137 +0,0 @@
-{
-  "delta": 1e-05,
-  "num_canaries": 500,
-  "num_members": 250,
-  "paper_guess_fraction": 0.2,
-  "paper_guess_steps": 20,
-  "loss": {
-    "auc": 0.538384,
-    "empirical_epsilon": {
-      "0.05": 0.0,
-      "0.01": 0.0
-    },
-    "empirical_epsilon_details": {
-      "0.05": {
-        "epsilon": 0.0,
-        "num_guesses": 0,
-        "correct_guesses": 0,
-        "candidate_num_guesses": [
-          5,
-          10,
-          15,
-          20,
-          25,
-          30,
-          35,
-          40,
-          45,
-          50,
-          55,
-          60,
-          65,
-          70,
-          75,
-          80,
-          85,
-          90,
-          95,
-          100
-        ],
-        "direction": "lower"
-      },
-      "0.01": {
-        "epsilon": 0.0,
-        "num_guesses": 0,
-        "correct_guesses": 0,
-        "candidate_num_guesses": [
-          5,
-          10,
-          15,
-          20,
-          25,
-          30,
-          35,
-          40,
-          45,
-          50,
-          55,
-          60,
-          65,
-          70,
-          75,
-          80,
-          85,
-          90,
-          95,
-          100
-        ],
-        "direction": "lower"
-      }
-    }
-  },
-  "embedding": {
-    "auc": 0.531696,
-    "empirical_epsilon": {
-      "0.05": 0.17315445840358734,
-      "0.01": 0.0
-    },
-    "empirical_epsilon_details": {
-      "0.05": {
-        "epsilon": 0.17315445840358734,
-        "num_guesses": 5,
-        "correct_guesses": 5,
-        "candidate_num_guesses": [
-          5,
-          10,
-          15,
-          20,
-          25,
-          30,
-          35,
-          40,
-          45,
-          50,
-          55,
-          60,
-          65,
-          70,
-          75,
-          80,
-          85,
-          90,
-          95,
-          100
-        ],
-        "direction": "higher"
-      },
-      "0.01": {
-        "epsilon": 0.0,
-        "num_guesses": 0,
-        "correct_guesses": 0,
-        "candidate_num_guesses": [
-          5,
-          10,
-          15,
-          20,
-          25,
-          30,
-          35,
-          40,
-          45,
-          50,
-          55,
-          60,
-          65,
-          70,
-          75,
-          80,
-          85,
-          90,
-          95,
-          100
-        ],
-        "direction": "lower"
-      }
-    }
-  }
-}
\ No newline at end of file
diff --git a/qwen1.5-moe-a2.7b/dp3_attn_v2/epochs/epoch_001/adapter/README.md b/qwen1.5-moe-a2.7b/dp3_attn_v2/epochs/epoch_001/adapter/README.md
deleted file mode 100644
index 3b9e588251d937f16d19c9a0dcc3c6c5e3404063..0000000000000000000000000000000000000000
--- a/qwen1.5-moe-a2.7b/dp3_attn_v2/epochs/epoch_001/adapter/README.md
+++ /dev/null
@@ -1,207 +0,0 @@
----
-base_model: Qwen/Qwen1.5-MoE-A2.7B
-library_name: peft
-pipeline_tag: text-generation
-tags:
-- base_model:adapter:Qwen/Qwen1.5-MoE-A2.7B
-- lora
-- transformers
----
-
-# Model Card for Model ID
-
-<!-- Provide a quick summary of what the model is/does. -->
-
-
-
-## Model Details
-
-### Model Description
-
-<!-- Provide a longer summary of what this model is. -->
-
-
-
-- **Developed by:** [More Information Needed]
-- **Funded by [optional]:** [More Information Needed]
-- **Shared by [optional]:** [More Information Needed]
-- **Model type:** [More Information Needed]
-- **Language(s) (NLP):** [More Information Needed]
-- **License:** [More Information Needed]
-- **Finetuned from model [optional]:** [More Information Needed]
-
-### Model Sources [optional]
-
-<!-- Provide the basic links for the model. -->
-
-- **Repository:** [More Information Needed]
-- **Paper [optional]:** [More Information Needed]
-- **Demo [optional]:** [More Information Needed]
-
-## Uses
-
-<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
-
-### Direct Use
-
-<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
-
-[More Information Needed]
-
-### Downstream Use [optional]
-
-<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
-
-[More Information Needed]
-
-### Out-of-Scope Use
-
-<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
-
-[More Information Needed]
-
-## Bias, Risks, and Limitations
-
-<!-- This section is meant to convey both technical and sociotechnical limitations. -->
-
-[More Information Needed]
-
-### Recommendations
-
-<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
-
-Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
-
-## How to Get Started with the Model
-
-Use the code below to get started with the model.
-
-[More Information Needed]
-
-## Training Details
-
-### Training Data
-
-<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
-
-[More Information Needed]
-
-### Training Procedure
-
-<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
-
-#### Preprocessing [optional]
-
-[More Information Needed]
-
-
-#### Training Hyperparameters
-
-- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
-
-#### Speeds, Sizes, Times [optional]
-
-<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
-
-[More Information Needed]
-
-## Evaluation
-
-<!-- This section describes the evaluation protocols and provides the results. -->
-
-### Testing Data, Factors & Metrics
-
-#### Testing Data
-
-<!-- This should link to a Dataset Card if possible. -->
-
-[More Information Needed]
-
-#### Factors
-
-<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
-
-[More Information Needed]
-
-#### Metrics
-
-<!-- These are the evaluation metrics being used, ideally with a description of why. -->
-
-[More Information Needed]
-
-### Results
-
-[More Information Needed]
-
-#### Summary
-
-
-
-## Model Examination [optional]
-
-<!-- Relevant interpretability work for the model goes here -->
-
-[More Information Needed]
-
-## Environmental Impact
-
-<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
-
-Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
-
-- **Hardware Type:** [More Information Needed]
-- **Hours used:** [More Information Needed]
-- **Cloud Provider:** [More Information Needed]
-- **Compute Region:** [More Information Needed]
-- **Carbon Emitted:** [More Information Needed]
-
-## Technical Specifications [optional]
-
-### Model Architecture and Objective
-
-[More Information Needed]
-
-### Compute Infrastructure
-
-[More Information Needed]
-
-#### Hardware
-
-[More Information Needed]
-
-#### Software
-
-[More Information Needed]
-
-## Citation [optional]
-
-<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
-
-**BibTeX:**
-
-[More Information Needed]
-
-**APA:**
-
-[More Information Needed]
-
-## Glossary [optional]
-
-<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
-
-[More Information Needed]
-
-## More Information [optional]
-
-[More Information Needed]
-
-## Model Card Authors [optional]
-
-[More Information Needed]
-
-## Model Card Contact
-
-[More Information Needed]
-### Framework versions
-
-- PEFT 0.18.1
\ No newline at end of file
diff --git a/qwen1.5-moe-a2.7b/dp3_attn_v2/epochs/epoch_001/adapter/adapter_config.json b/qwen1.5-moe-a2.7b/dp3_attn_v2/epochs/epoch_001/adapter/adapter_config.json
deleted file mode 100644
index 5cc38c385aa174ee4fc7225fc7482a1c8f9460b4..0000000000000000000000000000000000000000
--- a/qwen1.5-moe-a2.7b/dp3_attn_v2/epochs/epoch_001/adapter/adapter_config.json
+++ /dev/null
@@ -1,46 +0,0 @@
-{
-  "alora_invocation_tokens": null,
-  "alpha_pattern": {},
-  "arrow_config": null,
-  "auto_mapping": null,
-  "base_model_name_or_path": "Qwen/Qwen1.5-MoE-A2.7B",
-  "bias": "none",
-  "corda_config": null,
-  "ensure_weight_tying": true,
-  "eva_config": null,
-  "exclude_modules": null,
-  "fan_in_fan_out": false,
-  "inference_mode": true,
-  "init_lora_weights": true,
-  "layer_replication": null,
-  "layers_pattern": null,
-  "layers_to_transform": null,
-  "loftq_config": {},
-  "lora_alpha": 32,
-  "lora_bias": false,
-  "lora_dropout": 0.05,
-  "megatron_config": null,
-  "megatron_core": "megatron.core",
-  "modules_to_save": [
-    "lm_head",
-    "embed_tokens"
-  ],
-  "peft_type": "LORA",
-  "peft_version": "0.18.1",
-  "qalora_group_size": 16,
-  "r": 16,
-  "rank_pattern": {},
-  "revision": null,
-  "target_modules": [
-    "v_proj",
-    "k_proj",
-    "q_proj",
-    "o_proj"
-  ],
-  "target_parameters": null,
-  "task_type": "CAUSAL_LM",
-  "trainable_token_indices": null,
-  "use_dora": false,
-  "use_qalora": false,
-  "use_rslora": false
-}
\ No newline at end of file
diff --git a/qwen1.5-moe-a2.7b/dp3_attn_v2/epochs/epoch_001/audit_results.json b/qwen1.5-moe-a2.7b/dp3_attn_v2/epochs/epoch_001/audit_results.json
deleted file mode 100644
index fcad79aeeafa437e142220be28ea5b8bb7c17d0b..0000000000000000000000000000000000000000
--- a/qwen1.5-moe-a2.7b/dp3_attn_v2/epochs/epoch_001/audit_results.json
+++ /dev/null
@@ -1,137 +0,0 @@
-{
-  "delta": 1e-05,
-  "num_canaries": 500,
-  "num_members": 250,
-  "paper_guess_fraction": 0.2,
-  "paper_guess_steps": 20,
-  "loss": {
-    "auc": 0.50572,
-    "empirical_epsilon": {
-      "0.05": 0.0,
-      "0.01": 0.0
-    },
-    "empirical_epsilon_details": {
-      "0.05": {
-        "epsilon": 0.0,
-        "num_guesses": 0,
-        "correct_guesses": 0,
-        "candidate_num_guesses": [
-          5,
-          10,
-          15,
-          20,
-          25,
-          30,
-          35,
-          40,
-          45,
-          50,
-          55,
-          60,
-          65,
-          70,
-          75,
-          80,
-          85,
-          90,
-          95,
-          100
-        ],
-        "direction": "lower"
-      },
-      "0.01": {
-        "epsilon": 0.0,
-        "num_guesses": 0,
-        "correct_guesses": 0,
-        "candidate_num_guesses": [
-          5,
-          10,
-          15,
-          20,
-          25,
-          30,
-          35,
-          40,
-          45,
-          50,
-          55,
-          60,
-          65,
-          70,
-          75,
-          80,
-          85,
-          90,
-          95,
-          100
-        ],
-        "direction": "lower"
-      }
-    }
-  },
-  "embedding": {
-    "auc": 0.53652,
-    "empirical_epsilon": {
-      "0.05": 0.05696781910955906,
-      "0.01": 0.0
-    },
-    "empirical_epsilon_details": {
-      "0.05": {
-        "epsilon": 0.05696781910955906,
-        "num_guesses": 80,
-        "correct_guesses": 49,
-        "candidate_num_guesses": [
-          5,
-          10,
-          15,
-          20,
-          25,
-          30,
-          35,
-          40,
-          45,
-          50,
-          55,
-          60,
-          65,
-          70,
-          75,
-          80,
-          85,
-          90,
-          95,
-          100
-        ],
-        "direction": "higher"
-      },
-      "0.01": {
-        "epsilon": 0.0,
-        "num_guesses": 0,
-        "correct_guesses": 0,
-        "candidate_num_guesses": [
-          5,
-          10,
-          15,
-          20,
-          25,
-          30,
-          35,
-          40,
-          45,
-          50,
-          55,
-          60,
-          65,
-          70,
-          75,
-          80,
-          85,
-          90,
-          95,
-          100
-        ],
-        "direction": "lower"
-      }
-    }
-  }
-}
\ No newline at end of file
diff --git a/qwen1.5-moe-a2.7b/dp3_attn_v2/epochs/epoch_002/adapter/README.md b/qwen1.5-moe-a2.7b/dp3_attn_v2/epochs/epoch_002/adapter/README.md
deleted file mode 100644
index 3b9e588251d937f16d19c9a0dcc3c6c5e3404063..0000000000000000000000000000000000000000
--- a/qwen1.5-moe-a2.7b/dp3_attn_v2/epochs/epoch_002/adapter/README.md
+++ /dev/null
@@ -1,207 +0,0 @@
----
-base_model: Qwen/Qwen1.5-MoE-A2.7B
-library_name: peft
-pipeline_tag: text-generation
-tags:
-- base_model:adapter:Qwen/Qwen1.5-MoE-A2.7B
-- lora
-- transformers
----
-
-# Model Card for Model ID
-
-<!-- Provide a quick summary of what the model is/does. -->
-
-
-
-## Model Details
-
-### Model Description
-
-<!-- Provide a longer summary of what this model is. -->
-
-
-
-- **Developed by:** [More Information Needed]
-- **Funded by [optional]:** [More Information Needed]
-- **Shared by [optional]:** [More Information Needed]
-- **Model type:** [More Information Needed]
-- **Language(s) (NLP):** [More Information Needed]
-- **License:** [More Information Needed]
-- **Finetuned from model [optional]:** [More Information Needed]
-
-### Model Sources [optional]
-
-<!-- Provide the basic links for the model. -->
-
-- **Repository:** [More Information Needed]
-- **Paper [optional]:** [More Information Needed]
-- **Demo [optional]:** [More Information Needed]
-
-## Uses
-
-<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
-
-### Direct Use
-
-<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
-
-[More Information Needed]
-
-### Downstream Use [optional]
-
-<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
-
-[More Information Needed]
-
-### Out-of-Scope Use
-
-<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
-
-[More Information Needed]
-
-## Bias, Risks, and Limitations
-
-<!-- This section is meant to convey both technical and sociotechnical limitations. -->
-
-[More Information Needed]
-
-### Recommendations
-
-<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
-
-Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
-
-## How to Get Started with the Model
-
-Use the code below to get started with the model.
-
-[More Information Needed]
-
-## Training Details
-
-### Training Data
-
-<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
-
-[More Information Needed]
-
-### Training Procedure
-
-<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
-
-#### Preprocessing [optional]
-
-[More Information Needed]
-
-
-#### Training Hyperparameters
-
-- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
-
-#### Speeds, Sizes, Times [optional]
-
-<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
-
-[More Information Needed]
-
-## Evaluation
-
-<!-- This section describes the evaluation protocols and provides the results. -->
-
-### Testing Data, Factors & Metrics
-
-#### Testing Data
-
-<!-- This should link to a Dataset Card if possible. -->
-
-[More Information Needed]
-
-#### Factors
-
-<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
-
-[More Information Needed]
-
-#### Metrics
-
-<!-- These are the evaluation metrics being used, ideally with a description of why. -->
-
-[More Information Needed]
-
-### Results
-
-[More Information Needed]
-
-#### Summary
-
-
-
-## Model Examination [optional]
-
-<!-- Relevant interpretability work for the model goes here -->
-
-[More Information Needed]
-
-## Environmental Impact
-
-<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
-
-Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
-
-- **Hardware Type:** [More Information Needed]
-- **Hours used:** [More Information Needed]
-- **Cloud Provider:** [More Information Needed]
-- **Compute Region:** [More Information Needed]
-- **Carbon Emitted:** [More Information Needed]
-
-## Technical Specifications [optional]
-
-### Model Architecture and Objective
-
-[More Information Needed]
-
-### Compute Infrastructure
-
-[More Information Needed]
-
-#### Hardware
-
-[More Information Needed]
-
-#### Software
-
-[More Information Needed]
-
-## Citation [optional]
-
-<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
-
-**BibTeX:**
-
-[More Information Needed]
-
-**APA:**
-
-[More Information Needed]
-
-## Glossary [optional]
-
-<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
-
-[More Information Needed]
-
-## More Information [optional]
-
-[More Information Needed]
-
-## Model Card Authors [optional]
-
-[More Information Needed]
-
-## Model Card Contact
-
-[More Information Needed]
-### Framework versions
-
-- PEFT 0.18.1
\ No newline at end of file
diff --git a/qwen1.5-moe-a2.7b/dp3_attn_v2/epochs/epoch_002/adapter/adapter_config.json b/qwen1.5-moe-a2.7b/dp3_attn_v2/epochs/epoch_002/adapter/adapter_config.json
deleted file mode 100644
index 5cc38c385aa174ee4fc7225fc7482a1c8f9460b4..0000000000000000000000000000000000000000
--- a/qwen1.5-moe-a2.7b/dp3_attn_v2/epochs/epoch_002/adapter/adapter_config.json
+++ /dev/null
@@ -1,46 +0,0 @@
-{
-  "alora_invocation_tokens": null,
-  "alpha_pattern": {},
-  "arrow_config": null,
-  "auto_mapping": null,
-  "base_model_name_or_path": "Qwen/Qwen1.5-MoE-A2.7B",
-  "bias": "none",
-  "corda_config": null,
-  "ensure_weight_tying": true,
-  "eva_config": null,
-  "exclude_modules": null,
-  "fan_in_fan_out": false,
-  "inference_mode": true,
-  "init_lora_weights": true,
-  "layer_replication": null,
-  "layers_pattern": null,
-  "layers_to_transform": null,
-  "loftq_config": {},
-  "lora_alpha": 32,
-  "lora_bias": false,
-  "lora_dropout": 0.05,
-  "megatron_config": null,
-  "megatron_core": "megatron.core",
-  "modules_to_save": [
-    "lm_head",
-    "embed_tokens"
-  ],
-  "peft_type": "LORA",
-  "peft_version": "0.18.1",
-  "qalora_group_size": 16,
-  "r": 16,
-  "rank_pattern": {},
-  "revision": null,
-  "target_modules": [
-    "v_proj",
-    "k_proj",
-    "q_proj",
-    "o_proj"
-  ],
-  "target_parameters": null,
-  "task_type": "CAUSAL_LM",
-  "trainable_token_indices": null,
-  "use_dora": false,
-  "use_qalora": false,
-  "use_rslora": false
-}
\ No newline at end of file
diff --git a/qwen1.5-moe-a2.7b/dp3_attn_v2/epochs/epoch_002/audit_results.json b/qwen1.5-moe-a2.7b/dp3_attn_v2/epochs/epoch_002/audit_results.json
deleted file mode 100644
index 4c2ac43ae6085b5e7785f7c7203ceb88b233c045..0000000000000000000000000000000000000000
--- a/qwen1.5-moe-a2.7b/dp3_attn_v2/epochs/epoch_002/audit_results.json
+++ /dev/null
@@ -1,137 +0,0 @@
-{
-  "delta": 1e-05,
-  "num_canaries": 500,
-  "num_members": 250,
-  "paper_guess_fraction": 0.2,
-  "paper_guess_steps": 20,
-  "loss": {
-    "auc": 0.548672,
-    "empirical_epsilon": {
-      "0.05": 0.022048615850508213,
-      "0.01": 0.0
-    },
-    "empirical_epsilon_details": {
-      "0.05": {
-        "epsilon": 0.022048615850508213,
-        "num_guesses": 65,
-        "correct_guesses": 40,
-        "candidate_num_guesses": [
-          5,
-          10,
-          15,
-          20,
-          25,
-          30,
-          35,
-          40,
-          45,
-          50,
-          55,
-          60,
-          65,
-          70,
-          75,
-          80,
-          85,
-          90,
-          95,
-          100
-        ],
-        "direction": "lower"
-      },
-      "0.01": {
-        "epsilon": 0.0,
-        "num_guesses": 0,
-        "correct_guesses": 0,
-        "candidate_num_guesses": [
-          5,
-          10,
-          15,
-          20,
-          25,
-          30,
-          35,
-          40,
-          45,
-          50,
-          55,
-          60,
-          65,
-          70,
-          75,
-          80,
-          85,
-          90,
-          95,
-          100
-        ],
-        "direction": "lower"
-      }
-    }
-  },
-  "embedding": {
-    "auc": 0.529136,
-    "empirical_epsilon": {
-      "0.05": 0.0,
-      "0.01": 0.0
-    },
-    "empirical_epsilon_details": {
-      "0.05": {
-        "epsilon": 0.0,
-        "num_guesses": 0,
-        "correct_guesses": 0,
-        "candidate_num_guesses": [
-          5,
-          10,
-          15,
-          20,
-          25,
-          30,
-          35,
-          40,
-          45,
-          50,
-          55,
-          60,
-          65,
-          70,
-          75,
-          80,
-          85,
-          90,
-          95,
-          100
-        ],
-        "direction": "lower"
-      },
-      "0.01": {
-        "epsilon": 0.0,
-        "num_guesses": 0,
-        "correct_guesses": 0,
-        "candidate_num_guesses": [
-          5,
-          10,
-          15,
-          20,
-          25,
-          30,
-          35,
-          40,
-          45,
-          50,
-          55,
-          60,
-          65,
-          70,
-          75,
-          80,
-          85,
-          90,
-          95,
-          100
-        ],
-        "direction": "lower"
-      }
-    }
-  }
-}
\ No newline at end of file
diff --git a/qwen1.5-moe-a2.7b/dp3_attn_v2/epochs/epoch_003/adapter/README.md b/qwen1.5-moe-a2.7b/dp3_attn_v2/epochs/epoch_003/adapter/README.md
deleted file mode 100644
index 3b9e588251d937f16d19c9a0dcc3c6c5e3404063..0000000000000000000000000000000000000000
--- a/qwen1.5-moe-a2.7b/dp3_attn_v2/epochs/epoch_003/adapter/README.md
+++ /dev/null
@@ -1,207 +0,0 @@
----
-base_model: Qwen/Qwen1.5-MoE-A2.7B
-library_name: peft
-pipeline_tag: text-generation
-tags:
-- base_model:adapter:Qwen/Qwen1.5-MoE-A2.7B
-- lora
-- transformers
----
-
-# Model Card for Model ID
-
-<!-- Provide a quick summary of what the model is/does. -->
-
-
-
-## Model Details
-
-### Model Description
-
-<!-- Provide a longer summary of what this model is. -->
-
-
-
-- **Developed by:** [More Information Needed]
-- **Funded by [optional]:** [More Information Needed]
-- **Shared by [optional]:** [More Information Needed]
-- **Model type:** [More Information Needed]
-- **Language(s) (NLP):** [More Information Needed]
-- **License:** [More Information Needed]
-- **Finetuned from model [optional]:** [More Information Needed]
-
-### Model Sources [optional]
-
-<!-- Provide the basic links for the model. -->
-
-- **Repository:** [More Information Needed]
-- **Paper [optional]:** [More Information Needed]
-- **Demo [optional]:** [More Information Needed]
-
-## Uses
-
-<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
-
-### Direct Use
-
-<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
-
-[More Information Needed]
-
-### Downstream Use [optional]
-
-<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
-
-[More Information Needed]
-
-### Out-of-Scope Use
-
-<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
-
-[More Information Needed]
-
-## Bias, Risks, and Limitations
-
-<!-- This section is meant to convey both technical and sociotechnical limitations. -->
-
-[More Information Needed]
-
-### Recommendations
-
-<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
-
-Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
-
-## How to Get Started with the Model
-
-Use the code below to get started with the model.
-
-[More Information Needed]
-
-## Training Details
-
-### Training Data
-
-<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
-
-[More Information Needed]
-
-### Training Procedure
-
-<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
-
-#### Preprocessing [optional]
-
-[More Information Needed]
-
-
-#### Training Hyperparameters
-
-- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
-
-#### Speeds, Sizes, Times [optional]
-
-<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
-
-[More Information Needed]
-
-## Evaluation
-
-<!-- This section describes the evaluation protocols and provides the results. -->
-
-### Testing Data, Factors & Metrics
-
-#### Testing Data
-
-<!-- This should link to a Dataset Card if possible. -->
-
-[More Information Needed]
-
-#### Factors
-
-<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
-
-[More Information Needed]
-
-#### Metrics
-
-<!-- These are the evaluation metrics being used, ideally with a description of why. -->
-
-[More Information Needed]
-
-### Results
-
-[More Information Needed]
-
-#### Summary
-
-
-
-## Model Examination [optional]
-
-<!-- Relevant interpretability work for the model goes here -->
-
-[More Information Needed]
-
-## Environmental Impact
-
-<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
-
-Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
-
-- **Hardware Type:** [More Information Needed]
-- **Hours used:** [More Information Needed]
-- **Cloud Provider:** [More Information Needed]
-- **Compute Region:** [More Information Needed]
-- **Carbon Emitted:** [More Information Needed]
-
-## Technical Specifications [optional]
-
-### Model Architecture and Objective
-
-[More Information Needed]
-
-### Compute Infrastructure
-
-[More Information Needed]
-
-#### Hardware
-
-[More Information Needed]
-
-#### Software
-
-[More Information Needed]
-
-## Citation [optional]
-
-<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
-
-**BibTeX:**
-
-[More Information Needed]
-
-**APA:**
-
-[More Information Needed]
-
-## Glossary [optional]
-
-<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
-
-[More Information Needed]
-
-## More Information [optional]
-
-[More Information Needed]
-
-## Model Card Authors [optional]
-
-[More Information Needed]
-
-## Model Card Contact
-
-[More Information Needed]
-### Framework versions
-
-- PEFT 0.18.1
\ No newline at end of file
diff --git a/qwen1.5-moe-a2.7b/dp3_attn_v2/epochs/epoch_003/adapter/adapter_config.json b/qwen1.5-moe-a2.7b/dp3_attn_v2/epochs/epoch_003/adapter/adapter_config.json
deleted file mode 100644
index 5cc38c385aa174ee4fc7225fc7482a1c8f9460b4..0000000000000000000000000000000000000000
--- a/qwen1.5-moe-a2.7b/dp3_attn_v2/epochs/epoch_003/adapter/adapter_config.json
+++ /dev/null
@@ -1,46 +0,0 @@
-{
-  "alora_invocation_tokens": null,
-  "alpha_pattern": {},
-  "arrow_config": null,
-  "auto_mapping": null,
-  "base_model_name_or_path": "Qwen/Qwen1.5-MoE-A2.7B",
-  "bias": "none",
-  "corda_config": null,
-  "ensure_weight_tying": true,
-  "eva_config": null,
-  "exclude_modules": null,
-  "fan_in_fan_out": false,
-  "inference_mode": true,
-  "init_lora_weights": true,
-  "layer_replication": null,
-  "layers_pattern": null,
-  "layers_to_transform": null,
-  "loftq_config": {},
-  "lora_alpha": 32,
-  "lora_bias": false,
-  "lora_dropout": 0.05,
-  "megatron_config": null,
-  "megatron_core": "megatron.core",
-  "modules_to_save": [
-    "lm_head",
-    "embed_tokens"
-  ],
-  "peft_type": "LORA",
-  "peft_version": "0.18.1",
-  "qalora_group_size": 16,
-  "r": 16,
-  "rank_pattern": {},
-  "revision": null,
-  "target_modules": [
-    "v_proj",
-    "k_proj",
-    "q_proj",
-    "o_proj"
-  ],
-  "target_parameters": null,
-  "task_type": "CAUSAL_LM",
-  "trainable_token_indices": null,
-  "use_dora": false,
-  "use_qalora": false,
-  "use_rslora": false
-}
\ No newline at end of file
diff --git a/qwen1.5-moe-a2.7b/dp3_attn_v2/epochs/epoch_003/audit_results.json b/qwen1.5-moe-a2.7b/dp3_attn_v2/epochs/epoch_003/audit_results.json
deleted file mode 100644
index d241f093cd6384c84700120f1f48dcf2cb85d593..0000000000000000000000000000000000000000
--- a/qwen1.5-moe-a2.7b/dp3_attn_v2/epochs/epoch_003/audit_results.json
+++ /dev/null
@@ -1,137 +0,0 @@
-{
-  "delta": 1e-05,
-  "num_canaries": 500,
-  "num_members": 250,
-  "paper_guess_fraction": 0.2,
-  "paper_guess_steps": 20,
-  "loss": {
-    "auc": 0.533616,
-    "empirical_epsilon": {
-      "0.05": 0.172086076810956,
-      "0.01": 0.0
-    },
-    "empirical_epsilon_details": {
-      "0.05": {
-        "epsilon": 0.172086076810956,
-        "num_guesses": 20,
-        "correct_guesses": 15,
-        "candidate_num_guesses": [
-          5,
-          10,
-          15,
-          20,
-          25,
-          30,
-          35,
-          40,
-          45,
-          50,
-          55,
-          60,
-          65,
-          70,
-          75,
-          80,
-          85,
-          90,
-          95,
-          100
-        ],
-        "direction": "lower"
-      },
-      "0.01": {
-        "epsilon": 0.0,
-        "num_guesses": 0,
-        "correct_guesses": 0,
-        "candidate_num_guesses": [
-          5,
-          10,
-          15,
-          20,
-          25,
-          30,
-          35,
-          40,
-          45,
-          50,
-          55,
-          60,
-          65,
-          70,
-          75,
-          80,
-          85,
-          90,
-          95,
-          100
-        ],
-        "direction": "lower"
-      }
-    }
-  },
-  "embedding": {
-    "auc": 0.5622,
-    "empirical_epsilon": {
-      "0.05": 0.006439845077693462,
-      "0.01": 0.0
-    },
-    "empirical_epsilon_details": {
-      "0.05": {
-        "epsilon": 0.006439845077693462,
-        "num_guesses": 80,
-        "correct_guesses": 48,
-        "candidate_num_guesses": [
-          5,
-          10,
-          15,
-          20,
-          25,
-          30,
-          35,
-          40,
-          45,
-          50,
-          55,
-          60,
-          65,
-          70,
-          75,
-          80,
-          85,
-          90,
-          95,
-          100
-        ],
-        "direction": "higher"
-      },
-      "0.01": {
-        "epsilon": 0.0,
-        "num_guesses": 0,
-        "correct_guesses": 0,
-        "candidate_num_guesses": [
-          5,
-          10,
-          15,
-          20,
-          25,
-          30,
-          35,
-          40,
-          45,
-          50,
-          55,
-          60,
-          65,
-          70,
-          75,
-          80,
-          85,
-          90,
-          95,
-          100
-        ],
-        "direction": "lower"
-      }
-    }
-  }
-}
\ No newline at end of file
diff --git a/qwen1.5-moe-a2.7b/dp3_attn_v2/epochs/epoch_004/adapter/README.md b/qwen1.5-moe-a2.7b/dp3_attn_v2/epochs/epoch_004/adapter/README.md
deleted file mode 100644
index 3b9e588251d937f16d19c9a0dcc3c6c5e3404063..0000000000000000000000000000000000000000
--- a/qwen1.5-moe-a2.7b/dp3_attn_v2/epochs/epoch_004/adapter/README.md
+++ /dev/null
@@ -1,207 +0,0 @@
----
-base_model: Qwen/Qwen1.5-MoE-A2.7B
-library_name: peft
-pipeline_tag: text-generation
-tags:
-- base_model:adapter:Qwen/Qwen1.5-MoE-A2.7B
-- lora
-- transformers
----
-
-# Model Card for Model ID
-
-<!-- Provide a quick summary of what the model is/does. -->
-
-
-
-## Model Details
-
-### Model Description
-
-<!-- Provide a longer summary of what this model is. -->
-
-
-
-- **Developed by:** [More Information Needed]
-- **Funded by [optional]:** [More Information Needed]
-- **Shared by [optional]:** [More Information Needed]
-- **Model type:** [More Information Needed]
-- **Language(s) (NLP):** [More Information Needed]
-- **License:** [More Information Needed]
-- **Finetuned from model [optional]:** [More Information Needed]
-
-### Model Sources [optional]
-
-<!-- Provide the basic links for the model. -->
-
-- **Repository:** [More Information Needed]
-- **Paper [optional]:** [More Information Needed]
-- **Demo [optional]:** [More Information Needed]
-
-## Uses
-
-<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
-
-### Direct Use
-
-<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
-
-[More Information Needed]
-
-### Downstream Use [optional]
-
-<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
-
-[More Information Needed]
-
-### Out-of-Scope Use
-
-<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
-
-[More Information Needed]
-
-## Bias, Risks, and Limitations
-
-<!-- This section is meant to convey both technical and sociotechnical limitations. -->
-
-[More Information Needed]
-
-### Recommendations
-
-<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
-
-Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
-
-## How to Get Started with the Model
-
-Use the code below to get started with the model.
-
-[More Information Needed]
-
-## Training Details
-
-### Training Data
-
-<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
-
-[More Information Needed]
-
-### Training Procedure
-
-<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
-
-#### Preprocessing [optional]
-
-[More Information Needed]
-
-
-#### Training Hyperparameters
-
-- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
-
-#### Speeds, Sizes, Times [optional]
-
-<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
-
-[More Information Needed]
-
-## Evaluation
-
-<!-- This section describes the evaluation protocols and provides the results. -->
-
-### Testing Data, Factors & Metrics
-
-#### Testing Data
-
-<!-- This should link to a Dataset Card if possible. -->
-
-[More Information Needed]
-
-#### Factors
-
-<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
-
-[More Information Needed]
-
-#### Metrics
-
-<!-- These are the evaluation metrics being used, ideally with a description of why. -->
-
-[More Information Needed]
-
-### Results
-
-[More Information Needed]
-
-#### Summary
-
-
-
-## Model Examination [optional]
-
-<!-- Relevant interpretability work for the model goes here -->
-
-[More Information Needed]
-
-## Environmental Impact
-
-<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
-
-Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
-
-- **Hardware Type:** [More Information Needed]
-- **Hours used:** [More Information Needed]
-- **Cloud Provider:** [More Information Needed]
-- **Compute Region:** [More Information Needed]
-- **Carbon Emitted:** [More Information Needed]
-
-## Technical Specifications [optional]
-
-### Model Architecture and Objective
-
-[More Information Needed]
-
-### Compute Infrastructure
-
-[More Information Needed]
-
-#### Hardware
-
-[More Information Needed]
-
-#### Software
-
-[More Information Needed]
-
-## Citation [optional]
-
-<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
-
-**BibTeX:**
-
-[More Information Needed]
-
-**APA:**
-
-[More Information Needed]
-
-## Glossary [optional]
-
-<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
-
-[More Information Needed]
-
-## More Information [optional]
-
-[More Information Needed]
-
-## Model Card Authors [optional]
-
-[More Information Needed]
-
-## Model Card Contact
-
-[More Information Needed]
-### Framework versions
-
-- PEFT 0.18.1
\ No newline at end of file
diff --git a/qwen1.5-moe-a2.7b/dp3_attn_v2/epochs/epoch_004/adapter/adapter_config.json b/qwen1.5-moe-a2.7b/dp3_attn_v2/epochs/epoch_004/adapter/adapter_config.json
deleted file mode 100644
index 5cc38c385aa174ee4fc7225fc7482a1c8f9460b4..0000000000000000000000000000000000000000
--- a/qwen1.5-moe-a2.7b/dp3_attn_v2/epochs/epoch_004/adapter/adapter_config.json
+++ /dev/null
@@ -1,46 +0,0 @@
-{
-  "alora_invocation_tokens": null,
-  "alpha_pattern": {},
-  "arrow_config": null,
-  "auto_mapping": null,
-  "base_model_name_or_path": "Qwen/Qwen1.5-MoE-A2.7B",
-  "bias": "none",
-  "corda_config": null,
-  "ensure_weight_tying": true,
-  "eva_config": null,
-  "exclude_modules": null,
-  "fan_in_fan_out": false,
-  "inference_mode": true,
-  "init_lora_weights": true,
-  "layer_replication": null,
-  "layers_pattern": null,
-  "layers_to_transform": null,
-  "loftq_config": {},
-  "lora_alpha": 32,
-  "lora_bias": false,
-  "lora_dropout": 0.05,
-  "megatron_config": null,
-  "megatron_core": "megatron.core",
-  "modules_to_save": [
-    "lm_head",
-    "embed_tokens"
-  ],
-  "peft_type": "LORA",
-  "peft_version": "0.18.1",
-  "qalora_group_size": 16,
-  "r": 16,
-  "rank_pattern": {},
-  "revision": null,
-  "target_modules": [
-    "v_proj",
-    "k_proj",
-    "q_proj",
-    "o_proj"
-  ],
-  "target_parameters": null,
-  "task_type": "CAUSAL_LM",
-  "trainable_token_indices": null,
-  "use_dora": false,
-  "use_qalora": false,
-  "use_rslora": false
-}
\ No newline at end of file
diff --git a/qwen1.5-moe-a2.7b/dp3_attn_v2/epochs/epoch_004/audit_results.json b/qwen1.5-moe-a2.7b/dp3_attn_v2/epochs/epoch_004/audit_results.json
deleted file mode 100644
index 687cd4d44e14b125012b139b203c37cad17ff60b..0000000000000000000000000000000000000000
--- a/qwen1.5-moe-a2.7b/dp3_attn_v2/epochs/epoch_004/audit_results.json
+++ /dev/null
@@ -1,137 +0,0 @@
-{
-  "delta": 1e-05,
-  "num_canaries": 500,
-  "num_members": 250,
-  "paper_guess_fraction": 0.2,
-  "paper_guess_steps": 20,
-  "loss": {
-    "auc": 0.53848,
-    "empirical_epsilon": {
-      "0.05": 0.42648759204894304,
-      "0.01": 0.16005427204072475
-    },
-    "empirical_epsilon_details": {
-      "0.05": {
-        "epsilon": 0.42648759204894304,
-        "num_guesses": 30,
-        "correct_guesses": 23,
-        "candidate_num_guesses": [
-          5,
-          10,
-          15,
-          20,
-          25,
-          30,
-          35,
-          40,
-          45,
-          50,
-          55,
-          60,
-          65,
-          70,
-          75,
-          80,
-          85,
-          90,
-          95,
-          100
-        ],
-        "direction": "lower"
-      },
-      "0.01": {
-        "epsilon": 0.16005427204072475,
-        "num_guesses": 30,
-        "correct_guesses": 23,
-        "candidate_num_guesses": [
-          5,
-          10,
-          15,
-          20,
-          25,
-          30,
-          35,
-          40,
-          45,
-          50,
-          55,
-          60,
-          65,
-          70,
-          75,
-          80,
-          85,
-          90,
-          95,
-          100
-        ],
-        "direction": "lower"
-      }
-    }
-  },
-  "embedding": {
-    "auc": 0.57372,
-    "empirical_epsilon": {
-      "0.05": 0.0,
-      "0.01": 0.0
-    },
-    "empirical_epsilon_details": {
-      "0.05": {
-        "epsilon": 0.0,
-        "num_guesses": 0,
-        "correct_guesses": 0,
-        "candidate_num_guesses": [
-          5,
-          10,
-          15,
-          20,
-          25,
-          30,
-          35,
-          40,
-          45,
-          50,
-          55,
-          60,
-          65,
-          70,
-          75,
-          80,
-          85,
-          90,
-          95,
-          100
-        ],
-        "direction": "lower"
-      },
-      "0.01": {
-        "epsilon": 0.0,
-        "num_guesses": 0,
-        "correct_guesses": 0,
-        "candidate_num_guesses": [
-          5,
-          10,
-          15,
-          20,
-          25,
-          30,
-          35,
-          40,
-          45,
-          50,
-          55,
-          60,
-          65,
-          70,
-          75,
-          80,
-          85,
-          90,
-          95,
-          100
-        ],
-        "direction": "lower"
-      }
-    }
-  }
-}
\ No newline at end of file
diff --git a/qwen1.5-moe-a2.7b/dp3_attn_v2/epochs/epoch_005/adapter/README.md b/qwen1.5-moe-a2.7b/dp3_attn_v2/epochs/epoch_005/adapter/README.md
deleted file mode 100644
index 3b9e588251d937f16d19c9a0dcc3c6c5e3404063..0000000000000000000000000000000000000000
--- a/qwen1.5-moe-a2.7b/dp3_attn_v2/epochs/epoch_005/adapter/README.md
+++ /dev/null
@@ -1,207 +0,0 @@
----
-base_model: Qwen/Qwen1.5-MoE-A2.7B
-library_name: peft
-pipeline_tag: text-generation
-tags:
-- base_model:adapter:Qwen/Qwen1.5-MoE-A2.7B
-- lora
-- transformers
----
-
-# Model Card for Model ID
-
-<!-- Provide a quick summary of what the model is/does. -->
-
-
-
-## Model Details
-
-### Model Description
-
-<!-- Provide a longer summary of what this model is. -->
-
-
-
-- **Developed by:** [More Information Needed]
-- **Funded by [optional]:** [More Information Needed]
-- **Shared by [optional]:** [More Information Needed]
-- **Model type:** [More Information Needed]
-- **Language(s) (NLP):** [More Information Needed]
-- **License:** [More Information Needed]
-- **Finetuned from model [optional]:** [More Information Needed]
-
-### Model Sources [optional]
-
-<!-- Provide the basic links for the model. -->
-
-- **Repository:** [More Information Needed]
-- **Paper [optional]:** [More Information Needed]
-- **Demo [optional]:** [More Information Needed]
-
-## Uses
-
-<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
-
-### Direct Use
-
-<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
-
-[More Information Needed]
-
-### Downstream Use [optional]
-
-<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
-
-[More Information Needed]
-
-### Out-of-Scope Use
-
-<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
-
-[More Information Needed]
-
-## Bias, Risks, and Limitations
-
-<!-- This section is meant to convey both technical and sociotechnical limitations. -->
-
-[More Information Needed]
-
-### Recommendations
-
-<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
-
-Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
-
-## How to Get Started with the Model
-
-Use the code below to get started with the model.
-
-[More Information Needed]
-
-## Training Details
-
-### Training Data
-
-<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
-
-[More Information Needed]
-
-### Training Procedure
-
-<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
-
-#### Preprocessing [optional]
-
-[More Information Needed]
-
-
-#### Training Hyperparameters
-
-- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
-
-#### Speeds, Sizes, Times [optional]
-
-<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
-
-[More Information Needed]
-
-## Evaluation
-
-<!-- This section describes the evaluation protocols and provides the results. -->
-
-### Testing Data, Factors & Metrics
-
-#### Testing Data
-
-<!-- This should link to a Dataset Card if possible. -->
-
-[More Information Needed]
-
-#### Factors
-
-<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
-
-[More Information Needed]
-
-#### Metrics
-
-<!-- These are the evaluation metrics being used, ideally with a description of why. -->
-
-[More Information Needed]
-
-### Results
-
-[More Information Needed]
-
-#### Summary
-
-
-
-## Model Examination [optional]
-
-<!-- Relevant interpretability work for the model goes here -->
-
-[More Information Needed]
-
-## Environmental Impact
-
-<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
-
-Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
-
-- **Hardware Type:** [More Information Needed]
-- **Hours used:** [More Information Needed]
-- **Cloud Provider:** [More Information Needed]
-- **Compute Region:** [More Information Needed]
-- **Carbon Emitted:** [More Information Needed]
-
-## Technical Specifications [optional]
-
-### Model Architecture and Objective
-
-[More Information Needed]
-
-### Compute Infrastructure
-
-[More Information Needed]
-
-#### Hardware
-
-[More Information Needed]
-
-#### Software
-
-[More Information Needed]
-
-## Citation [optional]
-
-<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
-
-**BibTeX:**
-
-[More Information Needed]
-
-**APA:**
-
-[More Information Needed]
-
-## Glossary [optional]
-
-<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
-
-[More Information Needed]
-
-## More Information [optional]
-
-[More Information Needed]
-
-## Model Card Authors [optional]
-
-[More Information Needed]
-
-## Model Card Contact
-
-[More Information Needed]
-### Framework versions
-
-- PEFT 0.18.1
\ No newline at end of file
diff --git a/qwen1.5-moe-a2.7b/dp3_attn_v2/epochs/epoch_005/adapter/adapter_config.json b/qwen1.5-moe-a2.7b/dp3_attn_v2/epochs/epoch_005/adapter/adapter_config.json
deleted file mode 100644
index 5cc38c385aa174ee4fc7225fc7482a1c8f9460b4..0000000000000000000000000000000000000000
--- a/qwen1.5-moe-a2.7b/dp3_attn_v2/epochs/epoch_005/adapter/adapter_config.json
+++ /dev/null
@@ -1,46 +0,0 @@
-{
-  "alora_invocation_tokens": null,
-  "alpha_pattern": {},
-  "arrow_config": null,
-  "auto_mapping": null,
-  "base_model_name_or_path": "Qwen/Qwen1.5-MoE-A2.7B",
-  "bias": "none",
-  "corda_config": null,
-  "ensure_weight_tying": true,
-  "eva_config": null,
-  "exclude_modules": null,
-  "fan_in_fan_out": false,
-  "inference_mode": true,
-  "init_lora_weights": true,
-  "layer_replication": null,
-  "layers_pattern": null,
-  "layers_to_transform": null,
-  "loftq_config": {},
-  "lora_alpha": 32,
-  "lora_bias": false,
-  "lora_dropout": 0.05,
-  "megatron_config": null,
-  "megatron_core": "megatron.core",
-  "modules_to_save": [
-    "lm_head",
-    "embed_tokens"
-  ],
-  "peft_type": "LORA",
-  "peft_version": "0.18.1",
-  "qalora_group_size": 16,
-  "r": 16,
-  "rank_pattern": {},
-  "revision": null,
-  "target_modules": [
-    "v_proj",
-    "k_proj",
-    "q_proj",
-    "o_proj"
-  ],
-  "target_parameters": null,
-  "task_type": "CAUSAL_LM",
-  "trainable_token_indices": null,
-  "use_dora": false,
-  "use_qalora": false,
-  "use_rslora": false
-}
\ No newline at end of file
diff --git a/qwen1.5-moe-a2.7b/dp3_attn_v2/epochs/epoch_005/audit_results.json b/qwen1.5-moe-a2.7b/dp3_attn_v2/epochs/epoch_005/audit_results.json
deleted file mode 100644
index 36306c81e4d2f010e3847b6f8ba36a0951c77784..0000000000000000000000000000000000000000
--- a/qwen1.5-moe-a2.7b/dp3_attn_v2/epochs/epoch_005/audit_results.json
+++ /dev/null
@@ -1,137 +0,0 @@
-{
-  "delta": 1e-05,
-  "num_canaries": 500,
-  "num_members": 250,
-  "paper_guess_fraction": 0.2,
-  "paper_guess_steps": 20,
-  "loss": {
-    "auc": 0.540256,
-    "empirical_epsilon": {
-      "0.05": 0.3197584319859743,
-      "0.01": 0.030605491250753403
-    },
-    "empirical_epsilon_details": {
-      "0.05": {
-        "epsilon": 0.3197584319859743,
-        "num_guesses": 25,
-        "correct_guesses": 19,
-        "candidate_num_guesses": [
-          5,
-          10,
-          15,
-          20,
-          25,
-          30,
-          35,
-          40,
-          45,
-          50,
-          55,
-          60,
-          65,
-          70,
-          75,
-          80,
-          85,
-          90,
-          95,
-          100
-        ],
-        "direction": "lower"
-      },
-      "0.01": {
-        "epsilon": 0.030605491250753403,
-        "num_guesses": 25,
-        "correct_guesses": 19,
-        "candidate_num_guesses": [
-          5,
-          10,
-          15,
-          20,
-          25,
-          30,
-          35,
-          40,
-          45,
-          50,
-          55,
-          60,
-          65,
-          70,
-          75,
-          80,
-          85,
-          90,
-          95,
-          100
-        ],
-        "direction": "lower"
-      }
-    }
-  },
-  "embedding": {
-    "auc": 0.5672,
-    "empirical_epsilon": {
-      "0.05": 0.010390725918114185,
-      "0.01": 0.0
-    },
-    "empirical_epsilon_details": {
-      "0.05": {
-        "epsilon": 0.010390725918114185,
-        "num_guesses": 100,
-        "correct_guesses": 59,
-        "candidate_num_guesses": [
-          5,
-          10,
-          15,
-          20,
-          25,
-          30,
-          35,
-          40,
-          45,
-          50,
-          55,
-          60,
-          65,
-          70,
-          75,
-          80,
-          85,
-          90,
-          95,
-          100
-        ],
-        "direction": "higher"
-      },
-      "0.01": {
-        "epsilon": 0.0,
-        "num_guesses": 0,
-        "correct_guesses": 0,
-        "candidate_num_guesses": [
-          5,
-          10,
-          15,
-          20,
-          25,
-          30,
-          35,
-          40,
-          45,
-          50,
-          55,
-          60,
-          65,
-          70,
-          75,
-          80,
-          85,
-          90,
-          95,
-          100
-        ],
-        "direction": "lower"
-      }
-    }
-  }
-}
\ No newline at end of file
diff --git a/qwen1.5-moe-a2.7b/dp8_attn_v2/epochs/epoch_001/adapter/README.md b/qwen1.5-moe-a2.7b/dp8_attn_v2/epochs/epoch_001/adapter/README.md
deleted file mode 100644
index 3b9e588251d937f16d19c9a0dcc3c6c5e3404063..0000000000000000000000000000000000000000
--- a/qwen1.5-moe-a2.7b/dp8_attn_v2/epochs/epoch_001/adapter/README.md
+++ /dev/null
@@ -1,207 +0,0 @@
----
-base_model: Qwen/Qwen1.5-MoE-A2.7B
-library_name: peft
-pipeline_tag: text-generation
-tags:
-- base_model:adapter:Qwen/Qwen1.5-MoE-A2.7B
-- lora
-- transformers
----
-
-# Model Card for Model ID
-
-<!-- Provide a quick summary of what the model is/does. -->
-
-
-
-## Model Details
-
-### Model Description
-
-<!-- Provide a longer summary of what this model is. -->
-
-
-
-- **Developed by:** [More Information Needed]
-- **Funded by [optional]:** [More Information Needed]
-- **Shared by [optional]:** [More Information Needed]
-- **Model type:** [More Information Needed]
-- **Language(s) (NLP):** [More Information Needed]
-- **License:** [More Information Needed]
-- **Finetuned from model [optional]:** [More Information Needed]
-
-### Model Sources [optional]
-
-<!-- Provide the basic links for the model. -->
-
-- **Repository:** [More Information Needed]
-- **Paper [optional]:** [More Information Needed]
-- **Demo [optional]:** [More Information Needed]
-
-## Uses
-
-<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
-
-### Direct Use
-
-<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
-
-[More Information Needed]
-
-### Downstream Use [optional]
-
-<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
-
-[More Information Needed]
-
-### Out-of-Scope Use
-
-<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
-
-[More Information Needed]
-
-## Bias, Risks, and Limitations
-
-<!-- This section is meant to convey both technical and sociotechnical limitations. -->
-
-[More Information Needed]
-
-### Recommendations
-
-<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
-
-Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
-
-## How to Get Started with the Model
-
-Use the code below to get started with the model.
-
-[More Information Needed]
-
-## Training Details
-
-### Training Data
-
-<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
-
-[More Information Needed]
-
-### Training Procedure
-
-<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
-
-#### Preprocessing [optional]
-
-[More Information Needed]
-
-
-#### Training Hyperparameters
-
-- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
-
-#### Speeds, Sizes, Times [optional]
-
-<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
-
-[More Information Needed]
-
-## Evaluation
-
-<!-- This section describes the evaluation protocols and provides the results. -->
-
-### Testing Data, Factors & Metrics
-
-#### Testing Data
-
-<!-- This should link to a Dataset Card if possible. -->
-
-[More Information Needed]
-
-#### Factors
-
-<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
-
-[More Information Needed]
-
-#### Metrics
-
-<!-- These are the evaluation metrics being used, ideally with a description of why. -->
-
-[More Information Needed]
-
-### Results
-
-[More Information Needed]
-
-#### Summary
-
-
-
-## Model Examination [optional]
-
-<!-- Relevant interpretability work for the model goes here -->
-
-[More Information Needed]
-
-## Environmental Impact
-
-<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
-
-Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
-
-- **Hardware Type:** [More Information Needed]
-- **Hours used:** [More Information Needed]
-- **Cloud Provider:** [More Information Needed]
-- **Compute Region:** [More Information Needed]
-- **Carbon Emitted:** [More Information Needed]
-
-## Technical Specifications [optional]
-
-### Model Architecture and Objective
-
-[More Information Needed]
-
-### Compute Infrastructure
-
-[More Information Needed]
-
-#### Hardware
-
-[More Information Needed]
-
-#### Software
-
-[More Information Needed]
-
-## Citation [optional]
-
-<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
-
-**BibTeX:**
-
-[More Information Needed]
-
-**APA:**
-
-[More Information Needed]
-
-## Glossary [optional]
-
-<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
-
-[More Information Needed]
-
-## More Information [optional]
-
-[More Information Needed]
-
-## Model Card Authors [optional]
-
-[More Information Needed]
-
-## Model Card Contact
-
-[More Information Needed]
-### Framework versions
-
-- PEFT 0.18.1
\ No newline at end of file
diff --git a/qwen1.5-moe-a2.7b/dp8_attn_v2/epochs/epoch_001/adapter/adapter_config.json b/qwen1.5-moe-a2.7b/dp8_attn_v2/epochs/epoch_001/adapter/adapter_config.json
deleted file mode 100644
index 72a674c89ec3406f4e57c07f4178c67d553edb4d..0000000000000000000000000000000000000000
--- a/qwen1.5-moe-a2.7b/dp8_attn_v2/epochs/epoch_001/adapter/adapter_config.json
+++ /dev/null
@@ -1,46 +0,0 @@
-{
-  "alora_invocation_tokens": null,
-  "alpha_pattern": {},
-  "arrow_config": null,
-  "auto_mapping": null,
-  "base_model_name_or_path": "Qwen/Qwen1.5-MoE-A2.7B",
-  "bias": "none",
-  "corda_config": null,
-  "ensure_weight_tying": true,
-  "eva_config": null,
-  "exclude_modules": null,
-  "fan_in_fan_out": false,
-  "inference_mode": true,
-  "init_lora_weights": true,
-  "layer_replication": null,
-  "layers_pattern": null,
-  "layers_to_transform": null,
-  "loftq_config": {},
-  "lora_alpha": 32,
-  "lora_bias": false,
-  "lora_dropout": 0.05,
-  "megatron_config": null,
-  "megatron_core": "megatron.core",
-  "modules_to_save": [
-    "lm_head",
-    "embed_tokens"
-  ],
-  "peft_type": "LORA",
-  "peft_version": "0.18.1",
-  "qalora_group_size": 16,
-  "r": 16,
-  "rank_pattern": {},
-  "revision": null,
-  "target_modules": [
-    "q_proj",
-    "k_proj",
-    "v_proj",
-    "o_proj"
-  ],
-  "target_parameters": null,
-  "task_type": "CAUSAL_LM",
-  "trainable_token_indices": null,
-  "use_dora": false,
-  "use_qalora": false,
-  "use_rslora": false
-}
\ No newline at end of file
diff --git a/qwen1.5-moe-a2.7b/dp8_attn_v2/epochs/epoch_001/audit_results.json b/qwen1.5-moe-a2.7b/dp8_attn_v2/epochs/epoch_001/audit_results.json
deleted file mode 100644
index 1e22d3b7c20547f6b8906bf3d8a6138d0ab98678..0000000000000000000000000000000000000000
--- a/qwen1.5-moe-a2.7b/dp8_attn_v2/epochs/epoch_001/audit_results.json
+++ /dev/null
@@ -1,137 +0,0 @@
-{
-  "delta": 1e-05,
-  "num_canaries": 500,
-  "num_members": 250,
-  "paper_guess_fraction": 0.2,
-  "paper_guess_steps": 20,
-  "loss": {
-    "auc": 0.561928,
-    "empirical_epsilon": {
-      "0.05": 0.3776902826502919,
-      "0.01": 0.133380432613194
-    },
-    "empirical_epsilon_details": {
-      "0.05": {
-        "epsilon": 0.3776902826502919,
-        "num_guesses": 35,
-        "correct_guesses": 26,
-        "candidate_num_guesses": [
-          5,
-          10,
-          15,
-          20,
-          25,
-          30,
-          35,
-          40,
-          45,
-          50,
-          55,
-          60,
-          65,
-          70,
-          75,
-          80,
-          85,
-          90,
-          95,
-          100
-        ],
-        "direction": "lower"
-      },
-      "0.01": {
-        "epsilon": 0.133380432613194,
-        "num_guesses": 35,
-        "correct_guesses": 26,
-        "candidate_num_guesses": [
-          5,
-          10,
-          15,
-          20,
-          25,
-          30,
-          35,
-          40,
-          45,
-          50,
-          55,
-          60,
-          65,
-          70,
-          75,
-          80,
-          85,
-          90,
-          95,
-          100
-        ],
-        "direction": "lower"
-      }
-    }
-  },
-  "embedding": {
-    "auc": 0.536184,
-    "empirical_epsilon": {
-      "0.05": 0.09290625154972076,
-      "0.01": 0.0
-    },
-    "empirical_epsilon_details": {
-      "0.05": {
-        "epsilon": 0.09290625154972076,
-        "num_guesses": 70,
-        "correct_guesses": 44,
-        "candidate_num_guesses": [
-          5,
-          10,
-          15,
-          20,
-          25,
-          30,
-          35,
-          40,
-          45,
-          50,
-          55,
-          60,
-          65,
-          70,
-          75,
-          80,
-          85,
-          90,
-          95,
-          100
-        ],
-        "direction": "higher"
-      },
-      "0.01": {
-        "epsilon": 0.0,
-        "num_guesses": 0,
-        "correct_guesses": 0,
-        "candidate_num_guesses": [
-          5,
-          10,
-          15,
-          20,
-          25,
-          30,
-          35,
-          40,
-          45,
-          50,
-          55,
-          60,
-          65,
-          70,
-          75,
-          80,
-          85,
-          90,
-          95,
-          100
-        ],
-        "direction": "lower"
-      }
-    }
-  }
-}
\ No newline at end of file
diff --git a/qwen1.5-moe-a2.7b/dp8_attn_v2/epochs/epoch_002/adapter/README.md b/qwen1.5-moe-a2.7b/dp8_attn_v2/epochs/epoch_002/adapter/README.md
deleted file mode 100644
index 3b9e588251d937f16d19c9a0dcc3c6c5e3404063..0000000000000000000000000000000000000000
--- a/qwen1.5-moe-a2.7b/dp8_attn_v2/epochs/epoch_002/adapter/README.md
+++ /dev/null
@@ -1,207 +0,0 @@
----
-base_model: Qwen/Qwen1.5-MoE-A2.7B
-library_name: peft
-pipeline_tag: text-generation
-tags:
-- base_model:adapter:Qwen/Qwen1.5-MoE-A2.7B
-- lora
-- transformers
----
-
-# Model Card for Model ID
-
-<!-- Provide a quick summary of what the model is/does. -->
-
-
-
-## Model Details
-
-### Model Description
-
-<!-- Provide a longer summary of what this model is. -->
-
-
-
-- **Developed by:** [More Information Needed]
-- **Funded by [optional]:** [More Information Needed]
-- **Shared by [optional]:** [More Information Needed]
-- **Model type:** [More Information Needed]
-- **Language(s) (NLP):** [More Information Needed]
-- **License:** [More Information Needed]
-- **Finetuned from model [optional]:** [More Information Needed]
-
-### Model Sources [optional]
-
-<!-- Provide the basic links for the model. -->
-
-- **Repository:** [More Information Needed]
-- **Paper [optional]:** [More Information Needed]
-- **Demo [optional]:** [More Information Needed]
-
-## Uses
-
-<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
-
-### Direct Use
-
-<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
-
-[More Information Needed]
-
-### Downstream Use [optional]
-
-<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
-
-[More Information Needed]
-
-### Out-of-Scope Use
-
-<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
-
-[More Information Needed]
-
-## Bias, Risks, and Limitations
-
-<!-- This section is meant to convey both technical and sociotechnical limitations. -->
-
-[More Information Needed]
-
-### Recommendations
-
-<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
-
-Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
-
-## How to Get Started with the Model
-
-Use the code below to get started with the model.
-
-[More Information Needed]
-
-## Training Details
-
-### Training Data
-
-<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
-
-[More Information Needed]
-
-### Training Procedure
-
-<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
-
-#### Preprocessing [optional]
-
-[More Information Needed]
-
-
-#### Training Hyperparameters
-
-- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
-
-#### Speeds, Sizes, Times [optional]
-
-<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
-
-[More Information Needed]
-
-## Evaluation
-
-<!-- This section describes the evaluation protocols and provides the results. -->
-
-### Testing Data, Factors & Metrics
-
-#### Testing Data
-
-<!-- This should link to a Dataset Card if possible. -->
-
-[More Information Needed]
-
-#### Factors
-
-<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
-
-[More Information Needed]
-
-#### Metrics
-
-<!-- These are the evaluation metrics being used, ideally with a description of why. -->
-
-[More Information Needed]
-
-### Results
-
-[More Information Needed]
-
-#### Summary
-
-
-
-## Model Examination [optional]
-
-<!-- Relevant interpretability work for the model goes here -->
-
-[More Information Needed]
-
-## Environmental Impact
-
-<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
-
-Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
-
-- **Hardware Type:** [More Information Needed]
-- **Hours used:** [More Information Needed]
-- **Cloud Provider:** [More Information Needed]
-- **Compute Region:** [More Information Needed]
-- **Carbon Emitted:** [More Information Needed]
-
-## Technical Specifications [optional]
-
-### Model Architecture and Objective
-
-[More Information Needed]
-
-### Compute Infrastructure
-
-[More Information Needed]
-
-#### Hardware
-
-[More Information Needed]
-
-#### Software
-
-[More Information Needed]
-
-## Citation [optional]
-
-<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
-
-**BibTeX:**
-
-[More Information Needed]
-
-**APA:**
-
-[More Information Needed]
-
-## Glossary [optional]
-
-<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
-
-[More Information Needed]
-
-## More Information [optional]
-
-[More Information Needed]
-
-## Model Card Authors [optional]
-
-[More Information Needed]
-
-## Model Card Contact
-
-[More Information Needed]
-### Framework versions
-
-- PEFT 0.18.1
\ No newline at end of file
diff --git a/qwen1.5-moe-a2.7b/dp8_attn_v2/epochs/epoch_002/adapter/adapter_config.json b/qwen1.5-moe-a2.7b/dp8_attn_v2/epochs/epoch_002/adapter/adapter_config.json
deleted file mode 100644
index 72a674c89ec3406f4e57c07f4178c67d553edb4d..0000000000000000000000000000000000000000
--- a/qwen1.5-moe-a2.7b/dp8_attn_v2/epochs/epoch_002/adapter/adapter_config.json
+++ /dev/null
@@ -1,46 +0,0 @@
-{
-  "alora_invocation_tokens": null,
-  "alpha_pattern": {},
-  "arrow_config": null,
-  "auto_mapping": null,
-  "base_model_name_or_path": "Qwen/Qwen1.5-MoE-A2.7B",
-  "bias": "none",
-  "corda_config": null,
-  "ensure_weight_tying": true,
-  "eva_config": null,
-  "exclude_modules": null,
-  "fan_in_fan_out": false,
-  "inference_mode": true,
-  "init_lora_weights": true,
-  "layer_replication": null,
-  "layers_pattern": null,
-  "layers_to_transform": null,
-  "loftq_config": {},
-  "lora_alpha": 32,
-  "lora_bias": false,
-  "lora_dropout": 0.05,
-  "megatron_config": null,
-  "megatron_core": "megatron.core",
-  "modules_to_save": [
-    "lm_head",
-    "embed_tokens"
-  ],
-  "peft_type": "LORA",
-  "peft_version": "0.18.1",
-  "qalora_group_size": 16,
-  "r": 16,
-  "rank_pattern": {},
-  "revision": null,
-  "target_modules": [
-    "q_proj",
-    "k_proj",
-    "v_proj",
-    "o_proj"
-  ],
-  "target_parameters": null,
-  "task_type": "CAUSAL_LM",
-  "trainable_token_indices": null,
-  "use_dora": false,
-  "use_qalora": false,
-  "use_rslora": false
-}
\ No newline at end of file
diff --git a/qwen1.5-moe-a2.7b/dp8_attn_v2/epochs/epoch_002/audit_results.json b/qwen1.5-moe-a2.7b/dp8_attn_v2/epochs/epoch_002/audit_results.json
deleted file mode 100644
index 210278c885dbf2f290c2f4336aeec78ba9e78ae7..0000000000000000000000000000000000000000
--- a/qwen1.5-moe-a2.7b/dp8_attn_v2/epochs/epoch_002/audit_results.json
+++ /dev/null
@@ -1,137 +0,0 @@
-{
-  "delta": 1e-05,
-  "num_canaries": 500,
-  "num_members": 250,
-  "paper_guess_fraction": 0.2,
-  "paper_guess_steps": 20,
-  "loss": {
-    "auc": 0.537936,
-    "empirical_epsilon": {
-      "0.05": 0.2113842461258173,
-      "0.01": 0.03674145694822073
-    },
-    "empirical_epsilon_details": {
-      "0.05": {
-        "epsilon": 0.2113842461258173,
-        "num_guesses": 65,
-        "correct_guesses": 43,
-        "candidate_num_guesses": [
-          5,
-          10,
-          15,
-          20,
-          25,
-          30,
-          35,
-          40,
-          45,
-          50,
-          55,
-          60,
-          65,
-          70,
-          75,
-          80,
-          85,
-          90,
-          95,
-          100
-        ],
-        "direction": "lower"
-      },
-      "0.01": {
-        "epsilon": 0.03674145694822073,
-        "num_guesses": 65,
-        "correct_guesses": 43,
-        "candidate_num_guesses": [
-          5,
-          10,
-          15,
-          20,
-          25,
-          30,
-          35,
-          40,
-          45,
-          50,
-          55,
-          60,
-          65,
-          70,
-          75,
-          80,
-          85,
-          90,
-          95,
-          100
-        ],
-        "direction": "lower"
-      }
-    }
-  },
-  "embedding": {
-    "auc": 0.528328,
-    "empirical_epsilon": {
-      "0.05": 0.0,
-      "0.01": 0.0
-    },
-    "empirical_epsilon_details": {
-      "0.05": {
-        "epsilon": 0.0,
-        "num_guesses": 0,
-        "correct_guesses": 0,
-        "candidate_num_guesses": [
-          5,
-          10,
-          15,
-          20,
-          25,
-          30,
-          35,
-          40,
-          45,
-          50,
-          55,
-          60,
-          65,
-          70,
-          75,
-          80,
-          85,
-          90,
-          95,
-          100
-        ],
-        "direction": "lower"
-      },
-      "0.01": {
-        "epsilon": 0.0,
-        "num_guesses": 0,
-        "correct_guesses": 0,
-        "candidate_num_guesses": [
-          5,
-          10,
-          15,
-          20,
-          25,
-          30,
-          35,
-          40,
-          45,
-          50,
-          55,
-          60,
-          65,
-          70,
-          75,
-          80,
-          85,
-          90,
-          95,
-          100
-        ],
-        "direction": "lower"
-      }
-    }
-  }
-}
\ No newline at end of file
diff --git a/qwen1.5-moe-a2.7b/dp8_attn_v2/epochs/epoch_003/adapter/README.md b/qwen1.5-moe-a2.7b/dp8_attn_v2/epochs/epoch_003/adapter/README.md
deleted file mode 100644
index 3b9e588251d937f16d19c9a0dcc3c6c5e3404063..0000000000000000000000000000000000000000
--- a/qwen1.5-moe-a2.7b/dp8_attn_v2/epochs/epoch_003/adapter/README.md
+++ /dev/null
@@ -1,207 +0,0 @@
----
-base_model: Qwen/Qwen1.5-MoE-A2.7B
-library_name: peft
-pipeline_tag: text-generation
-tags:
-- base_model:adapter:Qwen/Qwen1.5-MoE-A2.7B
-- lora
-- transformers
----
-
-# Model Card for Model ID
-
-<!-- Provide a quick summary of what the model is/does. -->
-
-
-
-## Model Details
-
-### Model Description
-
-<!-- Provide a longer summary of what this model is. -->
-
-
-
-- **Developed by:** [More Information Needed]
-- **Funded by [optional]:** [More Information Needed]
-- **Shared by [optional]:** [More Information Needed]
-- **Model type:** [More Information Needed]
-- **Language(s) (NLP):** [More Information Needed]
-- **License:** [More Information Needed]
-- **Finetuned from model [optional]:** [More Information Needed]
-
-### Model Sources [optional]
-
-<!-- Provide the basic links for the model. -->
-
-- **Repository:** [More Information Needed]
-- **Paper [optional]:** [More Information Needed]
-- **Demo [optional]:** [More Information Needed]
-
-## Uses
-
-<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
-
-### Direct Use
-
-<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
-
-[More Information Needed]
-
-### Downstream Use [optional]
-
-<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
-
-[More Information Needed]
-
-### Out-of-Scope Use
-
-<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
-
-[More Information Needed]
-
-## Bias, Risks, and Limitations
-
-<!-- This section is meant to convey both technical and sociotechnical limitations. -->
-
-[More Information Needed]
-
-### Recommendations
-
-<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
-
-Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
-
-## How to Get Started with the Model
-
-Use the code below to get started with the model.
-
-[More Information Needed]
-
-## Training Details
-
-### Training Data
-
-<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
-
-[More Information Needed]
-
-### Training Procedure
-
-<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
-
-#### Preprocessing [optional]
-
-[More Information Needed]
-
-
-#### Training Hyperparameters
-
-- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
-
-#### Speeds, Sizes, Times [optional]
-
-<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
-
-[More Information Needed]
-
-## Evaluation
-
-<!-- This section describes the evaluation protocols and provides the results. -->
-
-### Testing Data, Factors & Metrics
-
-#### Testing Data
-
-<!-- This should link to a Dataset Card if possible. -->
-
-[More Information Needed]
-
-#### Factors
-
-<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
-
-[More Information Needed]
-
-#### Metrics
-
-<!-- These are the evaluation metrics being used, ideally with a description of why. -->
-
-[More Information Needed]
-
-### Results
-
-[More Information Needed]
-
-#### Summary
-
-
-
-## Model Examination [optional]
-
-<!-- Relevant interpretability work for the model goes here -->
-
-[More Information Needed]
-
-## Environmental Impact
-
-<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
-
-Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
-
-- **Hardware Type:** [More Information Needed]
-- **Hours used:** [More Information Needed]
-- **Cloud Provider:** [More Information Needed]
-- **Compute Region:** [More Information Needed]
-- **Carbon Emitted:** [More Information Needed]
-
-## Technical Specifications [optional]
-
-### Model Architecture and Objective
-
-[More Information Needed]
-
-### Compute Infrastructure
-
-[More Information Needed]
-
-#### Hardware
-
-[More Information Needed]
-
-#### Software
-
-[More Information Needed]
-
-## Citation [optional]
-
-<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
-
-**BibTeX:**
-
-[More Information Needed]
-
-**APA:**
-
-[More Information Needed]
-
-## Glossary [optional]
-
-<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
-
-[More Information Needed]
-
-## More Information [optional]
-
-[More Information Needed]
-
-## Model Card Authors [optional]
-
-[More Information Needed]
-
-## Model Card Contact
-
-[More Information Needed]
-### Framework versions
-
-- PEFT 0.18.1
\ No newline at end of file
diff --git a/qwen1.5-moe-a2.7b/dp8_attn_v2/epochs/epoch_003/adapter/adapter_config.json b/qwen1.5-moe-a2.7b/dp8_attn_v2/epochs/epoch_003/adapter/adapter_config.json
deleted file mode 100644
index 72a674c89ec3406f4e57c07f4178c67d553edb4d..0000000000000000000000000000000000000000
--- a/qwen1.5-moe-a2.7b/dp8_attn_v2/epochs/epoch_003/adapter/adapter_config.json
+++ /dev/null
@@ -1,46 +0,0 @@
-{
-  "alora_invocation_tokens": null,
-  "alpha_pattern": {},
-  "arrow_config": null,
-  "auto_mapping": null,
-  "base_model_name_or_path": "Qwen/Qwen1.5-MoE-A2.7B",
-  "bias": "none",
-  "corda_config": null,
-  "ensure_weight_tying": true,
-  "eva_config": null,
-  "exclude_modules": null,
-  "fan_in_fan_out": false,
-  "inference_mode": true,
-  "init_lora_weights": true,
-  "layer_replication": null,
-  "layers_pattern": null,
-  "layers_to_transform": null,
-  "loftq_config": {},
-  "lora_alpha": 32,
-  "lora_bias": false,
-  "lora_dropout": 0.05,
-  "megatron_config": null,
-  "megatron_core": "megatron.core",
-  "modules_to_save": [
-    "lm_head",
-    "embed_tokens"
-  ],
-  "peft_type": "LORA",
-  "peft_version": "0.18.1",
-  "qalora_group_size": 16,
-  "r": 16,
-  "rank_pattern": {},
-  "revision": null,
-  "target_modules": [
-    "q_proj",
-    "k_proj",
-    "v_proj",
-    "o_proj"
-  ],
-  "target_parameters": null,
-  "task_type": "CAUSAL_LM",
-  "trainable_token_indices": null,
-  "use_dora": false,
-  "use_qalora": false,
-  "use_rslora": false
-}
\ No newline at end of file
diff --git a/qwen1.5-moe-a2.7b/dp8_attn_v2/epochs/epoch_003/audit_results.json b/qwen1.5-moe-a2.7b/dp8_attn_v2/epochs/epoch_003/audit_results.json
deleted file mode 100644
index 5823a672a7705a24c53c4ea420c1a1700837aa68..0000000000000000000000000000000000000000
--- a/qwen1.5-moe-a2.7b/dp8_attn_v2/epochs/epoch_003/audit_results.json
+++ /dev/null
@@ -1,137 +0,0 @@
-{
-  "delta": 1e-05,
-  "num_canaries": 500,
-  "num_members": 250,
-  "paper_guess_fraction": 0.2,
-  "paper_guess_steps": 20,
-  "loss": {
-    "auc": 0.549096,
-    "empirical_epsilon": {
-      "0.05": 0.6400124887004495,
-      "0.01": 0.3146810317412019
-    },
-    "empirical_epsilon_details": {
-      "0.05": {
-        "epsilon": 0.6400124887004495,
-        "num_guesses": 20,
-        "correct_guesses": 17,
-        "candidate_num_guesses": [
-          5,
-          10,
-          15,
-          20,
-          25,
-          30,
-          35,
-          40,
-          45,
-          50,
-          55,
-          60,
-          65,
-          70,
-          75,
-          80,
-          85,
-          90,
-          95,
-          100
-        ],
-        "direction": "lower"
-      },
-      "0.01": {
-        "epsilon": 0.3146810317412019,
-        "num_guesses": 60,
-        "correct_guesses": 44,
-        "candidate_num_guesses": [
-          5,
-          10,
-          15,
-          20,
-          25,
-          30,
-          35,
-          40,
-          45,
-          50,
-          55,
-          60,
-          65,
-          70,
-          75,
-          80,
-          85,
-          90,
-          95,
-          100
-        ],
-        "direction": "lower"
-      }
-    }
-  },
-  "embedding": {
-    "auc": 0.562824,
-    "empirical_epsilon": {
-      "0.05": 0.0,
-      "0.01": 0.0
-    },
-    "empirical_epsilon_details": {
-      "0.05": {
-        "epsilon": 0.0,
-        "num_guesses": 0,
-        "correct_guesses": 0,
-        "candidate_num_guesses": [
-          5,
-          10,
-          15,
-          20,
-          25,
-          30,
-          35,
-          40,
-          45,
-          50,
-          55,
-          60,
-          65,
-          70,
-          75,
-          80,
-          85,
-          90,
-          95,
-          100
-        ],
-        "direction": "lower"
-      },
-      "0.01": {
-        "epsilon": 0.0,
-        "num_guesses": 0,
-        "correct_guesses": 0,
-        "candidate_num_guesses": [
-          5,
-          10,
-          15,
-          20,
-          25,
-          30,
-          35,
-          40,
-          45,
-          50,
-          55,
-          60,
-          65,
-          70,
-          75,
-          80,
-          85,
-          90,
-          95,
-          100
-        ],
-        "direction": "lower"
-      }
-    }
-  }
-}
\ No newline at end of file
diff --git a/qwen1.5-moe-a2.7b/dp8_attn_v2/epochs/epoch_004/adapter/README.md b/qwen1.5-moe-a2.7b/dp8_attn_v2/epochs/epoch_004/adapter/README.md
deleted file mode 100644
index 3b9e588251d937f16d19c9a0dcc3c6c5e3404063..0000000000000000000000000000000000000000
--- a/qwen1.5-moe-a2.7b/dp8_attn_v2/epochs/epoch_004/adapter/README.md
+++ /dev/null
@@ -1,207 +0,0 @@
----
-base_model: Qwen/Qwen1.5-MoE-A2.7B
-library_name: peft
-pipeline_tag: text-generation
-tags:
-- base_model:adapter:Qwen/Qwen1.5-MoE-A2.7B
-- lora
-- transformers
----
-
-# Model Card for Model ID
-
-<!-- Provide a quick summary of what the model is/does. -->
-
-
-
-## Model Details
-
-### Model Description
-
-<!-- Provide a longer summary of what this model is. -->
-
-
-
-- **Developed by:** [More Information Needed]
-- **Funded by [optional]:** [More Information Needed]
-- **Shared by [optional]:** [More Information Needed]
-- **Model type:** [More Information Needed]
-- **Language(s) (NLP):** [More Information Needed]
-- **License:** [More Information Needed]
-- **Finetuned from model [optional]:** [More Information Needed]
-
-### Model Sources [optional]
-
-<!-- Provide the basic links for the model. -->
-
-- **Repository:** [More Information Needed]
-- **Paper [optional]:** [More Information Needed]
-- **Demo [optional]:** [More Information Needed]
-
-## Uses
-
-<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
-
-### Direct Use
-
-<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
-
-[More Information Needed]
-
-### Downstream Use [optional]
-
-<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
-
-[More Information Needed]
-
-### Out-of-Scope Use
-
-<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
-
-[More Information Needed]
-
-## Bias, Risks, and Limitations
-
-<!-- This section is meant to convey both technical and sociotechnical limitations. -->
-
-[More Information Needed]
-
-### Recommendations
-
-<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
-
-Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
-
-## How to Get Started with the Model
-
-Use the code below to get started with the model.
-
-[More Information Needed]
-
-## Training Details
-
-### Training Data
-
-<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
-
-[More Information Needed]
-
-### Training Procedure
-
-<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
-
-#### Preprocessing [optional]
-
-[More Information Needed]
-
-
-#### Training Hyperparameters
-
-- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
-
-#### Speeds, Sizes, Times [optional]
-
-<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
-
-[More Information Needed]
-
-## Evaluation
-
-<!-- This section describes the evaluation protocols and provides the results. -->
-
-### Testing Data, Factors & Metrics
-
-#### Testing Data
-
-<!-- This should link to a Dataset Card if possible. -->
-
-[More Information Needed]
-
-#### Factors
-
-<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
-
-[More Information Needed]
-
-#### Metrics
-
-<!-- These are the evaluation metrics being used, ideally with a description of why. -->
-
-[More Information Needed]
-
-### Results
-
-[More Information Needed]
-
-#### Summary
-
-
-
-## Model Examination [optional]
-
-<!-- Relevant interpretability work for the model goes here -->
-
-[More Information Needed]
-
-## Environmental Impact
-
-<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
-
-Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
-
-- **Hardware Type:** [More Information Needed]
-- **Hours used:** [More Information Needed]
-- **Cloud Provider:** [More Information Needed]
-- **Compute Region:** [More Information Needed]
-- **Carbon Emitted:** [More Information Needed]
-
-## Technical Specifications [optional]
-
-### Model Architecture and Objective
-
-[More Information Needed]
-
-### Compute Infrastructure
-
-[More Information Needed]
-
-#### Hardware
-
-[More Information Needed]
-
-#### Software
-
-[More Information Needed]
-
-## Citation [optional]
-
-<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
-
-**BibTeX:**
-
-[More Information Needed]
-
-**APA:**
-
-[More Information Needed]
-
-## Glossary [optional]
-
-<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
-
-[More Information Needed]
-
-## More Information [optional]
-
-[More Information Needed]
-
-## Model Card Authors [optional]
-
-[More Information Needed]
-
-## Model Card Contact
-
-[More Information Needed]
-### Framework versions
-
-- PEFT 0.18.1
\ No newline at end of file
diff --git a/qwen1.5-moe-a2.7b/dp8_attn_v2/epochs/epoch_004/adapter/adapter_config.json b/qwen1.5-moe-a2.7b/dp8_attn_v2/epochs/epoch_004/adapter/adapter_config.json
deleted file mode 100644
index 72a674c89ec3406f4e57c07f4178c67d553edb4d..0000000000000000000000000000000000000000
--- a/qwen1.5-moe-a2.7b/dp8_attn_v2/epochs/epoch_004/adapter/adapter_config.json
+++ /dev/null
@@ -1,46 +0,0 @@
-{
-  "alora_invocation_tokens": null,
-  "alpha_pattern": {},
-  "arrow_config": null,
-  "auto_mapping": null,
-  "base_model_name_or_path": "Qwen/Qwen1.5-MoE-A2.7B",
-  "bias": "none",
-  "corda_config": null,
-  "ensure_weight_tying": true,
-  "eva_config": null,
-  "exclude_modules": null,
-  "fan_in_fan_out": false,
-  "inference_mode": true,
-  "init_lora_weights": true,
-  "layer_replication": null,
-  "layers_pattern": null,
-  "layers_to_transform": null,
-  "loftq_config": {},
-  "lora_alpha": 32,
-  "lora_bias": false,
-  "lora_dropout": 0.05,
-  "megatron_config": null,
-  "megatron_core": "megatron.core",
-  "modules_to_save": [
-    "lm_head",
-    "embed_tokens"
-  ],
-  "peft_type": "LORA",
-  "peft_version": "0.18.1",
-  "qalora_group_size": 16,
-  "r": 16,
-  "rank_pattern": {},
-  "revision": null,
-  "target_modules": [
-    "q_proj",
-    "k_proj",
-    "v_proj",
-    "o_proj"
-  ],
-  "target_parameters": null,
-  "task_type": "CAUSAL_LM",
-  "trainable_token_indices": null,
-  "use_dora": false,
-  "use_qalora": false,
-  "use_rslora": false
-}
\ No newline at end of file
diff --git a/qwen1.5-moe-a2.7b/dp8_attn_v2/epochs/epoch_004/audit_results.json b/qwen1.5-moe-a2.7b/dp8_attn_v2/epochs/epoch_004/audit_results.json
deleted file mode 100644
index 38db557b8aba4f6570fd5d45f0979b14173940e7..0000000000000000000000000000000000000000
--- a/qwen1.5-moe-a2.7b/dp8_attn_v2/epochs/epoch_004/audit_results.json
+++ /dev/null
@@ -1,137 +0,0 @@
-{
-  "delta": 1e-05,
-  "num_canaries": 500,
-  "num_members": 250,
-  "paper_guess_fraction": 0.2,
-  "paper_guess_steps": 20,
-  "loss": {
-    "auc": 0.560864,
-    "empirical_epsilon": {
-      "0.05": 0.7536821570247412,
-      "0.01": 0.47146937996149063
-    },
-    "empirical_epsilon_details": {
-      "0.05": {
-        "epsilon": 0.7536821570247412,
-        "num_guesses": 30,
-        "correct_guesses": 25,
-        "candidate_num_guesses": [
-          5,
-          10,
-          15,
-          20,
-          25,
-          30,
-          35,
-          40,
-          45,
-          50,
-          55,
-          60,
-          65,
-          70,
-          75,
-          80,
-          85,
-          90,
-          95,
-          100
-        ],
-        "direction": "lower"
-      },
-      "0.01": {
-        "epsilon": 0.47146937996149063,
-        "num_guesses": 30,
-        "correct_guesses": 25,
-        "candidate_num_guesses": [
-          5,
-          10,
-          15,
-          20,
-          25,
-          30,
-          35,
-          40,
-          45,
-          50,
-          55,
-          60,
-          65,
-          70,
-          75,
-          80,
-          85,
-          90,
-          95,
-          100
-        ],
-        "direction": "lower"
-      }
-    }
-  },
-  "embedding": {
-    "auc": 0.57056,
-    "empirical_epsilon": {
-      "0.05": 0.0,
-      "0.01": 0.0
-    },
-    "empirical_epsilon_details": {
-      "0.05": {
-        "epsilon": 0.0,
-        "num_guesses": 0,
-        "correct_guesses": 0,
-        "candidate_num_guesses": [
-          5,
-          10,
-          15,
-          20,
-          25,
-          30,
-          35,
-          40,
-          45,
-          50,
-          55,
-          60,
-          65,
-          70,
-          75,
-          80,
-          85,
-          90,
-          95,
-          100
-        ],
-        "direction": "lower"
-      },
-      "0.01": {
-        "epsilon": 0.0,
-        "num_guesses": 0,
-        "correct_guesses": 0,
-        "candidate_num_guesses": [
-          5,
-          10,
-          15,
-          20,
-          25,
-          30,
-          35,
-          40,
-          45,
-          50,
-          55,
-          60,
-          65,
-          70,
-          75,
-          80,
-          85,
-          90,
-          95,
-          100
-        ],
-        "direction": "lower"
-      }
-    }
-  }
-}
\ No newline at end of file
diff --git a/qwen1.5-moe-a2.7b/dp8_attn_v2/epochs/epoch_005/adapter/README.md b/qwen1.5-moe-a2.7b/dp8_attn_v2/epochs/epoch_005/adapter/README.md
deleted file mode 100644
index 3b9e588251d937f16d19c9a0dcc3c6c5e3404063..0000000000000000000000000000000000000000
--- a/qwen1.5-moe-a2.7b/dp8_attn_v2/epochs/epoch_005/adapter/README.md
+++ /dev/null
@@ -1,207 +0,0 @@
----
-base_model: Qwen/Qwen1.5-MoE-A2.7B
-library_name: peft
-pipeline_tag: text-generation
-tags:
-- base_model:adapter:Qwen/Qwen1.5-MoE-A2.7B
-- lora
-- transformers
----
-
-# Model Card for Model ID
-
-<!-- Provide a quick summary of what the model is/does. -->
-
-
-
-## Model Details
-
-### Model Description
-
-<!-- Provide a longer summary of what this model is. -->
-
-
-
-- **Developed by:** [More Information Needed]
-- **Funded by [optional]:** [More Information Needed]
-- **Shared by [optional]:** [More Information Needed]
-- **Model type:** [More Information Needed]
-- **Language(s) (NLP):** [More Information Needed]
-- **License:** [More Information Needed]
-- **Finetuned from model [optional]:** [More Information Needed]
-
-### Model Sources [optional]
-
-<!-- Provide the basic links for the model. -->
-
-- **Repository:** [More Information Needed]
-- **Paper [optional]:** [More Information Needed]
-- **Demo [optional]:** [More Information Needed]
-
-## Uses
-
-<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
-
-### Direct Use
-
-<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
-
-[More Information Needed]
-
-### Downstream Use [optional]
-
-<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
-
-[More Information Needed]
-
-### Out-of-Scope Use
-
-<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
-
-[More Information Needed]
-
-## Bias, Risks, and Limitations
-
-<!-- This section is meant to convey both technical and sociotechnical limitations. -->
-
-[More Information Needed]
-
-### Recommendations
-
-<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
-
-Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
-
-## How to Get Started with the Model
-
-Use the code below to get started with the model.
-
-[More Information Needed]
-
-## Training Details
-
-### Training Data
-
-<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
-
-[More Information Needed]
-
-### Training Procedure
-
-<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
-
-#### Preprocessing [optional]
-
-[More Information Needed]
-
-
-#### Training Hyperparameters
-
-- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
-
-#### Speeds, Sizes, Times [optional]
-
-<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
-
-[More Information Needed]
-
-## Evaluation
-
-<!-- This section describes the evaluation protocols and provides the results. -->
-
-### Testing Data, Factors & Metrics
-
-#### Testing Data
-
-<!-- This should link to a Dataset Card if possible. -->
-
-[More Information Needed]
-
-#### Factors
-
-<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
-
-[More Information Needed]
-
-#### Metrics
-
-<!-- These are the evaluation metrics being used, ideally with a description of why. -->
-
-[More Information Needed]
-
-### Results
-
-[More Information Needed]
-
-#### Summary
-
-
-
-## Model Examination [optional]
-
-<!-- Relevant interpretability work for the model goes here -->
-
-[More Information Needed]
-
-## Environmental Impact
-
-<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
-
-Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
-
-- **Hardware Type:** [More Information Needed]
-- **Hours used:** [More Information Needed]
-- **Cloud Provider:** [More Information Needed]
-- **Compute Region:** [More Information Needed]
-- **Carbon Emitted:** [More Information Needed]
-
-## Technical Specifications [optional]
-
-### Model Architecture and Objective
-
-[More Information Needed]
-
-### Compute Infrastructure
-
-[More Information Needed]
-
-#### Hardware
-
-[More Information Needed]
-
-#### Software
-
-[More Information Needed]
-
-## Citation [optional]
-
-<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
-
-**BibTeX:**
-
-[More Information Needed]
-
-**APA:**
-
-[More Information Needed]
-
-## Glossary [optional]
-
-<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
-
-[More Information Needed]
-
-## More Information [optional]
-
-[More Information Needed]
-
-## Model Card Authors [optional]
-
-[More Information Needed]
-
-## Model Card Contact
-
-[More Information Needed]
-### Framework versions
-
-- PEFT 0.18.1
\ No newline at end of file
diff --git a/qwen1.5-moe-a2.7b/dp8_attn_v2/epochs/epoch_005/adapter/adapter_config.json b/qwen1.5-moe-a2.7b/dp8_attn_v2/epochs/epoch_005/adapter/adapter_config.json
deleted file mode 100644
index 72a674c89ec3406f4e57c07f4178c67d553edb4d..0000000000000000000000000000000000000000
--- a/qwen1.5-moe-a2.7b/dp8_attn_v2/epochs/epoch_005/adapter/adapter_config.json
+++ /dev/null
@@ -1,46 +0,0 @@
-{
-  "alora_invocation_tokens": null,
-  "alpha_pattern": {},
-  "arrow_config": null,
-  "auto_mapping": null,
-  "base_model_name_or_path": "Qwen/Qwen1.5-MoE-A2.7B",
-  "bias": "none",
-  "corda_config": null,
-  "ensure_weight_tying": true,
-  "eva_config": null,
-  "exclude_modules": null,
-  "fan_in_fan_out": false,
-  "inference_mode": true,
-  "init_lora_weights": true,
-  "layer_replication": null,
-  "layers_pattern": null,
-  "layers_to_transform": null,
-  "loftq_config": {},
-  "lora_alpha": 32,
-  "lora_bias": false,
-  "lora_dropout": 0.05,
-  "megatron_config": null,
-  "megatron_core": "megatron.core",
-  "modules_to_save": [
-    "lm_head",
-    "embed_tokens"
-  ],
-  "peft_type": "LORA",
-  "peft_version": "0.18.1",
-  "qalora_group_size": 16,
-  "r": 16,
-  "rank_pattern": {},
-  "revision": null,
-  "target_modules": [
-    "q_proj",
-    "k_proj",
-    "v_proj",
-    "o_proj"
-  ],
-  "target_parameters": null,
-  "task_type": "CAUSAL_LM",
-  "trainable_token_indices": null,
-  "use_dora": false,
-  "use_qalora": false,
-  "use_rslora": false
-}
\ No newline at end of file
diff --git a/qwen1.5-moe-a2.7b/dp8_attn_v2/epochs/epoch_005/audit_results.json b/qwen1.5-moe-a2.7b/dp8_attn_v2/epochs/epoch_005/audit_results.json
deleted file mode 100644
index 891e68a58462346aab55c958357a56a94fe5c2f3..0000000000000000000000000000000000000000
--- a/qwen1.5-moe-a2.7b/dp8_attn_v2/epochs/epoch_005/audit_results.json
+++ /dev/null
@@ -1,137 +0,0 @@
-{
-  "delta": 1e-05,
-  "num_canaries": 500,
-  "num_members": 250,
-  "paper_guess_fraction": 0.2,
-  "paper_guess_steps": 20,
-  "loss": {
-    "auc": 0.568112,
-    "empirical_epsilon": {
-      "0.05": 0.9539302941411734,
-      "0.01": 0.6810795972123742
-    },
-    "empirical_epsilon_details": {
-      "0.05": {
-        "epsilon": 0.9539302941411734,
-        "num_guesses": 35,
-        "correct_guesses": 30,
-        "candidate_num_guesses": [
-          5,
-          10,
-          15,
-          20,
-          25,
-          30,
-          35,
-          40,
-          45,
-          50,
-          55,
-          60,
-          65,
-          70,
-          75,
-          80,
-          85,
-          90,
-          95,
-          100
-        ],
-        "direction": "lower"
-      },
-      "0.01": {
-        "epsilon": 0.6810795972123742,
-        "num_guesses": 35,
-        "correct_guesses": 30,
-        "candidate_num_guesses": [
-          5,
-          10,
-          15,
-          20,
-          25,
-          30,
-          35,
-          40,
-          45,
-          50,
-          55,
-          60,
-          65,
-          70,
-          75,
-          80,
-          85,
-          90,
-          95,
-          100
-        ],
-        "direction": "lower"
-      }
-    }
-  },
-  "embedding": {
-    "auc": 0.569768,
-    "empirical_epsilon": {
-      "0.05": 0.010390725918114185,
-      "0.01": 0.0
-    },
-    "empirical_epsilon_details": {
-      "0.05": {
-        "epsilon": 0.010390725918114185,
-        "num_guesses": 100,
-        "correct_guesses": 59,
-        "candidate_num_guesses": [
-          5,
-          10,
-          15,
-          20,
-          25,
-          30,
-          35,
-          40,
-          45,
-          50,
-          55,
-          60,
-          65,
-          70,
-          75,
-          80,
-          85,
-          90,
-          95,
-          100
-        ],
-        "direction": "higher"
-      },
-      "0.01": {
-        "epsilon": 0.0,
-        "num_guesses": 0,
-        "correct_guesses": 0,
-        "candidate_num_guesses": [
-          5,
-          10,
-          15,
-          20,
-          25,
-          30,
-          35,
-          40,
-          45,
-          50,
-          55,
-          60,
-          65,
-          70,
-          75,
-          80,
-          85,
-          90,
-          95,
-          100
-        ],
-        "direction": "lower"
-      }
-    }
-  }
-}
\ No newline at end of file
diff --git a/qwen3-8b-base/base/epochs/epoch_001/adapter/README.md b/qwen3-8b-base/base/epochs/epoch_001/adapter/README.md
deleted file mode 100644
index 3063755d5f621219b97f455d66479201adfb07ca..0000000000000000000000000000000000000000
--- a/qwen3-8b-base/base/epochs/epoch_001/adapter/README.md
+++ /dev/null
@@ -1,207 +0,0 @@
----
-base_model: Qwen/Qwen3-8B-Base
-library_name: peft
-pipeline_tag: text-generation
-tags:
-- base_model:adapter:Qwen/Qwen3-8B-Base
-- lora
-- transformers
----
-
-# Model Card for Model ID
-
-<!-- Provide a quick summary of what the model is/does. -->
-
-
-
-## Model Details
-
-### Model Description
-
-<!-- Provide a longer summary of what this model is. -->
-
-
-
-- **Developed by:** [More Information Needed]
-- **Funded by [optional]:** [More Information Needed]
-- **Shared by [optional]:** [More Information Needed]
-- **Model type:** [More Information Needed]
-- **Language(s) (NLP):** [More Information Needed]
-- **License:** [More Information Needed]
-- **Finetuned from model [optional]:** [More Information Needed]
-
-### Model Sources [optional]
-
-<!-- Provide the basic links for the model. -->
-
-- **Repository:** [More Information Needed]
-- **Paper [optional]:** [More Information Needed]
-- **Demo [optional]:** [More Information Needed]
-
-## Uses
-
-<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
-
-### Direct Use
-
-<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
-
-[More Information Needed]
-
-### Downstream Use [optional]
-
-<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
-
-[More Information Needed]
-
-### Out-of-Scope Use
-
-<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
-
-[More Information Needed]
-
-## Bias, Risks, and Limitations
-
-<!-- This section is meant to convey both technical and sociotechnical limitations. -->
-
-[More Information Needed]
-
-### Recommendations
-
-<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
-
-Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
-
-## How to Get Started with the Model
-
-Use the code below to get started with the model.
-
-[More Information Needed]
-
-## Training Details
-
-### Training Data
-
-<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
-
-[More Information Needed]
-
-### Training Procedure
-
-<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
-
-#### Preprocessing [optional]
-
-[More Information Needed]
-
-
-#### Training Hyperparameters
-
-- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
-
-#### Speeds, Sizes, Times [optional]
-
-<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
-
-[More Information Needed]
-
-## Evaluation
-
-<!-- This section describes the evaluation protocols and provides the results. -->
-
-### Testing Data, Factors & Metrics
-
-#### Testing Data
-
-<!-- This should link to a Dataset Card if possible. -->
-
-[More Information Needed]
-
-#### Factors
-
-<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
-
-[More Information Needed]
-
-#### Metrics
-
-<!-- These are the evaluation metrics being used, ideally with a description of why. -->
-
-[More Information Needed]
-
-### Results
-
-[More Information Needed]
-
-#### Summary
-
-
-
-## Model Examination [optional]
-
-<!-- Relevant interpretability work for the model goes here -->
-
-[More Information Needed]
-
-## Environmental Impact
-
-<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
-
-Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
-
-- **Hardware Type:** [More Information Needed]
-- **Hours used:** [More Information Needed]
-- **Cloud Provider:** [More Information Needed]
-- **Compute Region:** [More Information Needed]
-- **Carbon Emitted:** [More Information Needed]
-
-## Technical Specifications [optional]
-
-### Model Architecture and Objective
-
-[More Information Needed]
-
-### Compute Infrastructure
-
-[More Information Needed]
-
-#### Hardware
-
-[More Information Needed]
-
-#### Software
-
-[More Information Needed]
-
-## Citation [optional]
-
-<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
-
-**BibTeX:**
-
-[More Information Needed]
-
-**APA:**
-
-[More Information Needed]
-
-## Glossary [optional]
-
-<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
-
-[More Information Needed]
-
-## More Information [optional]
-
-[More Information Needed]
-
-## Model Card Authors [optional]
-
-[More Information Needed]
-
-## Model Card Contact
-
-[More Information Needed]
-### Framework versions
-
-- PEFT 0.18.1
\ No newline at end of file
diff --git a/qwen3-8b-base/base/epochs/epoch_001/adapter/adapter_config.json b/qwen3-8b-base/base/epochs/epoch_001/adapter/adapter_config.json
deleted file mode 100644
index a8ac90c700a739bb9c51dcfa69604cfd06ad0837..0000000000000000000000000000000000000000
--- a/qwen3-8b-base/base/epochs/epoch_001/adapter/adapter_config.json
+++ /dev/null
@@ -1,46 +0,0 @@
-{
-  "alora_invocation_tokens": null,
-  "alpha_pattern": {},
-  "arrow_config": null,
-  "auto_mapping": null,
-  "base_model_name_or_path": "Qwen/Qwen3-8B-Base",
-  "bias": "none",
-  "corda_config": null,
-  "ensure_weight_tying": true,
-  "eva_config": null,
-  "exclude_modules": null,
-  "fan_in_fan_out": false,
-  "inference_mode": true,
-  "init_lora_weights": true,
-  "layer_replication": null,
-  "layers_pattern": null,
-  "layers_to_transform": null,
-  "loftq_config": {},
-  "lora_alpha": 32,
-  "lora_bias": false,
-  "lora_dropout": 0.05,
-  "megatron_config": null,
-  "megatron_core": "megatron.core",
-  "modules_to_save": [
-    "lm_head",
-    "embed_tokens"
-  ],
-  "peft_type": "LORA",
-  "peft_version": "0.18.1",
-  "qalora_group_size": 16,
-  "r": 16,
-  "rank_pattern": {},
-  "revision": null,
-  "target_modules": [
-    "v_proj",
-    "o_proj",
-    "q_proj",
-    "k_proj"
-  ],
-  "target_parameters": null,
-  "task_type": "CAUSAL_LM",
-  "trainable_token_indices": null,
-  "use_dora": false,
-  "use_qalora": false,
-  "use_rslora": false
-}
\ No newline at end of file
diff --git a/qwen3-8b-base/base/epochs/epoch_001/audit_results.json b/qwen3-8b-base/base/epochs/epoch_001/audit_results.json
deleted file mode 100644
index dccd61f87686bf9a98c2d646d7f68afbcbb27fea..0000000000000000000000000000000000000000
--- a/qwen3-8b-base/base/epochs/epoch_001/audit_results.json
+++ /dev/null
@@ -1,137 +0,0 @@
-{
-  "delta": 1e-05,
-  "num_canaries": 500,
-  "num_members": 250,
-  "paper_guess_fraction": 0.2,
-  "paper_guess_steps": 20,
-  "loss": {
-    "auc": 0.9554,
-    "empirical_epsilon": {
-      "0.05": 3.4791953936219215,
-      "0.01": 3.023197554051876
-    },
-    "empirical_epsilon_details": {
-      "0.05": {
-        "epsilon": 3.4791953936219215,
-        "num_guesses": 100,
-        "correct_guesses": 100,
-        "candidate_num_guesses": [
-          5,
-          10,
-          15,
-          20,
-          25,
-          30,
-          35,
-          40,
-          45,
-          50,
-          55,
-          60,
-          65,
-          70,
-          75,
-          80,
-          85,
-          90,
-          95,
-          100
-        ],
-        "direction": "lower"
-      },
-      "0.01": {
-        "epsilon": 3.023197554051876,
-        "num_guesses": 100,
-        "correct_guesses": 100,
-        "candidate_num_guesses": [
-          5,
-          10,
-          15,
-          20,
-          25,
-          30,
-          35,
-          40,
-          45,
-          50,
-          55,
-          60,
-          65,
-          70,
-          75,
-          80,
-          85,
-          90,
-          95,
-          100
-        ],
-        "direction": "lower"
-      }
-    }
-  },
-  "embedding": {
-    "auc": 0.921392,
-    "empirical_epsilon": {
-      "0.05": 3.4791953936219215,
-      "0.01": 3.023197554051876
-    },
-    "empirical_epsilon_details": {
-      "0.05": {
-        "epsilon": 3.4791953936219215,
-        "num_guesses": 100,
-        "correct_guesses": 100,
-        "candidate_num_guesses": [
-          5,
-          10,
-          15,
-          20,
-          25,
-          30,
-          35,
-          40,
-          45,
-          50,
-          55,
-          60,
-          65,
-          70,
-          75,
-          80,
-          85,
-          90,
-          95,
-          100
-        ],
-        "direction": "lower"
-      },
-      "0.01": {
-        "epsilon": 3.023197554051876,
-        "num_guesses": 100,
-        "correct_guesses": 100,
-        "candidate_num_guesses": [
-          5,
-          10,
-          15,
-          20,
-          25,
-          30,
-          35,
-          40,
-          45,
-          50,
-          55,
-          60,
-          65,
-          70,
-          75,
-          80,
-          85,
-          90,
-          95,
-          100
-        ],
-        "direction": "lower"
-      }
-    }
-  }
-}
\ No newline at end of file
diff --git a/qwen3-8b-base/base/epochs/epoch_002/adapter/README.md b/qwen3-8b-base/base/epochs/epoch_002/adapter/README.md
deleted file mode 100644
index 3063755d5f621219b97f455d66479201adfb07ca..0000000000000000000000000000000000000000
--- a/qwen3-8b-base/base/epochs/epoch_002/adapter/README.md
+++ /dev/null
@@ -1,207 +0,0 @@
----
-base_model: Qwen/Qwen3-8B-Base
-library_name: peft
-pipeline_tag: text-generation
-tags:
-- base_model:adapter:Qwen/Qwen3-8B-Base
-- lora
-- transformers
----
-
-# Model Card for Model ID
-
-<!-- Provide a quick summary of what the model is/does. -->
-
-
-
-## Model Details
-
-### Model Description
-
-<!-- Provide a longer summary of what this model is. -->
-
-
-
-- **Developed by:** [More Information Needed]
-- **Funded by [optional]:** [More Information Needed]
-- **Shared by [optional]:** [More Information Needed]
-- **Model type:** [More Information Needed]
-- **Language(s) (NLP):** [More Information Needed]
-- **License:** [More Information Needed]
-- **Finetuned from model [optional]:** [More Information Needed]
-
-### Model Sources [optional]
-
-<!-- Provide the basic links for the model. -->
-
-- **Repository:** [More Information Needed]
-- **Paper [optional]:** [More Information Needed]
-- **Demo [optional]:** [More Information Needed]
-
-## Uses
-
-<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
-
-### Direct Use
-
-<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
-
-[More Information Needed]
-
-### Downstream Use [optional]
-
-<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
-
-[More Information Needed]
-
-### Out-of-Scope Use
-
-<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
-
-[More Information Needed]
-
-## Bias, Risks, and Limitations
-
-<!-- This section is meant to convey both technical and sociotechnical limitations. -->
-
-[More Information Needed]
-
-### Recommendations
-
-<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
-
-Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
-
-## How to Get Started with the Model
-
-Use the code below to get started with the model.
-
-[More Information Needed]
-
-## Training Details
-
-### Training Data
-
-<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
-
-[More Information Needed]
-
-### Training Procedure
-
-<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
-
-#### Preprocessing [optional]
-
-[More Information Needed]
-
-
-#### Training Hyperparameters
-
-- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
-
-#### Speeds, Sizes, Times [optional]
-
-<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
-
-[More Information Needed]
-
-## Evaluation
-
-<!-- This section describes the evaluation protocols and provides the results. -->
-
-### Testing Data, Factors & Metrics
-
-#### Testing Data
-
-<!-- This should link to a Dataset Card if possible. -->
-
-[More Information Needed]
-
-#### Factors
-
-<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
-
-[More Information Needed]
-
-#### Metrics
-
-<!-- These are the evaluation metrics being used, ideally with a description of why. -->
-
-[More Information Needed]
-
-### Results
-
-[More Information Needed]
-
-#### Summary
-
-
-
-## Model Examination [optional]
-
-<!-- Relevant interpretability work for the model goes here -->
-
-[More Information Needed]
-
-## Environmental Impact
-
-<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
-
-Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
-
-- **Hardware Type:** [More Information Needed]
-- **Hours used:** [More Information Needed]
-- **Cloud Provider:** [More Information Needed]
-- **Compute Region:** [More Information Needed]
-- **Carbon Emitted:** [More Information Needed]
-
-## Technical Specifications [optional]
-
-### Model Architecture and Objective
-
-[More Information Needed]
-
-### Compute Infrastructure
-
-[More Information Needed]
-
-#### Hardware
-
-[More Information Needed]
-
-#### Software
-
-[More Information Needed]
-
-## Citation [optional]
-
-<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
-
-**BibTeX:**
-
-[More Information Needed]
-
-**APA:**
-
-[More Information Needed]
-
-## Glossary [optional]
-
-<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
-
-[More Information Needed]
-
-## More Information [optional]
-
-[More Information Needed]
-
-## Model Card Authors [optional]
-
-[More Information Needed]
-
-## Model Card Contact
-
-[More Information Needed]
-### Framework versions
-
-- PEFT 0.18.1
\ No newline at end of file
diff --git a/qwen3-8b-base/base/epochs/epoch_002/adapter/adapter_config.json b/qwen3-8b-base/base/epochs/epoch_002/adapter/adapter_config.json
deleted file mode 100644
index a8ac90c700a739bb9c51dcfa69604cfd06ad0837..0000000000000000000000000000000000000000
--- a/qwen3-8b-base/base/epochs/epoch_002/adapter/adapter_config.json
+++ /dev/null
@@ -1,46 +0,0 @@
-{
-  "alora_invocation_tokens": null,
-  "alpha_pattern": {},
-  "arrow_config": null,
-  "auto_mapping": null,
-  "base_model_name_or_path": "Qwen/Qwen3-8B-Base",
-  "bias": "none",
-  "corda_config": null,
-  "ensure_weight_tying": true,
-  "eva_config": null,
-  "exclude_modules": null,
-  "fan_in_fan_out": false,
-  "inference_mode": true,
-  "init_lora_weights": true,
-  "layer_replication": null,
-  "layers_pattern": null,
-  "layers_to_transform": null,
-  "loftq_config": {},
-  "lora_alpha": 32,
-  "lora_bias": false,
-  "lora_dropout": 0.05,
-  "megatron_config": null,
-  "megatron_core": "megatron.core",
-  "modules_to_save": [
-    "lm_head",
-    "embed_tokens"
-  ],
-  "peft_type": "LORA",
-  "peft_version": "0.18.1",
-  "qalora_group_size": 16,
-  "r": 16,
-  "rank_pattern": {},
-  "revision": null,
-  "target_modules": [
-    "v_proj",
-    "o_proj",
-    "q_proj",
-    "k_proj"
-  ],
-  "target_parameters": null,
-  "task_type": "CAUSAL_LM",
-  "trainable_token_indices": null,
-  "use_dora": false,
-  "use_qalora": false,
-  "use_rslora": false
-}
\ No newline at end of file
diff --git a/qwen3-8b-base/base/epochs/epoch_002/audit_results.json b/qwen3-8b-base/base/epochs/epoch_002/audit_results.json
deleted file mode 100644
index 20fa16d6be1f5351d887cf45a29eeccd603dc0e1..0000000000000000000000000000000000000000
--- a/qwen3-8b-base/base/epochs/epoch_002/audit_results.json
+++ /dev/null
@@ -1,137 +0,0 @@
-{
-  "delta": 1e-05,
-  "num_canaries": 500,
-  "num_members": 250,
-  "paper_guess_fraction": 0.2,
-  "paper_guess_steps": 20,
-  "loss": {
-    "auc": 0.997936,
-    "empirical_epsilon": {
-      "0.05": 3.4791953936219215,
-      "0.01": 3.023197554051876
-    },
-    "empirical_epsilon_details": {
-      "0.05": {
-        "epsilon": 3.4791953936219215,
-        "num_guesses": 100,
-        "correct_guesses": 100,
-        "candidate_num_guesses": [
-          5,
-          10,
-          15,
-          20,
-          25,
-          30,
-          35,
-          40,
-          45,
-          50,
-          55,
-          60,
-          65,
-          70,
-          75,
-          80,
-          85,
-          90,
-          95,
-          100
-        ],
-        "direction": "lower"
-      },
-      "0.01": {
-        "epsilon": 3.023197554051876,
-        "num_guesses": 100,
-        "correct_guesses": 100,
-        "candidate_num_guesses": [
-          5,
-          10,
-          15,
-          20,
-          25,
-          30,
-          35,
-          40,
-          45,
-          50,
-          55,
-          60,
-          65,
-          70,
-          75,
-          80,
-          85,
-          90,
-          95,
-          100
-        ],
-        "direction": "lower"
-      }
-    }
-  },
-  "embedding": {
-    "auc": 0.901776,
-    "empirical_epsilon": {
-      "0.05": 3.4791953936219215,
-      "0.01": 3.023197554051876
-    },
-    "empirical_epsilon_details": {
-      "0.05": {
-        "epsilon": 3.4791953936219215,
-        "num_guesses": 100,
-        "correct_guesses": 100,
-        "candidate_num_guesses": [
-          5,
-          10,
-          15,
-          20,
-          25,
-          30,
-          35,
-          40,
-          45,
-          50,
-          55,
-          60,
-          65,
-          70,
-          75,
-          80,
-          85,
-          90,
-          95,
-          100
-        ],
-        "direction": "lower"
-      },
-      "0.01": {
-        "epsilon": 3.023197554051876,
-        "num_guesses": 100,
-        "correct_guesses": 100,
-        "candidate_num_guesses": [
-          5,
-          10,
-          15,
-          20,
-          25,
-          30,
-          35,
-          40,
-          45,
-          50,
-          55,
-          60,
-          65,
-          70,
-          75,
-          80,
-          85,
-          90,
-          95,
-          100
-        ],
-        "direction": "lower"
-      }
-    }
-  }
-}
\ No newline at end of file
diff --git a/qwen3-8b-base/base/epochs/epoch_003/adapter/README.md b/qwen3-8b-base/base/epochs/epoch_003/adapter/README.md
deleted file mode 100644
index 3063755d5f621219b97f455d66479201adfb07ca..0000000000000000000000000000000000000000
--- a/qwen3-8b-base/base/epochs/epoch_003/adapter/README.md
+++ /dev/null
@@ -1,207 +0,0 @@
----
-base_model: Qwen/Qwen3-8B-Base
-library_name: peft
-pipeline_tag: text-generation
-tags:
-- base_model:adapter:Qwen/Qwen3-8B-Base
-- lora
-- transformers
----
-
-# Model Card for Model ID
-
-<!-- Provide a quick summary of what the model is/does. -->
-
-
-
-## Model Details
-
-### Model Description
-
-<!-- Provide a longer summary of what this model is. -->
-
-
-
-- **Developed by:** [More Information Needed]
-- **Funded by [optional]:** [More Information Needed]
-- **Shared by [optional]:** [More Information Needed]
-- **Model type:** [More Information Needed]
-- **Language(s) (NLP):** [More Information Needed]
-- **License:** [More Information Needed]
-- **Finetuned from model [optional]:** [More Information Needed]
-
-### Model Sources [optional]
-
-<!-- Provide the basic links for the model. -->
-
-- **Repository:** [More Information Needed]
-- **Paper [optional]:** [More Information Needed]
-- **Demo [optional]:** [More Information Needed]
-
-## Uses
-
-<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
-
-### Direct Use
-
-<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
-
-[More Information Needed]
-
-### Downstream Use [optional]
-
-<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
-
-[More Information Needed]
-
-### Out-of-Scope Use
-
-<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
-
-[More Information Needed]
-
-## Bias, Risks, and Limitations
-
-<!-- This section is meant to convey both technical and sociotechnical limitations. -->
-
-[More Information Needed]
-
-### Recommendations
-
-<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
-
-Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
-
-## How to Get Started with the Model
-
-Use the code below to get started with the model.
-
-[More Information Needed]
-
-## Training Details
-
-### Training Data
-
-<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
-
-[More Information Needed]
-
-### Training Procedure
-
-<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
-
-#### Preprocessing [optional]
-
-[More Information Needed]
-
-
-#### Training Hyperparameters
-
-- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
-
-#### Speeds, Sizes, Times [optional]
-
-<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
-
-[More Information Needed]
-
-## Evaluation
-
-<!-- This section describes the evaluation protocols and provides the results. -->
-
-### Testing Data, Factors & Metrics
-
-#### Testing Data
-
-<!-- This should link to a Dataset Card if possible. -->
-
-[More Information Needed]
-
-#### Factors
-
-<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
-
-[More Information Needed]
-
-#### Metrics
-
-<!-- These are the evaluation metrics being used, ideally with a description of why. -->
-
-[More Information Needed]
-
-### Results
-
-[More Information Needed]
-
-#### Summary
-
-
-
-## Model Examination [optional]
-
-<!-- Relevant interpretability work for the model goes here -->
-
-[More Information Needed]
-
-## Environmental Impact
-
-<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
-
-Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
-
-- **Hardware Type:** [More Information Needed]
-- **Hours used:** [More Information Needed]
-- **Cloud Provider:** [More Information Needed]
-- **Compute Region:** [More Information Needed]
-- **Carbon Emitted:** [More Information Needed]
-
-## Technical Specifications [optional]
-
-### Model Architecture and Objective
-
-[More Information Needed]
-
-### Compute Infrastructure
-
-[More Information Needed]
-
-#### Hardware
-
-[More Information Needed]
-
-#### Software
-
-[More Information Needed]
-
-## Citation [optional]
-
-<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
-
-**BibTeX:**
-
-[More Information Needed]
-
-**APA:**
-
-[More Information Needed]
-
-## Glossary [optional]
-
-<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
-
-[More Information Needed]
-
-## More Information [optional]
-
-[More Information Needed]
-
-## Model Card Authors [optional]
-
-[More Information Needed]
-
-## Model Card Contact
-
-[More Information Needed]
-### Framework versions
-
-- PEFT 0.18.1
\ No newline at end of file
diff --git a/qwen3-8b-base/base/epochs/epoch_003/adapter/adapter_config.json b/qwen3-8b-base/base/epochs/epoch_003/adapter/adapter_config.json
deleted file mode 100644
index a8ac90c700a739bb9c51dcfa69604cfd06ad0837..0000000000000000000000000000000000000000
--- a/qwen3-8b-base/base/epochs/epoch_003/adapter/adapter_config.json
+++ /dev/null
@@ -1,46 +0,0 @@
-{
-  "alora_invocation_tokens": null,
-  "alpha_pattern": {},
-  "arrow_config": null,
-  "auto_mapping": null,
-  "base_model_name_or_path": "Qwen/Qwen3-8B-Base",
-  "bias": "none",
-  "corda_config": null,
-  "ensure_weight_tying": true,
-  "eva_config": null,
-  "exclude_modules": null,
-  "fan_in_fan_out": false,
-  "inference_mode": true,
-  "init_lora_weights": true,
-  "layer_replication": null,
-  "layers_pattern": null,
-  "layers_to_transform": null,
-  "loftq_config": {},
-  "lora_alpha": 32,
-  "lora_bias": false,
-  "lora_dropout": 0.05,
-  "megatron_config": null,
-  "megatron_core": "megatron.core",
-  "modules_to_save": [
-    "lm_head",
-    "embed_tokens"
-  ],
-  "peft_type": "LORA",
-  "peft_version": "0.18.1",
-  "qalora_group_size": 16,
-  "r": 16,
-  "rank_pattern": {},
-  "revision": null,
-  "target_modules": [
-    "v_proj",
-    "o_proj",
-    "q_proj",
-    "k_proj"
-  ],
-  "target_parameters": null,
-  "task_type": "CAUSAL_LM",
-  "trainable_token_indices": null,
-  "use_dora": false,
-  "use_qalora": false,
-  "use_rslora": false
-}
\ No newline at end of file
diff --git a/qwen3-8b-base/base/epochs/epoch_003/audit_results.json b/qwen3-8b-base/base/epochs/epoch_003/audit_results.json
deleted file mode 100644
index 56f3bb61422d6ee325051393b835b758c1c17c88..0000000000000000000000000000000000000000
--- a/qwen3-8b-base/base/epochs/epoch_003/audit_results.json
+++ /dev/null
@@ -1,137 +0,0 @@
-{
-  "delta": 1e-05,
-  "num_canaries": 500,
-  "num_members": 250,
-  "paper_guess_fraction": 0.2,
-  "paper_guess_steps": 20,
-  "loss": {
-    "auc": 1.0,
-    "empirical_epsilon": {
-      "0.05": 3.4791953936219215,
-      "0.01": 3.023197554051876
-    },
-    "empirical_epsilon_details": {
-      "0.05": {
-        "epsilon": 3.4791953936219215,
-        "num_guesses": 100,
-        "correct_guesses": 100,
-        "candidate_num_guesses": [
-          5,
-          10,
-          15,
-          20,
-          25,
-          30,
-          35,
-          40,
-          45,
-          50,
-          55,
-          60,
-          65,
-          70,
-          75,
-          80,
-          85,
-          90,
-          95,
-          100
-        ],
-        "direction": "lower"
-      },
-      "0.01": {
-        "epsilon": 3.023197554051876,
-        "num_guesses": 100,
-        "correct_guesses": 100,
-        "candidate_num_guesses": [
-          5,
-          10,
-          15,
-          20,
-          25,
-          30,
-          35,
-          40,
-          45,
-          50,
-          55,
-          60,
-          65,
-          70,
-          75,
-          80,
-          85,
-          90,
-          95,
-          100
-        ],
-        "direction": "lower"
-      }
-    }
-  },
-  "embedding": {
-    "auc": 0.90164,
-    "empirical_epsilon": {
-      "0.05": 3.4791953936219215,
-      "0.01": 3.023197554051876
-    },
-    "empirical_epsilon_details": {
-      "0.05": {
-        "epsilon": 3.4791953936219215,
-        "num_guesses": 100,
-        "correct_guesses": 100,
-        "candidate_num_guesses": [
-          5,
-          10,
-          15,
-          20,
-          25,
-          30,
-          35,
-          40,
-          45,
-          50,
-          55,
-          60,
-          65,
-          70,
-          75,
-          80,
-          85,
-          90,
-          95,
-          100
-        ],
-        "direction": "lower"
-      },
-      "0.01": {
-        "epsilon": 3.023197554051876,
-        "num_guesses": 100,
-        "correct_guesses": 100,
-        "candidate_num_guesses": [
-          5,
-          10,
-          15,
-          20,
-          25,
-          30,
-          35,
-          40,
-          45,
-          50,
-          55,
-          60,
-          65,
-          70,
-          75,
-          80,
-          85,
-          90,
-          95,
-          100
-        ],
-        "direction": "lower"
-      }
-    }
-  }
-}
\ No newline at end of file
diff --git a/qwen3-8b-base/dp3/epochs/epoch_001/adapter/README.md b/qwen3-8b-base/dp3/epochs/epoch_001/adapter/README.md
deleted file mode 100644
index 3063755d5f621219b97f455d66479201adfb07ca..0000000000000000000000000000000000000000
--- a/qwen3-8b-base/dp3/epochs/epoch_001/adapter/README.md
+++ /dev/null
@@ -1,207 +0,0 @@
----
-base_model: Qwen/Qwen3-8B-Base
-library_name: peft
-pipeline_tag: text-generation
-tags:
-- base_model:adapter:Qwen/Qwen3-8B-Base
-- lora
-- transformers
----
-
-# Model Card for Model ID
-
-<!-- Provide a quick summary of what the model is/does. -->
-
-
-
-## Model Details
-
-### Model Description
-
-<!-- Provide a longer summary of what this model is. -->
-
-
-
-- **Developed by:** [More Information Needed]
-- **Funded by [optional]:** [More Information Needed]
-- **Shared by [optional]:** [More Information Needed]
-- **Model type:** [More Information Needed]
-- **Language(s) (NLP):** [More Information Needed]
-- **License:** [More Information Needed]
-- **Finetuned from model [optional]:** [More Information Needed]
-
-### Model Sources [optional]
-
-<!-- Provide the basic links for the model. -->
-
-- **Repository:** [More Information Needed]
-- **Paper [optional]:** [More Information Needed]
-- **Demo [optional]:** [More Information Needed]
-
-## Uses
-
-<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
-
-### Direct Use
-
-<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
-
-[More Information Needed]
-
-### Downstream Use [optional]
-
-<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
-
-[More Information Needed]
-
-### Out-of-Scope Use
-
-<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
-
-[More Information Needed]
-
-## Bias, Risks, and Limitations
-
-<!-- This section is meant to convey both technical and sociotechnical limitations. -->
-
-[More Information Needed]
-
-### Recommendations
-
-<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
-
-Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
-
-## How to Get Started with the Model
-
-Use the code below to get started with the model.
-
-[More Information Needed]
-
-## Training Details
-
-### Training Data
-
-<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
-
-[More Information Needed]
-
-### Training Procedure
-
-<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
-
-#### Preprocessing [optional]
-
-[More Information Needed]
-
-
-#### Training Hyperparameters
-
-- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
-
-#### Speeds, Sizes, Times [optional]
-
-<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
-
-[More Information Needed]
-
-## Evaluation
-
-<!-- This section describes the evaluation protocols and provides the results. -->
-
-### Testing Data, Factors & Metrics
-
-#### Testing Data
-
-<!-- This should link to a Dataset Card if possible. -->
-
-[More Information Needed]
-
-#### Factors
-
-<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
-
-[More Information Needed]
-
-#### Metrics
-
-<!-- These are the evaluation metrics being used, ideally with a description of why. -->
-
-[More Information Needed]
-
-### Results
-
-[More Information Needed]
-
-#### Summary
-
-
-
-## Model Examination [optional]
-
-<!-- Relevant interpretability work for the model goes here -->
-
-[More Information Needed]
-
-## Environmental Impact
-
-<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
-
-Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
-
-- **Hardware Type:** [More Information Needed]
-- **Hours used:** [More Information Needed]
-- **Cloud Provider:** [More Information Needed]
-- **Compute Region:** [More Information Needed]
-- **Carbon Emitted:** [More Information Needed]
-
-## Technical Specifications [optional]
-
-### Model Architecture and Objective
-
-[More Information Needed]
-
-### Compute Infrastructure
-
-[More Information Needed]
-
-#### Hardware
-
-[More Information Needed]
-
-#### Software
-
-[More Information Needed]
-
-## Citation [optional]
-
-<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
-
-**BibTeX:**
-
-[More Information Needed]
-
-**APA:**
-
-[More Information Needed]
-
-## Glossary [optional]
-
-<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
-
-[More Information Needed]
-
-## More Information [optional]
-
-[More Information Needed]
-
-## Model Card Authors [optional]
-
-[More Information Needed]
-
-## Model Card Contact
-
-[More Information Needed]
-### Framework versions
-
-- PEFT 0.18.1
\ No newline at end of file
diff --git a/qwen3-8b-base/dp3/epochs/epoch_001/adapter/adapter_config.json b/qwen3-8b-base/dp3/epochs/epoch_001/adapter/adapter_config.json
deleted file mode 100644
index 3d1d4ffab7795dd087ec9f7f86b7b7698e41fa6e..0000000000000000000000000000000000000000
--- a/qwen3-8b-base/dp3/epochs/epoch_001/adapter/adapter_config.json
+++ /dev/null
@@ -1,46 +0,0 @@
-{
-  "alora_invocation_tokens": null,
-  "alpha_pattern": {},
-  "arrow_config": null,
-  "auto_mapping": null,
-  "base_model_name_or_path": "Qwen/Qwen3-8B-Base",
-  "bias": "none",
-  "corda_config": null,
-  "ensure_weight_tying": true,
-  "eva_config": null,
-  "exclude_modules": null,
-  "fan_in_fan_out": false,
-  "inference_mode": true,
-  "init_lora_weights": true,
-  "layer_replication": null,
-  "layers_pattern": null,
-  "layers_to_transform": null,
-  "loftq_config": {},
-  "lora_alpha": 32,
-  "lora_bias": false,
-  "lora_dropout": 0.05,
-  "megatron_config": null,
-  "megatron_core": "megatron.core",
-  "modules_to_save": [
-    "lm_head",
-    "embed_tokens"
-  ],
-  "peft_type": "LORA",
-  "peft_version": "0.18.1",
-  "qalora_group_size": 16,
-  "r": 16,
-  "rank_pattern": {},
-  "revision": null,
-  "target_modules": [
-    "o_proj",
-    "v_proj",
-    "q_proj",
-    "k_proj"
-  ],
-  "target_parameters": null,
-  "task_type": "CAUSAL_LM",
-  "trainable_token_indices": null,
-  "use_dora": false,
-  "use_qalora": false,
-  "use_rslora": false
-}
\ No newline at end of file
diff --git a/qwen3-8b-base/dp3/epochs/epoch_001/audit_results.json b/qwen3-8b-base/dp3/epochs/epoch_001/audit_results.json
deleted file mode 100644
index 783b24914182c70d6f3ca9f384fe72f835978cc0..0000000000000000000000000000000000000000
--- a/qwen3-8b-base/dp3/epochs/epoch_001/audit_results.json
+++ /dev/null
@@ -1,137 +0,0 @@
-{
-  "delta": 1e-05,
-  "num_canaries": 500,
-  "num_members": 250,
-  "paper_guess_fraction": 0.2,
-  "paper_guess_steps": 20,
-  "loss": {
-    "auc": 0.50132,
-    "empirical_epsilon": {
-      "0.05": 0.0,
-      "0.01": 0.0
-    },
-    "empirical_epsilon_details": {
-      "0.05": {
-        "epsilon": 0.0,
-        "num_guesses": 0,
-        "correct_guesses": 0,
-        "candidate_num_guesses": [
-          5,
-          10,
-          15,
-          20,
-          25,
-          30,
-          35,
-          40,
-          45,
-          50,
-          55,
-          60,
-          65,
-          70,
-          75,
-          80,
-          85,
-          90,
-          95,
-          100
-        ],
-        "direction": "lower"
-      },
-      "0.01": {
-        "epsilon": 0.0,
-        "num_guesses": 0,
-        "correct_guesses": 0,
-        "candidate_num_guesses": [
-          5,
-          10,
-          15,
-          20,
-          25,
-          30,
-          35,
-          40,
-          45,
-          50,
-          55,
-          60,
-          65,
-          70,
-          75,
-          80,
-          85,
-          90,
-          95,
-          100
-        ],
-        "direction": "lower"
-      }
-    }
-  },
-  "embedding": {
-    "auc": 0.502888,
-    "empirical_epsilon": {
-      "0.05": 0.0,
-      "0.01": 0.0
-    },
-    "empirical_epsilon_details": {
-      "0.05": {
-        "epsilon": 0.0,
-        "num_guesses": 0,
-        "correct_guesses": 0,
-        "candidate_num_guesses": [
-          5,
-          10,
-          15,
-          20,
-          25,
-          30,
-          35,
-          40,
-          45,
-          50,
-          55,
-          60,
-          65,
-          70,
-          75,
-          80,
-          85,
-          90,
-          95,
-          100
-        ],
-        "direction": "lower"
-      },
-      "0.01": {
-        "epsilon": 0.0,
-        "num_guesses": 0,
-        "correct_guesses": 0,
-        "candidate_num_guesses": [
-          5,
-          10,
-          15,
-          20,
-          25,
-          30,
-          35,
-          40,
-          45,
-          50,
-          55,
-          60,
-          65,
-          70,
-          75,
-          80,
-          85,
-          90,
-          95,
-          100
-        ],
-        "direction": "lower"
-      }
-    }
-  }
-}
\ No newline at end of file
diff --git a/qwen3-8b-base/dp3/epochs/epoch_002/adapter/README.md b/qwen3-8b-base/dp3/epochs/epoch_002/adapter/README.md
deleted file mode 100644
index 3063755d5f621219b97f455d66479201adfb07ca..0000000000000000000000000000000000000000
--- a/qwen3-8b-base/dp3/epochs/epoch_002/adapter/README.md
+++ /dev/null
@@ -1,207 +0,0 @@
----
-base_model: Qwen/Qwen3-8B-Base
-library_name: peft
-pipeline_tag: text-generation
-tags:
-- base_model:adapter:Qwen/Qwen3-8B-Base
-- lora
-- transformers
----
-
-# Model Card for Model ID
-
-<!-- Provide a quick summary of what the model is/does. -->
-
-
-
-## Model Details
-
-### Model Description
-
-<!-- Provide a longer summary of what this model is. -->
-
-
-
-- **Developed by:** [More Information Needed]
-- **Funded by [optional]:** [More Information Needed]
-- **Shared by [optional]:** [More Information Needed]
-- **Model type:** [More Information Needed]
-- **Language(s) (NLP):** [More Information Needed]
-- **License:** [More Information Needed]
-- **Finetuned from model [optional]:** [More Information Needed]
-
-### Model Sources [optional]
-
-<!-- Provide the basic links for the model. -->
-
-- **Repository:** [More Information Needed]
-- **Paper [optional]:** [More Information Needed]
-- **Demo [optional]:** [More Information Needed]
-
-## Uses
-
-<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
-
-### Direct Use
-
-<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
-
-[More Information Needed]
-
-### Downstream Use [optional]
-
-<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
-
-[More Information Needed]
-
-### Out-of-Scope Use
-
-<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
-
-[More Information Needed]
-
-## Bias, Risks, and Limitations
-
-<!-- This section is meant to convey both technical and sociotechnical limitations. -->
-
-[More Information Needed]
-
-### Recommendations
-
-<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
-
-Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
-
-## How to Get Started with the Model
-
-Use the code below to get started with the model.
-
-[More Information Needed]
-
-## Training Details
-
-### Training Data
-
-<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
-
-[More Information Needed]
-
-### Training Procedure
-
-<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
-
-#### Preprocessing [optional]
-
-[More Information Needed]
-
-
-#### Training Hyperparameters
-
-- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
-
-#### Speeds, Sizes, Times [optional]
-
-<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
-
-[More Information Needed]
-
-## Evaluation
-
-<!-- This section describes the evaluation protocols and provides the results. -->
-
-### Testing Data, Factors & Metrics
-
-#### Testing Data
-
-<!-- This should link to a Dataset Card if possible. -->
-
-[More Information Needed]
-
-#### Factors
-
-<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
-
-[More Information Needed]
-
-#### Metrics
-
-<!-- These are the evaluation metrics being used, ideally with a description of why. -->
-
-[More Information Needed]
-
-### Results
-
-[More Information Needed]
-
-#### Summary
-
-
-
-## Model Examination [optional]
-
-<!-- Relevant interpretability work for the model goes here -->
-
-[More Information Needed]
-
-## Environmental Impact
-
-<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
-
-Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
-
-- **Hardware Type:** [More Information Needed]
-- **Hours used:** [More Information Needed]
-- **Cloud Provider:** [More Information Needed]
-- **Compute Region:** [More Information Needed]
-- **Carbon Emitted:** [More Information Needed]
-
-## Technical Specifications [optional]
-
-### Model Architecture and Objective
-
-[More Information Needed]
-
-### Compute Infrastructure
-
-[More Information Needed]
-
-#### Hardware
-
-[More Information Needed]
-
-#### Software
-
-[More Information Needed]
-
-## Citation [optional]
-
-<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
-
-**BibTeX:**
-
-[More Information Needed]
-
-**APA:**
-
-[More Information Needed]
-
-## Glossary [optional]
-
-<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
-
-[More Information Needed]
-
-## More Information [optional]
-
-[More Information Needed]
-
-## Model Card Authors [optional]
-
-[More Information Needed]
-
-## Model Card Contact
-
-[More Information Needed]
-### Framework versions
-
-- PEFT 0.18.1
\ No newline at end of file
diff --git a/qwen3-8b-base/dp3/epochs/epoch_002/adapter/adapter_config.json b/qwen3-8b-base/dp3/epochs/epoch_002/adapter/adapter_config.json
deleted file mode 100644
index 3d1d4ffab7795dd087ec9f7f86b7b7698e41fa6e..0000000000000000000000000000000000000000
--- a/qwen3-8b-base/dp3/epochs/epoch_002/adapter/adapter_config.json
+++ /dev/null
@@ -1,46 +0,0 @@
-{
-  "alora_invocation_tokens": null,
-  "alpha_pattern": {},
-  "arrow_config": null,
-  "auto_mapping": null,
-  "base_model_name_or_path": "Qwen/Qwen3-8B-Base",
-  "bias": "none",
-  "corda_config": null,
-  "ensure_weight_tying": true,
-  "eva_config": null,
-  "exclude_modules": null,
-  "fan_in_fan_out": false,
-  "inference_mode": true,
-  "init_lora_weights": true,
-  "layer_replication": null,
-  "layers_pattern": null,
-  "layers_to_transform": null,
-  "loftq_config": {},
-  "lora_alpha": 32,
-  "lora_bias": false,
-  "lora_dropout": 0.05,
-  "megatron_config": null,
-  "megatron_core": "megatron.core",
-  "modules_to_save": [
-    "lm_head",
-    "embed_tokens"
-  ],
-  "peft_type": "LORA",
-  "peft_version": "0.18.1",
-  "qalora_group_size": 16,
-  "r": 16,
-  "rank_pattern": {},
-  "revision": null,
-  "target_modules": [
-    "o_proj",
-    "v_proj",
-    "q_proj",
-    "k_proj"
-  ],
-  "target_parameters": null,
-  "task_type": "CAUSAL_LM",
-  "trainable_token_indices": null,
-  "use_dora": false,
-  "use_qalora": false,
-  "use_rslora": false
-}
\ No newline at end of file
diff --git a/qwen3-8b-base/dp3/epochs/epoch_002/audit_results.json b/qwen3-8b-base/dp3/epochs/epoch_002/audit_results.json
deleted file mode 100644
index 58d1e7b7fd2b15675efc456071ae7e02a567aade..0000000000000000000000000000000000000000
--- a/qwen3-8b-base/dp3/epochs/epoch_002/audit_results.json
+++ /dev/null
@@ -1,137 +0,0 @@
-{
-  "delta": 1e-05,
-  "num_canaries": 500,
-  "num_members": 250,
-  "paper_guess_fraction": 0.2,
-  "paper_guess_steps": 20,
-  "loss": {
-    "auc": 0.561696,
-    "empirical_epsilon": {
-      "0.05": 0.0,
-      "0.01": 0.0
-    },
-    "empirical_epsilon_details": {
-      "0.05": {
-        "epsilon": 0.0,
-        "num_guesses": 0,
-        "correct_guesses": 0,
-        "candidate_num_guesses": [
-          5,
-          10,
-          15,
-          20,
-          25,
-          30,
-          35,
-          40,
-          45,
-          50,
-          55,
-          60,
-          65,
-          70,
-          75,
-          80,
-          85,
-          90,
-          95,
-          100
-        ],
-        "direction": "lower"
-      },
-      "0.01": {
-        "epsilon": 0.0,
-        "num_guesses": 0,
-        "correct_guesses": 0,
-        "candidate_num_guesses": [
-          5,
-          10,
-          15,
-          20,
-          25,
-          30,
-          35,
-          40,
-          45,
-          50,
-          55,
-          60,
-          65,
-          70,
-          75,
-          80,
-          85,
-          90,
-          95,
-          100
-        ],
-        "direction": "lower"
-      }
-    }
-  },
-  "embedding": {
-    "auc": 0.514544,
-    "empirical_epsilon": {
-      "0.05": 0.14663786999881268,
-      "0.01": 0.0
-    },
-    "empirical_epsilon_details": {
-      "0.05": {
-        "epsilon": 0.14663786999881268,
-        "num_guesses": 25,
-        "correct_guesses": 18,
-        "candidate_num_guesses": [
-          5,
-          10,
-          15,
-          20,
-          25,
-          30,
-          35,
-          40,
-          45,
-          50,
-          55,
-          60,
-          65,
-          70,
-          75,
-          80,
-          85,
-          90,
-          95,
-          100
-        ],
-        "direction": "higher"
-      },
-      "0.01": {
-        "epsilon": 0.0,
-        "num_guesses": 0,
-        "correct_guesses": 0,
-        "candidate_num_guesses": [
-          5,
-          10,
-          15,
-          20,
-          25,
-          30,
-          35,
-          40,
-          45,
-          50,
-          55,
-          60,
-          65,
-          70,
-          75,
-          80,
-          85,
-          90,
-          95,
-          100
-        ],
-        "direction": "lower"
-      }
-    }
-  }
-}
\ No newline at end of file
diff --git a/qwen3-8b-base/dp3/epochs/epoch_003/adapter/README.md b/qwen3-8b-base/dp3/epochs/epoch_003/adapter/README.md
deleted file mode 100644
index 3063755d5f621219b97f455d66479201adfb07ca..0000000000000000000000000000000000000000
--- a/qwen3-8b-base/dp3/epochs/epoch_003/adapter/README.md
+++ /dev/null
@@ -1,207 +0,0 @@
----
-base_model: Qwen/Qwen3-8B-Base
-library_name: peft
-pipeline_tag: text-generation
-tags:
-- base_model:adapter:Qwen/Qwen3-8B-Base
-- lora
-- transformers
----
-
-# Model Card for Model ID
-
-<!-- Provide a quick summary of what the model is/does. -->
-
-
-
-## Model Details
-
-### Model Description
-
-<!-- Provide a longer summary of what this model is. -->
-
-
-
-- **Developed by:** [More Information Needed]
-- **Funded by [optional]:** [More Information Needed]
-- **Shared by [optional]:** [More Information Needed]
-- **Model type:** [More Information Needed]
-- **Language(s) (NLP):** [More Information Needed]
-- **License:** [More Information Needed]
-- **Finetuned from model [optional]:** [More Information Needed]
-
-### Model Sources [optional]
-
-<!-- Provide the basic links for the model. -->
-
-- **Repository:** [More Information Needed]
-- **Paper [optional]:** [More Information Needed]
-- **Demo [optional]:** [More Information Needed]
-
-## Uses
-
-<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
-
-### Direct Use
-
-<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
-
-[More Information Needed]
-
-### Downstream Use [optional]
-
-<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
-
-[More Information Needed]
-
-### Out-of-Scope Use
-
-<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
-
-[More Information Needed]
-
-## Bias, Risks, and Limitations
-
-<!-- This section is meant to convey both technical and sociotechnical limitations. -->
-
-[More Information Needed]
-
-### Recommendations
-
-<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
-
-Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
-
-## How to Get Started with the Model
-
-Use the code below to get started with the model.
-
-[More Information Needed]
-
-## Training Details
-
-### Training Data
-
-<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
-
-[More Information Needed]
-
-### Training Procedure
-
-<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
-
-#### Preprocessing [optional]
-
-[More Information Needed]
-
-
-#### Training Hyperparameters
-
-- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
-
-#### Speeds, Sizes, Times [optional]
-
-<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
-
-[More Information Needed]
-
-## Evaluation
-
-<!-- This section describes the evaluation protocols and provides the results. -->
-
-### Testing Data, Factors & Metrics
-
-#### Testing Data
-
-<!-- This should link to a Dataset Card if possible. -->
-
-[More Information Needed]
-
-#### Factors
-
-<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
-
-[More Information Needed]
-
-#### Metrics
-
-<!-- These are the evaluation metrics being used, ideally with a description of why. -->
-
-[More Information Needed]
-
-### Results
-
-[More Information Needed]
-
-#### Summary
-
-
-
-## Model Examination [optional]
-
-<!-- Relevant interpretability work for the model goes here -->
-
-[More Information Needed]
-
-## Environmental Impact
-
-<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
-
-Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
-
-- **Hardware Type:** [More Information Needed]
-- **Hours used:** [More Information Needed]
-- **Cloud Provider:** [More Information Needed]
-- **Compute Region:** [More Information Needed]
-- **Carbon Emitted:** [More Information Needed]
-
-## Technical Specifications [optional]
-
-### Model Architecture and Objective
-
-[More Information Needed]
-
-### Compute Infrastructure
-
-[More Information Needed]
-
-#### Hardware
-
-[More Information Needed]
-
-#### Software
-
-[More Information Needed]
-
-## Citation [optional]
-
-<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
-
-**BibTeX:**
-
-[More Information Needed]
-
-**APA:**
-
-[More Information Needed]
-
-## Glossary [optional]
-
-<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
-
-[More Information Needed]
-
-## More Information [optional]
-
-[More Information Needed]
-
-## Model Card Authors [optional]
-
-[More Information Needed]
-
-## Model Card Contact
-
-[More Information Needed]
-### Framework versions
-
-- PEFT 0.18.1
\ No newline at end of file
diff --git a/qwen3-8b-base/dp3/epochs/epoch_003/adapter/adapter_config.json b/qwen3-8b-base/dp3/epochs/epoch_003/adapter/adapter_config.json
deleted file mode 100644
index 3d1d4ffab7795dd087ec9f7f86b7b7698e41fa6e..0000000000000000000000000000000000000000
--- a/qwen3-8b-base/dp3/epochs/epoch_003/adapter/adapter_config.json
+++ /dev/null
@@ -1,46 +0,0 @@
-{
-  "alora_invocation_tokens": null,
-  "alpha_pattern": {},
-  "arrow_config": null,
-  "auto_mapping": null,
-  "base_model_name_or_path": "Qwen/Qwen3-8B-Base",
-  "bias": "none",
-  "corda_config": null,
-  "ensure_weight_tying": true,
-  "eva_config": null,
-  "exclude_modules": null,
-  "fan_in_fan_out": false,
-  "inference_mode": true,
-  "init_lora_weights": true,
-  "layer_replication": null,
-  "layers_pattern": null,
-  "layers_to_transform": null,
-  "loftq_config": {},
-  "lora_alpha": 32,
-  "lora_bias": false,
-  "lora_dropout": 0.05,
-  "megatron_config": null,
-  "megatron_core": "megatron.core",
-  "modules_to_save": [
-    "lm_head",
-    "embed_tokens"
-  ],
-  "peft_type": "LORA",
-  "peft_version": "0.18.1",
-  "qalora_group_size": 16,
-  "r": 16,
-  "rank_pattern": {},
-  "revision": null,
-  "target_modules": [
-    "o_proj",
-    "v_proj",
-    "q_proj",
-    "k_proj"
-  ],
-  "target_parameters": null,
-  "task_type": "CAUSAL_LM",
-  "trainable_token_indices": null,
-  "use_dora": false,
-  "use_qalora": false,
-  "use_rslora": false
-}
\ No newline at end of file
diff --git a/qwen3-8b-base/dp3/epochs/epoch_003/audit_results.json b/qwen3-8b-base/dp3/epochs/epoch_003/audit_results.json
deleted file mode 100644
index 15009642e4e03a549fbc8c971359e99fe26be4e9..0000000000000000000000000000000000000000
--- a/qwen3-8b-base/dp3/epochs/epoch_003/audit_results.json
+++ /dev/null
@@ -1,137 +0,0 @@
-{
-  "delta": 1e-05,
-  "num_canaries": 500,
-  "num_members": 250,
-  "paper_guess_fraction": 0.2,
-  "paper_guess_steps": 20,
-  "loss": {
-    "auc": 0.548672,
-    "empirical_epsilon": {
-      "0.05": 0.0,
-      "0.01": 0.0
-    },
-    "empirical_epsilon_details": {
-      "0.05": {
-        "epsilon": 0.0,
-        "num_guesses": 0,
-        "correct_guesses": 0,
-        "candidate_num_guesses": [
-          5,
-          10,
-          15,
-          20,
-          25,
-          30,
-          35,
-          40,
-          45,
-          50,
-          55,
-          60,
-          65,
-          70,
-          75,
-          80,
-          85,
-          90,
-          95,
-          100
-        ],
-        "direction": "lower"
-      },
-      "0.01": {
-        "epsilon": 0.0,
-        "num_guesses": 0,
-        "correct_guesses": 0,
-        "candidate_num_guesses": [
-          5,
-          10,
-          15,
-          20,
-          25,
-          30,
-          35,
-          40,
-          45,
-          50,
-          55,
-          60,
-          65,
-          70,
-          75,
-          80,
-          85,
-          90,
-          95,
-          100
-        ],
-        "direction": "lower"
-      }
-    }
-  },
-  "embedding": {
-    "auc": 0.51744,
-    "empirical_epsilon": {
-      "0.05": 0.054578399285674095,
-      "0.01": 0.0
-    },
-    "empirical_epsilon_details": {
-      "0.05": {
-        "epsilon": 0.054578399285674095,
-        "num_guesses": 50,
-        "correct_guesses": 32,
-        "candidate_num_guesses": [
-          5,
-          10,
-          15,
-          20,
-          25,
-          30,
-          35,
-          40,
-          45,
-          50,
-          55,
-          60,
-          65,
-          70,
-          75,
-          80,
-          85,
-          90,
-          95,
-          100
-        ],
-        "direction": "higher"
-      },
-      "0.01": {
-        "epsilon": 0.0,
-        "num_guesses": 0,
-        "correct_guesses": 0,
-        "candidate_num_guesses": [
-          5,
-          10,
-          15,
-          20,
-          25,
-          30,
-          35,
-          40,
-          45,
-          50,
-          55,
-          60,
-          65,
-          70,
-          75,
-          80,
-          85,
-          90,
-          95,
-          100
-        ],
-        "direction": "lower"
-      }
-    }
-  }
-}
\ No newline at end of file
diff --git a/qwen3-8b-base/dp3_v2/epochs/epoch_001/adapter/README.md b/qwen3-8b-base/dp3_v2/epochs/epoch_001/adapter/README.md
deleted file mode 100644
index 3063755d5f621219b97f455d66479201adfb07ca..0000000000000000000000000000000000000000
--- a/qwen3-8b-base/dp3_v2/epochs/epoch_001/adapter/README.md
+++ /dev/null
@@ -1,207 +0,0 @@
----
-base_model: Qwen/Qwen3-8B-Base
-library_name: peft
-pipeline_tag: text-generation
-tags:
-- base_model:adapter:Qwen/Qwen3-8B-Base
-- lora
-- transformers
----
-
-# Model Card for Model ID
-
-<!-- Provide a quick summary of what the model is/does. -->
-
-
-
-## Model Details
-
-### Model Description
-
-<!-- Provide a longer summary of what this model is. -->
-
-
-
-- **Developed by:** [More Information Needed]
-- **Funded by [optional]:** [More Information Needed]
-- **Shared by [optional]:** [More Information Needed]
-- **Model type:** [More Information Needed]
-- **Language(s) (NLP):** [More Information Needed]
-- **License:** [More Information Needed]
-- **Finetuned from model [optional]:** [More Information Needed]
-
-### Model Sources [optional]
-
-<!-- Provide the basic links for the model. -->
-
-- **Repository:** [More Information Needed]
-- **Paper [optional]:** [More Information Needed]
-- **Demo [optional]:** [More Information Needed]
-
-## Uses
-
-<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
-
-### Direct Use
-
-<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
-
-[More Information Needed]
-
-### Downstream Use [optional]
-
-<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
-
-[More Information Needed]
-
-### Out-of-Scope Use
-
-<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
-
-[More Information Needed]
-
-## Bias, Risks, and Limitations
-
-<!-- This section is meant to convey both technical and sociotechnical limitations. -->
-
-[More Information Needed]
-
-### Recommendations
-
-<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
-
-Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
-
-## How to Get Started with the Model
-
-Use the code below to get started with the model.
-
-[More Information Needed]
-
-## Training Details
-
-### Training Data
-
-<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
-
-[More Information Needed]
-
-### Training Procedure
-
-<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
-
-#### Preprocessing [optional]
-
-[More Information Needed]
-
-
-#### Training Hyperparameters
-
-- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
-
-#### Speeds, Sizes, Times [optional]
-
-<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
-
-[More Information Needed]
-
-## Evaluation
-
-<!-- This section describes the evaluation protocols and provides the results. -->
-
-### Testing Data, Factors & Metrics
-
-#### Testing Data
-
-<!-- This should link to a Dataset Card if possible. -->
-
-[More Information Needed]
-
-#### Factors
-
-<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
-
-[More Information Needed]
-
-#### Metrics
-
-<!-- These are the evaluation metrics being used, ideally with a description of why. -->
-
-[More Information Needed]
-
-### Results
-
-[More Information Needed]
-
-#### Summary
-
-
-
-## Model Examination [optional]
-
-<!-- Relevant interpretability work for the model goes here -->
-
-[More Information Needed]
-
-## Environmental Impact
-
-<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
-
-Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
-
-- **Hardware Type:** [More Information Needed]
-- **Hours used:** [More Information Needed]
-- **Cloud Provider:** [More Information Needed]
-- **Compute Region:** [More Information Needed]
-- **Carbon Emitted:** [More Information Needed]
-
-## Technical Specifications [optional]
-
-### Model Architecture and Objective
-
-[More Information Needed]
-
-### Compute Infrastructure
-
-[More Information Needed]
-
-#### Hardware
-
-[More Information Needed]
-
-#### Software
-
-[More Information Needed]
-
-## Citation [optional]
-
-<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
-
-**BibTeX:**
-
-[More Information Needed]
-
-**APA:**
-
-[More Information Needed]
-
-## Glossary [optional]
-
-<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
-
-[More Information Needed]
-
-## More Information [optional]
-
-[More Information Needed]
-
-## Model Card Authors [optional]
-
-[More Information Needed]
-
-## Model Card Contact
-
-[More Information Needed]
-### Framework versions
-
-- PEFT 0.18.1
\ No newline at end of file
diff --git a/qwen3-8b-base/dp3_v2/epochs/epoch_001/adapter/adapter_config.json b/qwen3-8b-base/dp3_v2/epochs/epoch_001/adapter/adapter_config.json
deleted file mode 100644
index 7fce8872e904befaaa8e9691b3ef5f74b283e796..0000000000000000000000000000000000000000
--- a/qwen3-8b-base/dp3_v2/epochs/epoch_001/adapter/adapter_config.json
+++ /dev/null
@@ -1,46 +0,0 @@
-{
-  "alora_invocation_tokens": null,
-  "alpha_pattern": {},
-  "arrow_config": null,
-  "auto_mapping": null,
-  "base_model_name_or_path": "Qwen/Qwen3-8B-Base",
-  "bias": "none",
-  "corda_config": null,
-  "ensure_weight_tying": true,
-  "eva_config": null,
-  "exclude_modules": null,
-  "fan_in_fan_out": false,
-  "inference_mode": true,
-  "init_lora_weights": true,
-  "layer_replication": null,
-  "layers_pattern": null,
-  "layers_to_transform": null,
-  "loftq_config": {},
-  "lora_alpha": 32,
-  "lora_bias": false,
-  "lora_dropout": 0.05,
-  "megatron_config": null,
-  "megatron_core": "megatron.core",
-  "modules_to_save": [
-    "lm_head",
-    "embed_tokens"
-  ],
-  "peft_type": "LORA",
-  "peft_version": "0.18.1",
-  "qalora_group_size": 16,
-  "r": 16,
-  "rank_pattern": {},
-  "revision": null,
-  "target_modules": [
-    "q_proj",
-    "k_proj",
-    "v_proj",
-    "o_proj"
-  ],
-  "target_parameters": null,
-  "task_type": "CAUSAL_LM",
-  "trainable_token_indices": null,
-  "use_dora": false,
-  "use_qalora": false,
-  "use_rslora": false
-}
\ No newline at end of file
diff --git a/qwen3-8b-base/dp3_v2/epochs/epoch_001/audit_results.json b/qwen3-8b-base/dp3_v2/epochs/epoch_001/audit_results.json
deleted file mode 100644
index 048fa20f47b106b968feced59d04546bd08b685c..0000000000000000000000000000000000000000
--- a/qwen3-8b-base/dp3_v2/epochs/epoch_001/audit_results.json
+++ /dev/null
@@ -1,137 +0,0 @@
-{
-  "delta": 1e-05,
-  "num_canaries": 500,
-  "num_members": 250,
-  "paper_guess_fraction": 0.2,
-  "paper_guess_steps": 20,
-  "loss": {
-    "auc": 0.53972,
-    "empirical_epsilon": {
-      "0.05": 0.0,
-      "0.01": 0.0
-    },
-    "empirical_epsilon_details": {
-      "0.05": {
-        "epsilon": 0.0,
-        "num_guesses": 0,
-        "correct_guesses": 0,
-        "candidate_num_guesses": [
-          5,
-          10,
-          15,
-          20,
-          25,
-          30,
-          35,
-          40,
-          45,
-          50,
-          55,
-          60,
-          65,
-          70,
-          75,
-          80,
-          85,
-          90,
-          95,
-          100
-        ],
-        "direction": "lower"
-      },
-      "0.01": {
-        "epsilon": 0.0,
-        "num_guesses": 0,
-        "correct_guesses": 0,
-        "candidate_num_guesses": [
-          5,
-          10,
-          15,
-          20,
-          25,
-          30,
-          35,
-          40,
-          45,
-          50,
-          55,
-          60,
-          65,
-          70,
-          75,
-          80,
-          85,
-          90,
-          95,
-          100
-        ],
-        "direction": "lower"
-      }
-    }
-  },
-  "embedding": {
-    "auc": 0.502112,
-    "empirical_epsilon": {
-      "0.05": 0.0,
-      "0.01": 0.0
-    },
-    "empirical_epsilon_details": {
-      "0.05": {
-        "epsilon": 0.0,
-        "num_guesses": 0,
-        "correct_guesses": 0,
-        "candidate_num_guesses": [
-          5,
-          10,
-          15,
-          20,
-          25,
-          30,
-          35,
-          40,
-          45,
-          50,
-          55,
-          60,
-          65,
-          70,
-          75,
-          80,
-          85,
-          90,
-          95,
-          100
-        ],
-        "direction": "lower"
-      },
-      "0.01": {
-        "epsilon": 0.0,
-        "num_guesses": 0,
-        "correct_guesses": 0,
-        "candidate_num_guesses": [
-          5,
-          10,
-          15,
-          20,
-          25,
-          30,
-          35,
-          40,
-          45,
-          50,
-          55,
-          60,
-          65,
-          70,
-          75,
-          80,
-          85,
-          90,
-          95,
-          100
-        ],
-        "direction": "lower"
-      }
-    }
-  }
-}
\ No newline at end of file
diff --git a/qwen3-8b-base/dp3_v2/epochs/epoch_002/adapter/README.md b/qwen3-8b-base/dp3_v2/epochs/epoch_002/adapter/README.md
deleted file mode 100644
index 3063755d5f621219b97f455d66479201adfb07ca..0000000000000000000000000000000000000000
--- a/qwen3-8b-base/dp3_v2/epochs/epoch_002/adapter/README.md
+++ /dev/null
@@ -1,207 +0,0 @@
----
-base_model: Qwen/Qwen3-8B-Base
-library_name: peft
-pipeline_tag: text-generation
-tags:
-- base_model:adapter:Qwen/Qwen3-8B-Base
-- lora
-- transformers
----
-
-# Model Card for Model ID
-
-<!-- Provide a quick summary of what the model is/does. -->
-
-
-
-## Model Details
-
-### Model Description
-
-<!-- Provide a longer summary of what this model is. -->
-
-
-
-- **Developed by:** [More Information Needed]
-- **Funded by [optional]:** [More Information Needed]
-- **Shared by [optional]:** [More Information Needed]
-- **Model type:** [More Information Needed]
-- **Language(s) (NLP):** [More Information Needed]
-- **License:** [More Information Needed]
-- **Finetuned from model [optional]:** [More Information Needed]
-
-### Model Sources [optional]
-
-<!-- Provide the basic links for the model. -->
-
-- **Repository:** [More Information Needed]
-- **Paper [optional]:** [More Information Needed]
-- **Demo [optional]:** [More Information Needed]
-
-## Uses
-
-<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
-
-### Direct Use
-
-<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
-
-[More Information Needed]
-
-### Downstream Use [optional]
-
-<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
-
-[More Information Needed]
-
-### Out-of-Scope Use
-
-<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
-
-[More Information Needed]
-
-## Bias, Risks, and Limitations
-
-<!-- This section is meant to convey both technical and sociotechnical limitations. -->
-
-[More Information Needed]
-
-### Recommendations
-
-<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
-
-Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
-
-## How to Get Started with the Model
-
-Use the code below to get started with the model.
-
-[More Information Needed]
-
-## Training Details
-
-### Training Data
-
-<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
-
-[More Information Needed]
-
-### Training Procedure
-
-<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
-
-#### Preprocessing [optional]
-
-[More Information Needed]
-
-
-#### Training Hyperparameters
-
-- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
-
-#### Speeds, Sizes, Times [optional]
-
-<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
-
-[More Information Needed]
-
-## Evaluation
-
-<!-- This section describes the evaluation protocols and provides the results. -->
-
-### Testing Data, Factors & Metrics
-
-#### Testing Data
-
-<!-- This should link to a Dataset Card if possible. -->
-
-[More Information Needed]
-
-#### Factors
-
-<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
-
-[More Information Needed]
-
-#### Metrics
-
-<!-- These are the evaluation metrics being used, ideally with a description of why. -->
-
-[More Information Needed]
-
-### Results
-
-[More Information Needed]
-
-#### Summary
-
-
-
-## Model Examination [optional]
-
-<!-- Relevant interpretability work for the model goes here -->
-
-[More Information Needed]
-
-## Environmental Impact
-
-<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
-
-Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
-
-- **Hardware Type:** [More Information Needed]
-- **Hours used:** [More Information Needed]
-- **Cloud Provider:** [More Information Needed]
-- **Compute Region:** [More Information Needed]
-- **Carbon Emitted:** [More Information Needed]
-
-## Technical Specifications [optional]
-
-### Model Architecture and Objective
-
-[More Information Needed]
-
-### Compute Infrastructure
-
-[More Information Needed]
-
-#### Hardware
-
-[More Information Needed]
-
-#### Software
-
-[More Information Needed]
-
-## Citation [optional]
-
-<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
-
-**BibTeX:**
-
-[More Information Needed]
-
-**APA:**
-
-[More Information Needed]
-
-## Glossary [optional]
-
-<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
-
-[More Information Needed]
-
-## More Information [optional]
-
-[More Information Needed]
-
-## Model Card Authors [optional]
-
-[More Information Needed]
-
-## Model Card Contact
-
-[More Information Needed]
-### Framework versions
-
-- PEFT 0.18.1
\ No newline at end of file
diff --git a/qwen3-8b-base/dp3_v2/epochs/epoch_002/adapter/adapter_config.json b/qwen3-8b-base/dp3_v2/epochs/epoch_002/adapter/adapter_config.json
deleted file mode 100644
index 7fce8872e904befaaa8e9691b3ef5f74b283e796..0000000000000000000000000000000000000000
--- a/qwen3-8b-base/dp3_v2/epochs/epoch_002/adapter/adapter_config.json
+++ /dev/null
@@ -1,46 +0,0 @@
-{
-  "alora_invocation_tokens": null,
-  "alpha_pattern": {},
-  "arrow_config": null,
-  "auto_mapping": null,
-  "base_model_name_or_path": "Qwen/Qwen3-8B-Base",
-  "bias": "none",
-  "corda_config": null,
-  "ensure_weight_tying": true,
-  "eva_config": null,
-  "exclude_modules": null,
-  "fan_in_fan_out": false,
-  "inference_mode": true,
-  "init_lora_weights": true,
-  "layer_replication": null,
-  "layers_pattern": null,
-  "layers_to_transform": null,
-  "loftq_config": {},
-  "lora_alpha": 32,
-  "lora_bias": false,
-  "lora_dropout": 0.05,
-  "megatron_config": null,
-  "megatron_core": "megatron.core",
-  "modules_to_save": [
-    "lm_head",
-    "embed_tokens"
-  ],
-  "peft_type": "LORA",
-  "peft_version": "0.18.1",
-  "qalora_group_size": 16,
-  "r": 16,
-  "rank_pattern": {},
-  "revision": null,
-  "target_modules": [
-    "q_proj",
-    "k_proj",
-    "v_proj",
-    "o_proj"
-  ],
-  "target_parameters": null,
-  "task_type": "CAUSAL_LM",
-  "trainable_token_indices": null,
-  "use_dora": false,
-  "use_qalora": false,
-  "use_rslora": false
-}
\ No newline at end of file
diff --git a/qwen3-8b-base/dp3_v2/epochs/epoch_002/audit_results.json b/qwen3-8b-base/dp3_v2/epochs/epoch_002/audit_results.json
deleted file mode 100644
index 2a3d96758a732dddda42ac77677c42230481d961..0000000000000000000000000000000000000000
--- a/qwen3-8b-base/dp3_v2/epochs/epoch_002/audit_results.json
+++ /dev/null
@@ -1,137 +0,0 @@
-{
-  "delta": 1e-05,
-  "num_canaries": 500,
-  "num_members": 250,
-  "paper_guess_fraction": 0.2,
-  "paper_guess_steps": 20,
-  "loss": {
-    "auc": 0.527008,
-    "empirical_epsilon": {
-      "0.05": 0.03485510218888521,
-      "0.01": 0.0
-    },
-    "empirical_epsilon_details": {
-      "0.05": {
-        "epsilon": 0.03485510218888521,
-        "num_guesses": 70,
-        "correct_guesses": 43,
-        "candidate_num_guesses": [
-          5,
-          10,
-          15,
-          20,
-          25,
-          30,
-          35,
-          40,
-          45,
-          50,
-          55,
-          60,
-          65,
-          70,
-          75,
-          80,
-          85,
-          90,
-          95,
-          100
-        ],
-        "direction": "lower"
-      },
-      "0.01": {
-        "epsilon": 0.0,
-        "num_guesses": 0,
-        "correct_guesses": 0,
-        "candidate_num_guesses": [
-          5,
-          10,
-          15,
-          20,
-          25,
-          30,
-          35,
-          40,
-          45,
-          50,
-          55,
-          60,
-          65,
-          70,
-          75,
-          80,
-          85,
-          90,
-          95,
-          100
-        ],
-        "direction": "lower"
-      }
-    }
-  },
-  "embedding": {
-    "auc": 0.514304,
-    "empirical_epsilon": {
-      "0.05": 0.5080780601128936,
-      "0.01": 0.25933344289660454
-    },
-    "empirical_epsilon_details": {
-      "0.05": {
-        "epsilon": 0.5080780601128936,
-        "num_guesses": 35,
-        "correct_guesses": 27,
-        "candidate_num_guesses": [
-          5,
-          10,
-          15,
-          20,
-          25,
-          30,
-          35,
-          40,
-          45,
-          50,
-          55,
-          60,
-          65,
-          70,
-          75,
-          80,
-          85,
-          90,
-          95,
-          100
-        ],
-        "direction": "higher"
-      },
-      "0.01": {
-        "epsilon": 0.25933344289660454,
-        "num_guesses": 35,
-        "correct_guesses": 27,
-        "candidate_num_guesses": [
-          5,
-          10,
-          15,
-          20,
-          25,
-          30,
-          35,
-          40,
-          45,
-          50,
-          55,
-          60,
-          65,
-          70,
-          75,
-          80,
-          85,
-          90,
-          95,
-          100
-        ],
-        "direction": "higher"
-      }
-    }
-  }
-}
\ No newline at end of file
diff --git a/qwen3-8b-base/dp3_v2/epochs/epoch_003/adapter/README.md b/qwen3-8b-base/dp3_v2/epochs/epoch_003/adapter/README.md
deleted file mode 100644
index 3063755d5f621219b97f455d66479201adfb07ca..0000000000000000000000000000000000000000
--- a/qwen3-8b-base/dp3_v2/epochs/epoch_003/adapter/README.md
+++ /dev/null
@@ -1,207 +0,0 @@
----
-base_model: Qwen/Qwen3-8B-Base
-library_name: peft
-pipeline_tag: text-generation
-tags:
-- base_model:adapter:Qwen/Qwen3-8B-Base
-- lora
-- transformers
----
-
-# Model Card for Model ID
-
-<!-- Provide a quick summary of what the model is/does. -->
-
-
-
-## Model Details
-
-### Model Description
-
-<!-- Provide a longer summary of what this model is. -->
-
-
-
-- **Developed by:** [More Information Needed]
-- **Funded by [optional]:** [More Information Needed]
-- **Shared by [optional]:** [More Information Needed]
-- **Model type:** [More Information Needed]
-- **Language(s) (NLP):** [More Information Needed]
-- **License:** [More Information Needed]
-- **Finetuned from model [optional]:** [More Information Needed]
-
-### Model Sources [optional]
-
-<!-- Provide the basic links for the model. -->
-
-- **Repository:** [More Information Needed]
-- **Paper [optional]:** [More Information Needed]
-- **Demo [optional]:** [More Information Needed]
-
-## Uses
-
-<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
-
-### Direct Use
-
-<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
-
-[More Information Needed]
-
-### Downstream Use [optional]
-
-<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
-
-[More Information Needed]
-
-### Out-of-Scope Use
-
-<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
-
-[More Information Needed]
-
-## Bias, Risks, and Limitations
-
-<!-- This section is meant to convey both technical and sociotechnical limitations. -->
-
-[More Information Needed]
-
-### Recommendations
-
-<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
-
-Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
-
-## How to Get Started with the Model
-
-Use the code below to get started with the model.
-
-[More Information Needed]
-
-## Training Details
-
-### Training Data
-
-<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
-
-[More Information Needed]
-
-### Training Procedure
-
-<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
-
-#### Preprocessing [optional]
-
-[More Information Needed]
-
-
-#### Training Hyperparameters
-
-- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
-
-#### Speeds, Sizes, Times [optional]
-
-<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
-
-[More Information Needed]
-
-## Evaluation
-
-<!-- This section describes the evaluation protocols and provides the results. -->
-
-### Testing Data, Factors & Metrics
-
-#### Testing Data
-
-<!-- This should link to a Dataset Card if possible. -->
-
-[More Information Needed]
-
-#### Factors
-
-<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
-
-[More Information Needed]
-
-#### Metrics
-
-<!-- These are the evaluation metrics being used, ideally with a description of why. -->
-
-[More Information Needed]
-
-### Results
-
-[More Information Needed]
-
-#### Summary
-
-
-
-## Model Examination [optional]
-
-<!-- Relevant interpretability work for the model goes here -->
-
-[More Information Needed]
-
-## Environmental Impact
-
-<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
-
-Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
-
-- **Hardware Type:** [More Information Needed]
-- **Hours used:** [More Information Needed]
-- **Cloud Provider:** [More Information Needed]
-- **Compute Region:** [More Information Needed]
-- **Carbon Emitted:** [More Information Needed]
-
-## Technical Specifications [optional]
-
-### Model Architecture and Objective
-
-[More Information Needed]
-
-### Compute Infrastructure
-
-[More Information Needed]
-
-#### Hardware
-
-[More Information Needed]
-
-#### Software
-
-[More Information Needed]
-
-## Citation [optional]
-
-<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
-
-**BibTeX:**
-
-[More Information Needed]
-
-**APA:**
-
-[More Information Needed]
-
-## Glossary [optional]
-
-<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
-
-[More Information Needed]
-
-## More Information [optional]
-
-[More Information Needed]
-
-## Model Card Authors [optional]
-
-[More Information Needed]
-
-## Model Card Contact
-
-[More Information Needed]
-### Framework versions
-
-- PEFT 0.18.1
\ No newline at end of file
diff --git a/qwen3-8b-base/dp3_v2/epochs/epoch_003/adapter/adapter_config.json b/qwen3-8b-base/dp3_v2/epochs/epoch_003/adapter/adapter_config.json
deleted file mode 100644
index 7fce8872e904befaaa8e9691b3ef5f74b283e796..0000000000000000000000000000000000000000
--- a/qwen3-8b-base/dp3_v2/epochs/epoch_003/adapter/adapter_config.json
+++ /dev/null
@@ -1,46 +0,0 @@
-{
-  "alora_invocation_tokens": null,
-  "alpha_pattern": {},
-  "arrow_config": null,
-  "auto_mapping": null,
-  "base_model_name_or_path": "Qwen/Qwen3-8B-Base",
-  "bias": "none",
-  "corda_config": null,
-  "ensure_weight_tying": true,
-  "eva_config": null,
-  "exclude_modules": null,
-  "fan_in_fan_out": false,
-  "inference_mode": true,
-  "init_lora_weights": true,
-  "layer_replication": null,
-  "layers_pattern": null,
-  "layers_to_transform": null,
-  "loftq_config": {},
-  "lora_alpha": 32,
-  "lora_bias": false,
-  "lora_dropout": 0.05,
-  "megatron_config": null,
-  "megatron_core": "megatron.core",
-  "modules_to_save": [
-    "lm_head",
-    "embed_tokens"
-  ],
-  "peft_type": "LORA",
-  "peft_version": "0.18.1",
-  "qalora_group_size": 16,
-  "r": 16,
-  "rank_pattern": {},
-  "revision": null,
-  "target_modules": [
-    "q_proj",
-    "k_proj",
-    "v_proj",
-    "o_proj"
-  ],
-  "target_parameters": null,
-  "task_type": "CAUSAL_LM",
-  "trainable_token_indices": null,
-  "use_dora": false,
-  "use_qalora": false,
-  "use_rslora": false
-}
\ No newline at end of file
diff --git a/qwen3-8b-base/dp3_v2/epochs/epoch_003/audit_results.json b/qwen3-8b-base/dp3_v2/epochs/epoch_003/audit_results.json
deleted file mode 100644
index 9f595fb741ada20ca146339bc0ec6f87a5b7403c..0000000000000000000000000000000000000000
--- a/qwen3-8b-base/dp3_v2/epochs/epoch_003/audit_results.json
+++ /dev/null
@@ -1,137 +0,0 @@
-{
-  "delta": 1e-05,
-  "num_canaries": 500,
-  "num_members": 250,
-  "paper_guess_fraction": 0.2,
-  "paper_guess_steps": 20,
-  "loss": {
-    "auc": 0.519776,
-    "empirical_epsilon": {
-      "0.05": 0.41679731756448746,
-      "0.01": 0.0
-    },
-    "empirical_epsilon_details": {
-      "0.05": {
-        "epsilon": 0.41679731756448746,
-        "num_guesses": 10,
-        "correct_guesses": 9,
-        "candidate_num_guesses": [
-          5,
-          10,
-          15,
-          20,
-          25,
-          30,
-          35,
-          40,
-          45,
-          50,
-          55,
-          60,
-          65,
-          70,
-          75,
-          80,
-          85,
-          90,
-          95,
-          100
-        ],
-        "direction": "lower"
-      },
-      "0.01": {
-        "epsilon": 0.0,
-        "num_guesses": 0,
-        "correct_guesses": 0,
-        "candidate_num_guesses": [
-          5,
-          10,
-          15,
-          20,
-          25,
-          30,
-          35,
-          40,
-          45,
-          50,
-          55,
-          60,
-          65,
-          70,
-          75,
-          80,
-          85,
-          90,
-          95,
-          100
-        ],
-        "direction": "lower"
-      }
-    }
-  },
-  "embedding": {
-    "auc": 0.523704,
-    "empirical_epsilon": {
-      "0.05": 0.0,
-      "0.01": 0.0
-    },
-    "empirical_epsilon_details": {
-      "0.05": {
-        "epsilon": 0.0,
-        "num_guesses": 0,
-        "correct_guesses": 0,
-        "candidate_num_guesses": [
-          5,
-          10,
-          15,
-          20,
-          25,
-          30,
-          35,
-          40,
-          45,
-          50,
-          55,
-          60,
-          65,
-          70,
-          75,
-          80,
-          85,
-          90,
-          95,
-          100
-        ],
-        "direction": "lower"
-      },
-      "0.01": {
-        "epsilon": 0.0,
-        "num_guesses": 0,
-        "correct_guesses": 0,
-        "candidate_num_guesses": [
-          5,
-          10,
-          15,
-          20,
-          25,
-          30,
-          35,
-          40,
-          45,
-          50,
-          55,
-          60,
-          65,
-          70,
-          75,
-          80,
-          85,
-          90,
-          95,
-          100
-        ],
-        "direction": "lower"
-      }
-    }
-  }
-}
\ No newline at end of file
diff --git a/qwen3-8b-base/dp3_v2/epochs/epoch_004/adapter/README.md b/qwen3-8b-base/dp3_v2/epochs/epoch_004/adapter/README.md
deleted file mode 100644
index 3063755d5f621219b97f455d66479201adfb07ca..0000000000000000000000000000000000000000
--- a/qwen3-8b-base/dp3_v2/epochs/epoch_004/adapter/README.md
+++ /dev/null
@@ -1,207 +0,0 @@
----
-base_model: Qwen/Qwen3-8B-Base
-library_name: peft
-pipeline_tag: text-generation
-tags:
-- base_model:adapter:Qwen/Qwen3-8B-Base
-- lora
-- transformers
----
-
-# Model Card for Model ID
-
-<!-- Provide a quick summary of what the model is/does. -->
-
-
-
-## Model Details
-
-### Model Description
-
-<!-- Provide a longer summary of what this model is. -->
-
-
-
-- **Developed by:** [More Information Needed]
-- **Funded by [optional]:** [More Information Needed]
-- **Shared by [optional]:** [More Information Needed]
-- **Model type:** [More Information Needed]
-- **Language(s) (NLP):** [More Information Needed]
-- **License:** [More Information Needed]
-- **Finetuned from model [optional]:** [More Information Needed]
-
-### Model Sources [optional]
-
-<!-- Provide the basic links for the model. -->
-
-- **Repository:** [More Information Needed]
-- **Paper [optional]:** [More Information Needed]
-- **Demo [optional]:** [More Information Needed]
-
-## Uses
-
-<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
-
-### Direct Use
-
-<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
-
-[More Information Needed]
-
-### Downstream Use [optional]
-
-<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
-
-[More Information Needed]
-
-### Out-of-Scope Use
-
-<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
-
-[More Information Needed]
-
-## Bias, Risks, and Limitations
-
-<!-- This section is meant to convey both technical and sociotechnical limitations. -->
-
-[More Information Needed]
-
-### Recommendations
-
-<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
-
-Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
-
-## How to Get Started with the Model
-
-Use the code below to get started with the model.
-
-[More Information Needed]
-
-## Training Details
-
-### Training Data
-
-<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
-
-[More Information Needed]
-
-### Training Procedure
-
-<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
-
-#### Preprocessing [optional]
-
-[More Information Needed]
-
-
-#### Training Hyperparameters
-
-- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
-
-#### Speeds, Sizes, Times [optional]
-
-<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
-
-[More Information Needed]
-
-## Evaluation
-
-<!-- This section describes the evaluation protocols and provides the results. -->
-
-### Testing Data, Factors & Metrics
-
-#### Testing Data
-
-<!-- This should link to a Dataset Card if possible. -->
-
-[More Information Needed]
-
-#### Factors
-
-<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
-
-[More Information Needed]
-
-#### Metrics
-
-<!-- These are the evaluation metrics being used, ideally with a description of why. -->
-
-[More Information Needed]
-
-### Results
-
-[More Information Needed]
-
-#### Summary
-
-
-
-## Model Examination [optional]
-
-<!-- Relevant interpretability work for the model goes here -->
-
-[More Information Needed]
-
-## Environmental Impact
-
-<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
-
-Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
-
-- **Hardware Type:** [More Information Needed]
-- **Hours used:** [More Information Needed]
-- **Cloud Provider:** [More Information Needed]
-- **Compute Region:** [More Information Needed]
-- **Carbon Emitted:** [More Information Needed]
-
-## Technical Specifications [optional]
-
-### Model Architecture and Objective
-
-[More Information Needed]
-
-### Compute Infrastructure
-
-[More Information Needed]
-
-#### Hardware
-
-[More Information Needed]
-
-#### Software
-
-[More Information Needed]
-
-## Citation [optional]
-
-<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
-
-**BibTeX:**
-
-[More Information Needed]
-
-**APA:**
-
-[More Information Needed]
-
-## Glossary [optional]
-
-<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
-
-[More Information Needed]
-
-## More Information [optional]
-
-[More Information Needed]
-
-## Model Card Authors [optional]
-
-[More Information Needed]
-
-## Model Card Contact
-
-[More Information Needed]
-### Framework versions
-
-- PEFT 0.18.1
\ No newline at end of file
diff --git a/qwen3-8b-base/dp3_v2/epochs/epoch_004/adapter/adapter_config.json b/qwen3-8b-base/dp3_v2/epochs/epoch_004/adapter/adapter_config.json
deleted file mode 100644
index 7fce8872e904befaaa8e9691b3ef5f74b283e796..0000000000000000000000000000000000000000
--- a/qwen3-8b-base/dp3_v2/epochs/epoch_004/adapter/adapter_config.json
+++ /dev/null
@@ -1,46 +0,0 @@
-{
-  "alora_invocation_tokens": null,
-  "alpha_pattern": {},
-  "arrow_config": null,
-  "auto_mapping": null,
-  "base_model_name_or_path": "Qwen/Qwen3-8B-Base",
-  "bias": "none",
-  "corda_config": null,
-  "ensure_weight_tying": true,
-  "eva_config": null,
-  "exclude_modules": null,
-  "fan_in_fan_out": false,
-  "inference_mode": true,
-  "init_lora_weights": true,
-  "layer_replication": null,
-  "layers_pattern": null,
-  "layers_to_transform": null,
-  "loftq_config": {},
-  "lora_alpha": 32,
-  "lora_bias": false,
-  "lora_dropout": 0.05,
-  "megatron_config": null,
-  "megatron_core": "megatron.core",
-  "modules_to_save": [
-    "lm_head",
-    "embed_tokens"
-  ],
-  "peft_type": "LORA",
-  "peft_version": "0.18.1",
-  "qalora_group_size": 16,
-  "r": 16,
-  "rank_pattern": {},
-  "revision": null,
-  "target_modules": [
-    "q_proj",
-    "k_proj",
-    "v_proj",
-    "o_proj"
-  ],
-  "target_parameters": null,
-  "task_type": "CAUSAL_LM",
-  "trainable_token_indices": null,
-  "use_dora": false,
-  "use_qalora": false,
-  "use_rslora": false
-}
\ No newline at end of file
diff --git a/qwen3-8b-base/dp3_v2/epochs/epoch_004/audit_results.json b/qwen3-8b-base/dp3_v2/epochs/epoch_004/audit_results.json
deleted file mode 100644
index 41f1362cdba699973e42d76c2bf47c881b3a9e7f..0000000000000000000000000000000000000000
--- a/qwen3-8b-base/dp3_v2/epochs/epoch_004/audit_results.json
+++ /dev/null
@@ -1,137 +0,0 @@
-{
-  "delta": 1e-05,
-  "num_canaries": 500,
-  "num_members": 250,
-  "paper_guess_fraction": 0.2,
-  "paper_guess_steps": 20,
-  "loss": {
-    "auc": 0.511936,
-    "empirical_epsilon": {
-      "0.05": 0.41679731756448746,
-      "0.01": 0.01621033065021038
-    },
-    "empirical_epsilon_details": {
-      "0.05": {
-        "epsilon": 0.41679731756448746,
-        "num_guesses": 10,
-        "correct_guesses": 9,
-        "candidate_num_guesses": [
-          5,
-          10,
-          15,
-          20,
-          25,
-          30,
-          35,
-          40,
-          45,
-          50,
-          55,
-          60,
-          65,
-          70,
-          75,
-          80,
-          85,
-          90,
-          95,
-          100
-        ],
-        "direction": "lower"
-      },
-      "0.01": {
-        "epsilon": 0.01621033065021038,
-        "num_guesses": 30,
-        "correct_guesses": 22,
-        "candidate_num_guesses": [
-          5,
-          10,
-          15,
-          20,
-          25,
-          30,
-          35,
-          40,
-          45,
-          50,
-          55,
-          60,
-          65,
-          70,
-          75,
-          80,
-          85,
-          90,
-          95,
-          100
-        ],
-        "direction": "lower"
-      }
-    }
-  },
-  "embedding": {
-    "auc": 0.520328,
-    "empirical_epsilon": {
-      "0.05": 0.1552846673876047,
-      "0.01": 0.0
-    },
-    "empirical_epsilon_details": {
-      "0.05": {
-        "epsilon": 0.1552846673876047,
-        "num_guesses": 75,
-        "correct_guesses": 48,
-        "candidate_num_guesses": [
-          5,
-          10,
-          15,
-          20,
-          25,
-          30,
-          35,
-          40,
-          45,
-          50,
-          55,
-          60,
-          65,
-          70,
-          75,
-          80,
-          85,
-          90,
-          95,
-          100
-        ],
-        "direction": "higher"
-      },
-      "0.01": {
-        "epsilon": 0.0,
-        "num_guesses": 0,
-        "correct_guesses": 0,
-        "candidate_num_guesses": [
-          5,
-          10,
-          15,
-          20,
-          25,
-          30,
-          35,
-          40,
-          45,
-          50,
-          55,
-          60,
-          65,
-          70,
-          75,
-          80,
-          85,
-          90,
-          95,
-          100
-        ],
-        "direction": "lower"
-      }
-    }
-  }
-}
\ No newline at end of file
diff --git a/qwen3-8b-base/dp3_v2/epochs/epoch_005/adapter/README.md b/qwen3-8b-base/dp3_v2/epochs/epoch_005/adapter/README.md
deleted file mode 100644
index 3063755d5f621219b97f455d66479201adfb07ca..0000000000000000000000000000000000000000
--- a/qwen3-8b-base/dp3_v2/epochs/epoch_005/adapter/README.md
+++ /dev/null
@@ -1,207 +0,0 @@
----
-base_model: Qwen/Qwen3-8B-Base
-library_name: peft
-pipeline_tag: text-generation
-tags:
-- base_model:adapter:Qwen/Qwen3-8B-Base
-- lora
-- transformers
----
-
-# Model Card for Model ID
-
-<!-- Provide a quick summary of what the model is/does. -->
-
-
-
-## Model Details
-
-### Model Description
-
-<!-- Provide a longer summary of what this model is. -->
-
-
-
-- **Developed by:** [More Information Needed]
-- **Funded by [optional]:** [More Information Needed]
-- **Shared by [optional]:** [More Information Needed]
-- **Model type:** [More Information Needed]
-- **Language(s) (NLP):** [More Information Needed]
-- **License:** [More Information Needed]
-- **Finetuned from model [optional]:** [More Information Needed]
-
-### Model Sources [optional]
-
-<!-- Provide the basic links for the model. -->
-
-- **Repository:** [More Information Needed]
-- **Paper [optional]:** [More Information Needed]
-- **Demo [optional]:** [More Information Needed]
-
-## Uses
-
-<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
-
-### Direct Use
-
-<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
-
-[More Information Needed]
-
-### Downstream Use [optional]
-
-<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
-
-[More Information Needed]
-
-### Out-of-Scope Use
-
-<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
-
-[More Information Needed]
-
-## Bias, Risks, and Limitations
-
-<!-- This section is meant to convey both technical and sociotechnical limitations. -->
-
-[More Information Needed]
-
-### Recommendations
-
-<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
-
-Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
-
-## How to Get Started with the Model
-
-Use the code below to get started with the model.
-
-[More Information Needed]
-
-## Training Details
-
-### Training Data
-
-<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
-
-[More Information Needed]
-
-### Training Procedure
-
-<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
-
-#### Preprocessing [optional]
-
-[More Information Needed]
-
-
-#### Training Hyperparameters
-
-- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
-
-#### Speeds, Sizes, Times [optional]
-
-<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
-
-[More Information Needed]
-
-## Evaluation
-
-<!-- This section describes the evaluation protocols and provides the results. -->
-
-### Testing Data, Factors & Metrics
-
-#### Testing Data
-
-<!-- This should link to a Dataset Card if possible. -->
-
-[More Information Needed]
-
-#### Factors
-
-<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
-
-[More Information Needed]
-
-#### Metrics
-
-<!-- These are the evaluation metrics being used, ideally with a description of why. -->
-
-[More Information Needed]
-
-### Results
-
-[More Information Needed]
-
-#### Summary
-
-
-
-## Model Examination [optional]
-
-<!-- Relevant interpretability work for the model goes here -->
-
-[More Information Needed]
-
-## Environmental Impact
-
-<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
-
-Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
-
-- **Hardware Type:** [More Information Needed]
-- **Hours used:** [More Information Needed]
-- **Cloud Provider:** [More Information Needed]
-- **Compute Region:** [More Information Needed]
-- **Carbon Emitted:** [More Information Needed]
-
-## Technical Specifications [optional]
-
-### Model Architecture and Objective
-
-[More Information Needed]
-
-### Compute Infrastructure
-
-[More Information Needed]
-
-#### Hardware
-
-[More Information Needed]
-
-#### Software
-
-[More Information Needed]
-
-## Citation [optional]
-
-<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
-
-**BibTeX:**
-
-[More Information Needed]
-
-**APA:**
-
-[More Information Needed]
-
-## Glossary [optional]
-
-<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
-
-[More Information Needed]
-
-## More Information [optional]
-
-[More Information Needed]
-
-## Model Card Authors [optional]
-
-[More Information Needed]
-
-## Model Card Contact
-
-[More Information Needed]
-### Framework versions
-
-- PEFT 0.18.1
\ No newline at end of file
diff --git a/qwen3-8b-base/dp3_v2/epochs/epoch_005/adapter/adapter_config.json b/qwen3-8b-base/dp3_v2/epochs/epoch_005/adapter/adapter_config.json
deleted file mode 100644
index 7fce8872e904befaaa8e9691b3ef5f74b283e796..0000000000000000000000000000000000000000
--- a/qwen3-8b-base/dp3_v2/epochs/epoch_005/adapter/adapter_config.json
+++ /dev/null
@@ -1,46 +0,0 @@
-{
-  "alora_invocation_tokens": null,
-  "alpha_pattern": {},
-  "arrow_config": null,
-  "auto_mapping": null,
-  "base_model_name_or_path": "Qwen/Qwen3-8B-Base",
-  "bias": "none",
-  "corda_config": null,
-  "ensure_weight_tying": true,
-  "eva_config": null,
-  "exclude_modules": null,
-  "fan_in_fan_out": false,
-  "inference_mode": true,
-  "init_lora_weights": true,
-  "layer_replication": null,
-  "layers_pattern": null,
-  "layers_to_transform": null,
-  "loftq_config": {},
-  "lora_alpha": 32,
-  "lora_bias": false,
-  "lora_dropout": 0.05,
-  "megatron_config": null,
-  "megatron_core": "megatron.core",
-  "modules_to_save": [
-    "lm_head",
-    "embed_tokens"
-  ],
-  "peft_type": "LORA",
-  "peft_version": "0.18.1",
-  "qalora_group_size": 16,
-  "r": 16,
-  "rank_pattern": {},
-  "revision": null,
-  "target_modules": [
-    "q_proj",
-    "k_proj",
-    "v_proj",
-    "o_proj"
-  ],
-  "target_parameters": null,
-  "task_type": "CAUSAL_LM",
-  "trainable_token_indices": null,
-  "use_dora": false,
-  "use_qalora": false,
-  "use_rslora": false
-}
\ No newline at end of file
diff --git a/qwen3-8b-base/dp3_v2/epochs/epoch_005/audit_results.json b/qwen3-8b-base/dp3_v2/epochs/epoch_005/audit_results.json
deleted file mode 100644
index 1cc38257d021765b1bff788ea40131e43c1c3a73..0000000000000000000000000000000000000000
--- a/qwen3-8b-base/dp3_v2/epochs/epoch_005/audit_results.json
+++ /dev/null
@@ -1,137 +0,0 @@
-{
-  "delta": 1e-05,
-  "num_canaries": 500,
-  "num_members": 250,
-  "paper_guess_fraction": 0.2,
-  "paper_guess_steps": 20,
-  "loss": {
-    "auc": 0.508168,
-    "empirical_epsilon": {
-      "0.05": 0.23387643229216337,
-      "0.01": 0.0
-    },
-    "empirical_epsilon_details": {
-      "0.05": {
-        "epsilon": 0.23387643229216337,
-        "num_guesses": 15,
-        "correct_guesses": 12,
-        "candidate_num_guesses": [
-          5,
-          10,
-          15,
-          20,
-          25,
-          30,
-          35,
-          40,
-          45,
-          50,
-          55,
-          60,
-          65,
-          70,
-          75,
-          80,
-          85,
-          90,
-          95,
-          100
-        ],
-        "direction": "lower"
-      },
-      "0.01": {
-        "epsilon": 0.0,
-        "num_guesses": 0,
-        "correct_guesses": 0,
-        "candidate_num_guesses": [
-          5,
-          10,
-          15,
-          20,
-          25,
-          30,
-          35,
-          40,
-          45,
-          50,
-          55,
-          60,
-          65,
-          70,
-          75,
-          80,
-          85,
-          90,
-          95,
-          100
-        ],
-        "direction": "lower"
-      }
-    }
-  },
-  "embedding": {
-    "auc": 0.518896,
-    "empirical_epsilon": {
-      "0.05": 0.09125839080661535,
-      "0.01": 0.0
-    },
-    "empirical_epsilon_details": {
-      "0.05": {
-        "epsilon": 0.09125839080661535,
-        "num_guesses": 100,
-        "correct_guesses": 61,
-        "candidate_num_guesses": [
-          5,
-          10,
-          15,
-          20,
-          25,
-          30,
-          35,
-          40,
-          45,
-          50,
-          55,
-          60,
-          65,
-          70,
-          75,
-          80,
-          85,
-          90,
-          95,
-          100
-        ],
-        "direction": "higher"
-      },
-      "0.01": {
-        "epsilon": 0.0,
-        "num_guesses": 0,
-        "correct_guesses": 0,
-        "candidate_num_guesses": [
-          5,
-          10,
-          15,
-          20,
-          25,
-          30,
-          35,
-          40,
-          45,
-          50,
-          55,
-          60,
-          65,
-          70,
-          75,
-          80,
-          85,
-          90,
-          95,
-          100
-        ],
-        "direction": "lower"
-      }
-    }
-  }
-}
\ No newline at end of file
diff --git a/qwen3-8b-base/dp8/epochs/epoch_001/adapter/README.md b/qwen3-8b-base/dp8/epochs/epoch_001/adapter/README.md
deleted file mode 100644
index 3063755d5f621219b97f455d66479201adfb07ca..0000000000000000000000000000000000000000
--- a/qwen3-8b-base/dp8/epochs/epoch_001/adapter/README.md
+++ /dev/null
@@ -1,207 +0,0 @@
----
-base_model: Qwen/Qwen3-8B-Base
-library_name: peft
-pipeline_tag: text-generation
-tags:
-- base_model:adapter:Qwen/Qwen3-8B-Base
-- lora
-- transformers
----
-
-# Model Card for Model ID
-
-<!-- Provide a quick summary of what the model is/does. -->
-
-
-
-## Model Details
-
-### Model Description
-
-<!-- Provide a longer summary of what this model is. -->
-
-
-
-- **Developed by:** [More Information Needed]
-- **Funded by [optional]:** [More Information Needed]
-- **Shared by [optional]:** [More Information Needed]
-- **Model type:** [More Information Needed]
-- **Language(s) (NLP):** [More Information Needed]
-- **License:** [More Information Needed]
-- **Finetuned from model [optional]:** [More Information Needed]
-
-### Model Sources [optional]
-
-<!-- Provide the basic links for the model. -->
-
-- **Repository:** [More Information Needed]
-- **Paper [optional]:** [More Information Needed]
-- **Demo [optional]:** [More Information Needed]
-
-## Uses
-
-<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
-
-### Direct Use
-
-<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
-
-[More Information Needed]
-
-### Downstream Use [optional]
-
-<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
-
-[More Information Needed]
-
-### Out-of-Scope Use
-
-<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
-
-[More Information Needed]
-
-## Bias, Risks, and Limitations
-
-<!-- This section is meant to convey both technical and sociotechnical limitations. -->
-
-[More Information Needed]
-
-### Recommendations
-
-<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
-
-Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
-
-## How to Get Started with the Model
-
-Use the code below to get started with the model.
-
-[More Information Needed]
-
-## Training Details
-
-### Training Data
-
-<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
-
-[More Information Needed]
-
-### Training Procedure
-
-<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
-
-#### Preprocessing [optional]
-
-[More Information Needed]
-
-
-#### Training Hyperparameters
-
-- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
-
-#### Speeds, Sizes, Times [optional]
-
-<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
-
-[More Information Needed]
-
-## Evaluation
-
-<!-- This section describes the evaluation protocols and provides the results. -->
-
-### Testing Data, Factors & Metrics
-
-#### Testing Data
-
-<!-- This should link to a Dataset Card if possible. -->
-
-[More Information Needed]
-
-#### Factors
-
-<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
-
-[More Information Needed]
-
-#### Metrics
-
-<!-- These are the evaluation metrics being used, ideally with a description of why. -->
-
-[More Information Needed]
-
-### Results
-
-[More Information Needed]
-
-#### Summary
-
-
-
-## Model Examination [optional]
-
-<!-- Relevant interpretability work for the model goes here -->
-
-[More Information Needed]
-
-## Environmental Impact
-
-<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
-
-Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
-
-- **Hardware Type:** [More Information Needed]
-- **Hours used:** [More Information Needed]
-- **Cloud Provider:** [More Information Needed]
-- **Compute Region:** [More Information Needed]
-- **Carbon Emitted:** [More Information Needed]
-
-## Technical Specifications [optional]
-
-### Model Architecture and Objective
-
-[More Information Needed]
-
-### Compute Infrastructure
-
-[More Information Needed]
-
-#### Hardware
-
-[More Information Needed]
-
-#### Software
-
-[More Information Needed]
-
-## Citation [optional]
-
-<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
-
-**BibTeX:**
-
-[More Information Needed]
-
-**APA:**
-
-[More Information Needed]
-
-## Glossary [optional]
-
-<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
-
-[More Information Needed]
-
-## More Information [optional]
-
-[More Information Needed]
-
-## Model Card Authors [optional]
-
-[More Information Needed]
-
-## Model Card Contact
-
-[More Information Needed]
-### Framework versions
-
-- PEFT 0.18.1
\ No newline at end of file
diff --git a/qwen3-8b-base/dp8/epochs/epoch_001/adapter/adapter_config.json b/qwen3-8b-base/dp8/epochs/epoch_001/adapter/adapter_config.json
deleted file mode 100644
index 1bedcb13b14df08d2a6ab685b3aab1cb6bba6e58..0000000000000000000000000000000000000000
--- a/qwen3-8b-base/dp8/epochs/epoch_001/adapter/adapter_config.json
+++ /dev/null
@@ -1,46 +0,0 @@
-{
-  "alora_invocation_tokens": null,
-  "alpha_pattern": {},
-  "arrow_config": null,
-  "auto_mapping": null,
-  "base_model_name_or_path": "Qwen/Qwen3-8B-Base",
-  "bias": "none",
-  "corda_config": null,
-  "ensure_weight_tying": true,
-  "eva_config": null,
-  "exclude_modules": null,
-  "fan_in_fan_out": false,
-  "inference_mode": true,
-  "init_lora_weights": true,
-  "layer_replication": null,
-  "layers_pattern": null,
-  "layers_to_transform": null,
-  "loftq_config": {},
-  "lora_alpha": 32,
-  "lora_bias": false,
-  "lora_dropout": 0.05,
-  "megatron_config": null,
-  "megatron_core": "megatron.core",
-  "modules_to_save": [
-    "lm_head",
-    "embed_tokens"
-  ],
-  "peft_type": "LORA",
-  "peft_version": "0.18.1",
-  "qalora_group_size": 16,
-  "r": 16,
-  "rank_pattern": {},
-  "revision": null,
-  "target_modules": [
-    "q_proj",
-    "v_proj",
-    "k_proj",
-    "o_proj"
-  ],
-  "target_parameters": null,
-  "task_type": "CAUSAL_LM",
-  "trainable_token_indices": null,
-  "use_dora": false,
-  "use_qalora": false,
-  "use_rslora": false
-}
\ No newline at end of file
diff --git a/qwen3-8b-base/dp8/epochs/epoch_001/audit_results.json b/qwen3-8b-base/dp8/epochs/epoch_001/audit_results.json
deleted file mode 100644
index 3b2ebc30c6de7d4303da0c750aa216ad444e55a9..0000000000000000000000000000000000000000
--- a/qwen3-8b-base/dp8/epochs/epoch_001/audit_results.json
+++ /dev/null
@@ -1,137 +0,0 @@
-{
-  "delta": 1e-05,
-  "num_canaries": 500,
-  "num_members": 250,
-  "paper_guess_fraction": 0.2,
-  "paper_guess_steps": 20,
-  "loss": {
-    "auc": 0.558464,
-    "empirical_epsilon": {
-      "0.05": 0.0,
-      "0.01": 0.0
-    },
-    "empirical_epsilon_details": {
-      "0.05": {
-        "epsilon": 0.0,
-        "num_guesses": 0,
-        "correct_guesses": 0,
-        "candidate_num_guesses": [
-          5,
-          10,
-          15,
-          20,
-          25,
-          30,
-          35,
-          40,
-          45,
-          50,
-          55,
-          60,
-          65,
-          70,
-          75,
-          80,
-          85,
-          90,
-          95,
-          100
-        ],
-        "direction": "lower"
-      },
-      "0.01": {
-        "epsilon": 0.0,
-        "num_guesses": 0,
-        "correct_guesses": 0,
-        "candidate_num_guesses": [
-          5,
-          10,
-          15,
-          20,
-          25,
-          30,
-          35,
-          40,
-          45,
-          50,
-          55,
-          60,
-          65,
-          70,
-          75,
-          80,
-          85,
-          90,
-          95,
-          100
-        ],
-        "direction": "lower"
-      }
-    }
-  },
-  "embedding": {
-    "auc": 0.502584,
-    "empirical_epsilon": {
-      "0.05": 0.0,
-      "0.01": 0.0
-    },
-    "empirical_epsilon_details": {
-      "0.05": {
-        "epsilon": 0.0,
-        "num_guesses": 0,
-        "correct_guesses": 0,
-        "candidate_num_guesses": [
-          5,
-          10,
-          15,
-          20,
-          25,
-          30,
-          35,
-          40,
-          45,
-          50,
-          55,
-          60,
-          65,
-          70,
-          75,
-          80,
-          85,
-          90,
-          95,
-          100
-        ],
-        "direction": "lower"
-      },
-      "0.01": {
-        "epsilon": 0.0,
-        "num_guesses": 0,
-        "correct_guesses": 0,
-        "candidate_num_guesses": [
-          5,
-          10,
-          15,
-          20,
-          25,
-          30,
-          35,
-          40,
-          45,
-          50,
-          55,
-          60,
-          65,
-          70,
-          75,
-          80,
-          85,
-          90,
-          95,
-          100
-        ],
-        "direction": "lower"
-      }
-    }
-  }
-}
\ No newline at end of file
diff --git a/qwen3-8b-base/dp8/epochs/epoch_002/adapter/README.md b/qwen3-8b-base/dp8/epochs/epoch_002/adapter/README.md
deleted file mode 100644
index 3063755d5f621219b97f455d66479201adfb07ca..0000000000000000000000000000000000000000
--- a/qwen3-8b-base/dp8/epochs/epoch_002/adapter/README.md
+++ /dev/null
@@ -1,207 +0,0 @@
----
-base_model: Qwen/Qwen3-8B-Base
-library_name: peft
-pipeline_tag: text-generation
-tags:
-- base_model:adapter:Qwen/Qwen3-8B-Base
-- lora
-- transformers
----
-
-# Model Card for Model ID
-
-<!-- Provide a quick summary of what the model is/does. -->
-
-
-
-## Model Details
-
-### Model Description
-
-<!-- Provide a longer summary of what this model is. -->
-
-
-
-- **Developed by:** [More Information Needed]
-- **Funded by [optional]:** [More Information Needed]
-- **Shared by [optional]:** [More Information Needed]
-- **Model type:** [More Information Needed]
-- **Language(s) (NLP):** [More Information Needed]
-- **License:** [More Information Needed]
-- **Finetuned from model [optional]:** [More Information Needed]
-
-### Model Sources [optional]
-
-<!-- Provide the basic links for the model. -->
-
-- **Repository:** [More Information Needed]
-- **Paper [optional]:** [More Information Needed]
-- **Demo [optional]:** [More Information Needed]
-
-## Uses
-
-<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
-
-### Direct Use
-
-<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
-
-[More Information Needed]
-
-### Downstream Use [optional]
-
-<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
-
-[More Information Needed]
-
-### Out-of-Scope Use
-
-<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
-
-[More Information Needed]
-
-## Bias, Risks, and Limitations
-
-<!-- This section is meant to convey both technical and sociotechnical limitations. -->
-
-[More Information Needed]
-
-### Recommendations
-
-<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
-
-Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
-
-## How to Get Started with the Model
-
-Use the code below to get started with the model.
-
-[More Information Needed]
-
-## Training Details
-
-### Training Data
-
-<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
-
-[More Information Needed]
-
-### Training Procedure
-
-<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
-
-#### Preprocessing [optional]
-
-[More Information Needed]
-
-
-#### Training Hyperparameters
-
-- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
-
-#### Speeds, Sizes, Times [optional]
-
-<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
-
-[More Information Needed]
-
-## Evaluation
-
-<!-- This section describes the evaluation protocols and provides the results. -->
-
-### Testing Data, Factors & Metrics
-
-#### Testing Data
-
-<!-- This should link to a Dataset Card if possible. -->
-
-[More Information Needed]
-
-#### Factors
-
-<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
-
-[More Information Needed]
-
-#### Metrics
-
-<!-- These are the evaluation metrics being used, ideally with a description of why. -->
-
-[More Information Needed]
-
-### Results
-
-[More Information Needed]
-
-#### Summary
-
-
-
-## Model Examination [optional]
-
-<!-- Relevant interpretability work for the model goes here -->
-
-[More Information Needed]
-
-## Environmental Impact
-
-<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
-
-Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
-
-- **Hardware Type:** [More Information Needed]
-- **Hours used:** [More Information Needed]
-- **Cloud Provider:** [More Information Needed]
-- **Compute Region:** [More Information Needed]
-- **Carbon Emitted:** [More Information Needed]
-
-## Technical Specifications [optional]
-
-### Model Architecture and Objective
-
-[More Information Needed]
-
-### Compute Infrastructure
-
-[More Information Needed]
-
-#### Hardware
-
-[More Information Needed]
-
-#### Software
-
-[More Information Needed]
-
-## Citation [optional]
-
-<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
-
-**BibTeX:**
-
-[More Information Needed]
-
-**APA:**
-
-[More Information Needed]
-
-## Glossary [optional]
-
-<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
-
-[More Information Needed]
-
-## More Information [optional]
-
-[More Information Needed]
-
-## Model Card Authors [optional]
-
-[More Information Needed]
-
-## Model Card Contact
-
-[More Information Needed]
-### Framework versions
-
-- PEFT 0.18.1
\ No newline at end of file
diff --git a/qwen3-8b-base/dp8/epochs/epoch_002/adapter/adapter_config.json b/qwen3-8b-base/dp8/epochs/epoch_002/adapter/adapter_config.json
deleted file mode 100644
index 1bedcb13b14df08d2a6ab685b3aab1cb6bba6e58..0000000000000000000000000000000000000000
--- a/qwen3-8b-base/dp8/epochs/epoch_002/adapter/adapter_config.json
+++ /dev/null
@@ -1,46 +0,0 @@
-{
-  "alora_invocation_tokens": null,
-  "alpha_pattern": {},
-  "arrow_config": null,
-  "auto_mapping": null,
-  "base_model_name_or_path": "Qwen/Qwen3-8B-Base",
-  "bias": "none",
-  "corda_config": null,
-  "ensure_weight_tying": true,
-  "eva_config": null,
-  "exclude_modules": null,
-  "fan_in_fan_out": false,
-  "inference_mode": true,
-  "init_lora_weights": true,
-  "layer_replication": null,
-  "layers_pattern": null,
-  "layers_to_transform": null,
-  "loftq_config": {},
-  "lora_alpha": 32,
-  "lora_bias": false,
-  "lora_dropout": 0.05,
-  "megatron_config": null,
-  "megatron_core": "megatron.core",
-  "modules_to_save": [
-    "lm_head",
-    "embed_tokens"
-  ],
-  "peft_type": "LORA",
-  "peft_version": "0.18.1",
-  "qalora_group_size": 16,
-  "r": 16,
-  "rank_pattern": {},
-  "revision": null,
-  "target_modules": [
-    "q_proj",
-    "v_proj",
-    "k_proj",
-    "o_proj"
-  ],
-  "target_parameters": null,
-  "task_type": "CAUSAL_LM",
-  "trainable_token_indices": null,
-  "use_dora": false,
-  "use_qalora": false,
-  "use_rslora": false
-}
\ No newline at end of file
diff --git a/qwen3-8b-base/dp8/epochs/epoch_002/audit_results.json b/qwen3-8b-base/dp8/epochs/epoch_002/audit_results.json
deleted file mode 100644
index 220844846c5ff613342ba8da7bb91f9977afcd8b..0000000000000000000000000000000000000000
--- a/qwen3-8b-base/dp8/epochs/epoch_002/audit_results.json
+++ /dev/null
@@ -1,137 +0,0 @@
-{
-  "delta": 1e-05,
-  "num_canaries": 500,
-  "num_members": 250,
-  "paper_guess_fraction": 0.2,
-  "paper_guess_steps": 20,
-  "loss": {
-    "auc": 0.531888,
-    "empirical_epsilon": {
-      "0.05": 0.14663786999881268,
-      "0.01": 0.0
-    },
-    "empirical_epsilon_details": {
-      "0.05": {
-        "epsilon": 0.14663786999881268,
-        "num_guesses": 25,
-        "correct_guesses": 18,
-        "candidate_num_guesses": [
-          5,
-          10,
-          15,
-          20,
-          25,
-          30,
-          35,
-          40,
-          45,
-          50,
-          55,
-          60,
-          65,
-          70,
-          75,
-          80,
-          85,
-          90,
-          95,
-          100
-        ],
-        "direction": "lower"
-      },
-      "0.01": {
-        "epsilon": 0.0,
-        "num_guesses": 0,
-        "correct_guesses": 0,
-        "candidate_num_guesses": [
-          5,
-          10,
-          15,
-          20,
-          25,
-          30,
-          35,
-          40,
-          45,
-          50,
-          55,
-          60,
-          65,
-          70,
-          75,
-          80,
-          85,
-          90,
-          95,
-          100
-        ],
-        "direction": "lower"
-      }
-    }
-  },
-  "embedding": {
-    "auc": 0.513816,
-    "empirical_epsilon": {
-      "0.05": 0.14663786999881268,
-      "0.01": 0.0
-    },
-    "empirical_epsilon_details": {
-      "0.05": {
-        "epsilon": 0.14663786999881268,
-        "num_guesses": 25,
-        "correct_guesses": 18,
-        "candidate_num_guesses": [
-          5,
-          10,
-          15,
-          20,
-          25,
-          30,
-          35,
-          40,
-          45,
-          50,
-          55,
-          60,
-          65,
-          70,
-          75,
-          80,
-          85,
-          90,
-          95,
-          100
-        ],
-        "direction": "higher"
-      },
-      "0.01": {
-        "epsilon": 0.0,
-        "num_guesses": 0,
-        "correct_guesses": 0,
-        "candidate_num_guesses": [
-          5,
-          10,
-          15,
-          20,
-          25,
-          30,
-          35,
-          40,
-          45,
-          50,
-          55,
-          60,
-          65,
-          70,
-          75,
-          80,
-          85,
-          90,
-          95,
-          100
-        ],
-        "direction": "lower"
-      }
-    }
-  }
-}
\ No newline at end of file
diff --git a/qwen3-8b-base/dp8/epochs/epoch_003/adapter/README.md b/qwen3-8b-base/dp8/epochs/epoch_003/adapter/README.md
deleted file mode 100644
index 3063755d5f621219b97f455d66479201adfb07ca..0000000000000000000000000000000000000000
--- a/qwen3-8b-base/dp8/epochs/epoch_003/adapter/README.md
+++ /dev/null
@@ -1,207 +0,0 @@
----
-base_model: Qwen/Qwen3-8B-Base
-library_name: peft
-pipeline_tag: text-generation
-tags:
-- base_model:adapter:Qwen/Qwen3-8B-Base
-- lora
-- transformers
----
-
-# Model Card for Model ID
-
-<!-- Provide a quick summary of what the model is/does. -->
-
-
-
-## Model Details
-
-### Model Description
-
-<!-- Provide a longer summary of what this model is. -->
-
-
-
-- **Developed by:** [More Information Needed]
-- **Funded by [optional]:** [More Information Needed]
-- **Shared by [optional]:** [More Information Needed]
-- **Model type:** [More Information Needed]
-- **Language(s) (NLP):** [More Information Needed]
-- **License:** [More Information Needed]
-- **Finetuned from model [optional]:** [More Information Needed]
-
-### Model Sources [optional]
-
-<!-- Provide the basic links for the model. -->
-
-- **Repository:** [More Information Needed]
-- **Paper [optional]:** [More Information Needed]
-- **Demo [optional]:** [More Information Needed]
-
-## Uses
-
-<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
-
-### Direct Use
-
-<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
-
-[More Information Needed]
-
-### Downstream Use [optional]
-
-<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
-
-[More Information Needed]
-
-### Out-of-Scope Use
-
-<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
-
-[More Information Needed]
-
-## Bias, Risks, and Limitations
-
-<!-- This section is meant to convey both technical and sociotechnical limitations. -->
-
-[More Information Needed]
-
-### Recommendations
-
-<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
-
-Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
-
-## How to Get Started with the Model
-
-Use the code below to get started with the model.
-
-[More Information Needed]
-
-## Training Details
-
-### Training Data
-
-<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
-
-[More Information Needed]
-
-### Training Procedure
-
-<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
-
-#### Preprocessing [optional]
-
-[More Information Needed]
-
-
-#### Training Hyperparameters
-
-- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
-
-#### Speeds, Sizes, Times [optional]
-
-<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
-
-[More Information Needed]
-
-## Evaluation
-
-<!-- This section describes the evaluation protocols and provides the results. -->
-
-### Testing Data, Factors & Metrics
-
-#### Testing Data
-
-<!-- This should link to a Dataset Card if possible. -->
-
-[More Information Needed]
-
-#### Factors
-
-<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
-
-[More Information Needed]
-
-#### Metrics
-
-<!-- These are the evaluation metrics being used, ideally with a description of why. -->
-
-[More Information Needed]
-
-### Results
-
-[More Information Needed]
-
-#### Summary
-
-
-
-## Model Examination [optional]
-
-<!-- Relevant interpretability work for the model goes here -->
-
-[More Information Needed]
-
-## Environmental Impact
-
-<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
-
-Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
-
-- **Hardware Type:** [More Information Needed]
-- **Hours used:** [More Information Needed]
-- **Cloud Provider:** [More Information Needed]
-- **Compute Region:** [More Information Needed]
-- **Carbon Emitted:** [More Information Needed]
-
-## Technical Specifications [optional]
-
-### Model Architecture and Objective
-
-[More Information Needed]
-
-### Compute Infrastructure
-
-[More Information Needed]
-
-#### Hardware
-
-[More Information Needed]
-
-#### Software
-
-[More Information Needed]
-
-## Citation [optional]
-
-<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
-
-**BibTeX:**
-
-[More Information Needed]
-
-**APA:**
-
-[More Information Needed]
-
-## Glossary [optional]
-
-<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
-
-[More Information Needed]
-
-## More Information [optional]
-
-[More Information Needed]
-
-## Model Card Authors [optional]
-
-[More Information Needed]
-
-## Model Card Contact
-
-[More Information Needed]
-### Framework versions
-
-- PEFT 0.18.1
\ No newline at end of file
diff --git a/qwen3-8b-base/dp8/epochs/epoch_003/adapter/adapter_config.json b/qwen3-8b-base/dp8/epochs/epoch_003/adapter/adapter_config.json
deleted file mode 100644
index 1bedcb13b14df08d2a6ab685b3aab1cb6bba6e58..0000000000000000000000000000000000000000
--- a/qwen3-8b-base/dp8/epochs/epoch_003/adapter/adapter_config.json
+++ /dev/null
@@ -1,46 +0,0 @@
-{
-  "alora_invocation_tokens": null,
-  "alpha_pattern": {},
-  "arrow_config": null,
-  "auto_mapping": null,
-  "base_model_name_or_path": "Qwen/Qwen3-8B-Base",
-  "bias": "none",
-  "corda_config": null,
-  "ensure_weight_tying": true,
-  "eva_config": null,
-  "exclude_modules": null,
-  "fan_in_fan_out": false,
-  "inference_mode": true,
-  "init_lora_weights": true,
-  "layer_replication": null,
-  "layers_pattern": null,
-  "layers_to_transform": null,
-  "loftq_config": {},
-  "lora_alpha": 32,
-  "lora_bias": false,
-  "lora_dropout": 0.05,
-  "megatron_config": null,
-  "megatron_core": "megatron.core",
-  "modules_to_save": [
-    "lm_head",
-    "embed_tokens"
-  ],
-  "peft_type": "LORA",
-  "peft_version": "0.18.1",
-  "qalora_group_size": 16,
-  "r": 16,
-  "rank_pattern": {},
-  "revision": null,
-  "target_modules": [
-    "q_proj",
-    "v_proj",
-    "k_proj",
-    "o_proj"
-  ],
-  "target_parameters": null,
-  "task_type": "CAUSAL_LM",
-  "trainable_token_indices": null,
-  "use_dora": false,
-  "use_qalora": false,
-  "use_rslora": false
-}
\ No newline at end of file
diff --git a/qwen3-8b-base/dp8/epochs/epoch_003/audit_results.json b/qwen3-8b-base/dp8/epochs/epoch_003/audit_results.json
deleted file mode 100644
index 9c2e1e38f15f98fed8a9fec30f9152f9501b4cd8..0000000000000000000000000000000000000000
--- a/qwen3-8b-base/dp8/epochs/epoch_003/audit_results.json
+++ /dev/null
@@ -1,137 +0,0 @@
-{
-  "delta": 1e-05,
-  "num_canaries": 500,
-  "num_members": 250,
-  "paper_guess_fraction": 0.2,
-  "paper_guess_steps": 20,
-  "loss": {
-    "auc": 0.527816,
-    "empirical_epsilon": {
-      "0.05": 0.172086076810956,
-      "0.01": 0.0
-    },
-    "empirical_epsilon_details": {
-      "0.05": {
-        "epsilon": 0.172086076810956,
-        "num_guesses": 20,
-        "correct_guesses": 15,
-        "candidate_num_guesses": [
-          5,
-          10,
-          15,
-          20,
-          25,
-          30,
-          35,
-          40,
-          45,
-          50,
-          55,
-          60,
-          65,
-          70,
-          75,
-          80,
-          85,
-          90,
-          95,
-          100
-        ],
-        "direction": "lower"
-      },
-      "0.01": {
-        "epsilon": 0.0,
-        "num_guesses": 0,
-        "correct_guesses": 0,
-        "candidate_num_guesses": [
-          5,
-          10,
-          15,
-          20,
-          25,
-          30,
-          35,
-          40,
-          45,
-          50,
-          55,
-          60,
-          65,
-          70,
-          75,
-          80,
-          85,
-          90,
-          95,
-          100
-        ],
-        "direction": "lower"
-      }
-    }
-  },
-  "embedding": {
-    "auc": 0.516168,
-    "empirical_epsilon": {
-      "0.05": 0.13595904596149921,
-      "0.01": 0.0
-    },
-    "empirical_epsilon_details": {
-      "0.05": {
-        "epsilon": 0.13595904596149921,
-        "num_guesses": 30,
-        "correct_guesses": 21,
-        "candidate_num_guesses": [
-          5,
-          10,
-          15,
-          20,
-          25,
-          30,
-          35,
-          40,
-          45,
-          50,
-          55,
-          60,
-          65,
-          70,
-          75,
-          80,
-          85,
-          90,
-          95,
-          100
-        ],
-        "direction": "higher"
-      },
-      "0.01": {
-        "epsilon": 0.0,
-        "num_guesses": 0,
-        "correct_guesses": 0,
-        "candidate_num_guesses": [
-          5,
-          10,
-          15,
-          20,
-          25,
-          30,
-          35,
-          40,
-          45,
-          50,
-          55,
-          60,
-          65,
-          70,
-          75,
-          80,
-          85,
-          90,
-          95,
-          100
-        ],
-        "direction": "lower"
-      }
-    }
-  }
-}
\ No newline at end of file
diff --git a/qwen3-8b-base/dp8_v2/epochs/epoch_001/adapter/README.md b/qwen3-8b-base/dp8_v2/epochs/epoch_001/adapter/README.md
deleted file mode 100644
index 3063755d5f621219b97f455d66479201adfb07ca..0000000000000000000000000000000000000000
--- a/qwen3-8b-base/dp8_v2/epochs/epoch_001/adapter/README.md
+++ /dev/null
@@ -1,207 +0,0 @@
----
-base_model: Qwen/Qwen3-8B-Base
-library_name: peft
-pipeline_tag: text-generation
-tags:
-- base_model:adapter:Qwen/Qwen3-8B-Base
-- lora
-- transformers
----
-
-# Model Card for Model ID
-
-<!-- Provide a quick summary of what the model is/does. -->
-
-
-
-## Model Details
-
-### Model Description
-
-<!-- Provide a longer summary of what this model is. -->
-
-
-
-- **Developed by:** [More Information Needed]
-- **Funded by [optional]:** [More Information Needed]
-- **Shared by [optional]:** [More Information Needed]
-- **Model type:** [More Information Needed]
-- **Language(s) (NLP):** [More Information Needed]
-- **License:** [More Information Needed]
-- **Finetuned from model [optional]:** [More Information Needed]
-
-### Model Sources [optional]
-
-<!-- Provide the basic links for the model. -->
-
-- **Repository:** [More Information Needed]
-- **Paper [optional]:** [More Information Needed]
-- **Demo [optional]:** [More Information Needed]
-
-## Uses
-
-<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
-
-### Direct Use
-
-<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
-
-[More Information Needed]
-
-### Downstream Use [optional]
-
-<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
-
-[More Information Needed]
-
-### Out-of-Scope Use
-
-<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
-
-[More Information Needed]
-
-## Bias, Risks, and Limitations
-
-<!-- This section is meant to convey both technical and sociotechnical limitations. -->
-
-[More Information Needed]
-
-### Recommendations
-
-<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
-
-Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
-
-## How to Get Started with the Model
-
-Use the code below to get started with the model.
-
-[More Information Needed]
-
-## Training Details
-
-### Training Data
-
-<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
-
-[More Information Needed]
-
-### Training Procedure
-
-<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
-
-#### Preprocessing [optional]
-
-[More Information Needed]
-
-
-#### Training Hyperparameters
-
-- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
-
-#### Speeds, Sizes, Times [optional]
-
-<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
-
-[More Information Needed]
-
-## Evaluation
-
-<!-- This section describes the evaluation protocols and provides the results. -->
-
-### Testing Data, Factors & Metrics
-
-#### Testing Data
-
-<!-- This should link to a Dataset Card if possible. -->
-
-[More Information Needed]
-
-#### Factors
-
-<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
-
-[More Information Needed]
-
-#### Metrics
-
-<!-- These are the evaluation metrics being used, ideally with a description of why. -->
-
-[More Information Needed]
-
-### Results
-
-[More Information Needed]
-
-#### Summary
-
-
-
-## Model Examination [optional]
-
-<!-- Relevant interpretability work for the model goes here -->
-
-[More Information Needed]
-
-## Environmental Impact
-
-<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
-
-Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
-
-- **Hardware Type:** [More Information Needed]
-- **Hours used:** [More Information Needed]
-- **Cloud Provider:** [More Information Needed]
-- **Compute Region:** [More Information Needed]
-- **Carbon Emitted:** [More Information Needed]
-
-## Technical Specifications [optional]
-
-### Model Architecture and Objective
-
-[More Information Needed]
-
-### Compute Infrastructure
-
-[More Information Needed]
-
-#### Hardware
-
-[More Information Needed]
-
-#### Software
-
-[More Information Needed]
-
-## Citation [optional]
-
-<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
-
-**BibTeX:**
-
-[More Information Needed]
-
-**APA:**
-
-[More Information Needed]
-
-## Glossary [optional]
-
-<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
-
-[More Information Needed]
-
-## More Information [optional]
-
-[More Information Needed]
-
-## Model Card Authors [optional]
-
-[More Information Needed]
-
-## Model Card Contact
-
-[More Information Needed]
-### Framework versions
-
-- PEFT 0.18.1
\ No newline at end of file
diff --git a/qwen3-8b-base/dp8_v2/epochs/epoch_001/adapter/adapter_config.json b/qwen3-8b-base/dp8_v2/epochs/epoch_001/adapter/adapter_config.json
deleted file mode 100644
index 1c0a4183c0757ee663ad70a4cd91e31a9c09e5b6..0000000000000000000000000000000000000000
--- a/qwen3-8b-base/dp8_v2/epochs/epoch_001/adapter/adapter_config.json
+++ /dev/null
@@ -1,46 +0,0 @@
-{
-  "alora_invocation_tokens": null,
-  "alpha_pattern": {},
-  "arrow_config": null,
-  "auto_mapping": null,
-  "base_model_name_or_path": "Qwen/Qwen3-8B-Base",
-  "bias": "none",
-  "corda_config": null,
-  "ensure_weight_tying": true,
-  "eva_config": null,
-  "exclude_modules": null,
-  "fan_in_fan_out": false,
-  "inference_mode": true,
-  "init_lora_weights": true,
-  "layer_replication": null,
-  "layers_pattern": null,
-  "layers_to_transform": null,
-  "loftq_config": {},
-  "lora_alpha": 32,
-  "lora_bias": false,
-  "lora_dropout": 0.05,
-  "megatron_config": null,
-  "megatron_core": "megatron.core",
-  "modules_to_save": [
-    "lm_head",
-    "embed_tokens"
-  ],
-  "peft_type": "LORA",
-  "peft_version": "0.18.1",
-  "qalora_group_size": 16,
-  "r": 16,
-  "rank_pattern": {},
-  "revision": null,
-  "target_modules": [
-    "k_proj",
-    "q_proj",
-    "v_proj",
-    "o_proj"
-  ],
-  "target_parameters": null,
-  "task_type": "CAUSAL_LM",
-  "trainable_token_indices": null,
-  "use_dora": false,
-  "use_qalora": false,
-  "use_rslora": false
-}
\ No newline at end of file
diff --git a/qwen3-8b-base/dp8_v2/epochs/epoch_001/audit_results.json b/qwen3-8b-base/dp8_v2/epochs/epoch_001/audit_results.json
deleted file mode 100644
index 9a674dc822a970a6ba1faa2b7266cb0b02074d78..0000000000000000000000000000000000000000
--- a/qwen3-8b-base/dp8_v2/epochs/epoch_001/audit_results.json
+++ /dev/null
@@ -1,137 +0,0 @@
-{
-  "delta": 1e-05,
-  "num_canaries": 500,
-  "num_members": 250,
-  "paper_guess_fraction": 0.2,
-  "paper_guess_steps": 20,
-  "loss": {
-    "auc": 0.523144,
-    "empirical_epsilon": {
-      "0.05": 0.06548143830150366,
-      "0.01": 0.0
-    },
-    "empirical_epsilon_details": {
-      "0.05": {
-        "epsilon": 0.06548143830150366,
-        "num_guesses": 55,
-        "correct_guesses": 35,
-        "candidate_num_guesses": [
-          5,
-          10,
-          15,
-          20,
-          25,
-          30,
-          35,
-          40,
-          45,
-          50,
-          55,
-          60,
-          65,
-          70,
-          75,
-          80,
-          85,
-          90,
-          95,
-          100
-        ],
-        "direction": "lower"
-      },
-      "0.01": {
-        "epsilon": 0.0,
-        "num_guesses": 0,
-        "correct_guesses": 0,
-        "candidate_num_guesses": [
-          5,
-          10,
-          15,
-          20,
-          25,
-          30,
-          35,
-          40,
-          45,
-          50,
-          55,
-          60,
-          65,
-          70,
-          75,
-          80,
-          85,
-          90,
-          95,
-          100
-        ],
-        "direction": "lower"
-      }
-    }
-  },
-  "embedding": {
-    "auc": 0.5016,
-    "empirical_epsilon": {
-      "0.05": 0.0,
-      "0.01": 0.0
-    },
-    "empirical_epsilon_details": {
-      "0.05": {
-        "epsilon": 0.0,
-        "num_guesses": 0,
-        "correct_guesses": 0,
-        "candidate_num_guesses": [
-          5,
-          10,
-          15,
-          20,
-          25,
-          30,
-          35,
-          40,
-          45,
-          50,
-          55,
-          60,
-          65,
-          70,
-          75,
-          80,
-          85,
-          90,
-          95,
-          100
-        ],
-        "direction": "lower"
-      },
-      "0.01": {
-        "epsilon": 0.0,
-        "num_guesses": 0,
-        "correct_guesses": 0,
-        "candidate_num_guesses": [
-          5,
-          10,
-          15,
-          20,
-          25,
-          30,
-          35,
-          40,
-          45,
-          50,
-          55,
-          60,
-          65,
-          70,
-          75,
-          80,
-          85,
-          90,
-          95,
-          100
-        ],
-        "direction": "lower"
-      }
-    }
-  }
-}
\ No newline at end of file
diff --git a/qwen3-8b-base/dp8_v2/epochs/epoch_002/adapter/README.md b/qwen3-8b-base/dp8_v2/epochs/epoch_002/adapter/README.md
deleted file mode 100644
index 3063755d5f621219b97f455d66479201adfb07ca..0000000000000000000000000000000000000000
--- a/qwen3-8b-base/dp8_v2/epochs/epoch_002/adapter/README.md
+++ /dev/null
@@ -1,207 +0,0 @@
----
-base_model: Qwen/Qwen3-8B-Base
-library_name: peft
-pipeline_tag: text-generation
-tags:
-- base_model:adapter:Qwen/Qwen3-8B-Base
-- lora
-- transformers
----
-
-# Model Card for Model ID
-
-<!-- Provide a quick summary of what the model is/does. -->
-
-
-
-## Model Details
-
-### Model Description
-
-<!-- Provide a longer summary of what this model is. -->
-
-
-
-- **Developed by:** [More Information Needed]
-- **Funded by [optional]:** [More Information Needed]
-- **Shared by [optional]:** [More Information Needed]
-- **Model type:** [More Information Needed]
-- **Language(s) (NLP):** [More Information Needed]
-- **License:** [More Information Needed]
-- **Finetuned from model [optional]:** [More Information Needed]
-
-### Model Sources [optional]
-
-<!-- Provide the basic links for the model. -->
-
-- **Repository:** [More Information Needed]
-- **Paper [optional]:** [More Information Needed]
-- **Demo [optional]:** [More Information Needed]
-
-## Uses
-
-<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
-
-### Direct Use
-
-<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
-
-[More Information Needed]
-
-### Downstream Use [optional]
-
-<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
-
-[More Information Needed]
-
-### Out-of-Scope Use
-
-<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
-
-[More Information Needed]
-
-## Bias, Risks, and Limitations
-
-<!-- This section is meant to convey both technical and sociotechnical limitations. -->
-
-[More Information Needed]
-
-### Recommendations
-
-<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
-
-Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
-
-## How to Get Started with the Model
-
-Use the code below to get started with the model.
-
-[More Information Needed]
-
-## Training Details
-
-### Training Data
-
-<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
-
-[More Information Needed]
-
-### Training Procedure
-
-<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
-
-#### Preprocessing [optional]
-
-[More Information Needed]
-
-
-#### Training Hyperparameters
-
-- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
-
-#### Speeds, Sizes, Times [optional]
-
-<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
-
-[More Information Needed]
-
-## Evaluation
-
-<!-- This section describes the evaluation protocols and provides the results. -->
-
-### Testing Data, Factors & Metrics
-
-#### Testing Data
-
-<!-- This should link to a Dataset Card if possible. -->
-
-[More Information Needed]
-
-#### Factors
-
-<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
-
-[More Information Needed]
-
-#### Metrics
-
-<!-- These are the evaluation metrics being used, ideally with a description of why. -->
-
-[More Information Needed]
-
-### Results
-
-[More Information Needed]
-
-#### Summary
-
-
-
-## Model Examination [optional]
-
-<!-- Relevant interpretability work for the model goes here -->
-
-[More Information Needed]
-
-## Environmental Impact
-
-<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
-
-Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
-
-- **Hardware Type:** [More Information Needed]
-- **Hours used:** [More Information Needed]
-- **Cloud Provider:** [More Information Needed]
-- **Compute Region:** [More Information Needed]
-- **Carbon Emitted:** [More Information Needed]
-
-## Technical Specifications [optional]
-
-### Model Architecture and Objective
-
-[More Information Needed]
-
-### Compute Infrastructure
-
-[More Information Needed]
-
-#### Hardware
-
-[More Information Needed]
-
-#### Software
-
-[More Information Needed]
-
-## Citation [optional]
-
-<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
-
-**BibTeX:**
-
-[More Information Needed]
-
-**APA:**
-
-[More Information Needed]
-
-## Glossary [optional]
-
-<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
-
-[More Information Needed]
-
-## More Information [optional]
-
-[More Information Needed]
-
-## Model Card Authors [optional]
-
-[More Information Needed]
-
-## Model Card Contact
-
-[More Information Needed]
-### Framework versions
-
-- PEFT 0.18.1
\ No newline at end of file
diff --git a/qwen3-8b-base/dp8_v2/epochs/epoch_002/adapter/adapter_config.json b/qwen3-8b-base/dp8_v2/epochs/epoch_002/adapter/adapter_config.json
deleted file mode 100644
index 1c0a4183c0757ee663ad70a4cd91e31a9c09e5b6..0000000000000000000000000000000000000000
--- a/qwen3-8b-base/dp8_v2/epochs/epoch_002/adapter/adapter_config.json
+++ /dev/null
@@ -1,46 +0,0 @@
-{
-  "alora_invocation_tokens": null,
-  "alpha_pattern": {},
-  "arrow_config": null,
-  "auto_mapping": null,
-  "base_model_name_or_path": "Qwen/Qwen3-8B-Base",
-  "bias": "none",
-  "corda_config": null,
-  "ensure_weight_tying": true,
-  "eva_config": null,
-  "exclude_modules": null,
-  "fan_in_fan_out": false,
-  "inference_mode": true,
-  "init_lora_weights": true,
-  "layer_replication": null,
-  "layers_pattern": null,
-  "layers_to_transform": null,
-  "loftq_config": {},
-  "lora_alpha": 32,
-  "lora_bias": false,
-  "lora_dropout": 0.05,
-  "megatron_config": null,
-  "megatron_core": "megatron.core",
-  "modules_to_save": [
-    "lm_head",
-    "embed_tokens"
-  ],
-  "peft_type": "LORA",
-  "peft_version": "0.18.1",
-  "qalora_group_size": 16,
-  "r": 16,
-  "rank_pattern": {},
-  "revision": null,
-  "target_modules": [
-    "k_proj",
-    "q_proj",
-    "v_proj",
-    "o_proj"
-  ],
-  "target_parameters": null,
-  "task_type": "CAUSAL_LM",
-  "trainable_token_indices": null,
-  "use_dora": false,
-  "use_qalora": false,
-  "use_rslora": false
-}
\ No newline at end of file
diff --git a/qwen3-8b-base/dp8_v2/epochs/epoch_002/audit_results.json b/qwen3-8b-base/dp8_v2/epochs/epoch_002/audit_results.json
deleted file mode 100644
index 2a4b55beb149d54384e28f40c640a10d94591f77..0000000000000000000000000000000000000000
--- a/qwen3-8b-base/dp8_v2/epochs/epoch_002/audit_results.json
+++ /dev/null
@@ -1,137 +0,0 @@
-{
-  "delta": 1e-05,
-  "num_canaries": 500,
-  "num_members": 250,
-  "paper_guess_fraction": 0.2,
-  "paper_guess_steps": 20,
-  "loss": {
-    "auc": 0.525384,
-    "empirical_epsilon": {
-      "0.05": 0.32175926957279444,
-      "0.01": 0.1086258813738823
-    },
-    "empirical_epsilon_details": {
-      "0.05": {
-        "epsilon": 0.32175926957279444,
-        "num_guesses": 45,
-        "correct_guesses": 32,
-        "candidate_num_guesses": [
-          5,
-          10,
-          15,
-          20,
-          25,
-          30,
-          35,
-          40,
-          45,
-          50,
-          55,
-          60,
-          65,
-          70,
-          75,
-          80,
-          85,
-          90,
-          95,
-          100
-        ],
-        "direction": "lower"
-      },
-      "0.01": {
-        "epsilon": 0.1086258813738823,
-        "num_guesses": 45,
-        "correct_guesses": 32,
-        "candidate_num_guesses": [
-          5,
-          10,
-          15,
-          20,
-          25,
-          30,
-          35,
-          40,
-          45,
-          50,
-          55,
-          60,
-          65,
-          70,
-          75,
-          80,
-          85,
-          90,
-          95,
-          100
-        ],
-        "direction": "lower"
-      }
-    }
-  },
-  "embedding": {
-    "auc": 0.516648,
-    "empirical_epsilon": {
-      "0.05": 0.42648759204894304,
-      "0.01": 0.16005427204072475
-    },
-    "empirical_epsilon_details": {
-      "0.05": {
-        "epsilon": 0.42648759204894304,
-        "num_guesses": 30,
-        "correct_guesses": 23,
-        "candidate_num_guesses": [
-          5,
-          10,
-          15,
-          20,
-          25,
-          30,
-          35,
-          40,
-          45,
-          50,
-          55,
-          60,
-          65,
-          70,
-          75,
-          80,
-          85,
-          90,
-          95,
-          100
-        ],
-        "direction": "higher"
-      },
-      "0.01": {
-        "epsilon": 0.16005427204072475,
-        "num_guesses": 30,
-        "correct_guesses": 23,
-        "candidate_num_guesses": [
-          5,
-          10,
-          15,
-          20,
-          25,
-          30,
-          35,
-          40,
-          45,
-          50,
-          55,
-          60,
-          65,
-          70,
-          75,
-          80,
-          85,
-          90,
-          95,
-          100
-        ],
-        "direction": "higher"
-      }
-    }
-  }
-}
\ No newline at end of file
diff --git a/qwen3-8b-base/dp8_v2/epochs/epoch_003/adapter/README.md b/qwen3-8b-base/dp8_v2/epochs/epoch_003/adapter/README.md
deleted file mode 100644
index 3063755d5f621219b97f455d66479201adfb07ca..0000000000000000000000000000000000000000
--- a/qwen3-8b-base/dp8_v2/epochs/epoch_003/adapter/README.md
+++ /dev/null
@@ -1,207 +0,0 @@
----
-base_model: Qwen/Qwen3-8B-Base
-library_name: peft
-pipeline_tag: text-generation
-tags:
-- base_model:adapter:Qwen/Qwen3-8B-Base
-- lora
-- transformers
----
-
-# Model Card for Model ID
-
-<!-- Provide a quick summary of what the model is/does. -->
-
-
-
-## Model Details
-
-### Model Description
-
-<!-- Provide a longer summary of what this model is. -->
-
-
-
-- **Developed by:** [More Information Needed]
-- **Funded by [optional]:** [More Information Needed]
-- **Shared by [optional]:** [More Information Needed]
-- **Model type:** [More Information Needed]
-- **Language(s) (NLP):** [More Information Needed]
-- **License:** [More Information Needed]
-- **Finetuned from model [optional]:** [More Information Needed]
-
-### Model Sources [optional]
-
-<!-- Provide the basic links for the model. -->
-
-- **Repository:** [More Information Needed]
-- **Paper [optional]:** [More Information Needed]
-- **Demo [optional]:** [More Information Needed]
-
-## Uses
-
-<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
-
-### Direct Use
-
-<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
-
-[More Information Needed]
-
-### Downstream Use [optional]
-
-<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
-
-[More Information Needed]
-
-### Out-of-Scope Use
-
-<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
-
-[More Information Needed]
-
-## Bias, Risks, and Limitations
-
-<!-- This section is meant to convey both technical and sociotechnical limitations. -->
-
-[More Information Needed]
-
-### Recommendations
-
-<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
-
-Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
-
-## How to Get Started with the Model
-
-Use the code below to get started with the model.
-
-[More Information Needed]
-
-## Training Details
-
-### Training Data
-
-<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
-
-[More Information Needed]
-
-### Training Procedure
-
-<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
-
-#### Preprocessing [optional]
-
-[More Information Needed]
-
-
-#### Training Hyperparameters
-
-- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
-
-#### Speeds, Sizes, Times [optional]
-
-<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
-
-[More Information Needed]
-
-## Evaluation
-
-<!-- This section describes the evaluation protocols and provides the results. -->
-
-### Testing Data, Factors & Metrics
-
-#### Testing Data
-
-<!-- This should link to a Dataset Card if possible. -->
-
-[More Information Needed]
-
-#### Factors
-
-<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
-
-[More Information Needed]
-
-#### Metrics
-
-<!-- These are the evaluation metrics being used, ideally with a description of why. -->
-
-[More Information Needed]
-
-### Results
-
-[More Information Needed]
-
-#### Summary
-
-
-
-## Model Examination [optional]
-
-<!-- Relevant interpretability work for the model goes here -->
-
-[More Information Needed]
-
-## Environmental Impact
-
-<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
-
-Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
-
-- **Hardware Type:** [More Information Needed]
-- **Hours used:** [More Information Needed]
-- **Cloud Provider:** [More Information Needed]
-- **Compute Region:** [More Information Needed]
-- **Carbon Emitted:** [More Information Needed]
-
-## Technical Specifications [optional]
-
-### Model Architecture and Objective
-
-[More Information Needed]
-
-### Compute Infrastructure
-
-[More Information Needed]
-
-#### Hardware
-
-[More Information Needed]
-
-#### Software
-
-[More Information Needed]
-
-## Citation [optional]
-
-<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
-
-**BibTeX:**
-
-[More Information Needed]
-
-**APA:**
-
-[More Information Needed]
-
-## Glossary [optional]
-
-<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
-
-[More Information Needed]
-
-## More Information [optional]
-
-[More Information Needed]
-
-## Model Card Authors [optional]
-
-[More Information Needed]
-
-## Model Card Contact
-
-[More Information Needed]
-### Framework versions
-
-- PEFT 0.18.1
\ No newline at end of file
diff --git a/qwen3-8b-base/dp8_v2/epochs/epoch_003/adapter/adapter_config.json b/qwen3-8b-base/dp8_v2/epochs/epoch_003/adapter/adapter_config.json
deleted file mode 100644
index 1c0a4183c0757ee663ad70a4cd91e31a9c09e5b6..0000000000000000000000000000000000000000
--- a/qwen3-8b-base/dp8_v2/epochs/epoch_003/adapter/adapter_config.json
+++ /dev/null
@@ -1,46 +0,0 @@
-{
-  "alora_invocation_tokens": null,
-  "alpha_pattern": {},
-  "arrow_config": null,
-  "auto_mapping": null,
-  "base_model_name_or_path": "Qwen/Qwen3-8B-Base",
-  "bias": "none",
-  "corda_config": null,
-  "ensure_weight_tying": true,
-  "eva_config": null,
-  "exclude_modules": null,
-  "fan_in_fan_out": false,
-  "inference_mode": true,
-  "init_lora_weights": true,
-  "layer_replication": null,
-  "layers_pattern": null,
-  "layers_to_transform": null,
-  "loftq_config": {},
-  "lora_alpha": 32,
-  "lora_bias": false,
-  "lora_dropout": 0.05,
-  "megatron_config": null,
-  "megatron_core": "megatron.core",
-  "modules_to_save": [
-    "lm_head",
-    "embed_tokens"
-  ],
-  "peft_type": "LORA",
-  "peft_version": "0.18.1",
-  "qalora_group_size": 16,
-  "r": 16,
-  "rank_pattern": {},
-  "revision": null,
-  "target_modules": [
-    "k_proj",
-    "q_proj",
-    "v_proj",
-    "o_proj"
-  ],
-  "target_parameters": null,
-  "task_type": "CAUSAL_LM",
-  "trainable_token_indices": null,
-  "use_dora": false,
-  "use_qalora": false,
-  "use_rslora": false
-}
\ No newline at end of file
diff --git a/qwen3-8b-base/dp8_v2/epochs/epoch_003/audit_results.json b/qwen3-8b-base/dp8_v2/epochs/epoch_003/audit_results.json
deleted file mode 100644
index 0649e46d2949075353a30c88296c8723cb931525..0000000000000000000000000000000000000000
--- a/qwen3-8b-base/dp8_v2/epochs/epoch_003/audit_results.json
+++ /dev/null
@@ -1,137 +0,0 @@
-{
-  "delta": 1e-05,
-  "num_canaries": 500,
-  "num_members": 250,
-  "paper_guess_fraction": 0.2,
-  "paper_guess_steps": 20,
-  "loss": {
-    "auc": 0.523384,
-    "empirical_epsilon": {
-      "0.05": 0.5513502424582839,
-      "0.01": 0.1598324617370963
-    },
-    "empirical_epsilon_details": {
-      "0.05": {
-        "epsilon": 0.5513502424582839,
-        "num_guesses": 15,
-        "correct_guesses": 13,
-        "candidate_num_guesses": [
-          5,
-          10,
-          15,
-          20,
-          25,
-          30,
-          35,
-          40,
-          45,
-          50,
-          55,
-          60,
-          65,
-          70,
-          75,
-          80,
-          85,
-          90,
-          95,
-          100
-        ],
-        "direction": "lower"
-      },
-      "0.01": {
-        "epsilon": 0.1598324617370963,
-        "num_guesses": 15,
-        "correct_guesses": 13,
-        "candidate_num_guesses": [
-          5,
-          10,
-          15,
-          20,
-          25,
-          30,
-          35,
-          40,
-          45,
-          50,
-          55,
-          60,
-          65,
-          70,
-          75,
-          80,
-          85,
-          90,
-          95,
-          100
-        ],
-        "direction": "lower"
-      }
-    }
-  },
-  "embedding": {
-    "auc": 0.522584,
-    "empirical_epsilon": {
-      "0.05": 0.0,
-      "0.01": 0.0
-    },
-    "empirical_epsilon_details": {
-      "0.05": {
-        "epsilon": 0.0,
-        "num_guesses": 0,
-        "correct_guesses": 0,
-        "candidate_num_guesses": [
-          5,
-          10,
-          15,
-          20,
-          25,
-          30,
-          35,
-          40,
-          45,
-          50,
-          55,
-          60,
-          65,
-          70,
-          75,
-          80,
-          85,
-          90,
-          95,
-          100
-        ],
-        "direction": "lower"
-      },
-      "0.01": {
-        "epsilon": 0.0,
-        "num_guesses": 0,
-        "correct_guesses": 0,
-        "candidate_num_guesses": [
-          5,
-          10,
-          15,
-          20,
-          25,
-          30,
-          35,
-          40,
-          45,
-          50,
-          55,
-          60,
-          65,
-          70,
-          75,
-          80,
-          85,
-          90,
-          95,
-          100
-        ],
-        "direction": "lower"
-      }
-    }
-  }
-}
\ No newline at end of file
diff --git a/qwen3-8b-base/dp8_v2/epochs/epoch_004/adapter/README.md b/qwen3-8b-base/dp8_v2/epochs/epoch_004/adapter/README.md
deleted file mode 100644
index 3063755d5f621219b97f455d66479201adfb07ca..0000000000000000000000000000000000000000
--- a/qwen3-8b-base/dp8_v2/epochs/epoch_004/adapter/README.md
+++ /dev/null
@@ -1,207 +0,0 @@
----
-base_model: Qwen/Qwen3-8B-Base
-library_name: peft
-pipeline_tag: text-generation
-tags:
-- base_model:adapter:Qwen/Qwen3-8B-Base
-- lora
-- transformers
----
-
-# Model Card for Model ID
-
-<!-- Provide a quick summary of what the model is/does. -->
-
-
-
-## Model Details
-
-### Model Description
-
-<!-- Provide a longer summary of what this model is. -->
-
-
-
-- **Developed by:** [More Information Needed]
-- **Funded by [optional]:** [More Information Needed]
-- **Shared by [optional]:** [More Information Needed]
-- **Model type:** [More Information Needed]
-- **Language(s) (NLP):** [More Information Needed]
-- **License:** [More Information Needed]
-- **Finetuned from model [optional]:** [More Information Needed]
-
-### Model Sources [optional]
-
-<!-- Provide the basic links for the model. -->
-
-- **Repository:** [More Information Needed]
-- **Paper [optional]:** [More Information Needed]
-- **Demo [optional]:** [More Information Needed]
-
-## Uses
-
-<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
-
-### Direct Use
-
-<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
-
-[More Information Needed]
-
-### Downstream Use [optional]
-
-<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
-
-[More Information Needed]
-
-### Out-of-Scope Use
-
-<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
-
-[More Information Needed]
-
-## Bias, Risks, and Limitations
-
-<!-- This section is meant to convey both technical and sociotechnical limitations. -->
-
-[More Information Needed]
-
-### Recommendations
-
-<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
-
-Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
-
-## How to Get Started with the Model
-
-Use the code below to get started with the model.
-
-[More Information Needed]
-
-## Training Details
-
-### Training Data
-
-<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
-
-[More Information Needed]
-
-### Training Procedure
-
-<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
-
-#### Preprocessing [optional]
-
-[More Information Needed]
-
-
-#### Training Hyperparameters
-
-- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
-
-#### Speeds, Sizes, Times [optional]
-
-<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
-
-[More Information Needed]
-
-## Evaluation
-
-<!-- This section describes the evaluation protocols and provides the results. -->
-
-### Testing Data, Factors & Metrics
-
-#### Testing Data
-
-<!-- This should link to a Dataset Card if possible. -->
-
-[More Information Needed]
-
-#### Factors
-
-<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
-
-[More Information Needed]
-
-#### Metrics
-
-<!-- These are the evaluation metrics being used, ideally with a description of why. -->
-
-[More Information Needed]
-
-### Results
-
-[More Information Needed]
-
-#### Summary
-
-
-
-## Model Examination [optional]
-
-<!-- Relevant interpretability work for the model goes here -->
-
-[More Information Needed]
-
-## Environmental Impact
-
-<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
-
-Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
-
-- **Hardware Type:** [More Information Needed]
-- **Hours used:** [More Information Needed]
-- **Cloud Provider:** [More Information Needed]
-- **Compute Region:** [More Information Needed]
-- **Carbon Emitted:** [More Information Needed]
-
-## Technical Specifications [optional]
-
-### Model Architecture and Objective
-
-[More Information Needed]
-
-### Compute Infrastructure
-
-[More Information Needed]
-
-#### Hardware
-
-[More Information Needed]
-
-#### Software
-
-[More Information Needed]
-
-## Citation [optional]
-
-<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
-
-**BibTeX:**
-
-[More Information Needed]
-
-**APA:**
-
-[More Information Needed]
-
-## Glossary [optional]
-
-<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
-
-[More Information Needed]
-
-## More Information [optional]
-
-[More Information Needed]
-
-## Model Card Authors [optional]
-
-[More Information Needed]
-
-## Model Card Contact
-
-[More Information Needed]
-### Framework versions
-
-- PEFT 0.18.1
\ No newline at end of file
diff --git a/qwen3-8b-base/dp8_v2/epochs/epoch_004/adapter/adapter_config.json b/qwen3-8b-base/dp8_v2/epochs/epoch_004/adapter/adapter_config.json
deleted file mode 100644
index 1c0a4183c0757ee663ad70a4cd91e31a9c09e5b6..0000000000000000000000000000000000000000
--- a/qwen3-8b-base/dp8_v2/epochs/epoch_004/adapter/adapter_config.json
+++ /dev/null
@@ -1,46 +0,0 @@
-{
-  "alora_invocation_tokens": null,
-  "alpha_pattern": {},
-  "arrow_config": null,
-  "auto_mapping": null,
-  "base_model_name_or_path": "Qwen/Qwen3-8B-Base",
-  "bias": "none",
-  "corda_config": null,
-  "ensure_weight_tying": true,
-  "eva_config": null,
-  "exclude_modules": null,
-  "fan_in_fan_out": false,
-  "inference_mode": true,
-  "init_lora_weights": true,
-  "layer_replication": null,
-  "layers_pattern": null,
-  "layers_to_transform": null,
-  "loftq_config": {},
-  "lora_alpha": 32,
-  "lora_bias": false,
-  "lora_dropout": 0.05,
-  "megatron_config": null,
-  "megatron_core": "megatron.core",
-  "modules_to_save": [
-    "lm_head",
-    "embed_tokens"
-  ],
-  "peft_type": "LORA",
-  "peft_version": "0.18.1",
-  "qalora_group_size": 16,
-  "r": 16,
-  "rank_pattern": {},
-  "revision": null,
-  "target_modules": [
-    "k_proj",
-    "q_proj",
-    "v_proj",
-    "o_proj"
-  ],
-  "target_parameters": null,
-  "task_type": "CAUSAL_LM",
-  "trainable_token_indices": null,
-  "use_dora": false,
-  "use_qalora": false,
-  "use_rslora": false
-}
\ No newline at end of file
diff --git a/qwen3-8b-base/dp8_v2/epochs/epoch_004/audit_results.json b/qwen3-8b-base/dp8_v2/epochs/epoch_004/audit_results.json
deleted file mode 100644
index 1015c8da0dcda5aaeff32cb2dab3acf5bf182d2d..0000000000000000000000000000000000000000
--- a/qwen3-8b-base/dp8_v2/epochs/epoch_004/audit_results.json
+++ /dev/null
@@ -1,137 +0,0 @@
-{
-  "delta": 1e-05,
-  "num_canaries": 500,
-  "num_members": 250,
-  "paper_guess_fraction": 0.2,
-  "paper_guess_steps": 20,
-  "loss": {
-    "auc": 0.520672,
-    "empirical_epsilon": {
-      "0.05": 0.17315445840358734,
-      "0.01": 0.0
-    },
-    "empirical_epsilon_details": {
-      "0.05": {
-        "epsilon": 0.17315445840358734,
-        "num_guesses": 5,
-        "correct_guesses": 5,
-        "candidate_num_guesses": [
-          5,
-          10,
-          15,
-          20,
-          25,
-          30,
-          35,
-          40,
-          45,
-          50,
-          55,
-          60,
-          65,
-          70,
-          75,
-          80,
-          85,
-          90,
-          95,
-          100
-        ],
-        "direction": "lower"
-      },
-      "0.01": {
-        "epsilon": 0.0,
-        "num_guesses": 0,
-        "correct_guesses": 0,
-        "candidate_num_guesses": [
-          5,
-          10,
-          15,
-          20,
-          25,
-          30,
-          35,
-          40,
-          45,
-          50,
-          55,
-          60,
-          65,
-          70,
-          75,
-          80,
-          85,
-          90,
-          95,
-          100
-        ],
-        "direction": "lower"
-      }
-    }
-  },
-  "embedding": {
-    "auc": 0.522448,
-    "empirical_epsilon": {
-      "0.05": 0.05073561053723097,
-      "0.01": 0.0
-    },
-    "empirical_epsilon_details": {
-      "0.05": {
-        "epsilon": 0.05073561053723097,
-        "num_guesses": 100,
-        "correct_guesses": 60,
-        "candidate_num_guesses": [
-          5,
-          10,
-          15,
-          20,
-          25,
-          30,
-          35,
-          40,
-          45,
-          50,
-          55,
-          60,
-          65,
-          70,
-          75,
-          80,
-          85,
-          90,
-          95,
-          100
-        ],
-        "direction": "higher"
-      },
-      "0.01": {
-        "epsilon": 0.0,
-        "num_guesses": 0,
-        "correct_guesses": 0,
-        "candidate_num_guesses": [
-          5,
-          10,
-          15,
-          20,
-          25,
-          30,
-          35,
-          40,
-          45,
-          50,
-          55,
-          60,
-          65,
-          70,
-          75,
-          80,
-          85,
-          90,
-          95,
-          100
-        ],
-        "direction": "lower"
-      }
-    }
-  }
-}
\ No newline at end of file
diff --git a/qwen3-8b-base/dp8_v2/epochs/epoch_005/adapter/README.md b/qwen3-8b-base/dp8_v2/epochs/epoch_005/adapter/README.md
deleted file mode 100644
index 3063755d5f621219b97f455d66479201adfb07ca..0000000000000000000000000000000000000000
--- a/qwen3-8b-base/dp8_v2/epochs/epoch_005/adapter/README.md
+++ /dev/null
@@ -1,207 +0,0 @@
----
-base_model: Qwen/Qwen3-8B-Base
-library_name: peft
-pipeline_tag: text-generation
-tags:
-- base_model:adapter:Qwen/Qwen3-8B-Base
-- lora
-- transformers
----
-
-# Model Card for Model ID
-
-<!-- Provide a quick summary of what the model is/does. -->
-
-
-
-## Model Details
-
-### Model Description
-
-<!-- Provide a longer summary of what this model is. -->
-
-
-
-- **Developed by:** [More Information Needed]
-- **Funded by [optional]:** [More Information Needed]
-- **Shared by [optional]:** [More Information Needed]
-- **Model type:** [More Information Needed]
-- **Language(s) (NLP):** [More Information Needed]
-- **License:** [More Information Needed]
-- **Finetuned from model [optional]:** [More Information Needed]
-
-### Model Sources [optional]
-
-<!-- Provide the basic links for the model. -->
-
-- **Repository:** [More Information Needed]
-- **Paper [optional]:** [More Information Needed]
-- **Demo [optional]:** [More Information Needed]
-
-## Uses
-
-<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
-
-### Direct Use
-
-<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
-
-[More Information Needed]
-
-### Downstream Use [optional]
-
-<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
-
-[More Information Needed]
-
-### Out-of-Scope Use
-
-<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
-
-[More Information Needed]
-
-## Bias, Risks, and Limitations
-
-<!-- This section is meant to convey both technical and sociotechnical limitations. -->
-
-[More Information Needed]
-
-### Recommendations
-
-<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
-
-Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
-
-## How to Get Started with the Model
-
-Use the code below to get started with the model.
-
-[More Information Needed]
-
-## Training Details
-
-### Training Data
-
-<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
-
-[More Information Needed]
-
-### Training Procedure
-
-<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
-
-#### Preprocessing [optional]
-
-[More Information Needed]
-
-
-#### Training Hyperparameters
-
-- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
-
-#### Speeds, Sizes, Times [optional]
-
-<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
-
-[More Information Needed]
-
-## Evaluation
-
-<!-- This section describes the evaluation protocols and provides the results. -->
-
-### Testing Data, Factors & Metrics
-
-#### Testing Data
-
-<!-- This should link to a Dataset Card if possible. -->
-
-[More Information Needed]
-
-#### Factors
-
-<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
-
-[More Information Needed]
-
-#### Metrics
-
-<!-- These are the evaluation metrics being used, ideally with a description of why. -->
-
-[More Information Needed]
-
-### Results
-
-[More Information Needed]
-
-#### Summary
-
-
-
-## Model Examination [optional]
-
-<!-- Relevant interpretability work for the model goes here -->
-
-[More Information Needed]
-
-## Environmental Impact
-
-<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
-
-Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
-
-- **Hardware Type:** [More Information Needed]
-- **Hours used:** [More Information Needed]
-- **Cloud Provider:** [More Information Needed]
-- **Compute Region:** [More Information Needed]
-- **Carbon Emitted:** [More Information Needed]
-
-## Technical Specifications [optional]
-
-### Model Architecture and Objective
-
-[More Information Needed]
-
-### Compute Infrastructure
-
-[More Information Needed]
-
-#### Hardware
-
-[More Information Needed]
-
-#### Software
-
-[More Information Needed]
-
-## Citation [optional]
-
-<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
-
-**BibTeX:**
-
-[More Information Needed]
-
-**APA:**
-
-[More Information Needed]
-
-## Glossary [optional]
-
-<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
-
-[More Information Needed]
-
-## More Information [optional]
-
-[More Information Needed]
-
-## Model Card Authors [optional]
-
-[More Information Needed]
-
-## Model Card Contact
-
-[More Information Needed]
-### Framework versions
-
-- PEFT 0.18.1
\ No newline at end of file
diff --git a/qwen3-8b-base/dp8_v2/epochs/epoch_005/adapter/adapter_config.json b/qwen3-8b-base/dp8_v2/epochs/epoch_005/adapter/adapter_config.json
deleted file mode 100644
index 1c0a4183c0757ee663ad70a4cd91e31a9c09e5b6..0000000000000000000000000000000000000000
--- a/qwen3-8b-base/dp8_v2/epochs/epoch_005/adapter/adapter_config.json
+++ /dev/null
@@ -1,46 +0,0 @@
-{
-  "alora_invocation_tokens": null,
-  "alpha_pattern": {},
-  "arrow_config": null,
-  "auto_mapping": null,
-  "base_model_name_or_path": "Qwen/Qwen3-8B-Base",
-  "bias": "none",
-  "corda_config": null,
-  "ensure_weight_tying": true,
-  "eva_config": null,
-  "exclude_modules": null,
-  "fan_in_fan_out": false,
-  "inference_mode": true,
-  "init_lora_weights": true,
-  "layer_replication": null,
-  "layers_pattern": null,
-  "layers_to_transform": null,
-  "loftq_config": {},
-  "lora_alpha": 32,
-  "lora_bias": false,
-  "lora_dropout": 0.05,
-  "megatron_config": null,
-  "megatron_core": "megatron.core",
-  "modules_to_save": [
-    "lm_head",
-    "embed_tokens"
-  ],
-  "peft_type": "LORA",
-  "peft_version": "0.18.1",
-  "qalora_group_size": 16,
-  "r": 16,
-  "rank_pattern": {},
-  "revision": null,
-  "target_modules": [
-    "k_proj",
-    "q_proj",
-    "v_proj",
-    "o_proj"
-  ],
-  "target_parameters": null,
-  "task_type": "CAUSAL_LM",
-  "trainable_token_indices": null,
-  "use_dora": false,
-  "use_qalora": false,
-  "use_rslora": false
-}
\ No newline at end of file
diff --git a/qwen3-8b-base/dp8_v2/epochs/epoch_005/audit_results.json b/qwen3-8b-base/dp8_v2/epochs/epoch_005/audit_results.json
deleted file mode 100644
index 97143b6c5e63f566d5e9d5726f36e2fb43a8e444..0000000000000000000000000000000000000000
--- a/qwen3-8b-base/dp8_v2/epochs/epoch_005/audit_results.json
+++ /dev/null
@@ -1,137 +0,0 @@
-{
-  "delta": 1e-05,
-  "num_canaries": 500,
-  "num_members": 250,
-  "paper_guess_fraction": 0.2,
-  "paper_guess_steps": 20,
-  "loss": {
-    "auc": 0.517744,
-    "empirical_epsilon": {
-      "0.05": 0.41679731756448746,
-      "0.01": 0.0
-    },
-    "empirical_epsilon_details": {
-      "0.05": {
-        "epsilon": 0.41679731756448746,
-        "num_guesses": 10,
-        "correct_guesses": 9,
-        "candidate_num_guesses": [
-          5,
-          10,
-          15,
-          20,
-          25,
-          30,
-          35,
-          40,
-          45,
-          50,
-          55,
-          60,
-          65,
-          70,
-          75,
-          80,
-          85,
-          90,
-          95,
-          100
-        ],
-        "direction": "lower"
-      },
-      "0.01": {
-        "epsilon": 0.0,
-        "num_guesses": 0,
-        "correct_guesses": 0,
-        "candidate_num_guesses": [
-          5,
-          10,
-          15,
-          20,
-          25,
-          30,
-          35,
-          40,
-          45,
-          50,
-          55,
-          60,
-          65,
-          70,
-          75,
-          80,
-          85,
-          90,
-          95,
-          100
-        ],
-        "direction": "lower"
-      }
-    }
-  },
-  "embedding": {
-    "auc": 0.520904,
-    "empirical_epsilon": {
-      "0.05": 0.1320468671619892,
-      "0.01": 0.0
-    },
-    "empirical_epsilon_details": {
-      "0.05": {
-        "epsilon": 0.1320468671619892,
-        "num_guesses": 40,
-        "correct_guesses": 27,
-        "candidate_num_guesses": [
-          5,
-          10,
-          15,
-          20,
-          25,
-          30,
-          35,
-          40,
-          45,
-          50,
-          55,
-          60,
-          65,
-          70,
-          75,
-          80,
-          85,
-          90,
-          95,
-          100
-        ],
-        "direction": "higher"
-      },
-      "0.01": {
-        "epsilon": 0.0,
-        "num_guesses": 0,
-        "correct_guesses": 0,
-        "candidate_num_guesses": [
-          5,
-          10,
-          15,
-          20,
-          25,
-          30,
-          35,
-          40,
-          45,
-          50,
-          55,
-          60,
-          65,
-          70,
-          75,
-          80,
-          85,
-          90,
-          95,
-          100
-        ],
-        "direction": "lower"
-      }
-    }
-  }
-}
\ No newline at end of file
diff --git a/starcoder2-7b/base/epochs/epoch_001/adapter/README.md b/starcoder2-7b/base/epochs/epoch_001/adapter/README.md
deleted file mode 100644
index 9f7d6e699b0d658ac4c4291a23a0dafd21d1d0b8..0000000000000000000000000000000000000000
--- a/starcoder2-7b/base/epochs/epoch_001/adapter/README.md
+++ /dev/null
@@ -1,207 +0,0 @@
----
-base_model: bigcode/starcoder2-7b
-library_name: peft
-pipeline_tag: text-generation
-tags:
-- base_model:adapter:bigcode/starcoder2-7b
-- lora
-- transformers
----
-
-# Model Card for Model ID
-
-<!-- Provide a quick summary of what the model is/does. -->
-
-
-
-## Model Details
-
-### Model Description
-
-<!-- Provide a longer summary of what this model is. -->
-
-
-
-- **Developed by:** [More Information Needed]
-- **Funded by [optional]:** [More Information Needed]
-- **Shared by [optional]:** [More Information Needed]
-- **Model type:** [More Information Needed]
-- **Language(s) (NLP):** [More Information Needed]
-- **License:** [More Information Needed]
-- **Finetuned from model [optional]:** [More Information Needed]
-
-### Model Sources [optional]
-
-<!-- Provide the basic links for the model. -->
-
-- **Repository:** [More Information Needed]
-- **Paper [optional]:** [More Information Needed]
-- **Demo [optional]:** [More Information Needed]
-
-## Uses
-
-<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
-
-### Direct Use
-
-<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
-
-[More Information Needed]
-
-### Downstream Use [optional]
-
-<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
-
-[More Information Needed]
-
-### Out-of-Scope Use
-
-<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
-
-[More Information Needed]
-
-## Bias, Risks, and Limitations
-
-<!-- This section is meant to convey both technical and sociotechnical limitations. -->
-
-[More Information Needed]
-
-### Recommendations
-
-<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
-
-Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
-
-## How to Get Started with the Model
-
-Use the code below to get started with the model.
-
-[More Information Needed]
-
-## Training Details
-
-### Training Data
-
-<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
-
-[More Information Needed]
-
-### Training Procedure
-
-<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
-
-#### Preprocessing [optional]
-
-[More Information Needed]
-
-
-#### Training Hyperparameters
-
-- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
-
-#### Speeds, Sizes, Times [optional]
-
-<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
-
-[More Information Needed]
-
-## Evaluation
-
-<!-- This section describes the evaluation protocols and provides the results. -->
-
-### Testing Data, Factors & Metrics
-
-#### Testing Data
-
-<!-- This should link to a Dataset Card if possible. -->
-
-[More Information Needed]
-
-#### Factors
-
-<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
-
-[More Information Needed]
-
-#### Metrics
-
-<!-- These are the evaluation metrics being used, ideally with a description of why. -->
-
-[More Information Needed]
-
-### Results
-
-[More Information Needed]
-
-#### Summary
-
-
-
-## Model Examination [optional]
-
-<!-- Relevant interpretability work for the model goes here -->
-
-[More Information Needed]
-
-## Environmental Impact
-
-<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
-
-Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
-
-- **Hardware Type:** [More Information Needed]
-- **Hours used:** [More Information Needed]
-- **Cloud Provider:** [More Information Needed]
-- **Compute Region:** [More Information Needed]
-- **Carbon Emitted:** [More Information Needed]
-
-## Technical Specifications [optional]
-
-### Model Architecture and Objective
-
-[More Information Needed]
-
-### Compute Infrastructure
-
-[More Information Needed]
-
-#### Hardware
-
-[More Information Needed]
-
-#### Software
-
-[More Information Needed]
-
-## Citation [optional]
-
-<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
-
-**BibTeX:**
-
-[More Information Needed]
-
-**APA:**
-
-[More Information Needed]
-
-## Glossary [optional]
-
-<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
-
-[More Information Needed]
-
-## More Information [optional]
-
-[More Information Needed]
-
-## Model Card Authors [optional]
-
-[More Information Needed]
-
-## Model Card Contact
-
-[More Information Needed]
-### Framework versions
-
-- PEFT 0.18.1
\ No newline at end of file
diff --git a/starcoder2-7b/base/epochs/epoch_001/adapter/adapter_config.json b/starcoder2-7b/base/epochs/epoch_001/adapter/adapter_config.json
deleted file mode 100644
index 310e38ead0c24161335fe3d64ec37028fa9eb08d..0000000000000000000000000000000000000000
--- a/starcoder2-7b/base/epochs/epoch_001/adapter/adapter_config.json
+++ /dev/null
@@ -1,46 +0,0 @@
-{
-  "alora_invocation_tokens": null,
-  "alpha_pattern": {},
-  "arrow_config": null,
-  "auto_mapping": null,
-  "base_model_name_or_path": "bigcode/starcoder2-7b",
-  "bias": "none",
-  "corda_config": null,
-  "ensure_weight_tying": true,
-  "eva_config": null,
-  "exclude_modules": null,
-  "fan_in_fan_out": false,
-  "inference_mode": true,
-  "init_lora_weights": true,
-  "layer_replication": null,
-  "layers_pattern": null,
-  "layers_to_transform": null,
-  "loftq_config": {},
-  "lora_alpha": 32,
-  "lora_bias": false,
-  "lora_dropout": 0.05,
-  "megatron_config": null,
-  "megatron_core": "megatron.core",
-  "modules_to_save": [
-    "lm_head",
-    "embed_tokens"
-  ],
-  "peft_type": "LORA",
-  "peft_version": "0.18.1",
-  "qalora_group_size": 16,
-  "r": 16,
-  "rank_pattern": {},
-  "revision": null,
-  "target_modules": [
-    "q_proj",
-    "v_proj",
-    "o_proj",
-    "k_proj"
-  ],
-  "target_parameters": null,
-  "task_type": "CAUSAL_LM",
-  "trainable_token_indices": null,
-  "use_dora": false,
-  "use_qalora": false,
-  "use_rslora": false
-}
\ No newline at end of file
diff --git a/starcoder2-7b/base/epochs/epoch_001/audit_results.json b/starcoder2-7b/base/epochs/epoch_001/audit_results.json
deleted file mode 100644
index 9b76581d935713df7f9c3a55774bbe131e6946a7..0000000000000000000000000000000000000000
--- a/starcoder2-7b/base/epochs/epoch_001/audit_results.json
+++ /dev/null
@@ -1,137 +0,0 @@
-{
-  "delta": 1e-05,
-  "num_canaries": 500,
-  "num_members": 250,
-  "paper_guess_fraction": 0.2,
-  "paper_guess_steps": 20,
-  "loss": {
-    "auc": 0.865392,
-    "empirical_epsilon": {
-      "0.05": 3.4791953936219215,
-      "0.01": 3.023197554051876
-    },
-    "empirical_epsilon_details": {
-      "0.05": {
-        "epsilon": 3.4791953936219215,
-        "num_guesses": 100,
-        "correct_guesses": 100,
-        "candidate_num_guesses": [
-          5,
-          10,
-          15,
-          20,
-          25,
-          30,
-          35,
-          40,
-          45,
-          50,
-          55,
-          60,
-          65,
-          70,
-          75,
-          80,
-          85,
-          90,
-          95,
-          100
-        ],
-        "direction": "lower"
-      },
-      "0.01": {
-        "epsilon": 3.023197554051876,
-        "num_guesses": 100,
-        "correct_guesses": 100,
-        "candidate_num_guesses": [
-          5,
-          10,
-          15,
-          20,
-          25,
-          30,
-          35,
-          40,
-          45,
-          50,
-          55,
-          60,
-          65,
-          70,
-          75,
-          80,
-          85,
-          90,
-          95,
-          100
-        ],
-        "direction": "lower"
-      }
-    }
-  },
-  "embedding": {
-    "auc": 0.73,
-    "empirical_epsilon": {
-      "0.05": 3.4791953936219215,
-      "0.01": 3.023197554051876
-    },
-    "empirical_epsilon_details": {
-      "0.05": {
-        "epsilon": 3.4791953936219215,
-        "num_guesses": 100,
-        "correct_guesses": 100,
-        "candidate_num_guesses": [
-          5,
-          10,
-          15,
-          20,
-          25,
-          30,
-          35,
-          40,
-          45,
-          50,
-          55,
-          60,
-          65,
-          70,
-          75,
-          80,
-          85,
-          90,
-          95,
-          100
-        ],
-        "direction": "lower"
-      },
-      "0.01": {
-        "epsilon": 3.023197554051876,
-        "num_guesses": 100,
-        "correct_guesses": 100,
-        "candidate_num_guesses": [
-          5,
-          10,
-          15,
-          20,
-          25,
-          30,
-          35,
-          40,
-          45,
-          50,
-          55,
-          60,
-          65,
-          70,
-          75,
-          80,
-          85,
-          90,
-          95,
-          100
-        ],
-        "direction": "lower"
-      }
-    }
-  }
-}
\ No newline at end of file
diff --git a/starcoder2-7b/base/epochs/epoch_002/adapter/README.md b/starcoder2-7b/base/epochs/epoch_002/adapter/README.md
deleted file mode 100644
index 9f7d6e699b0d658ac4c4291a23a0dafd21d1d0b8..0000000000000000000000000000000000000000
--- a/starcoder2-7b/base/epochs/epoch_002/adapter/README.md
+++ /dev/null
@@ -1,207 +0,0 @@
----
-base_model: bigcode/starcoder2-7b
-library_name: peft
-pipeline_tag: text-generation
-tags:
-- base_model:adapter:bigcode/starcoder2-7b
-- lora
-- transformers
----
-
-# Model Card for Model ID
-
-<!-- Provide a quick summary of what the model is/does. -->
-
-
-
-## Model Details
-
-### Model Description
-
-<!-- Provide a longer summary of what this model is. -->
-
-
-
-- **Developed by:** [More Information Needed]
-- **Funded by [optional]:** [More Information Needed]
-- **Shared by [optional]:** [More Information Needed]
-- **Model type:** [More Information Needed]
-- **Language(s) (NLP):** [More Information Needed]
-- **License:** [More Information Needed]
-- **Finetuned from model [optional]:** [More Information Needed]
-
-### Model Sources [optional]
-
-<!-- Provide the basic links for the model. -->
-
-- **Repository:** [More Information Needed]
-- **Paper [optional]:** [More Information Needed]
-- **Demo [optional]:** [More Information Needed]
-
-## Uses
-
-<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
-
-### Direct Use
-
-<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
-
-[More Information Needed]
-
-### Downstream Use [optional]
-
-<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
-
-[More Information Needed]
-
-### Out-of-Scope Use
-
-<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
-
-[More Information Needed]
-
-## Bias, Risks, and Limitations
-
-<!-- This section is meant to convey both technical and sociotechnical limitations. -->
-
-[More Information Needed]
-
-### Recommendations
-
-<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
-
-Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
-
-## How to Get Started with the Model
-
-Use the code below to get started with the model.
-
-[More Information Needed]
-
-## Training Details
-
-### Training Data
-
-<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
-
-[More Information Needed]
-
-### Training Procedure
-
-<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
-
-#### Preprocessing [optional]
-
-[More Information Needed]
-
-
-#### Training Hyperparameters
-
-- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
-
-#### Speeds, Sizes, Times [optional]
-
-<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
-
-[More Information Needed]
-
-## Evaluation
-
-<!-- This section describes the evaluation protocols and provides the results. -->
-
-### Testing Data, Factors & Metrics
-
-#### Testing Data
-
-<!-- This should link to a Dataset Card if possible. -->
-
-[More Information Needed]
-
-#### Factors
-
-<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
-
-[More Information Needed]
-
-#### Metrics
-
-<!-- These are the evaluation metrics being used, ideally with a description of why. -->
-
-[More Information Needed]
-
-### Results
-
-[More Information Needed]
-
-#### Summary
-
-
-
-## Model Examination [optional]
-
-<!-- Relevant interpretability work for the model goes here -->
-
-[More Information Needed]
-
-## Environmental Impact
-
-<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
-
-Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
-
-- **Hardware Type:** [More Information Needed]
-- **Hours used:** [More Information Needed]
-- **Cloud Provider:** [More Information Needed]
-- **Compute Region:** [More Information Needed]
-- **Carbon Emitted:** [More Information Needed]
-
-## Technical Specifications [optional]
-
-### Model Architecture and Objective
-
-[More Information Needed]
-
-### Compute Infrastructure
-
-[More Information Needed]
-
-#### Hardware
-
-[More Information Needed]
-
-#### Software
-
-[More Information Needed]
-
-## Citation [optional]
-
-<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
-
-**BibTeX:**
-
-[More Information Needed]
-
-**APA:**
-
-[More Information Needed]
-
-## Glossary [optional]
-
-<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
-
-[More Information Needed]
-
-## More Information [optional]
-
-[More Information Needed]
-
-## Model Card Authors [optional]
-
-[More Information Needed]
-
-## Model Card Contact
-
-[More Information Needed]
-### Framework versions
-
-- PEFT 0.18.1
\ No newline at end of file
diff --git a/starcoder2-7b/base/epochs/epoch_002/adapter/adapter_config.json b/starcoder2-7b/base/epochs/epoch_002/adapter/adapter_config.json
deleted file mode 100644
index 310e38ead0c24161335fe3d64ec37028fa9eb08d..0000000000000000000000000000000000000000
--- a/starcoder2-7b/base/epochs/epoch_002/adapter/adapter_config.json
+++ /dev/null
@@ -1,46 +0,0 @@
-{
-  "alora_invocation_tokens": null,
-  "alpha_pattern": {},
-  "arrow_config": null,
-  "auto_mapping": null,
-  "base_model_name_or_path": "bigcode/starcoder2-7b",
-  "bias": "none",
-  "corda_config": null,
-  "ensure_weight_tying": true,
-  "eva_config": null,
-  "exclude_modules": null,
-  "fan_in_fan_out": false,
-  "inference_mode": true,
-  "init_lora_weights": true,
-  "layer_replication": null,
-  "layers_pattern": null,
-  "layers_to_transform": null,
-  "loftq_config": {},
-  "lora_alpha": 32,
-  "lora_bias": false,
-  "lora_dropout": 0.05,
-  "megatron_config": null,
-  "megatron_core": "megatron.core",
-  "modules_to_save": [
-    "lm_head",
-    "embed_tokens"
-  ],
-  "peft_type": "LORA",
-  "peft_version": "0.18.1",
-  "qalora_group_size": 16,
-  "r": 16,
-  "rank_pattern": {},
-  "revision": null,
-  "target_modules": [
-    "q_proj",
-    "v_proj",
-    "o_proj",
-    "k_proj"
-  ],
-  "target_parameters": null,
-  "task_type": "CAUSAL_LM",
-  "trainable_token_indices": null,
-  "use_dora": false,
-  "use_qalora": false,
-  "use_rslora": false
-}
\ No newline at end of file
diff --git a/starcoder2-7b/base/epochs/epoch_002/audit_results.json b/starcoder2-7b/base/epochs/epoch_002/audit_results.json
deleted file mode 100644
index addb90ca5e2b6d80d05dd2a64ae7c2c5f6798db8..0000000000000000000000000000000000000000
--- a/starcoder2-7b/base/epochs/epoch_002/audit_results.json
+++ /dev/null
@@ -1,137 +0,0 @@
-{
-  "delta": 1e-05,
-  "num_canaries": 500,
-  "num_members": 250,
-  "paper_guess_fraction": 0.2,
-  "paper_guess_steps": 20,
-  "loss": {
-    "auc": 0.962576,
-    "empirical_epsilon": {
-      "0.05": 3.4791953936219215,
-      "0.01": 3.023197554051876
-    },
-    "empirical_epsilon_details": {
-      "0.05": {
-        "epsilon": 3.4791953936219215,
-        "num_guesses": 100,
-        "correct_guesses": 100,
-        "candidate_num_guesses": [
-          5,
-          10,
-          15,
-          20,
-          25,
-          30,
-          35,
-          40,
-          45,
-          50,
-          55,
-          60,
-          65,
-          70,
-          75,
-          80,
-          85,
-          90,
-          95,
-          100
-        ],
-        "direction": "lower"
-      },
-      "0.01": {
-        "epsilon": 3.023197554051876,
-        "num_guesses": 100,
-        "correct_guesses": 100,
-        "candidate_num_guesses": [
-          5,
-          10,
-          15,
-          20,
-          25,
-          30,
-          35,
-          40,
-          45,
-          50,
-          55,
-          60,
-          65,
-          70,
-          75,
-          80,
-          85,
-          90,
-          95,
-          100
-        ],
-        "direction": "lower"
-      }
-    }
-  },
-  "embedding": {
-    "auc": 0.538,
-    "empirical_epsilon": {
-      "0.05": 3.4791953936219215,
-      "0.01": 3.023197554051876
-    },
-    "empirical_epsilon_details": {
-      "0.05": {
-        "epsilon": 3.4791953936219215,
-        "num_guesses": 100,
-        "correct_guesses": 100,
-        "candidate_num_guesses": [
-          5,
-          10,
-          15,
-          20,
-          25,
-          30,
-          35,
-          40,
-          45,
-          50,
-          55,
-          60,
-          65,
-          70,
-          75,
-          80,
-          85,
-          90,
-          95,
-          100
-        ],
-        "direction": "lower"
-      },
-      "0.01": {
-        "epsilon": 3.023197554051876,
-        "num_guesses": 100,
-        "correct_guesses": 100,
-        "candidate_num_guesses": [
-          5,
-          10,
-          15,
-          20,
-          25,
-          30,
-          35,
-          40,
-          45,
-          50,
-          55,
-          60,
-          65,
-          70,
-          75,
-          80,
-          85,
-          90,
-          95,
-          100
-        ],
-        "direction": "lower"
-      }
-    }
-  }
-}
\ No newline at end of file
diff --git a/starcoder2-7b/base/epochs/epoch_003/adapter/README.md b/starcoder2-7b/base/epochs/epoch_003/adapter/README.md
deleted file mode 100644
index 9f7d6e699b0d658ac4c4291a23a0dafd21d1d0b8..0000000000000000000000000000000000000000
--- a/starcoder2-7b/base/epochs/epoch_003/adapter/README.md
+++ /dev/null
@@ -1,207 +0,0 @@
----
-base_model: bigcode/starcoder2-7b
-library_name: peft
-pipeline_tag: text-generation
-tags:
-- base_model:adapter:bigcode/starcoder2-7b
-- lora
-- transformers
----
-
-# Model Card for Model ID
-
-<!-- Provide a quick summary of what the model is/does. -->
-
-
-
-## Model Details
-
-### Model Description
-
-<!-- Provide a longer summary of what this model is. -->
-
-
-
-- **Developed by:** [More Information Needed]
-- **Funded by [optional]:** [More Information Needed]
-- **Shared by [optional]:** [More Information Needed]
-- **Model type:** [More Information Needed]
-- **Language(s) (NLP):** [More Information Needed]
-- **License:** [More Information Needed]
-- **Finetuned from model [optional]:** [More Information Needed]
-
-### Model Sources [optional]
-
-<!-- Provide the basic links for the model. -->
-
-- **Repository:** [More Information Needed]
-- **Paper [optional]:** [More Information Needed]
-- **Demo [optional]:** [More Information Needed]
-
-## Uses
-
-<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
-
-### Direct Use
-
-<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
-
-[More Information Needed]
-
-### Downstream Use [optional]
-
-<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
-
-[More Information Needed]
-
-### Out-of-Scope Use
-
-<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
-
-[More Information Needed]
-
-## Bias, Risks, and Limitations
-
-<!-- This section is meant to convey both technical and sociotechnical limitations. -->
-
-[More Information Needed]
-
-### Recommendations
-
-<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
-
-Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
-
-## How to Get Started with the Model
-
-Use the code below to get started with the model.
-
-[More Information Needed]
-
-## Training Details
-
-### Training Data
-
-<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
-
-[More Information Needed]
-
-### Training Procedure
-
-<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
-
-#### Preprocessing [optional]
-
-[More Information Needed]
-
-
-#### Training Hyperparameters
-
-- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
-
-#### Speeds, Sizes, Times [optional]
-
-<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
-
-[More Information Needed]
-
-## Evaluation
-
-<!-- This section describes the evaluation protocols and provides the results. -->
-
-### Testing Data, Factors & Metrics
-
-#### Testing Data
-
-<!-- This should link to a Dataset Card if possible. -->
-
-[More Information Needed]
-
-#### Factors
-
-<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
-
-[More Information Needed]
-
-#### Metrics
-
-<!-- These are the evaluation metrics being used, ideally with a description of why. -->
-
-[More Information Needed]
-
-### Results
-
-[More Information Needed]
-
-#### Summary
-
-
-
-## Model Examination [optional]
-
-<!-- Relevant interpretability work for the model goes here -->
-
-[More Information Needed]
-
-## Environmental Impact
-
-<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
-
-Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
-
-- **Hardware Type:** [More Information Needed]
-- **Hours used:** [More Information Needed]
-- **Cloud Provider:** [More Information Needed]
-- **Compute Region:** [More Information Needed]
-- **Carbon Emitted:** [More Information Needed]
-
-## Technical Specifications [optional]
-
-### Model Architecture and Objective
-
-[More Information Needed]
-
-### Compute Infrastructure
-
-[More Information Needed]
-
-#### Hardware
-
-[More Information Needed]
-
-#### Software
-
-[More Information Needed]
-
-## Citation [optional]
-
-<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
-
-**BibTeX:**
-
-[More Information Needed]
-
-**APA:**
-
-[More Information Needed]
-
-## Glossary [optional]
-
-<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
-
-[More Information Needed]
-
-## More Information [optional]
-
-[More Information Needed]
-
-## Model Card Authors [optional]
-
-[More Information Needed]
-
-## Model Card Contact
-
-[More Information Needed]
-### Framework versions
-
-- PEFT 0.18.1
\ No newline at end of file
diff --git a/starcoder2-7b/base/epochs/epoch_003/adapter/adapter_config.json b/starcoder2-7b/base/epochs/epoch_003/adapter/adapter_config.json
deleted file mode 100644
index 310e38ead0c24161335fe3d64ec37028fa9eb08d..0000000000000000000000000000000000000000
--- a/starcoder2-7b/base/epochs/epoch_003/adapter/adapter_config.json
+++ /dev/null
@@ -1,46 +0,0 @@
-{
-  "alora_invocation_tokens": null,
-  "alpha_pattern": {},
-  "arrow_config": null,
-  "auto_mapping": null,
-  "base_model_name_or_path": "bigcode/starcoder2-7b",
-  "bias": "none",
-  "corda_config": null,
-  "ensure_weight_tying": true,
-  "eva_config": null,
-  "exclude_modules": null,
-  "fan_in_fan_out": false,
-  "inference_mode": true,
-  "init_lora_weights": true,
-  "layer_replication": null,
-  "layers_pattern": null,
-  "layers_to_transform": null,
-  "loftq_config": {},
-  "lora_alpha": 32,
-  "lora_bias": false,
-  "lora_dropout": 0.05,
-  "megatron_config": null,
-  "megatron_core": "megatron.core",
-  "modules_to_save": [
-    "lm_head",
-    "embed_tokens"
-  ],
-  "peft_type": "LORA",
-  "peft_version": "0.18.1",
-  "qalora_group_size": 16,
-  "r": 16,
-  "rank_pattern": {},
-  "revision": null,
-  "target_modules": [
-    "q_proj",
-    "v_proj",
-    "o_proj",
-    "k_proj"
-  ],
-  "target_parameters": null,
-  "task_type": "CAUSAL_LM",
-  "trainable_token_indices": null,
-  "use_dora": false,
-  "use_qalora": false,
-  "use_rslora": false
-}
\ No newline at end of file
diff --git a/starcoder2-7b/base/epochs/epoch_003/audit_results.json b/starcoder2-7b/base/epochs/epoch_003/audit_results.json
deleted file mode 100644
index 6fae29b929bbac0ad17b32da3211071ba4fec78a..0000000000000000000000000000000000000000
--- a/starcoder2-7b/base/epochs/epoch_003/audit_results.json
+++ /dev/null
@@ -1,137 +0,0 @@
-{
-  "delta": 1e-05,
-  "num_canaries": 500,
-  "num_members": 250,
-  "paper_guess_fraction": 0.2,
-  "paper_guess_steps": 20,
-  "loss": {
-    "auc": 0.997368,
-    "empirical_epsilon": {
-      "0.05": 3.4791953936219215,
-      "0.01": 3.023197554051876
-    },
-    "empirical_epsilon_details": {
-      "0.05": {
-        "epsilon": 3.4791953936219215,
-        "num_guesses": 100,
-        "correct_guesses": 100,
-        "candidate_num_guesses": [
-          5,
-          10,
-          15,
-          20,
-          25,
-          30,
-          35,
-          40,
-          45,
-          50,
-          55,
-          60,
-          65,
-          70,
-          75,
-          80,
-          85,
-          90,
-          95,
-          100
-        ],
-        "direction": "lower"
-      },
-      "0.01": {
-        "epsilon": 3.023197554051876,
-        "num_guesses": 100,
-        "correct_guesses": 100,
-        "candidate_num_guesses": [
-          5,
-          10,
-          15,
-          20,
-          25,
-          30,
-          35,
-          40,
-          45,
-          50,
-          55,
-          60,
-          65,
-          70,
-          75,
-          80,
-          85,
-          90,
-          95,
-          100
-        ],
-        "direction": "lower"
-      }
-    }
-  },
-  "embedding": {
-    "auc": 0.52,
-    "empirical_epsilon": {
-      "0.05": 3.4791953936219215,
-      "0.01": 3.023197554051876
-    },
-    "empirical_epsilon_details": {
-      "0.05": {
-        "epsilon": 3.4791953936219215,
-        "num_guesses": 100,
-        "correct_guesses": 100,
-        "candidate_num_guesses": [
-          5,
-          10,
-          15,
-          20,
-          25,
-          30,
-          35,
-          40,
-          45,
-          50,
-          55,
-          60,
-          65,
-          70,
-          75,
-          80,
-          85,
-          90,
-          95,
-          100
-        ],
-        "direction": "lower"
-      },
-      "0.01": {
-        "epsilon": 3.023197554051876,
-        "num_guesses": 100,
-        "correct_guesses": 100,
-        "candidate_num_guesses": [
-          5,
-          10,
-          15,
-          20,
-          25,
-          30,
-          35,
-          40,
-          45,
-          50,
-          55,
-          60,
-          65,
-          70,
-          75,
-          80,
-          85,
-          90,
-          95,
-          100
-        ],
-        "direction": "lower"
-      }
-    }
-  }
-}
\ No newline at end of file
diff --git a/starcoder2-7b/dp3/epochs/epoch_001/adapter/README.md b/starcoder2-7b/dp3/epochs/epoch_001/adapter/README.md
deleted file mode 100644
index 9f7d6e699b0d658ac4c4291a23a0dafd21d1d0b8..0000000000000000000000000000000000000000
--- a/starcoder2-7b/dp3/epochs/epoch_001/adapter/README.md
+++ /dev/null
@@ -1,207 +0,0 @@
----
-base_model: bigcode/starcoder2-7b
-library_name: peft
-pipeline_tag: text-generation
-tags:
-- base_model:adapter:bigcode/starcoder2-7b
-- lora
-- transformers
----
-
-# Model Card for Model ID
-
-<!-- Provide a quick summary of what the model is/does. -->
-
-
-
-## Model Details
-
-### Model Description
-
-<!-- Provide a longer summary of what this model is. -->
-
-
-
-- **Developed by:** [More Information Needed]
-- **Funded by [optional]:** [More Information Needed]
-- **Shared by [optional]:** [More Information Needed]
-- **Model type:** [More Information Needed]
-- **Language(s) (NLP):** [More Information Needed]
-- **License:** [More Information Needed]
-- **Finetuned from model [optional]:** [More Information Needed]
-
-### Model Sources [optional]
-
-<!-- Provide the basic links for the model. -->
-
-- **Repository:** [More Information Needed]
-- **Paper [optional]:** [More Information Needed]
-- **Demo [optional]:** [More Information Needed]
-
-## Uses
-
-<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
-
-### Direct Use
-
-<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
-
-[More Information Needed]
-
-### Downstream Use [optional]
-
-<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
-
-[More Information Needed]
-
-### Out-of-Scope Use
-
-<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
-
-[More Information Needed]
-
-## Bias, Risks, and Limitations
-
-<!-- This section is meant to convey both technical and sociotechnical limitations. -->
-
-[More Information Needed]
-
-### Recommendations
-
-<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
-
-Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
-
-## How to Get Started with the Model
-
-Use the code below to get started with the model.
-
-[More Information Needed]
-
-## Training Details
-
-### Training Data
-
-<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
-
-[More Information Needed]
-
-### Training Procedure
-
-<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
-
-#### Preprocessing [optional]
-
-[More Information Needed]
-
-
-#### Training Hyperparameters
-
-- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
-
-#### Speeds, Sizes, Times [optional]
-
-<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
-
-[More Information Needed]
-
-## Evaluation
-
-<!-- This section describes the evaluation protocols and provides the results. -->
-
-### Testing Data, Factors & Metrics
-
-#### Testing Data
-
-<!-- This should link to a Dataset Card if possible. -->
-
-[More Information Needed]
-
-#### Factors
-
-<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
-
-[More Information Needed]
-
-#### Metrics
-
-<!-- These are the evaluation metrics being used, ideally with a description of why. -->
-
-[More Information Needed]
-
-### Results
-
-[More Information Needed]
-
-#### Summary
-
-
-
-## Model Examination [optional]
-
-<!-- Relevant interpretability work for the model goes here -->
-
-[More Information Needed]
-
-## Environmental Impact
-
-<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
-
-Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
-
-- **Hardware Type:** [More Information Needed]
-- **Hours used:** [More Information Needed]
-- **Cloud Provider:** [More Information Needed]
-- **Compute Region:** [More Information Needed]
-- **Carbon Emitted:** [More Information Needed]
-
-## Technical Specifications [optional]
-
-### Model Architecture and Objective
-
-[More Information Needed]
-
-### Compute Infrastructure
-
-[More Information Needed]
-
-#### Hardware
-
-[More Information Needed]
-
-#### Software
-
-[More Information Needed]
-
-## Citation [optional]
-
-<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
-
-**BibTeX:**
-
-[More Information Needed]
-
-**APA:**
-
-[More Information Needed]
-
-## Glossary [optional]
-
-<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
-
-[More Information Needed]
-
-## More Information [optional]
-
-[More Information Needed]
-
-## Model Card Authors [optional]
-
-[More Information Needed]
-
-## Model Card Contact
-
-[More Information Needed]
-### Framework versions
-
-- PEFT 0.18.1
\ No newline at end of file
diff --git a/starcoder2-7b/dp3/epochs/epoch_001/adapter/adapter_config.json b/starcoder2-7b/dp3/epochs/epoch_001/adapter/adapter_config.json
deleted file mode 100644
index 0a76df9001e73d14444a2de3034d62c2c7986520..0000000000000000000000000000000000000000
--- a/starcoder2-7b/dp3/epochs/epoch_001/adapter/adapter_config.json
+++ /dev/null
@@ -1,46 +0,0 @@
-{
-  "alora_invocation_tokens": null,
-  "alpha_pattern": {},
-  "arrow_config": null,
-  "auto_mapping": null,
-  "base_model_name_or_path": "bigcode/starcoder2-7b",
-  "bias": "none",
-  "corda_config": null,
-  "ensure_weight_tying": true,
-  "eva_config": null,
-  "exclude_modules": null,
-  "fan_in_fan_out": false,
-  "inference_mode": true,
-  "init_lora_weights": true,
-  "layer_replication": null,
-  "layers_pattern": null,
-  "layers_to_transform": null,
-  "loftq_config": {},
-  "lora_alpha": 32,
-  "lora_bias": false,
-  "lora_dropout": 0.05,
-  "megatron_config": null,
-  "megatron_core": "megatron.core",
-  "modules_to_save": [
-    "lm_head",
-    "embed_tokens"
-  ],
-  "peft_type": "LORA",
-  "peft_version": "0.18.1",
-  "qalora_group_size": 16,
-  "r": 16,
-  "rank_pattern": {},
-  "revision": null,
-  "target_modules": [
-    "o_proj",
-    "k_proj",
-    "v_proj",
-    "q_proj"
-  ],
-  "target_parameters": null,
-  "task_type": "CAUSAL_LM",
-  "trainable_token_indices": null,
-  "use_dora": false,
-  "use_qalora": false,
-  "use_rslora": false
-}
\ No newline at end of file
diff --git a/starcoder2-7b/dp3/epochs/epoch_001/audit_results.json b/starcoder2-7b/dp3/epochs/epoch_001/audit_results.json
deleted file mode 100644
index 77802533d9338d9f34202149bb65d0cb1e8e9717..0000000000000000000000000000000000000000
--- a/starcoder2-7b/dp3/epochs/epoch_001/audit_results.json
+++ /dev/null
@@ -1,137 +0,0 @@
-{
-  "delta": 1e-05,
-  "num_canaries": 500,
-  "num_members": 250,
-  "paper_guess_fraction": 0.2,
-  "paper_guess_steps": 20,
-  "loss": {
-    "auc": 0.530384,
-    "empirical_epsilon": {
-      "0.05": 0.08366204984486103,
-      "0.01": 0.0
-    },
-    "empirical_epsilon_details": {
-      "0.05": {
-        "epsilon": 0.08366204984486103,
-        "num_guesses": 95,
-        "correct_guesses": 58,
-        "candidate_num_guesses": [
-          5,
-          10,
-          15,
-          20,
-          25,
-          30,
-          35,
-          40,
-          45,
-          50,
-          55,
-          60,
-          65,
-          70,
-          75,
-          80,
-          85,
-          90,
-          95,
-          100
-        ],
-        "direction": "higher"
-      },
-      "0.01": {
-        "epsilon": 0.0,
-        "num_guesses": 0,
-        "correct_guesses": 0,
-        "candidate_num_guesses": [
-          5,
-          10,
-          15,
-          20,
-          25,
-          30,
-          35,
-          40,
-          45,
-          50,
-          55,
-          60,
-          65,
-          70,
-          75,
-          80,
-          85,
-          90,
-          95,
-          100
-        ],
-        "direction": "lower"
-      }
-    }
-  },
-  "embedding": {
-    "auc": 0.531032,
-    "empirical_epsilon": {
-      "0.05": 0.0,
-      "0.01": 0.0
-    },
-    "empirical_epsilon_details": {
-      "0.05": {
-        "epsilon": 0.0,
-        "num_guesses": 0,
-        "correct_guesses": 0,
-        "candidate_num_guesses": [
-          5,
-          10,
-          15,
-          20,
-          25,
-          30,
-          35,
-          40,
-          45,
-          50,
-          55,
-          60,
-          65,
-          70,
-          75,
-          80,
-          85,
-          90,
-          95,
-          100
-        ],
-        "direction": "lower"
-      },
-      "0.01": {
-        "epsilon": 0.0,
-        "num_guesses": 0,
-        "correct_guesses": 0,
-        "candidate_num_guesses": [
-          5,
-          10,
-          15,
-          20,
-          25,
-          30,
-          35,
-          40,
-          45,
-          50,
-          55,
-          60,
-          65,
-          70,
-          75,
-          80,
-          85,
-          90,
-          95,
-          100
-        ],
-        "direction": "lower"
-      }
-    }
-  }
-}
\ No newline at end of file
diff --git a/starcoder2-7b/dp3/epochs/epoch_002/adapter/README.md b/starcoder2-7b/dp3/epochs/epoch_002/adapter/README.md
deleted file mode 100644
index 9f7d6e699b0d658ac4c4291a23a0dafd21d1d0b8..0000000000000000000000000000000000000000
--- a/starcoder2-7b/dp3/epochs/epoch_002/adapter/README.md
+++ /dev/null
@@ -1,207 +0,0 @@
----
-base_model: bigcode/starcoder2-7b
-library_name: peft
-pipeline_tag: text-generation
-tags:
-- base_model:adapter:bigcode/starcoder2-7b
-- lora
-- transformers
----
-
-# Model Card for Model ID
-
-<!-- Provide a quick summary of what the model is/does. -->
-
-
-
-## Model Details
-
-### Model Description
-
-<!-- Provide a longer summary of what this model is. -->
-
-
-
-- **Developed by:** [More Information Needed]
-- **Funded by [optional]:** [More Information Needed]
-- **Shared by [optional]:** [More Information Needed]
-- **Model type:** [More Information Needed]
-- **Language(s) (NLP):** [More Information Needed]
-- **License:** [More Information Needed]
-- **Finetuned from model [optional]:** [More Information Needed]
-
-### Model Sources [optional]
-
-<!-- Provide the basic links for the model. -->
-
-- **Repository:** [More Information Needed]
-- **Paper [optional]:** [More Information Needed]
-- **Demo [optional]:** [More Information Needed]
-
-## Uses
-
-<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
-
-### Direct Use
-
-<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
-
-[More Information Needed]
-
-### Downstream Use [optional]
-
-<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
-
-[More Information Needed]
-
-### Out-of-Scope Use
-
-<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
-
-[More Information Needed]
-
-## Bias, Risks, and Limitations
-
-<!-- This section is meant to convey both technical and sociotechnical limitations. -->
-
-[More Information Needed]
-
-### Recommendations
-
-<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
-
-Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
-
-## How to Get Started with the Model
-
-Use the code below to get started with the model.
-
-[More Information Needed]
-
-## Training Details
-
-### Training Data
-
-<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
-
-[More Information Needed]
-
-### Training Procedure
-
-<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
-
-#### Preprocessing [optional]
-
-[More Information Needed]
-
-
-#### Training Hyperparameters
-
-- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
-
-#### Speeds, Sizes, Times [optional]
-
-<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
-
-[More Information Needed]
-
-## Evaluation
-
-<!-- This section describes the evaluation protocols and provides the results. -->
-
-### Testing Data, Factors & Metrics
-
-#### Testing Data
-
-<!-- This should link to a Dataset Card if possible. -->
-
-[More Information Needed]
-
-#### Factors
-
-<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
-
-[More Information Needed]
-
-#### Metrics
-
-<!-- These are the evaluation metrics being used, ideally with a description of why. -->
-
-[More Information Needed]
-
-### Results
-
-[More Information Needed]
-
-#### Summary
-
-
-
-## Model Examination [optional]
-
-<!-- Relevant interpretability work for the model goes here -->
-
-[More Information Needed]
-
-## Environmental Impact
-
-<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
-
-Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
-
-- **Hardware Type:** [More Information Needed]
-- **Hours used:** [More Information Needed]
-- **Cloud Provider:** [More Information Needed]
-- **Compute Region:** [More Information Needed]
-- **Carbon Emitted:** [More Information Needed]
-
-## Technical Specifications [optional]
-
-### Model Architecture and Objective
-
-[More Information Needed]
-
-### Compute Infrastructure
-
-[More Information Needed]
-
-#### Hardware
-
-[More Information Needed]
-
-#### Software
-
-[More Information Needed]
-
-## Citation [optional]
-
-<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
-
-**BibTeX:**
-
-[More Information Needed]
-
-**APA:**
-
-[More Information Needed]
-
-## Glossary [optional]
-
-<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
-
-[More Information Needed]
-
-## More Information [optional]
-
-[More Information Needed]
-
-## Model Card Authors [optional]
-
-[More Information Needed]
-
-## Model Card Contact
-
-[More Information Needed]
-### Framework versions
-
-- PEFT 0.18.1
\ No newline at end of file
diff --git a/starcoder2-7b/dp3/epochs/epoch_002/adapter/adapter_config.json b/starcoder2-7b/dp3/epochs/epoch_002/adapter/adapter_config.json
deleted file mode 100644
index 0a76df9001e73d14444a2de3034d62c2c7986520..0000000000000000000000000000000000000000
--- a/starcoder2-7b/dp3/epochs/epoch_002/adapter/adapter_config.json
+++ /dev/null
@@ -1,46 +0,0 @@
-{
-  "alora_invocation_tokens": null,
-  "alpha_pattern": {},
-  "arrow_config": null,
-  "auto_mapping": null,
-  "base_model_name_or_path": "bigcode/starcoder2-7b",
-  "bias": "none",
-  "corda_config": null,
-  "ensure_weight_tying": true,
-  "eva_config": null,
-  "exclude_modules": null,
-  "fan_in_fan_out": false,
-  "inference_mode": true,
-  "init_lora_weights": true,
-  "layer_replication": null,
-  "layers_pattern": null,
-  "layers_to_transform": null,
-  "loftq_config": {},
-  "lora_alpha": 32,
-  "lora_bias": false,
-  "lora_dropout": 0.05,
-  "megatron_config": null,
-  "megatron_core": "megatron.core",
-  "modules_to_save": [
-    "lm_head",
-    "embed_tokens"
-  ],
-  "peft_type": "LORA",
-  "peft_version": "0.18.1",
-  "qalora_group_size": 16,
-  "r": 16,
-  "rank_pattern": {},
-  "revision": null,
-  "target_modules": [
-    "o_proj",
-    "k_proj",
-    "v_proj",
-    "q_proj"
-  ],
-  "target_parameters": null,
-  "task_type": "CAUSAL_LM",
-  "trainable_token_indices": null,
-  "use_dora": false,
-  "use_qalora": false,
-  "use_rslora": false
-}
\ No newline at end of file
diff --git a/starcoder2-7b/dp3/epochs/epoch_002/audit_results.json b/starcoder2-7b/dp3/epochs/epoch_002/audit_results.json
deleted file mode 100644
index aa86893e3c0f54cef51efbe18473d6f2c3cf2fb0..0000000000000000000000000000000000000000
--- a/starcoder2-7b/dp3/epochs/epoch_002/audit_results.json
+++ /dev/null
@@ -1,137 +0,0 @@
-{
-  "delta": 1e-05,
-  "num_canaries": 500,
-  "num_members": 250,
-  "paper_guess_fraction": 0.2,
-  "paper_guess_steps": 20,
-  "loss": {
-    "auc": 0.529296,
-    "empirical_epsilon": {
-      "0.05": 0.09125839080661535,
-      "0.01": 0.0
-    },
-    "empirical_epsilon_details": {
-      "0.05": {
-        "epsilon": 0.09125839080661535,
-        "num_guesses": 100,
-        "correct_guesses": 61,
-        "candidate_num_guesses": [
-          5,
-          10,
-          15,
-          20,
-          25,
-          30,
-          35,
-          40,
-          45,
-          50,
-          55,
-          60,
-          65,
-          70,
-          75,
-          80,
-          85,
-          90,
-          95,
-          100
-        ],
-        "direction": "higher"
-      },
-      "0.01": {
-        "epsilon": 0.0,
-        "num_guesses": 0,
-        "correct_guesses": 0,
-        "candidate_num_guesses": [
-          5,
-          10,
-          15,
-          20,
-          25,
-          30,
-          35,
-          40,
-          45,
-          50,
-          55,
-          60,
-          65,
-          70,
-          75,
-          80,
-          85,
-          90,
-          95,
-          100
-        ],
-        "direction": "lower"
-      }
-    }
-  },
-  "embedding": {
-    "auc": 0.542392,
-    "empirical_epsilon": {
-      "0.05": 0.0,
-      "0.01": 0.0
-    },
-    "empirical_epsilon_details": {
-      "0.05": {
-        "epsilon": 0.0,
-        "num_guesses": 0,
-        "correct_guesses": 0,
-        "candidate_num_guesses": [
-          5,
-          10,
-          15,
-          20,
-          25,
-          30,
-          35,
-          40,
-          45,
-          50,
-          55,
-          60,
-          65,
-          70,
-          75,
-          80,
-          85,
-          90,
-          95,
-          100
-        ],
-        "direction": "lower"
-      },
-      "0.01": {
-        "epsilon": 0.0,
-        "num_guesses": 0,
-        "correct_guesses": 0,
-        "candidate_num_guesses": [
-          5,
-          10,
-          15,
-          20,
-          25,
-          30,
-          35,
-          40,
-          45,
-          50,
-          55,
-          60,
-          65,
-          70,
-          75,
-          80,
-          85,
-          90,
-          95,
-          100
-        ],
-        "direction": "lower"
-      }
-    }
-  }
-}
\ No newline at end of file
diff --git a/starcoder2-7b/dp3/epochs/epoch_003/adapter/README.md b/starcoder2-7b/dp3/epochs/epoch_003/adapter/README.md
deleted file mode 100644
index 9f7d6e699b0d658ac4c4291a23a0dafd21d1d0b8..0000000000000000000000000000000000000000
--- a/starcoder2-7b/dp3/epochs/epoch_003/adapter/README.md
+++ /dev/null
@@ -1,207 +0,0 @@
----
-base_model: bigcode/starcoder2-7b
-library_name: peft
-pipeline_tag: text-generation
-tags:
-- base_model:adapter:bigcode/starcoder2-7b
-- lora
-- transformers
----
-
-# Model Card for Model ID
-
-<!-- Provide a quick summary of what the model is/does. -->
-
-
-
-## Model Details
-
-### Model Description
-
-<!-- Provide a longer summary of what this model is. -->
-
-
-
-- **Developed by:** [More Information Needed]
-- **Funded by [optional]:** [More Information Needed]
-- **Shared by [optional]:** [More Information Needed]
-- **Model type:** [More Information Needed]
-- **Language(s) (NLP):** [More Information Needed]
-- **License:** [More Information Needed]
-- **Finetuned from model [optional]:** [More Information Needed]
-
-### Model Sources [optional]
-
-<!-- Provide the basic links for the model. -->
-
-- **Repository:** [More Information Needed]
-- **Paper [optional]:** [More Information Needed]
-- **Demo [optional]:** [More Information Needed]
-
-## Uses
-
-<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
-
-### Direct Use
-
-<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
-
-[More Information Needed]
-
-### Downstream Use [optional]
-
-<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
-
-[More Information Needed]
-
-### Out-of-Scope Use
-
-<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
-
-[More Information Needed]
-
-## Bias, Risks, and Limitations
-
-<!-- This section is meant to convey both technical and sociotechnical limitations. -->
-
-[More Information Needed]
-
-### Recommendations
-
-<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
-
-Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
-
-## How to Get Started with the Model
-
-Use the code below to get started with the model.
-
-[More Information Needed]
-
-## Training Details
-
-### Training Data
-
-<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
-
-[More Information Needed]
-
-### Training Procedure
-
-<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
-
-#### Preprocessing [optional]
-
-[More Information Needed]
-
-
-#### Training Hyperparameters
-
-- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
-
-#### Speeds, Sizes, Times [optional]
-
-<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
-
-[More Information Needed]
-
-## Evaluation
-
-<!-- This section describes the evaluation protocols and provides the results. -->
-
-### Testing Data, Factors & Metrics
-
-#### Testing Data
-
-<!-- This should link to a Dataset Card if possible. -->
-
-[More Information Needed]
-
-#### Factors
-
-<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
-
-[More Information Needed]
-
-#### Metrics
-
-<!-- These are the evaluation metrics being used, ideally with a description of why. -->
-
-[More Information Needed]
-
-### Results
-
-[More Information Needed]
-
-#### Summary
-
-
-
-## Model Examination [optional]
-
-<!-- Relevant interpretability work for the model goes here -->
-
-[More Information Needed]
-
-## Environmental Impact
-
-<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
-
-Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
-
-- **Hardware Type:** [More Information Needed]
-- **Hours used:** [More Information Needed]
-- **Cloud Provider:** [More Information Needed]
-- **Compute Region:** [More Information Needed]
-- **Carbon Emitted:** [More Information Needed]
-
-## Technical Specifications [optional]
-
-### Model Architecture and Objective
-
-[More Information Needed]
-
-### Compute Infrastructure
-
-[More Information Needed]
-
-#### Hardware
-
-[More Information Needed]
-
-#### Software
-
-[More Information Needed]
-
-## Citation [optional]
-
-<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
-
-**BibTeX:**
-
-[More Information Needed]
-
-**APA:**
-
-[More Information Needed]
-
-## Glossary [optional]
-
-<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
-
-[More Information Needed]
-
-## More Information [optional]
-
-[More Information Needed]
-
-## Model Card Authors [optional]
-
-[More Information Needed]
-
-## Model Card Contact
-
-[More Information Needed]
-### Framework versions
-
-- PEFT 0.18.1
\ No newline at end of file
diff --git a/starcoder2-7b/dp3/epochs/epoch_003/adapter/adapter_config.json b/starcoder2-7b/dp3/epochs/epoch_003/adapter/adapter_config.json
deleted file mode 100644
index 0a76df9001e73d14444a2de3034d62c2c7986520..0000000000000000000000000000000000000000
--- a/starcoder2-7b/dp3/epochs/epoch_003/adapter/adapter_config.json
+++ /dev/null
@@ -1,46 +0,0 @@
-{
-  "alora_invocation_tokens": null,
-  "alpha_pattern": {},
-  "arrow_config": null,
-  "auto_mapping": null,
-  "base_model_name_or_path": "bigcode/starcoder2-7b",
-  "bias": "none",
-  "corda_config": null,
-  "ensure_weight_tying": true,
-  "eva_config": null,
-  "exclude_modules": null,
-  "fan_in_fan_out": false,
-  "inference_mode": true,
-  "init_lora_weights": true,
-  "layer_replication": null,
-  "layers_pattern": null,
-  "layers_to_transform": null,
-  "loftq_config": {},
-  "lora_alpha": 32,
-  "lora_bias": false,
-  "lora_dropout": 0.05,
-  "megatron_config": null,
-  "megatron_core": "megatron.core",
-  "modules_to_save": [
-    "lm_head",
-    "embed_tokens"
-  ],
-  "peft_type": "LORA",
-  "peft_version": "0.18.1",
-  "qalora_group_size": 16,
-  "r": 16,
-  "rank_pattern": {},
-  "revision": null,
-  "target_modules": [
-    "o_proj",
-    "k_proj",
-    "v_proj",
-    "q_proj"
-  ],
-  "target_parameters": null,
-  "task_type": "CAUSAL_LM",
-  "trainable_token_indices": null,
-  "use_dora": false,
-  "use_qalora": false,
-  "use_rslora": false
-}
\ No newline at end of file
diff --git a/starcoder2-7b/dp3/epochs/epoch_003/audit_results.json b/starcoder2-7b/dp3/epochs/epoch_003/audit_results.json
deleted file mode 100644
index 09b200140fb2b302f5524acfa8c6f01337d152e7..0000000000000000000000000000000000000000
--- a/starcoder2-7b/dp3/epochs/epoch_003/audit_results.json
+++ /dev/null
@@ -1,137 +0,0 @@
-{
-  "delta": 1e-05,
-  "num_canaries": 500,
-  "num_members": 250,
-  "paper_guess_fraction": 0.2,
-  "paper_guess_steps": 20,
-  "loss": {
-    "auc": 0.528976,
-    "empirical_epsilon": {
-      "0.05": 0.08366204984486103,
-      "0.01": 0.0
-    },
-    "empirical_epsilon_details": {
-      "0.05": {
-        "epsilon": 0.08366204984486103,
-        "num_guesses": 95,
-        "correct_guesses": 58,
-        "candidate_num_guesses": [
-          5,
-          10,
-          15,
-          20,
-          25,
-          30,
-          35,
-          40,
-          45,
-          50,
-          55,
-          60,
-          65,
-          70,
-          75,
-          80,
-          85,
-          90,
-          95,
-          100
-        ],
-        "direction": "higher"
-      },
-      "0.01": {
-        "epsilon": 0.0,
-        "num_guesses": 0,
-        "correct_guesses": 0,
-        "candidate_num_guesses": [
-          5,
-          10,
-          15,
-          20,
-          25,
-          30,
-          35,
-          40,
-          45,
-          50,
-          55,
-          60,
-          65,
-          70,
-          75,
-          80,
-          85,
-          90,
-          95,
-          100
-        ],
-        "direction": "lower"
-      }
-    }
-  },
-  "embedding": {
-    "auc": 0.540056,
-    "empirical_epsilon": {
-      "0.05": 0.0,
-      "0.01": 0.0
-    },
-    "empirical_epsilon_details": {
-      "0.05": {
-        "epsilon": 0.0,
-        "num_guesses": 0,
-        "correct_guesses": 0,
-        "candidate_num_guesses": [
-          5,
-          10,
-          15,
-          20,
-          25,
-          30,
-          35,
-          40,
-          45,
-          50,
-          55,
-          60,
-          65,
-          70,
-          75,
-          80,
-          85,
-          90,
-          95,
-          100
-        ],
-        "direction": "lower"
-      },
-      "0.01": {
-        "epsilon": 0.0,
-        "num_guesses": 0,
-        "correct_guesses": 0,
-        "candidate_num_guesses": [
-          5,
-          10,
-          15,
-          20,
-          25,
-          30,
-          35,
-          40,
-          45,
-          50,
-          55,
-          60,
-          65,
-          70,
-          75,
-          80,
-          85,
-          90,
-          95,
-          100
-        ],
-        "direction": "lower"
-      }
-    }
-  }
-}
\ No newline at end of file
diff --git a/starcoder2-7b/dp8/epochs/epoch_001/adapter/README.md b/starcoder2-7b/dp8/epochs/epoch_001/adapter/README.md
deleted file mode 100644
index 9f7d6e699b0d658ac4c4291a23a0dafd21d1d0b8..0000000000000000000000000000000000000000
--- a/starcoder2-7b/dp8/epochs/epoch_001/adapter/README.md
+++ /dev/null
@@ -1,207 +0,0 @@
----
-base_model: bigcode/starcoder2-7b
-library_name: peft
-pipeline_tag: text-generation
-tags:
-- base_model:adapter:bigcode/starcoder2-7b
-- lora
-- transformers
----
-
-# Model Card for Model ID
-
-<!-- Provide a quick summary of what the model is/does. -->
-
-
-
-## Model Details
-
-### Model Description
-
-<!-- Provide a longer summary of what this model is. -->
-
-
-
-- **Developed by:** [More Information Needed]
-- **Funded by [optional]:** [More Information Needed]
-- **Shared by [optional]:** [More Information Needed]
-- **Model type:** [More Information Needed]
-- **Language(s) (NLP):** [More Information Needed]
-- **License:** [More Information Needed]
-- **Finetuned from model [optional]:** [More Information Needed]
-
-### Model Sources [optional]
-
-<!-- Provide the basic links for the model. -->
-
-- **Repository:** [More Information Needed]
-- **Paper [optional]:** [More Information Needed]
-- **Demo [optional]:** [More Information Needed]
-
-## Uses
-
-<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
-
-### Direct Use
-
-<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
-
-[More Information Needed]
-
-### Downstream Use [optional]
-
-<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
-
-[More Information Needed]
-
-### Out-of-Scope Use
-
-<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
-
-[More Information Needed]
-
-## Bias, Risks, and Limitations
-
-<!-- This section is meant to convey both technical and sociotechnical limitations. -->
-
-[More Information Needed]
-
-### Recommendations
-
-<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
-
-Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
-
-## How to Get Started with the Model
-
-Use the code below to get started with the model.
-
-[More Information Needed]
-
-## Training Details
-
-### Training Data
-
-<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
-
-[More Information Needed]
-
-### Training Procedure
-
-<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
-
-#### Preprocessing [optional]
-
-[More Information Needed]
-
-
-#### Training Hyperparameters
-
-- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
-
-#### Speeds, Sizes, Times [optional]
-
-<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
-
-[More Information Needed]
-
-## Evaluation
-
-<!-- This section describes the evaluation protocols and provides the results. -->
-
-### Testing Data, Factors & Metrics
-
-#### Testing Data
-
-<!-- This should link to a Dataset Card if possible. -->
-
-[More Information Needed]
-
-#### Factors
-
-<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
-
-[More Information Needed]
-
-#### Metrics
-
-<!-- These are the evaluation metrics being used, ideally with a description of why. -->
-
-[More Information Needed]
-
-### Results
-
-[More Information Needed]
-
-#### Summary
-
-
-
-## Model Examination [optional]
-
-<!-- Relevant interpretability work for the model goes here -->
-
-[More Information Needed]
-
-## Environmental Impact
-
-<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
-
-Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
-
-- **Hardware Type:** [More Information Needed]
-- **Hours used:** [More Information Needed]
-- **Cloud Provider:** [More Information Needed]
-- **Compute Region:** [More Information Needed]
-- **Carbon Emitted:** [More Information Needed]
-
-## Technical Specifications [optional]
-
-### Model Architecture and Objective
-
-[More Information Needed]
-
-### Compute Infrastructure
-
-[More Information Needed]
-
-#### Hardware
-
-[More Information Needed]
-
-#### Software
-
-[More Information Needed]
-
-## Citation [optional]
-
-<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
-
-**BibTeX:**
-
-[More Information Needed]
-
-**APA:**
-
-[More Information Needed]
-
-## Glossary [optional]
-
-<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
-
-[More Information Needed]
-
-## More Information [optional]
-
-[More Information Needed]
-
-## Model Card Authors [optional]
-
-[More Information Needed]
-
-## Model Card Contact
-
-[More Information Needed]
-### Framework versions
-
-- PEFT 0.18.1
\ No newline at end of file
diff --git a/starcoder2-7b/dp8/epochs/epoch_001/adapter/adapter_config.json b/starcoder2-7b/dp8/epochs/epoch_001/adapter/adapter_config.json
deleted file mode 100644
index 4d946b27ca1339ce45b035b190c4cf07ce83dec0..0000000000000000000000000000000000000000
--- a/starcoder2-7b/dp8/epochs/epoch_001/adapter/adapter_config.json
+++ /dev/null
@@ -1,46 +0,0 @@
-{
-  "alora_invocation_tokens": null,
-  "alpha_pattern": {},
-  "arrow_config": null,
-  "auto_mapping": null,
-  "base_model_name_or_path": "bigcode/starcoder2-7b",
-  "bias": "none",
-  "corda_config": null,
-  "ensure_weight_tying": true,
-  "eva_config": null,
-  "exclude_modules": null,
-  "fan_in_fan_out": false,
-  "inference_mode": true,
-  "init_lora_weights": true,
-  "layer_replication": null,
-  "layers_pattern": null,
-  "layers_to_transform": null,
-  "loftq_config": {},
-  "lora_alpha": 32,
-  "lora_bias": false,
-  "lora_dropout": 0.05,
-  "megatron_config": null,
-  "megatron_core": "megatron.core",
-  "modules_to_save": [
-    "lm_head",
-    "embed_tokens"
-  ],
-  "peft_type": "LORA",
-  "peft_version": "0.18.1",
-  "qalora_group_size": 16,
-  "r": 16,
-  "rank_pattern": {},
-  "revision": null,
-  "target_modules": [
-    "q_proj",
-    "k_proj",
-    "v_proj",
-    "o_proj"
-  ],
-  "target_parameters": null,
-  "task_type": "CAUSAL_LM",
-  "trainable_token_indices": null,
-  "use_dora": false,
-  "use_qalora": false,
-  "use_rslora": false
-}
\ No newline at end of file
diff --git a/starcoder2-7b/dp8/epochs/epoch_001/audit_results.json b/starcoder2-7b/dp8/epochs/epoch_001/audit_results.json
deleted file mode 100644
index efb234ed9732ffde20d6740f79ec807f1481f122..0000000000000000000000000000000000000000
--- a/starcoder2-7b/dp8/epochs/epoch_001/audit_results.json
+++ /dev/null
@@ -1,137 +0,0 @@
-{
-  "delta": 1e-05,
-  "num_canaries": 500,
-  "num_members": 250,
-  "paper_guess_fraction": 0.2,
-  "paper_guess_steps": 20,
-  "loss": {
-    "auc": 0.529592,
-    "empirical_epsilon": {
-      "0.05": 0.007771402597427368,
-      "0.01": 0.0
-    },
-    "empirical_epsilon_details": {
-      "0.05": {
-        "epsilon": 0.007771402597427368,
-        "num_guesses": 60,
-        "correct_guesses": 37,
-        "candidate_num_guesses": [
-          5,
-          10,
-          15,
-          20,
-          25,
-          30,
-          35,
-          40,
-          45,
-          50,
-          55,
-          60,
-          65,
-          70,
-          75,
-          80,
-          85,
-          90,
-          95,
-          100
-        ],
-        "direction": "higher"
-      },
-      "0.01": {
-        "epsilon": 0.0,
-        "num_guesses": 0,
-        "correct_guesses": 0,
-        "candidate_num_guesses": [
-          5,
-          10,
-          15,
-          20,
-          25,
-          30,
-          35,
-          40,
-          45,
-          50,
-          55,
-          60,
-          65,
-          70,
-          75,
-          80,
-          85,
-          90,
-          95,
-          100
-        ],
-        "direction": "lower"
-      }
-    }
-  },
-  "embedding": {
-    "auc": 0.503408,
-    "empirical_epsilon": {
-      "0.05": 0.0,
-      "0.01": 0.0
-    },
-    "empirical_epsilon_details": {
-      "0.05": {
-        "epsilon": 0.0,
-        "num_guesses": 0,
-        "correct_guesses": 0,
-        "candidate_num_guesses": [
-          5,
-          10,
-          15,
-          20,
-          25,
-          30,
-          35,
-          40,
-          45,
-          50,
-          55,
-          60,
-          65,
-          70,
-          75,
-          80,
-          85,
-          90,
-          95,
-          100
-        ],
-        "direction": "lower"
-      },
-      "0.01": {
-        "epsilon": 0.0,
-        "num_guesses": 0,
-        "correct_guesses": 0,
-        "candidate_num_guesses": [
-          5,
-          10,
-          15,
-          20,
-          25,
-          30,
-          35,
-          40,
-          45,
-          50,
-          55,
-          60,
-          65,
-          70,
-          75,
-          80,
-          85,
-          90,
-          95,
-          100
-        ],
-        "direction": "lower"
-      }
-    }
-  }
-}
\ No newline at end of file
diff --git a/starcoder2-7b/dp8/epochs/epoch_002/adapter/README.md b/starcoder2-7b/dp8/epochs/epoch_002/adapter/README.md
deleted file mode 100644
index 9f7d6e699b0d658ac4c4291a23a0dafd21d1d0b8..0000000000000000000000000000000000000000
--- a/starcoder2-7b/dp8/epochs/epoch_002/adapter/README.md
+++ /dev/null
@@ -1,207 +0,0 @@
----
-base_model: bigcode/starcoder2-7b
-library_name: peft
-pipeline_tag: text-generation
-tags:
-- base_model:adapter:bigcode/starcoder2-7b
-- lora
-- transformers
----
-
-# Model Card for Model ID
-
-<!-- Provide a quick summary of what the model is/does. -->
-
-
-
-## Model Details
-
-### Model Description
-
-<!-- Provide a longer summary of what this model is. -->
-
-
-
-- **Developed by:** [More Information Needed]
-- **Funded by [optional]:** [More Information Needed]
-- **Shared by [optional]:** [More Information Needed]
-- **Model type:** [More Information Needed]
-- **Language(s) (NLP):** [More Information Needed]
-- **License:** [More Information Needed]
-- **Finetuned from model [optional]:** [More Information Needed]
-
-### Model Sources [optional]
-
-<!-- Provide the basic links for the model. -->
-
-- **Repository:** [More Information Needed]
-- **Paper [optional]:** [More Information Needed]
-- **Demo [optional]:** [More Information Needed]
-
-## Uses
-
-<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
-
-### Direct Use
-
-<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
-
-[More Information Needed]
-
-### Downstream Use [optional]
-
-<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
-
-[More Information Needed]
-
-### Out-of-Scope Use
-
-<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
-
-[More Information Needed]
-
-## Bias, Risks, and Limitations
-
-<!-- This section is meant to convey both technical and sociotechnical limitations. -->
-
-[More Information Needed]
-
-### Recommendations
-
-<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
-
-Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
-
-## How to Get Started with the Model
-
-Use the code below to get started with the model.
-
-[More Information Needed]
-
-## Training Details
-
-### Training Data
-
-<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
-
-[More Information Needed]
-
-### Training Procedure
-
-<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
-
-#### Preprocessing [optional]
-
-[More Information Needed]
-
-
-#### Training Hyperparameters
-
-- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
-
-#### Speeds, Sizes, Times [optional]
-
-<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
-
-[More Information Needed]
-
-## Evaluation
-
-<!-- This section describes the evaluation protocols and provides the results. -->
-
-### Testing Data, Factors & Metrics
-
-#### Testing Data
-
-<!-- This should link to a Dataset Card if possible. -->
-
-[More Information Needed]
-
-#### Factors
-
-<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
-
-[More Information Needed]
-
-#### Metrics
-
-<!-- These are the evaluation metrics being used, ideally with a description of why. -->
-
-[More Information Needed]
-
-### Results
-
-[More Information Needed]
-
-#### Summary
-
-
-
-## Model Examination [optional]
-
-<!-- Relevant interpretability work for the model goes here -->
-
-[More Information Needed]
-
-## Environmental Impact
-
-<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
-
-Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
-
-- **Hardware Type:** [More Information Needed]
-- **Hours used:** [More Information Needed]
-- **Cloud Provider:** [More Information Needed]
-- **Compute Region:** [More Information Needed]
-- **Carbon Emitted:** [More Information Needed]
-
-## Technical Specifications [optional]
-
-### Model Architecture and Objective
-
-[More Information Needed]
-
-### Compute Infrastructure
-
-[More Information Needed]
-
-#### Hardware
-
-[More Information Needed]
-
-#### Software
-
-[More Information Needed]
-
-## Citation [optional]
-
-<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
-
-**BibTeX:**
-
-[More Information Needed]
-
-**APA:**
-
-[More Information Needed]
-
-## Glossary [optional]
-
-<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
-
-[More Information Needed]
-
-## More Information [optional]
-
-[More Information Needed]
-
-## Model Card Authors [optional]
-
-[More Information Needed]
-
-## Model Card Contact
-
-[More Information Needed]
-### Framework versions
-
-- PEFT 0.18.1
\ No newline at end of file
diff --git a/starcoder2-7b/dp8/epochs/epoch_002/adapter/adapter_config.json b/starcoder2-7b/dp8/epochs/epoch_002/adapter/adapter_config.json
deleted file mode 100644
index 4d946b27ca1339ce45b035b190c4cf07ce83dec0..0000000000000000000000000000000000000000
--- a/starcoder2-7b/dp8/epochs/epoch_002/adapter/adapter_config.json
+++ /dev/null
@@ -1,46 +0,0 @@
-{
-  "alora_invocation_tokens": null,
-  "alpha_pattern": {},
-  "arrow_config": null,
-  "auto_mapping": null,
-  "base_model_name_or_path": "bigcode/starcoder2-7b",
-  "bias": "none",
-  "corda_config": null,
-  "ensure_weight_tying": true,
-  "eva_config": null,
-  "exclude_modules": null,
-  "fan_in_fan_out": false,
-  "inference_mode": true,
-  "init_lora_weights": true,
-  "layer_replication": null,
-  "layers_pattern": null,
-  "layers_to_transform": null,
-  "loftq_config": {},
-  "lora_alpha": 32,
-  "lora_bias": false,
-  "lora_dropout": 0.05,
-  "megatron_config": null,
-  "megatron_core": "megatron.core",
-  "modules_to_save": [
-    "lm_head",
-    "embed_tokens"
-  ],
-  "peft_type": "LORA",
-  "peft_version": "0.18.1",
-  "qalora_group_size": 16,
-  "r": 16,
-  "rank_pattern": {},
-  "revision": null,
-  "target_modules": [
-    "q_proj",
-    "k_proj",
-    "v_proj",
-    "o_proj"
-  ],
-  "target_parameters": null,
-  "task_type": "CAUSAL_LM",
-  "trainable_token_indices": null,
-  "use_dora": false,
-  "use_qalora": false,
-  "use_rslora": false
-}
\ No newline at end of file
diff --git a/starcoder2-7b/dp8/epochs/epoch_002/audit_results.json b/starcoder2-7b/dp8/epochs/epoch_002/audit_results.json
deleted file mode 100644
index 9f93bc5ec06ce4c904b0c0c3e153ba10ac40e25d..0000000000000000000000000000000000000000
--- a/starcoder2-7b/dp8/epochs/epoch_002/audit_results.json
+++ /dev/null
@@ -1,137 +0,0 @@
-{
-  "delta": 1e-05,
-  "num_canaries": 500,
-  "num_members": 250,
-  "paper_guess_fraction": 0.2,
-  "paper_guess_steps": 20,
-  "loss": {
-    "auc": 0.529288,
-    "empirical_epsilon": {
-      "0.05": 0.010390725918114185,
-      "0.01": 0.0
-    },
-    "empirical_epsilon_details": {
-      "0.05": {
-        "epsilon": 0.010390725918114185,
-        "num_guesses": 100,
-        "correct_guesses": 59,
-        "candidate_num_guesses": [
-          5,
-          10,
-          15,
-          20,
-          25,
-          30,
-          35,
-          40,
-          45,
-          50,
-          55,
-          60,
-          65,
-          70,
-          75,
-          80,
-          85,
-          90,
-          95,
-          100
-        ],
-        "direction": "higher"
-      },
-      "0.01": {
-        "epsilon": 0.0,
-        "num_guesses": 0,
-        "correct_guesses": 0,
-        "candidate_num_guesses": [
-          5,
-          10,
-          15,
-          20,
-          25,
-          30,
-          35,
-          40,
-          45,
-          50,
-          55,
-          60,
-          65,
-          70,
-          75,
-          80,
-          85,
-          90,
-          95,
-          100
-        ],
-        "direction": "lower"
-      }
-    }
-  },
-  "embedding": {
-    "auc": 0.549968,
-    "empirical_epsilon": {
-      "0.05": 0.0,
-      "0.01": 0.0
-    },
-    "empirical_epsilon_details": {
-      "0.05": {
-        "epsilon": 0.0,
-        "num_guesses": 0,
-        "correct_guesses": 0,
-        "candidate_num_guesses": [
-          5,
-          10,
-          15,
-          20,
-          25,
-          30,
-          35,
-          40,
-          45,
-          50,
-          55,
-          60,
-          65,
-          70,
-          75,
-          80,
-          85,
-          90,
-          95,
-          100
-        ],
-        "direction": "lower"
-      },
-      "0.01": {
-        "epsilon": 0.0,
-        "num_guesses": 0,
-        "correct_guesses": 0,
-        "candidate_num_guesses": [
-          5,
-          10,
-          15,
-          20,
-          25,
-          30,
-          35,
-          40,
-          45,
-          50,
-          55,
-          60,
-          65,
-          70,
-          75,
-          80,
-          85,
-          90,
-          95,
-          100
-        ],
-        "direction": "lower"
-      }
-    }
-  }
-}
\ No newline at end of file
diff --git a/starcoder2-7b/dp8/epochs/epoch_003/adapter/README.md b/starcoder2-7b/dp8/epochs/epoch_003/adapter/README.md
deleted file mode 100644
index 9f7d6e699b0d658ac4c4291a23a0dafd21d1d0b8..0000000000000000000000000000000000000000
--- a/starcoder2-7b/dp8/epochs/epoch_003/adapter/README.md
+++ /dev/null
@@ -1,207 +0,0 @@
----
-base_model: bigcode/starcoder2-7b
-library_name: peft
-pipeline_tag: text-generation
-tags:
-- base_model:adapter:bigcode/starcoder2-7b
-- lora
-- transformers
----
-
-# Model Card for Model ID
-
-<!-- Provide a quick summary of what the model is/does. -->
-
-
-
-## Model Details
-
-### Model Description
-
-<!-- Provide a longer summary of what this model is. -->
-
-
-
-- **Developed by:** [More Information Needed]
-- **Funded by [optional]:** [More Information Needed]
-- **Shared by [optional]:** [More Information Needed]
-- **Model type:** [More Information Needed]
-- **Language(s) (NLP):** [More Information Needed]
-- **License:** [More Information Needed]
-- **Finetuned from model [optional]:** [More Information Needed]
-
-### Model Sources [optional]
-
-<!-- Provide the basic links for the model. -->
-
-- **Repository:** [More Information Needed]
-- **Paper [optional]:** [More Information Needed]
-- **Demo [optional]:** [More Information Needed]
-
-## Uses
-
-<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
-
-### Direct Use
-
-<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
-
-[More Information Needed]
-
-### Downstream Use [optional]
-
-<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
-
-[More Information Needed]
-
-### Out-of-Scope Use
-
-<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
-
-[More Information Needed]
-
-## Bias, Risks, and Limitations
-
-<!-- This section is meant to convey both technical and sociotechnical limitations. -->
-
-[More Information Needed]
-
-### Recommendations
-
-<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
-
-Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
-
-## How to Get Started with the Model
-
-Use the code below to get started with the model.
-
-[More Information Needed]
-
-## Training Details
-
-### Training Data
-
-<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
-
-[More Information Needed]
-
-### Training Procedure
-
-<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
-
-#### Preprocessing [optional]
-
-[More Information Needed]
-
-
-#### Training Hyperparameters
-
-- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
-
-#### Speeds, Sizes, Times [optional]
-
-<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
-
-[More Information Needed]
-
-## Evaluation
-
-<!-- This section describes the evaluation protocols and provides the results. -->
-
-### Testing Data, Factors & Metrics
-
-#### Testing Data
-
-<!-- This should link to a Dataset Card if possible. -->
-
-[More Information Needed]
-
-#### Factors
-
-<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
-
-[More Information Needed]
-
-#### Metrics
-
-<!-- These are the evaluation metrics being used, ideally with a description of why. -->
-
-[More Information Needed]
-
-### Results
-
-[More Information Needed]
-
-#### Summary
-
-
-
-## Model Examination [optional]
-
-<!-- Relevant interpretability work for the model goes here -->
-
-[More Information Needed]
-
-## Environmental Impact
-
-<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
-
-Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
-
-- **Hardware Type:** [More Information Needed]
-- **Hours used:** [More Information Needed]
-- **Cloud Provider:** [More Information Needed]
-- **Compute Region:** [More Information Needed]
-- **Carbon Emitted:** [More Information Needed]
-
-## Technical Specifications [optional]
-
-### Model Architecture and Objective
-
-[More Information Needed]
-
-### Compute Infrastructure
-
-[More Information Needed]
-
-#### Hardware
-
-[More Information Needed]
-
-#### Software
-
-[More Information Needed]
-
-## Citation [optional]
-
-<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
-
-**BibTeX:**
-
-[More Information Needed]
-
-**APA:**
-
-[More Information Needed]
-
-## Glossary [optional]
-
-<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
-
-[More Information Needed]
-
-## More Information [optional]
-
-[More Information Needed]
-
-## Model Card Authors [optional]
-
-[More Information Needed]
-
-## Model Card Contact
-
-[More Information Needed]
-### Framework versions
-
-- PEFT 0.18.1
\ No newline at end of file
diff --git a/starcoder2-7b/dp8/epochs/epoch_003/adapter/adapter_config.json b/starcoder2-7b/dp8/epochs/epoch_003/adapter/adapter_config.json
deleted file mode 100644
index 4d946b27ca1339ce45b035b190c4cf07ce83dec0..0000000000000000000000000000000000000000
--- a/starcoder2-7b/dp8/epochs/epoch_003/adapter/adapter_config.json
+++ /dev/null
@@ -1,46 +0,0 @@
-{
-  "alora_invocation_tokens": null,
-  "alpha_pattern": {},
-  "arrow_config": null,
-  "auto_mapping": null,
-  "base_model_name_or_path": "bigcode/starcoder2-7b",
-  "bias": "none",
-  "corda_config": null,
-  "ensure_weight_tying": true,
-  "eva_config": null,
-  "exclude_modules": null,
-  "fan_in_fan_out": false,
-  "inference_mode": true,
-  "init_lora_weights": true,
-  "layer_replication": null,
-  "layers_pattern": null,
-  "layers_to_transform": null,
-  "loftq_config": {},
-  "lora_alpha": 32,
-  "lora_bias": false,
-  "lora_dropout": 0.05,
-  "megatron_config": null,
-  "megatron_core": "megatron.core",
-  "modules_to_save": [
-    "lm_head",
-    "embed_tokens"
-  ],
-  "peft_type": "LORA",
-  "peft_version": "0.18.1",
-  "qalora_group_size": 16,
-  "r": 16,
-  "rank_pattern": {},
-  "revision": null,
-  "target_modules": [
-    "q_proj",
-    "k_proj",
-    "v_proj",
-    "o_proj"
-  ],
-  "target_parameters": null,
-  "task_type": "CAUSAL_LM",
-  "trainable_token_indices": null,
-  "use_dora": false,
-  "use_qalora": false,
-  "use_rslora": false
-}
\ No newline at end of file
diff --git a/starcoder2-7b/dp8/epochs/epoch_003/audit_results.json b/starcoder2-7b/dp8/epochs/epoch_003/audit_results.json
deleted file mode 100644
index d773f8389b969df10f43cb2f789e32a3ecaaee76..0000000000000000000000000000000000000000
--- a/starcoder2-7b/dp8/epochs/epoch_003/audit_results.json
+++ /dev/null
@@ -1,137 +0,0 @@
-{
-  "delta": 1e-05,
-  "num_canaries": 500,
-  "num_members": 250,
-  "paper_guess_fraction": 0.2,
-  "paper_guess_steps": 20,
-  "loss": {
-    "auc": 0.52628,
-    "empirical_epsilon": {
-      "0.05": 0.0,
-      "0.01": 0.0
-    },
-    "empirical_epsilon_details": {
-      "0.05": {
-        "epsilon": 0.0,
-        "num_guesses": 0,
-        "correct_guesses": 0,
-        "candidate_num_guesses": [
-          5,
-          10,
-          15,
-          20,
-          25,
-          30,
-          35,
-          40,
-          45,
-          50,
-          55,
-          60,
-          65,
-          70,
-          75,
-          80,
-          85,
-          90,
-          95,
-          100
-        ],
-        "direction": "lower"
-      },
-      "0.01": {
-        "epsilon": 0.0,
-        "num_guesses": 0,
-        "correct_guesses": 0,
-        "candidate_num_guesses": [
-          5,
-          10,
-          15,
-          20,
-          25,
-          30,
-          35,
-          40,
-          45,
-          50,
-          55,
-          60,
-          65,
-          70,
-          75,
-          80,
-          85,
-          90,
-          95,
-          100
-        ],
-        "direction": "lower"
-      }
-    }
-  },
-  "embedding": {
-    "auc": 0.53716,
-    "empirical_epsilon": {
-      "0.05": 0.0,
-      "0.01": 0.0
-    },
-    "empirical_epsilon_details": {
-      "0.05": {
-        "epsilon": 0.0,
-        "num_guesses": 0,
-        "correct_guesses": 0,
-        "candidate_num_guesses": [
-          5,
-          10,
-          15,
-          20,
-          25,
-          30,
-          35,
-          40,
-          45,
-          50,
-          55,
-          60,
-          65,
-          70,
-          75,
-          80,
-          85,
-          90,
-          95,
-          100
-        ],
-        "direction": "lower"
-      },
-      "0.01": {
-        "epsilon": 0.0,
-        "num_guesses": 0,
-        "correct_guesses": 0,
-        "candidate_num_guesses": [
-          5,
-          10,
-          15,
-          20,
-          25,
-          30,
-          35,
-          40,
-          45,
-          50,
-          55,
-          60,
-          65,
-          70,
-          75,
-          80,
-          85,
-          90,
-          95,
-          100
-        ],
-        "direction": "lower"
-      }
-    }
-  }
-}
\ No newline at end of file